-- Logs begin at Sat 2022-08-20 20:01:16 UTC, end at Sat 2022-08-20 22:15:19 UTC. -- Aug 20 20:37:25.308449 ncdv-master-1 systemd[1]: Starting Devstack devstack@keystone.service... Aug 20 20:37:25.317543 ncdv-master-1 devstack@keystone.service[104107]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Aug 20 20:37:25 2022] *** Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: nodename: ncdv-master-1 Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: machine: x86_64 Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: clock source: unix Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: pcre jit disabled Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: detected number of CPU cores: 4 Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: current working directory: / Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: detected binary path: /usr/bin/uwsgi-core Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: your processes number limit is 47680 Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: your memory page size is 4096 bytes Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: detected max file descriptor number: 2048 Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: lock engine: pthread robust mutexes Aug 20 20:37:25.319706 ncdv-master-1 devstack@keystone.service[104107]: thunder lock: enabled Aug 20 20:37:25.320478 ncdv-master-1 devstack@keystone.service[104107]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 20 20:37:25.320478 ncdv-master-1 devstack@keystone.service[104107]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 20 20:37:25.363912 ncdv-master-1 devstack@keystone.service[104107]: Python main interpreter initialized at 0x562d8caf49c0 Aug 20 20:37:25.363912 ncdv-master-1 devstack@keystone.service[104107]: python threads support enabled Aug 20 20:37:25.363912 ncdv-master-1 devstack@keystone.service[104107]: your server socket listen backlog is limited to 100 connections Aug 20 20:37:25.363912 ncdv-master-1 devstack@keystone.service[104107]: your mercy for graceful operations on workers is 90 seconds Aug 20 20:37:25.364306 ncdv-master-1 devstack@keystone.service[104107]: mapped 403077 bytes (393 KB) for 2 cores Aug 20 20:37:25.364306 ncdv-master-1 devstack@keystone.service[104107]: *** Operational MODE: preforking *** Aug 20 20:37:25.364594 ncdv-master-1 devstack@keystone.service[104107]: *** uWSGI is running in multiple interpreter mode *** Aug 20 20:37:25.364594 ncdv-master-1 devstack@keystone.service[104107]: spawned uWSGI master process (pid: 104107) Aug 20 20:37:25.364813 ncdv-master-1 systemd[1]: Started Devstack devstack@keystone.service. Aug 20 20:37:25.365316 ncdv-master-1 devstack@keystone.service[104107]: spawned uWSGI worker 1 (pid: 104118, cores: 1) Aug 20 20:37:25.365671 ncdv-master-1 devstack@keystone.service[104107]: spawned uWSGI worker 2 (pid: 104119, cores: 1) Aug 20 20:37:25.365818 ncdv-master-1 devstack@keystone.service[104107]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 20 20:37:29.923335 ncdv-master-1 devstack@keystone.service[104118]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 20 20:37:29.923335 ncdv-master-1 devstack@keystone.service[104118]: from cryptography.utils import int_from_bytes Aug 20 20:37:29.924187 ncdv-master-1 devstack@keystone.service[104118]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 20 20:37:29.924187 ncdv-master-1 devstack@keystone.service[104118]: from cryptography.utils import int_from_bytes Aug 20 20:37:29.924546 ncdv-master-1 devstack@keystone.service[104119]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 20 20:37:29.924546 ncdv-master-1 devstack@keystone.service[104119]: from cryptography.utils import int_from_bytes Aug 20 20:37:29.925067 ncdv-master-1 devstack@keystone.service[104119]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 20 20:37:29.925067 ncdv-master-1 devstack@keystone.service[104119]: from cryptography.utils import int_from_bytes Aug 20 20:37:29.973776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 20 20:37:29.973976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 20 20:37:29.974135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] command line args: [] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 20 20:37:29.974348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 20 20:37:29.974459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 20 20:37:29.974538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ================================================================================ {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 20 20:37:29.974603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 20 20:37:29.974729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] command line args: [] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 20 20:37:29.974813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] admin_token = **** {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.975009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 20 20:37:29.975133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] config_dir = [] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.975208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ================================================================================ {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 20 20:37:29.975279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.975341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] config_source = [] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.975411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] admin_token = **** {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.975471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] debug = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.975617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] config_dir = [] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.975697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.976183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.976183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] config_source = [] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.976315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.976407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] debug = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.976487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] insecure_debug = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.976553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.976648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.976711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.976795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.976857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] insecure_debug = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.976916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] list_limit = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.976978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.977048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] log_config_append = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.977278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.977367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.977367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] log_dir = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.977485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] list_limit = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.977543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] log_file = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.977607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] log_config_append = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.977667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.977723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.977863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] log_dir = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.977938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.977998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] log_file = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.978183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.978275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.978458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.978572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.978642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.978710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.978778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.978912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.978999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.979060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.979214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.979316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.979466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.979601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.979686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.979749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.979876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.979995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.980076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.980197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.980304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.980428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.980588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.980734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] notification_format = cadf {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.980823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.980912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.981032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] notification_format = cadf {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.981316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.981439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.981537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] publish_errors = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.981721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.981853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.981947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.982022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] publish_errors = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.982159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.982274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.982386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.982512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.982512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.982731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.982846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.982966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] standard_threads = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.983157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.983322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] strict_password_check = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.983322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.983541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] standard_threads = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.983653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] use_eventlog = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.983653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] use_journal = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.983941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] strict_password_check = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.984057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] use_json = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.984057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] use_stderr = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.984266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.984401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] use_syslog = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.984535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] use_eventlog = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.984535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] use_journal = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.984748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] watch_log_file = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.984867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] use_json = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.984980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.985094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] use_stderr = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.985094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] use_syslog = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.985314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.985426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] watch_log_file = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 20 20:37:29.985544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.985663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.985871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.986000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.986115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.986235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.986355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.986469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.986582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.986702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.986821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.986933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.987044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.987155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.987310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.987437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.987550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.987667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.987785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.987897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.988014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.988134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.988248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.988365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.988498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.988631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.988746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.988872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.988985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.989097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.989214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.989327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.989438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.989630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.989752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.989864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.989987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.990101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.990220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.990341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.990341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.990557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.990676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.990788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.990788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.991012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.991124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.991235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.991332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.991427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.991530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.991626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.991719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.991813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.991813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.991987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.992088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.992197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.992296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.992436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.992552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.992663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.992783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.992896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.993013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.993125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.993236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.993347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.993468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.993581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.993695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.993806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.993918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.994035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.994159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.994272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.994383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.994494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.994613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.994725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.994913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.995027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.995138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.995249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.995362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.995479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.995602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.995715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.995833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.995946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.996058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.996172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.996290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.996446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.996554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.996656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.996766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.996877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.997002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.997123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.997236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.997382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.997497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.997611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.997733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.997846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.997958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.998069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.998181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.998327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.998449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.998569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.998705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.998821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.998934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.999039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.999154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.999251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.999347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.999449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.999563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.999685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.999815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:29.999815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.000042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.000164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.000278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.000405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.000521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.000647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.000762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.000931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.001055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.001209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.001333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.001488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.001607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.001751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.001868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.002080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.002201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.002353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.002470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.002614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.002729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.002872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.002989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.003148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.003266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.enabled = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.003411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.enabled = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.003526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.003676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.003795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.003955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.004073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.004216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.004331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.004522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.004637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.004763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.004884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.005006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.005120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.005233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.005355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.005477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.005592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.005704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.005818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.005930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.006044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.006165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.006278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.006391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.006504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.006616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.006729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.006923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.007039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.007153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.007265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.007412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.007538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.007660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.007782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.007907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.008022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.008136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.008254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.008381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.008511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.008634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.008749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.008879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.009008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.009121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.009244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.009358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.009471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.009583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.009695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.009807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.009928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.010042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.010155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.010269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.010381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] auth.external = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.010494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] auth.external = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.010614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] auth.mapped = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.010728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] auth.mapped = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.010841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.010984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.011110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.011231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.011347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] auth.password = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.011461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] auth.password = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.011587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] auth.token = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.011701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] auth.token = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.011821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.011937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.012050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] catalog.caching = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.012168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] catalog.caching = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.012291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.012431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.012549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.012668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.012783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.012897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.013019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.013134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.013254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.013376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.013376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] credential.caching = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.013591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] credential.caching = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.013713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] credential.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.013833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] credential.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.013947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.014061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.014180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.014350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.014481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.014597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.014710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.014823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.014936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.015055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.015181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.015296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.015410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.015524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.015644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.015767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.015882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.015995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.016109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.016222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.016335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.016481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.016599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.016713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.016713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.016933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.017048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.017048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] federation.caching = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.017269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.017390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.017542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] federation.caching = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.017660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] federation.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.017774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.017899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.018015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] federation.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.018134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.018247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.018360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.018473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.018598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.018713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.018827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.018950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.019064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.019181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.019303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.019418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.019530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.019643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.019757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.019870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.019997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.caching = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.020122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.020235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.020360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.caching = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.020497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.020611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.020803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.020921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.021035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.021148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.021262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.021375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.021497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.021611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.021724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.021838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.021951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.022065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.022190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.022305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.022419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.022538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.022653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.022766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.022888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.023003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.023116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.023230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.023343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.023456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.023581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.023696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.023810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.023925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.024038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.024159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.024275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.024441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.024559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.024673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.024786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.024911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.025028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.025147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.025264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.025377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.025490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.025610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.025731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.025847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.025962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.026075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.026188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.026309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.026424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.026546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.026661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.026775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.026888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.027010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.027125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.027247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.027362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.027476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.027629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.027754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.027869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.027983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.028096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.028209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.028322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.028513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.028638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.password = **** {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.028793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.028910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.029061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.password = **** {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.029178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.029334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.029563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.029684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.029831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.029995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.030255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.030446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.030587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.030691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.030842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.030959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.031117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.031246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.031394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.031510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.031655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.031772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.031922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.032047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.032196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.032313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.032467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.032467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.032700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.032822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.032936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.033049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.033162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.033274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.033388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.033509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.033509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.033723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.033836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.033949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.034062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.034062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.034283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.034398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.034511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.034624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.034745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.034859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.034971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.035084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.035197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.035311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.035434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.035549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.035662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.035776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.035888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.036001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.036122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.036122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.036334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.036524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.036639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.036753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.036878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.036994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.037108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.037220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.037333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.037333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.037554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.037554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.037805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.037805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.038017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.038017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.038237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.038352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.038465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.038681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.038800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.038913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.039035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] policy.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.039149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.039262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.039375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.039495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.039611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.039732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.039847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] policy.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.039960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] receipt.caching = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.040073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.040186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.040299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.040439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.040556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.040670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.040783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] receipt.caching = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.040896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.041009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.041130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.041251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.041366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] resource.caching = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.041478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.041593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.041707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.041828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] resource.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.041942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.042055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.042172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] resource.caching = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.042285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.042399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.042520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.042634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] resource.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.042749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] revoke.caching = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.042862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.042975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.043088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.043214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.043330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.043444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] role.cache_time = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.043557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] revoke.caching = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.043670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] role.caching = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.043784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.043974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] role.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.044091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.044205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] role.list_limit = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.044319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] role.cache_time = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.044459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.044574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] role.caching = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.044704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.044820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] role.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.044935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.045048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] role.list_limit = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.045161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.045276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.045397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.045536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.045693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.045808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.045922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.046021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.046126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.046224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.046339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.046452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.046565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.046686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.046802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.046916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.047031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.047145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.047258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.047380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.047380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.047594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.047746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.047864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.047979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.048101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.048101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.048316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.048465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.048580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.048693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.048819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.048935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.049050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.049164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.049277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.049390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.049512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.049627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.049741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.049854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.049968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.050081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.050081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.050304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.050304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.050518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.050631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.050744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.050865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.050980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.051093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.051207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.051320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.051442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.051556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.051669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] token.cache_time = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.051783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.051896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] token.caching = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.052009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.052133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.052250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.052363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] token.provider = fernet {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.052499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] token.cache_time = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.052612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.052725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] token.caching = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.052851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.052967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.053079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.053193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] token.provider = fernet {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.053307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.053419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.053540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.053663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.053777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.053891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.053995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] trust.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.054107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.054228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.054343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.054457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.054570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.054682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.054795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] trust.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.054915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.055030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.055143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.055255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.055369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.055482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.055645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.055764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.055877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.055990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.056103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.connection = **** {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.056225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.056348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.056502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.056629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.056745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.056859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.056972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.connection = **** {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.057094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.057217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.057332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.057455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.057570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.057683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.057832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.057949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.058064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.058185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.058300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.058413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.058527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.058640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.058754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.058875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.058990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.059104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.059217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.059331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.059445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.059567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.059683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.059797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.059910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.060023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.060136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.060258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.060420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.060537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.060653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=104118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 20 20:37:30.060768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.060768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.060768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 20 20:37:30.060768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=104119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 20 20:37:30.085698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.085836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.085950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.086048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.086172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.086252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.094422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.094561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.123994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.123994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.124719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.124719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.175610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.176072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.223996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.224211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.224324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.224460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.289941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.290134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.290373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.290503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.290646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.290751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.290878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.290957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.291023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.291096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.291333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.291539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.314356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.314546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.314772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.314875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.315022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.315114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.315210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.315299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.315455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.315596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.315736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.315818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.315881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.316085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.326413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.326646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.339170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.339384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.339622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.339842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.339989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.340075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.340150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.340222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.340294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.340549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=104118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.340675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.340753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.340970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.341173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=104119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 20 20:37:30.357351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.357521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 20 20:37:30.357895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.358087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.358216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 20 20:37:30.358308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 20 20:37:30.358391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.358649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.358775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.358853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 20 20:37:30.358918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.359055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.359135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.359342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.359594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.359843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.368533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.368808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.369072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.369196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.369396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.369527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.369741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.370016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.376521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.377245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.379384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.379663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.379944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.380070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.380224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.380353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.380513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.380708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.380800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.380928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.381021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.381145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.381316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.381443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.381573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.381703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.381832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.381984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.382113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.382241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.382370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.382625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.406199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.406739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.407651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.407931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.408186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.408485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.412466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.412737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.412947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.413048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.413245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.413517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.419542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.419824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.420017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.420306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.423849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.424141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.424268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.424492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.424590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.424721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.424821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.424948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.425151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.425343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.425424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.425662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.425799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.425928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.426055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.426267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.441783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.442063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.442328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.442602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.445935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.446445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.449027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.449321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.449509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.449821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.453754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.454178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.457049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.457330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.460141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.460347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.460533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.460657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.460750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.460954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.461082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.461235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.461320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.461484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.461605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.461870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.477352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.477580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.477707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.477857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.477939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.478060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.478197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.478361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.484880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.485411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.486525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.486780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.487003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.487303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.489571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.489856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.489982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.490153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.490288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.490413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.490538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.490713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.490850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.490975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.501995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.502277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.502405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.502559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.502651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.502789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.502996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.503089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.503202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.503326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.503419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.503617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.503759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.503898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.519159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.519288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.522041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.522171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.525039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.525168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.526469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.526597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.527931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.528081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.528223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.528305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.528491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.528619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.535050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.535405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.536531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.536880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.539392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.539683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.539826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.539967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.540049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.540239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.540365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.540537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.549175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.549413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.549537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.549675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.549755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.549952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.556456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=104118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.556659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=104119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 20 20:37:30.556811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.557033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.557178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.557316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.557435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.557514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.557658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.557797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.557934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.558014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.558187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.558312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.558447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.558541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.558732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.558861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.558979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.559086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.559230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.559308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.559521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.559647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.559823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.559950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.560124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=104119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.560248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=104118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 20 20:37:30.593207 ncdv-master-1 devstack@keystone.service[104118]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x562d8caf49c0 pid: 104118 (default app) Aug 20 20:37:30.593982 ncdv-master-1 devstack@keystone.service[104119]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x562d8caf49c0 pid: 104119 (default app) Aug 20 20:37:30.596693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb701358-6247-4abc-821b-d51085cb12a8 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:30.596975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb701358-6247-4abc-821b-d51085cb12a8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:30.597246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb701358-6247-4abc-821b-d51085cb12a8 None None] PATH_INFO: `/v3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:30.601416 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Sat Aug 20 20:37:30 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:37.767208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95594093-6f79-4693-b050-ba75993790ab None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:37.767744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95594093-6f79-4693-b050-ba75993790ab None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:37.767744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95594093-6f79-4693-b050-ba75993790ab None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:37.772092 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:37:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:37:37.778923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b590cb-d6f2-4190-956d-b33378afe6f3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:37.779635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b590cb-d6f2-4190-956d-b33378afe6f3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:37.779794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b590cb-d6f2-4190-956d-b33378afe6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:37.859393 ncdv-master-1 devstack@keystone.service[104118]: INFO dbcounter [None req-92b590cb-d6f2-4190-956d-b33378afe6f3 None None] Registered counter for database keystone Aug 20 20:37:37.880245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG oslo_db.sqlalchemy.engines [None req-92b590cb-d6f2-4190-956d-b33378afe6f3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=104118) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 20 20:37:37.891974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writer thread running {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 20 20:37:38.092298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG passlib.handlers.bcrypt [None req-92b590cb-d6f2-4190-956d-b33378afe6f3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=104118) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 20 20:37:38.093146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG passlib.handlers.bcrypt [None req-92b590cb-d6f2-4190-956d-b33378afe6f3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=104118) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 20 20:37:38.154377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-92b590cb-d6f2-4190-956d-b33378afe6f3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:37:38.156408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-92b590cb-d6f2-4190-956d-b33378afe6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:37:38.291355 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:37:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 514 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 20 20:37:38.866272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95594093-6f79-4693-b050-ba75993790ab None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:38.949734 ncdv-master-1 devstack@keystone.service[104119]: INFO dbcounter [None req-95594093-6f79-4693-b050-ba75993790ab None None] Registered counter for database keystone Aug 20 20:37:38.974666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG oslo_db.sqlalchemy.engines [None req-95594093-6f79-4693-b050-ba75993790ab None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=104119) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 20 20:37:38.985382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writer thread running {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 20 20:37:39.104281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95594093-6f79-4693-b050-ba75993790ab None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:39.109942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a992bc-3941-4fd5-ae5d-b97ccfd51d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:39.110550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a992bc-3941-4fd5-ae5d-b97ccfd51d23 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:39.110819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a992bc-3941-4fd5-ae5d-b97ccfd51d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:39.111085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a992bc-3941-4fd5-ae5d-b97ccfd51d23 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:39.120397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a992bc-3941-4fd5-ae5d-b97ccfd51d23 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:39.174298 ncdv-master-1 devstack@keystone.service[104119]: WARNING keystone.common.rbac_enforcer.enforcer [None req-17a992bc-3941-4fd5-ae5d-b97ccfd51d23 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 20 20:37:39.174631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a992bc-3941-4fd5-ae5d-b97ccfd51d23 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:39.184934 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Sat Aug 20 20:37:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 20:37:39.192309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b590cb-d6f2-4190-956d-b33378afe6f3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:39.218408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b590cb-d6f2-4190-956d-b33378afe6f3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:39.224194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49532db-7e32-4512-bac4-d6fd36ecd9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:39.225356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49532db-7e32-4512-bac4-d6fd36ecd9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:39.225628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49532db-7e32-4512-bac4-d6fd36ecd9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:39.225893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49532db-7e32-4512-bac4-d6fd36ecd9d9 admin admin] PATH_INFO: `/v3/projects/a5bed9a4b90f48f592cdc8d94741f821` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:39.227497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49532db-7e32-4512-bac4-d6fd36ecd9d9 admin admin] RBAC: Authorizing `identity:get_project(project_id=a5bed9a4b90f48f592cdc8d94741f821)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:39.278092 ncdv-master-1 devstack@keystone.service[104118]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b49532db-7e32-4512-bac4-d6fd36ecd9d9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 20 20:37:39.278443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49532db-7e32-4512-bac4-d6fd36ecd9d9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:39.283243 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Sat Aug 20 20:37:39 2022] GET /identity/v3/projects/a5bed9a4b90f48f592cdc8d94741f821 => generated 344 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:41.434569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ec4be-be99-47b7-b7e0-746636999fbf None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:41.435438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ec4be-be99-47b7-b7e0-746636999fbf None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:41.435438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ec4be-be99-47b7-b7e0-746636999fbf None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:41.440139 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:37:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:37:41.445980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95715c-7250-40e2-966c-b21150dea414 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:41.446408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95715c-7250-40e2-966c-b21150dea414 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:41.446507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95715c-7250-40e2-966c-b21150dea414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:41.466928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0b95715c-7250-40e2-966c-b21150dea414 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:37:41.468512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0b95715c-7250-40e2-966c-b21150dea414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:37:41.477235 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:37:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 20 20:37:41.809000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ec4be-be99-47b7-b7e0-746636999fbf None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:41.824059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ec4be-be99-47b7-b7e0-746636999fbf None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:41.829540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e507f4-b3a2-4144-b05c-4e5290c2f311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:41.830129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e507f4-b3a2-4144-b05c-4e5290c2f311 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:41.830398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e507f4-b3a2-4144-b05c-4e5290c2f311 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:41.830661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e507f4-b3a2-4144-b05c-4e5290c2f311 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:41.839089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e507f4-b3a2-4144-b05c-4e5290c2f311 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:41.840901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e507f4-b3a2-4144-b05c-4e5290c2f311 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:41.850584 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Sat Aug 20 20:37:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 20:37:41.857052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95715c-7250-40e2-966c-b21150dea414 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:41.868278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95715c-7250-40e2-966c-b21150dea414 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:41.878044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17857736-d162-4a4c-a774-7c0dbf717453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:41.879048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17857736-d162-4a4c-a774-7c0dbf717453 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:41.879514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17857736-d162-4a4c-a774-7c0dbf717453 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:41.879949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17857736-d162-4a4c-a774-7c0dbf717453 admin admin] PATH_INFO: `/v3/users/4ceb679399ff439a9bfe96f9758e269d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:41.882338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17857736-d162-4a4c-a774-7c0dbf717453 admin admin] RBAC: Authorizing `identity:get_user(user_id=4ceb679399ff439a9bfe96f9758e269d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:41.885429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17857736-d162-4a4c-a774-7c0dbf717453 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:41.887928 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Sat Aug 20 20:37:41 2022] GET /identity/v3/users/4ceb679399ff439a9bfe96f9758e269d => generated 247 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:44.162036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b74a4-3489-4f07-8441-06d1e58ba29f None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:44.162615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b74a4-3489-4f07-8441-06d1e58ba29f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:44.162615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b74a4-3489-4f07-8441-06d1e58ba29f None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:44.163898 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:37:44.166689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43faa20-cfc6-42fa-83bc-4a5f7776f75a None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:44.167136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43faa20-cfc6-42fa-83bc-4a5f7776f75a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:44.167236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43faa20-cfc6-42fa-83bc-4a5f7776f75a None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:44.168773 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:37:44.170115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b83f81-e2b2-4069-b428-14a0e08521f7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:44.170599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b83f81-e2b2-4069-b428-14a0e08521f7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:44.171021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b83f81-e2b2-4069-b428-14a0e08521f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:44.175123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72977669-9ad8-40cc-9026-27d2661ccf9d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:44.175400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72977669-9ad8-40cc-9026-27d2661ccf9d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:44.175663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72977669-9ad8-40cc-9026-27d2661ccf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:44.194299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-72977669-9ad8-40cc-9026-27d2661ccf9d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:37:44.195474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-72977669-9ad8-40cc-9026-27d2661ccf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:37:44.214011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG passlib.handlers.bcrypt [None req-a2b83f81-e2b2-4069-b428-14a0e08521f7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=104119) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 20 20:37:44.214424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG passlib.handlers.bcrypt [None req-a2b83f81-e2b2-4069-b428-14a0e08521f7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=104119) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 20 20:37:44.259454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a2b83f81-e2b2-4069-b428-14a0e08521f7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:37:44.261123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a2b83f81-e2b2-4069-b428-14a0e08521f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:37:44.384666 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:37:44.403219 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:37:44.708136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b83f81-e2b2-4069-b428-14a0e08521f7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:44.728878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72977669-9ad8-40cc-9026-27d2661ccf9d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:44.756959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:44.813760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:44.817289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b83f81-e2b2-4069-b428-14a0e08521f7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:44.836365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72977669-9ad8-40cc-9026-27d2661ccf9d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:44.853766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd82313-17fc-4a35-8fd7-55d2645a276d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:44.854874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd82313-17fc-4a35-8fd7-55d2645a276d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:44.855152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd82313-17fc-4a35-8fd7-55d2645a276d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:44.855419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd82313-17fc-4a35-8fd7-55d2645a276d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:44.867065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd82313-17fc-4a35-8fd7-55d2645a276d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:37:44.869489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ea7be-1021-4559-a97b-96f17af921b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:44.870096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ea7be-1021-4559-a97b-96f17af921b3 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:44.870372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ea7be-1021-4559-a97b-96f17af921b3 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:44.870636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ea7be-1021-4559-a97b-96f17af921b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-0dd82313-17fc-4a35-8fd7-55d2645a276d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:37:44.875699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:37:44.876688 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:37:44.877509 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:37:44.877509 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:37:44.877509 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:37:44.877509 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:37:44.877509 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:37:44.877509 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:37:44.877509 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:37:44.877509 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:37:44.877509 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:37:44.877509 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:37:44.877509 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:37:44.877509 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:37:44.877509 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:37:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:37:44.882716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd82313-17fc-4a35-8fd7-55d2645a276d None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:44.938212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ea7be-1021-4559-a97b-96f17af921b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:44.939683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ea7be-1021-4559-a97b-96f17af921b3 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:44.965498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:44.984486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd82313-17fc-4a35-8fd7-55d2645a276d None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:45.018328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c4e3a-b016-494c-a66d-39cc74601635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:45.018927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c4e3a-b016-494c-a66d-39cc74601635 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:45.019197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c4e3a-b016-494c-a66d-39cc74601635 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:45.019456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c4e3a-b016-494c-a66d-39cc74601635 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:45.020103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c4e3a-b016-494c-a66d-39cc74601635 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:37:45.020423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c4e3a-b016-494c-a66d-39cc74601635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:45.021858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c4e3a-b016-494c-a66d-39cc74601635 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:45.033866 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:37:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:37:45.036956 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 8/16] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:37:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 20 20:37:45.039928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c4e3a-b016-494c-a66d-39cc74601635 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:45.042614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ea7be-1021-4559-a97b-96f17af921b3 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:45.065879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:45.141203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ea7be-1021-4559-a97b-96f17af921b3 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:45.147613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c4e3a-b016-494c-a66d-39cc74601635 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:45.167812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:45.178050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6f276-583a-4d8a-8ea4-57df1f0e74ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:45.178699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6f276-583a-4d8a-8ea4-57df1f0e74ce None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:45.178969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6f276-583a-4d8a-8ea4-57df1f0e74ce None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:45.179230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6f276-583a-4d8a-8ea4-57df1f0e74ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:45.191600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6f276-583a-4d8a-8ea4-57df1f0e74ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:45.193168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6f276-583a-4d8a-8ea4-57df1f0e74ce None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-47d6f276-583a-4d8a-8ea4-57df1f0e74ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:37:45.213274 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:37:45.214275 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:37:45.215076 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:37:45.215076 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:37:45.215076 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:37:45.215076 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:37:45.221185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6f276-583a-4d8a-8ea4-57df1f0e74ce None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:45.293793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ecee5f-8306-47e1-a914-15e8e58923bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:45.294393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ecee5f-8306-47e1-a914-15e8e58923bf None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:45.294666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ecee5f-8306-47e1-a914-15e8e58923bf None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:45.294959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ecee5f-8306-47e1-a914-15e8e58923bf None admin] PATH_INFO: `/v3/users/4ceb679399ff439a9bfe96f9758e269d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:45.296879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ecee5f-8306-47e1-a914-15e8e58923bf None admin] RBAC: Authorizing `identity:get_user(user_id=4ceb679399ff439a9bfe96f9758e269d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:45.298331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ecee5f-8306-47e1-a914-15e8e58923bf None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:45.299933 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:37:45 2022] GET /identity/v3/users/4ceb679399ff439a9bfe96f9758e269d => generated 247 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:45.305729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ecee5f-8306-47e1-a914-15e8e58923bf None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:45.333933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6f276-583a-4d8a-8ea4-57df1f0e74ce None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:45.367395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f68dd7-7068-43e3-a3e8-6842d5b8f538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:45.367998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f68dd7-7068-43e3-a3e8-6842d5b8f538 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:45.368270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f68dd7-7068-43e3-a3e8-6842d5b8f538 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:45.368547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f68dd7-7068-43e3-a3e8-6842d5b8f538 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:45.369215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f68dd7-7068-43e3-a3e8-6842d5b8f538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:37:45.369518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f68dd7-7068-43e3-a3e8-6842d5b8f538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:45.370885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f68dd7-7068-43e3-a3e8-6842d5b8f538 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:45.388110 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:37:45.398877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:45.411136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ecee5f-8306-47e1-a914-15e8e58923bf None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:45.450234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6197f5-6963-41b6-8200-b7dffb3bff6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:45.450905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6197f5-6963-41b6-8200-b7dffb3bff6d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:45.451178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6197f5-6963-41b6-8200-b7dffb3bff6d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:45.451642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6197f5-6963-41b6-8200-b7dffb3bff6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:45.453523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6197f5-6963-41b6-8200-b7dffb3bff6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:45.454996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6197f5-6963-41b6-8200-b7dffb3bff6d None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:45.456703 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 10/20] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:37:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:45.462833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f68dd7-7068-43e3-a3e8-6842d5b8f538 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:45.508475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:45.573548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f68dd7-7068-43e3-a3e8-6842d5b8f538 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:45.609153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8adf1-32d7-4c9a-9817-9b5b933ed063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:45.609777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8adf1-32d7-4c9a-9817-9b5b933ed063 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:45.610068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8adf1-32d7-4c9a-9817-9b5b933ed063 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:45.610337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8adf1-32d7-4c9a-9817-9b5b933ed063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:45.611055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8adf1-32d7-4c9a-9817-9b5b933ed063 None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.domain.id=default, user.id=4ceb679399ff439a9bfe96f9758e269d) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:37:45.611358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8adf1-32d7-4c9a-9817-9b5b933ed063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:45.612859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8adf1-32d7-4c9a-9817-9b5b933ed063 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:45.619747 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 11/21] 10.222.0.23 () {56 vars in 1322 bytes} [Sat Aug 20 20:37:45 2022] GET /identity/v3/role_assignments?user.id=4ceb679399ff439a9bfe96f9758e269d&scope.domain.id=default&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:37:47.892711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5708b4-70b5-4dd9-a131-eda9aba4b5cc None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:47.893257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5708b4-70b5-4dd9-a131-eda9aba4b5cc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:47.893257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5708b4-70b5-4dd9-a131-eda9aba4b5cc None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:47.894733 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 11/22] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:37:47.900306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55541b09-6f2f-4732-9bc1-e6a709ebd0d1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:47.900731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55541b09-6f2f-4732-9bc1-e6a709ebd0d1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:47.900904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55541b09-6f2f-4732-9bc1-e6a709ebd0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:47.927765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-55541b09-6f2f-4732-9bc1-e6a709ebd0d1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:37:47.928991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-55541b09-6f2f-4732-9bc1-e6a709ebd0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:37:48.015150 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 12/23] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:37:48.343269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa6d3e-2577-4435-9599-d19b305e8ad7 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:48.343651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa6d3e-2577-4435-9599-d19b305e8ad7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:48.343839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa6d3e-2577-4435-9599-d19b305e8ad7 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:48.345327 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 12/24] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:37:48.362328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713a92c-693a-4e24-b1bd-64b85951ac2b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:48.362610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713a92c-693a-4e24-b1bd-64b85951ac2b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:48.363039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713a92c-693a-4e24-b1bd-64b85951ac2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:48.389536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d713a92c-693a-4e24-b1bd-64b85951ac2b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:37:48.391113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d713a92c-693a-4e24-b1bd-64b85951ac2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:37:48.425316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:48.482992 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 13/25] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:37:48.559999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8c3e2-2b3f-47c0-a0ba-7b2210191d92 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:48.561000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8c3e2-2b3f-47c0-a0ba-7b2210191d92 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:48.561430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8c3e2-2b3f-47c0-a0ba-7b2210191d92 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:48.562925 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 13/26] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:37:48.568303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566261b-e7ea-4f9e-b7df-4a3fa01e29ef None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:48.568836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566261b-e7ea-4f9e-b7df-4a3fa01e29ef None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:48.569319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566261b-e7ea-4f9e-b7df-4a3fa01e29ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:48.591687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f566261b-e7ea-4f9e-b7df-4a3fa01e29ef None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:37:48.593605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f566261b-e7ea-4f9e-b7df-4a3fa01e29ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:37:48.690744 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 14/27] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:37:48.696715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8c3e2-2b3f-47c0-a0ba-7b2210191d92 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:48.836072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8c3e2-2b3f-47c0-a0ba-7b2210191d92 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:48.876210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28d1f4-c586-429e-b1b5-aba9d275af60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:48.876875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28d1f4-c586-429e-b1b5-aba9d275af60 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:48.877147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28d1f4-c586-429e-b1b5-aba9d275af60 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:48.877411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28d1f4-c586-429e-b1b5-aba9d275af60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:48.902386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28d1f4-c586-429e-b1b5-aba9d275af60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:48.903859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28d1f4-c586-429e-b1b5-aba9d275af60 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-5e28d1f4-c586-429e-b1b5-aba9d275af60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:37:48.923886 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:37:48.924800 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:37:48.925605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:37:48.925605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:37:48.925605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:37:48.928528 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 14/28] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:37:48.932818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566261b-e7ea-4f9e-b7df-4a3fa01e29ef None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:48.938382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe62e05-8d76-48a4-8ee4-84f83009a899 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:48.938624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe62e05-8d76-48a4-8ee4-84f83009a899 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:48.938829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe62e05-8d76-48a4-8ee4-84f83009a899 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:48.940306 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 15/29] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:37:48.945304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae768800-c4f3-4ed1-af9d-4802919f092c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:48.945579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae768800-c4f3-4ed1-af9d-4802919f092c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:48.945845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae768800-c4f3-4ed1-af9d-4802919f092c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:48.973372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ae768800-c4f3-4ed1-af9d-4802919f092c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:37:48.974557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ae768800-c4f3-4ed1-af9d-4802919f092c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:37:48.983468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:49.122767 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 16/30] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:37:49.127836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae768800-c4f3-4ed1-af9d-4802919f092c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:49.129888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:49.154677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566261b-e7ea-4f9e-b7df-4a3fa01e29ef None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:49.258749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bde94-04bd-4201-9c47-27ad600895d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:49.259764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bde94-04bd-4201-9c47-27ad600895d8 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:49.260233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bde94-04bd-4201-9c47-27ad600895d8 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:49.260721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bde94-04bd-4201-9c47-27ad600895d8 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:49.261580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bde94-04bd-4201-9c47-27ad600895d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:37:49.262080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bde94-04bd-4201-9c47-27ad600895d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:49.263819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bde94-04bd-4201-9c47-27ad600895d8 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:49.291951 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 15/31] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:49.293173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bde94-04bd-4201-9c47-27ad600895d8 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:49.321607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae768800-c4f3-4ed1-af9d-4802919f092c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:49.416571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bde94-04bd-4201-9c47-27ad600895d8 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:49.417083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537d9b2-83fe-43d9-b52a-02f47b926805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:49.417773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537d9b2-83fe-43d9-b52a-02f47b926805 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:49.418046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537d9b2-83fe-43d9-b52a-02f47b926805 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:49.418311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537d9b2-83fe-43d9-b52a-02f47b926805 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:49.419047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537d9b2-83fe-43d9-b52a-02f47b926805 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:49.421190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537d9b2-83fe-43d9-b52a-02f47b926805 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:49.444356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:49.467526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354a6ed-10d7-41cf-a508-98b16e59eae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:49.468566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354a6ed-10d7-41cf-a508-98b16e59eae4 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:49.469066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354a6ed-10d7-41cf-a508-98b16e59eae4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:49.469536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354a6ed-10d7-41cf-a508-98b16e59eae4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:49.494775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354a6ed-10d7-41cf-a508-98b16e59eae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-a354a6ed-10d7-41cf-a508-98b16e59eae4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:37:49.501525 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:37:49.502372 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:37:49.503125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:37:49.503125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:37:49.503125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:37:49.503125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:37:49.503125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:37:49.503125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:37:49.503125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:37:49.503125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:37:49.503125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:37:49.503125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:37:49.503125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:37:49.503125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:37:49.514442 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 16/32] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:37:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:37:49.515481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354a6ed-10d7-41cf-a508-98b16e59eae4 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:49.536947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:49.581747 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 17/33] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Aug 20 20:37:49 2022] POST /identity/v3/roles => generated 219 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:37:49.582848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537d9b2-83fe-43d9-b52a-02f47b926805 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:49.710189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354a6ed-10d7-41cf-a508-98b16e59eae4 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:49.751781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450614e-3260-40d4-bb98-9aa3c505bff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:49.752475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450614e-3260-40d4-bb98-9aa3c505bff0 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:49.752935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450614e-3260-40d4-bb98-9aa3c505bff0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:49.753215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450614e-3260-40d4-bb98-9aa3c505bff0 None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:49.753884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450614e-3260-40d4-bb98-9aa3c505bff0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:49.755399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450614e-3260-40d4-bb98-9aa3c505bff0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:49.826736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537d9b2-83fe-43d9-b52a-02f47b926805 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:49.864565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbddb21-6401-4845-b1b4-49a09e590007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:49.865184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbddb21-6401-4845-b1b4-49a09e590007 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:49.865458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbddb21-6401-4845-b1b4-49a09e590007 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:49.865723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbddb21-6401-4845-b1b4-49a09e590007 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:49.866392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbddb21-6401-4845-b1b4-49a09e590007 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:37:49.866686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbddb21-6401-4845-b1b4-49a09e590007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:49.868187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbddb21-6401-4845-b1b4-49a09e590007 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:49.877402 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 18/34] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:37:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:49.878101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbddb21-6401-4845-b1b4-49a09e590007 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:49.932709 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 17/35] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:37:49 2022] POST /identity/v3/projects => generated 301 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:37:49.934039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6ce36-da94-4aac-b4bc-b865828c8080 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:49.934575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6ce36-da94-4aac-b4bc-b865828c8080 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:49.934575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6ce36-da94-4aac-b4bc-b865828c8080 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:49.936510 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 18/36] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:37:49.937318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6ce36-da94-4aac-b4bc-b865828c8080 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:49.950464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:49.994227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbddb21-6401-4845-b1b4-49a09e590007 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:50.016045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:50.031618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172fa629-fa0a-45e3-97e1-5f8a45c69b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:50.032415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fa629-fa0a-45e3-97e1-5f8a45c69b50 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:50.032687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fa629-fa0a-45e3-97e1-5f8a45c69b50 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:50.032953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fa629-fa0a-45e3-97e1-5f8a45c69b50 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:50.033625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172fa629-fa0a-45e3-97e1-5f8a45c69b50 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:50.035038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172fa629-fa0a-45e3-97e1-5f8a45c69b50 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:50.055180 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 19/37] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Aug 20 20:37:49 2022] POST /identity/v3/roles => generated 225 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:37:50.057536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3eadcd-a3fc-4f16-b6e7-449ec77fc377 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:50.058547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3eadcd-a3fc-4f16-b6e7-449ec77fc377 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:50.058974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3eadcd-a3fc-4f16-b6e7-449ec77fc377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:50.070651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6ce36-da94-4aac-b4bc-b865828c8080 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:50.088311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fe3eadcd-a3fc-4f16-b6e7-449ec77fc377 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:37:50.091534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fe3eadcd-a3fc-4f16-b6e7-449ec77fc377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:37:50.118600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a0ca0-6e7e-4cab-a2e1-73d564d435f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:50.119421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a0ca0-6e7e-4cab-a2e1-73d564d435f7 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:50.119765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a0ca0-6e7e-4cab-a2e1-73d564d435f7 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:50.120152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a0ca0-6e7e-4cab-a2e1-73d564d435f7 None admin] PATH_INFO: `/v3/users/4ceb679399ff439a9bfe96f9758e269d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:50.121803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a0ca0-6e7e-4cab-a2e1-73d564d435f7 None admin] RBAC: Authorizing `identity:get_user(user_id=4ceb679399ff439a9bfe96f9758e269d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:50.124124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a0ca0-6e7e-4cab-a2e1-73d564d435f7 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:50.126315 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 19/38] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:37:49 2022] GET /identity/v3/users/4ceb679399ff439a9bfe96f9758e269d => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:50.127683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86426c3-d535-40db-8719-8d1fdadbc303 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:50.128038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86426c3-d535-40db-8719-8d1fdadbc303 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:50.131586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86426c3-d535-40db-8719-8d1fdadbc303 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:50.131586 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 20/39] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:37:50.135708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86426c3-d535-40db-8719-8d1fdadbc303 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:50.183794 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 20/40] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:37:50.190093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df158a-de03-419b-847c-6565b14d0bef None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:50.190386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df158a-de03-419b-847c-6565b14d0bef None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:50.190662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df158a-de03-419b-847c-6565b14d0bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:50.216355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-28df158a-de03-419b-847c-6565b14d0bef None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:37:50.218416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-28df158a-de03-419b-847c-6565b14d0bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:37:50.274056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86426c3-d535-40db-8719-8d1fdadbc303 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:50.287697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:50.312514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:50.317179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee53004-cd90-470d-bec8-dd5f6fc5280d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:50.318266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee53004-cd90-470d-bec8-dd5f6fc5280d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:50.318529 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 21/41] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:37:50.318974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee53004-cd90-470d-bec8-dd5f6fc5280d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:50.319316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee53004-cd90-470d-bec8-dd5f6fc5280d None admin] PATH_INFO: `/v3/domains/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:50.321200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee53004-cd90-470d-bec8-dd5f6fc5280d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:50.322741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee53004-cd90-470d-bec8-dd5f6fc5280d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:50.324575 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 21/42] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:37:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:50.330826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df158a-de03-419b-847c-6565b14d0bef None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:50.432107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df158a-de03-419b-847c-6565b14d0bef None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:50.467406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55269fc6-c4d9-4771-869a-869b17184177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:50.468078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55269fc6-c4d9-4771-869a-869b17184177 None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:50.468359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55269fc6-c4d9-4771-869a-869b17184177 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:50.468670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55269fc6-c4d9-4771-869a-869b17184177 None admin] PATH_INFO: `/v3/domains/default/users/4ceb679399ff439a9bfe96f9758e269d/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:50.470726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55269fc6-c4d9-4771-869a-869b17184177 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4ceb679399ff439a9bfe96f9758e269d, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:50.472240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55269fc6-c4d9-4771-869a-869b17184177 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:50.489899 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 22/43] 10.222.0.23 () {58 vars in 1361 bytes} [Sat Aug 20 20:37:50 2022] PUT /identity/v3/domains/default/users/4ceb679399ff439a9bfe96f9758e269d/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:37:50.540822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee53004-cd90-470d-bec8-dd5f6fc5280d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:50.652203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55269fc6-c4d9-4771-869a-869b17184177 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:50.657606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee53004-cd90-470d-bec8-dd5f6fc5280d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:50.696879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096baf37-a1f0-435e-a879-4973af26a482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:50.697505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096baf37-a1f0-435e-a879-4973af26a482 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:50.697780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096baf37-a1f0-435e-a879-4973af26a482 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:50.698048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096baf37-a1f0-435e-a879-4973af26a482 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:50.698777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096baf37-a1f0-435e-a879-4973af26a482 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:50.700240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096baf37-a1f0-435e-a879-4973af26a482 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:50.703028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:50.720019 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 22/44] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Aug 20 20:37:50 2022] POST /identity/v3/roles => generated 223 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:37:50.768254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55269fc6-c4d9-4771-869a-869b17184177 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:50.812595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6a46d-f42d-4534-8894-086b5acf1f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:50.813171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6a46d-f42d-4534-8894-086b5acf1f41 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:50.813446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6a46d-f42d-4534-8894-086b5acf1f41 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:50.813713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6a46d-f42d-4534-8894-086b5acf1f41 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:50.815022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6a46d-f42d-4534-8894-086b5acf1f41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:50.816560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6a46d-f42d-4534-8894-086b5acf1f41 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:50.820441 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 23/45] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:37:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:50.824613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096baf37-a1f0-435e-a879-4973af26a482 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:50.892757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:50.942360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096baf37-a1f0-435e-a879-4973af26a482 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:50.980184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865a95e-e794-4233-9b73-4bbb3d9b9b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:50.980826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865a95e-e794-4233-9b73-4bbb3d9b9b60 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:50.981099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865a95e-e794-4233-9b73-4bbb3d9b9b60 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:50.981367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865a95e-e794-4233-9b73-4bbb3d9b9b60 None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:50.982012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865a95e-e794-4233-9b73-4bbb3d9b9b60 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:50.983605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865a95e-e794-4233-9b73-4bbb3d9b9b60 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:51.011003 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 23/46] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:37:50 2022] POST /identity/v3/projects => generated 312 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:37:52.722943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9715bda-d4ff-4425-951c-6526e79734a1 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:52.723467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9715bda-d4ff-4425-951c-6526e79734a1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:52.723467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9715bda-d4ff-4425-951c-6526e79734a1 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:52.724866 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 24/47] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:37:52.730323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae645a32-c41e-4d69-9146-7e2b421d3861 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:52.730746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae645a32-c41e-4d69-9146-7e2b421d3861 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:52.730971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae645a32-c41e-4d69-9146-7e2b421d3861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:52.749747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ae645a32-c41e-4d69-9146-7e2b421d3861 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:37:52.751251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ae645a32-c41e-4d69-9146-7e2b421d3861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:37:52.823919 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 24/48] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:37:53.149901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9715bda-d4ff-4425-951c-6526e79734a1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:53.175685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2908ca0-2c20-457a-b3c5-22b458f1f142 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:53.175958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2908ca0-2c20-457a-b3c5-22b458f1f142 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:53.176221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2908ca0-2c20-457a-b3c5-22b458f1f142 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:53.177597 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 25/49] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:37:53.182526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5666d2-c2e3-4bba-825e-4613427ef7a4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:53.182818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5666d2-c2e3-4bba-825e-4613427ef7a4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:53.183084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5666d2-c2e3-4bba-825e-4613427ef7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:53.200926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7a5666d2-c2e3-4bba-825e-4613427ef7a4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:37:53.202198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7a5666d2-c2e3-4bba-825e-4613427ef7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:37:53.237139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:53.262417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9715bda-d4ff-4425-951c-6526e79734a1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:53.270238 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 26/50] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:37:53.274327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:53.306778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b051de-7c5d-49d0-b47f-efed53196c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:53.307400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b051de-7c5d-49d0-b47f-efed53196c5d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:53.307676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b051de-7c5d-49d0-b47f-efed53196c5d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:53.307921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b051de-7c5d-49d0-b47f-efed53196c5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:53.318776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b051de-7c5d-49d0-b47f-efed53196c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-25b051de-7c5d-49d0-b47f-efed53196c5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:37:53.323395 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:37:53.324201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:37:53.325113 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:37:53.325113 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:37:53.325113 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:37:53.325113 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:37:53.325113 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:37:53.325113 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:37:53.325113 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:37:53.325113 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:37:53.325113 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:37:53.325113 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:37:53.325113 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:37:53.325113 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:37:53.325113 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 25/51] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:37:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:37:53.330441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5666d2-c2e3-4bba-825e-4613427ef7a4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:53.430061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5666d2-c2e3-4bba-825e-4613427ef7a4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:53.464177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661e91f-58fe-4b7a-96af-3d4076a92933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:53.464797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661e91f-58fe-4b7a-96af-3d4076a92933 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:53.465067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661e91f-58fe-4b7a-96af-3d4076a92933 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:53.465329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661e91f-58fe-4b7a-96af-3d4076a92933 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:53.465983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661e91f-58fe-4b7a-96af-3d4076a92933 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:37:53.466277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661e91f-58fe-4b7a-96af-3d4076a92933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:53.467931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661e91f-58fe-4b7a-96af-3d4076a92933 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:53.479839 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 27/52] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:37:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:53.486443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b051de-7c5d-49d0-b47f-efed53196c5d None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:53.593714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b051de-7c5d-49d0-b47f-efed53196c5d None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:53.603223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661e91f-58fe-4b7a-96af-3d4076a92933 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:53.629058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f15c0e-84ae-44e1-b389-8351b84d2358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:53.629691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f15c0e-84ae-44e1-b389-8351b84d2358 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:53.629968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f15c0e-84ae-44e1-b389-8351b84d2358 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:53.630237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f15c0e-84ae-44e1-b389-8351b84d2358 None admin] PATH_INFO: `/v3/users/4ceb679399ff439a9bfe96f9758e269d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:53.631352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f15c0e-84ae-44e1-b389-8351b84d2358 None admin] RBAC: Authorizing `identity:get_user(user_id=4ceb679399ff439a9bfe96f9758e269d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:53.632841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f15c0e-84ae-44e1-b389-8351b84d2358 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:53.634450 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 26/53] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:37:53 2022] GET /identity/v3/users/4ceb679399ff439a9bfe96f9758e269d => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:53.638310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:53.640493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f15c0e-84ae-44e1-b389-8351b84d2358 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:53.681126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:53.706547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661e91f-58fe-4b7a-96af-3d4076a92933 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:53.739496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87525272-9e7d-4f99-bc58-1e2f3744fd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:53.740151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87525272-9e7d-4f99-bc58-1e2f3744fd6a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:53.740448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87525272-9e7d-4f99-bc58-1e2f3744fd6a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:53.740721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87525272-9e7d-4f99-bc58-1e2f3744fd6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:53.740836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f15c0e-84ae-44e1-b389-8351b84d2358 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:53.741904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87525272-9e7d-4f99-bc58-1e2f3744fd6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:53.743350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87525272-9e7d-4f99-bc58-1e2f3744fd6a None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:53.744967 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 28/54] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:37:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:53.750857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87525272-9e7d-4f99-bc58-1e2f3744fd6a None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:53.774116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa4101-53ec-49eb-8e23-e0234e0dd256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:53.774745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa4101-53ec-49eb-8e23-e0234e0dd256 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:53.775020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa4101-53ec-49eb-8e23-e0234e0dd256 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:53.775292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa4101-53ec-49eb-8e23-e0234e0dd256 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:53.776429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa4101-53ec-49eb-8e23-e0234e0dd256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:53.777880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa4101-53ec-49eb-8e23-e0234e0dd256 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:53.779596 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 27/55] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:37:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:37:53.785747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa4101-53ec-49eb-8e23-e0234e0dd256 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:53.854172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87525272-9e7d-4f99-bc58-1e2f3744fd6a None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:53.855780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:53.887287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa4101-53ec-49eb-8e23-e0234e0dd256 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:53.891645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0c53f-5a34-4ce8-b589-22eb628eeaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:53.892251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0c53f-5a34-4ce8-b589-22eb628eeaf1 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:53.892564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0c53f-5a34-4ce8-b589-22eb628eeaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:53.892834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0c53f-5a34-4ce8-b589-22eb628eeaf1 None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:53.893483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0c53f-5a34-4ce8-b589-22eb628eeaf1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:53.894798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:53.894946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0c53f-5a34-4ce8-b589-22eb628eeaf1 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:53.920790 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 29/56] 10.222.0.23 () {60 vars in 1140 bytes} [Sat Aug 20 20:37:53 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:37:53.923102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b6a19-74c1-4b36-9f15-73f4e5a6e969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:53.923710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b6a19-74c1-4b36-9f15-73f4e5a6e969 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:53.923986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b6a19-74c1-4b36-9f15-73f4e5a6e969 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:53.924253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b6a19-74c1-4b36-9f15-73f4e5a6e969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:53.925007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b6a19-74c1-4b36-9f15-73f4e5a6e969 None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.domain.id=default, user.id=4ceb679399ff439a9bfe96f9758e269d) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:37:53.925309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b6a19-74c1-4b36-9f15-73f4e5a6e969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:53.926774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b6a19-74c1-4b36-9f15-73f4e5a6e969 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:53.938278 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 28/57] 10.222.0.23 () {56 vars in 1322 bytes} [Sat Aug 20 20:37:53 2022] GET /identity/v3/role_assignments?user.id=4ceb679399ff439a9bfe96f9758e269d&scope.domain.id=default&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:56.145752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcba61-9aa4-4ad9-9c23-b8198c690d0f None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:56.146584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcba61-9aa4-4ad9-9c23-b8198c690d0f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:56.146584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcba61-9aa4-4ad9-9c23-b8198c690d0f None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:56.148772 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 30/58] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:37:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:37:56.155431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c87a0-5588-4340-929b-a2904179f9f8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:56.156140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c87a0-5588-4340-929b-a2904179f9f8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:56.156289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c87a0-5588-4340-929b-a2904179f9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:56.185187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c68c87a0-5588-4340-929b-a2904179f9f8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:37:56.187064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c68c87a0-5588-4340-929b-a2904179f9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:37:56.268580 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 29/59] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:37:56.597272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcba61-9aa4-4ad9-9c23-b8198c690d0f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:56.713368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcba61-9aa4-4ad9-9c23-b8198c690d0f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:56.750922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbed8db-f254-46ca-ae9c-edf219b3cf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:56.751545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbed8db-f254-46ca-ae9c-edf219b3cf78 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:56.751822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbed8db-f254-46ca-ae9c-edf219b3cf78 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:56.752085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbed8db-f254-46ca-ae9c-edf219b3cf78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:56.753271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbed8db-f254-46ca-ae9c-edf219b3cf78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:56.754730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbed8db-f254-46ca-ae9c-edf219b3cf78 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:56.756276 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 31/60] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:37:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:37:56.764224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c87a0-5588-4340-929b-a2904179f9f8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:37:56.877876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:37:56.881483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c87a0-5588-4340-929b-a2904179f9f8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:37:56.918966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581b593-651c-4f77-ac77-4a35418b3e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:37:56.919672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581b593-651c-4f77-ac77-4a35418b3e74 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:37:56.919993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581b593-651c-4f77-ac77-4a35418b3e74 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:37:56.920263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581b593-651c-4f77-ac77-4a35418b3e74 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:37:56.920958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581b593-651c-4f77-ac77-4a35418b3e74 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:37:56.922570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581b593-651c-4f77-ac77-4a35418b3e74 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:37:56.955953 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 30/61] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:37:56 2022] POST /identity/v3/users => generated 275 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:38:00.354869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf082eb0-633c-481c-a180-ef899ee7b564 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:00.355714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf082eb0-633c-481c-a180-ef899ee7b564 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:00.356120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf082eb0-633c-481c-a180-ef899ee7b564 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:00.357717 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 32/62] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:00.363226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a3723-fc36-40f7-9aa5-1487a082af60 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:00.363690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a3723-fc36-40f7-9aa5-1487a082af60 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:00.363944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a3723-fc36-40f7-9aa5-1487a082af60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:00.393368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8e6a3723-fc36-40f7-9aa5-1487a082af60 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:00.394590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8e6a3723-fc36-40f7-9aa5-1487a082af60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:00.477828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb96dc-9279-433d-8fc3-e6a7be8232f7 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:00.477828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb96dc-9279-433d-8fc3-e6a7be8232f7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:00.477828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb96dc-9279-433d-8fc3-e6a7be8232f7 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:00.477828 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 33/63] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:38:00.484192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efa512-bca4-4cb1-b280-d5c2330cc88c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:00.484915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efa512-bca4-4cb1-b280-d5c2330cc88c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:00.485415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efa512-bca4-4cb1-b280-d5c2330cc88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:00.490222 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 31/64] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:00.505280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a8efa512-bca4-4cb1-b280-d5c2330cc88c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:00.512392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a8efa512-bca4-4cb1-b280-d5c2330cc88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:00.543683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:00.615473 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 34/65] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:00.635761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c9524-60ef-4859-9d1e-c0e0454ff876 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:00.636255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c9524-60ef-4859-9d1e-c0e0454ff876 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:00.636689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c9524-60ef-4859-9d1e-c0e0454ff876 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:00.638260 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 32/66] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:00.644293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbc1cd-34f1-4bf8-b2bc-df9aaaed0d9a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:00.644604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbc1cd-34f1-4bf8-b2bc-df9aaaed0d9a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:00.644875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbc1cd-34f1-4bf8-b2bc-df9aaaed0d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:00.672537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c5bbc1cd-34f1-4bf8-b2bc-df9aaaed0d9a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:00.673733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c5bbc1cd-34f1-4bf8-b2bc-df9aaaed0d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:00.766421 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 35/67] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:00.908088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f426dba-08fb-4fb3-89a8-5108503ce140 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:00.908806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f426dba-08fb-4fb3-89a8-5108503ce140 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:00.909274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f426dba-08fb-4fb3-89a8-5108503ce140 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:00.910812 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 33/68] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:00.916501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524399d3-467b-4db6-aaf5-149292b1e9dc None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:00.917014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524399d3-467b-4db6-aaf5-149292b1e9dc None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:00.917498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524399d3-467b-4db6-aaf5-149292b1e9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:00.936263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-524399d3-467b-4db6-aaf5-149292b1e9dc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:00.937753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-524399d3-467b-4db6-aaf5-149292b1e9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:01.021775 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 36/69] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:01.041186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f426dba-08fb-4fb3-89a8-5108503ce140 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:01.049665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524399d3-467b-4db6-aaf5-149292b1e9dc None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:01.069698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:01.183697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:01.221509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f426dba-08fb-4fb3-89a8-5108503ce140 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:01.230449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524399d3-467b-4db6-aaf5-149292b1e9dc None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:01.275843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01323f50-50cc-42e2-b371-60894b85e023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:01.276715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01323f50-50cc-42e2-b371-60894b85e023 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:01.277133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01323f50-50cc-42e2-b371-60894b85e023 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:01.277416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01323f50-50cc-42e2-b371-60894b85e023 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:01.281705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b8ef9-0cd0-41c7-877d-766ea07d90fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:01.282337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b8ef9-0cd0-41c7-877d-766ea07d90fa None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:01.282765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b8ef9-0cd0-41c7-877d-766ea07d90fa None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:01.283163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b8ef9-0cd0-41c7-877d-766ea07d90fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:01.287907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01323f50-50cc-42e2-b371-60894b85e023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-01323f50-50cc-42e2-b371-60894b85e023 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:01.292598 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:01.293414 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:01.294174 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:01.294174 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:01.294174 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:01.294174 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:01.294174 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:01.294174 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:01.294174 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:01.294174 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:01.294174 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:01.294174 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:01.294174 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:01.294174 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:01.294174 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 37/70] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:38:01.295743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f213017-3561-40a7-a09f-f5710b2be446 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:01.296085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f213017-3561-40a7-a09f-f5710b2be446 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:01.296353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f213017-3561-40a7-a09f-f5710b2be446 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:01.297731 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 38/71] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:01.299813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b8ef9-0cd0-41c7-877d-766ea07d90fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:01.301170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f213017-3561-40a7-a09f-f5710b2be446 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-ac2b8ef9-0cd0-41c7-877d-766ea07d90fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:01.304587 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:01.305474 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:01.306213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:01.306213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:01.306213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:01.306213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:01.306213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:01.306213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:01.306213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:01.306213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:01.306213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:01.306213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:01.306213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:01.306213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:01.306213 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 34/72] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:38:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:01.306834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b8ef9-0cd0-41c7-877d-766ea07d90fa None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:01.383183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:01.411360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b8ef9-0cd0-41c7-877d-766ea07d90fa None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:01.412796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f213017-3561-40a7-a09f-f5710b2be446 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:01.453507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a45abb-9fd6-4d03-b2d8-cd420990352a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:01.453712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4fbaf-d1b8-4936-a9b5-cb86fcc42f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:01.454106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a45abb-9fd6-4d03-b2d8-cd420990352a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:01.454377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a45abb-9fd6-4d03-b2d8-cd420990352a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:01.454466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4fbaf-d1b8-4936-a9b5-cb86fcc42f00 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:01.454639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a45abb-9fd6-4d03-b2d8-cd420990352a None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:01.454841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4fbaf-d1b8-4936-a9b5-cb86fcc42f00 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:01.455174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4fbaf-d1b8-4936-a9b5-cb86fcc42f00 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:01.455289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a45abb-9fd6-4d03-b2d8-cd420990352a None admin] RBAC: Adding query filter params (name=member) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:01.455592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a45abb-9fd6-4d03-b2d8-cd420990352a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:01.457191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a45abb-9fd6-4d03-b2d8-cd420990352a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:01.465415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4fbaf-d1b8-4936-a9b5-cb86fcc42f00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:01.465678 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 35/73] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:01.467107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6dc9e-7bba-4fdb-8d7b-203ae4809b75 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:01.467393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6dc9e-7bba-4fdb-8d7b-203ae4809b75 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:01.467657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6dc9e-7bba-4fdb-8d7b-203ae4809b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-a6c4fbaf-d1b8-4936-a9b5-cb86fcc42f00 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:01.470251 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:01.471462 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 39/74] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:38:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:01.472244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4fbaf-d1b8-4936-a9b5-cb86fcc42f00 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:01.486653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9fc6dc9e-7bba-4fdb-8d7b-203ae4809b75 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:01.487847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9fc6dc9e-7bba-4fdb-8d7b-203ae4809b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:01.489839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:01.555749 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 36/75] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:01.556812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6dc9e-7bba-4fdb-8d7b-203ae4809b75 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:01.585779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4fbaf-d1b8-4936-a9b5-cb86fcc42f00 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:01.616402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:01.621539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accdfbd-6848-4116-93c7-fc268dc4852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:01.622581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accdfbd-6848-4116-93c7-fc268dc4852a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:01.622854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accdfbd-6848-4116-93c7-fc268dc4852a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:01.623115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accdfbd-6848-4116-93c7-fc268dc4852a None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:01.623773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accdfbd-6848-4116-93c7-fc268dc4852a None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:01.624067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accdfbd-6848-4116-93c7-fc268dc4852a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:01.625491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accdfbd-6848-4116-93c7-fc268dc4852a None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:01.638803 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 40/76] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:01.638803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accdfbd-6848-4116-93c7-fc268dc4852a None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:01.664515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6dc9e-7bba-4fdb-8d7b-203ae4809b75 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:01.702339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7209bde-352c-476d-81b8-4cbabaf8f754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:01.702982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7209bde-352c-476d-81b8-4cbabaf8f754 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:01.703256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7209bde-352c-476d-81b8-4cbabaf8f754 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:01.703528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7209bde-352c-476d-81b8-4cbabaf8f754 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:01.710661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:01.714999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7209bde-352c-476d-81b8-4cbabaf8f754 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-b7209bde-352c-476d-81b8-4cbabaf8f754 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:01.720091 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:01.721134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:01.721987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:01.721987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:01.721987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:01.721987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:01.721987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:01.721987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:01.721987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:01.721987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:01.721987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:01.721987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:01.721987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:01.721987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:01.721987 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 37/77] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:01.722711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7209bde-352c-476d-81b8-4cbabaf8f754 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:01.748739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accdfbd-6848-4116-93c7-fc268dc4852a None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:01.784181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da38bd-1aaa-470e-93bf-83b6d4340b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:01.784840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da38bd-1aaa-470e-93bf-83b6d4340b33 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:01.785111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da38bd-1aaa-470e-93bf-83b6d4340b33 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:01.785379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da38bd-1aaa-470e-93bf-83b6d4340b33 None admin] PATH_INFO: `/v3/users/99df9e441d1e495695098147ca4290d9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:01.800893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da38bd-1aaa-470e-93bf-83b6d4340b33 None admin] RBAC: Authorizing `identity:get_user(user_id=99df9e441d1e495695098147ca4290d9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:01.802359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da38bd-1aaa-470e-93bf-83b6d4340b33 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:01.808815 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 41/78] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:01 2022] GET /identity/v3/users/99df9e441d1e495695098147ca4290d9 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:01.808815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da38bd-1aaa-470e-93bf-83b6d4340b33 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:01.832107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7209bde-352c-476d-81b8-4cbabaf8f754 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:01.869193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4ba62-e319-4154-a51a-7edeae2efb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:01.869363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:01.870204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4ba62-e319-4154-a51a-7edeae2efb1f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:01.870540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4ba62-e319-4154-a51a-7edeae2efb1f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:01.870810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4ba62-e319-4154-a51a-7edeae2efb1f None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:01.871473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4ba62-e319-4154-a51a-7edeae2efb1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:01.871820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4ba62-e319-4154-a51a-7edeae2efb1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:01.873918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4ba62-e319-4154-a51a-7edeae2efb1f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:01.882560 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 38/79] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:38:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:01.883541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4ba62-e319-4154-a51a-7edeae2efb1f None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:01.922880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da38bd-1aaa-470e-93bf-83b6d4340b33 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:01.945094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:01.957578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f61aa6-f867-414e-8ec0-26977cddb7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:01.958215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f61aa6-f867-414e-8ec0-26977cddb7df None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:01.958581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f61aa6-f867-414e-8ec0-26977cddb7df None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:01.958857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f61aa6-f867-414e-8ec0-26977cddb7df None admin] PATH_INFO: `/v3/users/4ceb679399ff439a9bfe96f9758e269d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:01.959971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f61aa6-f867-414e-8ec0-26977cddb7df None admin] RBAC: Authorizing `identity:get_user(user_id=4ceb679399ff439a9bfe96f9758e269d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:01.961452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f61aa6-f867-414e-8ec0-26977cddb7df None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:01.963023 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 42/80] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:01 2022] GET /identity/v3/users/4ceb679399ff439a9bfe96f9758e269d => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:01.963781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f61aa6-f867-414e-8ec0-26977cddb7df None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:01.987936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4ba62-e319-4154-a51a-7edeae2efb1f None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:02.020947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df2a09-4be6-490b-a760-d654fa2810c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:02.021597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df2a09-4be6-490b-a760-d654fa2810c9 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:02.021813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df2a09-4be6-490b-a760-d654fa2810c9 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:02.022082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df2a09-4be6-490b-a760-d654fa2810c9 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:02.022735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df2a09-4be6-490b-a760-d654fa2810c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:02.023037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df2a09-4be6-490b-a760-d654fa2810c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:02.024433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df2a09-4be6-490b-a760-d654fa2810c9 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:02.032785 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 39/81] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:02.033511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df2a09-4be6-490b-a760-d654fa2810c9 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:02.064864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f61aa6-f867-414e-8ec0-26977cddb7df None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:02.084525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:02.099491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc25c2-0cd0-4598-9649-c45198aaff94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:02.100093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc25c2-0cd0-4598-9649-c45198aaff94 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:02.100405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc25c2-0cd0-4598-9649-c45198aaff94 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:02.100661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc25c2-0cd0-4598-9649-c45198aaff94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:02.133411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df2a09-4be6-490b-a760-d654fa2810c9 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:02.156429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:02.169747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8369ee0-dcb0-4778-b702-863171fb6b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:02.170419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8369ee0-dcb0-4778-b702-863171fb6b35 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:02.170694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8369ee0-dcb0-4778-b702-863171fb6b35 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:02.170795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc25c2-0cd0-4598-9649-c45198aaff94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:02.170957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8369ee0-dcb0-4778-b702-863171fb6b35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:02.172113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8369ee0-dcb0-4778-b702-863171fb6b35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:02.172345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc25c2-0cd0-4598-9649-c45198aaff94 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:02.173565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8369ee0-dcb0-4778-b702-863171fb6b35 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:02.175344 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 40/82] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:38:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:02.177199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8369ee0-dcb0-4778-b702-863171fb6b35 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:02.273618 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 43/83] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:02.274461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc25c2-0cd0-4598-9649-c45198aaff94 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:02.278467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8369ee0-dcb0-4778-b702-863171fb6b35 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:02.295201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:02.312750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc8dcb-fe18-4a55-b025-f048563de293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:02.313389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc8dcb-fe18-4a55-b025-f048563de293 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:02.313660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc8dcb-fe18-4a55-b025-f048563de293 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:02.313930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc8dcb-fe18-4a55-b025-f048563de293 None admin] PATH_INFO: `/v3/users/99df9e441d1e495695098147ca4290d9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:02.315128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc8dcb-fe18-4a55-b025-f048563de293 None admin] RBAC: Authorizing `identity:get_user(user_id=99df9e441d1e495695098147ca4290d9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:02.316582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc8dcb-fe18-4a55-b025-f048563de293 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:02.318319 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 41/84] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:02 2022] GET /identity/v3/users/99df9e441d1e495695098147ca4290d9 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:02.318903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc8dcb-fe18-4a55-b025-f048563de293 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:02.372777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc25c2-0cd0-4598-9649-c45198aaff94 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:02.379375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:02.406862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f117909-d351-4bdd-967d-c22cd09c0bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:02.407461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f117909-d351-4bdd-967d-c22cd09c0bd0 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:02.407730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f117909-d351-4bdd-967d-c22cd09c0bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:02.407992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f117909-d351-4bdd-967d-c22cd09c0bd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:02.420811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc8dcb-fe18-4a55-b025-f048563de293 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:02.455395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e7182-0e38-437f-8304-00ed91a240bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:02.456006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e7182-0e38-437f-8304-00ed91a240bd None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:02.456276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e7182-0e38-437f-8304-00ed91a240bd None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:02.456570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e7182-0e38-437f-8304-00ed91a240bd None admin] PATH_INFO: `/v3/users/99df9e441d1e495695098147ca4290d9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:02.457716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e7182-0e38-437f-8304-00ed91a240bd None admin] RBAC: Authorizing `identity:get_user(user_id=99df9e441d1e495695098147ca4290d9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:02.459164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e7182-0e38-437f-8304-00ed91a240bd None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:02.460834 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 42/85] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:02 2022] GET /identity/v3/users/99df9e441d1e495695098147ca4290d9 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:02.460834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e7182-0e38-437f-8304-00ed91a240bd None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:02.485155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f117909-d351-4bdd-967d-c22cd09c0bd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:02.486646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f117909-d351-4bdd-967d-c22cd09c0bd0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:02.507316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:02.561114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e7182-0e38-437f-8304-00ed91a240bd None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:02.587714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:02.589236 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 44/86] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:02.590143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f117909-d351-4bdd-967d-c22cd09c0bd0 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:02.601466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d2ab9-b66b-4bbb-afba-8b373d673e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:02.602067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d2ab9-b66b-4bbb-afba-8b373d673e44 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:02.602337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d2ab9-b66b-4bbb-afba-8b373d673e44 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:02.602622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d2ab9-b66b-4bbb-afba-8b373d673e44 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:02.603345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d2ab9-b66b-4bbb-afba-8b373d673e44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:02.604827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d2ab9-b66b-4bbb-afba-8b373d673e44 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:02.626272 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 43/87] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:38:02 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:38:02.627082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d2ab9-b66b-4bbb-afba-8b373d673e44 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:02.695514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f117909-d351-4bdd-967d-c22cd09c0bd0 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:02.721325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:02.738350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e587685-a30d-4a81-81c9-fe79220aa0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:02.738992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e587685-a30d-4a81-81c9-fe79220aa0d5 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:02.739260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e587685-a30d-4a81-81c9-fe79220aa0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:02.739522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e587685-a30d-4a81-81c9-fe79220aa0d5 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:02.740710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e587685-a30d-4a81-81c9-fe79220aa0d5 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:02.742177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e587685-a30d-4a81-81c9-fe79220aa0d5 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:02.748867 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 45/88] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:02 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:02.748867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e587685-a30d-4a81-81c9-fe79220aa0d5 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:02.752924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d2ab9-b66b-4bbb-afba-8b373d673e44 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:02.792197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1dc72-b059-4123-8517-33681835b410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:02.792845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1dc72-b059-4123-8517-33681835b410 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:02.793122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1dc72-b059-4123-8517-33681835b410 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:02.793391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1dc72-b059-4123-8517-33681835b410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:02.853357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:02.858063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e587685-a30d-4a81-81c9-fe79220aa0d5 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:02.880236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1dc72-b059-4123-8517-33681835b410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:02.880236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1dc72-b059-4123-8517-33681835b410 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:02.898624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049dd63-f763-4bb5-8ff3-2d7f74d2c6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:02.899236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049dd63-f763-4bb5-8ff3-2d7f74d2c6f1 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:02.899506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049dd63-f763-4bb5-8ff3-2d7f74d2c6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:02.899769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049dd63-f763-4bb5-8ff3-2d7f74d2c6f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:02.973588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:02.987879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049dd63-f763-4bb5-8ff3-2d7f74d2c6f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:02.989378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049dd63-f763-4bb5-8ff3-2d7f74d2c6f1 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:03.007291 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 44/89] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:03.008025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1dc72-b059-4123-8517-33681835b410 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:03.105275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:03.126097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1dc72-b059-4123-8517-33681835b410 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:03.132517 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 46/90] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:03.133453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049dd63-f763-4bb5-8ff3-2d7f74d2c6f1 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:03.174580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97908c-a75c-499e-ba03-f361846d4341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:03.175628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97908c-a75c-499e-ba03-f361846d4341 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:03.176089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97908c-a75c-499e-ba03-f361846d4341 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:03.176585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97908c-a75c-499e-ba03-f361846d4341 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:03.177930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97908c-a75c-499e-ba03-f361846d4341 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:03.179774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97908c-a75c-499e-ba03-f361846d4341 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:03.181985 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 45/91] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:03 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:03.188168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97908c-a75c-499e-ba03-f361846d4341 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:03.248667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:03.262280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049dd63-f763-4bb5-8ff3-2d7f74d2c6f1 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:03.298500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42f74a-19fa-4235-9915-f3ad5782b046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:03.299482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42f74a-19fa-4235-9915-f3ad5782b046 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:03.299942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42f74a-19fa-4235-9915-f3ad5782b046 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:03.300567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42f74a-19fa-4235-9915-f3ad5782b046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:03.301495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42f74a-19fa-4235-9915-f3ad5782b046 None admin] RBAC: Adding query filter params (role.id=5eaf9701f37e480bb87c89f7391d5d3a, scope.project.id=4b6c3729d57640af9e56f5e19081b8b0, user.id=99df9e441d1e495695098147ca4290d9) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:03.301987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42f74a-19fa-4235-9915-f3ad5782b046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:03.303630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42f74a-19fa-4235-9915-f3ad5782b046 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:03.309936 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 47/92] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:03 2022] GET /identity/v3/role_assignments?user.id=99df9e441d1e495695098147ca4290d9&scope.project.id=4b6c3729d57640af9e56f5e19081b8b0&role.id=5eaf9701f37e480bb87c89f7391d5d3a => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:03.311363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42f74a-19fa-4235-9915-f3ad5782b046 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:03.317470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97908c-a75c-499e-ba03-f361846d4341 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:03.390579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e9a43-0b14-4614-b2e6-d8351bcce741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:03.391618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e9a43-0b14-4614-b2e6-d8351bcce741 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:03.392113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e9a43-0b14-4614-b2e6-d8351bcce741 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:03.392622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e9a43-0b14-4614-b2e6-d8351bcce741 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:03.394052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e9a43-0b14-4614-b2e6-d8351bcce741 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:03.395709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e9a43-0b14-4614-b2e6-d8351bcce741 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:03.397693 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 46/93] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:03 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:03.398670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e9a43-0b14-4614-b2e6-d8351bcce741 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:03.426145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:03.497885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42f74a-19fa-4235-9915-f3ad5782b046 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:03.566716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16ad67-50b2-48d7-bea0-2bafa8a2cf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:03.568005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16ad67-50b2-48d7-bea0-2bafa8a2cf33 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:03.571083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16ad67-50b2-48d7-bea0-2bafa8a2cf33 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:03.571780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16ad67-50b2-48d7-bea0-2bafa8a2cf33 None admin] PATH_INFO: `/v3/projects/ca388bda9756467c927de434026dce16` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:03.573170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16ad67-50b2-48d7-bea0-2bafa8a2cf33 None admin] RBAC: Authorizing `identity:get_project(project_id=ca388bda9756467c927de434026dce16)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:03.574947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16ad67-50b2-48d7-bea0-2bafa8a2cf33 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:03.577832 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 48/94] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:03 2022] GET /identity/v3/projects/ca388bda9756467c927de434026dce16 => generated 312 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:03.578958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16ad67-50b2-48d7-bea0-2bafa8a2cf33 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:03.597058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e9a43-0b14-4614-b2e6-d8351bcce741 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:03.622123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:03.637673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc5ad3-f60d-4099-a238-ae1d7abf443f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:03.638679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc5ad3-f60d-4099-a238-ae1d7abf443f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:03.639141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc5ad3-f60d-4099-a238-ae1d7abf443f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:03.639624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc5ad3-f60d-4099-a238-ae1d7abf443f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:03.640648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc5ad3-f60d-4099-a238-ae1d7abf443f None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=4b6c3729d57640af9e56f5e19081b8b0, user.id=4ceb679399ff439a9bfe96f9758e269d) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:03.641141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc5ad3-f60d-4099-a238-ae1d7abf443f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:03.642814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc5ad3-f60d-4099-a238-ae1d7abf443f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:03.653548 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 47/95] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:03 2022] GET /identity/v3/role_assignments?user.id=4ceb679399ff439a9bfe96f9758e269d&scope.project.id=4b6c3729d57640af9e56f5e19081b8b0&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:03.654890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc5ad3-f60d-4099-a238-ae1d7abf443f None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:03.794976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16ad67-50b2-48d7-bea0-2bafa8a2cf33 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:03.828480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:03.858452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc5ad3-f60d-4099-a238-ae1d7abf443f None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:03.880451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d2840-8c8c-418d-af2d-57808eb4ff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:03.881449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d2840-8c8c-418d-af2d-57808eb4ff7b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:03.881912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d2840-8c8c-418d-af2d-57808eb4ff7b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:03.882361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d2840-8c8c-418d-af2d-57808eb4ff7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:03.883727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d2840-8c8c-418d-af2d-57808eb4ff7b None admin] RBAC: Adding query filter params (role.id=f8a2ea13798443c48750e9baa83c7461, scope.project.id=4b6c3729d57640af9e56f5e19081b8b0, user.id=99df9e441d1e495695098147ca4290d9) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:03.884457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d2840-8c8c-418d-af2d-57808eb4ff7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:03.888583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d2840-8c8c-418d-af2d-57808eb4ff7b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:03.897291 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 49/96] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:03 2022] GET /identity/v3/role_assignments?user.id=99df9e441d1e495695098147ca4290d9&scope.project.id=4b6c3729d57640af9e56f5e19081b8b0&role.id=f8a2ea13798443c48750e9baa83c7461 => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:03.923225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0eebb-abc8-4691-bd47-29f2ea3643ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:03.924497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0eebb-abc8-4691-bd47-29f2ea3643ae None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:03.924992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0eebb-abc8-4691-bd47-29f2ea3643ae None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:03.925498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0eebb-abc8-4691-bd47-29f2ea3643ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:03.926438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0eebb-abc8-4691-bd47-29f2ea3643ae None admin] RBAC: Adding query filter params (role.id=5eaf9701f37e480bb87c89f7391d5d3a, scope.project.id=ca388bda9756467c927de434026dce16, user.id=99df9e441d1e495695098147ca4290d9) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:03.927018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0eebb-abc8-4691-bd47-29f2ea3643ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:03.928747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0eebb-abc8-4691-bd47-29f2ea3643ae None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:03.935996 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 48/97] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:03 2022] GET /identity/v3/role_assignments?user.id=99df9e441d1e495695098147ca4290d9&scope.project.id=ca388bda9756467c927de434026dce16&role.id=5eaf9701f37e480bb87c89f7391d5d3a => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:05.712984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bd752-6fd8-4e00-b547-30a2cb449aa2 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:05.713478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bd752-6fd8-4e00-b547-30a2cb449aa2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:05.713478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bd752-6fd8-4e00-b547-30a2cb449aa2 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:05.714798 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 50/98] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:05.725838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1764b1-215f-49bd-9aaf-0a38d86e3f12 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:05.726282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1764b1-215f-49bd-9aaf-0a38d86e3f12 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:05.726485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1764b1-215f-49bd-9aaf-0a38d86e3f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:05.748667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fd1764b1-215f-49bd-9aaf-0a38d86e3f12 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:05.749873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fd1764b1-215f-49bd-9aaf-0a38d86e3f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:05.839055 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 49/99] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:05.923292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324f59e-3503-4bfc-b73e-bd7a29a5fef0 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:05.923595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324f59e-3503-4bfc-b73e-bd7a29a5fef0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:05.923849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324f59e-3503-4bfc-b73e-bd7a29a5fef0 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:05.925481 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 51/100] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:05.931061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08be3e-ab4c-4bb3-95ef-10a31f0e2051 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:05.931349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08be3e-ab4c-4bb3-95ef-10a31f0e2051 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:05.931622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08be3e-ab4c-4bb3-95ef-10a31f0e2051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:05.957293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fc08be3e-ab4c-4bb3-95ef-10a31f0e2051 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:05.958474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fc08be3e-ab4c-4bb3-95ef-10a31f0e2051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:06.052855 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 50/101] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:06.361874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324f59e-3503-4bfc-b73e-bd7a29a5fef0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:06.469158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:06.491639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324f59e-3503-4bfc-b73e-bd7a29a5fef0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:06.543624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e5630-95a3-4359-bdc0-4051652263b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:06.544471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e5630-95a3-4359-bdc0-4051652263b2 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:06.544778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e5630-95a3-4359-bdc0-4051652263b2 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:06.545068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e5630-95a3-4359-bdc0-4051652263b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:06.565288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e5630-95a3-4359-bdc0-4051652263b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-1a3e5630-95a3-4359-bdc0-4051652263b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:06.570101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:06.570919 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:06.571679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:06.571679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:06.571679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:06.571679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:06.571679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:06.571679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:06.571679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:06.571679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:06.571679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:06.571679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:06.571679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:06.571679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:06.571679 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 52/102] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:06.579183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08be3e-ab4c-4bb3-95ef-10a31f0e2051 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:06.590873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:06.713619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ac3a3-c2f9-420c-8da2-afd0a83502bf None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:06.714104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ac3a3-c2f9-420c-8da2-afd0a83502bf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:06.714104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ac3a3-c2f9-420c-8da2-afd0a83502bf None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:06.715680 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 53/103] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:38:06.719955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ac3a3-c2f9-420c-8da2-afd0a83502bf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:06.726541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08be3e-ab4c-4bb3-95ef-10a31f0e2051 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:06.767160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef56399-e579-4d1a-85b8-a43a09980de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:06.767791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef56399-e579-4d1a-85b8-a43a09980de3 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:06.768071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef56399-e579-4d1a-85b8-a43a09980de3 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:06.768340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef56399-e579-4d1a-85b8-a43a09980de3 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:06.769059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef56399-e579-4d1a-85b8-a43a09980de3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:06.769363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef56399-e579-4d1a-85b8-a43a09980de3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:06.770774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef56399-e579-4d1a-85b8-a43a09980de3 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:06.791720 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 51/104] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:06.793167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18b599-bf0b-4878-ad11-6d951cc7f479 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:06.793453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18b599-bf0b-4878-ad11-6d951cc7f479 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:06.793724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18b599-bf0b-4878-ad11-6d951cc7f479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:06.829627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4e18b599-bf0b-4878-ad11-6d951cc7f479 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:06.831843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4e18b599-bf0b-4878-ad11-6d951cc7f479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:06.938358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:06.946571 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 52/105] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:06.948199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18b599-bf0b-4878-ad11-6d951cc7f479 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:06.957475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ac3a3-c2f9-420c-8da2-afd0a83502bf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:07.005411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96e0e2-5ae4-4c2a-a872-168f69711464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:07.006254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96e0e2-5ae4-4c2a-a872-168f69711464 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:07.006589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96e0e2-5ae4-4c2a-a872-168f69711464 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:07.006923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96e0e2-5ae4-4c2a-a872-168f69711464 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:07.008204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96e0e2-5ae4-4c2a-a872-168f69711464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:07.009741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96e0e2-5ae4-4c2a-a872-168f69711464 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:07.011712 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 54/106] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:38:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:07.017291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a290f9-2ccc-4591-b636-9bb7bd40878f None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:07.017627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a290f9-2ccc-4591-b636-9bb7bd40878f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:07.017903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a290f9-2ccc-4591-b636-9bb7bd40878f None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:07.019340 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 55/107] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:07.020170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a290f9-2ccc-4591-b636-9bb7bd40878f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:07.047165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:07.080832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18b599-bf0b-4878-ad11-6d951cc7f479 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:07.123388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def25a0-08c3-47a7-a231-8072c345a77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:07.124337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def25a0-08c3-47a7-a231-8072c345a77f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:07.124657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def25a0-08c3-47a7-a231-8072c345a77f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:07.124933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def25a0-08c3-47a7-a231-8072c345a77f None admin] PATH_INFO: `/v3/users/6866ea48849b4f40961322f484b0df1f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:07.144131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def25a0-08c3-47a7-a231-8072c345a77f None admin] RBAC: Authorizing `identity:get_user(user_id=6866ea48849b4f40961322f484b0df1f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:07.146274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def25a0-08c3-47a7-a231-8072c345a77f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:07.148133 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 53/108] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:06 2022] GET /identity/v3/users/6866ea48849b4f40961322f484b0df1f => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:07.149529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff893c1d-b47b-4807-88e3-8882b7c3bc55 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:07.150152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff893c1d-b47b-4807-88e3-8882b7c3bc55 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:07.150558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff893c1d-b47b-4807-88e3-8882b7c3bc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:07.175711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ff893c1d-b47b-4807-88e3-8882b7c3bc55 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:07.177020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ff893c1d-b47b-4807-88e3-8882b7c3bc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:07.211955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a290f9-2ccc-4591-b636-9bb7bd40878f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:07.255843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:07.265550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00e04e-0ce7-4e29-b35b-156a421bb36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:07.268089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00e04e-0ce7-4e29-b35b-156a421bb36e None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:07.268571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00e04e-0ce7-4e29-b35b-156a421bb36e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:07.269278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00e04e-0ce7-4e29-b35b-156a421bb36e None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:07.270318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00e04e-0ce7-4e29-b35b-156a421bb36e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:07.272811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00e04e-0ce7-4e29-b35b-156a421bb36e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:07.278929 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 54/109] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:07.279944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff893c1d-b47b-4807-88e3-8882b7c3bc55 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:07.317621 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 56/110] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:38:07 2022] POST /identity/v3/projects => generated 302 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:38:07.323260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f017f-bc36-46af-bbbb-e695393c3565 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:07.323833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f017f-bc36-46af-bbbb-e695393c3565 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:07.324319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f017f-bc36-46af-bbbb-e695393c3565 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:07.327148 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 57/111] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:07.335238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f017f-bc36-46af-bbbb-e695393c3565 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:07.435009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff893c1d-b47b-4807-88e3-8882b7c3bc55 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:07.464593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:07.488970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac7137-a686-4778-b415-885e77b414e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:07.490102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac7137-a686-4778-b415-885e77b414e3 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:07.490522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac7137-a686-4778-b415-885e77b414e3 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:07.490926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac7137-a686-4778-b415-885e77b414e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:07.518763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f017f-bc36-46af-bbbb-e695393c3565 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:07.561878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955bb5e-e3fc-4d00-96ab-0ce4e7154dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:07.562515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955bb5e-e3fc-4d00-96ab-0ce4e7154dd6 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:07.562834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955bb5e-e3fc-4d00-96ab-0ce4e7154dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:07.563101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955bb5e-e3fc-4d00-96ab-0ce4e7154dd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:07.569401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:07.577737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955bb5e-e3fc-4d00-96ab-0ce4e7154dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-3955bb5e-e3fc-4d00-96ab-0ce4e7154dd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:07.582480 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:07.583348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 58/112] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:07.585209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdec87e-e6df-4d71-9cb9-5b1fba83b124 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:07.585951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdec87e-e6df-4d71-9cb9-5b1fba83b124 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:07.585951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdec87e-e6df-4d71-9cb9-5b1fba83b124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:07.590480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac7137-a686-4778-b415-885e77b414e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:07.593767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac7137-a686-4778-b415-885e77b414e3 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:07.605948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ebdec87e-e6df-4d71-9cb9-5b1fba83b124 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:07.607253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ebdec87e-e6df-4d71-9cb9-5b1fba83b124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:07.691289 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 59/113] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:07.692570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd92ec4-3956-4683-8790-e935c93c03cf None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:07.692849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd92ec4-3956-4683-8790-e935c93c03cf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:07.693115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd92ec4-3956-4683-8790-e935c93c03cf None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:07.694338 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 60/114] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:07.695015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd92ec4-3956-4683-8790-e935c93c03cf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:07.708671 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 55/115] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:07.709380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac7137-a686-4778-b415-885e77b414e3 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:07.775473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:07.809376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:07.823660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac7137-a686-4778-b415-885e77b414e3 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:07.840392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd92ec4-3956-4683-8790-e935c93c03cf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:07.866523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a316d-a977-4304-adb7-dfb061013f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:07.867216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a316d-a977-4304-adb7-dfb061013f49 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:07.867589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a316d-a977-4304-adb7-dfb061013f49 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:07.867956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a316d-a977-4304-adb7-dfb061013f49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:07.879651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9c0b6-7c08-45c0-af73-6e809bf18753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:07.880615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9c0b6-7c08-45c0-af73-6e809bf18753 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:07.881022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9c0b6-7c08-45c0-af73-6e809bf18753 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:07.881464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9c0b6-7c08-45c0-af73-6e809bf18753 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:07.882341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9c0b6-7c08-45c0-af73-6e809bf18753 None admin] RBAC: Adding query filter params (name=member) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:07.882992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9c0b6-7c08-45c0-af73-6e809bf18753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:07.884889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9c0b6-7c08-45c0-af73-6e809bf18753 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:07.893839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a316d-a977-4304-adb7-dfb061013f49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:07.895351 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 61/116] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:07.896886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8a0a6-9c23-47e2-a2bb-28caabed8b27 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:07.897308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8a0a6-9c23-47e2-a2bb-28caabed8b27 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:07.897728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8a0a6-9c23-47e2-a2bb-28caabed8b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-3e4a316d-a977-4304-adb7-dfb061013f49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:07.906921 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:07.907840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:07.908652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:07.908652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:07.908652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:07.908652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:07.908652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:07.908652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:07.908652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:07.908652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:07.908652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:07.908652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:07.908652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:07.908652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:07.909221 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 56/117] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:38:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:07.915218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a316d-a977-4304-adb7-dfb061013f49 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:07.926581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f1a8a0a6-9c23-47e2-a2bb-28caabed8b27 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:07.927952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f1a8a0a6-9c23-47e2-a2bb-28caabed8b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:08.008251 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 62/118] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:08.009421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8a0a6-9c23-47e2-a2bb-28caabed8b27 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:08.047297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a316d-a977-4304-adb7-dfb061013f49 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:08.101017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c848a-71d3-43a5-bac1-93f4ce7711be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:08.101690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c848a-71d3-43a5-bac1-93f4ce7711be None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:08.101966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c848a-71d3-43a5-bac1-93f4ce7711be None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:08.102237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c848a-71d3-43a5-bac1-93f4ce7711be None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:08.103434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c848a-71d3-43a5-bac1-93f4ce7711be None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:08.105066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c848a-71d3-43a5-bac1-93f4ce7711be None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:08.107179 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 57/119] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:07 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:08.112998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:08.113433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c848a-71d3-43a5-bac1-93f4ce7711be None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:08.126415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8a0a6-9c23-47e2-a2bb-28caabed8b27 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:08.130973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:08.180399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410aa63d-2b68-463f-b68f-9e42d421631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:08.181677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410aa63d-2b68-463f-b68f-9e42d421631e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:08.182182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410aa63d-2b68-463f-b68f-9e42d421631e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:08.182646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410aa63d-2b68-463f-b68f-9e42d421631e None admin] PATH_INFO: `/v3/users/99df9e441d1e495695098147ca4290d9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:08.184093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410aa63d-2b68-463f-b68f-9e42d421631e None admin] RBAC: Authorizing `identity:get_user(user_id=99df9e441d1e495695098147ca4290d9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:08.186391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410aa63d-2b68-463f-b68f-9e42d421631e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:08.188904 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 63/120] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:08 2022] GET /identity/v3/users/99df9e441d1e495695098147ca4290d9 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:08.189856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410aa63d-2b68-463f-b68f-9e42d421631e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:08.235545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c848a-71d3-43a5-bac1-93f4ce7711be None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:08.271825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbc009-1bd4-4683-8a9c-80d976076eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:08.272560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbc009-1bd4-4683-8a9c-80d976076eef None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:08.272862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbc009-1bd4-4683-8a9c-80d976076eef None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:08.273285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbc009-1bd4-4683-8a9c-80d976076eef None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:08.274069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbc009-1bd4-4683-8a9c-80d976076eef None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:08.274304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbc009-1bd4-4683-8a9c-80d976076eef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:08.275948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbc009-1bd4-4683-8a9c-80d976076eef None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:08.291229 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 58/121] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:08.291229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbc009-1bd4-4683-8a9c-80d976076eef None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:08.335412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410aa63d-2b68-463f-b68f-9e42d421631e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:08.370973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:08.380976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:08.384791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42596d1f-cda1-47aa-ae3b-f716ce699772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:08.385875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42596d1f-cda1-47aa-ae3b-f716ce699772 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:08.386374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42596d1f-cda1-47aa-ae3b-f716ce699772 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:08.386987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42596d1f-cda1-47aa-ae3b-f716ce699772 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:08.396647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbc009-1bd4-4683-8a9c-80d976076eef None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:08.402050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42596d1f-cda1-47aa-ae3b-f716ce699772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-42596d1f-cda1-47aa-ae3b-f716ce699772 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:08.408798 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:08.409743 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:08.410553 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:08.410553 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:08.410553 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:08.410553 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:08.410553 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:08.410553 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:08.410553 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:08.410553 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:08.410553 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:08.410553 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:08.410553 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:08.410553 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:08.412527 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 64/122] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:38:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:08.413188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42596d1f-cda1-47aa-ae3b-f716ce699772 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:08.441121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741edfe0-4e32-4a8e-a90b-1859183fe54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:08.441549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741edfe0-4e32-4a8e-a90b-1859183fe54a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:08.441849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741edfe0-4e32-4a8e-a90b-1859183fe54a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:08.442131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741edfe0-4e32-4a8e-a90b-1859183fe54a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:08.442930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741edfe0-4e32-4a8e-a90b-1859183fe54a None admin] RBAC: Adding query filter params (role.id=bdbaea91119244b9aa4a822c613c2bb8, scope.project.id=4b6c3729d57640af9e56f5e19081b8b0, user.id=6866ea48849b4f40961322f484b0df1f) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:08.443268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741edfe0-4e32-4a8e-a90b-1859183fe54a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:08.444876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741edfe0-4e32-4a8e-a90b-1859183fe54a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:08.453103 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 59/123] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:08 2022] GET /identity/v3/role_assignments?user.id=6866ea48849b4f40961322f484b0df1f&scope.project.id=4b6c3729d57640af9e56f5e19081b8b0&role.id=bdbaea91119244b9aa4a822c613c2bb8 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:08.454869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741edfe0-4e32-4a8e-a90b-1859183fe54a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:08.577786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42596d1f-cda1-47aa-ae3b-f716ce699772 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:08.589759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741edfe0-4e32-4a8e-a90b-1859183fe54a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:08.638717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b5370-c50c-4ea0-9b60-2ed0ae801a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:08.639616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b5370-c50c-4ea0-9b60-2ed0ae801a4e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:08.639983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b5370-c50c-4ea0-9b60-2ed0ae801a4e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:08.640283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b5370-c50c-4ea0-9b60-2ed0ae801a4e None admin] PATH_INFO: `/v3/users/4ceb679399ff439a9bfe96f9758e269d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:08.641687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b5370-c50c-4ea0-9b60-2ed0ae801a4e None admin] RBAC: Authorizing `identity:get_user(user_id=4ceb679399ff439a9bfe96f9758e269d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:08.643346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b5370-c50c-4ea0-9b60-2ed0ae801a4e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:08.645569 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 60/124] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:08 2022] GET /identity/v3/users/4ceb679399ff439a9bfe96f9758e269d => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:08.646413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b5370-c50c-4ea0-9b60-2ed0ae801a4e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:08.660002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:08.663304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d795a3-9ce5-47ed-8d1f-344089831d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:08.664745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d795a3-9ce5-47ed-8d1f-344089831d01 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:08.665258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d795a3-9ce5-47ed-8d1f-344089831d01 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:08.667368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d795a3-9ce5-47ed-8d1f-344089831d01 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:08.669021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d795a3-9ce5-47ed-8d1f-344089831d01 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:08.670918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d795a3-9ce5-47ed-8d1f-344089831d01 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:08.673766 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 65/125] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:08 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:08.674863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d795a3-9ce5-47ed-8d1f-344089831d01 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:08.727943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:08.764670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b5370-c50c-4ea0-9b60-2ed0ae801a4e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:08.798145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d795a3-9ce5-47ed-8d1f-344089831d01 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:08.805326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73973c5-d6b1-4d15-afcc-aba2c772fd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:08.805976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73973c5-d6b1-4d15-afcc-aba2c772fd0d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:08.806371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73973c5-d6b1-4d15-afcc-aba2c772fd0d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:08.806665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73973c5-d6b1-4d15-afcc-aba2c772fd0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:08.819613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73973c5-d6b1-4d15-afcc-aba2c772fd0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-b73973c5-d6b1-4d15-afcc-aba2c772fd0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:08.825709 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:08.827261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:08.828114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:08.828114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:08.828114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:08.828114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:08.828114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:08.828114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:08.828114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:08.828114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:08.828114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:08.828114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:08.828114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:08.828114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:08.828114 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 61/126] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:08.828782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73973c5-d6b1-4d15-afcc-aba2c772fd0d None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:08.848118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9c72e-10bc-4298-b402-5e70e765305d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:08.849120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9c72e-10bc-4298-b402-5e70e765305d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:08.849640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9c72e-10bc-4298-b402-5e70e765305d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:08.850104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9c72e-10bc-4298-b402-5e70e765305d None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:08.851004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9c72e-10bc-4298-b402-5e70e765305d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:08.851508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9c72e-10bc-4298-b402-5e70e765305d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:08.853211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9c72e-10bc-4298-b402-5e70e765305d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:08.863389 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 66/127] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:38:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:08.864674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9c72e-10bc-4298-b402-5e70e765305d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:08.922557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:08.971558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73973c5-d6b1-4d15-afcc-aba2c772fd0d None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:09.004295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9c72e-10bc-4298-b402-5e70e765305d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:09.007066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:09.010645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d9906-c7af-4db7-a6fc-dc8dc71684b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:09.011529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d9906-c7af-4db7-a6fc-dc8dc71684b5 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:09.011902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d9906-c7af-4db7-a6fc-dc8dc71684b5 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:09.012273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d9906-c7af-4db7-a6fc-dc8dc71684b5 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:09.014014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d9906-c7af-4db7-a6fc-dc8dc71684b5 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:09.015713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d9906-c7af-4db7-a6fc-dc8dc71684b5 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:09.022257 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 62/128] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:08 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:09.022257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d9906-c7af-4db7-a6fc-dc8dc71684b5 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:09.055209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dd0f5-3f17-48eb-8e1d-b2dfdd2f4cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:09.056118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dd0f5-3f17-48eb-8e1d-b2dfdd2f4cb6 None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:09.056568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dd0f5-3f17-48eb-8e1d-b2dfdd2f4cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:09.056968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dd0f5-3f17-48eb-8e1d-b2dfdd2f4cb6 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0/users/99df9e441d1e495695098147ca4290d9/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:09.059295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2dd0f5-3f17-48eb-8e1d-b2dfdd2f4cb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b6c3729d57640af9e56f5e19081b8b0, user_id=99df9e441d1e495695098147ca4290d9, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:09.061394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2dd0f5-3f17-48eb-8e1d-b2dfdd2f4cb6 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:09.080831 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 67/129] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:38:08 2022] PUT /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0/users/99df9e441d1e495695098147ca4290d9/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:38:09.081917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dd0f5-3f17-48eb-8e1d-b2dfdd2f4cb6 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:09.214382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d9906-c7af-4db7-a6fc-dc8dc71684b5 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:09.223206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dd0f5-3f17-48eb-8e1d-b2dfdd2f4cb6 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:09.261804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f4dec-2a32-4c80-ac55-3a1d57d95dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:09.262417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f4dec-2a32-4c80-ac55-3a1d57d95dd2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:09.262693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f4dec-2a32-4c80-ac55-3a1d57d95dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:09.263010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8c9c8-4520-4571-9b85-de1fd54b23ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:09.263395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f4dec-2a32-4c80-ac55-3a1d57d95dd2 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:09.264067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8c9c8-4520-4571-9b85-de1fd54b23ae None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:09.264172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f4dec-2a32-4c80-ac55-3a1d57d95dd2 None admin] RBAC: Adding query filter params (name=member) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:09.264387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f4dec-2a32-4c80-ac55-3a1d57d95dd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:09.264886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8c9c8-4520-4571-9b85-de1fd54b23ae None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:09.265343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8c9c8-4520-4571-9b85-de1fd54b23ae None admin] PATH_INFO: `/v3/users/99df9e441d1e495695098147ca4290d9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:09.265836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f4dec-2a32-4c80-ac55-3a1d57d95dd2 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:09.267109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8c9c8-4520-4571-9b85-de1fd54b23ae None admin] RBAC: Authorizing `identity:get_user(user_id=99df9e441d1e495695098147ca4290d9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:09.268629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:09.269505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8c9c8-4520-4571-9b85-de1fd54b23ae None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:09.271977 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 68/130] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:09 2022] GET /identity/v3/users/99df9e441d1e495695098147ca4290d9 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:09.273284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8c9c8-4520-4571-9b85-de1fd54b23ae None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:09.278358 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 63/131] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:09.283053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f4dec-2a32-4c80-ac55-3a1d57d95dd2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:09.308933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:09.388337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8c9c8-4520-4571-9b85-de1fd54b23ae None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:09.393245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f4dec-2a32-4c80-ac55-3a1d57d95dd2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:09.428538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7906383-8594-463f-aacc-36af64c83a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:09.429172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7906383-8594-463f-aacc-36af64c83a9b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:09.432393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7906383-8594-463f-aacc-36af64c83a9b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:09.432393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7906383-8594-463f-aacc-36af64c83a9b None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:09.432393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7906383-8594-463f-aacc-36af64c83a9b None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:09.432393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7906383-8594-463f-aacc-36af64c83a9b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:09.433110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d23094-3596-4509-82af-4546bdf713c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:09.433795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d23094-3596-4509-82af-4546bdf713c9 None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:09.434068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d23094-3596-4509-82af-4546bdf713c9 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:09.434224 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 64/132] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:09 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:09.434365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d23094-3596-4509-82af-4546bdf713c9 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0/users/4ceb679399ff439a9bfe96f9758e269d/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:09.435134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7906383-8594-463f-aacc-36af64c83a9b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:09.437219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d23094-3596-4509-82af-4546bdf713c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b6c3729d57640af9e56f5e19081b8b0, user_id=4ceb679399ff439a9bfe96f9758e269d, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:09.438699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d23094-3596-4509-82af-4546bdf713c9 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:09.468548 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 69/133] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:38:09 2022] PUT /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0/users/4ceb679399ff439a9bfe96f9758e269d/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:38:09.469444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d23094-3596-4509-82af-4546bdf713c9 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:09.529279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:09.598670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7906383-8594-463f-aacc-36af64c83a9b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:09.672781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:09.676646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe1b21-cae0-4f01-8374-5a47ebeb5b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:09.677348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe1b21-cae0-4f01-8374-5a47ebeb5b34 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:09.677622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe1b21-cae0-4f01-8374-5a47ebeb5b34 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:09.677916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe1b21-cae0-4f01-8374-5a47ebeb5b34 None admin] PATH_INFO: `/v3/users/99df9e441d1e495695098147ca4290d9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:09.679819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe1b21-cae0-4f01-8374-5a47ebeb5b34 None admin] RBAC: Authorizing `identity:get_user(user_id=99df9e441d1e495695098147ca4290d9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:09.681313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe1b21-cae0-4f01-8374-5a47ebeb5b34 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:09.683253 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 65/134] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:09 2022] GET /identity/v3/users/99df9e441d1e495695098147ca4290d9 => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:09.690515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe1b21-cae0-4f01-8374-5a47ebeb5b34 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:09.698571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d23094-3596-4509-82af-4546bdf713c9 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:09.746630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f9c69-3825-422a-b502-69db93031185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:09.748333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f9c69-3825-422a-b502-69db93031185 None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:09.748644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f9c69-3825-422a-b502-69db93031185 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:09.748915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f9c69-3825-422a-b502-69db93031185 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0/users/99df9e441d1e495695098147ca4290d9/roles/f8a2ea13798443c48750e9baa83c7461` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:09.751978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f9c69-3825-422a-b502-69db93031185 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b6c3729d57640af9e56f5e19081b8b0, user_id=99df9e441d1e495695098147ca4290d9, role_id=f8a2ea13798443c48750e9baa83c7461)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:09.753874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f9c69-3825-422a-b502-69db93031185 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:09.775674 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 70/135] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:38:09 2022] PUT /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0/users/99df9e441d1e495695098147ca4290d9/roles/f8a2ea13798443c48750e9baa83c7461 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:38:09.822901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe1b21-cae0-4f01-8374-5a47ebeb5b34 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:09.862382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f7edd-ea96-456b-acb7-85e81f7012ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:09.863031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f7edd-ea96-456b-acb7-85e81f7012ef None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:09.863306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f7edd-ea96-456b-acb7-85e81f7012ef None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:09.863574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f7edd-ea96-456b-acb7-85e81f7012ef None admin] PATH_INFO: `/v3/projects/ca388bda9756467c927de434026dce16` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:09.864854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f7edd-ea96-456b-acb7-85e81f7012ef None admin] RBAC: Authorizing `identity:get_project(project_id=ca388bda9756467c927de434026dce16)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:09.866398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f7edd-ea96-456b-acb7-85e81f7012ef None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:09.868222 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 66/136] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:09 2022] GET /identity/v3/projects/ca388bda9756467c927de434026dce16 => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:09.874567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f9c69-3825-422a-b502-69db93031185 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:10.026302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f9c69-3825-422a-b502-69db93031185 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:10.064135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d72f3-eb0e-41de-a285-dbc3e9988b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:10.064919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d72f3-eb0e-41de-a285-dbc3e9988b16 None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:10.065196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d72f3-eb0e-41de-a285-dbc3e9988b16 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:10.065464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d72f3-eb0e-41de-a285-dbc3e9988b16 None admin] PATH_INFO: `/v3/projects/ca388bda9756467c927de434026dce16/users/99df9e441d1e495695098147ca4290d9/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:10.067439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d72f3-eb0e-41de-a285-dbc3e9988b16 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca388bda9756467c927de434026dce16, user_id=99df9e441d1e495695098147ca4290d9, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:10.069005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d72f3-eb0e-41de-a285-dbc3e9988b16 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:10.089737 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 71/137] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:38:09 2022] PUT /identity/v3/projects/ca388bda9756467c927de434026dce16/users/99df9e441d1e495695098147ca4290d9/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:38:10.264066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa1738-afff-42e0-bba5-83ef783e102c None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:10.264442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa1738-afff-42e0-bba5-83ef783e102c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:10.264675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa1738-afff-42e0-bba5-83ef783e102c None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:10.266085 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 67/138] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:10.271919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f616e-8bef-4003-b3d9-492c2d89c999 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:10.272193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f616e-8bef-4003-b3d9-492c2d89c999 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:10.272492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f616e-8bef-4003-b3d9-492c2d89c999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:10.295842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8c0f616e-8bef-4003-b3d9-492c2d89c999 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:10.297064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8c0f616e-8bef-4003-b3d9-492c2d89c999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:10.306306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:10.380872 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 72/139] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:11.046038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa1738-afff-42e0-bba5-83ef783e102c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:11.049083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:11.162149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa1738-afff-42e0-bba5-83ef783e102c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:11.199177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d561b2-a32b-47e6-9186-c9234629f3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:11.200230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d561b2-a32b-47e6-9186-c9234629f3e5 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:11.200761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d561b2-a32b-47e6-9186-c9234629f3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:11.201248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d561b2-a32b-47e6-9186-c9234629f3e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:11.202584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d561b2-a32b-47e6-9186-c9234629f3e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:11.204225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d561b2-a32b-47e6-9186-c9234629f3e5 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:11.206398 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 68/140] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:38:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:11.211683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f616e-8bef-4003-b3d9-492c2d89c999 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:11.372670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f616e-8bef-4003-b3d9-492c2d89c999 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:11.423788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd6a07-58ac-45d8-a0dc-042ce652e9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:11.424812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd6a07-58ac-45d8-a0dc-042ce652e9ea None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:11.425309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd6a07-58ac-45d8-a0dc-042ce652e9ea None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:11.425773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd6a07-58ac-45d8-a0dc-042ce652e9ea None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:11.426608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd6a07-58ac-45d8-a0dc-042ce652e9ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:11.428249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd6a07-58ac-45d8-a0dc-042ce652e9ea None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:11.459969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:11.467562 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 73/141] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:38:11 2022] POST /identity/v3/users => generated 283 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:38:11.937023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0932f1-7a46-4500-9a40-62f9ef3020a4 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:11.937435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0932f1-7a46-4500-9a40-62f9ef3020a4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:11.937558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0932f1-7a46-4500-9a40-62f9ef3020a4 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:11.938911 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 69/142] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:11.944433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfee85-722d-46ff-a759-ee78a7db5394 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:11.944880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfee85-722d-46ff-a759-ee78a7db5394 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:11.945303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfee85-722d-46ff-a759-ee78a7db5394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:11.973709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e1dfee85-722d-46ff-a759-ee78a7db5394 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:11.975621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e1dfee85-722d-46ff-a759-ee78a7db5394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:12.057072 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 74/143] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:12.219829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e993755-610f-4bc3-8f29-5db205dc0f6a None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:12.221131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e993755-610f-4bc3-8f29-5db205dc0f6a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:12.221644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e993755-610f-4bc3-8f29-5db205dc0f6a None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:12.223212 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 70/144] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:12.240831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49adb9-2ee0-4fea-a54e-52cc0ba657a2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:12.241847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49adb9-2ee0-4fea-a54e-52cc0ba657a2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:12.242312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49adb9-2ee0-4fea-a54e-52cc0ba657a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:12.267532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0d49adb9-2ee0-4fea-a54e-52cc0ba657a2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:12.268727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0d49adb9-2ee0-4fea-a54e-52cc0ba657a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:12.373854 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 75/145] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:12.607072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e993755-610f-4bc3-8f29-5db205dc0f6a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:12.704971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:12.737091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e993755-610f-4bc3-8f29-5db205dc0f6a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:12.776131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a267d2-5f2b-4f4a-a3d4-3a48098fa910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:12.776790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a267d2-5f2b-4f4a-a3d4-3a48098fa910 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:12.777059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a267d2-5f2b-4f4a-a3d4-3a48098fa910 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:12.777322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a267d2-5f2b-4f4a-a3d4-3a48098fa910 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:12.788862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a267d2-5f2b-4f4a-a3d4-3a48098fa910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-b4a267d2-5f2b-4f4a-a3d4-3a48098fa910 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:12.793765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:12.794698 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:12.795496 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:12.795496 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:12.795496 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:12.795496 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:12.795496 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:12.795496 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:12.795496 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:12.795496 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:12.795496 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:12.795496 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:12.795496 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:12.795496 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:12.795496 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 71/146] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:12.800870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49adb9-2ee0-4fea-a54e-52cc0ba657a2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:12.866866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6bf03-0202-447a-a583-b8dbb5679b8d None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:12.867048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6bf03-0202-447a-a583-b8dbb5679b8d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:12.867297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6bf03-0202-447a-a583-b8dbb5679b8d None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:12.868908 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 72/147] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:38:12.876677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3318611-42a2-4df3-9382-3a32732db6b2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:12.877408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3318611-42a2-4df3-9382-3a32732db6b2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:12.878280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3318611-42a2-4df3-9382-3a32732db6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:12.904269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:12.905774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d3318611-42a2-4df3-9382-3a32732db6b2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:12.907033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d3318611-42a2-4df3-9382-3a32732db6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:12.963837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49adb9-2ee0-4fea-a54e-52cc0ba657a2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:13.026679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22013869-742c-428f-9375-2f49992f8558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:13.027300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22013869-742c-428f-9375-2f49992f8558 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:13.027621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22013869-742c-428f-9375-2f49992f8558 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:13.027883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22013869-742c-428f-9375-2f49992f8558 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:13.028910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22013869-742c-428f-9375-2f49992f8558 None admin] RBAC: Adding query filter params (name=reader) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:13.029358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22013869-742c-428f-9375-2f49992f8558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:13.031189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22013869-742c-428f-9375-2f49992f8558 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:13.044205 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 76/148] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:13.044205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22013869-742c-428f-9375-2f49992f8558 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:13.051154 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 73/149] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:13.052670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203cef8-f09e-4036-afc3-a12312ecfd1c None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:13.053190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203cef8-f09e-4036-afc3-a12312ecfd1c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:13.053666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203cef8-f09e-4036-afc3-a12312ecfd1c None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:13.055152 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 74/150] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:13.056020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203cef8-f09e-4036-afc3-a12312ecfd1c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:13.135392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:13.174991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22013869-742c-428f-9375-2f49992f8558 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:13.202248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:13.209194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203cef8-f09e-4036-afc3-a12312ecfd1c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:13.218572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed105418-7be3-41ab-89f0-3109863b7231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:13.219427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105418-7be3-41ab-89f0-3109863b7231 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:13.219860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105418-7be3-41ab-89f0-3109863b7231 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:13.223435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105418-7be3-41ab-89f0-3109863b7231 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:13.236590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed105418-7be3-41ab-89f0-3109863b7231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-ed105418-7be3-41ab-89f0-3109863b7231 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:13.244706 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:13.246204 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:13.247294 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:13.247294 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:13.247294 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:13.247294 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:13.247294 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:13.247294 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:13.247294 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:13.247294 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:13.247294 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:13.247294 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:13.247294 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:13.247294 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:13.250870 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 77/151] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:13.253371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde93608-c0c5-49a4-947a-b3d782a6f6f9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:13.253812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde93608-c0c5-49a4-947a-b3d782a6f6f9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:13.254217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde93608-c0c5-49a4-947a-b3d782a6f6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:13.275277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dde93608-c0c5-49a4-947a-b3d782a6f6f9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:13.277173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dde93608-c0c5-49a4-947a-b3d782a6f6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:13.300560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee344be-260d-4c89-ae69-434d07bd04d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:13.302952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee344be-260d-4c89-ae69-434d07bd04d5 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:13.304661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee344be-260d-4c89-ae69-434d07bd04d5 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:13.305082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee344be-260d-4c89-ae69-434d07bd04d5 None admin] PATH_INFO: `/v3/users/6866ea48849b4f40961322f484b0df1f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:13.308566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee344be-260d-4c89-ae69-434d07bd04d5 None admin] RBAC: Authorizing `identity:get_user(user_id=6866ea48849b4f40961322f484b0df1f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:13.314363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee344be-260d-4c89-ae69-434d07bd04d5 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:13.316311 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 75/152] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:13 2022] GET /identity/v3/users/6866ea48849b4f40961322f484b0df1f => generated 289 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:13.321876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee344be-260d-4c89-ae69-434d07bd04d5 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:13.427506 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 78/153] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:13.428319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde93608-c0c5-49a4-947a-b3d782a6f6f9 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:13.469773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee344be-260d-4c89-ae69-434d07bd04d5 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:13.511241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:13.521261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b66d3a-eadf-4804-ba18-7528eac1473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:13.522168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b66d3a-eadf-4804-ba18-7528eac1473c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:13.522654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b66d3a-eadf-4804-ba18-7528eac1473c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:13.523101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b66d3a-eadf-4804-ba18-7528eac1473c None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:13.523979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b66d3a-eadf-4804-ba18-7528eac1473c None admin] RBAC: Adding query filter params (name=member) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:13.524486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b66d3a-eadf-4804-ba18-7528eac1473c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:13.526032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b66d3a-eadf-4804-ba18-7528eac1473c None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:13.536341 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 76/154] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:13.537909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c982d-0e8b-4c6f-a788-f18468c7d655 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:13.538400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c982d-0e8b-4c6f-a788-f18468c7d655 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:13.538858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c982d-0e8b-4c6f-a788-f18468c7d655 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:13.540238 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 77/155] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:13.541138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c982d-0e8b-4c6f-a788-f18468c7d655 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:13.546718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde93608-c0c5-49a4-947a-b3d782a6f6f9 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:13.564760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:13.592399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e44360-4ee1-48e1-b430-d146818fc4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:13.592399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e44360-4ee1-48e1-b430-d146818fc4f2 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:13.592399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e44360-4ee1-48e1-b430-d146818fc4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:13.592399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e44360-4ee1-48e1-b430-d146818fc4f2 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:13.592817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e44360-4ee1-48e1-b430-d146818fc4f2 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:13.592817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e44360-4ee1-48e1-b430-d146818fc4f2 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:13.593496 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 79/156] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:13 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:13.594771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae96500-4c62-4aec-bd58-8dbfe1f072e5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:13.595057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae96500-4c62-4aec-bd58-8dbfe1f072e5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:13.595366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae96500-4c62-4aec-bd58-8dbfe1f072e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:13.615244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3ae96500-4c62-4aec-bd58-8dbfe1f072e5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:13.616575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3ae96500-4c62-4aec-bd58-8dbfe1f072e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:13.734810 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 80/157] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:13.735746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae96500-4c62-4aec-bd58-8dbfe1f072e5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:13.760789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c982d-0e8b-4c6f-a788-f18468c7d655 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:13.880410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a15e86-e169-47c4-bbe8-f59851eaca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:13.880410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a15e86-e169-47c4-bbe8-f59851eaca16 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:13.880410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a15e86-e169-47c4-bbe8-f59851eaca16 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:13.880410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a15e86-e169-47c4-bbe8-f59851eaca16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:13.893213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae96500-4c62-4aec-bd58-8dbfe1f072e5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:13.896343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a15e86-e169-47c4-bbe8-f59851eaca16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-c7a15e86-e169-47c4-bbe8-f59851eaca16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:13.901195 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:13.902188 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:13.903031 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:13.903031 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:13.903031 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:13.903031 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:13.903031 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:13.903031 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:13.903031 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:13.903031 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:13.903031 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:13.903031 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:13.903031 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:13.903031 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:13.903031 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 78/158] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:38:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:13.903897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a15e86-e169-47c4-bbe8-f59851eaca16 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:13.928396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:13.949661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ed949-2e00-47f8-8378-63661d1a831e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:13.950680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ed949-2e00-47f8-8378-63661d1a831e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:13.951140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ed949-2e00-47f8-8378-63661d1a831e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:13.951598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ed949-2e00-47f8-8378-63661d1a831e None admin] PATH_INFO: `/v3/users/99df9e441d1e495695098147ca4290d9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:13.952936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ed949-2e00-47f8-8378-63661d1a831e None admin] RBAC: Authorizing `identity:get_user(user_id=99df9e441d1e495695098147ca4290d9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:13.954759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ed949-2e00-47f8-8378-63661d1a831e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:13.956622 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 81/159] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:13 2022] GET /identity/v3/users/99df9e441d1e495695098147ca4290d9 => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:13.957568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ed949-2e00-47f8-8378-63661d1a831e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:13.990889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:14.074628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a15e86-e169-47c4-bbe8-f59851eaca16 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:14.139451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928b27d-7832-4138-b4eb-686abdcd0854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:14.140170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928b27d-7832-4138-b4eb-686abdcd0854 None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:14.140496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928b27d-7832-4138-b4eb-686abdcd0854 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:14.140799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928b27d-7832-4138-b4eb-686abdcd0854 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0/users/6866ea48849b4f40961322f484b0df1f/roles/bdbaea91119244b9aa4a822c613c2bb8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:14.142859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928b27d-7832-4138-b4eb-686abdcd0854 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b6c3729d57640af9e56f5e19081b8b0, user_id=6866ea48849b4f40961322f484b0df1f, role_id=bdbaea91119244b9aa4a822c613c2bb8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:14.144448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928b27d-7832-4138-b4eb-686abdcd0854 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:14.165317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ed949-2e00-47f8-8378-63661d1a831e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:14.167026 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 79/160] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:38:13 2022] PUT /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0/users/6866ea48849b4f40961322f484b0df1f/roles/bdbaea91119244b9aa4a822c613c2bb8 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:38:14.167916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928b27d-7832-4138-b4eb-686abdcd0854 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:14.289208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29c1b0-9df1-4a9a-8911-823fd3dc7e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:14.289746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29c1b0-9df1-4a9a-8911-823fd3dc7e0d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:14.290011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29c1b0-9df1-4a9a-8911-823fd3dc7e0d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:14.290329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29c1b0-9df1-4a9a-8911-823fd3dc7e0d None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:14.291008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29c1b0-9df1-4a9a-8911-823fd3dc7e0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:14.291301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29c1b0-9df1-4a9a-8911-823fd3dc7e0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:14.292752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29c1b0-9df1-4a9a-8911-823fd3dc7e0d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:14.312759 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 82/161] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:38:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:14.313657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29c1b0-9df1-4a9a-8911-823fd3dc7e0d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:14.376594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928b27d-7832-4138-b4eb-686abdcd0854 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:14.393768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:14.398952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:14.460360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11c210-a367-4c25-8c84-d04a65360598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:14.461026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11c210-a367-4c25-8c84-d04a65360598 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:14.461302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11c210-a367-4c25-8c84-d04a65360598 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:14.461554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11c210-a367-4c25-8c84-d04a65360598 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:14.490067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11c210-a367-4c25-8c84-d04a65360598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-0f11c210-a367-4c25-8c84-d04a65360598 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:14.495052 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:14.495970 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:14.496770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:14.496770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:14.496770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:14.496770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:14.496770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:14.496770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:14.496770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:14.496770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:14.496770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:14.496770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:14.496770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:14.496770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:14.496770 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 80/162] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:38:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:14.497584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11c210-a367-4c25-8c84-d04a65360598 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:14.512268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29c1b0-9df1-4a9a-8911-823fd3dc7e0d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:14.580760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fdd87-51ab-4b2b-9d2a-08628760489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:14.581296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fdd87-51ab-4b2b-9d2a-08628760489d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:14.581565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fdd87-51ab-4b2b-9d2a-08628760489d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:14.581828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fdd87-51ab-4b2b-9d2a-08628760489d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:14.693128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11c210-a367-4c25-8c84-d04a65360598 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:14.716687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fdd87-51ab-4b2b-9d2a-08628760489d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:14.718217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fdd87-51ab-4b2b-9d2a-08628760489d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:14.762998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2059f-f277-4774-90c3-0980d452eb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:14.763657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2059f-f277-4774-90c3-0980d452eb20 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:14.763951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2059f-f277-4774-90c3-0980d452eb20 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:14.764224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2059f-f277-4774-90c3-0980d452eb20 None admin] PATH_INFO: `/v3/users/99df9e441d1e495695098147ca4290d9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:14.765486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f2059f-f277-4774-90c3-0980d452eb20 None admin] RBAC: Authorizing `identity:get_user(user_id=99df9e441d1e495695098147ca4290d9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:14.766944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f2059f-f277-4774-90c3-0980d452eb20 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:14.768569 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 81/163] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:14 2022] GET /identity/v3/users/99df9e441d1e495695098147ca4290d9 => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:14.769403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2059f-f277-4774-90c3-0980d452eb20 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:14.797538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:14.832909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:14.908870 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 83/164] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:14.913290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fdd87-51ab-4b2b-9d2a-08628760489d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:14.947541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2059f-f277-4774-90c3-0980d452eb20 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:15.039104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83142aa3-2f9d-4f38-bf15-49142d5d7102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:15.040271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83142aa3-2f9d-4f38-bf15-49142d5d7102 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:15.040793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83142aa3-2f9d-4f38-bf15-49142d5d7102 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:15.041291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83142aa3-2f9d-4f38-bf15-49142d5d7102 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:15.060622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83142aa3-2f9d-4f38-bf15-49142d5d7102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-83142aa3-2f9d-4f38-bf15-49142d5d7102 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:15.068415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:15.069544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:15.070321 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:15.070321 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:15.070321 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:15.070321 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:15.070321 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:15.070321 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:15.070321 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:15.070321 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:15.070321 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:15.070321 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:15.070321 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:15.070321 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:15.074573 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 82/165] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:15.077271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83142aa3-2f9d-4f38-bf15-49142d5d7102 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:15.144423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fdd87-51ab-4b2b-9d2a-08628760489d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:15.221898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ded9a-825b-4cd4-bacd-4c01b3586614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:15.222527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ded9a-825b-4cd4-bacd-4c01b3586614 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:15.222898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ded9a-825b-4cd4-bacd-4c01b3586614 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:15.223194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ded9a-825b-4cd4-bacd-4c01b3586614 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:15.223866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ded9a-825b-4cd4-bacd-4c01b3586614 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:15.224169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ded9a-825b-4cd4-bacd-4c01b3586614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:15.225624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ded9a-825b-4cd4-bacd-4c01b3586614 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:15.234835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:15.240265 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 84/166] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:15.241116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ded9a-825b-4cd4-bacd-4c01b3586614 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:15.353805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:15.370271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83142aa3-2f9d-4f38-bf15-49142d5d7102 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:15.380901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ded9a-825b-4cd4-bacd-4c01b3586614 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:15.429152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b509c-0480-4066-813a-842439bb299a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:15.429696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b509c-0480-4066-813a-842439bb299a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:15.429971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b509c-0480-4066-813a-842439bb299a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:15.430217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b509c-0480-4066-813a-842439bb299a None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:15.431553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b509c-0480-4066-813a-842439bb299a None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:15.433473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b509c-0480-4066-813a-842439bb299a None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:15.435858 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 85/167] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:15 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:15.437184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b509c-0480-4066-813a-842439bb299a None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:15.477653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442ee04-7716-4bbc-a858-bb6a1275c19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:15.478971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442ee04-7716-4bbc-a858-bb6a1275c19b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:15.479439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442ee04-7716-4bbc-a858-bb6a1275c19b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:15.479916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442ee04-7716-4bbc-a858-bb6a1275c19b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:15.566754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:15.631688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b509c-0480-4066-813a-842439bb299a None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:15.674963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442ee04-7716-4bbc-a858-bb6a1275c19b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:15.676820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442ee04-7716-4bbc-a858-bb6a1275c19b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:15.693272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75ec05-b5ec-4572-af6e-09786bbd7abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:15.693811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75ec05-b5ec-4572-af6e-09786bbd7abb None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:15.694082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75ec05-b5ec-4572-af6e-09786bbd7abb None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:15.694342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75ec05-b5ec-4572-af6e-09786bbd7abb None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:15.695000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75ec05-b5ec-4572-af6e-09786bbd7abb None admin] RBAC: Adding query filter params (name=member) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:15.695291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75ec05-b5ec-4572-af6e-09786bbd7abb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:15.700419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75ec05-b5ec-4572-af6e-09786bbd7abb None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:15.711662 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 86/168] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:15.712548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75ec05-b5ec-4572-af6e-09786bbd7abb None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:15.867746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:15.876087 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 83/169] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 800 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:15.876766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442ee04-7716-4bbc-a858-bb6a1275c19b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:15.930351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75ec05-b5ec-4572-af6e-09786bbd7abb None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:15.991060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:16.005448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f00b7-6832-4b21-9b5e-eb2a98d34c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:16.006090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f00b7-6832-4b21-9b5e-eb2a98d34c0e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:16.006358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f00b7-6832-4b21-9b5e-eb2a98d34c0e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:16.006622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f00b7-6832-4b21-9b5e-eb2a98d34c0e None admin] PATH_INFO: `/v3/users/4ceb679399ff439a9bfe96f9758e269d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:16.008707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f00b7-6832-4b21-9b5e-eb2a98d34c0e None admin] RBAC: Authorizing `identity:get_user(user_id=4ceb679399ff439a9bfe96f9758e269d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:16.010165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f00b7-6832-4b21-9b5e-eb2a98d34c0e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:16.017985 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 87/170] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:15 2022] GET /identity/v3/users/4ceb679399ff439a9bfe96f9758e269d => generated 247 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:16.018862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f00b7-6832-4b21-9b5e-eb2a98d34c0e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:16.080646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442ee04-7716-4bbc-a858-bb6a1275c19b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:16.141186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80318c94-d000-429b-823c-e98b2d39a659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:16.141861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80318c94-d000-429b-823c-e98b2d39a659 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:16.142160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80318c94-d000-429b-823c-e98b2d39a659 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:16.142426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80318c94-d000-429b-823c-e98b2d39a659 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:16.143176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80318c94-d000-429b-823c-e98b2d39a659 None admin] RBAC: Adding query filter params (role.id=5eaf9701f37e480bb87c89f7391d5d3a, scope.project.id=4b6c3729d57640af9e56f5e19081b8b0, user.id=99df9e441d1e495695098147ca4290d9) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:16.143481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80318c94-d000-429b-823c-e98b2d39a659 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:16.145170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80318c94-d000-429b-823c-e98b2d39a659 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:16.160263 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 84/171] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:15 2022] GET /identity/v3/role_assignments?user.id=99df9e441d1e495695098147ca4290d9&scope.project.id=4b6c3729d57640af9e56f5e19081b8b0&role.id=5eaf9701f37e480bb87c89f7391d5d3a => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:16.161685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338a6d9-8649-4205-bad8-dfa63a5e92c9 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:16.161965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338a6d9-8649-4205-bad8-dfa63a5e92c9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:16.162263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338a6d9-8649-4205-bad8-dfa63a5e92c9 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:16.166845 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 85/172] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:16.173614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338a6d9-8649-4205-bad8-dfa63a5e92c9 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:16.225825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f00b7-6832-4b21-9b5e-eb2a98d34c0e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:16.287880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:16.320739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2d192-2a43-47c3-97aa-2feb13d91bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:16.321765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2d192-2a43-47c3-97aa-2feb13d91bab None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:16.322224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2d192-2a43-47c3-97aa-2feb13d91bab None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:16.322674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2d192-2a43-47c3-97aa-2feb13d91bab None admin] PATH_INFO: `/v3/users/99df9e441d1e495695098147ca4290d9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:16.324158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2d192-2a43-47c3-97aa-2feb13d91bab None admin] RBAC: Authorizing `identity:get_user(user_id=99df9e441d1e495695098147ca4290d9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:16.325896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2d192-2a43-47c3-97aa-2feb13d91bab None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:16.336399 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 88/173] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:16 2022] GET /identity/v3/users/99df9e441d1e495695098147ca4290d9 => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:16.337868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2d192-2a43-47c3-97aa-2feb13d91bab None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:16.372273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338a6d9-8649-4205-bad8-dfa63a5e92c9 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:16.445148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94d877-b4d5-4bc2-acd7-92c32b0a7c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:16.445863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94d877-b4d5-4bc2-acd7-92c32b0a7c6d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:16.446137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94d877-b4d5-4bc2-acd7-92c32b0a7c6d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:16.446403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94d877-b4d5-4bc2-acd7-92c32b0a7c6d None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:16.447906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94d877-b4d5-4bc2-acd7-92c32b0a7c6d None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:16.449439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94d877-b4d5-4bc2-acd7-92c32b0a7c6d None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:16.455110 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 86/174] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:16 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:16.456474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfb20b-0a24-44cc-862b-d576a2d334cc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:16.459471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfb20b-0a24-44cc-862b-d576a2d334cc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:16.459767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfb20b-0a24-44cc-862b-d576a2d334cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:16.459883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:16.496055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8bcfb20b-0a24-44cc-862b-d576a2d334cc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:16.497545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8bcfb20b-0a24-44cc-862b-d576a2d334cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:16.515816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2d192-2a43-47c3-97aa-2feb13d91bab None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:16.565674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf5949-1b06-46ea-b568-01382771504b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:16.566372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf5949-1b06-46ea-b568-01382771504b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:16.566659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf5949-1b06-46ea-b568-01382771504b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:16.566928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf5949-1b06-46ea-b568-01382771504b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:16.598472 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 87/175] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:16.599345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcfb20b-0a24-44cc-862b-d576a2d334cc None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:16.680905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf5949-1b06-46ea-b568-01382771504b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:16.683823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf5949-1b06-46ea-b568-01382771504b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:16.701726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:16.754272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:16.759405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcfb20b-0a24-44cc-862b-d576a2d334cc None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:16.796959 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 89/176] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:16.798494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1691-447c-4cfd-b8b0-6f125a289b5a None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:16.798915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1691-447c-4cfd-b8b0-6f125a289b5a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:16.799324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1691-447c-4cfd-b8b0-6f125a289b5a None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:16.800759 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 90/177] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:16.801581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb1691-447c-4cfd-b8b0-6f125a289b5a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:16.816854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3e0cb-6c28-4b8a-8f49-8544630e29f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:16.817397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3e0cb-6c28-4b8a-8f49-8544630e29f5 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:16.817672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3e0cb-6c28-4b8a-8f49-8544630e29f5 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:16.817939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3e0cb-6c28-4b8a-8f49-8544630e29f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:16.912495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3e0cb-6c28-4b8a-8f49-8544630e29f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:16.914236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3e0cb-6c28-4b8a-8f49-8544630e29f5 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:16.920208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb1691-447c-4cfd-b8b0-6f125a289b5a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:16.967669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bda777-30f6-4c78-bd47-b494637d04ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:16.968287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bda777-30f6-4c78-bd47-b494637d04ee None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:16.968584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bda777-30f6-4c78-bd47-b494637d04ee None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:16.968853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bda777-30f6-4c78-bd47-b494637d04ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:16.969579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bda777-30f6-4c78-bd47-b494637d04ee None admin] RBAC: Adding query filter params (role.id=f8a2ea13798443c48750e9baa83c7461, scope.project.id=4b6c3729d57640af9e56f5e19081b8b0, user.id=99df9e441d1e495695098147ca4290d9) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:16.969881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bda777-30f6-4c78-bd47-b494637d04ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:16.972105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bda777-30f6-4c78-bd47-b494637d04ee None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:16.977420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:16.980156 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 91/178] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:16 2022] GET /identity/v3/role_assignments?user.id=99df9e441d1e495695098147ca4290d9&scope.project.id=4b6c3729d57640af9e56f5e19081b8b0&role.id=f8a2ea13798443c48750e9baa83c7461 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:16.981848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47f861-f171-444b-9af7-0aca6b260bc3 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:16.983120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47f861-f171-444b-9af7-0aca6b260bc3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:16.983120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47f861-f171-444b-9af7-0aca6b260bc3 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:16.983799 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 92/179] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:16.989058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47f861-f171-444b-9af7-0aca6b260bc3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:17.033514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:17.046938 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 88/180] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:17.049451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f331a6-1775-4dcc-98c5-27c5cbc82315 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:17.049930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f331a6-1775-4dcc-98c5-27c5cbc82315 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:17.050415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f331a6-1775-4dcc-98c5-27c5cbc82315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:17.078222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-83f331a6-1775-4dcc-98c5-27c5cbc82315 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:17.079897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-83f331a6-1775-4dcc-98c5-27c5cbc82315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:17.132597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47f861-f171-444b-9af7-0aca6b260bc3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:17.161487 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 89/181] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:17.162477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f331a6-1775-4dcc-98c5-27c5cbc82315 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:17.184264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261938a-bdf9-4a5b-a7d5-62a92cf731dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:17.187009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261938a-bdf9-4a5b-a7d5-62a92cf731dd None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:17.187825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261938a-bdf9-4a5b-a7d5-62a92cf731dd None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:17.188290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261938a-bdf9-4a5b-a7d5-62a92cf731dd None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:17.190769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261938a-bdf9-4a5b-a7d5-62a92cf731dd None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:17.193119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261938a-bdf9-4a5b-a7d5-62a92cf731dd None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:17.196570 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 93/182] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:16 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:17.198985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2187f1a-943f-4715-8012-23c2d1c8eb12 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:17.199512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2187f1a-943f-4715-8012-23c2d1c8eb12 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:17.200458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2187f1a-943f-4715-8012-23c2d1c8eb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:17.226897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e2187f1a-943f-4715-8012-23c2d1c8eb12 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:17.228486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e2187f1a-943f-4715-8012-23c2d1c8eb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:17.299937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:17.315994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f331a6-1775-4dcc-98c5-27c5cbc82315 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:17.316197 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 94/183] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:17.321816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2187f1a-943f-4715-8012-23c2d1c8eb12 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:17.375353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:17.384757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0915c8-9410-4326-b8de-5275dd3feec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:17.385384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0915c8-9410-4326-b8de-5275dd3feec9 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:17.385654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0915c8-9410-4326-b8de-5275dd3feec9 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:17.385914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0915c8-9410-4326-b8de-5275dd3feec9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:17.399806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0915c8-9410-4326-b8de-5275dd3feec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-ae0915c8-9410-4326-b8de-5275dd3feec9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:17.405593 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:17.406471 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 90/184] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:38:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:17.412127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0915c8-9410-4326-b8de-5275dd3feec9 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:17.470939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2187f1a-943f-4715-8012-23c2d1c8eb12 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:17.514767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b33095-0e8e-44e3-838e-d4fad5d03085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:17.515706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b33095-0e8e-44e3-838e-d4fad5d03085 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:17.516183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b33095-0e8e-44e3-838e-d4fad5d03085 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:17.516572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b33095-0e8e-44e3-838e-d4fad5d03085 None admin] PATH_INFO: `/v3/projects/ca388bda9756467c927de434026dce16` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:17.517813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b33095-0e8e-44e3-838e-d4fad5d03085 None admin] RBAC: Authorizing `identity:get_project(project_id=ca388bda9756467c927de434026dce16)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:17.519348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b33095-0e8e-44e3-838e-d4fad5d03085 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:17.521472 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 95/185] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:17 2022] GET /identity/v3/projects/ca388bda9756467c927de434026dce16 => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:17.522680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74a282-9ca2-47c9-8eb8-9dfcb504e18e None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:17.525173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74a282-9ca2-47c9-8eb8-9dfcb504e18e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:17.525288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74a282-9ca2-47c9-8eb8-9dfcb504e18e None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:17.526566 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 96/186] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:17.531179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74a282-9ca2-47c9-8eb8-9dfcb504e18e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:17.543756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0915c8-9410-4326-b8de-5275dd3feec9 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:17.584610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963652c9-5b19-4548-b93a-89251e001b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:17.585261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963652c9-5b19-4548-b93a-89251e001b36 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:17.585532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963652c9-5b19-4548-b93a-89251e001b36 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:17.585790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963652c9-5b19-4548-b93a-89251e001b36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:17.586499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963652c9-5b19-4548-b93a-89251e001b36 None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=4b6c3729d57640af9e56f5e19081b8b0, user.id=4ceb679399ff439a9bfe96f9758e269d) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:17.586815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963652c9-5b19-4548-b93a-89251e001b36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:17.588232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963652c9-5b19-4548-b93a-89251e001b36 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:17.594230 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 91/187] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:17 2022] GET /identity/v3/role_assignments?user.id=4ceb679399ff439a9bfe96f9758e269d&scope.project.id=4b6c3729d57640af9e56f5e19081b8b0&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:17.595026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963652c9-5b19-4548-b93a-89251e001b36 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:17.615197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:17.641699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:17.649761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74a282-9ca2-47c9-8eb8-9dfcb504e18e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:17.690422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50872c3-e9a1-4191-9ea5-039eee12b0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:17.691399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50872c3-e9a1-4191-9ea5-039eee12b0e3 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:17.691891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50872c3-e9a1-4191-9ea5-039eee12b0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:17.692340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50872c3-e9a1-4191-9ea5-039eee12b0e3 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:17.693268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50872c3-e9a1-4191-9ea5-039eee12b0e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:17.693757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50872c3-e9a1-4191-9ea5-039eee12b0e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:17.695353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50872c3-e9a1-4191-9ea5-039eee12b0e3 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:17.705478 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 97/188] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:17.712322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0d1f5-3237-4a47-acb9-362fa7003f22 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:17.713324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0d1f5-3237-4a47-acb9-362fa7003f22 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:17.713596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0d1f5-3237-4a47-acb9-362fa7003f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:17.728678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963652c9-5b19-4548-b93a-89251e001b36 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:17.731644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5da0d1f5-3237-4a47-acb9-362fa7003f22 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:17.732874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5da0d1f5-3237-4a47-acb9-362fa7003f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:17.766512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84246a6f-ba75-4daa-930a-2d39eb77af62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:17.767130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84246a6f-ba75-4daa-930a-2d39eb77af62 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:17.767402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84246a6f-ba75-4daa-930a-2d39eb77af62 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:17.767664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84246a6f-ba75-4daa-930a-2d39eb77af62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:17.768418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84246a6f-ba75-4daa-930a-2d39eb77af62 None admin] RBAC: Adding query filter params (role.id=5eaf9701f37e480bb87c89f7391d5d3a, scope.project.id=ca388bda9756467c927de434026dce16, user.id=99df9e441d1e495695098147ca4290d9) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:17.768727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84246a6f-ba75-4daa-930a-2d39eb77af62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:17.770181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84246a6f-ba75-4daa-930a-2d39eb77af62 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:17.778259 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 92/189] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:17 2022] GET /identity/v3/role_assignments?user.id=99df9e441d1e495695098147ca4290d9&scope.project.id=ca388bda9756467c927de434026dce16&role.id=5eaf9701f37e480bb87c89f7391d5d3a => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:17.779126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84246a6f-ba75-4daa-930a-2d39eb77af62 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:17.818828 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 98/190] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:17.819812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0d1f5-3237-4a47-acb9-362fa7003f22 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:17.895152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:17.919426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84246a6f-ba75-4daa-930a-2d39eb77af62 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:17.922354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:17.936861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0d1f5-3237-4a47-acb9-362fa7003f22 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:17.961308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c89331-d96f-4737-b05f-20858b54146b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:17.962420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c89331-d96f-4737-b05f-20858b54146b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:17.962882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c89331-d96f-4737-b05f-20858b54146b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:17.963355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c89331-d96f-4737-b05f-20858b54146b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:17.976162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c89331-d96f-4737-b05f-20858b54146b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-40c89331-d96f-4737-b05f-20858b54146b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:17.982366 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:17.983473 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:17.984487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:17.984487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:17.984487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:17.984487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:17.984487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:17.984487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:17.984487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:17.984487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:17.984487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:17.984487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:17.984487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:17.984487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:17.985226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72a589-f586-4e08-b8a0-c12b9bfb51dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:17.985226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72a589-f586-4e08-b8a0-c12b9bfb51dc None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:17.985226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72a589-f586-4e08-b8a0-c12b9bfb51dc None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:17.985226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72a589-f586-4e08-b8a0-c12b9bfb51dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:17.986281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72a589-f586-4e08-b8a0-c12b9bfb51dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:17.986922 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 93/191] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:38:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:17.987716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c89331-d96f-4737-b05f-20858b54146b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:17.987825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72a589-f586-4e08-b8a0-c12b9bfb51dc None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:17.989664 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 99/192] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:38:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:17.995596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72a589-f586-4e08-b8a0-c12b9bfb51dc None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:18.114342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c89331-d96f-4737-b05f-20858b54146b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:18.125955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72a589-f586-4e08-b8a0-c12b9bfb51dc None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:18.155299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50c41b-1128-4d90-904c-726e96a77985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:18.156135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50c41b-1128-4d90-904c-726e96a77985 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:18.156567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50c41b-1128-4d90-904c-726e96a77985 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:18.156971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50c41b-1128-4d90-904c-726e96a77985 None admin] PATH_INFO: `/v3/users/83bd8eb8f3bc449baaa07a558c5136d9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:18.165828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:18.170100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee125145-1dc3-4c35-81d5-b491048da7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:18.170719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee125145-1dc3-4c35-81d5-b491048da7ad None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:18.170988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee125145-1dc3-4c35-81d5-b491048da7ad None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:18.171251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee125145-1dc3-4c35-81d5-b491048da7ad None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:18.171906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee125145-1dc3-4c35-81d5-b491048da7ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:18.172770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50c41b-1128-4d90-904c-726e96a77985 None admin] RBAC: Authorizing `identity:get_user(user_id=83bd8eb8f3bc449baaa07a558c5136d9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:18.173444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee125145-1dc3-4c35-81d5-b491048da7ad None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:18.174307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50c41b-1128-4d90-904c-726e96a77985 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:18.176220 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 94/193] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:17 2022] GET /identity/v3/users/83bd8eb8f3bc449baaa07a558c5136d9 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:18.181213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50c41b-1128-4d90-904c-726e96a77985 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:18.196415 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 100/194] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:38:17 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:38:18.196415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee125145-1dc3-4c35-81d5-b491048da7ad None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:18.212743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:18.307014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50c41b-1128-4d90-904c-726e96a77985 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:18.323598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee125145-1dc3-4c35-81d5-b491048da7ad None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:18.346010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3b7a5-01da-4100-9e1d-8501d2d704ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:18.346623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3b7a5-01da-4100-9e1d-8501d2d704ae None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:18.346894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3b7a5-01da-4100-9e1d-8501d2d704ae None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:18.347157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3b7a5-01da-4100-9e1d-8501d2d704ae None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:18.347824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3b7a5-01da-4100-9e1d-8501d2d704ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:18.348118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3b7a5-01da-4100-9e1d-8501d2d704ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:18.349785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3b7a5-01da-4100-9e1d-8501d2d704ae None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:18.359603 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 95/195] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:38:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:18.360715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686071d-e9dc-46e9-a4bf-bd655f803dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:18.361269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686071d-e9dc-46e9-a4bf-bd655f803dfe None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:18.361538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686071d-e9dc-46e9-a4bf-bd655f803dfe None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:18.361799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686071d-e9dc-46e9-a4bf-bd655f803dfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:18.365324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3b7a5-01da-4100-9e1d-8501d2d704ae None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:18.431182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:18.445297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686071d-e9dc-46e9-a4bf-bd655f803dfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:18.447766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686071d-e9dc-46e9-a4bf-bd655f803dfe None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:18.464219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:18.482269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3b7a5-01da-4100-9e1d-8501d2d704ae None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:18.533759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b000c54-fc8f-4c35-9f12-f9942ea32e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:18.533759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b000c54-fc8f-4c35-9f12-f9942ea32e02 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:18.533759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b000c54-fc8f-4c35-9f12-f9942ea32e02 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:18.533759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b000c54-fc8f-4c35-9f12-f9942ea32e02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:18.546947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b000c54-fc8f-4c35-9f12-f9942ea32e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-4b000c54-fc8f-4c35-9f12-f9942ea32e02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:18.552315 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:18.553242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 96/196] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:18.554320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b000c54-fc8f-4c35-9f12-f9942ea32e02 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:18.575939 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 101/197] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:18.577442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686071d-e9dc-46e9-a4bf-bd655f803dfe None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:18.678799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b000c54-fc8f-4c35-9f12-f9942ea32e02 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:18.689153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686071d-e9dc-46e9-a4bf-bd655f803dfe None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:18.699582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:18.704695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:18.720066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3656700-088f-4726-a825-c913667da5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:18.720704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3656700-088f-4726-a825-c913667da5ca None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:18.720979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3656700-088f-4726-a825-c913667da5ca None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:18.721316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3656700-088f-4726-a825-c913667da5ca None admin] PATH_INFO: `/v3/users/83bd8eb8f3bc449baaa07a558c5136d9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:18.722372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3656700-088f-4726-a825-c913667da5ca None admin] RBAC: Authorizing `identity:get_user(user_id=83bd8eb8f3bc449baaa07a558c5136d9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:18.724001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3656700-088f-4726-a825-c913667da5ca None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:18.725891 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 97/198] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:18 2022] GET /identity/v3/users/83bd8eb8f3bc449baaa07a558c5136d9 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:18.726446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3656700-088f-4726-a825-c913667da5ca None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:18.729580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687a89d-3f1f-4cc0-8efc-cf977cd5b9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:18.730791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687a89d-3f1f-4cc0-8efc-cf977cd5b9e8 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:18.731354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687a89d-3f1f-4cc0-8efc-cf977cd5b9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:18.731810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687a89d-3f1f-4cc0-8efc-cf977cd5b9e8 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:18.732740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0687a89d-3f1f-4cc0-8efc-cf977cd5b9e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:18.733243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0687a89d-3f1f-4cc0-8efc-cf977cd5b9e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:18.734846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0687a89d-3f1f-4cc0-8efc-cf977cd5b9e8 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:18.744724 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 102/199] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:18.745848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687a89d-3f1f-4cc0-8efc-cf977cd5b9e8 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:18.834387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3656700-088f-4726-a825-c913667da5ca None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:18.852541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687a89d-3f1f-4cc0-8efc-cf977cd5b9e8 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:18.868228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36b2c3-d68b-4ac8-8133-ab38430a588e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:18.868880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36b2c3-d68b-4ac8-8133-ab38430a588e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:18.869152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36b2c3-d68b-4ac8-8133-ab38430a588e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:18.869411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36b2c3-d68b-4ac8-8133-ab38430a588e None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:18.870545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36b2c3-d68b-4ac8-8133-ab38430a588e None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:18.871996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36b2c3-d68b-4ac8-8133-ab38430a588e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:18.873679 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 98/200] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:18 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:18.874438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36b2c3-d68b-4ac8-8133-ab38430a588e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:18.891506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811d4ec-fe6e-4258-ac55-49b3c0aec6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:18.892695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811d4ec-fe6e-4258-ac55-49b3c0aec6c4 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:18.893210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811d4ec-fe6e-4258-ac55-49b3c0aec6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:18.893665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811d4ec-fe6e-4258-ac55-49b3c0aec6c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:18.936149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:18.960822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:18.972960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811d4ec-fe6e-4258-ac55-49b3c0aec6c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:18.975436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811d4ec-fe6e-4258-ac55-49b3c0aec6c4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:18.982350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36b2c3-d68b-4ac8-8133-ab38430a588e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:19.019098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91e7fa-b30f-45e5-b8c3-327910e6dd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:19.019814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91e7fa-b30f-45e5-b8c3-327910e6dd8d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:19.020179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91e7fa-b30f-45e5-b8c3-327910e6dd8d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:19.020486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91e7fa-b30f-45e5-b8c3-327910e6dd8d None admin] PATH_INFO: `/v3/users/6866ea48849b4f40961322f484b0df1f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:19.022178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91e7fa-b30f-45e5-b8c3-327910e6dd8d None admin] RBAC: Authorizing `identity:get_user(user_id=6866ea48849b4f40961322f484b0df1f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:19.023220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91e7fa-b30f-45e5-b8c3-327910e6dd8d None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:19.025288 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 99/201] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:18 2022] GET /identity/v3/users/6866ea48849b4f40961322f484b0df1f => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:19.026134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91e7fa-b30f-45e5-b8c3-327910e6dd8d None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:19.115062 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 103/202] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:19.117023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811d4ec-fe6e-4258-ac55-49b3c0aec6c4 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:19.138927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91e7fa-b30f-45e5-b8c3-327910e6dd8d None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:19.170061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:19.186582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d8cf6-a1ac-41e3-8492-5ce3f4bfd255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:19.188024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d8cf6-a1ac-41e3-8492-5ce3f4bfd255 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:19.188498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d8cf6-a1ac-41e3-8492-5ce3f4bfd255 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:19.189189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d8cf6-a1ac-41e3-8492-5ce3f4bfd255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:19.190337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d8cf6-a1ac-41e3-8492-5ce3f4bfd255 None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=66eda7a4830f4fdeb286e09ae72f278b, user.id=83bd8eb8f3bc449baaa07a558c5136d9) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:19.190815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d8cf6-a1ac-41e3-8492-5ce3f4bfd255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:19.193259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d8cf6-a1ac-41e3-8492-5ce3f4bfd255 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:19.196363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:19.200969 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 100/203] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:19 2022] GET /identity/v3/role_assignments?user.id=83bd8eb8f3bc449baaa07a558c5136d9&scope.project.id=66eda7a4830f4fdeb286e09ae72f278b&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:19.202080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d8cf6-a1ac-41e3-8492-5ce3f4bfd255 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:19.255005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811d4ec-fe6e-4258-ac55-49b3c0aec6c4 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:19.324679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d8cf6-a1ac-41e3-8492-5ce3f4bfd255 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:19.364682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d683603-cf69-4b92-b67f-f126f7021b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:19.365631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d683603-cf69-4b92-b67f-f126f7021b85 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:19.366103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d683603-cf69-4b92-b67f-f126f7021b85 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:19.366540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d683603-cf69-4b92-b67f-f126f7021b85 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:19.367974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d683603-cf69-4b92-b67f-f126f7021b85 None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:19.369744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d683603-cf69-4b92-b67f-f126f7021b85 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:19.372444 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 101/204] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:19 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:19.380505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d683603-cf69-4b92-b67f-f126f7021b85 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:19.394010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccec89-70e8-4c0e-bb0c-ea5bca7daee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:19.394925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccec89-70e8-4c0e-bb0c-ea5bca7daee4 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:19.395344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccec89-70e8-4c0e-bb0c-ea5bca7daee4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:19.395751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccec89-70e8-4c0e-bb0c-ea5bca7daee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:19.470730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:19.475651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccec89-70e8-4c0e-bb0c-ea5bca7daee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:19.477376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccec89-70e8-4c0e-bb0c-ea5bca7daee4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:19.527386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d683603-cf69-4b92-b67f-f126f7021b85 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:19.567190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e141e1e-cec0-4e81-9c8a-80b78f0da5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:19.568586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e141e1e-cec0-4e81-9c8a-80b78f0da5b7 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:19.569100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e141e1e-cec0-4e81-9c8a-80b78f0da5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:19.569555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e141e1e-cec0-4e81-9c8a-80b78f0da5b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:19.570443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e141e1e-cec0-4e81-9c8a-80b78f0da5b7 None admin] RBAC: Adding query filter params (role.id=f8a2ea13798443c48750e9baa83c7461, scope.project.id=66eda7a4830f4fdeb286e09ae72f278b, user.id=83bd8eb8f3bc449baaa07a558c5136d9) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:19.570911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e141e1e-cec0-4e81-9c8a-80b78f0da5b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:19.572542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e141e1e-cec0-4e81-9c8a-80b78f0da5b7 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:19.579787 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 102/205] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:19 2022] GET /identity/v3/role_assignments?user.id=83bd8eb8f3bc449baaa07a558c5136d9&scope.project.id=66eda7a4830f4fdeb286e09ae72f278b&role.id=f8a2ea13798443c48750e9baa83c7461 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:19.616822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:19.650206 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 104/206] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:19.656874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e141e1e-cec0-4e81-9c8a-80b78f0da5b7 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:19.798885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e141e1e-cec0-4e81-9c8a-80b78f0da5b7 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:19.822082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:19.842890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9dee2-d2c7-4291-bced-33ac796c4b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:19.843522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9dee2-d2c7-4291-bced-33ac796c4b08 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:19.843791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9dee2-d2c7-4291-bced-33ac796c4b08 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:19.844113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9dee2-d2c7-4291-bced-33ac796c4b08 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:19.849048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9dee2-d2c7-4291-bced-33ac796c4b08 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:19.850551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9dee2-d2c7-4291-bced-33ac796c4b08 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:19.852465 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 103/207] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:19 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:19.858554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccec89-70e8-4c0e-bb0c-ea5bca7daee4 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:19.965851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccec89-70e8-4c0e-bb0c-ea5bca7daee4 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:20.007929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6f8a6-339e-4491-ae92-3edf67571a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:20.008569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6f8a6-339e-4491-ae92-3edf67571a0a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:20.008838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6f8a6-339e-4491-ae92-3edf67571a0a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:20.009100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6f8a6-339e-4491-ae92-3edf67571a0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:20.010003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6f8a6-339e-4491-ae92-3edf67571a0a None admin] RBAC: Adding query filter params (role.id=bdbaea91119244b9aa4a822c613c2bb8, scope.project.id=4b6c3729d57640af9e56f5e19081b8b0, user.id=6866ea48849b4f40961322f484b0df1f) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:20.010358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6f8a6-339e-4491-ae92-3edf67571a0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:20.011857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6f8a6-339e-4491-ae92-3edf67571a0a None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:20.018479 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 105/208] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:19 2022] GET /identity/v3/role_assignments?user.id=6866ea48849b4f40961322f484b0df1f&scope.project.id=4b6c3729d57640af9e56f5e19081b8b0&role.id=bdbaea91119244b9aa4a822c613c2bb8 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:20.771584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c185e-7427-4d92-8e56-5b7537893b65 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:20.772139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c185e-7427-4d92-8e56-5b7537893b65 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:20.772139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c185e-7427-4d92-8e56-5b7537893b65 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:20.773579 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 104/209] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:20.779997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9a805-d130-44f2-b247-b5c6650de377 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:20.780526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9a805-d130-44f2-b247-b5c6650de377 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:20.780598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9a805-d130-44f2-b247-b5c6650de377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:20.800132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bef9a805-d130-44f2-b247-b5c6650de377 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:20.801340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bef9a805-d130-44f2-b247-b5c6650de377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:20.837483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:20.876719 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 106/210] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:20.920855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4073e-8146-4c60-921f-0fcb1663c046 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:20.921129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4073e-8146-4c60-921f-0fcb1663c046 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:20.921398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4073e-8146-4c60-921f-0fcb1663c046 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:20.922769 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 105/211] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:20.928624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba76d5-872a-4d25-9675-bda23cd0052a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:20.928904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba76d5-872a-4d25-9675-bda23cd0052a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:20.929228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba76d5-872a-4d25-9675-bda23cd0052a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:20.948794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f0ba76d5-872a-4d25-9675-bda23cd0052a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:20.950019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f0ba76d5-872a-4d25-9675-bda23cd0052a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:21.023682 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 107/212] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:21.209680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4073e-8146-4c60-921f-0fcb1663c046 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:21.325218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4073e-8146-4c60-921f-0fcb1663c046 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:21.360363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda06e5-bcd9-438a-bebf-62ffe5123aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:21.361018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda06e5-bcd9-438a-bebf-62ffe5123aae None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:21.361292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda06e5-bcd9-438a-bebf-62ffe5123aae None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:21.361560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda06e5-bcd9-438a-bebf-62ffe5123aae None admin] PATH_INFO: `/v3/roles/member` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:21.366726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba76d5-872a-4d25-9675-bda23cd0052a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:21.373025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda06e5-bcd9-438a-bebf-62ffe5123aae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-cfda06e5-bcd9-438a-bebf-62ffe5123aae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:21.377696 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:21.378574 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:21.379390 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:21.379390 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:21.379390 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:21.379390 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:21.379390 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:21.379390 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:21.379390 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:21.379390 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:21.379390 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:21.379390 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:21.379390 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:21.379390 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:21.379390 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 106/213] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:21.389361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda06e5-bcd9-438a-bebf-62ffe5123aae None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:21.446260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:21.483605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:21.487715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba76d5-872a-4d25-9675-bda23cd0052a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:21.514989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda06e5-bcd9-438a-bebf-62ffe5123aae None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:21.529577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbff8f6-cce0-4d2e-a223-2e2873d853f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:21.530763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbff8f6-cce0-4d2e-a223-2e2873d853f8 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:21.531230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbff8f6-cce0-4d2e-a223-2e2873d853f8 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:21.531683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbff8f6-cce0-4d2e-a223-2e2873d853f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:21.533207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbff8f6-cce0-4d2e-a223-2e2873d853f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:21.535207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbff8f6-cce0-4d2e-a223-2e2873d853f8 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:21.537268 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 108/214] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:38:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:21.543123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbff8f6-cce0-4d2e-a223-2e2873d853f8 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:21.558058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89588013-43ed-4412-a3b7-0af360bee10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:21.559013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89588013-43ed-4412-a3b7-0af360bee10a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:21.559508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89588013-43ed-4412-a3b7-0af360bee10a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:21.560016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89588013-43ed-4412-a3b7-0af360bee10a None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:21.561357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89588013-43ed-4412-a3b7-0af360bee10a None admin] RBAC: Adding query filter params (name=member) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:21.561875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89588013-43ed-4412-a3b7-0af360bee10a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:21.563497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89588013-43ed-4412-a3b7-0af360bee10a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:21.573201 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 107/215] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:21.579249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89588013-43ed-4412-a3b7-0af360bee10a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:21.662749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbff8f6-cce0-4d2e-a223-2e2873d853f8 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:21.701566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032757cd-83c0-4d6f-a94f-b00e5f69b6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:21.702497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032757cd-83c0-4d6f-a94f-b00e5f69b6c6 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:21.702887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032757cd-83c0-4d6f-a94f-b00e5f69b6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:21.703317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032757cd-83c0-4d6f-a94f-b00e5f69b6c6 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:21.703963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032757cd-83c0-4d6f-a94f-b00e5f69b6c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:21.705433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032757cd-83c0-4d6f-a94f-b00e5f69b6c6 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:21.707638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89588013-43ed-4412-a3b7-0af360bee10a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:21.718285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:21.731344 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 109/216] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:38:21 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:38:21.733733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a72b9-fd90-476e-b596-2c826cb3f44b None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:21.734387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a72b9-fd90-476e-b596-2c826cb3f44b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:21.734847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a72b9-fd90-476e-b596-2c826cb3f44b None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:21.737321 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 110/217] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:21.743367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2bd8b-06f9-4497-8b0d-71c42defc122 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:21.743661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2bd8b-06f9-4497-8b0d-71c42defc122 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:21.743896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2bd8b-06f9-4497-8b0d-71c42defc122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:21.756503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765faf5a-75bd-4b69-8116-78f93286a272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:21.757452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765faf5a-75bd-4b69-8116-78f93286a272 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:21.758009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765faf5a-75bd-4b69-8116-78f93286a272 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:21.758479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765faf5a-75bd-4b69-8116-78f93286a272 None admin] PATH_INFO: `/v3/users/7db0af1475de44c58b083d498c0d87dc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:21.766045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-05e2bd8b-06f9-4497-8b0d-71c42defc122 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:21.767426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-05e2bd8b-06f9-4497-8b0d-71c42defc122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:21.775662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765faf5a-75bd-4b69-8116-78f93286a272 None admin] RBAC: Authorizing `identity:get_user(user_id=7db0af1475de44c58b083d498c0d87dc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:21.778084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765faf5a-75bd-4b69-8116-78f93286a272 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:21.784977 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 108/218] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:21 2022] GET /identity/v3/users/7db0af1475de44c58b083d498c0d87dc => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:21.787973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765faf5a-75bd-4b69-8116-78f93286a272 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:21.790992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:21.875826 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 111/219] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:21.908449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765faf5a-75bd-4b69-8116-78f93286a272 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:21.947284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d97ae0-9bc6-4de6-8f2b-870c73a6c05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:21.947909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d97ae0-9bc6-4de6-8f2b-870c73a6c05a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:21.948174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d97ae0-9bc6-4de6-8f2b-870c73a6c05a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:21.948473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d97ae0-9bc6-4de6-8f2b-870c73a6c05a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:21.983453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:22.021290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d97ae0-9bc6-4de6-8f2b-870c73a6c05a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:22.023139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d97ae0-9bc6-4de6-8f2b-870c73a6c05a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:22.033493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2198c-6f2c-4164-ac16-19de53d93915 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:22.034072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2198c-6f2c-4164-ac16-19de53d93915 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:22.034427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2198c-6f2c-4164-ac16-19de53d93915 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:22.035918 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 112/220] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:38:22.041424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f73e9f-4bb6-4b23-8670-df17d6bee263 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:22.041699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f73e9f-4bb6-4b23-8670-df17d6bee263 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:22.041966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f73e9f-4bb6-4b23-8670-df17d6bee263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:22.065401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-18f73e9f-4bb6-4b23-8670-df17d6bee263 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:22.066567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-18f73e9f-4bb6-4b23-8670-df17d6bee263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:22.153166 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 109/221] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:22.154365 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 113/222] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:22.158923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d97ae0-9bc6-4de6-8f2b-870c73a6c05a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:22.223359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f73e9f-4bb6-4b23-8670-df17d6bee263 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:22.250592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:22.291230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d97ae0-9bc6-4de6-8f2b-870c73a6c05a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:22.336347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cee19-6815-4cd9-9569-c42bb851f262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:22.337029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cee19-6815-4cd9-9569-c42bb851f262 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:22.337311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cee19-6815-4cd9-9569-c42bb851f262 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:22.337815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cee19-6815-4cd9-9569-c42bb851f262 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:22.338978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cee19-6815-4cd9-9569-c42bb851f262 None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:22.341426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cee19-6815-4cd9-9569-c42bb851f262 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:22.343767 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 110/223] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:22 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:22.350034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cee19-6815-4cd9-9569-c42bb851f262 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:22.425994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:22.463680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f73e9f-4bb6-4b23-8670-df17d6bee263 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:22.487128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cee19-6815-4cd9-9569-c42bb851f262 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:22.512232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8055e-978d-49f6-91ee-ae99ef33f45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:22.512910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8055e-978d-49f6-91ee-ae99ef33f45d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:22.513116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8055e-978d-49f6-91ee-ae99ef33f45d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:22.513377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8055e-978d-49f6-91ee-ae99ef33f45d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:22.535758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8055e-978d-49f6-91ee-ae99ef33f45d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-b9b8055e-978d-49f6-91ee-ae99ef33f45d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:22.536299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:22.537209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:22.538019 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:22.538019 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:22.538019 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:22.538019 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:22.538019 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:22.538019 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:22.538019 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:22.538019 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:22.538019 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:22.538019 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:22.538019 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:22.538019 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:22.538019 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 114/224] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:38:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:38:22.543845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8055e-978d-49f6-91ee-ae99ef33f45d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:22.544959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5d742-d618-4792-9d2f-ca53982399e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:22.545575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5d742-d618-4792-9d2f-ca53982399e5 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:22.545845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5d742-d618-4792-9d2f-ca53982399e5 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:22.546104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5d742-d618-4792-9d2f-ca53982399e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:22.546829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5d742-d618-4792-9d2f-ca53982399e5 None admin] RBAC: Adding query filter params (role.id=5eaf9701f37e480bb87c89f7391d5d3a, scope.project.id=66eda7a4830f4fdeb286e09ae72f278b, user.id=7db0af1475de44c58b083d498c0d87dc) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:22.547130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5d742-d618-4792-9d2f-ca53982399e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:22.548593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5d742-d618-4792-9d2f-ca53982399e5 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:22.564766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:22.570303 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 111/225] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:22 2022] GET /identity/v3/role_assignments?user.id=7db0af1475de44c58b083d498c0d87dc&scope.project.id=66eda7a4830f4fdeb286e09ae72f278b&role.id=5eaf9701f37e480bb87c89f7391d5d3a => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:22.571974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5d742-d618-4792-9d2f-ca53982399e5 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:22.712795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8055e-978d-49f6-91ee-ae99ef33f45d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:22.761973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5d742-d618-4792-9d2f-ca53982399e5 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:22.766101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f7a61-c2ef-4edf-8893-f19c07a79a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:22.766728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f7a61-c2ef-4edf-8893-f19c07a79a0c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:22.767001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f7a61-c2ef-4edf-8893-f19c07a79a0c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:22.767262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f7a61-c2ef-4edf-8893-f19c07a79a0c None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:22.767934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f7a61-c2ef-4edf-8893-f19c07a79a0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:22.768229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f7a61-c2ef-4edf-8893-f19c07a79a0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:22.769642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f7a61-c2ef-4edf-8893-f19c07a79a0c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:22.780967 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 115/226] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:22.786897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f7a61-c2ef-4edf-8893-f19c07a79a0c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:22.792286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:22.815457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac2b9e-3ae5-42dd-9f42-d247f8d89922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:22.817776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac2b9e-3ae5-42dd-9f42-d247f8d89922 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:22.818861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac2b9e-3ae5-42dd-9f42-d247f8d89922 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:22.819310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac2b9e-3ae5-42dd-9f42-d247f8d89922 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:22.845970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac2b9e-3ae5-42dd-9f42-d247f8d89922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9bac2b9e-3ae5-42dd-9f42-d247f8d89922 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:22.854620 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:22.855716 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:22.856809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:22.856809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:22.856809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:22.856809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:22.856809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:22.856809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:22.856809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:22.856809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:22.856809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:22.856809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:22.856809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:22.856809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:22.862787 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 112/227] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:38:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:22.870051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac2b9e-3ae5-42dd-9f42-d247f8d89922 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:22.980835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:22.993311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac2b9e-3ae5-42dd-9f42-d247f8d89922 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:23.050204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e98fa-2680-4c81-88c9-b7ecaeb488fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:23.051062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e98fa-2680-4c81-88c9-b7ecaeb488fc None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:23.051639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e98fa-2680-4c81-88c9-b7ecaeb488fc None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:23.052063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e98fa-2680-4c81-88c9-b7ecaeb488fc None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:23.052945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e98fa-2680-4c81-88c9-b7ecaeb488fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:23.053389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e98fa-2680-4c81-88c9-b7ecaeb488fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:23.055153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e98fa-2680-4c81-88c9-b7ecaeb488fc None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:23.065242 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 113/228] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:38:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:23.067713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f7a61-c2ef-4edf-8893-f19c07a79a0c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:23.073624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e98fa-2680-4c81-88c9-b7ecaeb488fc None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:23.127627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e629d76-2eb9-4007-8fe4-93bd86f8972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:23.129879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e629d76-2eb9-4007-8fe4-93bd86f8972e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:23.131571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e629d76-2eb9-4007-8fe4-93bd86f8972e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:23.132035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e629d76-2eb9-4007-8fe4-93bd86f8972e None admin] PATH_INFO: `/v3/users/83bd8eb8f3bc449baaa07a558c5136d9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:23.134282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e629d76-2eb9-4007-8fe4-93bd86f8972e None admin] RBAC: Authorizing `identity:get_user(user_id=83bd8eb8f3bc449baaa07a558c5136d9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:23.137132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e629d76-2eb9-4007-8fe4-93bd86f8972e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:23.141078 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 116/229] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:22 2022] GET /identity/v3/users/83bd8eb8f3bc449baaa07a558c5136d9 => generated 283 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:23.146513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e629d76-2eb9-4007-8fe4-93bd86f8972e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:23.227254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e98fa-2680-4c81-88c9-b7ecaeb488fc None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:23.252788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:23.269808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8496bb0-a766-44c1-b46c-6d9adb212c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:23.270797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8496bb0-a766-44c1-b46c-6d9adb212c44 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:23.271263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8496bb0-a766-44c1-b46c-6d9adb212c44 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:23.271718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8496bb0-a766-44c1-b46c-6d9adb212c44 None admin] PATH_INFO: `/v3/users/83bd8eb8f3bc449baaa07a558c5136d9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:23.273086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8496bb0-a766-44c1-b46c-6d9adb212c44 None admin] RBAC: Authorizing `identity:get_user(user_id=83bd8eb8f3bc449baaa07a558c5136d9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:23.275201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8496bb0-a766-44c1-b46c-6d9adb212c44 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:23.277132 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 114/230] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:23 2022] GET /identity/v3/users/83bd8eb8f3bc449baaa07a558c5136d9 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:23.283703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8496bb0-a766-44c1-b46c-6d9adb212c44 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:23.314958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:23.353827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e629d76-2eb9-4007-8fe4-93bd86f8972e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:23.390223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2081960-869e-4d97-a520-1e8df5ae73b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:23.391226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2081960-869e-4d97-a520-1e8df5ae73b1 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:23.391721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2081960-869e-4d97-a520-1e8df5ae73b1 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:23.392178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2081960-869e-4d97-a520-1e8df5ae73b1 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:23.393789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2081960-869e-4d97-a520-1e8df5ae73b1 None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:23.395492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2081960-869e-4d97-a520-1e8df5ae73b1 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:23.398139 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 117/231] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:23 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:23.409448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2081960-869e-4d97-a520-1e8df5ae73b1 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:23.542639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8496bb0-a766-44c1-b46c-6d9adb212c44 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:23.567919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2081960-869e-4d97-a520-1e8df5ae73b1 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:23.605655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72d52c-c08a-4178-b983-38f20ffd5d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:23.606715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72d52c-c08a-4178-b983-38f20ffd5d2c None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:23.607124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72d52c-c08a-4178-b983-38f20ffd5d2c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:23.607523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72d52c-c08a-4178-b983-38f20ffd5d2c None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b/users/83bd8eb8f3bc449baaa07a558c5136d9/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:23.609558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72d52c-c08a-4178-b983-38f20ffd5d2c None admin] RBAC: Authorizing `identity:create_grant(project_id=66eda7a4830f4fdeb286e09ae72f278b, user_id=83bd8eb8f3bc449baaa07a558c5136d9, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:23.611134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72d52c-c08a-4178-b983-38f20ffd5d2c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:23.615000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203cb35-a1c4-4087-9de1-2cde0da3d483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:23.615669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203cb35-a1c4-4087-9de1-2cde0da3d483 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:23.615948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203cb35-a1c4-4087-9de1-2cde0da3d483 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:23.616258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203cb35-a1c4-4087-9de1-2cde0da3d483 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:23.617725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203cb35-a1c4-4087-9de1-2cde0da3d483 None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:23.619286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203cb35-a1c4-4087-9de1-2cde0da3d483 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:23.621337 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 115/232] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:23 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:23.623322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:23.628667 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 118/233] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:38:23 2022] PUT /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b/users/83bd8eb8f3bc449baaa07a558c5136d9/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:38:23.629652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203cb35-a1c4-4087-9de1-2cde0da3d483 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:23.709661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:23.762732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203cb35-a1c4-4087-9de1-2cde0da3d483 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:23.807816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea6036-cfb5-4913-aaeb-930c8a107c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:23.808419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea6036-cfb5-4913-aaeb-930c8a107c8b None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:23.808759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea6036-cfb5-4913-aaeb-930c8a107c8b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:23.809035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea6036-cfb5-4913-aaeb-930c8a107c8b None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b/users/83bd8eb8f3bc449baaa07a558c5136d9/roles/f8a2ea13798443c48750e9baa83c7461` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:23.812261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea6036-cfb5-4913-aaeb-930c8a107c8b None admin] RBAC: Authorizing `identity:create_grant(project_id=66eda7a4830f4fdeb286e09ae72f278b, user_id=83bd8eb8f3bc449baaa07a558c5136d9, role_id=f8a2ea13798443c48750e9baa83c7461)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:23.813768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea6036-cfb5-4913-aaeb-930c8a107c8b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:23.834059 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 116/234] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:38:23 2022] PUT /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b/users/83bd8eb8f3bc449baaa07a558c5136d9/roles/f8a2ea13798443c48750e9baa83c7461 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:38:24.513272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223ef01-f959-4d73-8fca-a8ee99cce114 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:24.513272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223ef01-f959-4d73-8fca-a8ee99cce114 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:24.513272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223ef01-f959-4d73-8fca-a8ee99cce114 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:24.514293 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 119/235] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:24.520060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa42c84-b706-476c-8d23-9e3784150d2f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:24.520339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa42c84-b706-476c-8d23-9e3784150d2f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:24.520630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa42c84-b706-476c-8d23-9e3784150d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:24.544215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1aa42c84-b706-476c-8d23-9e3784150d2f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:24.545427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1aa42c84-b706-476c-8d23-9e3784150d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:24.640522 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 117/236] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:24.884815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570f38d-a381-49fe-b487-e606f741cdb1 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:24.885002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570f38d-a381-49fe-b487-e606f741cdb1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:24.885262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570f38d-a381-49fe-b487-e606f741cdb1 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:24.886608 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 120/237] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:24.892058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae67191-260b-4466-8952-ef02dfe08a23 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:24.892547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae67191-260b-4466-8952-ef02dfe08a23 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:24.892632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae67191-260b-4466-8952-ef02dfe08a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:24.921717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cae67191-260b-4466-8952-ef02dfe08a23 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:24.922944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cae67191-260b-4466-8952-ef02dfe08a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:24.973795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:25.009718 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 118/238] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:25.095821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abacca-0963-46cc-ae3d-173bd420bd2d None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:25.096166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abacca-0963-46cc-ae3d-173bd420bd2d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:25.096358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abacca-0963-46cc-ae3d-173bd420bd2d None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:25.097888 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 121/239] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:25.103480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7405-1d82-48c6-917c-f6fccb8e83b5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:25.103761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7405-1d82-48c6-917c-f6fccb8e83b5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:25.104027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7405-1d82-48c6-917c-f6fccb8e83b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:25.125353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-276a7405-1d82-48c6-917c-f6fccb8e83b5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:25.126539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-276a7405-1d82-48c6-917c-f6fccb8e83b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:25.147789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abacca-0963-46cc-ae3d-173bd420bd2d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:25.230906 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 119/240] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:25.320940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abacca-0963-46cc-ae3d-173bd420bd2d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:25.368337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f69f2c-6aa5-4d28-87cc-09e898e5a39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:25.368999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f69f2c-6aa5-4d28-87cc-09e898e5a39b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:25.369272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f69f2c-6aa5-4d28-87cc-09e898e5a39b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:25.369533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f69f2c-6aa5-4d28-87cc-09e898e5a39b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:25.384906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f69f2c-6aa5-4d28-87cc-09e898e5a39b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-00f69f2c-6aa5-4d28-87cc-09e898e5a39b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:25.389820 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:25.390689 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:25.391470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:25.391470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:25.391470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:25.391470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:25.391470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:25.391470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:25.391470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:25.391470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:25.391470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:25.391470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:25.391470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:25.391470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:25.395503 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 122/241] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:25.396868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a7405-1d82-48c6-917c-f6fccb8e83b5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:25.425170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f69f2c-6aa5-4d28-87cc-09e898e5a39b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:25.547623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a7405-1d82-48c6-917c-f6fccb8e83b5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:25.564708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:25.568179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:25.604651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c4cdd-41d3-4f8c-beeb-189d3fd95d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:25.605355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c4cdd-41d3-4f8c-beeb-189d3fd95d65 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:25.605633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c4cdd-41d3-4f8c-beeb-189d3fd95d65 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:25.606347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c4cdd-41d3-4f8c-beeb-189d3fd95d65 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:25.607129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c4cdd-41d3-4f8c-beeb-189d3fd95d65 None admin] RBAC: Adding query filter params (name=reader) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:25.607433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c4cdd-41d3-4f8c-beeb-189d3fd95d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:25.608943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c4cdd-41d3-4f8c-beeb-189d3fd95d65 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:25.617845 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 120/242] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:25.618647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c4cdd-41d3-4f8c-beeb-189d3fd95d65 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:25.623789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f69f2c-6aa5-4d28-87cc-09e898e5a39b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:25.664794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb8693-3d7e-42cc-b814-25b72cf65672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:25.665743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb8693-3d7e-42cc-b814-25b72cf65672 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:25.666160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb8693-3d7e-42cc-b814-25b72cf65672 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:25.666561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb8693-3d7e-42cc-b814-25b72cf65672 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:25.667667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb8693-3d7e-42cc-b814-25b72cf65672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:25.669777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb8693-3d7e-42cc-b814-25b72cf65672 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:25.671947 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 123/243] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:38:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:25.673021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb8693-3d7e-42cc-b814-25b72cf65672 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:25.732166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c4cdd-41d3-4f8c-beeb-189d3fd95d65 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:25.768796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e72557-da61-4ea5-aefa-ef65b8a18d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:25.769671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e72557-da61-4ea5-aefa-ef65b8a18d3f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:25.770044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e72557-da61-4ea5-aefa-ef65b8a18d3f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:25.770408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e72557-da61-4ea5-aefa-ef65b8a18d3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:25.781969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e72557-da61-4ea5-aefa-ef65b8a18d3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-45e72557-da61-4ea5-aefa-ef65b8a18d3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:25.788592 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:25.790266 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:25.792177 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:25.792177 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:25.792177 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:25.792177 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:25.792177 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:25.792177 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:25.792177 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:25.792177 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:25.792177 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:25.792177 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:25.792177 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:25.792177 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:25.798494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb8693-3d7e-42cc-b814-25b72cf65672 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:25.798731 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 121/244] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:25.798731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e72557-da61-4ea5-aefa-ef65b8a18d3f None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:25.827060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:25.831894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:25.836089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554d835-899e-4b3a-9a08-453f46a85090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:25.836787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554d835-899e-4b3a-9a08-453f46a85090 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:25.837065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554d835-899e-4b3a-9a08-453f46a85090 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:25.837645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554d835-899e-4b3a-9a08-453f46a85090 None admin] PATH_INFO: `/v3/users/d199677cbf954c6da1ebc5074985bf89` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:25.854471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554d835-899e-4b3a-9a08-453f46a85090 None admin] RBAC: Authorizing `identity:get_user(user_id=d199677cbf954c6da1ebc5074985bf89)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:25.856976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554d835-899e-4b3a-9a08-453f46a85090 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:25.859096 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 124/245] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:25 2022] GET /identity/v3/users/d199677cbf954c6da1ebc5074985bf89 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:25.860144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554d835-899e-4b3a-9a08-453f46a85090 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:25.915817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e72557-da61-4ea5-aefa-ef65b8a18d3f None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:25.954250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8970f-b31d-4f5c-a1a9-ed8c58226183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:25.955148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8970f-b31d-4f5c-a1a9-ed8c58226183 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:25.955623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8970f-b31d-4f5c-a1a9-ed8c58226183 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:25.956064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8970f-b31d-4f5c-a1a9-ed8c58226183 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:25.956984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8970f-b31d-4f5c-a1a9-ed8c58226183 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:25.958699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8970f-b31d-4f5c-a1a9-ed8c58226183 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:25.985296 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 122/246] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:38:25 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:38:25.986451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8970f-b31d-4f5c-a1a9-ed8c58226183 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:25.989648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554d835-899e-4b3a-9a08-453f46a85090 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:26.037582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc14cb-2859-4af0-aa10-7d77d4db1532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:26.038919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc14cb-2859-4af0-aa10-7d77d4db1532 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:26.039504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc14cb-2859-4af0-aa10-7d77d4db1532 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:26.040106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc14cb-2859-4af0-aa10-7d77d4db1532 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:26.041262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc14cb-2859-4af0-aa10-7d77d4db1532 None admin] RBAC: Adding query filter params (name=member) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:26.041835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc14cb-2859-4af0-aa10-7d77d4db1532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:26.043970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc14cb-2859-4af0-aa10-7d77d4db1532 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:26.061248 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 125/247] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:26.070915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc14cb-2859-4af0-aa10-7d77d4db1532 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:26.165251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:26.181649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:26.204302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8970f-b31d-4f5c-a1a9-ed8c58226183 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:26.256161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40511c4-8d82-45c0-ad61-ceb0826bdade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:26.256766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40511c4-8d82-45c0-ad61-ceb0826bdade None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:26.257044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40511c4-8d82-45c0-ad61-ceb0826bdade None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:26.257304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40511c4-8d82-45c0-ad61-ceb0826bdade None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:26.267130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc14cb-2859-4af0-aa10-7d77d4db1532 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:26.313281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d915527-cb5d-4cb9-bff4-7ab279952b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:26.314109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d915527-cb5d-4cb9-bff4-7ab279952b3d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:26.314390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d915527-cb5d-4cb9-bff4-7ab279952b3d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:26.314659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d915527-cb5d-4cb9-bff4-7ab279952b3d None admin] PATH_INFO: `/v3/users/7db0af1475de44c58b083d498c0d87dc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:26.315814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d915527-cb5d-4cb9-bff4-7ab279952b3d None admin] RBAC: Authorizing `identity:get_user(user_id=7db0af1475de44c58b083d498c0d87dc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:26.317327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d915527-cb5d-4cb9-bff4-7ab279952b3d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:26.318974 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 126/248] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:26 2022] GET /identity/v3/users/7db0af1475de44c58b083d498c0d87dc => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:26.320113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd7e2d-1322-4ebc-aae8-97a3ebcc4af8 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:26.321922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd7e2d-1322-4ebc-aae8-97a3ebcc4af8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:26.321922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd7e2d-1322-4ebc-aae8-97a3ebcc4af8 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:26.322090 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 127/249] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:26.327663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd7e2d-1322-4ebc-aae8-97a3ebcc4af8 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:26.345787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40511c4-8d82-45c0-ad61-ceb0826bdade None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:26.347715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40511c4-8d82-45c0-ad61-ceb0826bdade None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:26.466881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd7e2d-1322-4ebc-aae8-97a3ebcc4af8 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:26.470055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:26.490266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:26.491798 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 123/250] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:26.493451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9b35c-8604-4cb7-b5e6-8e518b020eb6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:26.493909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9b35c-8604-4cb7-b5e6-8e518b020eb6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:26.494462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9b35c-8604-4cb7-b5e6-8e518b020eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:26.512750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c0932-8611-4572-9b2a-2dcce793b123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:26.513416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c0932-8611-4572-9b2a-2dcce793b123 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:26.513762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c0932-8611-4572-9b2a-2dcce793b123 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:26.514099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c0932-8611-4572-9b2a-2dcce793b123 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:26.515348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c0932-8611-4572-9b2a-2dcce793b123 None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:26.516179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-18f9b35c-8604-4cb7-b5e6-8e518b020eb6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:26.517074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c0932-8611-4572-9b2a-2dcce793b123 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:26.517827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-18f9b35c-8604-4cb7-b5e6-8e518b020eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:26.519289 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 128/251] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:26 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:26.520202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c0932-8611-4572-9b2a-2dcce793b123 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:26.608077 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 124/252] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:26.610043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9b35c-8604-4cb7-b5e6-8e518b020eb6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:26.669330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c0932-8611-4572-9b2a-2dcce793b123 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:26.707094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d0ede-7d95-49a7-b66f-7299537d3bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:26.707741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d0ede-7d95-49a7-b66f-7299537d3bc0 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:26.708068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d0ede-7d95-49a7-b66f-7299537d3bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:26.708334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d0ede-7d95-49a7-b66f-7299537d3bc0 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:26.709480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d0ede-7d95-49a7-b66f-7299537d3bc0 None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:26.710940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d0ede-7d95-49a7-b66f-7299537d3bc0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:26.713015 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 129/253] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:26 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:26.719429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d0ede-7d95-49a7-b66f-7299537d3bc0 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:26.766594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9b35c-8604-4cb7-b5e6-8e518b020eb6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:26.786474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:26.798500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:26.841178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d0ede-7d95-49a7-b66f-7299537d3bc0 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:26.853976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e6304-3cd7-4b87-aefb-f019a331b142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:26.855000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e6304-3cd7-4b87-aefb-f019a331b142 None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:26.855468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e6304-3cd7-4b87-aefb-f019a331b142 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:26.855982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e6304-3cd7-4b87-aefb-f019a331b142 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b/users/7db0af1475de44c58b083d498c0d87dc/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:26.861647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e6304-3cd7-4b87-aefb-f019a331b142 None admin] RBAC: Authorizing `identity:create_grant(project_id=66eda7a4830f4fdeb286e09ae72f278b, user_id=7db0af1475de44c58b083d498c0d87dc, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:26.863371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e6304-3cd7-4b87-aefb-f019a331b142 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:26.879525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763b254-af84-4284-bf02-f6ca78c32bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:26.880185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763b254-af84-4284-bf02-f6ca78c32bb9 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:26.880520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763b254-af84-4284-bf02-f6ca78c32bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:26.880785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763b254-af84-4284-bf02-f6ca78c32bb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:26.881567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763b254-af84-4284-bf02-f6ca78c32bb9 None admin] RBAC: Adding query filter params (role.id=bdbaea91119244b9aa4a822c613c2bb8, scope.project.id=66eda7a4830f4fdeb286e09ae72f278b, user.id=d199677cbf954c6da1ebc5074985bf89) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:26.881876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763b254-af84-4284-bf02-f6ca78c32bb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:26.883530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763b254-af84-4284-bf02-f6ca78c32bb9 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:26.884838 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 125/254] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:38:26 2022] PUT /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b/users/7db0af1475de44c58b083d498c0d87dc/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:38:26.886517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883924f-2df6-4548-acc8-079a443ab596 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:26.886982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883924f-2df6-4548-acc8-079a443ab596 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:26.887572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883924f-2df6-4548-acc8-079a443ab596 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:26.889162 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 126/255] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:26.895992 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 130/256] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:26 2022] GET /identity/v3/role_assignments?user.id=d199677cbf954c6da1ebc5074985bf89&scope.project.id=66eda7a4830f4fdeb286e09ae72f278b&role.id=bdbaea91119244b9aa4a822c613c2bb8 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:26.897698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decf770-a923-4702-8470-7b30a4d54d36 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:26.898126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decf770-a923-4702-8470-7b30a4d54d36 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:26.898555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decf770-a923-4702-8470-7b30a4d54d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:26.934466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9decf770-a923-4702-8470-7b30a4d54d36 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:26.936817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9decf770-a923-4702-8470-7b30a4d54d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:27.034712 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 127/257] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:27.100085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763b254-af84-4284-bf02-f6ca78c32bb9 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:27.220661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763b254-af84-4284-bf02-f6ca78c32bb9 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:27.238744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:27.258610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e37cd-6f87-4f8d-990b-89dd56c830f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:27.259251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e37cd-6f87-4f8d-990b-89dd56c830f6 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:27.259527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e37cd-6f87-4f8d-990b-89dd56c830f6 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:27.259852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e37cd-6f87-4f8d-990b-89dd56c830f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:27.270978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e37cd-6f87-4f8d-990b-89dd56c830f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-220e37cd-6f87-4f8d-990b-89dd56c830f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:27.276357 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:27.277421 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:27.278419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:27.278419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:27.278419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:27.278419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:27.278419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:27.278419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:27.278419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:27.278419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:27.278419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:27.278419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:27.278419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:27.278419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:27.278419 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 131/258] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:38:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:27.285377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decf770-a923-4702-8470-7b30a4d54d36 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:27.423089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decf770-a923-4702-8470-7b30a4d54d36 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:27.430266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:27.471894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a569159-18a9-4293-b1d0-1db60a7246d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:27.472536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a569159-18a9-4293-b1d0-1db60a7246d4 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:27.472814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a569159-18a9-4293-b1d0-1db60a7246d4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:27.473083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a569159-18a9-4293-b1d0-1db60a7246d4 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:27.473905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a569159-18a9-4293-b1d0-1db60a7246d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:27.474207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a569159-18a9-4293-b1d0-1db60a7246d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:27.475588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a569159-18a9-4293-b1d0-1db60a7246d4 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:27.487122 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 128/259] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:27.494200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e37cd-6f87-4f8d-990b-89dd56c830f6 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:27.551006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a569159-18a9-4293-b1d0-1db60a7246d4 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:27.662236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e37cd-6f87-4f8d-990b-89dd56c830f6 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:27.732592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a569159-18a9-4293-b1d0-1db60a7246d4 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:27.746173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29fe38-55be-4de7-91b3-3dc1f4b35cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:27.746950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29fe38-55be-4de7-91b3-3dc1f4b35cc0 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:27.747129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29fe38-55be-4de7-91b3-3dc1f4b35cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:27.747596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29fe38-55be-4de7-91b3-3dc1f4b35cc0 None admin] PATH_INFO: `/v3/users/83bd8eb8f3bc449baaa07a558c5136d9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:27.752415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29fe38-55be-4de7-91b3-3dc1f4b35cc0 None admin] RBAC: Authorizing `identity:get_user(user_id=83bd8eb8f3bc449baaa07a558c5136d9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:27.752415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29fe38-55be-4de7-91b3-3dc1f4b35cc0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:27.754538 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 132/260] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:27 2022] GET /identity/v3/users/83bd8eb8f3bc449baaa07a558c5136d9 => generated 283 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:27.763502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29fe38-55be-4de7-91b3-3dc1f4b35cc0 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:27.805214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8c318-8969-4524-9254-7e7868125692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:27.806192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8c318-8969-4524-9254-7e7868125692 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:27.806670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8c318-8969-4524-9254-7e7868125692 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:27.807121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8c318-8969-4524-9254-7e7868125692 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:27.829344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8c318-8969-4524-9254-7e7868125692 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-83e8c318-8969-4524-9254-7e7868125692 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:27.834530 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:27.835487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:27.836268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:27.836268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:27.836268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:27.836268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:27.836268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:27.836268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:27.836268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:27.836268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:27.836268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:27.836268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:27.836268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:27.836268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:27.836268 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 129/261] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:38:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:38:27.844237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8c318-8969-4524-9254-7e7868125692 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:27.861912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:27.881029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:27.963715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8c318-8969-4524-9254-7e7868125692 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:27.977979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29fe38-55be-4de7-91b3-3dc1f4b35cc0 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:28.016253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b7791-a2a1-45a0-ae07-4497541411d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:28.017138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b7791-a2a1-45a0-ae07-4497541411d6 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:28.017549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b7791-a2a1-45a0-ae07-4497541411d6 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:28.017947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b7791-a2a1-45a0-ae07-4497541411d6 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:28.018746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b7791-a2a1-45a0-ae07-4497541411d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:28.019207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b7791-a2a1-45a0-ae07-4497541411d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:28.020757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b7791-a2a1-45a0-ae07-4497541411d6 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:28.042184 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 130/262] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:38:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:28.049591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b7791-a2a1-45a0-ae07-4497541411d6 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:28.053188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e05c4a-ecd2-4d12-bc40-2b7279d9722d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:28.053814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e05c4a-ecd2-4d12-bc40-2b7279d9722d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:28.054135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e05c4a-ecd2-4d12-bc40-2b7279d9722d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:28.054409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e05c4a-ecd2-4d12-bc40-2b7279d9722d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:28.167548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e05c4a-ecd2-4d12-bc40-2b7279d9722d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:28.169114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e05c4a-ecd2-4d12-bc40-2b7279d9722d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:28.193447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:28.206749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b7791-a2a1-45a0-ae07-4497541411d6 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:28.253074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217a654-86b5-418a-a859-5d4f864534cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:28.253925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217a654-86b5-418a-a859-5d4f864534cf None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:28.254373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217a654-86b5-418a-a859-5d4f864534cf None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:28.254779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217a654-86b5-418a-a859-5d4f864534cf None admin] PATH_INFO: `/v3/users/83bd8eb8f3bc449baaa07a558c5136d9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:28.256087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217a654-86b5-418a-a859-5d4f864534cf None admin] RBAC: Authorizing `identity:get_user(user_id=83bd8eb8f3bc449baaa07a558c5136d9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:28.257714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217a654-86b5-418a-a859-5d4f864534cf None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:28.259886 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 131/263] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:28 2022] GET /identity/v3/users/83bd8eb8f3bc449baaa07a558c5136d9 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:28.265137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217a654-86b5-418a-a859-5d4f864534cf None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:28.294909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:28.406768 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 133/264] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:28.413381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e05c4a-ecd2-4d12-bc40-2b7279d9722d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:28.431125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217a654-86b5-418a-a859-5d4f864534cf None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:28.514078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c515508-6c96-4466-8137-916f19d419b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:28.514698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c515508-6c96-4466-8137-916f19d419b9 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:28.514968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c515508-6c96-4466-8137-916f19d419b9 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:28.515226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c515508-6c96-4466-8137-916f19d419b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:28.578516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:28.610965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e05c4a-ecd2-4d12-bc40-2b7279d9722d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:28.662995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd53b2c-c568-4672-b643-1f8f76805e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:28.663679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd53b2c-c568-4672-b643-1f8f76805e0f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:28.663956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd53b2c-c568-4672-b643-1f8f76805e0f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:28.664225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd53b2c-c568-4672-b643-1f8f76805e0f None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:28.665573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd53b2c-c568-4672-b643-1f8f76805e0f None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:28.667200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd53b2c-c568-4672-b643-1f8f76805e0f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:28.669674 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 134/265] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:28 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:28.675011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd53b2c-c568-4672-b643-1f8f76805e0f None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:28.694762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:28.698476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c515508-6c96-4466-8137-916f19d419b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:28.700024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c515508-6c96-4466-8137-916f19d419b9 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:28.828702 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 132/266] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:28.837323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c515508-6c96-4466-8137-916f19d419b9 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:28.842107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd53b2c-c568-4672-b643-1f8f76805e0f None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:28.908021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:28.910575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8af303-aef7-4b83-bbb2-c068c1e347cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:28.911199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8af303-aef7-4b83-bbb2-c068c1e347cd None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:28.911539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8af303-aef7-4b83-bbb2-c068c1e347cd None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:28.911810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8af303-aef7-4b83-bbb2-c068c1e347cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:28.912584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8af303-aef7-4b83-bbb2-c068c1e347cd None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=66eda7a4830f4fdeb286e09ae72f278b, user.id=83bd8eb8f3bc449baaa07a558c5136d9) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:28.912928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8af303-aef7-4b83-bbb2-c068c1e347cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:28.914420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8af303-aef7-4b83-bbb2-c068c1e347cd None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:28.922169 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 135/267] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:28 2022] GET /identity/v3/role_assignments?user.id=83bd8eb8f3bc449baaa07a558c5136d9&scope.project.id=66eda7a4830f4fdeb286e09ae72f278b&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:28.992152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c515508-6c96-4466-8137-916f19d419b9 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:29.045456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1d326-b7ba-4a71-ab7f-76e6e5da888f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:29.046090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1d326-b7ba-4a71-ab7f-76e6e5da888f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:29.046357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1d326-b7ba-4a71-ab7f-76e6e5da888f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:29.046620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1d326-b7ba-4a71-ab7f-76e6e5da888f None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:29.053132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1d326-b7ba-4a71-ab7f-76e6e5da888f None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:29.054640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1d326-b7ba-4a71-ab7f-76e6e5da888f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:29.056630 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 133/268] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:28 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:29.063246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8af303-aef7-4b83-bbb2-c068c1e347cd None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:29.173896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:29.201581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8af303-aef7-4b83-bbb2-c068c1e347cd None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:29.266826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148ad50-fb3a-46b1-91e0-9186d3bef84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:29.267353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148ad50-fb3a-46b1-91e0-9186d3bef84f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:29.267621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148ad50-fb3a-46b1-91e0-9186d3bef84f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:29.267881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148ad50-fb3a-46b1-91e0-9186d3bef84f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:29.268640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148ad50-fb3a-46b1-91e0-9186d3bef84f None admin] RBAC: Adding query filter params (role.id=f8a2ea13798443c48750e9baa83c7461, scope.project.id=66eda7a4830f4fdeb286e09ae72f278b, user.id=83bd8eb8f3bc449baaa07a558c5136d9) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:29.268942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148ad50-fb3a-46b1-91e0-9186d3bef84f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:29.270400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148ad50-fb3a-46b1-91e0-9186d3bef84f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:29.281079 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 136/269] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:29 2022] GET /identity/v3/role_assignments?user.id=83bd8eb8f3bc449baaa07a558c5136d9&scope.project.id=66eda7a4830f4fdeb286e09ae72f278b&role.id=f8a2ea13798443c48750e9baa83c7461 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:29.428894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d50a3-861c-4a12-9fed-13ee8e753a73 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:29.429252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d50a3-861c-4a12-9fed-13ee8e753a73 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:29.429445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d50a3-861c-4a12-9fed-13ee8e753a73 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:29.430848 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 134/270] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:29.438475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070c621-6cf1-4968-b987-48d97cd1c151 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:29.439120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070c621-6cf1-4968-b987-48d97cd1c151 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:29.439400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070c621-6cf1-4968-b987-48d97cd1c151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:29.464599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d070c621-6cf1-4968-b987-48d97cd1c151 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:29.466292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d070c621-6cf1-4968-b987-48d97cd1c151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:29.562916 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 137/271] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:29.643073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accc258-5a13-4af3-8fc8-e6912ab0dc4a None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:29.643396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accc258-5a13-4af3-8fc8-e6912ab0dc4a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:29.643605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accc258-5a13-4af3-8fc8-e6912ab0dc4a None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:29.645198 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 135/272] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:29.651027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18c2f8-f489-4a5f-bb73-df6726213547 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:29.651597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18c2f8-f489-4a5f-bb73-df6726213547 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:29.652101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18c2f8-f489-4a5f-bb73-df6726213547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:29.676242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-be18c2f8-f489-4a5f-bb73-df6726213547 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:29.677806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-be18c2f8-f489-4a5f-bb73-df6726213547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:29.761106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:29.768098 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 138/273] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:29.997338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accc258-5a13-4af3-8fc8-e6912ab0dc4a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:30.077876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef729eb-07f2-4aae-8e5b-c8f431ab8f15 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:30.078451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef729eb-07f2-4aae-8e5b-c8f431ab8f15 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:30.078974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef729eb-07f2-4aae-8e5b-c8f431ab8f15 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:30.080877 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 139/274] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:30.087004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805ac5a-ebde-4fe9-a4ba-04ad4971ab10 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:30.087282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805ac5a-ebde-4fe9-a4ba-04ad4971ab10 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:30.087550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805ac5a-ebde-4fe9-a4ba-04ad4971ab10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:30.116068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0805ac5a-ebde-4fe9-a4ba-04ad4971ab10 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:30.118163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0805ac5a-ebde-4fe9-a4ba-04ad4971ab10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:30.133867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accc258-5a13-4af3-8fc8-e6912ab0dc4a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:30.167937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:30.179855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bd2e5-2ab0-492a-a8a5-91d43ebcc09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:30.180511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bd2e5-2ab0-492a-a8a5-91d43ebcc09c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:30.180784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bd2e5-2ab0-492a-a8a5-91d43ebcc09c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:30.181048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bd2e5-2ab0-492a-a8a5-91d43ebcc09c None admin] PATH_INFO: `/v3/domains/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:30.182162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bd2e5-2ab0-492a-a8a5-91d43ebcc09c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:30.183766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bd2e5-2ab0-492a-a8a5-91d43ebcc09c None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:30.186783 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 136/275] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:38:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:30.187564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bd2e5-2ab0-492a-a8a5-91d43ebcc09c None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:30.213573 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 140/276] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:30.220629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805ac5a-ebde-4fe9-a4ba-04ad4971ab10 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:30.300059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bd2e5-2ab0-492a-a8a5-91d43ebcc09c None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:30.330796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805ac5a-ebde-4fe9-a4ba-04ad4971ab10 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:30.335914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139329c0-6699-4cad-ada5-0b4b084240fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:30.336569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139329c0-6699-4cad-ada5-0b4b084240fa None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:30.336839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139329c0-6699-4cad-ada5-0b4b084240fa None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:30.337099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139329c0-6699-4cad-ada5-0b4b084240fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:30.348047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139329c0-6699-4cad-ada5-0b4b084240fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-139329c0-6699-4cad-ada5-0b4b084240fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:30.352644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:30.353618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 137/277] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:30.360410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0567f1b-2640-492d-aaa6-9ad555d925ec None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:30.361287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0567f1b-2640-492d-aaa6-9ad555d925ec None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:30.361287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0567f1b-2640-492d-aaa6-9ad555d925ec None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:30.361287 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 138/278] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:30.362043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0567f1b-2640-492d-aaa6-9ad555d925ec None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:30.369146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:30.376025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25a2fb-6943-4eea-9718-d8475aa0d956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:30.376674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25a2fb-6943-4eea-9718-d8475aa0d956 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:30.376948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25a2fb-6943-4eea-9718-d8475aa0d956 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:30.377212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25a2fb-6943-4eea-9718-d8475aa0d956 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:30.377847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25a2fb-6943-4eea-9718-d8475aa0d956 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:30.379311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25a2fb-6943-4eea-9718-d8475aa0d956 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:30.411189 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 141/279] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:38:30 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:38:30.411189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d5854-6ae8-400b-9d20-234df9838881 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:30.411189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d5854-6ae8-400b-9d20-234df9838881 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:30.411189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d5854-6ae8-400b-9d20-234df9838881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:30.425597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c16d5854-6ae8-400b-9d20-234df9838881 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:30.426940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c16d5854-6ae8-400b-9d20-234df9838881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:30.435880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:30.508144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0567f1b-2640-492d-aaa6-9ad555d925ec None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:30.529705 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 142/280] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:30.551379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d5854-6ae8-400b-9d20-234df9838881 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:30.554413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425a80d-79df-4e71-b6f7-0769c5b182cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:30.556054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425a80d-79df-4e71-b6f7-0769c5b182cf None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:30.556336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425a80d-79df-4e71-b6f7-0769c5b182cf None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:30.556672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425a80d-79df-4e71-b6f7-0769c5b182cf None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:30.557359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425a80d-79df-4e71-b6f7-0769c5b182cf None admin] RBAC: Adding query filter params (name=member) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:30.557661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425a80d-79df-4e71-b6f7-0769c5b182cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:30.559228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425a80d-79df-4e71-b6f7-0769c5b182cf None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:30.570365 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 139/281] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:30.576710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425a80d-79df-4e71-b6f7-0769c5b182cf None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:30.668310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d5854-6ae8-400b-9d20-234df9838881 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:30.694288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:30.710312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425a80d-79df-4e71-b6f7-0769c5b182cf None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:30.716356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f9c9c-29d8-47c1-8c67-69da8312b84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:30.717037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f9c9c-29d8-47c1-8c67-69da8312b84a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:30.717311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f9c9c-29d8-47c1-8c67-69da8312b84a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:30.717581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f9c9c-29d8-47c1-8c67-69da8312b84a None admin] PATH_INFO: `/v3/roles/member` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:30.726517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:30.729384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f9c9c-29d8-47c1-8c67-69da8312b84a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-a87f9c9c-29d8-47c1-8c67-69da8312b84a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:30.734637 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:30.735522 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:30.736325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:30.736325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:30.736325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:30.736325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:30.736325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:30.736325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:30.736325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:30.736325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:30.736325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:30.736325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:30.736325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:30.736325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:30.738665 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 143/282] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:38:30.745364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f9c9c-29d8-47c1-8c67-69da8312b84a None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:30.751640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02dd85-06a6-4b7a-b71b-c81566fa812a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:30.752277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02dd85-06a6-4b7a-b71b-c81566fa812a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:30.752611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02dd85-06a6-4b7a-b71b-c81566fa812a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:30.752874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02dd85-06a6-4b7a-b71b-c81566fa812a None admin] PATH_INFO: `/v3/users/bca3a824455d45fbbdcd5016a27a2d43` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:30.769652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02dd85-06a6-4b7a-b71b-c81566fa812a None admin] RBAC: Authorizing `identity:get_user(user_id=bca3a824455d45fbbdcd5016a27a2d43)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:30.771138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02dd85-06a6-4b7a-b71b-c81566fa812a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:30.773190 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 140/283] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:30 2022] GET /identity/v3/users/bca3a824455d45fbbdcd5016a27a2d43 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:30.781065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02dd85-06a6-4b7a-b71b-c81566fa812a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:30.881913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f9c9c-29d8-47c1-8c67-69da8312b84a None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:30.919802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02dd85-06a6-4b7a-b71b-c81566fa812a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:30.923762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd017b6-c01d-437e-9473-b2fcbf8d9276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:30.924654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd017b6-c01d-437e-9473-b2fcbf8d9276 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:30.925125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd017b6-c01d-437e-9473-b2fcbf8d9276 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:30.925608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd017b6-c01d-437e-9473-b2fcbf8d9276 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:30.926467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd017b6-c01d-437e-9473-b2fcbf8d9276 None admin] RBAC: Adding query filter params (name=member) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:30.926959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd017b6-c01d-437e-9473-b2fcbf8d9276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:30.929000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd017b6-c01d-437e-9473-b2fcbf8d9276 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:30.943699 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 144/284] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:30.945359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd017b6-c01d-437e-9473-b2fcbf8d9276 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:30.958074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb72ddc-627d-4c41-9660-3b3dde4d83e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:30.958675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb72ddc-627d-4c41-9660-3b3dde4d83e7 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:30.958948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb72ddc-627d-4c41-9660-3b3dde4d83e7 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:30.959207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb72ddc-627d-4c41-9660-3b3dde4d83e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:30.959918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb72ddc-627d-4c41-9660-3b3dde4d83e7 None admin] RBAC: Adding query filter params (role.id=5eaf9701f37e480bb87c89f7391d5d3a, user.id=bca3a824455d45fbbdcd5016a27a2d43, scope.system=all) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:30.960215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb72ddc-627d-4c41-9660-3b3dde4d83e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:30.961766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb72ddc-627d-4c41-9660-3b3dde4d83e7 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:30.967741 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 141/285] 10.222.0.23 () {56 vars in 1308 bytes} [Sat Aug 20 20:38:30 2022] GET /identity/v3/role_assignments?user.id=bca3a824455d45fbbdcd5016a27a2d43&scope.system=all&role.id=5eaf9701f37e480bb87c89f7391d5d3a => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:30.972749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb72ddc-627d-4c41-9660-3b3dde4d83e7 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:30.998215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:31.062605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:31.077313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd017b6-c01d-437e-9473-b2fcbf8d9276 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:31.121120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3942e36-8e0d-4323-ba75-213ad530303e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:31.125617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3942e36-8e0d-4323-ba75-213ad530303e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:31.126930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3942e36-8e0d-4323-ba75-213ad530303e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:31.128197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3942e36-8e0d-4323-ba75-213ad530303e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:31.147925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3942e36-8e0d-4323-ba75-213ad530303e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-d3942e36-8e0d-4323-ba75-213ad530303e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:31.154912 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:31.155877 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:31.156696 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:31.156696 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:31.156696 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:31.156696 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:31.156696 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:31.156696 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:31.156696 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:31.156696 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:31.156696 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:31.156696 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:31.156696 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:31.156696 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:31.160531 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 145/286] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:31.167395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3942e36-8e0d-4323-ba75-213ad530303e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:31.235767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb72ddc-627d-4c41-9660-3b3dde4d83e7 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:31.300978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74807bb4-9ae4-4c08-82c7-44e2734e3d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:31.301613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74807bb4-9ae4-4c08-82c7-44e2734e3d27 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:31.301890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74807bb4-9ae4-4c08-82c7-44e2734e3d27 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:31.302149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74807bb4-9ae4-4c08-82c7-44e2734e3d27 None admin] PATH_INFO: `/v3/users/7db0af1475de44c58b083d498c0d87dc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:31.304244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74807bb4-9ae4-4c08-82c7-44e2734e3d27 None admin] RBAC: Authorizing `identity:get_user(user_id=7db0af1475de44c58b083d498c0d87dc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:31.305720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74807bb4-9ae4-4c08-82c7-44e2734e3d27 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:31.314012 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 142/287] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:30 2022] GET /identity/v3/users/7db0af1475de44c58b083d498c0d87dc => generated 297 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:31.320260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74807bb4-9ae4-4c08-82c7-44e2734e3d27 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:31.351581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:31.358010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3942e36-8e0d-4323-ba75-213ad530303e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:31.396568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6e3d5-0cd2-4753-87d4-e7fbeac8d3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:31.397603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6e3d5-0cd2-4753-87d4-e7fbeac8d3a0 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:31.398153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6e3d5-0cd2-4753-87d4-e7fbeac8d3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:31.398630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6e3d5-0cd2-4753-87d4-e7fbeac8d3a0 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:31.399575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6e3d5-0cd2-4753-87d4-e7fbeac8d3a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:31.400217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6e3d5-0cd2-4753-87d4-e7fbeac8d3a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:31.401917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6e3d5-0cd2-4753-87d4-e7fbeac8d3a0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:31.411707 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 146/288] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:31.419980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6e3d5-0cd2-4753-87d4-e7fbeac8d3a0 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:31.477043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74807bb4-9ae4-4c08-82c7-44e2734e3d27 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:31.479421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:31.513494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24776071-e7b2-4fda-aea5-5bcca3ca6c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:31.514099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24776071-e7b2-4fda-aea5-5bcca3ca6c98 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:31.514369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24776071-e7b2-4fda-aea5-5bcca3ca6c98 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:31.514631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24776071-e7b2-4fda-aea5-5bcca3ca6c98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:31.590312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24776071-e7b2-4fda-aea5-5bcca3ca6c98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:31.591787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24776071-e7b2-4fda-aea5-5bcca3ca6c98 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:31.686771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6e3d5-0cd2-4753-87d4-e7fbeac8d3a0 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:31.698220 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 143/289] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:31.704311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24776071-e7b2-4fda-aea5-5bcca3ca6c98 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:31.736743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:31.812915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24776071-e7b2-4fda-aea5-5bcca3ca6c98 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:31.813106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7466f6-104e-4e2c-88e2-7a50c046506c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:31.813714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7466f6-104e-4e2c-88e2-7a50c046506c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:31.813984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7466f6-104e-4e2c-88e2-7a50c046506c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:31.814250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7466f6-104e-4e2c-88e2-7a50c046506c None admin] PATH_INFO: `/v3/users/d199677cbf954c6da1ebc5074985bf89` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:31.817808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7466f6-104e-4e2c-88e2-7a50c046506c None admin] RBAC: Authorizing `identity:get_user(user_id=d199677cbf954c6da1ebc5074985bf89)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:31.819259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7466f6-104e-4e2c-88e2-7a50c046506c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:31.821191 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 147/290] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:31 2022] GET /identity/v3/users/d199677cbf954c6da1ebc5074985bf89 => generated 297 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:31.829292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7466f6-104e-4e2c-88e2-7a50c046506c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:31.849861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c417e-c6af-4667-880d-615126bbd3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:31.850503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c417e-c6af-4667-880d-615126bbd3da None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:31.850781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c417e-c6af-4667-880d-615126bbd3da None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:31.851050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c417e-c6af-4667-880d-615126bbd3da None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:31.852187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c417e-c6af-4667-880d-615126bbd3da None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:31.853697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c417e-c6af-4667-880d-615126bbd3da None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:31.858327 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 144/291] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:31 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:31.865443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c417e-c6af-4667-880d-615126bbd3da None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:31.882195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:31.946074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:31.983339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7466f6-104e-4e2c-88e2-7a50c046506c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:32.006447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c417e-c6af-4667-880d-615126bbd3da None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:32.017576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201b20c-a678-4b6c-bbee-8c25fdbf2175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:32.018210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201b20c-a678-4b6c-bbee-8c25fdbf2175 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:32.018476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201b20c-a678-4b6c-bbee-8c25fdbf2175 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:32.018735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201b20c-a678-4b6c-bbee-8c25fdbf2175 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:32.019837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201b20c-a678-4b6c-bbee-8c25fdbf2175 None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:32.021297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201b20c-a678-4b6c-bbee-8c25fdbf2175 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:32.022925 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 148/292] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:31 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:32.028758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201b20c-a678-4b6c-bbee-8c25fdbf2175 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:32.094151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0097d3c-6e41-41e6-be58-a7a12b85ff7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:32.094812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0097d3c-6e41-41e6-be58-a7a12b85ff7c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:32.095029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0097d3c-6e41-41e6-be58-a7a12b85ff7c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:32.095296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0097d3c-6e41-41e6-be58-a7a12b85ff7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:32.096023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0097d3c-6e41-41e6-be58-a7a12b85ff7c None admin] RBAC: Adding query filter params (role.id=5eaf9701f37e480bb87c89f7391d5d3a, scope.project.id=66eda7a4830f4fdeb286e09ae72f278b, user.id=7db0af1475de44c58b083d498c0d87dc) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:32.096442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0097d3c-6e41-41e6-be58-a7a12b85ff7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:32.097994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0097d3c-6e41-41e6-be58-a7a12b85ff7c None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:32.114165 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 145/293] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:31 2022] GET /identity/v3/role_assignments?user.id=7db0af1475de44c58b083d498c0d87dc&scope.project.id=66eda7a4830f4fdeb286e09ae72f278b&role.id=5eaf9701f37e480bb87c89f7391d5d3a => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:32.121582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:32.127917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201b20c-a678-4b6c-bbee-8c25fdbf2175 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:32.171681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e78b47-b170-4e45-9860-30d982f0064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:32.172381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e78b47-b170-4e45-9860-30d982f0064e None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:32.172665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e78b47-b170-4e45-9860-30d982f0064e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:32.172929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e78b47-b170-4e45-9860-30d982f0064e None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b/users/d199677cbf954c6da1ebc5074985bf89/roles/bdbaea91119244b9aa4a822c613c2bb8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:32.175078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e78b47-b170-4e45-9860-30d982f0064e None admin] RBAC: Authorizing `identity:create_grant(project_id=66eda7a4830f4fdeb286e09ae72f278b, user_id=d199677cbf954c6da1ebc5074985bf89, role_id=bdbaea91119244b9aa4a822c613c2bb8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:32.176633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e78b47-b170-4e45-9860-30d982f0064e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:32.194217 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 149/294] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:38:32 2022] PUT /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b/users/d199677cbf954c6da1ebc5074985bf89/roles/bdbaea91119244b9aa4a822c613c2bb8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:38:33.094933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653936e0-5552-486b-818e-a9b10d025b17 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:33.095497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653936e0-5552-486b-818e-a9b10d025b17 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:33.095497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653936e0-5552-486b-818e-a9b10d025b17 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:33.096854 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 146/295] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:33.102334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afb7bf-f8f7-4cfc-8989-9fb9d9dc1030 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:33.102616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afb7bf-f8f7-4cfc-8989-9fb9d9dc1030 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:33.102881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afb7bf-f8f7-4cfc-8989-9fb9d9dc1030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:33.120800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f6afb7bf-f8f7-4cfc-8989-9fb9d9dc1030 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:33.121979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f6afb7bf-f8f7-4cfc-8989-9fb9d9dc1030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:33.195857 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 150/296] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:33.380491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3648aa4-d49a-46ed-8705-fc6c481d2023 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:33.380703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3648aa4-d49a-46ed-8705-fc6c481d2023 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:33.380947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3648aa4-d49a-46ed-8705-fc6c481d2023 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:33.382610 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 147/297] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:33.388238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716b0ea-08d2-44b3-9b3e-f3edc82d342e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:33.388548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716b0ea-08d2-44b3-9b3e-f3edc82d342e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:33.388814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716b0ea-08d2-44b3-9b3e-f3edc82d342e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:33.392715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80efd98-8e02-4628-bc20-b07dbe13f99d None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:33.393006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80efd98-8e02-4628-bc20-b07dbe13f99d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:33.393351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80efd98-8e02-4628-bc20-b07dbe13f99d None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:33.395076 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 148/298] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:33.402286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121f8d3-8a16-4c1c-adce-172ee9e82ad0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:33.402629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121f8d3-8a16-4c1c-adce-172ee9e82ad0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:33.402919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121f8d3-8a16-4c1c-adce-172ee9e82ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:33.416173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c716b0ea-08d2-44b3-9b3e-f3edc82d342e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:33.417722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c716b0ea-08d2-44b3-9b3e-f3edc82d342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:33.421003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9121f8d3-8a16-4c1c-adce-172ee9e82ad0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:33.422168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9121f8d3-8a16-4c1c-adce-172ee9e82ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:33.500490 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 149/299] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:33.508564 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 151/300] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:33.527815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121f8d3-8a16-4c1c-adce-172ee9e82ad0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:33.593538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:33.634287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121f8d3-8a16-4c1c-adce-172ee9e82ad0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:33.669805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77089169-8161-4f3c-8115-e235a27cc137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:33.670430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77089169-8161-4f3c-8115-e235a27cc137 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:33.670705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77089169-8161-4f3c-8115-e235a27cc137 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:33.670972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77089169-8161-4f3c-8115-e235a27cc137 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:33.672073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77089169-8161-4f3c-8115-e235a27cc137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:33.673550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77089169-8161-4f3c-8115-e235a27cc137 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:33.675595 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 150/301] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:38:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:33.681026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716b0ea-08d2-44b3-9b3e-f3edc82d342e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:33.702252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:33.783298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716b0ea-08d2-44b3-9b3e-f3edc82d342e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:33.817675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7a2e0-d961-4ab7-87b2-b7d2ec2ddd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:33.818276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7a2e0-d961-4ab7-87b2-b7d2ec2ddd0f None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:33.818544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7a2e0-d961-4ab7-87b2-b7d2ec2ddd0f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:33.818798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7a2e0-d961-4ab7-87b2-b7d2ec2ddd0f None admin] PATH_INFO: `/v3/groups` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:33.819469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7a2e0-d961-4ab7-87b2-b7d2ec2ddd0f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:33.821000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7a2e0-d961-4ab7-87b2-b7d2ec2ddd0f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:33.837766 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 152/302] 10.222.0.23 () {60 vars in 1134 bytes} [Sat Aug 20 20:38:33 2022] POST /identity/v3/groups => generated 229 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:38:33.838512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77089169-8161-4f3c-8115-e235a27cc137 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:33.845135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7a2e0-d961-4ab7-87b2-b7d2ec2ddd0f None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:33.938733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:33.966096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7a2e0-d961-4ab7-87b2-b7d2ec2ddd0f None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:33.979798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77089169-8161-4f3c-8115-e235a27cc137 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:34.005294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9849cf-42d0-4e71-b3e7-8e7069855059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:34.005753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9849cf-42d0-4e71-b3e7-8e7069855059 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:34.006119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9849cf-42d0-4e71-b3e7-8e7069855059 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:34.006479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9849cf-42d0-4e71-b3e7-8e7069855059 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:34.018436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9849cf-42d0-4e71-b3e7-8e7069855059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:34.021509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2abfc-b0af-47b5-a074-6fe3fcd72cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:34.022304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2abfc-b0af-47b5-a074-6fe3fcd72cff None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:34.022741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2abfc-b0af-47b5-a074-6fe3fcd72cff None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:34.023149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2abfc-b0af-47b5-a074-6fe3fcd72cff None admin] PATH_INFO: `/v3/roles/member` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-1e9849cf-42d0-4e71-b3e7-8e7069855059 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:34.024419 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:34.025401 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:34.026191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:34.026191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:34.026191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:34.026191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:34.026191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:34.026191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:34.026191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:34.026191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:34.026191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:34.026191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:34.026191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:34.026191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:34.027350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:34.030600 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 153/303] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:38:34.032072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9849cf-42d0-4e71-b3e7-8e7069855059 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:34.044158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2abfc-b0af-47b5-a074-6fe3fcd72cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-01d2abfc-b0af-47b5-a074-6fe3fcd72cff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:34.047577 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:34.048724 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:34.049513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:34.049513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:34.049513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:34.049513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:34.049513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:34.049513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:34.049513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:34.049513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:34.049513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:34.049513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:34.049513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:34.049513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:34.049513 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 151/304] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:38:34.054653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2abfc-b0af-47b5-a074-6fe3fcd72cff None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:34.147013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9849cf-42d0-4e71-b3e7-8e7069855059 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:34.181458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2abfc-b0af-47b5-a074-6fe3fcd72cff None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:34.186195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cb826-7f24-4010-ab14-ac63cd34a098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:34.187086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cb826-7f24-4010-ab14-ac63cd34a098 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:34.187555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cb826-7f24-4010-ab14-ac63cd34a098 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:34.188011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cb826-7f24-4010-ab14-ac63cd34a098 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:34.188970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cb826-7f24-4010-ab14-ac63cd34a098 None admin] RBAC: Adding query filter params (name=reader) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:34.189481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cb826-7f24-4010-ab14-ac63cd34a098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:34.191478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cb826-7f24-4010-ab14-ac63cd34a098 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:34.198855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:34.204256 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 154/305] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:34.210572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cb826-7f24-4010-ab14-ac63cd34a098 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:34.229882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabc13a-03a4-4493-836e-cb6022f45b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:34.230530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabc13a-03a4-4493-836e-cb6022f45b51 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:34.230857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabc13a-03a4-4493-836e-cb6022f45b51 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:34.231214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabc13a-03a4-4493-836e-cb6022f45b51 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:34.231873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabc13a-03a4-4493-836e-cb6022f45b51 None admin] RBAC: Adding query filter params (name=member) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:34.232176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabc13a-03a4-4493-836e-cb6022f45b51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:34.233620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabc13a-03a4-4493-836e-cb6022f45b51 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:34.242960 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 152/306] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:34.248736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabc13a-03a4-4493-836e-cb6022f45b51 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:34.313073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:34.323546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cb826-7f24-4010-ab14-ac63cd34a098 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:34.360111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063f2d0-b9b4-4b9c-b632-d47528e70f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:34.361175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063f2d0-b9b4-4b9c-b632-d47528e70f66 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:34.361751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063f2d0-b9b4-4b9c-b632-d47528e70f66 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:34.362328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063f2d0-b9b4-4b9c-b632-d47528e70f66 None admin] PATH_INFO: `/v3/users/23c9c43472884295985af7257e127fcb` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:34.366253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabc13a-03a4-4493-836e-cb6022f45b51 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:34.378579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063f2d0-b9b4-4b9c-b632-d47528e70f66 None admin] RBAC: Authorizing `identity:get_user(user_id=23c9c43472884295985af7257e127fcb)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:34.380492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063f2d0-b9b4-4b9c-b632-d47528e70f66 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:34.382622 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 155/307] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:34 2022] GET /identity/v3/users/23c9c43472884295985af7257e127fcb => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:34.388720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063f2d0-b9b4-4b9c-b632-d47528e70f66 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:34.412975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a38c47-9890-4dbb-af18-44f975c9c75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:34.413889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a38c47-9890-4dbb-af18-44f975c9c75f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:34.414161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a38c47-9890-4dbb-af18-44f975c9c75f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:34.414430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a38c47-9890-4dbb-af18-44f975c9c75f None admin] PATH_INFO: `/v3/users/bca3a824455d45fbbdcd5016a27a2d43` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:34.415811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a38c47-9890-4dbb-af18-44f975c9c75f None admin] RBAC: Authorizing `identity:get_user(user_id=bca3a824455d45fbbdcd5016a27a2d43)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:34.418052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a38c47-9890-4dbb-af18-44f975c9c75f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:34.420644 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 153/308] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:34 2022] GET /identity/v3/users/bca3a824455d45fbbdcd5016a27a2d43 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:34.426071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a38c47-9890-4dbb-af18-44f975c9c75f None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:34.460145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:34.519491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063f2d0-b9b4-4b9c-b632-d47528e70f66 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:34.555679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a38c47-9890-4dbb-af18-44f975c9c75f None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:34.561935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb572dce-eb57-43e0-8d59-e44f8745f484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:34.562741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb572dce-eb57-43e0-8d59-e44f8745f484 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:34.563104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb572dce-eb57-43e0-8d59-e44f8745f484 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:34.563458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb572dce-eb57-43e0-8d59-e44f8745f484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:34.564310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb572dce-eb57-43e0-8d59-e44f8745f484 None admin] RBAC: Adding query filter params (role.id=bdbaea91119244b9aa4a822c613c2bb8, user.id=23c9c43472884295985af7257e127fcb, scope.system=all) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:34.564722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb572dce-eb57-43e0-8d59-e44f8745f484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:34.566761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb572dce-eb57-43e0-8d59-e44f8745f484 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:34.576536 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 156/309] 10.222.0.23 () {56 vars in 1308 bytes} [Sat Aug 20 20:38:34 2022] GET /identity/v3/role_assignments?user.id=23c9c43472884295985af7257e127fcb&scope.system=all&role.id=bdbaea91119244b9aa4a822c613c2bb8 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:34.591635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:34.604594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd4828-1de5-44a4-8f1e-f88509be1756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:34.605519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd4828-1de5-44a4-8f1e-f88509be1756 None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:34.606037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd4828-1de5-44a4-8f1e-f88509be1756 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:34.606471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd4828-1de5-44a4-8f1e-f88509be1756 None admin] PATH_INFO: `/v3/system/users/bca3a824455d45fbbdcd5016a27a2d43/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:34.608174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd4828-1de5-44a4-8f1e-f88509be1756 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bca3a824455d45fbbdcd5016a27a2d43, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:34.609794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd4828-1de5-44a4-8f1e-f88509be1756 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:34.620349 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 154/310] 10.222.0.23 () {58 vars in 1334 bytes} [Sat Aug 20 20:38:34 2022] PUT /identity/v3/system/users/bca3a824455d45fbbdcd5016a27a2d43/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:38:34.660913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe890b-a156-4dd0-aef6-8a7ec9f30de8 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:34.661188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe890b-a156-4dd0-aef6-8a7ec9f30de8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:34.661507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe890b-a156-4dd0-aef6-8a7ec9f30de8 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:34.663343 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 157/311] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:34.669912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefe469-d03d-4590-80e7-be346e3a271c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:34.670221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefe469-d03d-4590-80e7-be346e3a271c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:34.670464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefe469-d03d-4590-80e7-be346e3a271c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:34.691438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1eefe469-d03d-4590-80e7-be346e3a271c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:34.692724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1eefe469-d03d-4590-80e7-be346e3a271c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:34.775490 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 155/312] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:35.102762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe890b-a156-4dd0-aef6-8a7ec9f30de8 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:35.207440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe890b-a156-4dd0-aef6-8a7ec9f30de8 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:35.228717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:35.243773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c6385-e1f9-4761-9b67-8d24496bd0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:35.244419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c6385-e1f9-4761-9b67-8d24496bd0fb None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:35.244688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c6385-e1f9-4761-9b67-8d24496bd0fb None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:35.244946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c6385-e1f9-4761-9b67-8d24496bd0fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:35.258554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c6385-e1f9-4761-9b67-8d24496bd0fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-769c6385-e1f9-4761-9b67-8d24496bd0fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:35.263566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:35.264635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:35.265423 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:35.265423 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:35.265423 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:35.265423 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:35.265423 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:35.265423 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:35.265423 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:35.265423 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:35.265423 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:35.265423 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:35.265423 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:35.265423 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:35.265423 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 158/313] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:35.270733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefe469-d03d-4590-80e7-be346e3a271c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:35.370188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefe469-d03d-4590-80e7-be346e3a271c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:35.387784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:35.406486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90fb398-4ebe-4fdd-9e02-e87e5dd81db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:35.407395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fb398-4ebe-4fdd-9e02-e87e5dd81db2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:35.407864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fb398-4ebe-4fdd-9e02-e87e5dd81db2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:35.408353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fb398-4ebe-4fdd-9e02-e87e5dd81db2 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:35.409231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90fb398-4ebe-4fdd-9e02-e87e5dd81db2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:35.409723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90fb398-4ebe-4fdd-9e02-e87e5dd81db2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:35.411310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90fb398-4ebe-4fdd-9e02-e87e5dd81db2 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:35.420343 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 156/314] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:35.429446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c6385-e1f9-4761-9b67-8d24496bd0fb None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:35.535185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c6385-e1f9-4761-9b67-8d24496bd0fb None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:35.570173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2da67d-b0f2-45b6-9389-b18b3005f2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:35.570796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2da67d-b0f2-45b6-9389-b18b3005f2de None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:35.571061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2da67d-b0f2-45b6-9389-b18b3005f2de None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:35.571319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2da67d-b0f2-45b6-9389-b18b3005f2de None admin] PATH_INFO: `/v3/users/d199677cbf954c6da1ebc5074985bf89` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:35.572455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2da67d-b0f2-45b6-9389-b18b3005f2de None admin] RBAC: Authorizing `identity:get_user(user_id=d199677cbf954c6da1ebc5074985bf89)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:35.573915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2da67d-b0f2-45b6-9389-b18b3005f2de None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:35.575888 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 159/315] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:35 2022] GET /identity/v3/users/d199677cbf954c6da1ebc5074985bf89 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:35.581837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90fb398-4ebe-4fdd-9e02-e87e5dd81db2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:35.688306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90fb398-4ebe-4fdd-9e02-e87e5dd81db2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:35.723061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f7d3c-75aa-4dfa-a5eb-6f9cb303a3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:35.723666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f7d3c-75aa-4dfa-a5eb-6f9cb303a3b4 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:35.723942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f7d3c-75aa-4dfa-a5eb-6f9cb303a3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:35.724210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f7d3c-75aa-4dfa-a5eb-6f9cb303a3b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:35.778735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:35.802888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f7d3c-75aa-4dfa-a5eb-6f9cb303a3b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:35.804352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f7d3c-75aa-4dfa-a5eb-6f9cb303a3b4 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:35.908481 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 157/316] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:35.914268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2da67d-b0f2-45b6-9389-b18b3005f2de None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:35.966300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:36.018745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2da67d-b0f2-45b6-9389-b18b3005f2de None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:36.057273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f671d5-cd8b-4767-9804-b6fd9eacc5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:36.058318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f671d5-cd8b-4767-9804-b6fd9eacc5b5 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:36.058910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f671d5-cd8b-4767-9804-b6fd9eacc5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:36.059398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f671d5-cd8b-4767-9804-b6fd9eacc5b5 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:36.061277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f671d5-cd8b-4767-9804-b6fd9eacc5b5 None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:36.062952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f671d5-cd8b-4767-9804-b6fd9eacc5b5 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:36.068783 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 160/317] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:35 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:36.071764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f7d3c-75aa-4dfa-a5eb-6f9cb303a3b4 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:36.156413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:36.176804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f7d3c-75aa-4dfa-a5eb-6f9cb303a3b4 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:36.214031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb60c62-c24e-4b09-9ad3-94c1ad1bf641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:36.214636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb60c62-c24e-4b09-9ad3-94c1ad1bf641 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:36.214969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb60c62-c24e-4b09-9ad3-94c1ad1bf641 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:36.215516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb60c62-c24e-4b09-9ad3-94c1ad1bf641 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:36.216251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb60c62-c24e-4b09-9ad3-94c1ad1bf641 None admin] RBAC: Adding query filter params (role.id=bdbaea91119244b9aa4a822c613c2bb8, scope.project.id=66eda7a4830f4fdeb286e09ae72f278b, user.id=d199677cbf954c6da1ebc5074985bf89) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:36.216640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb60c62-c24e-4b09-9ad3-94c1ad1bf641 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:36.218031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb60c62-c24e-4b09-9ad3-94c1ad1bf641 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:36.221308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e217d7-231e-469b-9c46-e5c63ddd918c None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:36.222145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e217d7-231e-469b-9c46-e5c63ddd918c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:36.222601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e217d7-231e-469b-9c46-e5c63ddd918c None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:36.224204 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 161/318] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:36.229870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e1dd5-0eaa-4afc-a841-862069b1d056 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:36.230510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e1dd5-0eaa-4afc-a841-862069b1d056 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:36.231164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e1dd5-0eaa-4afc-a841-862069b1d056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:36.236442 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 158/319] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:38:36 2022] GET /identity/v3/role_assignments?user.id=d199677cbf954c6da1ebc5074985bf89&scope.project.id=66eda7a4830f4fdeb286e09ae72f278b&role.id=bdbaea91119244b9aa4a822c613c2bb8 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:36.257662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a12e1dd5-0eaa-4afc-a841-862069b1d056 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:36.259233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a12e1dd5-0eaa-4afc-a841-862069b1d056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:36.343079 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 162/320] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:36.662141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb60c62-c24e-4b09-9ad3-94c1ad1bf641 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:36.769942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb60c62-c24e-4b09-9ad3-94c1ad1bf641 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:36.808355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4343422-6b42-4146-bff0-7850cd1401b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:36.809243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4343422-6b42-4146-bff0-7850cd1401b4 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:36.809627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4343422-6b42-4146-bff0-7850cd1401b4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:36.811719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4343422-6b42-4146-bff0-7850cd1401b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:36.812132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4343422-6b42-4146-bff0-7850cd1401b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:36.813637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4343422-6b42-4146-bff0-7850cd1401b4 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:36.815425 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 159/321] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:38:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:36.821129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e1dd5-0eaa-4afc-a841-862069b1d056 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:36.859428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750ef8c-b516-4b92-be99-bb99d1955c05 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:36.859757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750ef8c-b516-4b92-be99-bb99d1955c05 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:36.859966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750ef8c-b516-4b92-be99-bb99d1955c05 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:36.861468 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 160/322] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:38:36.866379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466196c-65b7-418d-9ef5-209b9e2ecf90 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:36.866619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466196c-65b7-418d-9ef5-209b9e2ecf90 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:36.866886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466196c-65b7-418d-9ef5-209b9e2ecf90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:36.872540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:36.886609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8466196c-65b7-418d-9ef5-209b9e2ecf90 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:36.887853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8466196c-65b7-418d-9ef5-209b9e2ecf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:36.895612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:36.927353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e1dd5-0eaa-4afc-a841-862069b1d056 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:36.960539 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 161/323] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:36.960539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e797b-66d9-4b0e-8cbb-0de769b6e8b0 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:36.960539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e797b-66d9-4b0e-8cbb-0de769b6e8b0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:36.960539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e797b-66d9-4b0e-8cbb-0de769b6e8b0 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:36.960539 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 162/324] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:36.965348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c3f2b-3c44-40bd-968c-a95a072eda65 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:36.965771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c3f2b-3c44-40bd-968c-a95a072eda65 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:36.966038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c3f2b-3c44-40bd-968c-a95a072eda65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:36.969458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02126101-62c9-4428-8343-50115315d090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:36.970070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02126101-62c9-4428-8343-50115315d090 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:36.970342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02126101-62c9-4428-8343-50115315d090 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:36.970660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02126101-62c9-4428-8343-50115315d090 None admin] PATH_INFO: `/v3/groups` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:36.971221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02126101-62c9-4428-8343-50115315d090 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:36.972880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02126101-62c9-4428-8343-50115315d090 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:36.984251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-007c3f2b-3c44-40bd-968c-a95a072eda65 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:36.985494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-007c3f2b-3c44-40bd-968c-a95a072eda65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:36.992889 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 163/325] 10.222.0.23 () {60 vars in 1134 bytes} [Sat Aug 20 20:38:36 2022] POST /identity/v3/groups => generated 226 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:38:37.065912 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 163/326] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:37.293541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02126101-62c9-4428-8343-50115315d090 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:37.398115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c3f2b-3c44-40bd-968c-a95a072eda65 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:37.405183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02126101-62c9-4428-8343-50115315d090 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:37.413399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:37.454234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10097d0-ecee-42d1-a82d-b6e8c923e64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:37.454832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10097d0-ecee-42d1-a82d-b6e8c923e64c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:37.455100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10097d0-ecee-42d1-a82d-b6e8c923e64c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:37.455363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10097d0-ecee-42d1-a82d-b6e8c923e64c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:37.468451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10097d0-ecee-42d1-a82d-b6e8c923e64c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-f10097d0-ecee-42d1-a82d-b6e8c923e64c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:37.473645 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:37.474530 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:37.475311 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:37.475311 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:37.475311 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:37.475311 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:37.475311 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:37.475311 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:37.475311 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:37.475311 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:37.475311 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:37.475311 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:37.475311 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:37.475311 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:37.475311 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 164/327] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:37.481756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10097d0-ecee-42d1-a82d-b6e8c923e64c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:37.495704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:37.536007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c3f2b-3c44-40bd-968c-a95a072eda65 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:37.574638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2464a7-0934-471e-9368-fb901ce2e436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:37.575286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2464a7-0934-471e-9368-fb901ce2e436 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:37.575558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2464a7-0934-471e-9368-fb901ce2e436 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:37.575829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2464a7-0934-471e-9368-fb901ce2e436 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:37.587233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2464a7-0934-471e-9368-fb901ce2e436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-8a2464a7-0934-471e-9368-fb901ce2e436 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:37.592103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:37.593007 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:37.593793 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:37.593793 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:37.593793 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:37.593793 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:37.593793 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:37.593793 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:37.593793 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:37.593793 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:37.593793 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:37.593793 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:37.593793 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:37.593793 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:37.594394 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 164/328] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:38:37.600225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2464a7-0934-471e-9368-fb901ce2e436 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:37.602028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10097d0-ecee-42d1-a82d-b6e8c923e64c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:37.647365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130abd4-13f0-4abe-9d89-a1d3e5767fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:37.648039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130abd4-13f0-4abe-9d89-a1d3e5767fcb None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:37.648455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130abd4-13f0-4abe-9d89-a1d3e5767fcb None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:37.648834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130abd4-13f0-4abe-9d89-a1d3e5767fcb None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:37.649509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130abd4-13f0-4abe-9d89-a1d3e5767fcb None admin] RBAC: Adding query filter params (name=reader) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:37.650165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130abd4-13f0-4abe-9d89-a1d3e5767fcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:37.651665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130abd4-13f0-4abe-9d89-a1d3e5767fcb None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:37.662079 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 165/329] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:37.667557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130abd4-13f0-4abe-9d89-a1d3e5767fcb None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:37.699349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:37.712015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2464a7-0934-471e-9368-fb901ce2e436 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:37.745385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:37.751303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febab394-ed5a-4937-87c3-a836805247cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:37.751916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febab394-ed5a-4937-87c3-a836805247cc None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:37.753159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febab394-ed5a-4937-87c3-a836805247cc None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:37.753421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febab394-ed5a-4937-87c3-a836805247cc None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:37.754083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febab394-ed5a-4937-87c3-a836805247cc None admin] RBAC: Adding query filter params (name=member) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:37.754376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febab394-ed5a-4937-87c3-a836805247cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:37.755744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febab394-ed5a-4937-87c3-a836805247cc None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:37.764918 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 165/330] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:37.774946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febab394-ed5a-4937-87c3-a836805247cc None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:37.792305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130abd4-13f0-4abe-9d89-a1d3e5767fcb None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:37.846619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6afd65-d15a-4e01-9d40-deca85e36955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:37.847283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6afd65-d15a-4e01-9d40-deca85e36955 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:37.847549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6afd65-d15a-4e01-9d40-deca85e36955 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:37.847808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6afd65-d15a-4e01-9d40-deca85e36955 None admin] PATH_INFO: `/v3/users/23c9c43472884295985af7257e127fcb` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:37.848948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6afd65-d15a-4e01-9d40-deca85e36955 None admin] RBAC: Authorizing `identity:get_user(user_id=23c9c43472884295985af7257e127fcb)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:37.850442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6afd65-d15a-4e01-9d40-deca85e36955 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:37.852314 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 166/331] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:37 2022] GET /identity/v3/users/23c9c43472884295985af7257e127fcb => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:37.857733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6afd65-d15a-4e01-9d40-deca85e36955 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:37.911573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febab394-ed5a-4937-87c3-a836805247cc None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:37.946259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1334e3-7261-45a4-99af-216f0d9e68d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:37.946924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1334e3-7261-45a4-99af-216f0d9e68d1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:37.947178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1334e3-7261-45a4-99af-216f0d9e68d1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:37.947443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1334e3-7261-45a4-99af-216f0d9e68d1 None admin] PATH_INFO: `/v3/users/bca3a824455d45fbbdcd5016a27a2d43` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:37.948618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1334e3-7261-45a4-99af-216f0d9e68d1 None admin] RBAC: Authorizing `identity:get_user(user_id=bca3a824455d45fbbdcd5016a27a2d43)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:37.950785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1334e3-7261-45a4-99af-216f0d9e68d1 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:37.956402 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 166/332] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:37 2022] GET /identity/v3/users/bca3a824455d45fbbdcd5016a27a2d43 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:37.959212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1334e3-7261-45a4-99af-216f0d9e68d1 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:37.978846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:37.986876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6afd65-d15a-4e01-9d40-deca85e36955 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:38.036837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcd882-6089-46a8-9afc-50b4af627df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:38.037411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcd882-6089-46a8-9afc-50b4af627df8 None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:38.037677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcd882-6089-46a8-9afc-50b4af627df8 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:38.037940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcd882-6089-46a8-9afc-50b4af627df8 None admin] PATH_INFO: `/v3/system/users/23c9c43472884295985af7257e127fcb/roles/bdbaea91119244b9aa4a822c613c2bb8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:38.043526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcd882-6089-46a8-9afc-50b4af627df8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=23c9c43472884295985af7257e127fcb, role_id=bdbaea91119244b9aa4a822c613c2bb8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:38.044971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcd882-6089-46a8-9afc-50b4af627df8 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:38.059565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:38.063990 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 167/333] 10.222.0.23 () {58 vars in 1334 bytes} [Sat Aug 20 20:38:37 2022] PUT /identity/v3/system/users/23c9c43472884295985af7257e127fcb/roles/bdbaea91119244b9aa4a822c613c2bb8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:38:38.102942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1334e3-7261-45a4-99af-216f0d9e68d1 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:38.140694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e6533-8a38-44cd-9af2-6219bd95d18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:38.141452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e6533-8a38-44cd-9af2-6219bd95d18e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:38.141723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e6533-8a38-44cd-9af2-6219bd95d18e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:38.141984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e6533-8a38-44cd-9af2-6219bd95d18e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:38.142754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e6533-8a38-44cd-9af2-6219bd95d18e None admin] RBAC: Adding query filter params (role.id=5eaf9701f37e480bb87c89f7391d5d3a, user.id=bca3a824455d45fbbdcd5016a27a2d43, scope.system=all) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:38.143059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e6533-8a38-44cd-9af2-6219bd95d18e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:38.144587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e6533-8a38-44cd-9af2-6219bd95d18e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:38.158800 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 167/334] 10.222.0.23 () {56 vars in 1308 bytes} [Sat Aug 20 20:38:37 2022] GET /identity/v3/role_assignments?user.id=bca3a824455d45fbbdcd5016a27a2d43&scope.system=all&role.id=5eaf9701f37e480bb87c89f7391d5d3a => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:40.053538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f141791-a4b7-430b-8616-822628072160 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:40.054090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f141791-a4b7-430b-8616-822628072160 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:40.054337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f141791-a4b7-430b-8616-822628072160 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:40.055774 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 168/335] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:40.061946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c332a-69dc-4a76-b8c8-4bfff15b859c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:40.062376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c332a-69dc-4a76-b8c8-4bfff15b859c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:40.062551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c332a-69dc-4a76-b8c8-4bfff15b859c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:40.084303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c34c332a-69dc-4a76-b8c8-4bfff15b859c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:40.085792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c34c332a-69dc-4a76-b8c8-4bfff15b859c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:40.171790 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 168/336] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:40.484299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c84b5d-7e82-498a-86f7-0c7f4e84b04c None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:40.484671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c84b5d-7e82-498a-86f7-0c7f4e84b04c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:40.484968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c84b5d-7e82-498a-86f7-0c7f4e84b04c None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:40.486315 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 169/337] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:40.491928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c447a-97e6-4f2b-87b2-c45d0de2c9f7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:40.492211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c447a-97e6-4f2b-87b2-c45d0de2c9f7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:40.492498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c447a-97e6-4f2b-87b2-c45d0de2c9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:40.517381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-638c447a-97e6-4f2b-87b2-c45d0de2c9f7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:40.519003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-638c447a-97e6-4f2b-87b2-c45d0de2c9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:40.600741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:40.623222 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 169/338] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:40.643887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c84b5d-7e82-498a-86f7-0c7f4e84b04c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:40.770214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cd249-8125-4691-943d-c7ed84e3e860 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:40.770395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cd249-8125-4691-943d-c7ed84e3e860 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:40.770653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cd249-8125-4691-943d-c7ed84e3e860 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:40.772110 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 170/339] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:40.776978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c84b5d-7e82-498a-86f7-0c7f4e84b04c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:40.780958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d077d1-f740-40ea-b1f2-217f6333b036 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:40.781239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d077d1-f740-40ea-b1f2-217f6333b036 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:40.781507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d077d1-f740-40ea-b1f2-217f6333b036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:40.819126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af01db-efca-43d2-a101-43885035989a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:40.819797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af01db-efca-43d2-a101-43885035989a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:40.820065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af01db-efca-43d2-a101-43885035989a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:40.820329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af01db-efca-43d2-a101-43885035989a None admin] PATH_INFO: `/v3/roles/member` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:40.824613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-98d077d1-f740-40ea-b1f2-217f6333b036 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:40.826532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-98d077d1-f740-40ea-b1f2-217f6333b036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:40.838361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af01db-efca-43d2-a101-43885035989a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-61af01db-efca-43d2-a101-43885035989a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:40.846408 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:40.847333 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:40.848149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:40.848149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:40.848149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:40.848149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:40.848149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:40.848149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:40.848149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:40.848149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:40.848149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:40.848149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:40.848149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:40.848149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:40.848149 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 170/340] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:40.854776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af01db-efca-43d2-a101-43885035989a None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:40.894205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:40.973783 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 171/341] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:41.014428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af01db-efca-43d2-a101-43885035989a None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:41.054708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb39d44-d031-4a65-b248-2f2051d13086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:41.055308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb39d44-d031-4a65-b248-2f2051d13086 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:41.055521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb39d44-d031-4a65-b248-2f2051d13086 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:41.055777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb39d44-d031-4a65-b248-2f2051d13086 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:41.056520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb39d44-d031-4a65-b248-2f2051d13086 None admin] RBAC: Adding query filter params (name=member) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:41.056813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb39d44-d031-4a65-b248-2f2051d13086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:41.058225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb39d44-d031-4a65-b248-2f2051d13086 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:41.067543 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 171/342] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:41.073358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d077d1-f740-40ea-b1f2-217f6333b036 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:41.187870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d077d1-f740-40ea-b1f2-217f6333b036 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:41.201209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:41.231968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8f9ff-c3f3-491d-be6b-07260695161b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:41.232622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8f9ff-c3f3-491d-be6b-07260695161b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:41.232894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8f9ff-c3f3-491d-be6b-07260695161b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:41.233159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8f9ff-c3f3-491d-be6b-07260695161b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:41.274289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58561c96-fa4d-4d4c-9e64-d7b777dacac0 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:41.274618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58561c96-fa4d-4d4c-9e64-d7b777dacac0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:41.274817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58561c96-fa4d-4d4c-9e64-d7b777dacac0 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:41.276249 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 172/343] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:38:41.281656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42910d1-2d7a-4a10-bd16-25c2173f1bc5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:41.281915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42910d1-2d7a-4a10-bd16-25c2173f1bc5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:41.282176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42910d1-2d7a-4a10-bd16-25c2173f1bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:41.318226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e42910d1-2d7a-4a10-bd16-25c2173f1bc5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:41.319272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e42910d1-2d7a-4a10-bd16-25c2173f1bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:41.342475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa8f9ff-c3f3-491d-be6b-07260695161b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:41.344653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa8f9ff-c3f3-491d-be6b-07260695161b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:41.445010 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 173/344] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:41.445999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42910d1-2d7a-4a10-bd16-25c2173f1bc5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:41.461464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:41.557202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42910d1-2d7a-4a10-bd16-25c2173f1bc5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:41.598225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b588d-652e-4d31-a849-8620af193e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:41.598851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b588d-652e-4d31-a849-8620af193e65 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:41.599127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b588d-652e-4d31-a849-8620af193e65 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:41.599427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b588d-652e-4d31-a849-8620af193e65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:41.612557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b588d-652e-4d31-a849-8620af193e65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:41.615235 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 172/345] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:41.616113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8f9ff-c3f3-491d-be6b-07260695161b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-2c6b588d-652e-4d31-a849-8620af193e65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:41.617396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:41.618272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:41.619043 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:41.619043 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:41.619043 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:41.619043 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:41.619043 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:41.619043 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:41.619043 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:41.619043 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:41.619043 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:41.619043 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:41.619043 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:41.619043 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:41.622739 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 174/346] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:41.629748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b588d-652e-4d31-a849-8620af193e65 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:41.640753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:41.730820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:41.794348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b588d-652e-4d31-a849-8620af193e65 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:41.848598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8f9ff-c3f3-491d-be6b-07260695161b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:41.856354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88dee12-c5c4-4bce-bf4d-c2bc515ebe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:41.857353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dee12-c5c4-4bce-bf4d-c2bc515ebe7c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:41.857823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dee12-c5c4-4bce-bf4d-c2bc515ebe7c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:41.858281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dee12-c5c4-4bce-bf4d-c2bc515ebe7c None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:41.859139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88dee12-c5c4-4bce-bf4d-c2bc515ebe7c None admin] RBAC: Adding query filter params (name=member) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:41.859622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88dee12-c5c4-4bce-bf4d-c2bc515ebe7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:41.861870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88dee12-c5c4-4bce-bf4d-c2bc515ebe7c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:41.878127 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 175/347] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:41.879294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88dee12-c5c4-4bce-bf4d-c2bc515ebe7c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:41.891841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71db8cc-d253-4a68-aada-088c12ddfbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:41.896440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71db8cc-d253-4a68-aada-088c12ddfbd2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:41.896440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71db8cc-d253-4a68-aada-088c12ddfbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:41.896440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71db8cc-d253-4a68-aada-088c12ddfbd2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:41.910711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71db8cc-d253-4a68-aada-088c12ddfbd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-f71db8cc-d253-4a68-aada-088c12ddfbd2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:41.919391 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:41.920605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:41.921969 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:41.921969 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:41.921969 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:41.921969 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:41.921969 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:41.921969 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:41.921969 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:41.921969 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:41.921969 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:41.921969 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:41.921969 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:41.921969 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:41.921969 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 173/348] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:38:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:41.923168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71db8cc-d253-4a68-aada-088c12ddfbd2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:42.009765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:42.031399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88dee12-c5c4-4bce-bf4d-c2bc515ebe7c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:42.061072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71db8cc-d253-4a68-aada-088c12ddfbd2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:42.078674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a2b27-aad4-4ac1-9924-171b533b67a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:42.078674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a2b27-aad4-4ac1-9924-171b533b67a0 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:42.079281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a2b27-aad4-4ac1-9924-171b533b67a0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:42.079281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a2b27-aad4-4ac1-9924-171b533b67a0 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:42.081975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a2b27-aad4-4ac1-9924-171b533b67a0 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:42.083740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a2b27-aad4-4ac1-9924-171b533b67a0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:42.085154 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 176/349] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:41 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:42.085696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a2b27-aad4-4ac1-9924-171b533b67a0 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:42.091168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:42.110376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb3f9d-91ab-4eeb-9f56-73c64803fbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:42.111649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb3f9d-91ab-4eeb-9f56-73c64803fbbe None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:42.112152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb3f9d-91ab-4eeb-9f56-73c64803fbbe None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:42.113057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb3f9d-91ab-4eeb-9f56-73c64803fbbe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:42.128280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb3f9d-91ab-4eeb-9f56-73c64803fbbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-b9bb3f9d-91ab-4eeb-9f56-73c64803fbbe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:42.133083 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:42.134006 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:42.134784 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:42.134784 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:42.134784 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:42.134784 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:42.134784 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:42.134784 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:42.134784 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:42.134784 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:42.134784 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:42.134784 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:42.134784 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:42.134784 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:42.134784 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 174/350] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:38:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:42.135440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb3f9d-91ab-4eeb-9f56-73c64803fbbe None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:42.207385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a2b27-aad4-4ac1-9924-171b533b67a0 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:42.248341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec7650-077c-4727-b15b-b1dbe3ba0181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:42.249070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec7650-077c-4727-b15b-b1dbe3ba0181 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:42.249343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec7650-077c-4727-b15b-b1dbe3ba0181 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:42.249610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec7650-077c-4727-b15b-b1dbe3ba0181 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:42.249818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb3f9d-91ab-4eeb-9f56-73c64803fbbe None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:42.286035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:42.294937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402a743-0975-4d82-8696-221a96c617b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:42.295789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402a743-0975-4d82-8696-221a96c617b0 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:42.296206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402a743-0975-4d82-8696-221a96c617b0 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:42.296634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402a743-0975-4d82-8696-221a96c617b0 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:42.297463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402a743-0975-4d82-8696-221a96c617b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:42.297891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402a743-0975-4d82-8696-221a96c617b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:42.299912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402a743-0975-4d82-8696-221a96c617b0 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:42.309147 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 175/351] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:38:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:42.311055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018d575-cc60-4be6-8d70-aabf7dbc7184 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:42.311469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018d575-cc60-4be6-8d70-aabf7dbc7184 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:42.311878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018d575-cc60-4be6-8d70-aabf7dbc7184 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:42.313243 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 176/352] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:42.313786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018d575-cc60-4be6-8d70-aabf7dbc7184 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:42.331266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec7650-077c-4727-b15b-b1dbe3ba0181 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:42.332770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec7650-077c-4727-b15b-b1dbe3ba0181 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:42.334633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:42.443873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018d575-cc60-4be6-8d70-aabf7dbc7184 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:42.452790 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 177/353] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:42.454065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fa6de-12ce-4f7a-b62a-62676211307d None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:42.454339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fa6de-12ce-4f7a-b62a-62676211307d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:42.454603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fa6de-12ce-4f7a-b62a-62676211307d None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:42.456022 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 178/354] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:42.457886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fa6de-12ce-4f7a-b62a-62676211307d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:42.486366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476b5a2-1549-4f2a-ba97-2a9cbe75dab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:42.487021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476b5a2-1549-4f2a-ba97-2a9cbe75dab1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:42.487297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476b5a2-1549-4f2a-ba97-2a9cbe75dab1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:42.487558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476b5a2-1549-4f2a-ba97-2a9cbe75dab1 None admin] PATH_INFO: `/v3/groups/76923e35980d4ea7b0a2225c166281e3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:42.493905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0476b5a2-1549-4f2a-ba97-2a9cbe75dab1 None admin] RBAC: Authorizing `identity:get_group(group_id=76923e35980d4ea7b0a2225c166281e3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:42.495548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0476b5a2-1549-4f2a-ba97-2a9cbe75dab1 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:42.497469 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 177/355] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:42 2022] GET /identity/v3/groups/76923e35980d4ea7b0a2225c166281e3 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:42.499081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5bf82-b252-4d3f-8d26-7198007e8402 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:42.499081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5bf82-b252-4d3f-8d26-7198007e8402 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:42.499245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5bf82-b252-4d3f-8d26-7198007e8402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:42.517966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-28a5bf82-b252-4d3f-8d26-7198007e8402 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:42.519158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-28a5bf82-b252-4d3f-8d26-7198007e8402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:42.577256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fa6de-12ce-4f7a-b62a-62676211307d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:42.578847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:42.583292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:42.601638 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 178/356] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:42.602951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5bf82-b252-4d3f-8d26-7198007e8402 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:42.621398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025720e-3bd7-4b1e-8ae2-5b2d964e378f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:42.622006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025720e-3bd7-4b1e-8ae2-5b2d964e378f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:42.622274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025720e-3bd7-4b1e-8ae2-5b2d964e378f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:42.622534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025720e-3bd7-4b1e-8ae2-5b2d964e378f None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:42.623191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025720e-3bd7-4b1e-8ae2-5b2d964e378f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:42.623485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025720e-3bd7-4b1e-8ae2-5b2d964e378f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:42.624915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025720e-3bd7-4b1e-8ae2-5b2d964e378f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:42.634981 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 179/357] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:38:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:42.635868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025720e-3bd7-4b1e-8ae2-5b2d964e378f None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:42.725962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5bf82-b252-4d3f-8d26-7198007e8402 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:42.764759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025720e-3bd7-4b1e-8ae2-5b2d964e378f None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:42.771271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd343b79-5215-4b9e-80aa-d35b5684d9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:42.771880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd343b79-5215-4b9e-80aa-d35b5684d9b6 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:42.772147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd343b79-5215-4b9e-80aa-d35b5684d9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:42.772444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd343b79-5215-4b9e-80aa-d35b5684d9b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:42.805485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1addd602-4de9-4660-8a7e-fa1a8c2cbfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:42.806223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addd602-4de9-4660-8a7e-fa1a8c2cbfb8 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:42.806557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addd602-4de9-4660-8a7e-fa1a8c2cbfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:42.806885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addd602-4de9-4660-8a7e-fa1a8c2cbfb8 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:42.810456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1addd602-4de9-4660-8a7e-fa1a8c2cbfb8 None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:42.810456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1addd602-4de9-4660-8a7e-fa1a8c2cbfb8 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:42.812001 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 180/358] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:42 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:42.814157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6ab30-fd6f-4c5f-9eb8-efd547f0a354 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:42.814157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6ab30-fd6f-4c5f-9eb8-efd547f0a354 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:42.814157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6ab30-fd6f-4c5f-9eb8-efd547f0a354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:42.833721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2cc6ab30-fd6f-4c5f-9eb8-efd547f0a354 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:42.834903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2cc6ab30-fd6f-4c5f-9eb8-efd547f0a354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:42.842982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:42.857889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd343b79-5215-4b9e-80aa-d35b5684d9b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:42.859299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd343b79-5215-4b9e-80aa-d35b5684d9b6 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:42.870105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:42.912789 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 181/359] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:42.913751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6ab30-fd6f-4c5f-9eb8-efd547f0a354 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:42.978766 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 179/360] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:42.978766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd343b79-5215-4b9e-80aa-d35b5684d9b6 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:43.044424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6ab30-fd6f-4c5f-9eb8-efd547f0a354 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:43.077408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:43.091475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406f478-9589-4f58-ae80-c149cc6e21cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:43.092119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f478-9589-4f58-ae80-c149cc6e21cb None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:43.092418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f478-9589-4f58-ae80-c149cc6e21cb None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:43.092700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f478-9589-4f58-ae80-c149cc6e21cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:43.093421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406f478-9589-4f58-ae80-c149cc6e21cb None admin] RBAC: Adding query filter params (group.id=76923e35980d4ea7b0a2225c166281e3, role.id=5eaf9701f37e480bb87c89f7391d5d3a, scope.project.id=4b6c3729d57640af9e56f5e19081b8b0) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:43.093721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406f478-9589-4f58-ae80-c149cc6e21cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:43.094803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd343b79-5215-4b9e-80aa-d35b5684d9b6 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:43.095148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406f478-9589-4f58-ae80-c149cc6e21cb None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:43.101461 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 182/361] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Aug 20 20:38:42 2022] GET /identity/v3/role_assignments?group.id=76923e35980d4ea7b0a2225c166281e3&scope.project.id=4b6c3729d57640af9e56f5e19081b8b0&role.id=5eaf9701f37e480bb87c89f7391d5d3a => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:43.102164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406f478-9589-4f58-ae80-c149cc6e21cb None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:43.132312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:43.146861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b892a-be2b-46d7-84dd-6dd32033cc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:43.147544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b892a-be2b-46d7-84dd-6dd32033cc3f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:43.147735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b892a-be2b-46d7-84dd-6dd32033cc3f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:43.147998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b892a-be2b-46d7-84dd-6dd32033cc3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:43.237971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406f478-9589-4f58-ae80-c149cc6e21cb None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:43.261254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b892a-be2b-46d7-84dd-6dd32033cc3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:43.262781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b892a-be2b-46d7-84dd-6dd32033cc3f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:43.289446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b0e21-fb21-493f-87f5-8a477a126e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:43.290935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b0e21-fb21-493f-87f5-8a477a126e38 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:43.291421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b0e21-fb21-493f-87f5-8a477a126e38 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:43.291873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b0e21-fb21-493f-87f5-8a477a126e38 None admin] PATH_INFO: `/v3/groups/76923e35980d4ea7b0a2225c166281e3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:43.293793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b0e21-fb21-493f-87f5-8a477a126e38 None admin] RBAC: Authorizing `identity:get_group(group_id=76923e35980d4ea7b0a2225c166281e3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:43.296124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b0e21-fb21-493f-87f5-8a477a126e38 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:43.298550 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 183/362] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:43 2022] GET /identity/v3/groups/76923e35980d4ea7b0a2225c166281e3 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:43.299879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b0e21-fb21-493f-87f5-8a477a126e38 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:43.379409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:43.391187 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 180/363] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:43.392137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b892a-be2b-46d7-84dd-6dd32033cc3f None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:43.429827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:43.442388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b0e21-fb21-493f-87f5-8a477a126e38 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:43.490119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc6804-def2-4c31-9d38-9e394e500674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:43.490684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc6804-def2-4c31-9d38-9e394e500674 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:43.490955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc6804-def2-4c31-9d38-9e394e500674 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:43.491216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc6804-def2-4c31-9d38-9e394e500674 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:43.508984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc6804-def2-4c31-9d38-9e394e500674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-a6cc6804-def2-4c31-9d38-9e394e500674 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:43.517339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:43.518256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:43.519069 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:43.519069 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:43.519069 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:43.519069 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:43.519069 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:43.519069 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:43.519069 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:43.519069 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:43.519069 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:43.519069 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:43.519069 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 20 20:38:43.519069 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:43.521331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b892a-be2b-46d7-84dd-6dd32033cc3f None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:43.523662 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 184/364] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:43.524592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc6804-def2-4c31-9d38-9e394e500674 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:43.569222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160c7f5-2322-424f-8213-3ceda707e093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:43.569876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160c7f5-2322-424f-8213-3ceda707e093 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:43.570141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160c7f5-2322-424f-8213-3ceda707e093 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:43.570401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160c7f5-2322-424f-8213-3ceda707e093 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:43.571966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160c7f5-2322-424f-8213-3ceda707e093 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:43.574937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160c7f5-2322-424f-8213-3ceda707e093 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:43.576742 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 181/365] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:43 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:43.577509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160c7f5-2322-424f-8213-3ceda707e093 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:43.649385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc6804-def2-4c31-9d38-9e394e500674 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:43.659615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:43.697223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62973d-a51b-4f0e-a6eb-6f5119c6d05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:43.697866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62973d-a51b-4f0e-a6eb-6f5119c6d05a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:43.698138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62973d-a51b-4f0e-a6eb-6f5119c6d05a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:43.698406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62973d-a51b-4f0e-a6eb-6f5119c6d05a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:43.713525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62973d-a51b-4f0e-a6eb-6f5119c6d05a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-3e62973d-a51b-4f0e-a6eb-6f5119c6d05a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:43.719099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:43.720005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:43.720838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:43.720838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:43.720838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:43.720838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:43.720838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:43.720838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:43.720838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:43.720838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:43.720838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:43.720838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:43.720838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:43.720838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:43.720838 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 185/366] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:38:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:43.721497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62973d-a51b-4f0e-a6eb-6f5119c6d05a None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:43.722458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160c7f5-2322-424f-8213-3ceda707e093 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:43.738137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:43.778442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb12e1-654a-4269-8961-8612e34478f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:43.778965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb12e1-654a-4269-8961-8612e34478f2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:43.779235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb12e1-654a-4269-8961-8612e34478f2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:43.779496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb12e1-654a-4269-8961-8612e34478f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:43.780214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb12e1-654a-4269-8961-8612e34478f2 None admin] RBAC: Adding query filter params (group.id=76923e35980d4ea7b0a2225c166281e3, role.id=5eaf9701f37e480bb87c89f7391d5d3a, scope.project.id=66eda7a4830f4fdeb286e09ae72f278b) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:43.780548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb12e1-654a-4269-8961-8612e34478f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:43.782032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb12e1-654a-4269-8961-8612e34478f2 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:43.788029 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 182/367] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Aug 20 20:38:43 2022] GET /identity/v3/role_assignments?group.id=76923e35980d4ea7b0a2225c166281e3&scope.project.id=66eda7a4830f4fdeb286e09ae72f278b&role.id=5eaf9701f37e480bb87c89f7391d5d3a => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:43.788882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb12e1-654a-4269-8961-8612e34478f2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:43.871225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62973d-a51b-4f0e-a6eb-6f5119c6d05a None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:43.925900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e363a-aa4a-4b0f-a741-7a8c137c1477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:43.926509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e363a-aa4a-4b0f-a741-7a8c137c1477 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:43.926960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e363a-aa4a-4b0f-a741-7a8c137c1477 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:43.927302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e363a-aa4a-4b0f-a741-7a8c137c1477 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:43.928564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e363a-aa4a-4b0f-a741-7a8c137c1477 None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:43.929543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb12e1-654a-4269-8961-8612e34478f2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:43.930529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e363a-aa4a-4b0f-a741-7a8c137c1477 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:43.932697 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 186/368] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:43 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:43.933484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e363a-aa4a-4b0f-a741-7a8c137c1477 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:43.953108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:43.978290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed82bd-3404-4fe8-a263-6703af142f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:43.979179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed82bd-3404-4fe8-a263-6703af142f4c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:43.979881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed82bd-3404-4fe8-a263-6703af142f4c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:43.980434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed82bd-3404-4fe8-a263-6703af142f4c None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:43.982504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed82bd-3404-4fe8-a263-6703af142f4c None admin] RBAC: Adding query filter params (name=reader) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:43.983006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed82bd-3404-4fe8-a263-6703af142f4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:43.984576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed82bd-3404-4fe8-a263-6703af142f4c None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:43.997382 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 183/369] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:44.002859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed82bd-3404-4fe8-a263-6703af142f4c None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:44.061534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:44.082365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e363a-aa4a-4b0f-a741-7a8c137c1477 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:44.107188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed82bd-3404-4fe8-a263-6703af142f4c None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:44.127432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f6fc3-e4d2-4033-856b-01cf9d779a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:44.128791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f6fc3-e4d2-4033-856b-01cf9d779a75 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:44.129727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f6fc3-e4d2-4033-856b-01cf9d779a75 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:44.130209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f6fc3-e4d2-4033-856b-01cf9d779a75 None admin] PATH_INFO: `/v3/groups/76923e35980d4ea7b0a2225c166281e3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:44.132054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f6fc3-e4d2-4033-856b-01cf9d779a75 None admin] RBAC: Authorizing `identity:get_group(group_id=76923e35980d4ea7b0a2225c166281e3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:44.134589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f6fc3-e4d2-4033-856b-01cf9d779a75 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:44.137430 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 187/370] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:43 2022] GET /identity/v3/groups/76923e35980d4ea7b0a2225c166281e3 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:44.139197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f6fc3-e4d2-4033-856b-01cf9d779a75 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:44.143534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f1047-8ac7-4e48-b23f-2df7a9e9433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:44.144200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f1047-8ac7-4e48-b23f-2df7a9e9433a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:44.144511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f1047-8ac7-4e48-b23f-2df7a9e9433a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:44.144779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f1047-8ac7-4e48-b23f-2df7a9e9433a None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:44.145431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f1047-8ac7-4e48-b23f-2df7a9e9433a None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:44.145730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f1047-8ac7-4e48-b23f-2df7a9e9433a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:44.147072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f1047-8ac7-4e48-b23f-2df7a9e9433a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:44.155907 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 184/371] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:44.156767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f1047-8ac7-4e48-b23f-2df7a9e9433a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:44.201875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:44.276322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f6fc3-e4d2-4033-856b-01cf9d779a75 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:44.280407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f1047-8ac7-4e48-b23f-2df7a9e9433a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:44.320209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32c57b-9bc8-4a27-b7f7-1ee1cd17f338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:44.321126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32c57b-9bc8-4a27-b7f7-1ee1cd17f338 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:44.321537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32c57b-9bc8-4a27-b7f7-1ee1cd17f338 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:44.321876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32c57b-9bc8-4a27-b7f7-1ee1cd17f338 None admin] PATH_INFO: `/v3/users/23c9c43472884295985af7257e127fcb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:44.323062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cf7d9-9d81-4f77-931b-403eb4629413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:44.323188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32c57b-9bc8-4a27-b7f7-1ee1cd17f338 None admin] RBAC: Authorizing `identity:get_user(user_id=23c9c43472884295985af7257e127fcb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:44.324080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cf7d9-9d81-4f77-931b-403eb4629413 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:44.324582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cf7d9-9d81-4f77-931b-403eb4629413 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:44.325043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cf7d9-9d81-4f77-931b-403eb4629413 None admin] PATH_INFO: `/v3/groups/76923e35980d4ea7b0a2225c166281e3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:44.325152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32c57b-9bc8-4a27-b7f7-1ee1cd17f338 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:44.326457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cf7d9-9d81-4f77-931b-403eb4629413 None admin] RBAC: Authorizing `identity:get_group(group_id=76923e35980d4ea7b0a2225c166281e3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:44.331744 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 185/372] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:38:44 2022] GET /identity/v3/users/23c9c43472884295985af7257e127fcb => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:44.331744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32c57b-9bc8-4a27-b7f7-1ee1cd17f338 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:44.334363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cf7d9-9d81-4f77-931b-403eb4629413 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:44.336709 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 188/373] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:44 2022] GET /identity/v3/groups/76923e35980d4ea7b0a2225c166281e3 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:44.337718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cf7d9-9d81-4f77-931b-403eb4629413 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:44.362950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:44.446460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32c57b-9bc8-4a27-b7f7-1ee1cd17f338 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:44.458018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:44.473584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cf7d9-9d81-4f77-931b-403eb4629413 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:44.492949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b64fc-1db6-4089-b59e-22af540f902e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:44.494192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b64fc-1db6-4089-b59e-22af540f902e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:44.494664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b64fc-1db6-4089-b59e-22af540f902e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:44.495097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b64fc-1db6-4089-b59e-22af540f902e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:44.496209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b64fc-1db6-4089-b59e-22af540f902e None admin] RBAC: Adding query filter params (group.id=76923e35980d4ea7b0a2225c166281e3, role.id=f8a2ea13798443c48750e9baa83c7461, scope.project.id=4b6c3729d57640af9e56f5e19081b8b0) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:44.497312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b64fc-1db6-4089-b59e-22af540f902e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:44.498608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b64fc-1db6-4089-b59e-22af540f902e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:44.512424 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 186/374] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Aug 20 20:38:44 2022] GET /identity/v3/role_assignments?group.id=76923e35980d4ea7b0a2225c166281e3&scope.project.id=4b6c3729d57640af9e56f5e19081b8b0&role.id=f8a2ea13798443c48750e9baa83c7461 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:44.512424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b64fc-1db6-4089-b59e-22af540f902e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:44.527480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da32eca-38f3-49b2-b203-da6d2bdf005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:44.528420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da32eca-38f3-49b2-b203-da6d2bdf005c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:44.528420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da32eca-38f3-49b2-b203-da6d2bdf005c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:44.528735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da32eca-38f3-49b2-b203-da6d2bdf005c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:44.666924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da32eca-38f3-49b2-b203-da6d2bdf005c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:44.668913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da32eca-38f3-49b2-b203-da6d2bdf005c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:44.703374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:44.706089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b64fc-1db6-4089-b59e-22af540f902e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:44.771079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87b881-f115-4cdf-a996-f39d82b1a3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:44.771958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87b881-f115-4cdf-a996-f39d82b1a3b0 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:44.772462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87b881-f115-4cdf-a996-f39d82b1a3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:44.772932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87b881-f115-4cdf-a996-f39d82b1a3b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:44.774693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87b881-f115-4cdf-a996-f39d82b1a3b0 None admin] RBAC: Adding query filter params (role.id=bdbaea91119244b9aa4a822c613c2bb8, user.id=23c9c43472884295985af7257e127fcb, scope.system=all) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:44.775247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87b881-f115-4cdf-a996-f39d82b1a3b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:44.776995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87b881-f115-4cdf-a996-f39d82b1a3b0 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:44.798891 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 187/375] 10.222.0.23 () {56 vars in 1308 bytes} [Sat Aug 20 20:38:44 2022] GET /identity/v3/role_assignments?user.id=23c9c43472884295985af7257e127fcb&scope.system=all&role.id=bdbaea91119244b9aa4a822c613c2bb8 => generated 510 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:44.799751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87b881-f115-4cdf-a996-f39d82b1a3b0 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:44.846595 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 189/376] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:44.852973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da32eca-38f3-49b2-b203-da6d2bdf005c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:44.907174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:45.103483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87b881-f115-4cdf-a996-f39d82b1a3b0 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:45.122965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da32eca-38f3-49b2-b203-da6d2bdf005c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:45.156101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40684d-c544-4078-9825-286390742192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:45.158298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40684d-c544-4078-9825-286390742192 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:45.159567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40684d-c544-4078-9825-286390742192 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:45.160198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40684d-c544-4078-9825-286390742192 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:45.162371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40684d-c544-4078-9825-286390742192 None admin] RBAC: Adding query filter params (group.id=76923e35980d4ea7b0a2225c166281e3, role.id=f8a2ea13798443c48750e9baa83c7461, scope.project.id=66eda7a4830f4fdeb286e09ae72f278b) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:45.163614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40684d-c544-4078-9825-286390742192 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:45.163836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:45.165406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40684d-c544-4078-9825-286390742192 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:45.178684 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 188/377] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Aug 20 20:38:44 2022] GET /identity/v3/role_assignments?group.id=76923e35980d4ea7b0a2225c166281e3&scope.project.id=66eda7a4830f4fdeb286e09ae72f278b&role.id=f8a2ea13798443c48750e9baa83c7461 => generated 260 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:45.196743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa0491-d552-4420-b827-17991aa809d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:45.197433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa0491-d552-4420-b827-17991aa809d7 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:45.197702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa0491-d552-4420-b827-17991aa809d7 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:45.197967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa0491-d552-4420-b827-17991aa809d7 None admin] PATH_INFO: `/v3/projects/a5bed9a4b90f48f592cdc8d94741f821` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:45.199144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa0491-d552-4420-b827-17991aa809d7 None admin] RBAC: Authorizing `identity:get_project(project_id=a5bed9a4b90f48f592cdc8d94741f821)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:45.200721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa0491-d552-4420-b827-17991aa809d7 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:45.202802 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 190/378] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:44 2022] GET /identity/v3/projects/a5bed9a4b90f48f592cdc8d94741f821 => generated 344 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:45.208748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40684d-c544-4078-9825-286390742192 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:45.373195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:45.376672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40684d-c544-4078-9825-286390742192 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:45.425706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df735d-ce3b-47b1-8094-01ccf6e2a203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:45.426416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df735d-ce3b-47b1-8094-01ccf6e2a203 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:45.426694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df735d-ce3b-47b1-8094-01ccf6e2a203 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:45.426982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df735d-ce3b-47b1-8094-01ccf6e2a203 None admin] PATH_INFO: `/v3/groups/5779a29e6baf4173a51896d1215c1506` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:45.437208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df735d-ce3b-47b1-8094-01ccf6e2a203 None admin] RBAC: Authorizing `identity:get_group(group_id=5779a29e6baf4173a51896d1215c1506)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:45.439463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df735d-ce3b-47b1-8094-01ccf6e2a203 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:45.441674 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 189/379] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:45 2022] GET /identity/v3/groups/5779a29e6baf4173a51896d1215c1506 => generated 229 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:45.448099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa0491-d552-4420-b827-17991aa809d7 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:45.577472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa0491-d552-4420-b827-17991aa809d7 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:45.613317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf91ccc-c87f-4ff8-99b1-e50df19cfa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:45.613922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf91ccc-c87f-4ff8-99b1-e50df19cfa99 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:45.614188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf91ccc-c87f-4ff8-99b1-e50df19cfa99 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:45.614447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf91ccc-c87f-4ff8-99b1-e50df19cfa99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:45.615351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf91ccc-c87f-4ff8-99b1-e50df19cfa99 None admin] RBAC: Adding query filter params (group.id=5779a29e6baf4173a51896d1215c1506, role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=a5bed9a4b90f48f592cdc8d94741f821) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:45.615657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf91ccc-c87f-4ff8-99b1-e50df19cfa99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:45.617124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf91ccc-c87f-4ff8-99b1-e50df19cfa99 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:45.623470 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 191/380] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Aug 20 20:38:45 2022] GET /identity/v3/role_assignments?group.id=5779a29e6baf4173a51896d1215c1506&scope.project.id=a5bed9a4b90f48f592cdc8d94741f821&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:45.649871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a93c92-02f5-4aeb-a030-7753839c450d None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:45.650206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a93c92-02f5-4aeb-a030-7753839c450d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:45.650405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a93c92-02f5-4aeb-a030-7753839c450d None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:45.651742 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 190/381] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:45.658787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ac022-7677-41bd-9ca8-3313fcdebab0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:45.659115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ac022-7677-41bd-9ca8-3313fcdebab0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:45.659332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ac022-7677-41bd-9ca8-3313fcdebab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:45.682981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-040ac022-7677-41bd-9ca8-3313fcdebab0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:45.685097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-040ac022-7677-41bd-9ca8-3313fcdebab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:45.759356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:45.791075 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 192/382] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:46.167386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a93c92-02f5-4aeb-a030-7753839c450d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:46.299675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a93c92-02f5-4aeb-a030-7753839c450d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:46.339102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f847d7-c7ea-4319-880a-d4f8445673d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:46.339769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f847d7-c7ea-4319-880a-d4f8445673d3 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:46.340100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f847d7-c7ea-4319-880a-d4f8445673d3 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:46.340366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f847d7-c7ea-4319-880a-d4f8445673d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:46.357461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f847d7-c7ea-4319-880a-d4f8445673d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-52f847d7-c7ea-4319-880a-d4f8445673d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:46.362551 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:46.363515 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:46.364756 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:46.364756 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:46.364756 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:46.364756 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:46.364756 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:46.364756 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:46.364756 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:46.364756 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:46.364756 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:46.364756 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:46.364756 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:46.364756 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:46.364756 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 191/383] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:46.373784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ac022-7677-41bd-9ca8-3313fcdebab0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:46.503083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ac022-7677-41bd-9ca8-3313fcdebab0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:46.541105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efb1b3-f230-4c3f-9650-d63e8ecdd7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:46.541766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efb1b3-f230-4c3f-9650-d63e8ecdd7a8 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:46.542060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efb1b3-f230-4c3f-9650-d63e8ecdd7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:46.542346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efb1b3-f230-4c3f-9650-d63e8ecdd7a8 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:46.543209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efb1b3-f230-4c3f-9650-d63e8ecdd7a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:46.543209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efb1b3-f230-4c3f-9650-d63e8ecdd7a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:46.545094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efb1b3-f230-4c3f-9650-d63e8ecdd7a8 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:46.554908 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 193/384] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:46.561553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f847d7-c7ea-4319-880a-d4f8445673d3 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:46.569551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3b771-c447-491e-bdee-b10ac9d45f8c None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:46.569845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3b771-c447-491e-bdee-b10ac9d45f8c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:46.570151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3b771-c447-491e-bdee-b10ac9d45f8c None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:46.571866 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 194/385] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:38:46.577247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968e182-7426-4b7c-81fa-69414d5609bb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:46.577545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968e182-7426-4b7c-81fa-69414d5609bb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:46.577835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968e182-7426-4b7c-81fa-69414d5609bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:46.597823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4968e182-7426-4b7c-81fa-69414d5609bb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:46.599203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4968e182-7426-4b7c-81fa-69414d5609bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:46.601641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:46.660748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:46.674947 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 195/386] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:46.738196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f847d7-c7ea-4319-880a-d4f8445673d3 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:46.774200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5d4e1-53b9-488b-97c4-d4fe9674ed76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:46.774750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5d4e1-53b9-488b-97c4-d4fe9674ed76 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:46.775015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5d4e1-53b9-488b-97c4-d4fe9674ed76 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:46.775275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5d4e1-53b9-488b-97c4-d4fe9674ed76 None admin] PATH_INFO: `/v3/groups/76923e35980d4ea7b0a2225c166281e3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:46.776418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5d4e1-53b9-488b-97c4-d4fe9674ed76 None admin] RBAC: Authorizing `identity:get_group(group_id=76923e35980d4ea7b0a2225c166281e3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:46.778123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5d4e1-53b9-488b-97c4-d4fe9674ed76 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:46.779993 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 192/387] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:46 2022] GET /identity/v3/groups/76923e35980d4ea7b0a2225c166281e3 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:46.785412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968e182-7426-4b7c-81fa-69414d5609bb None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:46.892036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968e182-7426-4b7c-81fa-69414d5609bb None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:46.927372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d7087-0e68-4f7e-8312-ecf898eb4e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:46.928006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d7087-0e68-4f7e-8312-ecf898eb4e19 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:46.928275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d7087-0e68-4f7e-8312-ecf898eb4e19 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:46.928608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d7087-0e68-4f7e-8312-ecf898eb4e19 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:46.929708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d7087-0e68-4f7e-8312-ecf898eb4e19 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:46.931149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d7087-0e68-4f7e-8312-ecf898eb4e19 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:46.933104 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 196/388] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:46 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:46.941446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5d4e1-53b9-488b-97c4-d4fe9674ed76 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:47.057102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:47.073108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5d4e1-53b9-488b-97c4-d4fe9674ed76 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:47.114736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6dedb-1640-44f8-81a2-97477bbbf35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:47.115393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6dedb-1640-44f8-81a2-97477bbbf35d None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:47.115663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6dedb-1640-44f8-81a2-97477bbbf35d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:47.115930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6dedb-1640-44f8-81a2-97477bbbf35d None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0/groups/76923e35980d4ea7b0a2225c166281e3/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:47.117825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6dedb-1640-44f8-81a2-97477bbbf35d None admin] RBAC: Authorizing `identity:create_grant(project_id=4b6c3729d57640af9e56f5e19081b8b0, group_id=76923e35980d4ea7b0a2225c166281e3, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:47.119263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6dedb-1640-44f8-81a2-97477bbbf35d None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:47.122085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d7087-0e68-4f7e-8312-ecf898eb4e19 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:47.156579 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 193/389] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Aug 20 20:38:46 2022] PUT /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0/groups/76923e35980d4ea7b0a2225c166281e3/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:38:47.188265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:47.290629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d7087-0e68-4f7e-8312-ecf898eb4e19 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:47.333767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ea9f4-97f1-4fa6-bb6a-1aa04e5da459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:47.334728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ea9f4-97f1-4fa6-bb6a-1aa04e5da459 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:47.335193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ea9f4-97f1-4fa6-bb6a-1aa04e5da459 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:47.335645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ea9f4-97f1-4fa6-bb6a-1aa04e5da459 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:47.353405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ea9f4-97f1-4fa6-bb6a-1aa04e5da459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-c14ea9f4-97f1-4fa6-bb6a-1aa04e5da459 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:47.359164 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:47.360131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:47.360962 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:47.360962 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:47.360962 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:47.360962 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:47.360962 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:47.360962 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:47.360962 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:47.360962 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:47.360962 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:47.360962 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:47.360962 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:47.360962 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:47.363454 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 197/390] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:47.369737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6dedb-1640-44f8-81a2-97477bbbf35d None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:47.441161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f485f6-d0fd-4845-bf98-1d63a600b059 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:47.442163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f485f6-d0fd-4845-bf98-1d63a600b059 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:47.442571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f485f6-d0fd-4845-bf98-1d63a600b059 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:47.445139 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 198/391] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:47.450796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363280a-a87b-43bd-85b7-a0cc9d6bb5c1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:47.451329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363280a-a87b-43bd-85b7-a0cc9d6bb5c1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:47.451809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363280a-a87b-43bd-85b7-a0cc9d6bb5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:47.475788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8363280a-a87b-43bd-85b7-a0cc9d6bb5c1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:47.477414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8363280a-a87b-43bd-85b7-a0cc9d6bb5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:47.522064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6dedb-1640-44f8-81a2-97477bbbf35d None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:47.567383 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 199/392] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:47.581873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf08686-0d0c-49b1-9313-45f19c4ad518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:47.582673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf08686-0d0c-49b1-9313-45f19c4ad518 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:47.583106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf08686-0d0c-49b1-9313-45f19c4ad518 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:47.583514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf08686-0d0c-49b1-9313-45f19c4ad518 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:47.584394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf08686-0d0c-49b1-9313-45f19c4ad518 None admin] RBAC: Adding query filter params (name=member) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:47.584857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf08686-0d0c-49b1-9313-45f19c4ad518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:47.586414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf08686-0d0c-49b1-9313-45f19c4ad518 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:47.599853 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 194/393] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:47.606011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363280a-a87b-43bd-85b7-a0cc9d6bb5c1 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:47.648007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:47.743704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363280a-a87b-43bd-85b7-a0cc9d6bb5c1 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:47.781619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16290b-afaa-4e70-9d7c-861a20c7f8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:47.782525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16290b-afaa-4e70-9d7c-861a20c7f8cc None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:47.782934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16290b-afaa-4e70-9d7c-861a20c7f8cc None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:47.783336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16290b-afaa-4e70-9d7c-861a20c7f8cc None admin] PATH_INFO: `/v3/groups/76923e35980d4ea7b0a2225c166281e3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:47.784636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16290b-afaa-4e70-9d7c-861a20c7f8cc None admin] RBAC: Authorizing `identity:get_group(group_id=76923e35980d4ea7b0a2225c166281e3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:47.786205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16290b-afaa-4e70-9d7c-861a20c7f8cc None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:47.789116 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 200/394] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:47 2022] GET /identity/v3/groups/76923e35980d4ea7b0a2225c166281e3 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:47.797485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf08686-0d0c-49b1-9313-45f19c4ad518 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:47.824430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:47.961877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16290b-afaa-4e70-9d7c-861a20c7f8cc None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:47.970025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf08686-0d0c-49b1-9313-45f19c4ad518 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:48.007836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f91698-6776-4e50-a559-3032d9066b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:48.008540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f91698-6776-4e50-a559-3032d9066b5e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:48.008812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f91698-6776-4e50-a559-3032d9066b5e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:48.009076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f91698-6776-4e50-a559-3032d9066b5e None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:48.010284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f91698-6776-4e50-a559-3032d9066b5e None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:48.011969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f91698-6776-4e50-a559-3032d9066b5e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:48.014130 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 195/395] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:47 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:48.029560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f91698-6776-4e50-a559-3032d9066b5e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:48.101196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:48.112664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16290b-afaa-4e70-9d7c-861a20c7f8cc None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:48.119356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:48.147547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f91698-6776-4e50-a559-3032d9066b5e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:48.196960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f1f36-afaa-4510-9651-552c842e7f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:48.197976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f1f36-afaa-4510-9651-552c842e7f57 None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:48.198655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f1f36-afaa-4510-9651-552c842e7f57 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:48.199957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f1f36-afaa-4510-9651-552c842e7f57 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b/groups/76923e35980d4ea7b0a2225c166281e3/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:48.208982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f1f36-afaa-4510-9651-552c842e7f57 None admin] RBAC: Authorizing `identity:create_grant(project_id=66eda7a4830f4fdeb286e09ae72f278b, group_id=76923e35980d4ea7b0a2225c166281e3, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:48.218934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f1f36-afaa-4510-9651-552c842e7f57 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:48.242503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599168b9-187c-46e1-a7e3-9bccb699319a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:48.242503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599168b9-187c-46e1-a7e3-9bccb699319a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:48.242767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599168b9-187c-46e1-a7e3-9bccb699319a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:48.243053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599168b9-187c-46e1-a7e3-9bccb699319a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:48.272397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599168b9-187c-46e1-a7e3-9bccb699319a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:48.274848 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 196/396] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Aug 20 20:38:48 2022] PUT /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b/groups/76923e35980d4ea7b0a2225c166281e3/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-599168b9-187c-46e1-a7e3-9bccb699319a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:48.288334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:48.289394 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:48.290187 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:48.290187 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:48.290187 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:48.290187 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:48.290187 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:48.290187 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:48.290187 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:48.290187 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:48.290187 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:48.290187 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:48.290187 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:48.290187 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:48.290777 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 201/397] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:38:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:48.296342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f1f36-afaa-4510-9651-552c842e7f57 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:48.489117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f1f36-afaa-4510-9651-552c842e7f57 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:48.524726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116a69a-841d-4d73-b6f7-01391aadd71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:48.525324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116a69a-841d-4d73-b6f7-01391aadd71c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:48.525591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116a69a-841d-4d73-b6f7-01391aadd71c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:48.525850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116a69a-841d-4d73-b6f7-01391aadd71c None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:48.526498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116a69a-841d-4d73-b6f7-01391aadd71c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:48.526790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116a69a-841d-4d73-b6f7-01391aadd71c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:48.528205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116a69a-841d-4d73-b6f7-01391aadd71c None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:48.533668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:48.541175 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 197/398] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:38:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:48.547424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599168b9-187c-46e1-a7e3-9bccb699319a None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:48.650469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a85545-59d0-44fc-b5d5-e27023fe09fa None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:48.650818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a85545-59d0-44fc-b5d5-e27023fe09fa None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:48.651000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a85545-59d0-44fc-b5d5-e27023fe09fa None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:48.652653 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 198/399] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:48.660777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab13a70-5ad3-4946-9668-f1196e21d916 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:48.661060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab13a70-5ad3-4946-9668-f1196e21d916 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:48.661317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab13a70-5ad3-4946-9668-f1196e21d916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:48.682875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0ab13a70-5ad3-4946-9668-f1196e21d916 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:48.685592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0ab13a70-5ad3-4946-9668-f1196e21d916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:48.693176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599168b9-187c-46e1-a7e3-9bccb699319a None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:48.742193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff144a48-1f56-47cc-a199-c33cf1b8bc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:48.742739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff144a48-1f56-47cc-a199-c33cf1b8bc5e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:48.743007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff144a48-1f56-47cc-a199-c33cf1b8bc5e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:48.743263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff144a48-1f56-47cc-a199-c33cf1b8bc5e None admin] PATH_INFO: `/v3/groups/76923e35980d4ea7b0a2225c166281e3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:48.744344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff144a48-1f56-47cc-a199-c33cf1b8bc5e None admin] RBAC: Authorizing `identity:get_group(group_id=76923e35980d4ea7b0a2225c166281e3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:48.745942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff144a48-1f56-47cc-a199-c33cf1b8bc5e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:48.748401 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 202/400] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:48 2022] GET /identity/v3/groups/76923e35980d4ea7b0a2225c166281e3 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:48.749566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34282b9-51a2-4ac9-a95b-14a416e919d3 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:48.749836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34282b9-51a2-4ac9-a95b-14a416e919d3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:48.750099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34282b9-51a2-4ac9-a95b-14a416e919d3 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:48.751405 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 203/401] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:48.756588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d96ac-df44-4ff1-9d5e-efb8a00ed031 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:48.756855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d96ac-df44-4ff1-9d5e-efb8a00ed031 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:48.757118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d96ac-df44-4ff1-9d5e-efb8a00ed031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:48.776487 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 199/402] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:48.778123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab13a70-5ad3-4946-9668-f1196e21d916 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:48.787562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-330d96ac-df44-4ff1-9d5e-efb8a00ed031 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:48.789283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-330d96ac-df44-4ff1-9d5e-efb8a00ed031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:48.836742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:48.915266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab13a70-5ad3-4946-9668-f1196e21d916 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:48.930218 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 204/403] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:48.946774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:48.953440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c0410-a277-45f5-9555-64f6d818fad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:48.954113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c0410-a277-45f5-9555-64f6d818fad1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:48.954388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c0410-a277-45f5-9555-64f6d818fad1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:48.954655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c0410-a277-45f5-9555-64f6d818fad1 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:48.955792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c0410-a277-45f5-9555-64f6d818fad1 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:48.957341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c0410-a277-45f5-9555-64f6d818fad1 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:48.966700 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 200/404] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:48 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:48.973501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d96ac-df44-4ff1-9d5e-efb8a00ed031 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:49.136553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d96ac-df44-4ff1-9d5e-efb8a00ed031 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:49.181967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd0a46-60b6-44d4-b5c4-3de3d6bd12ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:49.182639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd0a46-60b6-44d4-b5c4-3de3d6bd12ba None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:49.182902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd0a46-60b6-44d4-b5c4-3de3d6bd12ba None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:49.183182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd0a46-60b6-44d4-b5c4-3de3d6bd12ba None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0/groups/76923e35980d4ea7b0a2225c166281e3/roles/f8a2ea13798443c48750e9baa83c7461` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:49.185989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd0a46-60b6-44d4-b5c4-3de3d6bd12ba None admin] RBAC: Authorizing `identity:create_grant(project_id=4b6c3729d57640af9e56f5e19081b8b0, group_id=76923e35980d4ea7b0a2225c166281e3, role_id=f8a2ea13798443c48750e9baa83c7461)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:49.187419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd0a46-60b6-44d4-b5c4-3de3d6bd12ba None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:49.205434 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 205/405] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Aug 20 20:38:48 2022] PUT /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0/groups/76923e35980d4ea7b0a2225c166281e3/roles/f8a2ea13798443c48750e9baa83c7461 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:38:49.249243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c0410-a277-45f5-9555-64f6d818fad1 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:49.366502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c0410-a277-45f5-9555-64f6d818fad1 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:49.405849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad5097-1fd1-480a-bb92-d7ac188d7a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:49.407466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad5097-1fd1-480a-bb92-d7ac188d7a60 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:49.407466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad5097-1fd1-480a-bb92-d7ac188d7a60 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:49.407466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad5097-1fd1-480a-bb92-d7ac188d7a60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:49.418536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad5097-1fd1-480a-bb92-d7ac188d7a60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-c7ad5097-1fd1-480a-bb92-d7ac188d7a60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:49.423389 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:49.425658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:49.427719 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:49.427719 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:49.427719 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:49.427719 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:49.427719 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:49.427719 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:49.427719 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:49.427719 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:49.427719 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:49.427719 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:49.427719 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:49.427719 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:49.429277 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 201/406] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:38:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:49.432283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd0a46-60b6-44d4-b5c4-3de3d6bd12ba None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:49.460775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:49.473143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad5097-1fd1-480a-bb92-d7ac188d7a60 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:49.542762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:49.591640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad5097-1fd1-480a-bb92-d7ac188d7a60 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:49.617423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd0a46-60b6-44d4-b5c4-3de3d6bd12ba None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:49.631571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be13053-789b-4369-9228-439db2cb9003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:49.632206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be13053-789b-4369-9228-439db2cb9003 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:49.632509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be13053-789b-4369-9228-439db2cb9003 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:49.632780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be13053-789b-4369-9228-439db2cb9003 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:49.644801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be13053-789b-4369-9228-439db2cb9003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-6be13053-789b-4369-9228-439db2cb9003 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:49.650647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:49.651673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:49.652927 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:49.652927 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:49.652927 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:49.652927 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:49.652927 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:49.652927 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:49.652927 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:49.652927 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:49.652927 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:49.652927 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:49.652927 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:49.652927 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:49.652927 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 202/407] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:38:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:38:49.658689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be13053-789b-4369-9228-439db2cb9003 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:49.694784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87210e58-12cb-4992-b3d4-748786c1d02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:49.695615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87210e58-12cb-4992-b3d4-748786c1d02d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:49.695881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87210e58-12cb-4992-b3d4-748786c1d02d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:49.697041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87210e58-12cb-4992-b3d4-748786c1d02d None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:49.697700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87210e58-12cb-4992-b3d4-748786c1d02d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:49.697992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87210e58-12cb-4992-b3d4-748786c1d02d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:49.702616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87210e58-12cb-4992-b3d4-748786c1d02d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:49.717719 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 206/408] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:38:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:49.723502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87210e58-12cb-4992-b3d4-748786c1d02d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:49.856738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:49.875212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be13053-789b-4369-9228-439db2cb9003 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:49.896576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87210e58-12cb-4992-b3d4-748786c1d02d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:49.901682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:49.938804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f96613-51a6-459b-b7f1-4cc80209683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:49.939373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f96613-51a6-459b-b7f1-4cc80209683e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:49.939648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f96613-51a6-459b-b7f1-4cc80209683e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:49.939937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f96613-51a6-459b-b7f1-4cc80209683e None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:49.940707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f96613-51a6-459b-b7f1-4cc80209683e None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:49.941025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f96613-51a6-459b-b7f1-4cc80209683e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:49.943650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f96613-51a6-459b-b7f1-4cc80209683e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:49.961584 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 203/409] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:49.966080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ede6b5-32fb-4c54-b493-ba5419e8c625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:49.967486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f96613-51a6-459b-b7f1-4cc80209683e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:49.968745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede6b5-32fb-4c54-b493-ba5419e8c625 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:49.969020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede6b5-32fb-4c54-b493-ba5419e8c625 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:49.969288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede6b5-32fb-4c54-b493-ba5419e8c625 None admin] PATH_INFO: `/v3/groups/76923e35980d4ea7b0a2225c166281e3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:49.972425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ede6b5-32fb-4c54-b493-ba5419e8c625 None admin] RBAC: Authorizing `identity:get_group(group_id=76923e35980d4ea7b0a2225c166281e3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:49.974935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ede6b5-32fb-4c54-b493-ba5419e8c625 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:49.976722 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 207/410] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:49 2022] GET /identity/v3/groups/76923e35980d4ea7b0a2225c166281e3 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:49.982611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ede6b5-32fb-4c54-b493-ba5419e8c625 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:50.104147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f96613-51a6-459b-b7f1-4cc80209683e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:50.149132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ede6b5-32fb-4c54-b493-ba5419e8c625 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:50.151708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603f87b-66a2-48de-94f8-8c268664e439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:50.153452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603f87b-66a2-48de-94f8-8c268664e439 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:50.153776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603f87b-66a2-48de-94f8-8c268664e439 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:50.154054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603f87b-66a2-48de-94f8-8c268664e439 None admin] PATH_INFO: `/v3/groups/5779a29e6baf4173a51896d1215c1506` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:50.155363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603f87b-66a2-48de-94f8-8c268664e439 None admin] RBAC: Authorizing `identity:get_group(group_id=5779a29e6baf4173a51896d1215c1506)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:50.156911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603f87b-66a2-48de-94f8-8c268664e439 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:50.159536 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 204/411] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:49 2022] GET /identity/v3/groups/5779a29e6baf4173a51896d1215c1506 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:50.161328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e9a27-39ad-4f39-8337-2b8e50b7e1e5 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:50.161610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e9a27-39ad-4f39-8337-2b8e50b7e1e5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:50.161882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e9a27-39ad-4f39-8337-2b8e50b7e1e5 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:50.164413 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 205/412] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:50.164712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e9a27-39ad-4f39-8337-2b8e50b7e1e5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:50.197451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57111c87-8617-4b48-9eb9-276ba3b58297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:50.198205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57111c87-8617-4b48-9eb9-276ba3b58297 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:50.198577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57111c87-8617-4b48-9eb9-276ba3b58297 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:50.199069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57111c87-8617-4b48-9eb9-276ba3b58297 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:50.200337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57111c87-8617-4b48-9eb9-276ba3b58297 None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:50.202023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57111c87-8617-4b48-9eb9-276ba3b58297 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:50.204254 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 208/413] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:49 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:50.205612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1429c1-543b-4d3e-b8d4-bf5d3449ad3f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:50.205984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1429c1-543b-4d3e-b8d4-bf5d3449ad3f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:50.206349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1429c1-543b-4d3e-b8d4-bf5d3449ad3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:50.224990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3f1429c1-543b-4d3e-b8d4-bf5d3449ad3f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:50.226395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3f1429c1-543b-4d3e-b8d4-bf5d3449ad3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:50.238113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:50.247590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:50.311919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e9a27-39ad-4f39-8337-2b8e50b7e1e5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:50.322122 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 209/414] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:50.323176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1429c1-543b-4d3e-b8d4-bf5d3449ad3f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:50.385348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec0845-b841-43e0-8dc9-ee1ad3cc297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:50.386912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec0845-b841-43e0-8dc9-ee1ad3cc297d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:50.387993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec0845-b841-43e0-8dc9-ee1ad3cc297d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:50.388700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec0845-b841-43e0-8dc9-ee1ad3cc297d None admin] PATH_INFO: `/v3/projects/a5bed9a4b90f48f592cdc8d94741f821` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:50.390930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec0845-b841-43e0-8dc9-ee1ad3cc297d None admin] RBAC: Authorizing `identity:get_project(project_id=a5bed9a4b90f48f592cdc8d94741f821)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:50.393545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec0845-b841-43e0-8dc9-ee1ad3cc297d None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:50.398710 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 206/415] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:50 2022] GET /identity/v3/projects/a5bed9a4b90f48f592cdc8d94741f821 => generated 344 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:50.404852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec0845-b841-43e0-8dc9-ee1ad3cc297d None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:50.547604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1429c1-543b-4d3e-b8d4-bf5d3449ad3f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:50.564624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec0845-b841-43e0-8dc9-ee1ad3cc297d None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:50.583620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:50.618758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64ffe7-789c-489d-a6a2-ada69de7d9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:50.620022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f296cb2-a15a-4d97-9125-0393ecf80957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:50.620801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f296cb2-a15a-4d97-9125-0393ecf80957 None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:50.621057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f296cb2-a15a-4d97-9125-0393ecf80957 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:50.621322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f296cb2-a15a-4d97-9125-0393ecf80957 None admin] PATH_INFO: `/v3/projects/a5bed9a4b90f48f592cdc8d94741f821/groups/5779a29e6baf4173a51896d1215c1506/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:50.622148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64ffe7-789c-489d-a6a2-ada69de7d9bc None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:50.622415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64ffe7-789c-489d-a6a2-ada69de7d9bc None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:50.622673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64ffe7-789c-489d-a6a2-ada69de7d9bc None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b/groups/76923e35980d4ea7b0a2225c166281e3/roles/f8a2ea13798443c48750e9baa83c7461` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:50.624699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f296cb2-a15a-4d97-9125-0393ecf80957 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5bed9a4b90f48f592cdc8d94741f821, group_id=5779a29e6baf4173a51896d1215c1506, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:50.626132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f296cb2-a15a-4d97-9125-0393ecf80957 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:50.629370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64ffe7-789c-489d-a6a2-ada69de7d9bc None admin] RBAC: Authorizing `identity:create_grant(project_id=66eda7a4830f4fdeb286e09ae72f278b, group_id=76923e35980d4ea7b0a2225c166281e3, role_id=f8a2ea13798443c48750e9baa83c7461)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:50.630855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64ffe7-789c-489d-a6a2-ada69de7d9bc None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:50.648195 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 207/416] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Aug 20 20:38:50 2022] PUT /identity/v3/projects/a5bed9a4b90f48f592cdc8d94741f821/groups/5779a29e6baf4173a51896d1215c1506/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:38:50.654771 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 210/417] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Aug 20 20:38:50 2022] PUT /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b/groups/76923e35980d4ea7b0a2225c166281e3/roles/f8a2ea13798443c48750e9baa83c7461 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:38:50.700111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f296cb2-a15a-4d97-9125-0393ecf80957 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:50.818320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f296cb2-a15a-4d97-9125-0393ecf80957 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:50.857245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa372e6-d955-49f5-bba2-0d3acaac5d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:50.857881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa372e6-d955-49f5-bba2-0d3acaac5d10 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:50.858151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa372e6-d955-49f5-bba2-0d3acaac5d10 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:50.858415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa372e6-d955-49f5-bba2-0d3acaac5d10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:50.870905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa372e6-d955-49f5-bba2-0d3acaac5d10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-ffa372e6-d955-49f5-bba2-0d3acaac5d10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:50.876117 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:50.877198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:50.878009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:50.878009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:50.878009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:50.878009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:50.878009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:50.878009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:50.878009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:50.878009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:50.878009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:50.878009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:50.878009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:50.878009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:50.878009 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 208/418] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:38:50.885169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64ffe7-789c-489d-a6a2-ada69de7d9bc None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:50.890380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:51.013400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64ffe7-789c-489d-a6a2-ada69de7d9bc None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:51.049613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee5fa7-64c3-49e3-981d-a94efbc6edb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:51.050215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee5fa7-64c3-49e3-981d-a94efbc6edb1 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:51.050482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee5fa7-64c3-49e3-981d-a94efbc6edb1 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:51.050741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee5fa7-64c3-49e3-981d-a94efbc6edb1 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:51.051393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee5fa7-64c3-49e3-981d-a94efbc6edb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:51.051681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee5fa7-64c3-49e3-981d-a94efbc6edb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:51.053075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee5fa7-64c3-49e3-981d-a94efbc6edb1 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:51.062731 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 211/419] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:51.068667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa372e6-d955-49f5-bba2-0d3acaac5d10 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:51.073972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589c73c-5352-4848-b9fe-55510b7d1af0 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:51.074378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589c73c-5352-4848-b9fe-55510b7d1af0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:51.074653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589c73c-5352-4848-b9fe-55510b7d1af0 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:51.077395 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 212/420] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:38:51.081575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546a3ae-031f-44a4-a07d-a3043efd33be None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:51.081849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546a3ae-031f-44a4-a07d-a3043efd33be None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:51.082110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546a3ae-031f-44a4-a07d-a3043efd33be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:51.101529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1546a3ae-031f-44a4-a07d-a3043efd33be None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:51.102859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1546a3ae-031f-44a4-a07d-a3043efd33be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:51.127933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:51.199510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa372e6-d955-49f5-bba2-0d3acaac5d10 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:51.208643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:51.233254 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 213/421] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:51.242264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba173e81-68f1-4d3b-9c03-15273d5b9a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:51.242883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba173e81-68f1-4d3b-9c03-15273d5b9a1f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:51.243155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba173e81-68f1-4d3b-9c03-15273d5b9a1f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:51.243418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba173e81-68f1-4d3b-9c03-15273d5b9a1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:51.323798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba173e81-68f1-4d3b-9c03-15273d5b9a1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:51.325374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba173e81-68f1-4d3b-9c03-15273d5b9a1f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:51.369447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:51.459894 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 209/422] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:51.465817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546a3ae-031f-44a4-a07d-a3043efd33be None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:51.590580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546a3ae-031f-44a4-a07d-a3043efd33be None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:51.633649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba173e81-68f1-4d3b-9c03-15273d5b9a1f None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:51.634941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907e3d6-f23e-4280-bd7d-91ab2ac9ad3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:51.636226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907e3d6-f23e-4280-bd7d-91ab2ac9ad3d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:51.636871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907e3d6-f23e-4280-bd7d-91ab2ac9ad3d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:51.637364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907e3d6-f23e-4280-bd7d-91ab2ac9ad3d None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:51.638779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907e3d6-f23e-4280-bd7d-91ab2ac9ad3d None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:51.640662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907e3d6-f23e-4280-bd7d-91ab2ac9ad3d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:51.643059 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 214/423] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:51 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:51.649318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907e3d6-f23e-4280-bd7d-91ab2ac9ad3d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:51.751698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:51.808470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907e3d6-f23e-4280-bd7d-91ab2ac9ad3d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:51.820244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba173e81-68f1-4d3b-9c03-15273d5b9a1f None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:51.851011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b549be-2fe8-48d6-b9d2-1f8a0ccd71ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:51.851626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b549be-2fe8-48d6-b9d2-1f8a0ccd71ec None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:51.851903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b549be-2fe8-48d6-b9d2-1f8a0ccd71ec None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:51.852243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b549be-2fe8-48d6-b9d2-1f8a0ccd71ec None admin] PATH_INFO: `/v3/groups/76923e35980d4ea7b0a2225c166281e3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:51.853359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b549be-2fe8-48d6-b9d2-1f8a0ccd71ec None admin] RBAC: Authorizing `identity:get_group(group_id=76923e35980d4ea7b0a2225c166281e3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:51.854784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b549be-2fe8-48d6-b9d2-1f8a0ccd71ec None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:51.857296 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 215/424] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:51 2022] GET /identity/v3/groups/76923e35980d4ea7b0a2225c166281e3 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:51.857685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d52c8-0dc2-4587-91e3-2eaa73ddbdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:51.858452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d52c8-0dc2-4587-91e3-2eaa73ddbdd9 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:51.858725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225e4f-823d-4ac8-9e70-05eed25158fb None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:51.858854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d52c8-0dc2-4587-91e3-2eaa73ddbdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:51.858967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225e4f-823d-4ac8-9e70-05eed25158fb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:51.859107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d52c8-0dc2-4587-91e3-2eaa73ddbdd9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:51.859216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225e4f-823d-4ac8-9e70-05eed25158fb None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:51.860390 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 216/425] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:51.865723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:51.867100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca379-f721-405e-906e-353166c39a4d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:51.867375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca379-f721-405e-906e-353166c39a4d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:51.867807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca379-f721-405e-906e-353166c39a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:51.872578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d52c8-0dc2-4587-91e3-2eaa73ddbdd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-3c4d52c8-0dc2-4587-91e3-2eaa73ddbdd9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:51.877727 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:51.878830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:51.879729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:51.879729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:51.879729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:51.879729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:51.879729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:51.879729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:51.879729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:51.879729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:51.879729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:51.879729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:51.879729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 20 20:38:51.879729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:51.881913 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 210/426] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:51.882825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d52c8-0dc2-4587-91e3-2eaa73ddbdd9 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:51.887474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-96dca379-f721-405e-906e-353166c39a4d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:51.888913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-96dca379-f721-405e-906e-353166c39a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:51.959728 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 217/427] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:51.961130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca379-f721-405e-906e-353166c39a4d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:51.989043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d52c8-0dc2-4587-91e3-2eaa73ddbdd9 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:52.024734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:52.027391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c71580-f1a1-4759-a7ea-52f6c636f539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:52.028425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c71580-f1a1-4759-a7ea-52f6c636f539 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:52.028844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c71580-f1a1-4759-a7ea-52f6c636f539 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:52.029269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c71580-f1a1-4759-a7ea-52f6c636f539 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:52.030476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c71580-f1a1-4759-a7ea-52f6c636f539 None admin] RBAC: Adding query filter params (group.id=76923e35980d4ea7b0a2225c166281e3, role.id=5eaf9701f37e480bb87c89f7391d5d3a, scope.project.id=4b6c3729d57640af9e56f5e19081b8b0) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:52.031376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c71580-f1a1-4759-a7ea-52f6c636f539 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:52.033025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c71580-f1a1-4759-a7ea-52f6c636f539 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:52.039905 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 211/428] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Aug 20 20:38:51 2022] GET /identity/v3/role_assignments?group.id=76923e35980d4ea7b0a2225c166281e3&scope.project.id=4b6c3729d57640af9e56f5e19081b8b0&role.id=5eaf9701f37e480bb87c89f7391d5d3a => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:52.068023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca379-f721-405e-906e-353166c39a4d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:52.105969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c14172-0d9f-4a71-a4a6-0ecab6376b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:52.106510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c14172-0d9f-4a71-a4a6-0ecab6376b40 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:52.106777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c14172-0d9f-4a71-a4a6-0ecab6376b40 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:52.107041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c14172-0d9f-4a71-a4a6-0ecab6376b40 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:52.107697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c14172-0d9f-4a71-a4a6-0ecab6376b40 None admin] RBAC: Adding query filter params (name=member) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:52.107988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c14172-0d9f-4a71-a4a6-0ecab6376b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:52.109419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c14172-0d9f-4a71-a4a6-0ecab6376b40 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:52.119319 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 218/429] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:38:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:52.125699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c71580-f1a1-4759-a7ea-52f6c636f539 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:52.244778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:52.296877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c71580-f1a1-4759-a7ea-52f6c636f539 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:52.337994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96f979-2c38-41e5-8ebf-9ffc605aec5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:52.338599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96f979-2c38-41e5-8ebf-9ffc605aec5c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:52.338873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96f979-2c38-41e5-8ebf-9ffc605aec5c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:52.339141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96f979-2c38-41e5-8ebf-9ffc605aec5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:52.418541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96f979-2c38-41e5-8ebf-9ffc605aec5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:52.420356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96f979-2c38-41e5-8ebf-9ffc605aec5c None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:52.449926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c14172-0d9f-4a71-a4a6-0ecab6376b40 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:52.506099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:52.571007 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 212/430] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:38:52.577537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96f979-2c38-41e5-8ebf-9ffc605aec5c None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:52.590034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c14172-0d9f-4a71-a4a6-0ecab6376b40 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:52.617498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:52.631003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a1e74-a6a5-49dc-a145-3efbe404dccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:52.631640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a1e74-a6a5-49dc-a145-3efbe404dccf None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:52.631912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a1e74-a6a5-49dc-a145-3efbe404dccf None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:52.632178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a1e74-a6a5-49dc-a145-3efbe404dccf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:52.645817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a1e74-a6a5-49dc-a145-3efbe404dccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-f15a1e74-a6a5-49dc-a145-3efbe404dccf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:52.652246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:52.653280 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:52.654130 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:52.654130 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:52.654130 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:52.654130 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:52.654130 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:52.654130 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:52.654130 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:52.654130 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:52.654130 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:52.654130 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:52.654130 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:52.654130 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:52.654130 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 219/431] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:38:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:52.660318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a1e74-a6a5-49dc-a145-3efbe404dccf None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:52.716968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96f979-2c38-41e5-8ebf-9ffc605aec5c None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:52.766125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f0d81-e34b-4315-8ada-09c63cd17ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:52.766791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f0d81-e34b-4315-8ada-09c63cd17ff3 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:52.767064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f0d81-e34b-4315-8ada-09c63cd17ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:52.767331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f0d81-e34b-4315-8ada-09c63cd17ff3 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:52.768516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f0d81-e34b-4315-8ada-09c63cd17ff3 None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:52.769968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f0d81-e34b-4315-8ada-09c63cd17ff3 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:52.777644 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 213/432] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:52 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:52.780996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f0d81-e34b-4315-8ada-09c63cd17ff3 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:52.815405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a1e74-a6a5-49dc-a145-3efbe404dccf None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:52.818006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:52.876350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381be66-3891-4b10-b5e8-63da10a1d7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:52.877269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381be66-3891-4b10-b5e8-63da10a1d7c4 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:52.877987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381be66-3891-4b10-b5e8-63da10a1d7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:52.878347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381be66-3891-4b10-b5e8-63da10a1d7c4 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:52.880235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381be66-3891-4b10-b5e8-63da10a1d7c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:52.880619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381be66-3891-4b10-b5e8-63da10a1d7c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:52.882236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381be66-3891-4b10-b5e8-63da10a1d7c4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:52.896017 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 220/433] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:38:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:52.904299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381be66-3891-4b10-b5e8-63da10a1d7c4 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:52.924344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f0d81-e34b-4315-8ada-09c63cd17ff3 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:52.964039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4c07a-5a0b-4c15-ba60-80930f7d9706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:52.965609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4c07a-5a0b-4c15-ba60-80930f7d9706 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:52.966178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4c07a-5a0b-4c15-ba60-80930f7d9706 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:52.966642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4c07a-5a0b-4c15-ba60-80930f7d9706 None admin] PATH_INFO: `/v3/groups/76923e35980d4ea7b0a2225c166281e3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:52.968077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4c07a-5a0b-4c15-ba60-80930f7d9706 None admin] RBAC: Authorizing `identity:get_group(group_id=76923e35980d4ea7b0a2225c166281e3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:52.969747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4c07a-5a0b-4c15-ba60-80930f7d9706 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:52.971842 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 214/434] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:52 2022] GET /identity/v3/groups/76923e35980d4ea7b0a2225c166281e3 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:52.975038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:52.981057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4c07a-5a0b-4c15-ba60-80930f7d9706 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:53.040988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381be66-3891-4b10-b5e8-63da10a1d7c4 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:53.081969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516a5c0-710d-4324-9c03-eb613f3d223c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:53.082999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516a5c0-710d-4324-9c03-eb613f3d223c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:53.083658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516a5c0-710d-4324-9c03-eb613f3d223c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:53.084122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516a5c0-710d-4324-9c03-eb613f3d223c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:53.118909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:53.127985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4c07a-5a0b-4c15-ba60-80930f7d9706 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:53.167812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a516a5c0-710d-4324-9c03-eb613f3d223c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:53.168893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d1d67-d5e7-4ae2-b142-4d0acafb3eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:53.169305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a516a5c0-710d-4324-9c03-eb613f3d223c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:53.170025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d1d67-d5e7-4ae2-b142-4d0acafb3eb9 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:53.170505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d1d67-d5e7-4ae2-b142-4d0acafb3eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:53.171096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d1d67-d5e7-4ae2-b142-4d0acafb3eb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:53.172708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d1d67-d5e7-4ae2-b142-4d0acafb3eb9 None admin] RBAC: Adding query filter params (group.id=76923e35980d4ea7b0a2225c166281e3, role.id=5eaf9701f37e480bb87c89f7391d5d3a, scope.project.id=66eda7a4830f4fdeb286e09ae72f278b) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:53.173240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d1d67-d5e7-4ae2-b142-4d0acafb3eb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:53.174884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d1d67-d5e7-4ae2-b142-4d0acafb3eb9 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:53.182496 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 215/435] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Aug 20 20:38:52 2022] GET /identity/v3/role_assignments?group.id=76923e35980d4ea7b0a2225c166281e3&scope.project.id=66eda7a4830f4fdeb286e09ae72f278b&role.id=5eaf9701f37e480bb87c89f7391d5d3a => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:53.286224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:53.432438 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 221/436] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 20 20:38:53.435564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d1d67-d5e7-4ae2-b142-4d0acafb3eb9 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:53.456606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245738f1-5adf-4097-9168-00cae54ad7c2 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:53.457205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245738f1-5adf-4097-9168-00cae54ad7c2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:53.457663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245738f1-5adf-4097-9168-00cae54ad7c2 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:53.463632 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 222/437] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:38:53.465128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab69be-5258-41f0-a758-75f46f57cf87 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:53.465554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab69be-5258-41f0-a758-75f46f57cf87 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:53.465967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab69be-5258-41f0-a758-75f46f57cf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:53.484136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b5ab69be-5258-41f0-a758-75f46f57cf87 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:53.485345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b5ab69be-5258-41f0-a758-75f46f57cf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:53.557851 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 223/438] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:53.559163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc6211-3566-4a1b-b23c-c6713c43506f None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:53.559437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc6211-3566-4a1b-b23c-c6713c43506f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:53.560022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc6211-3566-4a1b-b23c-c6713c43506f None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:53.564173 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 224/439] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:53.566711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68c221-e73e-42c3-8ea9-6924fc64685e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:53.567292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68c221-e73e-42c3-8ea9-6924fc64685e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:53.567570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68c221-e73e-42c3-8ea9-6924fc64685e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:53.569841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d1d67-d5e7-4ae2-b142-4d0acafb3eb9 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:53.588108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5c68c221-e73e-42c3-8ea9-6924fc64685e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:53.589432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5c68c221-e73e-42c3-8ea9-6924fc64685e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:53.605971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c83fe-a6de-4312-9659-fc46f0526ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:53.606600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c83fe-a6de-4312-9659-fc46f0526ff0 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:53.606986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c83fe-a6de-4312-9659-fc46f0526ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:53.607256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c83fe-a6de-4312-9659-fc46f0526ff0 None admin] PATH_INFO: `/v3/projects/4b6c3729d57640af9e56f5e19081b8b0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:53.608519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c83fe-a6de-4312-9659-fc46f0526ff0 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6c3729d57640af9e56f5e19081b8b0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:53.609942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c83fe-a6de-4312-9659-fc46f0526ff0 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:53.616320 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 216/440] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:53 2022] GET /identity/v3/projects/4b6c3729d57640af9e56f5e19081b8b0 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:53.617132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c83fe-a6de-4312-9659-fc46f0526ff0 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:53.647781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:53.668397 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 225/441] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:38:53.729821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c83fe-a6de-4312-9659-fc46f0526ff0 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:53.764226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3b382-db49-49c3-af94-08c3b3baf635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:53.764885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3b382-db49-49c3-af94-08c3b3baf635 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:53.765157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3b382-db49-49c3-af94-08c3b3baf635 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:53.765420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3b382-db49-49c3-af94-08c3b3baf635 None admin] PATH_INFO: `/v3/groups/76923e35980d4ea7b0a2225c166281e3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:53.766538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3b382-db49-49c3-af94-08c3b3baf635 None admin] RBAC: Authorizing `identity:get_group(group_id=76923e35980d4ea7b0a2225c166281e3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:53.767966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3b382-db49-49c3-af94-08c3b3baf635 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:53.769725 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 217/442] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:53 2022] GET /identity/v3/groups/76923e35980d4ea7b0a2225c166281e3 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:53.775478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68c221-e73e-42c3-8ea9-6924fc64685e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:53.784071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:53.880135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68c221-e73e-42c3-8ea9-6924fc64685e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:53.883203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3b382-db49-49c3-af94-08c3b3baf635 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:53.917583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c41cfa-e9cf-405c-8c83-f4fed9f2c34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:53.918222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c41cfa-e9cf-405c-8c83-f4fed9f2c34f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:53.918493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c41cfa-e9cf-405c-8c83-f4fed9f2c34f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:53.918858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c41cfa-e9cf-405c-8c83-f4fed9f2c34f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:53.920123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c41cfa-e9cf-405c-8c83-f4fed9f2c34f None admin] RBAC: Adding query filter params (group.id=76923e35980d4ea7b0a2225c166281e3, role.id=f8a2ea13798443c48750e9baa83c7461, scope.project.id=4b6c3729d57640af9e56f5e19081b8b0) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:53.920123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c41cfa-e9cf-405c-8c83-f4fed9f2c34f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:53.921366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c41cfa-e9cf-405c-8c83-f4fed9f2c34f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:53.927690 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 226/443] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Aug 20 20:38:53 2022] GET /identity/v3/role_assignments?group.id=76923e35980d4ea7b0a2225c166281e3&scope.project.id=4b6c3729d57640af9e56f5e19081b8b0&role.id=f8a2ea13798443c48750e9baa83c7461 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:53.988461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:53.995789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3b382-db49-49c3-af94-08c3b3baf635 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:54.008474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c41cfa-e9cf-405c-8c83-f4fed9f2c34f None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:54.034877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54c145-8e66-4d64-a3f4-554ecfcdce3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:54.035508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54c145-8e66-4d64-a3f4-554ecfcdce3f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:54.035791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54c145-8e66-4d64-a3f4-554ecfcdce3f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:54.036048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54c145-8e66-4d64-a3f4-554ecfcdce3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:54.047579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54c145-8e66-4d64-a3f4-554ecfcdce3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9b54c145-8e66-4d64-a3f4-554ecfcdce3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:54.053527 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:54.054493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:54.055276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:54.055276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:54.055276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:54.055276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:54.055276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:54.055276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:54.055276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:54.055276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:54.055276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:54.055276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:54.055276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:38:54.055276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:54.055276 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 218/444] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:38:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:54.061272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54c145-8e66-4d64-a3f4-554ecfcdce3f None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:54.090035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:54.126235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c41cfa-e9cf-405c-8c83-f4fed9f2c34f None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:54.164834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4f81a-2076-44ef-a178-ec892066a59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:54.165387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4f81a-2076-44ef-a178-ec892066a59e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:54.165660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4f81a-2076-44ef-a178-ec892066a59e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:54.165926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4f81a-2076-44ef-a178-ec892066a59e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:54.174625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54c145-8e66-4d64-a3f4-554ecfcdce3f None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:54.178671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4f81a-2076-44ef-a178-ec892066a59e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-c1d4f81a-2076-44ef-a178-ec892066a59e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:38:54.184245 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:54.185592 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:54.186411 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:54.186411 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:54.186411 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:54.186411 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:54.186411 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:38:54.186411 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:38:54.186411 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:38:54.186411 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:38:54.186411 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:38:54.186411 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:38:54.186411 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:38:54.186411 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:54.187004 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 227/445] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:38:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:38:54.192576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4f81a-2076-44ef-a178-ec892066a59e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:54.214035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8fea4-8636-495a-9fce-63661a638d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:54.214759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8fea4-8636-495a-9fce-63661a638d81 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:54.215035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8fea4-8636-495a-9fce-63661a638d81 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:54.215349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8fea4-8636-495a-9fce-63661a638d81 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:54.216018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8fea4-8636-495a-9fce-63661a638d81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:54.216311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8fea4-8636-495a-9fce-63661a638d81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:54.217830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8fea4-8636-495a-9fce-63661a638d81 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:54.228046 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 219/446] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:38:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:54.234467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8fea4-8636-495a-9fce-63661a638d81 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:54.258149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:54.331740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4f81a-2076-44ef-a178-ec892066a59e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:54.354418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8fea4-8636-495a-9fce-63661a638d81 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:54.367999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:54.376144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb9b65-e7b2-4367-b058-4b2141212675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:54.376791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb9b65-e7b2-4367-b058-4b2141212675 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:54.377064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb9b65-e7b2-4367-b058-4b2141212675 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:54.377328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb9b65-e7b2-4367-b058-4b2141212675 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:54.377990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb9b65-e7b2-4367-b058-4b2141212675 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:54.378288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb9b65-e7b2-4367-b058-4b2141212675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:54.379657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb9b65-e7b2-4367-b058-4b2141212675 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:54.388857 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 228/447] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:38:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:54.394573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dba4c-755c-4fa8-a004-a6aa3459800e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:54.394729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb9b65-e7b2-4367-b058-4b2141212675 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:54.395181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dba4c-755c-4fa8-a004-a6aa3459800e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:54.395456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dba4c-755c-4fa8-a004-a6aa3459800e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:54.395721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dba4c-755c-4fa8-a004-a6aa3459800e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:54.487717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5dba4c-755c-4fa8-a004-a6aa3459800e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:54.489249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5dba4c-755c-4fa8-a004-a6aa3459800e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:54.507954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:54.513287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb9b65-e7b2-4367-b058-4b2141212675 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:54.546286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cdc0c-814e-4a05-ac7f-6dc6cd359995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:54.546887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cdc0c-814e-4a05-ac7f-6dc6cd359995 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:54.547158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cdc0c-814e-4a05-ac7f-6dc6cd359995 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:54.547423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cdc0c-814e-4a05-ac7f-6dc6cd359995 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:54.590374 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 220/448] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 20 20:38:54.596196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dba4c-755c-4fa8-a004-a6aa3459800e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:54.605426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:54.615184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0cdc0c-814e-4a05-ac7f-6dc6cd359995 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:54.616675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0cdc0c-814e-4a05-ac7f-6dc6cd359995 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:54.697540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dba4c-755c-4fa8-a004-a6aa3459800e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:54.714349 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 229/449] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:38:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 20 20:38:54.715216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:54.720436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cdc0c-814e-4a05-ac7f-6dc6cd359995 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:54.732829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839b22b-ca57-4d81-a058-31d3c284998a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:54.733455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839b22b-ca57-4d81-a058-31d3c284998a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:54.733725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839b22b-ca57-4d81-a058-31d3c284998a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:54.733990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839b22b-ca57-4d81-a058-31d3c284998a None admin] PATH_INFO: `/v3/projects/a5bed9a4b90f48f592cdc8d94741f821` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:54.735124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839b22b-ca57-4d81-a058-31d3c284998a None admin] RBAC: Authorizing `identity:get_project(project_id=a5bed9a4b90f48f592cdc8d94741f821)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:54.736644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839b22b-ca57-4d81-a058-31d3c284998a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:54.738321 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 221/450] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:54 2022] GET /identity/v3/projects/a5bed9a4b90f48f592cdc8d94741f821 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:54.743572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839b22b-ca57-4d81-a058-31d3c284998a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:54.814075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:54.823267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cdc0c-814e-4a05-ac7f-6dc6cd359995 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:54.844328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839b22b-ca57-4d81-a058-31d3c284998a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:54.858329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fbf1c-26e6-4e5b-b4b8-04992763f9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:54.858992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fbf1c-26e6-4e5b-b4b8-04992763f9c6 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:54.859325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fbf1c-26e6-4e5b-b4b8-04992763f9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:54.859614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fbf1c-26e6-4e5b-b4b8-04992763f9c6 None admin] PATH_INFO: `/v3/projects/66eda7a4830f4fdeb286e09ae72f278b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:54.860856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fbf1c-26e6-4e5b-b4b8-04992763f9c6 None admin] RBAC: Authorizing `identity:get_project(project_id=66eda7a4830f4fdeb286e09ae72f278b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:54.862387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fbf1c-26e6-4e5b-b4b8-04992763f9c6 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:54.868610 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 230/451] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:38:54 2022] GET /identity/v3/projects/66eda7a4830f4fdeb286e09ae72f278b => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:54.869946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fbf1c-26e6-4e5b-b4b8-04992763f9c6 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:54.879811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853028f7-04dc-485e-baff-975daf9e52e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:54.880516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853028f7-04dc-485e-baff-975daf9e52e8 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:54.880790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853028f7-04dc-485e-baff-975daf9e52e8 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:54.881074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853028f7-04dc-485e-baff-975daf9e52e8 None admin] PATH_INFO: `/v3/groups/5779a29e6baf4173a51896d1215c1506` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:54.882322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853028f7-04dc-485e-baff-975daf9e52e8 None admin] RBAC: Authorizing `identity:get_group(group_id=5779a29e6baf4173a51896d1215c1506)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:54.883876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853028f7-04dc-485e-baff-975daf9e52e8 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:54.885578 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 222/452] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:54 2022] GET /identity/v3/groups/5779a29e6baf4173a51896d1215c1506 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:54.891775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853028f7-04dc-485e-baff-975daf9e52e8 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:54.950861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:54.969200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fbf1c-26e6-4e5b-b4b8-04992763f9c6 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:54.995034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853028f7-04dc-485e-baff-975daf9e52e8 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:55.003805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c9d2d-6c77-48c3-841f-bb07fb00ebbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:55.004459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c9d2d-6c77-48c3-841f-bb07fb00ebbc None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:55.004731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c9d2d-6c77-48c3-841f-bb07fb00ebbc None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:55.004996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c9d2d-6c77-48c3-841f-bb07fb00ebbc None admin] PATH_INFO: `/v3/groups/76923e35980d4ea7b0a2225c166281e3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:55.006144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c9d2d-6c77-48c3-841f-bb07fb00ebbc None admin] RBAC: Authorizing `identity:get_group(group_id=76923e35980d4ea7b0a2225c166281e3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:55.007604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c9d2d-6c77-48c3-841f-bb07fb00ebbc None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:55.009255 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 231/453] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Aug 20 20:38:54 2022] GET /identity/v3/groups/76923e35980d4ea7b0a2225c166281e3 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:55.015128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c9d2d-6c77-48c3-841f-bb07fb00ebbc None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:55.029133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c992ac-b393-452d-84ce-d11658f71d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:55.029723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c992ac-b393-452d-84ce-d11658f71d97 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:55.029990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c992ac-b393-452d-84ce-d11658f71d97 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:55.030251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c992ac-b393-452d-84ce-d11658f71d97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:55.030954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c992ac-b393-452d-84ce-d11658f71d97 None admin] RBAC: Adding query filter params (group.id=5779a29e6baf4173a51896d1215c1506, role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=a5bed9a4b90f48f592cdc8d94741f821) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:55.031251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c992ac-b393-452d-84ce-d11658f71d97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:55.032702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c992ac-b393-452d-84ce-d11658f71d97 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:55.038260 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 223/454] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Aug 20 20:38:54 2022] GET /identity/v3/role_assignments?group.id=5779a29e6baf4173a51896d1215c1506&scope.project.id=a5bed9a4b90f48f592cdc8d94741f821&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:55.044079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:55.121729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c9d2d-6c77-48c3-841f-bb07fb00ebbc None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:55.158977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92957d9a-ccb0-45e8-8572-22c8694d8d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:55.159587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92957d9a-ccb0-45e8-8572-22c8694d8d14 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:55.159852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92957d9a-ccb0-45e8-8572-22c8694d8d14 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:55.160110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92957d9a-ccb0-45e8-8572-22c8694d8d14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:55.160847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92957d9a-ccb0-45e8-8572-22c8694d8d14 None admin] RBAC: Adding query filter params (group.id=76923e35980d4ea7b0a2225c166281e3, role.id=f8a2ea13798443c48750e9baa83c7461, scope.project.id=66eda7a4830f4fdeb286e09ae72f278b) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:55.161141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92957d9a-ccb0-45e8-8572-22c8694d8d14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:55.162566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92957d9a-ccb0-45e8-8572-22c8694d8d14 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:55.169455 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 232/455] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Aug 20 20:38:55 2022] GET /identity/v3/role_assignments?group.id=76923e35980d4ea7b0a2225c166281e3&scope.project.id=66eda7a4830f4fdeb286e09ae72f278b&role.id=f8a2ea13798443c48750e9baa83c7461 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:57.938447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e84de9-2a48-4d32-a094-41ae77617ac2 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:57.939431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e84de9-2a48-4d32-a094-41ae77617ac2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:57.939927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e84de9-2a48-4d32-a094-41ae77617ac2 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:57.941733 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 224/456] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:57.946821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a99aa-a09f-4d07-8b88-a031dc91bfb1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:57.947599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a99aa-a09f-4d07-8b88-a031dc91bfb1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:57.948071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a99aa-a09f-4d07-8b88-a031dc91bfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:57.977335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ff6a99aa-a09f-4d07-8b88-a031dc91bfb1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:57.978827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ff6a99aa-a09f-4d07-8b88-a031dc91bfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:58.055958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c767d-2299-41c7-b8ca-8ce104f0d6e4 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:58.056644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c767d-2299-41c7-b8ca-8ce104f0d6e4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:58.057120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c767d-2299-41c7-b8ca-8ce104f0d6e4 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:58.058939 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 233/457] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:58.060162 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 225/458] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:58.065653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf11b4b-e9b7-4dde-854e-76215ee35f63 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:58.065927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf11b4b-e9b7-4dde-854e-76215ee35f63 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:58.066193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf11b4b-e9b7-4dde-854e-76215ee35f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:58.089442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5bf11b4b-e9b7-4dde-854e-76215ee35f63 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:58.090840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5bf11b4b-e9b7-4dde-854e-76215ee35f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:58.109770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:58.165802 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 234/459] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:58.305416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ac0c7-7712-472f-86cf-a8e5dca8bfa0 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:58.306056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ac0c7-7712-472f-86cf-a8e5dca8bfa0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:58.306520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ac0c7-7712-472f-86cf-a8e5dca8bfa0 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:58.308822 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 226/460] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:58.314004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599740b0-d83c-4880-8a73-511feb3897cf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:58.314277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599740b0-d83c-4880-8a73-511feb3897cf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:58.314541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599740b0-d83c-4880-8a73-511feb3897cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:58.341435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-599740b0-d83c-4880-8a73-511feb3897cf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:58.342621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-599740b0-d83c-4880-8a73-511feb3897cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:58.436508 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 235/461] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:58.515495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ac0c7-7712-472f-86cf-a8e5dca8bfa0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:58.603482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599740b0-d83c-4880-8a73-511feb3897cf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:58.680331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ac0c7-7712-472f-86cf-a8e5dca8bfa0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:58.705299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:58.738740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de399544-7bf4-45b4-945f-59173b002d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:58.739369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de399544-7bf4-45b4-945f-59173b002d7e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:58.739644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de399544-7bf4-45b4-945f-59173b002d7e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:58.739913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de399544-7bf4-45b4-945f-59173b002d7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:58.762078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de399544-7bf4-45b4-945f-59173b002d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:58.763539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de399544-7bf4-45b4-945f-59173b002d7e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:58.769269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-de399544-7bf4-45b4-945f-59173b002d7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:38:58.794815 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:58.795765 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:38:58.796634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:38:58.796634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:38:58.796634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:38:58.799254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599740b0-d83c-4880-8a73-511feb3897cf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:58.801577 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 227/462] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:58.807256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de399544-7bf4-45b4-945f-59173b002d7e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:58.868219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981c747-786d-46f1-9fc2-5b21d06ceaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:58.869318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981c747-786d-46f1-9fc2-5b21d06ceaaf None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:58.869911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981c747-786d-46f1-9fc2-5b21d06ceaaf None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:58.870561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981c747-786d-46f1-9fc2-5b21d06ceaaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:58.884193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981c747-786d-46f1-9fc2-5b21d06ceaaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:58.886705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981c747-786d-46f1-9fc2-5b21d06ceaaf None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-0981c747-786d-46f1-9fc2-5b21d06ceaaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:38:58.905565 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:58.906536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:38:58.907467 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:38:58.907467 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:38:58.907467 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:58.909442 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 236/463] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:38:58.910548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981c747-786d-46f1-9fc2-5b21d06ceaaf None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:58.944364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de399544-7bf4-45b4-945f-59173b002d7e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:58.977573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845fcec-13de-4a0b-a17b-0197859b661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:58.978447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845fcec-13de-4a0b-a17b-0197859b661a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:58.978853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845fcec-13de-4a0b-a17b-0197859b661a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:58.979270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845fcec-13de-4a0b-a17b-0197859b661a None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:58.980055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845fcec-13de-4a0b-a17b-0197859b661a None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:58.980536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845fcec-13de-4a0b-a17b-0197859b661a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:58.982024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845fcec-13de-4a0b-a17b-0197859b661a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:59.000044 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 228/464] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:38:59.001128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845fcec-13de-4a0b-a17b-0197859b661a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:59.022205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981c747-786d-46f1-9fc2-5b21d06ceaaf None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:59.038509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:59.059828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fcfbf-ffb1-4a20-a49b-bfcc3351b801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:59.060499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fcfbf-ffb1-4a20-a49b-bfcc3351b801 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:59.060777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fcfbf-ffb1-4a20-a49b-bfcc3351b801 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:59.061046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fcfbf-ffb1-4a20-a49b-bfcc3351b801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:59.063117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:59.072482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fcfbf-ffb1-4a20-a49b-bfcc3351b801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:59.074104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fcfbf-ffb1-4a20-a49b-bfcc3351b801 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-fa7fcfbf-ffb1-4a20-a49b-bfcc3351b801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:38:59.088476 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:38:59.089657 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:38:59.090555 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:38:59.090555 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:38:59.090555 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:38:59.090555 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 237/465] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:38:59.091269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fcfbf-ffb1-4a20-a49b-bfcc3351b801 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:59.122211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845fcec-13de-4a0b-a17b-0197859b661a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:59.159679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f1d20-625b-451b-82a5-55e2ee6f1fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:59.160309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f1d20-625b-451b-82a5-55e2ee6f1fae None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:59.160572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f1d20-625b-451b-82a5-55e2ee6f1fae None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:59.160908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f1d20-625b-451b-82a5-55e2ee6f1fae None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:59.161458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f1d20-625b-451b-82a5-55e2ee6f1fae None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:59.161750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f1d20-625b-451b-82a5-55e2ee6f1fae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:59.163124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f1d20-625b-451b-82a5-55e2ee6f1fae None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:59.179745 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 229/466] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:59.179745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00327093-7544-45e3-99d0-3db493f0788d None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:59.179745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00327093-7544-45e3-99d0-3db493f0788d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:59.179745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00327093-7544-45e3-99d0-3db493f0788d None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:59.179745 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 230/467] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:59.179745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00327093-7544-45e3-99d0-3db493f0788d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:59.208390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fcfbf-ffb1-4a20-a49b-bfcc3351b801 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:59.247359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a97a2a-0f8e-45d0-8376-c7d00c21152c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:59.247974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a97a2a-0f8e-45d0-8376-c7d00c21152c None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:59.248250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a97a2a-0f8e-45d0-8376-c7d00c21152c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:59.248548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a97a2a-0f8e-45d0-8376-c7d00c21152c None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:59.249186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a97a2a-0f8e-45d0-8376-c7d00c21152c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:59.250622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a97a2a-0f8e-45d0-8376-c7d00c21152c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:59.274745 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 238/468] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:38:59 2022] POST /identity/v3/users => generated 246 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:38:59.276586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a97a2a-0f8e-45d0-8376-c7d00c21152c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:59.299899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:59.313755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00327093-7544-45e3-99d0-3db493f0788d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:59.342065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:59.368013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89166117-61f7-46a2-9699-9a2b70c8ffca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:59.369098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89166117-61f7-46a2-9699-9a2b70c8ffca None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:59.369520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89166117-61f7-46a2-9699-9a2b70c8ffca None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:59.370455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89166117-61f7-46a2-9699-9a2b70c8ffca None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:59.371415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89166117-61f7-46a2-9699-9a2b70c8ffca None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:38:59.371968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89166117-61f7-46a2-9699-9a2b70c8ffca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:59.373553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89166117-61f7-46a2-9699-9a2b70c8ffca None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:59.391014 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 231/469] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:38:59.393021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1eb7c-4662-4ea9-93fe-72e1eccd46c5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:59.393631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1eb7c-4662-4ea9-93fe-72e1eccd46c5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:59.394213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1eb7c-4662-4ea9-93fe-72e1eccd46c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:59.417319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-afc1eb7c-4662-4ea9-93fe-72e1eccd46c5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:59.418839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-afc1eb7c-4662-4ea9-93fe-72e1eccd46c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:59.433268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a97a2a-0f8e-45d0-8376-c7d00c21152c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:59.483720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753a6ac-f32a-4cce-8407-748a051df4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:59.484697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753a6ac-f32a-4cce-8407-748a051df4d4 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:59.484981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753a6ac-f32a-4cce-8407-748a051df4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:59.485253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753a6ac-f32a-4cce-8407-748a051df4d4 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:59.487349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753a6ac-f32a-4cce-8407-748a051df4d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:59.488531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753a6ac-f32a-4cce-8407-748a051df4d4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:59.497354 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 232/470] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:59.499224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1eb7c-4662-4ea9-93fe-72e1eccd46c5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:59.515683 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 239/471] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:38:59 2022] POST /identity/v3/users => generated 248 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:38:59.608903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:59.623758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1eb7c-4662-4ea9-93fe-72e1eccd46c5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:38:59.667624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44106785-2555-48e4-b2c0-d67945475cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:38:59.668289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44106785-2555-48e4-b2c0-d67945475cbc None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:59.668598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44106785-2555-48e4-b2c0-d67945475cbc None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:59.668865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44106785-2555-48e4-b2c0-d67945475cbc None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:59.669515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44106785-2555-48e4-b2c0-d67945475cbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:38:59.670999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44106785-2555-48e4-b2c0-d67945475cbc None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:38:59.701166 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 233/472] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:38:59 2022] POST /identity/v3/users => generated 248 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:38:59.828169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7f30b-a586-462f-96d3-4db4e8c7e80e None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:59.828558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7f30b-a586-462f-96d3-4db4e8c7e80e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:59.828793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7f30b-a586-462f-96d3-4db4e8c7e80e None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:59.830152 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 240/473] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:38:59.836137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584cb43-c94a-4014-b755-9acca9b52b84 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:38:59.836445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584cb43-c94a-4014-b755-9acca9b52b84 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:38:59.836725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584cb43-c94a-4014-b755-9acca9b52b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:38:59.860876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d584cb43-c94a-4014-b755-9acca9b52b84 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:38:59.862038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d584cb43-c94a-4014-b755-9acca9b52b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:38:59.865133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7f30b-a586-462f-96d3-4db4e8c7e80e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:38:59.942767 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 234/474] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:38:59.990070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:38:59.998718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7f30b-a586-462f-96d3-4db4e8c7e80e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:00.041832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c6a01-d9c9-472a-9faa-60895ccd33c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:00.042560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c6a01-d9c9-472a-9faa-60895ccd33c8 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:00.042873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c6a01-d9c9-472a-9faa-60895ccd33c8 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:00.043083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c6a01-d9c9-472a-9faa-60895ccd33c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:00.054416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c6a01-d9c9-472a-9faa-60895ccd33c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:00.055866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c6a01-d9c9-472a-9faa-60895ccd33c8 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-4d0c6a01-d9c9-472a-9faa-60895ccd33c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:00.071067 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:00.072024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:00.074032 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:00.074032 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:00.074032 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:00.076523 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 241/475] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:00.078674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584cb43-c94a-4014-b755-9acca9b52b84 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:00.188924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:00.231751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584cb43-c94a-4014-b755-9acca9b52b84 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:00.275585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f734d-b7c9-4681-9ff5-0da9e8f4a7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:00.276193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f734d-b7c9-4681-9ff5-0da9e8f4a7f2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:00.276558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f734d-b7c9-4681-9ff5-0da9e8f4a7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:00.277048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f734d-b7c9-4681-9ff5-0da9e8f4a7f2 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:00.277715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f734d-b7c9-4681-9ff5-0da9e8f4a7f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:00.278014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f734d-b7c9-4681-9ff5-0da9e8f4a7f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:00.279396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f734d-b7c9-4681-9ff5-0da9e8f4a7f2 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:00.292072 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 235/476] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:00.292685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c6a01-d9c9-472a-9faa-60895ccd33c8 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:00.305495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f734d-b7c9-4681-9ff5-0da9e8f4a7f2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:00.418231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f734d-b7c9-4681-9ff5-0da9e8f4a7f2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:00.445963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:00.462974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77215b-aa38-417a-9450-158f44c8d4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:00.464912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77215b-aa38-417a-9450-158f44c8d4e4 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:00.466296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77215b-aa38-417a-9450-158f44c8d4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:00.466804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77215b-aa38-417a-9450-158f44c8d4e4 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:00.467746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77215b-aa38-417a-9450-158f44c8d4e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:00.470168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77215b-aa38-417a-9450-158f44c8d4e4 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:00.484085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c6a01-d9c9-472a-9faa-60895ccd33c8 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:00.508907 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 236/477] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:39:00 2022] POST /identity/v3/users => generated 249 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:39:00.544050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927dc4d-3c15-43a5-826f-5a1caf7049f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:00.545121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927dc4d-3c15-43a5-826f-5a1caf7049f0 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:00.545571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927dc4d-3c15-43a5-826f-5a1caf7049f0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:00.546043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927dc4d-3c15-43a5-826f-5a1caf7049f0 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:00.546903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927dc4d-3c15-43a5-826f-5a1caf7049f0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:00.551578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927dc4d-3c15-43a5-826f-5a1caf7049f0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:00.564607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.sql.core [None req-5927dc4d-3c15-43a5-826f-5a1caf7049f0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 20 20:39:00.564607 ncdv-master-1 devstack@keystone.service[104118]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 20 20:39:00.564607 ncdv-master-1 devstack@keystone.service[104118]: [parameters: {'id': 'd9a253f76f02474bbb66387745b7bf27', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 20 20:39:00.564607 ncdv-master-1 devstack@keystone.service[104118]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=104118) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-5927dc4d-3c15-43a5-826f-5a1caf7049f0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result = self._query(query) Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application conn.query(q) Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result.read() Aug 20 20:39:00.586750 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return ref.to_dict() Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 20 20:39:00.587854 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application next(self.gen) Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application yield resource Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application next(self.gen) Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.session.rollback() Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.force_reraise() Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise self.value Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application session.commit() Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 20 20:39:00.589216 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._prepare_impl() Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.session.flush() Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._flush(objects) Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application compat.raise_( Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application flush_context.execute() Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rec.execute(self) Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 20 20:39:00.590473 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 20 20:39:00.591532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result = self._query(query) Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application conn.query(q) Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result.read() Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 20 20:39:00.592579 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [parameters: {'id': 'd9a253f76f02474bbb66387745b7bf27', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:00.593544 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 20 20:39:00.594527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:00.597398 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 242/478] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Aug 20 20:39:00 2022] POST /identity/v3/roles => generated 144 bytes in 306 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:00.605779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77215b-aa38-417a-9450-158f44c8d4e4 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:00.734330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77215b-aa38-417a-9450-158f44c8d4e4 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:00.779672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8144ca-65a4-418a-947a-6f3b2eb68539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:00.780303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8144ca-65a4-418a-947a-6f3b2eb68539 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:00.780634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8144ca-65a4-418a-947a-6f3b2eb68539 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:00.780889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8144ca-65a4-418a-947a-6f3b2eb68539 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:00.793745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8144ca-65a4-418a-947a-6f3b2eb68539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-bd8144ca-65a4-418a-947a-6f3b2eb68539 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:00.800060 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:00.801023 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:00.801828 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:00.801828 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:00.801828 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:00.801828 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:00.801828 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:00.801828 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:00.801828 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:00.801828 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:00.801828 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:00.801828 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:00.801828 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 20 20:39:00.801828 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:00.801828 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 237/479] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:39:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:00.807754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927dc4d-3c15-43a5-826f-5a1caf7049f0 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:00.835501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:00.934416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927dc4d-3c15-43a5-826f-5a1caf7049f0 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:00.976286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c8f51-4091-41b4-ad8d-5a381859e721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:00.976941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c8f51-4091-41b4-ad8d-5a381859e721 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:00.977209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c8f51-4091-41b4-ad8d-5a381859e721 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:00.977471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c8f51-4091-41b4-ad8d-5a381859e721 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:00.978139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c8f51-4091-41b4-ad8d-5a381859e721 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:00.978433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c8f51-4091-41b4-ad8d-5a381859e721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:00.980175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c8f51-4091-41b4-ad8d-5a381859e721 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:00.992332 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 243/480] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:02.035457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5585f-4ed0-4932-9e23-8a6c132dcb5c None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:02.035996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5585f-4ed0-4932-9e23-8a6c132dcb5c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:02.035996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5585f-4ed0-4932-9e23-8a6c132dcb5c None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:02.037640 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 238/481] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:02.043166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337281b-4f5d-42dc-928e-ca2ed517fd9c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:02.043632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337281b-4f5d-42dc-928e-ca2ed517fd9c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:02.043709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337281b-4f5d-42dc-928e-ca2ed517fd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:02.062221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a337281b-4f5d-42dc-928e-ca2ed517fd9c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:02.063413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a337281b-4f5d-42dc-928e-ca2ed517fd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:02.148211 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 244/482] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:02.330351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f816d57-d87b-4b06-91dc-05ff87581a99 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:02.330647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f816d57-d87b-4b06-91dc-05ff87581a99 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:02.330801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f816d57-d87b-4b06-91dc-05ff87581a99 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:02.332428 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 239/483] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:02.337827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5132ae-d87e-45ea-94d1-dd89b1d14561 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:02.338105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5132ae-d87e-45ea-94d1-dd89b1d14561 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:02.338394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5132ae-d87e-45ea-94d1-dd89b1d14561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:02.354590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:02.360690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9a5132ae-d87e-45ea-94d1-dd89b1d14561 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:02.361880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9a5132ae-d87e-45ea-94d1-dd89b1d14561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:02.429384 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 245/484] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:02.545067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f816d57-d87b-4b06-91dc-05ff87581a99 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:02.603012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:02.665083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f816d57-d87b-4b06-91dc-05ff87581a99 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:02.701029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e2bec9-772d-437c-9d25-226abf3845a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:02.701656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2bec9-772d-437c-9d25-226abf3845a6 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:02.701933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2bec9-772d-437c-9d25-226abf3845a6 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:02.702193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2bec9-772d-437c-9d25-226abf3845a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:02.713727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e2bec9-772d-437c-9d25-226abf3845a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-61e2bec9-772d-437c-9d25-226abf3845a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:02.718493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:02.720510 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:02.721546 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:02.721546 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:02.721546 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:02.721546 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:02.721546 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:02.721546 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:02.721546 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:02.721546 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:02.721546 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:02.721546 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:02.721546 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:02.721546 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:02.721546 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 240/485] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:02.725738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5132ae-d87e-45ea-94d1-dd89b1d14561 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:02.799085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c667639-635a-4c05-8b34-0bab92b705e6 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:02.799368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c667639-635a-4c05-8b34-0bab92b705e6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:02.799641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c667639-635a-4c05-8b34-0bab92b705e6 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:02.801142 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 241/486] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:02.813699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c252be8-d1d5-427b-bd39-b58bf045ed9a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:02.813977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c252be8-d1d5-427b-bd39-b58bf045ed9a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:02.814251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c252be8-d1d5-427b-bd39-b58bf045ed9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:02.845483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9c252be8-d1d5-427b-bd39-b58bf045ed9a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:02.846684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9c252be8-d1d5-427b-bd39-b58bf045ed9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:02.858825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5132ae-d87e-45ea-94d1-dd89b1d14561 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:02.916690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047836a-2997-4262-b63b-c2a723106314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:02.917859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047836a-2997-4262-b63b-c2a723106314 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:02.918349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047836a-2997-4262-b63b-c2a723106314 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:02.918817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047836a-2997-4262-b63b-c2a723106314 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:02.919754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047836a-2997-4262-b63b-c2a723106314 None admin] RBAC: Adding query filter params (name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:02.920362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047836a-2997-4262-b63b-c2a723106314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:02.921823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047836a-2997-4262-b63b-c2a723106314 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:02.924192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:02.926420 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 242/487] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 20 20:39:02.927533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c252be8-d1d5-427b-bd39-b58bf045ed9a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:02.938601 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 246/488] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:02.944517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047836a-2997-4262-b63b-c2a723106314 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:02.975332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:03.100052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c252be8-d1d5-427b-bd39-b58bf045ed9a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:03.107582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047836a-2997-4262-b63b-c2a723106314 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:03.163279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6e6ca-9ece-4a11-b3f9-63fc88edd4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:03.163822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6e6ca-9ece-4a11-b3f9-63fc88edd4e2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:03.164094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6e6ca-9ece-4a11-b3f9-63fc88edd4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:03.164357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6e6ca-9ece-4a11-b3f9-63fc88edd4e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:03.166202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25fff4-3d74-434b-ba0e-5de34a7fc6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:03.166833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25fff4-3d74-434b-ba0e-5de34a7fc6c1 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:03.167318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25fff4-3d74-434b-ba0e-5de34a7fc6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:03.167498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25fff4-3d74-434b-ba0e-5de34a7fc6c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:03.185496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6e6ca-9ece-4a11-b3f9-63fc88edd4e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:03.190911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25fff4-3d74-434b-ba0e-5de34a7fc6c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:03.192431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25fff4-3d74-434b-ba0e-5de34a7fc6c1 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-38d6e6ca-9ece-4a11-b3f9-63fc88edd4e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:03.193966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:03.194858 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:03.195641 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:03.195641 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:03.195641 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:03.195641 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:03.195641 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:03.195641 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:03.195641 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:03.195641 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:03.195641 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:03.195641 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:03.195641 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:03.195641 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:03.195641 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 243/489] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:03.204192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6e6ca-9ece-4a11-b3f9-63fc88edd4e2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-0b25fff4-3d74-434b-ba0e-5de34a7fc6c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:03.218712 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:03.219644 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:03.220529 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:03.220529 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:03.220529 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:03.220529 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 247/490] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:03.226599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25fff4-3d74-434b-ba0e-5de34a7fc6c1 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:03.315590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:03.339851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25fff4-3d74-434b-ba0e-5de34a7fc6c1 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:03.371167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:03.381196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6e6ca-9ece-4a11-b3f9-63fc88edd4e2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:03.384507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0de55b-145b-451a-9442-5b3d95876806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:03.385149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0de55b-145b-451a-9442-5b3d95876806 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:03.385423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0de55b-145b-451a-9442-5b3d95876806 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:03.385690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0de55b-145b-451a-9442-5b3d95876806 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:03.386658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0de55b-145b-451a-9442-5b3d95876806 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:03.387059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0de55b-145b-451a-9442-5b3d95876806 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:03.388530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0de55b-145b-451a-9442-5b3d95876806 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:03.403545 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 248/491] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:03.404453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0de55b-145b-451a-9442-5b3d95876806 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:03.420164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2ce1f-2b2f-4bfd-86c1-bf89167bdf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:03.420712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2ce1f-2b2f-4bfd-86c1-bf89167bdf47 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:03.420989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2ce1f-2b2f-4bfd-86c1-bf89167bdf47 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:03.421569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2ce1f-2b2f-4bfd-86c1-bf89167bdf47 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:03.422691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2ce1f-2b2f-4bfd-86c1-bf89167bdf47 None admin] RBAC: Adding query filter params (name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:03.423174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2ce1f-2b2f-4bfd-86c1-bf89167bdf47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:03.425108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2ce1f-2b2f-4bfd-86c1-bf89167bdf47 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:03.435708 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 244/492] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:03.437007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2ce1f-2b2f-4bfd-86c1-bf89167bdf47 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:03.514447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0de55b-145b-451a-9442-5b3d95876806 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:03.526403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:03.542840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2ce1f-2b2f-4bfd-86c1-bf89167bdf47 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:03.556893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e793d5f-9054-42e2-bdae-de0ea4f27568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:03.557523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e793d5f-9054-42e2-bdae-de0ea4f27568 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:03.557850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e793d5f-9054-42e2-bdae-de0ea4f27568 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:03.558111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e793d5f-9054-42e2-bdae-de0ea4f27568 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:03.569771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e793d5f-9054-42e2-bdae-de0ea4f27568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-0e793d5f-9054-42e2-bdae-de0ea4f27568 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:03.574594 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:03.575475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:03.576267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:03.576267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:03.576267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:03.576267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:03.576267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:03.576267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:03.576267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:03.576267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:03.576267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:03.576267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:03.576267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:03.576267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:03.576267 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 249/493] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:03.577606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e793d5f-9054-42e2-bdae-de0ea4f27568 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:03.583639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8b381-784a-445c-aae2-4eb66896cbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:03.584265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8b381-784a-445c-aae2-4eb66896cbe3 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:03.584580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8b381-784a-445c-aae2-4eb66896cbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:03.584841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8b381-784a-445c-aae2-4eb66896cbe3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:03.593687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8b381-784a-445c-aae2-4eb66896cbe3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:03.595072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8b381-784a-445c-aae2-4eb66896cbe3 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-d3a8b381-784a-445c-aae2-4eb66896cbe3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:03.607768 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:03.608978 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:03.610242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:03.610242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:03.610242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:03.610242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:03.610242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:03.610242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:03.610242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:03.610242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:03.610242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:39:03.610242 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:03.610242 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 245/494] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Aug 20 20:39:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:03.610242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db939c-c3dc-4738-8fb1-d59f9a8bdcbc None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:03.610242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db939c-c3dc-4738-8fb1-d59f9a8bdcbc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:03.610982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db939c-c3dc-4738-8fb1-d59f9a8bdcbc None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:03.611920 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 246/495] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:03.616466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db939c-c3dc-4738-8fb1-d59f9a8bdcbc None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:03.633596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:03.687902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e793d5f-9054-42e2-bdae-de0ea4f27568 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:03.723212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44def99-5792-4494-b2a7-d6df5761060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:03.723839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44def99-5792-4494-b2a7-d6df5761060b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:03.724111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44def99-5792-4494-b2a7-d6df5761060b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:03.724211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db939c-c3dc-4738-8fb1-d59f9a8bdcbc None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:03.724440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44def99-5792-4494-b2a7-d6df5761060b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:03.735488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44def99-5792-4494-b2a7-d6df5761060b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:03.736989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44def99-5792-4494-b2a7-d6df5761060b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-b44def99-5792-4494-b2a7-d6df5761060b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:03.751018 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:03.752114 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:03.753007 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:03.753007 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:03.753007 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:03.753007 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 250/496] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:03.753223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44def99-5792-4494-b2a7-d6df5761060b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:03.766049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecef9f-5358-48d9-9d81-37d10ce51ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:03.766710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecef9f-5358-48d9-9d81-37d10ce51ca9 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:03.766985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecef9f-5358-48d9-9d81-37d10ce51ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:03.767243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecef9f-5358-48d9-9d81-37d10ce51ca9 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:03.767932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecef9f-5358-48d9-9d81-37d10ce51ca9 None admin] RBAC: Adding query filter params (name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:03.768224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecef9f-5358-48d9-9d81-37d10ce51ca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:03.769636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecef9f-5358-48d9-9d81-37d10ce51ca9 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:03.778814 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 247/497] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:03.784130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:03.784819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e99f6-3652-4153-985c-8d7fefec6aa9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:03.785094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e99f6-3652-4153-985c-8d7fefec6aa9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:03.785359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e99f6-3652-4153-985c-8d7fefec6aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:03.804059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8d1e99f6-3652-4153-985c-8d7fefec6aa9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:03.805269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8d1e99f6-3652-4153-985c-8d7fefec6aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:03.865282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44def99-5792-4494-b2a7-d6df5761060b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:03.875791 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 248/498] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:03.881419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e99f6-3652-4153-985c-8d7fefec6aa9 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:03.894811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:03.916083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48b5ed-1045-47db-bd0e-8a099a1853fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:03.917456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48b5ed-1045-47db-bd0e-8a099a1853fb None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:03.918172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48b5ed-1045-47db-bd0e-8a099a1853fb None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:03.918599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48b5ed-1045-47db-bd0e-8a099a1853fb None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:03.919824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48b5ed-1045-47db-bd0e-8a099a1853fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:03.920692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48b5ed-1045-47db-bd0e-8a099a1853fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:03.922955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48b5ed-1045-47db-bd0e-8a099a1853fb None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:03.972307 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 251/499] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Aug 20 20:39:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:03.973453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48b5ed-1045-47db-bd0e-8a099a1853fb None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:04.011159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e99f6-3652-4153-985c-8d7fefec6aa9 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:04.051534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5d387-5022-43bd-94f9-b02ccafe11a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:04.052144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5d387-5022-43bd-94f9-b02ccafe11a4 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:04.052445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5d387-5022-43bd-94f9-b02ccafe11a4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:04.052702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5d387-5022-43bd-94f9-b02ccafe11a4 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:04.053333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5d387-5022-43bd-94f9-b02ccafe11a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:04.053650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5d387-5022-43bd-94f9-b02ccafe11a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:04.055016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5d387-5022-43bd-94f9-b02ccafe11a4 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:04.066509 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 249/500] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:04.067303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5d387-5022-43bd-94f9-b02ccafe11a4 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:04.078115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:04.100760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48b5ed-1045-47db-bd0e-8a099a1853fb None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:04.135478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055090e-3762-4a37-a17a-9ffd04bdf728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:04.136184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055090e-3762-4a37-a17a-9ffd04bdf728 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:04.136591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055090e-3762-4a37-a17a-9ffd04bdf728 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:04.136809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055090e-3762-4a37-a17a-9ffd04bdf728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:04.137261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:04.147602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055090e-3762-4a37-a17a-9ffd04bdf728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:04.149211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055090e-3762-4a37-a17a-9ffd04bdf728 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-c055090e-3762-4a37-a17a-9ffd04bdf728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:04.164070 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:04.165042 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:04.165868 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:04.165868 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:04.165868 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:04.165868 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 252/501] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:04.166654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb49e3-2bbe-4723-b99c-d7bb526e6c74 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:04.166654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb49e3-2bbe-4723-b99c-d7bb526e6c74 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:04.167028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb49e3-2bbe-4723-b99c-d7bb526e6c74 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:04.172465 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 253/502] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:04.172465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb49e3-2bbe-4723-b99c-d7bb526e6c74 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:04.184236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5d387-5022-43bd-94f9-b02ccafe11a4 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:04.218286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06ff9e-6a58-4687-a3ba-9f9078b12995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:04.218878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06ff9e-6a58-4687-a3ba-9f9078b12995 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:04.219144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06ff9e-6a58-4687-a3ba-9f9078b12995 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:04.219403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06ff9e-6a58-4687-a3ba-9f9078b12995 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:04.273898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb49e3-2bbe-4723-b99c-d7bb526e6c74 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:04.286849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06ff9e-6a58-4687-a3ba-9f9078b12995 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:04.288316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06ff9e-6a58-4687-a3ba-9f9078b12995 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:04.301838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:04.307239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bad9c-cb4c-4917-b187-815a646983b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:04.307886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bad9c-cb4c-4917-b187-815a646983b7 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:04.308300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bad9c-cb4c-4917-b187-815a646983b7 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:04.308598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bad9c-cb4c-4917-b187-815a646983b7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:04.317333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bad9c-cb4c-4917-b187-815a646983b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:04.318790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bad9c-cb4c-4917-b187-815a646983b7 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-e96bad9c-cb4c-4917-b187-815a646983b7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:04.331699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:04.332695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:04.333612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:04.333612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:04.333612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:04.333612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:04.333612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:04.333612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:04.333612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:04.333612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:04.333612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 20 20:39:04.333612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:04.333612 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 254/503] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:39:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:04.333612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bad9c-cb4c-4917-b187-815a646983b7 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:04.360884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:04.403006 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 250/504] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:39:04.403006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b45a62-55b5-423c-b8d3-accc6ce0115d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:04.403006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b45a62-55b5-423c-b8d3-accc6ce0115d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:04.403006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b45a62-55b5-423c-b8d3-accc6ce0115d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:04.417526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-06b45a62-55b5-423c-b8d3-accc6ce0115d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:04.418714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-06b45a62-55b5-423c-b8d3-accc6ce0115d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:04.447332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bad9c-cb4c-4917-b187-815a646983b7 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:04.481189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcdb86-f7a8-4a74-8652-7848a7aa7e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:04.481702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcdb86-f7a8-4a74-8652-7848a7aa7e65 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:04.481973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcdb86-f7a8-4a74-8652-7848a7aa7e65 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:04.482219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcdb86-f7a8-4a74-8652-7848a7aa7e65 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:04.482839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcdb86-f7a8-4a74-8652-7848a7aa7e65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:04.483147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcdb86-f7a8-4a74-8652-7848a7aa7e65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:04.484578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcdb86-f7a8-4a74-8652-7848a7aa7e65 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:04.487863 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 251/505] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:04.488741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b45a62-55b5-423c-b8d3-accc6ce0115d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:04.502581 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 255/506] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:04.502581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcdb86-f7a8-4a74-8652-7848a7aa7e65 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:04.557312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:04.601701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:04.607002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcdb86-f7a8-4a74-8652-7848a7aa7e65 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:04.607886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b45a62-55b5-423c-b8d3-accc6ce0115d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:04.644430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9068c5-c8eb-4d29-88ba-ac6a1b9e5187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:04.644995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f2250-2c50-492d-8938-1aaf6c07e984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:04.645457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9068c5-c8eb-4d29-88ba-ac6a1b9e5187 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:04.645615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2250-2c50-492d-8938-1aaf6c07e984 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:04.645895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2250-2c50-492d-8938-1aaf6c07e984 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:04.646162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2250-2c50-492d-8938-1aaf6c07e984 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:04.646277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9068c5-c8eb-4d29-88ba-ac6a1b9e5187 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:04.646657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9068c5-c8eb-4d29-88ba-ac6a1b9e5187 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:04.647503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9068c5-c8eb-4d29-88ba-ac6a1b9e5187 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:04.648187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9068c5-c8eb-4d29-88ba-ac6a1b9e5187 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:04.650175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9068c5-c8eb-4d29-88ba-ac6a1b9e5187 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:04.657797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f2250-2c50-492d-8938-1aaf6c07e984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-3a7f2250-2c50-492d-8938-1aaf6c07e984 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:04.662545 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:04.663430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:04.664230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:04.664230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:04.664230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:04.664230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:04.664230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:04.664230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:04.664230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:04.664230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:04.664230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:04.664230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:04.664230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:04.664230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:04.668930 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 252/507] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:04.669150 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 256/508] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Aug 20 20:39:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:04.669638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f2250-2c50-492d-8938-1aaf6c07e984 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:04.669908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9068c5-c8eb-4d29-88ba-ac6a1b9e5187 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:04.773846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f2250-2c50-492d-8938-1aaf6c07e984 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:04.782915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9068c5-c8eb-4d29-88ba-ac6a1b9e5187 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:04.793711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:04.813066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad60c7-1a94-41e9-9640-36131a755387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:04.813697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad60c7-1a94-41e9-9640-36131a755387 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:04.813970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad60c7-1a94-41e9-9640-36131a755387 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:04.814432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad60c7-1a94-41e9-9640-36131a755387 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:04.819265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a85867-d55d-48c9-8fef-b831b80465da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:04.819886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a85867-d55d-48c9-8fef-b831b80465da None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:04.820156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a85867-d55d-48c9-8fef-b831b80465da None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:04.820457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a85867-d55d-48c9-8fef-b831b80465da None admin] PATH_INFO: `/v3/users/cinder` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:04.824831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad60c7-1a94-41e9-9640-36131a755387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:04.826395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad60c7-1a94-41e9-9640-36131a755387 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:04.827709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a85867-d55d-48c9-8fef-b831b80465da None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:04.829152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a85867-d55d-48c9-8fef-b831b80465da None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-d4a85867-d55d-48c9-8fef-b831b80465da None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:04.840711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:04.841606 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:04.842492 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:04.842492 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:04.842492 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:04.842492 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:04.842492 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:04.842492 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:04.842492 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:04.842492 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:04.842492 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 20 20:39:04.842492 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:04.842492 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 257/509] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:39:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-d3ad60c7-1a94-41e9-9640-36131a755387 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:04.843431 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:04.844945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:04.845966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:04.845966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:04.845966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:04.846125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a85867-d55d-48c9-8fef-b831b80465da None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:04.846405 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 253/510] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:04.846405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad60c7-1a94-41e9-9640-36131a755387 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:04.856132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:04.945841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a85867-d55d-48c9-8fef-b831b80465da None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:04.947388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad60c7-1a94-41e9-9640-36131a755387 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:04.978478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc512de0-9f5e-4aa5-8c5f-cad9b332f33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:04.979069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc512de0-9f5e-4aa5-8c5f-cad9b332f33c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:04.979343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc512de0-9f5e-4aa5-8c5f-cad9b332f33c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:04.979657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc512de0-9f5e-4aa5-8c5f-cad9b332f33c None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:04.980297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc512de0-9f5e-4aa5-8c5f-cad9b332f33c None admin] RBAC: Adding query filter params (name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:04.980622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc512de0-9f5e-4aa5-8c5f-cad9b332f33c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:04.980787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bb09f-654c-4632-acb8-e8efc7d287c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:04.981379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bb09f-654c-4632-acb8-e8efc7d287c6 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:04.981647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bb09f-654c-4632-acb8-e8efc7d287c6 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:04.981909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bb09f-654c-4632-acb8-e8efc7d287c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:04.981993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc512de0-9f5e-4aa5-8c5f-cad9b332f33c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:04.990036 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 258/511] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:04.990815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc512de0-9f5e-4aa5-8c5f-cad9b332f33c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:05.021850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:05.051388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bb09f-654c-4632-acb8-e8efc7d287c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:05.052833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bb09f-654c-4632-acb8-e8efc7d287c6 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:05.063316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:05.089754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc512de0-9f5e-4aa5-8c5f-cad9b332f33c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:05.122448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ba616-1b34-4e0b-98a1-db94cf00e7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:05.123066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ba616-1b34-4e0b-98a1-db94cf00e7e5 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:05.123336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ba616-1b34-4e0b-98a1-db94cf00e7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:05.123597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ba616-1b34-4e0b-98a1-db94cf00e7e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:05.133979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ba616-1b34-4e0b-98a1-db94cf00e7e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:05.135456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ba616-1b34-4e0b-98a1-db94cf00e7e5 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-cc8ba616-1b34-4e0b-98a1-db94cf00e7e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:05.149848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:05.151209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:05.152779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:05.152779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:05.152779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:05.152779 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 259/512] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:05.152779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ba616-1b34-4e0b-98a1-db94cf00e7e5 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:05.153074 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 254/513] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:39:05.153624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bb09f-654c-4632-acb8-e8efc7d287c6 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:05.240514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:05.253339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bb09f-654c-4632-acb8-e8efc7d287c6 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:05.258209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ba616-1b34-4e0b-98a1-db94cf00e7e5 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:05.262955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:05.288477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd4750-0cee-40fd-b6de-730ced03125c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:05.289060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd4750-0cee-40fd-b6de-730ced03125c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:05.289326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd4750-0cee-40fd-b6de-730ced03125c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:05.289581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd4750-0cee-40fd-b6de-730ced03125c None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:05.290205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd4750-0cee-40fd-b6de-730ced03125c None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:05.290494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd4750-0cee-40fd-b6de-730ced03125c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:05.291850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd4750-0cee-40fd-b6de-730ced03125c None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:05.292213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a8248-bac8-47a6-92d7-d375f8cc64f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:05.292834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a8248-bac8-47a6-92d7-d375f8cc64f7 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:05.293102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a8248-bac8-47a6-92d7-d375f8cc64f7 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:05.293450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a8248-bac8-47a6-92d7-d375f8cc64f7 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:05.294067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a8248-bac8-47a6-92d7-d375f8cc64f7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:05.294379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a8248-bac8-47a6-92d7-d375f8cc64f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:05.295776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a8248-bac8-47a6-92d7-d375f8cc64f7 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:05.303172 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 255/514] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:05.303919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd4750-0cee-40fd-b6de-730ced03125c None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:05.313391 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 260/515] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Aug 20 20:39:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:05.314196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a8248-bac8-47a6-92d7-d375f8cc64f7 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:05.414959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a8248-bac8-47a6-92d7-d375f8cc64f7 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:05.416233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd4750-0cee-40fd-b6de-730ced03125c None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:05.450431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91908581-3225-4f6e-8938-cd497c4d0da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:05.450639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72f9d4-691e-4386-95d0-b4f97a90d997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:05.451025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91908581-3225-4f6e-8938-cd497c4d0da9 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:05.451152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72f9d4-691e-4386-95d0-b4f97a90d997 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:05.451287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91908581-3225-4f6e-8938-cd497c4d0da9 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:05.451380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72f9d4-691e-4386-95d0-b4f97a90d997 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:05.451544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91908581-3225-4f6e-8938-cd497c4d0da9 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:05.451635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72f9d4-691e-4386-95d0-b4f97a90d997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:05.452158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91908581-3225-4f6e-8938-cd497c4d0da9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:05.452485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91908581-3225-4f6e-8938-cd497c4d0da9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:05.453858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91908581-3225-4f6e-8938-cd497c4d0da9 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:05.462342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72f9d4-691e-4386-95d0-b4f97a90d997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:05.463750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72f9d4-691e-4386-95d0-b4f97a90d997 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:05.464925 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 261/516] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:05.465703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91908581-3225-4f6e-8938-cd497c4d0da9 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:05.468224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-dc72f9d4-691e-4386-95d0-b4f97a90d997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:05.478505 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:05.479764 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:05.480683 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:05.480683 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:05.480683 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:05.480683 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 256/517] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:05.480918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72f9d4-691e-4386-95d0-b4f97a90d997 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:05.505256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:05.568415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91908581-3225-4f6e-8938-cd497c4d0da9 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:05.586898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72f9d4-691e-4386-95d0-b4f97a90d997 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:05.602773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d242a4-2670-4208-a1fb-7ce330fd1fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:05.603396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d242a4-2670-4208-a1fb-7ce330fd1fe9 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:05.603668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d242a4-2670-4208-a1fb-7ce330fd1fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:05.603935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d242a4-2670-4208-a1fb-7ce330fd1fe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:05.614321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d242a4-2670-4208-a1fb-7ce330fd1fe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:05.615779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d242a4-2670-4208-a1fb-7ce330fd1fe9 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:05.619265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b15d1-d2d5-48c0-b85a-a2510194bfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:05.619871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b15d1-d2d5-48c0-b85a-a2510194bfa0 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:05.620176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b15d1-d2d5-48c0-b85a-a2510194bfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:05.620788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b15d1-d2d5-48c0-b85a-a2510194bfa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:05.627187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b15d1-d2d5-48c0-b85a-a2510194bfa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:05.628637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b15d1-d2d5-48c0-b85a-a2510194bfa0 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-e5d242a4-2670-4208-a1fb-7ce330fd1fe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:05.629759 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:05.630655 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:05.631485 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:05.631485 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:05.631485 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:05.631485 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 262/518] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:05.631705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d242a4-2670-4208-a1fb-7ce330fd1fe9 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-c58b15d1-d2d5-48c0-b85a-a2510194bfa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:05.641932 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:05.642891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:05.643725 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:05.643725 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:05.643725 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:05.643725 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:05.643725 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:05.643725 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:05.643725 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:05.643725 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:05.643725 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 257/519] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:05.644358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b15d1-d2d5-48c0-b85a-a2510194bfa0 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:05.694891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:05.726317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:05.739913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d242a4-2670-4208-a1fb-7ce330fd1fe9 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:05.746039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b15d1-d2d5-48c0-b85a-a2510194bfa0 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:05.772616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad33ff0-80e6-462a-ba92-f565fe2b0cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:05.773211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad33ff0-80e6-462a-ba92-f565fe2b0cdf None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:05.773484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad33ff0-80e6-462a-ba92-f565fe2b0cdf None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:05.773741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad33ff0-80e6-462a-ba92-f565fe2b0cdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:05.780231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dabd8-73a7-471c-9f57-2d7143a5f7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:05.780863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dabd8-73a7-471c-9f57-2d7143a5f7ef None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:05.781135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dabd8-73a7-471c-9f57-2d7143a5f7ef None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:05.781401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dabd8-73a7-471c-9f57-2d7143a5f7ef None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:05.782017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1dabd8-73a7-471c-9f57-2d7143a5f7ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:05.783449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1dabd8-73a7-471c-9f57-2d7143a5f7ef None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:05.805521 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 258/520] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:39:05 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:39:05.806342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dabd8-73a7-471c-9f57-2d7143a5f7ef None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:05.850067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad33ff0-80e6-462a-ba92-f565fe2b0cdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:05.851461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad33ff0-80e6-462a-ba92-f565fe2b0cdf None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:05.914676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:05.921328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dabd8-73a7-471c-9f57-2d7143a5f7ef None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:05.960252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fe9d1-e1f9-4105-87fe-2c0644df1227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:05.960971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fe9d1-e1f9-4105-87fe-2c0644df1227 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:05.961288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fe9d1-e1f9-4105-87fe-2c0644df1227 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:05.961515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fe9d1-e1f9-4105-87fe-2c0644df1227 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:05.962181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fe9d1-e1f9-4105-87fe-2c0644df1227 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:05.962473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fe9d1-e1f9-4105-87fe-2c0644df1227 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:05.963967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fe9d1-e1f9-4105-87fe-2c0644df1227 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:05.970602 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 263/521] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:39:05.970602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad33ff0-80e6-462a-ba92-f565fe2b0cdf None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:05.973723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:05.987182 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 259/522] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:05.988051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fe9d1-e1f9-4105-87fe-2c0644df1227 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:06.085372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad33ff0-80e6-462a-ba92-f565fe2b0cdf None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:06.101577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fe9d1-e1f9-4105-87fe-2c0644df1227 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:06.125191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2495a-e4a5-44c9-bfcc-37748dd297c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:06.125890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2495a-e4a5-44c9-bfcc-37748dd297c4 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:06.126151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2495a-e4a5-44c9-bfcc-37748dd297c4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:06.126410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2495a-e4a5-44c9-bfcc-37748dd297c4 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:06.127040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2495a-e4a5-44c9-bfcc-37748dd297c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:06.127332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2495a-e4a5-44c9-bfcc-37748dd297c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:06.128943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2495a-e4a5-44c9-bfcc-37748dd297c4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:06.136559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a0702-6650-4f06-a643-de051af3b038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:06.137164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0702-6650-4f06-a643-de051af3b038 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:06.137436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0702-6650-4f06-a643-de051af3b038 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:06.137704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0702-6650-4f06-a643-de051af3b038 None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:06.138388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a0702-6650-4f06-a643-de051af3b038 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:06.138689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a0702-6650-4f06-a643-de051af3b038 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:06.140104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a0702-6650-4f06-a643-de051af3b038 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:06.145807 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 264/523] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:06.145807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2495a-e4a5-44c9-bfcc-37748dd297c4 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:06.158592 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 260/524] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:06.164076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a0702-6650-4f06-a643-de051af3b038 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:06.164656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:06.229463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:06.255848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2495a-e4a5-44c9-bfcc-37748dd297c4 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:06.275307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a0702-6650-4f06-a643-de051af3b038 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:06.289958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cccf5-3ecb-414d-bcd0-085c1de68c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:06.290716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cccf5-3ecb-414d-bcd0-085c1de68c93 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:06.291077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cccf5-3ecb-414d-bcd0-085c1de68c93 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:06.291409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cccf5-3ecb-414d-bcd0-085c1de68c93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:06.303010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670cccf5-3ecb-414d-bcd0-085c1de68c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:06.304483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670cccf5-3ecb-414d-bcd0-085c1de68c93 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:06.313664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a76295-b6ff-444e-8470-04eb0857aa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:06.314482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a76295-b6ff-444e-8470-04eb0857aa49 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:06.314911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a76295-b6ff-444e-8470-04eb0857aa49 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:06.315315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a76295-b6ff-444e-8470-04eb0857aa49 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-670cccf5-3ecb-414d-bcd0-085c1de68c93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:06.318915 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:06.320024 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:06.321025 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:06.321025 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:06.321025 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:06.321025 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 265/525] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:06.321025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cccf5-3ecb-414d-bcd0-085c1de68c93 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:06.326182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a76295-b6ff-444e-8470-04eb0857aa49 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:06.327849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a76295-b6ff-444e-8470-04eb0857aa49 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-93a76295-b6ff-444e-8470-04eb0857aa49 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:06.340804 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:06.341876 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:06.342965 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:06.342965 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:06.342965 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:06.342965 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:06.342965 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:06.342965 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:06.342965 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:06.342965 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:06.342965 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 20 20:39:06.342965 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:06.344794 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 261/526] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:06.345685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a76295-b6ff-444e-8470-04eb0857aa49 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:06.398249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:06.440366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cccf5-3ecb-414d-bcd0-085c1de68c93 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:06.467253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a76295-b6ff-444e-8470-04eb0857aa49 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:06.477231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08e9c4-21d7-455d-aec6-517c5b4cab2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:06.477863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08e9c4-21d7-455d-aec6-517c5b4cab2f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:06.478130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08e9c4-21d7-455d-aec6-517c5b4cab2f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:06.478389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08e9c4-21d7-455d-aec6-517c5b4cab2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:06.488448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:06.488876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08e9c4-21d7-455d-aec6-517c5b4cab2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:06.490329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08e9c4-21d7-455d-aec6-517c5b4cab2f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-9b08e9c4-21d7-455d-aec6-517c5b4cab2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:06.502894 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:06.503815 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:06.504702 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:06.504702 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:06.504702 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:06.504702 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:06.504702 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:06.504702 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:06.504702 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:06.504702 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:06.504702 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 266/527] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:06.505491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08e9c4-21d7-455d-aec6-517c5b4cab2f None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:06.513938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083bb4d-5584-4f8a-844b-dc6948e585f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:06.514858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083bb4d-5584-4f8a-844b-dc6948e585f4 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:06.514977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083bb4d-5584-4f8a-844b-dc6948e585f4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:06.515290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083bb4d-5584-4f8a-844b-dc6948e585f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:06.516073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083bb4d-5584-4f8a-844b-dc6948e585f4 None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=df852af59e7743f7a8798d9edf8ebd3e) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:06.516464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083bb4d-5584-4f8a-844b-dc6948e585f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:06.518069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083bb4d-5584-4f8a-844b-dc6948e585f4 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:06.524174 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 262/528] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:06 2022] GET /identity/v3/role_assignments?user.id=df852af59e7743f7a8798d9edf8ebd3e&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:06.525228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083bb4d-5584-4f8a-844b-dc6948e585f4 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:06.645674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08e9c4-21d7-455d-aec6-517c5b4cab2f None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:06.646287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083bb4d-5584-4f8a-844b-dc6948e585f4 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:06.682610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:06.687499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791eea9-5edf-4d68-b28f-2e85a8a67ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:06.688647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791eea9-5edf-4d68-b28f-2e85a8a67ade None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:06.689125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791eea9-5edf-4d68-b28f-2e85a8a67ade None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:06.689531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791eea9-5edf-4d68-b28f-2e85a8a67ade None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:06.690405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791eea9-5edf-4d68-b28f-2e85a8a67ade None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:06.690837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791eea9-5edf-4d68-b28f-2e85a8a67ade None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:06.692541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791eea9-5edf-4d68-b28f-2e85a8a67ade None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:06.693581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e451f-4cb1-404c-9843-834f6860820c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:06.694276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e451f-4cb1-404c-9843-834f6860820c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:06.694616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e451f-4cb1-404c-9843-834f6860820c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:06.694949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e451f-4cb1-404c-9843-834f6860820c None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:06.696014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e451f-4cb1-404c-9843-834f6860820c None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:06.696410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e451f-4cb1-404c-9843-834f6860820c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:06.697905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e451f-4cb1-404c-9843-834f6860820c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:06.717935 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 267/529] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:06.717935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e451f-4cb1-404c-9843-834f6860820c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:06.739262 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 263/530] 10.222.0.23 () {56 vars in 1139 bytes} [Sat Aug 20 20:39:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:06.740505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791eea9-5edf-4d68-b28f-2e85a8a67ade None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:06.788215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:06.850652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e451f-4cb1-404c-9843-834f6860820c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:06.873105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791eea9-5edf-4d68-b28f-2e85a8a67ade None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:06.898069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599e15e-ffd7-43ea-80d6-32b3a856d56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:06.899043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599e15e-ffd7-43ea-80d6-32b3a856d56d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:06.899476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599e15e-ffd7-43ea-80d6-32b3a856d56d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:06.899893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599e15e-ffd7-43ea-80d6-32b3a856d56d None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:06.901193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599e15e-ffd7-43ea-80d6-32b3a856d56d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:06.901539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599e15e-ffd7-43ea-80d6-32b3a856d56d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:06.903493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599e15e-ffd7-43ea-80d6-32b3a856d56d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:06.919483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d7883-acfc-4b15-86cc-5bec19d62f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:06.920425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d7883-acfc-4b15-86cc-5bec19d62f4a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:06.920718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d7883-acfc-4b15-86cc-5bec19d62f4a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:06.920981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d7883-acfc-4b15-86cc-5bec19d62f4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:06.922911 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 268/531] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:06.923969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599e15e-ffd7-43ea-80d6-32b3a856d56d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:06.934732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d7883-acfc-4b15-86cc-5bec19d62f4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:06.936543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d7883-acfc-4b15-86cc-5bec19d62f4a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-277d7883-acfc-4b15-86cc-5bec19d62f4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:06.950208 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:06.951176 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:06.952047 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:06.952047 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:06.952047 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:06.952047 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:06.952047 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:06.952047 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:06.952047 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:06.952047 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:06.956675 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 264/532] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:06.957756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d7883-acfc-4b15-86cc-5bec19d62f4a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:06.975814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:07.030151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599e15e-ffd7-43ea-80d6-32b3a856d56d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:07.064925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f50b69-304d-4424-822a-8891af1c50bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:07.065375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f50b69-304d-4424-822a-8891af1c50bb None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:07.065645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f50b69-304d-4424-822a-8891af1c50bb None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:07.065906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f50b69-304d-4424-822a-8891af1c50bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:07.067728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:07.072384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d7883-acfc-4b15-86cc-5bec19d62f4a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:07.111541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f471b-29cd-4d58-a433-3d8f16bb99ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:07.112429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f471b-29cd-4d58-a433-3d8f16bb99ea None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:07.113083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f471b-29cd-4d58-a433-3d8f16bb99ea None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:07.113941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f471b-29cd-4d58-a433-3d8f16bb99ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:07.115066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f471b-29cd-4d58-a433-3d8f16bb99ea None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=698d5a7fc34a46ba909152414a0860d7) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:07.115511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f471b-29cd-4d58-a433-3d8f16bb99ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:07.117802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f471b-29cd-4d58-a433-3d8f16bb99ea None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:07.125285 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 265/533] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:06 2022] GET /identity/v3/role_assignments?user.id=698d5a7fc34a46ba909152414a0860d7&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:07.126841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f471b-29cd-4d58-a433-3d8f16bb99ea None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:07.145285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f50b69-304d-4424-822a-8891af1c50bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:07.146800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f50b69-304d-4424-822a-8891af1c50bb None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:07.262977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:07.285048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f471b-29cd-4d58-a433-3d8f16bb99ea None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:07.333218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1919e-c73f-4db3-a281-9b284bbae303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:07.333836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1919e-c73f-4db3-a281-9b284bbae303 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:07.334113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1919e-c73f-4db3-a281-9b284bbae303 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:07.334364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1919e-c73f-4db3-a281-9b284bbae303 None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:07.335052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1919e-c73f-4db3-a281-9b284bbae303 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:07.335341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1919e-c73f-4db3-a281-9b284bbae303 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:07.338278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1919e-c73f-4db3-a281-9b284bbae303 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:07.358769 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 266/534] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:07.366787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1919e-c73f-4db3-a281-9b284bbae303 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:07.379107 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 269/535] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:39:07.384238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f50b69-304d-4424-822a-8891af1c50bb None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:07.416399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:07.492067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f50b69-304d-4424-822a-8891af1c50bb None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:07.529212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f3c07-89ce-489b-b832-92b1dcdb8cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:07.530199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f3c07-89ce-489b-b832-92b1dcdb8cd6 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:07.533923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f3c07-89ce-489b-b832-92b1dcdb8cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:07.534557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f3c07-89ce-489b-b832-92b1dcdb8cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:07.550809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f3c07-89ce-489b-b832-92b1dcdb8cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:07.553472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f3c07-89ce-489b-b832-92b1dcdb8cd6 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:07.570630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-1f8f3c07-89ce-489b-b832-92b1dcdb8cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:07.578710 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:07.579646 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:07.580520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:07.580520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:07.580520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:07.581238 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 270/536] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:07.589998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1919e-c73f-4db3-a281-9b284bbae303 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:07.592397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f3c07-89ce-489b-b832-92b1dcdb8cd6 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:07.659674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa3f04-5adb-41da-a7c5-62e30c89053c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:07.660278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa3f04-5adb-41da-a7c5-62e30c89053c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:07.660583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa3f04-5adb-41da-a7c5-62e30c89053c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:07.660852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa3f04-5adb-41da-a7c5-62e30c89053c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:07.661580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa3f04-5adb-41da-a7c5-62e30c89053c None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=53928836828d45fbb427dd92492cae20) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:07.661877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa3f04-5adb-41da-a7c5-62e30c89053c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:07.663396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa3f04-5adb-41da-a7c5-62e30c89053c None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:07.669414 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 267/537] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:07 2022] GET /identity/v3/role_assignments?user.id=53928836828d45fbb427dd92492cae20&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:07.706782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f3c07-89ce-489b-b832-92b1dcdb8cd6 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:07.743908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd495b75-1d31-4ef0-83ba-6445eeade6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:07.744551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd495b75-1d31-4ef0-83ba-6445eeade6bf None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:07.744825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd495b75-1d31-4ef0-83ba-6445eeade6bf None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:07.745152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd495b75-1d31-4ef0-83ba-6445eeade6bf None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:07.745826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd495b75-1d31-4ef0-83ba-6445eeade6bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:07.746122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd495b75-1d31-4ef0-83ba-6445eeade6bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:07.747737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd495b75-1d31-4ef0-83ba-6445eeade6bf None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:07.761489 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 271/538] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:07.767643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa3f04-5adb-41da-a7c5-62e30c89053c None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:07.873226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:07.890215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa3f04-5adb-41da-a7c5-62e30c89053c None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:07.928306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b4056-0bef-493f-9622-46ba5b1bb5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:07.929334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4056-0bef-493f-9622-46ba5b1bb5e2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:07.929807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4056-0bef-493f-9622-46ba5b1bb5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:07.930303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4056-0bef-493f-9622-46ba5b1bb5e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:07.939247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b4056-0bef-493f-9622-46ba5b1bb5e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:07.941300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b4056-0bef-493f-9622-46ba5b1bb5e2 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-873b4056-0bef-493f-9622-46ba5b1bb5e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:07.955245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:07.956182 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:07.957069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:07.957069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:07.957069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:07.957069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:07.957069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:07.957069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:07.957069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:07.957069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:07.964555 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 268/539] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:07.966711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd495b75-1d31-4ef0-83ba-6445eeade6bf None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:08.029695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:08.074021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd495b75-1d31-4ef0-83ba-6445eeade6bf None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:08.110477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669561f-cfe1-4e83-b93c-939f5ca1b9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:08.111092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669561f-cfe1-4e83-b93c-939f5ca1b9ed None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:08.111365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669561f-cfe1-4e83-b93c-939f5ca1b9ed None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:08.111634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669561f-cfe1-4e83-b93c-939f5ca1b9ed None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:08.112319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669561f-cfe1-4e83-b93c-939f5ca1b9ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:08.112661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669561f-cfe1-4e83-b93c-939f5ca1b9ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:08.114095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669561f-cfe1-4e83-b93c-939f5ca1b9ed None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:08.128603 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 272/540] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:08.132619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b4056-0bef-493f-9622-46ba5b1bb5e2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:08.145372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee567f8a-32ab-4a2d-a401-c2e45643f58c None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:08.145942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee567f8a-32ab-4a2d-a401-c2e45643f58c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:08.146447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee567f8a-32ab-4a2d-a401-c2e45643f58c None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:08.148211 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 273/541] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:08.154443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0babb4a-bebf-4417-b1dd-f7ff1b473cd9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:08.155098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0babb4a-bebf-4417-b1dd-f7ff1b473cd9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:08.155570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0babb4a-bebf-4417-b1dd-f7ff1b473cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:08.175733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a0babb4a-bebf-4417-b1dd-f7ff1b473cd9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:08.177500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a0babb4a-bebf-4417-b1dd-f7ff1b473cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:08.286414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b4056-0bef-493f-9622-46ba5b1bb5e2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:08.302077 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 274/542] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:08.325046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2c207-f35b-4337-ba48-db2dbd41ba47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:08.325651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2c207-f35b-4337-ba48-db2dbd41ba47 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:08.325924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2c207-f35b-4337-ba48-db2dbd41ba47 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:08.326188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2c207-f35b-4337-ba48-db2dbd41ba47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:08.326914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2c207-f35b-4337-ba48-db2dbd41ba47 None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=22bb014e04774c278c99496bf64b2319) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:08.327213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2c207-f35b-4337-ba48-db2dbd41ba47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:08.328830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2c207-f35b-4337-ba48-db2dbd41ba47 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:08.340545 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 269/543] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:08 2022] GET /identity/v3/role_assignments?user.id=22bb014e04774c278c99496bf64b2319&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:08.808721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0babb4a-bebf-4417-b1dd-f7ff1b473cd9 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:08.863679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:08.928000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0babb4a-bebf-4417-b1dd-f7ff1b473cd9 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:08.961374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f05b2-420e-4c7e-838d-8a5ea54184a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:08.961998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f05b2-420e-4c7e-838d-8a5ea54184a9 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:08.962265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f05b2-420e-4c7e-838d-8a5ea54184a9 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:08.962523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f05b2-420e-4c7e-838d-8a5ea54184a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:08.972587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f05b2-420e-4c7e-838d-8a5ea54184a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-af7f05b2-420e-4c7e-838d-8a5ea54184a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:08.977224 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:08.978987 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:08.980082 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:08.980082 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:08.980082 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:08.980082 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:08.980082 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:08.980082 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:08.980082 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:08.980082 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:08.980082 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:08.980082 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:08.980082 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:08.980082 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:08.980082 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 275/544] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:08.984386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2c207-f35b-4337-ba48-db2dbd41ba47 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:09.006257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:09.128762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2c207-f35b-4337-ba48-db2dbd41ba47 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:09.169428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93ecaa-88e2-439d-a815-9589c9ca4643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:09.170039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93ecaa-88e2-439d-a815-9589c9ca4643 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:09.170368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93ecaa-88e2-439d-a815-9589c9ca4643 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:09.170640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93ecaa-88e2-439d-a815-9589c9ca4643 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:09.171303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93ecaa-88e2-439d-a815-9589c9ca4643 None admin] RBAC: Adding query filter params (name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:09.171604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93ecaa-88e2-439d-a815-9589c9ca4643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:09.173015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93ecaa-88e2-439d-a815-9589c9ca4643 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:09.182026 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 270/545] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:09.187905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f05b2-420e-4c7e-838d-8a5ea54184a9 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:09.359988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f05b2-420e-4c7e-838d-8a5ea54184a9 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:09.372894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:09.412103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a056382-d0ce-44c0-b462-0efaeb1c9eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:09.414943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a056382-d0ce-44c0-b462-0efaeb1c9eee None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:09.415229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a056382-d0ce-44c0-b462-0efaeb1c9eee None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:09.415501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a056382-d0ce-44c0-b462-0efaeb1c9eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:09.436437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a056382-d0ce-44c0-b462-0efaeb1c9eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:09.437911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a056382-d0ce-44c0-b462-0efaeb1c9eee None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-0a056382-d0ce-44c0-b462-0efaeb1c9eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:09.455601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:09.456552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:09.457395 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:09.457395 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:09.457395 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:09.457395 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 276/546] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:09.462867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93ecaa-88e2-439d-a815-9589c9ca4643 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:09.471108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b911089-b921-432b-91d6-ac9e723fc808 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:09.471648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b911089-b921-432b-91d6-ac9e723fc808 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:09.472111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b911089-b921-432b-91d6-ac9e723fc808 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:09.473966 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 277/547] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:09.479209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa894b-9bae-4a9a-a3dd-133ff581484b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:09.479905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa894b-9bae-4a9a-a3dd-133ff581484b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:09.480437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa894b-9bae-4a9a-a3dd-133ff581484b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:09.509958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c8aa894b-9bae-4a9a-a3dd-133ff581484b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:09.511848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c8aa894b-9bae-4a9a-a3dd-133ff581484b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:09.538647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:09.596637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93ecaa-88e2-439d-a815-9589c9ca4643 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:09.655897 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 278/548] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:39:09.666282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e1909-162b-44f4-a391-6c6adb935bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:09.667388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e1909-162b-44f4-a391-6c6adb935bf7 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:09.667855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e1909-162b-44f4-a391-6c6adb935bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:09.668333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e1909-162b-44f4-a391-6c6adb935bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:09.669174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e1909-162b-44f4-a391-6c6adb935bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:09.669653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e1909-162b-44f4-a391-6c6adb935bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:09.671203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e1909-162b-44f4-a391-6c6adb935bf7 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:09.689062 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 271/549] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:09.697481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa894b-9bae-4a9a-a3dd-133ff581484b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:09.818664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:09.831293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa894b-9bae-4a9a-a3dd-133ff581484b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:09.870921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0bffb-d571-4c7b-a1a6-e615c1146f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:09.871900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0bffb-d571-4c7b-a1a6-e615c1146f7b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:09.872411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0bffb-d571-4c7b-a1a6-e615c1146f7b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:09.872864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0bffb-d571-4c7b-a1a6-e615c1146f7b None admin] PATH_INFO: `/v3/users/swift` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:09.884236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0bffb-d571-4c7b-a1a6-e615c1146f7b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:09.889030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0bffb-d571-4c7b-a1a6-e615c1146f7b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-eba0bffb-d571-4c7b-a1a6-e615c1146f7b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:09.908963 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:09.911121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:09.917828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:09.917828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:09.917828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:09.917828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:09.917828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:09.917828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:09.917828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:09.917828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:09.917828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 20 20:39:09.917828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:09.920511 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 279/550] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:39:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:09.926380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e1909-162b-44f4-a391-6c6adb935bf7 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:10.002500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e15da-6f11-4980-9a6b-39ac73dfa94c None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:10.003156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e15da-6f11-4980-9a6b-39ac73dfa94c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:10.003620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e15da-6f11-4980-9a6b-39ac73dfa94c None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:10.005513 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 280/551] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:10.013264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474903d-4e23-4265-99d0-366cc80ec528 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:10.015321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474903d-4e23-4265-99d0-366cc80ec528 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:10.015826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474903d-4e23-4265-99d0-366cc80ec528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:10.048105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4474903d-4e23-4265-99d0-366cc80ec528 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:10.055152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4474903d-4e23-4265-99d0-366cc80ec528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:10.112394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e1909-162b-44f4-a391-6c6adb935bf7 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:10.150091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:10.156870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950e4dd-bdd1-4946-8d94-0def7f8b93cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:10.158098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950e4dd-bdd1-4946-8d94-0def7f8b93cf None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:10.158355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950e4dd-bdd1-4946-8d94-0def7f8b93cf None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:10.158626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950e4dd-bdd1-4946-8d94-0def7f8b93cf None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:10.159840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950e4dd-bdd1-4946-8d94-0def7f8b93cf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:10.160029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950e4dd-bdd1-4946-8d94-0def7f8b93cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:10.161815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950e4dd-bdd1-4946-8d94-0def7f8b93cf None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:10.185646 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 272/552] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Aug 20 20:39:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:10.186587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950e4dd-bdd1-4946-8d94-0def7f8b93cf None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:10.187633 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 281/553] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:39:10.193694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474903d-4e23-4265-99d0-366cc80ec528 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:10.305106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950e4dd-bdd1-4946-8d94-0def7f8b93cf None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:10.343774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c851f-2b88-43e3-8c52-9028c9f9ca9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:10.344476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c851f-2b88-43e3-8c52-9028c9f9ca9d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:10.344799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c851f-2b88-43e3-8c52-9028c9f9ca9d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:10.345072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c851f-2b88-43e3-8c52-9028c9f9ca9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:10.359656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c851f-2b88-43e3-8c52-9028c9f9ca9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-bd0c851f-2b88-43e3-8c52-9028c9f9ca9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:10.368324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:10.369393 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:10.370567 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:10.370567 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:10.370567 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:10.370567 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:10.370567 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:10.370567 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:10.370567 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:10.370567 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:10.370567 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:10.370567 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:10.370567 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:10.370567 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:10.370567 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 273/554] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:10.380834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:10.381434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c851f-2b88-43e3-8c52-9028c9f9ca9d None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:10.399985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474903d-4e23-4265-99d0-366cc80ec528 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:10.439754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d68a4-656e-407c-b95a-5a9d9282a77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:10.440649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d68a4-656e-407c-b95a-5a9d9282a77f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:10.441091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d68a4-656e-407c-b95a-5a9d9282a77f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:10.441499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d68a4-656e-407c-b95a-5a9d9282a77f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:10.459169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:10.504284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c851f-2b88-43e3-8c52-9028c9f9ca9d None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:10.531787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d68a4-656e-407c-b95a-5a9d9282a77f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:10.533947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d68a4-656e-407c-b95a-5a9d9282a77f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:10.548395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd752e14-ad77-4811-a868-d9754467ae12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:10.549022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd752e14-ad77-4811-a868-d9754467ae12 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:10.549298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd752e14-ad77-4811-a868-d9754467ae12 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:10.549561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd752e14-ad77-4811-a868-d9754467ae12 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:10.550238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd752e14-ad77-4811-a868-d9754467ae12 None admin] RBAC: Adding query filter params (name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:10.550535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd752e14-ad77-4811-a868-d9754467ae12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:10.551927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd752e14-ad77-4811-a868-d9754467ae12 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:10.566757 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 274/555] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:10.568346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd752e14-ad77-4811-a868-d9754467ae12 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:10.690951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd752e14-ad77-4811-a868-d9754467ae12 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:10.713557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:10.719334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:10.731060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e7f40-8e6e-485a-86c2-6728099ebbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:10.732196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e7f40-8e6e-485a-86c2-6728099ebbbb None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:10.732836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e7f40-8e6e-485a-86c2-6728099ebbbb None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:10.733381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e7f40-8e6e-485a-86c2-6728099ebbbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:10.750650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e7f40-8e6e-485a-86c2-6728099ebbbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:10.753254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e7f40-8e6e-485a-86c2-6728099ebbbb None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:10.759524 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 282/556] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:39:10.760602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d68a4-656e-407c-b95a-5a9d9282a77f None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-7c3e7f40-8e6e-485a-86c2-6728099ebbbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:10.782373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:10.783460 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:10.784319 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:10.784319 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:10.784319 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:10.784319 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 275/557] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:10.788488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e7f40-8e6e-485a-86c2-6728099ebbbb None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:10.888872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d68a4-656e-407c-b95a-5a9d9282a77f None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:10.909264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e7f40-8e6e-485a-86c2-6728099ebbbb None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:10.926282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671a621-432a-4693-b284-b2a32f31d88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:10.926890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671a621-432a-4693-b284-b2a32f31d88e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:10.927155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671a621-432a-4693-b284-b2a32f31d88e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:10.927416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671a621-432a-4693-b284-b2a32f31d88e None admin] PATH_INFO: `/v3/roles/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:10.938421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671a621-432a-4693-b284-b2a32f31d88e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-7671a621-432a-4693-b284-b2a32f31d88e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:10.943219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:10.946804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 283/558] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:10.947671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671a621-432a-4693-b284-b2a32f31d88e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:10.949101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f906058-c781-4d51-8e54-fbf6af940a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:10.949322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f906058-c781-4d51-8e54-fbf6af940a0d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:10.949595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f906058-c781-4d51-8e54-fbf6af940a0d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:10.949857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f906058-c781-4d51-8e54-fbf6af940a0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:10.961053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f906058-c781-4d51-8e54-fbf6af940a0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:10.965502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f906058-c781-4d51-8e54-fbf6af940a0d None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-3f906058-c781-4d51-8e54-fbf6af940a0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:10.983198 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:10.984306 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:10.985324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:10.985324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:10.985324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:10.990187 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 276/559] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:10.991355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c14fc-75f1-4d92-ad18-5d7107bfef37 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:10.991634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c14fc-75f1-4d92-ad18-5d7107bfef37 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:10.992063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c14fc-75f1-4d92-ad18-5d7107bfef37 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:10.993405 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 277/560] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:10.994416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a34570-6239-4603-8efd-b569ea0c8801 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:10.994698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a34570-6239-4603-8efd-b569ea0c8801 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:10.995085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a34570-6239-4603-8efd-b569ea0c8801 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:10.996256 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 278/561] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:10.996991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a34570-6239-4603-8efd-b569ea0c8801 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:11.007086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:11.028290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:11.082648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671a621-432a-4693-b284-b2a32f31d88e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:11.116178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d805ed9-bbfc-4ad1-9bf7-32c2fe7c234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:11.116806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d805ed9-bbfc-4ad1-9bf7-32c2fe7c234d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:11.117080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d805ed9-bbfc-4ad1-9bf7-32c2fe7c234d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:11.117347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d805ed9-bbfc-4ad1-9bf7-32c2fe7c234d None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:11.117975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d805ed9-bbfc-4ad1-9bf7-32c2fe7c234d None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:11.118270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d805ed9-bbfc-4ad1-9bf7-32c2fe7c234d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:11.119267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a34570-6239-4603-8efd-b569ea0c8801 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:11.119652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d805ed9-bbfc-4ad1-9bf7-32c2fe7c234d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:11.131639 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 284/562] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:11.132253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d805ed9-bbfc-4ad1-9bf7-32c2fe7c234d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:11.165035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72762f60-fe00-4aec-ac72-2e19e7d5c714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:11.165653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72762f60-fe00-4aec-ac72-2e19e7d5c714 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:11.166003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72762f60-fe00-4aec-ac72-2e19e7d5c714 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:11.166275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72762f60-fe00-4aec-ac72-2e19e7d5c714 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:11.167054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72762f60-fe00-4aec-ac72-2e19e7d5c714 None admin] RBAC: Adding query filter params (name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:11.167345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72762f60-fe00-4aec-ac72-2e19e7d5c714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:11.168759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72762f60-fe00-4aec-ac72-2e19e7d5c714 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:11.177137 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 279/563] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:11.178283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa11099-c89c-4ae7-ade1-37a5cb16c040 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:11.178557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa11099-c89c-4ae7-ade1-37a5cb16c040 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:11.178820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa11099-c89c-4ae7-ade1-37a5cb16c040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:11.196249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bfa11099-c89c-4ae7-ade1-37a5cb16c040 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:11.197439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bfa11099-c89c-4ae7-ade1-37a5cb16c040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:11.238036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:11.241058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d805ed9-bbfc-4ad1-9bf7-32c2fe7c234d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:11.269539 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 280/564] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:11.270791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a543e-300c-47ac-bfd2-b200c22772e0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:11.271070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a543e-300c-47ac-bfd2-b200c22772e0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:11.271337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a543e-300c-47ac-bfd2-b200c22772e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:11.288672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c999a-8560-453b-b320-670e0c916e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:11.288867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-924a543e-300c-47ac-bfd2-b200c22772e0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:11.289374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c999a-8560-453b-b320-670e0c916e56 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:11.289721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c999a-8560-453b-b320-670e0c916e56 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:11.289896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-924a543e-300c-47ac-bfd2-b200c22772e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:11.290112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c999a-8560-453b-b320-670e0c916e56 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:11.290815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c999a-8560-453b-b320-670e0c916e56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:11.291178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c999a-8560-453b-b320-670e0c916e56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:11.292041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:11.292765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c999a-8560-453b-b320-670e0c916e56 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:11.309064 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 285/565] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:11.310298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c999a-8560-453b-b320-670e0c916e56 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:11.370630 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 281/566] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:11.371594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a543e-300c-47ac-bfd2-b200c22772e0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:11.428653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c999a-8560-453b-b320-670e0c916e56 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:11.465389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bc7f5-7ab5-4bd4-b272-6bf6cf9b8eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:11.466039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bc7f5-7ab5-4bd4-b272-6bf6cf9b8eff None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:11.466372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bc7f5-7ab5-4bd4-b272-6bf6cf9b8eff None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:11.466701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bc7f5-7ab5-4bd4-b272-6bf6cf9b8eff None admin] PATH_INFO: `/v3/users/nova` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:11.475215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bc7f5-7ab5-4bd4-b272-6bf6cf9b8eff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:11.476820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bc7f5-7ab5-4bd4-b272-6bf6cf9b8eff None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-311bc7f5-7ab5-4bd4-b272-6bf6cf9b8eff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:11.492122 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:11.493451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:11.494662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:11.494662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:11.494662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:11.494662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:11.494662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:11.494662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:11.494662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:11.494662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:11.494662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:39:11.494662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:11.500742 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 286/567] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Aug 20 20:39:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:11.501898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bc7f5-7ab5-4bd4-b272-6bf6cf9b8eff None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:11.514607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a543e-300c-47ac-bfd2-b200c22772e0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:11.527955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:11.559707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3639b00-f7b8-4031-9260-50e264ce3f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:11.560895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3639b00-f7b8-4031-9260-50e264ce3f6b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:11.561329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3639b00-f7b8-4031-9260-50e264ce3f6b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:11.561892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3639b00-f7b8-4031-9260-50e264ce3f6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:11.571179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:11.580683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3639b00-f7b8-4031-9260-50e264ce3f6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:11.582373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3639b00-f7b8-4031-9260-50e264ce3f6b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-e3639b00-f7b8-4031-9260-50e264ce3f6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:11.601771 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:11.602809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:11.603705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:11.603705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:11.603705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:11.603705 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 282/568] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:11.603705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3639b00-f7b8-4031-9260-50e264ce3f6b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:11.627581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bc7f5-7ab5-4bd4-b272-6bf6cf9b8eff None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:11.662133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a365c3b-2809-49d8-af0f-93f25d938fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:11.662779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a365c3b-2809-49d8-af0f-93f25d938fee None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:11.663047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a365c3b-2809-49d8-af0f-93f25d938fee None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:11.663307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a365c3b-2809-49d8-af0f-93f25d938fee None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:11.670110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a365c3b-2809-49d8-af0f-93f25d938fee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:11.671592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a365c3b-2809-49d8-af0f-93f25d938fee None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-4a365c3b-2809-49d8-af0f-93f25d938fee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:11.682621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:11.683581 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:11.689386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:11.689386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:11.689386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:11.689386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:11.689386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:11.689386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:11.689386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:11.689386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:11.689386 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 287/569] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:11.689386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a365c3b-2809-49d8-af0f-93f25d938fee None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:11.719718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3639b00-f7b8-4031-9260-50e264ce3f6b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:11.753703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4c808-ce57-4900-b4f5-f517eaa3cbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:11.754295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4c808-ce57-4900-b4f5-f517eaa3cbeb None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:11.754560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4c808-ce57-4900-b4f5-f517eaa3cbeb None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:11.754826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4c808-ce57-4900-b4f5-f517eaa3cbeb None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:11.755531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4c808-ce57-4900-b4f5-f517eaa3cbeb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:11.755827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4c808-ce57-4900-b4f5-f517eaa3cbeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:11.757260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4c808-ce57-4900-b4f5-f517eaa3cbeb None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:11.771973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:11.775695 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 283/570] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Aug 20 20:39:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:11.776616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4c808-ce57-4900-b4f5-f517eaa3cbeb None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:11.793275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a365c3b-2809-49d8-af0f-93f25d938fee None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:11.826814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:11.829939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2bc01-27e9-47cb-a2bf-b85a741269a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:11.830571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2bc01-27e9-47cb-a2bf-b85a741269a1 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:11.830840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2bc01-27e9-47cb-a2bf-b85a741269a1 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:11.831103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2bc01-27e9-47cb-a2bf-b85a741269a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:11.841232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2bc01-27e9-47cb-a2bf-b85a741269a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-e0a2bc01-27e9-47cb-a2bf-b85a741269a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:11.846381 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:11.847396 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:11.848214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:11.848214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:11.848214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:11.848214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:11.848214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:11.848214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:11.848214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:11.848214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:11.848214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:11.848214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:11.848214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:11.848214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:11.848214 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 288/571] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:11.848960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2bc01-27e9-47cb-a2bf-b85a741269a1 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:11.881513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4c808-ce57-4900-b4f5-f517eaa3cbeb None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:11.916231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c650d2-2648-463c-b25f-2c75565ca2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:11.916935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c650d2-2648-463c-b25f-2c75565ca2b0 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:11.917191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c650d2-2648-463c-b25f-2c75565ca2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:11.917478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c650d2-2648-463c-b25f-2c75565ca2b0 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:11.918329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c650d2-2648-463c-b25f-2c75565ca2b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:11.918682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c650d2-2648-463c-b25f-2c75565ca2b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:11.920037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c650d2-2648-463c-b25f-2c75565ca2b0 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:11.931629 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 284/572] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:11.932642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c650d2-2648-463c-b25f-2c75565ca2b0 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:11.948175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2bc01-27e9-47cb-a2bf-b85a741269a1 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:11.982668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e237676-aca3-4e14-9699-f1ef32bf9634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:11.983350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e237676-aca3-4e14-9699-f1ef32bf9634 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:11.983628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e237676-aca3-4e14-9699-f1ef32bf9634 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:11.983894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e237676-aca3-4e14-9699-f1ef32bf9634 None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:11.984612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e237676-aca3-4e14-9699-f1ef32bf9634 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:11.984939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e237676-aca3-4e14-9699-f1ef32bf9634 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:11.986361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e237676-aca3-4e14-9699-f1ef32bf9634 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:11.995355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:12.004064 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 289/573] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:12.004892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e237676-aca3-4e14-9699-f1ef32bf9634 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:12.030521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c650d2-2648-463c-b25f-2c75565ca2b0 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:12.039216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:12.065902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53598db-ab1f-4433-a437-011bec2dff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:12.066528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53598db-ab1f-4433-a437-011bec2dff1c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:12.066803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53598db-ab1f-4433-a437-011bec2dff1c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:12.067071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53598db-ab1f-4433-a437-011bec2dff1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:12.078146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53598db-ab1f-4433-a437-011bec2dff1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-d53598db-ab1f-4433-a437-011bec2dff1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:12.083299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:12.084430 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 285/574] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:12.090034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53598db-ab1f-4433-a437-011bec2dff1c None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:12.107549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e237676-aca3-4e14-9699-f1ef32bf9634 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:12.142218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4d39a-5d7c-4303-ac76-64e4082dcde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:12.142838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4d39a-5d7c-4303-ac76-64e4082dcde4 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:12.143115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4d39a-5d7c-4303-ac76-64e4082dcde4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:12.143380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4d39a-5d7c-4303-ac76-64e4082dcde4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:12.153774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4d39a-5d7c-4303-ac76-64e4082dcde4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:12.155220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4d39a-5d7c-4303-ac76-64e4082dcde4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-d0a4d39a-5d7c-4303-ac76-64e4082dcde4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:12.170546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:12.171519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:12.173123 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:12.173123 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:12.173123 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:12.173123 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 290/575] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:12.173123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4d39a-5d7c-4303-ac76-64e4082dcde4 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:12.189185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53598db-ab1f-4433-a437-011bec2dff1c None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:12.221423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb97a53-96bd-4eaa-b0be-1a8ec505eeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:12.222095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb97a53-96bd-4eaa-b0be-1a8ec505eeee None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:12.222439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb97a53-96bd-4eaa-b0be-1a8ec505eeee None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:12.222774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb97a53-96bd-4eaa-b0be-1a8ec505eeee None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:12.223501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb97a53-96bd-4eaa-b0be-1a8ec505eeee None admin] RBAC: Adding query filter params (name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:12.223798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb97a53-96bd-4eaa-b0be-1a8ec505eeee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:12.225193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb97a53-96bd-4eaa-b0be-1a8ec505eeee None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:12.232387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:12.240394 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 286/576] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:12.240394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb97a53-96bd-4eaa-b0be-1a8ec505eeee None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:12.271587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:12.281474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4d39a-5d7c-4303-ac76-64e4082dcde4 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:12.315050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e313c8-7f83-48be-8eca-a4c115a6ec1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:12.315670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e313c8-7f83-48be-8eca-a4c115a6ec1b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:12.315940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e313c8-7f83-48be-8eca-a4c115a6ec1b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:12.316203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e313c8-7f83-48be-8eca-a4c115a6ec1b None admin] PATH_INFO: `/v3/users/glance` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:12.323587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e313c8-7f83-48be-8eca-a4c115a6ec1b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:12.325107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e313c8-7f83-48be-8eca-a4c115a6ec1b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-42e313c8-7f83-48be-8eca-a4c115a6ec1b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:12.335774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:12.336886 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:12.337859 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:12.337859 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:12.337859 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:12.337859 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:12.337859 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:12.337859 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:12.337859 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:12.337859 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:12.337859 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 20 20:39:12.337859 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:12.337859 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 291/577] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:39:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:12.337859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e313c8-7f83-48be-8eca-a4c115a6ec1b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:12.341659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb97a53-96bd-4eaa-b0be-1a8ec505eeee None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:12.375373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618a398-a8ab-4863-831f-0b9e2f9dd60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:12.375957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618a398-a8ab-4863-831f-0b9e2f9dd60b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:12.376227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618a398-a8ab-4863-831f-0b9e2f9dd60b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:12.376535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618a398-a8ab-4863-831f-0b9e2f9dd60b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:12.377251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618a398-a8ab-4863-831f-0b9e2f9dd60b None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=d74f7cdc155a4397ad28a3fbca86f7f9) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:12.377558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618a398-a8ab-4863-831f-0b9e2f9dd60b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:12.379122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618a398-a8ab-4863-831f-0b9e2f9dd60b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:12.384453 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 287/578] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:12 2022] GET /identity/v3/role_assignments?user.id=d74f7cdc155a4397ad28a3fbca86f7f9&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:12.385224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618a398-a8ab-4863-831f-0b9e2f9dd60b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:12.447513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e313c8-7f83-48be-8eca-a4c115a6ec1b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:12.467914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:12.492921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ab109-1917-493c-aa4c-8b06a5796071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:12.493454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ab109-1917-493c-aa4c-8b06a5796071 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:12.493724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ab109-1917-493c-aa4c-8b06a5796071 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:12.493989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ab109-1917-493c-aa4c-8b06a5796071 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:12.494652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ab109-1917-493c-aa4c-8b06a5796071 None admin] RBAC: Adding query filter params (name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:12.495421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ab109-1917-493c-aa4c-8b06a5796071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:12.497038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ab109-1917-493c-aa4c-8b06a5796071 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:12.500150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:12.507376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618a398-a8ab-4863-831f-0b9e2f9dd60b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:12.513832 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 292/579] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:12.513832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ab109-1917-493c-aa4c-8b06a5796071 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:12.554354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bac8ce-2137-49ef-9483-7299fc82761a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:12.554967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bac8ce-2137-49ef-9483-7299fc82761a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:12.555222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bac8ce-2137-49ef-9483-7299fc82761a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:12.555485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bac8ce-2137-49ef-9483-7299fc82761a None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:12.556128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bac8ce-2137-49ef-9483-7299fc82761a None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:12.556447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bac8ce-2137-49ef-9483-7299fc82761a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:12.557958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bac8ce-2137-49ef-9483-7299fc82761a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:12.570695 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 288/580] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:12.571762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bac8ce-2137-49ef-9483-7299fc82761a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:12.630276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ab109-1917-493c-aa4c-8b06a5796071 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:12.670050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3092b-ada2-4c13-8c7a-4cf440600535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:12.670687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3092b-ada2-4c13-8c7a-4cf440600535 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:12.671025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3092b-ada2-4c13-8c7a-4cf440600535 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:12.671290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3092b-ada2-4c13-8c7a-4cf440600535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:12.680325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bac8ce-2137-49ef-9483-7299fc82761a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:12.681793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3092b-ada2-4c13-8c7a-4cf440600535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:12.683724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3092b-ada2-4c13-8c7a-4cf440600535 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-f3b3092b-ada2-4c13-8c7a-4cf440600535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:12.701888 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:12.703121 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:12.704198 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:12.704198 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:12.704198 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:12.710373 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 293/581] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:12.710373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3092b-ada2-4c13-8c7a-4cf440600535 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:12.724199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8882fa3-e054-43e3-9d57-0c5cd7d05521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:12.724869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8882fa3-e054-43e3-9d57-0c5cd7d05521 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:12.725162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8882fa3-e054-43e3-9d57-0c5cd7d05521 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:12.725403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8882fa3-e054-43e3-9d57-0c5cd7d05521 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:12.726133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8882fa3-e054-43e3-9d57-0c5cd7d05521 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:12.726434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8882fa3-e054-43e3-9d57-0c5cd7d05521 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:12.727857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8882fa3-e054-43e3-9d57-0c5cd7d05521 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:12.738325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:12.746771 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 289/582] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Aug 20 20:39:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:12.747864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8882fa3-e054-43e3-9d57-0c5cd7d05521 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:12.763673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:12.822134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3092b-ada2-4c13-8c7a-4cf440600535 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:12.854929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8882fa3-e054-43e3-9d57-0c5cd7d05521 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:12.859502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fa5d5-d60b-4562-ba2f-d36c81740742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:12.860127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fa5d5-d60b-4562-ba2f-d36c81740742 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:12.860411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fa5d5-d60b-4562-ba2f-d36c81740742 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:12.860677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fa5d5-d60b-4562-ba2f-d36c81740742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:12.872155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fa5d5-d60b-4562-ba2f-d36c81740742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:12.873671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fa5d5-d60b-4562-ba2f-d36c81740742 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-e64fa5d5-d60b-4562-ba2f-d36c81740742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:12.890170 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:12.891131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:12.892044 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:12.892044 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:12.892044 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:12.892044 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 294/583] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:12.894213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717980a1-2604-4be5-af00-41187f818d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:12.894882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717980a1-2604-4be5-af00-41187f818d71 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:12.895150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717980a1-2604-4be5-af00-41187f818d71 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:12.895433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717980a1-2604-4be5-af00-41187f818d71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:12.896693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fa5d5-d60b-4562-ba2f-d36c81740742 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:12.903055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717980a1-2604-4be5-af00-41187f818d71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:12.904527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717980a1-2604-4be5-af00-41187f818d71 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-717980a1-2604-4be5-af00-41187f818d71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:12.915576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:12.916487 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:12.917498 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:12.917498 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:12.917498 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:12.917498 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:12.917498 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:12.917498 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:12.917498 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:12.917498 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:12.917498 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 290/584] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:12.917976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717980a1-2604-4be5-af00-41187f818d71 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:12.980451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:12.989124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:13.003261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fa5d5-d60b-4562-ba2f-d36c81740742 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:13.018811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717980a1-2604-4be5-af00-41187f818d71 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:13.037311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a3761-8731-4c09-a079-7fc4722fe769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:13.037910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a3761-8731-4c09-a079-7fc4722fe769 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:13.038176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a3761-8731-4c09-a079-7fc4722fe769 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:13.038434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a3761-8731-4c09-a079-7fc4722fe769 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:13.039167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a3761-8731-4c09-a079-7fc4722fe769 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:13.039460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a3761-8731-4c09-a079-7fc4722fe769 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:13.040862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a3761-8731-4c09-a079-7fc4722fe769 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:13.053013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31130e5-b4fc-4cef-ac9b-470133033fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:13.053627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31130e5-b4fc-4cef-ac9b-470133033fa1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:13.053923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31130e5-b4fc-4cef-ac9b-470133033fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:13.054161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31130e5-b4fc-4cef-ac9b-470133033fa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:13.054688 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 295/585] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:13.055484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a3761-8731-4c09-a079-7fc4722fe769 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:13.070766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31130e5-b4fc-4cef-ac9b-470133033fa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:13.072409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31130e5-b4fc-4cef-ac9b-470133033fa1 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-d31130e5-b4fc-4cef-ac9b-470133033fa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:13.088889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:13.090035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:13.095834 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:13.095834 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:13.095834 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:13.095834 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 291/586] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:13.095834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31130e5-b4fc-4cef-ac9b-470133033fa1 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:13.160845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a3761-8731-4c09-a079-7fc4722fe769 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:13.195246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa1c16-427c-4e90-ae8e-8e16ca403bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:13.195926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa1c16-427c-4e90-ae8e-8e16ca403bfe None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:13.196299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa1c16-427c-4e90-ae8e-8e16ca403bfe None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:13.196477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31130e5-b4fc-4cef-ac9b-470133033fa1 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:13.196702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa1c16-427c-4e90-ae8e-8e16ca403bfe None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:13.197436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa1c16-427c-4e90-ae8e-8e16ca403bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:13.197813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa1c16-427c-4e90-ae8e-8e16ca403bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:13.199383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa1c16-427c-4e90-ae8e-8e16ca403bfe None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:13.205857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:13.216565 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 296/587] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:13.216565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa1c16-427c-4e90-ae8e-8e16ca403bfe None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:13.230212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:13.243487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5ce6a-4e70-4a40-822a-03d88a549124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:13.244213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5ce6a-4e70-4a40-822a-03d88a549124 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:13.244582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5ce6a-4e70-4a40-822a-03d88a549124 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:13.245023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5ce6a-4e70-4a40-822a-03d88a549124 None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:13.245765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5ce6a-4e70-4a40-822a-03d88a549124 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:13.246065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5ce6a-4e70-4a40-822a-03d88a549124 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:13.247518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5ce6a-4e70-4a40-822a-03d88a549124 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:13.260518 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 292/588] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:13.264914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5ce6a-4e70-4a40-822a-03d88a549124 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:13.330022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa1c16-427c-4e90-ae8e-8e16ca403bfe None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:13.365006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763146ef-5407-4db3-973e-42fa66250f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:13.365672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763146ef-5407-4db3-973e-42fa66250f21 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:13.365943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763146ef-5407-4db3-973e-42fa66250f21 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:13.366202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763146ef-5407-4db3-973e-42fa66250f21 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:13.372076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5ce6a-4e70-4a40-822a-03d88a549124 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:13.374566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763146ef-5407-4db3-973e-42fa66250f21 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:13.376022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763146ef-5407-4db3-973e-42fa66250f21 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-763146ef-5407-4db3-973e-42fa66250f21 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:13.393772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:13.394809 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:13.395808 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:13.395808 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:13.395808 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:13.395808 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:13.395808 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:13.395808 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:13.395808 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:13.395808 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:13.395808 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 20 20:39:13.395808 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:13.402326 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 297/589] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:13.402326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763146ef-5407-4db3-973e-42fa66250f21 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:13.409442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3fd2e-b9e4-4307-aff8-b8e27e4378c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:13.410330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3fd2e-b9e4-4307-aff8-b8e27e4378c8 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:13.410838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3fd2e-b9e4-4307-aff8-b8e27e4378c8 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:13.411314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3fd2e-b9e4-4307-aff8-b8e27e4378c8 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:13.412175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3fd2e-b9e4-4307-aff8-b8e27e4378c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:13.412690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3fd2e-b9e4-4307-aff8-b8e27e4378c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:13.414252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3fd2e-b9e4-4307-aff8-b8e27e4378c8 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:13.430828 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 293/590] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:13.431924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3fd2e-b9e4-4307-aff8-b8e27e4378c8 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:13.468354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:13.490516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:13.516638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763146ef-5407-4db3-973e-42fa66250f21 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:13.550390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3fd2e-b9e4-4307-aff8-b8e27e4378c8 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:13.552047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0123f9-8e3b-4c38-9aa5-81ba942bcd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:13.552704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0123f9-8e3b-4c38-9aa5-81ba942bcd84 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:13.552973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0123f9-8e3b-4c38-9aa5-81ba942bcd84 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:13.553229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0123f9-8e3b-4c38-9aa5-81ba942bcd84 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:13.561101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0123f9-8e3b-4c38-9aa5-81ba942bcd84 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:13.562519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0123f9-8e3b-4c38-9aa5-81ba942bcd84 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-2d0123f9-8e3b-4c38-9aa5-81ba942bcd84 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:13.573756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:13.574861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:13.576091 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:13.576091 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:13.576091 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:13.576091 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:13.576091 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:13.576091 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:13.576091 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:13.576091 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:13.576091 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 20 20:39:13.576091 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:13.579500 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 298/591] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:39:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:13.580325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0123f9-8e3b-4c38-9aa5-81ba942bcd84 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:13.594080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098558b5-88ee-4126-8ef2-9e8537e3d0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:13.594777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098558b5-88ee-4126-8ef2-9e8537e3d0a2 None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:13.594990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098558b5-88ee-4126-8ef2-9e8537e3d0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:13.595255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098558b5-88ee-4126-8ef2-9e8537e3d0a2 None admin] PATH_INFO: `/v3/projects/3b6c88840d764c2385aa678da4338213/users/df852af59e7743f7a8798d9edf8ebd3e/roles/04bd1260156b4de9bc4feff892a644a6` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:13.612203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098558b5-88ee-4126-8ef2-9e8537e3d0a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b6c88840d764c2385aa678da4338213, user_id=df852af59e7743f7a8798d9edf8ebd3e, role_id=04bd1260156b4de9bc4feff892a644a6)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:13.614770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098558b5-88ee-4126-8ef2-9e8537e3d0a2 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:13.631310 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 294/592] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:39:13 2022] PUT /identity/v3/projects/3b6c88840d764c2385aa678da4338213/users/df852af59e7743f7a8798d9edf8ebd3e/roles/04bd1260156b4de9bc4feff892a644a6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:39:13.632357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098558b5-88ee-4126-8ef2-9e8537e3d0a2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:13.704756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0123f9-8e3b-4c38-9aa5-81ba942bcd84 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:13.725344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:13.745065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fee55-6a39-4306-a587-077cb69ee808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:13.745898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fee55-6a39-4306-a587-077cb69ee808 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:13.746300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fee55-6a39-4306-a587-077cb69ee808 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:13.746776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fee55-6a39-4306-a587-077cb69ee808 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:13.747584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fee55-6a39-4306-a587-077cb69ee808 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:13.747957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fee55-6a39-4306-a587-077cb69ee808 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:13.749629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fee55-6a39-4306-a587-077cb69ee808 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:13.772338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:13.785328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098558b5-88ee-4126-8ef2-9e8537e3d0a2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:13.789478 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 299/593] 10.222.0.23 () {56 vars in 1139 bytes} [Sat Aug 20 20:39:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:13.790707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fee55-6a39-4306-a587-077cb69ee808 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:13.835424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09a6a0-0f10-47e9-9adb-b58275529ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:13.837106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09a6a0-0f10-47e9-9adb-b58275529ce1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:13.837564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09a6a0-0f10-47e9-9adb-b58275529ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:13.838040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09a6a0-0f10-47e9-9adb-b58275529ce1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:13.847461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09a6a0-0f10-47e9-9adb-b58275529ce1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:13.849377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09a6a0-0f10-47e9-9adb-b58275529ce1 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-7c09a6a0-0f10-47e9-9adb-b58275529ce1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:13.860713 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:13.861634 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:13.862721 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:13.862721 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:13.862721 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:13.862721 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:13.862721 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:13.862721 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:13.862721 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:13.862721 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:13.862721 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 295/594] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:13.862721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09a6a0-0f10-47e9-9adb-b58275529ce1 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:13.933261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fee55-6a39-4306-a587-077cb69ee808 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:13.973747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09a6a0-0f10-47e9-9adb-b58275529ce1 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:13.983604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475ba82-3df0-427f-aecc-72fb0599f240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:13.984598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475ba82-3df0-427f-aecc-72fb0599f240 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:13.984878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475ba82-3df0-427f-aecc-72fb0599f240 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:13.985146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475ba82-3df0-427f-aecc-72fb0599f240 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:13.986189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475ba82-3df0-427f-aecc-72fb0599f240 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:13.986493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475ba82-3df0-427f-aecc-72fb0599f240 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:13.988400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475ba82-3df0-427f-aecc-72fb0599f240 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:14.011777 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 300/595] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Aug 20 20:39:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:14.012661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475ba82-3df0-427f-aecc-72fb0599f240 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:14.016666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de1a4a-0270-4d13-a2ac-8811e754abe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:14.017463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de1a4a-0270-4d13-a2ac-8811e754abe4 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:14.017831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de1a4a-0270-4d13-a2ac-8811e754abe4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:14.018221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de1a4a-0270-4d13-a2ac-8811e754abe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:14.036040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de1a4a-0270-4d13-a2ac-8811e754abe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:14.038200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de1a4a-0270-4d13-a2ac-8811e754abe4 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:14.055245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-93de1a4a-0270-4d13-a2ac-8811e754abe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:14.058602 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:14.059987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:14.060911 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:14.060911 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:14.060911 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:14.069413 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 296/596] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:14.069413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de1a4a-0270-4d13-a2ac-8811e754abe4 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:14.078451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:14.138204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475ba82-3df0-427f-aecc-72fb0599f240 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:14.172294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de1a4a-0270-4d13-a2ac-8811e754abe4 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:14.176695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d531d4f-50e8-42b4-994e-4061c99a9f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:14.177463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d531d4f-50e8-42b4-994e-4061c99a9f2b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:14.177904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d531d4f-50e8-42b4-994e-4061c99a9f2b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:14.178306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d531d4f-50e8-42b4-994e-4061c99a9f2b None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:14.179120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d531d4f-50e8-42b4-994e-4061c99a9f2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:14.179553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d531d4f-50e8-42b4-994e-4061c99a9f2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:14.181418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d531d4f-50e8-42b4-994e-4061c99a9f2b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:14.195020 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 301/597] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:14.196195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d531d4f-50e8-42b4-994e-4061c99a9f2b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:14.215158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597c7b5-3c4b-445e-9142-989bcac605fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:14.215938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597c7b5-3c4b-445e-9142-989bcac605fd None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:14.216318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597c7b5-3c4b-445e-9142-989bcac605fd None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:14.216735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597c7b5-3c4b-445e-9142-989bcac605fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:14.230384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597c7b5-3c4b-445e-9142-989bcac605fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:14.230384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597c7b5-3c4b-445e-9142-989bcac605fd None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-d597c7b5-3c4b-445e-9142-989bcac605fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:14.244322 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:14.245579 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:14.246648 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:14.246648 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:14.246648 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:14.252417 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 297/598] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:14.253510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597c7b5-3c4b-445e-9142-989bcac605fd None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:14.312966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:14.314170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:14.322752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d531d4f-50e8-42b4-994e-4061c99a9f2b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:14.360962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597c7b5-3c4b-445e-9142-989bcac605fd None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:14.368747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214f358-c1fb-4d85-a133-08ce6a4ecd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:14.370174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214f358-c1fb-4d85-a133-08ce6a4ecd7d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:14.370750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214f358-c1fb-4d85-a133-08ce6a4ecd7d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:14.371198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214f358-c1fb-4d85-a133-08ce6a4ecd7d None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:14.372464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214f358-c1fb-4d85-a133-08ce6a4ecd7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:14.372918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214f358-c1fb-4d85-a133-08ce6a4ecd7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:14.375143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214f358-c1fb-4d85-a133-08ce6a4ecd7d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:14.395561 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 302/599] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:14.397461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214f358-c1fb-4d85-a133-08ce6a4ecd7d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:14.400623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f92f8-4fe3-4a28-a1f8-d47e2bea771e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:14.401527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f92f8-4fe3-4a28-a1f8-d47e2bea771e None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:14.401861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f92f8-4fe3-4a28-a1f8-d47e2bea771e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:14.402230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f92f8-4fe3-4a28-a1f8-d47e2bea771e None admin] PATH_INFO: `/v3/projects/3b6c88840d764c2385aa678da4338213/users/698d5a7fc34a46ba909152414a0860d7/roles/04bd1260156b4de9bc4feff892a644a6` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:14.420788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f92f8-4fe3-4a28-a1f8-d47e2bea771e None admin] RBAC: Authorizing `identity:create_grant(project_id=3b6c88840d764c2385aa678da4338213, user_id=698d5a7fc34a46ba909152414a0860d7, role_id=04bd1260156b4de9bc4feff892a644a6)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:14.422440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f92f8-4fe3-4a28-a1f8-d47e2bea771e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:14.438586 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 298/600] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:39:14 2022] PUT /identity/v3/projects/3b6c88840d764c2385aa678da4338213/users/698d5a7fc34a46ba909152414a0860d7/roles/04bd1260156b4de9bc4feff892a644a6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:39:14.445778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f92f8-4fe3-4a28-a1f8-d47e2bea771e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:14.529493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214f358-c1fb-4d85-a133-08ce6a4ecd7d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:14.572026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667d8f8-5d5e-4f6c-9171-355f428b8ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:14.574315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667d8f8-5d5e-4f6c-9171-355f428b8ea6 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:14.574806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667d8f8-5d5e-4f6c-9171-355f428b8ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:14.575219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667d8f8-5d5e-4f6c-9171-355f428b8ea6 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:14.576084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667d8f8-5d5e-4f6c-9171-355f428b8ea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:14.576566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667d8f8-5d5e-4f6c-9171-355f428b8ea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:14.578122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667d8f8-5d5e-4f6c-9171-355f428b8ea6 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:14.595605 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 303/601] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:14.597939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335507ac-0054-4998-9033-6d603bba40a2 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:14.598651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335507ac-0054-4998-9033-6d603bba40a2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:14.599076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335507ac-0054-4998-9033-6d603bba40a2 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:14.601504 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 304/602] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:14.602578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335507ac-0054-4998-9033-6d603bba40a2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:14.616859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f92f8-4fe3-4a28-a1f8-d47e2bea771e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:14.628817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:14.633259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:14.666069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248baf6-f2cd-4ac7-a8b2-b5d7fbe5134b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:14.666990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248baf6-f2cd-4ac7-a8b2-b5d7fbe5134b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:14.667451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248baf6-f2cd-4ac7-a8b2-b5d7fbe5134b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:14.667849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248baf6-f2cd-4ac7-a8b2-b5d7fbe5134b None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:14.676795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248baf6-f2cd-4ac7-a8b2-b5d7fbe5134b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:14.678543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248baf6-f2cd-4ac7-a8b2-b5d7fbe5134b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-c248baf6-f2cd-4ac7-a8b2-b5d7fbe5134b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:14.692809 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:14.693847 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:14.694746 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:14.694746 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:14.694746 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:14.694746 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:14.694746 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:14.694746 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:14.694746 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:14.694746 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:14.697108 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 299/603] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:14.698945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58079342-d5c4-4181-8a4c-8094082f3bd4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:14.699549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58079342-d5c4-4181-8a4c-8094082f3bd4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:14.700568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58079342-d5c4-4181-8a4c-8094082f3bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:14.721643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-58079342-d5c4-4181-8a4c-8094082f3bd4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:14.723195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-58079342-d5c4-4181-8a4c-8094082f3bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:14.731673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335507ac-0054-4998-9033-6d603bba40a2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:14.771001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9d505-c2aa-4499-ab38-778bacd887ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:14.771894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9d505-c2aa-4499-ab38-778bacd887ef None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:14.772487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9d505-c2aa-4499-ab38-778bacd887ef None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:14.773456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9d505-c2aa-4499-ab38-778bacd887ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:14.783573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9d505-c2aa-4499-ab38-778bacd887ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:14.785120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9d505-c2aa-4499-ab38-778bacd887ef None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-5af9d505-c2aa-4499-ab38-778bacd887ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:14.798386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:14.799326 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:14.800211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:14.800211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:14.800211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:14.800211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:14.800211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:14.800211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:14.800211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:14.800211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:14.800211 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 305/604] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:14.802385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9d505-c2aa-4499-ab38-778bacd887ef None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:14.817298 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 300/605] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:14.818820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58079342-d5c4-4181-8a4c-8094082f3bd4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:14.931515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:14.937822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58079342-d5c4-4181-8a4c-8094082f3bd4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:14.977025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b89c3-dcb4-40e8-af91-cd683aafe8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:14.978022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b89c3-dcb4-40e8-af91-cd683aafe8e4 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:14.978491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b89c3-dcb4-40e8-af91-cd683aafe8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:14.978958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b89c3-dcb4-40e8-af91-cd683aafe8e4 None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:14.979879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b89c3-dcb4-40e8-af91-cd683aafe8e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:14.981230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b89c3-dcb4-40e8-af91-cd683aafe8e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:14.982894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b89c3-dcb4-40e8-af91-cd683aafe8e4 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:14.985898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:14.996652 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 301/606] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:15.004039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b89c3-dcb4-40e8-af91-cd683aafe8e4 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:15.041182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9d505-c2aa-4499-ab38-778bacd887ef None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:15.103156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355841e-68f4-4c6d-b806-554cbbb9c97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:15.103746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355841e-68f4-4c6d-b806-554cbbb9c97e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:15.104011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355841e-68f4-4c6d-b806-554cbbb9c97e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:15.104321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355841e-68f4-4c6d-b806-554cbbb9c97e None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:15.105024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355841e-68f4-4c6d-b806-554cbbb9c97e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:15.105319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355841e-68f4-4c6d-b806-554cbbb9c97e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:15.106792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355841e-68f4-4c6d-b806-554cbbb9c97e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:15.128762 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 306/607] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:15.137162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355841e-68f4-4c6d-b806-554cbbb9c97e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:15.183047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b89c3-dcb4-40e8-af91-cd683aafe8e4 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:15.245062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bfcdda-3fd7-4b7c-8df3-d21dd0531245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:15.245747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bfcdda-3fd7-4b7c-8df3-d21dd0531245 None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:15.246057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bfcdda-3fd7-4b7c-8df3-d21dd0531245 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:15.246268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bfcdda-3fd7-4b7c-8df3-d21dd0531245 None admin] PATH_INFO: `/v3/projects/3b6c88840d764c2385aa678da4338213/users/53928836828d45fbb427dd92492cae20/roles/04bd1260156b4de9bc4feff892a644a6` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:15.268819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bfcdda-3fd7-4b7c-8df3-d21dd0531245 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b6c88840d764c2385aa678da4338213, user_id=53928836828d45fbb427dd92492cae20, role_id=04bd1260156b4de9bc4feff892a644a6)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:15.270463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bfcdda-3fd7-4b7c-8df3-d21dd0531245 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:15.287027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355841e-68f4-4c6d-b806-554cbbb9c97e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:15.290896 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 302/608] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:39:15 2022] PUT /identity/v3/projects/3b6c88840d764c2385aa678da4338213/users/53928836828d45fbb427dd92492cae20/roles/04bd1260156b4de9bc4feff892a644a6 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:39:15.291834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bfcdda-3fd7-4b7c-8df3-d21dd0531245 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:15.315508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:15.363917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7658e1-838d-4b6f-a367-8d02c3a72d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:15.364806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7658e1-838d-4b6f-a367-8d02c3a72d4b None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:15.365128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7658e1-838d-4b6f-a367-8d02c3a72d4b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:15.365395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7658e1-838d-4b6f-a367-8d02c3a72d4b None admin] PATH_INFO: `/v3/projects/3b6c88840d764c2385aa678da4338213/users/22bb014e04774c278c99496bf64b2319/roles/04bd1260156b4de9bc4feff892a644a6` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:15.368578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:15.385714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7658e1-838d-4b6f-a367-8d02c3a72d4b None admin] RBAC: Authorizing `identity:create_grant(project_id=3b6c88840d764c2385aa678da4338213, user_id=22bb014e04774c278c99496bf64b2319, role_id=04bd1260156b4de9bc4feff892a644a6)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:15.388323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7658e1-838d-4b6f-a367-8d02c3a72d4b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:15.408362 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 307/609] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:39:15 2022] PUT /identity/v3/projects/3b6c88840d764c2385aa678da4338213/users/22bb014e04774c278c99496bf64b2319/roles/04bd1260156b4de9bc4feff892a644a6 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:39:15.499302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bfcdda-3fd7-4b7c-8df3-d21dd0531245 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:15.562134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065dbba-68c8-4e49-a92c-177e6fecb6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:15.562783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065dbba-68c8-4e49-a92c-177e6fecb6c2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:15.563059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065dbba-68c8-4e49-a92c-177e6fecb6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:15.563327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065dbba-68c8-4e49-a92c-177e6fecb6c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:15.581966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065dbba-68c8-4e49-a92c-177e6fecb6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9065dbba-68c8-4e49-a92c-177e6fecb6c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:15.586810 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:15.587938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:15.589073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:15.589073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:15.589073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:15.589073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:15.589073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:15.589073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:15.589073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:15.589073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:15.589073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:15.589073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:15.589073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:15.589073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:15.590869 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 303/610] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:15.596016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7658e1-838d-4b6f-a367-8d02c3a72d4b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:15.731539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7658e1-838d-4b6f-a367-8d02c3a72d4b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:15.767010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58de34c-ebb6-4b96-8526-8a745f7e63e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:15.767531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58de34c-ebb6-4b96-8526-8a745f7e63e8 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:15.767796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58de34c-ebb6-4b96-8526-8a745f7e63e8 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:15.768056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58de34c-ebb6-4b96-8526-8a745f7e63e8 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:15.768804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58de34c-ebb6-4b96-8526-8a745f7e63e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:15.769097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58de34c-ebb6-4b96-8526-8a745f7e63e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:15.770506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58de34c-ebb6-4b96-8526-8a745f7e63e8 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:15.780429 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 308/611] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:15.785945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065dbba-68c8-4e49-a92c-177e6fecb6c2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:15.908794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:15.949997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065dbba-68c8-4e49-a92c-177e6fecb6c2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:15.990583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939cfe3-d822-4803-bc3f-c2d560d58e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:15.991217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939cfe3-d822-4803-bc3f-c2d560d58e66 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:15.991483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939cfe3-d822-4803-bc3f-c2d560d58e66 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:15.991741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939cfe3-d822-4803-bc3f-c2d560d58e66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:16.006994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e939cfe3-d822-4803-bc3f-c2d560d58e66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:16.008461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e939cfe3-d822-4803-bc3f-c2d560d58e66 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-e939cfe3-d822-4803-bc3f-c2d560d58e66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:16.024343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:16.025308 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:16.026265 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:16.026265 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:16.026265 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:16.026265 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 304/612] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:16.032616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58de34c-ebb6-4b96-8526-8a745f7e63e8 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:16.094655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:16.157848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58de34c-ebb6-4b96-8526-8a745f7e63e8 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:16.183354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b54843-3d5d-44dd-81f7-001bc089bfb4 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:16.183667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b54843-3d5d-44dd-81f7-001bc089bfb4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:16.183921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b54843-3d5d-44dd-81f7-001bc089bfb4 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:16.185979 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 305/613] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:39:16.191443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95019428-5e23-4f4e-a254-350de4e6cb79 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:16.191714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95019428-5e23-4f4e-a254-350de4e6cb79 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:16.191977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95019428-5e23-4f4e-a254-350de4e6cb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:16.205293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c477135-830d-4cda-a25a-deca6ea4e18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:16.205900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c477135-830d-4cda-a25a-deca6ea4e18e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:16.206169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c477135-830d-4cda-a25a-deca6ea4e18e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:16.206431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c477135-830d-4cda-a25a-deca6ea4e18e None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:16.207062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c477135-830d-4cda-a25a-deca6ea4e18e None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:16.207354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c477135-830d-4cda-a25a-deca6ea4e18e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:16.208741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c477135-830d-4cda-a25a-deca6ea4e18e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:16.210521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-95019428-5e23-4f4e-a254-350de4e6cb79 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:16.211690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-95019428-5e23-4f4e-a254-350de4e6cb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:16.227113 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 309/614] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:16.233088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c477135-830d-4cda-a25a-deca6ea4e18e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:16.281231 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 306/615] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:39:16.410858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c477135-830d-4cda-a25a-deca6ea4e18e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:16.423013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:16.463683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f92c2d-b0ee-4a74-8578-86743914300b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:16.464687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f92c2d-b0ee-4a74-8578-86743914300b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:16.464951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f92c2d-b0ee-4a74-8578-86743914300b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:16.465212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f92c2d-b0ee-4a74-8578-86743914300b None admin] PATH_INFO: `/v3/users/swift` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:16.476895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f92c2d-b0ee-4a74-8578-86743914300b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:16.478567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f92c2d-b0ee-4a74-8578-86743914300b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-23f92c2d-b0ee-4a74-8578-86743914300b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:16.494246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:16.495219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:16.496141 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:16.496141 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:16.496141 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:16.496141 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:16.496141 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:16.496141 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:16.496141 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:16.496141 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:16.496141 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 20 20:39:16.496141 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:16.498199 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 310/616] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:39:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:16.504024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95019428-5e23-4f4e-a254-350de4e6cb79 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:16.558764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:16.606844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95019428-5e23-4f4e-a254-350de4e6cb79 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:16.641457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051668e-a4db-4fd0-bb54-d7244cf0b991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:16.642063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051668e-a4db-4fd0-bb54-d7244cf0b991 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:16.642332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051668e-a4db-4fd0-bb54-d7244cf0b991 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:16.642590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051668e-a4db-4fd0-bb54-d7244cf0b991 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:16.643307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051668e-a4db-4fd0-bb54-d7244cf0b991 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:16.643600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051668e-a4db-4fd0-bb54-d7244cf0b991 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:16.645046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051668e-a4db-4fd0-bb54-d7244cf0b991 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:16.663388 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 307/617] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Aug 20 20:39:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:16.669063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f92c2d-b0ee-4a74-8578-86743914300b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:16.683395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051668e-a4db-4fd0-bb54-d7244cf0b991 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:16.788125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051668e-a4db-4fd0-bb54-d7244cf0b991 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:16.801188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:16.832101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0baa96-4469-4e7c-897b-f9aff2cd26c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:16.832810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0baa96-4469-4e7c-897b-f9aff2cd26c0 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:16.833283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0baa96-4469-4e7c-897b-f9aff2cd26c0 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:16.833646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0baa96-4469-4e7c-897b-f9aff2cd26c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:16.845391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0baa96-4469-4e7c-897b-f9aff2cd26c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-8d0baa96-4469-4e7c-897b-f9aff2cd26c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:16.850526 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:16.851953 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:16.853055 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:16.853055 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:16.853055 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:16.853055 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:16.853055 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:16.853055 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:16.853055 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:16.853055 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:16.853055 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:16.853055 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:16.853055 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:16.853055 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:16.855025 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 308/618] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:16.861588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0baa96-4469-4e7c-897b-f9aff2cd26c0 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:16.906611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f92c2d-b0ee-4a74-8578-86743914300b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:16.973238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c562a1b-93a0-40f9-a4f6-85cd7382544b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:16.973866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c562a1b-93a0-40f9-a4f6-85cd7382544b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:16.974137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c562a1b-93a0-40f9-a4f6-85cd7382544b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:16.974402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c562a1b-93a0-40f9-a4f6-85cd7382544b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:17.001395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c562a1b-93a0-40f9-a4f6-85cd7382544b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:17.002959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c562a1b-93a0-40f9-a4f6-85cd7382544b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-4c562a1b-93a0-40f9-a4f6-85cd7382544b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:17.033972 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:17.035133 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:17.036015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:17.036015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:17.036015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:17.036015 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 311/619] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:17.041267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c562a1b-93a0-40f9-a4f6-85cd7382544b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:17.043401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0baa96-4469-4e7c-897b-f9aff2cd26c0 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:17.080688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616a9e9-65b4-4511-826e-7d04990eaf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:17.081308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616a9e9-65b4-4511-826e-7d04990eaf64 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:17.081577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616a9e9-65b4-4511-826e-7d04990eaf64 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:17.081832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616a9e9-65b4-4511-826e-7d04990eaf64 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:17.082721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616a9e9-65b4-4511-826e-7d04990eaf64 None admin] RBAC: Adding query filter params (name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:17.082963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616a9e9-65b4-4511-826e-7d04990eaf64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:17.083317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:17.085533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616a9e9-65b4-4511-826e-7d04990eaf64 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:17.096930 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 309/620] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:17.103682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616a9e9-65b4-4511-826e-7d04990eaf64 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:17.162692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:17.211726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c562a1b-93a0-40f9-a4f6-85cd7382544b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:17.250972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616a9e9-65b4-4511-826e-7d04990eaf64 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:17.254803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33030a7-01be-46d2-b4dc-21dfc3ef6506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:17.255418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33030a7-01be-46d2-b4dc-21dfc3ef6506 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:17.255686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33030a7-01be-46d2-b4dc-21dfc3ef6506 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:17.255949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33030a7-01be-46d2-b4dc-21dfc3ef6506 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:17.256827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33030a7-01be-46d2-b4dc-21dfc3ef6506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:17.257124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33030a7-01be-46d2-b4dc-21dfc3ef6506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:17.258486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33030a7-01be-46d2-b4dc-21dfc3ef6506 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:17.282969 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 312/621] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:17.288782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0d8cf-8f2c-485d-a282-ce25aebb529c None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:17.289060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0d8cf-8f2c-485d-a282-ce25aebb529c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:17.289326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0d8cf-8f2c-485d-a282-ce25aebb529c None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:17.294683 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 313/622] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:17.294683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0d8cf-8f2c-485d-a282-ce25aebb529c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:17.319389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f781b3-0e4b-43f3-a7db-29f533e004b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:17.320622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f781b3-0e4b-43f3-a7db-29f533e004b9 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:17.321346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f781b3-0e4b-43f3-a7db-29f533e004b9 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:17.322194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f781b3-0e4b-43f3-a7db-29f533e004b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:17.339521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f781b3-0e4b-43f3-a7db-29f533e004b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:17.341757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f781b3-0e4b-43f3-a7db-29f533e004b9 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-00f781b3-0e4b-43f3-a7db-29f533e004b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:17.362294 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:17.363309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:17.364210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:17.364210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:17.364210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:17.371475 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 310/623] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:17.371475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f7175-0688-469c-8f5c-6814ae37caff None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:17.371475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f7175-0688-469c-8f5c-6814ae37caff None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:17.371475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f7175-0688-469c-8f5c-6814ae37caff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:17.387755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:17.395858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-506f7175-0688-469c-8f5c-6814ae37caff None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:17.397674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-506f7175-0688-469c-8f5c-6814ae37caff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:17.413022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0d8cf-8f2c-485d-a282-ce25aebb529c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:17.461843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05a726-d5d8-4aec-9798-c299b3a3618e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:17.463159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05a726-d5d8-4aec-9798-c299b3a3618e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:17.463685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05a726-d5d8-4aec-9798-c299b3a3618e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:17.464176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05a726-d5d8-4aec-9798-c299b3a3618e None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:17.473529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05a726-d5d8-4aec-9798-c299b3a3618e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:17.475730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05a726-d5d8-4aec-9798-c299b3a3618e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:17.488396 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 311/624] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:17.488396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f7175-0688-469c-8f5c-6814ae37caff None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-bf05a726-d5d8-4aec-9798-c299b3a3618e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:17.503772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:17.504777 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:17.505719 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:17.505719 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:17.505719 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:17.505719 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:17.505719 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:17.505719 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:17.505719 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:17.505719 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:17.512344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:17.513189 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 314/625] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:17.514023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05a726-d5d8-4aec-9798-c299b3a3618e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:17.685368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f7175-0688-469c-8f5c-6814ae37caff None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:17.704720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05a726-d5d8-4aec-9798-c299b3a3618e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:17.754877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b4eee-b0e3-4804-9055-5b72086ccbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:17.755482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b4eee-b0e3-4804-9055-5b72086ccbbc None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:17.755754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b4eee-b0e3-4804-9055-5b72086ccbbc None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:17.758242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b4eee-b0e3-4804-9055-5b72086ccbbc None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:17.758894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b4eee-b0e3-4804-9055-5b72086ccbbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:17.759193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b4eee-b0e3-4804-9055-5b72086ccbbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:17.760873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b4eee-b0e3-4804-9055-5b72086ccbbc None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:17.768269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ca9b2-6c1b-4b34-9e0d-488cffbcf84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:17.768960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ca9b2-6c1b-4b34-9e0d-488cffbcf84b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:17.769236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ca9b2-6c1b-4b34-9e0d-488cffbcf84b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:17.769487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ca9b2-6c1b-4b34-9e0d-488cffbcf84b None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:17.770360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ca9b2-6c1b-4b34-9e0d-488cffbcf84b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:17.770658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ca9b2-6c1b-4b34-9e0d-488cffbcf84b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:17.776462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ca9b2-6c1b-4b34-9e0d-488cffbcf84b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:17.790206 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 312/626] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:17.790706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:17.801400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b4eee-b0e3-4804-9055-5b72086ccbbc None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:17.803166 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 315/627] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:17.810294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ca9b2-6c1b-4b34-9e0d-488cffbcf84b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:17.903051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:17.928038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ca9b2-6c1b-4b34-9e0d-488cffbcf84b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:17.929628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b4eee-b0e3-4804-9055-5b72086ccbbc None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:17.972116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb14630-d92d-4b94-9957-536358120390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:17.972788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb14630-d92d-4b94-9957-536358120390 None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:17.973057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb14630-d92d-4b94-9957-536358120390 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:17.973315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb14630-d92d-4b94-9957-536358120390 None admin] PATH_INFO: `/v3/projects/3b6c88840d764c2385aa678da4338213/users/d74f7cdc155a4397ad28a3fbca86f7f9/roles/04bd1260156b4de9bc4feff892a644a6` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:17.976068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdd7c5-0e71-4136-bd86-bdb5b4823179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:17.977619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdd7c5-0e71-4136-bd86-bdb5b4823179 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:17.977892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdd7c5-0e71-4136-bd86-bdb5b4823179 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:17.978158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdd7c5-0e71-4136-bd86-bdb5b4823179 None admin] PATH_INFO: `/v3/users/nova` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:17.988150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdd7c5-0e71-4136-bd86-bdb5b4823179 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:17.988682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb14630-d92d-4b94-9957-536358120390 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b6c88840d764c2385aa678da4338213, user_id=d74f7cdc155a4397ad28a3fbca86f7f9, role_id=04bd1260156b4de9bc4feff892a644a6)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:17.989639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdd7c5-0e71-4136-bd86-bdb5b4823179 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:17.990106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb14630-d92d-4b94-9957-536358120390 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-6bfdd7c5-0e71-4136-bd86-bdb5b4823179 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:18.003100 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:18.004168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:18.005145 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:18.005145 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:18.005145 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:18.005145 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:18.005145 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:18.005145 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:18.005145 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:18.005145 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:18.005145 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:39:18.005145 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:18.005145 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 313/628] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Aug 20 20:39:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:18.006304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdd7c5-0e71-4136-bd86-bdb5b4823179 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:18.012307 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 316/629] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:39:17 2022] PUT /identity/v3/projects/3b6c88840d764c2385aa678da4338213/users/d74f7cdc155a4397ad28a3fbca86f7f9/roles/04bd1260156b4de9bc4feff892a644a6 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:39:18.013161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb14630-d92d-4b94-9957-536358120390 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:18.099295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:18.149002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdd7c5-0e71-4136-bd86-bdb5b4823179 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:18.149376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb14630-d92d-4b94-9957-536358120390 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:18.185967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:18.191275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac670a7f-f6aa-4253-91ce-5afbc8ad9d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:18.191888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac670a7f-f6aa-4253-91ce-5afbc8ad9d8d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:18.192152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac670a7f-f6aa-4253-91ce-5afbc8ad9d8d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:18.192344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc57969-3ba2-4733-a34f-f8477dd0b797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:18.192499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac670a7f-f6aa-4253-91ce-5afbc8ad9d8d None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:18.193036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc57969-3ba2-4733-a34f-f8477dd0b797 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:18.193135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac670a7f-f6aa-4253-91ce-5afbc8ad9d8d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:18.193311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc57969-3ba2-4733-a34f-f8477dd0b797 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:18.193418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac670a7f-f6aa-4253-91ce-5afbc8ad9d8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:18.193569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc57969-3ba2-4733-a34f-f8477dd0b797 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:18.194854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac670a7f-f6aa-4253-91ce-5afbc8ad9d8d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:18.206175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc57969-3ba2-4733-a34f-f8477dd0b797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-dfc57969-3ba2-4733-a34f-f8477dd0b797 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:18.212407 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:18.213440 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:18.214269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:18.214269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:18.214269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:18.214269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:18.214269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:18.214269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:18.214269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:18.214269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:18.214269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:18.214269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:18.214269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:18.214269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:18.214269 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 314/630] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:18.214962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f672d-1f98-4765-b4b2-b424e03f2b6d None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:18.215753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f672d-1f98-4765-b4b2-b424e03f2b6d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:18.215753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f672d-1f98-4765-b4b2-b424e03f2b6d None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:18.220480 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 315/631] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:18.220754 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 317/632] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Aug 20 20:39:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:18.221615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac670a7f-f6aa-4253-91ce-5afbc8ad9d8d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:18.221740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913aec6-4516-4088-a615-a0a31da5b556 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:18.222030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913aec6-4516-4088-a615-a0a31da5b556 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:18.222346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913aec6-4516-4088-a615-a0a31da5b556 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:18.232440 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 316/633] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:18 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:18.233354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913aec6-4516-4088-a615-a0a31da5b556 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:18.328839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac670a7f-f6aa-4253-91ce-5afbc8ad9d8d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:18.337126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:18.346179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913aec6-4516-4088-a615-a0a31da5b556 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:18.365709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8793d-b43e-443b-88e2-bd5c7eb786e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:18.366312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8793d-b43e-443b-88e2-bd5c7eb786e0 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:18.366577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8793d-b43e-443b-88e2-bd5c7eb786e0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:18.366834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8793d-b43e-443b-88e2-bd5c7eb786e0 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:18.367533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8793d-b43e-443b-88e2-bd5c7eb786e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:18.367826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8793d-b43e-443b-88e2-bd5c7eb786e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:18.369224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8793d-b43e-443b-88e2-bd5c7eb786e0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:18.377859 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 318/634] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:18.377859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4dc23-4520-4669-9207-1c518fc5a6bb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:18.377859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4dc23-4520-4669-9207-1c518fc5a6bb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:18.377859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4dc23-4520-4669-9207-1c518fc5a6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:18.387979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa35430-1135-4b56-8bfa-b41ee11988b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:18.388133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa35430-1135-4b56-8bfa-b41ee11988b0 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:18.388360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa35430-1135-4b56-8bfa-b41ee11988b0 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:18.388648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa35430-1135-4b56-8bfa-b41ee11988b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:18.398398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-faf4dc23-4520-4669-9207-1c518fc5a6bb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:18.399573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-faf4dc23-4520-4669-9207-1c518fc5a6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:18.450382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:18.462015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa35430-1135-4b56-8bfa-b41ee11988b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:18.463512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa35430-1135-4b56-8bfa-b41ee11988b0 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:18.468203 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 319/635] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:18.469712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56362-862b-4911-ba02-7b1b112f8b63 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:18.470182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56362-862b-4911-ba02-7b1b112f8b63 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:18.470631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56362-862b-4911-ba02-7b1b112f8b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:18.501534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9ed56362-862b-4911-ba02-7b1b112f8b63 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:18.503050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9ed56362-862b-4911-ba02-7b1b112f8b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:18.577225 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 317/636] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:39:18.578616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa35430-1135-4b56-8bfa-b41ee11988b0 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:18.589318 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 320/637] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:18.590587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56362-862b-4911-ba02-7b1b112f8b63 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:18.621281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:18.721535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:18.728328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56362-862b-4911-ba02-7b1b112f8b63 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:18.742918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa35430-1135-4b56-8bfa-b41ee11988b0 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:18.801545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73553af4-ea3a-488d-91dd-9389a7bdd06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:18.803068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73553af4-ea3a-488d-91dd-9389a7bdd06b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:18.804093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73553af4-ea3a-488d-91dd-9389a7bdd06b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:18.805006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73553af4-ea3a-488d-91dd-9389a7bdd06b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:18.808399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe1282-f48c-4662-87cc-2b526840257a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:18.810165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe1282-f48c-4662-87cc-2b526840257a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:18.811128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe1282-f48c-4662-87cc-2b526840257a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:18.812137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe1282-f48c-4662-87cc-2b526840257a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:18.827718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73553af4-ea3a-488d-91dd-9389a7bdd06b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:18.831851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73553af4-ea3a-488d-91dd-9389a7bdd06b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:18.833358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe1282-f48c-4662-87cc-2b526840257a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:18.835128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe1282-f48c-4662-87cc-2b526840257a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-73553af4-ea3a-488d-91dd-9389a7bdd06b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:18.850299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:18.851299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:18.852188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:18.852188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:18.852188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:18.854359 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 321/638] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:18.855323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73553af4-ea3a-488d-91dd-9389a7bdd06b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-4cbe1282-f48c-4662-87cc-2b526840257a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:18.862840 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:18.863806 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:18.864702 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:18.864702 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:18.864702 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:18.864702 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 318/639] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:18.865353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe1282-f48c-4662-87cc-2b526840257a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:18.943837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:18.968857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73553af4-ea3a-488d-91dd-9389a7bdd06b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:18.979912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe1282-f48c-4662-87cc-2b526840257a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:19.005054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6317b8-a93a-4ca7-9d5f-2788ab62bf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:19.006018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6317b8-a93a-4ca7-9d5f-2788ab62bf66 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:19.006579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6317b8-a93a-4ca7-9d5f-2788ab62bf66 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:19.007150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6317b8-a93a-4ca7-9d5f-2788ab62bf66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:19.017233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95383d-8392-4525-8087-10a351a6e08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:19.017811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6317b8-a93a-4ca7-9d5f-2788ab62bf66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:19.018073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95383d-8392-4525-8087-10a351a6e08d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:19.018343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95383d-8392-4525-8087-10a351a6e08d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:19.018605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95383d-8392-4525-8087-10a351a6e08d None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:19.019236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95383d-8392-4525-8087-10a351a6e08d None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:19.019532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95383d-8392-4525-8087-10a351a6e08d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:19.020944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95383d-8392-4525-8087-10a351a6e08d None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-ea6317b8-a93a-4ca7-9d5f-2788ab62bf66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:19.023099 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:19.024185 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 322/640] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:19.025083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6317b8-a93a-4ca7-9d5f-2788ab62bf66 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:19.041941 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 319/641] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:19.041941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95383d-8392-4525-8087-10a351a6e08d None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:19.046133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:19.132345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6317b8-a93a-4ca7-9d5f-2788ab62bf66 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:19.152664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95383d-8392-4525-8087-10a351a6e08d None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:19.169036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf4cad-2756-4339-b365-db68795d48c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:19.169643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf4cad-2756-4339-b365-db68795d48c1 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:19.169922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf4cad-2756-4339-b365-db68795d48c1 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:19.170189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf4cad-2756-4339-b365-db68795d48c1 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:19.170826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf4cad-2756-4339-b365-db68795d48c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:19.171140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf4cad-2756-4339-b365-db68795d48c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:19.172525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf4cad-2756-4339-b365-db68795d48c1 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:19.181682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:19.186086 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 323/642] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:19.186973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf4cad-2756-4339-b365-db68795d48c1 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:19.193837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d38fa-ea85-4efd-a7fb-3021ae862418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:19.194467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d38fa-ea85-4efd-a7fb-3021ae862418 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:19.194739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d38fa-ea85-4efd-a7fb-3021ae862418 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:19.195004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d38fa-ea85-4efd-a7fb-3021ae862418 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:19.206561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d38fa-ea85-4efd-a7fb-3021ae862418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-6b2d38fa-ea85-4efd-a7fb-3021ae862418 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:19.211249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:19.212220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 320/643] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:19.213123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d38fa-ea85-4efd-a7fb-3021ae862418 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:19.280297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:19.313802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf4cad-2756-4339-b365-db68795d48c1 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:19.325219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d38fa-ea85-4efd-a7fb-3021ae862418 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:19.350399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3a3f9-de3a-4d95-9149-43d95fe6d08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:19.351293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3a3f9-de3a-4d95-9149-43d95fe6d08b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:19.351569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3a3f9-de3a-4d95-9149-43d95fe6d08b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:19.351999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3a3f9-de3a-4d95-9149-43d95fe6d08b None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:19.352684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3a3f9-de3a-4d95-9149-43d95fe6d08b None admin] RBAC: Adding query filter params (name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:19.353030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3a3f9-de3a-4d95-9149-43d95fe6d08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:19.354626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3a3f9-de3a-4d95-9149-43d95fe6d08b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:19.359197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8097e-a092-42f0-8d12-ee42253712b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:19.360417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8097e-a092-42f0-8d12-ee42253712b3 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:19.361098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8097e-a092-42f0-8d12-ee42253712b3 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:19.361556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8097e-a092-42f0-8d12-ee42253712b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:19.362537 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 324/644] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:19.363353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3a3f9-de3a-4d95-9149-43d95fe6d08b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:19.368707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8097e-a092-42f0-8d12-ee42253712b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:19.371348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8097e-a092-42f0-8d12-ee42253712b3 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-d1a8097e-a092-42f0-8d12-ee42253712b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:19.384936 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:19.386026 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:19.387108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:19.387108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:19.387108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:19.387108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:19.387108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:19.387108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:19.387108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:19.387108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:19.388023 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 321/645] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:19.388874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8097e-a092-42f0-8d12-ee42253712b3 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:19.430282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:19.469792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3a3f9-de3a-4d95-9149-43d95fe6d08b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:19.497147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8097e-a092-42f0-8d12-ee42253712b3 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:19.504648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f736c8-672b-4ef8-94f3-38d19d2ad902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:19.505273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f736c8-672b-4ef8-94f3-38d19d2ad902 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:19.505539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f736c8-672b-4ef8-94f3-38d19d2ad902 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:19.505883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f736c8-672b-4ef8-94f3-38d19d2ad902 None admin] PATH_INFO: `/v3/users/glance` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:19.514678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f736c8-672b-4ef8-94f3-38d19d2ad902 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:19.516117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f736c8-672b-4ef8-94f3-38d19d2ad902 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:19.520641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-14f736c8-672b-4ef8-94f3-38d19d2ad902 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:19.528951 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:19.530104 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:19.531208 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:19.531208 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:19.531208 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:19.531208 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:19.531208 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:19.531208 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:19.531208 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:19.531208 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:19.531208 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 20 20:39:19.531208 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:19.535447 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 325/646] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:39:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:19.535756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f736c8-672b-4ef8-94f3-38d19d2ad902 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:19.540268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc40de7-55c4-4d1f-8112-b9f9cf461810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:19.540895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc40de7-55c4-4d1f-8112-b9f9cf461810 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:19.541195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc40de7-55c4-4d1f-8112-b9f9cf461810 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:19.541492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc40de7-55c4-4d1f-8112-b9f9cf461810 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:19.542487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc40de7-55c4-4d1f-8112-b9f9cf461810 None admin] RBAC: Adding query filter params (name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:19.542789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc40de7-55c4-4d1f-8112-b9f9cf461810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:19.544186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc40de7-55c4-4d1f-8112-b9f9cf461810 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:19.553086 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 322/647] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:19.553785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc40de7-55c4-4d1f-8112-b9f9cf461810 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:19.645555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f736c8-672b-4ef8-94f3-38d19d2ad902 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:19.659739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc40de7-55c4-4d1f-8112-b9f9cf461810 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:19.668623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:19.680395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d320611-3dc4-4659-a726-a3d726572676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:19.681173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d320611-3dc4-4659-a726-a3d726572676 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:19.681549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d320611-3dc4-4659-a726-a3d726572676 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:19.681910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d320611-3dc4-4659-a726-a3d726572676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:19.693297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d320611-3dc4-4659-a726-a3d726572676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:19.695036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d320611-3dc4-4659-a726-a3d726572676 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:19.699395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ab7bd-f18d-4ac7-bfd4-dbe7afae9fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:19.700103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ab7bd-f18d-4ac7-bfd4-dbe7afae9fab None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:19.700505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ab7bd-f18d-4ac7-bfd4-dbe7afae9fab None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:19.700866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ab7bd-f18d-4ac7-bfd4-dbe7afae9fab None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:19.702062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ab7bd-f18d-4ac7-bfd4-dbe7afae9fab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:19.702461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ab7bd-f18d-4ac7-bfd4-dbe7afae9fab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:19.704366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ab7bd-f18d-4ac7-bfd4-dbe7afae9fab None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-3d320611-3dc4-4659-a726-a3d726572676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:19.709246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:19.710334 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:19.711231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:19.711231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:19.711231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:19.711231 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 326/648] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:19.711231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d320611-3dc4-4659-a726-a3d726572676 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:19.728282 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 323/649] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:19.728282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ab7bd-f18d-4ac7-bfd4-dbe7afae9fab None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:19.778526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:19.824969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d320611-3dc4-4659-a726-a3d726572676 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:19.839797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ab7bd-f18d-4ac7-bfd4-dbe7afae9fab None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:19.860447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9edea75-0546-48aa-b59b-e6319194c828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:19.861052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edea75-0546-48aa-b59b-e6319194c828 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:19.861326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edea75-0546-48aa-b59b-e6319194c828 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:19.861590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edea75-0546-48aa-b59b-e6319194c828 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:19.862340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9edea75-0546-48aa-b59b-e6319194c828 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:19.862642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9edea75-0546-48aa-b59b-e6319194c828 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:19.864070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9edea75-0546-48aa-b59b-e6319194c828 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:19.878525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272a195-93b2-4bab-90b1-c76ab0a55d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:19.879173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272a195-93b2-4bab-90b1-c76ab0a55d47 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:19.879528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272a195-93b2-4bab-90b1-c76ab0a55d47 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:19.879731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272a195-93b2-4bab-90b1-c76ab0a55d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:19.886326 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 327/650] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Aug 20 20:39:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:19.886326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9edea75-0546-48aa-b59b-e6319194c828 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:19.895208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272a195-93b2-4bab-90b1-c76ab0a55d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:19.897082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272a195-93b2-4bab-90b1-c76ab0a55d47 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-b272a195-93b2-4bab-90b1-c76ab0a55d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:19.913133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:19.914095 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:19.914985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:19.914985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:19.914985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:19.914985 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 324/651] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:19.915208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272a195-93b2-4bab-90b1-c76ab0a55d47 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:19.923561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:19.999250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9edea75-0546-48aa-b59b-e6319194c828 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:20.023145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:20.028785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272a195-93b2-4bab-90b1-c76ab0a55d47 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:20.037722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a1f19-69ae-4a8f-854e-409455ec4299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:20.038331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a1f19-69ae-4a8f-854e-409455ec4299 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:20.038598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a1f19-69ae-4a8f-854e-409455ec4299 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:20.038850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a1f19-69ae-4a8f-854e-409455ec4299 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:20.039483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a1f19-69ae-4a8f-854e-409455ec4299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:20.039786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a1f19-69ae-4a8f-854e-409455ec4299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:20.041181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a1f19-69ae-4a8f-854e-409455ec4299 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:20.062095 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 328/652] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:20.062095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a1f19-69ae-4a8f-854e-409455ec4299 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:20.072270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d0d3a-092f-42f9-a1af-48a8f17c7d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:20.072996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d0d3a-092f-42f9-a1af-48a8f17c7d8c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:20.073978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d0d3a-092f-42f9-a1af-48a8f17c7d8c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:20.074262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d0d3a-092f-42f9-a1af-48a8f17c7d8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:20.074976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d0d3a-092f-42f9-a1af-48a8f17c7d8c None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=df852af59e7743f7a8798d9edf8ebd3e) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:20.075280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d0d3a-092f-42f9-a1af-48a8f17c7d8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:20.076814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d0d3a-092f-42f9-a1af-48a8f17c7d8c None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:20.083757 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 325/653] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:19 2022] GET /identity/v3/role_assignments?user.id=df852af59e7743f7a8798d9edf8ebd3e&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:20.084697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d0d3a-092f-42f9-a1af-48a8f17c7d8c None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:20.197431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:20.209832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a1f19-69ae-4a8f-854e-409455ec4299 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:20.210320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d0d3a-092f-42f9-a1af-48a8f17c7d8c None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:20.255359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99a027-338d-4a85-9acd-2585ecdf5fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:20.256445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99a027-338d-4a85-9acd-2585ecdf5fff None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:20.256866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99a027-338d-4a85-9acd-2585ecdf5fff None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:20.257276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99a027-338d-4a85-9acd-2585ecdf5fff None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:20.258135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6aa43-bad3-4c0e-930a-df1975d3cfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:20.258364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99a027-338d-4a85-9acd-2585ecdf5fff None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:20.258753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6aa43-bad3-4c0e-930a-df1975d3cfce None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:20.258958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99a027-338d-4a85-9acd-2585ecdf5fff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:20.259063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6aa43-bad3-4c0e-930a-df1975d3cfce None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:20.259269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6aa43-bad3-4c0e-930a-df1975d3cfce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:20.261000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99a027-338d-4a85-9acd-2585ecdf5fff None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:20.278760 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 326/654] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:20.279717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99a027-338d-4a85-9acd-2585ecdf5fff None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:20.315716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:20.346184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6aa43-bad3-4c0e-930a-df1975d3cfce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:20.347647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6aa43-bad3-4c0e-930a-df1975d3cfce None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:20.399002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99a027-338d-4a85-9acd-2585ecdf5fff None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:20.434159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e43be-0b25-439f-8dad-41504eda4cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:20.435090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e43be-0b25-439f-8dad-41504eda4cc4 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:20.435362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e43be-0b25-439f-8dad-41504eda4cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:20.435615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e43be-0b25-439f-8dad-41504eda4cc4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:20.442958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e43be-0b25-439f-8dad-41504eda4cc4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:20.444384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e43be-0b25-439f-8dad-41504eda4cc4 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:20.446450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-c99e43be-0b25-439f-8dad-41504eda4cc4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:20.456717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:20.457671 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:20.458624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:20.458624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:20.458624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:20.458624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:20.458624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:20.458624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:20.458624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:20.458624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:20.458624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 20 20:39:20.458624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:20.459167 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 329/655] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:39:20.459167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2206f-7c0b-4959-acba-381e8b78b869 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:20.459167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2206f-7c0b-4959-acba-381e8b78b869 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:20.459385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2206f-7c0b-4959-acba-381e8b78b869 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:20.464665 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 330/656] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:20.464665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2206f-7c0b-4959-acba-381e8b78b869 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:20.464894 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 327/657] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:20.464894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e43be-0b25-439f-8dad-41504eda4cc4 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:20.550063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:20.569217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e43be-0b25-439f-8dad-41504eda4cc4 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:20.580897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2206f-7c0b-4959-acba-381e8b78b869 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:20.604550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fe36b-af51-459a-94a5-41ee5537f684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:20.605171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fe36b-af51-459a-94a5-41ee5537f684 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:20.605445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fe36b-af51-459a-94a5-41ee5537f684 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:20.605709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fe36b-af51-459a-94a5-41ee5537f684 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:20.616778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fe36b-af51-459a-94a5-41ee5537f684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:20.618234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fe36b-af51-459a-94a5-41ee5537f684 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:20.621418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bbad0-705a-4386-9c7b-5d54188da541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:20.622040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bbad0-705a-4386-9c7b-5d54188da541 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:20.622312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bbad0-705a-4386-9c7b-5d54188da541 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:20.622584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bbad0-705a-4386-9c7b-5d54188da541 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:20.630524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640bbad0-705a-4386-9c7b-5d54188da541 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:20.631991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640bbad0-705a-4386-9c7b-5d54188da541 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-c71fe36b-af51-459a-94a5-41ee5537f684 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:20.633832 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:20.634737 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:20.635613 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:20.635613 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:20.635613 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:20.635613 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 328/658] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:20.636541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c08ba-9f58-4edc-8a69-8930693a6fa4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:20.636817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c08ba-9f58-4edc-8a69-8930693a6fa4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:20.637090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c08ba-9f58-4edc-8a69-8930693a6fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-640bbad0-705a-4386-9c7b-5d54188da541 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:20.648739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:20.649690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:20.650612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:20.650612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:20.650612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:20.650612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:20.650612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:20.650612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:20.650612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:20.650612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:20.650612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 20 20:39:20.650612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:20.650612 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 331/659] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:39:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:20.651294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bbad0-705a-4386-9c7b-5d54188da541 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:20.659867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-db3c08ba-9f58-4edc-8a69-8930693a6fa4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:20.661089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-db3c08ba-9f58-4edc-8a69-8930693a6fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:20.708193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:20.748560 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 329/660] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:20.749442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c08ba-9f58-4edc-8a69-8930693a6fa4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:20.757661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bbad0-705a-4386-9c7b-5d54188da541 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:20.804875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e76979-014c-4aba-9038-938f4c7b130d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:20.806192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e76979-014c-4aba-9038-938f4c7b130d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:20.806894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e76979-014c-4aba-9038-938f4c7b130d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:20.807406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e76979-014c-4aba-9038-938f4c7b130d None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:20.808601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e76979-014c-4aba-9038-938f4c7b130d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:20.809100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e76979-014c-4aba-9038-938f4c7b130d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:20.811489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e76979-014c-4aba-9038-938f4c7b130d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:20.823340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:20.838782 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 332/661] 10.222.0.23 () {56 vars in 1139 bytes} [Sat Aug 20 20:39:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:20.840259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e76979-014c-4aba-9038-938f4c7b130d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:20.870135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c08ba-9f58-4edc-8a69-8930693a6fa4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:20.906223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca8eb7-bd33-4e70-96f4-ef74187260e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:20.906978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca8eb7-bd33-4e70-96f4-ef74187260e8 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:20.907361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca8eb7-bd33-4e70-96f4-ef74187260e8 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:20.907691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca8eb7-bd33-4e70-96f4-ef74187260e8 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:20.908476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca8eb7-bd33-4e70-96f4-ef74187260e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:20.908879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca8eb7-bd33-4e70-96f4-ef74187260e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:20.910466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca8eb7-bd33-4e70-96f4-ef74187260e8 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:20.922403 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 330/662] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:20.923280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca8eb7-bd33-4e70-96f4-ef74187260e8 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:20.960058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e76979-014c-4aba-9038-938f4c7b130d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:20.971916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:20.998281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55186806-90da-4079-a429-894edb972939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:20.999191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55186806-90da-4079-a429-894edb972939 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:20.999658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55186806-90da-4079-a429-894edb972939 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:21.000087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55186806-90da-4079-a429-894edb972939 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:21.001089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55186806-90da-4079-a429-894edb972939 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:21.001557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55186806-90da-4079-a429-894edb972939 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:21.003253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55186806-90da-4079-a429-894edb972939 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:21.022993 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 333/663] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Aug 20 20:39:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:21.024310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55186806-90da-4079-a429-894edb972939 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:21.027109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca8eb7-bd33-4e70-96f4-ef74187260e8 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:21.050599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:21.064420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865018d-27a8-4eb9-9259-cd75b88bcfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:21.064727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865018d-27a8-4eb9-9259-cd75b88bcfcc None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:21.065045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865018d-27a8-4eb9-9259-cd75b88bcfcc None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:21.065389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865018d-27a8-4eb9-9259-cd75b88bcfcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:21.136410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55186806-90da-4079-a429-894edb972939 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:21.142023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a865018d-27a8-4eb9-9259-cd75b88bcfcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:21.143532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a865018d-27a8-4eb9-9259-cd75b88bcfcc None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:21.174717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78506a0f-0d41-4d11-9ca4-da4a01c87c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:21.175445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78506a0f-0d41-4d11-9ca4-da4a01c87c91 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:21.175810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78506a0f-0d41-4d11-9ca4-da4a01c87c91 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:21.176172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78506a0f-0d41-4d11-9ca4-da4a01c87c91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:21.183075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78506a0f-0d41-4d11-9ca4-da4a01c87c91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:21.184887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78506a0f-0d41-4d11-9ca4-da4a01c87c91 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-78506a0f-0d41-4d11-9ca4-da4a01c87c91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:21.195836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:21.196966 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:21.198088 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:21.198088 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:21.198088 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:21.198088 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:21.198088 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:21.198088 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:21.198088 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:21.198088 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:21.198088 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 334/664] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:21.198088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78506a0f-0d41-4d11-9ca4-da4a01c87c91 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:21.232765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:21.262439 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 331/665] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:39:21.263307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865018d-27a8-4eb9-9259-cd75b88bcfcc None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:21.291947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:21.311112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78506a0f-0d41-4d11-9ca4-da4a01c87c91 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:21.346350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f48024-4731-4ac8-bbad-fbfea953e0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:21.347057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f48024-4731-4ac8-bbad-fbfea953e0eb None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:21.347324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f48024-4731-4ac8-bbad-fbfea953e0eb None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:21.347583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f48024-4731-4ac8-bbad-fbfea953e0eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:21.370986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865018d-27a8-4eb9-9259-cd75b88bcfcc None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:21.408196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90478f-f657-45ae-8ee0-538baf576bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:21.408849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90478f-f657-45ae-8ee0-538baf576bf1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:21.409120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90478f-f657-45ae-8ee0-538baf576bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:21.409383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90478f-f657-45ae-8ee0-538baf576bf1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:21.414399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f48024-4731-4ac8-bbad-fbfea953e0eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:21.416014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f48024-4731-4ac8-bbad-fbfea953e0eb None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:21.419895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90478f-f657-45ae-8ee0-538baf576bf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-0e90478f-f657-45ae-8ee0-538baf576bf1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:21.424550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:21.425555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 332/666] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:21.426403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90478f-f657-45ae-8ee0-538baf576bf1 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:21.449786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:21.522388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:21.530638 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 335/667] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:39:21.530638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f48024-4731-4ac8-bbad-fbfea953e0eb None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:21.547100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90478f-f657-45ae-8ee0-538baf576bf1 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:21.581452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326889d6-e9c3-48ad-9bd4-651322367b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:21.582200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326889d6-e9c3-48ad-9bd4-651322367b7f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:21.582468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326889d6-e9c3-48ad-9bd4-651322367b7f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:21.582744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326889d6-e9c3-48ad-9bd4-651322367b7f None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:21.583395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326889d6-e9c3-48ad-9bd4-651322367b7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:21.583697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326889d6-e9c3-48ad-9bd4-651322367b7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:21.585150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326889d6-e9c3-48ad-9bd4-651322367b7f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:21.600534 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 333/668] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:21.601350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326889d6-e9c3-48ad-9bd4-651322367b7f None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:21.637315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f48024-4731-4ac8-bbad-fbfea953e0eb None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:21.665005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:21.674045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499c286-6748-4610-a827-ab43953b586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:21.674743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499c286-6748-4610-a827-ab43953b586c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:21.674977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499c286-6748-4610-a827-ab43953b586c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:21.675866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499c286-6748-4610-a827-ab43953b586c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:21.686278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499c286-6748-4610-a827-ab43953b586c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:21.687781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499c286-6748-4610-a827-ab43953b586c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-9499c286-6748-4610-a827-ab43953b586c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:21.702535 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:21.703528 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:21.704451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:21.704451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:21.704451 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:21.704451 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 336/669] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:21.704451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499c286-6748-4610-a827-ab43953b586c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:21.710882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326889d6-e9c3-48ad-9bd4-651322367b7f None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:21.749166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c48e2-b8f5-42f3-a518-cfa81ccb2c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:21.750245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c48e2-b8f5-42f3-a518-cfa81ccb2c35 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:21.750611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c48e2-b8f5-42f3-a518-cfa81ccb2c35 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:21.750967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c48e2-b8f5-42f3-a518-cfa81ccb2c35 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:21.751715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c48e2-b8f5-42f3-a518-cfa81ccb2c35 None admin] RBAC: Adding query filter params (name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:21.752117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c48e2-b8f5-42f3-a518-cfa81ccb2c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:21.753603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c48e2-b8f5-42f3-a518-cfa81ccb2c35 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:21.762088 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 334/670] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:21.762888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c48e2-b8f5-42f3-a518-cfa81ccb2c35 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:21.775884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:21.826411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499c286-6748-4610-a827-ab43953b586c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:21.865617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db138a4-5392-4411-bd1d-825009afd4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:21.866269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db138a4-5392-4411-bd1d-825009afd4b7 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:21.866629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db138a4-5392-4411-bd1d-825009afd4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:21.866952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db138a4-5392-4411-bd1d-825009afd4b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:21.873821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c48e2-b8f5-42f3-a518-cfa81ccb2c35 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:21.878207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db138a4-5392-4411-bd1d-825009afd4b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:21.879646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db138a4-5392-4411-bd1d-825009afd4b7 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-8db138a4-5392-4411-bd1d-825009afd4b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:21.897450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:21.898453 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:21.899342 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:21.899342 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:21.899342 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:21.899342 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 337/671] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:21.899342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db138a4-5392-4411-bd1d-825009afd4b7 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:21.915135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf82049-a2ca-4868-a086-5a7750585037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:21.915721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf82049-a2ca-4868-a086-5a7750585037 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:21.915987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf82049-a2ca-4868-a086-5a7750585037 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:21.916238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf82049-a2ca-4868-a086-5a7750585037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:21.917308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf82049-a2ca-4868-a086-5a7750585037 None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=698d5a7fc34a46ba909152414a0860d7) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:21.917682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf82049-a2ca-4868-a086-5a7750585037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:21.919014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf82049-a2ca-4868-a086-5a7750585037 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:21.932415 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 335/672] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:21 2022] GET /identity/v3/role_assignments?user.id=698d5a7fc34a46ba909152414a0860d7&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:21.932415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf82049-a2ca-4868-a086-5a7750585037 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:21.944596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:22.013313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:22.047790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db138a4-5392-4411-bd1d-825009afd4b7 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:22.052770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf82049-a2ca-4868-a086-5a7750585037 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:22.095407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b8d9d-2b01-408d-b140-c7dd296c99e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:22.096359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b8d9d-2b01-408d-b140-c7dd296c99e6 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:22.096682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfed28-5ce9-4761-8ed1-ddc463c9491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:22.097004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b8d9d-2b01-408d-b140-c7dd296c99e6 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:22.097412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b8d9d-2b01-408d-b140-c7dd296c99e6 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:22.097572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfed28-5ce9-4761-8ed1-ddc463c9491f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:22.097955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfed28-5ce9-4761-8ed1-ddc463c9491f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:22.098315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfed28-5ce9-4761-8ed1-ddc463c9491f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:22.098707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b8d9d-2b01-408d-b140-c7dd296c99e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:22.099147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b8d9d-2b01-408d-b140-c7dd296c99e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:22.101163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b8d9d-2b01-408d-b140-c7dd296c99e6 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:22.112682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfed28-5ce9-4761-8ed1-ddc463c9491f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:22.115055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfed28-5ce9-4761-8ed1-ddc463c9491f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:22.122992 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 338/673] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:22.122992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b8d9d-2b01-408d-b140-c7dd296c99e6 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-38dfed28-5ce9-4761-8ed1-ddc463c9491f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:22.137434 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:22.138448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:22.139344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:22.139344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:22.139344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:22.139344 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 336/674] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:22.139831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfed28-5ce9-4761-8ed1-ddc463c9491f None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:22.258186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:22.270810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfed28-5ce9-4761-8ed1-ddc463c9491f None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:22.270983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b8d9d-2b01-408d-b140-c7dd296c99e6 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:22.299503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:22.310896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0171b1-ea04-4ef0-80af-982d94b5baec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:22.311508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0171b1-ea04-4ef0-80af-982d94b5baec None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:22.311906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0171b1-ea04-4ef0-80af-982d94b5baec None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:22.312134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0171b1-ea04-4ef0-80af-982d94b5baec None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:22.313035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0171b1-ea04-4ef0-80af-982d94b5baec None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:22.313427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0171b1-ea04-4ef0-80af-982d94b5baec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:22.314792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0171b1-ea04-4ef0-80af-982d94b5baec None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:22.321096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2747ce7-0fe8-4dd0-ad78-43140e3c395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:22.322048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2747ce7-0fe8-4dd0-ad78-43140e3c395b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:22.323787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2747ce7-0fe8-4dd0-ad78-43140e3c395b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:22.324263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2747ce7-0fe8-4dd0-ad78-43140e3c395b None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:22.328306 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 339/675] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:22.329388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0171b1-ea04-4ef0-80af-982d94b5baec None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:22.335321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2747ce7-0fe8-4dd0-ad78-43140e3c395b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:22.337069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2747ce7-0fe8-4dd0-ad78-43140e3c395b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-b2747ce7-0fe8-4dd0-ad78-43140e3c395b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:22.354009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:22.355553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:22.356513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:22.356513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:22.356513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:22.356513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:22.356513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:22.356513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:22.356513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:22.356513 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:22.358046 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 337/676] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:22.360538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2747ce7-0fe8-4dd0-ad78-43140e3c395b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:22.443625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0171b1-ea04-4ef0-80af-982d94b5baec None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:22.481289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d724642-f6d6-48e5-b752-0f72a071b8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:22.481880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d724642-f6d6-48e5-b752-0f72a071b8a9 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:22.482151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d724642-f6d6-48e5-b752-0f72a071b8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:22.482414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d724642-f6d6-48e5-b752-0f72a071b8a9 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:22.483050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d724642-f6d6-48e5-b752-0f72a071b8a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:22.483211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2747ce7-0fe8-4dd0-ad78-43140e3c395b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:22.483558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d724642-f6d6-48e5-b752-0f72a071b8a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:22.485089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d724642-f6d6-48e5-b752-0f72a071b8a9 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:22.502699 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 340/677] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:22.502699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d724642-f6d6-48e5-b752-0f72a071b8a9 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:22.517188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:22.527010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c03b2-e186-4c12-9bc5-00e1c3c4c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:22.527670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c03b2-e186-4c12-9bc5-00e1c3c4c22f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:22.527941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c03b2-e186-4c12-9bc5-00e1c3c4c22f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:22.528177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c03b2-e186-4c12-9bc5-00e1c3c4c22f None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:22.535172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c03b2-e186-4c12-9bc5-00e1c3c4c22f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:22.536636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c03b2-e186-4c12-9bc5-00e1c3c4c22f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-bd0c03b2-e186-4c12-9bc5-00e1c3c4c22f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:22.549114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:22.550077 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:22.550986 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:22.550986 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:22.550986 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:22.550986 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:22.550986 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:22.550986 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:22.550986 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:22.550986 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:22.550986 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 338/678] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:22.551484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcd482-be7f-4a41-90b3-ee436cfe7cad None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:22.552023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcd482-be7f-4a41-90b3-ee436cfe7cad None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:22.552023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcd482-be7f-4a41-90b3-ee436cfe7cad None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:22.553226 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 339/679] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:22.553741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcd482-be7f-4a41-90b3-ee436cfe7cad None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:22.605897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:22.619971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d724642-f6d6-48e5-b752-0f72a071b8a9 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:22.662575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e459bc8-f607-4e74-86f0-0dda33d1fcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:22.663188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e459bc8-f607-4e74-86f0-0dda33d1fcdc None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:22.663458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e459bc8-f607-4e74-86f0-0dda33d1fcdc None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:22.663724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e459bc8-f607-4e74-86f0-0dda33d1fcdc None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:22.664434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e459bc8-f607-4e74-86f0-0dda33d1fcdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:22.664823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e459bc8-f607-4e74-86f0-0dda33d1fcdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:22.666224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e459bc8-f607-4e74-86f0-0dda33d1fcdc None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:22.668182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcd482-be7f-4a41-90b3-ee436cfe7cad None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:22.682591 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 341/680] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:22.682591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e459bc8-f607-4e74-86f0-0dda33d1fcdc None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:22.717682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575561c-27a2-48f2-b12f-42e8f2e94d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:22.718334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575561c-27a2-48f2-b12f-42e8f2e94d1e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:22.718609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575561c-27a2-48f2-b12f-42e8f2e94d1e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:22.718877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575561c-27a2-48f2-b12f-42e8f2e94d1e None admin] PATH_INFO: `/v3/users/swift` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:22.726458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575561c-27a2-48f2-b12f-42e8f2e94d1e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:22.727982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575561c-27a2-48f2-b12f-42e8f2e94d1e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-3575561c-27a2-48f2-b12f-42e8f2e94d1e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:22.739639 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:22.740616 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:22.741621 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:22.741621 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:22.741621 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:22.741621 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:22.741621 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:22.741621 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:22.741621 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:22.741621 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:22.741621 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 20 20:39:22.741621 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:22.741621 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 340/681] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:39:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:22.742241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50424f4-6bd7-4756-aac3-b59f1e1096c2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:22.742831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50424f4-6bd7-4756-aac3-b59f1e1096c2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:22.742831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50424f4-6bd7-4756-aac3-b59f1e1096c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:22.753284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:22.761732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c50424f4-6bd7-4756-aac3-b59f1e1096c2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:22.762922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c50424f4-6bd7-4756-aac3-b59f1e1096c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:22.786448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e459bc8-f607-4e74-86f0-0dda33d1fcdc None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:22.826175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c754f-f7c2-4e3a-baf5-f8ca2749d90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:22.827007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c754f-f7c2-4e3a-baf5-f8ca2749d90c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:22.827431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c754f-f7c2-4e3a-baf5-f8ca2749d90c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:22.827688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c754f-f7c2-4e3a-baf5-f8ca2749d90c None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:22.828365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c754f-f7c2-4e3a-baf5-f8ca2749d90c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:22.828705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c754f-f7c2-4e3a-baf5-f8ca2749d90c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:22.830169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c754f-f7c2-4e3a-baf5-f8ca2749d90c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:22.835451 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 341/682] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:22.836472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50424f4-6bd7-4756-aac3-b59f1e1096c2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:22.850445 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 342/683] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:22.850445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c754f-f7c2-4e3a-baf5-f8ca2749d90c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:22.886039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:22.959219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50424f4-6bd7-4756-aac3-b59f1e1096c2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:22.973267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c754f-f7c2-4e3a-baf5-f8ca2749d90c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:22.996471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548efc0f-e9f9-4b5b-8a3b-178eea231457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:22.997162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548efc0f-e9f9-4b5b-8a3b-178eea231457 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:22.997497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548efc0f-e9f9-4b5b-8a3b-178eea231457 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:22.997823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548efc0f-e9f9-4b5b-8a3b-178eea231457 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:22.998602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548efc0f-e9f9-4b5b-8a3b-178eea231457 None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=22bb014e04774c278c99496bf64b2319) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:22.999021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548efc0f-e9f9-4b5b-8a3b-178eea231457 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:23.000736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548efc0f-e9f9-4b5b-8a3b-178eea231457 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:23.007468 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 342/684] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:22 2022] GET /identity/v3/role_assignments?user.id=22bb014e04774c278c99496bf64b2319&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:23.008461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548efc0f-e9f9-4b5b-8a3b-178eea231457 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:23.011324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:23.031231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a1e4a-a4f5-450d-8bcb-5d8b30ed7a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:23.032301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a1e4a-a4f5-450d-8bcb-5d8b30ed7a4e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:23.032788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a1e4a-a4f5-450d-8bcb-5d8b30ed7a4e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:23.033243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a1e4a-a4f5-450d-8bcb-5d8b30ed7a4e None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:23.034235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a1e4a-a4f5-450d-8bcb-5d8b30ed7a4e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:23.036822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a1e4a-a4f5-450d-8bcb-5d8b30ed7a4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:23.038629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a1e4a-a4f5-450d-8bcb-5d8b30ed7a4e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:23.081221 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 343/685] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Aug 20 20:39:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:23.084834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a1e4a-a4f5-450d-8bcb-5d8b30ed7a4e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:23.217389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:23.228982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548efc0f-e9f9-4b5b-8a3b-178eea231457 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:23.265740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a1e4a-a4f5-450d-8bcb-5d8b30ed7a4e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:23.271571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2e666-27bb-405a-b5a6-8cb7711881c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:23.272169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2e666-27bb-405a-b5a6-8cb7711881c8 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:23.273114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2e666-27bb-405a-b5a6-8cb7711881c8 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:23.273114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2e666-27bb-405a-b5a6-8cb7711881c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:23.273812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2e666-27bb-405a-b5a6-8cb7711881c8 None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=53928836828d45fbb427dd92492cae20) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:23.273812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2e666-27bb-405a-b5a6-8cb7711881c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:23.275161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2e666-27bb-405a-b5a6-8cb7711881c8 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:23.283725 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 343/686] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:23 2022] GET /identity/v3/role_assignments?user.id=53928836828d45fbb427dd92492cae20&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:23.285197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2e666-27bb-405a-b5a6-8cb7711881c8 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:23.306078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041e35-31e9-4649-9be7-2e9be9c398cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:23.306706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041e35-31e9-4649-9be7-2e9be9c398cb None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:23.307077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041e35-31e9-4649-9be7-2e9be9c398cb None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:23.308416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041e35-31e9-4649-9be7-2e9be9c398cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:23.385251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:23.393801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b041e35-31e9-4649-9be7-2e9be9c398cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:23.395793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b041e35-31e9-4649-9be7-2e9be9c398cb None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:23.454595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2e666-27bb-405a-b5a6-8cb7711881c8 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:23.510429 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 344/687] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:39:23.515919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041e35-31e9-4649-9be7-2e9be9c398cb None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:23.549760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1997a-1b2f-46cc-a278-f419058b0fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:23.550413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1997a-1b2f-46cc-a278-f419058b0fab None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:23.550685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1997a-1b2f-46cc-a278-f419058b0fab None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:23.550946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1997a-1b2f-46cc-a278-f419058b0fab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:23.590053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1997a-1b2f-46cc-a278-f419058b0fab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-4bd1997a-1b2f-46cc-a278-f419058b0fab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:23.596766 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:23.597873 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:23.598897 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:23.598897 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:23.598897 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:23.598897 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:23.598897 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:23.598897 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:23.598897 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:23.598897 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:23.598897 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:23.598897 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:23.598897 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:39:23.598897 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:23.598897 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 344/688] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:39:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:23.603857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1997a-1b2f-46cc-a278-f419058b0fab None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:23.634642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:23.638405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:23.652101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041e35-31e9-4649-9be7-2e9be9c398cb None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:23.712759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023d127-0f73-481c-ae11-c4f10bc1af4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:23.713649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023d127-0f73-481c-ae11-c4f10bc1af4d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:23.714073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023d127-0f73-481c-ae11-c4f10bc1af4d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:23.714435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023d127-0f73-481c-ae11-c4f10bc1af4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:23.726190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1997a-1b2f-46cc-a278-f419058b0fab None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:23.730272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023d127-0f73-481c-ae11-c4f10bc1af4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:23.731835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023d127-0f73-481c-ae11-c4f10bc1af4d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-5023d127-0f73-481c-ae11-c4f10bc1af4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:23.750374 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:23.751587 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:23.752714 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:23.752714 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:23.752714 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:23.754097 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 345/689] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:23.759935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023d127-0f73-481c-ae11-c4f10bc1af4d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:23.768693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de318b-374d-458f-b7b2-d5a4cbb0dc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:23.769315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de318b-374d-458f-b7b2-d5a4cbb0dc2a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:23.769578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de318b-374d-458f-b7b2-d5a4cbb0dc2a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:23.769842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de318b-374d-458f-b7b2-d5a4cbb0dc2a None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:23.770507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de318b-374d-458f-b7b2-d5a4cbb0dc2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:23.770805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de318b-374d-458f-b7b2-d5a4cbb0dc2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:23.772176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de318b-374d-458f-b7b2-d5a4cbb0dc2a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:23.781449 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 345/690] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:39:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:23.790084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de318b-374d-458f-b7b2-d5a4cbb0dc2a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:23.874405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023d127-0f73-481c-ae11-c4f10bc1af4d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:23.947843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcba46-5fd8-4bb7-b3cd-c84a2af218fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:23.948362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcba46-5fd8-4bb7-b3cd-c84a2af218fe None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:23.948672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcba46-5fd8-4bb7-b3cd-c84a2af218fe None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:23.948941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcba46-5fd8-4bb7-b3cd-c84a2af218fe None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:23.949236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:23.949681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dcba46-5fd8-4bb7-b3cd-c84a2af218fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:23.949868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dcba46-5fd8-4bb7-b3cd-c84a2af218fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:23.952317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dcba46-5fd8-4bb7-b3cd-c84a2af218fe None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:23.978694 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 346/691] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:23.982508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de318b-374d-458f-b7b2-d5a4cbb0dc2a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:23.985297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcba46-5fd8-4bb7-b3cd-c84a2af218fe None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:23.988040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:24.073184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cd505-f904-4281-94f5-ca9754bbf240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:24.074224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cd505-f904-4281-94f5-ca9754bbf240 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:24.074695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cd505-f904-4281-94f5-ca9754bbf240 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:24.075153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cd505-f904-4281-94f5-ca9754bbf240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:24.096057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cd505-f904-4281-94f5-ca9754bbf240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:24.097839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cd505-f904-4281-94f5-ca9754bbf240 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:24.108151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcba46-5fd8-4bb7-b3cd-c84a2af218fe None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-431cd505-f904-4281-94f5-ca9754bbf240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:24.124422 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:24.125483 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:24.126355 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:24.126355 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:24.126355 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:24.132859 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 346/692] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:24.138495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cd505-f904-4281-94f5-ca9754bbf240 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:24.143501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f5729-1998-41cf-aa46-9778a205aa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:24.144134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f5729-1998-41cf-aa46-9778a205aa54 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:24.144419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f5729-1998-41cf-aa46-9778a205aa54 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:24.144686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f5729-1998-41cf-aa46-9778a205aa54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:24.152013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f5729-1998-41cf-aa46-9778a205aa54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:24.156393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f5729-1998-41cf-aa46-9778a205aa54 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-e79f5729-1998-41cf-aa46-9778a205aa54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:24.164834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:24.165739 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:24.167142 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:24.167142 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:24.167142 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:24.167142 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:24.167142 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:24.167142 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:24.167142 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:24.167142 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:24.167142 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 347/693] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:24.172196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f5729-1998-41cf-aa46-9778a205aa54 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:24.237060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:24.278180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f5729-1998-41cf-aa46-9778a205aa54 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:24.313615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ca32f-1d1a-47ca-afd0-d876254df1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:24.314220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ca32f-1d1a-47ca-afd0-d876254df1fa None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:24.314487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ca32f-1d1a-47ca-afd0-d876254df1fa None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:24.314747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ca32f-1d1a-47ca-afd0-d876254df1fa None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:24.315424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ca32f-1d1a-47ca-afd0-d876254df1fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:24.315717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ca32f-1d1a-47ca-afd0-d876254df1fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:24.317273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ca32f-1d1a-47ca-afd0-d876254df1fa None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:24.330315 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 348/694] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:24.336776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ca32f-1d1a-47ca-afd0-d876254df1fa None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:24.403073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cd505-f904-4281-94f5-ca9754bbf240 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:24.447762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ca32f-1d1a-47ca-afd0-d876254df1fa None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:24.473807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:24.474083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c829c21-ace4-4b3a-b95e-98d77bb0f66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:24.474684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c829c21-ace4-4b3a-b95e-98d77bb0f66a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:24.474962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c829c21-ace4-4b3a-b95e-98d77bb0f66a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:24.475277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c829c21-ace4-4b3a-b95e-98d77bb0f66a None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:24.475924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c829c21-ace4-4b3a-b95e-98d77bb0f66a None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:24.476332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c829c21-ace4-4b3a-b95e-98d77bb0f66a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:24.478871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c829c21-ace4-4b3a-b95e-98d77bb0f66a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:24.486996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6a8f7-4aba-411d-b3f5-b8cb47a67613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:24.487653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6a8f7-4aba-411d-b3f5-b8cb47a67613 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:24.487865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6a8f7-4aba-411d-b3f5-b8cb47a67613 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:24.488122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6a8f7-4aba-411d-b3f5-b8cb47a67613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:24.488876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6a8f7-4aba-411d-b3f5-b8cb47a67613 None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=d74f7cdc155a4397ad28a3fbca86f7f9) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:24.489168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6a8f7-4aba-411d-b3f5-b8cb47a67613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:24.489409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:24.490571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6a8f7-4aba-411d-b3f5-b8cb47a67613 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:24.497363 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 349/695] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:24 2022] GET /identity/v3/role_assignments?user.id=d74f7cdc155a4397ad28a3fbca86f7f9&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:24.502231 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 347/696] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:24.507888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6a8f7-4aba-411d-b3f5-b8cb47a67613 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:24.564035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95959fdd-f339-40fe-9266-e0f58ac4942a None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:24.564418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95959fdd-f339-40fe-9266-e0f58ac4942a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:24.564587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95959fdd-f339-40fe-9266-e0f58ac4942a None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:24.567624 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 348/697] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:24.572659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f770a-cbfb-4f57-9b4c-339daefb06af None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:24.573133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f770a-cbfb-4f57-9b4c-339daefb06af None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:24.573366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f770a-cbfb-4f57-9b4c-339daefb06af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:24.620671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b90f770a-cbfb-4f57-9b4c-339daefb06af None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:24.621941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b90f770a-cbfb-4f57-9b4c-339daefb06af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:24.631162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6a8f7-4aba-411d-b3f5-b8cb47a67613 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:24.667286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff27cc-5ecd-4ce1-8bb8-c7e0abe8637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:24.667935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff27cc-5ecd-4ce1-8bb8-c7e0abe8637a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:24.668200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff27cc-5ecd-4ce1-8bb8-c7e0abe8637a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:24.668492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff27cc-5ecd-4ce1-8bb8-c7e0abe8637a None admin] PATH_INFO: `/v3/users/nova` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:24.677192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff27cc-5ecd-4ce1-8bb8-c7e0abe8637a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:24.678681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff27cc-5ecd-4ce1-8bb8-c7e0abe8637a None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-e6ff27cc-5ecd-4ce1-8bb8-c7e0abe8637a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:24.691432 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:24.692466 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:24.693391 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:24.693391 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:24.693391 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:24.693391 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:24.693391 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:24.693391 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:24.693391 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:24.693391 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:24.693391 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:39:24.693391 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:24.693391 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 350/698] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Aug 20 20:39:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:24.700190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff27cc-5ecd-4ce1-8bb8-c7e0abe8637a None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:24.758089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:24.781707 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 349/699] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 20:39:24.817906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff27cc-5ecd-4ce1-8bb8-c7e0abe8637a None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:24.852424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f649d-e4d2-4d87-9f76-37fe0b3a16e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:24.852950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f649d-e4d2-4d87-9f76-37fe0b3a16e0 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:24.853218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f649d-e4d2-4d87-9f76-37fe0b3a16e0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:24.853477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f649d-e4d2-4d87-9f76-37fe0b3a16e0 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:24.854661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f649d-e4d2-4d87-9f76-37fe0b3a16e0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:24.854962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f649d-e4d2-4d87-9f76-37fe0b3a16e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:24.856415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f649d-e4d2-4d87-9f76-37fe0b3a16e0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:24.874785 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 351/700] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Aug 20 20:39:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:24.880434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f770a-cbfb-4f57-9b4c-339daefb06af None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:24.993825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f770a-cbfb-4f57-9b4c-339daefb06af None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:25.019635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:25.038441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f389657-5482-442c-a2ab-4af53e2e6cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:25.039049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f389657-5482-442c-a2ab-4af53e2e6cea None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:25.039319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f389657-5482-442c-a2ab-4af53e2e6cea None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:25.039581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f389657-5482-442c-a2ab-4af53e2e6cea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:25.115122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f389657-5482-442c-a2ab-4af53e2e6cea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:25.116660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f389657-5482-442c-a2ab-4af53e2e6cea None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:25.179777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f649d-e4d2-4d87-9f76-37fe0b3a16e0 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:25.228331 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 350/701] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:39:25.240748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f389657-5482-442c-a2ab-4af53e2e6cea None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:25.285881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:25.316169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f649d-e4d2-4d87-9f76-37fe0b3a16e0 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:25.319141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:25.356217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2b963-b56c-4e95-878c-8c18f0d456ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:25.357370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2b963-b56c-4e95-878c-8c18f0d456ed None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:25.357841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2b963-b56c-4e95-878c-8c18f0d456ed None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:25.358337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2b963-b56c-4e95-878c-8c18f0d456ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:25.369471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2b963-b56c-4e95-878c-8c18f0d456ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:25.371347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2b963-b56c-4e95-878c-8c18f0d456ed None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:25.381852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f389657-5482-442c-a2ab-4af53e2e6cea None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-9bf2b963-b56c-4e95-878c-8c18f0d456ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:25.387936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:25.388917 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:25.389885 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:25.389885 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:25.389885 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:25.392008 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 352/702] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:25.397615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2b963-b56c-4e95-878c-8c18f0d456ed None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:25.437194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657934fb-f04a-4bf2-8098-741d2a04cdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:25.437852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657934fb-f04a-4bf2-8098-741d2a04cdfc None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:25.438124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657934fb-f04a-4bf2-8098-741d2a04cdfc None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:25.438386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657934fb-f04a-4bf2-8098-741d2a04cdfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:25.458420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657934fb-f04a-4bf2-8098-741d2a04cdfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:25.459871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657934fb-f04a-4bf2-8098-741d2a04cdfc None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-657934fb-f04a-4bf2-8098-741d2a04cdfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:25.496309 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:25.497365 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:25.498249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:25.498249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:25.498249 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:25.498249 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 351/703] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:25.504339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657934fb-f04a-4bf2-8098-741d2a04cdfc None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:25.566277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2b963-b56c-4e95-878c-8c18f0d456ed None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:25.601942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094e743-1ca3-46fe-8a70-b4060ecd40c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:25.602543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094e743-1ca3-46fe-8a70-b4060ecd40c3 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:25.602809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094e743-1ca3-46fe-8a70-b4060ecd40c3 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:25.603068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094e743-1ca3-46fe-8a70-b4060ecd40c3 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:25.603728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094e743-1ca3-46fe-8a70-b4060ecd40c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:25.604048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094e743-1ca3-46fe-8a70-b4060ecd40c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:25.605447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094e743-1ca3-46fe-8a70-b4060ecd40c3 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:25.618232 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 353/704] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:25.619439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8470f-00dc-4a0a-a159-0600ed93ccb3 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:25.619730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8470f-00dc-4a0a-a159-0600ed93ccb3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:25.619986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8470f-00dc-4a0a-a159-0600ed93ccb3 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:25.630048 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 354/705] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:25.630048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8470f-00dc-4a0a-a159-0600ed93ccb3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:25.645902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:25.653931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:25.677264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657934fb-f04a-4bf2-8098-741d2a04cdfc None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:25.723342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3db518-47ea-4e24-ac0f-bc14cec71e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:25.724824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3db518-47ea-4e24-ac0f-bc14cec71e2f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:25.725111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3db518-47ea-4e24-ac0f-bc14cec71e2f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:25.725372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3db518-47ea-4e24-ac0f-bc14cec71e2f None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:25.726005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3db518-47ea-4e24-ac0f-bc14cec71e2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:25.726298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3db518-47ea-4e24-ac0f-bc14cec71e2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:25.727662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3db518-47ea-4e24-ac0f-bc14cec71e2f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:25.743956 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 352/706] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:25.745332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebbe07-bcef-4492-8cbb-951d0cbc1735 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:25.745606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebbe07-bcef-4492-8cbb-951d0cbc1735 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:25.745869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebbe07-bcef-4492-8cbb-951d0cbc1735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:25.764312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8470f-00dc-4a0a-a159-0600ed93ccb3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:25.768591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f0ebbe07-bcef-4492-8cbb-951d0cbc1735 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:25.769764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f0ebbe07-bcef-4492-8cbb-951d0cbc1735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:25.810633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab01ad-eb3d-4668-be75-7bf3dd1994dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:25.811769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab01ad-eb3d-4668-be75-7bf3dd1994dc None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:25.812306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab01ad-eb3d-4668-be75-7bf3dd1994dc None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:25.812902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab01ad-eb3d-4668-be75-7bf3dd1994dc None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:25.813827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab01ad-eb3d-4668-be75-7bf3dd1994dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:25.815612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab01ad-eb3d-4668-be75-7bf3dd1994dc None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:25.844333 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 355/707] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:39:25 2022] POST /identity/v3/users => generated 254 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:39:25.845427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab01ad-eb3d-4668-be75-7bf3dd1994dc None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:25.868668 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 353/708] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:25.955441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:25.989101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab01ad-eb3d-4668-be75-7bf3dd1994dc None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:26.027804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8ed66-6004-4fc9-b328-c245a6f4f778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:26.028790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8ed66-6004-4fc9-b328-c245a6f4f778 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:26.029245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8ed66-6004-4fc9-b328-c245a6f4f778 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:26.029688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8ed66-6004-4fc9-b328-c245a6f4f778 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:26.037805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8ed66-6004-4fc9-b328-c245a6f4f778 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:26.039675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8ed66-6004-4fc9-b328-c245a6f4f778 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-19b8ed66-6004-4fc9-b328-c245a6f4f778 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:26.052362 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:26.053385 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:26.054332 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:26.054332 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:26.054332 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:26.054332 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:26.054332 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:26.054332 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:26.054332 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:26.054332 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:26.056710 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 356/709] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:26.064423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ebbe07-bcef-4492-8cbb-951d0cbc1735 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:26.140896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:26.179474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ebbe07-bcef-4492-8cbb-951d0cbc1735 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:26.240918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c383063-ce95-4e15-9893-2dd7332acdc1 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:26.241444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c383063-ce95-4e15-9893-2dd7332acdc1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:26.241444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c383063-ce95-4e15-9893-2dd7332acdc1 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:26.243169 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 357/710] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:26.249941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6bc9e-0f2e-4d6a-9596-b7e94670c030 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:26.250256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6bc9e-0f2e-4d6a-9596-b7e94670c030 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:26.250580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6bc9e-0f2e-4d6a-9596-b7e94670c030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:26.271259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-58e6bc9e-0f2e-4d6a-9596-b7e94670c030 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:26.272586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-58e6bc9e-0f2e-4d6a-9596-b7e94670c030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:26.347620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbf630-2933-4d3c-a3c1-adb3131cc532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:26.348226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbf630-2933-4d3c-a3c1-adb3131cc532 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:26.348581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbf630-2933-4d3c-a3c1-adb3131cc532 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:26.348806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbf630-2933-4d3c-a3c1-adb3131cc532 None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:26.349508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbf630-2933-4d3c-a3c1-adb3131cc532 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:26.349772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbf630-2933-4d3c-a3c1-adb3131cc532 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:26.351277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbf630-2933-4d3c-a3c1-adb3131cc532 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:26.352308 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 358/711] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:26.360128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6bc9e-0f2e-4d6a-9596-b7e94670c030 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:26.371700 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 354/712] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:26.377233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbf630-2933-4d3c-a3c1-adb3131cc532 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:26.472385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:26.516156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6bc9e-0f2e-4d6a-9596-b7e94670c030 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:26.551145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbf630-2933-4d3c-a3c1-adb3131cc532 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:26.558475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ad56f-f2a3-4a70-abc0-141c28fa9a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:26.559417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ad56f-f2a3-4a70-abc0-141c28fa9a26 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:26.559691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ad56f-f2a3-4a70-abc0-141c28fa9a26 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:26.559949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ad56f-f2a3-4a70-abc0-141c28fa9a26 None admin] PATH_INFO: `/v3/services/network` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:26.560569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ad56f-f2a3-4a70-abc0-141c28fa9a26 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:26.562006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ad56f-f2a3-4a70-abc0-141c28fa9a26 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-d65ad56f-f2a3-4a70-abc0-141c28fa9a26 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:26.575006 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:26.576113 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:26.577457 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 359/713] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:26 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:26.589005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ad56f-f2a3-4a70-abc0-141c28fa9a26 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:26.614514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:26.641098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8f42e-c700-4e07-82f8-1bae2f595db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:26.641871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8f42e-c700-4e07-82f8-1bae2f595db7 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:26.642455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8f42e-c700-4e07-82f8-1bae2f595db7 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:26.642573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8f42e-c700-4e07-82f8-1bae2f595db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:26.643478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8f42e-c700-4e07-82f8-1bae2f595db7 None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=df852af59e7743f7a8798d9edf8ebd3e) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:26.643910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8f42e-c700-4e07-82f8-1bae2f595db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:26.645426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8f42e-c700-4e07-82f8-1bae2f595db7 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:26.662788 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 355/714] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:26 2022] GET /identity/v3/role_assignments?user.id=df852af59e7743f7a8798d9edf8ebd3e&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:26.703154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ad56f-f2a3-4a70-abc0-141c28fa9a26 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:26.732498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:26.750083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8e7d-6db7-4659-bfcb-5544571ed701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:26.751110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8e7d-6db7-4659-bfcb-5544571ed701 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:26.751525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8e7d-6db7-4659-bfcb-5544571ed701 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:26.751927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8e7d-6db7-4659-bfcb-5544571ed701 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:26.752742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe8e7d-6db7-4659-bfcb-5544571ed701 None admin] RBAC: Adding query filter params (name=network) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:26.753191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe8e7d-6db7-4659-bfcb-5544571ed701 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:26.754783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe8e7d-6db7-4659-bfcb-5544571ed701 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:26.761356 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 360/715] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Aug 20 20:39:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:26.767183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8f42e-c700-4e07-82f8-1bae2f595db7 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:26.777528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8e7d-6db7-4659-bfcb-5544571ed701 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:26.892885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8e7d-6db7-4659-bfcb-5544571ed701 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:26.940571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a76ebe-2266-4c9e-a52a-62f34fa819c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:26.941534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a76ebe-2266-4c9e-a52a-62f34fa819c1 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:26.941947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a76ebe-2266-4c9e-a52a-62f34fa819c1 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:26.942363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a76ebe-2266-4c9e-a52a-62f34fa819c1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:26.943082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a76ebe-2266-4c9e-a52a-62f34fa819c1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:26.944744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a76ebe-2266-4c9e-a52a-62f34fa819c1 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:26.953177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8f42e-c700-4e07-82f8-1bae2f595db7 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-a7a76ebe-2266-4c9e-a52a-62f34fa819c1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:26.955779 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:26.956940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 20 20:39:26.957771 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:26.960450 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 361/716] 10.222.0.23 () {56 vars in 1129 bytes} [Sat Aug 20 20:39:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:26.966356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a76ebe-2266-4c9e-a52a-62f34fa819c1 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:27.017757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a915c5f-cbcf-405e-b2c1-238311e6c6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:27.018299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915c5f-cbcf-405e-b2c1-238311e6c6b4 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:27.018572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915c5f-cbcf-405e-b2c1-238311e6c6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:27.018840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915c5f-cbcf-405e-b2c1-238311e6c6b4 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:27.019468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a915c5f-cbcf-405e-b2c1-238311e6c6b4 None admin] RBAC: Adding query filter params (type=network) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:27.019764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a915c5f-cbcf-405e-b2c1-238311e6c6b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:27.021155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a915c5f-cbcf-405e-b2c1-238311e6c6b4 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:27.027568 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 356/717] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Aug 20 20:39:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:27.044253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:27.101630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a76ebe-2266-4c9e-a52a-62f34fa819c1 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:27.158020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8afead-86d9-440a-9fd4-676f9685b274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:27.159017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8afead-86d9-440a-9fd4-676f9685b274 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:27.159513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8afead-86d9-440a-9fd4-676f9685b274 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:27.159972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8afead-86d9-440a-9fd4-676f9685b274 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:27.160818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8afead-86d9-440a-9fd4-676f9685b274 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:27.161315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8afead-86d9-440a-9fd4-676f9685b274 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:27.162868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8afead-86d9-440a-9fd4-676f9685b274 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:27.170717 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 362/718] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Aug 20 20:39:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:27.176718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a915c5f-cbcf-405e-b2c1-238311e6c6b4 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:27.264761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:27.333781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a915c5f-cbcf-405e-b2c1-238311e6c6b4 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:27.376400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e73a6-9a6d-42b7-999e-5253cd4e346c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:27.376400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e73a6-9a6d-42b7-999e-5253cd4e346c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:27.376400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e73a6-9a6d-42b7-999e-5253cd4e346c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:27.376400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e73a6-9a6d-42b7-999e-5253cd4e346c None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:27.376835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e73a6-9a6d-42b7-999e-5253cd4e346c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:27.376835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e73a6-9a6d-42b7-999e-5253cd4e346c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:27.377790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e73a6-9a6d-42b7-999e-5253cd4e346c None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:27.384119 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 357/719] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Aug 20 20:39:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:27.429550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68850358-538b-4560-8d57-8eadf2119259 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:27.430490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68850358-538b-4560-8d57-8eadf2119259 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:27.430957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68850358-538b-4560-8d57-8eadf2119259 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:27.433122 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 363/720] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:27.438957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5d9bf-5246-4369-9c7b-37a2d241cad3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:27.439301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5d9bf-5246-4369-9c7b-37a2d241cad3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:27.439499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5d9bf-5246-4369-9c7b-37a2d241cad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:27.465664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-64e5d9bf-5246-4369-9c7b-37a2d241cad3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:27.466877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-64e5d9bf-5246-4369-9c7b-37a2d241cad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:27.556272 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 358/721] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:27.915231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68850358-538b-4560-8d57-8eadf2119259 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:28.035303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68850358-538b-4560-8d57-8eadf2119259 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:28.057964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:28.081116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529f817-ca9c-402d-9bdd-240585556534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:28.081750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529f817-ca9c-402d-9bdd-240585556534 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:28.082025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529f817-ca9c-402d-9bdd-240585556534 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:28.082290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529f817-ca9c-402d-9bdd-240585556534 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:28.098302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529f817-ca9c-402d-9bdd-240585556534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-5529f817-ca9c-402d-9bdd-240585556534 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:28.103012 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:28.104112 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:28.105108 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:28.105108 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:28.105108 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:28.105108 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:28.105108 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:28.105108 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:28.105108 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:28.105108 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:28.105108 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:28.105108 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:28.105108 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:39:28.105108 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:28.105108 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 364/722] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:39:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:28.112003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5d9bf-5246-4369-9c7b-37a2d241cad3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:28.174437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:28.246273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5d9bf-5246-4369-9c7b-37a2d241cad3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:28.295215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa3a68-20dc-4d13-bcca-999b319da207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:28.295930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa3a68-20dc-4d13-bcca-999b319da207 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:28.296236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa3a68-20dc-4d13-bcca-999b319da207 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:28.296532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa3a68-20dc-4d13-bcca-999b319da207 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:28.297196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa3a68-20dc-4d13-bcca-999b319da207 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:28.297492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa3a68-20dc-4d13-bcca-999b319da207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:28.298861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa3a68-20dc-4d13-bcca-999b319da207 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:28.312906 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 359/723] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:39:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:28.318849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529f817-ca9c-402d-9bdd-240585556534 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:28.457458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529f817-ca9c-402d-9bdd-240585556534 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:28.504997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14082d1-8f03-4091-bf92-276a9b86169f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:28.505629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14082d1-8f03-4091-bf92-276a9b86169f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:28.505900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14082d1-8f03-4091-bf92-276a9b86169f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:28.506167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14082d1-8f03-4091-bf92-276a9b86169f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:28.517139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14082d1-8f03-4091-bf92-276a9b86169f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:28.518586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14082d1-8f03-4091-bf92-276a9b86169f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-e14082d1-8f03-4091-bf92-276a9b86169f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:28.532861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:28.535039 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:28.537084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:28.537084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:28.537084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:28.537084 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 365/724] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:28.540036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa3a68-20dc-4d13-bcca-999b319da207 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:28.680405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa3a68-20dc-4d13-bcca-999b319da207 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:28.693585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:28.724584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15128bbf-8d18-4e3d-b230-9dbe0ebb14a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:28.725191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15128bbf-8d18-4e3d-b230-9dbe0ebb14a7 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:28.725458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15128bbf-8d18-4e3d-b230-9dbe0ebb14a7 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:28.725720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15128bbf-8d18-4e3d-b230-9dbe0ebb14a7 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:28.726352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15128bbf-8d18-4e3d-b230-9dbe0ebb14a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:28.726646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15128bbf-8d18-4e3d-b230-9dbe0ebb14a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:28.727996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15128bbf-8d18-4e3d-b230-9dbe0ebb14a7 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:28.747875 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 360/725] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:28.753530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14082d1-8f03-4091-bf92-276a9b86169f None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:28.789530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:28.795489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3b0eb-a660-40aa-8422-2fc25ec7512d None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:28.795758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3b0eb-a660-40aa-8422-2fc25ec7512d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:28.796021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3b0eb-a660-40aa-8422-2fc25ec7512d None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:28.797693 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 361/726] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:39:28.804000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194a065-9a1e-4c8d-b7df-01e416e5136e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:28.804250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194a065-9a1e-4c8d-b7df-01e416e5136e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:28.804486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194a065-9a1e-4c8d-b7df-01e416e5136e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:28.826241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9194a065-9a1e-4c8d-b7df-01e416e5136e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:28.827839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9194a065-9a1e-4c8d-b7df-01e416e5136e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:28.918085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14082d1-8f03-4091-bf92-276a9b86169f None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:28.943470 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 362/727] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:28.963059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4f5d0-f478-4413-9cf2-b0170ab0b870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:28.963940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4f5d0-f478-4413-9cf2-b0170ab0b870 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:28.964216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4f5d0-f478-4413-9cf2-b0170ab0b870 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:28.964515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4f5d0-f478-4413-9cf2-b0170ab0b870 None admin] PATH_INFO: `/v3/users/swift` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:28.972102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4f5d0-f478-4413-9cf2-b0170ab0b870 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:28.973632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4f5d0-f478-4413-9cf2-b0170ab0b870 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-5bd4f5d0-f478-4413-9cf2-b0170ab0b870 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:28.985054 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:28.987115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:28.989272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:28.989272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:28.989272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:28.989272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:28.989272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:28.989272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:28.989272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:28.989272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:28.989272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 20 20:39:28.989272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:28.989272 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 366/728] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:39:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:28.992131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194a065-9a1e-4c8d-b7df-01e416e5136e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:29.095043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:29.098887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194a065-9a1e-4c8d-b7df-01e416e5136e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:29.139831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25b0c0-9cfb-4f4a-a31a-a3e04425ac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:29.140467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25b0c0-9cfb-4f4a-a31a-a3e04425ac56 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:29.140741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25b0c0-9cfb-4f4a-a31a-a3e04425ac56 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:29.141007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25b0c0-9cfb-4f4a-a31a-a3e04425ac56 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:29.141735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25b0c0-9cfb-4f4a-a31a-a3e04425ac56 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:29.142031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25b0c0-9cfb-4f4a-a31a-a3e04425ac56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:29.143469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25b0c0-9cfb-4f4a-a31a-a3e04425ac56 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:29.162532 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 363/729] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Aug 20 20:39:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:29.167990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4f5d0-f478-4413-9cf2-b0170ab0b870 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:29.293941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:29.304394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4f5d0-f478-4413-9cf2-b0170ab0b870 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:29.352180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7e3d0-8748-4a85-980f-d651557ac3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:29.352876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7e3d0-8748-4a85-980f-d651557ac3cf None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:29.353148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7e3d0-8748-4a85-980f-d651557ac3cf None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:29.353411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7e3d0-8748-4a85-980f-d651557ac3cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:29.384715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25b0c0-9cfb-4f4a-a31a-a3e04425ac56 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:29.443932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7e3d0-8748-4a85-980f-d651557ac3cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:29.446438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7e3d0-8748-4a85-980f-d651557ac3cf None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:29.602529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25b0c0-9cfb-4f4a-a31a-a3e04425ac56 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:29.620732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:29.637819 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 367/730] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:39:29.643617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7e3d0-8748-4a85-980f-d651557ac3cf None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:29.685073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb8ac8-a84d-4cdb-a1ec-8ce7460c1e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:29.685703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb8ac8-a84d-4cdb-a1ec-8ce7460c1e69 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:29.685979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb8ac8-a84d-4cdb-a1ec-8ce7460c1e69 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:29.686243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb8ac8-a84d-4cdb-a1ec-8ce7460c1e69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:29.708065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb8ac8-a84d-4cdb-a1ec-8ce7460c1e69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-7fcb8ac8-a84d-4cdb-a1ec-8ce7460c1e69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:29.713615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:29.714538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:29.715367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:29.715367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:29.715367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:29.715367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:29.715367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:29.715367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:29.715367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:29.715367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:29.715367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:29.715367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:29.715367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:29.715367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:29.717461 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 364/731] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:29.724290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb8ac8-a84d-4cdb-a1ec-8ce7460c1e69 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:29.750427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:29.778129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7e3d0-8748-4a85-980f-d651557ac3cf None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:29.813871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc3845-5190-4898-aec5-b6a123d9233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:29.814603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc3845-5190-4898-aec5-b6a123d9233b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:29.814871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc3845-5190-4898-aec5-b6a123d9233b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:29.815128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc3845-5190-4898-aec5-b6a123d9233b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:29.826539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc3845-5190-4898-aec5-b6a123d9233b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:29.828428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc3845-5190-4898-aec5-b6a123d9233b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:29.840944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb8ac8-a84d-4cdb-a1ec-8ce7460c1e69 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-4bcc3845-5190-4898-aec5-b6a123d9233b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:29.843422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:29.844335 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:29.845267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:29.845267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:29.845267 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:29.845267 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 368/732] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:29.845818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae76add-ad51-4d64-aed8-8dcd1da57283 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:29.846094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae76add-ad51-4d64-aed8-8dcd1da57283 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:29.846354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae76add-ad51-4d64-aed8-8dcd1da57283 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:29.851422 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 369/733] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:29.851422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884b942-6d7f-4e01-b30a-5781448753bc None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:29.851422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884b942-6d7f-4e01-b30a-5781448753bc None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:29.851422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884b942-6d7f-4e01-b30a-5781448753bc None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:29.851422 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 370/734] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:29.851422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884b942-6d7f-4e01-b30a-5781448753bc None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:29.889945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b4f09-f346-4fc3-989f-4efce850bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:29.890842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b4f09-f346-4fc3-989f-4efce850bb94 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:29.891395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b4f09-f346-4fc3-989f-4efce850bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:29.892261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b4f09-f346-4fc3-989f-4efce850bb94 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:29.893124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b4f09-f346-4fc3-989f-4efce850bb94 None admin] RBAC: Adding query filter params (name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:29.893608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b4f09-f346-4fc3-989f-4efce850bb94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:29.895419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:29.896065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b4f09-f346-4fc3-989f-4efce850bb94 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:29.911017 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 365/735] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:29.911017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f29b2f-2e38-49e9-8614-4e36f918ec11 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:29.911017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f29b2f-2e38-49e9-8614-4e36f918ec11 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:29.911017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f29b2f-2e38-49e9-8614-4e36f918ec11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:29.927435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d1f29b2f-2e38-49e9-8614-4e36f918ec11 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:29.928288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d1f29b2f-2e38-49e9-8614-4e36f918ec11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:29.966113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884b942-6d7f-4e01-b30a-5781448753bc None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:30.008406 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 366/736] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:30.008406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592e5c6-3eff-49dd-97d2-ce0c0ffba80c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:30.008406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592e5c6-3eff-49dd-97d2-ce0c0ffba80c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:30.008406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592e5c6-3eff-49dd-97d2-ce0c0ffba80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:30.011102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdb0a8-154b-4b17-95be-9961fd702549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:30.011773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdb0a8-154b-4b17-95be-9961fd702549 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:30.012050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdb0a8-154b-4b17-95be-9961fd702549 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:30.012314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdb0a8-154b-4b17-95be-9961fd702549 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:30.013049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdb0a8-154b-4b17-95be-9961fd702549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:30.013548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdb0a8-154b-4b17-95be-9961fd702549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:30.015208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdb0a8-154b-4b17-95be-9961fd702549 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:30.025935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8592e5c6-3eff-49dd-97d2-ce0c0ffba80c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:30.027453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8592e5c6-3eff-49dd-97d2-ce0c0ffba80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:30.028782 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 371/737] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:30.029617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdb0a8-154b-4b17-95be-9961fd702549 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:30.035333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:30.125697 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 367/738] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:30.126566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592e5c6-3eff-49dd-97d2-ce0c0ffba80c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:30.161775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdb0a8-154b-4b17-95be-9961fd702549 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:30.163555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:30.243492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592e5c6-3eff-49dd-97d2-ce0c0ffba80c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:30.249794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d6f6b-0e26-408d-95e5-78f2e9ee54fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:30.250453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d6f6b-0e26-408d-95e5-78f2e9ee54fe None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:30.250727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d6f6b-0e26-408d-95e5-78f2e9ee54fe None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:30.250993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d6f6b-0e26-408d-95e5-78f2e9ee54fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:30.281240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d6f6b-0e26-408d-95e5-78f2e9ee54fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:30.282407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19622c31-a56c-495d-8738-19c0a379a377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:30.282822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d6f6b-0e26-408d-95e5-78f2e9ee54fe None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:30.283043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19622c31-a56c-495d-8738-19c0a379a377 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:30.283415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19622c31-a56c-495d-8738-19c0a379a377 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:30.283760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19622c31-a56c-495d-8738-19c0a379a377 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:30.291253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19622c31-a56c-495d-8738-19c0a379a377 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:30.293233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19622c31-a56c-495d-8738-19c0a379a377 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:30.295810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-19622c31-a56c-495d-8738-19c0a379a377 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:30.306997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:30.307972 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:30.308907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:30.308907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:30.308907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:30.308907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:30.308907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:30.308907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:30.308907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:30.308907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-e10d6f6b-0e26-408d-95e5-78f2e9ee54fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:30.309651 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:30.310493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:30.311373 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:30.311373 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:30.311373 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:30.311536 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 368/739] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:30.313890 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 372/740] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:30.314101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19622c31-a56c-495d-8738-19c0a379a377 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:30.319508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d6f6b-0e26-408d-95e5-78f2e9ee54fe None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:30.453347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19622c31-a56c-495d-8738-19c0a379a377 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:30.472421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d6f6b-0e26-408d-95e5-78f2e9ee54fe None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:30.493291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593d3c4-2110-462f-93c4-5a76e107e4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:30.494491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593d3c4-2110-462f-93c4-5a76e107e4e5 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:30.495093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593d3c4-2110-462f-93c4-5a76e107e4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:30.495804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593d3c4-2110-462f-93c4-5a76e107e4e5 None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:30.496719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593d3c4-2110-462f-93c4-5a76e107e4e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:30.497216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593d3c4-2110-462f-93c4-5a76e107e4e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:30.499260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593d3c4-2110-462f-93c4-5a76e107e4e5 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:30.514387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef733e-c087-4528-8cb8-fe65c0de7c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:30.515012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef733e-c087-4528-8cb8-fe65c0de7c9d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:30.515278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef733e-c087-4528-8cb8-fe65c0de7c9d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:30.515534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef733e-c087-4528-8cb8-fe65c0de7c9d None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:30.516199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef733e-c087-4528-8cb8-fe65c0de7c9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:30.516423 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 369/741] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:30.516522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef733e-c087-4528-8cb8-fe65c0de7c9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:30.517567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593d3c4-2110-462f-93c4-5a76e107e4e5 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:30.517816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef733e-c087-4528-8cb8-fe65c0de7c9d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:30.537521 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 373/742] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:30.538775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef733e-c087-4528-8cb8-fe65c0de7c9d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:30.556106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:30.593983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:30.643354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593d3c4-2110-462f-93c4-5a76e107e4e5 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:30.647766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef733e-c087-4528-8cb8-fe65c0de7c9d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:30.680020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992ddf1-8fe5-49de-bc0f-65abbdd1bb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:30.680657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992ddf1-8fe5-49de-bc0f-65abbdd1bb61 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:30.680926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992ddf1-8fe5-49de-bc0f-65abbdd1bb61 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:30.681184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992ddf1-8fe5-49de-bc0f-65abbdd1bb61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:30.682643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9d0be-9369-4d87-9d5a-e96832ee5c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:30.683387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9d0be-9369-4d87-9d5a-e96832ee5c06 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:30.683661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9d0be-9369-4d87-9d5a-e96832ee5c06 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:30.684581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9d0be-9369-4d87-9d5a-e96832ee5c06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:30.685294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9d0be-9369-4d87-9d5a-e96832ee5c06 None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=d74f7cdc155a4397ad28a3fbca86f7f9) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:30.685593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9d0be-9369-4d87-9d5a-e96832ee5c06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:30.687037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9d0be-9369-4d87-9d5a-e96832ee5c06 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:30.691651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992ddf1-8fe5-49de-bc0f-65abbdd1bb61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:30.692614 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 374/743] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:30 2022] GET /identity/v3/role_assignments?user.id=d74f7cdc155a4397ad28a3fbca86f7f9&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:30.693381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9d0be-9369-4d87-9d5a-e96832ee5c06 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-5992ddf1-8fe5-49de-bc0f-65abbdd1bb61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:30.696168 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:30.697140 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:30.699981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:30.699981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:30.699981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:30.699981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:30.699981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:30.699981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:30.699981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:30.699981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:30.699981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:30.699981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:30.699981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:39:30.699981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:30.702688 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 370/744] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:39:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:30.703739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992ddf1-8fe5-49de-bc0f-65abbdd1bb61 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:30.790678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:30.816731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9d0be-9369-4d87-9d5a-e96832ee5c06 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:30.824283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992ddf1-8fe5-49de-bc0f-65abbdd1bb61 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:30.848755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:30.857552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83446e-d1f9-41ca-9bea-63d575eaa636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:30.858204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83446e-d1f9-41ca-9bea-63d575eaa636 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:30.858496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83446e-d1f9-41ca-9bea-63d575eaa636 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:30.858698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83446e-d1f9-41ca-9bea-63d575eaa636 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:30.865828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934c7f9-7ac5-4b1a-a621-d12a1b0c7011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:30.866445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934c7f9-7ac5-4b1a-a621-d12a1b0c7011 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:30.866713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934c7f9-7ac5-4b1a-a621-d12a1b0c7011 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:30.866977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934c7f9-7ac5-4b1a-a621-d12a1b0c7011 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:30.867374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83446e-d1f9-41ca-9bea-63d575eaa636 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:30.867614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934c7f9-7ac5-4b1a-a621-d12a1b0c7011 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:30.868889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83446e-d1f9-41ca-9bea-63d575eaa636 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:30.868988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934c7f9-7ac5-4b1a-a621-d12a1b0c7011 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-2b83446e-d1f9-41ca-9bea-63d575eaa636 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:30.882806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:30.883875 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:30.885060 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:30.885060 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:30.885060 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:30.885060 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:30.885060 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:30.885060 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:30.885060 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:30.885060 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:30.885060 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 20 20:39:30.885060 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:30.885060 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 375/745] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Aug 20 20:39:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:30.885733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aedd18-ca20-4794-a0cb-e12d6c0737a4 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:30.885733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aedd18-ca20-4794-a0cb-e12d6c0737a4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:30.886026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aedd18-ca20-4794-a0cb-e12d6c0737a4 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:30.888036 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 376/746] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:30.892247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aedd18-ca20-4794-a0cb-e12d6c0737a4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:30.896753 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 371/747] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:39:30 2022] POST /identity/v3/services => generated 240 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:39:30.899039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934c7f9-7ac5-4b1a-a621-d12a1b0c7011 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:31.038828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934c7f9-7ac5-4b1a-a621-d12a1b0c7011 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:31.081977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82b35d-ded8-4ae2-a9bc-48f88a7c3343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:31.082606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82b35d-ded8-4ae2-a9bc-48f88a7c3343 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:31.082875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82b35d-ded8-4ae2-a9bc-48f88a7c3343 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:31.083138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82b35d-ded8-4ae2-a9bc-48f88a7c3343 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:31.084033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82b35d-ded8-4ae2-a9bc-48f88a7c3343 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:31.084329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82b35d-ded8-4ae2-a9bc-48f88a7c3343 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:31.085838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82b35d-ded8-4ae2-a9bc-48f88a7c3343 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:31.107587 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 372/748] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Aug 20 20:39:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:31.109214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42521178-95d9-418d-8376-9893023cc309 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:31.109492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42521178-95d9-418d-8376-9893023cc309 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:31.109760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42521178-95d9-418d-8376-9893023cc309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:31.129176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-42521178-95d9-418d-8376-9893023cc309 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:31.130424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-42521178-95d9-418d-8376-9893023cc309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:31.165311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:31.183152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aedd18-ca20-4794-a0cb-e12d6c0737a4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:31.215190 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 373/749] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:31 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 20:39:31.217399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42521178-95d9-418d-8376-9893023cc309 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:31.244732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82159b-2f8e-4010-834a-248fe408efa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:31.245345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82159b-2f8e-4010-834a-248fe408efa3 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:31.245614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82159b-2f8e-4010-834a-248fe408efa3 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:31.245876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82159b-2f8e-4010-834a-248fe408efa3 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:31.246536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82159b-2f8e-4010-834a-248fe408efa3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:31.246830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82159b-2f8e-4010-834a-248fe408efa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:31.248192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82159b-2f8e-4010-834a-248fe408efa3 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:31.257882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:31.264038 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 377/750] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:39:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:31.269767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82159b-2f8e-4010-834a-248fe408efa3 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:31.376492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42521178-95d9-418d-8376-9893023cc309 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:31.453258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d3f8f-918f-41ff-bccb-6ccf4cba7d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:31.453870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d3f8f-918f-41ff-bccb-6ccf4cba7d32 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:31.454436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d3f8f-918f-41ff-bccb-6ccf4cba7d32 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:31.454436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d3f8f-918f-41ff-bccb-6ccf4cba7d32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:31.457924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82159b-2f8e-4010-834a-248fe408efa3 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:31.499499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5102393-f915-4391-8a11-28bd6af226e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:31.500135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5102393-f915-4391-8a11-28bd6af226e7 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:31.500446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5102393-f915-4391-8a11-28bd6af226e7 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:31.500721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5102393-f915-4391-8a11-28bd6af226e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:31.511568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5102393-f915-4391-8a11-28bd6af226e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:31.513057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5102393-f915-4391-8a11-28bd6af226e7 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-d5102393-f915-4391-8a11-28bd6af226e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:31.529196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:31.530136 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:31.531621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:31.531621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:31.531621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:31.531621 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 378/751] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:31.537314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5102393-f915-4391-8a11-28bd6af226e7 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:31.541703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:31.606096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d3f8f-918f-41ff-bccb-6ccf4cba7d32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:31.607562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d3f8f-918f-41ff-bccb-6ccf4cba7d32 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:31.633630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:31.693473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5102393-f915-4391-8a11-28bd6af226e7 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:31.736757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f989250-de4c-4a37-86f9-4e87654314f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:31.737356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f989250-de4c-4a37-86f9-4e87654314f1 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:31.737621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f989250-de4c-4a37-86f9-4e87654314f1 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:31.737880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f989250-de4c-4a37-86f9-4e87654314f1 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:31.738506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f989250-de4c-4a37-86f9-4e87654314f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:31.738796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f989250-de4c-4a37-86f9-4e87654314f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:31.740141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f989250-de4c-4a37-86f9-4e87654314f1 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:31.746632 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 374/752] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:31 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 20:39:31.747591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d3f8f-918f-41ff-bccb-6ccf4cba7d32 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:31.756603 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 379/753] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:31.757444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f989250-de4c-4a37-86f9-4e87654314f1 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:31.823537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:31.857095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d3f8f-918f-41ff-bccb-6ccf4cba7d32 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:31.886028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f989250-de4c-4a37-86f9-4e87654314f1 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:31.893292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950ef91-e60c-4f19-94ea-e9c390d3c488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:31.893903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950ef91-e60c-4f19-94ea-e9c390d3c488 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:31.894175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950ef91-e60c-4f19-94ea-e9c390d3c488 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:31.894444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950ef91-e60c-4f19-94ea-e9c390d3c488 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:31.895187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950ef91-e60c-4f19-94ea-e9c390d3c488 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:31.896766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950ef91-e60c-4f19-94ea-e9c390d3c488 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:31.898881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:31.920759 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 375/754] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:39:31 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:39:31.921955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950ef91-e60c-4f19-94ea-e9c390d3c488 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:31.940349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe9878-2842-450a-9ae6-d6840e8a3eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:31.941099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe9878-2842-450a-9ae6-d6840e8a3eed None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:31.941377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe9878-2842-450a-9ae6-d6840e8a3eed None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:31.941679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe9878-2842-450a-9ae6-d6840e8a3eed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:31.955199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe9878-2842-450a-9ae6-d6840e8a3eed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:31.956760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe9878-2842-450a-9ae6-d6840e8a3eed None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-90fe9878-2842-450a-9ae6-d6840e8a3eed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:31.973532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:31.974546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:31.975427 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:31.975427 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:31.975427 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:31.975427 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 380/755] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:31.983806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe9878-2842-450a-9ae6-d6840e8a3eed None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:32.148362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950ef91-e60c-4f19-94ea-e9c390d3c488 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:32.169452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe9878-2842-450a-9ae6-d6840e8a3eed None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:32.182437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:32.195690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b457ee-a72b-4aac-be49-de47cd36f24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:32.196331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b457ee-a72b-4aac-be49-de47cd36f24b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:32.196637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b457ee-a72b-4aac-be49-de47cd36f24b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:32.196899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b457ee-a72b-4aac-be49-de47cd36f24b None admin] PATH_INFO: `/v3/users/nova` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:32.206066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b457ee-a72b-4aac-be49-de47cd36f24b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:32.207577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b457ee-a72b-4aac-be49-de47cd36f24b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:32.218318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53794de7-109c-41b0-9d2b-b03f40b6a8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:32.219260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53794de7-109c-41b0-9d2b-b03f40b6a8c5 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:32.219727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53794de7-109c-41b0-9d2b-b03f40b6a8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:32.220205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53794de7-109c-41b0-9d2b-b03f40b6a8c5 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:32.221063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53794de7-109c-41b0-9d2b-b03f40b6a8c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:32.221574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53794de7-109c-41b0-9d2b-b03f40b6a8c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-86b457ee-a72b-4aac-be49-de47cd36f24b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:32.221867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:32.222856 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:32.223813 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:32.223813 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:32.223813 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:32.223813 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:32.223813 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:32.223813 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:32.223813 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:32.223813 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:32.223813 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:39:32.223813 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:32.226192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53794de7-109c-41b0-9d2b-b03f40b6a8c5 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:32.234106 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 376/756] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Aug 20 20:39:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:32.239241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b457ee-a72b-4aac-be49-de47cd36f24b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:32.240093 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 381/757] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:32.245599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53794de7-109c-41b0-9d2b-b03f40b6a8c5 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:32.276920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:32.368420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53794de7-109c-41b0-9d2b-b03f40b6a8c5 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:32.385074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b457ee-a72b-4aac-be49-de47cd36f24b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:32.411805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a3721-8d55-4365-bd1e-9653b1606d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:32.412497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a3721-8d55-4365-bd1e-9653b1606d34 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:32.412765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a3721-8d55-4365-bd1e-9653b1606d34 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:32.413019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a3721-8d55-4365-bd1e-9653b1606d34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:32.430107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24a94a-0078-4fd6-8829-96cb894594f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:32.431047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24a94a-0078-4fd6-8829-96cb894594f1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:32.431842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24a94a-0078-4fd6-8829-96cb894594f1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:32.432324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24a94a-0078-4fd6-8829-96cb894594f1 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:32.433154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a3721-8d55-4365-bd1e-9653b1606d34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:32.433520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24a94a-0078-4fd6-8829-96cb894594f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:32.434316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24a94a-0078-4fd6-8829-96cb894594f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:32.434934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a3721-8d55-4365-bd1e-9653b1606d34 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:32.436104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24a94a-0078-4fd6-8829-96cb894594f1 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:32.456185 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 377/758] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Aug 20 20:39:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-183a3721-8d55-4365-bd1e-9653b1606d34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:32.464402 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:32.465301 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:32.466214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:32.466214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:32.466214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:32.466214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:32.466214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:32.466214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:32.466214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:32.466214 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:32.466214 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 382/759] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:32.473450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24a94a-0078-4fd6-8829-96cb894594f1 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:32.474693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a3721-8d55-4365-bd1e-9653b1606d34 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:32.551020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:32.621890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:32.654075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a3721-8d55-4365-bd1e-9653b1606d34 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:32.657495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24a94a-0078-4fd6-8829-96cb894594f1 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:32.711274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bf879-e82c-45d4-b1ac-811f934da1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:32.711925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258176c-c9c0-4ab8-9900-26928c5f64a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:32.712105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bf879-e82c-45d4-b1ac-811f934da1b1 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:32.712105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bf879-e82c-45d4-b1ac-811f934da1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:32.712440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258176c-c9c0-4ab8-9900-26928c5f64a5 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:32.712596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bf879-e82c-45d4-b1ac-811f934da1b1 None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:32.712741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258176c-c9c0-4ab8-9900-26928c5f64a5 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:32.712741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258176c-c9c0-4ab8-9900-26928c5f64a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:32.713256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bf879-e82c-45d4-b1ac-811f934da1b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:32.713256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bf879-e82c-45d4-b1ac-811f934da1b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:32.715016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bf879-e82c-45d4-b1ac-811f934da1b1 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:32.728568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0258176c-c9c0-4ab8-9900-26928c5f64a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:32.730007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0258176c-c9c0-4ab8-9900-26928c5f64a5 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-0258176c-c9c0-4ab8-9900-26928c5f64a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:32.745373 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:32.746375 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:32.747269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:32.747269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:32.747269 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:32.747269 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 378/760] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:32.750067 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 383/761] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:32.752250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258176c-c9c0-4ab8-9900-26928c5f64a5 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:32.755993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bf879-e82c-45d4-b1ac-811f934da1b1 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:32.903933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:32.926718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258176c-c9c0-4ab8-9900-26928c5f64a5 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:32.935576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bf879-e82c-45d4-b1ac-811f934da1b1 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:32.991929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e561dcb-46cc-450e-b678-6caea88e32a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:32.992899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e561dcb-46cc-450e-b678-6caea88e32a1 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:32.993794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e561dcb-46cc-450e-b678-6caea88e32a1 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:32.994425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e561dcb-46cc-450e-b678-6caea88e32a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:32.996183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e561dcb-46cc-450e-b678-6caea88e32a1 None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=1547cb99abe642ae86db8de1fa80041f) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:32.997008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e561dcb-46cc-450e-b678-6caea88e32a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:32.999112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e561dcb-46cc-450e-b678-6caea88e32a1 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:32.999916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b40ba-d511-43ee-80d1-b86a7304e45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:33.000736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b40ba-d511-43ee-80d1-b86a7304e45a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:33.001180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b40ba-d511-43ee-80d1-b86a7304e45a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:33.001633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b40ba-d511-43ee-80d1-b86a7304e45a None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:33.002445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b40ba-d511-43ee-80d1-b86a7304e45a None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:33.002915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b40ba-d511-43ee-80d1-b86a7304e45a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:33.004442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b40ba-d511-43ee-80d1-b86a7304e45a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:33.014798 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 384/762] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:32 2022] GET /identity/v3/role_assignments?user.id=1547cb99abe642ae86db8de1fa80041f&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:33.018373 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 379/763] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:33.023778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e561dcb-46cc-450e-b678-6caea88e32a1 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:33.037770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:33.106083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739c19d-5b80-4b62-992f-d3e0967834fd None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:33.106447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739c19d-5b80-4b62-992f-d3e0967834fd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:33.106641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739c19d-5b80-4b62-992f-d3e0967834fd None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:33.108206 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 380/764] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:33.115823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd211f74-d615-4073-9544-dce6188e4d3f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:33.117364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd211f74-d615-4073-9544-dce6188e4d3f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:33.117992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd211f74-d615-4073-9544-dce6188e4d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:33.153713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cd211f74-d615-4073-9544-dce6188e4d3f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:33.155065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cd211f74-d615-4073-9544-dce6188e4d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:33.218224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e561dcb-46cc-450e-b678-6caea88e32a1 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:33.285993 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 381/765] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:39:33.288103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3eb293-bdf5-4efa-ba32-592abf3ff411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:33.288994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3eb293-bdf5-4efa-ba32-592abf3ff411 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:33.289389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3eb293-bdf5-4efa-ba32-592abf3ff411 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:33.289787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3eb293-bdf5-4efa-ba32-592abf3ff411 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:33.299820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3eb293-bdf5-4efa-ba32-592abf3ff411 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:33.301664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3eb293-bdf5-4efa-ba32-592abf3ff411 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-9d3eb293-bdf5-4efa-ba32-592abf3ff411 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:33.314501 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:33.315622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:33.316780 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:33.316780 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:33.316780 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:33.316780 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:33.316780 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:33.316780 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:33.316780 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:33.316780 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:33.318893 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 385/766] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:33.327655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd211f74-d615-4073-9544-dce6188e4d3f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:33.403539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9fa54-1903-47d1-ab4a-5b01367c43ef None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:33.403896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9fa54-1903-47d1-ab4a-5b01367c43ef None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:33.404061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9fa54-1903-47d1-ab4a-5b01367c43ef None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:33.407043 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 386/767] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:39:33.413844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bd869-2bc6-40c3-9c30-aa93b8ec2f3f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:33.414115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bd869-2bc6-40c3-9c30-aa93b8ec2f3f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:33.414380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bd869-2bc6-40c3-9c30-aa93b8ec2f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:33.453204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-df7bd869-2bc6-40c3-9c30-aa93b8ec2f3f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:33.454749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-df7bd869-2bc6-40c3-9c30-aa93b8ec2f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:33.455302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:33.486471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd211f74-d615-4073-9544-dce6188e4d3f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:33.557984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d44c1-5895-4fe5-bb64-8f4a27a50544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:33.558553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d44c1-5895-4fe5-bb64-8f4a27a50544 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:33.558838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d44c1-5895-4fe5-bb64-8f4a27a50544 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:33.559412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d44c1-5895-4fe5-bb64-8f4a27a50544 None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:33.560101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d44c1-5895-4fe5-bb64-8f4a27a50544 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:33.560417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d44c1-5895-4fe5-bb64-8f4a27a50544 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:33.561846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d44c1-5895-4fe5-bb64-8f4a27a50544 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:33.584602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:33.588540 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 382/768] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:33.595427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d44c1-5895-4fe5-bb64-8f4a27a50544 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:33.599555 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 387/769] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:39:33.663373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bd869-2bc6-40c3-9c30-aa93b8ec2f3f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:33.764759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d44c1-5895-4fe5-bb64-8f4a27a50544 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:33.794029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bd869-2bc6-40c3-9c30-aa93b8ec2f3f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:33.832975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080698-37b7-475c-a8c8-bf90eb2104ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:33.833931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080698-37b7-475c-a8c8-bf90eb2104ed None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:33.834392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080698-37b7-475c-a8c8-bf90eb2104ed None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:33.834860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080698-37b7-475c-a8c8-bf90eb2104ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:33.845804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5080698-37b7-475c-a8c8-bf90eb2104ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:33.849059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a377618-6713-4e26-b73c-f6c3c09c5d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:33.849730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a377618-6713-4e26-b73c-f6c3c09c5d6a None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:33.850008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a377618-6713-4e26-b73c-f6c3c09c5d6a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:33.850278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a377618-6713-4e26-b73c-f6c3c09c5d6a None admin] PATH_INFO: `/v3/projects/3b6c88840d764c2385aa678da4338213/users/df852af59e7743f7a8798d9edf8ebd3e/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-e5080698-37b7-475c-a8c8-bf90eb2104ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:33.851752 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:33.852681 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:33.853591 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:33.853591 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:33.853591 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:33.853591 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:33.853591 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:33.853591 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:33.853591 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:33.853591 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:33.853591 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:33.853591 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:33.853591 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:39:33.853591 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:33.854213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a377618-6713-4e26-b73c-f6c3c09c5d6a None admin] RBAC: Authorizing `identity:create_grant(project_id=3b6c88840d764c2385aa678da4338213, user_id=df852af59e7743f7a8798d9edf8ebd3e, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:33.854213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a377618-6713-4e26-b73c-f6c3c09c5d6a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:33.856457 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 388/770] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:39:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:33.862494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080698-37b7-475c-a8c8-bf90eb2104ed None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:33.890821 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 383/771] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:39:33 2022] PUT /identity/v3/projects/3b6c88840d764c2385aa678da4338213/users/df852af59e7743f7a8798d9edf8ebd3e/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:39:33.922709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:33.989145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080698-37b7-475c-a8c8-bf90eb2104ed None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:34.039703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5c49-0701-430a-a029-f7359bca4c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:34.040746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5c49-0701-430a-a029-f7359bca4c9a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:34.041223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5c49-0701-430a-a029-f7359bca4c9a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:34.041710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5c49-0701-430a-a029-f7359bca4c9a None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:34.042596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d5c49-0701-430a-a029-f7359bca4c9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:34.043096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d5c49-0701-430a-a029-f7359bca4c9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:34.044722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d5c49-0701-430a-a029-f7359bca4c9a None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:34.053845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a377618-6713-4e26-b73c-f6c3c09c5d6a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:34.056848 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 389/772] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:39:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:34.063006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5c49-0701-430a-a029-f7359bca4c9a None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:34.072216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:34.189021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a377618-6713-4e26-b73c-f6c3c09c5d6a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:34.202988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5c49-0701-430a-a029-f7359bca4c9a None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:34.221198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:34.229363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06373550-58e6-4232-bebe-9232ca25573f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:34.229363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06373550-58e6-4232-bebe-9232ca25573f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:34.229363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06373550-58e6-4232-bebe-9232ca25573f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:34.229363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06373550-58e6-4232-bebe-9232ca25573f None admin] PATH_INFO: `/v3/services/network` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:34.229759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06373550-58e6-4232-bebe-9232ca25573f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:34.230502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06373550-58e6-4232-bebe-9232ca25573f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-06373550-58e6-4232-bebe-9232ca25573f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:34.241956 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:34.242966 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:34.244706 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 384/773] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:34 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:34.249380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06373550-58e6-4232-bebe-9232ca25573f None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:34.256581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80c284-24d5-4cb8-bba3-531f58354042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:34.257570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80c284-24d5-4cb8-bba3-531f58354042 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:34.258033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80c284-24d5-4cb8-bba3-531f58354042 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:34.258544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80c284-24d5-4cb8-bba3-531f58354042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:34.270497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80c284-24d5-4cb8-bba3-531f58354042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:34.272586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80c284-24d5-4cb8-bba3-531f58354042 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-2b80c284-24d5-4cb8-bba3-531f58354042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:34.288678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:34.289669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:34.290552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:34.290552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:34.290552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:34.296563 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 390/774] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:34.299392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80c284-24d5-4cb8-bba3-531f58354042 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:34.319669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:34.408473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80c284-24d5-4cb8-bba3-531f58354042 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:34.422651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06373550-58e6-4232-bebe-9232ca25573f None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:34.445097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a3092-f929-403f-8b22-49501f3c829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:34.446021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a3092-f929-403f-8b22-49501f3c829f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:34.446544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a3092-f929-403f-8b22-49501f3c829f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:34.446994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a3092-f929-403f-8b22-49501f3c829f None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:34.448451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a3092-f929-403f-8b22-49501f3c829f None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:34.448992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a3092-f929-403f-8b22-49501f3c829f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:34.450539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a3092-f929-403f-8b22-49501f3c829f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:34.468425 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 391/775] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:34.470050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a3092-f929-403f-8b22-49501f3c829f None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:34.494519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c98be4-51ae-438b-b460-7f20899532e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:34.495258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c98be4-51ae-438b-b460-7f20899532e2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:34.495540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c98be4-51ae-438b-b460-7f20899532e2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:34.495807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c98be4-51ae-438b-b460-7f20899532e2 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:34.496475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c98be4-51ae-438b-b460-7f20899532e2 None admin] RBAC: Adding query filter params (name=network) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:34.496778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c98be4-51ae-438b-b460-7f20899532e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:34.498154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c98be4-51ae-438b-b460-7f20899532e2 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:34.502977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:34.510646 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 385/776] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Aug 20 20:39:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:34.516837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c98be4-51ae-438b-b460-7f20899532e2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:34.577237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a3092-f929-403f-8b22-49501f3c829f None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:34.614957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f752c0-fbba-4877-bab7-43d3de22174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:34.615977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f752c0-fbba-4877-bab7-43d3de22174c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:34.616483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f752c0-fbba-4877-bab7-43d3de22174c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:34.617049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f752c0-fbba-4877-bab7-43d3de22174c None admin] PATH_INFO: `/v3/users/swift` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:34.625658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f752c0-fbba-4877-bab7-43d3de22174c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:34.627486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f752c0-fbba-4877-bab7-43d3de22174c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-00f752c0-fbba-4877-bab7-43d3de22174c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:34.642834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:34.644075 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:34.645609 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:34.645609 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:34.645609 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:34.645609 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:34.645609 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:34.645609 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:34.645609 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:34.645609 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:34.645609 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 20 20:39:34.645609 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:34.652520 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 392/777] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:39:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:34.655315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f752c0-fbba-4877-bab7-43d3de22174c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:34.688479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c98be4-51ae-438b-b460-7f20899532e2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:34.746560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:34.747908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:34.758998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde60db-63af-4f3a-8c6c-a88555a59f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:34.759656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde60db-63af-4f3a-8c6c-a88555a59f1b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:34.762926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f752c0-fbba-4877-bab7-43d3de22174c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:34.764742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde60db-63af-4f3a-8c6c-a88555a59f1b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:34.765030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde60db-63af-4f3a-8c6c-a88555a59f1b None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:34.765682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde60db-63af-4f3a-8c6c-a88555a59f1b None admin] RBAC: Adding query filter params (type=network) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:34.765988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde60db-63af-4f3a-8c6c-a88555a59f1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:34.767354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde60db-63af-4f3a-8c6c-a88555a59f1b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:34.772829 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 386/778] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Aug 20 20:39:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:34.778622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde60db-63af-4f3a-8c6c-a88555a59f1b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:34.800524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e3431-ec99-4f20-8ab7-52e9893d1b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:34.801839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e3431-ec99-4f20-8ab7-52e9893d1b1a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:34.802269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e3431-ec99-4f20-8ab7-52e9893d1b1a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:34.802760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e3431-ec99-4f20-8ab7-52e9893d1b1a None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:34.803827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e3431-ec99-4f20-8ab7-52e9893d1b1a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:34.804338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e3431-ec99-4f20-8ab7-52e9893d1b1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:34.806003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e3431-ec99-4f20-8ab7-52e9893d1b1a None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:34.829692 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 393/779] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Aug 20 20:39:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:34.831419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e3431-ec99-4f20-8ab7-52e9893d1b1a None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:34.934520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e3431-ec99-4f20-8ab7-52e9893d1b1a None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:34.967287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde60db-63af-4f3a-8c6c-a88555a59f1b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:34.970780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5d3e3-1ced-470a-aa67-d3a872e11e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:34.971654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5d3e3-1ced-470a-aa67-d3a872e11e78 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:34.972113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5d3e3-1ced-470a-aa67-d3a872e11e78 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:34.972588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5d3e3-1ced-470a-aa67-d3a872e11e78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:34.983884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5d3e3-1ced-470a-aa67-d3a872e11e78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:34.985899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5d3e3-1ced-470a-aa67-d3a872e11e78 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-2eb5d3e3-1ced-470a-aa67-d3a872e11e78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:35.002236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:35.003193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:35.004090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:35.004090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:35.004090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:35.004270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b8113-428a-457d-9b14-964d4b035aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:35.004270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b8113-428a-457d-9b14-964d4b035aac None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:35.004270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b8113-428a-457d-9b14-964d4b035aac None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:35.004270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b8113-428a-457d-9b14-964d4b035aac None admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:35.005035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b8113-428a-457d-9b14-964d4b035aac None admin] RBAC: Adding query filter params (interface=public, service_id=b379381cbce74126b433110079309a68, region_id=RegionOne) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:35.005035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b8113-428a-457d-9b14-964d4b035aac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:35.006323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b8113-428a-457d-9b14-964d4b035aac None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:35.008436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:35.011468 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 394/780] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:35.012871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0b338-2201-4fda-bcc3-5e573b1cff34 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:35.013240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0b338-2201-4fda-bcc3-5e573b1cff34 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:35.013597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0b338-2201-4fda-bcc3-5e573b1cff34 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:35.015083 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 395/781] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:35.015196 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 387/782] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Aug 20 20:39:34 2022] GET /identity/v3/endpoints?service_id=b379381cbce74126b433110079309a68&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:35.016177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0b338-2201-4fda-bcc3-5e573b1cff34 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:35.020732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397edc4e-3e34-4077-9a62-3c184ba349dc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:35.021009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397edc4e-3e34-4077-9a62-3c184ba349dc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:35.021281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397edc4e-3e34-4077-9a62-3c184ba349dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:35.039401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-397edc4e-3e34-4077-9a62-3c184ba349dc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:35.040617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-397edc4e-3e34-4077-9a62-3c184ba349dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:35.100736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:35.116485 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 388/783] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:39:35.117362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397edc4e-3e34-4077-9a62-3c184ba349dc None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:35.132819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0b338-2201-4fda-bcc3-5e573b1cff34 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:35.205545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65bc23-0e42-4f93-83bd-d02d37c8857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:35.206142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65bc23-0e42-4f93-83bd-d02d37c8857b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:35.206406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65bc23-0e42-4f93-83bd-d02d37c8857b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:35.206666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65bc23-0e42-4f93-83bd-d02d37c8857b None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:35.210025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65bc23-0e42-4f93-83bd-d02d37c8857b None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:35.210334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65bc23-0e42-4f93-83bd-d02d37c8857b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:35.211691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65bc23-0e42-4f93-83bd-d02d37c8857b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:35.246494 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 396/784] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:35.264771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65bc23-0e42-4f93-83bd-d02d37c8857b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:35.287834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397edc4e-3e34-4077-9a62-3c184ba349dc None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:35.334829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558319c2-8928-417a-8880-7c219038ece7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:35.335451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558319c2-8928-417a-8880-7c219038ece7 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:35.335717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558319c2-8928-417a-8880-7c219038ece7 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:35.335977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558319c2-8928-417a-8880-7c219038ece7 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:35.336600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558319c2-8928-417a-8880-7c219038ece7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:35.338103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558319c2-8928-417a-8880-7c219038ece7 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:35.344544 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 389/785] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Aug 20 20:39:35 2022] GET /identity/v3/services => generated 772 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:35.396148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:35.461919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65bc23-0e42-4f93-83bd-d02d37c8857b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:35.514245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67ad4f-d871-4630-bb13-b98d7214ce43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:35.514482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558319c2-8928-417a-8880-7c219038ece7 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:35.515127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67ad4f-d871-4630-bb13-b98d7214ce43 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:35.515297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67ad4f-d871-4630-bb13-b98d7214ce43 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:35.515556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67ad4f-d871-4630-bb13-b98d7214ce43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:35.543305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67ad4f-d871-4630-bb13-b98d7214ce43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:35.544818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67ad4f-d871-4630-bb13-b98d7214ce43 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-9c67ad4f-d871-4630-bb13-b98d7214ce43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:35.568666 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:35.570806 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:35.573211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:35.573211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:35.573211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:35.573211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:35.573211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:35.573211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:35.573211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:35.573211 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:35.581543 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 397/786] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:35.586146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67ad4f-d871-4630-bb13-b98d7214ce43 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:35.629281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:35.683406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558319c2-8928-417a-8880-7c219038ece7 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:35.717076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67ad4f-d871-4630-bb13-b98d7214ce43 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:35.731159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25249f-6a31-452e-9078-546a8de259ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:35.732649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25249f-6a31-452e-9078-546a8de259ef None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:35.733354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25249f-6a31-452e-9078-546a8de259ef None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:35.733805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25249f-6a31-452e-9078-546a8de259ef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:35.734804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25249f-6a31-452e-9078-546a8de259ef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:35.735718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:35.737516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25249f-6a31-452e-9078-546a8de259ef None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9f25249f-6a31-452e-9078-546a8de259ef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:35.751345 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:35.752493 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 20 20:39:35.753357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:35.754293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df9c5c-d868-4062-b4bd-61d647c2a9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:35.755253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df9c5c-d868-4062-b4bd-61d647c2a9e2 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:35.755614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df9c5c-d868-4062-b4bd-61d647c2a9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:35.755992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df9c5c-d868-4062-b4bd-61d647c2a9e2 None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:35.756392 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 390/787] 10.222.0.23 () {56 vars in 1129 bytes} [Sat Aug 20 20:39:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:35.757037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df9c5c-d868-4062-b4bd-61d647c2a9e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:35.757424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df9c5c-d868-4062-b4bd-61d647c2a9e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:35.758068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fba3e-2279-4935-8b5e-37578156265d None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:35.758484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fba3e-2279-4935-8b5e-37578156265d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:35.758889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fba3e-2279-4935-8b5e-37578156265d None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:35.759253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df9c5c-d868-4062-b4bd-61d647c2a9e2 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:35.760725 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 391/788] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:35.765458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fba3e-2279-4935-8b5e-37578156265d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:35.781644 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 398/789] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:35.783216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9e19f-65cf-417a-b5df-ba88eb7de0e7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:35.783694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9e19f-65cf-417a-b5df-ba88eb7de0e7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:35.784184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9e19f-65cf-417a-b5df-ba88eb7de0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:35.810036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e8b9e19f-65cf-417a-b5df-ba88eb7de0e7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:35.812005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e8b9e19f-65cf-417a-b5df-ba88eb7de0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:35.907583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fba3e-2279-4935-8b5e-37578156265d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:35.909160 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 399/790] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:35.911345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9e19f-65cf-417a-b5df-ba88eb7de0e7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:35.940034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:35.976521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6b6fc-8694-4500-bf6a-93b1157197f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:35.977123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6b6fc-8694-4500-bf6a-93b1157197f1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:35.977395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6b6fc-8694-4500-bf6a-93b1157197f1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:35.980393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6b6fc-8694-4500-bf6a-93b1157197f1 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:35.980393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6b6fc-8694-4500-bf6a-93b1157197f1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:35.980393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6b6fc-8694-4500-bf6a-93b1157197f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:35.980393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6b6fc-8694-4500-bf6a-93b1157197f1 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:35.993525 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 392/791] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Aug 20 20:39:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:36.000081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6b6fc-8694-4500-bf6a-93b1157197f1 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:36.129365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9e19f-65cf-417a-b5df-ba88eb7de0e7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:36.148850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:36.169535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298e554-565a-4108-9513-cd91cb0463bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:36.170194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298e554-565a-4108-9513-cd91cb0463bb None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:36.170460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298e554-565a-4108-9513-cd91cb0463bb None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:36.170720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298e554-565a-4108-9513-cd91cb0463bb None admin] PATH_INFO: `/v3/projects/3b6c88840d764c2385aa678da4338213/users/d74f7cdc155a4397ad28a3fbca86f7f9/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:36.172622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298e554-565a-4108-9513-cd91cb0463bb None admin] RBAC: Authorizing `identity:create_grant(project_id=3b6c88840d764c2385aa678da4338213, user_id=d74f7cdc155a4397ad28a3fbca86f7f9, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:36.174047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298e554-565a-4108-9513-cd91cb0463bb None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:36.175276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6b6fc-8694-4500-bf6a-93b1157197f1 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:36.196454 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 400/792] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:39:35 2022] PUT /identity/v3/projects/3b6c88840d764c2385aa678da4338213/users/d74f7cdc155a4397ad28a3fbca86f7f9/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:39:36.223909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb03ff9-8eea-4433-a7ce-74bb48e202eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:36.224543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb03ff9-8eea-4433-a7ce-74bb48e202eb None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:36.224861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb03ff9-8eea-4433-a7ce-74bb48e202eb None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:36.225128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb03ff9-8eea-4433-a7ce-74bb48e202eb None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:36.225748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb03ff9-8eea-4433-a7ce-74bb48e202eb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:36.226040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb03ff9-8eea-4433-a7ce-74bb48e202eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:36.227457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb03ff9-8eea-4433-a7ce-74bb48e202eb None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:36.246811 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 393/793] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Aug 20 20:39:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:36.252883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298e554-565a-4108-9513-cd91cb0463bb None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:36.374088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298e554-565a-4108-9513-cd91cb0463bb None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:36.397741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb03ff9-8eea-4433-a7ce-74bb48e202eb None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:36.416907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47941d78-f6b0-470e-aad1-3dc959385713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:36.417640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47941d78-f6b0-470e-aad1-3dc959385713 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:36.417945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47941d78-f6b0-470e-aad1-3dc959385713 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:36.418215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47941d78-f6b0-470e-aad1-3dc959385713 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:36.419210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47941d78-f6b0-470e-aad1-3dc959385713 None admin] RBAC: Adding query filter params (interface=public, service_id=5ed48038e5f04a84814349bea72350fe, region_id=RegionOne) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:36.419404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47941d78-f6b0-470e-aad1-3dc959385713 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:36.421010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47941d78-f6b0-470e-aad1-3dc959385713 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:36.430019 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 401/794] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Aug 20 20:39:36 2022] GET /identity/v3/endpoints?service_id=5ed48038e5f04a84814349bea72350fe&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:36.437813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47941d78-f6b0-470e-aad1-3dc959385713 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:36.474553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:36.477312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:36.558891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb03ff9-8eea-4433-a7ce-74bb48e202eb None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:36.560079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47941d78-f6b0-470e-aad1-3dc959385713 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:36.601827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e0a91-e52c-447e-94dd-467f3b579e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:36.602707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e0a91-e52c-447e-94dd-467f3b579e20 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:36.603083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e0a91-e52c-447e-94dd-467f3b579e20 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:36.603791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e0a91-e52c-447e-94dd-467f3b579e20 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:36.604428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e0a91-e52c-447e-94dd-467f3b579e20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:36.606030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e0a91-e52c-447e-94dd-467f3b579e20 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:36.606911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa7079-24a1-4c93-a29e-0238ad361924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:36.607642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa7079-24a1-4c93-a29e-0238ad361924 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:36.608014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa7079-24a1-4c93-a29e-0238ad361924 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:36.608404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa7079-24a1-4c93-a29e-0238ad361924 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:36.616485 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 402/795] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Aug 20 20:39:36 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:36.618404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a86438-fa41-4bcd-b5af-be168b426677 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:36.618711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a86438-fa41-4bcd-b5af-be168b426677 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:36.619015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a86438-fa41-4bcd-b5af-be168b426677 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:36.622253 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 403/796] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:36.629835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa7079-24a1-4c93-a29e-0238ad361924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:36.630110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b903-69bd-4fa0-896d-235f53dfaa29 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:36.630197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b903-69bd-4fa0-896d-235f53dfaa29 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:36.630429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b903-69bd-4fa0-896d-235f53dfaa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9caa7079-24a1-4c93-a29e-0238ad361924 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:36.635356 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:36.636866 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:36.638525 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:36.638525 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:36.638525 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:36.638525 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:36.638525 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:36.638525 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:36.638525 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:36.638525 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:36.638525 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:36.638525 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:36.638525 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:36.638525 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:36.638525 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 394/797] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:36.645349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa7079-24a1-4c93-a29e-0238ad361924 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:36.662506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e8f2b903-69bd-4fa0-896d-235f53dfaa29 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:36.663700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e8f2b903-69bd-4fa0-896d-235f53dfaa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:36.741207 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 404/798] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:36.847522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa7079-24a1-4c93-a29e-0238ad361924 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:36.856402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:36.885783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f5f7f-bfbe-4cc7-ad87-42b1c4671cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:36.886388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f5f7f-bfbe-4cc7-ad87-42b1c4671cd5 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:36.886665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f5f7f-bfbe-4cc7-ad87-42b1c4671cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:36.886932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f5f7f-bfbe-4cc7-ad87-42b1c4671cd5 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:36.887595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f5f7f-bfbe-4cc7-ad87-42b1c4671cd5 None admin] RBAC: Adding query filter params (name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:36.887901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f5f7f-bfbe-4cc7-ad87-42b1c4671cd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:36.889403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f5f7f-bfbe-4cc7-ad87-42b1c4671cd5 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:36.898337 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 395/799] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:36.905310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2b903-69bd-4fa0-896d-235f53dfaa29 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:36.936924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:37.008123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2b903-69bd-4fa0-896d-235f53dfaa29 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:37.042821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db562d7b-c1be-45db-b525-1c7ee7619402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:37.043454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db562d7b-c1be-45db-b525-1c7ee7619402 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:37.043725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db562d7b-c1be-45db-b525-1c7ee7619402 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:37.043993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db562d7b-c1be-45db-b525-1c7ee7619402 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:37.054821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db562d7b-c1be-45db-b525-1c7ee7619402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:37.056337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db562d7b-c1be-45db-b525-1c7ee7619402 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-db562d7b-c1be-45db-b525-1c7ee7619402 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:37.072221 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:37.073356 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:37.074690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:37.074690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:37.074690 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:37.074690 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 405/800] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:37.081539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f5f7f-bfbe-4cc7-ad87-42b1c4671cd5 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:37.197552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f5f7f-bfbe-4cc7-ad87-42b1c4671cd5 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:37.215510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db562d7b-c1be-45db-b525-1c7ee7619402 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:37.263099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4cfb83-6264-4d8a-b199-ccaa70f3e01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:37.263703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cfb83-6264-4d8a-b199-ccaa70f3e01a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:37.263974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cfb83-6264-4d8a-b199-ccaa70f3e01a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:37.264239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cfb83-6264-4d8a-b199-ccaa70f3e01a None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:37.264910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4cfb83-6264-4d8a-b199-ccaa70f3e01a None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:37.265206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4cfb83-6264-4d8a-b199-ccaa70f3e01a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:37.266572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4cfb83-6264-4d8a-b199-ccaa70f3e01a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:37.291044 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 396/801] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:37.297392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4cfb83-6264-4d8a-b199-ccaa70f3e01a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:37.332723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:37.349596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:37.385139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db562d7b-c1be-45db-b525-1c7ee7619402 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:37.433339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4cfb83-6264-4d8a-b199-ccaa70f3e01a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:37.461010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92f204-aa78-4055-b855-7a5ca76e8c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:37.461574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92f204-aa78-4055-b855-7a5ca76e8c02 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:37.461841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92f204-aa78-4055-b855-7a5ca76e8c02 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:37.462098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92f204-aa78-4055-b855-7a5ca76e8c02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:37.478076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d044b84-29b7-4d82-97d4-4c51e628e91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:37.479068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d044b84-29b7-4d82-97d4-4c51e628e91a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:37.479346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d044b84-29b7-4d82-97d4-4c51e628e91a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:37.479612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d044b84-29b7-4d82-97d4-4c51e628e91a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:37.487222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92f204-aa78-4055-b855-7a5ca76e8c02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-1c92f204-aa78-4055-b855-7a5ca76e8c02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:37.493475 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:37.494489 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:37.495312 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:37.495312 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:37.495312 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:37.495312 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:37.495312 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:37.495312 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:37.495312 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:37.495312 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:37.495312 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:37.495312 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:37.495312 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:39:37.495312 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:37.498214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d044b84-29b7-4d82-97d4-4c51e628e91a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:37.499661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d044b84-29b7-4d82-97d4-4c51e628e91a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:37.501536 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 406/802] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:39:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:37.508621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92f204-aa78-4055-b855-7a5ca76e8c02 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9d044b84-29b7-4d82-97d4-4c51e628e91a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:37.533411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:37.535814 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:37.538073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:37.538073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:37.538073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:37.538073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:37.538073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:37.538073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:37.538073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:37.538073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:37.538073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 20 20:39:37.538073 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:37.538073 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 397/803] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Aug 20 20:39:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:37.543237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d044b84-29b7-4d82-97d4-4c51e628e91a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:37.654834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:37.668139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d044b84-29b7-4d82-97d4-4c51e628e91a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:37.703446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740d5a7-9f8e-4477-8588-fbefff0ff525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:37.704047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740d5a7-9f8e-4477-8588-fbefff0ff525 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:37.704320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740d5a7-9f8e-4477-8588-fbefff0ff525 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:37.704890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740d5a7-9f8e-4477-8588-fbefff0ff525 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:37.705628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740d5a7-9f8e-4477-8588-fbefff0ff525 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:37.706053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740d5a7-9f8e-4477-8588-fbefff0ff525 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:37.707689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740d5a7-9f8e-4477-8588-fbefff0ff525 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:37.726188 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 398/804] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Aug 20 20:39:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:37.731854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740d5a7-9f8e-4477-8588-fbefff0ff525 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:37.779070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92f204-aa78-4055-b855-7a5ca76e8c02 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:37.850797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740d5a7-9f8e-4477-8588-fbefff0ff525 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:37.900550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214b199-f500-4272-91f6-d4c4c40f775e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:37.901079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214b199-f500-4272-91f6-d4c4c40f775e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:37.901351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214b199-f500-4272-91f6-d4c4c40f775e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:37.901615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214b199-f500-4272-91f6-d4c4c40f775e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:37.909167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006d993-0b25-4122-ac82-23cfe5fdf0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:37.909788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006d993-0b25-4122-ac82-23cfe5fdf0a2 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:37.910056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006d993-0b25-4122-ac82-23cfe5fdf0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:37.910316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006d993-0b25-4122-ac82-23cfe5fdf0a2 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:37.911081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006d993-0b25-4122-ac82-23cfe5fdf0a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:37.911376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006d993-0b25-4122-ac82-23cfe5fdf0a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:37.912753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006d993-0b25-4122-ac82-23cfe5fdf0a2 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:37.912861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214b199-f500-4272-91f6-d4c4c40f775e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:37.914511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214b199-f500-4272-91f6-d4c4c40f775e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:37.924735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:37.932670 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 407/805] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:39:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9214b199-f500-4272-91f6-d4c4c40f775e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:37.933647 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:37.934891 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:37.935780 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:37.935780 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:37.935780 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:37.936629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:37.941314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006d993-0b25-4122-ac82-23cfe5fdf0a2 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:37.943364 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 399/806] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:37.946208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214b199-f500-4272-91f6-d4c4c40f775e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:38.071115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214b199-f500-4272-91f6-d4c4c40f775e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:38.090098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006d993-0b25-4122-ac82-23cfe5fdf0a2 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:38.115690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b13a6-39d8-41cf-9fc0-53a59a53c29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:38.116477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b13a6-39d8-41cf-9fc0-53a59a53c29a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:38.116755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b13a6-39d8-41cf-9fc0-53a59a53c29a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:38.117336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b13a6-39d8-41cf-9fc0-53a59a53c29a None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:38.118250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b13a6-39d8-41cf-9fc0-53a59a53c29a None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:38.118550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b13a6-39d8-41cf-9fc0-53a59a53c29a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:38.120672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b13a6-39d8-41cf-9fc0-53a59a53c29a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:38.125119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da206d1f-ba01-49dd-b089-5c705fe813f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:38.125859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da206d1f-ba01-49dd-b089-5c705fe813f1 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:38.126193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da206d1f-ba01-49dd-b089-5c705fe813f1 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:38.126521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da206d1f-ba01-49dd-b089-5c705fe813f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:38.140472 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 400/807] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:38.141951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf929b6-1567-4d34-aaa8-92494affcaa4 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:38.142299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf929b6-1567-4d34-aaa8-92494affcaa4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:38.142560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf929b6-1567-4d34-aaa8-92494affcaa4 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:38.143979 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 401/808] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:38.144531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf929b6-1567-4d34-aaa8-92494affcaa4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:38.145898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da206d1f-ba01-49dd-b089-5c705fe813f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:38.148136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da206d1f-ba01-49dd-b089-5c705fe813f1 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-da206d1f-ba01-49dd-b089-5c705fe813f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:38.168678 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:38.169796 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:38.170702 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:38.170702 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:38.170702 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:38.177302 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 408/809] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:38.178755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358ad95-502e-41ad-818d-e4b3fe6de81a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:38.179025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358ad95-502e-41ad-818d-e4b3fe6de81a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:38.179287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358ad95-502e-41ad-818d-e4b3fe6de81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:38.197942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7358ad95-502e-41ad-818d-e4b3fe6de81a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:38.199111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7358ad95-502e-41ad-818d-e4b3fe6de81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:38.233595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:38.264794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:38.278449 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 409/810] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:38.279613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358ad95-502e-41ad-818d-e4b3fe6de81a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:38.292795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf929b6-1567-4d34-aaa8-92494affcaa4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:38.340580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e561b9-cee5-4f77-a058-995ac9496809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:38.341444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e561b9-cee5-4f77-a058-995ac9496809 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:38.341995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e561b9-cee5-4f77-a058-995ac9496809 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:38.342409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e561b9-cee5-4f77-a058-995ac9496809 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:38.351857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e561b9-cee5-4f77-a058-995ac9496809 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:38.354215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e561b9-cee5-4f77-a058-995ac9496809 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-a4e561b9-cee5-4f77-a058-995ac9496809 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:38.368830 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:38.369954 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:38.371046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:38.371046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:38.371046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:38.371046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:38.371046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:38.371046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:38.371046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:38.371046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:38.377687 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 402/811] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:38.385777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e561b9-cee5-4f77-a058-995ac9496809 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:38.402367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358ad95-502e-41ad-818d-e4b3fe6de81a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:38.441432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5f629-2c6d-4ad3-9eaa-484e83df2286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:38.442052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5f629-2c6d-4ad3-9eaa-484e83df2286 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:38.442326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5f629-2c6d-4ad3-9eaa-484e83df2286 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:38.442591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5f629-2c6d-4ad3-9eaa-484e83df2286 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:38.443408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5f629-2c6d-4ad3-9eaa-484e83df2286 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:38.443713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5f629-2c6d-4ad3-9eaa-484e83df2286 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:38.445273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5f629-2c6d-4ad3-9eaa-484e83df2286 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:38.459406 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 410/812] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:38.465233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5f629-2c6d-4ad3-9eaa-484e83df2286 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:38.520729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:38.613216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e561b9-cee5-4f77-a058-995ac9496809 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:38.632336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:38.647505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5f629-2c6d-4ad3-9eaa-484e83df2286 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:38.663391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35fb0e-767b-4e51-a16d-9c108e3f7358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:38.664019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35fb0e-767b-4e51-a16d-9c108e3f7358 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:38.664267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35fb0e-767b-4e51-a16d-9c108e3f7358 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:38.664565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35fb0e-767b-4e51-a16d-9c108e3f7358 None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:38.665254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35fb0e-767b-4e51-a16d-9c108e3f7358 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:38.665557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35fb0e-767b-4e51-a16d-9c108e3f7358 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:38.667035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35fb0e-767b-4e51-a16d-9c108e3f7358 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:38.683309 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 403/813] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:38.684575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35fb0e-767b-4e51-a16d-9c108e3f7358 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:38.695629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6dbff-4d78-4c25-9e7b-1f764dcbf01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:38.696259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6dbff-4d78-4c25-9e7b-1f764dcbf01f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:38.696616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6dbff-4d78-4c25-9e7b-1f764dcbf01f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:38.696885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6dbff-4d78-4c25-9e7b-1f764dcbf01f None admin] PATH_INFO: `/v3/users/nova` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:38.707004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6dbff-4d78-4c25-9e7b-1f764dcbf01f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:38.708488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6dbff-4d78-4c25-9e7b-1f764dcbf01f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-5ee6dbff-4d78-4c25-9e7b-1f764dcbf01f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:38.724504 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:38.725491 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:38.726861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:38.726861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:38.726861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:38.726861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:38.726861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:38.726861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:38.726861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:38.726861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:38.726861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:39:38.726861 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:38.726861 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 411/814] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Aug 20 20:39:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:38.726861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6dbff-4d78-4c25-9e7b-1f764dcbf01f None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:38.796089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35fb0e-767b-4e51-a16d-9c108e3f7358 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:38.832498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc3e57-b15c-4027-80ca-c5b898a061e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:38.833093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc3e57-b15c-4027-80ca-c5b898a061e6 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:38.833365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc3e57-b15c-4027-80ca-c5b898a061e6 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:38.833662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc3e57-b15c-4027-80ca-c5b898a061e6 None admin] PATH_INFO: `/v3/services/network` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:38.834211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc3e57-b15c-4027-80ca-c5b898a061e6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:38.836102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc3e57-b15c-4027-80ca-c5b898a061e6 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:38.837509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6dbff-4d78-4c25-9e7b-1f764dcbf01f None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:38.839535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-15fc3e57-b15c-4027-80ca-c5b898a061e6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:38.845134 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:38.846181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:38.847013 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 404/815] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:38 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:38.847807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc3e57-b15c-4027-80ca-c5b898a061e6 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:38.879030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b35bad-f504-4241-8fe0-6766e3e8015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:38.879931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b35bad-f504-4241-8fe0-6766e3e8015f None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:38.880200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b35bad-f504-4241-8fe0-6766e3e8015f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:38.880484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b35bad-f504-4241-8fe0-6766e3e8015f None admin] PATH_INFO: `/v3/projects/3b6c88840d764c2385aa678da4338213/users/1547cb99abe642ae86db8de1fa80041f/roles/04bd1260156b4de9bc4feff892a644a6` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:38.893975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:38.896396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b35bad-f504-4241-8fe0-6766e3e8015f None admin] RBAC: Authorizing `identity:create_grant(project_id=3b6c88840d764c2385aa678da4338213, user_id=1547cb99abe642ae86db8de1fa80041f, role_id=04bd1260156b4de9bc4feff892a644a6)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:38.897972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b35bad-f504-4241-8fe0-6766e3e8015f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:38.913255 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 412/816] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:39:38 2022] PUT /identity/v3/projects/3b6c88840d764c2385aa678da4338213/users/1547cb99abe642ae86db8de1fa80041f/roles/04bd1260156b4de9bc4feff892a644a6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:39:38.914501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000afcd-9813-48ed-84a2-b551f50aa166 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:38.914781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000afcd-9813-48ed-84a2-b551f50aa166 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:38.915048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000afcd-9813-48ed-84a2-b551f50aa166 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:38.920538 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 413/817] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:38.920538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000afcd-9813-48ed-84a2-b551f50aa166 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:38.966990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc3e57-b15c-4027-80ca-c5b898a061e6 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:39.005363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1a015-0e7a-404c-a9f8-00af495c2b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:39.006341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1a015-0e7a-404c-a9f8-00af495c2b97 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:39.006612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1a015-0e7a-404c-a9f8-00af495c2b97 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:39.006875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1a015-0e7a-404c-a9f8-00af495c2b97 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:39.007634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1a015-0e7a-404c-a9f8-00af495c2b97 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:39.007908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1a015-0e7a-404c-a9f8-00af495c2b97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:39.009399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1a015-0e7a-404c-a9f8-00af495c2b97 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:39.034253 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 405/818] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Aug 20 20:39:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:39.035541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778d047-3f03-4ba9-a450-b778e05de03c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:39.035818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778d047-3f03-4ba9-a450-b778e05de03c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:39.036082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778d047-3f03-4ba9-a450-b778e05de03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:39.058761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7778d047-3f03-4ba9-a450-b778e05de03c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:39.060854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7778d047-3f03-4ba9-a450-b778e05de03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:39.074276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000afcd-9813-48ed-84a2-b551f50aa166 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:39.112925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11812864-381d-4958-a912-0b67c3072385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:39.114276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11812864-381d-4958-a912-0b67c3072385 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:39.114566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11812864-381d-4958-a912-0b67c3072385 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:39.114833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11812864-381d-4958-a912-0b67c3072385 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:39.115470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11812864-381d-4958-a912-0b67c3072385 None admin] RBAC: Adding query filter params (name=network) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:39.115767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11812864-381d-4958-a912-0b67c3072385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:39.118084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11812864-381d-4958-a912-0b67c3072385 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:39.123421 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 414/819] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Aug 20 20:39:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:39.124024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11812864-381d-4958-a912-0b67c3072385 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:39.143753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:39.152533 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 406/820] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:39.153448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778d047-3f03-4ba9-a450-b778e05de03c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:39.200653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:39.253268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11812864-381d-4958-a912-0b67c3072385 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:39.319634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d35be-e6fe-4b2c-aecd-34bcbc89a287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:39.320285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d35be-e6fe-4b2c-aecd-34bcbc89a287 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:39.320586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d35be-e6fe-4b2c-aecd-34bcbc89a287 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:39.320848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d35be-e6fe-4b2c-aecd-34bcbc89a287 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:39.338640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778d047-3f03-4ba9-a450-b778e05de03c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:39.399327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d35be-e6fe-4b2c-aecd-34bcbc89a287 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:39.400965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d35be-e6fe-4b2c-aecd-34bcbc89a287 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:39.419990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc371b4e-4762-4b60-8d8a-5caf0c5bc9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:39.420767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc371b4e-4762-4b60-8d8a-5caf0c5bc9a3 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:39.421038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc371b4e-4762-4b60-8d8a-5caf0c5bc9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:39.421445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc371b4e-4762-4b60-8d8a-5caf0c5bc9a3 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:39.422070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc371b4e-4762-4b60-8d8a-5caf0c5bc9a3 None admin] RBAC: Adding query filter params (type=network) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:39.422365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc371b4e-4762-4b60-8d8a-5caf0c5bc9a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:39.423721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc371b4e-4762-4b60-8d8a-5caf0c5bc9a3 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:39.424652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:39.434673 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 407/821] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Aug 20 20:39:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:39.434673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaa221-0eab-497a-9330-0862d67e287f None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:39.434673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaa221-0eab-497a-9330-0862d67e287f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:39.434673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaa221-0eab-497a-9330-0862d67e287f None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:39.434673 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 408/822] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:39.435785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adaa221-0eab-497a-9330-0862d67e287f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:39.535527 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 415/823] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:39:39.537845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bd5a8-90c6-42bf-b87a-ba0e433c84a3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:39.538377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bd5a8-90c6-42bf-b87a-ba0e433c84a3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:39.538819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bd5a8-90c6-42bf-b87a-ba0e433c84a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:39.539612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:39.548954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adaa221-0eab-497a-9330-0862d67e287f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:39.563883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1e6bd5a8-90c6-42bf-b87a-ba0e433c84a3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:39.565510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1e6bd5a8-90c6-42bf-b87a-ba0e433c84a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:39.584594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631832b-9c6d-4a1c-9870-412fa77c5546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:39.585415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631832b-9c6d-4a1c-9870-412fa77c5546 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:39.585834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631832b-9c6d-4a1c-9870-412fa77c5546 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:39.586244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631832b-9c6d-4a1c-9870-412fa77c5546 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:39.587038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631832b-9c6d-4a1c-9870-412fa77c5546 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:39.588675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631832b-9c6d-4a1c-9870-412fa77c5546 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:39.620408 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 409/824] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:39:39 2022] POST /identity/v3/endpoints => generated 337 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:39:39.620408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631832b-9c6d-4a1c-9870-412fa77c5546 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:39.668396 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 416/825] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:39.669402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bd5a8-90c6-42bf-b87a-ba0e433c84a3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:39.727098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:39.788231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631832b-9c6d-4a1c-9870-412fa77c5546 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:39.797093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bd5a8-90c6-42bf-b87a-ba0e433c84a3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:39.839046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b87626-23d8-4404-8606-6dec1171b112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:39.839805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b87626-23d8-4404-8606-6dec1171b112 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:39.840068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b87626-23d8-4404-8606-6dec1171b112 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:39.840435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b87626-23d8-4404-8606-6dec1171b112 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:39.851701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b87626-23d8-4404-8606-6dec1171b112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-69b87626-23d8-4404-8606-6dec1171b112 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:39.856430 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:39.857363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:39.858193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:39.858193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:39.858193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:39.858193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:39.858193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:39.858193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:39.858193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:39.858193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:39.858193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:39.858193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:39.858193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:39:39.858193 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:39.858193 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 417/826] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:39:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:39.864505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b87626-23d8-4404-8606-6dec1171b112 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:39.876948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0253cfc-a5fb-4939-8783-f106499e74fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:39.877996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0253cfc-a5fb-4939-8783-f106499e74fe None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:39.878474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0253cfc-a5fb-4939-8783-f106499e74fe None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:39.878960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0253cfc-a5fb-4939-8783-f106499e74fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:39.892656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0253cfc-a5fb-4939-8783-f106499e74fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:39.894549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0253cfc-a5fb-4939-8783-f106499e74fe None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-c0253cfc-a5fb-4939-8783-f106499e74fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:39.910658 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:39.911662 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:39.913029 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:39.913029 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:39.913029 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:39.915005 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 410/827] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:39.925446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0253cfc-a5fb-4939-8783-f106499e74fe None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:39.929810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:39.987174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b87626-23d8-4404-8606-6dec1171b112 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:39.988748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:40.033705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483c17c-3a58-4c24-a5e4-7fcb5941daf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:40.034313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483c17c-3a58-4c24-a5e4-7fcb5941daf7 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:40.034586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483c17c-3a58-4c24-a5e4-7fcb5941daf7 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:40.034845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483c17c-3a58-4c24-a5e4-7fcb5941daf7 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:40.035503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483c17c-3a58-4c24-a5e4-7fcb5941daf7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:40.035796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483c17c-3a58-4c24-a5e4-7fcb5941daf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:40.037186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483c17c-3a58-4c24-a5e4-7fcb5941daf7 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:40.046848 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 418/828] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:39:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:40.052714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483c17c-3a58-4c24-a5e4-7fcb5941daf7 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:40.054925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0253cfc-a5fb-4939-8783-f106499e74fe None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:40.090985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ceccc-0ea8-4fa2-8420-38690ad28279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:40.091597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ceccc-0ea8-4fa2-8420-38690ad28279 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:40.091866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ceccc-0ea8-4fa2-8420-38690ad28279 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:40.092127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ceccc-0ea8-4fa2-8420-38690ad28279 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:40.092793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ceccc-0ea8-4fa2-8420-38690ad28279 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:40.093141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ceccc-0ea8-4fa2-8420-38690ad28279 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:40.094560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ceccc-0ea8-4fa2-8420-38690ad28279 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:40.108126 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 411/829] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:40.109036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ceccc-0ea8-4fa2-8420-38690ad28279 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:40.163080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483c17c-3a58-4c24-a5e4-7fcb5941daf7 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:40.170164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:40.201587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954deb0f-98e9-4ead-b356-250cfed020d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:40.202250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954deb0f-98e9-4ead-b356-250cfed020d1 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:40.202519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954deb0f-98e9-4ead-b356-250cfed020d1 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:40.202775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954deb0f-98e9-4ead-b356-250cfed020d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:40.209885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ceccc-0ea8-4fa2-8420-38690ad28279 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:40.215474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954deb0f-98e9-4ead-b356-250cfed020d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:40.216990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954deb0f-98e9-4ead-b356-250cfed020d1 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-954deb0f-98e9-4ead-b356-250cfed020d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:40.237873 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:40.238996 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:40.240736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:40.240736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:40.240736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:40.240736 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 419/830] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:40.240736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954deb0f-98e9-4ead-b356-250cfed020d1 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:40.254832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c500ca2-8f57-4486-89ac-1404ab6b9c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:40.255453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c500ca2-8f57-4486-89ac-1404ab6b9c51 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:40.255979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c500ca2-8f57-4486-89ac-1404ab6b9c51 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:40.256089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c500ca2-8f57-4486-89ac-1404ab6b9c51 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:40.256664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c500ca2-8f57-4486-89ac-1404ab6b9c51 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:40.258121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c500ca2-8f57-4486-89ac-1404ab6b9c51 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:40.263021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-5c500ca2-8f57-4486-89ac-1404ab6b9c51 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:40.268503 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:40.269544 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:40.270399 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 412/831] 10.222.0.23 () {56 vars in 1129 bytes} [Sat Aug 20 20:39:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:40.271216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c500ca2-8f57-4486-89ac-1404ab6b9c51 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:40.354570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954deb0f-98e9-4ead-b356-250cfed020d1 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:40.378469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c500ca2-8f57-4486-89ac-1404ab6b9c51 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:40.388470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2e3af-3aaa-4449-bb86-632cc13bdef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:40.389072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2e3af-3aaa-4449-bb86-632cc13bdef6 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:40.389324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2e3af-3aaa-4449-bb86-632cc13bdef6 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:40.389583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2e3af-3aaa-4449-bb86-632cc13bdef6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:40.396670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2e3af-3aaa-4449-bb86-632cc13bdef6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:40.398288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2e3af-3aaa-4449-bb86-632cc13bdef6 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:40.407447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-12f2e3af-3aaa-4449-bb86-632cc13bdef6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:40.410699 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:40.411582 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:40.413061 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:40.413061 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:40.413061 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:40.413061 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:40.413061 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:40.413061 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:40.413061 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:40.413061 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:40.413061 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 420/832] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:40.413061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2e3af-3aaa-4449-bb86-632cc13bdef6 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:40.422654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f542a7-8d3b-414b-ab1a-7265a2177cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:40.423253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f542a7-8d3b-414b-ab1a-7265a2177cf1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:40.423503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f542a7-8d3b-414b-ab1a-7265a2177cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:40.423766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f542a7-8d3b-414b-ab1a-7265a2177cf1 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:40.424414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f542a7-8d3b-414b-ab1a-7265a2177cf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:40.424720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f542a7-8d3b-414b-ab1a-7265a2177cf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:40.426072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f542a7-8d3b-414b-ab1a-7265a2177cf1 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:40.438411 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 413/833] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:40.439185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f542a7-8d3b-414b-ab1a-7265a2177cf1 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:40.501941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:40.532582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2e3af-3aaa-4449-bb86-632cc13bdef6 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:40.540940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f542a7-8d3b-414b-ab1a-7265a2177cf1 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:40.567316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7ded6-71e4-48e4-a268-284fe133c247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:40.568176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7ded6-71e4-48e4-a268-284fe133c247 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:40.568625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7ded6-71e4-48e4-a268-284fe133c247 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:40.569033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7ded6-71e4-48e4-a268-284fe133c247 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:40.569801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7ded6-71e4-48e4-a268-284fe133c247 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:40.570098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7ded6-71e4-48e4-a268-284fe133c247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:40.571617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7ded6-71e4-48e4-a268-284fe133c247 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:40.581030 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 421/834] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Aug 20 20:39:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:40.581030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7ded6-71e4-48e4-a268-284fe133c247 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:40.582919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8662203-bc6d-4356-bf31-b3d303c992e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:40.583535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8662203-bc6d-4356-bf31-b3d303c992e9 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:40.583807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8662203-bc6d-4356-bf31-b3d303c992e9 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:40.584078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8662203-bc6d-4356-bf31-b3d303c992e9 None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:40.584793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8662203-bc6d-4356-bf31-b3d303c992e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:40.585096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8662203-bc6d-4356-bf31-b3d303c992e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:40.586556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8662203-bc6d-4356-bf31-b3d303c992e9 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:40.599302 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 414/835] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:40.604980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8662203-bc6d-4356-bf31-b3d303c992e9 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:40.653662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:40.686698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7ded6-71e4-48e4-a268-284fe133c247 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:40.711327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8662203-bc6d-4356-bf31-b3d303c992e9 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:40.725196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6be5cb-27fe-41a7-947b-99179b2ebc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:40.726166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6be5cb-27fe-41a7-947b-99179b2ebc8a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:40.726615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6be5cb-27fe-41a7-947b-99179b2ebc8a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:40.727054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6be5cb-27fe-41a7-947b-99179b2ebc8a None admin] PATH_INFO: `/v3/users/swift` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:40.735042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6be5cb-27fe-41a7-947b-99179b2ebc8a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:40.736722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6be5cb-27fe-41a7-947b-99179b2ebc8a None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:40.739589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:40.749988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fcbf2-9b31-43e7-8974-712cac3e2c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:40.750598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fcbf2-9b31-43e7-8974-712cac3e2c2b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-0b6be5cb-27fe-41a7-947b-99179b2ebc8a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:40.750825 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:40.751722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:40.753342 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:40.753342 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:40.753342 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:40.753342 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:40.753342 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:40.753342 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:40.753342 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:40.753342 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:40.753342 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 20 20:39:40.753342 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:40.753906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fcbf2-9b31-43e7-8974-712cac3e2c2b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:40.753906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fcbf2-9b31-43e7-8974-712cac3e2c2b None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:40.753906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fcbf2-9b31-43e7-8974-712cac3e2c2b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:40.754098 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 422/836] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:39:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:40.754098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6be5cb-27fe-41a7-947b-99179b2ebc8a None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:40.756516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fcbf2-9b31-43e7-8974-712cac3e2c2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:40.758113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fcbf2-9b31-43e7-8974-712cac3e2c2b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:40.764401 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 415/837] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Aug 20 20:39:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:40.765214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fcbf2-9b31-43e7-8974-712cac3e2c2b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:40.870612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6be5cb-27fe-41a7-947b-99179b2ebc8a None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:40.881038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fcbf2-9b31-43e7-8974-712cac3e2c2b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:40.893521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:40.912046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a21992-9986-467a-95e9-648ca7b9e9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:40.912684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a21992-9986-467a-95e9-648ca7b9e9c4 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:40.912956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a21992-9986-467a-95e9-648ca7b9e9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:40.913220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a21992-9986-467a-95e9-648ca7b9e9c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:40.913935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a21992-9986-467a-95e9-648ca7b9e9c4 None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=df852af59e7743f7a8798d9edf8ebd3e) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:40.914237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a21992-9986-467a-95e9-648ca7b9e9c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:40.915796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a21992-9986-467a-95e9-648ca7b9e9c4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:40.918312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d5046-23f5-4f54-945d-5ccde37d80e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:40.919022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d5046-23f5-4f54-945d-5ccde37d80e6 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:40.919294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d5046-23f5-4f54-945d-5ccde37d80e6 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:40.919557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d5046-23f5-4f54-945d-5ccde37d80e6 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:40.920271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d5046-23f5-4f54-945d-5ccde37d80e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:40.920599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d5046-23f5-4f54-945d-5ccde37d80e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:40.932829 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 423/838] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:40 2022] GET /identity/v3/role_assignments?user.id=df852af59e7743f7a8798d9edf8ebd3e&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:40.933049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d5046-23f5-4f54-945d-5ccde37d80e6 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:40.933169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a21992-9986-467a-95e9-648ca7b9e9c4 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:40.958044 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 416/839] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Aug 20 20:39:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:40.959198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d5046-23f5-4f54-945d-5ccde37d80e6 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:41.008300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:41.062777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a21992-9986-467a-95e9-648ca7b9e9c4 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:41.090662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d5046-23f5-4f54-945d-5ccde37d80e6 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:41.106766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c2e0b-6ddb-4229-a500-b2ad4e45cc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:41.107485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c2e0b-6ddb-4229-a500-b2ad4e45cc42 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:41.107754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c2e0b-6ddb-4229-a500-b2ad4e45cc42 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:41.108017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c2e0b-6ddb-4229-a500-b2ad4e45cc42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:41.108713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c2e0b-6ddb-4229-a500-b2ad4e45cc42 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:41.110095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c2e0b-6ddb-4229-a500-b2ad4e45cc42 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:41.139792 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 424/840] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:39:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:39:41.149685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36323d44-7bb9-4962-8e2b-58ae7fd0e654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:41.150528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36323d44-7bb9-4962-8e2b-58ae7fd0e654 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:41.150920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36323d44-7bb9-4962-8e2b-58ae7fd0e654 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:41.151250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36323d44-7bb9-4962-8e2b-58ae7fd0e654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:41.194617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:41.242025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36323d44-7bb9-4962-8e2b-58ae7fd0e654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:41.243580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36323d44-7bb9-4962-8e2b-58ae7fd0e654 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:41.305382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e3431-93f9-40f2-bd98-1080c6a46e3a None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:41.305923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e3431-93f9-40f2-bd98-1080c6a46e3a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:41.306325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e3431-93f9-40f2-bd98-1080c6a46e3a None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:41.308806 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 425/841] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:39:41.314896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3028e-35b6-484e-8226-caea01e540fe None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:41.315199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3028e-35b6-484e-8226-caea01e540fe None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:41.315441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3028e-35b6-484e-8226-caea01e540fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:41.336820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0ea3028e-35b6-484e-8226-caea01e540fe None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:41.338667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0ea3028e-35b6-484e-8226-caea01e540fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:41.370853 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 417/842] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:39:41.375161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36323d44-7bb9-4962-8e2b-58ae7fd0e654 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:41.437314 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 426/843] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:39:41.438360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:41.476720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36323d44-7bb9-4962-8e2b-58ae7fd0e654 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:41.510959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097ccc8-f00b-4e0a-bbcb-52b6e457292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:41.511587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097ccc8-f00b-4e0a-bbcb-52b6e457292d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:41.511855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097ccc8-f00b-4e0a-bbcb-52b6e457292d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:41.512118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097ccc8-f00b-4e0a-bbcb-52b6e457292d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:41.522367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097ccc8-f00b-4e0a-bbcb-52b6e457292d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:41.523800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097ccc8-f00b-4e0a-bbcb-52b6e457292d None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-a097ccc8-f00b-4e0a-bbcb-52b6e457292d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:41.538491 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:41.539472 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:41.540411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:41.540411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:41.540411 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:41.540411 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 418/844] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:41.546868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3028e-35b6-484e-8226-caea01e540fe None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:41.600294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:41.652222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3028e-35b6-484e-8226-caea01e540fe None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:41.688035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50dd75-7c01-4887-90b9-05b195390286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:41.688662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50dd75-7c01-4887-90b9-05b195390286 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:41.688928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50dd75-7c01-4887-90b9-05b195390286 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:41.689186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50dd75-7c01-4887-90b9-05b195390286 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:41.689810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50dd75-7c01-4887-90b9-05b195390286 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:41.690101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50dd75-7c01-4887-90b9-05b195390286 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:41.691524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50dd75-7c01-4887-90b9-05b195390286 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:41.704637 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 427/845] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:41.710120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097ccc8-f00b-4e0a-bbcb-52b6e457292d None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:41.803488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50dd75-7c01-4887-90b9-05b195390286 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:41.813241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097ccc8-f00b-4e0a-bbcb-52b6e457292d None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:41.834727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:41.862022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da0931-6ec5-4656-a633-ecca8101b91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:41.862707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da0931-6ec5-4656-a633-ecca8101b91d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:41.863075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da0931-6ec5-4656-a633-ecca8101b91d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:41.863434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da0931-6ec5-4656-a633-ecca8101b91d None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:41.873841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da0931-6ec5-4656-a633-ecca8101b91d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:41.875710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da0931-6ec5-4656-a633-ecca8101b91d None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-13da0931-6ec5-4656-a633-ecca8101b91d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:41.891569 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:41.892993 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:41.893895 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:41.893895 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:41.893895 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:41.893895 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:41.893895 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:41.893895 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:41.893895 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:41.893895 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:41.893895 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 419/846] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:41.898798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da0931-6ec5-4656-a633-ecca8101b91d None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:41.918660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50dd75-7c01-4887-90b9-05b195390286 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:41.929645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:41.955630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20b57b-9b29-425a-9606-5689fce9635f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:41.956438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20b57b-9b29-425a-9606-5689fce9635f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:41.956816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20b57b-9b29-425a-9606-5689fce9635f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:41.957218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20b57b-9b29-425a-9606-5689fce9635f None admin] PATH_INFO: `/v3/roles/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:41.968282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20b57b-9b29-425a-9606-5689fce9635f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-9e20b57b-9b29-425a-9606-5689fce9635f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:41.973635 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:41.974548 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:41.975379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:41.975379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:41.975379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:41.975379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:41.975379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:41.975379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:41.975379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:41.975379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:41.975379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:41.975379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:41.975379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:39:41.975379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:41.979908 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 428/847] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:39:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:41.981007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20b57b-9b29-425a-9606-5689fce9635f None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:42.015168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da0931-6ec5-4656-a633-ecca8101b91d None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:42.068584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcfba6-bfc5-46bd-aef2-fcb4e7d94a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:42.069289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcfba6-bfc5-46bd-aef2-fcb4e7d94a7a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:42.069562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcfba6-bfc5-46bd-aef2-fcb4e7d94a7a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:42.069823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcfba6-bfc5-46bd-aef2-fcb4e7d94a7a None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:42.070501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcfba6-bfc5-46bd-aef2-fcb4e7d94a7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:42.070795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcfba6-bfc5-46bd-aef2-fcb4e7d94a7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:42.072498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcfba6-bfc5-46bd-aef2-fcb4e7d94a7a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:42.088541 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 420/848] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:42.094494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcfba6-bfc5-46bd-aef2-fcb4e7d94a7a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:42.102461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20b57b-9b29-425a-9606-5689fce9635f None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:42.131787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:42.149525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72f7c0-4d7b-4960-8a40-1df0f13ec14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:42.150976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72f7c0-4d7b-4960-8a40-1df0f13ec14b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:42.151416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72f7c0-4d7b-4960-8a40-1df0f13ec14b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:42.151838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72f7c0-4d7b-4960-8a40-1df0f13ec14b None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:42.152706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72f7c0-4d7b-4960-8a40-1df0f13ec14b None admin] RBAC: Adding query filter params (name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:42.153208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72f7c0-4d7b-4960-8a40-1df0f13ec14b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:42.155575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72f7c0-4d7b-4960-8a40-1df0f13ec14b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:42.171762 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 429/849] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:39:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:42.173899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72f7c0-4d7b-4960-8a40-1df0f13ec14b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:42.213795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcfba6-bfc5-46bd-aef2-fcb4e7d94a7a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:42.244954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:42.250702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b541a8-3942-4b38-a8c9-3df168eb007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:42.251440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b541a8-3942-4b38-a8c9-3df168eb007a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:42.251810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b541a8-3942-4b38-a8c9-3df168eb007a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:42.252419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b541a8-3942-4b38-a8c9-3df168eb007a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:42.253205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b541a8-3942-4b38-a8c9-3df168eb007a None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=d74f7cdc155a4397ad28a3fbca86f7f9) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:42.253609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b541a8-3942-4b38-a8c9-3df168eb007a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:42.255220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b541a8-3942-4b38-a8c9-3df168eb007a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:42.268780 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 421/850] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:42 2022] GET /identity/v3/role_assignments?user.id=d74f7cdc155a4397ad28a3fbca86f7f9&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:42.305718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72f7c0-4d7b-4960-8a40-1df0f13ec14b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:42.344308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab5021-2cb9-4441-ab2d-5570bca90a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:42.345282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab5021-2cb9-4441-ab2d-5570bca90a7e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:42.345711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab5021-2cb9-4441-ab2d-5570bca90a7e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:42.346130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab5021-2cb9-4441-ab2d-5570bca90a7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:42.358597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab5021-2cb9-4441-ab2d-5570bca90a7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:42.360401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab5021-2cb9-4441-ab2d-5570bca90a7e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-90ab5021-2cb9-4441-ab2d-5570bca90a7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:42.376299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:42.377660 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:42.378868 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:42.378868 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:42.378868 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:42.380731 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 430/851] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:42.387131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b541a8-3942-4b38-a8c9-3df168eb007a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:42.498331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:42.501736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b541a8-3942-4b38-a8c9-3df168eb007a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:42.537529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0472a6-6900-4ae3-9c61-749c61a47d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:42.538393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0472a6-6900-4ae3-9c61-749c61a47d6e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:42.538836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0472a6-6900-4ae3-9c61-749c61a47d6e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:42.539260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0472a6-6900-4ae3-9c61-749c61a47d6e None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:42.540049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0472a6-6900-4ae3-9c61-749c61a47d6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:42.540552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0472a6-6900-4ae3-9c61-749c61a47d6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:42.542297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0472a6-6900-4ae3-9c61-749c61a47d6e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:42.555415 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 422/852] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:42.561722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab5021-2cb9-4441-ab2d-5570bca90a7e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:42.656836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:42.665848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab5021-2cb9-4441-ab2d-5570bca90a7e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:42.701428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68cfc7-8164-4565-98d1-88b85893cf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:42.702050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68cfc7-8164-4565-98d1-88b85893cf96 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:42.702372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68cfc7-8164-4565-98d1-88b85893cf96 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:42.702645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68cfc7-8164-4565-98d1-88b85893cf96 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:42.711562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68cfc7-8164-4565-98d1-88b85893cf96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:42.713078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68cfc7-8164-4565-98d1-88b85893cf96 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-5c68cfc7-8164-4565-98d1-88b85893cf96 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:42.724701 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:42.726527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:42.728470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:42.728470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:42.728470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:42.728470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:42.728470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:42.728470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:42.728470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:42.728470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:42.728470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 20 20:39:42.728470 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:42.728470 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 431/853] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Aug 20 20:39:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:42.731998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0472a6-6900-4ae3-9c61-749c61a47d6e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:42.838651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0472a6-6900-4ae3-9c61-749c61a47d6e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:42.874529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bcee2-e7ca-424c-987e-da674d783895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:42.875139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bcee2-e7ca-424c-987e-da674d783895 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:42.875411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bcee2-e7ca-424c-987e-da674d783895 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:42.875673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bcee2-e7ca-424c-987e-da674d783895 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:42.876417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bcee2-e7ca-424c-987e-da674d783895 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:42.876722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bcee2-e7ca-424c-987e-da674d783895 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:42.878146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bcee2-e7ca-424c-987e-da674d783895 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:42.897413 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 423/854] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Aug 20 20:39:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:42.902907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68cfc7-8164-4565-98d1-88b85893cf96 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:43.007603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68cfc7-8164-4565-98d1-88b85893cf96 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:43.042147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02beb531-bafb-4128-bf5d-8a2016cfb884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:43.042750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beb531-bafb-4128-bf5d-8a2016cfb884 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:43.043072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beb531-bafb-4128-bf5d-8a2016cfb884 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:43.043352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beb531-bafb-4128-bf5d-8a2016cfb884 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:43.059289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:43.115330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02beb531-bafb-4128-bf5d-8a2016cfb884 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:43.116825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02beb531-bafb-4128-bf5d-8a2016cfb884 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:43.226548 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 432/855] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:42 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:39:43.232486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bcee2-e7ca-424c-987e-da674d783895 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:43.251830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:43.313710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea3fc0-ad3a-4d8e-b80b-70dd4c4f7c63 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:43.313916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea3fc0-ad3a-4d8e-b80b-70dd4c4f7c63 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:43.314148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea3fc0-ad3a-4d8e-b80b-70dd4c4f7c63 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:43.315745 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 433/856] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:39:43.320686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68be5f-699d-419b-8756-4a9563a49618 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:43.320964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68be5f-699d-419b-8756-4a9563a49618 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:43.321229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68be5f-699d-419b-8756-4a9563a49618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:43.338364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bcee2-e7ca-424c-987e-da674d783895 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:43.345011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4e68be5f-699d-419b-8756-4a9563a49618 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:43.346342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4e68be5f-699d-419b-8756-4a9563a49618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:43.387496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53516e-20d6-493f-8461-c673ce4604be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:43.388339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53516e-20d6-493f-8461-c673ce4604be None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:43.388655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53516e-20d6-493f-8461-c673ce4604be None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:43.389022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53516e-20d6-493f-8461-c673ce4604be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:43.404243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53516e-20d6-493f-8461-c673ce4604be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:43.406544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53516e-20d6-493f-8461-c673ce4604be None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:43.408741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-5f53516e-20d6-493f-8461-c673ce4604be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:43.427589 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:43.428622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:43.429514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:43.429514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:43.429514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:43.429514 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 424/857] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:43.435518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53516e-20d6-493f-8461-c673ce4604be None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:43.436345 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 434/858] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:39:43.512881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:43.548321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53516e-20d6-493f-8461-c673ce4604be None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:43.554070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e40768-c785-4ebd-8722-750f40aae77e None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:43.554342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e40768-c785-4ebd-8722-750f40aae77e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:43.554606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e40768-c785-4ebd-8722-750f40aae77e None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:43.560196 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 435/859] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:39:43.561682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951db63b-6acf-496c-b295-9f5a87e6dc68 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:43.561954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951db63b-6acf-496c-b295-9f5a87e6dc68 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:43.562216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951db63b-6acf-496c-b295-9f5a87e6dc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:43.581544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-951db63b-6acf-496c-b295-9f5a87e6dc68 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:43.582708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-951db63b-6acf-496c-b295-9f5a87e6dc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:43.589533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce279c-98b0-4aa8-b264-5f496bce029e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:43.590141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce279c-98b0-4aa8-b264-5f496bce029e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:43.590412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce279c-98b0-4aa8-b264-5f496bce029e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:43.590698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce279c-98b0-4aa8-b264-5f496bce029e None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:43.591810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce279c-98b0-4aa8-b264-5f496bce029e None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:43.592105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce279c-98b0-4aa8-b264-5f496bce029e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:43.593947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce279c-98b0-4aa8-b264-5f496bce029e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:43.611409 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 425/860] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:43.617155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce279c-98b0-4aa8-b264-5f496bce029e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:43.659557 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 436/861] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:39:43.729911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce279c-98b0-4aa8-b264-5f496bce029e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:43.759556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:43.764995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c5f07-2fd9-42ad-9deb-1e5f9cd48790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:43.765614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5f07-2fd9-42ad-9deb-1e5f9cd48790 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:43.765887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5f07-2fd9-42ad-9deb-1e5f9cd48790 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:43.766147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5f07-2fd9-42ad-9deb-1e5f9cd48790 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:43.772885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c5f07-2fd9-42ad-9deb-1e5f9cd48790 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:43.774323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c5f07-2fd9-42ad-9deb-1e5f9cd48790 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:43.781058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951db63b-6acf-496c-b295-9f5a87e6dc68 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-6b2c5f07-2fd9-42ad-9deb-1e5f9cd48790 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:43.787403 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:43.789289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:43.790439 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:43.790439 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:43.790439 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:43.790439 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:43.790439 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:43.790439 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:43.790439 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:43.790439 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:43.790439 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 426/862] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:43.795264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c5f07-2fd9-42ad-9deb-1e5f9cd48790 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:43.913092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c5f07-2fd9-42ad-9deb-1e5f9cd48790 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:43.913755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951db63b-6acf-496c-b295-9f5a87e6dc68 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:43.917856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:43.952711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75696b10-45f0-4e3e-9f98-120f56620425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:43.953511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75696b10-45f0-4e3e-9f98-120f56620425 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:43.954071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75696b10-45f0-4e3e-9f98-120f56620425 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:43.954245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75696b10-45f0-4e3e-9f98-120f56620425 None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:43.955040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75696b10-45f0-4e3e-9f98-120f56620425 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:43.955397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75696b10-45f0-4e3e-9f98-120f56620425 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:43.957000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75696b10-45f0-4e3e-9f98-120f56620425 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:43.963686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9624707-162a-4cf5-a143-1e16cb875e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:43.964592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9624707-162a-4cf5-a143-1e16cb875e9b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:43.965007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9624707-162a-4cf5-a143-1e16cb875e9b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:43.966000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9624707-162a-4cf5-a143-1e16cb875e9b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:43.966725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9624707-162a-4cf5-a143-1e16cb875e9b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:43.968918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9624707-162a-4cf5-a143-1e16cb875e9b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:43.976281 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 427/863] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:43.977292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75696b10-45f0-4e3e-9f98-120f56620425 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-b9624707-162a-4cf5-a143-1e16cb875e9b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:43.979355 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:43.980363 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:43.981733 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 437/864] 10.222.0.23 () {56 vars in 1132 bytes} [Sat Aug 20 20:39:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:43.982913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9624707-162a-4cf5-a143-1e16cb875e9b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:44.030347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:44.085294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75696b10-45f0-4e3e-9f98-120f56620425 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:44.093683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9624707-162a-4cf5-a143-1e16cb875e9b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:44.125082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a10c8-addc-4833-b8c1-452f0db61822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:44.125683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a10c8-addc-4833-b8c1-452f0db61822 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:44.125946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a10c8-addc-4833-b8c1-452f0db61822 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:44.126212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a10c8-addc-4833-b8c1-452f0db61822 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:44.127018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a10c8-addc-4833-b8c1-452f0db61822 None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=1547cb99abe642ae86db8de1fa80041f) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:44.127325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a10c8-addc-4833-b8c1-452f0db61822 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:44.128788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a10c8-addc-4833-b8c1-452f0db61822 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:44.129382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f2f05-3561-49b8-aca3-e0eae400fcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:44.130011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f2f05-3561-49b8-aca3-e0eae400fcca None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:44.130280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f2f05-3561-49b8-aca3-e0eae400fcca None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:44.130566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f2f05-3561-49b8-aca3-e0eae400fcca None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:44.131111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f2f05-3561-49b8-aca3-e0eae400fcca None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:44.132497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f2f05-3561-49b8-aca3-e0eae400fcca None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:44.145237 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 428/865] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:43 2022] GET /identity/v3/role_assignments?user.id=1547cb99abe642ae86db8de1fa80041f&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:44.145237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a10c8-addc-4833-b8c1-452f0db61822 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-b78f2f05-3561-49b8-aca3-e0eae400fcca None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:44.153829 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:44.154921 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:44.155755 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 438/866] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:39:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:44.161321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f2f05-3561-49b8-aca3-e0eae400fcca None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:44.208087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:44.271523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a10c8-addc-4833-b8c1-452f0db61822 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:44.288940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:44.316632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a9de1-31b0-4cab-a3fc-664e9ce24553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:44.317300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a9de1-31b0-4cab-a3fc-664e9ce24553 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:44.317585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a9de1-31b0-4cab-a3fc-664e9ce24553 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:44.317851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a9de1-31b0-4cab-a3fc-664e9ce24553 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:44.318484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a9de1-31b0-4cab-a3fc-664e9ce24553 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:44.318779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a9de1-31b0-4cab-a3fc-664e9ce24553 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:44.319492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f2f05-3561-49b8-aca3-e0eae400fcca None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:44.320284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a9de1-31b0-4cab-a3fc-664e9ce24553 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:44.330484 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 429/867] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Aug 20 20:39:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:44.333529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a9de1-31b0-4cab-a3fc-664e9ce24553 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:44.372363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ca203-efed-4934-8215-1afb894bbb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:44.373001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ca203-efed-4934-8215-1afb894bbb66 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:44.373266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ca203-efed-4934-8215-1afb894bbb66 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:44.373526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ca203-efed-4934-8215-1afb894bbb66 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:44.374136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ca203-efed-4934-8215-1afb894bbb66 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:44.374427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ca203-efed-4934-8215-1afb894bbb66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:44.375773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ca203-efed-4934-8215-1afb894bbb66 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:44.382146 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 439/868] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:39:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:44.389892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ca203-efed-4934-8215-1afb894bbb66 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:44.455807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a9de1-31b0-4cab-a3fc-664e9ce24553 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:44.503222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae251f6-e1ca-4214-8a9c-fddbf7e12a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:44.503854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:44.504826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae251f6-e1ca-4214-8a9c-fddbf7e12a5f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:44.505329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae251f6-e1ca-4214-8a9c-fddbf7e12a5f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:44.505630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae251f6-e1ca-4214-8a9c-fddbf7e12a5f None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:44.506599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae251f6-e1ca-4214-8a9c-fddbf7e12a5f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:44.506892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae251f6-e1ca-4214-8a9c-fddbf7e12a5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:44.508408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae251f6-e1ca-4214-8a9c-fddbf7e12a5f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:44.511563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ca203-efed-4934-8215-1afb894bbb66 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:44.513895 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 430/869] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Aug 20 20:39:44 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:44.561197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797404b-c779-433d-adbe-a73d860ba8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:44.561862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797404b-c779-433d-adbe-a73d860ba8a7 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:44.562140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797404b-c779-433d-adbe-a73d860ba8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:44.562404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797404b-c779-433d-adbe-a73d860ba8a7 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:44.563040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797404b-c779-433d-adbe-a73d860ba8a7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:44.563337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797404b-c779-433d-adbe-a73d860ba8a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:44.564755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797404b-c779-433d-adbe-a73d860ba8a7 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:44.571633 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 440/870] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:39:44 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:44.660621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cdb9f-c5c6-4c0b-a738-3be3b5491f99 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:44.660885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cdb9f-c5c6-4c0b-a738-3be3b5491f99 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:44.661152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cdb9f-c5c6-4c0b-a738-3be3b5491f99 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:44.662889 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 431/871] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:44.669023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efabfc-f2fe-4fc2-922b-df15257182cf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:44.669436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efabfc-f2fe-4fc2-922b-df15257182cf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:44.669802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efabfc-f2fe-4fc2-922b-df15257182cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:44.691016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-34efabfc-f2fe-4fc2-922b-df15257182cf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:44.692249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-34efabfc-f2fe-4fc2-922b-df15257182cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:44.771345 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 441/872] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:45.089490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cdb9f-c5c6-4c0b-a738-3be3b5491f99 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:45.137024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:45.199291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cdb9f-c5c6-4c0b-a738-3be3b5491f99 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:45.234646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f2bf6-adb4-4eca-9ff3-f2e7b162d1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:45.235276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f2bf6-adb4-4eca-9ff3-f2e7b162d1f2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:45.235555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f2bf6-adb4-4eca-9ff3-f2e7b162d1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:45.235790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f2bf6-adb4-4eca-9ff3-f2e7b162d1f2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:45.236390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f2bf6-adb4-4eca-9ff3-f2e7b162d1f2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:45.237757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f2bf6-adb4-4eca-9ff3-f2e7b162d1f2 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-1d0f2bf6-adb4-4eca-9ff3-f2e7b162d1f2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:45.247485 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:45.249835 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:45.251879 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 432/873] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Aug 20 20:39:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:45.258076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efabfc-f2fe-4fc2-922b-df15257182cf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:45.348798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:45.358873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efabfc-f2fe-4fc2-922b-df15257182cf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:45.395777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5010a9-05de-4ea4-87b6-43451a0cb4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:45.396479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5010a9-05de-4ea4-87b6-43451a0cb4be None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:45.396918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5010a9-05de-4ea4-87b6-43451a0cb4be None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:45.397206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5010a9-05de-4ea4-87b6-43451a0cb4be None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:45.397819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5010a9-05de-4ea4-87b6-43451a0cb4be None admin] RBAC: Adding query filter params (name=object-store) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:45.398120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5010a9-05de-4ea4-87b6-43451a0cb4be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:45.399499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5010a9-05de-4ea4-87b6-43451a0cb4be None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:45.405191 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 442/874] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Aug 20 20:39:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:45.411655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f2bf6-adb4-4eca-9ff3-f2e7b162d1f2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:45.517415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f2bf6-adb4-4eca-9ff3-f2e7b162d1f2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:45.529914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:45.559175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dbc8d-c3cc-4ed1-af83-e1d725c2510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:45.559783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dbc8d-c3cc-4ed1-af83-e1d725c2510f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:45.560052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dbc8d-c3cc-4ed1-af83-e1d725c2510f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:45.560316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dbc8d-c3cc-4ed1-af83-e1d725c2510f None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:45.560976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dbc8d-c3cc-4ed1-af83-e1d725c2510f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:45.561271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dbc8d-c3cc-4ed1-af83-e1d725c2510f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:45.562631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dbc8d-c3cc-4ed1-af83-e1d725c2510f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:45.567840 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 433/875] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Aug 20 20:39:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:46.655876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffc881-419a-495a-ba93-656d3c8bce46 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:46.655876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffc881-419a-495a-ba93-656d3c8bce46 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:46.655876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffc881-419a-495a-ba93-656d3c8bce46 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:46.655876 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 443/876] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:39:46.663004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf36489-fc2c-469b-9feb-4f8627cb704e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:46.667089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf36489-fc2c-469b-9feb-4f8627cb704e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:46.667089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf36489-fc2c-469b-9feb-4f8627cb704e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:46.683639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1bf36489-fc2c-469b-9feb-4f8627cb704e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:46.685510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1bf36489-fc2c-469b-9feb-4f8627cb704e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:46.760785 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 434/877] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:46.817722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4c9fd-bc0c-4e1c-91bb-a94aca899c45 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:46.818041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4c9fd-bc0c-4e1c-91bb-a94aca899c45 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:46.818248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4c9fd-bc0c-4e1c-91bb-a94aca899c45 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:46.819888 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 444/878] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:46.825541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58800b-cf5a-4e63-8d30-a4ca4231cb80 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:46.825688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58800b-cf5a-4e63-8d30-a4ca4231cb80 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:46.825856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58800b-cf5a-4e63-8d30-a4ca4231cb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:46.844938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3f58800b-cf5a-4e63-8d30-a4ca4231cb80 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:46.846096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3f58800b-cf5a-4e63-8d30-a4ca4231cb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:46.916251 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 435/879] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:46.964300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c911f-6b05-429d-babd-79067b05b79a None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:46.964623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c911f-6b05-429d-babd-79067b05b79a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:46.964853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c911f-6b05-429d-babd-79067b05b79a None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:46.966461 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 445/880] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:46.973692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08569f2b-121a-40d6-ab7a-7275c635d9fb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:46.973972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08569f2b-121a-40d6-ab7a-7275c635d9fb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:46.974237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08569f2b-121a-40d6-ab7a-7275c635d9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:46.992981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-08569f2b-121a-40d6-ab7a-7275c635d9fb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:46.994234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-08569f2b-121a-40d6-ab7a-7275c635d9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:47.017266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:47.061517 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 436/881] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:47.103758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c911f-6b05-429d-babd-79067b05b79a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:47.206928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c911f-6b05-429d-babd-79067b05b79a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:47.240627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b07ad3-0ee3-4473-b9d0-4aff4236651c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:47.241250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b07ad3-0ee3-4473-b9d0-4aff4236651c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:47.241517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b07ad3-0ee3-4473-b9d0-4aff4236651c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:47.241778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b07ad3-0ee3-4473-b9d0-4aff4236651c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:47.252201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b07ad3-0ee3-4473-b9d0-4aff4236651c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-f6b07ad3-0ee3-4473-b9d0-4aff4236651c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:47.256928 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:47.257866 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:47.258722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:47.258722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:47.258722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:47.258722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:47.258722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:47.258722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:47.258722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:47.258722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:47.258722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:47.258722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:47.258722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 20 20:39:47.258722 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:47.258722 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 446/882] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Aug 20 20:39:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:47.265468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08569f2b-121a-40d6-ab7a-7275c635d9fb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:47.266808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b07ad3-0ee3-4473-b9d0-4aff4236651c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:47.285211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:47.378964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b07ad3-0ee3-4473-b9d0-4aff4236651c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:47.402501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08569f2b-121a-40d6-ab7a-7275c635d9fb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:47.414450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffd87d-44ba-4325-9e82-caf0544d27f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:47.415061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffd87d-44ba-4325-9e82-caf0544d27f0 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:47.415368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffd87d-44ba-4325-9e82-caf0544d27f0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:47.415697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffd87d-44ba-4325-9e82-caf0544d27f0 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:47.416509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffd87d-44ba-4325-9e82-caf0544d27f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:47.417724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffd87d-44ba-4325-9e82-caf0544d27f0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:47.440627 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 447/883] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:39:47 2022] POST /identity/v3/services => generated 269 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:39:47.442084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffd87d-44ba-4325-9e82-caf0544d27f0 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:47.452447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd14d36-4c03-49de-ae1e-38977dcc93ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:47.453180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd14d36-4c03-49de-ae1e-38977dcc93ff None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:47.453528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd14d36-4c03-49de-ae1e-38977dcc93ff None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:47.453872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd14d36-4c03-49de-ae1e-38977dcc93ff None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:47.454640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd14d36-4c03-49de-ae1e-38977dcc93ff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:47.455016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd14d36-4c03-49de-ae1e-38977dcc93ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:47.457216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd14d36-4c03-49de-ae1e-38977dcc93ff None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:47.472133 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 437/884] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Aug 20 20:39:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:47.478090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd14d36-4c03-49de-ae1e-38977dcc93ff None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:47.506026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:47.546150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:47.579357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffd87d-44ba-4325-9e82-caf0544d27f0 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:47.603374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd14d36-4c03-49de-ae1e-38977dcc93ff None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:47.623080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562a447-7c73-4195-bf0b-1c53ab468814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:47.623947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562a447-7c73-4195-bf0b-1c53ab468814 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:47.624361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562a447-7c73-4195-bf0b-1c53ab468814 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:47.624810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562a447-7c73-4195-bf0b-1c53ab468814 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:47.625728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562a447-7c73-4195-bf0b-1c53ab468814 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:47.627111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562a447-7c73-4195-bf0b-1c53ab468814 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:47.642379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e325bd4-4099-4351-a4ac-826fe6c8a6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:47.643012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e325bd4-4099-4351-a4ac-826fe6c8a6bf None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:47.643294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e325bd4-4099-4351-a4ac-826fe6c8a6bf None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:47.643544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e325bd4-4099-4351-a4ac-826fe6c8a6bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:47.648500 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 448/885] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:39:47 2022] POST /identity/v3/services => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:39:47.657071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e325bd4-4099-4351-a4ac-826fe6c8a6bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:47.658686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e325bd4-4099-4351-a4ac-826fe6c8a6bf None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-2e325bd4-4099-4351-a4ac-826fe6c8a6bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:47.675699 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:47.677222 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:47.678223 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:47.678223 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:47.678223 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:47.678223 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 438/886] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:47.683012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562a447-7c73-4195-bf0b-1c53ab468814 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:47.824907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562a447-7c73-4195-bf0b-1c53ab468814 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:47.857785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:47.863714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b94b13-f5a0-4de4-b870-85b59bce0d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:47.864586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b94b13-f5a0-4de4-b870-85b59bce0d92 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:47.864864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b94b13-f5a0-4de4-b870-85b59bce0d92 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:47.865177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b94b13-f5a0-4de4-b870-85b59bce0d92 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:47.865841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b94b13-f5a0-4de4-b870-85b59bce0d92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:47.866135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b94b13-f5a0-4de4-b870-85b59bce0d92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:47.868142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b94b13-f5a0-4de4-b870-85b59bce0d92 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:47.884305 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 449/887] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:47.889689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e325bd4-4099-4351-a4ac-826fe6c8a6bf None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:47.910061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661daae0-3d34-4539-90f7-6999581b53f3 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:47.910382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661daae0-3d34-4539-90f7-6999581b53f3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:47.910634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661daae0-3d34-4539-90f7-6999581b53f3 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:47.912152 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 450/888] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:39:47.917715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be15f2-81c7-4998-a1ce-9f0cb59557fe None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:47.917981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be15f2-81c7-4998-a1ce-9f0cb59557fe None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:47.918232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be15f2-81c7-4998-a1ce-9f0cb59557fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:47.942295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d1be15f2-81c7-4998-a1ce-9f0cb59557fe None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:47.943475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d1be15f2-81c7-4998-a1ce-9f0cb59557fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:47.972590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:48.013362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e325bd4-4099-4351-a4ac-826fe6c8a6bf None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:48.015802 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 451/889] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:48.059344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad03875-2d8e-40f0-8adc-53b13be49fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:48.059982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03875-2d8e-40f0-8adc-53b13be49fce None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:48.060254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03875-2d8e-40f0-8adc-53b13be49fce None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:48.060556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03875-2d8e-40f0-8adc-53b13be49fce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:48.068830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad03875-2d8e-40f0-8adc-53b13be49fce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:48.070275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad03875-2d8e-40f0-8adc-53b13be49fce None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-dad03875-2d8e-40f0-8adc-53b13be49fce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:48.082538 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:48.083484 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:48.084445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:48.084445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:48.084445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:48.084445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:48.084445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:48.084445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:48.084445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:48.084445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:48.084445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 20 20:39:48.084445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:48.084445 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 439/890] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Aug 20 20:39:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:48.089639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be15f2-81c7-4998-a1ce-9f0cb59557fe None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:48.196359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be15f2-81c7-4998-a1ce-9f0cb59557fe None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:48.217291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:48.235119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa11d0-7b3b-46e3-aa7b-ccfe10c49378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:48.235739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa11d0-7b3b-46e3-aa7b-ccfe10c49378 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:48.236011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa11d0-7b3b-46e3-aa7b-ccfe10c49378 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:48.236276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa11d0-7b3b-46e3-aa7b-ccfe10c49378 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:48.237151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa11d0-7b3b-46e3-aa7b-ccfe10c49378 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:48.237446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa11d0-7b3b-46e3-aa7b-ccfe10c49378 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:48.238931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa11d0-7b3b-46e3-aa7b-ccfe10c49378 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:48.257509 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 452/891] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Aug 20 20:39:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:48.263355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad03875-2d8e-40f0-8adc-53b13be49fce None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:48.365247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa11d0-7b3b-46e3-aa7b-ccfe10c49378 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:48.397416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad03875-2d8e-40f0-8adc-53b13be49fce None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:48.463143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:48.485728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d293824-8ac7-4c8c-851e-b6714ab7aeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:48.486397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d293824-8ac7-4c8c-851e-b6714ab7aeb0 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:48.486678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d293824-8ac7-4c8c-851e-b6714ab7aeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:48.486949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d293824-8ac7-4c8c-851e-b6714ab7aeb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:48.500665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa11d0-7b3b-46e3-aa7b-ccfe10c49378 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:48.536830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a17a85-d22a-4213-9bc8-98a0a8bad539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:48.537773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a17a85-d22a-4213-9bc8-98a0a8bad539 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:48.538266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a17a85-d22a-4213-9bc8-98a0a8bad539 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:48.538752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a17a85-d22a-4213-9bc8-98a0a8bad539 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:48.539627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a17a85-d22a-4213-9bc8-98a0a8bad539 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:48.541298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a17a85-d22a-4213-9bc8-98a0a8bad539 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:48.580948 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 453/892] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:39:48 2022] POST /identity/v3/services => generated 241 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:39:48.593969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d293824-8ac7-4c8c-851e-b6714ab7aeb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:48.595611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d293824-8ac7-4c8c-851e-b6714ab7aeb0 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:48.704396 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 440/893] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:48 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:39:48.719171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a17a85-d22a-4213-9bc8-98a0a8bad539 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:48.788918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:48.874609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a17a85-d22a-4213-9bc8-98a0a8bad539 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:48.912924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bae03-3dcd-4a8c-8065-cef5e1378ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:48.914493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bae03-3dcd-4a8c-8065-cef5e1378ca0 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:48.915051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bae03-3dcd-4a8c-8065-cef5e1378ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:48.915425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bae03-3dcd-4a8c-8065-cef5e1378ca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:48.929233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bae03-3dcd-4a8c-8065-cef5e1378ca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:48.930865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bae03-3dcd-4a8c-8065-cef5e1378ca0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-bf0bae03-3dcd-4a8c-8065-cef5e1378ca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:48.949736 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:48.950734 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:48.951631 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:48.951631 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:48.951631 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:48.951631 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 454/894] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:48.961575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d293824-8ac7-4c8c-851e-b6714ab7aeb0 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:49.008831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:49.087595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d293824-8ac7-4c8c-851e-b6714ab7aeb0 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:49.123746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e70fe-97d4-4615-8a69-eb89cf5dc111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:49.124730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e70fe-97d4-4615-8a69-eb89cf5dc111 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:49.125193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e70fe-97d4-4615-8a69-eb89cf5dc111 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:49.125630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e70fe-97d4-4615-8a69-eb89cf5dc111 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:49.126441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e70fe-97d4-4615-8a69-eb89cf5dc111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:49.126936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e70fe-97d4-4615-8a69-eb89cf5dc111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:49.128479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e70fe-97d4-4615-8a69-eb89cf5dc111 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:49.140925 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 441/895] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:49.146830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bae03-3dcd-4a8c-8065-cef5e1378ca0 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:49.241997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:49.256940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bae03-3dcd-4a8c-8065-cef5e1378ca0 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:49.293161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92392a1-7509-41ad-81f5-5667317f45db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:49.293794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92392a1-7509-41ad-81f5-5667317f45db None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:49.294061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92392a1-7509-41ad-81f5-5667317f45db None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:49.294326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92392a1-7509-41ad-81f5-5667317f45db None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:49.301608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92392a1-7509-41ad-81f5-5667317f45db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:49.303061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92392a1-7509-41ad-81f5-5667317f45db None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-e92392a1-7509-41ad-81f5-5667317f45db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:49.314941 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:49.315874 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:49.316783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:49.316783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:49.316783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:49.316783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:49.316783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:49.316783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:49.316783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:49.316783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:49.316783 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 455/896] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:49.322559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e70fe-97d4-4615-8a69-eb89cf5dc111 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:49.419367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:49.431226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e70fe-97d4-4615-8a69-eb89cf5dc111 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:49.468475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd971f-4b48-4b80-8bd0-4b8efd7a732e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:49.469165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd971f-4b48-4b80-8bd0-4b8efd7a732e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:49.469437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd971f-4b48-4b80-8bd0-4b8efd7a732e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:49.469755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd971f-4b48-4b80-8bd0-4b8efd7a732e None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:49.470564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd971f-4b48-4b80-8bd0-4b8efd7a732e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:49.470870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd971f-4b48-4b80-8bd0-4b8efd7a732e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:49.472531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd971f-4b48-4b80-8bd0-4b8efd7a732e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:49.486277 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 442/897] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:49.492320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92392a1-7509-41ad-81f5-5667317f45db None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:49.596325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92392a1-7509-41ad-81f5-5667317f45db None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:49.631041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836d0cd-45ee-42eb-ae9b-3a8e4014fb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:49.631642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836d0cd-45ee-42eb-ae9b-3a8e4014fb40 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:49.631911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836d0cd-45ee-42eb-ae9b-3a8e4014fb40 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:49.632263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836d0cd-45ee-42eb-ae9b-3a8e4014fb40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:49.633255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836d0cd-45ee-42eb-ae9b-3a8e4014fb40 None admin] RBAC: Adding query filter params (role.id=d78b7ae7d2c541498b055d1e2822e9b1, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=1547cb99abe642ae86db8de1fa80041f) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:49.633709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836d0cd-45ee-42eb-ae9b-3a8e4014fb40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:49.635388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836d0cd-45ee-42eb-ae9b-3a8e4014fb40 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:49.641454 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 456/898] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:49 2022] GET /identity/v3/role_assignments?user.id=1547cb99abe642ae86db8de1fa80041f&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=d78b7ae7d2c541498b055d1e2822e9b1 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:49.832647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a8a43-13fc-439b-bbf0-fd2d63bd4b15 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:49.833266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a8a43-13fc-439b-bbf0-fd2d63bd4b15 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:49.833266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a8a43-13fc-439b-bbf0-fd2d63bd4b15 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:49.834656 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 443/899] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:49.840138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf5dc2-4688-48c0-88d7-d84860d3c17a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:49.840983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf5dc2-4688-48c0-88d7-d84860d3c17a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:49.841478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf5dc2-4688-48c0-88d7-d84860d3c17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:49.860748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9adf5dc2-4688-48c0-88d7-d84860d3c17a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:49.862265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9adf5dc2-4688-48c0-88d7-d84860d3c17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:49.865491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:49.936819 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 457/900] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:50.098063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97fd03-b87f-4e5b-bb25-25a43c6ec002 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:50.098238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97fd03-b87f-4e5b-bb25-25a43c6ec002 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:50.098511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97fd03-b87f-4e5b-bb25-25a43c6ec002 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:50.100101 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 444/901] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:50.105759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6e61e-a7a5-4ffc-a8ad-b85f1a5cc665 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:50.106039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6e61e-a7a5-4ffc-a8ad-b85f1a5cc665 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:50.106312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6e61e-a7a5-4ffc-a8ad-b85f1a5cc665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:50.126239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ceb6e61e-a7a5-4ffc-a8ad-b85f1a5cc665 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:50.127426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ceb6e61e-a7a5-4ffc-a8ad-b85f1a5cc665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:50.199022 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 458/902] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:50.259930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97fd03-b87f-4e5b-bb25-25a43c6ec002 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:50.366885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97fd03-b87f-4e5b-bb25-25a43c6ec002 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:50.402412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca44f94-b934-4e0a-838a-4320206e9b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:50.403034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca44f94-b934-4e0a-838a-4320206e9b17 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:50.403300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca44f94-b934-4e0a-838a-4320206e9b17 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:50.403560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca44f94-b934-4e0a-838a-4320206e9b17 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:50.404133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca44f94-b934-4e0a-838a-4320206e9b17 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:50.405526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca44f94-b934-4e0a-838a-4320206e9b17 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-1ca44f94-b934-4e0a-838a-4320206e9b17 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:50.414206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:50.415112 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:50.416087 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 445/903] 10.222.0.23 () {56 vars in 1132 bytes} [Sat Aug 20 20:39:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:50.423351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6e61e-a7a5-4ffc-a8ad-b85f1a5cc665 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:50.504784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:50.527742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6e61e-a7a5-4ffc-a8ad-b85f1a5cc665 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:50.565130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b7b3d-90c6-41c3-bf5d-543fce0d6b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:50.565739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b7b3d-90c6-41c3-bf5d-543fce0d6b54 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:50.566005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b7b3d-90c6-41c3-bf5d-543fce0d6b54 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:50.566262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b7b3d-90c6-41c3-bf5d-543fce0d6b54 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:50.566883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b7b3d-90c6-41c3-bf5d-543fce0d6b54 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:50.567175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b7b3d-90c6-41c3-bf5d-543fce0d6b54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:50.568556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b7b3d-90c6-41c3-bf5d-543fce0d6b54 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:50.573820 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 459/904] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Aug 20 20:39:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:50.576419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca44f94-b934-4e0a-838a-4320206e9b17 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:50.579541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b7b3d-90c6-41c3-bf5d-543fce0d6b54 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:50.593858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:50.696594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b7b3d-90c6-41c3-bf5d-543fce0d6b54 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:50.697410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca44f94-b934-4e0a-838a-4320206e9b17 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:50.736336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1745f2-fde1-49c8-a535-141d94e54381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:50.737582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1745f2-fde1-49c8-a535-141d94e54381 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:50.738096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1745f2-fde1-49c8-a535-141d94e54381 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:50.738624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1745f2-fde1-49c8-a535-141d94e54381 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:50.738942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ead5b-fc3d-4acb-80aa-eafe5e1208fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:50.739540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ead5b-fc3d-4acb-80aa-eafe5e1208fc None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:50.739807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ead5b-fc3d-4acb-80aa-eafe5e1208fc None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:50.740068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ead5b-fc3d-4acb-80aa-eafe5e1208fc None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:50.740253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1745f2-fde1-49c8-a535-141d94e54381 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:50.740728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ead5b-fc3d-4acb-80aa-eafe5e1208fc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:50.741020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ead5b-fc3d-4acb-80aa-eafe5e1208fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:50.742268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1745f2-fde1-49c8-a535-141d94e54381 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:50.742391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ead5b-fc3d-4acb-80aa-eafe5e1208fc None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:50.743693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:50.753417 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 460/905] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Aug 20 20:39:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:50.757324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ead5b-fc3d-4acb-80aa-eafe5e1208fc None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9c1745f2-fde1-49c8-a535-141d94e54381 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:50.757594 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:50.758652 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:50.760445 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 446/906] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:39:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:50.765156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1745f2-fde1-49c8-a535-141d94e54381 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:50.881061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ead5b-fc3d-4acb-80aa-eafe5e1208fc None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:50.884502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:50.911559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1745f2-fde1-49c8-a535-141d94e54381 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:50.921155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c9aee-5a7a-450f-b0cd-32919488ce1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:50.922194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c9aee-5a7a-450f-b0cd-32919488ce1e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:50.922668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c9aee-5a7a-450f-b0cd-32919488ce1e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:50.923261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c9aee-5a7a-450f-b0cd-32919488ce1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:50.924155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c9aee-5a7a-450f-b0cd-32919488ce1e None admin] RBAC: Adding query filter params (interface=public, service_id=a92a651fe2ce4a02a4d4b601c7ee4ce8, region_id=RegionOne) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:50.924705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c9aee-5a7a-450f-b0cd-32919488ce1e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:50.926487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c9aee-5a7a-450f-b0cd-32919488ce1e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:50.937517 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 461/907] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Aug 20 20:39:50 2022] GET /identity/v3/endpoints?service_id=a92a651fe2ce4a02a4d4b601c7ee4ce8&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:50.939255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c9aee-5a7a-450f-b0cd-32919488ce1e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:50.949010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf0e0-454a-4b35-898e-eae5ee008a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:50.949958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf0e0-454a-4b35-898e-eae5ee008a4c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:50.950234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf0e0-454a-4b35-898e-eae5ee008a4c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:50.950497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf0e0-454a-4b35-898e-eae5ee008a4c None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:50.951117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebf0e0-454a-4b35-898e-eae5ee008a4c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:50.951409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebf0e0-454a-4b35-898e-eae5ee008a4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:50.953171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebf0e0-454a-4b35-898e-eae5ee008a4c None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:50.960440 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 447/908] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:39:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:50.964863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf0e0-454a-4b35-898e-eae5ee008a4c None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:51.024933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:51.071954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c9aee-5a7a-450f-b0cd-32919488ce1e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:51.079915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf0e0-454a-4b35-898e-eae5ee008a4c None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:51.118596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c4e55-4c6d-43fd-97c0-9ebce4607874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:51.119638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c4e55-4c6d-43fd-97c0-9ebce4607874 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:51.120138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c4e55-4c6d-43fd-97c0-9ebce4607874 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:51.120716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c4e55-4c6d-43fd-97c0-9ebce4607874 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:51.121747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c4e55-4c6d-43fd-97c0-9ebce4607874 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:51.122244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c4e55-4c6d-43fd-97c0-9ebce4607874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:51.124062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c4e55-4c6d-43fd-97c0-9ebce4607874 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:51.124174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757c4d7-0d44-4820-9d3b-ee05d98cb302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:51.124986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757c4d7-0d44-4820-9d3b-ee05d98cb302 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:51.125349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757c4d7-0d44-4820-9d3b-ee05d98cb302 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:51.125710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757c4d7-0d44-4820-9d3b-ee05d98cb302 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:51.126357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4757c4d7-0d44-4820-9d3b-ee05d98cb302 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:51.129187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4757c4d7-0d44-4820-9d3b-ee05d98cb302 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:51.131753 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 448/909] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:39:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:51.137605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c4e55-4c6d-43fd-97c0-9ebce4607874 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:51.147777 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 462/910] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Aug 20 20:39:50 2022] GET /identity/v3/services => generated 1497 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:39:51.148008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:51.167041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3c742-2db3-4d33-a59a-45211bbefe3b None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:51.167291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3c742-2db3-4d33-a59a-45211bbefe3b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:51.167546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3c742-2db3-4d33-a59a-45211bbefe3b None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:51.169131 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 463/911] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:39:51.175130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9eb7a-3d3e-407b-a237-59e7b0a6c3af None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:51.175444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9eb7a-3d3e-407b-a237-59e7b0a6c3af None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:51.175661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9eb7a-3d3e-407b-a237-59e7b0a6c3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:51.198589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-efb9eb7a-3d3e-407b-a237-59e7b0a6c3af None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:51.200324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-efb9eb7a-3d3e-407b-a237-59e7b0a6c3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:51.271927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c4e55-4c6d-43fd-97c0-9ebce4607874 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:51.300404 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 464/912] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:39:51.325450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441123d9-a2a3-40a1-98c0-30c10f30cb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:51.326069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441123d9-a2a3-40a1-98c0-30c10f30cb28 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:51.326338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441123d9-a2a3-40a1-98c0-30c10f30cb28 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:51.326661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441123d9-a2a3-40a1-98c0-30c10f30cb28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:51.327370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441123d9-a2a3-40a1-98c0-30c10f30cb28 None admin] RBAC: Adding query filter params (interface=public, service_id=ff48adf9e3ff47baaebd03dd0e40d260, region_id=RegionOne) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:51.327674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441123d9-a2a3-40a1-98c0-30c10f30cb28 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:51.329137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441123d9-a2a3-40a1-98c0-30c10f30cb28 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:51.335650 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 449/913] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Aug 20 20:39:51 2022] GET /identity/v3/endpoints?service_id=ff48adf9e3ff47baaebd03dd0e40d260&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:51.341679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9eb7a-3d3e-407b-a237-59e7b0a6c3af None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:51.408552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:51.456460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9eb7a-3d3e-407b-a237-59e7b0a6c3af None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:51.493313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2f6d0-cee9-460b-b169-66cb749bcae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:51.493951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2f6d0-cee9-460b-b169-66cb749bcae5 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:51.493951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2f6d0-cee9-460b-b169-66cb749bcae5 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:51.494525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2f6d0-cee9-460b-b169-66cb749bcae5 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:51.495046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2f6d0-cee9-460b-b169-66cb749bcae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:51.496422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2f6d0-cee9-460b-b169-66cb749bcae5 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:51.503191 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 465/914] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Aug 20 20:39:51 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:39:51.644924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441123d9-a2a3-40a1-98c0-30c10f30cb28 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:51.721524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:51.759231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441123d9-a2a3-40a1-98c0-30c10f30cb28 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:51.797792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850905b9-b0bf-43b4-8b7d-e049e10f2015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:51.798425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850905b9-b0bf-43b4-8b7d-e049e10f2015 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:51.798697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850905b9-b0bf-43b4-8b7d-e049e10f2015 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:51.798963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850905b9-b0bf-43b4-8b7d-e049e10f2015 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:51.799546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850905b9-b0bf-43b4-8b7d-e049e10f2015 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:51.800947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850905b9-b0bf-43b4-8b7d-e049e10f2015 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-850905b9-b0bf-43b4-8b7d-e049e10f2015 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:51.810838 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:51.811860 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:51.812706 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 450/915] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Aug 20 20:39:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:51.818223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2f6d0-cee9-460b-b169-66cb749bcae5 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:51.910911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5516e-3cf6-4015-bd00-4272fd89b899 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:51.911442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5516e-3cf6-4015-bd00-4272fd89b899 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:51.911442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5516e-3cf6-4015-bd00-4272fd89b899 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:51.913053 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 451/916] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:39:51.918181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5466a4b-c172-4fb5-9243-5be0520cba90 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:51.918451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5466a4b-c172-4fb5-9243-5be0520cba90 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:51.918717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5466a4b-c172-4fb5-9243-5be0520cba90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:51.933172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2f6d0-cee9-460b-b169-66cb749bcae5 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:51.938110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a5466a4b-c172-4fb5-9243-5be0520cba90 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:51.939387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a5466a4b-c172-4fb5-9243-5be0520cba90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:51.961071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:51.982209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20e7f8-856f-49e9-8aa6-d33617a1a6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:51.982447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20e7f8-856f-49e9-8aa6-d33617a1a6de None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:51.982816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20e7f8-856f-49e9-8aa6-d33617a1a6de None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:51.983188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20e7f8-856f-49e9-8aa6-d33617a1a6de None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:51.983902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20e7f8-856f-49e9-8aa6-d33617a1a6de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:51.984290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20e7f8-856f-49e9-8aa6-d33617a1a6de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:51.985770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20e7f8-856f-49e9-8aa6-d33617a1a6de None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:51.991404 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 466/917] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Aug 20 20:39:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:51.997068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20e7f8-856f-49e9-8aa6-d33617a1a6de None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:52.025437 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 452/918] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:39:52.106680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20e7f8-856f-49e9-8aa6-d33617a1a6de None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:52.143195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e58450-2f86-4756-acca-3a8d0673f539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:52.144048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e58450-2f86-4756-acca-3a8d0673f539 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:52.144499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e58450-2f86-4756-acca-3a8d0673f539 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:52.144905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e58450-2f86-4756-acca-3a8d0673f539 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:52.145659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e58450-2f86-4756-acca-3a8d0673f539 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:52.146096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e58450-2f86-4756-acca-3a8d0673f539 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:52.147617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e58450-2f86-4756-acca-3a8d0673f539 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:52.153564 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 467/919] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Aug 20 20:39:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:52.162102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5466a4b-c172-4fb5-9243-5be0520cba90 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:52.226447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:52.266498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5466a4b-c172-4fb5-9243-5be0520cba90 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:52.301015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d77b3-b05f-46df-9c69-ae652230ed8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:52.301620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d77b3-b05f-46df-9c69-ae652230ed8c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:52.301894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d77b3-b05f-46df-9c69-ae652230ed8c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:52.302160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d77b3-b05f-46df-9c69-ae652230ed8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:52.302852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d77b3-b05f-46df-9c69-ae652230ed8c None admin] RBAC: Adding query filter params (interface=public, service_id=947a8d19050649afa19b265f1180ea15, region_id=RegionOne) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:52.303153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d77b3-b05f-46df-9c69-ae652230ed8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:52.304604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d77b3-b05f-46df-9c69-ae652230ed8c None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:52.310851 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 453/920] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Aug 20 20:39:52 2022] GET /identity/v3/endpoints?service_id=947a8d19050649afa19b265f1180ea15&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:52.316490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e58450-2f86-4756-acca-3a8d0673f539 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:52.358500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d77b3-b05f-46df-9c69-ae652230ed8c None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:52.371210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:52.439012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e58450-2f86-4756-acca-3a8d0673f539 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:52.485105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcd48d-5372-45a4-834b-f9811d6751ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:52.485990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcd48d-5372-45a4-834b-f9811d6751ee None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:52.486467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcd48d-5372-45a4-834b-f9811d6751ee None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:52.487574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcd48d-5372-45a4-834b-f9811d6751ee None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:52.488896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bcd48d-5372-45a4-834b-f9811d6751ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:52.490554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bcd48d-5372-45a4-834b-f9811d6751ee None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:52.492590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d77b3-b05f-46df-9c69-ae652230ed8c None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:52.504719 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 468/921] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Aug 20 20:39:52 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:39:52.523727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:52.533774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47783ed1-68e4-49ac-9733-86e2ffc295e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:52.534416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47783ed1-68e4-49ac-9733-86e2ffc295e2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:52.534684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47783ed1-68e4-49ac-9733-86e2ffc295e2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:52.534946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47783ed1-68e4-49ac-9733-86e2ffc295e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:52.546988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47783ed1-68e4-49ac-9733-86e2ffc295e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-47783ed1-68e4-49ac-9733-86e2ffc295e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:52.551774 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:52.552736 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:52.553585 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:52.553585 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:52.553585 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:52.553585 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:52.553585 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:52.553585 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:52.553585 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:52.553585 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:52.553585 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:52.553585 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:52.553585 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 20 20:39:52.553585 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:52.553585 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 454/922] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Aug 20 20:39:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:52.560097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcd48d-5372-45a4-834b-f9811d6751ee None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:52.678155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcd48d-5372-45a4-834b-f9811d6751ee None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:52.693966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:52.721358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8212de-7e1d-46ae-970f-50ebe423a0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:52.722300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8212de-7e1d-46ae-970f-50ebe423a0cf None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:52.722747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8212de-7e1d-46ae-970f-50ebe423a0cf None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:52.723179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8212de-7e1d-46ae-970f-50ebe423a0cf None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:52.724098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8212de-7e1d-46ae-970f-50ebe423a0cf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:52.724596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8212de-7e1d-46ae-970f-50ebe423a0cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:52.726195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8212de-7e1d-46ae-970f-50ebe423a0cf None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:52.736156 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 469/923] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Aug 20 20:39:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:52.742225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47783ed1-68e4-49ac-9733-86e2ffc295e2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:52.854574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47783ed1-68e4-49ac-9733-86e2ffc295e2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:52.892326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b2497-b022-4e35-95ca-86296ff99cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:52.893334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b2497-b022-4e35-95ca-86296ff99cf0 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:52.893784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b2497-b022-4e35-95ca-86296ff99cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:52.894220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b2497-b022-4e35-95ca-86296ff99cf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:52.905668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b2497-b022-4e35-95ca-86296ff99cf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:52.907307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b2497-b022-4e35-95ca-86296ff99cf0 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-5d5b2497-b022-4e35-95ca-86296ff99cf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:52.922653 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:52.924114 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:52.925030 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:52.925030 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:52.925030 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:52.926936 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 455/924] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:52.932721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8212de-7e1d-46ae-970f-50ebe423a0cf None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:53.042373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8212de-7e1d-46ae-970f-50ebe423a0cf None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:53.080028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8258f70-78a1-40f1-9c5d-58b27cf70240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:53.080948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8258f70-78a1-40f1-9c5d-58b27cf70240 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:53.081382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8258f70-78a1-40f1-9c5d-58b27cf70240 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:53.081799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8258f70-78a1-40f1-9c5d-58b27cf70240 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:53.082573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8258f70-78a1-40f1-9c5d-58b27cf70240 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:53.083009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8258f70-78a1-40f1-9c5d-58b27cf70240 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:53.084535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8258f70-78a1-40f1-9c5d-58b27cf70240 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:53.098988 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 470/925] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:53.105571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b2497-b022-4e35-95ca-86296ff99cf0 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:53.151517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:53.213662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b2497-b022-4e35-95ca-86296ff99cf0 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:53.247941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe41eb1-7f1c-46ca-ab7c-ce68bd623be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:53.248640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe41eb1-7f1c-46ca-ab7c-ce68bd623be8 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:53.248911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe41eb1-7f1c-46ca-ab7c-ce68bd623be8 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:53.249174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe41eb1-7f1c-46ca-ab7c-ce68bd623be8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:53.257142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe41eb1-7f1c-46ca-ab7c-ce68bd623be8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:53.258637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe41eb1-7f1c-46ca-ab7c-ce68bd623be8 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-afe41eb1-7f1c-46ca-ab7c-ce68bd623be8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:53.270103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:53.271024 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:53.271945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:53.271945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:53.271945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:53.271945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:53.271945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:53.271945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:53.271945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:53.271945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:53.271945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 20 20:39:53.271945 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:53.271945 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 456/926] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Aug 20 20:39:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:53.281462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8258f70-78a1-40f1-9c5d-58b27cf70240 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:53.321593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:53.401300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8258f70-78a1-40f1-9c5d-58b27cf70240 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:53.438709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bdc20-cb8e-45bc-a699-1f5cf7e72949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:53.439314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bdc20-cb8e-45bc-a699-1f5cf7e72949 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:53.439585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bdc20-cb8e-45bc-a699-1f5cf7e72949 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:53.439846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bdc20-cb8e-45bc-a699-1f5cf7e72949 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:53.440665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4bdc20-cb8e-45bc-a699-1f5cf7e72949 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:53.440962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4bdc20-cb8e-45bc-a699-1f5cf7e72949 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:53.442389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4bdc20-cb8e-45bc-a699-1f5cf7e72949 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:53.460777 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 471/927] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Aug 20 20:39:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:53.466426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe41eb1-7f1c-46ca-ab7c-ce68bd623be8 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:53.539410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b634097-eedc-4213-b061-714243addd9c None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:53.539747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b634097-eedc-4213-b061-714243addd9c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:53.539939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b634097-eedc-4213-b061-714243addd9c None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:53.541898 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 472/928] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:39:53.547407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79865dad-c0bd-47ad-80d3-0b9f558bad50 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:53.547677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79865dad-c0bd-47ad-80d3-0b9f558bad50 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:53.547940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79865dad-c0bd-47ad-80d3-0b9f558bad50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:53.566270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-79865dad-c0bd-47ad-80d3-0b9f558bad50 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:53.567431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-79865dad-c0bd-47ad-80d3-0b9f558bad50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:53.569295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe41eb1-7f1c-46ca-ab7c-ce68bd623be8 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:53.594401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:53.607060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e12c88-7f6d-4024-b925-68917f20e8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:53.608133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e12c88-7f6d-4024-b925-68917f20e8ed None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:53.608500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e12c88-7f6d-4024-b925-68917f20e8ed None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:53.608828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e12c88-7f6d-4024-b925-68917f20e8ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:53.621291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e12c88-7f6d-4024-b925-68917f20e8ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:53.622829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e12c88-7f6d-4024-b925-68917f20e8ed None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-48e12c88-7f6d-4024-b925-68917f20e8ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:53.638234 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:53.639233 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:53.640542 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:53.640542 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:53.640542 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:53.640542 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 457/929] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:53.645357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e12c88-7f6d-4024-b925-68917f20e8ed None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:53.646787 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 473/930] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:39:53.747190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e12c88-7f6d-4024-b925-68917f20e8ed None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:53.770558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0946fd-cf8c-46f3-8f12-4bd77c33be71 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:53.770845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0946fd-cf8c-46f3-8f12-4bd77c33be71 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:53.771374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0946fd-cf8c-46f3-8f12-4bd77c33be71 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:53.773464 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 474/931] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:53.778738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a352d-1cac-4c04-83bb-ebcc3ddc0429 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:53.779007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a352d-1cac-4c04-83bb-ebcc3ddc0429 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:53.779276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a352d-1cac-4c04-83bb-ebcc3ddc0429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:53.784221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53633ff8-1060-46be-a0c8-509bbc395808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:53.784860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53633ff8-1060-46be-a0c8-509bbc395808 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:53.785127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53633ff8-1060-46be-a0c8-509bbc395808 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:53.785464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53633ff8-1060-46be-a0c8-509bbc395808 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:53.786115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53633ff8-1060-46be-a0c8-509bbc395808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:53.786410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53633ff8-1060-46be-a0c8-509bbc395808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:53.787845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53633ff8-1060-46be-a0c8-509bbc395808 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:53.799277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5b7a352d-1cac-4c04-83bb-ebcc3ddc0429 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:53.800857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5b7a352d-1cac-4c04-83bb-ebcc3ddc0429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:53.801734 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 458/932] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:53.808319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53633ff8-1060-46be-a0c8-509bbc395808 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:53.820893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:53.826060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:53.896788 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 475/933] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:39:53.923329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53633ff8-1060-46be-a0c8-509bbc395808 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:53.959622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553d7ab-6218-4427-9fe0-7f610c0951ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:53.960388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553d7ab-6218-4427-9fe0-7f610c0951ad None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:53.960663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553d7ab-6218-4427-9fe0-7f610c0951ad None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:53.960924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553d7ab-6218-4427-9fe0-7f610c0951ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:53.967743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553d7ab-6218-4427-9fe0-7f610c0951ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:53.969246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553d7ab-6218-4427-9fe0-7f610c0951ad None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:53.979442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a352d-1cac-4c04-83bb-ebcc3ddc0429 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-3553d7ab-6218-4427-9fe0-7f610c0951ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:53.980522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:53.981381 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:53.982367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:53.982367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:53.982367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:53.982367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:53.982367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:53.982367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:53.982367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:53.982367 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:53.982367 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 459/934] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:53.987955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553d7ab-6218-4427-9fe0-7f610c0951ad None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:54.082865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:54.084499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a352d-1cac-4c04-83bb-ebcc3ddc0429 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:54.121729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bcf6b-fde9-46b8-ab3e-df06b9dc306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:54.122281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553d7ab-6218-4427-9fe0-7f610c0951ad None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:54.122395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bcf6b-fde9-46b8-ab3e-df06b9dc306c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:54.122615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bcf6b-fde9-46b8-ab3e-df06b9dc306c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:54.122880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bcf6b-fde9-46b8-ab3e-df06b9dc306c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:54.123459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99bcf6b-fde9-46b8-ab3e-df06b9dc306c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:54.125064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99bcf6b-fde9-46b8-ab3e-df06b9dc306c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-f99bcf6b-fde9-46b8-ab3e-df06b9dc306c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:54.134441 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:54.135448 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 20 20:39:54.136339 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:54.140673 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 476/935] 10.222.0.23 () {56 vars in 1132 bytes} [Sat Aug 20 20:39:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:54.141995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bcf6b-fde9-46b8-ab3e-df06b9dc306c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:54.162232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:54.165638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e11613-8635-4b33-b1b4-3bb4248f0551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:54.166578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e11613-8635-4b33-b1b4-3bb4248f0551 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:54.167750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e11613-8635-4b33-b1b4-3bb4248f0551 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:54.168511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e11613-8635-4b33-b1b4-3bb4248f0551 None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:54.169795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e11613-8635-4b33-b1b4-3bb4248f0551 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:54.170289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e11613-8635-4b33-b1b4-3bb4248f0551 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:54.172768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e11613-8635-4b33-b1b4-3bb4248f0551 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:54.194596 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 460/936] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:54.196976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e11613-8635-4b33-b1b4-3bb4248f0551 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:54.266650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bcf6b-fde9-46b8-ab3e-df06b9dc306c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:54.300996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500de90f-d245-4e7d-a145-bf580d534fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:54.301597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500de90f-d245-4e7d-a145-bf580d534fe4 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:54.301864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500de90f-d245-4e7d-a145-bf580d534fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:54.302126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500de90f-d245-4e7d-a145-bf580d534fe4 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:54.302736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500de90f-d245-4e7d-a145-bf580d534fe4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:54.303028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500de90f-d245-4e7d-a145-bf580d534fe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:54.304403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500de90f-d245-4e7d-a145-bf580d534fe4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:54.308297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e11613-8635-4b33-b1b4-3bb4248f0551 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:54.309679 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 477/937] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Aug 20 20:39:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:54.310475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500de90f-d245-4e7d-a145-bf580d534fe4 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:54.343451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:54.349836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5ab84-19a6-4f88-82a8-0231071632cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:54.350482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5ab84-19a6-4f88-82a8-0231071632cd None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:54.350755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5ab84-19a6-4f88-82a8-0231071632cd None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:54.351018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5ab84-19a6-4f88-82a8-0231071632cd None admin] PATH_INFO: `/v3/projects/3b6c88840d764c2385aa678da4338213/users/1547cb99abe642ae86db8de1fa80041f/roles/d78b7ae7d2c541498b055d1e2822e9b1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:54.354452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5ab84-19a6-4f88-82a8-0231071632cd None admin] RBAC: Authorizing `identity:create_grant(project_id=3b6c88840d764c2385aa678da4338213, user_id=1547cb99abe642ae86db8de1fa80041f, role_id=d78b7ae7d2c541498b055d1e2822e9b1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:54.356782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5ab84-19a6-4f88-82a8-0231071632cd None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:54.376533 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 461/938] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:39:54 2022] PUT /identity/v3/projects/3b6c88840d764c2385aa678da4338213/users/1547cb99abe642ae86db8de1fa80041f/roles/d78b7ae7d2c541498b055d1e2822e9b1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:39:54.377539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5ab84-19a6-4f88-82a8-0231071632cd None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:54.419303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:54.442930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500de90f-d245-4e7d-a145-bf580d534fe4 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:54.498071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5ab84-19a6-4f88-82a8-0231071632cd None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:54.502338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17675454-9f6f-4000-b63d-b8b9a4f841ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:54.503002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17675454-9f6f-4000-b63d-b8b9a4f841ff None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:54.503280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17675454-9f6f-4000-b63d-b8b9a4f841ff None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:54.503549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17675454-9f6f-4000-b63d-b8b9a4f841ff None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:54.504140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17675454-9f6f-4000-b63d-b8b9a4f841ff None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:54.505580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17675454-9f6f-4000-b63d-b8b9a4f841ff None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-17675454-9f6f-4000-b63d-b8b9a4f841ff None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:54.518551 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:54.519652 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:54.526050 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 478/939] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:39:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:54.529079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17675454-9f6f-4000-b63d-b8b9a4f841ff None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:54.550297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ef53b-d6aa-4665-9803-078e5fca8de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:54.550926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ef53b-d6aa-4665-9803-078e5fca8de4 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:54.551201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ef53b-d6aa-4665-9803-078e5fca8de4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:54.551466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ef53b-d6aa-4665-9803-078e5fca8de4 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:54.552429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ef53b-d6aa-4665-9803-078e5fca8de4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:54.552738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ef53b-d6aa-4665-9803-078e5fca8de4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:54.554704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ef53b-d6aa-4665-9803-078e5fca8de4 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:54.568461 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 462/940] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Aug 20 20:39:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:54.570478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ef53b-d6aa-4665-9803-078e5fca8de4 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:54.643947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:54.666444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17675454-9f6f-4000-b63d-b8b9a4f841ff None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:54.704129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ef53b-d6aa-4665-9803-078e5fca8de4 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:54.709474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233698bd-a38c-4233-a9c9-10da9009c5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:54.710430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233698bd-a38c-4233-a9c9-10da9009c5f7 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:54.710904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233698bd-a38c-4233-a9c9-10da9009c5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:54.711401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233698bd-a38c-4233-a9c9-10da9009c5f7 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:54.712750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233698bd-a38c-4233-a9c9-10da9009c5f7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:54.713431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233698bd-a38c-4233-a9c9-10da9009c5f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:54.715546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233698bd-a38c-4233-a9c9-10da9009c5f7 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:54.722362 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 479/941] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:39:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:54.729778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233698bd-a38c-4233-a9c9-10da9009c5f7 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:54.742040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:54.753831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120677c9-3ab9-4c8e-a315-142e4da45c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:54.754838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120677c9-3ab9-4c8e-a315-142e4da45c0b None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:54.755314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120677c9-3ab9-4c8e-a315-142e4da45c0b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:54.755778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120677c9-3ab9-4c8e-a315-142e4da45c0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:54.757529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120677c9-3ab9-4c8e-a315-142e4da45c0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:54.759628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120677c9-3ab9-4c8e-a315-142e4da45c0b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:54.796948 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 463/942] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:39:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:39:54.863712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233698bd-a38c-4233-a9c9-10da9009c5f7 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:54.913873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345412f-4bbe-4dec-8d82-e783fe9beb7b None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:54.914071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345412f-4bbe-4dec-8d82-e783fe9beb7b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:54.914312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345412f-4bbe-4dec-8d82-e783fe9beb7b None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:54.916725 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 464/943] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:54.917795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729b347-cd91-4949-87ff-9439ba66c74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:54.918523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729b347-cd91-4949-87ff-9439ba66c74a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:54.918849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729b347-cd91-4949-87ff-9439ba66c74a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:54.919174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729b347-cd91-4949-87ff-9439ba66c74a None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:54.919799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729b347-cd91-4949-87ff-9439ba66c74a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:54.920146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729b347-cd91-4949-87ff-9439ba66c74a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:54.921652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729b347-cd91-4949-87ff-9439ba66c74a None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:54.927325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c57f7-38b5-48cf-b2c1-96e792df572b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:54.928188 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 480/944] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Aug 20 20:39:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:54.928531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c57f7-38b5-48cf-b2c1-96e792df572b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:54.928998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c57f7-38b5-48cf-b2c1-96e792df572b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:54.935191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729b347-cd91-4949-87ff-9439ba66c74a None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:54.955449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-261c57f7-38b5-48cf-b2c1-96e792df572b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:54.957033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-261c57f7-38b5-48cf-b2c1-96e792df572b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:55.023350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:55.062310 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 465/945] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:39:55.070663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729b347-cd91-4949-87ff-9439ba66c74a None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:55.105789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82c893-eda3-485a-9526-6c903f6d13c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:55.106474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82c893-eda3-485a-9526-6c903f6d13c9 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:55.106687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82c893-eda3-485a-9526-6c903f6d13c9 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:55.106909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82c893-eda3-485a-9526-6c903f6d13c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:55.107562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82c893-eda3-485a-9526-6c903f6d13c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:55.108994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82c893-eda3-485a-9526-6c903f6d13c9 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:55.140590 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 481/946] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:39:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:39:55.401479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c57f7-38b5-48cf-b2c1-96e792df572b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:55.446802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:55.506861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c57f7-38b5-48cf-b2c1-96e792df572b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:55.548686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3cea1-7169-4c9d-a547-5168acb73f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:55.549318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3cea1-7169-4c9d-a547-5168acb73f27 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:55.549593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3cea1-7169-4c9d-a547-5168acb73f27 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:55.549857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3cea1-7169-4c9d-a547-5168acb73f27 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:55.550430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3cea1-7169-4c9d-a547-5168acb73f27 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:55.551800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3cea1-7169-4c9d-a547-5168acb73f27 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-ece3cea1-7169-4c9d-a547-5168acb73f27 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:55.561337 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:55.562323 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:55.563183 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 466/947] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Aug 20 20:39:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:55.570537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82c893-eda3-485a-9526-6c903f6d13c9 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:55.686050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82c893-eda3-485a-9526-6c903f6d13c9 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:55.711454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:55.724764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f246fef-173a-465c-b13b-90644a68c974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:55.725428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f246fef-173a-465c-b13b-90644a68c974 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:55.725707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f246fef-173a-465c-b13b-90644a68c974 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:55.725973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f246fef-173a-465c-b13b-90644a68c974 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:55.726647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f246fef-173a-465c-b13b-90644a68c974 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:55.726949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f246fef-173a-465c-b13b-90644a68c974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:55.728426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f246fef-173a-465c-b13b-90644a68c974 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:55.733512 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 482/948] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Aug 20 20:39:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:55.741449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3cea1-7169-4c9d-a547-5168acb73f27 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:55.848765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3cea1-7169-4c9d-a547-5168acb73f27 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:55.850228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:55.883702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de8026-6a48-4f64-ba0b-713f283e7766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:55.884507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de8026-6a48-4f64-ba0b-713f283e7766 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:55.884786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de8026-6a48-4f64-ba0b-713f283e7766 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:55.885057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de8026-6a48-4f64-ba0b-713f283e7766 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:55.885692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de8026-6a48-4f64-ba0b-713f283e7766 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:55.885994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de8026-6a48-4f64-ba0b-713f283e7766 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:55.887361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de8026-6a48-4f64-ba0b-713f283e7766 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:55.892953 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 467/949] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Aug 20 20:39:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:55.901466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f246fef-173a-465c-b13b-90644a68c974 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:56.006534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f246fef-173a-465c-b13b-90644a68c974 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:56.042460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b98a44-f633-469a-92a5-840d60b1f300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:56.043074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b98a44-f633-469a-92a5-840d60b1f300 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:56.043347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b98a44-f633-469a-92a5-840d60b1f300 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:56.043614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b98a44-f633-469a-92a5-840d60b1f300 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:56.044274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b98a44-f633-469a-92a5-840d60b1f300 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:56.045837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b98a44-f633-469a-92a5-840d60b1f300 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:56.069665 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 483/950] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:39:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:39:56.756326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f3cfa-d339-4f7c-96cb-9221389fc6d4 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:56.756886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f3cfa-d339-4f7c-96cb-9221389fc6d4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:56.756886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f3cfa-d339-4f7c-96cb-9221389fc6d4 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:56.758407 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 468/951] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:56.764871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cb791-2dcb-4bba-b152-5f5656741709 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:56.765314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cb791-2dcb-4bba-b152-5f5656741709 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:56.765390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cb791-2dcb-4bba-b152-5f5656741709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:56.784324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e62cb791-2dcb-4bba-b152-5f5656741709 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:56.785525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e62cb791-2dcb-4bba-b152-5f5656741709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:56.828115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:56.868724 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 484/952] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:57.076427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08cf04-d98b-4847-a1f6-de1a750dcfae None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:57.076769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08cf04-d98b-4847-a1f6-de1a750dcfae None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:57.076944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08cf04-d98b-4847-a1f6-de1a750dcfae None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:57.078481 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 469/953] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:57.083782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8764-edf9-4d2c-83ed-165e31becd6e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:57.084060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8764-edf9-4d2c-83ed-165e31becd6e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:57.084322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8764-edf9-4d2c-83ed-165e31becd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:57.102377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-456f8764-edf9-4d2c-83ed-165e31becd6e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:57.103878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-456f8764-edf9-4d2c-83ed-165e31becd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:57.169786 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 485/954] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:57.199286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08cf04-d98b-4847-a1f6-de1a750dcfae None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:57.310373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08cf04-d98b-4847-a1f6-de1a750dcfae None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:57.345224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6312ad-ee91-4f75-a825-d5d2d50edf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:57.345766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6312ad-ee91-4f75-a825-d5d2d50edf02 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:57.346037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6312ad-ee91-4f75-a825-d5d2d50edf02 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:57.346304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6312ad-ee91-4f75-a825-d5d2d50edf02 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:57.359888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6312ad-ee91-4f75-a825-d5d2d50edf02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-ba6312ad-ee91-4f75-a825-d5d2d50edf02 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:39:57.364867 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:57.365899 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:57.366729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:57.366729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:57.366729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:57.366729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:57.366729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:39:57.366729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:39:57.366729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:39:57.366729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:39:57.366729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:39:57.366729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:39:57.366729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 20 20:39:57.366729 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:57.366729 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 470/955] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Aug 20 20:39:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:57.372559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f8764-edf9-4d2c-83ed-165e31becd6e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:57.480594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:57.484903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f8764-edf9-4d2c-83ed-165e31becd6e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:57.520219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392e7c7-bb73-400a-84d5-24f0990a79c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:57.520868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392e7c7-bb73-400a-84d5-24f0990a79c5 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:57.521140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392e7c7-bb73-400a-84d5-24f0990a79c5 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:57.521424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392e7c7-bb73-400a-84d5-24f0990a79c5 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:57.522181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392e7c7-bb73-400a-84d5-24f0990a79c5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:57.522453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392e7c7-bb73-400a-84d5-24f0990a79c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:57.523806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392e7c7-bb73-400a-84d5-24f0990a79c5 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:57.523967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6312ad-ee91-4f75-a825-d5d2d50edf02 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:57.534394 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 486/956] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Aug 20 20:39:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:57.538128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607e48e-fff0-4ba1-9268-50acc2a94b6d None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:57.538406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607e48e-fff0-4ba1-9268-50acc2a94b6d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:57.538676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607e48e-fff0-4ba1-9268-50acc2a94b6d None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:57.541133 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 487/957] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:57.543169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607e48e-fff0-4ba1-9268-50acc2a94b6d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:57.566139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:57.643420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6312ad-ee91-4f75-a825-d5d2d50edf02 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:57.657633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607e48e-fff0-4ba1-9268-50acc2a94b6d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:57.677505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07c459-a3ac-4838-8969-0ad91b035a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:57.678127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07c459-a3ac-4838-8969-0ad91b035a9f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:57.678396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07c459-a3ac-4838-8969-0ad91b035a9f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:57.678654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07c459-a3ac-4838-8969-0ad91b035a9f None admin] PATH_INFO: `/v3/services/compute` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:57.679226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07c459-a3ac-4838-8969-0ad91b035a9f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:57.680630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07c459-a3ac-4838-8969-0ad91b035a9f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-1e07c459-a3ac-4838-8969-0ad91b035a9f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:57.689977 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:57.691035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:57.691864 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 471/958] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:57.693481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d699c-caed-4a3c-a794-a9dae0641657 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:57.693481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d699c-caed-4a3c-a794-a9dae0641657 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:57.693481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d699c-caed-4a3c-a794-a9dae0641657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:57.700790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6883043-7394-41d4-afaf-cc75b72a1bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:57.701426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883043-7394-41d4-afaf-cc75b72a1bff None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:57.701700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883043-7394-41d4-afaf-cc75b72a1bff None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:57.701966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883043-7394-41d4-afaf-cc75b72a1bff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:57.711301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-451d699c-caed-4a3c-a794-a9dae0641657 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:57.712457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-451d699c-caed-4a3c-a794-a9dae0641657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:57.713590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6883043-7394-41d4-afaf-cc75b72a1bff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:57.715091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6883043-7394-41d4-afaf-cc75b72a1bff None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-f6883043-7394-41d4-afaf-cc75b72a1bff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:57.729845 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:57.730909 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:57.731823 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:57.731823 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:57.731823 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:57.731823 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 488/959] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:57.732618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6883043-7394-41d4-afaf-cc75b72a1bff None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:57.749475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:57.791125 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 472/960] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:57.792634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d699c-caed-4a3c-a794-a9dae0641657 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:57.821768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:57.854095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6883043-7394-41d4-afaf-cc75b72a1bff None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:57.894165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f0bce-bad4-4a23-b0ac-3d03704dfd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:57.894686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f0bce-bad4-4a23-b0ac-3d03704dfd74 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:57.894961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f0bce-bad4-4a23-b0ac-3d03704dfd74 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:57.895227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f0bce-bad4-4a23-b0ac-3d03704dfd74 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:57.895954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f0bce-bad4-4a23-b0ac-3d03704dfd74 None admin] RBAC: Adding query filter params (name=compute) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:57.896258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f0bce-bad4-4a23-b0ac-3d03704dfd74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:57.897760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f0bce-bad4-4a23-b0ac-3d03704dfd74 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:57.903213 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 489/961] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Aug 20 20:39:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:57.908663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d699c-caed-4a3c-a794-a9dae0641657 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:57.909664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f0bce-bad4-4a23-b0ac-3d03704dfd74 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:57.951354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26502c3f-c0f4-4262-a2d8-6c5ef7272542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:57.952342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26502c3f-c0f4-4262-a2d8-6c5ef7272542 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:57.952843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26502c3f-c0f4-4262-a2d8-6c5ef7272542 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:57.953363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26502c3f-c0f4-4262-a2d8-6c5ef7272542 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:57.954240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26502c3f-c0f4-4262-a2d8-6c5ef7272542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:57.954787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26502c3f-c0f4-4262-a2d8-6c5ef7272542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:57.956346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26502c3f-c0f4-4262-a2d8-6c5ef7272542 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:57.968783 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 473/962] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:57.974241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26502c3f-c0f4-4262-a2d8-6c5ef7272542 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:57.997452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:58.025998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f0bce-bad4-4a23-b0ac-3d03704dfd74 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:58.062442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77885944-f313-4dd8-a7cb-004db4715620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:58.063041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77885944-f313-4dd8-a7cb-004db4715620 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:58.063305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77885944-f313-4dd8-a7cb-004db4715620 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:58.063579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77885944-f313-4dd8-a7cb-004db4715620 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:58.064289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77885944-f313-4dd8-a7cb-004db4715620 None admin] RBAC: Adding query filter params (type=compute) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:58.064547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77885944-f313-4dd8-a7cb-004db4715620 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:58.065915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77885944-f313-4dd8-a7cb-004db4715620 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:58.076541 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 490/963] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Aug 20 20:39:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:58.092117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:58.115696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26502c3f-c0f4-4262-a2d8-6c5ef7272542 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:58.133190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77885944-f313-4dd8-a7cb-004db4715620 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:58.162359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94540e8-ab35-439c-ae39-381760fb6207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:58.164086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94540e8-ab35-439c-ae39-381760fb6207 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:58.164617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94540e8-ab35-439c-ae39-381760fb6207 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:58.165660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94540e8-ab35-439c-ae39-381760fb6207 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:58.178126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94540e8-ab35-439c-ae39-381760fb6207 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:58.180650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94540e8-ab35-439c-ae39-381760fb6207 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-f94540e8-ab35-439c-ae39-381760fb6207 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:39:58.199878 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:58.201051 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:58.202014 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:58.202014 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:58.202014 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:39:58.202014 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:39:58.202014 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:39:58.202014 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:39:58.202014 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:39:58.202014 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:39:58.202014 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 20 20:39:58.202014 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:58.204533 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 474/964] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Aug 20 20:39:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:58.207683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94540e8-ab35-439c-ae39-381760fb6207 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:58.285765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77885944-f313-4dd8-a7cb-004db4715620 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:58.321310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1dbca-7308-4baf-86d4-67e3f188da9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:58.322048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1dbca-7308-4baf-86d4-67e3f188da9a None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:58.322320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1dbca-7308-4baf-86d4-67e3f188da9a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:58.322590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1dbca-7308-4baf-86d4-67e3f188da9a None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:58.323231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1dbca-7308-4baf-86d4-67e3f188da9a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:58.324873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1dbca-7308-4baf-86d4-67e3f188da9a None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:58.327774 ncdv-master-1 devstack@keystone.service[104118]: WARNING keystone.server.flask.common [None req-03b1dbca-7308-4baf-86d4-67e3f188da9a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 20 20:39:58.334577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94540e8-ab35-439c-ae39-381760fb6207 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:58.344491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:58.349896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.sql.core [None req-03b1dbca-7308-4baf-86d4-67e3f188da9a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 20 20:39:58.349896 ncdv-master-1 devstack@keystone.service[104118]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 20 20:39:58.349896 ncdv-master-1 devstack@keystone.service[104118]: [parameters: {'id': '6120bc2967bd48fabe055918b89fee56', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 20 20:39:58.349896 ncdv-master-1 devstack@keystone.service[104118]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=104118) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 20 20:39:58.358005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-03b1dbca-7308-4baf-86d4-67e3f188da9a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6120bc2967bd48fabe055918b89fee56.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6120bc2967bd48fabe055918b89fee56. Aug 20 20:39:58.358005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:58.358005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 20 20:39:58.358005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 20 20:39:58.358005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 20 20:39:58.358005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 20 20:39:58.358005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 20 20:39:58.358005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result = self._query(query) Aug 20 20:39:58.358005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 20 20:39:58.358005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application conn.query(q) Aug 20 20:39:58.358005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 20 20:39:58.358005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 20 20:39:58.358005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result.read() Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 20 20:39:58.359184 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application next(self.gen) Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application yield resource Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application next(self.gen) Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.session.rollback() Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.force_reraise() Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise self.value Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application session.commit() Aug 20 20:39:58.360386 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._prepare_impl() Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.session.flush() Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._flush(objects) Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application compat.raise_( Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application flush_context.execute() Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 20 20:39:58.361387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rec.execute(self) Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 20 20:39:58.362304 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result = self._query(query) Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application conn.query(q) Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result.read() Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 20 20:39:58.363256 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [parameters: {'id': '6120bc2967bd48fabe055918b89fee56', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:58.364270 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:58.369936 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:58.370838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:58.370838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 20 20:39:58.370838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 20 20:39:58.370838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:58.370838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:58.370838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 20 20:39:58.370838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 20 20:39:58.370838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6120bc2967bd48fabe055918b89fee56. Aug 20 20:39:58.370838 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:58.370838 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 491/965] 10.222.0.23 () {60 vars in 1140 bytes} [Sat Aug 20 20:39:58 2022] POST /identity/v3/projects => generated 262 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:58.371930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1dbca-7308-4baf-86d4-67e3f188da9a None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:58.396635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da9b00-40ba-4edd-ae99-3af3cba2cb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:58.397680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da9b00-40ba-4edd-ae99-3af3cba2cb18 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:58.398145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da9b00-40ba-4edd-ae99-3af3cba2cb18 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:58.398760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da9b00-40ba-4edd-ae99-3af3cba2cb18 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:58.399726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da9b00-40ba-4edd-ae99-3af3cba2cb18 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:58.400548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da9b00-40ba-4edd-ae99-3af3cba2cb18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:58.402329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da9b00-40ba-4edd-ae99-3af3cba2cb18 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:58.413683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:58.427304 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 475/966] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Aug 20 20:39:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:58.428926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d2f38-aeec-4584-bd92-764e50a0561a None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:58.429364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d2f38-aeec-4584-bd92-764e50a0561a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:58.429818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d2f38-aeec-4584-bd92-764e50a0561a None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:58.431547 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 476/967] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:39:58.432890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d2f38-aeec-4584-bd92-764e50a0561a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:58.491322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1dbca-7308-4baf-86d4-67e3f188da9a None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:58.525365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18ecf2-a30d-4b90-9f88-2af236b40ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:58.526101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18ecf2-a30d-4b90-9f88-2af236b40ef4 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:58.526370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18ecf2-a30d-4b90-9f88-2af236b40ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:58.526631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18ecf2-a30d-4b90-9f88-2af236b40ef4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:58.534018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18ecf2-a30d-4b90-9f88-2af236b40ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:58.535660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18ecf2-a30d-4b90-9f88-2af236b40ef4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:58.539940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d2f38-aeec-4584-bd92-764e50a0561a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-4f18ecf2-a30d-4b90-9f88-2af236b40ef4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:58.547266 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:58.548160 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:58.549128 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:58.549128 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:58.549128 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:58.549128 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:58.549128 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:58.549128 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:58.549128 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:58.549128 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:58.553198 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 492/968] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:58.554650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521e15b-57d5-4f27-8ad8-726cb7da4073 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:58.554920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521e15b-57d5-4f27-8ad8-726cb7da4073 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:58.555181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521e15b-57d5-4f27-8ad8-726cb7da4073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:58.574963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7521e15b-57d5-4f27-8ad8-726cb7da4073 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:39:58.576553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7521e15b-57d5-4f27-8ad8-726cb7da4073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:39:58.583880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4c546-3302-4640-be16-3ea823f089f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:58.584916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4c546-3302-4640-be16-3ea823f089f7 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:58.585189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4c546-3302-4640-be16-3ea823f089f7 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:58.585448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4c546-3302-4640-be16-3ea823f089f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:58.640993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:58.657978 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 493/969] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:39:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:39:58.658809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521e15b-57d5-4f27-8ad8-726cb7da4073 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:58.665155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:58.669578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4c546-3302-4640-be16-3ea823f089f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:58.671383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4c546-3302-4640-be16-3ea823f089f7 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:58.782788 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 477/970] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:39:58 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:39:58.783002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521e15b-57d5-4f27-8ad8-726cb7da4073 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:58.784258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4c546-3302-4640-be16-3ea823f089f7 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:58.827359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab2adb-1ad0-4654-9708-3d6e1c082de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:58.828435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab2adb-1ad0-4654-9708-3d6e1c082de2 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:58.828902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab2adb-1ad0-4654-9708-3d6e1c082de2 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:58.829441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab2adb-1ad0-4654-9708-3d6e1c082de2 None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:58.830296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab2adb-1ad0-4654-9708-3d6e1c082de2 None admin] RBAC: Adding query filter params (name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:58.830809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab2adb-1ad0-4654-9708-3d6e1c082de2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:58.832687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab2adb-1ad0-4654-9708-3d6e1c082de2 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:58.855109 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 494/971] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Aug 20 20:39:58 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:58.891005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:58.895141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4c546-3302-4640-be16-3ea823f089f7 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:58.935368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b40669-f0f5-4fe3-9382-fa3141f2ea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:58.935919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b40669-f0f5-4fe3-9382-fa3141f2ea57 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:58.936190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b40669-f0f5-4fe3-9382-fa3141f2ea57 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:58.936493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b40669-f0f5-4fe3-9382-fa3141f2ea57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:58.948428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b40669-f0f5-4fe3-9382-fa3141f2ea57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:58.949896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b40669-f0f5-4fe3-9382-fa3141f2ea57 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-05b40669-f0f5-4fe3-9382-fa3141f2ea57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:39:58.966405 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:58.967409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:39:58.968296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:39:58.968296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:39:58.968296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:58.968296 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 478/972] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:58.974778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab2adb-1ad0-4654-9708-3d6e1c082de2 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:59.013210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b40669-f0f5-4fe3-9382-fa3141f2ea57 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:59.033202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:59.089109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab2adb-1ad0-4654-9708-3d6e1c082de2 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:59.126595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b40669-f0f5-4fe3-9382-fa3141f2ea57 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:59.127661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb243371-3311-4e43-94bb-6dc414e12f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:59.128734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb243371-3311-4e43-94bb-6dc414e12f7d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:59.129064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb243371-3311-4e43-94bb-6dc414e12f7d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:59.129330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb243371-3311-4e43-94bb-6dc414e12f7d None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:59.130801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb243371-3311-4e43-94bb-6dc414e12f7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:59.131108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb243371-3311-4e43-94bb-6dc414e12f7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:59.132513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb243371-3311-4e43-94bb-6dc414e12f7d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:59.146652 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 495/973] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:59.152246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb243371-3311-4e43-94bb-6dc414e12f7d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:59.172858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b1786-f3c4-46b9-ab0a-d9cecf7b43e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:59.173878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1786-f3c4-46b9-ab0a-d9cecf7b43e0 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:59.174250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1786-f3c4-46b9-ab0a-d9cecf7b43e0 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:59.174629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1786-f3c4-46b9-ab0a-d9cecf7b43e0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:59.175586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b1786-f3c4-46b9-ab0a-d9cecf7b43e0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:59.176973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b1786-f3c4-46b9-ab0a-d9cecf7b43e0 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-d82b1786-f3c4-46b9-ab0a-d9cecf7b43e0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:59.188644 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:59.189827 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:39:59.190825 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:39:59.193403 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 479/974] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Aug 20 20:39:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:39:59.200502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b1786-f3c4-46b9-ab0a-d9cecf7b43e0 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:59.248248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:59.291157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb243371-3311-4e43-94bb-6dc414e12f7d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:59.302912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:59.312768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b1786-f3c4-46b9-ab0a-d9cecf7b43e0 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:59.334201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6daacbf-de39-4cfd-a396-502ef51a8b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:59.334985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6daacbf-de39-4cfd-a396-502ef51a8b60 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:59.335351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6daacbf-de39-4cfd-a396-502ef51a8b60 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:59.335891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6daacbf-de39-4cfd-a396-502ef51a8b60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:59.345026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6daacbf-de39-4cfd-a396-502ef51a8b60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:59.346811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6daacbf-de39-4cfd-a396-502ef51a8b60 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:59.349581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58827f-4106-492e-ad3e-7b0a744d25dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:59.350379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58827f-4106-492e-ad3e-7b0a744d25dd None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:59.350944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58827f-4106-492e-ad3e-7b0a744d25dd None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:59.351361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58827f-4106-492e-ad3e-7b0a744d25dd None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:59.352124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58827f-4106-492e-ad3e-7b0a744d25dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:59.352668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58827f-4106-492e-ad3e-7b0a744d25dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:59.354208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58827f-4106-492e-ad3e-7b0a744d25dd None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:59.359567 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 480/975] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Aug 20 20:39:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-f6daacbf-de39-4cfd-a396-502ef51a8b60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:39:59.360776 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:39:59.361662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:39:59.362568 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:39:59.362568 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:39:59.362568 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:39:59.362568 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:39:59.362568 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:39:59.362568 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:39:59.362568 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:39:59.362568 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:39:59.364496 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 496/976] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:39:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:39:59.365710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58827f-4106-492e-ad3e-7b0a744d25dd None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:59.370063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6daacbf-de39-4cfd-a396-502ef51a8b60 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:59.482600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:59.522879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58827f-4106-492e-ad3e-7b0a744d25dd None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:59.524298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6daacbf-de39-4cfd-a396-502ef51a8b60 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:59.592432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81c0da-c650-4635-9b16-6bf60e5cd2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:59.593687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81c0da-c650-4635-9b16-6bf60e5cd2f5 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:59.594512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81c0da-c650-4635-9b16-6bf60e5cd2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:59.595214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81c0da-c650-4635-9b16-6bf60e5cd2f5 None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:59.596267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81c0da-c650-4635-9b16-6bf60e5cd2f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:59.596853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81c0da-c650-4635-9b16-6bf60e5cd2f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:59.598753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81c0da-c650-4635-9b16-6bf60e5cd2f5 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:59.599794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17bee5-564b-44f8-a5a7-96a545c81e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:59.600545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17bee5-564b-44f8-a5a7-96a545c81e11 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:59.600938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17bee5-564b-44f8-a5a7-96a545c81e11 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:59.601562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17bee5-564b-44f8-a5a7-96a545c81e11 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:59.602309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17bee5-564b-44f8-a5a7-96a545c81e11 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:59.603822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17bee5-564b-44f8-a5a7-96a545c81e11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:59.605350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17bee5-564b-44f8-a5a7-96a545c81e11 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:59.612450 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 481/977] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Aug 20 20:39:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:39:59.615016 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 497/978] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:39:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:59.619958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17bee5-564b-44f8-a5a7-96a545c81e11 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:59.621205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81c0da-c650-4635-9b16-6bf60e5cd2f5 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:59.656153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:59.728552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17bee5-564b-44f8-a5a7-96a545c81e11 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:59.755807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81c0da-c650-4635-9b16-6bf60e5cd2f5 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:59.768647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db026a-74db-47fd-ae86-fe327976161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:59.769656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db026a-74db-47fd-ae86-fe327976161e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:59.770005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db026a-74db-47fd-ae86-fe327976161e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:59.770362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db026a-74db-47fd-ae86-fe327976161e None admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:59.771144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db026a-74db-47fd-ae86-fe327976161e None admin] RBAC: Adding query filter params (interface=admin, service_id=947a8d19050649afa19b265f1180ea15, region_id=RegionOne) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:59.771749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db026a-74db-47fd-ae86-fe327976161e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:59.773007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db026a-74db-47fd-ae86-fe327976161e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:59.783749 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 482/979] 10.222.0.23 () {56 vars in 1249 bytes} [Sat Aug 20 20:39:59 2022] GET /identity/v3/endpoints?service_id=947a8d19050649afa19b265f1180ea15&interface=admin®ion_id=RegionOne => generated 193 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:59.784771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db026a-74db-47fd-ae86-fe327976161e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:39:59.794868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:39:59.802914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701de570-be9e-4da8-82b3-14030c56c39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:59.804319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701de570-be9e-4da8-82b3-14030c56c39c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:59.804880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701de570-be9e-4da8-82b3-14030c56c39c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:59.805371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701de570-be9e-4da8-82b3-14030c56c39c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:59.806682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701de570-be9e-4da8-82b3-14030c56c39c None admin] RBAC: Adding query filter params (role.id=d78b7ae7d2c541498b055d1e2822e9b1, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=1547cb99abe642ae86db8de1fa80041f) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:39:59.807133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701de570-be9e-4da8-82b3-14030c56c39c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:59.809906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701de570-be9e-4da8-82b3-14030c56c39c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:59.827373 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 498/980] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:39:59 2022] GET /identity/v3/role_assignments?user.id=1547cb99abe642ae86db8de1fa80041f&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=d78b7ae7d2c541498b055d1e2822e9b1 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:39:59.912060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db026a-74db-47fd-ae86-fe327976161e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:39:59.949867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a1f3b-ca21-4c64-b4b1-2f26b6c55abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:39:59.950485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a1f3b-ca21-4c64-b4b1-2f26b6c55abf None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:39:59.950759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a1f3b-ca21-4c64-b4b1-2f26b6c55abf None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:39:59.951022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a1f3b-ca21-4c64-b4b1-2f26b6c55abf None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:39:59.951630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a1f3b-ca21-4c64-b4b1-2f26b6c55abf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:39:59.953057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a1f3b-ca21-4c64-b4b1-2f26b6c55abf None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:39:59.960233 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 483/981] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Aug 20 20:39:59 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:40:00.403483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025761f6-52d9-4bc9-86a9-ba421c2b57cb None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:00.403996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025761f6-52d9-4bc9-86a9-ba421c2b57cb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:00.403996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025761f6-52d9-4bc9-86a9-ba421c2b57cb None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:00.405611 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 499/982] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:00.411189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8dac-06de-4f5f-ac29-49ac1559f50d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:00.411477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8dac-06de-4f5f-ac29-49ac1559f50d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:00.411748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8dac-06de-4f5f-ac29-49ac1559f50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:00.430111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7cdf8dac-06de-4f5f-ac29-49ac1559f50d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:00.431562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7cdf8dac-06de-4f5f-ac29-49ac1559f50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:00.498684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:00.504224 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 484/983] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:00.842389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025761f6-52d9-4bc9-86a9-ba421c2b57cb None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:00.949875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:00.960574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025761f6-52d9-4bc9-86a9-ba421c2b57cb None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:00.996010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4735aad-ac18-4cf7-9ca2-b52690f2cfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:00.996652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4735aad-ac18-4cf7-9ca2-b52690f2cfce None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:00.996915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4735aad-ac18-4cf7-9ca2-b52690f2cfce None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:00.997173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4735aad-ac18-4cf7-9ca2-b52690f2cfce None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:00.997823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4735aad-ac18-4cf7-9ca2-b52690f2cfce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:00.999178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4735aad-ac18-4cf7-9ca2-b52690f2cfce None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:01.017354 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 500/984] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:40:00 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:40:01.297678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4128a7-b466-4f9e-ab89-07f613b95d60 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:01.298205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4128a7-b466-4f9e-ab89-07f613b95d60 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:01.298205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4128a7-b466-4f9e-ab89-07f613b95d60 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:01.299797 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 485/985] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:01.305814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c6aa2-51c2-403d-ab33-b1c1602ed255 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:01.306273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c6aa2-51c2-403d-ab33-b1c1602ed255 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:01.306746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c6aa2-51c2-403d-ab33-b1c1602ed255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:01.329913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4a4c6aa2-51c2-403d-ab33-b1c1602ed255 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:01.329913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4a4c6aa2-51c2-403d-ab33-b1c1602ed255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:01.402620 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 501/986] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:01.739215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4128a7-b466-4f9e-ab89-07f613b95d60 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:01.844663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4128a7-b466-4f9e-ab89-07f613b95d60 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:01.878483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55687f6c-9814-4339-b03e-84b308784170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:01.879079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55687f6c-9814-4339-b03e-84b308784170 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:01.879348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55687f6c-9814-4339-b03e-84b308784170 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:01.879609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55687f6c-9814-4339-b03e-84b308784170 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:01.880398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55687f6c-9814-4339-b03e-84b308784170 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:01.881826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55687f6c-9814-4339-b03e-84b308784170 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:01.884656 ncdv-master-1 devstack@keystone.service[104119]: WARNING keystone.server.flask.common [None req-55687f6c-9814-4339-b03e-84b308784170 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 20 20:40:01.899096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.sql.core [None req-55687f6c-9814-4339-b03e-84b308784170 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 20 20:40:01.899096 ncdv-master-1 devstack@keystone.service[104119]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 20 20:40:01.899096 ncdv-master-1 devstack@keystone.service[104119]: [parameters: {'user_id': '28c9181a91aa4fea91470d4f70a5c401', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 20 20:40:01.899096 ncdv-master-1 devstack@keystone.service[104119]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=104119) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-55687f6c-9814-4339-b03e-84b308784170 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application result = self._query(query) Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application conn.query(q) Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application result.read() Aug 20 20:40:01.915314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 20 20:40:01.917791 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application next(self.gen) Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application yield resource Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application next(self.gen) Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.session.rollback() Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.force_reraise() Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise self.value Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application session.commit() Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 20 20:40:01.920180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._prepare_impl() Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.session.flush() Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._flush(objects) Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application compat.raise_( Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application flush_context.execute() Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rec.execute(self) Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 20 20:40:01.922461 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 20 20:40:01.924573 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application result = self._query(query) Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application conn.query(q) Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application result.read() Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 20 20:40:01.932199 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [parameters: {'user_id': '28c9181a91aa4fea91470d4f70a5c401', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:01.934622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 20 20:40:01.936543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 20 20:40:01.938622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 20 20:40:01.938622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 20 20:40:01.938622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 20 20:40:01.938622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 20 20:40:01.938622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 20 20:40:01.938622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:01.938622 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 486/987] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Aug 20 20:40:01 2022] POST /identity/v3/users => generated 160 bytes in 178 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:01.939521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c6aa2-51c2-403d-ab33-b1c1602ed255 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:02.006121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:02.029929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c6aa2-51c2-403d-ab33-b1c1602ed255 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:02.066769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba616c-f260-4f18-bad0-170ee3c7fe6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:02.067424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba616c-f260-4f18-bad0-170ee3c7fe6a None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:02.067690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba616c-f260-4f18-bad0-170ee3c7fe6a None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:02.067955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba616c-f260-4f18-bad0-170ee3c7fe6a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:02.076609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba616c-f260-4f18-bad0-170ee3c7fe6a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:02.078100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba616c-f260-4f18-bad0-170ee3c7fe6a None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-46ba616c-f260-4f18-bad0-170ee3c7fe6a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:40:02.091520 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:02.093756 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:02.096004 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:02.096004 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:02.096004 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:40:02.096004 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:40:02.096004 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:40:02.096004 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:40:02.096004 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:40:02.096004 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:40:02.096004 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 20 20:40:02.096004 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:02.096004 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 502/988] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:40:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:02.101861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55687f6c-9814-4339-b03e-84b308784170 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:02.167221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:02.212646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55687f6c-9814-4339-b03e-84b308784170 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:02.247055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9daacef-3c3a-4651-b534-6b39a84b9eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:02.247658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daacef-3c3a-4651-b534-6b39a84b9eca None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:02.247984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daacef-3c3a-4651-b534-6b39a84b9eca None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:02.248254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daacef-3c3a-4651-b534-6b39a84b9eca None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:02.249351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9daacef-3c3a-4651-b534-6b39a84b9eca None admin] RBAC: Adding query filter params (name=cinder) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:02.250749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f91943-eb98-47f9-93d7-631fff7a74ae None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:02.251009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f91943-eb98-47f9-93d7-631fff7a74ae None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:02.251268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f91943-eb98-47f9-93d7-631fff7a74ae None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:02.252765 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 503/989] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:02.252944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9daacef-3c3a-4651-b534-6b39a84b9eca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:02.254410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9daacef-3c3a-4651-b534-6b39a84b9eca None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:02.258497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ce093-f245-43ed-a81e-f8074edfdf66 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:02.259007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ce093-f245-43ed-a81e-f8074edfdf66 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:02.259716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ce093-f245-43ed-a81e-f8074edfdf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:02.278945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-029ce093-f245-43ed-a81e-f8074edfdf66 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:02.280107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-029ce093-f245-43ed-a81e-f8074edfdf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:02.298047 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 487/990] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Aug 20 20:40:02 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:02.299664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283eb5-40c9-47bc-8437-264f062f10b4 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:02.300040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283eb5-40c9-47bc-8437-264f062f10b4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:02.300430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283eb5-40c9-47bc-8437-264f062f10b4 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:02.301824 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 488/991] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:02.308100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7c1c2-7b50-4fb8-baea-e7cc0061f0ff None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:02.308547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7c1c2-7b50-4fb8-baea-e7cc0061f0ff None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:02.309111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7c1c2-7b50-4fb8-baea-e7cc0061f0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:02.329603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fed7c1c2-7b50-4fb8-baea-e7cc0061f0ff None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:02.331173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fed7c1c2-7b50-4fb8-baea-e7cc0061f0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:02.376337 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 504/992] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:40:02.413969 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 489/993] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:40:02.705162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ce093-f245-43ed-a81e-f8074edfdf66 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:02.750585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7c1c2-7b50-4fb8-baea-e7cc0061f0ff None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:02.776879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:02.805927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:02.817168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ce093-f245-43ed-a81e-f8074edfdf66 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:02.858451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77a3fe-b8cd-447c-a3cc-b85b12b3ea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:02.859085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77a3fe-b8cd-447c-a3cc-b85b12b3ea7b None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:02.859357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77a3fe-b8cd-447c-a3cc-b85b12b3ea7b None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:02.859621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77a3fe-b8cd-447c-a3cc-b85b12b3ea7b None admin] PATH_INFO: `/v3/services/image` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:02.860199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77a3fe-b8cd-447c-a3cc-b85b12b3ea7b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:02.860455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7c1c2-7b50-4fb8-baea-e7cc0061f0ff None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:02.861607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77a3fe-b8cd-447c-a3cc-b85b12b3ea7b None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-5c77a3fe-b8cd-447c-a3cc-b85b12b3ea7b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:02.872669 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:02.873926 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:02.874789 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 505/994] 10.222.0.23 () {56 vars in 1105 bytes} [Sat Aug 20 20:40:02 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:02.880199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77a3fe-b8cd-447c-a3cc-b85b12b3ea7b None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:02.903047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9b46e-ab40-49c0-b1e1-932d2fa48022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:02.903806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9b46e-ab40-49c0-b1e1-932d2fa48022 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:02.904249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9b46e-ab40-49c0-b1e1-932d2fa48022 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:02.904560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9b46e-ab40-49c0-b1e1-932d2fa48022 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:02.905146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9b46e-ab40-49c0-b1e1-932d2fa48022 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:02.907047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9b46e-ab40-49c0-b1e1-932d2fa48022 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9ed9b46e-ab40-49c0-b1e1-932d2fa48022 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:02.917605 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:02.918805 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 20 20:40:02.919824 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:02.923668 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 490/995] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Aug 20 20:40:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:02.924511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9b46e-ab40-49c0-b1e1-932d2fa48022 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:02.992099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77a3fe-b8cd-447c-a3cc-b85b12b3ea7b None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:03.023711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:03.030318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33166432-ae6e-4edc-b50a-aac5b44cb35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:03.030925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166432-ae6e-4edc-b50a-aac5b44cb35c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:03.031195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166432-ae6e-4edc-b50a-aac5b44cb35c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:03.031455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166432-ae6e-4edc-b50a-aac5b44cb35c None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:03.032079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33166432-ae6e-4edc-b50a-aac5b44cb35c None admin] RBAC: Adding query filter params (name=image) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:03.032390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33166432-ae6e-4edc-b50a-aac5b44cb35c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:03.033753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33166432-ae6e-4edc-b50a-aac5b44cb35c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:03.035151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9b46e-ab40-49c0-b1e1-932d2fa48022 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:03.038805 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 506/996] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:03.045199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33166432-ae6e-4edc-b50a-aac5b44cb35c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:03.051006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:03.075935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae197f80-534d-418f-9117-5fc82f0c6e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:03.076832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae197f80-534d-418f-9117-5fc82f0c6e1d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:03.076832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae197f80-534d-418f-9117-5fc82f0c6e1d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:03.077073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae197f80-534d-418f-9117-5fc82f0c6e1d None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:03.077685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae197f80-534d-418f-9117-5fc82f0c6e1d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:03.077981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae197f80-534d-418f-9117-5fc82f0c6e1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:03.079340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae197f80-534d-418f-9117-5fc82f0c6e1d None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:03.084091 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 491/997] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Aug 20 20:40:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:03.090082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae197f80-534d-418f-9117-5fc82f0c6e1d None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:03.170086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33166432-ae6e-4edc-b50a-aac5b44cb35c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:03.201560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae197f80-534d-418f-9117-5fc82f0c6e1d None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:03.205805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b44019-d9d9-4bd8-816e-446a62740ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:03.206437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b44019-d9d9-4bd8-816e-446a62740ed9 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:03.206707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b44019-d9d9-4bd8-816e-446a62740ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:03.206968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b44019-d9d9-4bd8-816e-446a62740ed9 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:03.207589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b44019-d9d9-4bd8-816e-446a62740ed9 None admin] RBAC: Adding query filter params (type=image) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:03.207880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b44019-d9d9-4bd8-816e-446a62740ed9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:03.209278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b44019-d9d9-4bd8-816e-446a62740ed9 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:03.214350 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 507/998] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:03.247648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77ca8b-f109-46b7-9b62-ed6205144d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:03.248182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77ca8b-f109-46b7-9b62-ed6205144d7e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:03.248493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77ca8b-f109-46b7-9b62-ed6205144d7e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:03.248759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77ca8b-f109-46b7-9b62-ed6205144d7e None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:03.249483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77ca8b-f109-46b7-9b62-ed6205144d7e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:03.249782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77ca8b-f109-46b7-9b62-ed6205144d7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:03.251230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77ca8b-f109-46b7-9b62-ed6205144d7e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:03.257100 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 492/999] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Aug 20 20:40:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:03.263024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b44019-d9d9-4bd8-816e-446a62740ed9 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:03.297577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e5dca-2a2e-473c-9b79-72075311799c None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:03.297859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e5dca-2a2e-473c-9b79-72075311799c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:03.298127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e5dca-2a2e-473c-9b79-72075311799c None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:03.299651 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 493/1000] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:03.305658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdad50-9064-40b6-9f03-2d7109b6c073 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:03.306021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdad50-9064-40b6-9f03-2d7109b6c073 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:03.306202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdad50-9064-40b6-9f03-2d7109b6c073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:03.326347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-69cdad50-9064-40b6-9f03-2d7109b6c073 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:03.327696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-69cdad50-9064-40b6-9f03-2d7109b6c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:03.335437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:03.377003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:03.393003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b44019-d9d9-4bd8-816e-446a62740ed9 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:03.420130 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 494/1001] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:40:03.432644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83383fa9-2843-420d-bc42-4551400db8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:03.433247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83383fa9-2843-420d-bc42-4551400db8bc None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:03.433522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83383fa9-2843-420d-bc42-4551400db8bc None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:03.434129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83383fa9-2843-420d-bc42-4551400db8bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:03.434809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83383fa9-2843-420d-bc42-4551400db8bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:03.436161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83383fa9-2843-420d-bc42-4551400db8bc None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:03.453523 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 508/1002] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:40:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:40:03.752271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdad50-9064-40b6-9f03-2d7109b6c073 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:03.858960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdad50-9064-40b6-9f03-2d7109b6c073 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:03.901789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0d843-e472-4aaa-ab57-580f04293616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:03.902422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0d843-e472-4aaa-ab57-580f04293616 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:03.902693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0d843-e472-4aaa-ab57-580f04293616 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:03.902954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0d843-e472-4aaa-ab57-580f04293616 None admin] PATH_INFO: `/v3/services/compute` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:03.903524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0d843-e472-4aaa-ab57-580f04293616 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:03.904908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0d843-e472-4aaa-ab57-580f04293616 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-6fe0d843-e472-4aaa-ab57-580f04293616 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:03.913789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:03.914939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 20 20:40:03.915794 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:03.917167 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 495/1003] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:40:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:03.921322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83383fa9-2843-420d-bc42-4551400db8bc None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:04.031253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83383fa9-2843-420d-bc42-4551400db8bc None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:04.051489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:04.068911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bee06-7577-42d3-ba21-9efa58a44c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:04.069561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bee06-7577-42d3-ba21-9efa58a44c43 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:04.069829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bee06-7577-42d3-ba21-9efa58a44c43 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:04.070087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bee06-7577-42d3-ba21-9efa58a44c43 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:04.070703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758bee06-7577-42d3-ba21-9efa58a44c43 None admin] RBAC: Adding query filter params (name=compute) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:04.071029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758bee06-7577-42d3-ba21-9efa58a44c43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:04.072411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758bee06-7577-42d3-ba21-9efa58a44c43 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:04.077537 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 509/1004] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Aug 20 20:40:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:04.083188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0d843-e472-4aaa-ab57-580f04293616 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:04.115315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:04.195708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0d843-e472-4aaa-ab57-580f04293616 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:04.238848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a363a8-03a2-4d8e-8355-0b5bac2d8c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:04.239451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a363a8-03a2-4d8e-8355-0b5bac2d8c4b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:04.239692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a363a8-03a2-4d8e-8355-0b5bac2d8c4b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:04.239980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a363a8-03a2-4d8e-8355-0b5bac2d8c4b None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:04.240637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a363a8-03a2-4d8e-8355-0b5bac2d8c4b None admin] RBAC: Adding query filter params (type=compute) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:04.240961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a363a8-03a2-4d8e-8355-0b5bac2d8c4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:04.242323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a363a8-03a2-4d8e-8355-0b5bac2d8c4b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:04.252548 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 496/1005] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Aug 20 20:40:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:04.253520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bee06-7577-42d3-ba21-9efa58a44c43 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:04.360594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bee06-7577-42d3-ba21-9efa58a44c43 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:04.397844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0dd0d7-4630-49e3-877c-7b5adaa167f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:04.398451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dd0d7-4630-49e3-877c-7b5adaa167f7 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:04.398721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dd0d7-4630-49e3-877c-7b5adaa167f7 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:04.398985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dd0d7-4630-49e3-877c-7b5adaa167f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:04.399704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0dd0d7-4630-49e3-877c-7b5adaa167f7 None admin] RBAC: Adding query filter params (interface=public, service_id=35fb49457a7a44b2900b9c346a51555f, region_id=RegionOne) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:04.400198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0dd0d7-4630-49e3-877c-7b5adaa167f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:04.401658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0dd0d7-4630-49e3-877c-7b5adaa167f7 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:04.407038 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 510/1006] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Aug 20 20:40:04 2022] GET /identity/v3/endpoints?service_id=35fb49457a7a44b2900b9c346a51555f&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:04.412520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a363a8-03a2-4d8e-8355-0b5bac2d8c4b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:04.510813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:04.518408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a363a8-03a2-4d8e-8355-0b5bac2d8c4b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:04.553382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3cc24-3d65-4077-8bfc-7446e15b52f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:04.553980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3cc24-3d65-4077-8bfc-7446e15b52f1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:04.554253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3cc24-3d65-4077-8bfc-7446e15b52f1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:04.554517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3cc24-3d65-4077-8bfc-7446e15b52f1 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:04.555111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3cc24-3d65-4077-8bfc-7446e15b52f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:04.556565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3cc24-3d65-4077-8bfc-7446e15b52f1 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:04.563313 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 497/1007] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Aug 20 20:40:04 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:40:05.412432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493538d-56d3-4636-a51b-c9a17c3471e9 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:05.413016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493538d-56d3-4636-a51b-c9a17c3471e9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:05.413016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493538d-56d3-4636-a51b-c9a17c3471e9 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:05.414471 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 511/1008] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:05.419615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56321c-d629-4f99-8a44-acd4d6afaa83 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:05.420059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56321c-d629-4f99-8a44-acd4d6afaa83 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:05.420140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56321c-d629-4f99-8a44-acd4d6afaa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:05.438684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bd56321c-d629-4f99-8a44-acd4d6afaa83 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:05.439836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bd56321c-d629-4f99-8a44-acd4d6afaa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:05.511606 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 498/1009] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:05.727301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbea61e-560c-484a-990f-56da33896254 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:05.727724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbea61e-560c-484a-990f-56da33896254 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:05.727991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbea61e-560c-484a-990f-56da33896254 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:05.729538 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 512/1010] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:05.734859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c86ffd-8b04-4ec1-b64b-b401a4a8ba2e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:05.735118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c86ffd-8b04-4ec1-b64b-b401a4a8ba2e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:05.735384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c86ffd-8b04-4ec1-b64b-b401a4a8ba2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:05.753863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-17c86ffd-8b04-4ec1-b64b-b401a4a8ba2e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:05.755037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-17c86ffd-8b04-4ec1-b64b-b401a4a8ba2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:05.828012 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 499/1011] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:40:05.834518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbea61e-560c-484a-990f-56da33896254 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:05.884933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:05.938616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbea61e-560c-484a-990f-56da33896254 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:05.971637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc613126-4dcb-4c37-bf79-e231a3fba031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:05.972225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc613126-4dcb-4c37-bf79-e231a3fba031 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:05.972534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc613126-4dcb-4c37-bf79-e231a3fba031 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:05.972796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc613126-4dcb-4c37-bf79-e231a3fba031 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:05.973414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc613126-4dcb-4c37-bf79-e231a3fba031 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:05.974764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc613126-4dcb-4c37-bf79-e231a3fba031 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:05.990835 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 513/1012] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:40:05 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:40:06.174417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c86ffd-8b04-4ec1-b64b-b401a4a8ba2e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:06.198766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:06.283684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c86ffd-8b04-4ec1-b64b-b401a4a8ba2e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:06.327870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1864c-d25f-43f9-a3ec-176b010aef84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:06.328526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1864c-d25f-43f9-a3ec-176b010aef84 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:06.328801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1864c-d25f-43f9-a3ec-176b010aef84 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:06.329058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1864c-d25f-43f9-a3ec-176b010aef84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:06.330164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1864c-d25f-43f9-a3ec-176b010aef84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:06.331595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1864c-d25f-43f9-a3ec-176b010aef84 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:06.333219 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 500/1013] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:06.340077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc613126-4dcb-4c37-bf79-e231a3fba031 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:06.468663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc613126-4dcb-4c37-bf79-e231a3fba031 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:06.479573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:06.508834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e6953-52f9-478e-9384-aff1cfdea1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:06.509440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e6953-52f9-478e-9384-aff1cfdea1b7 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:06.509679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e6953-52f9-478e-9384-aff1cfdea1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:06.509933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e6953-52f9-478e-9384-aff1cfdea1b7 None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:06.510569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e6953-52f9-478e-9384-aff1cfdea1b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:06.512007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e6953-52f9-478e-9384-aff1cfdea1b7 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:06.548427 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 514/1014] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:40:06 2022] POST /identity/v3/projects => generated 311 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:40:06.777477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a628047-853b-42da-9820-51f13e2394b1 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:06.778047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a628047-853b-42da-9820-51f13e2394b1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:06.778047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a628047-853b-42da-9820-51f13e2394b1 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:06.779597 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 501/1015] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:06.784854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c9bd2-173f-4bcd-891b-2768183f345f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:06.785130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c9bd2-173f-4bcd-891b-2768183f345f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:06.785400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c9bd2-173f-4bcd-891b-2768183f345f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:06.805404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f35c9bd2-173f-4bcd-891b-2768183f345f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:06.807012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f35c9bd2-173f-4bcd-891b-2768183f345f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:06.879071 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 515/1016] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:06 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:07.202726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a628047-853b-42da-9820-51f13e2394b1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:07.301857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:07.319864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a628047-853b-42da-9820-51f13e2394b1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:07.353646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e03ff9-f6e5-4c89-8afb-6c486fffda62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:07.354488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03ff9-f6e5-4c89-8afb-6c486fffda62 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:07.354733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03ff9-f6e5-4c89-8afb-6c486fffda62 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:07.355006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03ff9-f6e5-4c89-8afb-6c486fffda62 None admin] PATH_INFO: `/v3/services/compute` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:07.355626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e03ff9-f6e5-4c89-8afb-6c486fffda62 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:07.357021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e03ff9-f6e5-4c89-8afb-6c486fffda62 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-30e03ff9-f6e5-4c89-8afb-6c486fffda62 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:07.365673 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:07.366677 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:07.369477 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 502/1017] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:40:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:07.372901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c9bd2-173f-4bcd-891b-2768183f345f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:07.468250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:07.479217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c9bd2-173f-4bcd-891b-2768183f345f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:07.513854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324429d9-34e5-4899-8755-72b45b280184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:07.514469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324429d9-34e5-4899-8755-72b45b280184 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:07.514733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324429d9-34e5-4899-8755-72b45b280184 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:07.514988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324429d9-34e5-4899-8755-72b45b280184 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:07.515647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324429d9-34e5-4899-8755-72b45b280184 None admin] RBAC: Adding query filter params (name=compute) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:07.515953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324429d9-34e5-4899-8755-72b45b280184 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:07.517384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324429d9-34e5-4899-8755-72b45b280184 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:07.527046 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 516/1018] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Aug 20 20:40:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:07.528266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e03ff9-f6e5-4c89-8afb-6c486fffda62 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:07.633737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e03ff9-f6e5-4c89-8afb-6c486fffda62 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:07.668337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f378eac-806e-4f7d-81f5-cca949c2a977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:07.668964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f378eac-806e-4f7d-81f5-cca949c2a977 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:07.669230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f378eac-806e-4f7d-81f5-cca949c2a977 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:07.669492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f378eac-806e-4f7d-81f5-cca949c2a977 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:07.670110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f378eac-806e-4f7d-81f5-cca949c2a977 None admin] RBAC: Adding query filter params (type=compute) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:07.670403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f378eac-806e-4f7d-81f5-cca949c2a977 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:07.671739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f378eac-806e-4f7d-81f5-cca949c2a977 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:07.677030 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 503/1019] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Aug 20 20:40:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:07.683003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324429d9-34e5-4899-8755-72b45b280184 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:07.790679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324429d9-34e5-4899-8755-72b45b280184 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:07.826640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373ec12-a2fc-4fbd-95d2-c87fd45258b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:07.827244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373ec12-a2fc-4fbd-95d2-c87fd45258b7 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:07.827514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373ec12-a2fc-4fbd-95d2-c87fd45258b7 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:07.827779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373ec12-a2fc-4fbd-95d2-c87fd45258b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:07.828471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373ec12-a2fc-4fbd-95d2-c87fd45258b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:07.829837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373ec12-a2fc-4fbd-95d2-c87fd45258b7 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:07.850211 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 517/1020] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:40:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:40:08.221836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8ac88-d219-4e72-ad2b-3237915835e3 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:08.222351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8ac88-d219-4e72-ad2b-3237915835e3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:08.222351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8ac88-d219-4e72-ad2b-3237915835e3 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:08.224469 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 504/1021] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:08.229513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6c703-0151-4d53-af3e-ce5f337f5cc1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:08.229774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6c703-0151-4d53-af3e-ce5f337f5cc1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:08.230120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6c703-0151-4d53-af3e-ce5f337f5cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:08.247931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c7f6c703-0151-4d53-af3e-ce5f337f5cc1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:08.249135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c7f6c703-0151-4d53-af3e-ce5f337f5cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:08.252005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:08.328116 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 518/1022] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:08.657158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8ac88-d219-4e72-ad2b-3237915835e3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:08.672563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:08.762511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8ac88-d219-4e72-ad2b-3237915835e3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:08.777612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be317b-5c30-4a35-b454-357c923643d5 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:08.778134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be317b-5c30-4a35-b454-357c923643d5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:08.778134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be317b-5c30-4a35-b454-357c923643d5 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:08.779754 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 519/1023] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:08.784423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb6bda-8d17-4b01-8c52-93deb1aa8349 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:08.784716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb6bda-8d17-4b01-8c52-93deb1aa8349 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:08.784977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb6bda-8d17-4b01-8c52-93deb1aa8349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:08.797568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26d073-8e69-46dd-99d5-bc108e7906e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:08.798186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26d073-8e69-46dd-99d5-bc108e7906e6 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:08.798452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26d073-8e69-46dd-99d5-bc108e7906e6 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:08.798707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26d073-8e69-46dd-99d5-bc108e7906e6 None admin] PATH_INFO: `/v3/services/image` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:08.799275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26d073-8e69-46dd-99d5-bc108e7906e6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:08.800637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26d073-8e69-46dd-99d5-bc108e7906e6 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:08.802727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c9bb6bda-8d17-4b01-8c52-93deb1aa8349 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:08.803930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c9bb6bda-8d17-4b01-8c52-93deb1aa8349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-de26d073-8e69-46dd-99d5-bc108e7906e6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:08.810230 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:08.811240 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:08.812098 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 505/1024] 10.222.0.23 () {56 vars in 1105 bytes} [Sat Aug 20 20:40:08 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:08.819152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26d073-8e69-46dd-99d5-bc108e7906e6 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:08.875038 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 520/1025] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:40:08.903026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:08.927667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26d073-8e69-46dd-99d5-bc108e7906e6 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:08.963517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010a56c-ef0b-43bb-ab32-1ab7746d5b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:08.964123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010a56c-ef0b-43bb-ab32-1ab7746d5b76 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:08.964419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010a56c-ef0b-43bb-ab32-1ab7746d5b76 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:08.964691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010a56c-ef0b-43bb-ab32-1ab7746d5b76 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:08.965314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010a56c-ef0b-43bb-ab32-1ab7746d5b76 None admin] RBAC: Adding query filter params (name=image) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:08.965608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010a56c-ef0b-43bb-ab32-1ab7746d5b76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:08.966959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010a56c-ef0b-43bb-ab32-1ab7746d5b76 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:08.972309 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 506/1026] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:08.978087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb6bda-8d17-4b01-8c52-93deb1aa8349 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:09.045206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:09.082940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb6bda-8d17-4b01-8c52-93deb1aa8349 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:09.116318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309e4f5-9802-40d9-b621-e42b1a1c368f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:09.116935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309e4f5-9802-40d9-b621-e42b1a1c368f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:09.117220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309e4f5-9802-40d9-b621-e42b1a1c368f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:09.117483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309e4f5-9802-40d9-b621-e42b1a1c368f None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:09.118102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309e4f5-9802-40d9-b621-e42b1a1c368f None admin] RBAC: Adding query filter params (type=image) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:09.118391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309e4f5-9802-40d9-b621-e42b1a1c368f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:09.119741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309e4f5-9802-40d9-b621-e42b1a1c368f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:09.124744 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 521/1027] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:09.130654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010a56c-ef0b-43bb-ab32-1ab7746d5b76 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:09.206792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309e4f5-9802-40d9-b621-e42b1a1c368f None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:09.241981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010a56c-ef0b-43bb-ab32-1ab7746d5b76 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:09.277442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c58b88-1b01-4e76-a83b-8e15386d4000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:09.278042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c58b88-1b01-4e76-a83b-8e15386d4000 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:09.278311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c58b88-1b01-4e76-a83b-8e15386d4000 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:09.278571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c58b88-1b01-4e76-a83b-8e15386d4000 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:09.279250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c58b88-1b01-4e76-a83b-8e15386d4000 None admin] RBAC: Adding query filter params (interface=public, service_id=9ea06ddc94974a30b466f65ffc4f2e7e, region_id=RegionOne) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:09.279548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c58b88-1b01-4e76-a83b-8e15386d4000 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:09.280929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c58b88-1b01-4e76-a83b-8e15386d4000 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:09.286646 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 507/1028] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Aug 20 20:40:09 2022] GET /identity/v3/endpoints?service_id=9ea06ddc94974a30b466f65ffc4f2e7e&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:09.292009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c58b88-1b01-4e76-a83b-8e15386d4000 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:09.295453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:09.315382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309e4f5-9802-40d9-b621-e42b1a1c368f None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:09.345613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:09.352247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd43b1-90e0-4901-bc1f-2b560186caa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:09.352973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd43b1-90e0-4901-bc1f-2b560186caa8 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:09.353242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd43b1-90e0-4901-bc1f-2b560186caa8 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:09.353501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd43b1-90e0-4901-bc1f-2b560186caa8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:09.354680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd43b1-90e0-4901-bc1f-2b560186caa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:09.356108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd43b1-90e0-4901-bc1f-2b560186caa8 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:09.357708 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 522/1029] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:09.363424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd43b1-90e0-4901-bc1f-2b560186caa8 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:09.400203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c58b88-1b01-4e76-a83b-8e15386d4000 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:09.440162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bea11-3d8f-41af-b628-3e8670801c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:09.440795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bea11-3d8f-41af-b628-3e8670801c0a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:09.441070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bea11-3d8f-41af-b628-3e8670801c0a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:09.441332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bea11-3d8f-41af-b628-3e8670801c0a None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:09.441934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bea11-3d8f-41af-b628-3e8670801c0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:09.443290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bea11-3d8f-41af-b628-3e8670801c0a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:09.450145 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 508/1030] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Aug 20 20:40:09 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:40:09.476827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd43b1-90e0-4901-bc1f-2b560186caa8 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:09.514683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b960ca-95b6-48ad-83bf-4e0f48095e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:09.515307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b960ca-95b6-48ad-83bf-4e0f48095e14 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:09.515568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b960ca-95b6-48ad-83bf-4e0f48095e14 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:09.515828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b960ca-95b6-48ad-83bf-4e0f48095e14 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:09.516509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b960ca-95b6-48ad-83bf-4e0f48095e14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:09.517973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b960ca-95b6-48ad-83bf-4e0f48095e14 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:09.543804 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 523/1031] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:40:09 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:40:10.165660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160a5b4-6045-4029-9138-c4a087127a3f None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:10.165904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160a5b4-6045-4029-9138-c4a087127a3f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:10.166195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160a5b4-6045-4029-9138-c4a087127a3f None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:10.167727 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 509/1032] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:10.173205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c04bb0-548a-4614-a39d-b663112d1800 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:10.173664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c04bb0-548a-4614-a39d-b663112d1800 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:10.173739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c04bb0-548a-4614-a39d-b663112d1800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:10.192248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f9c04bb0-548a-4614-a39d-b663112d1800 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:10.193454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f9c04bb0-548a-4614-a39d-b663112d1800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:10.250070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:10.270698 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 524/1033] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:10.596676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160a5b4-6045-4029-9138-c4a087127a3f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:10.667908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:10.706592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160a5b4-6045-4029-9138-c4a087127a3f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:10.741820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f3271-4a67-4cc9-be8e-5298f2ff8159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:10.742435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f3271-4a67-4cc9-be8e-5298f2ff8159 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:10.742874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f3271-4a67-4cc9-be8e-5298f2ff8159 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:10.743193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f3271-4a67-4cc9-be8e-5298f2ff8159 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:10.754019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f3271-4a67-4cc9-be8e-5298f2ff8159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-2d3f3271-4a67-4cc9-be8e-5298f2ff8159 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:40:10.758769 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:10.759694 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:10.760985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:10.760985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:10.760985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:10.760985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:10.760985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:40:10.760985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:40:10.760985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:40:10.760985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:40:10.760985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:40:10.760985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:40:10.760985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 20 20:40:10.760985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:10.764730 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 510/1034] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Aug 20 20:40:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:10.766187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c04bb0-548a-4614-a39d-b663112d1800 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:10.885945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c04bb0-548a-4614-a39d-b663112d1800 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:10.925847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663aeb3-ceae-40de-bf9d-6f1d67285a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:10.926452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663aeb3-ceae-40de-bf9d-6f1d67285a4f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:10.926718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663aeb3-ceae-40de-bf9d-6f1d67285a4f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:10.926981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663aeb3-ceae-40de-bf9d-6f1d67285a4f None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:10.927631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663aeb3-ceae-40de-bf9d-6f1d67285a4f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:10.927920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663aeb3-ceae-40de-bf9d-6f1d67285a4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:10.929317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663aeb3-ceae-40de-bf9d-6f1d67285a4f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:10.942833 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 525/1035] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Aug 20 20:40:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:10.944133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f3271-4a67-4cc9-be8e-5298f2ff8159 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:11.056485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f3271-4a67-4cc9-be8e-5298f2ff8159 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:11.085328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:11.095818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17e84e-4180-4bbd-9454-79f9f1d2e115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:11.096462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17e84e-4180-4bbd-9454-79f9f1d2e115 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:11.096778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17e84e-4180-4bbd-9454-79f9f1d2e115 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:11.097053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17e84e-4180-4bbd-9454-79f9f1d2e115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:11.107943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17e84e-4180-4bbd-9454-79f9f1d2e115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:11.109496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17e84e-4180-4bbd-9454-79f9f1d2e115 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9a17e84e-4180-4bbd-9454-79f9f1d2e115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:40:11.124599 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:11.125650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:40:11.131795 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:40:11.131795 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:11.131795 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:11.131795 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 511/1036] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:11.132046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663aeb3-ceae-40de-bf9d-6f1d67285a4f None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:11.183096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:11.250863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663aeb3-ceae-40de-bf9d-6f1d67285a4f None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:11.286233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421d7dc-b28a-4472-b3d8-ea2ea29dfc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:11.286853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421d7dc-b28a-4472-b3d8-ea2ea29dfc3d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:11.287119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421d7dc-b28a-4472-b3d8-ea2ea29dfc3d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:11.287377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421d7dc-b28a-4472-b3d8-ea2ea29dfc3d None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:11.288000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421d7dc-b28a-4472-b3d8-ea2ea29dfc3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:11.288293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421d7dc-b28a-4472-b3d8-ea2ea29dfc3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:11.289723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421d7dc-b28a-4472-b3d8-ea2ea29dfc3d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:11.302309 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 526/1037] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:11.308997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17e84e-4180-4bbd-9454-79f9f1d2e115 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:11.416076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17e84e-4180-4bbd-9454-79f9f1d2e115 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:11.457954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec32563-e22a-4a99-8941-6416cbaffdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:11.458494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec32563-e22a-4a99-8941-6416cbaffdc8 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:11.458764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec32563-e22a-4a99-8941-6416cbaffdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:11.459079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec32563-e22a-4a99-8941-6416cbaffdc8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:11.467355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec32563-e22a-4a99-8941-6416cbaffdc8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:11.468817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec32563-e22a-4a99-8941-6416cbaffdc8 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-2ec32563-e22a-4a99-8941-6416cbaffdc8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:40:11.481624 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:11.482659 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:11.483669 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:11.483669 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:11.483669 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:40:11.483669 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:40:11.483669 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:40:11.483669 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:40:11.483669 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:40:11.483669 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:40:11.483669 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:40:11.483669 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:11.488415 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 512/1038] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Aug 20 20:40:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:11.489350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421d7dc-b28a-4472-b3d8-ea2ea29dfc3d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:11.602680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421d7dc-b28a-4472-b3d8-ea2ea29dfc3d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:11.611902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:11.642553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a9438-258f-4d37-ac6d-7e2d0bfc3620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:11.643165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a9438-258f-4d37-ac6d-7e2d0bfc3620 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:11.643428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a9438-258f-4d37-ac6d-7e2d0bfc3620 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:11.643682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a9438-258f-4d37-ac6d-7e2d0bfc3620 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:11.644401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a9438-258f-4d37-ac6d-7e2d0bfc3620 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:11.644707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a9438-258f-4d37-ac6d-7e2d0bfc3620 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:11.646188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a9438-258f-4d37-ac6d-7e2d0bfc3620 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:11.668571 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 527/1039] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Aug 20 20:40:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:11.670742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec32563-e22a-4a99-8941-6416cbaffdc8 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:11.717579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:11.738700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f9efe-d18a-4e1f-933c-abcca66476ce None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:11.738984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f9efe-d18a-4e1f-933c-abcca66476ce None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:11.739231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f9efe-d18a-4e1f-933c-abcca66476ce None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:11.741198 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 528/1040] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:11.747640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb772615-e67b-42ea-ad73-3403c40f0bcc None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:11.747913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb772615-e67b-42ea-ad73-3403c40f0bcc None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:11.748177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb772615-e67b-42ea-ad73-3403c40f0bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:11.766762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fb772615-e67b-42ea-ad73-3403c40f0bcc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:11.767943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fb772615-e67b-42ea-ad73-3403c40f0bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:11.792121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec32563-e22a-4a99-8941-6416cbaffdc8 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:11.833449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf98a6-88dc-4051-a6d9-93acefe54545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:11.834225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf98a6-88dc-4051-a6d9-93acefe54545 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:11.834807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf98a6-88dc-4051-a6d9-93acefe54545 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:11.835133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf98a6-88dc-4051-a6d9-93acefe54545 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:11.844989 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 529/1041] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:40:11.909966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf98a6-88dc-4051-a6d9-93acefe54545 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:11.910312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf98a6-88dc-4051-a6d9-93acefe54545 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:11.917718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b18a57-0862-4e78-8c02-b9e3bbe6d07d None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:11.917993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b18a57-0862-4e78-8c02-b9e3bbe6d07d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:11.918253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b18a57-0862-4e78-8c02-b9e3bbe6d07d None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:11.920176 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 530/1042] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:40:11.926375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3712e37-19f1-4907-971f-7fe3bfb29391 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:11.926650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3712e37-19f1-4907-971f-7fe3bfb29391 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:11.926923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3712e37-19f1-4907-971f-7fe3bfb29391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:11.949050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b3712e37-19f1-4907-971f-7fe3bfb29391 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:11.950875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b3712e37-19f1-4907-971f-7fe3bfb29391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:11.959724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:12.033557 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 531/1043] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 20 20:40:12.049541 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 513/1044] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:40:11 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:40:12.055705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3712e37-19f1-4907-971f-7fe3bfb29391 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:12.084077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:12.161610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3712e37-19f1-4907-971f-7fe3bfb29391 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:12.195735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723eacb-8d77-41a5-be9c-2c9096095d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:12.196648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723eacb-8d77-41a5-be9c-2c9096095d8c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:12.196648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723eacb-8d77-41a5-be9c-2c9096095d8c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:12.196941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723eacb-8d77-41a5-be9c-2c9096095d8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:12.206914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723eacb-8d77-41a5-be9c-2c9096095d8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:12.208405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723eacb-8d77-41a5-be9c-2c9096095d8c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:12.215604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf98a6-88dc-4051-a6d9-93acefe54545 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-e723eacb-8d77-41a5-be9c-2c9096095d8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:40:12.224190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:12.225299 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:40:12.226198 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:40:12.226198 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:12.226198 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:12.228584 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 532/1045] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:12.232854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723eacb-8d77-41a5-be9c-2c9096095d8c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:12.313373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:12.327307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf98a6-88dc-4051-a6d9-93acefe54545 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:12.338381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723eacb-8d77-41a5-be9c-2c9096095d8c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:12.363597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc01851-c375-4aac-8cae-1b72365dd6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:12.364234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc01851-c375-4aac-8cae-1b72365dd6fc None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:12.364532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc01851-c375-4aac-8cae-1b72365dd6fc None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:12.364801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc01851-c375-4aac-8cae-1b72365dd6fc None admin] PATH_INFO: `/v3/services/image` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:12.365379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc01851-c375-4aac-8cae-1b72365dd6fc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:12.366744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc01851-c375-4aac-8cae-1b72365dd6fc None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:12.372398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc49ff-2d25-4697-afd3-322341e57400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:12.373041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc49ff-2d25-4697-afd3-322341e57400 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:12.373309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc49ff-2d25-4697-afd3-322341e57400 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:12.373559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc49ff-2d25-4697-afd3-322341e57400 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:12.374201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc49ff-2d25-4697-afd3-322341e57400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:12.374496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc49ff-2d25-4697-afd3-322341e57400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-7fc01851-c375-4aac-8cae-1b72365dd6fc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:12.375734 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:12.376917 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 20 20:40:12.377770 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:12.378558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc49ff-2d25-4697-afd3-322341e57400 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:12.378637 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 514/1046] 10.222.0.23 () {56 vars in 1105 bytes} [Sat Aug 20 20:40:12 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:40:12.378637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc01851-c375-4aac-8cae-1b72365dd6fc None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:12.382862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:12.392505 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 533/1047] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:12.393206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc49ff-2d25-4697-afd3-322341e57400 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:12.485514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc01851-c375-4aac-8cae-1b72365dd6fc None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:12.497263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc49ff-2d25-4697-afd3-322341e57400 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:12.521176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46f72d-5831-474e-946c-5573d2241b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:12.521798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46f72d-5831-474e-946c-5573d2241b0c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:12.522072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46f72d-5831-474e-946c-5573d2241b0c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:12.522341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46f72d-5831-474e-946c-5573d2241b0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:12.531241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d444e-371d-4a8d-a259-5d4d28fa1f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:12.531846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d444e-371d-4a8d-a259-5d4d28fa1f6d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:12.532116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d444e-371d-4a8d-a259-5d4d28fa1f6d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:12.532409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d444e-371d-4a8d-a259-5d4d28fa1f6d None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:12.532806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46f72d-5831-474e-946c-5573d2241b0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:40:12.533027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d444e-371d-4a8d-a259-5d4d28fa1f6d None admin] RBAC: Adding query filter params (name=image) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:12.533327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d444e-371d-4a8d-a259-5d4d28fa1f6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:12.534697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d444e-371d-4a8d-a259-5d4d28fa1f6d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:12.536422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-2c46f72d-5831-474e-946c-5573d2241b0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:40:12.537342 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:12.538519 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:12.539415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:12.539415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:12.539415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:12.539415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:12.539415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:40:12.539415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:40:12.539415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:40:12.539415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:40:12.539415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:40:12.539415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:40:12.539415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:40:12.539415 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:12.539415 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 515/1048] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:40:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:12.540168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46f72d-5831-474e-946c-5573d2241b0c None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:12.540961 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 534/1049] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:12.545005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d444e-371d-4a8d-a259-5d4d28fa1f6d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:12.601468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:12.638636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46f72d-5831-474e-946c-5573d2241b0c None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:12.643140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d444e-371d-4a8d-a259-5d4d28fa1f6d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:12.672338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf32d5-33df-4843-8e26-75a213eb3efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:12.672982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf32d5-33df-4843-8e26-75a213eb3efb None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:12.673251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf32d5-33df-4843-8e26-75a213eb3efb None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:12.673508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf32d5-33df-4843-8e26-75a213eb3efb None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:12.677287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e1ef2-6134-4a2c-8e56-389fcb272d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:12.677887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e1ef2-6134-4a2c-8e56-389fcb272d30 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:12.678158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e1ef2-6134-4a2c-8e56-389fcb272d30 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:12.678421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e1ef2-6134-4a2c-8e56-389fcb272d30 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:12.679073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e1ef2-6134-4a2c-8e56-389fcb272d30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:12.679371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e1ef2-6134-4a2c-8e56-389fcb272d30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:12.680267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf32d5-33df-4843-8e26-75a213eb3efb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:12.681284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e1ef2-6134-4a2c-8e56-389fcb272d30 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:12.681781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf32d5-33df-4843-8e26-75a213eb3efb None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:12.690146 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 535/1050] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:40:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:12.690933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e1ef2-6134-4a2c-8e56-389fcb272d30 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9aaf32d5-33df-4843-8e26-75a213eb3efb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:40:12.693207 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:12.694067 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:12.694979 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:40:12.694979 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:40:12.694979 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:40:12.694979 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:40:12.694979 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:40:12.694979 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:40:12.694979 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:40:12.694979 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:12.694979 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 516/1051] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:40:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:12.696149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf32d5-33df-4843-8e26-75a213eb3efb None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:12.758075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:12.798144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e1ef2-6134-4a2c-8e56-389fcb272d30 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:12.804160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf32d5-33df-4843-8e26-75a213eb3efb None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:12.829340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:12.832172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea736d-312b-4db8-92d2-102181b81995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:12.833016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea736d-312b-4db8-92d2-102181b81995 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:12.833301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea736d-312b-4db8-92d2-102181b81995 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:12.833592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea736d-312b-4db8-92d2-102181b81995 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:12.834255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea736d-312b-4db8-92d2-102181b81995 None admin] RBAC: Adding query filter params (type=image) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:12.834576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea736d-312b-4db8-92d2-102181b81995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:12.836016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea736d-312b-4db8-92d2-102181b81995 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:12.841157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200fcf0-ecfa-4de0-9199-3041c78d8e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:12.841788 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 536/1052] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:12.841925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200fcf0-ecfa-4de0-9199-3041c78d8e7b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:12.842073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200fcf0-ecfa-4de0-9199-3041c78d8e7b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:12.842369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200fcf0-ecfa-4de0-9199-3041c78d8e7b None admin] PATH_INFO: `/v3/users/962a3d33783d4f7caae4fb00802375fb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:12.842507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea736d-312b-4db8-92d2-102181b81995 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:12.857731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200fcf0-ecfa-4de0-9199-3041c78d8e7b None admin] RBAC: Authorizing `identity:get_user(user_id=962a3d33783d4f7caae4fb00802375fb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:12.859053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200fcf0-ecfa-4de0-9199-3041c78d8e7b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:12.860815 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 517/1053] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:40:12 2022] GET /identity/v3/users/962a3d33783d4f7caae4fb00802375fb => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:12.861606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200fcf0-ecfa-4de0-9199-3041c78d8e7b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:12.947069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea736d-312b-4db8-92d2-102181b81995 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:12.959460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200fcf0-ecfa-4de0-9199-3041c78d8e7b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:12.971944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:12.981305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e423adc-6898-4269-a194-4e26d2e5b4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:12.981898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e423adc-6898-4269-a194-4e26d2e5b4e2 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:12.982173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e423adc-6898-4269-a194-4e26d2e5b4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:12.982431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e423adc-6898-4269-a194-4e26d2e5b4e2 None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:12.983114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e423adc-6898-4269-a194-4e26d2e5b4e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:12.983415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e423adc-6898-4269-a194-4e26d2e5b4e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:12.984879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e423adc-6898-4269-a194-4e26d2e5b4e2 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:12.993635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526c96-f23e-4f68-ba29-7c77b07a9d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:12.994238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526c96-f23e-4f68-ba29-7c77b07a9d1b None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:12.994503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526c96-f23e-4f68-ba29-7c77b07a9d1b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:12.994769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526c96-f23e-4f68-ba29-7c77b07a9d1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:12.995397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526c96-f23e-4f68-ba29-7c77b07a9d1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:12.996743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526c96-f23e-4f68-ba29-7c77b07a9d1b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:12.997592 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 537/1054] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:40:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:12.998391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e423adc-6898-4269-a194-4e26d2e5b4e2 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:13.024435 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 518/1055] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:40:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:40:13.024435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526c96-f23e-4f68-ba29-7c77b07a9d1b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:13.070035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:13.131298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e423adc-6898-4269-a194-4e26d2e5b4e2 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:13.151420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526c96-f23e-4f68-ba29-7c77b07a9d1b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:13.167587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43e4b2-c024-444d-893d-549c04fd7322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:13.168203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43e4b2-c024-444d-893d-549c04fd7322 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:13.168512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43e4b2-c024-444d-893d-549c04fd7322 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:13.168893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43e4b2-c024-444d-893d-549c04fd7322 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:13.191313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee7cf2-6c2d-440d-90d4-5b77eacdae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:13.192392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee7cf2-6c2d-440d-90d4-5b77eacdae59 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:13.192896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee7cf2-6c2d-440d-90d4-5b77eacdae59 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:13.193337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee7cf2-6c2d-440d-90d4-5b77eacdae59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:13.194226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee7cf2-6c2d-440d-90d4-5b77eacdae59 None admin] RBAC: Adding query filter params (role.id=d78b7ae7d2c541498b055d1e2822e9b1, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=df852af59e7743f7a8798d9edf8ebd3e) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:13.194769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee7cf2-6c2d-440d-90d4-5b77eacdae59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:13.196948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee7cf2-6c2d-440d-90d4-5b77eacdae59 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:13.204772 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 519/1056] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:40:13 2022] GET /identity/v3/role_assignments?user.id=df852af59e7743f7a8798d9edf8ebd3e&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=d78b7ae7d2c541498b055d1e2822e9b1 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:13.240801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:13.258532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d43e4b2-c024-444d-893d-549c04fd7322 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:13.260002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d43e4b2-c024-444d-893d-549c04fd7322 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:13.377830 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 538/1057] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:40:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:40:13.384897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee7cf2-6c2d-440d-90d4-5b77eacdae59 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:13.492978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:13.497107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee7cf2-6c2d-440d-90d4-5b77eacdae59 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:13.531534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0621ea-a5f6-449f-8dcd-1d1219b63689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:13.532156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0621ea-a5f6-449f-8dcd-1d1219b63689 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:13.532449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0621ea-a5f6-449f-8dcd-1d1219b63689 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:13.532718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0621ea-a5f6-449f-8dcd-1d1219b63689 None admin] PATH_INFO: `/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:13.533838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0621ea-a5f6-449f-8dcd-1d1219b63689 None admin] RBAC: Authorizing `identity:get_project(project_id=f71d9e3993b94aed8bdbeeb9d55811f5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:13.535267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0621ea-a5f6-449f-8dcd-1d1219b63689 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:13.536940 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 520/1058] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:40:13 2022] GET /identity/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:13.543268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43e4b2-c024-444d-893d-549c04fd7322 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:13.632076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:13.657966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43e4b2-c024-444d-893d-549c04fd7322 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:13.693193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5ceae-4846-425f-ab70-691e317f2bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:13.693798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5ceae-4846-425f-ab70-691e317f2bac None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:13.694068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5ceae-4846-425f-ab70-691e317f2bac None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:13.694327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5ceae-4846-425f-ab70-691e317f2bac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:13.695042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5ceae-4846-425f-ab70-691e317f2bac None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=f71d9e3993b94aed8bdbeeb9d55811f5, user.id=962a3d33783d4f7caae4fb00802375fb) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:13.695346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5ceae-4846-425f-ab70-691e317f2bac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:13.696964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5ceae-4846-425f-ab70-691e317f2bac None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:13.703083 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 539/1059] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:40:13 2022] GET /identity/v3/role_assignments?user.id=962a3d33783d4f7caae4fb00802375fb&scope.project.id=f71d9e3993b94aed8bdbeeb9d55811f5&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:15.276159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708215b6-29b5-4b74-ae63-5482722dd4d8 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:15.276766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708215b6-29b5-4b74-ae63-5482722dd4d8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:15.276766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708215b6-29b5-4b74-ae63-5482722dd4d8 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:15.278339 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 521/1060] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:40:15.284196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f5e96-c35e-47ff-ba84-24a030bcbc77 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:15.284808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f5e96-c35e-47ff-ba84-24a030bcbc77 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:15.284808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f5e96-c35e-47ff-ba84-24a030bcbc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:15.303659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-de4f5e96-c35e-47ff-ba84-24a030bcbc77 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:15.304899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-de4f5e96-c35e-47ff-ba84-24a030bcbc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:15.372073 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 540/1061] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:15.442786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9fb6b-d4c0-41aa-97b5-836b2fb1f61e None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:15.443025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9fb6b-d4c0-41aa-97b5-836b2fb1f61e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:15.443267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9fb6b-d4c0-41aa-97b5-836b2fb1f61e None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:15.444824 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 522/1062] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:15.449938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ecf76-33e4-44d0-8dd7-8367e2a7b715 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:15.450179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ecf76-33e4-44d0-8dd7-8367e2a7b715 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:15.450476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ecf76-33e4-44d0-8dd7-8367e2a7b715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:15.475276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-712ecf76-33e4-44d0-8dd7-8367e2a7b715 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:15.476548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-712ecf76-33e4-44d0-8dd7-8367e2a7b715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:15.548705 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 541/1063] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:15.696979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9fb6b-d4c0-41aa-97b5-836b2fb1f61e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:15.813752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9fb6b-d4c0-41aa-97b5-836b2fb1f61e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:15.850437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e001b-a4ea-41e2-ac29-9552d1f58f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:15.851027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e001b-a4ea-41e2-ac29-9552d1f58f59 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:15.851295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e001b-a4ea-41e2-ac29-9552d1f58f59 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:15.851554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e001b-a4ea-41e2-ac29-9552d1f58f59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:15.877580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ecf76-33e4-44d0-8dd7-8367e2a7b715 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:15.889511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:15.890330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:15.926379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e001b-a4ea-41e2-ac29-9552d1f58f59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:15.927844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e001b-a4ea-41e2-ac29-9552d1f58f59 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:15.983203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ecf76-33e4-44d0-8dd7-8367e2a7b715 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:16.015957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2520096-d83c-4a02-aae2-e46ab272f7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:16.016602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2520096-d83c-4a02-aae2-e46ab272f7d0 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:16.016874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2520096-d83c-4a02-aae2-e46ab272f7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:16.017138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2520096-d83c-4a02-aae2-e46ab272f7d0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:16.026939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2520096-d83c-4a02-aae2-e46ab272f7d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:40:16.027099 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 523/1064] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:40:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:40:16.028232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542163a-c50e-4dac-9c3e-5eaf41c766df None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:16.028541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542163a-c50e-4dac-9c3e-5eaf41c766df None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:16.028850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542163a-c50e-4dac-9c3e-5eaf41c766df None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:16.030213 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 524/1065] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-b2520096-d83c-4a02-aae2-e46ab272f7d0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:40:16.031906 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:16.032952 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:16.033896 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:16.033896 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:16.033896 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:16.033896 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:16.033896 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:40:16.033896 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:40:16.033896 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:40:16.033896 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:40:16.033896 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:40:16.033896 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:40:16.033896 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 20 20:40:16.033896 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:16.035599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542163a-c50e-4dac-9c3e-5eaf41c766df None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:16.036511 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 542/1066] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Aug 20 20:40:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:16.038274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cb594-9e92-4afd-aebd-42679578a092 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:16.038699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cb594-9e92-4afd-aebd-42679578a092 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:16.038964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cb594-9e92-4afd-aebd-42679578a092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:16.056915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-028cb594-9e92-4afd-aebd-42679578a092 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:16.058041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-028cb594-9e92-4afd-aebd-42679578a092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:16.101052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:16.120317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:16.132694 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 543/1067] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:16.133626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cb594-9e92-4afd-aebd-42679578a092 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:16.138700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542163a-c50e-4dac-9c3e-5eaf41c766df None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:16.181507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25864c-68d1-4c11-bdf6-11958ebf1e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:16.182216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25864c-68d1-4c11-bdf6-11958ebf1e3b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:16.187138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25864c-68d1-4c11-bdf6-11958ebf1e3b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:16.188998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25864c-68d1-4c11-bdf6-11958ebf1e3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:16.203864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25864c-68d1-4c11-bdf6-11958ebf1e3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:16.205343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25864c-68d1-4c11-bdf6-11958ebf1e3b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9f25864c-68d1-4c11-bdf6-11958ebf1e3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:40:16.221020 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:16.222057 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:40:16.222995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:40:16.222995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:16.222995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:16.222995 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 525/1068] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:16.229193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25864c-68d1-4c11-bdf6-11958ebf1e3b None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:16.246502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cb594-9e92-4afd-aebd-42679578a092 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:16.280983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23f672-a4da-41f8-ac97-750045199e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:16.281585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23f672-a4da-41f8-ac97-750045199e90 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:16.281851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23f672-a4da-41f8-ac97-750045199e90 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:16.282107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23f672-a4da-41f8-ac97-750045199e90 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:16.282748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23f672-a4da-41f8-ac97-750045199e90 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:16.283032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23f672-a4da-41f8-ac97-750045199e90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:16.284398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23f672-a4da-41f8-ac97-750045199e90 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:16.296422 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 544/1069] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Aug 20 20:40:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:16.298457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23f672-a4da-41f8-ac97-750045199e90 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:16.335500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25864c-68d1-4c11-bdf6-11958ebf1e3b None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:16.352224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:16.363623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:16.372478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b434a-6320-4bb1-9299-c37a18d6ecc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:16.373053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b434a-6320-4bb1-9299-c37a18d6ecc1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:16.373309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b434a-6320-4bb1-9299-c37a18d6ecc1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:16.373565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b434a-6320-4bb1-9299-c37a18d6ecc1 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:16.374193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b434a-6320-4bb1-9299-c37a18d6ecc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:16.374481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b434a-6320-4bb1-9299-c37a18d6ecc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:16.375829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b434a-6320-4bb1-9299-c37a18d6ecc1 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:16.388413 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 526/1070] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:16.408290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23f672-a4da-41f8-ac97-750045199e90 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:16.448184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0041557-10c2-4878-94f0-4ec03399859e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:16.449182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0041557-10c2-4878-94f0-4ec03399859e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:16.449610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0041557-10c2-4878-94f0-4ec03399859e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:16.450028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0041557-10c2-4878-94f0-4ec03399859e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:16.462230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0041557-10c2-4878-94f0-4ec03399859e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:16.463976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0041557-10c2-4878-94f0-4ec03399859e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:16.465133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b434a-6320-4bb1-9299-c37a18d6ecc1 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-f0041557-10c2-4878-94f0-4ec03399859e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:40:16.482236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:16.483965 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:40:16.485138 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:40:16.485138 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:16.485138 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:16.492081 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 545/1071] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:40:16.493412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0041557-10c2-4878-94f0-4ec03399859e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:16.590491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b434a-6320-4bb1-9299-c37a18d6ecc1 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:16.610477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0041557-10c2-4878-94f0-4ec03399859e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:16.624895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:16.629997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd56fb-bc3d-4941-acb0-1cf7e935a26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:16.630632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd56fb-bc3d-4941-acb0-1cf7e935a26c None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:16.630904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd56fb-bc3d-4941-acb0-1cf7e935a26c None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:16.631169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd56fb-bc3d-4941-acb0-1cf7e935a26c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:16.644004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd56fb-bc3d-4941-acb0-1cf7e935a26c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:40:16.648327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a676e6-9ee0-4e65-930e-270c4eab43f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:16.649299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a676e6-9ee0-4e65-930e-270c4eab43f3 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-f8bd56fb-bc3d-4941-acb0-1cf7e935a26c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:40:16.649547 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:16.650469 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:16.651329 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:16.651329 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:16.651329 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:16.651329 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:16.651329 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:40:16.651329 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:40:16.651329 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:40:16.651329 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:40:16.651329 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:40:16.651329 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:40:16.651329 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:40:16.651329 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:16.651979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a676e6-9ee0-4e65-930e-270c4eab43f3 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:16.651979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a676e6-9ee0-4e65-930e-270c4eab43f3 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:16.651979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a676e6-9ee0-4e65-930e-270c4eab43f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:16.651979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a676e6-9ee0-4e65-930e-270c4eab43f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:16.652244 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 527/1072] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:40:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:40:16.652319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a676e6-9ee0-4e65-930e-270c4eab43f3 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:16.656874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd56fb-bc3d-4941-acb0-1cf7e935a26c None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:16.667307 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 546/1073] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:16.674656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a676e6-9ee0-4e65-930e-270c4eab43f3 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:16.705151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:16.771527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd56fb-bc3d-4941-acb0-1cf7e935a26c None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:16.781925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a676e6-9ee0-4e65-930e-270c4eab43f3 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:16.808318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe479f4-155b-4d41-9705-7499bbaef803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:16.812840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe479f4-155b-4d41-9705-7499bbaef803 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:16.812840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe479f4-155b-4d41-9705-7499bbaef803 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:16.812840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe479f4-155b-4d41-9705-7499bbaef803 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:16.812840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe479f4-155b-4d41-9705-7499bbaef803 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:16.812840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe479f4-155b-4d41-9705-7499bbaef803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:16.812840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe479f4-155b-4d41-9705-7499bbaef803 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:16.816212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cb696-633f-4b9e-8849-d06e46c9b7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:16.816874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cb696-633f-4b9e-8849-d06e46c9b7a3 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:16.817144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cb696-633f-4b9e-8849-d06e46c9b7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:16.817408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cb696-633f-4b9e-8849-d06e46c9b7a3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:16.822065 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 528/1074] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:40:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:16.825080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36cb696-633f-4b9e-8849-d06e46c9b7a3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:16.826514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36cb696-633f-4b9e-8849-d06e46c9b7a3 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:16.828431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe479f4-155b-4d41-9705-7499bbaef803 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-f36cb696-633f-4b9e-8849-d06e46c9b7a3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:40:16.839098 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:16.840109 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:16.841125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:16.841125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:16.841125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:40:16.841125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:40:16.841125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:40:16.841125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:40:16.841125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:40:16.841125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:40:16.841125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:40:16.841125 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:16.845096 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 547/1075] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Aug 20 20:40:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:40:16.845861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cb696-633f-4b9e-8849-d06e46c9b7a3 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:16.880974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:16.930770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:16.934134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe479f4-155b-4d41-9705-7499bbaef803 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:16.946847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cb696-633f-4b9e-8849-d06e46c9b7a3 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:16.967648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3b810-0265-42f9-a4cc-b61263a3fa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:16.968278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3b810-0265-42f9-a4cc-b61263a3fa89 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:16.968593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3b810-0265-42f9-a4cc-b61263a3fa89 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:16.968859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3b810-0265-42f9-a4cc-b61263a3fa89 None admin] PATH_INFO: `/v3/users/962a3d33783d4f7caae4fb00802375fb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:16.969981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3b810-0265-42f9-a4cc-b61263a3fa89 None admin] RBAC: Authorizing `identity:get_user(user_id=962a3d33783d4f7caae4fb00802375fb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:16.971408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3b810-0265-42f9-a4cc-b61263a3fa89 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:16.973036 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 529/1076] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:40:16 2022] GET /identity/v3/users/962a3d33783d4f7caae4fb00802375fb => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:16.978906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3b810-0265-42f9-a4cc-b61263a3fa89 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:16.979873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2975693-f940-42d9-956b-b7796db96ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:16.980521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975693-f940-42d9-956b-b7796db96ef8 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:16.980793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975693-f940-42d9-956b-b7796db96ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:16.981058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975693-f940-42d9-956b-b7796db96ef8 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:16.981775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2975693-f940-42d9-956b-b7796db96ef8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:16.982072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2975693-f940-42d9-956b-b7796db96ef8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:16.983522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2975693-f940-42d9-956b-b7796db96ef8 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:17.000622 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 548/1077] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Aug 20 20:40:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:17.006352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2975693-f940-42d9-956b-b7796db96ef8 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:17.077277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3b810-0265-42f9-a4cc-b61263a3fa89 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:17.096351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:17.106680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2975693-f940-42d9-956b-b7796db96ef8 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:17.115977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6bd71-629b-452b-ad02-5deaf6cb0590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:17.116718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6bd71-629b-452b-ad02-5deaf6cb0590 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:17.117098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6bd71-629b-452b-ad02-5deaf6cb0590 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:17.117933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6bd71-629b-452b-ad02-5deaf6cb0590 None admin] PATH_INFO: `/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:17.119508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6bd71-629b-452b-ad02-5deaf6cb0590 None admin] RBAC: Authorizing `identity:get_project(project_id=f71d9e3993b94aed8bdbeeb9d55811f5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:17.121293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6bd71-629b-452b-ad02-5deaf6cb0590 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:17.123550 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 530/1078] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:40:16 2022] GET /identity/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:17.129725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6bd71-629b-452b-ad02-5deaf6cb0590 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:17.153343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ae52b-dc2e-4f59-895d-350dbd024413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:17.153907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ae52b-dc2e-4f59-895d-350dbd024413 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:17.154166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ae52b-dc2e-4f59-895d-350dbd024413 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:17.154427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ae52b-dc2e-4f59-895d-350dbd024413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:17.164763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ae52b-dc2e-4f59-895d-350dbd024413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:17.166239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ae52b-dc2e-4f59-895d-350dbd024413 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:17.167441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-c60ae52b-dc2e-4f59-895d-350dbd024413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:40:17.181974 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:17.183102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:40:17.184398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:40:17.184398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:17.184398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:17.184398 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 549/1079] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:40:17.189229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ae52b-dc2e-4f59-895d-350dbd024413 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:17.244967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6bd71-629b-452b-ad02-5deaf6cb0590 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:17.279859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053fd6e-811c-46a3-aae7-903d46309da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:17.280537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053fd6e-811c-46a3-aae7-903d46309da4 None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:17.280805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053fd6e-811c-46a3-aae7-903d46309da4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:17.281059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053fd6e-811c-46a3-aae7-903d46309da4 None admin] PATH_INFO: `/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5/users/962a3d33783d4f7caae4fb00802375fb/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:17.282958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053fd6e-811c-46a3-aae7-903d46309da4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f71d9e3993b94aed8bdbeeb9d55811f5, user_id=962a3d33783d4f7caae4fb00802375fb, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:17.284389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053fd6e-811c-46a3-aae7-903d46309da4 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:17.290114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ae52b-dc2e-4f59-895d-350dbd024413 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:17.297903 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 531/1080] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:40:17 2022] PUT /identity/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5/users/962a3d33783d4f7caae4fb00802375fb/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:40:17.326937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278f42e-619c-46af-84c2-300b9c9011a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:17.327561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278f42e-619c-46af-84c2-300b9c9011a4 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:17.327832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278f42e-619c-46af-84c2-300b9c9011a4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:17.328101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278f42e-619c-46af-84c2-300b9c9011a4 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:17.328791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278f42e-619c-46af-84c2-300b9c9011a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:17.329089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278f42e-619c-46af-84c2-300b9c9011a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:17.330540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278f42e-619c-46af-84c2-300b9c9011a4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:17.343984 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 550/1081] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:17.350007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053fd6e-811c-46a3-aae7-903d46309da4 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:17.449741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:17.457554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053fd6e-811c-46a3-aae7-903d46309da4 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:17.496136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b23f1-5bee-426d-80d9-806d37d95370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:17.496813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b23f1-5bee-426d-80d9-806d37d95370 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:17.497084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b23f1-5bee-426d-80d9-806d37d95370 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:17.497348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b23f1-5bee-426d-80d9-806d37d95370 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:17.504877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b23f1-5bee-426d-80d9-806d37d95370 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:17.506334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b23f1-5bee-426d-80d9-806d37d95370 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-af8b23f1-5bee-426d-80d9-806d37d95370 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:40:17.517934 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:17.518904 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:17.519789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:40:17.519789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:40:17.519789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:40:17.519789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:40:17.519789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:40:17.519789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:40:17.519789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:40:17.519789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:17.519789 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 532/1082] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:40:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:17.525187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278f42e-619c-46af-84c2-300b9c9011a4 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:17.529231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:17.633084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278f42e-619c-46af-84c2-300b9c9011a4 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:17.670562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd227e1-f619-4ecc-91ef-cfc665da75aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:17.671200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd227e1-f619-4ecc-91ef-cfc665da75aa None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:17.671468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd227e1-f619-4ecc-91ef-cfc665da75aa None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:17.671728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd227e1-f619-4ecc-91ef-cfc665da75aa None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:17.672404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd227e1-f619-4ecc-91ef-cfc665da75aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:17.672732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd227e1-f619-4ecc-91ef-cfc665da75aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:17.674150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd227e1-f619-4ecc-91ef-cfc665da75aa None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:17.686894 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 551/1083] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:40:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:17.693563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b23f1-5bee-426d-80d9-806d37d95370 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:17.799682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b23f1-5bee-426d-80d9-806d37d95370 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:17.836869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70479d1b-d140-4c39-ac3e-3a20b3a1c2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:17.837533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70479d1b-d140-4c39-ac3e-3a20b3a1c2a3 None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:17.837799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70479d1b-d140-4c39-ac3e-3a20b3a1c2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:17.838064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70479d1b-d140-4c39-ac3e-3a20b3a1c2a3 None admin] PATH_INFO: `/v3/projects/3b6c88840d764c2385aa678da4338213/users/df852af59e7743f7a8798d9edf8ebd3e/roles/d78b7ae7d2c541498b055d1e2822e9b1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:17.840336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70479d1b-d140-4c39-ac3e-3a20b3a1c2a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b6c88840d764c2385aa678da4338213, user_id=df852af59e7743f7a8798d9edf8ebd3e, role_id=d78b7ae7d2c541498b055d1e2822e9b1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:17.841798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70479d1b-d140-4c39-ac3e-3a20b3a1c2a3 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:17.855888 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 533/1084] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:40:17 2022] PUT /identity/v3/projects/3b6c88840d764c2385aa678da4338213/users/df852af59e7743f7a8798d9edf8ebd3e/roles/d78b7ae7d2c541498b055d1e2822e9b1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:40:19.483371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23850a89-906c-43f0-bf61-1e3d084f13da None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:19.483887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23850a89-906c-43f0-bf61-1e3d084f13da None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:19.483887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23850a89-906c-43f0-bf61-1e3d084f13da None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:19.485170 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 552/1085] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:19.490601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6aac9f-13a1-45fc-9c4f-b92feb84cfdd None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:19.491071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6aac9f-13a1-45fc-9c4f-b92feb84cfdd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:19.491234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6aac9f-13a1-45fc-9c4f-b92feb84cfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:19.513264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ca6aac9f-13a1-45fc-9c4f-b92feb84cfdd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:19.515208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ca6aac9f-13a1-45fc-9c4f-b92feb84cfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:19.528050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:19.591485 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 534/1086] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:19.917710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23850a89-906c-43f0-bf61-1e3d084f13da None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:19.981604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:20.020794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23850a89-906c-43f0-bf61-1e3d084f13da None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:20.048224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78d07c-6192-4831-abe4-19244e11778c None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:20.048482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78d07c-6192-4831-abe4-19244e11778c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:20.048780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78d07c-6192-4831-abe4-19244e11778c None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:20.050099 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 535/1087] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:20.055381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e2428-b0bd-471f-ab65-8a60f1c49fde None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:20.055538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4f6d7-b3df-4002-81f9-b73557a10c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:20.055648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e2428-b0bd-471f-ab65-8a60f1c49fde None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:20.055904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e2428-b0bd-471f-ab65-8a60f1c49fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:20.056330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4f6d7-b3df-4002-81f9-b73557a10c9d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:20.056715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4f6d7-b3df-4002-81f9-b73557a10c9d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:20.057046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4f6d7-b3df-4002-81f9-b73557a10c9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:20.068048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4f6d7-b3df-4002-81f9-b73557a10c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-0fd4f6d7-b3df-4002-81f9-b73557a10c9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:40:20.073041 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:20.074023 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:20.074858 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:20.074858 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:20.074858 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:20.074858 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:20.074858 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:40:20.074858 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:40:20.074858 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:40:20.074858 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:40:20.074858 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:40:20.074858 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:40:20.074858 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:40:20.074858 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:20.074858 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 553/1088] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:40:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:20.075549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6d9e2428-b0bd-471f-ab65-8a60f1c49fde None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:20.075802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6d9e2428-b0bd-471f-ab65-8a60f1c49fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:20.080231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4f6d7-b3df-4002-81f9-b73557a10c9d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:20.141646 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 536/1089] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:20.178143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4f6d7-b3df-4002-81f9-b73557a10c9d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:20.202201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:20.213426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7ff62-ad5d-4b50-97f3-85aab74628fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:20.214033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7ff62-ad5d-4b50-97f3-85aab74628fd None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:20.214301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7ff62-ad5d-4b50-97f3-85aab74628fd None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:20.214567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7ff62-ad5d-4b50-97f3-85aab74628fd None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:20.215220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7ff62-ad5d-4b50-97f3-85aab74628fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:20.215513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7ff62-ad5d-4b50-97f3-85aab74628fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:20.216902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7ff62-ad5d-4b50-97f3-85aab74628fd None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:20.226243 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 554/1090] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:40:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:20.232590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e2428-b0bd-471f-ab65-8a60f1c49fde None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:20.306559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:20.331012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e2428-b0bd-471f-ab65-8a60f1c49fde None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:20.365744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e7044-1ba1-4c54-a198-adee160d1ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:20.366365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e7044-1ba1-4c54-a198-adee160d1ac5 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:20.366635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e7044-1ba1-4c54-a198-adee160d1ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:20.366897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e7044-1ba1-4c54-a198-adee160d1ac5 None admin] PATH_INFO: `/v3/users/962a3d33783d4f7caae4fb00802375fb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:20.368072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e7044-1ba1-4c54-a198-adee160d1ac5 None admin] RBAC: Authorizing `identity:get_user(user_id=962a3d33783d4f7caae4fb00802375fb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:20.369531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e7044-1ba1-4c54-a198-adee160d1ac5 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:20.371108 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 537/1091] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:40:20 2022] GET /identity/v3/users/962a3d33783d4f7caae4fb00802375fb => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:20.376978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7ff62-ad5d-4b50-97f3-85aab74628fd None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:20.474726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e7044-1ba1-4c54-a198-adee160d1ac5 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:20.485448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7ff62-ad5d-4b50-97f3-85aab74628fd None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:20.520982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fb5d3-aac7-44de-a325-7b2d8a90df54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:20.521581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fb5d3-aac7-44de-a325-7b2d8a90df54 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:20.521851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fb5d3-aac7-44de-a325-7b2d8a90df54 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:20.522110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fb5d3-aac7-44de-a325-7b2d8a90df54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:20.575555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:20.575789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e7044-1ba1-4c54-a198-adee160d1ac5 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:20.588650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723fb5d3-aac7-44de-a325-7b2d8a90df54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:20.590090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723fb5d3-aac7-44de-a325-7b2d8a90df54 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:20.609919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3070924-952a-465d-8a5f-8c281d441c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:20.610527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3070924-952a-465d-8a5f-8c281d441c15 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:20.610794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3070924-952a-465d-8a5f-8c281d441c15 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:20.611052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3070924-952a-465d-8a5f-8c281d441c15 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:20.620827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3070924-952a-465d-8a5f-8c281d441c15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-f3070924-952a-465d-8a5f-8c281d441c15 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:40:20.625397 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:20.626343 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:20.627193 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:20.627193 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:20.627193 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:20.627193 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:20.627193 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:40:20.627193 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:40:20.627193 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:40:20.627193 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:40:20.627193 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:40:20.627193 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:40:20.627193 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 20 20:40:20.627193 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:20.627193 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 538/1092] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Aug 20 20:40:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:20.632729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3070924-952a-465d-8a5f-8c281d441c15 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:20.637084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:20.688144 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 555/1093] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:40:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 20 20:40:20.693927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fb5d3-aac7-44de-a325-7b2d8a90df54 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:20.735549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3070924-952a-465d-8a5f-8c281d441c15 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:20.769645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750154a-d633-4851-a941-04f67db92719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:20.770244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750154a-d633-4851-a941-04f67db92719 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:20.770516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750154a-d633-4851-a941-04f67db92719 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:20.770779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750154a-d633-4851-a941-04f67db92719 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:20.771434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750154a-d633-4851-a941-04f67db92719 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:20.771729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750154a-d633-4851-a941-04f67db92719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:20.773128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750154a-d633-4851-a941-04f67db92719 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:20.775493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:20.782063 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 539/1094] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Aug 20 20:40:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:20.788179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750154a-d633-4851-a941-04f67db92719 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:20.790728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fb5d3-aac7-44de-a325-7b2d8a90df54 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:20.824729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c277e3c-1c22-4e1d-b921-678298b59cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:20.825347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c277e3c-1c22-4e1d-b921-678298b59cb7 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:20.825612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c277e3c-1c22-4e1d-b921-678298b59cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:20.825874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c277e3c-1c22-4e1d-b921-678298b59cb7 None admin] PATH_INFO: `/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:20.826959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c277e3c-1c22-4e1d-b921-678298b59cb7 None admin] RBAC: Authorizing `identity:get_project(project_id=f71d9e3993b94aed8bdbeeb9d55811f5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:20.828404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c277e3c-1c22-4e1d-b921-678298b59cb7 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:20.830027 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 556/1095] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:40:20 2022] GET /identity/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:20.836454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c277e3c-1c22-4e1d-b921-678298b59cb7 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:20.856393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:20.892247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750154a-d633-4851-a941-04f67db92719 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:20.924744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f493b5f-7cd8-4b5d-ab85-a369f7fbad3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:20.925366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f493b5f-7cd8-4b5d-ab85-a369f7fbad3e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:20.925634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f493b5f-7cd8-4b5d-ab85-a369f7fbad3e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:20.925931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f493b5f-7cd8-4b5d-ab85-a369f7fbad3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:20.934977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c277e3c-1c22-4e1d-b921-678298b59cb7 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:20.937278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f493b5f-7cd8-4b5d-ab85-a369f7fbad3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:20.938740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f493b5f-7cd8-4b5d-ab85-a369f7fbad3e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-5f493b5f-7cd8-4b5d-ab85-a369f7fbad3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:40:20.952665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:20.953682 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:40:20.954576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:40:20.954576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:20.954576 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:20.954576 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 540/1096] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:20.959649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f493b5f-7cd8-4b5d-ab85-a369f7fbad3e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:20.968557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b62b80-cbc3-42c5-8e7b-48320f25024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:20.969167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b62b80-cbc3-42c5-8e7b-48320f25024c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:20.969431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b62b80-cbc3-42c5-8e7b-48320f25024c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:20.969688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b62b80-cbc3-42c5-8e7b-48320f25024c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:20.970390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b62b80-cbc3-42c5-8e7b-48320f25024c None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=f71d9e3993b94aed8bdbeeb9d55811f5, user.id=962a3d33783d4f7caae4fb00802375fb) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:20.970689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b62b80-cbc3-42c5-8e7b-48320f25024c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:20.972097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b62b80-cbc3-42c5-8e7b-48320f25024c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:20.978084 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 557/1097] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:40:20 2022] GET /identity/v3/role_assignments?user.id=962a3d33783d4f7caae4fb00802375fb&scope.project.id=f71d9e3993b94aed8bdbeeb9d55811f5&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:21.068884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f493b5f-7cd8-4b5d-ab85-a369f7fbad3e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:21.088080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:21.109778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa680d5a-c412-4223-a5d1-dad0577e52df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:21.110398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa680d5a-c412-4223-a5d1-dad0577e52df None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:21.110669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa680d5a-c412-4223-a5d1-dad0577e52df None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:21.110972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa680d5a-c412-4223-a5d1-dad0577e52df None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:21.111618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa680d5a-c412-4223-a5d1-dad0577e52df None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:21.111914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa680d5a-c412-4223-a5d1-dad0577e52df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:21.113375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa680d5a-c412-4223-a5d1-dad0577e52df None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:21.126954 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 541/1098] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:40:21.133079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b62b80-cbc3-42c5-8e7b-48320f25024c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:21.154200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:21.241443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b62b80-cbc3-42c5-8e7b-48320f25024c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:21.274706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5655e-eb82-4ae1-9590-716c32cb49ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:21.275325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5655e-eb82-4ae1-9590-716c32cb49ca None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:21.275595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5655e-eb82-4ae1-9590-716c32cb49ca None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:21.275855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5655e-eb82-4ae1-9590-716c32cb49ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:21.283275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5655e-eb82-4ae1-9590-716c32cb49ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:21.284747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5655e-eb82-4ae1-9590-716c32cb49ca None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-3ad5655e-eb82-4ae1-9590-716c32cb49ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:40:21.295679 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:21.296704 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:21.297648 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:21.297648 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:21.297648 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:40:21.297648 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:40:21.297648 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:40:21.297648 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:40:21.297648 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:40:21.297648 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:40:21.297648 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 20 20:40:21.297648 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:21.297648 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 558/1099] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Aug 20 20:40:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:21.302821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa680d5a-c412-4223-a5d1-dad0577e52df None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:21.405737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa680d5a-c412-4223-a5d1-dad0577e52df None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:21.443279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7869a-e3ae-4748-b7ea-3ac26e10a7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:21.443894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7869a-e3ae-4748-b7ea-3ac26e10a7c0 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:21.444160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7869a-e3ae-4748-b7ea-3ac26e10a7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:21.444450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7869a-e3ae-4748-b7ea-3ac26e10a7c0 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:21.445167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7869a-e3ae-4748-b7ea-3ac26e10a7c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:21.445460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7869a-e3ae-4748-b7ea-3ac26e10a7c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:21.446859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7869a-e3ae-4748-b7ea-3ac26e10a7c0 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:21.464743 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 542/1100] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Aug 20 20:40:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:21.470594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5655e-eb82-4ae1-9590-716c32cb49ca None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:21.550963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:21.574023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5655e-eb82-4ae1-9590-716c32cb49ca None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:21.608037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f6417-e1bd-4ffc-a8ed-0eb13e8672ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:21.608652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f6417-e1bd-4ffc-a8ed-0eb13e8672ff None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:21.608916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f6417-e1bd-4ffc-a8ed-0eb13e8672ff None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:21.609176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f6417-e1bd-4ffc-a8ed-0eb13e8672ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:21.676723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f6417-e1bd-4ffc-a8ed-0eb13e8672ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:21.678070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f6417-e1bd-4ffc-a8ed-0eb13e8672ff None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:21.754093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:21.777292 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 559/1101] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:40:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:40:21.783357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7869a-e3ae-4748-b7ea-3ac26e10a7c0 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:21.819626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:21.885310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7869a-e3ae-4748-b7ea-3ac26e10a7c0 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:21.922072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa5262-36ca-44d6-b45d-0caa4742d829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:21.922691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa5262-36ca-44d6-b45d-0caa4742d829 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:21.922966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa5262-36ca-44d6-b45d-0caa4742d829 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:21.923307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa5262-36ca-44d6-b45d-0caa4742d829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:21.934529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa5262-36ca-44d6-b45d-0caa4742d829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:21.935908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa5262-36ca-44d6-b45d-0caa4742d829 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-c3aa5262-36ca-44d6-b45d-0caa4742d829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:40:21.951050 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:21.952046 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:40:21.952981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:40:21.952981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:40:21.952981 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:21.952981 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 543/1102] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:21.958210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f6417-e1bd-4ffc-a8ed-0eb13e8672ff None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:22.066112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f6417-e1bd-4ffc-a8ed-0eb13e8672ff None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:22.101662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6961f-0ce1-4b89-8e1e-c4c12e3404d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:22.102918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6961f-0ce1-4b89-8e1e-c4c12e3404d2 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:22.103199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6961f-0ce1-4b89-8e1e-c4c12e3404d2 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:22.103449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6961f-0ce1-4b89-8e1e-c4c12e3404d2 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:22.104104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6961f-0ce1-4b89-8e1e-c4c12e3404d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:22.104407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6961f-0ce1-4b89-8e1e-c4c12e3404d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:22.105845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6961f-0ce1-4b89-8e1e-c4c12e3404d2 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:22.119196 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 560/1103] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:22.124983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa5262-36ca-44d6-b45d-0caa4742d829 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:22.231329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:22.240192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa5262-36ca-44d6-b45d-0caa4742d829 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:22.274088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415db0a3-d595-4d75-8fd6-c4d307564050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:22.274702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415db0a3-d595-4d75-8fd6-c4d307564050 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:22.274975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415db0a3-d595-4d75-8fd6-c4d307564050 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:22.275233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415db0a3-d595-4d75-8fd6-c4d307564050 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:22.281878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415db0a3-d595-4d75-8fd6-c4d307564050 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:22.283305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415db0a3-d595-4d75-8fd6-c4d307564050 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-415db0a3-d595-4d75-8fd6-c4d307564050 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:40:22.294344 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:22.295261 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:22.296213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:40:22.296213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:40:22.296213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:40:22.296213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:40:22.296213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:40:22.296213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:40:22.296213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:40:22.296213 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:22.296213 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 544/1104] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:40:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:22.301611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6961f-0ce1-4b89-8e1e-c4c12e3404d2 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:22.342282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:22.403115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6961f-0ce1-4b89-8e1e-c4c12e3404d2 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:22.443330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ad016-5480-4883-a2cd-bd17992fa00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:22.444349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ad016-5480-4883-a2cd-bd17992fa00d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:22.444812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ad016-5480-4883-a2cd-bd17992fa00d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:22.445241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ad016-5480-4883-a2cd-bd17992fa00d None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:22.446325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ad016-5480-4883-a2cd-bd17992fa00d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:22.446781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ad016-5480-4883-a2cd-bd17992fa00d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:22.449005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ad016-5480-4883-a2cd-bd17992fa00d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:22.461986 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 561/1105] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:40:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:22.468328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415db0a3-d595-4d75-8fd6-c4d307564050 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:22.566484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415db0a3-d595-4d75-8fd6-c4d307564050 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:22.601273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7f5ba-c3e4-407e-9b88-7ba8b2539b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:22.601940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7f5ba-c3e4-407e-9b88-7ba8b2539b57 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:22.602207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7f5ba-c3e4-407e-9b88-7ba8b2539b57 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:22.602467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7f5ba-c3e4-407e-9b88-7ba8b2539b57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:22.603172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7f5ba-c3e4-407e-9b88-7ba8b2539b57 None admin] RBAC: Adding query filter params (role.id=d78b7ae7d2c541498b055d1e2822e9b1, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=df852af59e7743f7a8798d9edf8ebd3e) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:22.603467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7f5ba-c3e4-407e-9b88-7ba8b2539b57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:22.604907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7f5ba-c3e4-407e-9b88-7ba8b2539b57 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:22.610841 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 545/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:40:22 2022] GET /identity/v3/role_assignments?user.id=df852af59e7743f7a8798d9edf8ebd3e&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=d78b7ae7d2c541498b055d1e2822e9b1 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:23.206561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba5ab8-e131-44e1-aa9d-df0a1d4dea28 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:23.207110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba5ab8-e131-44e1-aa9d-df0a1d4dea28 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:23.207110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba5ab8-e131-44e1-aa9d-df0a1d4dea28 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:23.208486 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 562/1107] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:23.215012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6b163-5190-4c20-9647-d68d5413270d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:23.215462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6b163-5190-4c20-9647-d68d5413270d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:23.215551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6b163-5190-4c20-9647-d68d5413270d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:23.234676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-aad6b163-5190-4c20-9647-d68d5413270d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:23.235858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-aad6b163-5190-4c20-9647-d68d5413270d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:23.243939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:23.308171 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 546/1108] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:23.636939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba5ab8-e131-44e1-aa9d-df0a1d4dea28 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:23.744976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:23.751517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba5ab8-e131-44e1-aa9d-df0a1d4dea28 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:23.795673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b5fe1-4412-408d-95f5-21d5685e43ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:23.796282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b5fe1-4412-408d-95f5-21d5685e43ad None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:23.796583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b5fe1-4412-408d-95f5-21d5685e43ad None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:23.796845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b5fe1-4412-408d-95f5-21d5685e43ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:23.797987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b5fe1-4412-408d-95f5-21d5685e43ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:23.799413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b5fe1-4412-408d-95f5-21d5685e43ad None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:23.800990 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 563/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:23.809421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6b163-5190-4c20-9647-d68d5413270d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:23.920776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6b163-5190-4c20-9647-d68d5413270d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:23.955717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ec10f-0905-4d57-b2bc-3aacd7688c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:23.956307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ec10f-0905-4d57-b2bc-3aacd7688c70 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:23.956613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ec10f-0905-4d57-b2bc-3aacd7688c70 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:23.956882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ec10f-0905-4d57-b2bc-3aacd7688c70 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:23.957515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ec10f-0905-4d57-b2bc-3aacd7688c70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:23.958953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ec10f-0905-4d57-b2bc-3aacd7688c70 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:23.974904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:23.982630 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 547/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:40:23 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:40:26.190782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37639781-900b-40a0-8095-fd03408db092 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:26.191684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37639781-900b-40a0-8095-fd03408db092 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:26.191684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37639781-900b-40a0-8095-fd03408db092 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:26.193724 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 564/1111] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:26.201482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c5cc-e0bf-407c-bc0f-3700669a7c76 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:26.202214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c5cc-e0bf-407c-bc0f-3700669a7c76 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:26.202376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c5cc-e0bf-407c-bc0f-3700669a7c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:26.230556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0823c5cc-e0bf-407c-bc0f-3700669a7c76 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:26.232105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0823c5cc-e0bf-407c-bc0f-3700669a7c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:26.305510 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 548/1112] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:26.632259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37639781-900b-40a0-8095-fd03408db092 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:26.740145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37639781-900b-40a0-8095-fd03408db092 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:26.775394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f1879-7fa8-49cb-b665-95b94e18cd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:26.776002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f1879-7fa8-49cb-b665-95b94e18cd97 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:26.776271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f1879-7fa8-49cb-b665-95b94e18cd97 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:26.776569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f1879-7fa8-49cb-b665-95b94e18cd97 None admin] PATH_INFO: `/v3/roles/f8a2ea13798443c48750e9baa83c7461` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:26.778075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f1879-7fa8-49cb-b665-95b94e18cd97 None admin] RBAC: Authorizing `identity:get_role(role_id=f8a2ea13798443c48750e9baa83c7461)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:26.779443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f1879-7fa8-49cb-b665-95b94e18cd97 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:26.780802 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 565/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:40:26 2022] GET /identity/v3/roles/f8a2ea13798443c48750e9baa83c7461 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:26.787571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823c5cc-e0bf-407c-bc0f-3700669a7c76 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:26.892611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823c5cc-e0bf-407c-bc0f-3700669a7c76 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:26.919446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:26.933870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911028a9-89d8-4f94-9fb1-c2ce076e9ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:26.934781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911028a9-89d8-4f94-9fb1-c2ce076e9ceb None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:26.935167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911028a9-89d8-4f94-9fb1-c2ce076e9ceb None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:26.935545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911028a9-89d8-4f94-9fb1-c2ce076e9ceb None admin] PATH_INFO: `/v3/users/4419145e992445c6aa397b8f8353faff` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:26.953371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911028a9-89d8-4f94-9fb1-c2ce076e9ceb None admin] RBAC: Authorizing `identity:get_user(user_id=4419145e992445c6aa397b8f8353faff)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:26.954796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911028a9-89d8-4f94-9fb1-c2ce076e9ceb None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:26.956435 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 549/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:40:26 2022] GET /identity/v3/users/4419145e992445c6aa397b8f8353faff => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:26.962720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f1879-7fa8-49cb-b665-95b94e18cd97 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:26.998755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:27.071493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f1879-7fa8-49cb-b665-95b94e18cd97 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:27.107774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbabcb-4ea9-4e51-8d41-ee11d12e477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:27.108355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbabcb-4ea9-4e51-8d41-ee11d12e477c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:27.108655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbabcb-4ea9-4e51-8d41-ee11d12e477c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:27.108922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbabcb-4ea9-4e51-8d41-ee11d12e477c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:27.189162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbabcb-4ea9-4e51-8d41-ee11d12e477c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:27.190530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbabcb-4ea9-4e51-8d41-ee11d12e477c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:27.224399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:27.301050 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 566/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:40:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:40:27.309149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911028a9-89d8-4f94-9fb1-c2ce076e9ceb None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:27.425246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911028a9-89d8-4f94-9fb1-c2ce076e9ceb None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:27.468681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee691ce9-0576-430d-816f-1d2fc0f25dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:27.469256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee691ce9-0576-430d-816f-1d2fc0f25dcc None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:27.469529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee691ce9-0576-430d-816f-1d2fc0f25dcc None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:27.469802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee691ce9-0576-430d-816f-1d2fc0f25dcc None admin] PATH_INFO: `/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:27.471050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee691ce9-0576-430d-816f-1d2fc0f25dcc None admin] RBAC: Authorizing `identity:get_project(project_id=f71d9e3993b94aed8bdbeeb9d55811f5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:27.472533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee691ce9-0576-430d-816f-1d2fc0f25dcc None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:27.474199 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 550/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:40:27 2022] GET /identity/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:27.480684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbabcb-4ea9-4e51-8d41-ee11d12e477c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:27.592612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbabcb-4ea9-4e51-8d41-ee11d12e477c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:27.641446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ffae6-53b8-4b09-9ee9-240683dacdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:27.642046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ffae6-53b8-4b09-9ee9-240683dacdd4 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:27.642376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ffae6-53b8-4b09-9ee9-240683dacdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:27.642648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ffae6-53b8-4b09-9ee9-240683dacdd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:27.643354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ffae6-53b8-4b09-9ee9-240683dacdd4 None admin] RBAC: Adding query filter params (role.id=f8a2ea13798443c48750e9baa83c7461, scope.project.id=f71d9e3993b94aed8bdbeeb9d55811f5, user.id=4419145e992445c6aa397b8f8353faff) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:27.643649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ffae6-53b8-4b09-9ee9-240683dacdd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:27.645110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ffae6-53b8-4b09-9ee9-240683dacdd4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:27.646879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:27.652569 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 567/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:40:27 2022] GET /identity/v3/role_assignments?user.id=4419145e992445c6aa397b8f8353faff&scope.project.id=f71d9e3993b94aed8bdbeeb9d55811f5&role.id=f8a2ea13798443c48750e9baa83c7461 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:29.824244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728ce17-bc5b-4add-ac8f-0e9097053385 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:29.824856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728ce17-bc5b-4add-ac8f-0e9097053385 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:29.824856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728ce17-bc5b-4add-ac8f-0e9097053385 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:29.826131 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 551/1118] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:29.832354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bbfc3-a347-4aac-a52c-5a9f25218492 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:29.833071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bbfc3-a347-4aac-a52c-5a9f25218492 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:29.833211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bbfc3-a347-4aac-a52c-5a9f25218492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:29.859637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c38bbfc3-a347-4aac-a52c-5a9f25218492 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:29.860849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c38bbfc3-a347-4aac-a52c-5a9f25218492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:29.931744 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 568/1119] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:30.267638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728ce17-bc5b-4add-ac8f-0e9097053385 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:30.330297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:30.378930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728ce17-bc5b-4add-ac8f-0e9097053385 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:30.425246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef498a4-b968-45a5-bb0c-a31fbdc5db58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:30.425855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef498a4-b968-45a5-bb0c-a31fbdc5db58 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:30.426122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef498a4-b968-45a5-bb0c-a31fbdc5db58 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:30.426381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef498a4-b968-45a5-bb0c-a31fbdc5db58 None admin] PATH_INFO: `/v3/roles/f8a2ea13798443c48750e9baa83c7461` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:30.427907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef498a4-b968-45a5-bb0c-a31fbdc5db58 None admin] RBAC: Authorizing `identity:get_role(role_id=f8a2ea13798443c48750e9baa83c7461)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:30.429295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef498a4-b968-45a5-bb0c-a31fbdc5db58 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:30.430770 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 552/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:40:30 2022] GET /identity/v3/roles/f8a2ea13798443c48750e9baa83c7461 => generated 223 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:30.437293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38bbfc3-a347-4aac-a52c-5a9f25218492 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:30.548348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38bbfc3-a347-4aac-a52c-5a9f25218492 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:30.574981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:30.584265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5c328-9846-4163-aa32-4b03750ce0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:30.584941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5c328-9846-4163-aa32-4b03750ce0f5 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:30.585210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5c328-9846-4163-aa32-4b03750ce0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:30.585480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5c328-9846-4163-aa32-4b03750ce0f5 None admin] PATH_INFO: `/v3/users/4419145e992445c6aa397b8f8353faff` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:30.586679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5c328-9846-4163-aa32-4b03750ce0f5 None admin] RBAC: Authorizing `identity:get_user(user_id=4419145e992445c6aa397b8f8353faff)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:30.588113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5c328-9846-4163-aa32-4b03750ce0f5 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:30.589734 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 569/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:40:30 2022] GET /identity/v3/users/4419145e992445c6aa397b8f8353faff => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:30.598418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef498a4-b968-45a5-bb0c-a31fbdc5db58 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:30.701038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef498a4-b968-45a5-bb0c-a31fbdc5db58 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:30.724464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:30.743119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da13fcf-51c7-4ee0-b632-74d7a5f72d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:30.744018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13fcf-51c7-4ee0-b632-74d7a5f72d1b None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:30.744426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13fcf-51c7-4ee0-b632-74d7a5f72d1b None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:30.744806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13fcf-51c7-4ee0-b632-74d7a5f72d1b None admin] PATH_INFO: `/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:30.747202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da13fcf-51c7-4ee0-b632-74d7a5f72d1b None admin] RBAC: Authorizing `identity:get_project(project_id=f71d9e3993b94aed8bdbeeb9d55811f5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:30.750818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da13fcf-51c7-4ee0-b632-74d7a5f72d1b None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:30.754973 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 553/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:40:30 2022] GET /identity/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:30.761364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5c328-9846-4163-aa32-4b03750ce0f5 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:30.872624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5c328-9846-4163-aa32-4b03750ce0f5 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:30.910048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334a56f-315e-4804-8153-7437bb502fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:30.911284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334a56f-315e-4804-8153-7437bb502fc0 None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:30.911606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334a56f-315e-4804-8153-7437bb502fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:30.911948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334a56f-315e-4804-8153-7437bb502fc0 None admin] PATH_INFO: `/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5/users/4419145e992445c6aa397b8f8353faff/roles/f8a2ea13798443c48750e9baa83c7461` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:30.915238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334a56f-315e-4804-8153-7437bb502fc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f71d9e3993b94aed8bdbeeb9d55811f5, user_id=4419145e992445c6aa397b8f8353faff, role_id=f8a2ea13798443c48750e9baa83c7461)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:30.916681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334a56f-315e-4804-8153-7437bb502fc0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:30.931991 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 570/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:40:30 2022] PUT /identity/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5/users/4419145e992445c6aa397b8f8353faff/roles/f8a2ea13798443c48750e9baa83c7461 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:40:33.114548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cc69b-37c5-4d51-91b7-5c29d8cf0044 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:33.115142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cc69b-37c5-4d51-91b7-5c29d8cf0044 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:33.115142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cc69b-37c5-4d51-91b7-5c29d8cf0044 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:33.116406 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 554/1124] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:33.122821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dbc34-b1c9-4d4c-a245-cf2270afc1b5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:33.123608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dbc34-b1c9-4d4c-a245-cf2270afc1b5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:33.123733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dbc34-b1c9-4d4c-a245-cf2270afc1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:33.146617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c21dbc34-b1c9-4d4c-a245-cf2270afc1b5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:33.147805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c21dbc34-b1c9-4d4c-a245-cf2270afc1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:33.208832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:33.227519 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 571/1125] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:33.556092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cc69b-37c5-4d51-91b7-5c29d8cf0044 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:33.679296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cc69b-37c5-4d51-91b7-5c29d8cf0044 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:33.715491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e864607-efa2-4f22-b7dc-73c46381abb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:33.716088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e864607-efa2-4f22-b7dc-73c46381abb1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:33.716353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e864607-efa2-4f22-b7dc-73c46381abb1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:33.716643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e864607-efa2-4f22-b7dc-73c46381abb1 None admin] PATH_INFO: `/v3/roles/f8a2ea13798443c48750e9baa83c7461` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:33.718130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e864607-efa2-4f22-b7dc-73c46381abb1 None admin] RBAC: Authorizing `identity:get_role(role_id=f8a2ea13798443c48750e9baa83c7461)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:33.719482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e864607-efa2-4f22-b7dc-73c46381abb1 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:33.720812 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 555/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:40:33 2022] GET /identity/v3/roles/f8a2ea13798443c48750e9baa83c7461 => generated 223 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:33.727214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21dbc34-b1c9-4d4c-a245-cf2270afc1b5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:33.826273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21dbc34-b1c9-4d4c-a245-cf2270afc1b5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:33.861985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978ecdd-6ec5-4007-829b-c754f82e55f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:33.862644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978ecdd-6ec5-4007-829b-c754f82e55f5 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:33.862915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978ecdd-6ec5-4007-829b-c754f82e55f5 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:33.863206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978ecdd-6ec5-4007-829b-c754f82e55f5 None admin] PATH_INFO: `/v3/users/4419145e992445c6aa397b8f8353faff` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:33.864511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978ecdd-6ec5-4007-829b-c754f82e55f5 None admin] RBAC: Authorizing `identity:get_user(user_id=4419145e992445c6aa397b8f8353faff)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:33.865968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978ecdd-6ec5-4007-829b-c754f82e55f5 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:33.867748 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 572/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:40:33 2022] GET /identity/v3/users/4419145e992445c6aa397b8f8353faff => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:33.874110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e864607-efa2-4f22-b7dc-73c46381abb1 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:33.938016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:33.983147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e864607-efa2-4f22-b7dc-73c46381abb1 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:34.017627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016dd64-8e2c-47b7-a287-4dd011a6a0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:34.018221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016dd64-8e2c-47b7-a287-4dd011a6a0c4 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:34.018496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016dd64-8e2c-47b7-a287-4dd011a6a0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:34.018820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016dd64-8e2c-47b7-a287-4dd011a6a0c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:34.094394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016dd64-8e2c-47b7-a287-4dd011a6a0c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:34.095778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016dd64-8e2c-47b7-a287-4dd011a6a0c4 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:34.165077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:34.210602 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 556/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:40:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:40:34.217587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978ecdd-6ec5-4007-829b-c754f82e55f5 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:34.292638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:34.340765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978ecdd-6ec5-4007-829b-c754f82e55f5 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:34.386119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce45d6-7b73-4732-8043-d6be9b8c036f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:34.386648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce45d6-7b73-4732-8043-d6be9b8c036f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:34.386912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce45d6-7b73-4732-8043-d6be9b8c036f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:34.387168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce45d6-7b73-4732-8043-d6be9b8c036f None admin] PATH_INFO: `/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:34.388419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce45d6-7b73-4732-8043-d6be9b8c036f None admin] RBAC: Authorizing `identity:get_project(project_id=f71d9e3993b94aed8bdbeeb9d55811f5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:34.389832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce45d6-7b73-4732-8043-d6be9b8c036f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:34.391457 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 573/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:40:34 2022] GET /identity/v3/projects/f71d9e3993b94aed8bdbeeb9d55811f5 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:34.398647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016dd64-8e2c-47b7-a287-4dd011a6a0c4 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:34.515070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016dd64-8e2c-47b7-a287-4dd011a6a0c4 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:34.550374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b562a-5c56-42e3-bf05-d16704313315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:34.550960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b562a-5c56-42e3-bf05-d16704313315 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:34.551204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b562a-5c56-42e3-bf05-d16704313315 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:34.551508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b562a-5c56-42e3-bf05-d16704313315 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:34.552215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b562a-5c56-42e3-bf05-d16704313315 None admin] RBAC: Adding query filter params (role.id=f8a2ea13798443c48750e9baa83c7461, scope.project.id=f71d9e3993b94aed8bdbeeb9d55811f5, user.id=4419145e992445c6aa397b8f8353faff) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:34.552550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b562a-5c56-42e3-bf05-d16704313315 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:34.553994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b562a-5c56-42e3-bf05-d16704313315 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:34.560407 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 557/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:40:34 2022] GET /identity/v3/role_assignments?user.id=4419145e992445c6aa397b8f8353faff&scope.project.id=f71d9e3993b94aed8bdbeeb9d55811f5&role.id=f8a2ea13798443c48750e9baa83c7461 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:36.783873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8eef85-4d5a-46fd-b5f5-483a2b7926d6 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:36.784806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8eef85-4d5a-46fd-b5f5-483a2b7926d6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:36.784806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8eef85-4d5a-46fd-b5f5-483a2b7926d6 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:36.786861 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 574/1131] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:36.792625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32681941-9bf1-40e2-aee1-2e9fe5cf8f4a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:36.793066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32681941-9bf1-40e2-aee1-2e9fe5cf8f4a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:36.793174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32681941-9bf1-40e2-aee1-2e9fe5cf8f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:36.814518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-32681941-9bf1-40e2-aee1-2e9fe5cf8f4a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:36.815698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-32681941-9bf1-40e2-aee1-2e9fe5cf8f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:36.833671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:36.887106 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 558/1132] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:37.227261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8eef85-4d5a-46fd-b5f5-483a2b7926d6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:37.356743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8eef85-4d5a-46fd-b5f5-483a2b7926d6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:37.383327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:37.397101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6fbfc-cdf3-4de9-b0e9-22113e92ff82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:37.397749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6fbfc-cdf3-4de9-b0e9-22113e92ff82 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:37.398055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6fbfc-cdf3-4de9-b0e9-22113e92ff82 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:37.398334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6fbfc-cdf3-4de9-b0e9-22113e92ff82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:37.399485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6fbfc-cdf3-4de9-b0e9-22113e92ff82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:37.400945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6fbfc-cdf3-4de9-b0e9-22113e92ff82 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:37.402516 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 575/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:37.411985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32681941-9bf1-40e2-aee1-2e9fe5cf8f4a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:37.521883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32681941-9bf1-40e2-aee1-2e9fe5cf8f4a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:37.560710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d9dfc-295f-42fa-9c31-d47ab8c1d5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:37.561301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d9dfc-295f-42fa-9c31-d47ab8c1d5b5 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:37.561556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d9dfc-295f-42fa-9c31-d47ab8c1d5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:37.561825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d9dfc-295f-42fa-9c31-d47ab8c1d5b5 None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:37.562457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d9dfc-295f-42fa-9c31-d47ab8c1d5b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:37.563887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d9dfc-295f-42fa-9c31-d47ab8c1d5b5 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:37.594700 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 559/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:40:37 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:40:39.787050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062511b-4bb4-47e9-bc3d-35a8b598f110 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:39.787890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062511b-4bb4-47e9-bc3d-35a8b598f110 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:39.787890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062511b-4bb4-47e9-bc3d-35a8b598f110 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:39.789586 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 576/1135] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:39.796012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b6fc8-8ae2-4094-b20a-b2ee770c6c7d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:39.796482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b6fc8-8ae2-4094-b20a-b2ee770c6c7d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:39.796590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b6fc8-8ae2-4094-b20a-b2ee770c6c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:39.809401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:39.824987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-003b6fc8-8ae2-4094-b20a-b2ee770c6c7d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:39.826189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-003b6fc8-8ae2-4094-b20a-b2ee770c6c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:39.912076 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 560/1136] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:40.252933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062511b-4bb4-47e9-bc3d-35a8b598f110 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:40.361679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062511b-4bb4-47e9-bc3d-35a8b598f110 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:40.399495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d9138-3896-4702-ab21-93185b466767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:40.400425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d9138-3896-4702-ab21-93185b466767 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:40.400801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d9138-3896-4702-ab21-93185b466767 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:40.401198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d9138-3896-4702-ab21-93185b466767 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:40.403099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d9138-3896-4702-ab21-93185b466767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:40.405477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d9138-3896-4702-ab21-93185b466767 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:40.407868 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 577/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:40:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:40.416359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b6fc8-8ae2-4094-b20a-b2ee770c6c7d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:40.527346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b6fc8-8ae2-4094-b20a-b2ee770c6c7d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:40.562942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebaf128-a292-4325-9275-4332f20c0cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:40.563537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaf128-a292-4325-9275-4332f20c0cf6 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:40.563804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaf128-a292-4325-9275-4332f20c0cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:40.564066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaf128-a292-4325-9275-4332f20c0cf6 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:40.564734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebaf128-a292-4325-9275-4332f20c0cf6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:40.566162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebaf128-a292-4325-9275-4332f20c0cf6 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:40.574202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:40.591605 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 561/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:40:40 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:40:42.814319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbfaed-5a89-487f-af3b-b4ea289acc39 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:42.814879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbfaed-5a89-487f-af3b-b4ea289acc39 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:42.814879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbfaed-5a89-487f-af3b-b4ea289acc39 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:42.816163 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 578/1139] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:42.822643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f139bdd-7f29-4de2-8a69-8f4ca19d25ca None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:42.823062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f139bdd-7f29-4de2-8a69-8f4ca19d25ca None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:42.823227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f139bdd-7f29-4de2-8a69-8f4ca19d25ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:42.852890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9f139bdd-7f29-4de2-8a69-8f4ca19d25ca None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:42.854083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9f139bdd-7f29-4de2-8a69-8f4ca19d25ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:42.941763 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 562/1140] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:43.265130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbfaed-5a89-487f-af3b-b4ea289acc39 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:43.333667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:43.376877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbfaed-5a89-487f-af3b-b4ea289acc39 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:43.414987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce551c3f-1d51-4e86-9da4-871eb99cfd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:43.415621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce551c3f-1d51-4e86-9da4-871eb99cfd9d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:43.415897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce551c3f-1d51-4e86-9da4-871eb99cfd9d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:43.416163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce551c3f-1d51-4e86-9da4-871eb99cfd9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:43.426468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce551c3f-1d51-4e86-9da4-871eb99cfd9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-ce551c3f-1d51-4e86-9da4-871eb99cfd9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:40:43.431171 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:43.432285 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:43.433515 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:43.433515 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:43.433515 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:43.433515 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:43.433515 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:40:43.433515 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:40:43.433515 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:40:43.433515 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:40:43.433515 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:40:43.433515 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:40:43.433515 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:40:43.433515 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:43.433515 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 579/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:40:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:43.439128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f139bdd-7f29-4de2-8a69-8f4ca19d25ca None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:43.550270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f139bdd-7f29-4de2-8a69-8f4ca19d25ca None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:43.577924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:43.600487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81061b61-ed09-4643-9c69-964116ec0ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:43.601515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81061b61-ed09-4643-9c69-964116ec0ee2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:43.601789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81061b61-ed09-4643-9c69-964116ec0ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:43.602057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81061b61-ed09-4643-9c69-964116ec0ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:43.602712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81061b61-ed09-4643-9c69-964116ec0ee2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:43.603008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81061b61-ed09-4643-9c69-964116ec0ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:43.604394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81061b61-ed09-4643-9c69-964116ec0ee2 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:43.612312 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 563/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:40:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:43.618768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce551c3f-1d51-4e86-9da4-871eb99cfd9d None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:43.738905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce551c3f-1d51-4e86-9da4-871eb99cfd9d None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:43.755945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:43.774428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22c717-5d4e-4294-837b-042a07f0e100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:43.775029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22c717-5d4e-4294-837b-042a07f0e100 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:43.775297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22c717-5d4e-4294-837b-042a07f0e100 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:43.775553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22c717-5d4e-4294-837b-042a07f0e100 None admin] PATH_INFO: `/v3/users/794f52b2d7a748b98027a568f871ab55` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:43.789473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22c717-5d4e-4294-837b-042a07f0e100 None admin] RBAC: Authorizing `identity:get_user(user_id=794f52b2d7a748b98027a568f871ab55)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:43.790907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22c717-5d4e-4294-837b-042a07f0e100 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:43.792583 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 580/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:40:43 2022] GET /identity/v3/users/794f52b2d7a748b98027a568f871ab55 => generated 286 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:43.798660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81061b61-ed09-4643-9c69-964116ec0ee2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:43.903351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81061b61-ed09-4643-9c69-964116ec0ee2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:43.939191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133977a5-2e31-4223-8b73-a6228ed1df92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:43.939786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133977a5-2e31-4223-8b73-a6228ed1df92 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:43.940045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133977a5-2e31-4223-8b73-a6228ed1df92 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:43.940302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133977a5-2e31-4223-8b73-a6228ed1df92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:43.989894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:44.012178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133977a5-2e31-4223-8b73-a6228ed1df92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:44.013717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133977a5-2e31-4223-8b73-a6228ed1df92 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:44.114513 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 564/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:40:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:40:44.121031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22c717-5d4e-4294-837b-042a07f0e100 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:44.233757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22c717-5d4e-4294-837b-042a07f0e100 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:44.268991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d61273-83f8-4263-b451-a4c9cd06c226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:44.269601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d61273-83f8-4263-b451-a4c9cd06c226 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:44.269867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d61273-83f8-4263-b451-a4c9cd06c226 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:44.270123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d61273-83f8-4263-b451-a4c9cd06c226 None admin] PATH_INFO: `/v3/projects/239fe605f3a449f1ad0086af17b6fe6e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:44.271258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d61273-83f8-4263-b451-a4c9cd06c226 None admin] RBAC: Authorizing `identity:get_project(project_id=239fe605f3a449f1ad0086af17b6fe6e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:44.272752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d61273-83f8-4263-b451-a4c9cd06c226 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:44.274366 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 581/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:40:44 2022] GET /identity/v3/projects/239fe605f3a449f1ad0086af17b6fe6e => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:44.280803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133977a5-2e31-4223-8b73-a6228ed1df92 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:44.375592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:44.398179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133977a5-2e31-4223-8b73-a6228ed1df92 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:44.435359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1127c8-e09d-4a86-aff0-285ac80a5255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:44.435937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1127c8-e09d-4a86-aff0-285ac80a5255 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:44.436205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1127c8-e09d-4a86-aff0-285ac80a5255 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:44.436495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1127c8-e09d-4a86-aff0-285ac80a5255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:44.437208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1127c8-e09d-4a86-aff0-285ac80a5255 None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=239fe605f3a449f1ad0086af17b6fe6e, user.id=794f52b2d7a748b98027a568f871ab55) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:44.437507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1127c8-e09d-4a86-aff0-285ac80a5255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:44.438928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1127c8-e09d-4a86-aff0-285ac80a5255 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:44.444637 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 565/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:40:44 2022] GET /identity/v3/role_assignments?user.id=794f52b2d7a748b98027a568f871ab55&scope.project.id=239fe605f3a449f1ad0086af17b6fe6e&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:46.649988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6008ba0-c9e9-44d2-a61a-ff1541c2decc None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:46.650528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6008ba0-c9e9-44d2-a61a-ff1541c2decc None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:46.650528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6008ba0-c9e9-44d2-a61a-ff1541c2decc None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:46.651788 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 582/1147] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:46.658128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a469a4-f2d1-423f-ac89-a8e9a96ed1c3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:46.658828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a469a4-f2d1-423f-ac89-a8e9a96ed1c3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:46.659026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a469a4-f2d1-423f-ac89-a8e9a96ed1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:46.681220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e5a469a4-f2d1-423f-ac89-a8e9a96ed1c3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:46.682395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e5a469a4-f2d1-423f-ac89-a8e9a96ed1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:46.763910 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 566/1148] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:47.089875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6008ba0-c9e9-44d2-a61a-ff1541c2decc None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:47.143821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:47.207323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6008ba0-c9e9-44d2-a61a-ff1541c2decc None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:47.244558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c289c81-05ad-48e1-88bf-674fece081b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:47.245188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c289c81-05ad-48e1-88bf-674fece081b9 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:47.245466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c289c81-05ad-48e1-88bf-674fece081b9 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:47.245737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c289c81-05ad-48e1-88bf-674fece081b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:47.256598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c289c81-05ad-48e1-88bf-674fece081b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-3c289c81-05ad-48e1-88bf-674fece081b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:40:47.261621 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:47.262836 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:47.264115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:47.264115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:47.264115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:47.264115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:47.264115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:40:47.264115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:40:47.264115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:40:47.264115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:40:47.264115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:40:47.264115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:40:47.264115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:40:47.264115 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:47.264115 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 583/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:40:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:47.268941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a469a4-f2d1-423f-ac89-a8e9a96ed1c3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:47.353674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:47.377943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a469a4-f2d1-423f-ac89-a8e9a96ed1c3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:47.417221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c201774-9203-44ff-82d2-62bbbaebe103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:47.417809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c201774-9203-44ff-82d2-62bbbaebe103 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:47.418078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c201774-9203-44ff-82d2-62bbbaebe103 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:47.418343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c201774-9203-44ff-82d2-62bbbaebe103 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:47.418992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c201774-9203-44ff-82d2-62bbbaebe103 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:47.419288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c201774-9203-44ff-82d2-62bbbaebe103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:47.420676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c201774-9203-44ff-82d2-62bbbaebe103 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:47.429141 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 567/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:40:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:47.438825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c289c81-05ad-48e1-88bf-674fece081b9 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:47.557776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c289c81-05ad-48e1-88bf-674fece081b9 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:47.560662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:47.602154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873457ea-7b88-4fd2-8edf-e1c2a1654c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:47.602769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873457ea-7b88-4fd2-8edf-e1c2a1654c99 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:47.603040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873457ea-7b88-4fd2-8edf-e1c2a1654c99 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:47.603308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873457ea-7b88-4fd2-8edf-e1c2a1654c99 None admin] PATH_INFO: `/v3/users/794f52b2d7a748b98027a568f871ab55` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:47.604587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873457ea-7b88-4fd2-8edf-e1c2a1654c99 None admin] RBAC: Authorizing `identity:get_user(user_id=794f52b2d7a748b98027a568f871ab55)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:47.605968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873457ea-7b88-4fd2-8edf-e1c2a1654c99 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:47.607567 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 584/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:40:47 2022] GET /identity/v3/users/794f52b2d7a748b98027a568f871ab55 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:47.613717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c201774-9203-44ff-82d2-62bbbaebe103 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:47.725942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c201774-9203-44ff-82d2-62bbbaebe103 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:47.760650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c7208-4f6a-4592-a9ef-b55f407e4510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:47.761268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c7208-4f6a-4592-a9ef-b55f407e4510 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:47.761535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c7208-4f6a-4592-a9ef-b55f407e4510 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:47.761805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c7208-4f6a-4592-a9ef-b55f407e4510 None admin] PATH_INFO: `/v3/projects/239fe605f3a449f1ad0086af17b6fe6e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:47.762996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c7208-4f6a-4592-a9ef-b55f407e4510 None admin] RBAC: Authorizing `identity:get_project(project_id=239fe605f3a449f1ad0086af17b6fe6e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:47.764449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c7208-4f6a-4592-a9ef-b55f407e4510 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:47.766067 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 568/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:40:47 2022] GET /identity/v3/projects/239fe605f3a449f1ad0086af17b6fe6e => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:47.773083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873457ea-7b88-4fd2-8edf-e1c2a1654c99 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:47.882062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873457ea-7b88-4fd2-8edf-e1c2a1654c99 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:47.926042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2b6bf-e2cb-4863-b104-d6adb8fc0fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:47.926941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2b6bf-e2cb-4863-b104-d6adb8fc0fe4 None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:47.927083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2b6bf-e2cb-4863-b104-d6adb8fc0fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:47.927348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2b6bf-e2cb-4863-b104-d6adb8fc0fe4 None admin] PATH_INFO: `/v3/projects/239fe605f3a449f1ad0086af17b6fe6e/users/794f52b2d7a748b98027a568f871ab55/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:47.930274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2b6bf-e2cb-4863-b104-d6adb8fc0fe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=239fe605f3a449f1ad0086af17b6fe6e, user_id=794f52b2d7a748b98027a568f871ab55, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:47.931777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2b6bf-e2cb-4863-b104-d6adb8fc0fe4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:47.951202 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 585/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:40:47 2022] PUT /identity/v3/projects/239fe605f3a449f1ad0086af17b6fe6e/users/794f52b2d7a748b98027a568f871ab55/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:40:50.127794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49e5f6-b3ba-44d6-87f3-5839b6851293 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:50.128713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49e5f6-b3ba-44d6-87f3-5839b6851293 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:50.128713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49e5f6-b3ba-44d6-87f3-5839b6851293 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:50.130826 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 569/1154] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:50.137534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1508c-6352-4ea2-a61d-f7ea8da47354 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:50.137989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1508c-6352-4ea2-a61d-f7ea8da47354 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:50.138088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1508c-6352-4ea2-a61d-f7ea8da47354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:50.166253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bda1508c-6352-4ea2-a61d-f7ea8da47354 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:50.167425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bda1508c-6352-4ea2-a61d-f7ea8da47354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:50.195489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:50.242167 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 586/1155] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:50.567149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49e5f6-b3ba-44d6-87f3-5839b6851293 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:50.581457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:50.682718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49e5f6-b3ba-44d6-87f3-5839b6851293 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:50.721304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a733619-1d10-4372-9755-6841ee8d2877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:50.721923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a733619-1d10-4372-9755-6841ee8d2877 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:50.722194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a733619-1d10-4372-9755-6841ee8d2877 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:50.722459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a733619-1d10-4372-9755-6841ee8d2877 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:50.735872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a733619-1d10-4372-9755-6841ee8d2877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9a733619-1d10-4372-9755-6841ee8d2877 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:40:50.742785 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:50.743883 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:50.744871 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:50.744871 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:50.744871 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:50.744871 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:50.744871 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:40:50.744871 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:40:50.744871 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:40:50.744871 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:40:50.744871 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:40:50.744871 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:40:50.744871 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:40:50.744871 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:40:50.744871 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 570/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:40:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:50.750811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1508c-6352-4ea2-a61d-f7ea8da47354 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:50.877506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1508c-6352-4ea2-a61d-f7ea8da47354 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:50.919055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f490b-929f-4ebe-9663-cc7a84d25158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:50.919641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f490b-929f-4ebe-9663-cc7a84d25158 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:50.919909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f490b-929f-4ebe-9663-cc7a84d25158 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:50.920172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f490b-929f-4ebe-9663-cc7a84d25158 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:50.920856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f490b-929f-4ebe-9663-cc7a84d25158 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:50.921153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f490b-929f-4ebe-9663-cc7a84d25158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:50.922517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f490b-929f-4ebe-9663-cc7a84d25158 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:50.930393 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 587/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:40:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:50.937579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a733619-1d10-4372-9755-6841ee8d2877 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:51.013778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:51.060816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a733619-1d10-4372-9755-6841ee8d2877 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:51.095407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf60cc-ca64-43af-be0d-e16f2097afc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:51.096007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf60cc-ca64-43af-be0d-e16f2097afc3 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:51.096266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf60cc-ca64-43af-be0d-e16f2097afc3 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:51.096562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf60cc-ca64-43af-be0d-e16f2097afc3 None admin] PATH_INFO: `/v3/users/794f52b2d7a748b98027a568f871ab55` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:51.097781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf60cc-ca64-43af-be0d-e16f2097afc3 None admin] RBAC: Authorizing `identity:get_user(user_id=794f52b2d7a748b98027a568f871ab55)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:51.099198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf60cc-ca64-43af-be0d-e16f2097afc3 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:51.100899 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 571/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:40:50 2022] GET /identity/v3/users/794f52b2d7a748b98027a568f871ab55 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:51.110504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f490b-929f-4ebe-9663-cc7a84d25158 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:51.145405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:51.230318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f490b-929f-4ebe-9663-cc7a84d25158 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:51.264610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fd7ef-6824-4c85-a3ae-dfd895518a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:51.265209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fd7ef-6824-4c85-a3ae-dfd895518a52 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:51.265477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fd7ef-6824-4c85-a3ae-dfd895518a52 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:51.265741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fd7ef-6824-4c85-a3ae-dfd895518a52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:51.341646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71fd7ef-6824-4c85-a3ae-dfd895518a52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:51.344307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71fd7ef-6824-4c85-a3ae-dfd895518a52 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:51.371390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:51.451240 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 588/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:40:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:40:51.460580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf60cc-ca64-43af-be0d-e16f2097afc3 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:51.567599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf60cc-ca64-43af-be0d-e16f2097afc3 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:51.602907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060d4e4-2c39-4623-81f3-d4dbf40e3ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:51.603514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060d4e4-2c39-4623-81f3-d4dbf40e3ca5 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:51.603784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060d4e4-2c39-4623-81f3-d4dbf40e3ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:51.604049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060d4e4-2c39-4623-81f3-d4dbf40e3ca5 None admin] PATH_INFO: `/v3/projects/239fe605f3a449f1ad0086af17b6fe6e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:51.605314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060d4e4-2c39-4623-81f3-d4dbf40e3ca5 None admin] RBAC: Authorizing `identity:get_project(project_id=239fe605f3a449f1ad0086af17b6fe6e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:51.606761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060d4e4-2c39-4623-81f3-d4dbf40e3ca5 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:51.608393 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 572/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:40:51 2022] GET /identity/v3/projects/239fe605f3a449f1ad0086af17b6fe6e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:51.615093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fd7ef-6824-4c85-a3ae-dfd895518a52 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:51.728028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fd7ef-6824-4c85-a3ae-dfd895518a52 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:51.751329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:51.768539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18022f80-4ac9-41af-b934-7d99f27020c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:51.769406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18022f80-4ac9-41af-b934-7d99f27020c0 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:51.769761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18022f80-4ac9-41af-b934-7d99f27020c0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:51.770103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18022f80-4ac9-41af-b934-7d99f27020c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:51.771204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18022f80-4ac9-41af-b934-7d99f27020c0 None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=239fe605f3a449f1ad0086af17b6fe6e, user.id=794f52b2d7a748b98027a568f871ab55) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:51.771617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18022f80-4ac9-41af-b934-7d99f27020c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:51.774192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18022f80-4ac9-41af-b934-7d99f27020c0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:51.783631 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 589/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:40:51 2022] GET /identity/v3/role_assignments?user.id=794f52b2d7a748b98027a568f871ab55&scope.project.id=239fe605f3a449f1ad0086af17b6fe6e&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:53.963984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fea8b6-68d2-4c7f-8f94-9e6a174ea07e None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:53.964894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fea8b6-68d2-4c7f-8f94-9e6a174ea07e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:53.964894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fea8b6-68d2-4c7f-8f94-9e6a174ea07e None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:53.966914 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 573/1162] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:53.975864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533bed4-525f-43d9-a3e8-89219467422a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:53.976484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533bed4-525f-43d9-a3e8-89219467422a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:53.976484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533bed4-525f-43d9-a3e8-89219467422a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:54.003062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2533bed4-525f-43d9-a3e8-89219467422a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:54.004986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2533bed4-525f-43d9-a3e8-89219467422a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:54.083021 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 590/1163] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:54.404687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fea8b6-68d2-4c7f-8f94-9e6a174ea07e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:54.409548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:54.522043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fea8b6-68d2-4c7f-8f94-9e6a174ea07e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:54.556825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25186741-ccbe-483a-a641-bf3f464fa9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:54.557413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25186741-ccbe-483a-a641-bf3f464fa9a1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:54.557679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25186741-ccbe-483a-a641-bf3f464fa9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:54.558245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25186741-ccbe-483a-a641-bf3f464fa9a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:54.629952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25186741-ccbe-483a-a641-bf3f464fa9a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:54.631398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25186741-ccbe-483a-a641-bf3f464fa9a1 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:54.633340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:54.730389 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 574/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:40:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:40:54.740290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533bed4-525f-43d9-a3e8-89219467422a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:54.852633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533bed4-525f-43d9-a3e8-89219467422a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:54.870564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:54.889873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae6e17-2913-46f6-aaef-fc0011f7e364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:54.890488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae6e17-2913-46f6-aaef-fc0011f7e364 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:54.890753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae6e17-2913-46f6-aaef-fc0011f7e364 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:54.891015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae6e17-2913-46f6-aaef-fc0011f7e364 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:54.897595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae6e17-2913-46f6-aaef-fc0011f7e364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:54.899003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae6e17-2913-46f6-aaef-fc0011f7e364 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-45ae6e17-2913-46f6-aaef-fc0011f7e364 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:40:54.910720 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:40:54.911864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:40:54.912863 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 591/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Aug 20 20:40:54 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:40:54.918089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25186741-ccbe-483a-a641-bf3f464fa9a1 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:55.021985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25186741-ccbe-483a-a641-bf3f464fa9a1 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:55.023717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:40:55.059937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a916509-1c9f-478d-bbd3-e336579646a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:55.060548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a916509-1c9f-478d-bbd3-e336579646a8 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:55.060820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a916509-1c9f-478d-bbd3-e336579646a8 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:55.061078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a916509-1c9f-478d-bbd3-e336579646a8 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:55.061703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a916509-1c9f-478d-bbd3-e336579646a8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:40:55.061991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a916509-1c9f-478d-bbd3-e336579646a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:55.063356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a916509-1c9f-478d-bbd3-e336579646a8 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:55.070718 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 575/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Aug 20 20:40:54 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:40:57.254988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e75394-a195-4276-bb89-56cf97bd01fd None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:57.255500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e75394-a195-4276-bb89-56cf97bd01fd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:57.255500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e75394-a195-4276-bb89-56cf97bd01fd None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:57.256917 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 592/1167] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:40:57.263389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c7321-70ef-48c8-9475-6554e88ad179 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:57.264097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c7321-70ef-48c8-9475-6554e88ad179 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:57.264189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c7321-70ef-48c8-9475-6554e88ad179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:57.286094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-dc3c7321-70ef-48c8-9475-6554e88ad179 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:40:57.287356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dc3c7321-70ef-48c8-9475-6554e88ad179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:40:57.357762 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 576/1168] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:40:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:40:57.696163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e75394-a195-4276-bb89-56cf97bd01fd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:40:57.813236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e75394-a195-4276-bb89-56cf97bd01fd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:40:57.847515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf2764-55cf-44f1-ba77-9af5731c1220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:40:57.848101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf2764-55cf-44f1-ba77-9af5731c1220 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:40:57.848398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf2764-55cf-44f1-ba77-9af5731c1220 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:40:57.848649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf2764-55cf-44f1-ba77-9af5731c1220 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:40:57.849293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf2764-55cf-44f1-ba77-9af5731c1220 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:40:57.850988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf2764-55cf-44f1-ba77-9af5731c1220 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:40:57.870140 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 593/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Sat Aug 20 20:40:57 2022] POST /identity/v3/domains => generated 267 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:41:00.053404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dba1d-b3f7-457d-adb0-8ad3447aa6ad None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:00.055047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dba1d-b3f7-457d-adb0-8ad3447aa6ad None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:00.055047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dba1d-b3f7-457d-adb0-8ad3447aa6ad None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:00.056634 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 577/1170] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:41:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:41:00.063215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b35173-e37d-47c1-ba7e-14e82546fb73 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:00.063643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b35173-e37d-47c1-ba7e-14e82546fb73 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:00.063732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b35173-e37d-47c1-ba7e-14e82546fb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:00.084665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-97b35173-e37d-47c1-ba7e-14e82546fb73 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:41:00.085841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-97b35173-e37d-47c1-ba7e-14e82546fb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:41:00.116583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:00.156672 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 594/1171] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:41:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:41:00.490446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9dba1d-b3f7-457d-adb0-8ad3447aa6ad None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:00.609349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:00.617764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9dba1d-b3f7-457d-adb0-8ad3447aa6ad None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:00.661642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216f393-95a2-4374-8ece-f6f26da1d046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:00.662502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216f393-95a2-4374-8ece-f6f26da1d046 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:00.662966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216f393-95a2-4374-8ece-f6f26da1d046 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:00.663349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216f393-95a2-4374-8ece-f6f26da1d046 None admin] PATH_INFO: `/v3/domains/e60c71bec714485e83015e98ecce2ede` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:00.676319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216f393-95a2-4374-8ece-f6f26da1d046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e60c71bec714485e83015e98ecce2ede)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:00.677794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216f393-95a2-4374-8ece-f6f26da1d046 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:00.679880 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 578/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Sat Aug 20 20:41:00 2022] GET /identity/v3/domains/e60c71bec714485e83015e98ecce2ede => generated 267 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:41:00.686493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b35173-e37d-47c1-ba7e-14e82546fb73 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:00.809554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b35173-e37d-47c1-ba7e-14e82546fb73 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:00.844685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d1cae-dce7-487f-9288-d14b5f4abdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:00.845278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1cae-dce7-487f-9288-d14b5f4abdb5 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:00.845547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1cae-dce7-487f-9288-d14b5f4abdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:00.845806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1cae-dce7-487f-9288-d14b5f4abdb5 None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:00.846433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d1cae-dce7-487f-9288-d14b5f4abdb5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:00.847859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d1cae-dce7-487f-9288-d14b5f4abdb5 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:00.876769 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 595/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:41:00 2022] POST /identity/v3/projects => generated 361 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:41:03.129098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60050c8d-d05a-4e6d-9a8f-aafc166487a7 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:03.130000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60050c8d-d05a-4e6d-9a8f-aafc166487a7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:03.130000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60050c8d-d05a-4e6d-9a8f-aafc166487a7 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:03.131913 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 579/1174] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:41:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:41:03.140739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65579a5-5259-4685-b3bc-24bd20b35f0c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:03.141454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65579a5-5259-4685-b3bc-24bd20b35f0c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:03.141631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65579a5-5259-4685-b3bc-24bd20b35f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:03.168190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e65579a5-5259-4685-b3bc-24bd20b35f0c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:41:03.169429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e65579a5-5259-4685-b3bc-24bd20b35f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:41:03.185052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:03.242614 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 596/1175] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:41:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:41:03.570890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60050c8d-d05a-4e6d-9a8f-aafc166487a7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:03.692340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60050c8d-d05a-4e6d-9a8f-aafc166487a7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:03.733874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33021f-45d8-46f4-8e0d-daa64bdbcb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:03.734498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33021f-45d8-46f4-8e0d-daa64bdbcb11 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:03.734773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33021f-45d8-46f4-8e0d-daa64bdbcb11 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:03.735025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33021f-45d8-46f4-8e0d-daa64bdbcb11 None admin] PATH_INFO: `/v3/domains/e60c71bec714485e83015e98ecce2ede` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:03.736299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33021f-45d8-46f4-8e0d-daa64bdbcb11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e60c71bec714485e83015e98ecce2ede)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:03.737790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33021f-45d8-46f4-8e0d-daa64bdbcb11 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:03.739394 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 580/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Sat Aug 20 20:41:03 2022] GET /identity/v3/domains/e60c71bec714485e83015e98ecce2ede => generated 267 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:41:03.746256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65579a5-5259-4685-b3bc-24bd20b35f0c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:03.866702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65579a5-5259-4685-b3bc-24bd20b35f0c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:03.908345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33a70f-f64a-4399-91b0-f05ed96eed55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:03.909327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33a70f-f64a-4399-91b0-f05ed96eed55 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:03.909698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33a70f-f64a-4399-91b0-f05ed96eed55 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:03.910085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33a70f-f64a-4399-91b0-f05ed96eed55 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:03.911061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33a70f-f64a-4399-91b0-f05ed96eed55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:03.913346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33a70f-f64a-4399-91b0-f05ed96eed55 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:03.937283 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 597/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:41:03 2022] POST /identity/v3/users => generated 311 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:41:06.165718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247144c2-eff8-4e50-8c25-d08258d87170 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:06.166641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247144c2-eff8-4e50-8c25-d08258d87170 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:06.166641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247144c2-eff8-4e50-8c25-d08258d87170 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:06.168669 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 581/1178] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:41:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:41:06.177423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5eb5b-11aa-4a2e-9e18-b05eced4299e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:06.177863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5eb5b-11aa-4a2e-9e18-b05eced4299e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:06.177993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5eb5b-11aa-4a2e-9e18-b05eced4299e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:06.191815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:06.198043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-03d5eb5b-11aa-4a2e-9e18-b05eced4299e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:41:06.199806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-03d5eb5b-11aa-4a2e-9e18-b05eced4299e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:41:06.269667 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 598/1179] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:41:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:41:06.594516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247144c2-eff8-4e50-8c25-d08258d87170 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:06.627274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:06.716773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247144c2-eff8-4e50-8c25-d08258d87170 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:06.752920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19d510-21a1-422f-8dcd-8174ee5c15cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:06.753530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19d510-21a1-422f-8dcd-8174ee5c15cd None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:06.753803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19d510-21a1-422f-8dcd-8174ee5c15cd None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:06.754068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19d510-21a1-422f-8dcd-8174ee5c15cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:06.765568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19d510-21a1-422f-8dcd-8174ee5c15cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-0b19d510-21a1-422f-8dcd-8174ee5c15cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:41:06.770210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:41:06.771302 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:41:06.772622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:41:06.772622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:41:06.772622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:41:06.772622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:41:06.772622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:41:06.772622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:41:06.772622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:41:06.772622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:41:06.772622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:41:06.772622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:41:06.772622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:41:06.772622 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:41:06.772622 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 582/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:41:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:41:06.777986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5eb5b-11aa-4a2e-9e18-b05eced4299e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:06.884476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5eb5b-11aa-4a2e-9e18-b05eced4299e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:06.931212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d10492-0a51-4fdc-970e-9848cab40afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:06.931801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d10492-0a51-4fdc-970e-9848cab40afd None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:06.932039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d10492-0a51-4fdc-970e-9848cab40afd None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:06.932320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d10492-0a51-4fdc-970e-9848cab40afd None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:06.933330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d10492-0a51-4fdc-970e-9848cab40afd None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:41:06.933619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d10492-0a51-4fdc-970e-9848cab40afd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:06.934982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d10492-0a51-4fdc-970e-9848cab40afd None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:06.937203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:06.945017 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 599/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:41:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:41:06.951672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19d510-21a1-422f-8dcd-8174ee5c15cd None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:07.042885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:07.061695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19d510-21a1-422f-8dcd-8174ee5c15cd None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:07.098211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45a464-72d7-40e6-9bbc-635250de4b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:07.098826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45a464-72d7-40e6-9bbc-635250de4b59 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:07.099098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45a464-72d7-40e6-9bbc-635250de4b59 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:07.099360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45a464-72d7-40e6-9bbc-635250de4b59 None admin] PATH_INFO: `/v3/users/06af1d8d27ec4e5ebf3dc2c952b21782` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:07.112745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45a464-72d7-40e6-9bbc-635250de4b59 None admin] RBAC: Authorizing `identity:get_user(user_id=06af1d8d27ec4e5ebf3dc2c952b21782)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:07.114189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45a464-72d7-40e6-9bbc-635250de4b59 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:07.115854 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 583/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:41:06 2022] GET /identity/v3/users/06af1d8d27ec4e5ebf3dc2c952b21782 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:41:07.124624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d10492-0a51-4fdc-970e-9848cab40afd None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:07.226069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d10492-0a51-4fdc-970e-9848cab40afd None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:07.262380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aeb6f0-04f9-4ce1-ad24-500beb865f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:07.262974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aeb6f0-04f9-4ce1-ad24-500beb865f18 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:07.263243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aeb6f0-04f9-4ce1-ad24-500beb865f18 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:07.263504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aeb6f0-04f9-4ce1-ad24-500beb865f18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:07.337122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:07.340887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aeb6f0-04f9-4ce1-ad24-500beb865f18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:07.343589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aeb6f0-04f9-4ce1-ad24-500beb865f18 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:07.450106 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 600/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:41:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:41:07.461190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45a464-72d7-40e6-9bbc-635250de4b59 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:07.575458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45a464-72d7-40e6-9bbc-635250de4b59 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:07.612687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d8a9e-9ea7-4f6e-9258-027358c1847a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:07.613298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d8a9e-9ea7-4f6e-9258-027358c1847a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:07.613569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d8a9e-9ea7-4f6e-9258-027358c1847a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:07.613829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d8a9e-9ea7-4f6e-9258-027358c1847a None admin] PATH_INFO: `/v3/projects/6e0f1e06a07147a8bed7570a308efcfe` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:07.614985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d8a9e-9ea7-4f6e-9258-027358c1847a None admin] RBAC: Authorizing `identity:get_project(project_id=6e0f1e06a07147a8bed7570a308efcfe)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:07.616486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d8a9e-9ea7-4f6e-9258-027358c1847a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:07.618107 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 584/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:41:07 2022] GET /identity/v3/projects/6e0f1e06a07147a8bed7570a308efcfe => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:41:07.625042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aeb6f0-04f9-4ce1-ad24-500beb865f18 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:07.736634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:07.743766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aeb6f0-04f9-4ce1-ad24-500beb865f18 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:07.798075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d7f08-7a3b-40c9-8544-9bec5ecd7634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:07.798925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d7f08-7a3b-40c9-8544-9bec5ecd7634 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:07.799312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d7f08-7a3b-40c9-8544-9bec5ecd7634 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:07.799629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d7f08-7a3b-40c9-8544-9bec5ecd7634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:07.800747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d7f08-7a3b-40c9-8544-9bec5ecd7634 None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=6e0f1e06a07147a8bed7570a308efcfe, user.id=06af1d8d27ec4e5ebf3dc2c952b21782) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:41:07.801119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d7f08-7a3b-40c9-8544-9bec5ecd7634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:07.803407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d7f08-7a3b-40c9-8544-9bec5ecd7634 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:07.809586 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 601/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:41:07 2022] GET /identity/v3/role_assignments?user.id=06af1d8d27ec4e5ebf3dc2c952b21782&scope.project.id=6e0f1e06a07147a8bed7570a308efcfe&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:41:10.028866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c0af7-6604-4753-84e7-4820d497a5f7 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:10.029459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c0af7-6604-4753-84e7-4820d497a5f7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:10.029459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c0af7-6604-4753-84e7-4820d497a5f7 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:10.030744 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 585/1186] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:41:10.036352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff2a69-033c-4896-853a-6b4373d898d7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:10.036821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff2a69-033c-4896-853a-6b4373d898d7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:10.036925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff2a69-033c-4896-853a-6b4373d898d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:10.057280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-16ff2a69-033c-4896-853a-6b4373d898d7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:41:10.058770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-16ff2a69-033c-4896-853a-6b4373d898d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:41:10.139387 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 602/1187] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:41:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:41:10.468029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c0af7-6604-4753-84e7-4820d497a5f7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:10.484085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:10.576232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c0af7-6604-4753-84e7-4820d497a5f7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:10.615807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c63f4-700c-43da-9729-d81fedc98fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:10.616454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c63f4-700c-43da-9729-d81fedc98fe2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:10.616727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c63f4-700c-43da-9729-d81fedc98fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:10.616989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c63f4-700c-43da-9729-d81fedc98fe2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:10.628820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c63f4-700c-43da-9729-d81fedc98fe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-f34c63f4-700c-43da-9729-d81fedc98fe2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:41:10.633495 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:41:10.634550 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:41:10.635700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:41:10.635700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:41:10.635700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:41:10.635700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:41:10.635700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:41:10.635700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:41:10.635700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:41:10.635700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:41:10.635700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:41:10.635700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:41:10.635700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:41:10.635700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:41:10.635700 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 586/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:41:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:41:10.644209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff2a69-033c-4896-853a-6b4373d898d7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:10.754823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:10.764951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff2a69-033c-4896-853a-6b4373d898d7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:10.814383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7e1ae-f64c-4cab-981b-51c8f400db8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:10.815006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7e1ae-f64c-4cab-981b-51c8f400db8c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:10.815254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7e1ae-f64c-4cab-981b-51c8f400db8c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:10.815517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7e1ae-f64c-4cab-981b-51c8f400db8c None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:10.816166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7e1ae-f64c-4cab-981b-51c8f400db8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:41:10.816491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7e1ae-f64c-4cab-981b-51c8f400db8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:10.817868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7e1ae-f64c-4cab-981b-51c8f400db8c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:10.826827 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 603/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:41:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:41:10.833702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c63f4-700c-43da-9729-d81fedc98fe2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:10.938247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:10.970813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c63f4-700c-43da-9729-d81fedc98fe2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:11.009470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8d689-50ee-4c89-963c-356cacb11ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:11.010824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8d689-50ee-4c89-963c-356cacb11ba2 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:11.011100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8d689-50ee-4c89-963c-356cacb11ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:11.011367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8d689-50ee-4c89-963c-356cacb11ba2 None admin] PATH_INFO: `/v3/users/06af1d8d27ec4e5ebf3dc2c952b21782` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:11.012593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8d689-50ee-4c89-963c-356cacb11ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=06af1d8d27ec4e5ebf3dc2c952b21782)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:11.014038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8d689-50ee-4c89-963c-356cacb11ba2 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:11.015671 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 587/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:41:10 2022] GET /identity/v3/users/06af1d8d27ec4e5ebf3dc2c952b21782 => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:41:11.022203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7e1ae-f64c-4cab-981b-51c8f400db8c None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:11.159873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7e1ae-f64c-4cab-981b-51c8f400db8c None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:11.206125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db63b61-7609-43f9-837d-4d15387f589c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:11.207109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db63b61-7609-43f9-837d-4d15387f589c None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:11.207533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db63b61-7609-43f9-837d-4d15387f589c None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:11.207804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db63b61-7609-43f9-837d-4d15387f589c None admin] PATH_INFO: `/v3/projects/6e0f1e06a07147a8bed7570a308efcfe` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:11.209132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db63b61-7609-43f9-837d-4d15387f589c None admin] RBAC: Authorizing `identity:get_project(project_id=6e0f1e06a07147a8bed7570a308efcfe)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:11.210585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db63b61-7609-43f9-837d-4d15387f589c None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:11.212286 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 604/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:41:11 2022] GET /identity/v3/projects/6e0f1e06a07147a8bed7570a308efcfe => generated 361 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:41:11.219702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8d689-50ee-4c89-963c-356cacb11ba2 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:11.353852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8d689-50ee-4c89-963c-356cacb11ba2 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:11.403631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b514d-3baf-4c13-a5ab-e33b3d4855eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:11.404281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b514d-3baf-4c13-a5ab-e33b3d4855eb None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:11.404596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b514d-3baf-4c13-a5ab-e33b3d4855eb None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:11.404837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b514d-3baf-4c13-a5ab-e33b3d4855eb None admin] PATH_INFO: `/v3/projects/6e0f1e06a07147a8bed7570a308efcfe/users/06af1d8d27ec4e5ebf3dc2c952b21782/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:11.409204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b514d-3baf-4c13-a5ab-e33b3d4855eb None admin] RBAC: Authorizing `identity:create_grant(project_id=6e0f1e06a07147a8bed7570a308efcfe, user_id=06af1d8d27ec4e5ebf3dc2c952b21782, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:11.411088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b514d-3baf-4c13-a5ab-e33b3d4855eb None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:11.424145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:11.431149 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 588/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:41:11 2022] PUT /identity/v3/projects/6e0f1e06a07147a8bed7570a308efcfe/users/06af1d8d27ec4e5ebf3dc2c952b21782/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:41:13.643810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47439202-8e7a-46d9-8a9e-b2b0857b1ac3 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:13.644823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47439202-8e7a-46d9-8a9e-b2b0857b1ac3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:13.644823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47439202-8e7a-46d9-8a9e-b2b0857b1ac3 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:13.646872 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 605/1193] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:41:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:41:13.652487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78bff-43db-4b70-b99b-730e31f8362c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:13.652906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78bff-43db-4b70-b99b-730e31f8362c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:13.652995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78bff-43db-4b70-b99b-730e31f8362c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:13.671968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5de78bff-43db-4b70-b99b-730e31f8362c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:41:13.673164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5de78bff-43db-4b70-b99b-730e31f8362c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:41:13.742296 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 589/1194] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:41:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:41:14.064748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47439202-8e7a-46d9-8a9e-b2b0857b1ac3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:14.095949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:14.195565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47439202-8e7a-46d9-8a9e-b2b0857b1ac3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:14.230855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb60840-8b78-4954-ab99-360e0b0d0b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:14.231423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb60840-8b78-4954-ab99-360e0b0d0b65 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:14.231691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb60840-8b78-4954-ab99-360e0b0d0b65 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:14.231951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb60840-8b78-4954-ab99-360e0b0d0b65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:14.243211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb60840-8b78-4954-ab99-360e0b0d0b65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-fcb60840-8b78-4954-ab99-360e0b0d0b65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:41:14.247774 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:41:14.248927 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:41:14.249864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:41:14.249864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:41:14.249864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:41:14.249864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:41:14.249864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:41:14.249864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:41:14.249864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:41:14.249864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:41:14.249864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:41:14.249864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:41:14.249864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:41:14.249864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:41:14.252572 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 606/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:41:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:41:14.255224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de78bff-43db-4b70-b99b-730e31f8362c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:14.359425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de78bff-43db-4b70-b99b-730e31f8362c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:14.387552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:14.397016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745eeb5-2405-4daf-8cc3-35e2f53e1bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:14.397602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745eeb5-2405-4daf-8cc3-35e2f53e1bc1 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:14.397855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745eeb5-2405-4daf-8cc3-35e2f53e1bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:14.398136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745eeb5-2405-4daf-8cc3-35e2f53e1bc1 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:14.398780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745eeb5-2405-4daf-8cc3-35e2f53e1bc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:41:14.399072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745eeb5-2405-4daf-8cc3-35e2f53e1bc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:14.400413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745eeb5-2405-4daf-8cc3-35e2f53e1bc1 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:14.410468 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 590/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:41:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:41:14.417682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb60840-8b78-4954-ab99-360e0b0d0b65 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:14.507252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:14.527712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb60840-8b78-4954-ab99-360e0b0d0b65 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:14.562443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd9c0e-a122-47ed-ace8-772ed1adcff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:14.563057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd9c0e-a122-47ed-ace8-772ed1adcff4 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:14.563322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd9c0e-a122-47ed-ace8-772ed1adcff4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:14.563578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd9c0e-a122-47ed-ace8-772ed1adcff4 None admin] PATH_INFO: `/v3/users/06af1d8d27ec4e5ebf3dc2c952b21782` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:14.564746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd9c0e-a122-47ed-ace8-772ed1adcff4 None admin] RBAC: Authorizing `identity:get_user(user_id=06af1d8d27ec4e5ebf3dc2c952b21782)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:14.566172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd9c0e-a122-47ed-ace8-772ed1adcff4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:14.567732 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 607/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Aug 20 20:41:14 2022] GET /identity/v3/users/06af1d8d27ec4e5ebf3dc2c952b21782 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:41:14.574159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745eeb5-2405-4daf-8cc3-35e2f53e1bc1 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:14.698489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745eeb5-2405-4daf-8cc3-35e2f53e1bc1 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:14.735757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461fd8b-756c-467a-839c-c4dd899a3928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:14.736351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461fd8b-756c-467a-839c-c4dd899a3928 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:14.736648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461fd8b-756c-467a-839c-c4dd899a3928 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:14.736910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461fd8b-756c-467a-839c-c4dd899a3928 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:14.795943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:14.810889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461fd8b-756c-467a-839c-c4dd899a3928 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:14.812898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461fd8b-756c-467a-839c-c4dd899a3928 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:14.937111 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 591/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:41:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:41:14.944076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd9c0e-a122-47ed-ace8-772ed1adcff4 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:15.054724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd9c0e-a122-47ed-ace8-772ed1adcff4 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:15.090897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14b907-3757-41bc-80a0-b37bf0808109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:15.091513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14b907-3757-41bc-80a0-b37bf0808109 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:15.091799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14b907-3757-41bc-80a0-b37bf0808109 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:15.092047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14b907-3757-41bc-80a0-b37bf0808109 None admin] PATH_INFO: `/v3/projects/6e0f1e06a07147a8bed7570a308efcfe` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:15.093206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14b907-3757-41bc-80a0-b37bf0808109 None admin] RBAC: Authorizing `identity:get_project(project_id=6e0f1e06a07147a8bed7570a308efcfe)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:15.094640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14b907-3757-41bc-80a0-b37bf0808109 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:15.096267 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 608/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Aug 20 20:41:14 2022] GET /identity/v3/projects/6e0f1e06a07147a8bed7570a308efcfe => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:41:15.102849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461fd8b-756c-467a-839c-c4dd899a3928 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:41:15.205991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:15.217674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461fd8b-756c-467a-839c-c4dd899a3928 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:41:15.256140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77dd98a-a423-4174-be65-a54956cded72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:41:15.256783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dd98a-a423-4174-be65-a54956cded72 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:41:15.257052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dd98a-a423-4174-be65-a54956cded72 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:41:15.257320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dd98a-a423-4174-be65-a54956cded72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:41:15.258029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77dd98a-a423-4174-be65-a54956cded72 None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=6e0f1e06a07147a8bed7570a308efcfe, user.id=06af1d8d27ec4e5ebf3dc2c952b21782) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:41:15.258331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77dd98a-a423-4174-be65-a54956cded72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:41:15.259763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77dd98a-a423-4174-be65-a54956cded72 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:41:15.265681 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 592/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:41:15 2022] GET /identity/v3/role_assignments?user.id=06af1d8d27ec4e5ebf3dc2c952b21782&scope.project.id=6e0f1e06a07147a8bed7570a308efcfe&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:41:25.086153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:41:25.262943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=23 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:18.307218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd180252-e471-4010-bb31-16c20a4142fd None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:18.307794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd180252-e471-4010-bb31-16c20a4142fd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:18.307889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd180252-e471-4010-bb31-16c20a4142fd None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:18.309524 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 609/1201] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:42:18.317243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e7db6-9921-4d43-bc7c-ea2ff01f056b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:18.317919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e7db6-9921-4d43-bc7c-ea2ff01f056b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:18.317919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e7db6-9921-4d43-bc7c-ea2ff01f056b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:18.336787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4b6e7db6-9921-4d43-bc7c-ea2ff01f056b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:42:18.337992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4b6e7db6-9921-4d43-bc7c-ea2ff01f056b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:42:18.431820 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 593/1202] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:42:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:42:18.777957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd180252-e471-4010-bb31-16c20a4142fd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:18.915005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd180252-e471-4010-bb31-16c20a4142fd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:18.962713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da60fa-fdd7-4d13-ab76-854bb0f57a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:18.965152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da60fa-fdd7-4d13-ab76-854bb0f57a27 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:18.965152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da60fa-fdd7-4d13-ab76-854bb0f57a27 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:18.965152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da60fa-fdd7-4d13-ab76-854bb0f57a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:18.982378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da60fa-fdd7-4d13-ab76-854bb0f57a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:18.983942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da60fa-fdd7-4d13-ab76-854bb0f57a27 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-f8da60fa-fdd7-4d13-ab76-854bb0f57a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:42:19.002622 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:19.003754 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:42:19.005443 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:42:19.005443 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:19.005443 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:19.005443 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 610/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:19.010200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e7db6-9921-4d43-bc7c-ea2ff01f056b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:19.137139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e7db6-9921-4d43-bc7c-ea2ff01f056b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:19.171477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:19.178759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3480d6-7c08-4185-8a30-be1339394946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:19.179723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3480d6-7c08-4185-8a30-be1339394946 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:19.180255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3480d6-7c08-4185-8a30-be1339394946 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:19.180595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3480d6-7c08-4185-8a30-be1339394946 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:19.181239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3480d6-7c08-4185-8a30-be1339394946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:19.181557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3480d6-7c08-4185-8a30-be1339394946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:19.182920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3480d6-7c08-4185-8a30-be1339394946 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:19.200581 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 594/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:19.202718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da60fa-fdd7-4d13-ab76-854bb0f57a27 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:19.287053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:19.338288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da60fa-fdd7-4d13-ab76-854bb0f57a27 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:19.375390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916be891-963e-4396-90ee-e48c50ce3973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:19.376030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916be891-963e-4396-90ee-e48c50ce3973 None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:19.376314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916be891-963e-4396-90ee-e48c50ce3973 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:19.376598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916be891-963e-4396-90ee-e48c50ce3973 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:19.377302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916be891-963e-4396-90ee-e48c50ce3973 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:19.378829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916be891-963e-4396-90ee-e48c50ce3973 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:19.405277 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 611/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Aug 20 20:42:19 2022] POST /identity/v3/users => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:42:21.990759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbae73-53d8-4837-a1da-45202142eb0b None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:21.991325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbae73-53d8-4837-a1da-45202142eb0b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:21.991619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbae73-53d8-4837-a1da-45202142eb0b None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:21.993374 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 595/1206] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:42:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:42:21.999889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd384b-9bc5-4c35-95c1-0b9dac1dd45b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:22.000426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd384b-9bc5-4c35-95c1-0b9dac1dd45b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:22.000536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd384b-9bc5-4c35-95c1-0b9dac1dd45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:22.022534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5ebd384b-9bc5-4c35-95c1-0b9dac1dd45b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:42:22.023893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5ebd384b-9bc5-4c35-95c1-0b9dac1dd45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:42:22.111512 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 612/1207] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:42:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:42:22.498447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbae73-53d8-4837-a1da-45202142eb0b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:22.619334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbae73-53d8-4837-a1da-45202142eb0b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:22.656159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad62fd-62d3-43c4-a6e3-e833c9f16731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:22.657099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad62fd-62d3-43c4-a6e3-e833c9f16731 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:22.657395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad62fd-62d3-43c4-a6e3-e833c9f16731 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:22.657861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad62fd-62d3-43c4-a6e3-e833c9f16731 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:22.672673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad62fd-62d3-43c4-a6e3-e833c9f16731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-edad62fd-62d3-43c4-a6e3-e833c9f16731 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:42:22.678596 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:22.679684 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:22.680680 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:22.680680 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:22.680680 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:22.680680 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:22.680680 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:42:22.680680 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:42:22.680680 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:42:22.680680 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:42:22.680680 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:42:22.680680 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:42:22.680680 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:42:22.680680 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:22.680680 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 596/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:42:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:22.687782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd384b-9bc5-4c35-95c1-0b9dac1dd45b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:22.736348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:22.817911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd384b-9bc5-4c35-95c1-0b9dac1dd45b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:22.871707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d4f8d-0786-434d-8cbc-bebc71bffb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:22.872512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d4f8d-0786-434d-8cbc-bebc71bffb9e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:22.873072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d4f8d-0786-434d-8cbc-bebc71bffb9e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:22.873421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d4f8d-0786-434d-8cbc-bebc71bffb9e None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:22.874104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d4f8d-0786-434d-8cbc-bebc71bffb9e None admin] RBAC: Adding query filter params (name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:22.874456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d4f8d-0786-434d-8cbc-bebc71bffb9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:22.875886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d4f8d-0786-434d-8cbc-bebc71bffb9e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:22.889273 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 613/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:42:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:22.891677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad62fd-62d3-43c4-a6e3-e833c9f16731 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:22.964630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:23.022285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad62fd-62d3-43c4-a6e3-e833c9f16731 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:23.062813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126dd98-19c4-4c92-bb04-a1fd3f444e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:23.063491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126dd98-19c4-4c92-bb04-a1fd3f444e2a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:23.063791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126dd98-19c4-4c92-bb04-a1fd3f444e2a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:23.064087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126dd98-19c4-4c92-bb04-a1fd3f444e2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:23.077682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126dd98-19c4-4c92-bb04-a1fd3f444e2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:23.079062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126dd98-19c4-4c92-bb04-a1fd3f444e2a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-1126dd98-19c4-4c92-bb04-a1fd3f444e2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:42:23.096069 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:23.097181 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:42:23.098081 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:42:23.098081 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:23.098081 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:23.098081 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 597/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:23.104331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d4f8d-0786-434d-8cbc-bebc71bffb9e None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:23.212743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:23.216862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d4f8d-0786-434d-8cbc-bebc71bffb9e None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:23.263468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f63626-901e-4e7f-8e88-73a5f4a644db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:23.264086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f63626-901e-4e7f-8e88-73a5f4a644db None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:23.264356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f63626-901e-4e7f-8e88-73a5f4a644db None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:23.265773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f63626-901e-4e7f-8e88-73a5f4a644db None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:23.265773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f63626-901e-4e7f-8e88-73a5f4a644db None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:23.265773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f63626-901e-4e7f-8e88-73a5f4a644db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:23.266910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f63626-901e-4e7f-8e88-73a5f4a644db None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:23.281082 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 614/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:23.287303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126dd98-19c4-4c92-bb04-a1fd3f444e2a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:23.403686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126dd98-19c4-4c92-bb04-a1fd3f444e2a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:23.405320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:23.441078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570efa2f-e90e-4288-bf74-3d8d43e649b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:23.441737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570efa2f-e90e-4288-bf74-3d8d43e649b3 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:23.441980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570efa2f-e90e-4288-bf74-3d8d43e649b3 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:23.442242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570efa2f-e90e-4288-bf74-3d8d43e649b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:23.450516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570efa2f-e90e-4288-bf74-3d8d43e649b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:23.451967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570efa2f-e90e-4288-bf74-3d8d43e649b3 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-570efa2f-e90e-4288-bf74-3d8d43e649b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:42:23.463571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:23.464678 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:23.466289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:23.466289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:23.466289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:42:23.466289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:42:23.466289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:42:23.466289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:42:23.466289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:42:23.466289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:42:23.466289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 20 20:42:23.466289 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:23.469580 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 598/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:23.470585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f63626-901e-4e7f-8e88-73a5f4a644db None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:23.579781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f63626-901e-4e7f-8e88-73a5f4a644db None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:23.616091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1b213-2401-4ef9-95ae-0f1e6dd3bf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:23.616754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1b213-2401-4ef9-95ae-0f1e6dd3bf98 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:23.617027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1b213-2401-4ef9-95ae-0f1e6dd3bf98 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:23.617284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1b213-2401-4ef9-95ae-0f1e6dd3bf98 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:23.618000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1b213-2401-4ef9-95ae-0f1e6dd3bf98 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:23.618290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1b213-2401-4ef9-95ae-0f1e6dd3bf98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:23.619696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1b213-2401-4ef9-95ae-0f1e6dd3bf98 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:23.641847 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 615/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Aug 20 20:42:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:42:23.643131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570efa2f-e90e-4288-bf74-3d8d43e649b3 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:23.764219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570efa2f-e90e-4288-bf74-3d8d43e649b3 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:23.802299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfe890-720c-4178-9979-2597d8f163a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:23.803479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfe890-720c-4178-9979-2597d8f163a6 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:23.803815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfe890-720c-4178-9979-2597d8f163a6 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:23.804140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfe890-720c-4178-9979-2597d8f163a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:23.845155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:23.900072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcfe890-720c-4178-9979-2597d8f163a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:23.901899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcfe890-720c-4178-9979-2597d8f163a6 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:24.020502 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 599/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:42:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:42:24.025572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1b213-2401-4ef9-95ae-0f1e6dd3bf98 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:24.050021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:24.143341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1b213-2401-4ef9-95ae-0f1e6dd3bf98 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:24.187382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04336f2-bd2a-4542-9003-9296c4e02ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:24.188111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04336f2-bd2a-4542-9003-9296c4e02ce5 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:24.188438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04336f2-bd2a-4542-9003-9296c4e02ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:24.188696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04336f2-bd2a-4542-9003-9296c4e02ce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:24.199684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04336f2-bd2a-4542-9003-9296c4e02ce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:24.201159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04336f2-bd2a-4542-9003-9296c4e02ce5 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-b04336f2-bd2a-4542-9003-9296c4e02ce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:42:24.216493 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:24.222695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:42:24.223632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:42:24.223632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:24.223632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:24.223632 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 616/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:24.223869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfe890-720c-4178-9979-2597d8f163a6 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:24.294688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:24.338723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfe890-720c-4178-9979-2597d8f163a6 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:24.378508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f83ed4-b595-4757-9511-84a1f14d8338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:24.379201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f83ed4-b595-4757-9511-84a1f14d8338 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:24.379449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f83ed4-b595-4757-9511-84a1f14d8338 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:24.379715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f83ed4-b595-4757-9511-84a1f14d8338 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:24.380420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f83ed4-b595-4757-9511-84a1f14d8338 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:24.380794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f83ed4-b595-4757-9511-84a1f14d8338 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:24.382712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f83ed4-b595-4757-9511-84a1f14d8338 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:24.395456 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 600/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:24.401622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04336f2-bd2a-4542-9003-9296c4e02ce5 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:24.479464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:24.514497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04336f2-bd2a-4542-9003-9296c4e02ce5 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:24.554677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d710a46-3a09-4f00-a06b-4985a13223ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:24.555321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d710a46-3a09-4f00-a06b-4985a13223ec None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:24.555677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d710a46-3a09-4f00-a06b-4985a13223ec None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:24.555998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d710a46-3a09-4f00-a06b-4985a13223ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:24.565573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d710a46-3a09-4f00-a06b-4985a13223ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:24.567150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d710a46-3a09-4f00-a06b-4985a13223ec None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-7d710a46-3a09-4f00-a06b-4985a13223ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:42:24.580028 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:24.581132 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:24.582036 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:42:24.582036 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:42:24.582036 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:42:24.582036 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:42:24.582036 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:42:24.582036 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:42:24.582036 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:42:24.582036 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:24.586825 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 617/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:42:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:42:24.588100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f83ed4-b595-4757-9511-84a1f14d8338 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:24.713097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f83ed4-b595-4757-9511-84a1f14d8338 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:24.739572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:24.755630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f93be4-3249-4c8f-8e1f-31691ee695df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:24.756819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f93be4-3249-4c8f-8e1f-31691ee695df None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:24.757482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f93be4-3249-4c8f-8e1f-31691ee695df None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:24.757777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f93be4-3249-4c8f-8e1f-31691ee695df None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:24.758704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f93be4-3249-4c8f-8e1f-31691ee695df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:24.759239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f93be4-3249-4c8f-8e1f-31691ee695df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:24.761617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f93be4-3249-4c8f-8e1f-31691ee695df None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:24.783129 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 601/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:42:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:24.784309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d710a46-3a09-4f00-a06b-4985a13223ec None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:24.907842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d710a46-3a09-4f00-a06b-4985a13223ec None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:24.939261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:24.945823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d743d-2d1d-4ddd-b4e8-5e0e3f397b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:24.946427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d743d-2d1d-4ddd-b4e8-5e0e3f397b90 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:24.946714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d743d-2d1d-4ddd-b4e8-5e0e3f397b90 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:24.947046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d743d-2d1d-4ddd-b4e8-5e0e3f397b90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:24.947903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d743d-2d1d-4ddd-b4e8-5e0e3f397b90 None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=2b04f1e0e8094a14a3c1384faa601aa9) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:24.948272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d743d-2d1d-4ddd-b4e8-5e0e3f397b90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:24.949884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d743d-2d1d-4ddd-b4e8-5e0e3f397b90 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:24.956725 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 618/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:42:24 2022] GET /identity/v3/role_assignments?user.id=2b04f1e0e8094a14a3c1384faa601aa9&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:27.267010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb34dfb-35e1-4c0a-a605-1e59cca0b4ef None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:27.267629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb34dfb-35e1-4c0a-a605-1e59cca0b4ef None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:27.267629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb34dfb-35e1-4c0a-a605-1e59cca0b4ef None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:27.269237 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 602/1220] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:42:27.278383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e085105-45cf-438c-b1b1-393d889ff7c6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:27.278383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e085105-45cf-438c-b1b1-393d889ff7c6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:27.278383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e085105-45cf-438c-b1b1-393d889ff7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:27.296854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5e085105-45cf-438c-b1b1-393d889ff7c6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:42:27.298808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5e085105-45cf-438c-b1b1-393d889ff7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:42:27.383341 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 619/1221] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:42:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:42:27.723044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb34dfb-35e1-4c0a-a605-1e59cca0b4ef None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:27.852179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb34dfb-35e1-4c0a-a605-1e59cca0b4ef None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:27.898423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030716dd-4122-4588-91eb-b4bb920026b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:27.899046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030716dd-4122-4588-91eb-b4bb920026b4 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:27.899313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030716dd-4122-4588-91eb-b4bb920026b4 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:27.899575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030716dd-4122-4588-91eb-b4bb920026b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:27.909908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030716dd-4122-4588-91eb-b4bb920026b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-030716dd-4122-4588-91eb-b4bb920026b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:42:27.915082 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:27.916325 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:27.917520 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:27.917520 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:27.917520 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:27.917520 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:27.917520 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:42:27.917520 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:42:27.917520 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:42:27.917520 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:42:27.917520 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:42:27.917520 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:42:27.917520 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:42:27.917520 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:27.917520 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 603/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:42:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:27.922555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e085105-45cf-438c-b1b1-393d889ff7c6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:28.032877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e085105-45cf-438c-b1b1-393d889ff7c6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:28.059558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:28.073860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd53f8-f9bb-450c-aa3b-2fa7e1098375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:28.074590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd53f8-f9bb-450c-aa3b-2fa7e1098375 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:28.074861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd53f8-f9bb-450c-aa3b-2fa7e1098375 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:28.075122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd53f8-f9bb-450c-aa3b-2fa7e1098375 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:28.075774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd53f8-f9bb-450c-aa3b-2fa7e1098375 None admin] RBAC: Adding query filter params (name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:28.076072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd53f8-f9bb-450c-aa3b-2fa7e1098375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:28.077471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd53f8-f9bb-450c-aa3b-2fa7e1098375 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:28.087163 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 620/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:42:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:28.093582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030716dd-4122-4588-91eb-b4bb920026b4 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:28.143015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:28.202086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030716dd-4122-4588-91eb-b4bb920026b4 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:28.245664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006cdb2-937d-4ec1-aa60-ead5a803928f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:28.246286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006cdb2-937d-4ec1-aa60-ead5a803928f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:28.246551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006cdb2-937d-4ec1-aa60-ead5a803928f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:28.246806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006cdb2-937d-4ec1-aa60-ead5a803928f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:28.258240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006cdb2-937d-4ec1-aa60-ead5a803928f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:28.259766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006cdb2-937d-4ec1-aa60-ead5a803928f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-1006cdb2-937d-4ec1-aa60-ead5a803928f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:42:28.275214 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:28.281733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:42:28.282665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:42:28.282665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:28.282665 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:28.282665 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 604/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:28.282977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd53f8-f9bb-450c-aa3b-2fa7e1098375 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:28.399474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd53f8-f9bb-450c-aa3b-2fa7e1098375 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:28.437199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cb6bd-f0eb-42a2-9258-5dd9969ef053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:28.437803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cb6bd-f0eb-42a2-9258-5dd9969ef053 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:28.438108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cb6bd-f0eb-42a2-9258-5dd9969ef053 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:28.438331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cb6bd-f0eb-42a2-9258-5dd9969ef053 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:28.438991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84cb6bd-f0eb-42a2-9258-5dd9969ef053 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:28.439295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84cb6bd-f0eb-42a2-9258-5dd9969ef053 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:28.440704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84cb6bd-f0eb-42a2-9258-5dd9969ef053 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:28.453366 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 621/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:28.460118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006cdb2-937d-4ec1-aa60-ead5a803928f None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:28.565543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:28.574208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006cdb2-937d-4ec1-aa60-ead5a803928f None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:28.613352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f482e-c05b-48d8-8756-554aceb9641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:28.613989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f482e-c05b-48d8-8756-554aceb9641f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:28.614260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f482e-c05b-48d8-8756-554aceb9641f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:28.614510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f482e-c05b-48d8-8756-554aceb9641f None admin] PATH_INFO: `/v3/users/placement` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:28.624295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f482e-c05b-48d8-8756-554aceb9641f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:28.626049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f482e-c05b-48d8-8756-554aceb9641f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-8f8f482e-c05b-48d8-8756-554aceb9641f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:42:28.640362 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:28.641801 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:28.642741 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:28.642741 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:28.642741 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:42:28.642741 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:42:28.642741 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:42:28.642741 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:42:28.642741 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:42:28.642741 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:42:28.642741 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 20 20:42:28.642741 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:28.642741 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 605/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:28.649382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cb6bd-f0eb-42a2-9258-5dd9969ef053 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:28.702835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:28.768171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cb6bd-f0eb-42a2-9258-5dd9969ef053 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:28.805852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5499fa-a4e3-4d26-8469-9c989bcbd8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:28.806432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5499fa-a4e3-4d26-8469-9c989bcbd8c4 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:28.806889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5499fa-a4e3-4d26-8469-9c989bcbd8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:28.807166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5499fa-a4e3-4d26-8469-9c989bcbd8c4 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:28.807965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5499fa-a4e3-4d26-8469-9c989bcbd8c4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:28.808251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5499fa-a4e3-4d26-8469-9c989bcbd8c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:28.810011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5499fa-a4e3-4d26-8469-9c989bcbd8c4 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:28.829367 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 622/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Aug 20 20:42:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:28.835905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f482e-c05b-48d8-8756-554aceb9641f None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:28.946599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f482e-c05b-48d8-8756-554aceb9641f None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:28.985070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755a939-4fc4-40e2-b796-63d7e774f79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:28.986595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755a939-4fc4-40e2-b796-63d7e774f79e None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:28.986902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755a939-4fc4-40e2-b796-63d7e774f79e None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:28.987150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755a939-4fc4-40e2-b796-63d7e774f79e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:29.000843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755a939-4fc4-40e2-b796-63d7e774f79e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:29.002511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755a939-4fc4-40e2-b796-63d7e774f79e None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-5755a939-4fc4-40e2-b796-63d7e774f79e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:42:29.020327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:29.021578 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:42:29.022731 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:42:29.022731 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:29.022731 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:29.022731 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 606/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:29.028534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5499fa-a4e3-4d26-8469-9c989bcbd8c4 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:29.127466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5499fa-a4e3-4d26-8469-9c989bcbd8c4 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:29.129967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:29.171553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cec736-823f-4ee8-bc71-9a4a8ab794ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:29.172187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cec736-823f-4ee8-bc71-9a4a8ab794ad None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:29.172469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cec736-823f-4ee8-bc71-9a4a8ab794ad None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:29.172771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cec736-823f-4ee8-bc71-9a4a8ab794ad None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:29.173368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cec736-823f-4ee8-bc71-9a4a8ab794ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:29.173664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cec736-823f-4ee8-bc71-9a4a8ab794ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:29.175062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cec736-823f-4ee8-bc71-9a4a8ab794ad None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:29.187760 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 623/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:42:29.194470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755a939-4fc4-40e2-b796-63d7e774f79e None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:29.205251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:29.312258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755a939-4fc4-40e2-b796-63d7e774f79e None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:29.349469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094e93e-8e37-4298-90ba-331a8c720029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:29.350138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094e93e-8e37-4298-90ba-331a8c720029 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:29.350371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094e93e-8e37-4298-90ba-331a8c720029 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:29.350683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094e93e-8e37-4298-90ba-331a8c720029 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:29.357763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094e93e-8e37-4298-90ba-331a8c720029 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:29.359217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094e93e-8e37-4298-90ba-331a8c720029 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-f094e93e-8e37-4298-90ba-331a8c720029 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:42:29.370553 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:29.372191 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:29.373164 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:42:29.373164 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:42:29.373164 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:42:29.373164 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:42:29.373164 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:42:29.373164 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:42:29.373164 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:42:29.373164 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:29.373164 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 607/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:29.378403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cec736-823f-4ee8-bc71-9a4a8ab794ad None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:29.485918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cec736-823f-4ee8-bc71-9a4a8ab794ad None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:29.520433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0a43b-424a-4ee9-9a66-a059165b571e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:29.521497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0a43b-424a-4ee9-9a66-a059165b571e None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:29.521497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0a43b-424a-4ee9-9a66-a059165b571e None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:29.522147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0a43b-424a-4ee9-9a66-a059165b571e None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:29.525045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0a43b-424a-4ee9-9a66-a059165b571e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:29.525045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0a43b-424a-4ee9-9a66-a059165b571e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:29.525045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0a43b-424a-4ee9-9a66-a059165b571e None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:29.537619 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 624/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:42:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:29.545130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094e93e-8e37-4298-90ba-331a8c720029 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:29.614505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:29.652323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094e93e-8e37-4298-90ba-331a8c720029 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:29.688940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac61f02-7e8b-4c6c-9f94-ce8000a7deb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:29.689587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac61f02-7e8b-4c6c-9f94-ce8000a7deb2 None admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:29.689853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac61f02-7e8b-4c6c-9f94-ce8000a7deb2 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:29.690117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac61f02-7e8b-4c6c-9f94-ce8000a7deb2 None admin] PATH_INFO: `/v3/projects/3b6c88840d764c2385aa678da4338213/users/2b04f1e0e8094a14a3c1384faa601aa9/roles/04bd1260156b4de9bc4feff892a644a6` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:29.706085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac61f02-7e8b-4c6c-9f94-ce8000a7deb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b6c88840d764c2385aa678da4338213, user_id=2b04f1e0e8094a14a3c1384faa601aa9, role_id=04bd1260156b4de9bc4feff892a644a6)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:29.707521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac61f02-7e8b-4c6c-9f94-ce8000a7deb2 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:29.719911 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 608/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:42:29 2022] PUT /identity/v3/projects/3b6c88840d764c2385aa678da4338213/users/2b04f1e0e8094a14a3c1384faa601aa9/roles/04bd1260156b4de9bc4feff892a644a6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:42:32.024775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560eab9-61e0-4411-8053-b79f30c47aae None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:32.025433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560eab9-61e0-4411-8053-b79f30c47aae None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:32.025433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560eab9-61e0-4411-8053-b79f30c47aae None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:32.026660 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 625/1233] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:42:32.032363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95309425-6012-4bf6-80fc-f190055ece73 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:32.032888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95309425-6012-4bf6-80fc-f190055ece73 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:32.033063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95309425-6012-4bf6-80fc-f190055ece73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:32.053826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-95309425-6012-4bf6-80fc-f190055ece73 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:42:32.055062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-95309425-6012-4bf6-80fc-f190055ece73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:42:32.127700 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 609/1234] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:42:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:42:32.456649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560eab9-61e0-4411-8053-b79f30c47aae None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:32.482845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:32.563596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560eab9-61e0-4411-8053-b79f30c47aae None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:32.600476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54462641-173a-453c-91de-cf5fea235a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:32.601143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54462641-173a-453c-91de-cf5fea235a32 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:32.601388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54462641-173a-453c-91de-cf5fea235a32 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:32.601652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54462641-173a-453c-91de-cf5fea235a32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:32.611885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54462641-173a-453c-91de-cf5fea235a32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-54462641-173a-453c-91de-cf5fea235a32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:42:32.616783 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:32.618059 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:32.619124 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:32.619124 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:32.619124 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:32.619124 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:32.619124 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:42:32.619124 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:42:32.619124 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:42:32.619124 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:42:32.619124 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:42:32.619124 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:42:32.619124 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 20 20:42:32.619124 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:32.619124 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 626/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Aug 20 20:42:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:32.624158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95309425-6012-4bf6-80fc-f190055ece73 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:32.672472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:32.727368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95309425-6012-4bf6-80fc-f190055ece73 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:32.762922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee426ec3-a3a5-4dce-aaff-483da7e01a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:32.763494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee426ec3-a3a5-4dce-aaff-483da7e01a9a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:32.763764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee426ec3-a3a5-4dce-aaff-483da7e01a9a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:32.764024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee426ec3-a3a5-4dce-aaff-483da7e01a9a None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:32.764719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee426ec3-a3a5-4dce-aaff-483da7e01a9a None admin] RBAC: Adding query filter params (name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:32.765019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee426ec3-a3a5-4dce-aaff-483da7e01a9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:32.766380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee426ec3-a3a5-4dce-aaff-483da7e01a9a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:32.779927 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 610/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Aug 20 20:42:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:32.781216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54462641-173a-453c-91de-cf5fea235a32 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:32.884744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:32.903203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54462641-173a-453c-91de-cf5fea235a32 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:32.941348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cdd91-c5d6-43da-8537-da8d97832926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:32.943251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cdd91-c5d6-43da-8537-da8d97832926 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:32.943251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cdd91-c5d6-43da-8537-da8d97832926 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:32.943251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cdd91-c5d6-43da-8537-da8d97832926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:32.953004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cdd91-c5d6-43da-8537-da8d97832926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:32.954459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cdd91-c5d6-43da-8537-da8d97832926 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-703cdd91-c5d6-43da-8537-da8d97832926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:42:32.969005 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:32.970131 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:42:32.971011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:42:32.971011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:32.971011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:32.971011 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 627/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:32.976595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee426ec3-a3a5-4dce-aaff-483da7e01a9a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:33.089682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee426ec3-a3a5-4dce-aaff-483da7e01a9a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:33.099845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:33.127525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829cd9b-600d-4f30-896b-bb4e051c5444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:33.128120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829cd9b-600d-4f30-896b-bb4e051c5444 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:33.128417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829cd9b-600d-4f30-896b-bb4e051c5444 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:33.128712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829cd9b-600d-4f30-896b-bb4e051c5444 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:33.129343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829cd9b-600d-4f30-896b-bb4e051c5444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:33.129638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829cd9b-600d-4f30-896b-bb4e051c5444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:33.131357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829cd9b-600d-4f30-896b-bb4e051c5444 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:33.143865 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 611/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:33.149846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cdd91-c5d6-43da-8537-da8d97832926 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:33.254118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cdd91-c5d6-43da-8537-da8d97832926 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:33.290875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c35ec-25dd-44ca-b63f-9eb6bb81ef79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:33.291518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c35ec-25dd-44ca-b63f-9eb6bb81ef79 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:33.291766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c35ec-25dd-44ca-b63f-9eb6bb81ef79 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:33.292030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c35ec-25dd-44ca-b63f-9eb6bb81ef79 None admin] PATH_INFO: `/v3/users/placement` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:33.294795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:33.300240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c35ec-25dd-44ca-b63f-9eb6bb81ef79 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:33.301963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c35ec-25dd-44ca-b63f-9eb6bb81ef79 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-f89c35ec-25dd-44ca-b63f-9eb6bb81ef79 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:42:33.315084 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:33.316247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:33.317261 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:33.317261 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:33.317261 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:42:33.317261 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:42:33.317261 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:42:33.317261 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:42:33.317261 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:42:33.317261 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:42:33.317261 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 20 20:42:33.317261 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:33.317261 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 628/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:33.322831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829cd9b-600d-4f30-896b-bb4e051c5444 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:33.440566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829cd9b-600d-4f30-896b-bb4e051c5444 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:33.478597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7c79b-2e52-4db8-8013-b27f3418cb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:33.479207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7c79b-2e52-4db8-8013-b27f3418cb06 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:33.479480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7c79b-2e52-4db8-8013-b27f3418cb06 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:33.479745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7c79b-2e52-4db8-8013-b27f3418cb06 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:33.480516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7c79b-2e52-4db8-8013-b27f3418cb06 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:33.480807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7c79b-2e52-4db8-8013-b27f3418cb06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:33.482262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7c79b-2e52-4db8-8013-b27f3418cb06 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:33.500016 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 612/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Aug 20 20:42:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:33.506249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c35ec-25dd-44ca-b63f-9eb6bb81ef79 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:33.621922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c35ec-25dd-44ca-b63f-9eb6bb81ef79 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:33.657848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c0f8f-151d-4c01-9b53-e00fb85a3899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:33.658454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c0f8f-151d-4c01-9b53-e00fb85a3899 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:33.658730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c0f8f-151d-4c01-9b53-e00fb85a3899 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:33.659036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c0f8f-151d-4c01-9b53-e00fb85a3899 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:33.724526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:33.731664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c0f8f-151d-4c01-9b53-e00fb85a3899 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:33.733261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c0f8f-151d-4c01-9b53-e00fb85a3899 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:33.854005 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 629/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:42:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:42:33.855252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7c79b-2e52-4db8-8013-b27f3418cb06 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:33.886407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:33.966408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7c79b-2e52-4db8-8013-b27f3418cb06 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:34.003833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540884f0-3472-4c0f-95be-70eda89f05ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:34.004406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540884f0-3472-4c0f-95be-70eda89f05ec None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:34.004765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540884f0-3472-4c0f-95be-70eda89f05ec None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:34.005030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540884f0-3472-4c0f-95be-70eda89f05ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:34.017027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540884f0-3472-4c0f-95be-70eda89f05ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:34.018814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540884f0-3472-4c0f-95be-70eda89f05ec None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-540884f0-3472-4c0f-95be-70eda89f05ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:42:34.033930 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:34.041153 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:42:34.042236 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:42:34.042236 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:34.042236 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:34.042236 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 613/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:34.042496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c0f8f-151d-4c01-9b53-e00fb85a3899 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:34.155724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:34.163956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c0f8f-151d-4c01-9b53-e00fb85a3899 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:34.204401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620496a-5bb6-4397-b598-d7a510889fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:34.205077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620496a-5bb6-4397-b598-d7a510889fe9 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:34.205416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620496a-5bb6-4397-b598-d7a510889fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:34.205689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620496a-5bb6-4397-b598-d7a510889fe9 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:34.206475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620496a-5bb6-4397-b598-d7a510889fe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:34.206778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620496a-5bb6-4397-b598-d7a510889fe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:34.208216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620496a-5bb6-4397-b598-d7a510889fe9 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:34.222554 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 630/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:34.229968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540884f0-3472-4c0f-95be-70eda89f05ec None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:34.327542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:34.357369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540884f0-3472-4c0f-95be-70eda89f05ec None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:34.398239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76efdb8-929a-4b8c-8a63-7228557b03d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:34.398936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76efdb8-929a-4b8c-8a63-7228557b03d0 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:34.399251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76efdb8-929a-4b8c-8a63-7228557b03d0 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:34.399543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76efdb8-929a-4b8c-8a63-7228557b03d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:34.407228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76efdb8-929a-4b8c-8a63-7228557b03d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:34.408983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76efdb8-929a-4b8c-8a63-7228557b03d0 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-f76efdb8-929a-4b8c-8a63-7228557b03d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:42:34.422555 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:34.423654 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:34.424821 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:42:34.424821 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:42:34.424821 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:42:34.424821 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:42:34.424821 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:42:34.424821 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:42:34.424821 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:42:34.424821 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:34.424821 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 614/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:42:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:34.430878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620496a-5bb6-4397-b598-d7a510889fe9 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:34.550359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620496a-5bb6-4397-b598-d7a510889fe9 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:34.592489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb043b-967c-499c-a46f-ec8dc2bdd4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:34.593049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb043b-967c-499c-a46f-ec8dc2bdd4ad None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:34.593293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb043b-967c-499c-a46f-ec8dc2bdd4ad None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:34.593562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb043b-967c-499c-a46f-ec8dc2bdd4ad None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:34.594272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb043b-967c-499c-a46f-ec8dc2bdd4ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:34.594547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb043b-967c-499c-a46f-ec8dc2bdd4ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:34.595982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb043b-967c-499c-a46f-ec8dc2bdd4ad None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:34.608464 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 631/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:42:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:34.614777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76efdb8-929a-4b8c-8a63-7228557b03d0 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:34.726040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76efdb8-929a-4b8c-8a63-7228557b03d0 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:34.760847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e500c-4c7a-477f-8492-5c2bfdd091d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:34.761456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e500c-4c7a-477f-8492-5c2bfdd091d0 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:34.761728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e500c-4c7a-477f-8492-5c2bfdd091d0 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:34.761992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e500c-4c7a-477f-8492-5c2bfdd091d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:34.762714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e500c-4c7a-477f-8492-5c2bfdd091d0 None admin] RBAC: Adding query filter params (role.id=04bd1260156b4de9bc4feff892a644a6, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=2b04f1e0e8094a14a3c1384faa601aa9) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:34.763025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e500c-4c7a-477f-8492-5c2bfdd091d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:34.764493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e500c-4c7a-477f-8492-5c2bfdd091d0 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:34.773576 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 615/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:42:34 2022] GET /identity/v3/role_assignments?user.id=2b04f1e0e8094a14a3c1384faa601aa9&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=04bd1260156b4de9bc4feff892a644a6 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:37.029704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d19c6e-4e94-40df-9683-43de7e5bcfc8 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:37.030285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d19c6e-4e94-40df-9683-43de7e5bcfc8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:37.030285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d19c6e-4e94-40df-9683-43de7e5bcfc8 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:37.031599 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 632/1247] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:42:37.037960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e487dd-5de9-4e5b-8832-fcb8ba6ef654 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:37.038441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e487dd-5de9-4e5b-8832-fcb8ba6ef654 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:37.038666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e487dd-5de9-4e5b-8832-fcb8ba6ef654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:37.045540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:37.061173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e2e487dd-5de9-4e5b-8832-fcb8ba6ef654 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:42:37.062795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e2e487dd-5de9-4e5b-8832-fcb8ba6ef654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:42:37.135977 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 616/1248] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:42:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:42:37.492330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d19c6e-4e94-40df-9683-43de7e5bcfc8 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:37.518570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:37.611443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d19c6e-4e94-40df-9683-43de7e5bcfc8 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:37.645645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4600ca-3b70-4647-a67c-740068429701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:37.646275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4600ca-3b70-4647-a67c-740068429701 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:37.648517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4600ca-3b70-4647-a67c-740068429701 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:37.648517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4600ca-3b70-4647-a67c-740068429701 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:37.658484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4600ca-3b70-4647-a67c-740068429701 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-2b4600ca-3b70-4647-a67c-740068429701 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:42:37.665026 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:37.666188 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:37.667161 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:37.667161 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:37.667161 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:37.667161 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:37.667161 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:42:37.667161 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:42:37.667161 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:42:37.667161 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:42:37.667161 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:42:37.667161 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:42:37.667161 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:42:37.667161 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:37.667161 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 633/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:42:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:37.672858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e487dd-5de9-4e5b-8832-fcb8ba6ef654 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:37.784531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e487dd-5de9-4e5b-8832-fcb8ba6ef654 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:37.817113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:37.823609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e5ad5-f38f-409c-acf6-b9497ab73940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:37.824225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e5ad5-f38f-409c-acf6-b9497ab73940 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:37.824528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e5ad5-f38f-409c-acf6-b9497ab73940 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:37.824772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e5ad5-f38f-409c-acf6-b9497ab73940 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:37.825420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e5ad5-f38f-409c-acf6-b9497ab73940 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:37.825712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e5ad5-f38f-409c-acf6-b9497ab73940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:37.827100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e5ad5-f38f-409c-acf6-b9497ab73940 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:37.841490 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 617/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:42:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:37.842982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4600ca-3b70-4647-a67c-740068429701 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:37.935104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:37.959926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4600ca-3b70-4647-a67c-740068429701 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:37.995221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286738b4-8c1a-4404-a6c4-3f180cd4a9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:37.995856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286738b4-8c1a-4404-a6c4-3f180cd4a9bf None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:37.996121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286738b4-8c1a-4404-a6c4-3f180cd4a9bf None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:37.996417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286738b4-8c1a-4404-a6c4-3f180cd4a9bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:38.007282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286738b4-8c1a-4404-a6c4-3f180cd4a9bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:38.008864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286738b4-8c1a-4404-a6c4-3f180cd4a9bf None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-286738b4-8c1a-4404-a6c4-3f180cd4a9bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:42:38.023500 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:38.024755 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:42:38.025831 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:42:38.025831 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:38.025831 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:38.025831 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 634/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:38.032349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e5ad5-f38f-409c-acf6-b9497ab73940 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:38.147180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e5ad5-f38f-409c-acf6-b9497ab73940 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:38.182738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d856dd-5950-429f-bda4-9a0d114b2ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:38.183450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d856dd-5950-429f-bda4-9a0d114b2ae5 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:38.183748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d856dd-5950-429f-bda4-9a0d114b2ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:38.184051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d856dd-5950-429f-bda4-9a0d114b2ae5 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:38.185069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d856dd-5950-429f-bda4-9a0d114b2ae5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:38.185427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d856dd-5950-429f-bda4-9a0d114b2ae5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:38.186974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d856dd-5950-429f-bda4-9a0d114b2ae5 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:38.200248 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 618/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:38.206567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286738b4-8c1a-4404-a6c4-3f180cd4a9bf None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:38.312268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286738b4-8c1a-4404-a6c4-3f180cd4a9bf None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:38.340439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:38.351357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21456442-1d5b-4248-9ea3-157d349a5a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:38.351995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21456442-1d5b-4248-9ea3-157d349a5a69 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:38.352267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21456442-1d5b-4248-9ea3-157d349a5a69 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:38.352578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21456442-1d5b-4248-9ea3-157d349a5a69 None admin] PATH_INFO: `/v3/users/placement` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:38.360961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21456442-1d5b-4248-9ea3-157d349a5a69 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:38.362521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21456442-1d5b-4248-9ea3-157d349a5a69 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-21456442-1d5b-4248-9ea3-157d349a5a69 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:42:38.374834 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:38.375940 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:38.376956 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:38.376956 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:38.376956 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:42:38.376956 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:42:38.376956 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:42:38.376956 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:42:38.376956 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:42:38.376956 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:42:38.376956 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 20 20:42:38.376956 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:38.376956 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 635/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:42:38.382446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d856dd-5950-429f-bda4-9a0d114b2ae5 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:38.443062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:38.499038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d856dd-5950-429f-bda4-9a0d114b2ae5 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:38.543229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbb1fd-0a1d-4630-8aa8-0285f9f62cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:38.544052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbb1fd-0a1d-4630-8aa8-0285f9f62cbf None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:38.544560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbb1fd-0a1d-4630-8aa8-0285f9f62cbf None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:38.544918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbb1fd-0a1d-4630-8aa8-0285f9f62cbf None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:38.545715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbb1fd-0a1d-4630-8aa8-0285f9f62cbf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:38.546064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbb1fd-0a1d-4630-8aa8-0285f9f62cbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:38.548279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbb1fd-0a1d-4630-8aa8-0285f9f62cbf None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:38.570292 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 619/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Aug 20 20:42:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:42:38.577402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21456442-1d5b-4248-9ea3-157d349a5a69 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:38.698221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21456442-1d5b-4248-9ea3-157d349a5a69 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:38.734089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b3753-86a9-4a43-9da4-e95f260b1233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:38.734735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b3753-86a9-4a43-9da4-e95f260b1233 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:38.735016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b3753-86a9-4a43-9da4-e95f260b1233 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:38.735287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b3753-86a9-4a43-9da4-e95f260b1233 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:38.798300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:38.813301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b3753-86a9-4a43-9da4-e95f260b1233 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:38.814781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b3753-86a9-4a43-9da4-e95f260b1233 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:38.944611 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 636/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:42:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:42:38.950612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbb1fd-0a1d-4630-8aa8-0285f9f62cbf None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:39.069838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbb1fd-0a1d-4630-8aa8-0285f9f62cbf None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:39.083742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:39.106391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1dc0b-8c02-4783-8b53-5f0539177904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:39.107017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1dc0b-8c02-4783-8b53-5f0539177904 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:39.107293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1dc0b-8c02-4783-8b53-5f0539177904 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:39.107598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1dc0b-8c02-4783-8b53-5f0539177904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:39.117878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1dc0b-8c02-4783-8b53-5f0539177904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:39.119331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1dc0b-8c02-4783-8b53-5f0539177904 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-23d1dc0b-8c02-4783-8b53-5f0539177904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:42:39.134122 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:39.135103 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:42:39.136021 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:42:39.136021 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:39.136021 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:39.136021 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 620/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:39.141239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b3753-86a9-4a43-9da4-e95f260b1233 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:39.238606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:39.251662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b3753-86a9-4a43-9da4-e95f260b1233 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:39.285833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d09d4-6b46-40e5-8b92-d23dcc8e74e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:39.286452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d09d4-6b46-40e5-8b92-d23dcc8e74e0 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:39.286725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d09d4-6b46-40e5-8b92-d23dcc8e74e0 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:39.286985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d09d4-6b46-40e5-8b92-d23dcc8e74e0 None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:39.288050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d09d4-6b46-40e5-8b92-d23dcc8e74e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:39.288432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d09d4-6b46-40e5-8b92-d23dcc8e74e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:39.289802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d09d4-6b46-40e5-8b92-d23dcc8e74e0 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:39.301643 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 637/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:39.307233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1dc0b-8c02-4783-8b53-5f0539177904 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:39.417442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1dc0b-8c02-4783-8b53-5f0539177904 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:39.462455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aee03c-7f62-442a-b658-31106409e690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:39.463066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aee03c-7f62-442a-b658-31106409e690 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:39.463327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aee03c-7f62-442a-b658-31106409e690 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:39.463594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aee03c-7f62-442a-b658-31106409e690 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:39.471380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aee03c-7f62-442a-b658-31106409e690 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:39.472837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aee03c-7f62-442a-b658-31106409e690 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-f2aee03c-7f62-442a-b658-31106409e690 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:42:39.491009 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:39.492018 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:39.493394 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:42:39.493394 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:42:39.493394 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:42:39.493394 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:42:39.493394 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:42:39.493394 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:42:39.493394 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:42:39.493394 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:39.493394 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 621/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:42:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:39.498950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d09d4-6b46-40e5-8b92-d23dcc8e74e0 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:39.614662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d09d4-6b46-40e5-8b92-d23dcc8e74e0 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:39.660803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8fbb6-50c4-437f-aaa3-aba3a2b0e21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:39.661658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8fbb6-50c4-437f-aaa3-aba3a2b0e21f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:39.662037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8fbb6-50c4-437f-aaa3-aba3a2b0e21f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:39.662575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8fbb6-50c4-437f-aaa3-aba3a2b0e21f None admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:39.663606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8fbb6-50c4-437f-aaa3-aba3a2b0e21f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:39.663908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8fbb6-50c4-437f-aaa3-aba3a2b0e21f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:39.665376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8fbb6-50c4-437f-aaa3-aba3a2b0e21f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:39.677848 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 638/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:42:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:39.685165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aee03c-7f62-442a-b658-31106409e690 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:39.713975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:39.797879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aee03c-7f62-442a-b658-31106409e690 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:39.838206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18622c-f905-4993-8dfa-5e6c4b4dc071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:39.839288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18622c-f905-4993-8dfa-5e6c4b4dc071 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:39.839288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18622c-f905-4993-8dfa-5e6c4b4dc071 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:39.839513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18622c-f905-4993-8dfa-5e6c4b4dc071 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:39.840147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18622c-f905-4993-8dfa-5e6c4b4dc071 None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=2b04f1e0e8094a14a3c1384faa601aa9) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:39.840484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18622c-f905-4993-8dfa-5e6c4b4dc071 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:39.841918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18622c-f905-4993-8dfa-5e6c4b4dc071 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:39.847976 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 622/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:42:39 2022] GET /identity/v3/role_assignments?user.id=2b04f1e0e8094a14a3c1384faa601aa9&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:42.055312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377936de-f24b-480a-ad31-5759790d7a23 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:42.056220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377936de-f24b-480a-ad31-5759790d7a23 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:42.056220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377936de-f24b-480a-ad31-5759790d7a23 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:42.058368 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 639/1261] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:42:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:42:42.064921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb55e2-70f9-4dc5-8898-3423e94a0bd0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:42.065683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb55e2-70f9-4dc5-8898-3423e94a0bd0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:42.065811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb55e2-70f9-4dc5-8898-3423e94a0bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:42.096227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fecb55e2-70f9-4dc5-8898-3423e94a0bd0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:42:42.098062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fecb55e2-70f9-4dc5-8898-3423e94a0bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:42:42.175176 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 623/1262] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:42:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:42:42.498503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377936de-f24b-480a-ad31-5759790d7a23 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:42.516627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:42.618963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377936de-f24b-480a-ad31-5759790d7a23 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:42.655301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5e77a-09fc-4031-a81b-4892366342b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:42.655917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5e77a-09fc-4031-a81b-4892366342b6 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:42.656187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5e77a-09fc-4031-a81b-4892366342b6 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:42.656476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5e77a-09fc-4031-a81b-4892366342b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:42.667376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5e77a-09fc-4031-a81b-4892366342b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-47a5e77a-09fc-4031-a81b-4892366342b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:42:42.672015 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:42.673102 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:42.674231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:42.674231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:42.674231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:42.674231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:42.674231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:42:42.674231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:42:42.674231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:42:42.674231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:42:42.674231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:42:42.674231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:42:42.674231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:42:42.674231 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:42.674231 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 640/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:42:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:42.679821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb55e2-70f9-4dc5-8898-3423e94a0bd0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:42.699348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:42.786247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb55e2-70f9-4dc5-8898-3423e94a0bd0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:42.826611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936395-90e0-4289-a42e-881b25d43786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:42.827214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936395-90e0-4289-a42e-881b25d43786 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:42.827489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936395-90e0-4289-a42e-881b25d43786 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:42.827755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936395-90e0-4289-a42e-881b25d43786 None admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:42.828564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e936395-90e0-4289-a42e-881b25d43786 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:42.828755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e936395-90e0-4289-a42e-881b25d43786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:42.831103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e936395-90e0-4289-a42e-881b25d43786 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:42.841241 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 624/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:42:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:42.847484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5e77a-09fc-4031-a81b-4892366342b6 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:42.925659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:42.960240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5e77a-09fc-4031-a81b-4892366342b6 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:42.992968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06d444-c891-44e4-be4d-5763fb976bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:42.993575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06d444-c891-44e4-be4d-5763fb976bf5 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:42.993843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06d444-c891-44e4-be4d-5763fb976bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:42.994098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06d444-c891-44e4-be4d-5763fb976bf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:43.005322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06d444-c891-44e4-be4d-5763fb976bf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:43.007041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06d444-c891-44e4-be4d-5763fb976bf5 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-8e06d444-c891-44e4-be4d-5763fb976bf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:42:43.022775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:43.026379 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:42:43.028000 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:42:43.028000 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:43.028000 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:43.028000 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 641/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:43.032082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936395-90e0-4289-a42e-881b25d43786 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:43.126668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:43.149669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936395-90e0-4289-a42e-881b25d43786 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:43.185786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166abd7-5876-4680-9a53-2d59dd5a5f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:43.186684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166abd7-5876-4680-9a53-2d59dd5a5f27 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:43.187067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166abd7-5876-4680-9a53-2d59dd5a5f27 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:43.187458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166abd7-5876-4680-9a53-2d59dd5a5f27 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:43.188421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166abd7-5876-4680-9a53-2d59dd5a5f27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:43.188890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166abd7-5876-4680-9a53-2d59dd5a5f27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:43.191029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166abd7-5876-4680-9a53-2d59dd5a5f27 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:43.210427 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 625/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:43.216400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06d444-c891-44e4-be4d-5763fb976bf5 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:43.318110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06d444-c891-44e4-be4d-5763fb976bf5 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:43.337644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:43.353399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74e9df-4d24-4152-b341-6d37ffabb3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:43.354021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74e9df-4d24-4152-b341-6d37ffabb3af None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:43.354286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74e9df-4d24-4152-b341-6d37ffabb3af None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:43.354529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74e9df-4d24-4152-b341-6d37ffabb3af None admin] PATH_INFO: `/v3/users/placement` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:43.362080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74e9df-4d24-4152-b341-6d37ffabb3af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:43.363504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74e9df-4d24-4152-b341-6d37ffabb3af None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-5a74e9df-4d24-4152-b341-6d37ffabb3af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:42:43.374514 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:43.375559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:43.376566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:43.376566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:43.376566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:42:43.376566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:42:43.376566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:42:43.376566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:42:43.376566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:42:43.376566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:42:43.376566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 20 20:42:43.376566 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:43.376566 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 642/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 20:42:43.381844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166abd7-5876-4680-9a53-2d59dd5a5f27 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:43.485354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166abd7-5876-4680-9a53-2d59dd5a5f27 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:43.521616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ae6e8-81dd-4a27-bba9-b8b00ab0a911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:43.522359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ae6e8-81dd-4a27-bba9-b8b00ab0a911 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:43.522662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ae6e8-81dd-4a27-bba9-b8b00ab0a911 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:43.522946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ae6e8-81dd-4a27-bba9-b8b00ab0a911 None admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:43.523813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ae6e8-81dd-4a27-bba9-b8b00ab0a911 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:43.524180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ae6e8-81dd-4a27-bba9-b8b00ab0a911 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:43.525852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ae6e8-81dd-4a27-bba9-b8b00ab0a911 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:43.537068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:43.546535 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 626/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Aug 20 20:42:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:43.552673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74e9df-4d24-4152-b341-6d37ffabb3af None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:43.658043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74e9df-4d24-4152-b341-6d37ffabb3af None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:43.693587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bef1d-7995-44f3-81ff-a44cb21f9f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:43.694640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bef1d-7995-44f3-81ff-a44cb21f9f25 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:43.694911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bef1d-7995-44f3-81ff-a44cb21f9f25 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:43.695176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bef1d-7995-44f3-81ff-a44cb21f9f25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:43.705963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bef1d-7995-44f3-81ff-a44cb21f9f25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:43.707396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bef1d-7995-44f3-81ff-a44cb21f9f25 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-3c6bef1d-7995-44f3-81ff-a44cb21f9f25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:42:43.721983 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:43.723105 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:42:43.723990 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:42:43.723990 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:43.723990 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:43.723990 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 643/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:43.731890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ae6e8-81dd-4a27-bba9-b8b00ab0a911 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:43.840303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ae6e8-81dd-4a27-bba9-b8b00ab0a911 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:43.877667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12318df-9d98-442b-91f2-3233c564c0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:43.878294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12318df-9d98-442b-91f2-3233c564c0fa None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:43.878565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12318df-9d98-442b-91f2-3233c564c0fa None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:43.878825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12318df-9d98-442b-91f2-3233c564c0fa None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:43.879460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12318df-9d98-442b-91f2-3233c564c0fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:43.879755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12318df-9d98-442b-91f2-3233c564c0fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:43.881155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12318df-9d98-442b-91f2-3233c564c0fa None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:43.893298 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 627/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:43.900540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bef1d-7995-44f3-81ff-a44cb21f9f25 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:43.933284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:44.019350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bef1d-7995-44f3-81ff-a44cb21f9f25 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:44.055957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d9cee-6c7a-4313-8434-90b76ef34022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:44.056583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d9cee-6c7a-4313-8434-90b76ef34022 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:44.056856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d9cee-6c7a-4313-8434-90b76ef34022 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:44.057117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d9cee-6c7a-4313-8434-90b76ef34022 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:44.064186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d9cee-6c7a-4313-8434-90b76ef34022 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:44.065667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d9cee-6c7a-4313-8434-90b76ef34022 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-275d9cee-6c7a-4313-8434-90b76ef34022 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:42:44.079856 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:44.080924 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:44.081964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:42:44.081964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:42:44.081964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:42:44.081964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:42:44.081964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:42:44.081964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:42:44.081964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:42:44.081964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:44.081964 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 644/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:42:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:44.087609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12318df-9d98-442b-91f2-3233c564c0fa None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:44.157062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:44.200894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12318df-9d98-442b-91f2-3233c564c0fa None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:44.239527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd6a17-7ede-4494-9811-52f26794c078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:44.240130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd6a17-7ede-4494-9811-52f26794c078 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:44.240405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd6a17-7ede-4494-9811-52f26794c078 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:44.240667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd6a17-7ede-4494-9811-52f26794c078 None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:44.241368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd6a17-7ede-4494-9811-52f26794c078 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:44.241665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd6a17-7ede-4494-9811-52f26794c078 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:44.243106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd6a17-7ede-4494-9811-52f26794c078 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:44.255123 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 628/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:42:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:44.262285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d9cee-6c7a-4313-8434-90b76ef34022 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:44.357485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:44.372202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d9cee-6c7a-4313-8434-90b76ef34022 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:44.410353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724acee-e86d-44fa-883c-74684cbb3706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:44.410999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724acee-e86d-44fa-883c-74684cbb3706 None admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:44.411270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724acee-e86d-44fa-883c-74684cbb3706 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:44.411543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724acee-e86d-44fa-883c-74684cbb3706 None admin] PATH_INFO: `/v3/projects/3b6c88840d764c2385aa678da4338213/users/2b04f1e0e8094a14a3c1384faa601aa9/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:44.414066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724acee-e86d-44fa-883c-74684cbb3706 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b6c88840d764c2385aa678da4338213, user_id=2b04f1e0e8094a14a3c1384faa601aa9, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:44.415501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724acee-e86d-44fa-883c-74684cbb3706 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:44.431462 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 645/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Aug 20 20:42:44 2022] PUT /identity/v3/projects/3b6c88840d764c2385aa678da4338213/users/2b04f1e0e8094a14a3c1384faa601aa9/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:42:46.590699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4499ec-4eff-4832-a4dd-1ccfcf706ee5 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:46.591692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4499ec-4eff-4832-a4dd-1ccfcf706ee5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:46.591692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4499ec-4eff-4832-a4dd-1ccfcf706ee5 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:46.593576 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 629/1274] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:42:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:42:46.600118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6db93-976f-4337-a4b4-fece4f94523c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:46.601207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6db93-976f-4337-a4b4-fece4f94523c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:46.601207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6db93-976f-4337-a4b4-fece4f94523c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:46.623334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f3c6db93-976f-4337-a4b4-fece4f94523c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:42:46.625275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f3c6db93-976f-4337-a4b4-fece4f94523c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:42:46.716858 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 646/1275] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:42:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:42:47.072616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4499ec-4eff-4832-a4dd-1ccfcf706ee5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:47.191181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4499ec-4eff-4832-a4dd-1ccfcf706ee5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:47.209802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:47.230340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9da32-e5d6-4a10-afc4-3eba051ee92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:47.230963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9da32-e5d6-4a10-afc4-3eba051ee92a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:47.231230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9da32-e5d6-4a10-afc4-3eba051ee92a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:47.231487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9da32-e5d6-4a10-afc4-3eba051ee92a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:47.242078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9da32-e5d6-4a10-afc4-3eba051ee92a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-b5a9da32-e5d6-4a10-afc4-3eba051ee92a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 20 20:42:47.246686 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:47.247859 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:47.248889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:47.248889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:47.248889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:47.248889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:47.248889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 20:42:47.248889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 20:42:47.248889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 20:42:47.248889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 20:42:47.248889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 20:42:47.248889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 20:42:47.248889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 20 20:42:47.248889 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:47.248889 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 630/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Aug 20 20:42:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:47.254292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6db93-976f-4337-a4b4-fece4f94523c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:47.341368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:47.363280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6db93-976f-4337-a4b4-fece4f94523c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:47.396662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ba007-2d4a-4aae-a8e7-fd40951c0d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:47.397313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ba007-2d4a-4aae-a8e7-fd40951c0d03 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:47.397583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ba007-2d4a-4aae-a8e7-fd40951c0d03 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:47.397845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ba007-2d4a-4aae-a8e7-fd40951c0d03 None admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:47.398492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ba007-2d4a-4aae-a8e7-fd40951c0d03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:47.398783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ba007-2d4a-4aae-a8e7-fd40951c0d03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:47.400177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ba007-2d4a-4aae-a8e7-fd40951c0d03 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:47.407882 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 647/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Aug 20 20:42:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:47.414280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9da32-e5d6-4a10-afc4-3eba051ee92a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:47.524407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9da32-e5d6-4a10-afc4-3eba051ee92a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:47.561085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c010f-21a5-404c-96cb-250f74d06832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:47.561701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c010f-21a5-404c-96cb-250f74d06832 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:47.561972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c010f-21a5-404c-96cb-250f74d06832 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:47.562238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c010f-21a5-404c-96cb-250f74d06832 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:47.573136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c010f-21a5-404c-96cb-250f74d06832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:47.574581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c010f-21a5-404c-96cb-250f74d06832 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-191c010f-21a5-404c-96cb-250f74d06832 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:42:47.588909 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:47.589997 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:42:47.591522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:42:47.591522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:47.591522 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:47.591522 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 631/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:47.596883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ba007-2d4a-4aae-a8e7-fd40951c0d03 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:47.710566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ba007-2d4a-4aae-a8e7-fd40951c0d03 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:47.744644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e942c0-91dd-49b0-a2e5-6a4830c5566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:47.745222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e942c0-91dd-49b0-a2e5-6a4830c5566f None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:47.745486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e942c0-91dd-49b0-a2e5-6a4830c5566f None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:47.745741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e942c0-91dd-49b0-a2e5-6a4830c5566f None admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:47.746362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e942c0-91dd-49b0-a2e5-6a4830c5566f None admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:47.746649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e942c0-91dd-49b0-a2e5-6a4830c5566f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:47.747996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e942c0-91dd-49b0-a2e5-6a4830c5566f None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:47.759622 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 648/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:47.765803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c010f-21a5-404c-96cb-250f74d06832 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:47.801113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:47.878501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c010f-21a5-404c-96cb-250f74d06832 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:47.918009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d6124-e431-439f-8c7e-d1099c9ab108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:47.918854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d6124-e431-439f-8c7e-d1099c9ab108 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:47.919212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d6124-e431-439f-8c7e-d1099c9ab108 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:47.919560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d6124-e431-439f-8c7e-d1099c9ab108 None admin] PATH_INFO: `/v3/users/placement` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:47.933079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d6124-e431-439f-8c7e-d1099c9ab108 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:47.935609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d6124-e431-439f-8c7e-d1099c9ab108 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-c42d6124-e431-439f-8c7e-d1099c9ab108 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 20:42:47.950807 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:47.951942 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:47.953938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:47.953938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:47.953938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 20:42:47.953938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 20:42:47.953938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 20:42:47.953938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 20:42:47.953938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 20:42:47.953938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 20:42:47.953938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 20 20:42:47.953938 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:47.953938 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 632/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:47.961467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e942c0-91dd-49b0-a2e5-6a4830c5566f None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:47.967914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:48.081035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e942c0-91dd-49b0-a2e5-6a4830c5566f None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:48.121843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e874f5b-c86f-40bb-8b45-84a916464960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:48.122446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e874f5b-c86f-40bb-8b45-84a916464960 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:48.122714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e874f5b-c86f-40bb-8b45-84a916464960 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:48.123621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e874f5b-c86f-40bb-8b45-84a916464960 None admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:48.124349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e874f5b-c86f-40bb-8b45-84a916464960 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:48.124674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e874f5b-c86f-40bb-8b45-84a916464960 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:48.126098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e874f5b-c86f-40bb-8b45-84a916464960 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:48.145124 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 649/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Aug 20 20:42:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:48.151297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d6124-e431-439f-8c7e-d1099c9ab108 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:48.245502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:48.255891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d6124-e431-439f-8c7e-d1099c9ab108 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:48.290523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1fe99-3464-4bfb-b207-5bf6b0080419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:48.291106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1fe99-3464-4bfb-b207-5bf6b0080419 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:48.291380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1fe99-3464-4bfb-b207-5bf6b0080419 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:48.291642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1fe99-3464-4bfb-b207-5bf6b0080419 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:48.359530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1fe99-3464-4bfb-b207-5bf6b0080419 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:48.360995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1fe99-3464-4bfb-b207-5bf6b0080419 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:48.460905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:48.472974 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 633/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Aug 20 20:42:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 20 20:42:48.479211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e874f5b-c86f-40bb-8b45-84a916464960 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:48.550953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:48.592096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e874f5b-c86f-40bb-8b45-84a916464960 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:48.630172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b5f63-3d9b-439f-a311-5c1c468ab777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:48.630786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b5f63-3d9b-439f-a311-5c1c468ab777 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:48.631052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b5f63-3d9b-439f-a311-5c1c468ab777 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:48.631500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b5f63-3d9b-439f-a311-5c1c468ab777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:48.641512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b5f63-3d9b-439f-a311-5c1c468ab777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:48.642948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b5f63-3d9b-439f-a311-5c1c468ab777 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-968b5f63-3d9b-439f-a311-5c1c468ab777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 20:42:48.657618 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:48.658672 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 20 20:42:48.659599 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 20:42:48.659599 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 20 20:42:48.659599 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:48.659599 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 650/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:48.665371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1fe99-3464-4bfb-b207-5bf6b0080419 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:48.765058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1fe99-3464-4bfb-b207-5bf6b0080419 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:48.800408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7f658-bc7e-4d15-b4b6-706bbbb77962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:48.801347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7f658-bc7e-4d15-b4b6-706bbbb77962 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:48.801726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7f658-bc7e-4d15-b4b6-706bbbb77962 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:48.802107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7f658-bc7e-4d15-b4b6-706bbbb77962 None admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:48.803452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7f658-bc7e-4d15-b4b6-706bbbb77962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:48.804011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7f658-bc7e-4d15-b4b6-706bbbb77962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:48.806388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7f658-bc7e-4d15-b4b6-706bbbb77962 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:48.824576 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 634/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Aug 20 20:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:48.831099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b5f63-3d9b-439f-a311-5c1c468ab777 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:48.950174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b5f63-3d9b-439f-a311-5c1c468ab777 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:48.964641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:48.988217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9481e-5920-4b40-a288-bb0b4577c3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:48.989053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9481e-5920-4b40-a288-bb0b4577c3b8 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:48.989330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9481e-5920-4b40-a288-bb0b4577c3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:48.989597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9481e-5920-4b40-a288-bb0b4577c3b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:48.997337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9481e-5920-4b40-a288-bb0b4577c3b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:48.998783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9481e-5920-4b40-a288-bb0b4577c3b8 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-c5d9481e-5920-4b40-a288-bb0b4577c3b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 20 20:42:49.010433 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:49.011536 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:49.012695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 20 20:42:49.012695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 20 20:42:49.012695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 20:42:49.012695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 20:42:49.012695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 20:42:49.012695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 20:42:49.012695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 20 20:42:49.012695 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 20:42:49.012695 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 651/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Aug 20 20:42:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:49.017762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7f658-bc7e-4d15-b4b6-706bbbb77962 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:49.071440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:49.135369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7f658-bc7e-4d15-b4b6-706bbbb77962 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:49.176838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f314a7-3ae6-4633-a56a-c10b09ec0a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:49.177416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f314a7-3ae6-4633-a56a-c10b09ec0a6f None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:49.177685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f314a7-3ae6-4633-a56a-c10b09ec0a6f None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:49.177958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f314a7-3ae6-4633-a56a-c10b09ec0a6f None admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:49.178675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f314a7-3ae6-4633-a56a-c10b09ec0a6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:49.178970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f314a7-3ae6-4633-a56a-c10b09ec0a6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:49.180401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f314a7-3ae6-4633-a56a-c10b09ec0a6f None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:49.192744 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 635/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Aug 20 20:42:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:49.202548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9481e-5920-4b40-a288-bb0b4577c3b8 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:49.312891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9481e-5920-4b40-a288-bb0b4577c3b8 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:49.355663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43729cb-bb2e-4c7d-8986-e839b5c0839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:49.356264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43729cb-bb2e-4c7d-8986-e839b5c0839d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:49.356575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43729cb-bb2e-4c7d-8986-e839b5c0839d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:49.356837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43729cb-bb2e-4c7d-8986-e839b5c0839d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:49.357540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43729cb-bb2e-4c7d-8986-e839b5c0839d None admin] RBAC: Adding query filter params (role.id=1e7f1276e60442b0bb784f6e1d0a4647, scope.project.id=3b6c88840d764c2385aa678da4338213, user.id=2b04f1e0e8094a14a3c1384faa601aa9) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:49.357840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43729cb-bb2e-4c7d-8986-e839b5c0839d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:49.359266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43729cb-bb2e-4c7d-8986-e839b5c0839d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:49.365450 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 652/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Aug 20 20:42:49 2022] GET /identity/v3/role_assignments?user.id=2b04f1e0e8094a14a3c1384faa601aa9&scope.project.id=3b6c88840d764c2385aa678da4338213&role.id=1e7f1276e60442b0bb784f6e1d0a4647 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:51.560020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c865df5-95a9-4888-acdc-f8ea877a8c90 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:51.561223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c865df5-95a9-4888-acdc-f8ea877a8c90 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:51.561223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c865df5-95a9-4888-acdc-f8ea877a8c90 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:51.563199 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 636/1288] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:42:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:42:51.572601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51e890-43d3-477e-a0dc-190c53e47bc7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:51.573016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51e890-43d3-477e-a0dc-190c53e47bc7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:51.573149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51e890-43d3-477e-a0dc-190c53e47bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:51.602418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7b51e890-43d3-477e-a0dc-190c53e47bc7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:42:51.603622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7b51e890-43d3-477e-a0dc-190c53e47bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:42:51.634842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:51.675870 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 653/1289] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:42:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:42:52.008640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c865df5-95a9-4888-acdc-f8ea877a8c90 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:52.121361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c865df5-95a9-4888-acdc-f8ea877a8c90 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:52.132556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:52.158531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d630b-d2ff-4840-93de-1e60db2c05d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:52.159155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d630b-d2ff-4840-93de-1e60db2c05d9 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:52.159426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d630b-d2ff-4840-93de-1e60db2c05d9 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:52.159691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d630b-d2ff-4840-93de-1e60db2c05d9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:52.160269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d630b-d2ff-4840-93de-1e60db2c05d9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:52.161664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d630b-d2ff-4840-93de-1e60db2c05d9 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-7b2d630b-d2ff-4840-93de-1e60db2c05d9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:52.171039 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:52.172201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:52.173150 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 637/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Aug 20 20:42:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:52.178670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51e890-43d3-477e-a0dc-190c53e47bc7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:52.288983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51e890-43d3-477e-a0dc-190c53e47bc7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:52.333648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206ce71-fd58-491e-a405-e40ef31fa396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:52.334243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206ce71-fd58-491e-a405-e40ef31fa396 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:52.334527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206ce71-fd58-491e-a405-e40ef31fa396 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:52.334813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206ce71-fd58-491e-a405-e40ef31fa396 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:52.335573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206ce71-fd58-491e-a405-e40ef31fa396 None admin] RBAC: Adding query filter params (name=placement) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:52.335864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206ce71-fd58-491e-a405-e40ef31fa396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:52.337262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206ce71-fd58-491e-a405-e40ef31fa396 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:52.342193 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 654/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Sat Aug 20 20:42:52 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:52.348399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d630b-d2ff-4840-93de-1e60db2c05d9 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:52.450809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d630b-d2ff-4840-93de-1e60db2c05d9 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:52.483685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f12eb2-090d-4ad5-9cd6-8bfb887cba79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:52.484263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f12eb2-090d-4ad5-9cd6-8bfb887cba79 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:52.484567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f12eb2-090d-4ad5-9cd6-8bfb887cba79 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:52.484841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f12eb2-090d-4ad5-9cd6-8bfb887cba79 None admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:52.485448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f12eb2-090d-4ad5-9cd6-8bfb887cba79 None admin] RBAC: Adding query filter params (type=placement) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:52.485747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f12eb2-090d-4ad5-9cd6-8bfb887cba79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:52.487101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f12eb2-090d-4ad5-9cd6-8bfb887cba79 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:52.491814 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 638/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Sat Aug 20 20:42:52 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:42:54.665832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d22503-8548-4d56-9de0-a32df6496421 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:54.666386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d22503-8548-4d56-9de0-a32df6496421 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:54.666386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d22503-8548-4d56-9de0-a32df6496421 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:54.667623 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 655/1293] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:42:54.674097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86c23a-a0ac-45d5-87f1-aa1847aeb344 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:54.674793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86c23a-a0ac-45d5-87f1-aa1847aeb344 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:54.674904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86c23a-a0ac-45d5-87f1-aa1847aeb344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:54.701790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7e86c23a-a0ac-45d5-87f1-aa1847aeb344 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:42:54.702988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7e86c23a-a0ac-45d5-87f1-aa1847aeb344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:42:54.734164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:54.775464 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 639/1294] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:42:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:42:55.124902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d22503-8548-4d56-9de0-a32df6496421 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:55.166271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:55.253602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d22503-8548-4d56-9de0-a32df6496421 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:55.296240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b2df6-c05c-4db3-b0fc-0b2f43d2fccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:55.296863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b2df6-c05c-4db3-b0fc-0b2f43d2fccb None admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:55.297129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b2df6-c05c-4db3-b0fc-0b2f43d2fccb None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:55.297388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b2df6-c05c-4db3-b0fc-0b2f43d2fccb None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:55.298018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b2df6-c05c-4db3-b0fc-0b2f43d2fccb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:55.299381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b2df6-c05c-4db3-b0fc-0b2f43d2fccb None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:55.318121 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 656/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Aug 20 20:42:55 2022] POST /identity/v3/services => generated 246 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:42:57.508198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a0677-6836-4c0c-96fd-72254b6854e4 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:57.508800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a0677-6836-4c0c-96fd-72254b6854e4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:57.508800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a0677-6836-4c0c-96fd-72254b6854e4 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:57.510013 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 640/1296] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:42:57.516433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab20f86-e4dd-4824-9d69-3ac965650854 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:57.517141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab20f86-e4dd-4824-9d69-3ac965650854 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:57.517321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab20f86-e4dd-4824-9d69-3ac965650854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:57.538823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dab20f86-e4dd-4824-9d69-3ac965650854 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:42:57.539991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dab20f86-e4dd-4824-9d69-3ac965650854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:42:57.615203 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 657/1297] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:42:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:42:57.949832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a0677-6836-4c0c-96fd-72254b6854e4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:58.059656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a0677-6836-4c0c-96fd-72254b6854e4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:58.098193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635326e8-d051-4079-849b-ecd223e5cc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:58.098814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635326e8-d051-4079-849b-ecd223e5cc6a None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:58.099080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635326e8-d051-4079-849b-ecd223e5cc6a None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:58.099344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635326e8-d051-4079-849b-ecd223e5cc6a None admin] PATH_INFO: `/v3/services/placement` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:58.099922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635326e8-d051-4079-849b-ecd223e5cc6a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:58.101331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635326e8-d051-4079-849b-ecd223e5cc6a None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-635326e8-d051-4079-849b-ecd223e5cc6a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:42:58.110180 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:42:58.111299 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:42:58.112210 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 641/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Aug 20 20:42:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:42:58.118648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab20f86-e4dd-4824-9d69-3ac965650854 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:58.141679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:58.233287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab20f86-e4dd-4824-9d69-3ac965650854 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:58.273825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db044183-5996-4b86-9565-7233f8e7cd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:58.274418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db044183-5996-4b86-9565-7233f8e7cd57 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:58.274681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db044183-5996-4b86-9565-7233f8e7cd57 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:58.274943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db044183-5996-4b86-9565-7233f8e7cd57 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:58.275559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db044183-5996-4b86-9565-7233f8e7cd57 None admin] RBAC: Adding query filter params (name=placement) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:58.275835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db044183-5996-4b86-9565-7233f8e7cd57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:58.277230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db044183-5996-4b86-9565-7233f8e7cd57 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:58.282598 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 658/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Sat Aug 20 20:42:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:58.289444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635326e8-d051-4079-849b-ecd223e5cc6a None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:58.323706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:58.410649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635326e8-d051-4079-849b-ecd223e5cc6a None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:58.449159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b32a41-3b3d-4799-af1a-5d188eb6435d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:58.449750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b32a41-3b3d-4799-af1a-5d188eb6435d None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:58.450025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b32a41-3b3d-4799-af1a-5d188eb6435d None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:58.450290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b32a41-3b3d-4799-af1a-5d188eb6435d None admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:58.450980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b32a41-3b3d-4799-af1a-5d188eb6435d None admin] RBAC: Adding query filter params (interface=public, service_id=99f2ea27694441eeaa8e082fe929f26d, region_id=RegionOne) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:42:58.451303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b32a41-3b3d-4799-af1a-5d188eb6435d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:58.452683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b32a41-3b3d-4799-af1a-5d188eb6435d None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:58.458009 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 642/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Aug 20 20:42:58 2022] GET /identity/v3/endpoints?service_id=99f2ea27694441eeaa8e082fe929f26d&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:42:58.464484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db044183-5996-4b86-9565-7233f8e7cd57 None admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:42:58.551559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:42:58.570160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db044183-5996-4b86-9565-7233f8e7cd57 None admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:42:58.603565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95692304-5638-4dc9-9b8e-6e2dff175440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:42:58.604145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95692304-5638-4dc9-9b8e-6e2dff175440 None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:42:58.604417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95692304-5638-4dc9-9b8e-6e2dff175440 None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:42:58.604694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95692304-5638-4dc9-9b8e-6e2dff175440 None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:42:58.605281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95692304-5638-4dc9-9b8e-6e2dff175440 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:42:58.606630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95692304-5638-4dc9-9b8e-6e2dff175440 None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:42:58.613875 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 659/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Aug 20 20:42:58 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:43:00.780099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f619e-13ad-42e3-a552-b9d9388246ce None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:43:00.780671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f619e-13ad-42e3-a552-b9d9388246ce None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:43:00.780671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f619e-13ad-42e3-a552-b9d9388246ce None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:43:00.782006 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 643/1302] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:43:00.788969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e133f-0b85-4ce6-88fc-2beb063d838e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:43:00.789719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e133f-0b85-4ce6-88fc-2beb063d838e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:43:00.789854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e133f-0b85-4ce6-88fc-2beb063d838e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:43:00.820161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-260e133f-0b85-4ce6-88fc-2beb063d838e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:43:00.821367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-260e133f-0b85-4ce6-88fc-2beb063d838e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:43:00.890265 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 660/1303] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:43:00 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 20:43:01.230086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f619e-13ad-42e3-a552-b9d9388246ce None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:43:01.330860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:43:01.339205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f619e-13ad-42e3-a552-b9d9388246ce None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:43:01.377596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5961a-6fcb-4007-b865-246a2fa0f9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:43:01.378203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5961a-6fcb-4007-b865-246a2fa0f9a7 None admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:43:01.378471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5961a-6fcb-4007-b865-246a2fa0f9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:43:01.378731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5961a-6fcb-4007-b865-246a2fa0f9a7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:43:01.379294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5961a-6fcb-4007-b865-246a2fa0f9a7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:43:01.380737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5961a-6fcb-4007-b865-246a2fa0f9a7 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-dca5961a-6fcb-4007-b865-246a2fa0f9a7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 20:43:01.389506 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 20:43:01.390651 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 20:43:01.391987 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 644/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Aug 20 20:43:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 20:43:01.399880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e133f-0b85-4ce6-88fc-2beb063d838e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:43:01.488655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:43:01.522503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e133f-0b85-4ce6-88fc-2beb063d838e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:43:01.563717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16054f3-1336-4783-addf-abaa58cf4b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:43:01.564655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16054f3-1336-4783-addf-abaa58cf4b5d None admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:43:01.565057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16054f3-1336-4783-addf-abaa58cf4b5d None admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:43:01.565459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16054f3-1336-4783-addf-abaa58cf4b5d None admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:43:01.566434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16054f3-1336-4783-addf-abaa58cf4b5d None admin] RBAC: Adding query filter params (name=placement) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:43:01.566891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16054f3-1336-4783-addf-abaa58cf4b5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:43:01.569356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16054f3-1336-4783-addf-abaa58cf4b5d None admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:43:01.575549 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 661/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Sat Aug 20 20:43:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:43:01.581533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5961a-6fcb-4007-b865-246a2fa0f9a7 None admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:43:01.695920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5961a-6fcb-4007-b865-246a2fa0f9a7 None admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:43:01.732052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d190d-2acb-42a6-9ca8-476e62d93478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:43:01.732680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d190d-2acb-42a6-9ca8-476e62d93478 None admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:43:01.732948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d190d-2acb-42a6-9ca8-476e62d93478 None admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:43:01.733215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d190d-2acb-42a6-9ca8-476e62d93478 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:43:01.733875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d190d-2acb-42a6-9ca8-476e62d93478 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:43:01.735232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d190d-2acb-42a6-9ca8-476e62d93478 None admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:43:01.759798 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 645/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:43:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:43:11.571895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=30 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:43:11.754010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:15.599188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970358fc-0d0f-45ee-a16f-d1696ebff71f None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:15.600613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970358fc-0d0f-45ee-a16f-d1696ebff71f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:15.601110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970358fc-0d0f-45ee-a16f-d1696ebff71f None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:15.609581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfed554-fd62-45b2-a851-8689c16e0787 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:15.609979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfed554-fd62-45b2-a851-8689c16e0787 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:15.610230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfed554-fd62-45b2-a851-8689c16e0787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:15.610444 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 662/1307] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:44:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:15.730526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5cfed554-fd62-45b2-a851-8689c16e0787 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:15.731965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5cfed554-fd62-45b2-a851-8689c16e0787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:15.884119 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 646/1308] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:44:16.240032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d9fde-a8e1-455d-929b-583d7fde4715 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:16.240336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d9fde-a8e1-455d-929b-583d7fde4715 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:16.240609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d9fde-a8e1-455d-929b-583d7fde4715 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:16.242464 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 663/1309] 10.222.0.23 () {54 vars in 965 bytes} [Sat Aug 20 20:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:16.248952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff96ed-b3df-4b20-83cf-6d42d19c550b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:16.249249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff96ed-b3df-4b20-83cf-6d42d19c550b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:16.249532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff96ed-b3df-4b20-83cf-6d42d19c550b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:16.332436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f4ff96ed-b3df-4b20-83cf-6d42d19c550b None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:16.333675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f4ff96ed-b3df-4b20-83cf-6d42d19c550b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:16.355156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:16.417835 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 647/1310] 10.222.0.23 () {58 vars in 1062 bytes} [Sat Aug 20 20:44:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:44:16.426597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d9fde-a8e1-455d-929b-583d7fde4715 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:44:16.428355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-095d9fde-a8e1-455d-929b-583d7fde4715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:16.542698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d9fde-a8e1-455d-929b-583d7fde4715 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:44:16.632310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eecca1-27ab-4dd4-8d9d-73a37cd58b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:44:16.633770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eecca1-27ab-4dd4-8d9d-73a37cd58b35 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:16.634255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eecca1-27ab-4dd4-8d9d-73a37cd58b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:16.634813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eecca1-27ab-4dd4-8d9d-73a37cd58b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:16.636800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e5eecca1-27ab-4dd4-8d9d-73a37cd58b35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:16.697917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:16.760089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eecca1-27ab-4dd4-8d9d-73a37cd58b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:44:16.761816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eecca1-27ab-4dd4-8d9d-73a37cd58b35 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:44:16.802030 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 664/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:44:19.370796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31fd3d-0a7c-445e-843b-e3421afec168 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:19.371407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31fd3d-0a7c-445e-843b-e3421afec168 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:19.371407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31fd3d-0a7c-445e-843b-e3421afec168 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:19.372878 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 648/1312] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:19.378002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a6d59-8074-46b6-b7e8-fc1819f36438 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:19.378442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a6d59-8074-46b6-b7e8-fc1819f36438 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:19.378533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a6d59-8074-46b6-b7e8-fc1819f36438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:19.452932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-680a6d59-8074-46b6-b7e8-fc1819f36438 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:19.454131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-680a6d59-8074-46b6-b7e8-fc1819f36438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:19.567712 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 665/1313] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:44:19.906619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31fd3d-0a7c-445e-843b-e3421afec168 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:44:19.908175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9f31fd3d-0a7c-445e-843b-e3421afec168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:20.018133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31fd3d-0a7c-445e-843b-e3421afec168 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:44:20.093105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:20.095848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649560c-0b7d-48f6-8867-5e53c529dada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:44:20.096512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649560c-0b7d-48f6-8867-5e53c529dada service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:20.096780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649560c-0b7d-48f6-8867-5e53c529dada service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:20.097068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649560c-0b7d-48f6-8867-5e53c529dada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:20.099167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6649560c-0b7d-48f6-8867-5e53c529dada service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:20.214606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649560c-0b7d-48f6-8867-5e53c529dada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:44:20.215973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649560c-0b7d-48f6-8867-5e53c529dada service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:44:20.237007 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 649/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:44:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:44:22.741286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88710f51-03ca-4945-9bfc-4c22d2c89b6d None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:22.741870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88710f51-03ca-4945-9bfc-4c22d2c89b6d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:22.741870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88710f51-03ca-4945-9bfc-4c22d2c89b6d None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:22.743191 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 666/1315] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:22.749957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7de48-0d9b-4594-a13e-2147a37ab9bb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:22.750699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7de48-0d9b-4594-a13e-2147a37ab9bb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:22.750885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7de48-0d9b-4594-a13e-2147a37ab9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:22.833753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-09c7de48-0d9b-4594-a13e-2147a37ab9bb None None] MFA Rules not processed for user `99df9e441d1e495695098147ca4290d9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:22.834980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-09c7de48-0d9b-4594-a13e-2147a37ab9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:22.904395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:22.952548 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 650/1316] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:44:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:44:23.302171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88710f51-03ca-4945-9bfc-4c22d2c89b6d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:44:23.304689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-88710f51-03ca-4945-9bfc-4c22d2c89b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:23.322515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:23.422145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88710f51-03ca-4945-9bfc-4c22d2c89b6d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:44:23.500480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36466748-e7cd-440c-adc4-0777f0f2c5e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:44:23.501016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36466748-e7cd-440c-adc4-0777f0f2c5e6 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:23.501283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36466748-e7cd-440c-adc4-0777f0f2c5e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:23.501542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36466748-e7cd-440c-adc4-0777f0f2c5e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:23.503147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-36466748-e7cd-440c-adc4-0777f0f2c5e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:23.568823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:23.624241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36466748-e7cd-440c-adc4-0777f0f2c5e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:44:23.625723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36466748-e7cd-440c-adc4-0777f0f2c5e6 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:44:23.647075 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 667/1317] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:44:23 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:44:26.382692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9b75a-fd7a-436d-b984-7e6fa9fbdf7c None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:26.383609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9b75a-fd7a-436d-b984-7e6fa9fbdf7c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:26.383609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9b75a-fd7a-436d-b984-7e6fa9fbdf7c None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:26.385674 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 651/1318] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:44:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:26.394780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9713b4b-469c-43bf-b2a4-9345b8286b27 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:26.395468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9713b4b-469c-43bf-b2a4-9345b8286b27 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:26.395641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9713b4b-469c-43bf-b2a4-9345b8286b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:26.467187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b9713b4b-469c-43bf-b2a4-9345b8286b27 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:26.468398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b9713b4b-469c-43bf-b2a4-9345b8286b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:26.574587 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 668/1319] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:44:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:44:26.941650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9b75a-fd7a-436d-b984-7e6fa9fbdf7c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:44:26.944135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a9b9b75a-fd7a-436d-b984-7e6fa9fbdf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:27.054875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9b75a-fd7a-436d-b984-7e6fa9fbdf7c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:44:27.131902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee1c5c-bb4e-45be-9e50-6bebf08d1b5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:44:27.132548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee1c5c-bb4e-45be-9e50-6bebf08d1b5d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:27.132815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee1c5c-bb4e-45be-9e50-6bebf08d1b5d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:27.133068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee1c5c-bb4e-45be-9e50-6bebf08d1b5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:27.134726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5aee1c5c-bb4e-45be-9e50-6bebf08d1b5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:27.142507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:27.275164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee1c5c-bb4e-45be-9e50-6bebf08d1b5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:44:27.276863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee1c5c-bb4e-45be-9e50-6bebf08d1b5d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:44:27.303615 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 652/1320] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:44:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:44:30.340929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361f712-8e63-47f6-81c7-67a9246f4569 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:30.341484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361f712-8e63-47f6-81c7-67a9246f4569 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:30.341484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361f712-8e63-47f6-81c7-67a9246f4569 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:30.342896 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 669/1321] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:30.348253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae853a9-4947-4352-817e-36b76af9ae54 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:30.348765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae853a9-4947-4352-817e-36b76af9ae54 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:30.348860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae853a9-4947-4352-817e-36b76af9ae54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:30.416802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-eae853a9-4947-4352-817e-36b76af9ae54 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:30.418028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-eae853a9-4947-4352-817e-36b76af9ae54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:30.480739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:30.520865 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 653/1322] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:44:30.875903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9351046-c576-47a0-93c7-d5e6dcde7017 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:30.876131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9351046-c576-47a0-93c7-d5e6dcde7017 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:30.876499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9351046-c576-47a0-93c7-d5e6dcde7017 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:30.877863 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 670/1323] 10.222.0.23 () {54 vars in 965 bytes} [Sat Aug 20 20:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:30.884430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e05f01-bcfd-4bd6-9e22-59f912498ad5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:30.885285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e05f01-bcfd-4bd6-9e22-59f912498ad5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:30.886036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e05f01-bcfd-4bd6-9e22-59f912498ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:30.971982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-48e05f01-bcfd-4bd6-9e22-59f912498ad5 None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:30.973230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-48e05f01-bcfd-4bd6-9e22-59f912498ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:31.057822 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 654/1324] 10.222.0.23 () {58 vars in 1062 bytes} [Sat Aug 20 20:44:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:44:31.067435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9351046-c576-47a0-93c7-d5e6dcde7017 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:44:31.069021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b9351046-c576-47a0-93c7-d5e6dcde7017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:31.089963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:31.187896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9351046-c576-47a0-93c7-d5e6dcde7017 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:44:31.272919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c228411-1b03-4a45-9873-59466dad6d9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:44:31.273526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c228411-1b03-4a45-9873-59466dad6d9a service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:31.273794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c228411-1b03-4a45-9873-59466dad6d9a service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:31.274061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c228411-1b03-4a45-9873-59466dad6d9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:31.275843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3c228411-1b03-4a45-9873-59466dad6d9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:31.365413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:31.408849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c228411-1b03-4a45-9873-59466dad6d9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:44:31.410376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c228411-1b03-4a45-9873-59466dad6d9a service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:44:31.430961 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 671/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:44:33.969077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5292a-8826-48ea-9a55-f9c0ae32ef21 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:33.969627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5292a-8826-48ea-9a55-f9c0ae32ef21 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:33.969627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5292a-8826-48ea-9a55-f9c0ae32ef21 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:33.970854 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 655/1326] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:33.976481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df4b23-1edf-4a6f-9685-d61ca3a19f35 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:33.976922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df4b23-1edf-4a6f-9685-d61ca3a19f35 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:33.977015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df4b23-1edf-4a6f-9685-d61ca3a19f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:34.048584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f0df4b23-1edf-4a6f-9685-d61ca3a19f35 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:34.049797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f0df4b23-1edf-4a6f-9685-d61ca3a19f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:34.150708 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 672/1327] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:44:34.492066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5292a-8826-48ea-9a55-f9c0ae32ef21 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:44:34.493678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-23e5292a-8826-48ea-9a55-f9c0ae32ef21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:34.558163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:34.602030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5292a-8826-48ea-9a55-f9c0ae32ef21 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:44:34.679012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7143eb7-e3b5-4007-ad0e-4e63229e413d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:44:34.679609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7143eb7-e3b5-4007-ad0e-4e63229e413d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:34.679869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7143eb7-e3b5-4007-ad0e-4e63229e413d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:34.680123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7143eb7-e3b5-4007-ad0e-4e63229e413d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:34.682196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d7143eb7-e3b5-4007-ad0e-4e63229e413d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:34.800869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:34.803666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7143eb7-e3b5-4007-ad0e-4e63229e413d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:44:34.805184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7143eb7-e3b5-4007-ad0e-4e63229e413d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:44:34.827343 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 656/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:44:37.065370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746b320-1242-4242-a1da-d112e93ab1f2 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:37.065974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746b320-1242-4242-a1da-d112e93ab1f2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:37.065974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746b320-1242-4242-a1da-d112e93ab1f2 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:37.067411 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 673/1329] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:37.073015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c5fe5-87b0-4e70-80aa-ab2b64189318 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:37.073471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c5fe5-87b0-4e70-80aa-ab2b64189318 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:37.073587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c5fe5-87b0-4e70-80aa-ab2b64189318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:37.140222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-603c5fe5-87b0-4e70-80aa-ab2b64189318 None None] MFA Rules not processed for user `99df9e441d1e495695098147ca4290d9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:37.141439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-603c5fe5-87b0-4e70-80aa-ab2b64189318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:37.242655 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 657/1330] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:44:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:44:37.584701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746b320-1242-4242-a1da-d112e93ab1f2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:44:37.586287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b746b320-1242-4242-a1da-d112e93ab1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:37.620778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:37.694881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746b320-1242-4242-a1da-d112e93ab1f2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:44:37.771118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653700e-d910-4e35-ad3c-c2a39fda65f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:44:37.771720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653700e-d910-4e35-ad3c-c2a39fda65f2 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:37.771993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653700e-d910-4e35-ad3c-c2a39fda65f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:37.772238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653700e-d910-4e35-ad3c-c2a39fda65f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:37.773857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3653700e-d910-4e35-ad3c-c2a39fda65f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:37.850856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:37.882785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653700e-d910-4e35-ad3c-c2a39fda65f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:44:37.884390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653700e-d910-4e35-ad3c-c2a39fda65f2 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:44:37.902412 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 674/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:44:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:44:40.537729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8939b0-714c-45d0-b0b7-b9b3294265bf None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:40.538277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8939b0-714c-45d0-b0b7-b9b3294265bf None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:40.538277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8939b0-714c-45d0-b0b7-b9b3294265bf None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:40.539628 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 658/1332] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:40.545538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf99ac9-a02b-439e-8e48-cd2cec15bb01 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:40.546037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf99ac9-a02b-439e-8e48-cd2cec15bb01 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:40.546037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf99ac9-a02b-439e-8e48-cd2cec15bb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:40.616912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bbf99ac9-a02b-439e-8e48-cd2cec15bb01 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:40.618114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bbf99ac9-a02b-439e-8e48-cd2cec15bb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:40.720148 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 675/1333] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:44:41.059573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8939b0-714c-45d0-b0b7-b9b3294265bf None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:44:41.061240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ea8939b0-714c-45d0-b0b7-b9b3294265bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:41.134918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:41.170312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8939b0-714c-45d0-b0b7-b9b3294265bf None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:44:41.250849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe61fa-c657-4e54-be4b-14ac12f8ae86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:44:41.251466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe61fa-c657-4e54-be4b-14ac12f8ae86 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:41.251739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe61fa-c657-4e54-be4b-14ac12f8ae86 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:41.252057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe61fa-c657-4e54-be4b-14ac12f8ae86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:41.253760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7fbe61fa-c657-4e54-be4b-14ac12f8ae86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:41.370192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe61fa-c657-4e54-be4b-14ac12f8ae86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:44:41.371552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe61fa-c657-4e54-be4b-14ac12f8ae86 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:44:41.383523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:41.393344 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 659/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:44:43.626644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326669a-8124-4992-9b85-4c5c8fc22f64 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:43.627205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326669a-8124-4992-9b85-4c5c8fc22f64 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:43.627205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326669a-8124-4992-9b85-4c5c8fc22f64 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:43.628428 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 676/1335] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:43.634773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb566e-74c2-42d8-909d-dda4a985e542 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:43.635705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb566e-74c2-42d8-909d-dda4a985e542 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:43.635705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb566e-74c2-42d8-909d-dda4a985e542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:43.714436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f4cb566e-74c2-42d8-909d-dda4a985e542 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:43.715650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f4cb566e-74c2-42d8-909d-dda4a985e542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:43.829898 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 660/1336] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:44:44.176362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326669a-8124-4992-9b85-4c5c8fc22f64 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:44:44.178872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6326669a-8124-4992-9b85-4c5c8fc22f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:44.230603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:44.306129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326669a-8124-4992-9b85-4c5c8fc22f64 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:44:44.388301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73897c19-fd41-43f3-8bb2-954c7716cee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:44:44.388928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73897c19-fd41-43f3-8bb2-954c7716cee0 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:44.389194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73897c19-fd41-43f3-8bb2-954c7716cee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:44.389457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73897c19-fd41-43f3-8bb2-954c7716cee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:44.391046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-73897c19-fd41-43f3-8bb2-954c7716cee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:44.478101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:44.505250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73897c19-fd41-43f3-8bb2-954c7716cee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:44:44.506719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73897c19-fd41-43f3-8bb2-954c7716cee0 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:44:44.525275 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 677/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:44:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:44:47.234018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610297c-2347-4d09-8bf7-4e5ff950c09f None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:47.234459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610297c-2347-4d09-8bf7-4e5ff950c09f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:47.234567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610297c-2347-4d09-8bf7-4e5ff950c09f None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:47.235920 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 661/1338] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:47.241441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5904f-1ed8-46f6-9bb9-d9fcbe245bb4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:47.241869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5904f-1ed8-46f6-9bb9-d9fcbe245bb4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:47.241982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5904f-1ed8-46f6-9bb9-d9fcbe245bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:47.309213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f3a5904f-1ed8-46f6-9bb9-d9fcbe245bb4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:47.310383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f3a5904f-1ed8-46f6-9bb9-d9fcbe245bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:47.419144 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 678/1339] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:44:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:44:47.760494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610297c-2347-4d09-8bf7-4e5ff950c09f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:44:47.762107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1610297c-2347-4d09-8bf7-4e5ff950c09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:47.842056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:47.872268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610297c-2347-4d09-8bf7-4e5ff950c09f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:44:47.949405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c0648-4611-4bda-9afb-c53e2750e0c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:44:47.949919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c0648-4611-4bda-9afb-c53e2750e0c5 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:47.950184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c0648-4611-4bda-9afb-c53e2750e0c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:47.950443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c0648-4611-4bda-9afb-c53e2750e0c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:47.952015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-538c0648-4611-4bda-9afb-c53e2750e0c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:48.073097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c0648-4611-4bda-9afb-c53e2750e0c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:44:48.074539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c0648-4611-4bda-9afb-c53e2750e0c5 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:44:48.094572 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 662/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:44:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:44:52.338012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e077e8-6e1b-47ea-b2ac-2b4518f7a2d9 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:52.338592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e077e8-6e1b-47ea-b2ac-2b4518f7a2d9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:52.338592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e077e8-6e1b-47ea-b2ac-2b4518f7a2d9 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:52.340055 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 679/1341] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:52.345485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c50f9a-d364-4da5-b835-1b499524d92f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:52.345935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c50f9a-d364-4da5-b835-1b499524d92f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:52.346040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c50f9a-d364-4da5-b835-1b499524d92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:52.351825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:52.415076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-30c50f9a-d364-4da5-b835-1b499524d92f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:52.416270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-30c50f9a-d364-4da5-b835-1b499524d92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:52.517134 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 663/1342] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:44:52.861450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e077e8-6e1b-47ea-b2ac-2b4518f7a2d9 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:44:52.863070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-67e077e8-6e1b-47ea-b2ac-2b4518f7a2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:52.916753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:52.982273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e077e8-6e1b-47ea-b2ac-2b4518f7a2d9 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:44:53.063218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba04465-e6e8-4b8a-b337-ae3f91eb0d4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:44:53.063885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba04465-e6e8-4b8a-b337-ae3f91eb0d4f service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:53.064163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba04465-e6e8-4b8a-b337-ae3f91eb0d4f service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:53.064429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba04465-e6e8-4b8a-b337-ae3f91eb0d4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:53.066513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dba04465-e6e8-4b8a-b337-ae3f91eb0d4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:53.167459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:53.186074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba04465-e6e8-4b8a-b337-ae3f91eb0d4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:44:53.187519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba04465-e6e8-4b8a-b337-ae3f91eb0d4f service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:44:53.207299 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 680/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:44:56.049310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83819d0b-87c8-42a7-afcc-eb75d6fa7ec0 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:56.049923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83819d0b-87c8-42a7-afcc-eb75d6fa7ec0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:56.049923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83819d0b-87c8-42a7-afcc-eb75d6fa7ec0 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:56.051427 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 664/1344] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:44:56.060565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a12a56-c20b-499d-b9d4-84871c72304b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:56.061094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a12a56-c20b-499d-b9d4-84871c72304b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:56.061094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a12a56-c20b-499d-b9d4-84871c72304b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:56.141581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b7a12a56-c20b-499d-b9d4-84871c72304b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:44:56.142807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b7a12a56-c20b-499d-b9d4-84871c72304b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:56.255948 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 681/1345] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:44:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:44:56.593396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83819d0b-87c8-42a7-afcc-eb75d6fa7ec0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:44:56.594937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-83819d0b-87c8-42a7-afcc-eb75d6fa7ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:56.686084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:44:56.704845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83819d0b-87c8-42a7-afcc-eb75d6fa7ec0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:44:56.777239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf0a01-fa5b-49ad-a7bb-86cd7f0b37c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:44:56.777839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf0a01-fa5b-49ad-a7bb-86cd7f0b37c5 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:44:56.778103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf0a01-fa5b-49ad-a7bb-86cd7f0b37c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:44:56.778369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf0a01-fa5b-49ad-a7bb-86cd7f0b37c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:44:56.779967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e2bf0a01-fa5b-49ad-a7bb-86cd7f0b37c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:44:56.903498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf0a01-fa5b-49ad-a7bb-86cd7f0b37c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:44:56.904968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf0a01-fa5b-49ad-a7bb-86cd7f0b37c5 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:44:56.928775 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 665/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:44:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:45:01.384808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65f3ce-7ed4-4eba-bbd9-1bc69a679cd6 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:01.385347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65f3ce-7ed4-4eba-bbd9-1bc69a679cd6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:01.385347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65f3ce-7ed4-4eba-bbd9-1bc69a679cd6 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:01.386884 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 682/1347] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:01.392547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0d23d-39fb-4522-bda1-24e3d82df5e7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:01.393025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0d23d-39fb-4522-bda1-24e3d82df5e7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:01.393116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0d23d-39fb-4522-bda1-24e3d82df5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:01.409544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:01.462052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6cc0d23d-39fb-4522-bda1-24e3d82df5e7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:01.463159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6cc0d23d-39fb-4522-bda1-24e3d82df5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:01.565156 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 666/1348] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:01.909842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65f3ce-7ed4-4eba-bbd9-1bc69a679cd6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:01.911359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4d65f3ce-7ed4-4eba-bbd9-1bc69a679cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:01.974865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:02.022651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65f3ce-7ed4-4eba-bbd9-1bc69a679cd6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:02.097286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d7a08-9734-4034-8e71-5772599afac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:02.097890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d7a08-9734-4034-8e71-5772599afac4 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:02.098155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d7a08-9734-4034-8e71-5772599afac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:02.098420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d7a08-9734-4034-8e71-5772599afac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:02.100030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-501d7a08-9734-4034-8e71-5772599afac4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:02.212687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:02.220030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d7a08-9734-4034-8e71-5772599afac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:02.221585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d7a08-9734-4034-8e71-5772599afac4 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:02.243170 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 683/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:07.699777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3244836-554b-439d-a7f1-9ed35d390222 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:07.700283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3244836-554b-439d-a7f1-9ed35d390222 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:07.700283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3244836-554b-439d-a7f1-9ed35d390222 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:07.701819 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 667/1350] 10.222.0.23 () {54 vars in 921 bytes} [Sat Aug 20 20:45:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:07.707022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad5e62-640c-49c9-9ff4-cb3a62c10efa None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:07.707437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad5e62-640c-49c9-9ff4-cb3a62c10efa None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:07.707766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad5e62-640c-49c9-9ff4-cb3a62c10efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:07.800328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-93ad5e62-640c-49c9-9ff4-cb3a62c10efa None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:07.801948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-93ad5e62-640c-49c9-9ff4-cb3a62c10efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:07.988340 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 684/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 20:45:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:11.563190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=59 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:12.313239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4612790-f4f4-408b-a90c-d8b5bc2198d5 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:12.313460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4612790-f4f4-408b-a90c-d8b5bc2198d5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:12.313726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4612790-f4f4-408b-a90c-d8b5bc2198d5 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:12.316356 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 668/1352] 10.222.0.23 () {54 vars in 921 bytes} [Sat Aug 20 20:45:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:12.323871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de22036-b03b-40c2-b775-b9740c065d53 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:12.329038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de22036-b03b-40c2-b775-b9740c065d53 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:12.329219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de22036-b03b-40c2-b775-b9740c065d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:12.399922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:12.425984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1de22036-b03b-40c2-b775-b9740c065d53 None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:12.427178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1de22036-b03b-40c2-b775-b9740c065d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:12.547789 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 685/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 20:45:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:14.080609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74317a0-9d61-4b78-9151-e55144eccabd None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:14.081207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74317a0-9d61-4b78-9151-e55144eccabd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:14.081207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74317a0-9d61-4b78-9151-e55144eccabd None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:14.082527 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 669/1354] 10.222.0.23 () {54 vars in 921 bytes} [Sat Aug 20 20:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:14.089119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e50229-d66a-4475-80d2-035a2d87b00b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:14.089685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e50229-d66a-4475-80d2-035a2d87b00b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:14.090350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e50229-d66a-4475-80d2-035a2d87b00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:14.193998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-67e50229-d66a-4475-80d2-035a2d87b00b None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:14.195442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-67e50229-d66a-4475-80d2-035a2d87b00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:14.249340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:14.341780 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 686/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 20:45:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:18.110341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86b2f2-2d0b-4ec4-b4fb-6ca65add0f4c None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:18.111165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86b2f2-2d0b-4ec4-b4fb-6ca65add0f4c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:18.111574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86b2f2-2d0b-4ec4-b4fb-6ca65add0f4c None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:18.113107 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 670/1356] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:18.118841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2d8d3-3029-46d9-be20-aed6ef535502 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:18.119622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2d8d3-3029-46d9-be20-aed6ef535502 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:18.120033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2d8d3-3029-46d9-be20-aed6ef535502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:18.195732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-33e2d8d3-3029-46d9-be20-aed6ef535502 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:18.197215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-33e2d8d3-3029-46d9-be20-aed6ef535502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:18.306592 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 687/1357] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:18.659428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5606a8-d693-437b-91f3-bf00861bb9d3 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:18.659704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5606a8-d693-437b-91f3-bf00861bb9d3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:18.659968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5606a8-d693-437b-91f3-bf00861bb9d3 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:18.661533 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 671/1358] 10.222.0.23 () {54 vars in 962 bytes} [Sat Aug 20 20:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:18.670700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4f1ab-8c0e-412c-adce-c501d7615f25 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:18.671301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4f1ab-8c0e-412c-adce-c501d7615f25 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:18.672022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4f1ab-8c0e-412c-adce-c501d7615f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:18.700274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:18.767611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6ac4f1ab-8c0e-412c-adce-c501d7615f25 None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:18.768857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6ac4f1ab-8c0e-412c-adce-c501d7615f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:18.896429 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 688/1359] 10.222.0.23 () {58 vars in 1059 bytes} [Sat Aug 20 20:45:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:45:18.904419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5606a8-d693-437b-91f3-bf00861bb9d3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:18.906121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6f5606a8-d693-437b-91f3-bf00861bb9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:19.059961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5606a8-d693-437b-91f3-bf00861bb9d3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:19.136498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:19.180501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3c93c-16c0-4f96-b680-524b4b2a6882 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:19.180919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3c93c-16c0-4f96-b680-524b4b2a6882 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:19.181193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3c93c-16c0-4f96-b680-524b4b2a6882 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:19.181472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3c93c-16c0-4f96-b680-524b4b2a6882 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:19.183152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-12f3c93c-16c0-4f96-b680-524b4b2a6882 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:19.275539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3c93c-16c0-4f96-b680-524b4b2a6882 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:19.277192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3c93c-16c0-4f96-b680-524b4b2a6882 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:19.295835 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 672/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:21.734326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ee62c-6f9d-485b-9e66-7e2bb7586a0c None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:21.734929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ee62c-6f9d-485b-9e66-7e2bb7586a0c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:21.735070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ee62c-6f9d-485b-9e66-7e2bb7586a0c None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:21.736800 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 689/1361] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:21.742095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f053f9e-502e-4b18-ad20-3a174c634e8b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:21.742587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f053f9e-502e-4b18-ad20-3a174c634e8b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:21.742587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f053f9e-502e-4b18-ad20-3a174c634e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:21.820765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9f053f9e-502e-4b18-ad20-3a174c634e8b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:21.823317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9f053f9e-502e-4b18-ad20-3a174c634e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:21.870304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:21.941411 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 673/1362] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:22.293151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ee62c-6f9d-485b-9e66-7e2bb7586a0c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:22.294864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1b0ee62c-6f9d-485b-9e66-7e2bb7586a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:22.363340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:22.451799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ee62c-6f9d-485b-9e66-7e2bb7586a0c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:22.580363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8b457-f566-4a6b-9122-b95f30d6b08a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:22.580983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8b457-f566-4a6b-9122-b95f30d6b08a service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:22.581253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8b457-f566-4a6b-9122-b95f30d6b08a service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:22.581477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8b457-f566-4a6b-9122-b95f30d6b08a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:22.583125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8ad8b457-f566-4a6b-9122-b95f30d6b08a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:22.606645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:22.676558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8b457-f566-4a6b-9122-b95f30d6b08a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:22.678555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8b457-f566-4a6b-9122-b95f30d6b08a service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:22.699590 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 690/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:25.234097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b301bb-af69-4f4a-8c41-06bb37f51a1a None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:25.236728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b301bb-af69-4f4a-8c41-06bb37f51a1a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:25.238028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b301bb-af69-4f4a-8c41-06bb37f51a1a None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:25.239630 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 674/1364] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:25.249492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48685f4b-a8f6-4c27-9081-db354ffe3028 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:25.249920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48685f4b-a8f6-4c27-9081-db354ffe3028 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:25.250032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48685f4b-a8f6-4c27-9081-db354ffe3028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:25.355787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-48685f4b-a8f6-4c27-9081-db354ffe3028 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:25.357344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-48685f4b-a8f6-4c27-9081-db354ffe3028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:25.459390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:25.468530 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 691/1365] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:25.824304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9c785-bd14-4b36-9075-ad2f9c6a3019 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:25.824883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9c785-bd14-4b36-9075-ad2f9c6a3019 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:25.825295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9c785-bd14-4b36-9075-ad2f9c6a3019 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:25.827535 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 675/1366] 10.222.0.23 () {54 vars in 962 bytes} [Sat Aug 20 20:45:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:25.833533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51dd15-288e-4f07-ac9d-7b9e0089d74a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:25.833928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51dd15-288e-4f07-ac9d-7b9e0089d74a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:25.834267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51dd15-288e-4f07-ac9d-7b9e0089d74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:25.913547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2c51dd15-288e-4f07-ac9d-7b9e0089d74a None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:25.914951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2c51dd15-288e-4f07-ac9d-7b9e0089d74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:26.041085 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 692/1367] 10.222.0.23 () {58 vars in 1059 bytes} [Sat Aug 20 20:45:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:26.060029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9c785-bd14-4b36-9075-ad2f9c6a3019 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:26.062073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cca9c785-bd14-4b36-9075-ad2f9c6a3019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:26.259146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9c785-bd14-4b36-9075-ad2f9c6a3019 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:26.286988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:26.411273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81d116-0d5b-40a3-b34c-ab64c38d121e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:26.411806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81d116-0d5b-40a3-b34c-ab64c38d121e service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:26.412077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81d116-0d5b-40a3-b34c-ab64c38d121e service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:26.412702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81d116-0d5b-40a3-b34c-ab64c38d121e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:26.415393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6a81d116-0d5b-40a3-b34c-ab64c38d121e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:26.527714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81d116-0d5b-40a3-b34c-ab64c38d121e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:26.529723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81d116-0d5b-40a3-b34c-ab64c38d121e service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:26.552776 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 676/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:29.172863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61ab7c-a849-40f6-a6ca-baba3a6f6e4e None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:29.196567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61ab7c-a849-40f6-a6ca-baba3a6f6e4e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:29.196567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61ab7c-a849-40f6-a6ca-baba3a6f6e4e None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:29.196567 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 693/1369] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:29.196909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8eece-6882-4c1a-af86-c5a81c20da34 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:29.196909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8eece-6882-4c1a-af86-c5a81c20da34 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:29.196909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8eece-6882-4c1a-af86-c5a81c20da34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:29.248745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:29.299839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3aa8eece-6882-4c1a-af86-c5a81c20da34 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:29.301442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3aa8eece-6882-4c1a-af86-c5a81c20da34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:29.432926 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 677/1370] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:29.814917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61ab7c-a849-40f6-a6ca-baba3a6f6e4e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:29.816898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fa61ab7c-a849-40f6-a6ca-baba3a6f6e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:29.880984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:29.987481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61ab7c-a849-40f6-a6ca-baba3a6f6e4e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:30.145883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1335c81-5dda-496c-858a-e45674913ac5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:30.147490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1335c81-5dda-496c-858a-e45674913ac5 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:30.148358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1335c81-5dda-496c-858a-e45674913ac5 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:30.159911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1335c81-5dda-496c-858a-e45674913ac5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:30.164007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d1335c81-5dda-496c-858a-e45674913ac5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:30.187481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:30.322297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1335c81-5dda-496c-858a-e45674913ac5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:30.324645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1335c81-5dda-496c-858a-e45674913ac5 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:30.348180 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 694/1371] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:30.873804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15813361-1183-4155-b263-11a10094b274 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:30.874764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15813361-1183-4155-b263-11a10094b274 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:30.874764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15813361-1183-4155-b263-11a10094b274 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:30.875731 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 678/1372] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:30.881767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c7a51-30f9-48fa-bfb5-3232a74d76c4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:30.882271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c7a51-30f9-48fa-bfb5-3232a74d76c4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:30.882691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c7a51-30f9-48fa-bfb5-3232a74d76c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:30.965506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-149c7a51-30f9-48fa-bfb5-3232a74d76c4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:30.966803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-149c7a51-30f9-48fa-bfb5-3232a74d76c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:31.048967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:31.070533 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 695/1373] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:31.677128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf5a3e-19cf-4175-a9ad-7bfbe1bea2ed None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:31.677991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf5a3e-19cf-4175-a9ad-7bfbe1bea2ed None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:31.678109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf5a3e-19cf-4175-a9ad-7bfbe1bea2ed None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:31.679541 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 679/1374] 10.222.0.23 () {54 vars in 964 bytes} [Sat Aug 20 20:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:31.685553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bbb13-962d-4ced-81b3-57cbcf4cc179 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:31.686023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bbb13-962d-4ced-81b3-57cbcf4cc179 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:31.686108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bbb13-962d-4ced-81b3-57cbcf4cc179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:31.774476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f48bbb13-962d-4ced-81b3-57cbcf4cc179 None None] MFA Rules not processed for user `53928836828d45fbb427dd92492cae20`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:31.774476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f48bbb13-962d-4ced-81b3-57cbcf4cc179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:31.854813 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 696/1375] 10.222.0.23 () {58 vars in 1061 bytes} [Sat Aug 20 20:45:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:31.862934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf5a3e-19cf-4175-a9ad-7bfbe1bea2ed None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:31.864476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-76bf5a3e-19cf-4175-a9ad-7bfbe1bea2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:31.986294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf5a3e-19cf-4175-a9ad-7bfbe1bea2ed None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:31.991244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:32.071342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28314d-960b-485d-b3d8-49ff2cf64d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:32.072193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28314d-960b-485d-b3d8-49ff2cf64d6b service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:32.072569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28314d-960b-485d-b3d8-49ff2cf64d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:32.073025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28314d-960b-485d-b3d8-49ff2cf64d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:32.075176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3f28314d-960b-485d-b3d8-49ff2cf64d6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:32.207013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28314d-960b-485d-b3d8-49ff2cf64d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:32.208441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28314d-960b-485d-b3d8-49ff2cf64d6b service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:32.231461 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 680/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:32.818403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb74b9-47e5-4141-bfb9-c0b737374b10 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:32.819328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb74b9-47e5-4141-bfb9-c0b737374b10 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:32.819597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb74b9-47e5-4141-bfb9-c0b737374b10 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:32.821032 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 697/1377] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:32.827989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35db0e1-11ad-43d5-8b2c-d01837e5c47d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:32.828533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35db0e1-11ad-43d5-8b2c-d01837e5c47d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:32.828774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35db0e1-11ad-43d5-8b2c-d01837e5c47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:32.899377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:32.939975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d35db0e1-11ad-43d5-8b2c-d01837e5c47d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:32.941745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d35db0e1-11ad-43d5-8b2c-d01837e5c47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:33.054790 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 681/1378] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:45:33.596112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb74b9-47e5-4141-bfb9-c0b737374b10 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:33.597692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fcbb74b9-47e5-4141-bfb9-c0b737374b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:33.679697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:33.766438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb74b9-47e5-4141-bfb9-c0b737374b10 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:33.884825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0758a-1afc-42f2-9ab8-e6abe6a9ea7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:33.886086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0758a-1afc-42f2-9ab8-e6abe6a9ea7a service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:33.886453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0758a-1afc-42f2-9ab8-e6abe6a9ea7a service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:33.886893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0758a-1afc-42f2-9ab8-e6abe6a9ea7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:33.888706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-05d0758a-1afc-42f2-9ab8-e6abe6a9ea7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:33.930952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:33.981620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0758a-1afc-42f2-9ab8-e6abe6a9ea7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:33.983241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0758a-1afc-42f2-9ab8-e6abe6a9ea7a service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:34.003696 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 698/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:36.406793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa681f-f1b2-4c7e-91f0-41438ab6dddb None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:36.407380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa681f-f1b2-4c7e-91f0-41438ab6dddb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:36.407380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa681f-f1b2-4c7e-91f0-41438ab6dddb None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:36.410324 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 682/1380] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:36.414627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e18275-3f0d-4d0b-bbd7-753b38cfdf56 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:36.415090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e18275-3f0d-4d0b-bbd7-753b38cfdf56 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:36.415194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e18275-3f0d-4d0b-bbd7-753b38cfdf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:36.498973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e0e18275-3f0d-4d0b-bbd7-753b38cfdf56 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:36.500235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e0e18275-3f0d-4d0b-bbd7-753b38cfdf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:36.620500 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 699/1381] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:36.964097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa681f-f1b2-4c7e-91f0-41438ab6dddb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:36.965884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3faa681f-f1b2-4c7e-91f0-41438ab6dddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:37.086900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:37.127036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa681f-f1b2-4c7e-91f0-41438ab6dddb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:37.251248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4aabe-e500-49d6-a1e4-483da7696af3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:37.252271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4aabe-e500-49d6-a1e4-483da7696af3 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:37.252790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4aabe-e500-49d6-a1e4-483da7696af3 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:37.253249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4aabe-e500-49d6-a1e4-483da7696af3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:37.256084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3ee4aabe-e500-49d6-a1e4-483da7696af3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:37.342053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:37.358721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4aabe-e500-49d6-a1e4-483da7696af3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:37.360321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4aabe-e500-49d6-a1e4-483da7696af3 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:37.379111 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 683/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:39.664102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac5378-6a93-490c-b04a-08a9d4a77865 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:39.664706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac5378-6a93-490c-b04a-08a9d4a77865 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:39.665147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac5378-6a93-490c-b04a-08a9d4a77865 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:39.666641 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 700/1383] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:39.672348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0b485-b59e-4cd0-ae18-aeef6992f896 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:39.672854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0b485-b59e-4cd0-ae18-aeef6992f896 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:39.672934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0b485-b59e-4cd0-ae18-aeef6992f896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:39.774404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b3e0b485-b59e-4cd0-ae18-aeef6992f896 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:39.775692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b3e0b485-b59e-4cd0-ae18-aeef6992f896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:39.888835 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 684/1384] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:40.250005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac5378-6a93-490c-b04a-08a9d4a77865 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:40.251498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-74ac5378-6a93-490c-b04a-08a9d4a77865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:40.272843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:40.406236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac5378-6a93-490c-b04a-08a9d4a77865 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:40.504094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:40.533106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c33185-89f5-4c3e-b9d4-c0821d329a3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:40.533783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c33185-89f5-4c3e-b9d4-c0821d329a3b service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:40.534115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c33185-89f5-4c3e-b9d4-c0821d329a3b service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:40.534544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c33185-89f5-4c3e-b9d4-c0821d329a3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:40.536287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-24c33185-89f5-4c3e-b9d4-c0821d329a3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:40.644810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c33185-89f5-4c3e-b9d4-c0821d329a3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:40.646441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c33185-89f5-4c3e-b9d4-c0821d329a3b service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:40.668407 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 701/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:43.075110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844a1d7-5df9-489a-b21b-686ba30273b8 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:43.075831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844a1d7-5df9-489a-b21b-686ba30273b8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:43.075831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844a1d7-5df9-489a-b21b-686ba30273b8 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:43.077921 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 685/1386] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:43.084538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197c26-a908-44ae-a1b2-41ba9a609f4c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:43.085243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197c26-a908-44ae-a1b2-41ba9a609f4c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:43.085243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197c26-a908-44ae-a1b2-41ba9a609f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:43.170139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3d197c26-a908-44ae-a1b2-41ba9a609f4c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:43.170803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3d197c26-a908-44ae-a1b2-41ba9a609f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:43.247020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:43.295703 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 702/1387] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:43.662304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844a1d7-5df9-489a-b21b-686ba30273b8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:43.664587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e844a1d7-5df9-489a-b21b-686ba30273b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:43.735236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:43.860676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844a1d7-5df9-489a-b21b-686ba30273b8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:44.008205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff492c-d00c-4642-8dcb-92ad9eaf3165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:44.008912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff492c-d00c-4642-8dcb-92ad9eaf3165 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:44.009312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff492c-d00c-4642-8dcb-92ad9eaf3165 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:44.009588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff492c-d00c-4642-8dcb-92ad9eaf3165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:44.011409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f7ff492c-d00c-4642-8dcb-92ad9eaf3165 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:44.026449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:44.132879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff492c-d00c-4642-8dcb-92ad9eaf3165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:44.134981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff492c-d00c-4642-8dcb-92ad9eaf3165 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:44.159033 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 686/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:46.632870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50570d8b-7257-405a-b5e7-3bde815cd329 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:46.633505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50570d8b-7257-405a-b5e7-3bde815cd329 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:46.633505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50570d8b-7257-405a-b5e7-3bde815cd329 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:46.634803 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 703/1389] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:46.640427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601fb93-9627-46d0-b512-6cf9527f1681 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:46.640920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601fb93-9627-46d0-b512-6cf9527f1681 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:46.641038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601fb93-9627-46d0-b512-6cf9527f1681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:46.715218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4601fb93-9627-46d0-b512-6cf9527f1681 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:46.716402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4601fb93-9627-46d0-b512-6cf9527f1681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:46.809052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:46.820709 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 687/1390] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:47.166556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50570d8b-7257-405a-b5e7-3bde815cd329 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:47.168170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-50570d8b-7257-405a-b5e7-3bde815cd329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:47.322722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50570d8b-7257-405a-b5e7-3bde815cd329 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:47.361706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:47.444007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31554c84-f3de-4114-8339-5d4ae2f13c98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:47.444563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31554c84-f3de-4114-8339-5d4ae2f13c98 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:47.444831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31554c84-f3de-4114-8339-5d4ae2f13c98 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:47.445096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31554c84-f3de-4114-8339-5d4ae2f13c98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:47.446753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-31554c84-f3de-4114-8339-5d4ae2f13c98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:47.540097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31554c84-f3de-4114-8339-5d4ae2f13c98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:47.541718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31554c84-f3de-4114-8339-5d4ae2f13c98 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:47.559479 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 704/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:48.554851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dd76f-686a-4188-82b0-b6ef9ea9dab7 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:48.555494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dd76f-686a-4188-82b0-b6ef9ea9dab7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:48.555494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dd76f-686a-4188-82b0-b6ef9ea9dab7 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:48.556778 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 688/1392] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:48.562878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21518b-c46c-4f73-aa07-7de68182955c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:48.563360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21518b-c46c-4f73-aa07-7de68182955c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:48.563360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21518b-c46c-4f73-aa07-7de68182955c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:48.622015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:48.639638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0a21518b-c46c-4f73-aa07-7de68182955c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:48.641039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0a21518b-c46c-4f73-aa07-7de68182955c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:48.748719 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 705/1393] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:49.110649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734c772-9ace-4953-8e83-5e76055df054 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:49.110912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734c772-9ace-4953-8e83-5e76055df054 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:49.111177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734c772-9ace-4953-8e83-5e76055df054 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:49.112565 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 689/1394] 10.222.0.23 () {54 vars in 967 bytes} [Sat Aug 20 20:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:49.117678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b7cc8-a79d-4515-9a3b-7d01ddbe98fe None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:49.117959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b7cc8-a79d-4515-9a3b-7d01ddbe98fe None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:49.118227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b7cc8-a79d-4515-9a3b-7d01ddbe98fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:49.196814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1e2b7cc8-a79d-4515-9a3b-7d01ddbe98fe None None] MFA Rules not processed for user `698d5a7fc34a46ba909152414a0860d7`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:49.198005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1e2b7cc8-a79d-4515-9a3b-7d01ddbe98fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:49.274143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:49.288278 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 706/1395] 10.222.0.23 () {58 vars in 1064 bytes} [Sat Aug 20 20:45:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:49.297574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734c772-9ace-4953-8e83-5e76055df054 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:49.298077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2734c772-9ace-4953-8e83-5e76055df054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:49.406250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734c772-9ace-4953-8e83-5e76055df054 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:49.485252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954931e-acb5-484d-8463-531eae45baea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:49.485882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954931e-acb5-484d-8463-531eae45baea service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:49.486161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954931e-acb5-484d-8463-531eae45baea service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:49.486474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954931e-acb5-484d-8463-531eae45baea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:49.488194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5954931e-acb5-484d-8463-531eae45baea service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:49.536528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:49.606591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954931e-acb5-484d-8463-531eae45baea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:49.608049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954931e-acb5-484d-8463-531eae45baea service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:49.631903 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 690/1396] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:49.782998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fd7a5-a66a-4dc9-a4a6-ba1f487407e8 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:49.783599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fd7a5-a66a-4dc9-a4a6-ba1f487407e8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:49.783820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fd7a5-a66a-4dc9-a4a6-ba1f487407e8 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:49.785342 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 707/1397] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:49.790706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2a648-d004-4197-b8fe-c94df9006ff5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:49.790986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2a648-d004-4197-b8fe-c94df9006ff5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:49.791254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2a648-d004-4197-b8fe-c94df9006ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:49.862981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-67a2a648-d004-4197-b8fe-c94df9006ff5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:49.864210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-67a2a648-d004-4197-b8fe-c94df9006ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:49.974283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:49.977195 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 691/1398] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:50.219837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa8bf5-724f-494a-bda7-283a204afc1e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:50.220112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa8bf5-724f-494a-bda7-283a204afc1e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:50.220411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa8bf5-724f-494a-bda7-283a204afc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:50.289509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8ffa8bf5-724f-494a-bda7-283a204afc1e None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:50.291510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8ffa8bf5-724f-494a-bda7-283a204afc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:50.337534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2a648-d004-4197-b8fe-c94df9006ff5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:50.338914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-67a2a648-d004-4197-b8fe-c94df9006ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:50.392966 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 708/1399] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 20:45:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:50.445354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448f7f7-6456-43ed-a2db-dae52fdb2ce5 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:50.445654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448f7f7-6456-43ed-a2db-dae52fdb2ce5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:50.445904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448f7f7-6456-43ed-a2db-dae52fdb2ce5 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:50.447486 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 709/1400] 10.222.0.23 () {54 vars in 943 bytes} [Sat Aug 20 20:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:50.455587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab97a9-95c9-447f-8366-e79c4a8a7a69 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:50.456070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab97a9-95c9-447f-8366-e79c4a8a7a69 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:50.456779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab97a9-95c9-447f-8366-e79c4a8a7a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:50.490372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2a648-d004-4197-b8fe-c94df9006ff5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:50.538449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-20ab97a9-95c9-447f-8366-e79c4a8a7a69 None None] MFA Rules not processed for user `d74f7cdc155a4397ad28a3fbca86f7f9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:50.539788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-20ab97a9-95c9-447f-8366-e79c4a8a7a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:50.567810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:50.611347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:50.617453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc0689-09b0-477c-bd96-2fe5ebaa61b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:50.618067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc0689-09b0-477c-bd96-2fe5ebaa61b3 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:50.618336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc0689-09b0-477c-bd96-2fe5ebaa61b3 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:50.618598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc0689-09b0-477c-bd96-2fe5ebaa61b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:50.620183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ebdc0689-09b0-477c-bd96-2fe5ebaa61b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:50.663236 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 710/1401] 10.222.0.23 () {58 vars in 1040 bytes} [Sat Aug 20 20:45:50 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:50.672822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab97a9-95c9-447f-8366-e79c4a8a7a69 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:50.682120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab97a9-95c9-447f-8366-e79c4a8a7a69 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:50.716498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc0689-09b0-477c-bd96-2fe5ebaa61b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:50.718099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc0689-09b0-477c-bd96-2fe5ebaa61b3 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:50.743821 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 692/1402] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:50.791780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019b196-416d-4d19-8391-d4d75be1fce3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:50.792396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019b196-416d-4d19-8391-d4d75be1fce3 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:50.792753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019b196-416d-4d19-8391-d4d75be1fce3 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:50.793009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019b196-416d-4d19-8391-d4d75be1fce3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:50.794744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b019b196-416d-4d19-8391-d4d75be1fce3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:50.823796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:50.894593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019b196-416d-4d19-8391-d4d75be1fce3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:50.897074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019b196-416d-4d19-8391-d4d75be1fce3 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:50.907631 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 711/1403] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 20:45:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 20:45:53.045436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64a2c0-f4cd-4a0d-8269-d038e2a7d5fa None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:53.046117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64a2c0-f4cd-4a0d-8269-d038e2a7d5fa None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:53.046117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64a2c0-f4cd-4a0d-8269-d038e2a7d5fa None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:53.047451 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 693/1404] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:53.052661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a971d7-44e4-421f-ba72-46621cdcdcad None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:53.053128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a971d7-44e4-421f-ba72-46621cdcdcad None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:53.053221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a971d7-44e4-421f-ba72-46621cdcdcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:53.126098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f2a971d7-44e4-421f-ba72-46621cdcdcad None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:53.127280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f2a971d7-44e4-421f-ba72-46621cdcdcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:53.256476 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 712/1405] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:53.609338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64a2c0-f4cd-4a0d-8269-d038e2a7d5fa None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:53.611337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-aa64a2c0-f4cd-4a0d-8269-d038e2a7d5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:53.743494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:53.784361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64a2c0-f4cd-4a0d-8269-d038e2a7d5fa None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:53.919624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027910a8-be37-4be6-b64b-053b5b9483e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:53.920149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027910a8-be37-4be6-b64b-053b5b9483e1 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:53.920442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027910a8-be37-4be6-b64b-053b5b9483e1 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:53.920707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027910a8-be37-4be6-b64b-053b5b9483e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:53.922458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-027910a8-be37-4be6-b64b-053b5b9483e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:54.010960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:54.027345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027910a8-be37-4be6-b64b-053b5b9483e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:54.029011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027910a8-be37-4be6-b64b-053b5b9483e1 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:54.051200 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 694/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:45:56.348282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d27822-a378-4248-8d36-a64604db8fff None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:56.348955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d27822-a378-4248-8d36-a64604db8fff None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:56.349392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d27822-a378-4248-8d36-a64604db8fff None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:56.352424 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 713/1407] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:45:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:45:56.356640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c136d43-993e-4161-9f14-81f34a20de64 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:56.357151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c136d43-993e-4161-9f14-81f34a20de64 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:56.357258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c136d43-993e-4161-9f14-81f34a20de64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:56.425751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7c136d43-993e-4161-9f14-81f34a20de64 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:45:56.434549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7c136d43-993e-4161-9f14-81f34a20de64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:56.552181 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 695/1408] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:45:56.913123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d27822-a378-4248-8d36-a64604db8fff None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:45:56.914685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a7d27822-a378-4248-8d36-a64604db8fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:56.934486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:57.138468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d27822-a378-4248-8d36-a64604db8fff None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:45:57.255583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:45:57.286183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acf977-324b-4a68-b454-bdaa624da387 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:45:57.287278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acf977-324b-4a68-b454-bdaa624da387 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:45:57.287738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acf977-324b-4a68-b454-bdaa624da387 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:45:57.288005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acf977-324b-4a68-b454-bdaa624da387 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:45:57.289726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-44acf977-324b-4a68-b454-bdaa624da387 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:45:57.389422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44acf977-324b-4a68-b454-bdaa624da387 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:45:57.391142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44acf977-324b-4a68-b454-bdaa624da387 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:45:57.411340 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 714/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:45:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:06.549883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=74 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:07.408973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=54 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:14.035175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1e90d-761b-4a2d-9793-a676dfd2d08e None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:14.035789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1e90d-761b-4a2d-9793-a676dfd2d08e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:14.035789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1e90d-761b-4a2d-9793-a676dfd2d08e None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:14.037052 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 696/1410] 10.222.0.23 () {54 vars in 918 bytes} [Sat Aug 20 20:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:14.042199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ce50e-80d8-46ce-84c5-ace2b07d7328 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:14.042657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ce50e-80d8-46ce-84c5-ace2b07d7328 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:14.042782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ce50e-80d8-46ce-84c5-ace2b07d7328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:14.125262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f56ce50e-80d8-46ce-84c5-ace2b07d7328 None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:14.126463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f56ce50e-80d8-46ce-84c5-ace2b07d7328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:14.246955 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 715/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Sat Aug 20 20:46:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:14.261403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ca76c-5f32-4d93-8d51-33a8b5222731 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:14.261680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ca76c-5f32-4d93-8d51-33a8b5222731 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:14.261944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ca76c-5f32-4d93-8d51-33a8b5222731 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:14.263381 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 697/1412] 10.222.0.23 () {54 vars in 918 bytes} [Sat Aug 20 20:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:46:14.268846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87e2cd-c081-4e64-9675-325fca9566c5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:14.269186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87e2cd-c081-4e64-9675-325fca9566c5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:14.269502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87e2cd-c081-4e64-9675-325fca9566c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:14.353295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:14.356630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ba87e2cd-c081-4e64-9675-325fca9566c5 None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:14.358255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ba87e2cd-c081-4e64-9675-325fca9566c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:14.486015 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 716/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Sat Aug 20 20:46:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:17.348407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf1f91-cbe0-4e46-96c7-8a484b7b559d None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:17.349001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf1f91-cbe0-4e46-96c7-8a484b7b559d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:17.349001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf1f91-cbe0-4e46-96c7-8a484b7b559d None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:17.350257 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 20 20:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:17.357294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f1f15-bed0-490e-803f-c09f57f85a3c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:17.357993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f1f15-bed0-490e-803f-c09f57f85a3c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:17.357993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f1f15-bed0-490e-803f-c09f57f85a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:17.434844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fa1f1f15-bed0-490e-803f-c09f57f85a3c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:17.436097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fa1f1f15-bed0-490e-803f-c09f57f85a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:17.497604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:17.543270 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 20 20:46:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:17.896146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf1f91-cbe0-4e46-96c7-8a484b7b559d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:17.897637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c3cf1f91-cbe0-4e46-96c7-8a484b7b559d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:18.062208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf1f91-cbe0-4e46-96c7-8a484b7b559d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:18.137015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:18.177032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29459009-2775-493d-96da-b59e9808b22b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:18.177631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29459009-2775-493d-96da-b59e9808b22b service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:18.177900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29459009-2775-493d-96da-b59e9808b22b service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:18.178163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29459009-2775-493d-96da-b59e9808b22b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:18.179745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-29459009-2775-493d-96da-b59e9808b22b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:18.275750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29459009-2775-493d-96da-b59e9808b22b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:18.277417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29459009-2775-493d-96da-b59e9808b22b service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:18.296194 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 699/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:19.255146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5d943-202e-4357-ae3d-0975121711b3 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:19.255626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5d943-202e-4357-ae3d-0975121711b3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:19.255626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5d943-202e-4357-ae3d-0975121711b3 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:19.257096 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 20 20:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:19.262187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3fd56-d9b0-427f-88d8-08f98f9a1580 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:19.262775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3fd56-d9b0-427f-88d8-08f98f9a1580 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:19.262775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3fd56-d9b0-427f-88d8-08f98f9a1580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:19.348247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a7a3fd56-d9b0-427f-88d8-08f98f9a1580 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:19.349491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a7a3fd56-d9b0-427f-88d8-08f98f9a1580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:19.378275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:19.450947 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 20 20:46:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:19.693397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5d943-202e-4357-ae3d-0975121711b3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:19.694790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-66b5d943-202e-4357-ae3d-0975121711b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:19.850556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5d943-202e-4357-ae3d-0975121711b3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:19.893752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:19.987739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bf1fd-348d-4833-820a-03154800ef6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:19.988345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bf1fd-348d-4833-820a-03154800ef6f service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:19.988649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bf1fd-348d-4833-820a-03154800ef6f service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:19.988920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bf1fd-348d-4833-820a-03154800ef6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:19.990515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2c1bf1fd-348d-4833-820a-03154800ef6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:20.084276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bf1fd-348d-4833-820a-03154800ef6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:20.085901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bf1fd-348d-4833-820a-03154800ef6f service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:20.104491 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 719/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:21.015012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5e472-4abb-45db-8f90-dd517eb8fcc1 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:21.015894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5e472-4abb-45db-8f90-dd517eb8fcc1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:21.015894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5e472-4abb-45db-8f90-dd517eb8fcc1 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:21.017928 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 20 20:46:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:21.022897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901dcf6-19ff-4abf-9d07-833e6916de88 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:21.023323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901dcf6-19ff-4abf-9d07-833e6916de88 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:21.023440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901dcf6-19ff-4abf-9d07-833e6916de88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:21.086306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:21.111119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c901dcf6-19ff-4abf-9d07-833e6916de88 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:21.112470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c901dcf6-19ff-4abf-9d07-833e6916de88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:21.221505 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 20 20:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:21.382718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5e472-4abb-45db-8f90-dd517eb8fcc1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:21.384951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7fd5e472-4abb-45db-8f90-dd517eb8fcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:21.551067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5e472-4abb-45db-8f90-dd517eb8fcc1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:21.563966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:21.677410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3a0b6-49ac-430c-9337-8ea76214c134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:21.677955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3a0b6-49ac-430c-9337-8ea76214c134 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:21.678225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3a0b6-49ac-430c-9337-8ea76214c134 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:21.678486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3a0b6-49ac-430c-9337-8ea76214c134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:21.680098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-69f3a0b6-49ac-430c-9337-8ea76214c134 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:21.777638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3a0b6-49ac-430c-9337-8ea76214c134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:21.779364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3a0b6-49ac-430c-9337-8ea76214c134 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:21.798077 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 702/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:22.669901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe70210-f555-4e4e-a658-51e48651dbc6 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:22.670777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe70210-f555-4e4e-a658-51e48651dbc6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:22.670777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe70210-f555-4e4e-a658-51e48651dbc6 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:22.672841 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 20 20:46:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:22.678496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76d262-adef-4370-84f3-80d845bc48b0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:22.679262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76d262-adef-4370-84f3-80d845bc48b0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:22.679413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76d262-adef-4370-84f3-80d845bc48b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:22.702175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:22.758752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2a76d262-adef-4370-84f3-80d845bc48b0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:22.759949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2a76d262-adef-4370-84f3-80d845bc48b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:22.862629 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 20 20:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:23.036050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe70210-f555-4e4e-a658-51e48651dbc6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:23.038451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ebe70210-f555-4e4e-a658-51e48651dbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:23.165605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:23.203321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe70210-f555-4e4e-a658-51e48651dbc6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:23.344566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf724d33-5f40-4041-add8-0002516e6226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:23.345102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf724d33-5f40-4041-add8-0002516e6226 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:23.345371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf724d33-5f40-4041-add8-0002516e6226 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:23.345625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf724d33-5f40-4041-add8-0002516e6226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:23.347221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bf724d33-5f40-4041-add8-0002516e6226 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:23.426767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:23.440103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf724d33-5f40-4041-add8-0002516e6226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:23.441752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf724d33-5f40-4041-add8-0002516e6226 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:23.459901 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 722/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:24.343336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84497cf8-1e89-41ec-a480-6380ec527a43 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:24.343922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84497cf8-1e89-41ec-a480-6380ec527a43 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:24.343922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84497cf8-1e89-41ec-a480-6380ec527a43 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:24.345248 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 20 20:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:24.350377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f478d9b-1148-4ce0-ace8-7e8bc2d43e3c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:24.350832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f478d9b-1148-4ce0-ace8-7e8bc2d43e3c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:24.350929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f478d9b-1148-4ce0-ace8-7e8bc2d43e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:24.433651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1f478d9b-1148-4ce0-ace8-7e8bc2d43e3c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:24.435367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1f478d9b-1148-4ce0-ace8-7e8bc2d43e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:24.547028 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 20 20:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:24.717005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84497cf8-1e89-41ec-a480-6380ec527a43 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:24.719702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-84497cf8-1e89-41ec-a480-6380ec527a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:24.818958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:24.883693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84497cf8-1e89-41ec-a480-6380ec527a43 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:25.003700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef8c61-cf8b-44f3-9284-4faa6fce4932 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:25.004299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef8c61-cf8b-44f3-9284-4faa6fce4932 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:25.004608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef8c61-cf8b-44f3-9284-4faa6fce4932 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:25.004851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef8c61-cf8b-44f3-9284-4faa6fce4932 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:25.006462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2eef8c61-cf8b-44f3-9284-4faa6fce4932 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:25.065290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:25.108682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef8c61-cf8b-44f3-9284-4faa6fce4932 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:25.110183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef8c61-cf8b-44f3-9284-4faa6fce4932 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:25.129060 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 705/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:25.953083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193a0fe-6674-41ed-b36a-c9c1bddb6b8c None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:25.953715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193a0fe-6674-41ed-b36a-c9c1bddb6b8c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:25.953715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193a0fe-6674-41ed-b36a-c9c1bddb6b8c None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:25.955116 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 20 20:46:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:25.961279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6aa0a-093d-4ea9-bc5f-a3ef19c30c07 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:25.961753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6aa0a-093d-4ea9-bc5f-a3ef19c30c07 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:25.961861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6aa0a-093d-4ea9-bc5f-a3ef19c30c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:26.041072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-19e6aa0a-093d-4ea9-bc5f-a3ef19c30c07 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:26.042307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-19e6aa0a-093d-4ea9-bc5f-a3ef19c30c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:26.153440 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 20 20:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:46:26.321105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193a0fe-6674-41ed-b36a-c9c1bddb6b8c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:26.323426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0193a0fe-6674-41ed-b36a-c9c1bddb6b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:26.382634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:26.497661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193a0fe-6674-41ed-b36a-c9c1bddb6b8c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:26.617599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4b804-b73b-414b-9418-88db09ef66ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:26.618140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4b804-b73b-414b-9418-88db09ef66ad service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:26.618403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4b804-b73b-414b-9418-88db09ef66ad service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:26.618652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4b804-b73b-414b-9418-88db09ef66ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:26.620254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e9a4b804-b73b-414b-9418-88db09ef66ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:26.627549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:26.719200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4b804-b73b-414b-9418-88db09ef66ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:26.720886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4b804-b73b-414b-9418-88db09ef66ad service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:26.739284 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 725/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:27.575388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe589a65-b772-4139-97b5-a718f17c7c85 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:27.576350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe589a65-b772-4139-97b5-a718f17c7c85 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:27.576350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe589a65-b772-4139-97b5-a718f17c7c85 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:27.578630 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 20 20:46:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:27.584731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a485c-1732-4fd0-acaf-3d4a9dbab9d6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:27.585227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a485c-1732-4fd0-acaf-3d4a9dbab9d6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:27.585227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a485c-1732-4fd0-acaf-3d4a9dbab9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:27.671806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-636a485c-1732-4fd0-acaf-3d4a9dbab9d6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:27.673065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-636a485c-1732-4fd0-acaf-3d4a9dbab9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:27.743661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:27.774710 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 20 20:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:27.938752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe589a65-b772-4139-97b5-a718f17c7c85 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:27.940984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fe589a65-b772-4139-97b5-a718f17c7c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:27.970659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:28.118844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe589a65-b772-4139-97b5-a718f17c7c85 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:28.211493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:28.238665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678152f-5925-4150-9bf1-b037853aa7a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:28.239279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678152f-5925-4150-9bf1-b037853aa7a3 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:28.239559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678152f-5925-4150-9bf1-b037853aa7a3 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:28.239823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678152f-5925-4150-9bf1-b037853aa7a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:28.241464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4678152f-5925-4150-9bf1-b037853aa7a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:28.337213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678152f-5925-4150-9bf1-b037853aa7a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:28.339810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678152f-5925-4150-9bf1-b037853aa7a3 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:28.370319 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 708/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:29.213802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e35506-86ed-4ba0-ab7a-9444ba92c451 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:29.214369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e35506-86ed-4ba0-ab7a-9444ba92c451 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:29.214369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e35506-86ed-4ba0-ab7a-9444ba92c451 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:29.215589 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 20 20:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:29.224848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6a03f-13ce-42df-b5ca-289b0cd9d928 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:29.225639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6a03f-13ce-42df-b5ca-289b0cd9d928 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:29.225639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6a03f-13ce-42df-b5ca-289b0cd9d928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:29.305648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-afa6a03f-13ce-42df-b5ca-289b0cd9d928 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:29.306836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-afa6a03f-13ce-42df-b5ca-289b0cd9d928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:29.354962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:29.411746 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 20 20:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:29.578665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e35506-86ed-4ba0-ab7a-9444ba92c451 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:29.581167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e5e35506-86ed-4ba0-ab7a-9444ba92c451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:29.748849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e35506-86ed-4ba0-ab7a-9444ba92c451 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:29.806956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:29.873943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e3c6b-b87e-41a9-8cc6-6ffd5d2b2f90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:29.874463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e3c6b-b87e-41a9-8cc6-6ffd5d2b2f90 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:29.874729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e3c6b-b87e-41a9-8cc6-6ffd5d2b2f90 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:29.874990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e3c6b-b87e-41a9-8cc6-6ffd5d2b2f90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:29.876618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-279e3c6b-b87e-41a9-8cc6-6ffd5d2b2f90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:29.975045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e3c6b-b87e-41a9-8cc6-6ffd5d2b2f90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:29.976670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e3c6b-b87e-41a9-8cc6-6ffd5d2b2f90 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:29.998793 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 728/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:30.819195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5b4bc-3c2c-4301-9e69-b25287393935 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:30.819806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5b4bc-3c2c-4301-9e69-b25287393935 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:30.819806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5b4bc-3c2c-4301-9e69-b25287393935 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:30.821199 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 20 20:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:30.827356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde4f1d-17d7-4e35-96ac-ba0eec62a105 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:30.828126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde4f1d-17d7-4e35-96ac-ba0eec62a105 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:30.828126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde4f1d-17d7-4e35-96ac-ba0eec62a105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:30.912156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fdde4f1d-17d7-4e35-96ac-ba0eec62a105 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:30.913380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fdde4f1d-17d7-4e35-96ac-ba0eec62a105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:30.915500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:31.014350 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 20 20:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:31.179865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5b4bc-3c2c-4301-9e69-b25287393935 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:31.182196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dbe5b4bc-3c2c-4301-9e69-b25287393935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:31.343213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5b4bc-3c2c-4301-9e69-b25287393935 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:31.375689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:31.477552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72310cb-fc67-41f6-aa4e-e854a76eb5c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:31.478367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72310cb-fc67-41f6-aa4e-e854a76eb5c7 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:31.478720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72310cb-fc67-41f6-aa4e-e854a76eb5c7 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:31.478971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72310cb-fc67-41f6-aa4e-e854a76eb5c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:31.481520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e72310cb-fc67-41f6-aa4e-e854a76eb5c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:31.582558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72310cb-fc67-41f6-aa4e-e854a76eb5c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:31.584169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72310cb-fc67-41f6-aa4e-e854a76eb5c7 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:31.603802 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 711/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:33.659181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9273e57-af81-428b-abf2-c0d76f8189fd None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:33.659767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9273e57-af81-428b-abf2-c0d76f8189fd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:33.659767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9273e57-af81-428b-abf2-c0d76f8189fd None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:33.660986 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 20 20:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:33.667596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb216d-eb19-43fb-a908-87e7fd8e5c25 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:33.668316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb216d-eb19-43fb-a908-87e7fd8e5c25 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:33.668447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb216d-eb19-43fb-a908-87e7fd8e5c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:33.714569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:33.743507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-61bb216d-eb19-43fb-a908-87e7fd8e5c25 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:33.744752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-61bb216d-eb19-43fb-a908-87e7fd8e5c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:33.856479 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 20 20:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:34.204956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9273e57-af81-428b-abf2-c0d76f8189fd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:34.206548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e9273e57-af81-428b-abf2-c0d76f8189fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:34.316700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9273e57-af81-428b-abf2-c0d76f8189fd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:34.349853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:34.395052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75790d1-0843-410d-a420-3ea48011fecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:34.395659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75790d1-0843-410d-a420-3ea48011fecd service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:34.395931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75790d1-0843-410d-a420-3ea48011fecd service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:34.396192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75790d1-0843-410d-a420-3ea48011fecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:34.397869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d75790d1-0843-410d-a420-3ea48011fecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:34.509660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75790d1-0843-410d-a420-3ea48011fecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:34.511102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75790d1-0843-410d-a420-3ea48011fecd service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:34.531192 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 731/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:36.909997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9b430-830f-4408-8002-be2d9c6baab1 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:36.910604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9b430-830f-4408-8002-be2d9c6baab1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:36.910604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9b430-830f-4408-8002-be2d9c6baab1 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:36.912628 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 20 20:46:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:36.921762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab24ab-eaaa-4175-97e4-f95765af8b09 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:36.922187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab24ab-eaaa-4175-97e4-f95765af8b09 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:36.922317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab24ab-eaaa-4175-97e4-f95765af8b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:36.997041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-96ab24ab-eaaa-4175-97e4-f95765af8b09 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:36.998330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-96ab24ab-eaaa-4175-97e4-f95765af8b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:37.011653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:37.113275 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 20 20:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:37.464122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9b430-830f-4408-8002-be2d9c6baab1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:37.467401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-85c9b430-830f-4408-8002-be2d9c6baab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:37.597384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9b430-830f-4408-8002-be2d9c6baab1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:37.611137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:37.678775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182756c-76d4-4661-b957-4f60522aa91a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:37.679370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182756c-76d4-4661-b957-4f60522aa91a service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:37.679634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182756c-76d4-4661-b957-4f60522aa91a service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:37.679898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182756c-76d4-4661-b957-4f60522aa91a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:37.681550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e182756c-76d4-4661-b957-4f60522aa91a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:37.792715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182756c-76d4-4661-b957-4f60522aa91a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:37.794132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182756c-76d4-4661-b957-4f60522aa91a service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:37.812498 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 714/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:40.455867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac3e66-b941-45f5-bbdf-ab4f4bd8c9b8 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:40.456497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac3e66-b941-45f5-bbdf-ab4f4bd8c9b8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:40.456497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac3e66-b941-45f5-bbdf-ab4f4bd8c9b8 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:40.457859 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 20 20:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:40.463437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bffb08-1b66-4bff-aa88-a74401363736 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:40.463861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bffb08-1b66-4bff-aa88-a74401363736 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:40.463985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bffb08-1b66-4bff-aa88-a74401363736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:40.513957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:40.532534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b2bffb08-1b66-4bff-aa88-a74401363736 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:40.533749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b2bffb08-1b66-4bff-aa88-a74401363736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:40.632309 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 20 20:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:40.989371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac3e66-b941-45f5-bbdf-ab4f4bd8c9b8 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:40.991864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4aac3e66-b941-45f5-bbdf-ab4f4bd8c9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:41.107928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac3e66-b941-45f5-bbdf-ab4f4bd8c9b8 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:41.148949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:41.195286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d64966-1fd3-4eff-88b8-1ceced54ff82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:41.196094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d64966-1fd3-4eff-88b8-1ceced54ff82 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:41.196448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d64966-1fd3-4eff-88b8-1ceced54ff82 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:41.196775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d64966-1fd3-4eff-88b8-1ceced54ff82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:41.199326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-01d64966-1fd3-4eff-88b8-1ceced54ff82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:41.318829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d64966-1fd3-4eff-88b8-1ceced54ff82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:41.320185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d64966-1fd3-4eff-88b8-1ceced54ff82 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:41.339763 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 734/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:43.686034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef39803-3b28-4fe9-85e4-089ac95c251b None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:43.686967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef39803-3b28-4fe9-85e4-089ac95c251b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:43.686967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef39803-3b28-4fe9-85e4-089ac95c251b None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:43.689291 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 20 20:46:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:43.695354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e5c65-4ff7-471b-a69d-4e6d1acd0a1d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:43.695809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e5c65-4ff7-471b-a69d-4e6d1acd0a1d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:43.695913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e5c65-4ff7-471b-a69d-4e6d1acd0a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:43.770318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4a6e5c65-4ff7-471b-a69d-4e6d1acd0a1d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:43.771521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4a6e5c65-4ff7-471b-a69d-4e6d1acd0a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:43.792971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:43.881256 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 20 20:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:44.247111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef39803-3b28-4fe9-85e4-089ac95c251b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:44.249048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7ef39803-3b28-4fe9-85e4-089ac95c251b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:44.368461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef39803-3b28-4fe9-85e4-089ac95c251b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:44.390619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:44.447790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260b880-d84c-4ac1-b8f8-429122fd0394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:44.448408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260b880-d84c-4ac1-b8f8-429122fd0394 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:44.448667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260b880-d84c-4ac1-b8f8-429122fd0394 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:44.448955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260b880-d84c-4ac1-b8f8-429122fd0394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:44.450590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e260b880-d84c-4ac1-b8f8-429122fd0394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:44.585908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260b880-d84c-4ac1-b8f8-429122fd0394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:44.587284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260b880-d84c-4ac1-b8f8-429122fd0394 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:44.608309 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 717/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:46.858961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8602e-6bb6-4a1f-8a92-450771407765 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:46.859620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8602e-6bb6-4a1f-8a92-450771407765 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:46.859620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8602e-6bb6-4a1f-8a92-450771407765 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:46.861067 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 20 20:46:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:46.869024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45190d7-676f-4503-b5af-7787f6380a86 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:46.869469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45190d7-676f-4503-b5af-7787f6380a86 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:46.869575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45190d7-676f-4503-b5af-7787f6380a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:46.938985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:46.942409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b45190d7-676f-4503-b5af-7787f6380a86 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:46.944922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b45190d7-676f-4503-b5af-7787f6380a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:47.070812 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 20 20:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:46:47.415292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8602e-6bb6-4a1f-8a92-450771407765 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:47.417841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-88f8602e-6bb6-4a1f-8a92-450771407765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:47.547742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8602e-6bb6-4a1f-8a92-450771407765 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:47.600165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:47.626677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c42e5-4ab0-4a50-bb86-5ed7c19c4c68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:47.627274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c42e5-4ab0-4a50-bb86-5ed7c19c4c68 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:47.627542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c42e5-4ab0-4a50-bb86-5ed7c19c4c68 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:47.628050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c42e5-4ab0-4a50-bb86-5ed7c19c4c68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:47.629682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d60c42e5-4ab0-4a50-bb86-5ed7c19c4c68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:47.746460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c42e5-4ab0-4a50-bb86-5ed7c19c4c68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:47.748168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c42e5-4ab0-4a50-bb86-5ed7c19c4c68 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:47.767564 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 737/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:50.047499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf11bef-7563-440e-8b1b-de2692ccbff7 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:50.048349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf11bef-7563-440e-8b1b-de2692ccbff7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:50.048349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf11bef-7563-440e-8b1b-de2692ccbff7 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:50.050372 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 20 20:46:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:50.057026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d1672-02b6-4a78-81ae-613a5627a2ff None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:50.057477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d1672-02b6-4a78-81ae-613a5627a2ff None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:50.057582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d1672-02b6-4a78-81ae-613a5627a2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:50.138782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d84d1672-02b6-4a78-81ae-613a5627a2ff None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:50.140000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d84d1672-02b6-4a78-81ae-613a5627a2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:50.171043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:50.242767 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 20 20:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:50.593645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf11bef-7563-440e-8b1b-de2692ccbff7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:50.595989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dbf11bef-7563-440e-8b1b-de2692ccbff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:50.710877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf11bef-7563-440e-8b1b-de2692ccbff7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:50.740993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:50.788229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81053265-5dc2-41aa-94c5-9e4972495d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:50.788886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81053265-5dc2-41aa-94c5-9e4972495d8e service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:50.789156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81053265-5dc2-41aa-94c5-9e4972495d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:50.789418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81053265-5dc2-41aa-94c5-9e4972495d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:50.791005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-81053265-5dc2-41aa-94c5-9e4972495d8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:51.039480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81053265-5dc2-41aa-94c5-9e4972495d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:51.041127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81053265-5dc2-41aa-94c5-9e4972495d8e service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:51.059653 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 720/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:46:53.313958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dbf83-6973-4a39-8aef-eaf75297c6a2 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:53.314558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dbf83-6973-4a39-8aef-eaf75297c6a2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:53.314558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dbf83-6973-4a39-8aef-eaf75297c6a2 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:53.315844 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 20 20:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:53.321920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60cf5b-95a0-4704-8855-fda490b152af None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:53.322363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60cf5b-95a0-4704-8855-fda490b152af None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:53.322458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60cf5b-95a0-4704-8855-fda490b152af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:53.390767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ed60cf5b-95a0-4704-8855-fda490b152af None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:53.391879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ed60cf5b-95a0-4704-8855-fda490b152af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:53.400889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:53.505189 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 20 20:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:53.860205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dbf83-6973-4a39-8aef-eaf75297c6a2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:53.861993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0e3dbf83-6973-4a39-8aef-eaf75297c6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:53.976512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dbf83-6973-4a39-8aef-eaf75297c6a2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:54.045146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:54.064035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f49ae4-fa19-4bb3-82da-ea9d50cd15cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:54.064668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f49ae4-fa19-4bb3-82da-ea9d50cd15cc service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:54.064938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f49ae4-fa19-4bb3-82da-ea9d50cd15cc service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:54.065186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f49ae4-fa19-4bb3-82da-ea9d50cd15cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:54.066787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-66f49ae4-fa19-4bb3-82da-ea9d50cd15cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:54.184063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f49ae4-fa19-4bb3-82da-ea9d50cd15cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:54.186163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f49ae4-fa19-4bb3-82da-ea9d50cd15cc service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:54.205332 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 740/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:56.422751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae63d7d-f48a-4c34-849f-37703715154d None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:56.423637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae63d7d-f48a-4c34-849f-37703715154d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:56.423637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae63d7d-f48a-4c34-849f-37703715154d None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:56.425812 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 20 20:46:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:56.431352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c0d8-738a-46c6-a6cd-5adb760ea374 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:56.431822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c0d8-738a-46c6-a6cd-5adb760ea374 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:56.431945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c0d8-738a-46c6-a6cd-5adb760ea374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:56.498504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b280c0d8-738a-46c6-a6cd-5adb760ea374 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:56.499607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b280c0d8-738a-46c6-a6cd-5adb760ea374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:56.537939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:56.596968 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 20 20:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:46:56.944519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae63d7d-f48a-4c34-849f-37703715154d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:46:56.946795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3ae63d7d-f48a-4c34-849f-37703715154d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:57.068621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae63d7d-f48a-4c34-849f-37703715154d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:46:57.115537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:57.163120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91844f-bd04-4412-8809-c110070cdf48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:46:57.163720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91844f-bd04-4412-8809-c110070cdf48 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:57.164016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91844f-bd04-4412-8809-c110070cdf48 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:57.164270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91844f-bd04-4412-8809-c110070cdf48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:57.165895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fd91844f-bd04-4412-8809-c110070cdf48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:57.280777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91844f-bd04-4412-8809-c110070cdf48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:46:57.282237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91844f-bd04-4412-8809-c110070cdf48 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:46:57.300844 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 723/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:46:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:46:59.549232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a98dc8-08d8-4d28-b901-2a2abe131c6b None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:59.550152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a98dc8-08d8-4d28-b901-2a2abe131c6b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:59.550152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a98dc8-08d8-4d28-b901-2a2abe131c6b None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:59.552204 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 20 20:46:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:46:59.561058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87397f-fd7f-4e37-9ea7-933744887292 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:46:59.561545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87397f-fd7f-4e37-9ea7-933744887292 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:46:59.561660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87397f-fd7f-4e37-9ea7-933744887292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:46:59.642742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9a87397f-fd7f-4e37-9ea7-933744887292 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:46:59.643938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9a87397f-fd7f-4e37-9ea7-933744887292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:46:59.663477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:46:59.750287 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 20 20:46:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:47:00.116254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a98dc8-08d8-4d28-b901-2a2abe131c6b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:47:00.117854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-87a98dc8-08d8-4d28-b901-2a2abe131c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:00.231202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a98dc8-08d8-4d28-b901-2a2abe131c6b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:47:00.309770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:00.318497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b596c-aa0f-43b8-beef-548d289bc3e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:47:00.319104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b596c-aa0f-43b8-beef-548d289bc3e4 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:00.319372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b596c-aa0f-43b8-beef-548d289bc3e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:00.319628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b596c-aa0f-43b8-beef-548d289bc3e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:00.321282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fd0b596c-aa0f-43b8-beef-548d289bc3e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:00.437971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b596c-aa0f-43b8-beef-548d289bc3e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:47:00.439297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b596c-aa0f-43b8-beef-548d289bc3e4 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:47:00.459709 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 743/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:47:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:47:02.684848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255b515-b36c-4ca5-94ef-936ec99ef296 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:02.685585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255b515-b36c-4ca5-94ef-936ec99ef296 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:02.685856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255b515-b36c-4ca5-94ef-936ec99ef296 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:02.687267 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 20 20:47:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:47:02.694382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e7282-264e-4ee4-b6db-a42cd698a2e4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:02.694868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e7282-264e-4ee4-b6db-a42cd698a2e4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:02.694978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e7282-264e-4ee4-b6db-a42cd698a2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:02.765545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-626e7282-264e-4ee4-b6db-a42cd698a2e4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:47:02.766779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-626e7282-264e-4ee4-b6db-a42cd698a2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:02.820136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:02.869901 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 20 20:47:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:47:03.226045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255b515-b36c-4ca5-94ef-936ec99ef296 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:47:03.228662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7255b515-b36c-4ca5-94ef-936ec99ef296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:03.351349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255b515-b36c-4ca5-94ef-936ec99ef296 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:47:03.400891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:03.428093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cd202-8bf1-432b-a974-cdd7a816cead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:47:03.428749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cd202-8bf1-432b-a974-cdd7a816cead service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:03.429017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cd202-8bf1-432b-a974-cdd7a816cead service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:03.429279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cd202-8bf1-432b-a974-cdd7a816cead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:03.430935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d57cd202-8bf1-432b-a974-cdd7a816cead service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:03.551015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cd202-8bf1-432b-a974-cdd7a816cead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:47:03.552499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cd202-8bf1-432b-a974-cdd7a816cead service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:47:03.570854 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 726/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:47:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:47:12.868113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=22 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:13.568644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=66 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:28.085396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0ea08-0bef-45a7-940b-f8d7278beba1 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:28.085931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0ea08-0bef-45a7-940b-f8d7278beba1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:28.085931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0ea08-0bef-45a7-940b-f8d7278beba1 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:28.087162 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 745/1471] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:47:28.093043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafff272-a509-4abf-8b20-e03a816f689d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:28.093502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafff272-a509-4abf-8b20-e03a816f689d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:28.093606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafff272-a509-4abf-8b20-e03a816f689d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:28.164685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fafff272-a509-4abf-8b20-e03a816f689d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:47:28.165972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fafff272-a509-4abf-8b20-e03a816f689d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:28.266969 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 727/1472] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:47:28.627253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10ae28-a0fb-4644-b293-0ab3be88259b None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:28.627500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10ae28-a0fb-4644-b293-0ab3be88259b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:28.627777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10ae28-a0fb-4644-b293-0ab3be88259b None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:28.629139 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 746/1473] 10.222.0.23 () {54 vars in 967 bytes} [Sat Aug 20 20:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:47:28.634048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccca91-44d2-44a0-a18b-b5850bc7ec00 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:28.634323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccca91-44d2-44a0-a18b-b5850bc7ec00 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:28.634582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccca91-44d2-44a0-a18b-b5850bc7ec00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:28.720317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fbccca91-44d2-44a0-a18b-b5850bc7ec00 None None] MFA Rules not processed for user `698d5a7fc34a46ba909152414a0860d7`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:47:28.721562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fbccca91-44d2-44a0-a18b-b5850bc7ec00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:28.748693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:28.821084 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 728/1474] 10.222.0.23 () {58 vars in 1064 bytes} [Sat Aug 20 20:47:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:47:28.828933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10ae28-a0fb-4644-b293-0ab3be88259b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:47:28.830421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fd10ae28-a0fb-4644-b293-0ab3be88259b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:28.942512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10ae28-a0fb-4644-b293-0ab3be88259b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:47:29.018415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca094ce-3956-4b93-832e-a23213667b29 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:47:29.019017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca094ce-3956-4b93-832e-a23213667b29 service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:29.019286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca094ce-3956-4b93-832e-a23213667b29 service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:29.019547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca094ce-3956-4b93-832e-a23213667b29 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:29.021187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cca094ce-3956-4b93-832e-a23213667b29 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:29.081164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:29.143680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca094ce-3956-4b93-832e-a23213667b29 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:47:29.145229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca094ce-3956-4b93-832e-a23213667b29 service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:47:29.166856 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 747/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:47:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:47:31.507113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897ad0d-3d78-476f-929d-afbd86e6d812 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:31.507928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897ad0d-3d78-476f-929d-afbd86e6d812 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:31.507928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897ad0d-3d78-476f-929d-afbd86e6d812 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:31.510004 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 729/1476] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:47:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:47:31.516888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf44248-e6d3-419f-99d0-1542304f42d6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:31.517368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf44248-e6d3-419f-99d0-1542304f42d6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:31.517368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf44248-e6d3-419f-99d0-1542304f42d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:31.587887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9bf44248-e6d3-419f-99d0-1542304f42d6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:47:31.589188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9bf44248-e6d3-419f-99d0-1542304f42d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:31.696453 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 748/1477] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:47:32.032219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897ad0d-3d78-476f-929d-afbd86e6d812 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:47:32.034718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c897ad0d-3d78-476f-929d-afbd86e6d812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:32.184214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897ad0d-3d78-476f-929d-afbd86e6d812 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:47:32.244637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:32.292054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57da26-2807-4217-bd1d-b16e5081fee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:47:32.292602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57da26-2807-4217-bd1d-b16e5081fee1 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:32.292868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57da26-2807-4217-bd1d-b16e5081fee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:32.293128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57da26-2807-4217-bd1d-b16e5081fee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:32.293754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57da26-2807-4217-bd1d-b16e5081fee1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:47:32.295513 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-df57da26-2807-4217-bd1d-b16e5081fee1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:47:32.295513 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 20:47:32.295513 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 20:47:32.296692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57da26-2807-4217-bd1d-b16e5081fee1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:47:32.316868 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 730/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Aug 20 20:47:32 2022] GET /identity/v3/projects => generated 2553 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:47:34.576440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fabf1-da95-49de-b3ca-93d7ad03302e None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:34.577018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fabf1-da95-49de-b3ca-93d7ad03302e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:34.577018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fabf1-da95-49de-b3ca-93d7ad03302e None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:34.578273 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 749/1479] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:47:34.583752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efc8ec-fe82-4de2-b517-9a1fd644f00b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:34.584203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efc8ec-fe82-4de2-b517-9a1fd644f00b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:34.584308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efc8ec-fe82-4de2-b517-9a1fd644f00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:34.650923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-03efc8ec-fe82-4de2-b517-9a1fd644f00b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:47:34.652199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-03efc8ec-fe82-4de2-b517-9a1fd644f00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:34.762392 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 731/1480] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:47:35.133093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fabf1-da95-49de-b3ca-93d7ad03302e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:47:35.135537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0f1fabf1-da95-49de-b3ca-93d7ad03302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:35.146287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:35.288113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fabf1-da95-49de-b3ca-93d7ad03302e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:47:35.380537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:35.413837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee52094-a690-4671-a334-ee3840d76080 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:47:35.414644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee52094-a690-4671-a334-ee3840d76080 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:35.414994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee52094-a690-4671-a334-ee3840d76080 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:35.415302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee52094-a690-4671-a334-ee3840d76080 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:35.417858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2ee52094-a690-4671-a334-ee3840d76080 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:35.517977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee52094-a690-4671-a334-ee3840d76080 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:47:35.519560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee52094-a690-4671-a334-ee3840d76080 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:47:35.537857 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 750/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:47:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:47:37.925047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc5a49-7f66-4ad2-b50f-e116eb0c0765 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:37.925943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc5a49-7f66-4ad2-b50f-e116eb0c0765 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:37.925943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc5a49-7f66-4ad2-b50f-e116eb0c0765 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:37.927463 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 732/1482] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:47:37.936834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2f73f-ab41-4afb-aad0-7f0c1cf4943b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:37.937534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2f73f-ab41-4afb-aad0-7f0c1cf4943b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:37.937534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2f73f-ab41-4afb-aad0-7f0c1cf4943b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:38.022726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1ad2f73f-ab41-4afb-aad0-7f0c1cf4943b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:47:38.023877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1ad2f73f-ab41-4afb-aad0-7f0c1cf4943b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:38.071008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:38.128425 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 751/1483] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:47:38.474552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc5a49-7f66-4ad2-b50f-e116eb0c0765 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:47:38.476901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6ffc5a49-7f66-4ad2-b50f-e116eb0c0765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:38.509343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:38.594697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc5a49-7f66-4ad2-b50f-e116eb0c0765 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:47:38.679934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1ba9c-558b-4c48-8569-e74f2c4a367f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:47:38.680578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1ba9c-558b-4c48-8569-e74f2c4a367f service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:38.680882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1ba9c-558b-4c48-8569-e74f2c4a367f service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:38.681115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1ba9c-558b-4c48-8569-e74f2c4a367f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:38.682751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-26f1ba9c-558b-4c48-8569-e74f2c4a367f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:38.764901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:38.801750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1ba9c-558b-4c48-8569-e74f2c4a367f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:47:38.803231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1ba9c-558b-4c48-8569-e74f2c4a367f service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:47:38.822945 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 733/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:47:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:47:41.064163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a3da3-9618-4726-8363-f2fb5ab6ef57 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:41.064800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a3da3-9618-4726-8363-f2fb5ab6ef57 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:41.064800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a3da3-9618-4726-8363-f2fb5ab6ef57 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:41.066061 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 752/1485] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:47:41.071885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a3f45-b73d-4919-b7b0-cbb485f6e342 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:41.072359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a3f45-b73d-4919-b7b0-cbb485f6e342 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:41.072489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a3f45-b73d-4919-b7b0-cbb485f6e342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:41.143519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-517a3f45-b73d-4919-b7b0-cbb485f6e342 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:47:41.144744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-517a3f45-b73d-4919-b7b0-cbb485f6e342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:41.249850 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 734/1486] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:47:41.600830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a3da3-9618-4726-8363-f2fb5ab6ef57 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:47:41.603226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-265a3da3-9618-4726-8363-f2fb5ab6ef57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:41.727150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a3da3-9618-4726-8363-f2fb5ab6ef57 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:47:41.813148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:41.815311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8bb63-f3f3-4ea5-b3d9-42758418c5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:47:41.815989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8bb63-f3f3-4ea5-b3d9-42758418c5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:41.816265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8bb63-f3f3-4ea5-b3d9-42758418c5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:41.816603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8bb63-f3f3-4ea5-b3d9-42758418c5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:41.819490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2ef8bb63-f3f3-4ea5-b3d9-42758418c5d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:41.933189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8bb63-f3f3-4ea5-b3d9-42758418c5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:47:41.934541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8bb63-f3f3-4ea5-b3d9-42758418c5d5 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:47:41.954153 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 753/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:47:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:47:44.393020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f87a7-5209-490e-b4a4-edf607dc00a2 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:44.393623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f87a7-5209-490e-b4a4-edf607dc00a2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:44.393623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f87a7-5209-490e-b4a4-edf607dc00a2 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:44.394890 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 735/1488] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:47:44.401218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96501c77-8c0f-4761-8f39-0f9c7788ae2a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:44.401663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96501c77-8c0f-4761-8f39-0f9c7788ae2a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:44.401772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96501c77-8c0f-4761-8f39-0f9c7788ae2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:44.472328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-96501c77-8c0f-4761-8f39-0f9c7788ae2a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:47:44.473538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-96501c77-8c0f-4761-8f39-0f9c7788ae2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:44.528240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:44.578641 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 754/1489] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:47:44.937228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f87a7-5209-490e-b4a4-edf607dc00a2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:47:44.939317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ab2f87a7-5209-490e-b4a4-edf607dc00a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:44.982793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:45.055449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f87a7-5209-490e-b4a4-edf607dc00a2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:47:45.137534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bfac9-f81f-4315-a129-f15f4bbefd6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:47:45.138197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bfac9-f81f-4315-a129-f15f4bbefd6b service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:45.138528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bfac9-f81f-4315-a129-f15f4bbefd6b service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:45.138793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bfac9-f81f-4315-a129-f15f4bbefd6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:45.140411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-855bfac9-f81f-4315-a129-f15f4bbefd6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:45.236018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:45.269376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bfac9-f81f-4315-a129-f15f4bbefd6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:47:45.270814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bfac9-f81f-4315-a129-f15f4bbefd6b service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:47:45.291554 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 736/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:47:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:47:47.663995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd525a-869b-45bb-a99a-161177ad3060 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:47.664582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd525a-869b-45bb-a99a-161177ad3060 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:47.664582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd525a-869b-45bb-a99a-161177ad3060 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:47.665813 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 755/1491] 10.222.0.23 () {54 vars in 927 bytes} [Sat Aug 20 20:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:47:47.673137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578843a-1c78-475f-b448-16aa2763774d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:47.673881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578843a-1c78-475f-b448-16aa2763774d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:47.674047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578843a-1c78-475f-b448-16aa2763774d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:47.753794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9578843a-1c78-475f-b448-16aa2763774d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:47:47.755002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9578843a-1c78-475f-b448-16aa2763774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:47.859382 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 737/1492] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Aug 20 20:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:47:48.218736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd525a-869b-45bb-a99a-161177ad3060 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:47:48.220269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-67cd525a-869b-45bb-a99a-161177ad3060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:48.334368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd525a-869b-45bb-a99a-161177ad3060 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:47:48.414138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58c7a5-89c1-4746-be91-ceb41358d13c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:47:48.414748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58c7a5-89c1-4746-be91-ceb41358d13c service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:47:48.415014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58c7a5-89c1-4746-be91-ceb41358d13c service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:47:48.415272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58c7a5-89c1-4746-be91-ceb41358d13c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:47:48.416933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fe58c7a5-89c1-4746-be91-ceb41358d13c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:47:48.427003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:48.562423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58c7a5-89c1-4746-be91-ceb41358d13c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:47:48.563875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58c7a5-89c1-4746-be91-ceb41358d13c service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:47:48.583991 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 756/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:47:57.856858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=82 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:47:58.582031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=50 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:49:13.900590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a63854-6eac-4432-8401-f7404f215bc9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:13.901244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a63854-6eac-4432-8401-f7404f215bc9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:13.901244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a63854-6eac-4432-8401-f7404f215bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:13.990611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-36a63854-6eac-4432-8401-f7404f215bc9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:49:13.991966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-36a63854-6eac-4432-8401-f7404f215bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:14.100701 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 738/1494] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:49:14.112079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2509a8-8906-4ffd-a6db-5b0b31e99a79 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:14.112647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2509a8-8906-4ffd-a6db-5b0b31e99a79 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:14.112647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2509a8-8906-4ffd-a6db-5b0b31e99a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:14.189630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-eb2509a8-8906-4ffd-a6db-5b0b31e99a79 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:49:14.190892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-eb2509a8-8906-4ffd-a6db-5b0b31e99a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:14.297854 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 757/1495] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:49:14.302969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a63854-6eac-4432-8401-f7404f215bc9 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:49:14.304771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-36a63854-6eac-4432-8401-f7404f215bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:14.317000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a63854-6eac-4432-8401-f7404f215bc9 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:49:14.390093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:49:14.430493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba089be-12f1-4022-b92f-4cfbcdd8d7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:49:14.431111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba089be-12f1-4022-b92f-4cfbcdd8d7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:14.431418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba089be-12f1-4022-b92f-4cfbcdd8d7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:14.431681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba089be-12f1-4022-b92f-4cfbcdd8d7d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:14.432355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba089be-12f1-4022-b92f-4cfbcdd8d7d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:49:14.432693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba089be-12f1-4022-b92f-4cfbcdd8d7d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:49:14.433716 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-7ba089be-12f1-4022-b92f-4cfbcdd8d7d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:49:14.433716 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 20:49:14.433716 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 20:49:14.434614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba089be-12f1-4022-b92f-4cfbcdd8d7d6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:49:14.447132 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 739/1496] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:49:14.455835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2509a8-8906-4ffd-a6db-5b0b31e99a79 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:49:14.465610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2509a8-8906-4ffd-a6db-5b0b31e99a79 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:49:14.662285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:49:14.698120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bc90d-1a60-4ed9-aab6-cd5c3d6fa62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:49:14.698744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bc90d-1a60-4ed9-aab6-cd5c3d6fa62e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:14.699010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bc90d-1a60-4ed9-aab6-cd5c3d6fa62e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:14.699274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bc90d-1a60-4ed9-aab6-cd5c3d6fa62e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:14.699881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bc90d-1a60-4ed9-aab6-cd5c3d6fa62e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:49:14.701718 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-1e1bc90d-1a60-4ed9-aab6-cd5c3d6fa62e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:49:14.701718 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 20:49:14.701718 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 20:49:14.702913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bc90d-1a60-4ed9-aab6-cd5c3d6fa62e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:49:14.768250 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 758/1497] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:49:14 2022] POST /identity/v3/projects => generated 379 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:49:14.771961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba089be-12f1-4022-b92f-4cfbcdd8d7d6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:49:14.773594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7ba089be-12f1-4022-b92f-4cfbcdd8d7d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:14.809799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba089be-12f1-4022-b92f-4cfbcdd8d7d6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:49:14.910637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15125e51-d2a2-4da3-ae1f-85ae37bc5030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:49:14.911447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15125e51-d2a2-4da3-ae1f-85ae37bc5030 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:14.911447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15125e51-d2a2-4da3-ae1f-85ae37bc5030 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:14.911743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15125e51-d2a2-4da3-ae1f-85ae37bc5030 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:14.912442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15125e51-d2a2-4da3-ae1f-85ae37bc5030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:49:14.913663 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-15125e51-d2a2-4da3-ae1f-85ae37bc5030 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:49:14.913663 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 20:49:14.913663 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 20:49:14.914872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15125e51-d2a2-4da3-ae1f-85ae37bc5030 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:49:14.960861 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 740/1498] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:49:14 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:49:14.965323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bc90d-1a60-4ed9-aab6-cd5c3d6fa62e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:49:14.966942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1e1bc90d-1a60-4ed9-aab6-cd5c3d6fa62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:15.027686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bc90d-1a60-4ed9-aab6-cd5c3d6fa62e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:49:15.114037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:49:15.128091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02dfcb-d227-4c32-b849-366aff8a67af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:49:15.128743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02dfcb-d227-4c32-b849-366aff8a67af admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:15.129009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02dfcb-d227-4c32-b849-366aff8a67af admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:15.129289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02dfcb-d227-4c32-b849-366aff8a67af admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:15.129972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02dfcb-d227-4c32-b849-366aff8a67af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:49:15.131100 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-ab02dfcb-d227-4c32-b849-366aff8a67af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:49:15.131100 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 20:49:15.131100 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 20:49:15.132768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02dfcb-d227-4c32-b849-366aff8a67af admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:49:15.146660 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 759/1499] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:49:15.151211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15125e51-d2a2-4da3-ae1f-85ae37bc5030 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:49:15.152876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-15125e51-d2a2-4da3-ae1f-85ae37bc5030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:15.164846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15125e51-d2a2-4da3-ae1f-85ae37bc5030 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:49:15.191801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:49:15.278996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95a71a-9c61-4310-a520-5b38f33eebe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:49:15.279597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95a71a-9c61-4310-a520-5b38f33eebe7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:15.279864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95a71a-9c61-4310-a520-5b38f33eebe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:15.280123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95a71a-9c61-4310-a520-5b38f33eebe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:15.280795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95a71a-9c61-4310-a520-5b38f33eebe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:49:15.281843 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-6e95a71a-9c61-4310-a520-5b38f33eebe7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:49:15.281843 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 20:49:15.281843 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 20:49:15.282740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95a71a-9c61-4310-a520-5b38f33eebe7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:49:15.296087 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 741/1500] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:49:15.300584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02dfcb-d227-4c32-b849-366aff8a67af admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:49:15.302141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ab02dfcb-d227-4c32-b849-366aff8a67af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:15.349576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02dfcb-d227-4c32-b849-366aff8a67af admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:49:15.440668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b1633-db5c-45d8-a7ef-0ca5f8dd9471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:49:15.441235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b1633-db5c-45d8-a7ef-0ca5f8dd9471 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:15.441499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b1633-db5c-45d8-a7ef-0ca5f8dd9471 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:15.441761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b1633-db5c-45d8-a7ef-0ca5f8dd9471 admin admin] PATH_INFO: `/v3/projects/fa25fbf1a3b04391940bd0de1179ce1e/users/dd00e0d52d194a3d834b34ec79a19acc/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:15.467491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b1633-db5c-45d8-a7ef-0ca5f8dd9471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa25fbf1a3b04391940bd0de1179ce1e, user_id=dd00e0d52d194a3d834b34ec79a19acc, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:49:15.468566 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-592b1633-db5c-45d8-a7ef-0ca5f8dd9471 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:49:15.468566 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 20:49:15.468566 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 20:49:15.470469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b1633-db5c-45d8-a7ef-0ca5f8dd9471 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:49:15.496753 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 760/1501] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:49:15 2022] PUT /identity/v3/projects/fa25fbf1a3b04391940bd0de1179ce1e/users/dd00e0d52d194a3d834b34ec79a19acc/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:49:15.510052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9777140-d9c8-4497-adac-7863d10cd744 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:15.510328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9777140-d9c8-4497-adac-7863d10cd744 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:15.510595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9777140-d9c8-4497-adac-7863d10cd744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:15.571067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a9777140-d9c8-4497-adac-7863d10cd744 None None] MFA Rules not processed for user `dd00e0d52d194a3d834b34ec79a19acc`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:49:15.572294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a9777140-d9c8-4497-adac-7863d10cd744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:15.662863 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 742/1502] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:49:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:49:15.676742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c469e8e-49e8-4165-90c1-1726f209b881 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:15.677019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c469e8e-49e8-4165-90c1-1726f209b881 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:15.677285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c469e8e-49e8-4165-90c1-1726f209b881 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:15.678671 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 761/1503] 10.222.0.23 () {54 vars in 964 bytes} [Sat Aug 20 20:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:49:15.683796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3407f5c-0c37-4c73-905e-11ec2728a6c9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:15.684080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3407f5c-0c37-4c73-905e-11ec2728a6c9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:15.684349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3407f5c-0c37-4c73-905e-11ec2728a6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:15.723511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:49:15.783469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c3407f5c-0c37-4c73-905e-11ec2728a6c9 None None] MFA Rules not processed for user `53928836828d45fbb427dd92492cae20`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:49:15.784980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c3407f5c-0c37-4c73-905e-11ec2728a6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:15.915788 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 743/1504] 10.222.0.23 () {58 vars in 1061 bytes} [Sat Aug 20 20:49:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:49:15.925314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c469e8e-49e8-4165-90c1-1726f209b881 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:49:15.927906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1c469e8e-49e8-4165-90c1-1726f209b881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:15.995226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:49:16.059219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c469e8e-49e8-4165-90c1-1726f209b881 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:49:16.148199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecadea-17cf-4db0-b92d-46081720c998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:49:16.148980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecadea-17cf-4db0-b92d-46081720c998 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:16.149245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecadea-17cf-4db0-b92d-46081720c998 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:16.149502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecadea-17cf-4db0-b92d-46081720c998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:16.151144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-81ecadea-17cf-4db0-b92d-46081720c998 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:16.286021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecadea-17cf-4db0-b92d-46081720c998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:49:16.288555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecadea-17cf-4db0-b92d-46081720c998 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:49:16.292483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:49:16.310507 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 762/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:49:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:49:16.370565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b47b97-3388-4e68-805d-3c79f8b1ebb3 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:16.370904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b47b97-3388-4e68-805d-3c79f8b1ebb3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:16.371158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b47b97-3388-4e68-805d-3c79f8b1ebb3 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:16.373596 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 744/1506] 10.222.0.23 () {52 vars in 830 bytes} [Sat Aug 20 20:49:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:49:16.379292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecadea-17cf-4db0-b92d-46081720c998 service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:49:16.380580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-81ecadea-17cf-4db0-b92d-46081720c998 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:16.507804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecadea-17cf-4db0-b92d-46081720c998 service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:49:16.611440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:49:16.611964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:16.612249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:16.612542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] PATH_INFO: `/v3/users/dd00e0d52d194a3d834b34ec79a19acc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:16.627678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd00e0d52d194a3d834b34ec79a19acc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:49:16.628565 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:49:16.628565 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 20:49:16.628565 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 20:49:16.629855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:49:16.652505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:49:16.721470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd00e0d52d194a3d834b34ec79a19acc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:49:16.734899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd00e0d52d194a3d834b34ec79a19acc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:49:16.755143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd00e0d52d194a3d834b34ec79a19acc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:49:16.790108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd00e0d52d194a3d834b34ec79a19acc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:49:16.790853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd00e0d52d194a3d834b34ec79a19acc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:49:16.797224 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 763/1507] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:49:16 2022] DELETE /identity/v3/users/dd00e0d52d194a3d834b34ec79a19acc => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:49:16.802621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accadb83-709e-4355-b6cd-231129588be4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:16.803216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accadb83-709e-4355-b6cd-231129588be4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:16.804174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accadb83-709e-4355-b6cd-231129588be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:16.857549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-accadb83-709e-4355-b6cd-231129588be4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:49:16.858831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-accadb83-709e-4355-b6cd-231129588be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:16.930380 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 745/1508] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:49:16.941505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:49:16.943161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:17.028470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3ee87-d08c-43a1-9886-c30dedc00f9b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:49:17.032287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f267515-2bb1-4060-af76-bb549297dd1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:49:17.032949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f267515-2bb1-4060-af76-bb549297dd1b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:17.033217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f267515-2bb1-4060-af76-bb549297dd1b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:17.033473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f267515-2bb1-4060-af76-bb549297dd1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:17.043775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f267515-2bb1-4060-af76-bb549297dd1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:49:17.045301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f267515-2bb1-4060-af76-bb549297dd1b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:49:17.055404 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 764/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:49:17.406510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accadb83-709e-4355-b6cd-231129588be4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:49:17.408024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-accadb83-709e-4355-b6cd-231129588be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:49:17.421415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accadb83-709e-4355-b6cd-231129588be4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:49:17.429170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4681c7b-413b-4260-bf6c-0d35906dd8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:49:17.429814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4681c7b-413b-4260-bf6c-0d35906dd8a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:49:17.430084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4681c7b-413b-4260-bf6c-0d35906dd8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:49:17.430359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4681c7b-413b-4260-bf6c-0d35906dd8a1 admin admin] PATH_INFO: `/v3/projects/fa25fbf1a3b04391940bd0de1179ce1e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:49:17.443615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4681c7b-413b-4260-bf6c-0d35906dd8a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa25fbf1a3b04391940bd0de1179ce1e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:49:17.445827 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-f4681c7b-413b-4260-bf6c-0d35906dd8a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:49:17.445827 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 20:49:17.445827 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 20:49:17.447055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4681c7b-413b-4260-bf6c-0d35906dd8a1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:49:17.482933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:49:17.511226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4681c7b-413b-4260-bf6c-0d35906dd8a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa25fbf1a3b04391940bd0de1179ce1e', 'request_id': 'req-f4681c7b-413b-4260-bf6c-0d35906dd8a1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:49:17.525377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4681c7b-413b-4260-bf6c-0d35906dd8a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa25fbf1a3b04391940bd0de1179ce1e', 'request_id': 'req-f4681c7b-413b-4260-bf6c-0d35906dd8a1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:49:17.535506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4681c7b-413b-4260-bf6c-0d35906dd8a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa25fbf1a3b04391940bd0de1179ce1e', 'request_id': 'req-f4681c7b-413b-4260-bf6c-0d35906dd8a1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:49:17.546953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4681c7b-413b-4260-bf6c-0d35906dd8a1 admin admin] The token cache is being invalidate because project fa25fbf1a3b04391940bd0de1179ce1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:49:17.547252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4681c7b-413b-4260-bf6c-0d35906dd8a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:49:17.548022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4681c7b-413b-4260-bf6c-0d35906dd8a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:49:17.549620 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 746/1510] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:49:17 2022] DELETE /identity/v3/projects/fa25fbf1a3b04391940bd0de1179ce1e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:49:27.051038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=56,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:49:27.544159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:52:35.820908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eac265-8598-41e2-b66b-0fba8a794e25 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:52:35.821860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eac265-8598-41e2-b66b-0fba8a794e25 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:52:35.821860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eac265-8598-41e2-b66b-0fba8a794e25 None None] PATH_INFO: `/v3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:52:35.823890 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 765/1511] 10.222.0.30 () {54 vars in 925 bytes} [Sat Aug 20 20:52:35 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:52:35.832220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66273e-06e5-4496-93ce-a4b94c8052e6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:52:35.832714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66273e-06e5-4496-93ce-a4b94c8052e6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:52:35.832830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66273e-06e5-4496-93ce-a4b94c8052e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:52:35.889358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-be66273e-06e5-4496-93ce-a4b94c8052e6 None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:52:35.890485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-be66273e-06e5-4496-93ce-a4b94c8052e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:52:35.964025 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 747/1512] 10.222.0.30 () {58 vars in 1015 bytes} [Sat Aug 20 20:52:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:52:38.083093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df688b52-4a99-4c71-a4d0-2cbc46b940e3 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:52:38.083907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df688b52-4a99-4c71-a4d0-2cbc46b940e3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:52:38.083907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df688b52-4a99-4c71-a4d0-2cbc46b940e3 None None] PATH_INFO: `/v3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:52:38.086053 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 766/1513] 10.222.0.16 () {54 vars in 925 bytes} [Sat Aug 20 20:52:38 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:52:38.093300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420105a1-36ba-4ce4-8b8a-c45e8d904d1f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:52:38.093992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420105a1-36ba-4ce4-8b8a-c45e8d904d1f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:52:38.094146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420105a1-36ba-4ce4-8b8a-c45e8d904d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:52:38.122119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-420105a1-36ba-4ce4-8b8a-c45e8d904d1f None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:52:38.123361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-420105a1-36ba-4ce4-8b8a-c45e8d904d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:52:38.134652 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 748/1514] 10.222.0.16 () {58 vars in 1015 bytes} [Sat Aug 20 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:52:40.290900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84d0c1-56f5-4328-838b-6630f6bec9b6 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:52:40.291897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84d0c1-56f5-4328-838b-6630f6bec9b6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:52:40.292566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84d0c1-56f5-4328-838b-6630f6bec9b6 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:52:40.294147 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 767/1515] 10.222.0.23 () {54 vars in 961 bytes} [Sat Aug 20 20:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:52:40.300912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbe71c-cc13-447a-9d0d-a19281b929ab None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:52:40.301572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbe71c-cc13-447a-9d0d-a19281b929ab None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:52:40.301572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbe71c-cc13-447a-9d0d-a19281b929ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:52:40.322232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-acbbe71c-cc13-447a-9d0d-a19281b929ab None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:52:40.323495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-acbbe71c-cc13-447a-9d0d-a19281b929ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:52:40.332349 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 749/1516] 10.222.0.23 () {58 vars in 1058 bytes} [Sat Aug 20 20:52:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:52:40.340055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84d0c1-56f5-4328-838b-6630f6bec9b6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:52:40.355058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84d0c1-56f5-4328-838b-6630f6bec9b6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:52:40.361159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9daa3d3-32b8-427a-a1eb-27df6bea890c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:52:40.361767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daa3d3-32b8-427a-a1eb-27df6bea890c service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:52:40.362034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daa3d3-32b8-427a-a1eb-27df6bea890c service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:52:40.362321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daa3d3-32b8-427a-a1eb-27df6bea890c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:52:40.370734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9daa3d3-32b8-427a-a1eb-27df6bea890c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:52:40.372346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9daa3d3-32b8-427a-a1eb-27df6bea890c service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:52:40.382205 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 768/1517] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:52:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:52:41.810242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63f605-141f-465f-a1ab-d1f0aa094e20 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:52:41.810715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63f605-141f-465f-a1ab-d1f0aa094e20 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:52:41.810840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63f605-141f-465f-a1ab-d1f0aa094e20 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:52:41.812242 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 750/1518] 10.222.0.23 () {54 vars in 961 bytes} [Sat Aug 20 20:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:52:41.818647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84835747-fba7-41b0-bb08-3601119a2916 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:52:41.819160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84835747-fba7-41b0-bb08-3601119a2916 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:52:41.819277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84835747-fba7-41b0-bb08-3601119a2916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:52:41.843287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-84835747-fba7-41b0-bb08-3601119a2916 None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:52:41.844514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-84835747-fba7-41b0-bb08-3601119a2916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:52:41.853250 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 769/1519] 10.222.0.23 () {58 vars in 1058 bytes} [Sat Aug 20 20:52:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:52:41.860997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63f605-141f-465f-a1ab-d1f0aa094e20 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:52:41.875472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63f605-141f-465f-a1ab-d1f0aa094e20 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:52:41.881162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aecf55-55f9-43dc-99d3-92936b07ac86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:52:41.881784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aecf55-55f9-43dc-99d3-92936b07ac86 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:52:41.882051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aecf55-55f9-43dc-99d3-92936b07ac86 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:52:41.882317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aecf55-55f9-43dc-99d3-92936b07ac86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:52:41.890722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aecf55-55f9-43dc-99d3-92936b07ac86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:52:41.892358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aecf55-55f9-43dc-99d3-92936b07ac86 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:52:41.902799 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 751/1520] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:52:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:52:51.838307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=9 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:52:51.898971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=56 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:53:10.103565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992a45e-e755-48b7-870a-942e3118783d None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:53:10.104164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992a45e-e755-48b7-870a-942e3118783d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:53:10.104164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992a45e-e755-48b7-870a-942e3118783d None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:53:10.105457 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 770/1521] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 20 20:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:53:10.111717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d940f-e4d0-424e-b3b8-07522a7dfb24 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:53:10.112165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d940f-e4d0-424e-b3b8-07522a7dfb24 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:53:10.112402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d940f-e4d0-424e-b3b8-07522a7dfb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:53:10.161509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7e5d940f-e4d0-424e-b3b8-07522a7dfb24 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:53:10.162711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7e5d940f-e4d0-424e-b3b8-07522a7dfb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:53:10.228731 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 752/1522] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 20 20:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:53:10.398092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992a45e-e755-48b7-870a-942e3118783d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:53:10.401112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0992a45e-e755-48b7-870a-942e3118783d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:53:10.509550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992a45e-e755-48b7-870a-942e3118783d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:53:10.515420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a180ed-ad62-408e-aa93-febf278002da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:53:10.516016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a180ed-ad62-408e-aa93-febf278002da service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:53:10.516282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a180ed-ad62-408e-aa93-febf278002da service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:53:10.516579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a180ed-ad62-408e-aa93-febf278002da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:53:10.525513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a180ed-ad62-408e-aa93-febf278002da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:53:10.527097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a180ed-ad62-408e-aa93-febf278002da service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:53:10.536962 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 771/1523] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:53:11.406008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aa748-fb76-4063-894b-504e637f00f1 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:53:11.406917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aa748-fb76-4063-894b-504e637f00f1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:53:11.406917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aa748-fb76-4063-894b-504e637f00f1 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:53:11.409122 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 753/1524] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 20 20:53:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:53:11.415122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31766cc-8100-441c-958a-b9e98d7befa9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:53:11.415876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31766cc-8100-441c-958a-b9e98d7befa9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:53:11.416036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31766cc-8100-441c-958a-b9e98d7befa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:53:11.438187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a31766cc-8100-441c-958a-b9e98d7befa9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:53:11.439375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a31766cc-8100-441c-958a-b9e98d7befa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:53:11.449015 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 772/1525] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 20 20:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:53:11.617700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aa748-fb76-4063-894b-504e637f00f1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:53:11.640397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aa748-fb76-4063-894b-504e637f00f1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:53:11.646195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3ce24-aff4-486b-a2fb-da4016b67ba5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:53:11.646786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3ce24-aff4-486b-a2fb-da4016b67ba5 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:53:11.647050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3ce24-aff4-486b-a2fb-da4016b67ba5 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:53:11.647313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3ce24-aff4-486b-a2fb-da4016b67ba5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:53:11.655761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3ce24-aff4-486b-a2fb-da4016b67ba5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:53:11.657380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3ce24-aff4-486b-a2fb-da4016b67ba5 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:53:11.666838 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 754/1526] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:53:15.711202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9ad3-9d0f-4d8e-af01-bd151985b705 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:53:15.711847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9ad3-9d0f-4d8e-af01-bd151985b705 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:53:15.711847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9ad3-9d0f-4d8e-af01-bd151985b705 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:53:15.713103 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 773/1527] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 20 20:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:53:15.718553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86058cb7-1924-4a0a-887f-a34cacf6955a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:53:15.719077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86058cb7-1924-4a0a-887f-a34cacf6955a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:53:15.719077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86058cb7-1924-4a0a-887f-a34cacf6955a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:53:15.742261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-86058cb7-1924-4a0a-887f-a34cacf6955a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:53:15.743473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-86058cb7-1924-4a0a-887f-a34cacf6955a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:53:15.754533 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 755/1528] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 20 20:53:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:53:16.116426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a9ad3-9d0f-4d8e-af01-bd151985b705 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:53:16.119297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5b5a9ad3-9d0f-4d8e-af01-bd151985b705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:53:16.186230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a9ad3-9d0f-4d8e-af01-bd151985b705 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:53:16.191249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d318d-c392-4521-8d8e-02f7a85e4515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:53:16.191879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d318d-c392-4521-8d8e-02f7a85e4515 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:53:16.192151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d318d-c392-4521-8d8e-02f7a85e4515 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:53:16.192412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d318d-c392-4521-8d8e-02f7a85e4515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:53:16.202304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d318d-c392-4521-8d8e-02f7a85e4515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:53:16.203926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d318d-c392-4521-8d8e-02f7a85e4515 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:53:16.213433 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 774/1529] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:53:18.526639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591919bc-3186-4b3f-81ae-3d35fbbcf245 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:53:18.527542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591919bc-3186-4b3f-81ae-3d35fbbcf245 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:53:18.527542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591919bc-3186-4b3f-81ae-3d35fbbcf245 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:53:18.529756 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 756/1530] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 20 20:53:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:53:18.535582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bbcbe-01b1-483e-85b0-4704f85de78e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:53:18.536068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bbcbe-01b1-483e-85b0-4704f85de78e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:53:18.536068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bbcbe-01b1-483e-85b0-4704f85de78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:53:18.559404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b51bbcbe-01b1-483e-85b0-4704f85de78e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:53:18.560638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b51bbcbe-01b1-483e-85b0-4704f85de78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:53:18.569954 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 775/1531] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 20 20:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:53:18.922184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591919bc-3186-4b3f-81ae-3d35fbbcf245 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:53:18.935096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591919bc-3186-4b3f-81ae-3d35fbbcf245 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:53:18.938922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d121e-3297-4bdc-bbc3-93566f06a116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:53:18.939524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d121e-3297-4bdc-bbc3-93566f06a116 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:53:18.939798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d121e-3297-4bdc-bbc3-93566f06a116 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:53:18.940189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d121e-3297-4bdc-bbc3-93566f06a116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:53:18.949813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d121e-3297-4bdc-bbc3-93566f06a116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:53:18.951316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d121e-3297-4bdc-bbc3-93566f06a116 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:53:18.961276 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 757/1532] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:53:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:53:28.554493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=78 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:53:28.957077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=49 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:10.643040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0941c-603d-499c-b034-d927c13e37c1 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:10.643585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0941c-603d-499c-b034-d927c13e37c1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:10.643585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0941c-603d-499c-b034-d927c13e37c1 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:10.645196 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 776/1533] 10.222.0.23 () {54 vars in 921 bytes} [Sat Aug 20 20:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:54:10.650417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d054f-ee8e-431a-8da4-2c4da4184b6d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:10.650906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d054f-ee8e-431a-8da4-2c4da4184b6d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:10.650986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d054f-ee8e-431a-8da4-2c4da4184b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:10.671909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-af1d054f-ee8e-431a-8da4-2c4da4184b6d None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:10.673176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-af1d054f-ee8e-431a-8da4-2c4da4184b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:10.683531 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 758/1534] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 20:54:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:54:20.667046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:43.683959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c741e6-dba4-461c-8426-073bca556f9b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:43.685235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c741e6-dba4-461c-8426-073bca556f9b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:43.685702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c741e6-dba4-461c-8426-073bca556f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:43.713432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a6c741e6-dba4-461c-8426-073bca556f9b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:43.714967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a6c741e6-dba4-461c-8426-073bca556f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:43.724445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10664161-abad-4035-a69a-3f398f4457c0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:43.725375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10664161-abad-4035-a69a-3f398f4457c0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:43.725836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10664161-abad-4035-a69a-3f398f4457c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:43.737855 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 777/1535] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:54:43.751234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2b15a-7e14-4978-8c69-ef8b2728884d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:43.751866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2b15a-7e14-4978-8c69-ef8b2728884d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:43.752324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2b15a-7e14-4978-8c69-ef8b2728884d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:43.752743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-10664161-abad-4035-a69a-3f398f4457c0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:43.753989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-10664161-abad-4035-a69a-3f398f4457c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:43.770934 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 759/1536] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:54:43.780284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d2a68-0c9f-408e-aa41-3706320aaa2b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:43.780583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d2a68-0c9f-408e-aa41-3706320aaa2b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:43.780855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d2a68-0c9f-408e-aa41-3706320aaa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:43.789260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2aa2b15a-7e14-4978-8c69-ef8b2728884d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:43.790995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2aa2b15a-7e14-4978-8c69-ef8b2728884d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:43.811343 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 778/1537] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:54:43.817620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2b15a-7e14-4978-8c69-ef8b2728884d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:43.821753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-945d2a68-0c9f-408e-aa41-3706320aaa2b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:43.823008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-945d2a68-0c9f-408e-aa41-3706320aaa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:43.837769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2b15a-7e14-4978-8c69-ef8b2728884d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:43.850056 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 760/1538] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:43.851074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1e1d1-94b7-4438-8470-e0db087f5b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:43.851925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1e1d1-94b7-4438-8470-e0db087f5b1d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:43.852416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1e1d1-94b7-4438-8470-e0db087f5b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:43.853587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1e1d1-94b7-4438-8470-e0db087f5b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:43.855205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1e1d1-94b7-4438-8470-e0db087f5b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:54:43.855701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1e1d1-94b7-4438-8470-e0db087f5b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:43.857019 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-e4d1e1d1-94b7-4438-8470-e0db087f5b1d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:54:43.857019 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 20:54:43.857019 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 20:54:43.857520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d2a68-0c9f-408e-aa41-3706320aaa2b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:43.858491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1e1d1-94b7-4438-8470-e0db087f5b1d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:43.871725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d2a68-0c9f-408e-aa41-3706320aaa2b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:43.884436 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 779/1539] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:54:43.893017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1e1d1-94b7-4438-8470-e0db087f5b1d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:43.897833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc39e3-d66b-49d7-9608-1e5dc6c736b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:43.898467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc39e3-d66b-49d7-9608-1e5dc6c736b3 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:43.898738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc39e3-d66b-49d7-9608-1e5dc6c736b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:43.899131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc39e3-d66b-49d7-9608-1e5dc6c736b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:43.899787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc39e3-d66b-49d7-9608-1e5dc6c736b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:54:43.900085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc39e3-d66b-49d7-9608-1e5dc6c736b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:43.901722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc39e3-d66b-49d7-9608-1e5dc6c736b3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:43.915232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1e1d1-94b7-4438-8470-e0db087f5b1d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:43.923592 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 761/1540] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:54:43.926794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6523c-f52e-4fce-a397-34053e472405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:43.928287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6523c-f52e-4fce-a397-34053e472405 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:43.928779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6523c-f52e-4fce-a397-34053e472405 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:43.929238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6523c-f52e-4fce-a397-34053e472405 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:43.930058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6523c-f52e-4fce-a397-34053e472405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:43.932225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6523c-f52e-4fce-a397-34053e472405 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:43.940393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc39e3-d66b-49d7-9608-1e5dc6c736b3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:43.953671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc39e3-d66b-49d7-9608-1e5dc6c736b3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:43.977212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835fd7c-5196-4aab-8031-82fb80bbd05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:43.977857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835fd7c-5196-4aab-8031-82fb80bbd05c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:43.978132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835fd7c-5196-4aab-8031-82fb80bbd05c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:43.978393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835fd7c-5196-4aab-8031-82fb80bbd05c admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:43.979026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835fd7c-5196-4aab-8031-82fb80bbd05c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:43.980100 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-5835fd7c-5196-4aab-8031-82fb80bbd05c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:54:43.980100 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 20:54:43.980100 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 20:54:43.981587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835fd7c-5196-4aab-8031-82fb80bbd05c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:44.001865 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 780/1541] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:43 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:54:44.006074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6523c-f52e-4fce-a397-34053e472405 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:44.030358 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 762/1542] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:43 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:44.034793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835fd7c-5196-4aab-8031-82fb80bbd05c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:44.134770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835fd7c-5196-4aab-8031-82fb80bbd05c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:44.137245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6523c-f52e-4fce-a397-34053e472405 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:44.140178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889c7ac-4805-49c9-9fb9-2df2347c03d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:44.140846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889c7ac-4805-49c9-9fb9-2df2347c03d5 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.141135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889c7ac-4805-49c9-9fb9-2df2347c03d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.141394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889c7ac-4805-49c9-9fb9-2df2347c03d5 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.142077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5889c7ac-4805-49c9-9fb9-2df2347c03d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:44.144538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5889c7ac-4805-49c9-9fb9-2df2347c03d5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:44.151091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5feb5-1d12-42b6-af75-47c592e7a708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:44.152449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5feb5-1d12-42b6-af75-47c592e7a708 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.152956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5feb5-1d12-42b6-af75-47c592e7a708 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.153410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5feb5-1d12-42b6-af75-47c592e7a708 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.154448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5feb5-1d12-42b6-af75-47c592e7a708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:44.155876 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-1fe5feb5-1d12-42b6-af75-47c592e7a708 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:54:44.155876 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 20:54:44.155876 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 20:54:44.157606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5feb5-1d12-42b6-af75-47c592e7a708 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:44.186116 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 763/1543] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:44.187653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ca20e-1525-43ed-b408-4e50012a1ef2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.187936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ca20e-1525-43ed-b408-4e50012a1ef2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.188198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ca20e-1525-43ed-b408-4e50012a1ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.204849 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 781/1544] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:54:44.205780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5feb5-1d12-42b6-af75-47c592e7a708 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:44.224708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5feb5-1d12-42b6-af75-47c592e7a708 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:44.239278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3a2ca20e-1525-43ed-b408-4e50012a1ef2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:44.241050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3a2ca20e-1525-43ed-b408-4e50012a1ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:44.257218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898adb7-5d62-4cd8-ae3b-047cbd06a017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:44.257881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898adb7-5d62-4cd8-ae3b-047cbd06a017 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.258159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898adb7-5d62-4cd8-ae3b-047cbd06a017 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.258422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898adb7-5d62-4cd8-ae3b-047cbd06a017 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.259163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898adb7-5d62-4cd8-ae3b-047cbd06a017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:44.260822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898adb7-5d62-4cd8-ae3b-047cbd06a017 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:44.270825 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 764/1545] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:44.272134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ca20e-1525-43ed-b408-4e50012a1ef2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:44.289289 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 782/1546] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:44.292568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0350106-c282-4b78-b03b-9e14f599d4e9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.292920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0350106-c282-4b78-b03b-9e14f599d4e9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.293175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0350106-c282-4b78-b03b-9e14f599d4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.298294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ca20e-1525-43ed-b408-4e50012a1ef2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:44.307230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dfe71-f7a6-4134-a626-6e6c859f9b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:44.307875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dfe71-f7a6-4134-a626-6e6c859f9b30 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.308150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dfe71-f7a6-4134-a626-6e6c859f9b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.308448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dfe71-f7a6-4134-a626-6e6c859f9b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.309110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240dfe71-f7a6-4134-a626-6e6c859f9b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:44.310807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240dfe71-f7a6-4134-a626-6e6c859f9b30 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:44.337307 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 765/1547] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:44.338240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dfe71-f7a6-4134-a626-6e6c859f9b30 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:44.342945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a0350106-c282-4b78-b03b-9e14f599d4e9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:44.344233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a0350106-c282-4b78-b03b-9e14f599d4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:44.360773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dfe71-f7a6-4134-a626-6e6c859f9b30 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:44.365105 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 783/1548] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:44.366042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0350106-c282-4b78-b03b-9e14f599d4e9 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:44.375372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6066cf-10d7-41a3-968c-811f64ca4556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:44.376000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6066cf-10d7-41a3-968c-811f64ca4556 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.376276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6066cf-10d7-41a3-968c-811f64ca4556 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.376565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6066cf-10d7-41a3-968c-811f64ca4556 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.377215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6066cf-10d7-41a3-968c-811f64ca4556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:44.378859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6066cf-10d7-41a3-968c-811f64ca4556 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:44.383930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0350106-c282-4b78-b03b-9e14f599d4e9 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:44.395577 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 766/1549] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:44.399367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6066cf-10d7-41a3-968c-811f64ca4556 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:44.400176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3ccdd-56c5-48b3-b7ec-bd91cc2d811f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:44.400862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3ccdd-56c5-48b3-b7ec-bd91cc2d811f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.401136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3ccdd-56c5-48b3-b7ec-bd91cc2d811f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.401398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3ccdd-56c5-48b3-b7ec-bd91cc2d811f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.402060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3ccdd-56c5-48b3-b7ec-bd91cc2d811f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:44.403683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3ccdd-56c5-48b3-b7ec-bd91cc2d811f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:44.410194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6066cf-10d7-41a3-968c-811f64ca4556 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:44.429137 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 784/1550] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:44.430100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3ccdd-56c5-48b3-b7ec-bd91cc2d811f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:44.433204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67354e-1c7c-445f-b78d-1b1030959f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:44.433837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67354e-1c7c-445f-b78d-1b1030959f48 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.434108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67354e-1c7c-445f-b78d-1b1030959f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.434374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67354e-1c7c-445f-b78d-1b1030959f48 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.435030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67354e-1c7c-445f-b78d-1b1030959f48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:54:44.435336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67354e-1c7c-445f-b78d-1b1030959f48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:44.445574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67354e-1c7c-445f-b78d-1b1030959f48 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:44.445972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3ccdd-56c5-48b3-b7ec-bd91cc2d811f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:44.454219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1393e-e81f-4c7b-ba04-88a23b5a43ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:44.454845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1393e-e81f-4c7b-ba04-88a23b5a43ec admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.455119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1393e-e81f-4c7b-ba04-88a23b5a43ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.455389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1393e-e81f-4c7b-ba04-88a23b5a43ec admin admin] PATH_INFO: `/v3/projects/11dbc98ec6164065a662c7995d15040c/users/876d24cb5a3e4f2ab6154315ceeabde8/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.464194 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 767/1551] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:54:44.465226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67354e-1c7c-445f-b78d-1b1030959f48 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:44.480601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1393e-e81f-4c7b-ba04-88a23b5a43ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=11dbc98ec6164065a662c7995d15040c, user_id=876d24cb5a3e4f2ab6154315ceeabde8, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:44.483309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1393e-e81f-4c7b-ba04-88a23b5a43ec admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:44.485304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67354e-1c7c-445f-b78d-1b1030959f48 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:44.511053 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 785/1552] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:44 2022] PUT /identity/v3/projects/11dbc98ec6164065a662c7995d15040c/users/876d24cb5a3e4f2ab6154315ceeabde8/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:44.511939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1393e-e81f-4c7b-ba04-88a23b5a43ec admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:44.513091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b3d5f-796b-4a18-86b4-a6264e018622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:44.513747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b3d5f-796b-4a18-86b4-a6264e018622 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.514021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b3d5f-796b-4a18-86b4-a6264e018622 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.514473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b3d5f-796b-4a18-86b4-a6264e018622 admin admin] PATH_INFO: `/v3/projects/3fca15c7755e4b9ca531cf6b3d266a03/users/1b96f59ee1304302bc8f1e0b728f1df3/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.539276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b3d5f-796b-4a18-86b4-a6264e018622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fca15c7755e4b9ca531cf6b3d266a03, user_id=1b96f59ee1304302bc8f1e0b728f1df3, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:44.540320 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-a09b3d5f-796b-4a18-86b4-a6264e018622 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:54:44.540320 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 20:54:44.540320 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 20:54:44.550633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b3d5f-796b-4a18-86b4-a6264e018622 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:44.574758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:44.587821 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 768/1553] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:44 2022] PUT /identity/v3/projects/3fca15c7755e4b9ca531cf6b3d266a03/users/1b96f59ee1304302bc8f1e0b728f1df3/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:44.590906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99be95-e53f-464b-b718-032a143fd209 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.590906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99be95-e53f-464b-b718-032a143fd209 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.590906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99be95-e53f-464b-b718-032a143fd209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.616330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4c99be95-e53f-464b-b718-032a143fd209 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:44.617588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4c99be95-e53f-464b-b718-032a143fd209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:44.621104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1393e-e81f-4c7b-ba04-88a23b5a43ec admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:44.667990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:44.703081 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 769/1554] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:44.704794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97c7c-53c6-468d-aaf2-a34673755fa1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.705278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97c7c-53c6-468d-aaf2-a34673755fa1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.705397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166548fb-038a-4881-a5d3-924f77f46ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:44.706062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166548fb-038a-4881-a5d3-924f77f46ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.706274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97c7c-53c6-468d-aaf2-a34673755fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.706436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166548fb-038a-4881-a5d3-924f77f46ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.706783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166548fb-038a-4881-a5d3-924f77f46ea3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.707472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166548fb-038a-4881-a5d3-924f77f46ea3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:44.709397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166548fb-038a-4881-a5d3-924f77f46ea3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:44.734898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4bc97c7c-53c6-468d-aaf2-a34673755fa1 None None] MFA Rules not processed for user `876d24cb5a3e4f2ab6154315ceeabde8`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:44.736111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4bc97c7c-53c6-468d-aaf2-a34673755fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:44.752428 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 786/1555] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/projects => generated 381 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:44.752428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cac8d-e8b9-4c63-a844-2694b725013b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.752686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cac8d-e8b9-4c63-a844-2694b725013b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.752867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cac8d-e8b9-4c63-a844-2694b725013b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.772657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c71cac8d-e8b9-4c63-a844-2694b725013b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:44.773862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c71cac8d-e8b9-4c63-a844-2694b725013b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:44.806074 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 770/1556] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:44.807229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cabf9-931e-44ba-97f5-a6297ade6e85 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.807508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cabf9-931e-44ba-97f5-a6297ade6e85 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.807777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cabf9-931e-44ba-97f5-a6297ade6e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.827172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e29cabf9-931e-44ba-97f5-a6297ade6e85 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:44.828363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e29cabf9-931e-44ba-97f5-a6297ade6e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:44.833755 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 787/1557] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:44.835092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60013b27-9b87-40de-ab27-c53e7e87bc57 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.835374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60013b27-9b87-40de-ab27-c53e7e87bc57 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.835647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60013b27-9b87-40de-ab27-c53e7e87bc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.837521 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 771/1558] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:44.838930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd41b8-cba7-422d-a1a6-67e8d83725d5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.839332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd41b8-cba7-422d-a1a6-67e8d83725d5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.839722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd41b8-cba7-422d-a1a6-67e8d83725d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.862212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8abd41b8-cba7-422d-a1a6-67e8d83725d5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:44.863524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8abd41b8-cba7-422d-a1a6-67e8d83725d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:44.872021 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 772/1559] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:44.873646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6be8a-ce4a-46cf-885d-3d45b4db599d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.873942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6be8a-ce4a-46cf-885d-3d45b4db599d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.874202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6be8a-ce4a-46cf-885d-3d45b4db599d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.875602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-60013b27-9b87-40de-ab27-c53e7e87bc57 None None] MFA Rules not processed for user `1b96f59ee1304302bc8f1e0b728f1df3`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:44.876830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-60013b27-9b87-40de-ab27-c53e7e87bc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:44.895606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c8e6be8a-ce4a-46cf-885d-3d45b4db599d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:44.896840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c8e6be8a-ce4a-46cf-885d-3d45b4db599d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:44.905231 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 773/1560] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:44.906456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b0984-563a-4f0a-91c4-27dd04d3bd20 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.906732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b0984-563a-4f0a-91c4-27dd04d3bd20 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.907009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b0984-563a-4f0a-91c4-27dd04d3bd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.927733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ee1b0984-563a-4f0a-91c4-27dd04d3bd20 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:44.928906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ee1b0984-563a-4f0a-91c4-27dd04d3bd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:44.936788 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 774/1561] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:44.937593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b0984-563a-4f0a-91c4-27dd04d3bd20 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:44.947005 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 788/1562] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:44.947309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b0984-563a-4f0a-91c4-27dd04d3bd20 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:44.948102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60013b27-9b87-40de-ab27-c53e7e87bc57 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:44.952087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91954642-852c-479b-b91a-abd62ec59fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:44.952738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91954642-852c-479b-b91a-abd62ec59fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.953015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91954642-852c-479b-b91a-abd62ec59fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.953282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91954642-852c-479b-b91a-abd62ec59fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:44.953568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:44.953899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91954642-852c-479b-b91a-abd62ec59fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:44.955884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91954642-852c-479b-b91a-abd62ec59fc7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:44.979108 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 775/1563] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/users => generated 291 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:44.983450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc475f55-f7ab-4f88-9932-15a4b048f26f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:44.983731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc475f55-f7ab-4f88-9932-15a4b048f26f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:44.984008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc475f55-f7ab-4f88-9932-15a4b048f26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.003025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60013b27-9b87-40de-ab27-c53e7e87bc57 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.003723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cc475f55-f7ab-4f88-9932-15a4b048f26f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:45.005029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cc475f55-f7ab-4f88-9932-15a4b048f26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:45.007396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8171e-e7f9-4fa1-8fdb-f470d7a48d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.008004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8171e-e7f9-4fa1-8fdb-f470d7a48d7c service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.008272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8171e-e7f9-4fa1-8fdb-f470d7a48d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.008566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8171e-e7f9-4fa1-8fdb-f470d7a48d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.013143 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 776/1564] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:45.014315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962f79-81e4-40a8-8995-d6c97482b6a9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.014593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962f79-81e4-40a8-8995-d6c97482b6a9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.014858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962f79-81e4-40a8-8995-d6c97482b6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.033623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9a962f79-81e4-40a8-8995-d6c97482b6a9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:45.034851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9a962f79-81e4-40a8-8995-d6c97482b6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:45.043010 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 777/1565] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:45.044154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da7df4-5d8b-46e2-a366-202302b191cb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.044463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da7df4-5d8b-46e2-a366-202302b191cb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.044730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da7df4-5d8b-46e2-a366-202302b191cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.063852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-30da7df4-5d8b-46e2-a366-202302b191cb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:45.065062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-30da7df4-5d8b-46e2-a366-202302b191cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:45.072026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8171e-e7f9-4fa1-8fdb-f470d7a48d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.073184 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 778/1566] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:45.073498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8171e-e7f9-4fa1-8fdb-f470d7a48d7c service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.074505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e93ab-5da4-4683-8bbf-71340f580a4d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.074615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e93ab-5da4-4683-8bbf-71340f580a4d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.074867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e93ab-5da4-4683-8bbf-71340f580a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.084228 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 789/1567] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:45.085088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8171e-e7f9-4fa1-8fdb-f470d7a48d7c service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.094661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-518e93ab-5da4-4683-8bbf-71340f580a4d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:45.095403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8171e-e7f9-4fa1-8fdb-f470d7a48d7c service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.095867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-518e93ab-5da4-4683-8bbf-71340f580a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:45.100812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d1409-1b2b-4386-8abf-dc1a28482928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.101417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d1409-1b2b-4386-8abf-dc1a28482928 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.101685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d1409-1b2b-4386-8abf-dc1a28482928 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.101949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d1409-1b2b-4386-8abf-dc1a28482928 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.102585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d1409-1b2b-4386-8abf-dc1a28482928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:54:45.102883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d1409-1b2b-4386-8abf-dc1a28482928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.104575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d1409-1b2b-4386-8abf-dc1a28482928 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.104736 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 779/1568] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:45.105630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e93ab-5da4-4683-8bbf-71340f580a4d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.118978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e93ab-5da4-4683-8bbf-71340f580a4d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.122196 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 790/1569] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:54:45.123193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d1409-1b2b-4386-8abf-dc1a28482928 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.130058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a4886-dfa4-4d26-8a03-a42f2f00d2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.130686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a4886-dfa4-4d26-8a03-a42f2f00d2b3 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.130958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a4886-dfa4-4d26-8a03-a42f2f00d2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.131227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a4886-dfa4-4d26-8a03-a42f2f00d2b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.131908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a4886-dfa4-4d26-8a03-a42f2f00d2b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.133933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a4886-dfa4-4d26-8a03-a42f2f00d2b3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.137201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d1409-1b2b-4386-8abf-dc1a28482928 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.141904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd4604-90db-405b-a6c0-00c3115f1e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.142504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd4604-90db-405b-a6c0-00c3115f1e2f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.142771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd4604-90db-405b-a6c0-00c3115f1e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.143032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd4604-90db-405b-a6c0-00c3115f1e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.143661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd4604-90db-405b-a6c0-00c3115f1e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.145481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd4604-90db-405b-a6c0-00c3115f1e2f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.157899 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 791/1570] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:45.161891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd4604-90db-405b-a6c0-00c3115f1e2f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.166370 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 780/1571] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:45 2022] POST /identity/v3/projects => generated 389 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:45.167056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a4886-dfa4-4d26-8a03-a42f2f00d2b3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.174959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd4604-90db-405b-a6c0-00c3115f1e2f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.207324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:45.243697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a4886-dfa4-4d26-8a03-a42f2f00d2b3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.251977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a1e7b-9605-436c-93ce-4e6d4df89ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.252998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a1e7b-9605-436c-93ce-4e6d4df89ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.253767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a1e7b-9605-436c-93ce-4e6d4df89ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.254220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a1e7b-9605-436c-93ce-4e6d4df89ec0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.254980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f9a3d-c04d-4f25-9bd2-945901750ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.255257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a1e7b-9605-436c-93ce-4e6d4df89ec0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:54:45.255783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a1e7b-9605-436c-93ce-4e6d4df89ec0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.255890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f9a3d-c04d-4f25-9bd2-945901750ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.256071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f9a3d-c04d-4f25-9bd2-945901750ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.257400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f9a3d-c04d-4f25-9bd2-945901750ff0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.258055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f9a3d-c04d-4f25-9bd2-945901750ff0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:54:45.258307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a1e7b-9605-436c-93ce-4e6d4df89ec0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.258419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f9a3d-c04d-4f25-9bd2-945901750ff0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.259933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f9a3d-c04d-4f25-9bd2-945901750ff0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.271711 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 781/1572] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:54:45.272951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a1e7b-9605-436c-93ce-4e6d4df89ec0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.281590 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 792/1573] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:54:45.283009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f9a3d-c04d-4f25-9bd2-945901750ff0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.284954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a1e7b-9605-436c-93ce-4e6d4df89ec0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.290983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fc4c5-d9c4-4986-9791-aa6fd274ba34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.291665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fc4c5-d9c4-4986-9791-aa6fd274ba34 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.292008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fc4c5-d9c4-4986-9791-aa6fd274ba34 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.292341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fc4c5-d9c4-4986-9791-aa6fd274ba34 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.293087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fc4c5-d9c4-4986-9791-aa6fd274ba34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:54:45.293390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fc4c5-d9c4-4986-9791-aa6fd274ba34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.295072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fc4c5-d9c4-4986-9791-aa6fd274ba34 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.295284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f9a3d-c04d-4f25-9bd2-945901750ff0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.299961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba1a5d-2783-4d03-a655-ac89d5319905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.300982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba1a5d-2783-4d03-a655-ac89d5319905 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.301255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba1a5d-2783-4d03-a655-ac89d5319905 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.301518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba1a5d-2783-4d03-a655-ac89d5319905 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.302152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba1a5d-2783-4d03-a655-ac89d5319905 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:54:45.302445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba1a5d-2783-4d03-a655-ac89d5319905 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.304201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba1a5d-2783-4d03-a655-ac89d5319905 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.309534 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 782/1574] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:54:45.310286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fc4c5-d9c4-4986-9791-aa6fd274ba34 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.318694 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 793/1575] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:54:45.319665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba1a5d-2783-4d03-a655-ac89d5319905 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.322720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fc4c5-d9c4-4986-9791-aa6fd274ba34 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.329774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13cf3b-23b7-4ccd-ae4d-870ea6afd6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.330746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13cf3b-23b7-4ccd-ae4d-870ea6afd6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.331270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13cf3b-23b7-4ccd-ae4d-870ea6afd6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.332432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13cf3b-23b7-4ccd-ae4d-870ea6afd6e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.333221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba1a5d-2783-4d03-a655-ac89d5319905 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.333735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13cf3b-23b7-4ccd-ae4d-870ea6afd6e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.335915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13cf3b-23b7-4ccd-ae4d-870ea6afd6e2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.337874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf721ff2-188e-454b-aefd-e3f28e575173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.338526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721ff2-188e-454b-aefd-e3f28e575173 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.338800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721ff2-188e-454b-aefd-e3f28e575173 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.339064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721ff2-188e-454b-aefd-e3f28e575173 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.339753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf721ff2-188e-454b-aefd-e3f28e575173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.341379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf721ff2-188e-454b-aefd-e3f28e575173 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.351966 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 794/1576] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:45.352815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf721ff2-188e-454b-aefd-e3f28e575173 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.360593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:45.366625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf721ff2-188e-454b-aefd-e3f28e575173 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.369368 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 783/1577] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:45 2022] POST /identity/v3/projects => generated 401 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:45.370066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13cf3b-23b7-4ccd-ae4d-870ea6afd6e2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.435853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d4fd9-7c44-421c-8dbd-e476d4659757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.436601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d4fd9-7c44-421c-8dbd-e476d4659757 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.436757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13cf3b-23b7-4ccd-ae4d-870ea6afd6e2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.436986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d4fd9-7c44-421c-8dbd-e476d4659757 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.437331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d4fd9-7c44-421c-8dbd-e476d4659757 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.437982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d4fd9-7c44-421c-8dbd-e476d4659757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.439897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d4fd9-7c44-421c-8dbd-e476d4659757 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.441448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12719c0-5cbc-4853-ada0-88c45f424b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.442047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12719c0-5cbc-4853-ada0-88c45f424b1c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.442319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12719c0-5cbc-4853-ada0-88c45f424b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.442583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12719c0-5cbc-4853-ada0-88c45f424b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.443193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12719c0-5cbc-4853-ada0-88c45f424b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.445544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12719c0-5cbc-4853-ada0-88c45f424b1c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.465772 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 795/1578] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:45 2022] POST /identity/v3/users => generated 301 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:45.466636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d4fd9-7c44-421c-8dbd-e476d4659757 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.477976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d4fd9-7c44-421c-8dbd-e476d4659757 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.479456 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 784/1579] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:45 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:45.480277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12719c0-5cbc-4853-ada0-88c45f424b1c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.545475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f3993-14f9-4e36-bbc1-8518e4ccccbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.546087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f3993-14f9-4e36-bbc1-8518e4ccccbb admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.546347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f3993-14f9-4e36-bbc1-8518e4ccccbb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.546617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f3993-14f9-4e36-bbc1-8518e4ccccbb admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.547036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12719c0-5cbc-4853-ada0-88c45f424b1c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.547408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f3993-14f9-4e36-bbc1-8518e4ccccbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.549391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f3993-14f9-4e36-bbc1-8518e4ccccbb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.553692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfd66e-6c11-450f-a01e-4ab9820de4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.554302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfd66e-6c11-450f-a01e-4ab9820de4bf admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.554568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfd66e-6c11-450f-a01e-4ab9820de4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.554831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfd66e-6c11-450f-a01e-4ab9820de4bf admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.555455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfd66e-6c11-450f-a01e-4ab9820de4bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.557370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfd66e-6c11-450f-a01e-4ab9820de4bf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.584884 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 796/1580] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:45 2022] POST /identity/v3/projects => generated 409 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:45.584884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f3993-14f9-4e36-bbc1-8518e4ccccbb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.589041 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 785/1581] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:45 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:45.589894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfd66e-6c11-450f-a01e-4ab9820de4bf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.596392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:45.652341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f3993-14f9-4e36-bbc1-8518e4ccccbb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.665399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfd66e-6c11-450f-a01e-4ab9820de4bf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.670804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83a424-256b-4698-b23e-b3c84a895057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.671881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83a424-256b-4698-b23e-b3c84a895057 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.672271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83a424-256b-4698-b23e-b3c84a895057 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.672656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83a424-256b-4698-b23e-b3c84a895057 admin admin] PATH_INFO: `/v3/projects/a6c380136f644611bfcb756f87feba71/users/3f85bb62037044ebae5a0a83194f9e9c/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.674321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:45.691752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83a424-256b-4698-b23e-b3c84a895057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6c380136f644611bfcb756f87feba71, user_id=3f85bb62037044ebae5a0a83194f9e9c, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.695611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83a424-256b-4698-b23e-b3c84a895057 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.711190 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 786/1582] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:45 2022] PUT /identity/v3/projects/a6c380136f644611bfcb756f87feba71/users/3f85bb62037044ebae5a0a83194f9e9c/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:45.712263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83a424-256b-4698-b23e-b3c84a895057 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.723631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2233596-44a1-4e82-89d9-ce58b260d8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.724268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2233596-44a1-4e82-89d9-ce58b260d8eb admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.724760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2233596-44a1-4e82-89d9-ce58b260d8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.725095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2233596-44a1-4e82-89d9-ce58b260d8eb admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.725742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2233596-44a1-4e82-89d9-ce58b260d8eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.727763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2233596-44a1-4e82-89d9-ce58b260d8eb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.762055 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 797/1583] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:45 2022] POST /identity/v3/projects => generated 397 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:45.762055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2233596-44a1-4e82-89d9-ce58b260d8eb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.787681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83a424-256b-4698-b23e-b3c84a895057 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.826041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2233596-44a1-4e82-89d9-ce58b260d8eb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.830650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.831286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.831641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.831911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] PATH_INFO: `/v3/users/876d24cb5a3e4f2ab6154315ceeabde8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.833199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=876d24cb5a3e4f2ab6154315ceeabde8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.835175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.849237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08114ce9-f674-4b27-8523-8939518855e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.849869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08114ce9-f674-4b27-8523-8939518855e7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.850169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08114ce9-f674-4b27-8523-8939518855e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.850438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08114ce9-f674-4b27-8523-8939518855e7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.851114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08114ce9-f674-4b27-8523-8939518855e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.853166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08114ce9-f674-4b27-8523-8939518855e7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.876403 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 787/1584] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:45 2022] POST /identity/v3/users => generated 301 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:45.877372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08114ce9-f674-4b27-8523-8939518855e7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.886458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '876d24cb5a3e4f2ab6154315ceeabde8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:45.890112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08114ce9-f674-4b27-8523-8939518855e7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.895387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '876d24cb5a3e4f2ab6154315ceeabde8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:45.896111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bd1a3-79f9-47fa-adf7-28b5a2cfde8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.896798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bd1a3-79f9-47fa-adf7-28b5a2cfde8e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.897073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bd1a3-79f9-47fa-adf7-28b5a2cfde8e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.897338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bd1a3-79f9-47fa-adf7-28b5a2cfde8e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.897976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bd1a3-79f9-47fa-adf7-28b5a2cfde8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:45.899625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bd1a3-79f9-47fa-adf7-28b5a2cfde8e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:45.903180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '876d24cb5a3e4f2ab6154315ceeabde8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:45.910533 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 788/1585] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:45.914791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bd1a3-79f9-47fa-adf7-28b5a2cfde8e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.916932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '876d24cb5a3e4f2ab6154315ceeabde8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:45.917644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '876d24cb5a3e4f2ab6154315ceeabde8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:45.923839 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 798/1586] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:54:45 2022] DELETE /identity/v3/users/876d24cb5a3e4f2ab6154315ceeabde8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:45.927084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bd1a3-79f9-47fa-adf7-28b5a2cfde8e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:45.927841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:45.929419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:45.938826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:45.964613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:45.997015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4696b9-9a1d-4430-8962-f180094f7f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:45.997642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4696b9-9a1d-4430-8962-f180094f7f2a admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:45.998066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4696b9-9a1d-4430-8962-f180094f7f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:45.998354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4696b9-9a1d-4430-8962-f180094f7f2a admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:45.998983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4696b9-9a1d-4430-8962-f180094f7f2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.000209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9108d-cd2a-4589-a4bf-3bbe72f95d82 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.000988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4696b9-9a1d-4430-8962-f180094f7f2a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.005115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bb901-ee5e-4808-a5bf-4b6e9d7d75a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.005717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bb901-ee5e-4808-a5bf-4b6e9d7d75a7 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.005983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bb901-ee5e-4808-a5bf-4b6e9d7d75a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.006243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bb901-ee5e-4808-a5bf-4b6e9d7d75a7 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.006853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bb901-ee5e-4808-a5bf-4b6e9d7d75a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.008796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bb901-ee5e-4808-a5bf-4b6e9d7d75a7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.025074 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 789/1587] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:45 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:46.025965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4696b9-9a1d-4430-8962-f180094f7f2a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.027527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6e4696b9-9a1d-4430-8962-f180094f7f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:46.030700 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 799/1588] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:45 2022] POST /identity/v3/users => generated 305 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:46.031830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4acf5c-33b0-4085-83cb-f726835cc0eb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.032111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4acf5c-33b0-4085-83cb-f726835cc0eb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.032399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4acf5c-33b0-4085-83cb-f726835cc0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.040080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4696b9-9a1d-4430-8962-f180094f7f2a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.044818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd05286-2428-4d13-924f-66ce270f6f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.045404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd05286-2428-4d13-924f-66ce270f6f2b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.045672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd05286-2428-4d13-924f-66ce270f6f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.045930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd05286-2428-4d13-924f-66ce270f6f2b admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.046539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd05286-2428-4d13-924f-66ce270f6f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.048568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd05286-2428-4d13-924f-66ce270f6f2b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.060026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fd4acf5c-33b0-4085-83cb-f726835cc0eb None None] MFA Rules not processed for user `3f85bb62037044ebae5a0a83194f9e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:46.061561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fd4acf5c-33b0-4085-83cb-f726835cc0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:46.072219 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 790/1589] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:46 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:46.076273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd05286-2428-4d13-924f-66ce270f6f2b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.077888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0fd05286-2428-4d13-924f-66ce270f6f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:46.089747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd05286-2428-4d13-924f-66ce270f6f2b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.094422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fd0c8-168d-41db-9788-b4feb5d44ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.095016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fd0c8-168d-41db-9788-b4feb5d44ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.095286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fd0c8-168d-41db-9788-b4feb5d44ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.095549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fd0c8-168d-41db-9788-b4feb5d44ef9 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.096171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fd0c8-168d-41db-9788-b4feb5d44ef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.098393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fd0c8-168d-41db-9788-b4feb5d44ef9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.124644 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 791/1590] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:46 2022] POST /identity/v3/users => generated 299 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:46.124644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fd0c8-168d-41db-9788-b4feb5d44ef9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.127696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-522fd0c8-168d-41db-9788-b4feb5d44ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:46.141492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fd0c8-168d-41db-9788-b4feb5d44ef9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.143465 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 800/1591] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:46.144341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4acf5c-33b0-4085-83cb-f726835cc0eb None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.145677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fd4acf5c-33b0-4085-83cb-f726835cc0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:46.146453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066d0a8-0e4d-4c78-82be-6ed46a0c7d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.147180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066d0a8-0e4d-4c78-82be-6ed46a0c7d85 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.147622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066d0a8-0e4d-4c78-82be-6ed46a0c7d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.147794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066d0a8-0e4d-4c78-82be-6ed46a0c7d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.148691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066d0a8-0e4d-4c78-82be-6ed46a0c7d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.150295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066d0a8-0e4d-4c78-82be-6ed46a0c7d85 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.158994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4acf5c-33b0-4085-83cb-f726835cc0eb None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.161338 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 792/1592] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:46.162732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f135847-56f9-49ee-96ef-098b25bed74b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.163024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f135847-56f9-49ee-96ef-098b25bed74b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.163281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f135847-56f9-49ee-96ef-098b25bed74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.163945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6203c1-2b62-4968-aa00-cbc932a839de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.165038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6203c1-2b62-4968-aa00-cbc932a839de admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.165316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6203c1-2b62-4968-aa00-cbc932a839de admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.165581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6203c1-2b62-4968-aa00-cbc932a839de admin admin] PATH_INFO: `/v3/projects/a466057405124a4cb8f212e31a08984f/users/429681ce41994ec197658f17573c6081/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.180586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6203c1-2b62-4968-aa00-cbc932a839de admin admin] RBAC: Authorizing `identity:create_grant(project_id=a466057405124a4cb8f212e31a08984f, user_id=429681ce41994ec197658f17573c6081, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.183249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6203c1-2b62-4968-aa00-cbc932a839de admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.184422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8f135847-56f9-49ee-96ef-098b25bed74b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:46.185629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8f135847-56f9-49ee-96ef-098b25bed74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:46.194159 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 793/1593] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:46.195354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f135847-56f9-49ee-96ef-098b25bed74b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.199878 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 801/1594] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:46 2022] PUT /identity/v3/projects/a466057405124a4cb8f212e31a08984f/users/429681ce41994ec197658f17573c6081/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:46.200991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6203c1-2b62-4968-aa00-cbc932a839de admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.201669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8f135847-56f9-49ee-96ef-098b25bed74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:46.214639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f135847-56f9-49ee-96ef-098b25bed74b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.265475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6203c1-2b62-4968-aa00-cbc932a839de admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.270494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ae3b1-2341-4242-bddd-d184a86eb082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.271099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ae3b1-2341-4242-bddd-d184a86eb082 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.271369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ae3b1-2341-4242-bddd-d184a86eb082 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.271640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ae3b1-2341-4242-bddd-d184a86eb082 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.272304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ae3b1-2341-4242-bddd-d184a86eb082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.273977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ae3b1-2341-4242-bddd-d184a86eb082 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.278223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b60f8-4c16-45fb-9a1d-c04888a7713a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.278912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b60f8-4c16-45fb-9a1d-c04888a7713a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.279195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b60f8-4c16-45fb-9a1d-c04888a7713a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.279466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b60f8-4c16-45fb-9a1d-c04888a7713a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.280142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b60f8-4c16-45fb-9a1d-c04888a7713a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.282066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b60f8-4c16-45fb-9a1d-c04888a7713a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.283917 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 802/1595] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:46.284850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ae3b1-2341-4242-bddd-d184a86eb082 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.297045 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 794/1596] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:46.297201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ae3b1-2341-4242-bddd-d184a86eb082 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.297295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b60f8-4c16-45fb-9a1d-c04888a7713a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.299419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5a3a1-0f83-4922-9a2f-3c4a7d505f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.300027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5a3a1-0f83-4922-9a2f-3c4a7d505f02 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.300405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5a3a1-0f83-4922-9a2f-3c4a7d505f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.300671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5a3a1-0f83-4922-9a2f-3c4a7d505f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.301331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5a3a1-0f83-4922-9a2f-3c4a7d505f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.302920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5a3a1-0f83-4922-9a2f-3c4a7d505f02 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.304910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b60f8-4c16-45fb-9a1d-c04888a7713a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.309830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b760e-dce0-484d-bf45-19639a569d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.310698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b760e-dce0-484d-bf45-19639a569d68 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.311099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b760e-dce0-484d-bf45-19639a569d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.311373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b760e-dce0-484d-bf45-19639a569d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.312026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b760e-dce0-484d-bf45-19639a569d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.316857 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 803/1597] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:46.317060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b760e-dce0-484d-bf45-19639a569d68 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.317188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5a3a1-0f83-4922-9a2f-3c4a7d505f02 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.317188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-eaf5a3a1-0f83-4922-9a2f-3c4a7d505f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:46.327658 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 795/1598] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:46.328548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b760e-dce0-484d-bf45-19639a569d68 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.333356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5a3a1-0f83-4922-9a2f-3c4a7d505f02 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.338818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbde0b-3243-4ad4-bbba-22074068dbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.338982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b760e-dce0-484d-bf45-19639a569d68 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.339471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbde0b-3243-4ad4-bbba-22074068dbe8 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.339824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbde0b-3243-4ad4-bbba-22074068dbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.340085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbde0b-3243-4ad4-bbba-22074068dbe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.340725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbde0b-3243-4ad4-bbba-22074068dbe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.342666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbde0b-3243-4ad4-bbba-22074068dbe8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.343814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3b3a7-7121-48e4-aa48-22931b717bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.344432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3b3a7-7121-48e4-aa48-22931b717bfd admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.344707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3b3a7-7121-48e4-aa48-22931b717bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.344974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3b3a7-7121-48e4-aa48-22931b717bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.345661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3b3a7-7121-48e4-aa48-22931b717bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.347410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3b3a7-7121-48e4-aa48-22931b717bfd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.349026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:46.358802 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 796/1599] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:46.359628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3b3a7-7121-48e4-aa48-22931b717bfd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.370501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3b3a7-7121-48e4-aa48-22931b717bfd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.375399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5160d-50b6-428c-af67-c28dd4b11195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.375540 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 804/1600] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:46 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:46.376000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5160d-50b6-428c-af67-c28dd4b11195 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.376290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbde0b-3243-4ad4-bbba-22074068dbe8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.376418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5160d-50b6-428c-af67-c28dd4b11195 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.376624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5160d-50b6-428c-af67-c28dd4b11195 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.377531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5160d-50b6-428c-af67-c28dd4b11195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.380196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5160d-50b6-428c-af67-c28dd4b11195 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.381333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-61bbde0b-3243-4ad4-bbba-22074068dbe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:46.395126 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 797/1601] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:46.395126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5160d-50b6-428c-af67-c28dd4b11195 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.404706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5160d-50b6-428c-af67-c28dd4b11195 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.412721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:46.439645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbde0b-3243-4ad4-bbba-22074068dbe8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.444380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c09a6-981b-42db-a3a3-6af3b48aa869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.445332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c09a6-981b-42db-a3a3-6af3b48aa869 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.445602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c09a6-981b-42db-a3a3-6af3b48aa869 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.445873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c09a6-981b-42db-a3a3-6af3b48aa869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.469977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a6b4f-ff06-4250-b318-25b5f51e4f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.470642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a6b4f-ff06-4250-b318-25b5f51e4f05 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.470914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a6b4f-ff06-4250-b318-25b5f51e4f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.471174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a6b4f-ff06-4250-b318-25b5f51e4f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.471853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a6b4f-ff06-4250-b318-25b5f51e4f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.473911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a6b4f-ff06-4250-b318-25b5f51e4f05 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.486045 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 798/1602] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:46.487266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a6b4f-ff06-4250-b318-25b5f51e4f05 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.497569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a6b4f-ff06-4250-b318-25b5f51e4f05 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.502663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e20a5d-e7f8-48a4-a095-d9f5448a30d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.503611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e20a5d-e7f8-48a4-a095-d9f5448a30d5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.503741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e20a5d-e7f8-48a4-a095-d9f5448a30d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.503939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e20a5d-e7f8-48a4-a095-d9f5448a30d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.504682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e20a5d-e7f8-48a4-a095-d9f5448a30d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.507446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e20a5d-e7f8-48a4-a095-d9f5448a30d5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.510472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c09a6-981b-42db-a3a3-6af3b48aa869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.511951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c09a6-981b-42db-a3a3-6af3b48aa869 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.523311 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 799/1603] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:46.523311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e20a5d-e7f8-48a4-a095-d9f5448a30d5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.525629 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 805/1604] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:46.526635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c09a6-981b-42db-a3a3-6af3b48aa869 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.534913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e20a5d-e7f8-48a4-a095-d9f5448a30d5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.536339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c09a6-981b-42db-a3a3-6af3b48aa869 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.540422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d1ad8-a109-40be-ad34-85e1f639ac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.540967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc7ec2-d1c4-45b5-a342-20691544b4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.541262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d1ad8-a109-40be-ad34-85e1f639ac13 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.541559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc7ec2-d1c4-45b5-a342-20691544b4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.541666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d1ad8-a109-40be-ad34-85e1f639ac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.541829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc7ec2-d1c4-45b5-a342-20691544b4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.541919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d1ad8-a109-40be-ad34-85e1f639ac13 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.542090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc7ec2-d1c4-45b5-a342-20691544b4a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.542736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d1ad8-a109-40be-ad34-85e1f639ac13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.542893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc7ec2-d1c4-45b5-a342-20691544b4a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.544986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc7ec2-d1c4-45b5-a342-20691544b4a9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.545070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d1ad8-a109-40be-ad34-85e1f639ac13 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.558878 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 806/1605] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:46.558878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc7ec2-d1c4-45b5-a342-20691544b4a9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.559180 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 800/1606] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:46.559634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d1ad8-a109-40be-ad34-85e1f639ac13 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.573967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc7ec2-d1c4-45b5-a342-20691544b4a9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.574381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d1ad8-a109-40be-ad34-85e1f639ac13 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.579779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368811a-7fa2-4d00-a4ea-b956f340f079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.580119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa246ee-da9f-4509-a108-720860e0b9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.580732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368811a-7fa2-4d00-a4ea-b956f340f079 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.580871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa246ee-da9f-4509-a108-720860e0b9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.581060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa246ee-da9f-4509-a108-720860e0b9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.581301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368811a-7fa2-4d00-a4ea-b956f340f079 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.581416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa246ee-da9f-4509-a108-720860e0b9d9 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.581794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368811a-7fa2-4d00-a4ea-b956f340f079 admin admin] PATH_INFO: `/v3/projects/42cbcac0c2d54eaea98f071074003560/users/a3378bf4fa90489cb103378667f00399/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.582780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa246ee-da9f-4509-a108-720860e0b9d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.585678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa246ee-da9f-4509-a108-720860e0b9d9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.602044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368811a-7fa2-4d00-a4ea-b956f340f079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42cbcac0c2d54eaea98f071074003560, user_id=a3378bf4fa90489cb103378667f00399, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.605194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368811a-7fa2-4d00-a4ea-b956f340f079 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.610947 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 801/1607] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:46 2022] POST /identity/v3/users => generated 298 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:46.611824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa246ee-da9f-4509-a108-720860e0b9d9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.623142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa246ee-da9f-4509-a108-720860e0b9d9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.625052 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 807/1608] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:46 2022] PUT /identity/v3/projects/42cbcac0c2d54eaea98f071074003560/users/a3378bf4fa90489cb103378667f00399/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:46.626010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368811a-7fa2-4d00-a4ea-b956f340f079 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.699425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368811a-7fa2-4d00-a4ea-b956f340f079 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.704806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27667de4-4f35-43a5-8ae0-59bd0d2816a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.705935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27667de4-4f35-43a5-8ae0-59bd0d2816a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.706878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27667de4-4f35-43a5-8ae0-59bd0d2816a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.707263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27667de4-4f35-43a5-8ae0-59bd0d2816a1 admin admin] PATH_INFO: `/v3/projects/5c01dcbd100e4a418094b329a46fdc3d/users/4983cbd777b244ab9badec08f75708ba/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.711436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:46.719673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bed1e4-9845-4d63-9e5c-169616af625f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.720543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bed1e4-9845-4d63-9e5c-169616af625f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.720899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bed1e4-9845-4d63-9e5c-169616af625f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.721066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bed1e4-9845-4d63-9e5c-169616af625f admin admin] PATH_INFO: `/v3/projects/a466057405124a4cb8f212e31a08984f/users/429681ce41994ec197658f17573c6081/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.723644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bed1e4-9845-4d63-9e5c-169616af625f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a466057405124a4cb8f212e31a08984f, user_id=429681ce41994ec197658f17573c6081, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.726641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bed1e4-9845-4d63-9e5c-169616af625f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.727731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27667de4-4f35-43a5-8ae0-59bd0d2816a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c01dcbd100e4a418094b329a46fdc3d, user_id=4983cbd777b244ab9badec08f75708ba, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.730768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27667de4-4f35-43a5-8ae0-59bd0d2816a1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.739637 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 802/1609] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:46 2022] PUT /identity/v3/projects/a466057405124a4cb8f212e31a08984f/users/429681ce41994ec197658f17573c6081/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:46.740611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bed1e4-9845-4d63-9e5c-169616af625f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.756578 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 808/1610] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:46 2022] PUT /identity/v3/projects/5c01dcbd100e4a418094b329a46fdc3d/users/4983cbd777b244ab9badec08f75708ba/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:46.757758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27667de4-4f35-43a5-8ae0-59bd0d2816a1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.827359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27667de4-4f35-43a5-8ae0-59bd0d2816a1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.831456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bed1e4-9845-4d63-9e5c-169616af625f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.832859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec2245-1865-4413-8c36-a4bf15711cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.833750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec2245-1865-4413-8c36-a4bf15711cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.834159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec2245-1865-4413-8c36-a4bf15711cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.834583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec2245-1865-4413-8c36-a4bf15711cfe admin admin] PATH_INFO: `/v3/projects/9a703d5afdeb467eab897527129a2d01/users/44f542f6dc164350b3ffa192616d0f95/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.837274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91a8c1-e5a0-4528-af0c-f927a1fdd384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.839025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91a8c1-e5a0-4528-af0c-f927a1fdd384 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.839333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91a8c1-e5a0-4528-af0c-f927a1fdd384 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.839603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91a8c1-e5a0-4528-af0c-f927a1fdd384 admin admin] PATH_INFO: `/v3/projects/28b6481e72f84322b4090cf6c2a20927/users/4cdb1c73a7de40f8962ed40eef69641a/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.844412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:46.854360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec2245-1865-4413-8c36-a4bf15711cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a703d5afdeb467eab897527129a2d01, user_id=44f542f6dc164350b3ffa192616d0f95, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.858082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec2245-1865-4413-8c36-a4bf15711cfe admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.861287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91a8c1-e5a0-4528-af0c-f927a1fdd384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b6481e72f84322b4090cf6c2a20927, user_id=4cdb1c73a7de40f8962ed40eef69641a, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.864836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91a8c1-e5a0-4528-af0c-f927a1fdd384 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.876406 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 809/1611] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:46 2022] PUT /identity/v3/projects/9a703d5afdeb467eab897527129a2d01/users/44f542f6dc164350b3ffa192616d0f95/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:46.877433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec2245-1865-4413-8c36-a4bf15711cfe admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.888860 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 803/1612] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:46 2022] PUT /identity/v3/projects/28b6481e72f84322b4090cf6c2a20927/users/4cdb1c73a7de40f8962ed40eef69641a/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:46.889880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91a8c1-e5a0-4528-af0c-f927a1fdd384 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:46.949229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec2245-1865-4413-8c36-a4bf15711cfe admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.965577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91a8c1-e5a0-4528-af0c-f927a1fdd384 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:46.971006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030a235-56b3-4110-ad4c-1dd7d8a56a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:46.971642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030a235-56b3-4110-ad4c-1dd7d8a56a2f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.971915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030a235-56b3-4110-ad4c-1dd7d8a56a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.972311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030a235-56b3-4110-ad4c-1dd7d8a56a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:46.972993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030a235-56b3-4110-ad4c-1dd7d8a56a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:46.974780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030a235-56b3-4110-ad4c-1dd7d8a56a2f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:46.985845 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 804/1613] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:46.987180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba4d77-604d-4c2f-bf59-d1347c8a550f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:46.987463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba4d77-604d-4c2f-bf59-d1347c8a550f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:46.987796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba4d77-604d-4c2f-bf59-d1347c8a550f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.007796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-00ba4d77-604d-4c2f-bf59-d1347c8a550f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:47.009070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-00ba4d77-604d-4c2f-bf59-d1347c8a550f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.009519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a06d0-75cb-4896-933c-acd09146d93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:47.010204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a06d0-75cb-4896-933c-acd09146d93e admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.010516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a06d0-75cb-4896-933c-acd09146d93e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.010834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a06d0-75cb-4896-933c-acd09146d93e admin admin] PATH_INFO: `/v3/projects/2e2f79566b574520aabb42684e035f10/users/2733d0669fd24ed68b312925763fa212/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.017232 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 805/1614] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:47.018527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd33f7-a92f-45fa-8a62-0c64be3800a1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.019465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd33f7-a92f-45fa-8a62-0c64be3800a1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.019682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd33f7-a92f-45fa-8a62-0c64be3800a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.037621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a06d0-75cb-4896-933c-acd09146d93e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2f79566b574520aabb42684e035f10, user_id=2733d0669fd24ed68b312925763fa212, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:47.040571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a06d0-75cb-4896-933c-acd09146d93e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:47.050125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:47.052252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ecfd33f7-a92f-45fa-8a62-0c64be3800a1 None None] MFA Rules not processed for user `429681ce41994ec197658f17573c6081`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:47.053702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ecfd33f7-a92f-45fa-8a62-0c64be3800a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.057316 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 810/1615] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:46 2022] PUT /identity/v3/projects/2e2f79566b574520aabb42684e035f10/users/2733d0669fd24ed68b312925763fa212/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:47.058763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1c5c5-8a81-4948-84fa-7a88a24d21c2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.059046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1c5c5-8a81-4948-84fa-7a88a24d21c2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.059310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1c5c5-8a81-4948-84fa-7a88a24d21c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.091424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9fc1c5c5-8a81-4948-84fa-7a88a24d21c2 None None] MFA Rules not processed for user `4983cbd777b244ab9badec08f75708ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:47.092638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9fc1c5c5-8a81-4948-84fa-7a88a24d21c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.136450 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 806/1616] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:47.137306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd33f7-a92f-45fa-8a62-0c64be3800a1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:47.138599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ecfd33f7-a92f-45fa-8a62-0c64be3800a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.158710 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 811/1617] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:47.160149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b461de-07f0-4fe8-9f03-1725ca51f1d9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.160670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b461de-07f0-4fe8-9f03-1725ca51f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.160868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b461de-07f0-4fe8-9f03-1725ca51f1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.188247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a9b461de-07f0-4fe8-9f03-1725ca51f1d9 None None] MFA Rules not processed for user `44f542f6dc164350b3ffa192616d0f95`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:47.189440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a9b461de-07f0-4fe8-9f03-1725ca51f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.210374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd33f7-a92f-45fa-8a62-0c64be3800a1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:47.214948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:47.215584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.215857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.216152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] PATH_INFO: `/v3/projects/11dbc98ec6164065a662c7995d15040c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.217321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11dbc98ec6164065a662c7995d15040c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:47.219236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:47.221219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:47.259157 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 812/1618] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:47.263105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273042f5-ed16-4870-8f95-688035ccc791 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.263105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273042f5-ed16-4870-8f95-688035ccc791 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.263105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273042f5-ed16-4870-8f95-688035ccc791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.280144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11dbc98ec6164065a662c7995d15040c', 'request_id': 'req-15c5f834-8fef-4cde-94e3-478b0c626294'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:47.287179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11dbc98ec6164065a662c7995d15040c', 'request_id': 'req-15c5f834-8fef-4cde-94e3-478b0c626294'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:47.291879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-273042f5-ed16-4870-8f95-688035ccc791 None None] MFA Rules not processed for user `4cdb1c73a7de40f8962ed40eef69641a`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:47.293190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-273042f5-ed16-4870-8f95-688035ccc791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.295393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11dbc98ec6164065a662c7995d15040c', 'request_id': 'req-15c5f834-8fef-4cde-94e3-478b0c626294'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:47.306678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] The token cache is being invalidate because project 11dbc98ec6164065a662c7995d15040c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:54:47.306998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:47.307651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:47.309265 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 807/1619] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:54:47 2022] DELETE /identity/v3/projects/11dbc98ec6164065a662c7995d15040c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:47.310144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:47.311962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.366355 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 813/1620] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:47.367663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273042f5-ed16-4870-8f95-688035ccc791 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:47.369306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-273042f5-ed16-4870-8f95-688035ccc791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.381288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:47.396841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5f834-8fef-4cde-94e3-478b0c626294 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:47.401842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cef8e0-cef4-4536-8f71-8bb88a7992d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:47.402485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cef8e0-cef4-4536-8f71-8bb88a7992d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.402763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cef8e0-cef4-4536-8f71-8bb88a7992d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.403031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cef8e0-cef4-4536-8f71-8bb88a7992d7 admin admin] PATH_INFO: `/v3/projects/022df6b28f2644038160415fe5fbf339/users/218942b27bdb401ca842ee6ef27670d2/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.420144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cef8e0-cef4-4536-8f71-8bb88a7992d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=022df6b28f2644038160415fe5fbf339, user_id=218942b27bdb401ca842ee6ef27670d2, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:47.422892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cef8e0-cef4-4536-8f71-8bb88a7992d7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:47.433809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273042f5-ed16-4870-8f95-688035ccc791 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:47.437531 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 808/1621] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:47 2022] PUT /identity/v3/projects/022df6b28f2644038160415fe5fbf339/users/218942b27bdb401ca842ee6ef27670d2/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:47.438775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4461ce-be95-46ad-b3f1-22c3c02cf197 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.439051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4461ce-be95-46ad-b3f1-22c3c02cf197 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.439324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4461ce-be95-46ad-b3f1-22c3c02cf197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.470084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0b4461ce-be95-46ad-b3f1-22c3c02cf197 None None] MFA Rules not processed for user `2733d0669fd24ed68b312925763fa212`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:47.471569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0b4461ce-be95-46ad-b3f1-22c3c02cf197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.486820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e3c93-5ccc-45b4-ba1e-3f0d296f681a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:47.487433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e3c93-5ccc-45b4-ba1e-3f0d296f681a service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.487875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e3c93-5ccc-45b4-ba1e-3f0d296f681a service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.488278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e3c93-5ccc-45b4-ba1e-3f0d296f681a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.490300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a90e3c93-5ccc-45b4-ba1e-3f0d296f681a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.536309 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 809/1622] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:47.537606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feac559-b46a-45c8-b62e-659121c683aa None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.538000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feac559-b46a-45c8-b62e-659121c683aa None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.538494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feac559-b46a-45c8-b62e-659121c683aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.545399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:47.557649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e3c93-5ccc-45b4-ba1e-3f0d296f681a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:47.559098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e3c93-5ccc-45b4-ba1e-3f0d296f681a service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:47.559751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1feac559-b46a-45c8-b62e-659121c683aa None None] MFA Rules not processed for user `1b96f59ee1304302bc8f1e0b728f1df3`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:47.560977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1feac559-b46a-45c8-b62e-659121c683aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.569669 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 814/1623] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:47.570653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e3c93-5ccc-45b4-ba1e-3f0d296f681a service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:47.572051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a90e3c93-5ccc-45b4-ba1e-3f0d296f681a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.586169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e3c93-5ccc-45b4-ba1e-3f0d296f681a service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:47.591910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396bd71-0674-41ae-9b90-aeef80f100ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:47.592528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396bd71-0674-41ae-9b90-aeef80f100ea admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.592892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396bd71-0674-41ae-9b90-aeef80f100ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.593132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396bd71-0674-41ae-9b90-aeef80f100ea admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.593924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396bd71-0674-41ae-9b90-aeef80f100ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:47.595935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396bd71-0674-41ae-9b90-aeef80f100ea admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:47.626926 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 810/1624] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:47.627938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feac559-b46a-45c8-b62e-659121c683aa None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:47.629621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1feac559-b46a-45c8-b62e-659121c683aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.631694 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 815/1625] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:47 2022] POST /identity/v3/projects => generated 411 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:47.633445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c335ea-8f50-4376-b5d9-490a04fdb997 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.633873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c335ea-8f50-4376-b5d9-490a04fdb997 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.634295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c335ea-8f50-4376-b5d9-490a04fdb997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.658883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b6c335ea-8f50-4376-b5d9-490a04fdb997 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:47.660892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b6c335ea-8f50-4376-b5d9-490a04fdb997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.662560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feac559-b46a-45c8-b62e-659121c683aa None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:47.719811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:47.740756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49364881-9fa2-49c9-9a1b-e4c8e0840ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:47.741471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49364881-9fa2-49c9-9a1b-e4c8e0840ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.741861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49364881-9fa2-49c9-9a1b-e4c8e0840ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.742157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49364881-9fa2-49c9-9a1b-e4c8e0840ad8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.742966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49364881-9fa2-49c9-9a1b-e4c8e0840ad8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:47.745032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49364881-9fa2-49c9-9a1b-e4c8e0840ad8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:47.748556 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 816/1626] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:47.749528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c335ea-8f50-4376-b5d9-490a04fdb997 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:47.751062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b6c335ea-8f50-4376-b5d9-490a04fdb997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.764422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c335ea-8f50-4376-b5d9-490a04fdb997 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:47.771141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ec9ab-7871-4d8f-842d-0535514332cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:47.771973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ec9ab-7871-4d8f-842d-0535514332cd admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.772564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ec9ab-7871-4d8f-842d-0535514332cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.772644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ec9ab-7871-4d8f-842d-0535514332cd admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.773356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ec9ab-7871-4d8f-842d-0535514332cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:47.775564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ec9ab-7871-4d8f-842d-0535514332cd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:47.788194 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 811/1627] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:47 2022] POST /identity/v3/projects => generated 399 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:47.789594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49364881-9fa2-49c9-9a1b-e4c8e0840ad8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:47.807182 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 817/1628] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:47 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:47.807802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ec9ab-7871-4d8f-842d-0535514332cd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:47.811629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0d9ec9ab-7871-4d8f-842d-0535514332cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:47.866014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49364881-9fa2-49c9-9a1b-e4c8e0840ad8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:47.884887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ec9ab-7871-4d8f-842d-0535514332cd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:47.890450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af8dbd-685f-4971-b83b-e364f249b5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:47.891163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af8dbd-685f-4971-b83b-e364f249b5ec admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.891523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af8dbd-685f-4971-b83b-e364f249b5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.891859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af8dbd-685f-4971-b83b-e364f249b5ec admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.892645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af8dbd-685f-4971-b83b-e364f249b5ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:47.894799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af8dbd-685f-4971-b83b-e364f249b5ec admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:47.913529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:47.930533 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 818/1629] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:47 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:47.931762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af8dbd-685f-4971-b83b-e364f249b5ec admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:47.937691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49630ee2-6c2c-4b59-ab6b-b0b1abef7fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:47.938554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49630ee2-6c2c-4b59-ab6b-b0b1abef7fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.938913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49630ee2-6c2c-4b59-ab6b-b0b1abef7fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.939282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49630ee2-6c2c-4b59-ab6b-b0b1abef7fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.940145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49630ee2-6c2c-4b59-ab6b-b0b1abef7fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:47.942143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49630ee2-6c2c-4b59-ab6b-b0b1abef7fe7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:47.956732 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 812/1630] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:47.959368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9353c-9c58-4006-9cc3-69165e5e3402 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.960257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9353c-9c58-4006-9cc3-69165e5e3402 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.960583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9353c-9c58-4006-9cc3-69165e5e3402 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.968835 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 813/1631] 10.222.0.23 () {54 vars in 962 bytes} [Sat Aug 20 20:54:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:54:47.970147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd73ac-7b3c-4f2c-b869-134b77e5351e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:47.970554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd73ac-7b3c-4f2c-b869-134b77e5351e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:47.970830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd73ac-7b3c-4f2c-b869-134b77e5351e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:47.994019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a8cd73ac-7b3c-4f2c-b869-134b77e5351e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:47.995342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a8cd73ac-7b3c-4f2c-b869-134b77e5351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:48.004723 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 814/1632] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:48.005591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd73ac-7b3c-4f2c-b869-134b77e5351e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.007759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af8dbd-685f-4971-b83b-e364f249b5ec admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.013591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa38700-cf95-420b-93b0-7ab924ca0612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.014218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa38700-cf95-420b-93b0-7ab924ca0612 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.014479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa38700-cf95-420b-93b0-7ab924ca0612 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.014745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa38700-cf95-420b-93b0-7ab924ca0612 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.015411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa38700-cf95-420b-93b0-7ab924ca0612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.017671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa38700-cf95-420b-93b0-7ab924ca0612 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.018263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd73ac-7b3c-4f2c-b869-134b77e5351e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.032202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb1020-a785-4432-b4be-6766ba8a3a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.032202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb1020-a785-4432-b4be-6766ba8a3a1a admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.032202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb1020-a785-4432-b4be-6766ba8a3a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.032202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb1020-a785-4432-b4be-6766ba8a3a1a admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.032714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb1020-a785-4432-b4be-6766ba8a3a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.033270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb1020-a785-4432-b4be-6766ba8a3a1a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.052480 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 819/1633] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:47 2022] POST /identity/v3/users => generated 312 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:48.053576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa38700-cf95-420b-93b0-7ab924ca0612 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.074892 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 815/1634] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:48 2022] POST /identity/v3/users => generated 306 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:48.076467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb1020-a785-4432-b4be-6766ba8a3a1a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.077215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa38700-cf95-420b-93b0-7ab924ca0612 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.085553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3a7d1-dab8-47be-a46a-ffe39b8d6640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.086512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3a7d1-dab8-47be-a46a-ffe39b8d6640 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.086957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3a7d1-dab8-47be-a46a-ffe39b8d6640 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.087498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3a7d1-dab8-47be-a46a-ffe39b8d6640 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.088548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3a7d1-dab8-47be-a46a-ffe39b8d6640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.092863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3a7d1-dab8-47be-a46a-ffe39b8d6640 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.098875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb1020-a785-4432-b4be-6766ba8a3a1a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.106488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c7351-92ca-44a3-9c65-764ece581c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.107491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c7351-92ca-44a3-9c65-764ece581c0d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.107942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c7351-92ca-44a3-9c65-764ece581c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.108417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c7351-92ca-44a3-9c65-764ece581c0d admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.109574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c7351-92ca-44a3-9c65-764ece581c0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.112843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c7351-92ca-44a3-9c65-764ece581c0d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.129605 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 820/1635] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:48 2022] POST /identity/v3/users => generated 297 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:48.131778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3a7d1-dab8-47be-a46a-ffe39b8d6640 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.154573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3a7d1-dab8-47be-a46a-ffe39b8d6640 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.156639 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 816/1636] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:48 2022] POST /identity/v3/users => generated 296 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:48.159360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd664be2-f58d-4965-91ae-1560ee69109a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.160078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd664be2-f58d-4965-91ae-1560ee69109a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.160572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd664be2-f58d-4965-91ae-1560ee69109a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.165608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be75dc2-a9ac-4a78-8ebe-4a503097b77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.166566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be75dc2-a9ac-4a78-8ebe-4a503097b77e admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.166884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be75dc2-a9ac-4a78-8ebe-4a503097b77e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.167212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be75dc2-a9ac-4a78-8ebe-4a503097b77e admin admin] PATH_INFO: `/v3/projects/022df6b28f2644038160415fe5fbf339/users/218942b27bdb401ca842ee6ef27670d2/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.169816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be75dc2-a9ac-4a78-8ebe-4a503097b77e admin admin] RBAC: Authorizing `identity:create_grant(project_id=022df6b28f2644038160415fe5fbf339, user_id=218942b27bdb401ca842ee6ef27670d2, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.173816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be75dc2-a9ac-4a78-8ebe-4a503097b77e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.195779 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 821/1637] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:48 2022] PUT /identity/v3/projects/022df6b28f2644038160415fe5fbf339/users/218942b27bdb401ca842ee6ef27670d2/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:48.195779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be75dc2-a9ac-4a78-8ebe-4a503097b77e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.210222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:48.223860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cd664be2-f58d-4965-91ae-1560ee69109a None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:48.225166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cd664be2-f58d-4965-91ae-1560ee69109a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:48.283497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be75dc2-a9ac-4a78-8ebe-4a503097b77e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.289898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c47513-f5b3-49b3-b602-99e0682b91ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.290523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c47513-f5b3-49b3-b602-99e0682b91ef admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.290789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c47513-f5b3-49b3-b602-99e0682b91ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.291050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c47513-f5b3-49b3-b602-99e0682b91ef admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.291700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c47513-f5b3-49b3-b602-99e0682b91ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:54:48.292117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c47513-f5b3-49b3-b602-99e0682b91ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.293642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c47513-f5b3-49b3-b602-99e0682b91ef admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.304600 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 817/1638] 10.222.0.23 () {58 vars in 1059 bytes} [Sat Aug 20 20:54:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:48.305488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd664be2-f58d-4965-91ae-1560ee69109a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.306779 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 822/1639] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:54:48.307692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c47513-f5b3-49b3-b602-99e0682b91ef admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.319617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c47513-f5b3-49b3-b602-99e0682b91ef admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.325311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b926b09-a9d6-465f-9879-47869da2746b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.326332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b926b09-a9d6-465f-9879-47869da2746b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.326612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b926b09-a9d6-465f-9879-47869da2746b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.326711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd664be2-f58d-4965-91ae-1560ee69109a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.326869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b926b09-a9d6-465f-9879-47869da2746b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.327514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b926b09-a9d6-465f-9879-47869da2746b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.329177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b926b09-a9d6-465f-9879-47869da2746b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.331777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebac46-08ed-455d-ba7b-c395148e9b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.332247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebac46-08ed-455d-ba7b-c395148e9b9a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.332560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebac46-08ed-455d-ba7b-c395148e9b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.332826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebac46-08ed-455d-ba7b-c395148e9b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.333474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cebac46-08ed-455d-ba7b-c395148e9b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.335258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cebac46-08ed-455d-ba7b-c395148e9b9a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.341999 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 823/1640] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:48.342953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b926b09-a9d6-465f-9879-47869da2746b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.350031 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 818/1641] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:48.350999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebac46-08ed-455d-ba7b-c395148e9b9a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.355835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b926b09-a9d6-465f-9879-47869da2746b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.361217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20caf9b1-3c83-4d54-a7d8-c9d9bd27a278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.362025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20caf9b1-3c83-4d54-a7d8-c9d9bd27a278 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.362313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20caf9b1-3c83-4d54-a7d8-c9d9bd27a278 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.362866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20caf9b1-3c83-4d54-a7d8-c9d9bd27a278 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.363472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20caf9b1-3c83-4d54-a7d8-c9d9bd27a278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.365140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebac46-08ed-455d-ba7b-c395148e9b9a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.365653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20caf9b1-3c83-4d54-a7d8-c9d9bd27a278 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.370300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f35e9-6eac-4354-8d04-ca2a5ae973a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.370928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f35e9-6eac-4354-8d04-ca2a5ae973a6 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.371197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f35e9-6eac-4354-8d04-ca2a5ae973a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.371466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f35e9-6eac-4354-8d04-ca2a5ae973a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.372085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f35e9-6eac-4354-8d04-ca2a5ae973a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.373709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f35e9-6eac-4354-8d04-ca2a5ae973a6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.379173 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 824/1642] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:48.380502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21490f-507d-4d92-8311-e363ef23f6f3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.380855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21490f-507d-4d92-8311-e363ef23f6f3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.381173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21490f-507d-4d92-8311-e363ef23f6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.395255 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 819/1643] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:48.396334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f35e9-6eac-4354-8d04-ca2a5ae973a6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.408873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f35e9-6eac-4354-8d04-ca2a5ae973a6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.414681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ae21490f-507d-4d92-8311-e363ef23f6f3 None None] MFA Rules not processed for user `218942b27bdb401ca842ee6ef27670d2`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:48.416283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe9168-3536-422e-9e01-9d2aea38eaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.416495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ae21490f-507d-4d92-8311-e363ef23f6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:48.417051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe9168-3536-422e-9e01-9d2aea38eaac admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.417415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe9168-3536-422e-9e01-9d2aea38eaac admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.417753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe9168-3536-422e-9e01-9d2aea38eaac admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.418604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe9168-3536-422e-9e01-9d2aea38eaac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.428285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe9168-3536-422e-9e01-9d2aea38eaac admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.479295 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 820/1644] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:48 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:48.481143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe9168-3536-422e-9e01-9d2aea38eaac admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.526177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:48.544891 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 825/1645] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:48.545914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21490f-507d-4d92-8311-e363ef23f6f3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.569220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21490f-507d-4d92-8311-e363ef23f6f3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.582890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe9168-3536-422e-9e01-9d2aea38eaac admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.590363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11daf258-87a9-4086-90fa-34f82dd7ce20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.591150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daf258-87a9-4086-90fa-34f82dd7ce20 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.591427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daf258-87a9-4086-90fa-34f82dd7ce20 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.591826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daf258-87a9-4086-90fa-34f82dd7ce20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.593849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-11daf258-87a9-4086-90fa-34f82dd7ce20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:48.645654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211978c-2422-4958-9268-b156fd186853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.646324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211978c-2422-4958-9268-b156fd186853 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.646590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211978c-2422-4958-9268-b156fd186853 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.646851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211978c-2422-4958-9268-b156fd186853 admin admin] PATH_INFO: `/v3/projects/5a6ddebe39bc4c32914dab26c3b7d437/users/e8ee40e4a8b7453e8f3b394f7cf81094/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.663745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211978c-2422-4958-9268-b156fd186853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6ddebe39bc4c32914dab26c3b7d437, user_id=e8ee40e4a8b7453e8f3b394f7cf81094, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.666673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211978c-2422-4958-9268-b156fd186853 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.669087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:48.685649 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 826/1646] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:48 2022] PUT /identity/v3/projects/5a6ddebe39bc4c32914dab26c3b7d437/users/e8ee40e4a8b7453e8f3b394f7cf81094/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:48.686838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211978c-2422-4958-9268-b156fd186853 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.698674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11daf258-87a9-4086-90fa-34f82dd7ce20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.702238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11daf258-87a9-4086-90fa-34f82dd7ce20 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.717736 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 821/1647] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:48.719437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11daf258-87a9-4086-90fa-34f82dd7ce20 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.738893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11daf258-87a9-4086-90fa-34f82dd7ce20 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.802462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211978c-2422-4958-9268-b156fd186853 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:48.807528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d767f7-3e06-4321-a498-89269d62fcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.808255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d767f7-3e06-4321-a498-89269d62fcc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.808558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d767f7-3e06-4321-a498-89269d62fcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.808820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d767f7-3e06-4321-a498-89269d62fcc1 admin admin] PATH_INFO: `/v3/projects/3e9bcfa245224652a766f2310c25e7d2/users/fa81cae2d2064780832aaef27161951b/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.827635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d767f7-3e06-4321-a498-89269d62fcc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e9bcfa245224652a766f2310c25e7d2, user_id=fa81cae2d2064780832aaef27161951b, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.830751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d767f7-3e06-4321-a498-89269d62fcc1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.852394 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 827/1648] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:48 2022] PUT /identity/v3/projects/3e9bcfa245224652a766f2310c25e7d2/users/fa81cae2d2064780832aaef27161951b/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:48.854193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d767f7-3e06-4321-a498-89269d62fcc1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:48.870940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed6f54-fd4d-475c-b039-21bf4a397644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:48.872993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed6f54-fd4d-475c-b039-21bf4a397644 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:48.873511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed6f54-fd4d-475c-b039-21bf4a397644 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:48.873786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed6f54-fd4d-475c-b039-21bf4a397644 admin admin] PATH_INFO: `/v3/projects/06a86e61472b4ad69a50b4308ff3a7b7/users/77b94de1ecd64564bac67b21996158d9/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:48.913220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed6f54-fd4d-475c-b039-21bf4a397644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a86e61472b4ad69a50b4308ff3a7b7, user_id=77b94de1ecd64564bac67b21996158d9, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:48.922108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed6f54-fd4d-475c-b039-21bf4a397644 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:48.950106 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 822/1649] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:48 2022] PUT /identity/v3/projects/06a86e61472b4ad69a50b4308ff3a7b7/users/77b94de1ecd64564bac67b21996158d9/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:48.951033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed6f54-fd4d-475c-b039-21bf4a397644 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:49.008085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d767f7-3e06-4321-a498-89269d62fcc1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:49.059836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:49.078926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed6f54-fd4d-475c-b039-21bf4a397644 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:49.084401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fbce9-0ce1-4777-94e7-53078e0a36f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:49.085030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fbce9-0ce1-4777-94e7-53078e0a36f7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.085383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fbce9-0ce1-4777-94e7-53078e0a36f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.085658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fbce9-0ce1-4777-94e7-53078e0a36f7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.086385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fbce9-0ce1-4777-94e7-53078e0a36f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:49.088649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fbce9-0ce1-4777-94e7-53078e0a36f7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:49.104246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e260e83-6450-4f1c-ad8e-8cbac19436b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:49.105461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260e83-6450-4f1c-ad8e-8cbac19436b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.105989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260e83-6450-4f1c-ad8e-8cbac19436b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.106501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260e83-6450-4f1c-ad8e-8cbac19436b2 admin admin] PATH_INFO: `/v3/projects/1cc9b7e63d3b4b7fb38b1b1855a22778/users/d32ef91f38084fe09f80557ba543f7c4/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.119084 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 823/1650] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:48 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:49.120677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b43a5-646d-426d-b2bb-1a71fdc279d1 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.121130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b43a5-646d-426d-b2bb-1a71fdc279d1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.121799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b43a5-646d-426d-b2bb-1a71fdc279d1 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.123235 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 824/1651] 10.222.0.23 () {54 vars in 962 bytes} [Sat Aug 20 20:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:54:49.124068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b43a5-646d-426d-b2bb-1a71fdc279d1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:49.136659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e260e83-6450-4f1c-ad8e-8cbac19436b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc9b7e63d3b4b7fb38b1b1855a22778, user_id=d32ef91f38084fe09f80557ba543f7c4, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:49.139348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e260e83-6450-4f1c-ad8e-8cbac19436b2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:49.140400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b43a5-646d-426d-b2bb-1a71fdc279d1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:49.145778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e7a97-f39a-448d-89c9-3512ced4e75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:49.146634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e7a97-f39a-448d-89c9-3512ced4e75c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.146634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e7a97-f39a-448d-89c9-3512ced4e75c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.146903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e7a97-f39a-448d-89c9-3512ced4e75c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.147562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e7a97-f39a-448d-89c9-3512ced4e75c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:49.149318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e7a97-f39a-448d-89c9-3512ced4e75c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:49.163646 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 825/1652] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:49.164724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e7a97-f39a-448d-89c9-3512ced4e75c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:49.167110 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 828/1653] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:48 2022] PUT /identity/v3/projects/1cc9b7e63d3b4b7fb38b1b1855a22778/users/d32ef91f38084fe09f80557ba543f7c4/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:49.168394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e260e83-6450-4f1c-ad8e-8cbac19436b2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:49.178345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e7a97-f39a-448d-89c9-3512ced4e75c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:49.181588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:49.267678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e260e83-6450-4f1c-ad8e-8cbac19436b2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:49.273151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc16b9a-88eb-46fb-8b30-cdbb5acf13ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:49.274196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6b6a3-955d-45ff-a64b-a22b82dbe288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:49.274888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6b6a3-955d-45ff-a64b-a22b82dbe288 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.275226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6b6a3-955d-45ff-a64b-a22b82dbe288 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.275699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6b6a3-955d-45ff-a64b-a22b82dbe288 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.276519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6b6a3-955d-45ff-a64b-a22b82dbe288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:49.279082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6b6a3-955d-45ff-a64b-a22b82dbe288 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:49.285041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc16b9a-88eb-46fb-8b30-cdbb5acf13ca admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.285617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc16b9a-88eb-46fb-8b30-cdbb5acf13ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.289751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc16b9a-88eb-46fb-8b30-cdbb5acf13ca admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.291446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc16b9a-88eb-46fb-8b30-cdbb5acf13ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:49.293899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc16b9a-88eb-46fb-8b30-cdbb5acf13ca admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:49.301292 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 829/1654] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:49.303562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d6b1d-b623-4b29-8253-922adf001c5b None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.303909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d6b1d-b623-4b29-8253-922adf001c5b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.304254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d6b1d-b623-4b29-8253-922adf001c5b None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.306743 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 830/1655] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:54:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:54:49.307493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d6b1d-b623-4b29-8253-922adf001c5b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:49.309388 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 826/1656] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:49.311241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990893c-10f6-4d22-9b40-387a930a9074 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.311735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990893c-10f6-4d22-9b40-387a930a9074 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.312210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990893c-10f6-4d22-9b40-387a930a9074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.329805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d6b1d-b623-4b29-8253-922adf001c5b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:49.335424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b154596-5cc7-443b-bf0d-8798f3ce378d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:49.336904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b154596-5cc7-443b-bf0d-8798f3ce378d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.337170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b154596-5cc7-443b-bf0d-8798f3ce378d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.337436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b154596-5cc7-443b-bf0d-8798f3ce378d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.338262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b154596-5cc7-443b-bf0d-8798f3ce378d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:49.341358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b154596-5cc7-443b-bf0d-8798f3ce378d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:49.345326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a990893c-10f6-4d22-9b40-387a930a9074 None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:49.346741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a990893c-10f6-4d22-9b40-387a930a9074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:49.361870 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 831/1657] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:49.361870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b154596-5cc7-443b-bf0d-8798f3ce378d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:49.374016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b154596-5cc7-443b-bf0d-8798f3ce378d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:49.382253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e9cb0-2463-4d1a-9c10-e47694b70be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:49.383034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e9cb0-2463-4d1a-9c10-e47694b70be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.383459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e9cb0-2463-4d1a-9c10-e47694b70be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.383904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e9cb0-2463-4d1a-9c10-e47694b70be7 admin admin] PATH_INFO: `/v3/projects/5a6ddebe39bc4c32914dab26c3b7d437/users/e8ee40e4a8b7453e8f3b394f7cf81094/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.386089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e9cb0-2463-4d1a-9c10-e47694b70be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6ddebe39bc4c32914dab26c3b7d437, user_id=e8ee40e4a8b7453e8f3b394f7cf81094, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:49.389615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e9cb0-2463-4d1a-9c10-e47694b70be7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:49.412458 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 832/1658] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:49 2022] PUT /identity/v3/projects/5a6ddebe39bc4c32914dab26c3b7d437/users/e8ee40e4a8b7453e8f3b394f7cf81094/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:49.419184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e9cb0-2463-4d1a-9c10-e47694b70be7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:49.427776 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 827/1659] 10.222.0.23 () {58 vars in 1059 bytes} [Sat Aug 20 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:49.428985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990893c-10f6-4d22-9b40-387a930a9074 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:49.440893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990893c-10f6-4d22-9b40-387a930a9074 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:49.525960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e9cb0-2463-4d1a-9c10-e47694b70be7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:49.532727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f31296-c871-4ca7-87c0-5a42e1ff5908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:49.533365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f31296-c871-4ca7-87c0-5a42e1ff5908 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.533640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f31296-c871-4ca7-87c0-5a42e1ff5908 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.533910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f31296-c871-4ca7-87c0-5a42e1ff5908 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.534621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f31296-c871-4ca7-87c0-5a42e1ff5908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:49.536311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f31296-c871-4ca7-87c0-5a42e1ff5908 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:49.544083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:49.549168 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 833/1660] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:49.549890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f31296-c871-4ca7-87c0-5a42e1ff5908 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:49.560617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418b9ba-0898-417b-b82f-f6cc1a28b323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:49.561315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418b9ba-0898-417b-b82f-f6cc1a28b323 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.561604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418b9ba-0898-417b-b82f-f6cc1a28b323 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.561875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418b9ba-0898-417b-b82f-f6cc1a28b323 admin admin] PATH_INFO: `/v3/projects/06a86e61472b4ad69a50b4308ff3a7b7/users/77b94de1ecd64564bac67b21996158d9/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.563716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f31296-c871-4ca7-87c0-5a42e1ff5908 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:49.571479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418b9ba-0898-417b-b82f-f6cc1a28b323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a86e61472b4ad69a50b4308ff3a7b7, user_id=77b94de1ecd64564bac67b21996158d9, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:49.574839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20ee12-118f-435e-b0e4-aef91b223880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:49.575601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20ee12-118f-435e-b0e4-aef91b223880 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.575939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20ee12-118f-435e-b0e4-aef91b223880 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.576309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20ee12-118f-435e-b0e4-aef91b223880 admin admin] PATH_INFO: `/v3/projects/3e9bcfa245224652a766f2310c25e7d2/users/fa81cae2d2064780832aaef27161951b/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.579617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418b9ba-0898-417b-b82f-f6cc1a28b323 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:49.582994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20ee12-118f-435e-b0e4-aef91b223880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e9bcfa245224652a766f2310c25e7d2, user_id=fa81cae2d2064780832aaef27161951b, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:49.592514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20ee12-118f-435e-b0e4-aef91b223880 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:49.596813 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 828/1661] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:49 2022] PUT /identity/v3/projects/06a86e61472b4ad69a50b4308ff3a7b7/users/77b94de1ecd64564bac67b21996158d9/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:49.599163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6be87-5fcb-4f9e-ac29-0dc4e6a802cf None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.599446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6be87-5fcb-4f9e-ac29-0dc4e6a802cf None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.599712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6be87-5fcb-4f9e-ac29-0dc4e6a802cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.609600 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 834/1662] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:49 2022] PUT /identity/v3/projects/3e9bcfa245224652a766f2310c25e7d2/users/fa81cae2d2064780832aaef27161951b/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:49.610982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20ee12-118f-435e-b0e4-aef91b223880 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:49.636396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-87d6be87-5fcb-4f9e-ac29-0dc4e6a802cf None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:49.642318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-87d6be87-5fcb-4f9e-ac29-0dc4e6a802cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:49.695716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20ee12-118f-435e-b0e4-aef91b223880 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:49.700493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184f0cc-7f29-4ef1-9e7d-dccbabd84854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:49.701424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184f0cc-7f29-4ef1-9e7d-dccbabd84854 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.701694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184f0cc-7f29-4ef1-9e7d-dccbabd84854 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.701950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184f0cc-7f29-4ef1-9e7d-dccbabd84854 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.702581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184f0cc-7f29-4ef1-9e7d-dccbabd84854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:49.704179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184f0cc-7f29-4ef1-9e7d-dccbabd84854 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:49.712891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:49.719183 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 835/1663] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:49.720844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862082f1-9543-4189-adf6-4d51e809d121 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.721172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862082f1-9543-4189-adf6-4d51e809d121 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.721485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862082f1-9543-4189-adf6-4d51e809d121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.734164 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 829/1664] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:49.737302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6be87-5fcb-4f9e-ac29-0dc4e6a802cf None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:49.752004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6be87-5fcb-4f9e-ac29-0dc4e6a802cf None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:49.752699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-862082f1-9543-4189-adf6-4d51e809d121 None None] MFA Rules not processed for user `e8ee40e4a8b7453e8f3b394f7cf81094`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:49.753984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-862082f1-9543-4189-adf6-4d51e809d121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:49.758726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06288a-e7e0-470e-b620-8565ba3b4042 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:49.759338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06288a-e7e0-470e-b620-8565ba3b4042 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.759697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06288a-e7e0-470e-b620-8565ba3b4042 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.760029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06288a-e7e0-470e-b620-8565ba3b4042 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.824533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06288a-e7e0-470e-b620-8565ba3b4042 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:49.826236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06288a-e7e0-470e-b620-8565ba3b4042 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:49.836359 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 830/1665] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:49.836947 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 836/1666] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:49.837416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06288a-e7e0-470e-b620-8565ba3b4042 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:49.838292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e87f8-16c3-4331-9f3a-19464f33c60f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.838573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e87f8-16c3-4331-9f3a-19464f33c60f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.838839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e87f8-16c3-4331-9f3a-19464f33c60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.848096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06288a-e7e0-470e-b620-8565ba3b4042 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:49.852824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a226e-8c6b-4d26-bc76-a0d5050a15c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:49.853479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a226e-8c6b-4d26-bc76-a0d5050a15c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.853752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a226e-8c6b-4d26-bc76-a0d5050a15c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.854019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a226e-8c6b-4d26-bc76-a0d5050a15c3 admin admin] PATH_INFO: `/v3/projects/1cc9b7e63d3b4b7fb38b1b1855a22778/users/d32ef91f38084fe09f80557ba543f7c4/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.855920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a226e-8c6b-4d26-bc76-a0d5050a15c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc9b7e63d3b4b7fb38b1b1855a22778, user_id=d32ef91f38084fe09f80557ba543f7c4, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:49.858605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a226e-8c6b-4d26-bc76-a0d5050a15c3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:49.869980 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 831/1667] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:49 2022] PUT /identity/v3/projects/1cc9b7e63d3b4b7fb38b1b1855a22778/users/d32ef91f38084fe09f80557ba543f7c4/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:49.870943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678fc9f-2694-40a6-b8f5-8e0fbd0ae4a2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:49.871217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678fc9f-2694-40a6-b8f5-8e0fbd0ae4a2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:49.871782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678fc9f-2694-40a6-b8f5-8e0fbd0ae4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:49.883042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-535e87f8-16c3-4331-9f3a-19464f33c60f None None] MFA Rules not processed for user `77b94de1ecd64564bac67b21996158d9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:49.884262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-535e87f8-16c3-4331-9f3a-19464f33c60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:49.916198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0678fc9f-2694-40a6-b8f5-8e0fbd0ae4a2 None None] MFA Rules not processed for user `fa81cae2d2064780832aaef27161951b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:49.917949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0678fc9f-2694-40a6-b8f5-8e0fbd0ae4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:49.978183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:49.986444 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 837/1668] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:49.987817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e87f8-16c3-4331-9f3a-19464f33c60f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:50.011667 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 832/1669] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:50.011667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678fc9f-2694-40a6-b8f5-8e0fbd0ae4a2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:50.013374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0678fc9f-2694-40a6-b8f5-8e0fbd0ae4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:50.115348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e87f8-16c3-4331-9f3a-19464f33c60f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:50.120554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f7ab5-eba7-48ea-a644-5b11a6564443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:50.121515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f7ab5-eba7-48ea-a644-5b11a6564443 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:50.121988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f7ab5-eba7-48ea-a644-5b11a6564443 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:50.122444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f7ab5-eba7-48ea-a644-5b11a6564443 admin admin] PATH_INFO: `/v3/projects/8c05a02088d7460aa8b889c9fe6278bd/users/1d816c885f6f4b7bb3f3208c3d8c7388/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:50.138833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:50.143313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f7ab5-eba7-48ea-a644-5b11a6564443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c05a02088d7460aa8b889c9fe6278bd, user_id=1d816c885f6f4b7bb3f3208c3d8c7388, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:50.147484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f7ab5-eba7-48ea-a644-5b11a6564443 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:50.172178 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 838/1670] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:49 2022] PUT /identity/v3/projects/8c05a02088d7460aa8b889c9fe6278bd/users/1d816c885f6f4b7bb3f3208c3d8c7388/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:50.173752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd9eef-fddf-407d-845b-0d23129e0dee None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:50.174477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd9eef-fddf-407d-845b-0d23129e0dee None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:50.174796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd9eef-fddf-407d-845b-0d23129e0dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:50.180824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678fc9f-2694-40a6-b8f5-8e0fbd0ae4a2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:50.258782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-66fd9eef-fddf-407d-845b-0d23129e0dee None None] MFA Rules not processed for user `d32ef91f38084fe09f80557ba543f7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:50.260318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-66fd9eef-fddf-407d-845b-0d23129e0dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:50.285563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de48098-0b81-4c3e-b5ae-d76c00d158b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:50.285787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de48098-0b81-4c3e-b5ae-d76c00d158b3 service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:50.286054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de48098-0b81-4c3e-b5ae-d76c00d158b3 service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:50.286512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de48098-0b81-4c3e-b5ae-d76c00d158b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:50.398455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de48098-0b81-4c3e-b5ae-d76c00d158b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:50.399926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de48098-0b81-4c3e-b5ae-d76c00d158b3 service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:50.424437 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 833/1671] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:50.425510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0200949-b662-4634-9582-c80d6b9d5434 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:50.425790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0200949-b662-4634-9582-c80d6b9d5434 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:50.426055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0200949-b662-4634-9582-c80d6b9d5434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:50.452716 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 839/1672] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:50.453728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd9eef-fddf-407d-845b-0d23129e0dee None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:50.493168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e0200949-b662-4634-9582-c80d6b9d5434 None None] MFA Rules not processed for user `1d816c885f6f4b7bb3f3208c3d8c7388`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:50.494752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e0200949-b662-4634-9582-c80d6b9d5434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:50.514903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:50.531063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd9eef-fddf-407d-845b-0d23129e0dee None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:50.545698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48b05e-d4ec-4dfa-baa4-16ab355c101a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:50.546365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48b05e-d4ec-4dfa-baa4-16ab355c101a service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:50.546637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48b05e-d4ec-4dfa-baa4-16ab355c101a service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:50.546904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48b05e-d4ec-4dfa-baa4-16ab355c101a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:50.671240 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 834/1673] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:50.675176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0200949-b662-4634-9582-c80d6b9d5434 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:50.679228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48b05e-d4ec-4dfa-baa4-16ab355c101a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:50.681107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48b05e-d4ec-4dfa-baa4-16ab355c101a service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:50.704411 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 840/1674] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:50 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:50.848718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0200949-b662-4634-9582-c80d6b9d5434 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:50.869361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cad9af-0e70-4dfa-a13b-8a61c412afcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:50.870200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cad9af-0e70-4dfa-a13b-8a61c412afcd admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:50.870583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cad9af-0e70-4dfa-a13b-8a61c412afcd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:50.870920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cad9af-0e70-4dfa-a13b-8a61c412afcd admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:50.871757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cad9af-0e70-4dfa-a13b-8a61c412afcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:50.873859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cad9af-0e70-4dfa-a13b-8a61c412afcd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:50.910708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48b05e-d4ec-4dfa-baa4-16ab355c101a service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:50.921464 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 835/1675] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:50 2022] POST /identity/v3/projects => generated 369 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:50.924652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cad9af-0e70-4dfa-a13b-8a61c412afcd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:50.939053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48b05e-d4ec-4dfa-baa4-16ab355c101a service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:51.090699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1286765-55e7-4ac5-b1c9-a3536ce4a1ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:51.091379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1286765-55e7-4ac5-b1c9-a3536ce4a1ac service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:51.091643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1286765-55e7-4ac5-b1c9-a3536ce4a1ac service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:51.095671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1286765-55e7-4ac5-b1c9-a3536ce4a1ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:51.104864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cad9af-0e70-4dfa-a13b-8a61c412afcd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:51.107160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:51.116043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873a9b0-f72d-4419-9263-e70348c9f64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:51.116737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873a9b0-f72d-4419-9263-e70348c9f64b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:51.117011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873a9b0-f72d-4419-9263-e70348c9f64b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:51.117278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873a9b0-f72d-4419-9263-e70348c9f64b admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:51.117955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873a9b0-f72d-4419-9263-e70348c9f64b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:51.120072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873a9b0-f72d-4419-9263-e70348c9f64b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:51.172248 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 836/1676] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:50 2022] POST /identity/v3/users => generated 291 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:51.173399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873a9b0-f72d-4419-9263-e70348c9f64b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:51.197784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873a9b0-f72d-4419-9263-e70348c9f64b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:51.216489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c999354-7242-4600-9e36-6b4f6dacf526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:51.217116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c999354-7242-4600-9e36-6b4f6dacf526 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:51.217632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c999354-7242-4600-9e36-6b4f6dacf526 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:51.217937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c999354-7242-4600-9e36-6b4f6dacf526 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:51.220159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c999354-7242-4600-9e36-6b4f6dacf526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:51.234788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c999354-7242-4600-9e36-6b4f6dacf526 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:51.251133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1286765-55e7-4ac5-b1c9-a3536ce4a1ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:51.258116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1286765-55e7-4ac5-b1c9-a3536ce4a1ac service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:51.258383 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 837/1677] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:51.259336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c999354-7242-4600-9e36-6b4f6dacf526 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:51.280964 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 841/1678] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:54:51.291245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c999354-7242-4600-9e36-6b4f6dacf526 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:51.299336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1286765-55e7-4ac5-b1c9-a3536ce4a1ac service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:51.313509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1286765-55e7-4ac5-b1c9-a3536ce4a1ac service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:51.313725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950f596-318d-49f3-99e4-9cadb04ba5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:51.314448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950f596-318d-49f3-99e4-9cadb04ba5c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:51.314828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950f596-318d-49f3-99e4-9cadb04ba5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:51.314978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950f596-318d-49f3-99e4-9cadb04ba5c9 admin admin] PATH_INFO: `/v3/projects/1fb1f1597c4f45acba73e980ef469fb4/users/05ef6806617041188e892355fa4ce41a/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:51.322394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca7a22-6875-4830-b204-088553601ea9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:51.323532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca7a22-6875-4830-b204-088553601ea9 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:51.323994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca7a22-6875-4830-b204-088553601ea9 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:51.324564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca7a22-6875-4830-b204-088553601ea9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:51.366288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950f596-318d-49f3-99e4-9cadb04ba5c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb1f1597c4f45acba73e980ef469fb4, user_id=05ef6806617041188e892355fa4ce41a, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:51.372403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950f596-318d-49f3-99e4-9cadb04ba5c9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:51.377312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:51.405618 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 838/1679] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:51 2022] PUT /identity/v3/projects/1fb1f1597c4f45acba73e980ef469fb4/users/05ef6806617041188e892355fa4ce41a/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:51.410398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950f596-318d-49f3-99e4-9cadb04ba5c9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:51.474993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca7a22-6875-4830-b204-088553601ea9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:51.476783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca7a22-6875-4830-b204-088553601ea9 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:51.489488 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 842/1680] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:51 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:54:51.491241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca7a22-6875-4830-b204-088553601ea9 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:51.493055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5cca7a22-6875-4830-b204-088553601ea9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:51.571108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950f596-318d-49f3-99e4-9cadb04ba5c9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:51.578773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22cc17-cf79-481f-960b-c1941f3b88a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:51.579928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22cc17-cf79-481f-960b-c1941f3b88a0 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:51.580284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22cc17-cf79-481f-960b-c1941f3b88a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:51.580662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22cc17-cf79-481f-960b-c1941f3b88a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:51.581558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22cc17-cf79-481f-960b-c1941f3b88a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:51.583831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22cc17-cf79-481f-960b-c1941f3b88a0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:51.599216 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 839/1681] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:51.600613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22cc17-cf79-481f-960b-c1941f3b88a0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:51.603236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca7a22-6875-4830-b204-088553601ea9 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:51.668654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:51.695646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdeb850-fda0-45b1-ad8e-eab59ed0aa85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:51.697114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeb850-fda0-45b1-ad8e-eab59ed0aa85 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:51.697403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeb850-fda0-45b1-ad8e-eab59ed0aa85 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:51.697909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeb850-fda0-45b1-ad8e-eab59ed0aa85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:51.700332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6cdeb850-fda0-45b1-ad8e-eab59ed0aa85 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:51.713901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdeb850-fda0-45b1-ad8e-eab59ed0aa85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:51.716100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdeb850-fda0-45b1-ad8e-eab59ed0aa85 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:51.726714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22cc17-cf79-481f-960b-c1941f3b88a0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:51.740815 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 843/1682] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:51.741789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdeb850-fda0-45b1-ad8e-eab59ed0aa85 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:51.742348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5da86b-10fa-4f33-b79d-8a3354e738a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:51.742985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da86b-10fa-4f33-b79d-8a3354e738a2 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:51.743324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da86b-10fa-4f33-b79d-8a3354e738a2 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:51.743594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da86b-10fa-4f33-b79d-8a3354e738a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:51.762233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdeb850-fda0-45b1-ad8e-eab59ed0aa85 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:51.777269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c2463-a7b8-4545-99ef-12f520041d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:51.778172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c2463-a7b8-4545-99ef-12f520041d23 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:51.778694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c2463-a7b8-4545-99ef-12f520041d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:51.779652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c2463-a7b8-4545-99ef-12f520041d23 admin admin] PATH_INFO: `/v3/projects/1fb1f1597c4f45acba73e980ef469fb4/users/05ef6806617041188e892355fa4ce41a/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:51.784017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c2463-a7b8-4545-99ef-12f520041d23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb1f1597c4f45acba73e980ef469fb4, user_id=05ef6806617041188e892355fa4ce41a, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:51.790757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c2463-a7b8-4545-99ef-12f520041d23 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:51.822257 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 844/1683] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:51 2022] PUT /identity/v3/projects/1fb1f1597c4f45acba73e980ef469fb4/users/05ef6806617041188e892355fa4ce41a/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:51.824401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd7fd7-bcc1-4291-a3ff-63b587f7259d None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:51.824774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd7fd7-bcc1-4291-a3ff-63b587f7259d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:51.825132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd7fd7-bcc1-4291-a3ff-63b587f7259d None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:51.832871 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 845/1684] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:54:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:54:51.835702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dddfd0-cb5d-4e2e-ab12-80e9679b609d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:51.836911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dddfd0-cb5d-4e2e-ab12-80e9679b609d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:51.837708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dddfd0-cb5d-4e2e-ab12-80e9679b609d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:51.883507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c5dddfd0-cb5d-4e2e-ab12-80e9679b609d None None] MFA Rules not processed for user `05ef6806617041188e892355fa4ce41a`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:51.885055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c5dddfd0-cb5d-4e2e-ab12-80e9679b609d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:51.917157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5da86b-10fa-4f33-b79d-8a3354e738a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:51.922176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5da86b-10fa-4f33-b79d-8a3354e738a2 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:51.952519 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 840/1685] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:51.969609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11754f9e-aa07-49cb-95f4-e33d1b1ef9c0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:51.969905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11754f9e-aa07-49cb-95f4-e33d1b1ef9c0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:51.970463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11754f9e-aa07-49cb-95f4-e33d1b1ef9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:52.018791 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 846/1686] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:52.074849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-11754f9e-aa07-49cb-95f4-e33d1b1ef9c0 None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:52.074849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-11754f9e-aa07-49cb-95f4-e33d1b1ef9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:52.084647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dddfd0-cb5d-4e2e-ab12-80e9679b609d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:52.120818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:52.226387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dddfd0-cb5d-4e2e-ab12-80e9679b609d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:52.234767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a284442-faa5-4775-9733-9e4a52e9597d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:52.236637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a284442-faa5-4775-9733-9e4a52e9597d service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:52.237004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a284442-faa5-4775-9733-9e4a52e9597d service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:52.237371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a284442-faa5-4775-9733-9e4a52e9597d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:52.241038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3a284442-faa5-4775-9733-9e4a52e9597d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:52.309046 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 841/1687] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 20:54:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:52.321498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11754f9e-aa07-49cb-95f4-e33d1b1ef9c0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:52.323211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-11754f9e-aa07-49cb-95f4-e33d1b1ef9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:52.327526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:52.380590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a284442-faa5-4775-9733-9e4a52e9597d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:52.382545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a284442-faa5-4775-9733-9e4a52e9597d service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:52.401485 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 847/1688] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:52 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:52.449703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11754f9e-aa07-49cb-95f4-e33d1b1ef9c0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:52.462244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042397a7-3e1a-47c4-a82a-c76ec0dcf472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:52.462888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042397a7-3e1a-47c4-a82a-c76ec0dcf472 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:52.463168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042397a7-3e1a-47c4-a82a-c76ec0dcf472 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:52.463669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042397a7-3e1a-47c4-a82a-c76ec0dcf472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:52.496626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042397a7-3e1a-47c4-a82a-c76ec0dcf472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:52.497354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a284442-faa5-4775-9733-9e4a52e9597d service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:52.498091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042397a7-3e1a-47c4-a82a-c76ec0dcf472 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:52.523675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a284442-faa5-4775-9733-9e4a52e9597d service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:52.528711 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 842/1689] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:54:52.548585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57cd31-d90c-424b-864c-5fe4a85f04a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:52.549425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57cd31-d90c-424b-864c-5fe4a85f04a0 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:52.549905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57cd31-d90c-424b-864c-5fe4a85f04a0 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:52.550362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57cd31-d90c-424b-864c-5fe4a85f04a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:52.560290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57cd31-d90c-424b-864c-5fe4a85f04a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:52.561958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57cd31-d90c-424b-864c-5fe4a85f04a0 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:52.576062 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 848/1690] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:54:52.670036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42619c4-5102-4318-b4f0-8ed94659c291 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:52.670302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42619c4-5102-4318-b4f0-8ed94659c291 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:52.670565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42619c4-5102-4318-b4f0-8ed94659c291 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:52.672319 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 843/1691] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:54:52.678093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57cd31-d90c-424b-864c-5fe4a85f04a0 service nova] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:54:52.696516 ncdv-master-1 devstack@keystone.service[104118]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-4e57cd31-d90c-424b-864c-5fe4a85f04a0 service nova] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Aug 20 20:54:52.696516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57cd31-d90c-424b-864c-5fe4a85f04a0 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:52.726963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42619c4-5102-4318-b4f0-8ed94659c291 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:52.894582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57cd31-d90c-424b-864c-5fe4a85f04a0 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:52.904654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c163fa-d48c-4488-afcf-33eb2b5b7083 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32ef91f38084fe09f80557ba543f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cc9b7e63d3b4b7fb38b1b1855a22778', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:52.905221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42619c4-5102-4318-b4f0-8ed94659c291 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:52.910955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c163fa-d48c-4488-afcf-33eb2b5b7083 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:52.914038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0514d9-8b0b-44c0-9199-d0132f7be3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:52.915259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0514d9-8b0b-44c0-9199-d0132f7be3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:52.915787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0514d9-8b0b-44c0-9199-d0132f7be3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:52.916225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0514d9-8b0b-44c0-9199-d0132f7be3ff admin admin] PATH_INFO: `/v3/users/4983cbd777b244ab9badec08f75708ba` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:52.917537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c163fa-d48c-4488-afcf-33eb2b5b7083 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:52.917883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c163fa-d48c-4488-afcf-33eb2b5b7083 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] PATH_INFO: `/v3/projects/2e2f79566b574520aabb42684e035f10` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:52.918263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0514d9-8b0b-44c0-9199-d0132f7be3ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=4983cbd777b244ab9badec08f75708ba)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:52.919396 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-7d0514d9-8b0b-44c0-9199-d0132f7be3ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:54:52.919396 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 20:54:52.919396 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 20:54:52.921111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0514d9-8b0b-44c0-9199-d0132f7be3ff admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:52.923040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c163fa-d48c-4488-afcf-33eb2b5b7083 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] RBAC: Authorizing `identity:get_project(project_id=2e2f79566b574520aabb42684e035f10)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:52.932943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c163fa-d48c-4488-afcf-33eb2b5b7083 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:52.951518 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 849/1692] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:54:52 2022] GET /identity/v3/projects/2e2f79566b574520aabb42684e035f10 => generated 377 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:54:53.034185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226dec8-11f3-47bf-ad03-497ab46e48b1 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:53.034984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226dec8-11f3-47bf-ad03-497ab46e48b1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:53.034984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226dec8-11f3-47bf-ad03-497ab46e48b1 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:53.036889 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 850/1693] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:54:53.042383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98274002-d1d1-4628-8f7b-df75bd3e32ba None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:53.042651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98274002-d1d1-4628-8f7b-df75bd3e32ba None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:53.042913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98274002-d1d1-4628-8f7b-df75bd3e32ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:53.114268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-98274002-d1d1-4628-8f7b-df75bd3e32ba None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:53.117512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-98274002-d1d1-4628-8f7b-df75bd3e32ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:53.143112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7d0514d9-8b0b-44c0-9199-d0132f7be3ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4983cbd777b244ab9badec08f75708ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:53.162956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7d0514d9-8b0b-44c0-9199-d0132f7be3ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4983cbd777b244ab9badec08f75708ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:53.178752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:53.211554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7d0514d9-8b0b-44c0-9199-d0132f7be3ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4983cbd777b244ab9badec08f75708ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:53.213055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7d0514d9-8b0b-44c0-9199-d0132f7be3ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4983cbd777b244ab9badec08f75708ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:53.215468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7d0514d9-8b0b-44c0-9199-d0132f7be3ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4983cbd777b244ab9badec08f75708ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:53.241835 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 844/1694] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:54:52 2022] DELETE /identity/v3/users/4983cbd777b244ab9badec08f75708ba => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:53.246658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d681883-7721-4414-8fda-6d3ae8cc4d8a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:53.247401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d681883-7721-4414-8fda-6d3ae8cc4d8a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:53.247872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d681883-7721-4414-8fda-6d3ae8cc4d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:53.289127 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 851/1695] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 20:54:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 20 20:54:53.302316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98274002-d1d1-4628-8f7b-df75bd3e32ba None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:53.305436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6d681883-7721-4414-8fda-6d3ae8cc4d8a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:53.306694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6d681883-7721-4414-8fda-6d3ae8cc4d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:53.309551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-98274002-d1d1-4628-8f7b-df75bd3e32ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:53.329976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98274002-d1d1-4628-8f7b-df75bd3e32ba None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:53.436477 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 845/1696] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:54:53.437505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d681883-7721-4414-8fda-6d3ae8cc4d8a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:53.438763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6d681883-7721-4414-8fda-6d3ae8cc4d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:53.454220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:53.469176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d681883-7721-4414-8fda-6d3ae8cc4d8a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:53.475815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:53.476733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:53.476982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:53.477248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] PATH_INFO: `/v3/users/1d816c885f6f4b7bb3f3208c3d8c7388` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:53.479089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d816c885f6f4b7bb3f3208c3d8c7388)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:53.482057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:53.487456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfc697-4ad0-444e-9867-a3e542d3bcd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:53.488860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfc697-4ad0-444e-9867-a3e542d3bcd8 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:53.489115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfc697-4ad0-444e-9867-a3e542d3bcd8 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:53.489724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfc697-4ad0-444e-9867-a3e542d3bcd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:53.492910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-25bfc697-4ad0-444e-9867-a3e542d3bcd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:53.588899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d816c885f6f4b7bb3f3208c3d8c7388'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:53.600273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfc697-4ad0-444e-9867-a3e542d3bcd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:53.601765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d816c885f6f4b7bb3f3208c3d8c7388'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:53.602182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfc697-4ad0-444e-9867-a3e542d3bcd8 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:53.622644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d816c885f6f4b7bb3f3208c3d8c7388'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:53.624099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d816c885f6f4b7bb3f3208c3d8c7388'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:53.624906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d816c885f6f4b7bb3f3208c3d8c7388'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:53.627916 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 852/1697] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:53.632521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfc697-4ad0-444e-9867-a3e542d3bcd8 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:53.638310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-25bfc697-4ad0-444e-9867-a3e542d3bcd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:53.643336 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 846/1698] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:54:53 2022] DELETE /identity/v3/users/1d816c885f6f4b7bb3f3208c3d8c7388 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:53.644671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:53.647939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:53.731558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:53.737456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac11f38-ee89-403d-a6f0-6d91f9a64726 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:53.744578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1ce27-450b-4382-b797-d154142e3da8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:53.745395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1ce27-450b-4382-b797-d154142e3da8 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:53.746791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1ce27-450b-4382-b797-d154142e3da8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:53.747343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1ce27-450b-4382-b797-d154142e3da8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:53.757778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-41e1ce27-450b-4382-b797-d154142e3da8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:53.794024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfc697-4ad0-444e-9867-a3e542d3bcd8 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:53.901471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1ce27-450b-4382-b797-d154142e3da8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:53.903995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1ce27-450b-4382-b797-d154142e3da8 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:53.915105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee216d1-0104-4c2d-adb7-0b1d3d14ef67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:53.915805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee216d1-0104-4c2d-adb7-0b1d3d14ef67 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:53.916083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee216d1-0104-4c2d-adb7-0b1d3d14ef67 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:53.916529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee216d1-0104-4c2d-adb7-0b1d3d14ef67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:53.919526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bee216d1-0104-4c2d-adb7-0b1d3d14ef67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:53.921471 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 847/1699] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:53.924075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439bcdc-3ca5-4e3e-8792-6350ea23d364 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:53.924624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439bcdc-3ca5-4e3e-8792-6350ea23d364 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:53.925387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439bcdc-3ca5-4e3e-8792-6350ea23d364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:53.950493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee216d1-0104-4c2d-adb7-0b1d3d14ef67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:53.952217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee216d1-0104-4c2d-adb7-0b1d3d14ef67 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:53.957805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f439bcdc-3ca5-4e3e-8792-6350ea23d364 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:53.960463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f439bcdc-3ca5-4e3e-8792-6350ea23d364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:53.970571 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 848/1700] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:53.972900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f87cd-0bb1-42d6-889b-1de0232fab2e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:53.973640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f87cd-0bb1-42d6-889b-1de0232fab2e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:53.977909 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 853/1701] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:53.982175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f87cd-0bb1-42d6-889b-1de0232fab2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:53.983518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee216d1-0104-4c2d-adb7-0b1d3d14ef67 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:53.986477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bee216d1-0104-4c2d-adb7-0b1d3d14ef67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:54.011427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee216d1-0104-4c2d-adb7-0b1d3d14ef67 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:54.019184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080d7de-ec9b-45c5-b546-32f103b64244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:54.019968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080d7de-ec9b-45c5-b546-32f103b64244 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:54.020943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080d7de-ec9b-45c5-b546-32f103b64244 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:54.021683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080d7de-ec9b-45c5-b546-32f103b64244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:54.055603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080d7de-ec9b-45c5-b546-32f103b64244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:54.057984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080d7de-ec9b-45c5-b546-32f103b64244 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:54.075815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0d6f87cd-0bb1-42d6-889b-1de0232fab2e None None] MFA Rules not processed for user `a3378bf4fa90489cb103378667f00399`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:54.079989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0d6f87cd-0bb1-42d6-889b-1de0232fab2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:54.106011 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 854/1702] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:54.108963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b89598-b31a-4f40-ade2-4544df3b1e25 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:54.110167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b89598-b31a-4f40-ade2-4544df3b1e25 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:54.110505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b89598-b31a-4f40-ade2-4544df3b1e25 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:54.114365 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 855/1703] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:54:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:54:54.133406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b89598-b31a-4f40-ade2-4544df3b1e25 None None] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:54:54.135661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f9b89598-b31a-4f40-ade2-4544df3b1e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:54.179667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:54.283924 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 849/1704] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:53 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:54.286701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74799fc8-3459-42a9-828a-1638115ce9b1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:54.287257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74799fc8-3459-42a9-828a-1638115ce9b1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:54.287430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74799fc8-3459-42a9-828a-1638115ce9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:54.302529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b89598-b31a-4f40-ade2-4544df3b1e25 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:54.303909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f9b89598-b31a-4f40-ade2-4544df3b1e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:54.312141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-74799fc8-3459-42a9-828a-1638115ce9b1 None None] MFA Rules not processed for user `3f85bb62037044ebae5a0a83194f9e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:54.313488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-74799fc8-3459-42a9-828a-1638115ce9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:54.433362 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 850/1705] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:54.433362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74799fc8-3459-42a9-828a-1638115ce9b1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:54.433362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-74799fc8-3459-42a9-828a-1638115ce9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:54.455494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b89598-b31a-4f40-ade2-4544df3b1e25 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:54.462483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74799fc8-3459-42a9-828a-1638115ce9b1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:54.464086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b273656-55cd-414b-aeb2-0fe2b08f6f44 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32ef91f38084fe09f80557ba543f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cc9b7e63d3b4b7fb38b1b1855a22778', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:54.464873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b273656-55cd-414b-aeb2-0fe2b08f6f44 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:54.465214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b273656-55cd-414b-aeb2-0fe2b08f6f44 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:54.465549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b273656-55cd-414b-aeb2-0fe2b08f6f44 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] PATH_INFO: `/v3/projects/2e2f79566b574520aabb42684e035f10` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:54.466857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b273656-55cd-414b-aeb2-0fe2b08f6f44 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] RBAC: Authorizing `identity:get_project(project_id=2e2f79566b574520aabb42684e035f10)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:54.468799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843136a8-6462-4a35-8d88-940b35e3ca00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:54.470411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843136a8-6462-4a35-8d88-940b35e3ca00 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:54.470785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b273656-55cd-414b-aeb2-0fe2b08f6f44 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:54.471208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843136a8-6462-4a35-8d88-940b35e3ca00 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:54.471757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843136a8-6462-4a35-8d88-940b35e3ca00 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:54.473459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843136a8-6462-4a35-8d88-940b35e3ca00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:54.477190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843136a8-6462-4a35-8d88-940b35e3ca00 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:54.487370 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 856/1706] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:54:54 2022] GET /identity/v3/projects/2e2f79566b574520aabb42684e035f10 => generated 377 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:54:54.490640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b273656-55cd-414b-aeb2-0fe2b08f6f44 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:54.494675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2b273656-55cd-414b-aeb2-0fe2b08f6f44 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:54.536596 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 851/1707] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:54 2022] POST /identity/v3/projects => generated 401 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:54.546414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843136a8-6462-4a35-8d88-940b35e3ca00 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:54.549940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:54.568691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b273656-55cd-414b-aeb2-0fe2b08f6f44 tempest-FlavorsAccessTestJSON-1063194532 tempest-FlavorsAccessTestJSON-1063194532-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:54.678973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843136a8-6462-4a35-8d88-940b35e3ca00 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:54.688246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f286f-e6ee-490f-9ec1-bf06c447f752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:54.707025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f286f-e6ee-490f-9ec1-bf06c447f752 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:54.709695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f286f-e6ee-490f-9ec1-bf06c447f752 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:54.710159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f286f-e6ee-490f-9ec1-bf06c447f752 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:54.711137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f286f-e6ee-490f-9ec1-bf06c447f752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:54.717655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f286f-e6ee-490f-9ec1-bf06c447f752 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:54.722002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e4739-11c2-4143-9359-68a34a388003 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:54.722980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e4739-11c2-4143-9359-68a34a388003 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:54.723273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e4739-11c2-4143-9359-68a34a388003 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:54.723541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e4739-11c2-4143-9359-68a34a388003 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:54.729635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0e4e4739-11c2-4143-9359-68a34a388003 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:54.764410 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 852/1708] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:54 2022] POST /identity/v3/users => generated 307 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:54.765726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f286f-e6ee-490f-9ec1-bf06c447f752 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:54.785858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f286f-e6ee-490f-9ec1-bf06c447f752 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:54.795889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fc2cd-d14e-4b08-89fd-743647b4c776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:54.796662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fc2cd-d14e-4b08-89fd-743647b4c776 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:54.796938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fc2cd-d14e-4b08-89fd-743647b4c776 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:54.797209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fc2cd-d14e-4b08-89fd-743647b4c776 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:54.797870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fc2cd-d14e-4b08-89fd-743647b4c776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:54.799967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fc2cd-d14e-4b08-89fd-743647b4c776 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:54.822395 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 853/1709] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:54:54.824623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fc2cd-d14e-4b08-89fd-743647b4c776 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:54.840013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-209fc2cd-d14e-4b08-89fd-743647b4c776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:54.882699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fc2cd-d14e-4b08-89fd-743647b4c776 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:54.893289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:54.894273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:54.894682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:54.895109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] PATH_INFO: `/v3/projects/8c05a02088d7460aa8b889c9fe6278bd` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:54.896509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c05a02088d7460aa8b889c9fe6278bd)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:54.898468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:54.902328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:54.919340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e4739-11c2-4143-9359-68a34a388003 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:54.923337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e4739-11c2-4143-9359-68a34a388003 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:54.935642 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 857/1710] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:54.936804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e4739-11c2-4143-9359-68a34a388003 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:54.952587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e4739-11c2-4143-9359-68a34a388003 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:54.958564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c9fde-66c9-4d9a-a982-9e76f7d39ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:54.959397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c9fde-66c9-4d9a-a982-9e76f7d39ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:54.959754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c9fde-66c9-4d9a-a982-9e76f7d39ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:54.960207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c9fde-66c9-4d9a-a982-9e76f7d39ec2 admin admin] PATH_INFO: `/v3/projects/ecc2ab17545442f68aa78857a0140871/users/112c5395a1b243e684a5ec3db9990e9a/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:54.980403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c9fde-66c9-4d9a-a982-9e76f7d39ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc2ab17545442f68aa78857a0140871, user_id=112c5395a1b243e684a5ec3db9990e9a, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:54.983624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c9fde-66c9-4d9a-a982-9e76f7d39ec2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:55.022077 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 858/1711] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:54 2022] PUT /identity/v3/projects/ecc2ab17545442f68aa78857a0140871/users/112c5395a1b243e684a5ec3db9990e9a/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:55.024200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c9fde-66c9-4d9a-a982-9e76f7d39ec2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:55.026806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9a4c9fde-66c9-4d9a-a982-9e76f7d39ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:55.034288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c05a02088d7460aa8b889c9fe6278bd', 'request_id': 'req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.053286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c05a02088d7460aa8b889c9fe6278bd', 'request_id': 'req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.076487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c05a02088d7460aa8b889c9fe6278bd', 'request_id': 'req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.104030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] The token cache is being invalidate because project 8c05a02088d7460aa8b889c9fe6278bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:54:55.104640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.105904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.109875 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 854/1712] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:54:54 2022] DELETE /identity/v3/projects/8c05a02088d7460aa8b889c9fe6278bd => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:55.110887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:55.115168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:55.133709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c9fde-66c9-4d9a-a982-9e76f7d39ec2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:55.145741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc768f-a4dc-4864-8a36-ab42572dd948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:55.146455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc768f-a4dc-4864-8a36-ab42572dd948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:55.146730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc768f-a4dc-4864-8a36-ab42572dd948 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:55.146993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc768f-a4dc-4864-8a36-ab42572dd948 admin admin] PATH_INFO: `/v3/projects/5c01dcbd100e4a418094b329a46fdc3d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:55.148716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc768f-a4dc-4864-8a36-ab42572dd948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c01dcbd100e4a418094b329a46fdc3d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:55.150470 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-0abc768f-a4dc-4864-8a36-ab42572dd948 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 20:54:55.150470 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 20:54:55.150470 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 20:54:55.152626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc768f-a4dc-4864-8a36-ab42572dd948 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:55.212501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfca47b-5faa-4e46-b4f3-e2dfdec66ac5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:55.225761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:55.232212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0abc768f-a4dc-4864-8a36-ab42572dd948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c01dcbd100e4a418094b329a46fdc3d', 'request_id': 'req-0abc768f-a4dc-4864-8a36-ab42572dd948'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.245395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0abc768f-a4dc-4864-8a36-ab42572dd948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c01dcbd100e4a418094b329a46fdc3d', 'request_id': 'req-0abc768f-a4dc-4864-8a36-ab42572dd948'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.256128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0abc768f-a4dc-4864-8a36-ab42572dd948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c01dcbd100e4a418094b329a46fdc3d', 'request_id': 'req-0abc768f-a4dc-4864-8a36-ab42572dd948'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.281307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0abc768f-a4dc-4864-8a36-ab42572dd948 admin admin] The token cache is being invalidate because project 5c01dcbd100e4a418094b329a46fdc3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:54:55.281601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0abc768f-a4dc-4864-8a36-ab42572dd948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.282622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0abc768f-a4dc-4864-8a36-ab42572dd948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.288511 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 859/1713] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:54:55 2022] DELETE /identity/v3/projects/5c01dcbd100e4a418094b329a46fdc3d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:55.290292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc793f4-c1fd-4d30-9a06-96b65adc24ab None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:55.290563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc793f4-c1fd-4d30-9a06-96b65adc24ab None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:55.290822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc793f4-c1fd-4d30-9a06-96b65adc24ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:55.298872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927b411-879c-4e35-83f3-0c3cdd8bd48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:55.299558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927b411-879c-4e35-83f3-0c3cdd8bd48b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:55.299955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927b411-879c-4e35-83f3-0c3cdd8bd48b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:55.300327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927b411-879c-4e35-83f3-0c3cdd8bd48b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:55.301357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927b411-879c-4e35-83f3-0c3cdd8bd48b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:55.303266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927b411-879c-4e35-83f3-0c3cdd8bd48b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:55.328120 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 855/1714] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:55.329805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927b411-879c-4e35-83f3-0c3cdd8bd48b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:55.333412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4927b411-879c-4e35-83f3-0c3cdd8bd48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:55.357779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927b411-879c-4e35-83f3-0c3cdd8bd48b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:55.363744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:55.364536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:55.364868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:55.365203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] PATH_INFO: `/v3/users/2733d0669fd24ed68b312925763fa212` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:55.366640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2733d0669fd24ed68b312925763fa212)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:55.368834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:55.369454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fbc793f4-c1fd-4d30-9a06-96b65adc24ab None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:55.370678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fbc793f4-c1fd-4d30-9a06-96b65adc24ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:55.410863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:55.443967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2733d0669fd24ed68b312925763fa212'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.454642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2733d0669fd24ed68b312925763fa212'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.470541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2733d0669fd24ed68b312925763fa212'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.471522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2733d0669fd24ed68b312925763fa212'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.472708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2733d0669fd24ed68b312925763fa212'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.486699 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 860/1715] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 20:54:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:55.487821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc793f4-c1fd-4d30-9a06-96b65adc24ab None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:55.489808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fbc793f4-c1fd-4d30-9a06-96b65adc24ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:55.496590 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 856/1716] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:54:55 2022] DELETE /identity/v3/users/2733d0669fd24ed68b312925763fa212 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:55.497508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:55.499214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:55.516059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc793f4-c1fd-4d30-9a06-96b65adc24ab None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:55.607288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80561af5-26f7-4bc0-a34d-8af29e0c32e7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:55.614981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ba7a6-e1c8-4627-9a3a-69bffee13e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:55.615678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ba7a6-e1c8-4627-9a3a-69bffee13e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:55.615951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ba7a6-e1c8-4627-9a3a-69bffee13e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:55.616758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ba7a6-e1c8-4627-9a3a-69bffee13e84 admin admin] PATH_INFO: `/v3/users/fa81cae2d2064780832aaef27161951b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:55.618148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ba7a6-e1c8-4627-9a3a-69bffee13e84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa81cae2d2064780832aaef27161951b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:55.619182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:55.620620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:55.620914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ba7a6-e1c8-4627-9a3a-69bffee13e84 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:55.621219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:55.621801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] PATH_INFO: `/v3/users/05ef6806617041188e892355fa4ce41a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:55.623657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05ef6806617041188e892355fa4ce41a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:55.626943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:55.651840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:55.693059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-672ba7a6-e1c8-4627-9a3a-69bffee13e84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa81cae2d2064780832aaef27161951b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.705344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-672ba7a6-e1c8-4627-9a3a-69bffee13e84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa81cae2d2064780832aaef27161951b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.718611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05ef6806617041188e892355fa4ce41a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.729445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05ef6806617041188e892355fa4ce41a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.743689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05ef6806617041188e892355fa4ce41a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.762435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-672ba7a6-e1c8-4627-9a3a-69bffee13e84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa81cae2d2064780832aaef27161951b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.764551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-672ba7a6-e1c8-4627-9a3a-69bffee13e84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa81cae2d2064780832aaef27161951b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.765542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-672ba7a6-e1c8-4627-9a3a-69bffee13e84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa81cae2d2064780832aaef27161951b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.776005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05ef6806617041188e892355fa4ce41a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.776816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05ef6806617041188e892355fa4ce41a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:55.790431 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 861/1718] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:54:55 2022] DELETE /identity/v3/users/05ef6806617041188e892355fa4ce41a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:55.793619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:55.794902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:55.806809 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 857/1718] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:54:55 2022] DELETE /identity/v3/users/fa81cae2d2064780832aaef27161951b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:55.813376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab137560-d3eb-4417-8da1-14803c6fa68c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:55.813965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab137560-d3eb-4417-8da1-14803c6fa68c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:55.816126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab137560-d3eb-4417-8da1-14803c6fa68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:55.871481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ab137560-d3eb-4417-8da1-14803c6fa68c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:55.877044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ab137560-d3eb-4417-8da1-14803c6fa68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:55.893926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:55.935263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895845c-2d99-45cc-9a20-bd2efc9f48c2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:55.945396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bde46f-6c43-4ef1-90a8-ddecd005bb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:55.947095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bde46f-6c43-4ef1-90a8-ddecd005bb7c admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:55.947756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bde46f-6c43-4ef1-90a8-ddecd005bb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:55.948344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bde46f-6c43-4ef1-90a8-ddecd005bb7c admin admin] PATH_INFO: `/v3/projects/ecc2ab17545442f68aa78857a0140871/users/112c5395a1b243e684a5ec3db9990e9a/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:55.952523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bde46f-6c43-4ef1-90a8-ddecd005bb7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc2ab17545442f68aa78857a0140871, user_id=112c5395a1b243e684a5ec3db9990e9a, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:55.960409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bde46f-6c43-4ef1-90a8-ddecd005bb7c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:55.980558 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 862/1719] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:55 2022] PUT /identity/v3/projects/ecc2ab17545442f68aa78857a0140871/users/112c5395a1b243e684a5ec3db9990e9a/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:55.981393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bde46f-6c43-4ef1-90a8-ddecd005bb7c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:55.984818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c6bde46f-6c43-4ef1-90a8-ddecd005bb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:56.027509 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 858/1720] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:56.056117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab137560-d3eb-4417-8da1-14803c6fa68c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:56.057882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ab137560-d3eb-4417-8da1-14803c6fa68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:56.180132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab137560-d3eb-4417-8da1-14803c6fa68c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:56.288697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bde46f-6c43-4ef1-90a8-ddecd005bb7c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:56.312101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659952a0-d172-425c-8cc8-0a9f6ba004c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:56.312852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659952a0-d172-425c-8cc8-0a9f6ba004c8 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:56.324902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659952a0-d172-425c-8cc8-0a9f6ba004c8 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:56.325633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659952a0-d172-425c-8cc8-0a9f6ba004c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:56.336224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-659952a0-d172-425c-8cc8-0a9f6ba004c8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:56.341343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a22c68-6588-45a8-bbba-a45a5dcd631e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:56.342428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a22c68-6588-45a8-bbba-a45a5dcd631e service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:56.344058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a22c68-6588-45a8-bbba-a45a5dcd631e service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:56.344405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a22c68-6588-45a8-bbba-a45a5dcd631e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:56.409744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:56.504426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659952a0-d172-425c-8cc8-0a9f6ba004c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:56.504426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659952a0-d172-425c-8cc8-0a9f6ba004c8 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:56.535963 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 863/1721] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 20:54:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 20:54:56.594783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659952a0-d172-425c-8cc8-0a9f6ba004c8 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:56.596412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-659952a0-d172-425c-8cc8-0a9f6ba004c8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:56.619342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a22c68-6588-45a8-bbba-a45a5dcd631e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:56.621768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a22c68-6588-45a8-bbba-a45a5dcd631e service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:56.663854 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 859/1722] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:56.766965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659952a0-d172-425c-8cc8-0a9f6ba004c8 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:56.778281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493580e-222d-49ac-9a43-18080d29b55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:56.781778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493580e-222d-49ac-9a43-18080d29b55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:56.783949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493580e-222d-49ac-9a43-18080d29b55c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:56.786111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493580e-222d-49ac-9a43-18080d29b55c admin admin] PATH_INFO: `/v3/projects/1fb1f1597c4f45acba73e980ef469fb4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:56.789562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493580e-222d-49ac-9a43-18080d29b55c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fb1f1597c4f45acba73e980ef469fb4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:56.796878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493580e-222d-49ac-9a43-18080d29b55c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:56.804082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:56.942972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a493580e-222d-49ac-9a43-18080d29b55c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fb1f1597c4f45acba73e980ef469fb4', 'request_id': 'req-a493580e-222d-49ac-9a43-18080d29b55c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:56.954693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a493580e-222d-49ac-9a43-18080d29b55c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fb1f1597c4f45acba73e980ef469fb4', 'request_id': 'req-a493580e-222d-49ac-9a43-18080d29b55c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:56.969234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a493580e-222d-49ac-9a43-18080d29b55c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fb1f1597c4f45acba73e980ef469fb4', 'request_id': 'req-a493580e-222d-49ac-9a43-18080d29b55c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:56.997938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a493580e-222d-49ac-9a43-18080d29b55c admin admin] The token cache is being invalidate because project 1fb1f1597c4f45acba73e980ef469fb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:54:56.998302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a493580e-222d-49ac-9a43-18080d29b55c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:56.999789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a493580e-222d-49ac-9a43-18080d29b55c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:57.002346 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 864/1723] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:54:56 2022] DELETE /identity/v3/projects/1fb1f1597c4f45acba73e980ef469fb4 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:57.008938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb9907-253f-4fa1-9cd7-dfe70fbd745f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:57.009510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb9907-253f-4fa1-9cd7-dfe70fbd745f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:57.009844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb9907-253f-4fa1-9cd7-dfe70fbd745f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:57.058387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-dbfb9907-253f-4fa1-9cd7-dfe70fbd745f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:57.058387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dbfb9907-253f-4fa1-9cd7-dfe70fbd745f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:57.200241 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 860/1724] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:57.209614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dacc4c-dccc-4264-b1dd-c32dec2e73f0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:57.209892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dacc4c-dccc-4264-b1dd-c32dec2e73f0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:57.210351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dacc4c-dccc-4264-b1dd-c32dec2e73f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:57.266221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-79dacc4c-dccc-4264-b1dd-c32dec2e73f0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:57.268136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-79dacc4c-dccc-4264-b1dd-c32dec2e73f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:57.308131 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 865/1725] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:57.311207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb9907-253f-4fa1-9cd7-dfe70fbd745f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:57.328350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb9907-253f-4fa1-9cd7-dfe70fbd745f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:57.334114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb910e37-03a5-4cec-98bc-d2460db1d7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:57.335869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb910e37-03a5-4cec-98bc-d2460db1d7ea admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:57.336177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb910e37-03a5-4cec-98bc-d2460db1d7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:57.338042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb910e37-03a5-4cec-98bc-d2460db1d7ea admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:57.342568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb910e37-03a5-4cec-98bc-d2460db1d7ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:54:57.342869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb910e37-03a5-4cec-98bc-d2460db1d7ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:57.344640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb910e37-03a5-4cec-98bc-d2460db1d7ea admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:57.375775 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 861/1726] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:54:57.381853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dacc4c-dccc-4264-b1dd-c32dec2e73f0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:57.393776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dacc4c-dccc-4264-b1dd-c32dec2e73f0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:57.411151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1698975-d8d2-40ad-9a9a-cb17ec6a7bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:57.411695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1698975-d8d2-40ad-9a9a-cb17ec6a7bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:57.412019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1698975-d8d2-40ad-9a9a-cb17ec6a7bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:57.412119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1698975-d8d2-40ad-9a9a-cb17ec6a7bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:57.417205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1698975-d8d2-40ad-9a9a-cb17ec6a7bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:57.437830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1698975-d8d2-40ad-9a9a-cb17ec6a7bd6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:57.499705 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 866/1727] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:57 2022] POST /identity/v3/projects => generated 409 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:57.504869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb910e37-03a5-4cec-98bc-d2460db1d7ea admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:57.519693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb910e37-03a5-4cec-98bc-d2460db1d7ea admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:57.662196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da780b-a44c-4cde-b77b-84240a9f597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:57.663118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da780b-a44c-4cde-b77b-84240a9f597a admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:57.663392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da780b-a44c-4cde-b77b-84240a9f597a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:57.663662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da780b-a44c-4cde-b77b-84240a9f597a admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:57.664329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da780b-a44c-4cde-b77b-84240a9f597a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:57.668524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da780b-a44c-4cde-b77b-84240a9f597a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:57.716336 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 862/1728] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:57 2022] POST /identity/v3/users => generated 305 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:57.720642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1698975-d8d2-40ad-9a9a-cb17ec6a7bd6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:57.757330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1698975-d8d2-40ad-9a9a-cb17ec6a7bd6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:57.769114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495ae8f-542a-4161-8785-1ddb12811595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:57.770805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495ae8f-542a-4161-8785-1ddb12811595 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:57.772255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495ae8f-542a-4161-8785-1ddb12811595 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:57.772767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495ae8f-542a-4161-8785-1ddb12811595 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:57.773687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495ae8f-542a-4161-8785-1ddb12811595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:57.776032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495ae8f-542a-4161-8785-1ddb12811595 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:57.801584 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 867/1729] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:57.806795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da780b-a44c-4cde-b77b-84240a9f597a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:57.847857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da780b-a44c-4cde-b77b-84240a9f597a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:57.848084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495ae8f-542a-4161-8785-1ddb12811595 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:57.851666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f495ae8f-542a-4161-8785-1ddb12811595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:57.876408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc246c-f93b-4376-9811-383f8ceac2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:57.876408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc246c-f93b-4376-9811-383f8ceac2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:57.876408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc246c-f93b-4376-9811-383f8ceac2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:57.876408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc246c-f93b-4376-9811-383f8ceac2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:57.876735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc246c-f93b-4376-9811-383f8ceac2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:57.878493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc246c-f93b-4376-9811-383f8ceac2d8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:57.891431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495ae8f-542a-4161-8785-1ddb12811595 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:57.908426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:57.910896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:57.913246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:57.913555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] PATH_INFO: `/v3/projects/2e2f79566b574520aabb42684e035f10` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:57.921415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2f79566b574520aabb42684e035f10)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:57.928302 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 863/1730] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:54:57.929482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc246c-f93b-4376-9811-383f8ceac2d8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:57.930791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:57.960449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc246c-f93b-4376-9811-383f8ceac2d8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:57.971610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911802e-6dc9-4cee-9e9e-c7c09a70336a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:57.971610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911802e-6dc9-4cee-9e9e-c7c09a70336a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:57.971969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911802e-6dc9-4cee-9e9e-c7c09a70336a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:57.972201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911802e-6dc9-4cee-9e9e-c7c09a70336a admin admin] PATH_INFO: `/v3/projects/5fd09d159bdc4888b9c1b260af72cdee/users/1a4fbbe792484d33a2774eb66ee42f08/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:58.007419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:58.017452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911802e-6dc9-4cee-9e9e-c7c09a70336a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fd09d159bdc4888b9c1b260af72cdee, user_id=1a4fbbe792484d33a2774eb66ee42f08, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:58.024040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911802e-6dc9-4cee-9e9e-c7c09a70336a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:58.067644 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 864/1731] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:54:57 2022] PUT /identity/v3/projects/5fd09d159bdc4888b9c1b260af72cdee/users/1a4fbbe792484d33a2774eb66ee42f08/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:58.068556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2f79566b574520aabb42684e035f10', 'request_id': 'req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:58.079595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261692a7-897e-470a-b46c-94f3a4f23535 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:58.079884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261692a7-897e-470a-b46c-94f3a4f23535 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:58.080149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261692a7-897e-470a-b46c-94f3a4f23535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:58.093905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2f79566b574520aabb42684e035f10', 'request_id': 'req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:58.110185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2f79566b574520aabb42684e035f10', 'request_id': 'req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:58.131096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] The token cache is being invalidate because project 2e2f79566b574520aabb42684e035f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:54:58.131410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:58.135403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:58.139373 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 868/1732] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:54:57 2022] DELETE /identity/v3/projects/2e2f79566b574520aabb42684e035f10 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:58.145367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:58.148138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:58.150918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-261692a7-897e-470a-b46c-94f3a4f23535 None None] MFA Rules not processed for user `1a4fbbe792484d33a2774eb66ee42f08`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:58.165701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-261692a7-897e-470a-b46c-94f3a4f23535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:58.282651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:58.323459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a6f4e-d0e3-4ac5-9b7b-122d28e0f8d3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:58.335196 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 865/1733] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:58 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:54:58.336198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261692a7-897e-470a-b46c-94f3a4f23535 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:58.343384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-261692a7-897e-470a-b46c-94f3a4f23535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:58.482507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261692a7-897e-470a-b46c-94f3a4f23535 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:58.498713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cfe22-1d13-497d-9f9f-e4e0d5f3865a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:58.499842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cfe22-1d13-497d-9f9f-e4e0d5f3865a admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:58.500150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cfe22-1d13-497d-9f9f-e4e0d5f3865a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:58.502204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cfe22-1d13-497d-9f9f-e4e0d5f3865a admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:58.503092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cfe22-1d13-497d-9f9f-e4e0d5f3865a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:58.507799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cfe22-1d13-497d-9f9f-e4e0d5f3865a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:58.617550 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 866/1734] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:54:58 2022] POST /identity/v3/projects => generated 407 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:58.638276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cfe22-1d13-497d-9f9f-e4e0d5f3865a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:58.643994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d16cfe22-1d13-497d-9f9f-e4e0d5f3865a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:58.729185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689ac24-d214-4219-9017-b72e8e0cd91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:58.732906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689ac24-d214-4219-9017-b72e8e0cd91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:58.733404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689ac24-d214-4219-9017-b72e8e0cd91b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:58.733885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689ac24-d214-4219-9017-b72e8e0cd91b admin admin] PATH_INFO: `/v3/users/d32ef91f38084fe09f80557ba543f7c4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:58.740681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689ac24-d214-4219-9017-b72e8e0cd91b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d32ef91f38084fe09f80557ba543f7c4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:58.770437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689ac24-d214-4219-9017-b72e8e0cd91b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:58.897527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cfe22-1d13-497d-9f9f-e4e0d5f3865a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:58.914350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:58.916989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:58.917278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:58.917571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] PATH_INFO: `/v3/projects/3e9bcfa245224652a766f2310c25e7d2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:58.921985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e9bcfa245224652a766f2310c25e7d2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:58.928909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:58.953090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:58.965811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9689ac24-d214-4219-9017-b72e8e0cd91b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd32ef91f38084fe09f80557ba543f7c4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:58.990180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9689ac24-d214-4219-9017-b72e8e0cd91b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd32ef91f38084fe09f80557ba543f7c4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:59.011274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9689ac24-d214-4219-9017-b72e8e0cd91b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd32ef91f38084fe09f80557ba543f7c4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:59.063210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9689ac24-d214-4219-9017-b72e8e0cd91b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd32ef91f38084fe09f80557ba543f7c4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:59.063852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9689ac24-d214-4219-9017-b72e8e0cd91b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd32ef91f38084fe09f80557ba543f7c4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:59.068681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e9bcfa245224652a766f2310c25e7d2', 'request_id': 'req-c1c41984-acdb-4949-8847-5ba7bf0eac98'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:59.074606 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 869/1735] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:54:58 2022] DELETE /identity/v3/users/d32ef91f38084fe09f80557ba543f7c4 => generated 0 bytes in 928 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:59.074606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46335b-128b-4519-891e-afb8e7c9b9c4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:59.074606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46335b-128b-4519-891e-afb8e7c9b9c4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:59.074606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46335b-128b-4519-891e-afb8e7c9b9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:59.099225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e9bcfa245224652a766f2310c25e7d2', 'request_id': 'req-c1c41984-acdb-4949-8847-5ba7bf0eac98'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:59.117105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e9bcfa245224652a766f2310c25e7d2', 'request_id': 'req-c1c41984-acdb-4949-8847-5ba7bf0eac98'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:59.146591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] The token cache is being invalidate because project 3e9bcfa245224652a766f2310c25e7d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:54:59.147260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:59.149581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:54:59.153552 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 867/1736] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:54:58 2022] DELETE /identity/v3/projects/3e9bcfa245224652a766f2310c25e7d2 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:54:59.155584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:59.166139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:59.203182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9f46335b-128b-4519-891e-afb8e7c9b9c4 None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:59.204547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9f46335b-128b-4519-891e-afb8e7c9b9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:59.484178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c41984-acdb-4949-8847-5ba7bf0eac98 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:59.521500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:54:59.543944 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 870/1737] 10.222.0.16 () {58 vars in 1015 bytes} [Sat Aug 20 20:54:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:59.548857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b9bf0-ee48-4cff-8246-a5273f3c5497 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:59.549161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b9bf0-ee48-4cff-8246-a5273f3c5497 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:59.549398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b9bf0-ee48-4cff-8246-a5273f3c5497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:59.665993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-524b9bf0-ee48-4cff-8246-a5273f3c5497 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:59.669059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-524b9bf0-ee48-4cff-8246-a5273f3c5497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:59.785559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e9fc4-50a9-4a3e-aa06-4b00282057b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:59.786234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e9fc4-50a9-4a3e-aa06-4b00282057b3 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:59.786507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e9fc4-50a9-4a3e-aa06-4b00282057b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:59.786770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e9fc4-50a9-4a3e-aa06-4b00282057b3 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:59.787408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e9fc4-50a9-4a3e-aa06-4b00282057b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:59.802176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e9fc4-50a9-4a3e-aa06-4b00282057b3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:59.850289 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 871/1738] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:59.850289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b9bf0-ee48-4cff-8246-a5273f3c5497 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:59.852440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-524b9bf0-ee48-4cff-8246-a5273f3c5497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:59.879804 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 868/1739] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:54:59 2022] POST /identity/v3/users => generated 310 bytes in 726 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:54:59.881363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4c34d-e5bb-4c75-9e40-47f9314e97fe None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:59.881648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4c34d-e5bb-4c75-9e40-47f9314e97fe None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:59.881906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4c34d-e5bb-4c75-9e40-47f9314e97fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:59.899786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b9bf0-ee48-4cff-8246-a5273f3c5497 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:54:59.917915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a110589-b432-4f5c-9798-31c3fe3d9f21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:54:59.918540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a110589-b432-4f5c-9798-31c3fe3d9f21 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:59.918826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a110589-b432-4f5c-9798-31c3fe3d9f21 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:59.919116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a110589-b432-4f5c-9798-31c3fe3d9f21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:54:59.935066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a4a4c34d-e5bb-4c75-9e40-47f9314e97fe None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:54:59.936413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a4a4c34d-e5bb-4c75-9e40-47f9314e97fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:54:59.947094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a110589-b432-4f5c-9798-31c3fe3d9f21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:54:59.948803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a110589-b432-4f5c-9798-31c3fe3d9f21 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:54:59.975571 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 869/1740] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:54:59.975571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4c34d-e5bb-4c75-9e40-47f9314e97fe None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:54:59.975859 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 872/1741] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:54:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:54:59.975859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799afb5c-1906-410b-a9ce-c54d80a06f8c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:54:59.975998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799afb5c-1906-410b-a9ce-c54d80a06f8c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:54:59.976269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799afb5c-1906-410b-a9ce-c54d80a06f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:00.010921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4c34d-e5bb-4c75-9e40-47f9314e97fe None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:00.030810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009f7ae-4606-4d8c-abf6-be2725403799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:00.031436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009f7ae-4606-4d8c-abf6-be2725403799 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:00.032072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009f7ae-4606-4d8c-abf6-be2725403799 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:00.032072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009f7ae-4606-4d8c-abf6-be2725403799 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:00.032862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009f7ae-4606-4d8c-abf6-be2725403799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:00.034428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009f7ae-4606-4d8c-abf6-be2725403799 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:00.043921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-799afb5c-1906-410b-a9ce-c54d80a06f8c None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:00.046030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-799afb5c-1906-410b-a9ce-c54d80a06f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:00.076231 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 870/1742] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:00.078950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009f7ae-4606-4d8c-abf6-be2725403799 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:00.114966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009f7ae-4606-4d8c-abf6-be2725403799 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:00.157262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05488ece-b562-40d7-9b09-12c3d60abcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:00.158027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05488ece-b562-40d7-9b09-12c3d60abcf6 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:00.158326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05488ece-b562-40d7-9b09-12c3d60abcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:00.158823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05488ece-b562-40d7-9b09-12c3d60abcf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:00.159325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05488ece-b562-40d7-9b09-12c3d60abcf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:55:00.159844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05488ece-b562-40d7-9b09-12c3d60abcf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:00.165893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05488ece-b562-40d7-9b09-12c3d60abcf6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:00.187862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:00.222081 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 871/1743] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:55:00.222081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05488ece-b562-40d7-9b09-12c3d60abcf6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:00.242014 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 873/1744] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 20:54:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:00.242907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799afb5c-1906-410b-a9ce-c54d80a06f8c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:00.245113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05488ece-b562-40d7-9b09-12c3d60abcf6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:00.259793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d03661-b3b4-47e4-97bf-91688b66056b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:00.260521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d03661-b3b4-47e4-97bf-91688b66056b admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:00.260828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d03661-b3b4-47e4-97bf-91688b66056b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:00.261075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d03661-b3b4-47e4-97bf-91688b66056b admin admin] PATH_INFO: `/v3/projects/9168c4b22fa045a3808cd7352573b0cf/users/f442d104723f49f0883deb94bc992fd8/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:00.285754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799afb5c-1906-410b-a9ce-c54d80a06f8c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:00.300016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d03661-b3b4-47e4-97bf-91688b66056b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9168c4b22fa045a3808cd7352573b0cf, user_id=f442d104723f49f0883deb94bc992fd8, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:00.307934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30dfb83-1f0b-4f82-a2a9-0d7b93fbb950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:00.310696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d03661-b3b4-47e4-97bf-91688b66056b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:00.310837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dfb83-1f0b-4f82-a2a9-0d7b93fbb950 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:00.310837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dfb83-1f0b-4f82-a2a9-0d7b93fbb950 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:00.311088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dfb83-1f0b-4f82-a2a9-0d7b93fbb950 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:00.311716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30dfb83-1f0b-4f82-a2a9-0d7b93fbb950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:00.334953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30dfb83-1f0b-4f82-a2a9-0d7b93fbb950 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:00.343926 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 872/1745] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:00 2022] PUT /identity/v3/projects/9168c4b22fa045a3808cd7352573b0cf/users/f442d104723f49f0883deb94bc992fd8/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:00.344995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d03661-b3b4-47e4-97bf-91688b66056b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:00.348800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c9d03661-b3b4-47e4-97bf-91688b66056b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:00.433141 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 874/1746] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:00 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:00.433141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30dfb83-1f0b-4f82-a2a9-0d7b93fbb950 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:00.606509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d03661-b3b4-47e4-97bf-91688b66056b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:00.616333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30dfb83-1f0b-4f82-a2a9-0d7b93fbb950 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:00.646461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce500eee-3198-432a-9832-437524d84922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:00.647104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce500eee-3198-432a-9832-437524d84922 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:00.647375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce500eee-3198-432a-9832-437524d84922 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:00.647641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce500eee-3198-432a-9832-437524d84922 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:00.648450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce500eee-3198-432a-9832-437524d84922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:00.649941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce500eee-3198-432a-9832-437524d84922 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:00.700900 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 875/1747] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:00.704118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce500eee-3198-432a-9832-437524d84922 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:00.708235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ce500eee-3198-432a-9832-437524d84922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:00.736228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:00.756190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce500eee-3198-432a-9832-437524d84922 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:00.757155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236e0f0-1b55-4463-85af-793ef9228ffc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:00.758057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236e0f0-1b55-4463-85af-793ef9228ffc service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:00.758457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236e0f0-1b55-4463-85af-793ef9228ffc service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:00.758731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236e0f0-1b55-4463-85af-793ef9228ffc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:00.777092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:00.777748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:00.778018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:00.778302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] PATH_INFO: `/v3/projects/1cc9b7e63d3b4b7fb38b1b1855a22778` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:00.780609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc9b7e63d3b4b7fb38b1b1855a22778)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:00.786562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:00.952719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236e0f0-1b55-4463-85af-793ef9228ffc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:00.975236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236e0f0-1b55-4463-85af-793ef9228ffc service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:00.995787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc9b7e63d3b4b7fb38b1b1855a22778', 'request_id': 'req-bc60b617-8d5c-47fa-9b61-eb5044b80e02'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:01.003572 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 873/1748] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 20:55:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 20:55:01.006618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236e0f0-1b55-4463-85af-793ef9228ffc service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:01.014878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc9b7e63d3b4b7fb38b1b1855a22778', 'request_id': 'req-bc60b617-8d5c-47fa-9b61-eb5044b80e02'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:01.033869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc9b7e63d3b4b7fb38b1b1855a22778', 'request_id': 'req-bc60b617-8d5c-47fa-9b61-eb5044b80e02'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:01.037563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236e0f0-1b55-4463-85af-793ef9228ffc service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:01.096108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] The token cache is being invalidate because project 1cc9b7e63d3b4b7fb38b1b1855a22778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:01.096901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:01.099782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:01.117014 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 876/1749] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:00 2022] DELETE /identity/v3/projects/1cc9b7e63d3b4b7fb38b1b1855a22778 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:01.117014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:01.138796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:01.159753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:01.392274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816df91b-6e94-44d1-89af-8350fa44503e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:01.393039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816df91b-6e94-44d1-89af-8350fa44503e admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:01.393321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816df91b-6e94-44d1-89af-8350fa44503e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:01.393617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816df91b-6e94-44d1-89af-8350fa44503e admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:01.394282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816df91b-6e94-44d1-89af-8350fa44503e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:01.404565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816df91b-6e94-44d1-89af-8350fa44503e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:01.495963 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 874/1750] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:01 2022] POST /identity/v3/users => generated 293 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:01.515544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96070441-6613-4a10-a0a6-d2349afbc456 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:01.516387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96070441-6613-4a10-a0a6-d2349afbc456 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:01.518187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96070441-6613-4a10-a0a6-d2349afbc456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:01.567920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60b617-8d5c-47fa-9b61-eb5044b80e02 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:01.648035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca610c84-e21c-46a3-bf42-fdd56312a774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:01.650977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca610c84-e21c-46a3-bf42-fdd56312a774 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:01.651259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca610c84-e21c-46a3-bf42-fdd56312a774 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:01.659753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca610c84-e21c-46a3-bf42-fdd56312a774 admin admin] PATH_INFO: `/v3/projects/9168c4b22fa045a3808cd7352573b0cf/users/f442d104723f49f0883deb94bc992fd8/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:01.667513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca610c84-e21c-46a3-bf42-fdd56312a774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9168c4b22fa045a3808cd7352573b0cf, user_id=f442d104723f49f0883deb94bc992fd8, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:01.681701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-96070441-6613-4a10-a0a6-d2349afbc456 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:01.684709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-96070441-6613-4a10-a0a6-d2349afbc456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:01.690315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca610c84-e21c-46a3-bf42-fdd56312a774 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:01.757668 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 877/1751] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:01 2022] PUT /identity/v3/projects/9168c4b22fa045a3808cd7352573b0cf/users/f442d104723f49f0883deb94bc992fd8/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:01.757668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca610c84-e21c-46a3-bf42-fdd56312a774 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:01.757668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ca610c84-e21c-46a3-bf42-fdd56312a774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:01.766133 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 875/1752] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:01.769874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986136c-5669-4c42-b90e-1f53af82b228 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:01.773999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986136c-5669-4c42-b90e-1f53af82b228 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:01.774563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986136c-5669-4c42-b90e-1f53af82b228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:01.853462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a986136c-5669-4c42-b90e-1f53af82b228 None None] MFA Rules not processed for user `f442d104723f49f0883deb94bc992fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:01.856237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a986136c-5669-4c42-b90e-1f53af82b228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:02.034859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca610c84-e21c-46a3-bf42-fdd56312a774 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:02.040585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c86b7-6cbc-4ef9-af0c-61e3d61bb332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:02.041595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c86b7-6cbc-4ef9-af0c-61e3d61bb332 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:02.042125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c86b7-6cbc-4ef9-af0c-61e3d61bb332 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:02.042846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c86b7-6cbc-4ef9-af0c-61e3d61bb332 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:02.044028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c86b7-6cbc-4ef9-af0c-61e3d61bb332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:02.046289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c86b7-6cbc-4ef9-af0c-61e3d61bb332 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:02.080702 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 878/1753] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:02.082632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97731e-4058-4165-a6b6-abdb251b46af None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:02.083102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97731e-4058-4165-a6b6-abdb251b46af None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:02.083258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97731e-4058-4165-a6b6-abdb251b46af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:02.133421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-aa97731e-4058-4165-a6b6-abdb251b46af None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:02.135289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aa97731e-4058-4165-a6b6-abdb251b46af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:02.148822 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 879/1754] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:02.150209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97731e-4058-4165-a6b6-abdb251b46af None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:02.170058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97731e-4058-4165-a6b6-abdb251b46af None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:02.179275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e8b00-59ec-4e87-abe3-78eb34696458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:02.181535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e8b00-59ec-4e87-abe3-78eb34696458 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:02.182233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e8b00-59ec-4e87-abe3-78eb34696458 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:02.183220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e8b00-59ec-4e87-abe3-78eb34696458 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:02.186529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e8b00-59ec-4e87-abe3-78eb34696458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:02.192873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e8b00-59ec-4e87-abe3-78eb34696458 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:02.204067 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 876/1755] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:01 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:02.205656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986136c-5669-4c42-b90e-1f53af82b228 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:02.207097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:02.220123 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 880/1756] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:02.224875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e8b00-59ec-4e87-abe3-78eb34696458 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:02.226986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a67e8b00-59ec-4e87-abe3-78eb34696458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:02.242570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986136c-5669-4c42-b90e-1f53af82b228 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:02.263015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a31ba-8e22-42cd-8a95-57da41c07672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:02.263738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a31ba-8e22-42cd-8a95-57da41c07672 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:02.264011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a31ba-8e22-42cd-8a95-57da41c07672 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:02.264451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a31ba-8e22-42cd-8a95-57da41c07672 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:02.265025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a31ba-8e22-42cd-8a95-57da41c07672 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:55:02.274349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a31ba-8e22-42cd-8a95-57da41c07672 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:02.292446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a31ba-8e22-42cd-8a95-57da41c07672 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:02.340188 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 877/1757] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:55:02.340188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a31ba-8e22-42cd-8a95-57da41c07672 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:02.359559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a31ba-8e22-42cd-8a95-57da41c07672 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:02.366525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189801c-557a-453d-b84f-f95883aec2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:02.367213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189801c-557a-453d-b84f-f95883aec2d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:02.367479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189801c-557a-453d-b84f-f95883aec2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:02.367839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189801c-557a-453d-b84f-f95883aec2d4 admin admin] PATH_INFO: `/v3/projects/1e753a6065044fcd81165523768fe2f4/users/c3adc70e73614d48be23e3542424ee1d/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:02.404324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189801c-557a-453d-b84f-f95883aec2d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e753a6065044fcd81165523768fe2f4, user_id=c3adc70e73614d48be23e3542424ee1d, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:02.410399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189801c-557a-453d-b84f-f95883aec2d4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:02.447366 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 878/1758] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:02 2022] PUT /identity/v3/projects/1e753a6065044fcd81165523768fe2f4/users/c3adc70e73614d48be23e3542424ee1d/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:02.448099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189801c-557a-453d-b84f-f95883aec2d4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:02.556205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e8b00-59ec-4e87-abe3-78eb34696458 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:02.674985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:02.716735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189801c-557a-453d-b84f-f95883aec2d4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:02.728081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7ac60-cd24-459a-95da-c8869c3f2c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:02.733388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7ac60-cd24-459a-95da-c8869c3f2c59 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:02.734153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7ac60-cd24-459a-95da-c8869c3f2c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:02.734654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7ac60-cd24-459a-95da-c8869c3f2c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:02.736856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7ac60-cd24-459a-95da-c8869c3f2c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:02.741332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7ac60-cd24-459a-95da-c8869c3f2c59 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:02.830014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55ece7-b7bc-4e1e-ad62-abed617f0fdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:02.833460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55ece7-b7bc-4e1e-ad62-abed617f0fdc service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:02.834026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55ece7-b7bc-4e1e-ad62-abed617f0fdc service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:02.834901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55ece7-b7bc-4e1e-ad62-abed617f0fdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:02.837482 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 879/1759] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:02 2022] POST /identity/v3/projects => generated 403 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:02.841924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7adb2-36d2-434a-933a-3fa351ac50e8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:02.842233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7adb2-36d2-434a-933a-3fa351ac50e8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:02.842822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7adb2-36d2-434a-933a-3fa351ac50e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:02.935852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b0b7adb2-36d2-434a-933a-3fa351ac50e8 None None] MFA Rules not processed for user `c3adc70e73614d48be23e3542424ee1d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:02.941328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b0b7adb2-36d2-434a-933a-3fa351ac50e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:03.027252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55ece7-b7bc-4e1e-ad62-abed617f0fdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:03.029222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55ece7-b7bc-4e1e-ad62-abed617f0fdc service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:03.052437 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 881/1760] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:02 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 831 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:03.055077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55ece7-b7bc-4e1e-ad62-abed617f0fdc service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:03.083800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55ece7-b7bc-4e1e-ad62-abed617f0fdc service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:03.183928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:03.306170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78f528-5799-4170-bd37-68e07eb29007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:03.307414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78f528-5799-4170-bd37-68e07eb29007 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:03.308680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78f528-5799-4170-bd37-68e07eb29007 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:03.309247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78f528-5799-4170-bd37-68e07eb29007 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:03.310523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78f528-5799-4170-bd37-68e07eb29007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:03.312874 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 880/1761] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:03.313691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78f528-5799-4170-bd37-68e07eb29007 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:03.318800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7adb2-36d2-434a-933a-3fa351ac50e8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:03.348143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7adb2-36d2-434a-933a-3fa351ac50e8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:03.350930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b253cb-8342-4a1d-96e8-a75e75dff619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:03.352073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b253cb-8342-4a1d-96e8-a75e75dff619 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:03.352527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b253cb-8342-4a1d-96e8-a75e75dff619 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:03.352932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b253cb-8342-4a1d-96e8-a75e75dff619 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:03.353758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b253cb-8342-4a1d-96e8-a75e75dff619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:03.366106 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 882/1762] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:03 2022] POST /identity/v3/users => generated 302 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:03.370489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78f528-5799-4170-bd37-68e07eb29007 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:03.371338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b253cb-8342-4a1d-96e8-a75e75dff619 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:03.406781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78f528-5799-4170-bd37-68e07eb29007 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:03.425882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8087d6-f12a-46bb-ad43-ed6a42fd668c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:03.430152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8087d6-f12a-46bb-ad43-ed6a42fd668c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:03.430152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8087d6-f12a-46bb-ad43-ed6a42fd668c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:03.430152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8087d6-f12a-46bb-ad43-ed6a42fd668c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:03.433167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8087d6-f12a-46bb-ad43-ed6a42fd668c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:03.441042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8087d6-f12a-46bb-ad43-ed6a42fd668c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:03.474294 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 881/1763] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:03 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:03.475525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b253cb-8342-4a1d-96e8-a75e75dff619 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:03.487160 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 883/1764] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:55:03.488835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8087d6-f12a-46bb-ad43-ed6a42fd668c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:03.532051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8087d6-f12a-46bb-ad43-ed6a42fd668c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:03.667900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b253cb-8342-4a1d-96e8-a75e75dff619 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:03.685470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470b62-2dde-4d15-9cde-2a49716317fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:03.687081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470b62-2dde-4d15-9cde-2a49716317fb admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:03.687421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470b62-2dde-4d15-9cde-2a49716317fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:03.691751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470b62-2dde-4d15-9cde-2a49716317fb admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:03.696863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb470b62-2dde-4d15-9cde-2a49716317fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:03.705306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb470b62-2dde-4d15-9cde-2a49716317fb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:03.707734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6d530-4689-48b7-866f-ec6059324caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:03.717960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6d530-4689-48b7-866f-ec6059324caf admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:03.718265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6d530-4689-48b7-866f-ec6059324caf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:03.718529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6d530-4689-48b7-866f-ec6059324caf admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:03.721357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6d530-4689-48b7-866f-ec6059324caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:03.729634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6d530-4689-48b7-866f-ec6059324caf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:03.759810 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 884/1765] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:03.766745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6d530-4689-48b7-866f-ec6059324caf admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:03.775851 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 882/1766] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:03 2022] POST /identity/v3/users => generated 300 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:03.779952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470b62-2dde-4d15-9cde-2a49716317fb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:03.796556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6d530-4689-48b7-866f-ec6059324caf admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:03.813061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470b62-2dde-4d15-9cde-2a49716317fb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:03.813594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41065c-58ee-4938-8a6d-c8110ab75dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:03.815892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41065c-58ee-4938-8a6d-c8110ab75dab admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:03.816632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41065c-58ee-4938-8a6d-c8110ab75dab admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:03.817594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41065c-58ee-4938-8a6d-c8110ab75dab admin admin] PATH_INFO: `/v3/projects/7ac138e2ccba4251851f97d850689015/users/aef4fc482ab1475fa24637f6991ae876/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:03.826446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b756b8-aa71-47af-843e-a9f06e7fdf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:03.827767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b756b8-aa71-47af-843e-a9f06e7fdf6c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:03.828065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b756b8-aa71-47af-843e-a9f06e7fdf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:03.828556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b756b8-aa71-47af-843e-a9f06e7fdf6c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:03.832052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b756b8-aa71-47af-843e-a9f06e7fdf6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:03.836401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b756b8-aa71-47af-843e-a9f06e7fdf6c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:03.866638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41065c-58ee-4938-8a6d-c8110ab75dab admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac138e2ccba4251851f97d850689015, user_id=aef4fc482ab1475fa24637f6991ae876, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:03.869357 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 883/1767] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:03.871474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41065c-58ee-4938-8a6d-c8110ab75dab admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:03.874285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b756b8-aa71-47af-843e-a9f06e7fdf6c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:03.894154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b756b8-aa71-47af-843e-a9f06e7fdf6c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:03.920131 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 885/1768] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:03 2022] PUT /identity/v3/projects/7ac138e2ccba4251851f97d850689015/users/aef4fc482ab1475fa24637f6991ae876/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:03.926676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c07c4-eb39-4e0e-9e80-bac1cb3a3255 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:03.927367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c07c4-eb39-4e0e-9e80-bac1cb3a3255 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:03.927857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c07c4-eb39-4e0e-9e80-bac1cb3a3255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:03.937742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:03.980935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e37c07c4-eb39-4e0e-9e80-bac1cb3a3255 None None] MFA Rules not processed for user `aef4fc482ab1475fa24637f6991ae876`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:03.982685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e37c07c4-eb39-4e0e-9e80-bac1cb3a3255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:04.054121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7990780-dc95-4ceb-984f-d962feb5773e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:04.054808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7990780-dc95-4ceb-984f-d962feb5773e admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:04.055087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7990780-dc95-4ceb-984f-d962feb5773e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:04.055358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7990780-dc95-4ceb-984f-d962feb5773e admin admin] PATH_INFO: `/v3/projects/ffc82a7c80da462980f6a6d9ca5f8303/users/bb0837c4526042d59310159de6b9383f/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:04.100801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7990780-dc95-4ceb-984f-d962feb5773e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc82a7c80da462980f6a6d9ca5f8303, user_id=bb0837c4526042d59310159de6b9383f, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:04.107262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7990780-dc95-4ceb-984f-d962feb5773e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:04.148316 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 884/1769] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:03 2022] PUT /identity/v3/projects/ffc82a7c80da462980f6a6d9ca5f8303/users/bb0837c4526042d59310159de6b9383f/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:04.149760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7990780-dc95-4ceb-984f-d962feb5773e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:04.153926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b7990780-dc95-4ceb-984f-d962feb5773e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:04.221308 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 886/1770] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:03 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:04.233272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c07c4-eb39-4e0e-9e80-bac1cb3a3255 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:04.273621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c07c4-eb39-4e0e-9e80-bac1cb3a3255 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:04.296355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:04.399999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7990780-dc95-4ceb-984f-d962feb5773e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:04.407633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee9096-503d-4528-acaa-59f9c9785b13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:04.408694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee9096-503d-4528-acaa-59f9c9785b13 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:04.409405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee9096-503d-4528-acaa-59f9c9785b13 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:04.410712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee9096-503d-4528-acaa-59f9c9785b13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:04.413590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-59ee9096-503d-4528-acaa-59f9c9785b13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:04.457162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c5e08-c1d2-435c-a64e-18aa9be58356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:04.460606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c5e08-c1d2-435c-a64e-18aa9be58356 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:04.461856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c5e08-c1d2-435c-a64e-18aa9be58356 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:04.465195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c5e08-c1d2-435c-a64e-18aa9be58356 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:04.466197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c5e08-c1d2-435c-a64e-18aa9be58356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:04.479495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c5e08-c1d2-435c-a64e-18aa9be58356 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:04.517070 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 887/1771] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:04.520258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c5e08-c1d2-435c-a64e-18aa9be58356 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:04.554328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c5e08-c1d2-435c-a64e-18aa9be58356 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:04.571827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e65e3c-8b77-4ebb-8661-e69468b3ac34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:04.574346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e65e3c-8b77-4ebb-8661-e69468b3ac34 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:04.575126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e65e3c-8b77-4ebb-8661-e69468b3ac34 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:04.576557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e65e3c-8b77-4ebb-8661-e69468b3ac34 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:04.579402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e65e3c-8b77-4ebb-8661-e69468b3ac34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:04.638826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e65e3c-8b77-4ebb-8661-e69468b3ac34 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:04.680457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee9096-503d-4528-acaa-59f9c9785b13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:04.683637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee9096-503d-4528-acaa-59f9c9785b13 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:04.715825 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 888/1772] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:04 2022] POST /identity/v3/projects => generated 401 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:04.715825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e65e3c-8b77-4ebb-8661-e69468b3ac34 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:04.730385 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 885/1773] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:04 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:04.731798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee9096-503d-4528-acaa-59f9c9785b13 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:04.756336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee9096-503d-4528-acaa-59f9c9785b13 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:04.810214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:04.836365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617e206-1631-4448-8d65-803491df575c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:04.837252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617e206-1631-4448-8d65-803491df575c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:04.837689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617e206-1631-4448-8d65-803491df575c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:04.837946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617e206-1631-4448-8d65-803491df575c admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:04.838649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617e206-1631-4448-8d65-803491df575c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:04.842757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617e206-1631-4448-8d65-803491df575c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:04.916528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e65e3c-8b77-4ebb-8661-e69468b3ac34 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:04.924518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3568b9b-616f-4be4-9f23-002788031001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:04.926081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3568b9b-616f-4be4-9f23-002788031001 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:04.927034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3568b9b-616f-4be4-9f23-002788031001 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:04.928929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3568b9b-616f-4be4-9f23-002788031001 admin admin] PATH_INFO: `/v3/projects/ffc82a7c80da462980f6a6d9ca5f8303/users/bb0837c4526042d59310159de6b9383f/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:04.929396 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 886/1774] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:04 2022] POST /identity/v3/users => generated 307 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:04.932728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617e206-1631-4448-8d65-803491df575c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:04.933685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3568b9b-616f-4be4-9f23-002788031001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc82a7c80da462980f6a6d9ca5f8303, user_id=bb0837c4526042d59310159de6b9383f, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:04.940315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3568b9b-616f-4be4-9f23-002788031001 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:04.961956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617e206-1631-4448-8d65-803491df575c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:04.966090 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 889/1775] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:04 2022] PUT /identity/v3/projects/ffc82a7c80da462980f6a6d9ca5f8303/users/bb0837c4526042d59310159de6b9383f/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:04.967922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375ad49-07ad-47fa-a956-7e26e65e478b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:04.968208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375ad49-07ad-47fa-a956-7e26e65e478b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:04.972543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375ad49-07ad-47fa-a956-7e26e65e478b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:05.090045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e375ad49-07ad-47fa-a956-7e26e65e478b None None] MFA Rules not processed for user `bb0837c4526042d59310159de6b9383f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:05.091303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e375ad49-07ad-47fa-a956-7e26e65e478b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:05.146099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355f6d5-1fc3-4b18-8589-50c38720b26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:05.147119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355f6d5-1fc3-4b18-8589-50c38720b26c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:05.147706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355f6d5-1fc3-4b18-8589-50c38720b26c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:05.148195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355f6d5-1fc3-4b18-8589-50c38720b26c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:05.149111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355f6d5-1fc3-4b18-8589-50c38720b26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:05.153871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355f6d5-1fc3-4b18-8589-50c38720b26c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:05.199943 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 887/1776] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:05.201394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355f6d5-1fc3-4b18-8589-50c38720b26c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:05.230915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355f6d5-1fc3-4b18-8589-50c38720b26c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:05.257487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294acf0a-a18f-4253-b6bb-de22dd2f899f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:05.265618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294acf0a-a18f-4253-b6bb-de22dd2f899f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:05.266303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294acf0a-a18f-4253-b6bb-de22dd2f899f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:05.266831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294acf0a-a18f-4253-b6bb-de22dd2f899f admin admin] PATH_INFO: `/v3/users/1a4fbbe792484d33a2774eb66ee42f08` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:05.268357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294acf0a-a18f-4253-b6bb-de22dd2f899f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a4fbbe792484d33a2774eb66ee42f08)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:05.271209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294acf0a-a18f-4253-b6bb-de22dd2f899f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:05.328960 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 890/1777] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:05.330068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375ad49-07ad-47fa-a956-7e26e65e478b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:05.363945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:05.385945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375ad49-07ad-47fa-a956-7e26e65e478b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:05.410933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-294acf0a-a18f-4253-b6bb-de22dd2f899f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a4fbbe792484d33a2774eb66ee42f08'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:05.413291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a6001-aa30-4996-89c8-7b5efb4e1c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:05.417203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a6001-aa30-4996-89c8-7b5efb4e1c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:05.417498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a6001-aa30-4996-89c8-7b5efb4e1c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:05.421141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a6001-aa30-4996-89c8-7b5efb4e1c3c admin admin] PATH_INFO: `/v3/projects/519e44ad07dc4153a8736e73ae1dd963/users/efc81a87adbf4b10a17eed2b8621b05c/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:05.431195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-294acf0a-a18f-4253-b6bb-de22dd2f899f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a4fbbe792484d33a2774eb66ee42f08'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:05.463147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-294acf0a-a18f-4253-b6bb-de22dd2f899f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a4fbbe792484d33a2774eb66ee42f08'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:05.466631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-294acf0a-a18f-4253-b6bb-de22dd2f899f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a4fbbe792484d33a2774eb66ee42f08'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:05.469225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-294acf0a-a18f-4253-b6bb-de22dd2f899f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a4fbbe792484d33a2774eb66ee42f08'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:05.476495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a6001-aa30-4996-89c8-7b5efb4e1c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=519e44ad07dc4153a8736e73ae1dd963, user_id=efc81a87adbf4b10a17eed2b8621b05c, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:05.481693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a6001-aa30-4996-89c8-7b5efb4e1c3c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:05.508935 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 888/1778] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:05 2022] DELETE /identity/v3/users/1a4fbbe792484d33a2774eb66ee42f08 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:05.514073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dfc94-441a-4398-af31-4c12b9aa2fb1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:05.516120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dfc94-441a-4398-af31-4c12b9aa2fb1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:05.517220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dfc94-441a-4398-af31-4c12b9aa2fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:05.519584 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 891/1779] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:05 2022] PUT /identity/v3/projects/519e44ad07dc4153a8736e73ae1dd963/users/efc81a87adbf4b10a17eed2b8621b05c/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:05.522987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32811cff-2a18-4bb4-bc31-2189d7b007c5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:05.524280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32811cff-2a18-4bb4-bc31-2189d7b007c5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:05.524905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32811cff-2a18-4bb4-bc31-2189d7b007c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:05.603206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-dd8dfc94-441a-4398-af31-4c12b9aa2fb1 None None] MFA Rules not processed for user `c3adc70e73614d48be23e3542424ee1d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:05.606319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dd8dfc94-441a-4398-af31-4c12b9aa2fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:05.618364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-32811cff-2a18-4bb4-bc31-2189d7b007c5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:05.619811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-32811cff-2a18-4bb4-bc31-2189d7b007c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:05.694905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:05.767995 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 889/1780] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:05.769226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8dfc94-441a-4398-af31-4c12b9aa2fb1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:05.773305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dd8dfc94-441a-4398-af31-4c12b9aa2fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:05.832477 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 892/1781] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:05.870742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32811cff-2a18-4bb4-bc31-2189d7b007c5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:05.883089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-32811cff-2a18-4bb4-bc31-2189d7b007c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:05.947214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8dfc94-441a-4398-af31-4c12b9aa2fb1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:05.960804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ceb686-258e-48c8-9939-9911f0e9d951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:05.961321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ceb686-258e-48c8-9939-9911f0e9d951 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:05.961618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ceb686-258e-48c8-9939-9911f0e9d951 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:05.962053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ceb686-258e-48c8-9939-9911f0e9d951 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:05.964462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ceb686-258e-48c8-9939-9911f0e9d951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:05.966257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ceb686-258e-48c8-9939-9911f0e9d951 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:05.993910 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 890/1782] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:05.996760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f40ea-6cc3-4212-b68a-06fc921c8f33 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:05.997839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f40ea-6cc3-4212-b68a-06fc921c8f33 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:05.998144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f40ea-6cc3-4212-b68a-06fc921c8f33 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:06.001415 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 891/1783] 10.222.0.23 () {54 vars in 921 bytes} [Sat Aug 20 20:55:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:55:06.004407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f40ea-6cc3-4212-b68a-06fc921c8f33 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:06.014736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32811cff-2a18-4bb4-bc31-2189d7b007c5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:06.018929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcbe37-7376-48a3-b05c-c8c6759beb4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:06.019891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcbe37-7376-48a3-b05c-c8c6759beb4e service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:06.020234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcbe37-7376-48a3-b05c-c8c6759beb4e service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:06.020940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcbe37-7376-48a3-b05c-c8c6759beb4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:06.033487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcbe37-7376-48a3-b05c-c8c6759beb4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:06.034584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f40ea-6cc3-4212-b68a-06fc921c8f33 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:06.035445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcbe37-7376-48a3-b05c-c8c6759beb4e service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:06.048283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e86ce-c6a2-4e11-a573-7776c49ab9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:06.048283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e86ce-c6a2-4e11-a573-7776c49ab9a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:06.048636 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 893/1784] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:06.048824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e86ce-c6a2-4e11-a573-7776c49ab9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:06.050126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e86ce-c6a2-4e11-a573-7776c49ab9a9 admin admin] PATH_INFO: `/v3/projects/519e44ad07dc4153a8736e73ae1dd963/users/efc81a87adbf4b10a17eed2b8621b05c/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:06.052405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e86ce-c6a2-4e11-a573-7776c49ab9a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=519e44ad07dc4153a8736e73ae1dd963, user_id=efc81a87adbf4b10a17eed2b8621b05c, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:06.058593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e86ce-c6a2-4e11-a573-7776c49ab9a9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:06.078093 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 892/1785] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:06 2022] PUT /identity/v3/projects/519e44ad07dc4153a8736e73ae1dd963/users/efc81a87adbf4b10a17eed2b8621b05c/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:06.091113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b4e3d-b624-4caa-a9d9-2cf98677a606 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:06.091464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b4e3d-b624-4caa-a9d9-2cf98677a606 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:06.091788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b4e3d-b624-4caa-a9d9-2cf98677a606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:06.153565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ad0b4e3d-b624-4caa-a9d9-2cf98677a606 None None] MFA Rules not processed for user `efc81a87adbf4b10a17eed2b8621b05c`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:06.157845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ad0b4e3d-b624-4caa-a9d9-2cf98677a606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:06.284550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e86ce-c6a2-4e11-a573-7776c49ab9a9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:06.287252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0c2e86ce-c6a2-4e11-a573-7776c49ab9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:06.385753 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 894/1786] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:06 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:06.387873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a0aa2-4117-4c6f-9f07-f5c9acaf540b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:06.387873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a0aa2-4117-4c6f-9f07-f5c9acaf540b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:06.388117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a0aa2-4117-4c6f-9f07-f5c9acaf540b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:06.461908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e86ce-c6a2-4e11-a573-7776c49ab9a9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:06.468989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fc0a0aa2-4117-4c6f-9f07-f5c9acaf540b None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:06.469793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d34170-8b0b-4b15-bc05-bd653ffc5fc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:06.470573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d34170-8b0b-4b15-bc05-bd653ffc5fc0 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:06.470936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d34170-8b0b-4b15-bc05-bd653ffc5fc0 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:06.471455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d34170-8b0b-4b15-bc05-bd653ffc5fc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:06.480469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-32d34170-8b0b-4b15-bc05-bd653ffc5fc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:06.480678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fc0a0aa2-4117-4c6f-9f07-f5c9acaf540b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:06.513708 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 895/1787] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 20:55:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:06.604204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d34170-8b0b-4b15-bc05-bd653ffc5fc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:06.607464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d34170-8b0b-4b15-bc05-bd653ffc5fc0 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:06.642684 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 893/1788] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:06.729175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a0aa2-4117-4c6f-9f07-f5c9acaf540b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:06.730625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fc0a0aa2-4117-4c6f-9f07-f5c9acaf540b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:06.740535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:06.760751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a0aa2-4117-4c6f-9f07-f5c9acaf540b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:06.796236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab5ed2-427f-4f69-b675-9e3a3bec4229 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:06.797229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab5ed2-427f-4f69-b675-9e3a3bec4229 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:06.797527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab5ed2-427f-4f69-b675-9e3a3bec4229 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:06.797803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab5ed2-427f-4f69-b675-9e3a3bec4229 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:06.833872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab5ed2-427f-4f69-b675-9e3a3bec4229 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:06.835699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab5ed2-427f-4f69-b675-9e3a3bec4229 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:06.886807 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 896/1789] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:06 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:07.000153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d34170-8b0b-4b15-bc05-bd653ffc5fc0 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:07.009593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:07.020071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d34170-8b0b-4b15-bc05-bd653ffc5fc0 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:07.045269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebcb05-db6c-480d-851c-910da2a1af8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:07.047946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebcb05-db6c-480d-851c-910da2a1af8a service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:07.048745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebcb05-db6c-480d-851c-910da2a1af8a service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:07.051717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebcb05-db6c-480d-851c-910da2a1af8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:07.081732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebcb05-db6c-480d-851c-910da2a1af8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:07.089377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebcb05-db6c-480d-851c-910da2a1af8a service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:07.119817 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 894/1790] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:07.409848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab5ed2-427f-4f69-b675-9e3a3bec4229 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:07.426623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab5ed2-427f-4f69-b675-9e3a3bec4229 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:07.437312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabe5a6-f2c4-4942-be7a-8d10b35ee79f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:07.440068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabe5a6-f2c4-4942-be7a-8d10b35ee79f service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:07.440748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabe5a6-f2c4-4942-be7a-8d10b35ee79f service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:07.441218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabe5a6-f2c4-4942-be7a-8d10b35ee79f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:07.444201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ffabe5a6-f2c4-4942-be7a-8d10b35ee79f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:07.573826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cb2f1-c152-4063-8783-6e465dc57579 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:07.574383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cb2f1-c152-4063-8783-6e465dc57579 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:07.574383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cb2f1-c152-4063-8783-6e465dc57579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:07.578442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabe5a6-f2c4-4942-be7a-8d10b35ee79f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:07.580273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabe5a6-f2c4-4942-be7a-8d10b35ee79f service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:07.621989 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 897/1791] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:07 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:07.630930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c56cb2f1-c152-4063-8783-6e465dc57579 None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:07.642078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c56cb2f1-c152-4063-8783-6e465dc57579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:07.872489 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 895/1792] 10.222.0.30 () {58 vars in 1015 bytes} [Sat Aug 20 20:55:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:08.051796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabe5a6-f2c4-4942-be7a-8d10b35ee79f service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:08.053453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ffabe5a6-f2c4-4942-be7a-8d10b35ee79f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:08.085619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabe5a6-f2c4-4942-be7a-8d10b35ee79f service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:08.114209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3e036-9bc1-4e47-b1a9-c37a08378438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:08.115113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3e036-9bc1-4e47-b1a9-c37a08378438 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:08.115562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3e036-9bc1-4e47-b1a9-c37a08378438 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:08.115950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3e036-9bc1-4e47-b1a9-c37a08378438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:08.159575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3e036-9bc1-4e47-b1a9-c37a08378438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:08.162023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3e036-9bc1-4e47-b1a9-c37a08378438 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:08.218562 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 898/1793] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:55:08.552304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cb2f1-c152-4063-8783-6e465dc57579 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:08.554327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c56cb2f1-c152-4063-8783-6e465dc57579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:08.724302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cb2f1-c152-4063-8783-6e465dc57579 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:08.730171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad3c2a-bf54-4665-acb2-587d44357298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:08.731032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad3c2a-bf54-4665-acb2-587d44357298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:08.731306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad3c2a-bf54-4665-acb2-587d44357298 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:08.731572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad3c2a-bf54-4665-acb2-587d44357298 admin admin] PATH_INFO: `/v3/projects/5fd09d159bdc4888b9c1b260af72cdee` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:08.734729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad3c2a-bf54-4665-acb2-587d44357298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fd09d159bdc4888b9c1b260af72cdee)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:08.751505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad3c2a-bf54-4665-acb2-587d44357298 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:08.916678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9cad3c2a-bf54-4665-acb2-587d44357298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fd09d159bdc4888b9c1b260af72cdee', 'request_id': 'req-9cad3c2a-bf54-4665-acb2-587d44357298'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:08.928009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9cad3c2a-bf54-4665-acb2-587d44357298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fd09d159bdc4888b9c1b260af72cdee', 'request_id': 'req-9cad3c2a-bf54-4665-acb2-587d44357298'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:08.950878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9cad3c2a-bf54-4665-acb2-587d44357298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fd09d159bdc4888b9c1b260af72cdee', 'request_id': 'req-9cad3c2a-bf54-4665-acb2-587d44357298'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:08.980196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9cad3c2a-bf54-4665-acb2-587d44357298 admin admin] The token cache is being invalidate because project 5fd09d159bdc4888b9c1b260af72cdee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:08.980566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9cad3c2a-bf54-4665-acb2-587d44357298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:08.981323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9cad3c2a-bf54-4665-acb2-587d44357298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:08.988802 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 896/1794] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:08 2022] DELETE /identity/v3/projects/5fd09d159bdc4888b9c1b260af72cdee => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:08.989970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3e036-9bc1-4e47-b1a9-c37a08378438 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:09.033809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3e036-9bc1-4e47-b1a9-c37a08378438 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:09.132060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:09.132936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:09.133487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:09.134140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] PATH_INFO: `/v3/users/f442d104723f49f0883deb94bc992fd8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:09.135427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f442d104723f49f0883deb94bc992fd8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:09.141793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:09.248151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f442d104723f49f0883deb94bc992fd8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:09.270655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f442d104723f49f0883deb94bc992fd8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:09.291173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f442d104723f49f0883deb94bc992fd8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:09.308494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:09.328797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b768a-10bf-464d-8def-9bc7f51f5b63 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:09.329079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b768a-10bf-464d-8def-9bc7f51f5b63 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:09.329348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b768a-10bf-464d-8def-9bc7f51f5b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:09.334977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f442d104723f49f0883deb94bc992fd8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:09.337462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f442d104723f49f0883deb94bc992fd8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:09.351223 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 899/1795] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:08 2022] DELETE /identity/v3/users/f442d104723f49f0883deb94bc992fd8 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:09.387816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1b7b768a-10bf-464d-8def-9bc7f51f5b63 None None] MFA Rules not processed for user `112c5395a1b243e684a5ec3db9990e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:09.389093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1b7b768a-10bf-464d-8def-9bc7f51f5b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:09.430408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:09.597355 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 897/1796] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:09 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:09.895025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:09.898574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:10.101026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b768a-10bf-464d-8def-9bc7f51f5b63 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:10.102759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1b7b768a-10bf-464d-8def-9bc7f51f5b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:10.187560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474334bf-e3f0-4497-8772-4f6a8ff02ecc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:10.205730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4628d-a055-4e25-912b-e7516f722776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:10.208441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4628d-a055-4e25-912b-e7516f722776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:10.209175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4628d-a055-4e25-912b-e7516f722776 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:10.210296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4628d-a055-4e25-912b-e7516f722776 admin admin] PATH_INFO: `/v3/projects/9168c4b22fa045a3808cd7352573b0cf` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:10.214307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4628d-a055-4e25-912b-e7516f722776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9168c4b22fa045a3808cd7352573b0cf)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:10.218789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4628d-a055-4e25-912b-e7516f722776 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:10.341019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b768a-10bf-464d-8def-9bc7f51f5b63 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:10.369821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-86b4628d-a055-4e25-912b-e7516f722776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9168c4b22fa045a3808cd7352573b0cf', 'request_id': 'req-86b4628d-a055-4e25-912b-e7516f722776'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:10.394425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-86b4628d-a055-4e25-912b-e7516f722776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9168c4b22fa045a3808cd7352573b0cf', 'request_id': 'req-86b4628d-a055-4e25-912b-e7516f722776'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:10.402261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-86b4628d-a055-4e25-912b-e7516f722776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9168c4b22fa045a3808cd7352573b0cf', 'request_id': 'req-86b4628d-a055-4e25-912b-e7516f722776'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:10.432812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-86b4628d-a055-4e25-912b-e7516f722776 admin admin] The token cache is being invalidate because project 9168c4b22fa045a3808cd7352573b0cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:10.433202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-86b4628d-a055-4e25-912b-e7516f722776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:10.439352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-86b4628d-a055-4e25-912b-e7516f722776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:10.449068 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 900/1797] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:09 2022] DELETE /identity/v3/projects/9168c4b22fa045a3808cd7352573b0cf => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:10.461290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34058687-3ec4-431d-8557-f072136d6c07 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:10.462827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34058687-3ec4-431d-8557-f072136d6c07 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:10.463647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34058687-3ec4-431d-8557-f072136d6c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:10.526024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43316a88-14b5-44cc-b67a-11960f3cc751 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:10.527223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43316a88-14b5-44cc-b67a-11960f3cc751 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:10.527625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43316a88-14b5-44cc-b67a-11960f3cc751 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:10.529039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43316a88-14b5-44cc-b67a-11960f3cc751 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:10.533375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-43316a88-14b5-44cc-b67a-11960f3cc751 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:10.538196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-34058687-3ec4-431d-8557-f072136d6c07 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:10.541216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-34058687-3ec4-431d-8557-f072136d6c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:10.646392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:10.672142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43316a88-14b5-44cc-b67a-11960f3cc751 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:10.674206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43316a88-14b5-44cc-b67a-11960f3cc751 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:10.704740 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 898/1798] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:10 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:55:10.725125 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 901/1799] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:10.737748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65db26-e231-457d-ac96-eb8cad0eacb3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:10.738183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65db26-e231-457d-ac96-eb8cad0eacb3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:10.738515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65db26-e231-457d-ac96-eb8cad0eacb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:10.806827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-dd65db26-e231-457d-ac96-eb8cad0eacb3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:10.811467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dd65db26-e231-457d-ac96-eb8cad0eacb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:10.836429 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 899/1800] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:10.838072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34058687-3ec4-431d-8557-f072136d6c07 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:10.854217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34058687-3ec4-431d-8557-f072136d6c07 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:10.868744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1b66d-f400-4411-929d-55b0bc98439a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:10.869585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1b66d-f400-4411-929d-55b0bc98439a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:10.869966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1b66d-f400-4411-929d-55b0bc98439a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:10.870357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1b66d-f400-4411-929d-55b0bc98439a admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:10.871135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1b66d-f400-4411-929d-55b0bc98439a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:55:10.872064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1b66d-f400-4411-929d-55b0bc98439a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:10.875913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1b66d-f400-4411-929d-55b0bc98439a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:10.905984 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 902/1801] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:55:10.906488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65db26-e231-457d-ac96-eb8cad0eacb3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:10.919809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65db26-e231-457d-ac96-eb8cad0eacb3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:10.932420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337d750-b6d2-4b40-b73e-1b34971e6dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:10.933425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337d750-b6d2-4b40-b73e-1b34971e6dbf admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:10.933795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337d750-b6d2-4b40-b73e-1b34971e6dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:10.934241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337d750-b6d2-4b40-b73e-1b34971e6dbf admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:10.935106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337d750-b6d2-4b40-b73e-1b34971e6dbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:10.938352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337d750-b6d2-4b40-b73e-1b34971e6dbf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:11.031405 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 900/1802] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:10 2022] POST /identity/v3/projects => generated 361 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:11.035925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1b66d-f400-4411-929d-55b0bc98439a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:11.049550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1b66d-f400-4411-929d-55b0bc98439a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:11.072491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:11.147447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0581b6-d2a3-43b4-b6c8-d21913813421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:11.148072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0581b6-d2a3-43b4-b6c8-d21913813421 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:11.148355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0581b6-d2a3-43b4-b6c8-d21913813421 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:11.148674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0581b6-d2a3-43b4-b6c8-d21913813421 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:11.149597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0581b6-d2a3-43b4-b6c8-d21913813421 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:11.152406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0581b6-d2a3-43b4-b6c8-d21913813421 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:11.186680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337d750-b6d2-4b40-b73e-1b34971e6dbf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:11.192761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5337d750-b6d2-4b40-b73e-1b34971e6dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:11.212525 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 903/1803] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:11 2022] POST /identity/v3/users => generated 281 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:55:11.213523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0581b6-d2a3-43b4-b6c8-d21913813421 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:11.246843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0581b6-d2a3-43b4-b6c8-d21913813421 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:11.252337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbafd49-21f9-4df4-9634-34f36f0a53a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:11.253178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbafd49-21f9-4df4-9634-34f36f0a53a2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:11.253513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbafd49-21f9-4df4-9634-34f36f0a53a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:11.253907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbafd49-21f9-4df4-9634-34f36f0a53a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:11.254982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbafd49-21f9-4df4-9634-34f36f0a53a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:11.256925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbafd49-21f9-4df4-9634-34f36f0a53a2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:11.271261 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 904/1804] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:11.275441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbafd49-21f9-4df4-9634-34f36f0a53a2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:11.293575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbafd49-21f9-4df4-9634-34f36f0a53a2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:11.300435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf5b51-6ce7-450e-910f-039ae9dbb30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:11.301118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf5b51-6ce7-450e-910f-039ae9dbb30a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:11.302232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf5b51-6ce7-450e-910f-039ae9dbb30a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:11.302585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf5b51-6ce7-450e-910f-039ae9dbb30a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:11.303714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf5b51-6ce7-450e-910f-039ae9dbb30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:11.306480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf5b51-6ce7-450e-910f-039ae9dbb30a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:11.334333 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 905/1805] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:55:11.334333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf5b51-6ce7-450e-910f-039ae9dbb30a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:11.357111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf5b51-6ce7-450e-910f-039ae9dbb30a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:11.369884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c97b6b-6d51-4d06-88a0-c359e6445293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:11.372158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c97b6b-6d51-4d06-88a0-c359e6445293 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:11.373973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c97b6b-6d51-4d06-88a0-c359e6445293 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:11.374386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c97b6b-6d51-4d06-88a0-c359e6445293 admin admin] PATH_INFO: `/v3/projects/ce651839d9ec4f42b83073e0134a6db7/users/f15d453b100f4b99ae3ab285cdbb88f0/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:11.430001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337d750-b6d2-4b40-b73e-1b34971e6dbf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:11.439335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580ebf5-ab3a-484b-b6c4-75444afa45af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:11.452300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c97b6b-6d51-4d06-88a0-c359e6445293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce651839d9ec4f42b83073e0134a6db7, user_id=f15d453b100f4b99ae3ab285cdbb88f0, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:11.452300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c97b6b-6d51-4d06-88a0-c359e6445293 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:11.471096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580ebf5-ab3a-484b-b6c4-75444afa45af service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:11.471907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580ebf5-ab3a-484b-b6c4-75444afa45af service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:11.473058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580ebf5-ab3a-484b-b6c4-75444afa45af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:11.476946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8580ebf5-ab3a-484b-b6c4-75444afa45af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:11.487918 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 906/1806] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:11 2022] PUT /identity/v3/projects/ce651839d9ec4f42b83073e0134a6db7/users/f15d453b100f4b99ae3ab285cdbb88f0/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:11.504856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b782cde-7845-4a4b-9d80-c956cacfcff8 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:11.505428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b782cde-7845-4a4b-9d80-c956cacfcff8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:11.505803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b782cde-7845-4a4b-9d80-c956cacfcff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:11.579684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580ebf5-ab3a-484b-b6c4-75444afa45af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:11.581924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580ebf5-ab3a-484b-b6c4-75444afa45af service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:11.588707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1b782cde-7845-4a4b-9d80-c956cacfcff8 None None] MFA Rules not processed for user `f15d453b100f4b99ae3ab285cdbb88f0`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:11.592635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1b782cde-7845-4a4b-9d80-c956cacfcff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:11.598750 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 901/1807] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:11 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:11.748069 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 907/1808] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:11.753376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580ebf5-ab3a-484b-b6c4-75444afa45af service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:11.810244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580ebf5-ab3a-484b-b6c4-75444afa45af service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:11.886404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:11.898867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd3cca-ee64-4fc2-86aa-3448823231de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:11.899620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd3cca-ee64-4fc2-86aa-3448823231de admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:11.900170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd3cca-ee64-4fc2-86aa-3448823231de admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:11.900779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd3cca-ee64-4fc2-86aa-3448823231de admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:11.902790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd3cca-ee64-4fc2-86aa-3448823231de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:11.906819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd3cca-ee64-4fc2-86aa-3448823231de admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:11.960759 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 902/1809] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:11 2022] POST /identity/v3/projects => generated 359 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:11.966354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b782cde-7845-4a4b-9d80-c956cacfcff8 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:11.992532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b782cde-7845-4a4b-9d80-c956cacfcff8 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:12.053425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:12.081107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e659f-13d7-4065-83a5-f069ff69084a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:12.087633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e659f-13d7-4065-83a5-f069ff69084a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:12.087961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e659f-13d7-4065-83a5-f069ff69084a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:12.088357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e659f-13d7-4065-83a5-f069ff69084a admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:12.089153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e659f-13d7-4065-83a5-f069ff69084a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:12.092941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e659f-13d7-4065-83a5-f069ff69084a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:12.156466 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 908/1810] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:11 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:12.157770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd3cca-ee64-4fc2-86aa-3448823231de admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:12.173136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd3cca-ee64-4fc2-86aa-3448823231de admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:12.179252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ab7f4-b7d1-4ee8-adfb-f96b9c3aa47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:12.180195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ab7f4-b7d1-4ee8-adfb-f96b9c3aa47e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:12.180542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ab7f4-b7d1-4ee8-adfb-f96b9c3aa47e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:12.180817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ab7f4-b7d1-4ee8-adfb-f96b9c3aa47e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:12.181518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ab7f4-b7d1-4ee8-adfb-f96b9c3aa47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:12.183229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ab7f4-b7d1-4ee8-adfb-f96b9c3aa47e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:12.200432 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 903/1811] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:12.212353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e659f-13d7-4065-83a5-f069ff69084a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:12.224176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e659f-13d7-4065-83a5-f069ff69084a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:12.233167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb8049-ff60-40b5-9ef0-fea34eda06f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:12.235018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb8049-ff60-40b5-9ef0-fea34eda06f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:12.235444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb8049-ff60-40b5-9ef0-fea34eda06f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:12.235866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb8049-ff60-40b5-9ef0-fea34eda06f4 admin admin] PATH_INFO: `/v3/projects/5ee1b0e5dfab4110a164e7285698eee6/users/96663d32565b4af795f80c58423c31ad/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:12.274334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb8049-ff60-40b5-9ef0-fea34eda06f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ee1b0e5dfab4110a164e7285698eee6, user_id=96663d32565b4af795f80c58423c31ad, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:12.278167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb8049-ff60-40b5-9ef0-fea34eda06f4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:12.306483 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 909/1812] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:12 2022] PUT /identity/v3/projects/5ee1b0e5dfab4110a164e7285698eee6/users/96663d32565b4af795f80c58423c31ad/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:12.310648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ab7f4-b7d1-4ee8-adfb-f96b9c3aa47e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:12.335996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ab7f4-b7d1-4ee8-adfb-f96b9c3aa47e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:12.569488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f96c2-76a5-4e61-a853-a7622194f7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:12.570994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f96c2-76a5-4e61-a853-a7622194f7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:12.572088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f96c2-76a5-4e61-a853-a7622194f7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:12.572444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f96c2-76a5-4e61-a853-a7622194f7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:12.573576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f96c2-76a5-4e61-a853-a7622194f7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:12.576479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f96c2-76a5-4e61-a853-a7622194f7c9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:12.623403 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 904/1813] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:12.627557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb8049-ff60-40b5-9ef0-fea34eda06f4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:12.647185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb8049-ff60-40b5-9ef0-fea34eda06f4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:12.667053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e389ae-7192-4640-ba90-b26823b2ef9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:12.667763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e389ae-7192-4640-ba90-b26823b2ef9c admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:12.668040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e389ae-7192-4640-ba90-b26823b2ef9c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:12.668309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e389ae-7192-4640-ba90-b26823b2ef9c admin admin] PATH_INFO: `/v3/projects/5ee1b0e5dfab4110a164e7285698eee6/users/96663d32565b4af795f80c58423c31ad/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:12.670497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e389ae-7192-4640-ba90-b26823b2ef9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ee1b0e5dfab4110a164e7285698eee6, user_id=96663d32565b4af795f80c58423c31ad, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:12.673290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e389ae-7192-4640-ba90-b26823b2ef9c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:12.698807 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 910/1814] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:12 2022] PUT /identity/v3/projects/5ee1b0e5dfab4110a164e7285698eee6/users/96663d32565b4af795f80c58423c31ad/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:12.715018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d0f85-582a-42fd-bdff-863615ec741c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:12.715308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d0f85-582a-42fd-bdff-863615ec741c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:12.715661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d0f85-582a-42fd-bdff-863615ec741c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:12.787094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c63d0f85-582a-42fd-bdff-863615ec741c None None] MFA Rules not processed for user `96663d32565b4af795f80c58423c31ad`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:12.788427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c63d0f85-582a-42fd-bdff-863615ec741c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:12.980101 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 905/1815] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:12 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:13.196141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e389ae-7192-4640-ba90-b26823b2ef9c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:13.390997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e389ae-7192-4640-ba90-b26823b2ef9c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:13.397526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf230e7-69cf-4e04-9fcb-554c7f161516 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:13.399056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf230e7-69cf-4e04-9fcb-554c7f161516 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:13.399272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf230e7-69cf-4e04-9fcb-554c7f161516 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:13.399548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf230e7-69cf-4e04-9fcb-554c7f161516 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:13.414750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf230e7-69cf-4e04-9fcb-554c7f161516 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:13.416923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf230e7-69cf-4e04-9fcb-554c7f161516 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:13.430306 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 911/1816] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:13 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:15.064189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d0f85-582a-42fd-bdff-863615ec741c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:15.211397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf230e7-69cf-4e04-9fcb-554c7f161516 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:15.221362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9bf230e7-69cf-4e04-9fcb-554c7f161516 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:15.241970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:15.279491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d0f85-582a-42fd-bdff-863615ec741c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:15.288413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8fa9b-ec1f-432c-a99a-869c188f0101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:15.293642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8fa9b-ec1f-432c-a99a-869c188f0101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:15.293924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8fa9b-ec1f-432c-a99a-869c188f0101 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:15.294272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8fa9b-ec1f-432c-a99a-869c188f0101 admin admin] PATH_INFO: `/v3/users/f15d453b100f4b99ae3ab285cdbb88f0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:15.295764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8fa9b-ec1f-432c-a99a-869c188f0101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f15d453b100f4b99ae3ab285cdbb88f0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:15.299216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8fa9b-ec1f-432c-a99a-869c188f0101 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:15.362619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf230e7-69cf-4e04-9fcb-554c7f161516 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:15.373165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e2231-058e-4a55-ba12-e6bda9c355f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:15.374311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e2231-058e-4a55-ba12-e6bda9c355f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:15.374650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e2231-058e-4a55-ba12-e6bda9c355f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:15.374967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e2231-058e-4a55-ba12-e6bda9c355f6 admin admin] PATH_INFO: `/v3/users/aef4fc482ab1475fa24637f6991ae876` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:15.379072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e2231-058e-4a55-ba12-e6bda9c355f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef4fc482ab1475fa24637f6991ae876)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:15.386403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e2231-058e-4a55-ba12-e6bda9c355f6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:15.409556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c3d8fa9b-ec1f-432c-a99a-869c188f0101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f15d453b100f4b99ae3ab285cdbb88f0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:15.426436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c3d8fa9b-ec1f-432c-a99a-869c188f0101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f15d453b100f4b99ae3ab285cdbb88f0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:15.451257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:15.453923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c3d8fa9b-ec1f-432c-a99a-869c188f0101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f15d453b100f4b99ae3ab285cdbb88f0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:15.455331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c3d8fa9b-ec1f-432c-a99a-869c188f0101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f15d453b100f4b99ae3ab285cdbb88f0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:15.457384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c3d8fa9b-ec1f-432c-a99a-869c188f0101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f15d453b100f4b99ae3ab285cdbb88f0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:15.482801 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 906/1817] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:15 2022] DELETE /identity/v3/users/f15d453b100f4b99ae3ab285cdbb88f0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:15.489403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df8bca-6b43-4e60-be03-7077b2cb935a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:15.489950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df8bca-6b43-4e60-be03-7077b2cb935a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:15.490370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df8bca-6b43-4e60-be03-7077b2cb935a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:15.509011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e58e2231-058e-4a55-ba12-e6bda9c355f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef4fc482ab1475fa24637f6991ae876'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:15.526470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e58e2231-058e-4a55-ba12-e6bda9c355f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef4fc482ab1475fa24637f6991ae876'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:15.549660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e58e2231-058e-4a55-ba12-e6bda9c355f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef4fc482ab1475fa24637f6991ae876'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:15.553240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-37df8bca-6b43-4e60-be03-7077b2cb935a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:15.554810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-37df8bca-6b43-4e60-be03-7077b2cb935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:15.581912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e58e2231-058e-4a55-ba12-e6bda9c355f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef4fc482ab1475fa24637f6991ae876'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:15.583209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e58e2231-058e-4a55-ba12-e6bda9c355f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef4fc482ab1475fa24637f6991ae876'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:15.595985 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 912/1818] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:15 2022] DELETE /identity/v3/users/aef4fc482ab1475fa24637f6991ae876 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:15.598394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0bd58-ce7c-4b4f-9a00-44dce61219fe None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:15.598712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0bd58-ce7c-4b4f-9a00-44dce61219fe None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:15.598984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0bd58-ce7c-4b4f-9a00-44dce61219fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:15.631378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6bd0bd58-ce7c-4b4f-9a00-44dce61219fe None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:15.632751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6bd0bd58-ce7c-4b4f-9a00-44dce61219fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:15.644489 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 907/1819] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:15.661395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df8bca-6b43-4e60-be03-7077b2cb935a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:15.673815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-37df8bca-6b43-4e60-be03-7077b2cb935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:15.716704 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 913/1820] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:15.735071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0bd58-ce7c-4b4f-9a00-44dce61219fe None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:15.737169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6bd0bd58-ce7c-4b4f-9a00-44dce61219fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:15.791416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df8bca-6b43-4e60-be03-7077b2cb935a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:15.816999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0bd58-ce7c-4b4f-9a00-44dce61219fe None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:15.821594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6156de-cd5b-431c-8e22-4e2b7c682f51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:15.822245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6156de-cd5b-431c-8e22-4e2b7c682f51 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:15.822677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6156de-cd5b-431c-8e22-4e2b7c682f51 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:15.822947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6156de-cd5b-431c-8e22-4e2b7c682f51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:15.824833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:15.842526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6156de-cd5b-431c-8e22-4e2b7c682f51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:15.844953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6156de-cd5b-431c-8e22-4e2b7c682f51 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:15.865749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0bccd-2406-4d3b-aa8c-bc69445923a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:15.866449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0bccd-2406-4d3b-aa8c-bc69445923a3 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:15.869945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0bccd-2406-4d3b-aa8c-bc69445923a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:15.870242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0bccd-2406-4d3b-aa8c-bc69445923a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:15.872470 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 914/1821] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:55:15.886952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0bccd-2406-4d3b-aa8c-bc69445923a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:15.888939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0bccd-2406-4d3b-aa8c-bc69445923a3 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:15.908555 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 908/1822] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:16.605508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6156de-cd5b-431c-8e22-4e2b7c682f51 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:16.607227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9a6156de-cd5b-431c-8e22-4e2b7c682f51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:16.619964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0bccd-2406-4d3b-aa8c-bc69445923a3 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:16.621979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6156de-cd5b-431c-8e22-4e2b7c682f51 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:16.622803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8ab0bccd-2406-4d3b-aa8c-bc69445923a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:16.627876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:16.628581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:16.629002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:16.629405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] PATH_INFO: `/v3/projects/7ac138e2ccba4251851f97d850689015` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:16.630845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac138e2ccba4251851f97d850689015)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:16.633027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:16.668724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0bccd-2406-4d3b-aa8c-bc69445923a3 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:16.675557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:16.676436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:16.676833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:16.677089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] PATH_INFO: `/v3/projects/ce651839d9ec4f42b83073e0134a6db7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:16.678408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce651839d9ec4f42b83073e0134a6db7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:16.680583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:16.701587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:16.711842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac138e2ccba4251851f97d850689015', 'request_id': 'req-5d7f49a9-791b-4e78-9d18-9dfde94523de'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.723018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac138e2ccba4251851f97d850689015', 'request_id': 'req-5d7f49a9-791b-4e78-9d18-9dfde94523de'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.729872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac138e2ccba4251851f97d850689015', 'request_id': 'req-5d7f49a9-791b-4e78-9d18-9dfde94523de'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.742576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] The token cache is being invalidate because project 7ac138e2ccba4251851f97d850689015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:16.742878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.743756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.746275 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 915/1823] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:16 2022] DELETE /identity/v3/projects/7ac138e2ccba4251851f97d850689015 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:16.751591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:16.753091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:16.755788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce651839d9ec4f42b83073e0134a6db7', 'request_id': 'req-8472ec32-0eb9-4957-ae37-29727cda5519'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.765641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce651839d9ec4f42b83073e0134a6db7', 'request_id': 'req-8472ec32-0eb9-4957-ae37-29727cda5519'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.776435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce651839d9ec4f42b83073e0134a6db7', 'request_id': 'req-8472ec32-0eb9-4957-ae37-29727cda5519'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.789003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] The token cache is being invalidate because project ce651839d9ec4f42b83073e0134a6db7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:16.789319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.790364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.792302 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 909/1824] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:16 2022] DELETE /identity/v3/projects/ce651839d9ec4f42b83073e0134a6db7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:16.796417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:16.797957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:16.837977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f49a9-791b-4e78-9d18-9dfde94523de admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:16.892260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472ec32-0eb9-4957-ae37-29727cda5519 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:16.898204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:16.898733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:16.899061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:16.899481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] PATH_INFO: `/v3/users/96663d32565b4af795f80c58423c31ad` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:16.901248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=96663d32565b4af795f80c58423c31ad)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:16.903195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:16.912137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f96e0-6a70-47b4-87f2-69093beb1bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:16.913027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f96e0-6a70-47b4-87f2-69093beb1bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:16.913314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f96e0-6a70-47b4-87f2-69093beb1bee admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:16.913563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f96e0-6a70-47b4-87f2-69093beb1bee admin admin] PATH_INFO: `/v3/users/efc81a87adbf4b10a17eed2b8621b05c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:16.914767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f96e0-6a70-47b4-87f2-69093beb1bee admin admin] RBAC: Authorizing `identity:delete_user(user_id=efc81a87adbf4b10a17eed2b8621b05c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:16.916898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f96e0-6a70-47b4-87f2-69093beb1bee admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:16.978741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-458f96e0-6a70-47b4-87f2-69093beb1bee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efc81a87adbf4b10a17eed2b8621b05c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.979412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96663d32565b4af795f80c58423c31ad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.988168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-458f96e0-6a70-47b4-87f2-69093beb1bee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efc81a87adbf4b10a17eed2b8621b05c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.996885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-458f96e0-6a70-47b4-87f2-69093beb1bee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efc81a87adbf4b10a17eed2b8621b05c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:16.998903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96663d32565b4af795f80c58423c31ad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.000294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:17.011531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:17.014117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-458f96e0-6a70-47b4-87f2-69093beb1bee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efc81a87adbf4b10a17eed2b8621b05c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.015510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-458f96e0-6a70-47b4-87f2-69093beb1bee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efc81a87adbf4b10a17eed2b8621b05c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.019152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96663d32565b4af795f80c58423c31ad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.020027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96663d32565b4af795f80c58423c31ad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.020699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96663d32565b4af795f80c58423c31ad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.024774 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 916/1825] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:16 2022] DELETE /identity/v3/users/efc81a87adbf4b10a17eed2b8621b05c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:17.039582 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 910/1826] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:16 2022] DELETE /identity/v3/users/96663d32565b4af795f80c58423c31ad => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:17.333264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8dd04-14a3-4472-84f3-625d99b3a8b3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:17.333585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8dd04-14a3-4472-84f3-625d99b3a8b3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:17.333960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8dd04-14a3-4472-84f3-625d99b3a8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:17.358557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7bc8dd04-14a3-4472-84f3-625d99b3a8b3 None None] MFA Rules not processed for user `a3378bf4fa90489cb103378667f00399`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:17.359725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7bc8dd04-14a3-4472-84f3-625d99b3a8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:17.435467 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 917/1827] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:17 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:17.509541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:17.511357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:17.570327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8dd04-14a3-4472-84f3-625d99b3a8b3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:17.572409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7bc8dd04-14a3-4472-84f3-625d99b3a8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:17.599785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa421a-1c8e-4ec8-9e0d-72d397683c1b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:17.606057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2a05c-b49e-415c-992c-4d6421408ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:17.607359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2a05c-b49e-415c-992c-4d6421408ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:17.607635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2a05c-b49e-415c-992c-4d6421408ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:17.608114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2a05c-b49e-415c-992c-4d6421408ee2 admin admin] PATH_INFO: `/v3/projects/519e44ad07dc4153a8736e73ae1dd963` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:17.609473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2a05c-b49e-415c-992c-4d6421408ee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=519e44ad07dc4153a8736e73ae1dd963)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:17.611782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2a05c-b49e-415c-992c-4d6421408ee2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:17.670549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3b2a05c-b49e-415c-992c-4d6421408ee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '519e44ad07dc4153a8736e73ae1dd963', 'request_id': 'req-e3b2a05c-b49e-415c-992c-4d6421408ee2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.677688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8dd04-14a3-4472-84f3-625d99b3a8b3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:17.678185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3b2a05c-b49e-415c-992c-4d6421408ee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '519e44ad07dc4153a8736e73ae1dd963', 'request_id': 'req-e3b2a05c-b49e-415c-992c-4d6421408ee2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.687412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3b2a05c-b49e-415c-992c-4d6421408ee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '519e44ad07dc4153a8736e73ae1dd963', 'request_id': 'req-e3b2a05c-b49e-415c-992c-4d6421408ee2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.701990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3b2a05c-b49e-415c-992c-4d6421408ee2 admin admin] The token cache is being invalidate because project 519e44ad07dc4153a8736e73ae1dd963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:17.702659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3b2a05c-b49e-415c-992c-4d6421408ee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.703772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3b2a05c-b49e-415c-992c-4d6421408ee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.706148 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 911/1828] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:17 2022] DELETE /identity/v3/projects/519e44ad07dc4153a8736e73ae1dd963 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:17.711675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf840c1-298f-4f2f-90fd-8b7b1a13235d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:17.711966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf840c1-298f-4f2f-90fd-8b7b1a13235d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:17.712219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf840c1-298f-4f2f-90fd-8b7b1a13235d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:17.736702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9bf840c1-298f-4f2f-90fd-8b7b1a13235d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:17.738018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9bf840c1-298f-4f2f-90fd-8b7b1a13235d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:17.763389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d24f0-50ca-412d-9939-a7c4972fe8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:17.764509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d24f0-50ca-412d-9939-a7c4972fe8f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:17.765020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d24f0-50ca-412d-9939-a7c4972fe8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:17.765357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d24f0-50ca-412d-9939-a7c4972fe8f6 admin admin] PATH_INFO: `/v3/projects/5ee1b0e5dfab4110a164e7285698eee6` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:17.766969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d24f0-50ca-412d-9939-a7c4972fe8f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ee1b0e5dfab4110a164e7285698eee6)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:17.769479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d24f0-50ca-412d-9939-a7c4972fe8f6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:17.834749 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 912/1829] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:17.836751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24da29-4f7c-4647-9b54-b319f2c1a560 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:17.837029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24da29-4f7c-4647-9b54-b319f2c1a560 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:17.837291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24da29-4f7c-4647-9b54-b319f2c1a560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:17.843250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=5,keystone:SELECT=95 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:17.845684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a67d24f0-50ca-412d-9939-a7c4972fe8f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ee1b0e5dfab4110a164e7285698eee6', 'request_id': 'req-a67d24f0-50ca-412d-9939-a7c4972fe8f6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.848945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:17.856626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a67d24f0-50ca-412d-9939-a7c4972fe8f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ee1b0e5dfab4110a164e7285698eee6', 'request_id': 'req-a67d24f0-50ca-412d-9939-a7c4972fe8f6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.864917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a67d24f0-50ca-412d-9939-a7c4972fe8f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ee1b0e5dfab4110a164e7285698eee6', 'request_id': 'req-a67d24f0-50ca-412d-9939-a7c4972fe8f6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.875924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-df24da29-4f7c-4647-9b54-b319f2c1a560 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:17.877465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-df24da29-4f7c-4647-9b54-b319f2c1a560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:17.879790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a67d24f0-50ca-412d-9939-a7c4972fe8f6 admin admin] The token cache is being invalidate because project 5ee1b0e5dfab4110a164e7285698eee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:17.880148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a67d24f0-50ca-412d-9939-a7c4972fe8f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.882842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a67d24f0-50ca-412d-9939-a7c4972fe8f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:17.884732 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 918/1830] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:17 2022] DELETE /identity/v3/projects/5ee1b0e5dfab4110a164e7285698eee6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:17.890384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a1175-a711-4698-9a63-867c1ee29a52 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:17.890882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a1175-a711-4698-9a63-867c1ee29a52 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:17.891337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a1175-a711-4698-9a63-867c1ee29a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:17.915715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d97a1175-a711-4698-9a63-867c1ee29a52 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:17.917363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d97a1175-a711-4698-9a63-867c1ee29a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:17.961542 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 913/1831] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:17.967009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24da29-4f7c-4647-9b54-b319f2c1a560 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:17.982855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24da29-4f7c-4647-9b54-b319f2c1a560 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:17.990763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aef60c-ecf7-4e57-b527-1f7deb397d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:17.991666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aef60c-ecf7-4e57-b527-1f7deb397d4e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:17.991946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aef60c-ecf7-4e57-b527-1f7deb397d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:17.992202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aef60c-ecf7-4e57-b527-1f7deb397d4e admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:17.992934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aef60c-ecf7-4e57-b527-1f7deb397d4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:55:17.993394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aef60c-ecf7-4e57-b527-1f7deb397d4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:17.996204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aef60c-ecf7-4e57-b527-1f7deb397d4e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.008749 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 919/1832] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:18.015007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b530753-e6e0-4108-9d61-89ed5b052ac0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.015449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b530753-e6e0-4108-9d61-89ed5b052ac0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.015712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b530753-e6e0-4108-9d61-89ed5b052ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.022626 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 914/1833] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:55:18.030402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aef60c-ecf7-4e57-b527-1f7deb397d4e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.042164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5b530753-e6e0-4108-9d61-89ed5b052ac0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:18.043379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5b530753-e6e0-4108-9d61-89ed5b052ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:18.045319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aef60c-ecf7-4e57-b527-1f7deb397d4e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.051632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924fabb-c9a9-431a-8411-f639c92e17da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:18.052627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924fabb-c9a9-431a-8411-f639c92e17da admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.055082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924fabb-c9a9-431a-8411-f639c92e17da admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.055492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924fabb-c9a9-431a-8411-f639c92e17da admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.056220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924fabb-c9a9-431a-8411-f639c92e17da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:18.061996 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 920/1834] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:18.063138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b530753-e6e0-4108-9d61-89ed5b052ac0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.064350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924fabb-c9a9-431a-8411-f639c92e17da admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.074226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b530753-e6e0-4108-9d61-89ed5b052ac0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.096234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e867b9-23d7-4460-937d-fae93686b68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:18.097223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e867b9-23d7-4460-937d-fae93686b68a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.097736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e867b9-23d7-4460-937d-fae93686b68a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.098043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e867b9-23d7-4460-937d-fae93686b68a admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.099216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e867b9-23d7-4460-937d-fae93686b68a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:55:18.099559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e867b9-23d7-4460-937d-fae93686b68a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:18.101671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e867b9-23d7-4460-937d-fae93686b68a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.114018 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 915/1835] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:18 2022] POST /identity/v3/projects => generated 395 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:55:18.115643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924fabb-c9a9-431a-8411-f639c92e17da admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.120579 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 921/1836] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:55:18.125013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e867b9-23d7-4460-937d-fae93686b68a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.141314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e867b9-23d7-4460-937d-fae93686b68a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.212275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924fabb-c9a9-431a-8411-f639c92e17da admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.219258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748c5a9-85b1-4f13-8d6c-23abeee2d432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:18.220014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748c5a9-85b1-4f13-8d6c-23abeee2d432 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.220169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748c5a9-85b1-4f13-8d6c-23abeee2d432 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.220463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748c5a9-85b1-4f13-8d6c-23abeee2d432 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.221231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748c5a9-85b1-4f13-8d6c-23abeee2d432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:18.226912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748c5a9-85b1-4f13-8d6c-23abeee2d432 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.229213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa2041-f587-48e2-a986-527d17b65c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:18.230109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa2041-f587-48e2-a986-527d17b65c5e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.230734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa2041-f587-48e2-a986-527d17b65c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.231317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa2041-f587-48e2-a986-527d17b65c5e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.232268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa2041-f587-48e2-a986-527d17b65c5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:18.234565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa2041-f587-48e2-a986-527d17b65c5e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.264140 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 916/1837] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:18 2022] POST /identity/v3/users => generated 298 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:18.265630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0f86b-87b1-446c-8ed0-ca77d239e230 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.268075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0f86b-87b1-446c-8ed0-ca77d239e230 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.268075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0f86b-87b1-446c-8ed0-ca77d239e230 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.268075 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 917/1838] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:55:18.270027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0f86b-87b1-446c-8ed0-ca77d239e230 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.284395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0f86b-87b1-446c-8ed0-ca77d239e230 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.288529 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 922/1839] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:18 2022] POST /identity/v3/projects => generated 407 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:55:18.288529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db00bb6-be79-4946-83ad-5f7ac50ea3ae None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.288529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db00bb6-be79-4946-83ad-5f7ac50ea3ae None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.288529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db00bb6-be79-4946-83ad-5f7ac50ea3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.310745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5db00bb6-be79-4946-83ad-5f7ac50ea3ae None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:18.312174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5db00bb6-be79-4946-83ad-5f7ac50ea3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:18.361449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:18.366263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:18.370232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8509b-a331-4baa-947e-6c29d8c61d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:18.371207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8509b-a331-4baa-947e-6c29d8c61d00 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.371485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8509b-a331-4baa-947e-6c29d8c61d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.372258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8509b-a331-4baa-947e-6c29d8c61d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.373333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8509b-a331-4baa-947e-6c29d8c61d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:18.376055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8509b-a331-4baa-947e-6c29d8c61d00 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.385565 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 923/1840] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 20:55:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:18.386923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db00bb6-be79-4946-83ad-5f7ac50ea3ae None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.387280 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 918/1841] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:18.391202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8509b-a331-4baa-947e-6c29d8c61d00 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.404152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8509b-a331-4baa-947e-6c29d8c61d00 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.407705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db00bb6-be79-4946-83ad-5f7ac50ea3ae None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.410570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae317b-86ad-4063-9e4e-c5d9950c708f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:18.411199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae317b-86ad-4063-9e4e-c5d9950c708f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.411490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae317b-86ad-4063-9e4e-c5d9950c708f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.411798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae317b-86ad-4063-9e4e-c5d9950c708f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.412425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae317b-86ad-4063-9e4e-c5d9950c708f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:18.414050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae317b-86ad-4063-9e4e-c5d9950c708f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.416366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bbad4-9e8d-4dc1-8047-b291933164b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:18.417518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bbad4-9e8d-4dc1-8047-b291933164b0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.419855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bbad4-9e8d-4dc1-8047-b291933164b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.420144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bbad4-9e8d-4dc1-8047-b291933164b0 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.422226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bbad4-9e8d-4dc1-8047-b291933164b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:18.423284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bbad4-9e8d-4dc1-8047-b291933164b0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.433004 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 919/1842] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:18.435773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae317b-86ad-4063-9e4e-c5d9950c708f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.439482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5aae317b-86ad-4063-9e4e-c5d9950c708f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:18.452033 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 924/1843] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:18 2022] POST /identity/v3/users => generated 304 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:18.454294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bbad4-9e8d-4dc1-8047-b291933164b0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.459509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae317b-86ad-4063-9e4e-c5d9950c708f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.465981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430b260-8563-45e8-8a50-46ffad9e4222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:18.466207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430b260-8563-45e8-8a50-46ffad9e4222 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.466479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430b260-8563-45e8-8a50-46ffad9e4222 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.466747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430b260-8563-45e8-8a50-46ffad9e4222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.467855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bbad4-9e8d-4dc1-8047-b291933164b0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.477801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3b4bf-db53-4697-9a66-85a9273f992b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:18.479347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430b260-8563-45e8-8a50-46ffad9e4222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:18.479643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3b4bf-db53-4697-9a66-85a9273f992b admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.479876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3b4bf-db53-4697-9a66-85a9273f992b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.480411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3b4bf-db53-4697-9a66-85a9273f992b admin admin] PATH_INFO: `/v3/projects/dfe65efd22e2487fa097923a1e086dfc/users/92cefa53390047978d2b5de9e7556434/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.481049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430b260-8563-45e8-8a50-46ffad9e4222 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.492790 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 920/1844] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:18.493788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430b260-8563-45e8-8a50-46ffad9e4222 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.501921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3b4bf-db53-4697-9a66-85a9273f992b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfe65efd22e2487fa097923a1e086dfc, user_id=92cefa53390047978d2b5de9e7556434, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:18.508638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3b4bf-db53-4697-9a66-85a9273f992b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.510236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430b260-8563-45e8-8a50-46ffad9e4222 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.530435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0448-10ae-410e-8b47-45250d802938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:18.533266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0448-10ae-410e-8b47-45250d802938 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.533266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0448-10ae-410e-8b47-45250d802938 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.533266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0448-10ae-410e-8b47-45250d802938 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.533266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b0448-10ae-410e-8b47-45250d802938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:18.533803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b0448-10ae-410e-8b47-45250d802938 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.555211 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 925/1845] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:18 2022] PUT /identity/v3/projects/dfe65efd22e2487fa097923a1e086dfc/users/92cefa53390047978d2b5de9e7556434/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:18.557280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0a46d-e33f-404b-aa80-07e56f508211 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.557835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0a46d-e33f-404b-aa80-07e56f508211 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.558304 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 921/1846] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:18.558527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0a46d-e33f-404b-aa80-07e56f508211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.559524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0448-10ae-410e-8b47-45250d802938 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.572131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0448-10ae-410e-8b47-45250d802938 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.605946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-59e0a46d-e33f-404b-aa80-07e56f508211 None None] MFA Rules not processed for user `92cefa53390047978d2b5de9e7556434`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:18.607582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-59e0a46d-e33f-404b-aa80-07e56f508211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:18.660118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5baee-d134-4886-9f93-b2f13d918289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:18.662069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5baee-d134-4886-9f93-b2f13d918289 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.662707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5baee-d134-4886-9f93-b2f13d918289 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.663302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5baee-d134-4886-9f93-b2f13d918289 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.664312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5baee-d134-4886-9f93-b2f13d918289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:18.667019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5baee-d134-4886-9f93-b2f13d918289 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.695578 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 922/1847] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:18.701773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5baee-d134-4886-9f93-b2f13d918289 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.704154 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 926/1848] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:18 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:18.709095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0a46d-e33f-404b-aa80-07e56f508211 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.720494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5baee-d134-4886-9f93-b2f13d918289 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.723527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0a46d-e33f-404b-aa80-07e56f508211 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.728170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a0391-b208-429d-8096-0d3ba15698ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:18.729350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a0391-b208-429d-8096-0d3ba15698ca admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.729839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a0391-b208-429d-8096-0d3ba15698ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.730347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a0391-b208-429d-8096-0d3ba15698ca admin admin] PATH_INFO: `/v3/projects/25f24ca21e7a4097a26931975999fe68/users/7d30d0c627674cf58a5fec86d69e85b0/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.730687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df59b31-3d42-4555-9e96-44a6e5f3d1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:18.731352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df59b31-3d42-4555-9e96-44a6e5f3d1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.731621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df59b31-3d42-4555-9e96-44a6e5f3d1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.731881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df59b31-3d42-4555-9e96-44a6e5f3d1e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.732584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df59b31-3d42-4555-9e96-44a6e5f3d1e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:18.735380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df59b31-3d42-4555-9e96-44a6e5f3d1e5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.759445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a0391-b208-429d-8096-0d3ba15698ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f24ca21e7a4097a26931975999fe68, user_id=7d30d0c627674cf58a5fec86d69e85b0, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:18.765419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a0391-b208-429d-8096-0d3ba15698ca admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:18.776427 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 927/1849] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:18 2022] POST /identity/v3/projects => generated 393 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:18.781567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df59b31-3d42-4555-9e96-44a6e5f3d1e5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:18.811710 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 923/1850] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:18 2022] PUT /identity/v3/projects/25f24ca21e7a4097a26931975999fe68/users/7d30d0c627674cf58a5fec86d69e85b0/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:18.838185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39bbc3-eb62-46d5-a2b4-946303a86cde None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:18.838681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39bbc3-eb62-46d5-a2b4-946303a86cde None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:18.839257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39bbc3-eb62-46d5-a2b4-946303a86cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:18.905122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9b39bbc3-eb62-46d5-a2b4-946303a86cde None None] MFA Rules not processed for user `7d30d0c627674cf58a5fec86d69e85b0`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:18.906111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df59b31-3d42-4555-9e96-44a6e5f3d1e5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:18.909115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9b39bbc3-eb62-46d5-a2b4-946303a86cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:18.957734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:19.064288 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 924/1851] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:18 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:19.070493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39bbc3-eb62-46d5-a2b4-946303a86cde None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:19.072969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e062c4-91a9-45a3-b9f0-2234da9f4740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:19.073695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e062c4-91a9-45a3-b9f0-2234da9f4740 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.073993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e062c4-91a9-45a3-b9f0-2234da9f4740 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.074264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e062c4-91a9-45a3-b9f0-2234da9f4740 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.074993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e062c4-91a9-45a3-b9f0-2234da9f4740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:19.077197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e062c4-91a9-45a3-b9f0-2234da9f4740 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:19.090816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39bbc3-eb62-46d5-a2b4-946303a86cde None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:19.098599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682e80e-7845-4175-b212-94f14026e3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:19.099598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682e80e-7845-4175-b212-94f14026e3a2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.099906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682e80e-7845-4175-b212-94f14026e3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.100293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682e80e-7845-4175-b212-94f14026e3a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.101037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682e80e-7845-4175-b212-94f14026e3a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:19.103138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682e80e-7845-4175-b212-94f14026e3a2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:19.109418 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 928/1852] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:18 2022] POST /identity/v3/users => generated 303 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:19.111751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e062c4-91a9-45a3-b9f0-2234da9f4740 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:19.139706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e062c4-91a9-45a3-b9f0-2234da9f4740 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:19.150441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ac3e-1e97-41d4-a065-a405e614e764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:19.151706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ac3e-1e97-41d4-a065-a405e614e764 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.155723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ac3e-1e97-41d4-a065-a405e614e764 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.156242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:19.156417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ac3e-1e97-41d4-a065-a405e614e764 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.157190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1ac3e-1e97-41d4-a065-a405e614e764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:19.160647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1ac3e-1e97-41d4-a065-a405e614e764 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:19.169039 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 925/1853] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:19 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:55:19.174134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682e80e-7845-4175-b212-94f14026e3a2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:19.179654 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 929/1854] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:55:19.184430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ac3e-1e97-41d4-a065-a405e614e764 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:19.198264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ac3e-1e97-41d4-a065-a405e614e764 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:19.281745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f0656-b36a-45e9-86d7-6e222fcc4fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:19.282453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f0656-b36a-45e9-86d7-6e222fcc4fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.284167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f0656-b36a-45e9-86d7-6e222fcc4fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.285436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f0656-b36a-45e9-86d7-6e222fcc4fc8 admin admin] PATH_INFO: `/v3/projects/4ddaa9764826480592f1dd6255e88836/users/1607cdce313b44d8bba69fd486edc7e1/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.294155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682e80e-7845-4175-b212-94f14026e3a2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:19.299043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94bb07-8f2e-400a-9675-041693c01092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:19.299717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94bb07-8f2e-400a-9675-041693c01092 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.300067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94bb07-8f2e-400a-9675-041693c01092 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.300324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94bb07-8f2e-400a-9675-041693c01092 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.301416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94bb07-8f2e-400a-9675-041693c01092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:19.302953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94bb07-8f2e-400a-9675-041693c01092 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:19.309241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f0656-b36a-45e9-86d7-6e222fcc4fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ddaa9764826480592f1dd6255e88836, user_id=1607cdce313b44d8bba69fd486edc7e1, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:19.314481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f0656-b36a-45e9-86d7-6e222fcc4fc8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:19.333289 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 926/1855] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:19 2022] POST /identity/v3/users => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:55:19.337442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94bb07-8f2e-400a-9675-041693c01092 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:19.337745 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 930/1856] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:19 2022] PUT /identity/v3/projects/4ddaa9764826480592f1dd6255e88836/users/1607cdce313b44d8bba69fd486edc7e1/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:19.338685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f0656-b36a-45e9-86d7-6e222fcc4fc8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:19.350345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94bb07-8f2e-400a-9675-041693c01092 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:19.420303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f0656-b36a-45e9-86d7-6e222fcc4fc8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:19.425264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6235a9-4bb2-4695-9cb4-2aeb186577d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:19.425799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6235a9-4bb2-4695-9cb4-2aeb186577d2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.426066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6235a9-4bb2-4695-9cb4-2aeb186577d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.426327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6235a9-4bb2-4695-9cb4-2aeb186577d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.426986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6235a9-4bb2-4695-9cb4-2aeb186577d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:19.428835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6235a9-4bb2-4695-9cb4-2aeb186577d2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:19.431540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7b97d-f4d6-403f-9e1d-11f0b8374cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:19.432164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7b97d-f4d6-403f-9e1d-11f0b8374cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.432491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7b97d-f4d6-403f-9e1d-11f0b8374cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.432910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7b97d-f4d6-403f-9e1d-11f0b8374cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.434645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7b97d-f4d6-403f-9e1d-11f0b8374cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:19.436717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7b97d-f4d6-403f-9e1d-11f0b8374cc7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:19.442711 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 931/1857] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:19.447531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6235a9-4bb2-4695-9cb4-2aeb186577d2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:19.451698 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 927/1858] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:55:19.456817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7b97d-f4d6-403f-9e1d-11f0b8374cc7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:19.460485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6235a9-4bb2-4695-9cb4-2aeb186577d2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:19.465814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec599e-f222-4b39-8b1e-30eec7dcb2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:19.466568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec599e-f222-4b39-8b1e-30eec7dcb2ca admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.466847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec599e-f222-4b39-8b1e-30eec7dcb2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.467112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec599e-f222-4b39-8b1e-30eec7dcb2ca admin admin] PATH_INFO: `/v3/projects/4ddaa9764826480592f1dd6255e88836/users/1607cdce313b44d8bba69fd486edc7e1/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.469971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec599e-f222-4b39-8b1e-30eec7dcb2ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ddaa9764826480592f1dd6255e88836, user_id=1607cdce313b44d8bba69fd486edc7e1, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:19.472558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7b97d-f4d6-403f-9e1d-11f0b8374cc7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:19.473001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec599e-f222-4b39-8b1e-30eec7dcb2ca admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:19.480260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a54a2-cc8c-4874-913d-49c2d0fa0693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:19.480984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a54a2-cc8c-4874-913d-49c2d0fa0693 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.481309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a54a2-cc8c-4874-913d-49c2d0fa0693 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.481611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a54a2-cc8c-4874-913d-49c2d0fa0693 admin admin] PATH_INFO: `/v3/projects/c3fb7f3499a24028a93a66fb3ef836df/users/1d71cc59306e4061b2b80838a116e007/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.490967 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 932/1859] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:19 2022] PUT /identity/v3/projects/4ddaa9764826480592f1dd6255e88836/users/1607cdce313b44d8bba69fd486edc7e1/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:19.505373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d730180-cb55-4598-8b47-7c8480caeaff None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.505737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d730180-cb55-4598-8b47-7c8480caeaff None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.506068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d730180-cb55-4598-8b47-7c8480caeaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.509358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a54a2-cc8c-4874-913d-49c2d0fa0693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3fb7f3499a24028a93a66fb3ef836df, user_id=1d71cc59306e4061b2b80838a116e007, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:19.513135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a54a2-cc8c-4874-913d-49c2d0fa0693 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:19.539313 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 928/1860] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:19 2022] PUT /identity/v3/projects/c3fb7f3499a24028a93a66fb3ef836df/users/1d71cc59306e4061b2b80838a116e007/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:19.541779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:19.542895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a54a2-cc8c-4874-913d-49c2d0fa0693 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:19.561525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1d730180-cb55-4598-8b47-7c8480caeaff None None] MFA Rules not processed for user `1607cdce313b44d8bba69fd486edc7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:19.563209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1d730180-cb55-4598-8b47-7c8480caeaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:19.639398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a54a2-cc8c-4874-913d-49c2d0fa0693 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:19.646542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146b95f-f12e-47c0-a375-1f3b674c5457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:19.647192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146b95f-f12e-47c0-a375-1f3b674c5457 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.647432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146b95f-f12e-47c0-a375-1f3b674c5457 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.647693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146b95f-f12e-47c0-a375-1f3b674c5457 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.648405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146b95f-f12e-47c0-a375-1f3b674c5457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:19.650597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146b95f-f12e-47c0-a375-1f3b674c5457 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:19.652192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:19.664155 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 933/1861] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:19 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:19.667368 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 929/1862] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:19.672412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d730180-cb55-4598-8b47-7c8480caeaff None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:19.678818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146b95f-f12e-47c0-a375-1f3b674c5457 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:19.682932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1146b95f-f12e-47c0-a375-1f3b674c5457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:19.687446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d730180-cb55-4598-8b47-7c8480caeaff None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:19.694345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df69f54-fdc2-49cf-9bc2-c4bef3cc2227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:19.695418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df69f54-fdc2-49cf-9bc2-c4bef3cc2227 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.695695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df69f54-fdc2-49cf-9bc2-c4bef3cc2227 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.695996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df69f54-fdc2-49cf-9bc2-c4bef3cc2227 admin admin] PATH_INFO: `/v3/projects/c3fb7f3499a24028a93a66fb3ef836df/users/1d71cc59306e4061b2b80838a116e007/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.698628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df69f54-fdc2-49cf-9bc2-c4bef3cc2227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3fb7f3499a24028a93a66fb3ef836df, user_id=1d71cc59306e4061b2b80838a116e007, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:19.702084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df69f54-fdc2-49cf-9bc2-c4bef3cc2227 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:19.716121 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 934/1863] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:19 2022] PUT /identity/v3/projects/c3fb7f3499a24028a93a66fb3ef836df/users/1d71cc59306e4061b2b80838a116e007/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:19.727452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ad94d-7de2-451a-acad-289a20a0c1e6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.728240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ad94d-7de2-451a-acad-289a20a0c1e6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.728623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ad94d-7de2-451a-acad-289a20a0c1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.775300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-166ad94d-7de2-451a-acad-289a20a0c1e6 None None] MFA Rules not processed for user `1d71cc59306e4061b2b80838a116e007`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:19.776905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-166ad94d-7de2-451a-acad-289a20a0c1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:19.793141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146b95f-f12e-47c0-a375-1f3b674c5457 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:19.872252 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 935/1864] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:19 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:19.884301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ad94d-7de2-451a-acad-289a20a0c1e6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:19.898176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020162e-4d75-47da-98e6-3df9b861daeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:19.901021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020162e-4d75-47da-98e6-3df9b861daeb service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.901462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020162e-4d75-47da-98e6-3df9b861daeb service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.901908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020162e-4d75-47da-98e6-3df9b861daeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.972131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ad94d-7de2-451a-acad-289a20a0c1e6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:19.979547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:19.980634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:19.981000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:19.981362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] PATH_INFO: `/v3/users/7d30d0c627674cf58a5fec86d69e85b0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:19.982741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d30d0c627674cf58a5fec86d69e85b0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:19.985247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:19.992951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:20.017878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020162e-4d75-47da-98e6-3df9b861daeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:20.019943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020162e-4d75-47da-98e6-3df9b861daeb service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:20.044110 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 930/1865] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:19 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:20.057423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d30d0c627674cf58a5fec86d69e85b0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:20.073960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d30d0c627674cf58a5fec86d69e85b0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:20.086159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d30d0c627674cf58a5fec86d69e85b0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:20.104508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d30d0c627674cf58a5fec86d69e85b0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:20.106218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d30d0c627674cf58a5fec86d69e85b0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:20.117550 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 936/1866] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:19 2022] DELETE /identity/v3/users/7d30d0c627674cf58a5fec86d69e85b0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:20.119386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b10c9-bd12-4974-9a50-d670bac7d1f0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:20.119712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b10c9-bd12-4974-9a50-d670bac7d1f0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:20.119972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b10c9-bd12-4974-9a50-d670bac7d1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:20.153313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-358b10c9-bd12-4974-9a50-d670bac7d1f0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:20.155219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-358b10c9-bd12-4974-9a50-d670bac7d1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:20.206396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:20.226879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:20.228280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:20.296614 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 931/1867] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:20.298823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b10c9-bd12-4974-9a50-d670bac7d1f0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:20.302660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-358b10c9-bd12-4974-9a50-d670bac7d1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:20.362917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589bddc-1e9f-491f-8e5f-83efc4ee803d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:20.374121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:20.375602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:20.376035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:20.376418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] PATH_INFO: `/v3/users/92cefa53390047978d2b5de9e7556434` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:20.378609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] RBAC: Authorizing `identity:delete_user(user_id=92cefa53390047978d2b5de9e7556434)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:20.382610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:20.396616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b10c9-bd12-4974-9a50-d670bac7d1f0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:20.402868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae822d4a-7364-4bfe-9352-08f802d508d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:20.403811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae822d4a-7364-4bfe-9352-08f802d508d2 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:20.404229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae822d4a-7364-4bfe-9352-08f802d508d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:20.404631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae822d4a-7364-4bfe-9352-08f802d508d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:20.416583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae822d4a-7364-4bfe-9352-08f802d508d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:20.418576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae822d4a-7364-4bfe-9352-08f802d508d2 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:20.439886 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 932/1868] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:20.481758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92cefa53390047978d2b5de9e7556434'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:20.494766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92cefa53390047978d2b5de9e7556434'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:20.511352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92cefa53390047978d2b5de9e7556434'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:20.530282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92cefa53390047978d2b5de9e7556434'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:20.530557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92cefa53390047978d2b5de9e7556434'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:20.537959 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 937/1869] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:20 2022] DELETE /identity/v3/users/92cefa53390047978d2b5de9e7556434 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:20.543089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000978f-2839-4ebb-9422-e4011e79df7e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:20.543361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000978f-2839-4ebb-9422-e4011e79df7e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:20.543650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000978f-2839-4ebb-9422-e4011e79df7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:20.575978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7000978f-2839-4ebb-9422-e4011e79df7e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:20.578268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7000978f-2839-4ebb-9422-e4011e79df7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:20.678803 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 933/1870] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:20.699633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:20.701386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:20.708984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:20.841187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a4a06-bbd4-4458-8eeb-9f61103ddadd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:20.845367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176d290-ddac-452f-87f0-12389b36a25c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:20.846090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176d290-ddac-452f-87f0-12389b36a25c service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:20.846492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176d290-ddac-452f-87f0-12389b36a25c service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:20.846841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176d290-ddac-452f-87f0-12389b36a25c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:20.862719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176d290-ddac-452f-87f0-12389b36a25c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:20.864765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176d290-ddac-452f-87f0-12389b36a25c service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:20.884776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000978f-2839-4ebb-9422-e4011e79df7e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:20.886690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7000978f-2839-4ebb-9422-e4011e79df7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:20.915172 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 938/1871] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:20.918676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000978f-2839-4ebb-9422-e4011e79df7e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:20.927895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:20.928834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:20.929102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:20.929365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] PATH_INFO: `/v3/projects/25f24ca21e7a4097a26931975999fe68` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:20.944559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] RBAC: Authorizing `identity:delete_project(project_id=25f24ca21e7a4097a26931975999fe68)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:20.948587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:21.034870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25f24ca21e7a4097a26931975999fe68', 'request_id': 'req-c43b0350-fe59-4795-9832-823df1cf242a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.046998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25f24ca21e7a4097a26931975999fe68', 'request_id': 'req-c43b0350-fe59-4795-9832-823df1cf242a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.076022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25f24ca21e7a4097a26931975999fe68', 'request_id': 'req-c43b0350-fe59-4795-9832-823df1cf242a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.086234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:21.106395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] The token cache is being invalidate because project 25f24ca21e7a4097a26931975999fe68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:21.107074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.107337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.109443 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 934/1872] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:20 2022] DELETE /identity/v3/projects/25f24ca21e7a4097a26931975999fe68 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:21.115591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176d290-ddac-452f-87f0-12389b36a25c service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:21.131537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176d290-ddac-452f-87f0-12389b36a25c service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:21.242854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:21.243529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:21.243799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:21.244747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] PATH_INFO: `/v3/users/1d71cc59306e4061b2b80838a116e007` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:21.246414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d71cc59306e4061b2b80838a116e007)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:21.248400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:21.319143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d71cc59306e4061b2b80838a116e007'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.331610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d71cc59306e4061b2b80838a116e007'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.345468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d71cc59306e4061b2b80838a116e007'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.363712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d71cc59306e4061b2b80838a116e007'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.364716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d71cc59306e4061b2b80838a116e007'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.373251 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 939/1873] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:21 2022] DELETE /identity/v3/users/1d71cc59306e4061b2b80838a116e007 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:21.386339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:21.387926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:21.408112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b0350-fe59-4795-9832-823df1cf242a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:21.510449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:21.511114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:21.511388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:21.511655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] PATH_INFO: `/v3/projects/dfe65efd22e2487fa097923a1e086dfc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:21.512833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfe65efd22e2487fa097923a1e086dfc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:21.514923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:21.578362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfe65efd22e2487fa097923a1e086dfc', 'request_id': 'req-e3825dca-0825-4170-b11a-f604402ed29c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.585956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfe65efd22e2487fa097923a1e086dfc', 'request_id': 'req-e3825dca-0825-4170-b11a-f604402ed29c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.596028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfe65efd22e2487fa097923a1e086dfc', 'request_id': 'req-e3825dca-0825-4170-b11a-f604402ed29c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.614551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] The token cache is being invalidate because project dfe65efd22e2487fa097923a1e086dfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:21.615553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.615553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.617343 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 935/1874] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:21 2022] DELETE /identity/v3/projects/dfe65efd22e2487fa097923a1e086dfc => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:21.621366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:21.622988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:21.639598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80b215-a08c-4d4c-85d3-3c138e073e92 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:21.667917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:21.709223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6102bf-065d-423c-a1b9-855a3c1f6576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:21.710302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6102bf-065d-423c-a1b9-855a3c1f6576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:21.710302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6102bf-065d-423c-a1b9-855a3c1f6576 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:21.710563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6102bf-065d-423c-a1b9-855a3c1f6576 admin admin] PATH_INFO: `/v3/users/1607cdce313b44d8bba69fd486edc7e1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:21.712221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6102bf-065d-423c-a1b9-855a3c1f6576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1607cdce313b44d8bba69fd486edc7e1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:21.714173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6102bf-065d-423c-a1b9-855a3c1f6576 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:21.727614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:21.729131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:21.753432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3825dca-0825-4170-b11a-f604402ed29c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:21.762789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2093c73-9803-4be8-8321-76e84e94213f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:21.763489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2093c73-9803-4be8-8321-76e84e94213f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:21.763866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2093c73-9803-4be8-8321-76e84e94213f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:21.764240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2093c73-9803-4be8-8321-76e84e94213f admin admin] PATH_INFO: `/v3/projects/c3fb7f3499a24028a93a66fb3ef836df` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:21.766224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2093c73-9803-4be8-8321-76e84e94213f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3fb7f3499a24028a93a66fb3ef836df)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:21.768308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2093c73-9803-4be8-8321-76e84e94213f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:21.798858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1b6102bf-065d-423c-a1b9-855a3c1f6576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1607cdce313b44d8bba69fd486edc7e1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.817247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1b6102bf-065d-423c-a1b9-855a3c1f6576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1607cdce313b44d8bba69fd486edc7e1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.832298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1b6102bf-065d-423c-a1b9-855a3c1f6576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1607cdce313b44d8bba69fd486edc7e1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.849592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1b6102bf-065d-423c-a1b9-855a3c1f6576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1607cdce313b44d8bba69fd486edc7e1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.850953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1b6102bf-065d-423c-a1b9-855a3c1f6576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1607cdce313b44d8bba69fd486edc7e1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.869963 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 940/1875] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:21 2022] DELETE /identity/v3/users/1607cdce313b44d8bba69fd486edc7e1 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:21.899471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b2093c73-9803-4be8-8321-76e84e94213f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3fb7f3499a24028a93a66fb3ef836df', 'request_id': 'req-b2093c73-9803-4be8-8321-76e84e94213f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.907342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b2093c73-9803-4be8-8321-76e84e94213f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3fb7f3499a24028a93a66fb3ef836df', 'request_id': 'req-b2093c73-9803-4be8-8321-76e84e94213f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.919838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b2093c73-9803-4be8-8321-76e84e94213f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3fb7f3499a24028a93a66fb3ef836df', 'request_id': 'req-b2093c73-9803-4be8-8321-76e84e94213f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.941172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b2093c73-9803-4be8-8321-76e84e94213f admin admin] The token cache is being invalidate because project c3fb7f3499a24028a93a66fb3ef836df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:21.941799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b2093c73-9803-4be8-8321-76e84e94213f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.942799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b2093c73-9803-4be8-8321-76e84e94213f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:21.946358 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 936/1876] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:21 2022] DELETE /identity/v3/projects/c3fb7f3499a24028a93a66fb3ef836df => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:21.952428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c07613-620e-49b3-9b33-4dbfa03b16b5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:21.952711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c07613-620e-49b3-9b33-4dbfa03b16b5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:21.952978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c07613-620e-49b3-9b33-4dbfa03b16b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:21.979790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-45c07613-620e-49b3-9b33-4dbfa03b16b5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:21.982118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-45c07613-620e-49b3-9b33-4dbfa03b16b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:22.077232 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 941/1877] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:22.086551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f9b0f-0c3e-4fea-a63d-ed0962cf9995 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.086929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f9b0f-0c3e-4fea-a63d-ed0962cf9995 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.087198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f9b0f-0c3e-4fea-a63d-ed0962cf9995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.113138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:22.129730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-af6f9b0f-0c3e-4fea-a63d-ed0962cf9995 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:22.131353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-af6f9b0f-0c3e-4fea-a63d-ed0962cf9995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:22.147879 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 937/1878] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:22.155603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c07613-620e-49b3-9b33-4dbfa03b16b5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:22.176726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c07613-620e-49b3-9b33-4dbfa03b16b5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:22.182270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6b081-9cc3-4a2c-be06-b4eb77f5cbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:22.183492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6b081-9cc3-4a2c-be06-b4eb77f5cbef admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.183772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6b081-9cc3-4a2c-be06-b4eb77f5cbef admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.184039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6b081-9cc3-4a2c-be06-b4eb77f5cbef admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.184742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6b081-9cc3-4a2c-be06-b4eb77f5cbef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:55:22.185041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6b081-9cc3-4a2c-be06-b4eb77f5cbef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:22.186990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6b081-9cc3-4a2c-be06-b4eb77f5cbef admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:22.210614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f9b0f-0c3e-4fea-a63d-ed0962cf9995 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:22.217905 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 942/1879] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:55:22.239558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f9b0f-0c3e-4fea-a63d-ed0962cf9995 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:22.250288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116ac38-0893-4da8-8356-4d06617b1a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:22.251958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116ac38-0893-4da8-8356-4d06617b1a8d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.252269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116ac38-0893-4da8-8356-4d06617b1a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.252635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116ac38-0893-4da8-8356-4d06617b1a8d admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.253948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116ac38-0893-4da8-8356-4d06617b1a8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:22.260956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116ac38-0893-4da8-8356-4d06617b1a8d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:22.289971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6b081-9cc3-4a2c-be06-b4eb77f5cbef admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:22.292415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-55b6b081-9cc3-4a2c-be06-b4eb77f5cbef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:22.309419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6b081-9cc3-4a2c-be06-b4eb77f5cbef admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:22.312409 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 938/1880] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:22 2022] POST /identity/v3/projects => generated 397 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:22.314290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116ac38-0893-4da8-8356-4d06617b1a8d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:22.358780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:22.401091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116ac38-0893-4da8-8356-4d06617b1a8d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:22.408982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff538c-b920-4b30-89b9-5cd9be37ef7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:22.409966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff538c-b920-4b30-89b9-5cd9be37ef7d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.410382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff538c-b920-4b30-89b9-5cd9be37ef7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.410850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff538c-b920-4b30-89b9-5cd9be37ef7d admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.411835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff538c-b920-4b30-89b9-5cd9be37ef7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:22.414697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:22.415424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.415689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.415958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf admin admin] PATH_INFO: `/v3/projects/4ddaa9764826480592f1dd6255e88836` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.417555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ddaa9764826480592f1dd6255e88836)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:22.420039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:22.428802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff538c-b920-4b30-89b9-5cd9be37ef7d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:22.471195 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 939/1881] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:22 2022] POST /identity/v3/users => generated 299 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:55:22.471195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a18cd-e978-4374-9b37-8b8700b54628 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.471195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a18cd-e978-4374-9b37-8b8700b54628 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.471195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a18cd-e978-4374-9b37-8b8700b54628 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.472514 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 940/1882] 10.222.0.23 () {54 vars in 921 bytes} [Sat Aug 20 20:55:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:55:22.473985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a18cd-e978-4374-9b37-8b8700b54628 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:22.492594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a18cd-e978-4374-9b37-8b8700b54628 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:22.502762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ddaa9764826480592f1dd6255e88836', 'request_id': 'req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:22.516071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ddaa9764826480592f1dd6255e88836', 'request_id': 'req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:22.543982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ddaa9764826480592f1dd6255e88836', 'request_id': 'req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:22.575516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf admin admin] The token cache is being invalidate because project 4ddaa9764826480592f1dd6255e88836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:22.575940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:22.577455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8c72c0b1-fcf6-4aac-a30f-a991df4f3dbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:22.580249 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 943/1883] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:22 2022] DELETE /identity/v3/projects/4ddaa9764826480592f1dd6255e88836 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:22.582034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c7f0f-9121-4aab-8d84-5009af3afb8b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.582662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c7f0f-9121-4aab-8d84-5009af3afb8b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.582991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c7f0f-9121-4aab-8d84-5009af3afb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.599788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9cd0c-78c7-4272-9bb3-2c0d8ea6a14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:22.600605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9cd0c-78c7-4272-9bb3-2c0d8ea6a14b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.600980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9cd0c-78c7-4272-9bb3-2c0d8ea6a14b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.601347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9cd0c-78c7-4272-9bb3-2c0d8ea6a14b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.602103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9cd0c-78c7-4272-9bb3-2c0d8ea6a14b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:22.603821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9cd0c-78c7-4272-9bb3-2c0d8ea6a14b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:22.622835 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 941/1884] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:22.625111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8a3af-3d1c-4ca9-bd22-a07e54b3a8b8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.625349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d78c7f0f-9121-4aab-8d84-5009af3afb8b None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:22.625572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8a3af-3d1c-4ca9-bd22-a07e54b3a8b8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.625938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8a3af-3d1c-4ca9-bd22-a07e54b3a8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.627041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d78c7f0f-9121-4aab-8d84-5009af3afb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:22.651161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2fb8a3af-3d1c-4ca9-bd22-a07e54b3a8b8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:22.652683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2fb8a3af-3d1c-4ca9-bd22-a07e54b3a8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:22.663521 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 942/1885] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:22.664821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8a3af-3d1c-4ca9-bd22-a07e54b3a8b8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:22.666555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2fb8a3af-3d1c-4ca9-bd22-a07e54b3a8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:22.682313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8a3af-3d1c-4ca9-bd22-a07e54b3a8b8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:22.687733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c34a8-33ce-4ea2-9bd2-7a06badbaf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:22.688669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c34a8-33ce-4ea2-9bd2-7a06badbaf3b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.689148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c34a8-33ce-4ea2-9bd2-7a06badbaf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.689685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c34a8-33ce-4ea2-9bd2-7a06badbaf3b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.690567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c34a8-33ce-4ea2-9bd2-7a06badbaf3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:22.692556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c34a8-33ce-4ea2-9bd2-7a06badbaf3b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:22.708330 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 943/1886] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:22.709938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc337f-a1d6-4d49-b84d-528bafe0e9f4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.710215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc337f-a1d6-4d49-b84d-528bafe0e9f4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.710486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc337f-a1d6-4d49-b84d-528bafe0e9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.716093 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 944/1887] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 20:55:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:22.717239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c7f0f-9121-4aab-8d84-5009af3afb8b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:22.729425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c7f0f-9121-4aab-8d84-5009af3afb8b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:22.735841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74ec33-79a8-49ac-a3a9-f0ca2a41ffd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:22.736599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-79cc337f-a1d6-4d49-b84d-528bafe0e9f4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:22.736900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74ec33-79a8-49ac-a3a9-f0ca2a41ffd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.737283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74ec33-79a8-49ac-a3a9-f0ca2a41ffd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.737628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74ec33-79a8-49ac-a3a9-f0ca2a41ffd7 admin admin] PATH_INFO: `/v3/projects/54cdd4799ffa49d5bcca23b8d5cbd2b2/users/a3f7a1130dff44708fc03a7c5757efa7/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.738747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-79cc337f-a1d6-4d49-b84d-528bafe0e9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:22.748766 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 944/1888] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:22.753836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc337f-a1d6-4d49-b84d-528bafe0e9f4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:22.758229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d74ec33-79a8-49ac-a3a9-f0ca2a41ffd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54cdd4799ffa49d5bcca23b8d5cbd2b2, user_id=a3f7a1130dff44708fc03a7c5757efa7, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:22.761922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d74ec33-79a8-49ac-a3a9-f0ca2a41ffd7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:22.766846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc337f-a1d6-4d49-b84d-528bafe0e9f4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:22.775305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c3955-8bd0-4c10-a264-2d964c8ed0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:22.776161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c3955-8bd0-4c10-a264-2d964c8ed0be admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.776476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c3955-8bd0-4c10-a264-2d964c8ed0be admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.777430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c3955-8bd0-4c10-a264-2d964c8ed0be admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.778241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c3955-8bd0-4c10-a264-2d964c8ed0be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:55:22.778652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c3955-8bd0-4c10-a264-2d964c8ed0be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:22.780322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c3955-8bd0-4c10-a264-2d964c8ed0be admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:22.783480 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 945/1889] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:22 2022] PUT /identity/v3/projects/54cdd4799ffa49d5bcca23b8d5cbd2b2/users/a3f7a1130dff44708fc03a7c5757efa7/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:22.789387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df9274-5a6f-4f57-bc99-4124ccaf7969 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.790477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df9274-5a6f-4f57-bc99-4124ccaf7969 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.790477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df9274-5a6f-4f57-bc99-4124ccaf7969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.804261 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 945/1890] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:55:22.809125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c3955-8bd0-4c10-a264-2d964c8ed0be admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:22.814195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-53df9274-5a6f-4f57-bc99-4124ccaf7969 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:22.815644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-53df9274-5a6f-4f57-bc99-4124ccaf7969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:22.833837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c3955-8bd0-4c10-a264-2d964c8ed0be admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:22.879921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:22.899770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:22.961256 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 946/1891] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:22.965495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dcb76-414f-40c1-a4eb-64c3308f41d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:22.966122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dcb76-414f-40c1-a4eb-64c3308f41d6 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:22.966397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dcb76-414f-40c1-a4eb-64c3308f41d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:22.966664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dcb76-414f-40c1-a4eb-64c3308f41d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:22.967296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dcb76-414f-40c1-a4eb-64c3308f41d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:22.969252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dcb76-414f-40c1-a4eb-64c3308f41d6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:22.970811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df9274-5a6f-4f57-bc99-4124ccaf7969 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:22.974209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-53df9274-5a6f-4f57-bc99-4124ccaf7969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:23.017604 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 946/1892] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:22 2022] POST /identity/v3/projects => generated 405 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:55:23.018696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dcb76-414f-40c1-a4eb-64c3308f41d6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:23.025187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b97dcb76-414f-40c1-a4eb-64c3308f41d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:23.091466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df9274-5a6f-4f57-bc99-4124ccaf7969 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:23.121982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dcb76-414f-40c1-a4eb-64c3308f41d6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:23.125807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b7a06-2c8f-44f0-9f0d-9d5d236a1d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:23.126422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b7a06-2c8f-44f0-9f0d-9d5d236a1d10 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:23.126740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b7a06-2c8f-44f0-9f0d-9d5d236a1d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:23.127006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b7a06-2c8f-44f0-9f0d-9d5d236a1d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:23.179955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b34b5-76b6-46a8-8fce-14e2d0d1d1e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:23.180627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b34b5-76b6-46a8-8fce-14e2d0d1d1e6 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:23.181094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b34b5-76b6-46a8-8fce-14e2d0d1d1e6 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:23.181365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b34b5-76b6-46a8-8fce-14e2d0d1d1e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:23.192292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b34b5-76b6-46a8-8fce-14e2d0d1d1e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:23.194251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b34b5-76b6-46a8-8fce-14e2d0d1d1e6 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:23.207339 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 947/1893] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:23.208461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b34b5-76b6-46a8-8fce-14e2d0d1d1e6 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:23.211100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b7a06-2c8f-44f0-9f0d-9d5d236a1d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:23.212766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b7a06-2c8f-44f0-9f0d-9d5d236a1d10 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:23.221774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b34b5-76b6-46a8-8fce-14e2d0d1d1e6 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:23.227485 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 947/1894] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:23.259535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97a66f-9e63-4906-9901-9bdd6fc29725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:23.260514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97a66f-9e63-4906-9901-9bdd6fc29725 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:23.260838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97a66f-9e63-4906-9901-9bdd6fc29725 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:23.261215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97a66f-9e63-4906-9901-9bdd6fc29725 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:23.261928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97a66f-9e63-4906-9901-9bdd6fc29725 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:23.264296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97a66f-9e63-4906-9901-9bdd6fc29725 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:23.300543 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 948/1895] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:23 2022] POST /identity/v3/users => generated 303 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:23.303666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b7a06-2c8f-44f0-9f0d-9d5d236a1d10 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:23.322314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b7a06-2c8f-44f0-9f0d-9d5d236a1d10 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:23.330192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630c7c4-6a13-4c92-843f-73b30ba9522a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:23.330935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630c7c4-6a13-4c92-843f-73b30ba9522a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:23.331382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630c7c4-6a13-4c92-843f-73b30ba9522a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:23.332263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630c7c4-6a13-4c92-843f-73b30ba9522a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:23.333172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630c7c4-6a13-4c92-843f-73b30ba9522a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:23.335496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630c7c4-6a13-4c92-843f-73b30ba9522a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:23.355804 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 948/1896] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:55:23.357790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97a66f-9e63-4906-9901-9bdd6fc29725 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:23.371897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97a66f-9e63-4906-9901-9bdd6fc29725 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:23.376999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df09bdc-c581-4920-8130-a2518c7c6f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:23.377784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df09bdc-c581-4920-8130-a2518c7c6f22 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:23.378814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df09bdc-c581-4920-8130-a2518c7c6f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:23.378814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df09bdc-c581-4920-8130-a2518c7c6f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:23.379226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df09bdc-c581-4920-8130-a2518c7c6f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:23.380968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df09bdc-c581-4920-8130-a2518c7c6f22 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:23.393358 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 949/1897] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:23.397844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630c7c4-6a13-4c92-843f-73b30ba9522a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:23.410492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630c7c4-6a13-4c92-843f-73b30ba9522a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:23.416297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982713d-e64d-43a9-9f90-28bf95e78287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:23.417372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982713d-e64d-43a9-9f90-28bf95e78287 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:23.417803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982713d-e64d-43a9-9f90-28bf95e78287 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:23.418217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982713d-e64d-43a9-9f90-28bf95e78287 admin admin] PATH_INFO: `/v3/projects/7f2a18b59cac4d7385a897ef2695b826/users/b9ac6652707a4819a2b1229bd08ca524/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:23.436553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982713d-e64d-43a9-9f90-28bf95e78287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f2a18b59cac4d7385a897ef2695b826, user_id=b9ac6652707a4819a2b1229bd08ca524, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:23.439389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982713d-e64d-43a9-9f90-28bf95e78287 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:23.463032 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 949/1898] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:23 2022] PUT /identity/v3/projects/7f2a18b59cac4d7385a897ef2695b826/users/b9ac6652707a4819a2b1229bd08ca524/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:23.465205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28199db4-fbe5-47b1-942d-fe5bdad1c9c8 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:23.465548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28199db4-fbe5-47b1-942d-fe5bdad1c9c8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:23.465962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28199db4-fbe5-47b1-942d-fe5bdad1c9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:23.487869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-28199db4-fbe5-47b1-942d-fe5bdad1c9c8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:23.490500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-28199db4-fbe5-47b1-942d-fe5bdad1c9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:23.515943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:23.592459 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 950/1899] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:23.620517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982713d-e64d-43a9-9f90-28bf95e78287 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:23.658786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:23.695656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982713d-e64d-43a9-9f90-28bf95e78287 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:23.700823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20552b9a-0234-43b6-b77e-60c100fb42ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:23.701761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20552b9a-0234-43b6-b77e-60c100fb42ac service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:23.702213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20552b9a-0234-43b6-b77e-60c100fb42ac service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:23.702607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20552b9a-0234-43b6-b77e-60c100fb42ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:23.715526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20552b9a-0234-43b6-b77e-60c100fb42ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:23.718734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20552b9a-0234-43b6-b77e-60c100fb42ac service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:23.734953 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 950/1900] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:29.294729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28199db4-fbe5-47b1-942d-fe5bdad1c9c8 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:29.296763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-28199db4-fbe5-47b1-942d-fe5bdad1c9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:29.430730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28199db4-fbe5-47b1-942d-fe5bdad1c9c8 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:29.437739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b34484-a601-4302-ac40-893650c71d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:29.438646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b34484-a601-4302-ac40-893650c71d77 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:29.438765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b34484-a601-4302-ac40-893650c71d77 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:29.439451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b34484-a601-4302-ac40-893650c71d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:29.441552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a2b34484-a601-4302-ac40-893650c71d77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:29.563630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b34484-a601-4302-ac40-893650c71d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:29.566549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b34484-a601-4302-ac40-893650c71d77 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:29.598895 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 951/1901] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:29 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:31.935623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43c7ab-fb30-4ee0-b440-5eac957b8128 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:31.937977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43c7ab-fb30-4ee0-b440-5eac957b8128 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:31.938668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43c7ab-fb30-4ee0-b440-5eac957b8128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:31.996939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3c43c7ab-fb30-4ee0-b440-5eac957b8128 None None] MFA Rules not processed for user `b9ac6652707a4819a2b1229bd08ca524`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:31.998387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3c43c7ab-fb30-4ee0-b440-5eac957b8128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:32.100332 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 951/1902] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:31 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:32.103410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b34484-a601-4302-ac40-893650c71d77 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:32.117731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b34484-a601-4302-ac40-893650c71d77 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:32.124600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b15ab-d8fd-47c6-8687-bc3338aab3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:32.125278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b15ab-d8fd-47c6-8687-bc3338aab3dc admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:32.125598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b15ab-d8fd-47c6-8687-bc3338aab3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:32.125888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b15ab-d8fd-47c6-8687-bc3338aab3dc admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:32.126593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b15ab-d8fd-47c6-8687-bc3338aab3dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:32.129208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b15ab-d8fd-47c6-8687-bc3338aab3dc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:32.208870 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 952/1903] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:32 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:32.210147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43c7ab-fb30-4ee0-b440-5eac957b8128 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:32.242677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43c7ab-fb30-4ee0-b440-5eac957b8128 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:32.359089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf99199-b5aa-4f8c-aaec-57d4a0ddad8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:32.360329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf99199-b5aa-4f8c-aaec-57d4a0ddad8f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:32.361714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf99199-b5aa-4f8c-aaec-57d4a0ddad8f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:32.362181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf99199-b5aa-4f8c-aaec-57d4a0ddad8f admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:32.363073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf99199-b5aa-4f8c-aaec-57d4a0ddad8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:32.366517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf99199-b5aa-4f8c-aaec-57d4a0ddad8f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:32.417418 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 952/1904] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:32 2022] POST /identity/v3/users => generated 309 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:32.421034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b15ab-d8fd-47c6-8687-bc3338aab3dc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:32.438351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b15ab-d8fd-47c6-8687-bc3338aab3dc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:32.454030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120c2bb-1c05-4ec1-bcde-607d92042564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:32.455155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c2bb-1c05-4ec1-bcde-607d92042564 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:32.455640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c2bb-1c05-4ec1-bcde-607d92042564 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:32.456878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c2bb-1c05-4ec1-bcde-607d92042564 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:32.457546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4120c2bb-1c05-4ec1-bcde-607d92042564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:32.465824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4120c2bb-1c05-4ec1-bcde-607d92042564 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:32.499884 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 953/1905] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:32.501373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf99199-b5aa-4f8c-aaec-57d4a0ddad8f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:32.520023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf99199-b5aa-4f8c-aaec-57d4a0ddad8f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:32.537263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165947ad-fd0e-4a9e-806c-79bb41ed477f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:32.540267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165947ad-fd0e-4a9e-806c-79bb41ed477f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:32.540580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165947ad-fd0e-4a9e-806c-79bb41ed477f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:32.540879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165947ad-fd0e-4a9e-806c-79bb41ed477f admin admin] PATH_INFO: `/v3/projects/4019258611094cbf979dece7390ba297/users/d83156b45fed412f8506ef2d8af25339/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:32.596827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165947ad-fd0e-4a9e-806c-79bb41ed477f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4019258611094cbf979dece7390ba297, user_id=d83156b45fed412f8506ef2d8af25339, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:32.599680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165947ad-fd0e-4a9e-806c-79bb41ed477f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:32.639015 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 953/1906] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:32 2022] PUT /identity/v3/projects/4019258611094cbf979dece7390ba297/users/d83156b45fed412f8506ef2d8af25339/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:32.649182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120c2bb-1c05-4ec1-bcde-607d92042564 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:32.706266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120c2bb-1c05-4ec1-bcde-607d92042564 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:32.772302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:32.812508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08870c59-45de-48d8-8d0e-e2eaa87ec666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:32.813258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08870c59-45de-48d8-8d0e-e2eaa87ec666 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:32.813837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08870c59-45de-48d8-8d0e-e2eaa87ec666 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:32.816669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08870c59-45de-48d8-8d0e-e2eaa87ec666 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:32.817453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08870c59-45de-48d8-8d0e-e2eaa87ec666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:32.819144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08870c59-45de-48d8-8d0e-e2eaa87ec666 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:32.831716 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 954/1907] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:32.835833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165947ad-fd0e-4a9e-806c-79bb41ed477f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:32.855970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165947ad-fd0e-4a9e-806c-79bb41ed477f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:32.862320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd69a74-7343-4dda-81fa-5393916d1db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:32.863350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd69a74-7343-4dda-81fa-5393916d1db4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:32.863754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd69a74-7343-4dda-81fa-5393916d1db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:32.864159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd69a74-7343-4dda-81fa-5393916d1db4 admin admin] PATH_INFO: `/v3/projects/4019258611094cbf979dece7390ba297/users/d83156b45fed412f8506ef2d8af25339/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:32.866401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd69a74-7343-4dda-81fa-5393916d1db4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4019258611094cbf979dece7390ba297, user_id=d83156b45fed412f8506ef2d8af25339, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:32.869251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd69a74-7343-4dda-81fa-5393916d1db4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:32.903330 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 954/1908] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:32 2022] PUT /identity/v3/projects/4019258611094cbf979dece7390ba297/users/d83156b45fed412f8506ef2d8af25339/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:33.110272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b532c09-7ed1-485a-b8dc-895dfc8ebccb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:33.110870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b532c09-7ed1-485a-b8dc-895dfc8ebccb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:33.110870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b532c09-7ed1-485a-b8dc-895dfc8ebccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:33.202992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2b532c09-7ed1-485a-b8dc-895dfc8ebccb None None] MFA Rules not processed for user `a3f7a1130dff44708fc03a7c5757efa7`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:33.206116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2b532c09-7ed1-485a-b8dc-895dfc8ebccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:33.464128 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 955/1909] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:33 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:33.530723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd69a74-7343-4dda-81fa-5393916d1db4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:33.548059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:33.655530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd69a74-7343-4dda-81fa-5393916d1db4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:33.662406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313c68f-771f-4215-b60a-4d7d8712f485 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:33.663063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313c68f-771f-4215-b60a-4d7d8712f485 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:33.664108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313c68f-771f-4215-b60a-4d7d8712f485 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:33.664437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313c68f-771f-4215-b60a-4d7d8712f485 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:33.675041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313c68f-771f-4215-b60a-4d7d8712f485 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:33.677672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313c68f-771f-4215-b60a-4d7d8712f485 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:33.712632 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 955/1910] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:33 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:35.944171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b532c09-7ed1-485a-b8dc-895dfc8ebccb None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:35.971758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b532c09-7ed1-485a-b8dc-895dfc8ebccb None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:35.979675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549368d8-58e2-4726-b261-61f5048828f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:35.980861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549368d8-58e2-4726-b261-61f5048828f5 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:35.981318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549368d8-58e2-4726-b261-61f5048828f5 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:35.981732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549368d8-58e2-4726-b261-61f5048828f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:35.986032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-549368d8-58e2-4726-b261-61f5048828f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:36.138697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549368d8-58e2-4726-b261-61f5048828f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:36.140313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549368d8-58e2-4726-b261-61f5048828f5 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:36.177449 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 956/1911] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:36.324512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c15fc-55c0-48a9-89c5-07835b812759 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:36.324512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c15fc-55c0-48a9-89c5-07835b812759 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:36.324512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c15fc-55c0-48a9-89c5-07835b812759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:36.389799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-710c15fc-55c0-48a9-89c5-07835b812759 None None] MFA Rules not processed for user `1b96f59ee1304302bc8f1e0b728f1df3`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:36.391027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-710c15fc-55c0-48a9-89c5-07835b812759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:36.404060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17e05d-7e16-48d1-ab13-342d7fd7f858 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:36.404334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17e05d-7e16-48d1-ab13-342d7fd7f858 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:36.404795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17e05d-7e16-48d1-ab13-342d7fd7f858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:36.452958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ea17e05d-7e16-48d1-ab13-342d7fd7f858 None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:36.454169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ea17e05d-7e16-48d1-ab13-342d7fd7f858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:36.590176 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 956/1912] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:36.592037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:36.623734 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 957/1913] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 20:55:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:36.646189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c15fc-55c0-48a9-89c5-07835b812759 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:36.647650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-710c15fc-55c0-48a9-89c5-07835b812759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:36.674220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c15fc-55c0-48a9-89c5-07835b812759 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:36.677846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eff0bf-8f08-4d7f-be4f-0f337f69594d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:36.678466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eff0bf-8f08-4d7f-be4f-0f337f69594d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:36.678737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eff0bf-8f08-4d7f-be4f-0f337f69594d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:36.679001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eff0bf-8f08-4d7f-be4f-0f337f69594d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:36.697012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eff0bf-8f08-4d7f-be4f-0f337f69594d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:36.698475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eff0bf-8f08-4d7f-be4f-0f337f69594d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:36.722133 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 957/1914] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:37.146538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab872dca-a6a5-45c5-a3c0-dd3b2a053619 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:37.147066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab872dca-a6a5-45c5-a3c0-dd3b2a053619 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:37.147066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab872dca-a6a5-45c5-a3c0-dd3b2a053619 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:37.148838 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 958/1915] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:55:37.154117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019d298-5611-4586-ae2e-c1748cf85115 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:37.154390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019d298-5611-4586-ae2e-c1748cf85115 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:37.154816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019d298-5611-4586-ae2e-c1748cf85115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:37.212967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4019d298-5611-4586-ae2e-c1748cf85115 None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:37.214208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4019d298-5611-4586-ae2e-c1748cf85115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:37.354766 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 958/1916] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 20:55:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:37.386391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab872dca-a6a5-45c5-a3c0-dd3b2a053619 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:37.388351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ab872dca-a6a5-45c5-a3c0-dd3b2a053619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:37.420863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab872dca-a6a5-45c5-a3c0-dd3b2a053619 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:37.436571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be34df-8153-46fb-862e-0df2ad9ff147 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:37.437217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be34df-8153-46fb-862e-0df2ad9ff147 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:37.437645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be34df-8153-46fb-862e-0df2ad9ff147 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:37.438062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be34df-8153-46fb-862e-0df2ad9ff147 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:37.463320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be34df-8153-46fb-862e-0df2ad9ff147 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:37.465372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be34df-8153-46fb-862e-0df2ad9ff147 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:37.488543 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 959/1917] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:38.753012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9361143-fc68-4ce8-b971-54d4874985d1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:38.753626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9361143-fc68-4ce8-b971-54d4874985d1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:38.753626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9361143-fc68-4ce8-b971-54d4874985d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:38.801860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b9361143-fc68-4ce8-b971-54d4874985d1 None None] MFA Rules not processed for user `c3adc70e73614d48be23e3542424ee1d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:38.803325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b9361143-fc68-4ce8-b971-54d4874985d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:38.861117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:38.984739 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 959/1918] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:42.006724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be34df-8153-46fb-862e-0df2ad9ff147 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:42.008294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-69be34df-8153-46fb-862e-0df2ad9ff147 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:42.154437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e901a3-c39a-4996-a380-266f8f8fea19 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:42.154982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e901a3-c39a-4996-a380-266f8f8fea19 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:42.154982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e901a3-c39a-4996-a380-266f8f8fea19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:42.200401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be34df-8153-46fb-862e-0df2ad9ff147 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:42.214096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93eb1bb-550a-4069-b679-ba2590bafdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:42.215001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93eb1bb-550a-4069-b679-ba2590bafdc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:42.215871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93eb1bb-550a-4069-b679-ba2590bafdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:42.217266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93eb1bb-550a-4069-b679-ba2590bafdc6 admin admin] PATH_INFO: `/v3/users/4cdb1c73a7de40f8962ed40eef69641a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:42.219343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93eb1bb-550a-4069-b679-ba2590bafdc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cdb1c73a7de40f8962ed40eef69641a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:42.237370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93eb1bb-550a-4069-b679-ba2590bafdc6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:42.249952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d4e901a3-c39a-4996-a380-266f8f8fea19 None None] MFA Rules not processed for user `d83156b45fed412f8506ef2d8af25339`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:42.255616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d4e901a3-c39a-4996-a380-266f8f8fea19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:42.375155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e93eb1bb-550a-4069-b679-ba2590bafdc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cdb1c73a7de40f8962ed40eef69641a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:42.393667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e93eb1bb-550a-4069-b679-ba2590bafdc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cdb1c73a7de40f8962ed40eef69641a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:42.406911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e93eb1bb-550a-4069-b679-ba2590bafdc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cdb1c73a7de40f8962ed40eef69641a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:42.434112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e93eb1bb-550a-4069-b679-ba2590bafdc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cdb1c73a7de40f8962ed40eef69641a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:42.435295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e93eb1bb-550a-4069-b679-ba2590bafdc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cdb1c73a7de40f8962ed40eef69641a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:42.448940 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 960/1919] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:42 2022] DELETE /identity/v3/users/4cdb1c73a7de40f8962ed40eef69641a => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:42.456507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9e2c5-2499-4bb6-aa19-7dbd1b0b0cb0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:42.457071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9e2c5-2499-4bb6-aa19-7dbd1b0b0cb0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:42.457443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9e2c5-2499-4bb6-aa19-7dbd1b0b0cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:42.468403 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 960/1920] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:42 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:42.498329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f3d9e2c5-2499-4bb6-aa19-7dbd1b0b0cb0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:42.500079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f3d9e2c5-2499-4bb6-aa19-7dbd1b0b0cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:42.729275 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 961/1921] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:42.821430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e901a3-c39a-4996-a380-266f8f8fea19 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:42.843873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e901a3-c39a-4996-a380-266f8f8fea19 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:43.081616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1d263-a2eb-41ed-aae9-98b8c3e14b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:43.082567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1d263-a2eb-41ed-aae9-98b8c3e14b9d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:43.083001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1d263-a2eb-41ed-aae9-98b8c3e14b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:43.083452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1d263-a2eb-41ed-aae9-98b8c3e14b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:43.128282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1d263-a2eb-41ed-aae9-98b8c3e14b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:43.128282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1d263-a2eb-41ed-aae9-98b8c3e14b9d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:43.152480 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 961/1922] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:43.655826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9e2c5-2499-4bb6-aa19-7dbd1b0b0cb0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:43.658371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f3d9e2c5-2499-4bb6-aa19-7dbd1b0b0cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:43.670254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:43.678237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9e2c5-2499-4bb6-aa19-7dbd1b0b0cb0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:43.685851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293222b-2cfa-43d6-bdb8-2547469606b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:43.687374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293222b-2cfa-43d6-bdb8-2547469606b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:43.687897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293222b-2cfa-43d6-bdb8-2547469606b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:43.688403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293222b-2cfa-43d6-bdb8-2547469606b9 admin admin] PATH_INFO: `/v3/users/3f85bb62037044ebae5a0a83194f9e9c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:43.690307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293222b-2cfa-43d6-bdb8-2547469606b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f85bb62037044ebae5a0a83194f9e9c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:43.695328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293222b-2cfa-43d6-bdb8-2547469606b9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:43.733054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1d263-a2eb-41ed-aae9-98b8c3e14b9d service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:43.734869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f0d1d263-a2eb-41ed-aae9-98b8c3e14b9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:43.799746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c293222b-2cfa-43d6-bdb8-2547469606b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f85bb62037044ebae5a0a83194f9e9c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:43.817032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c293222b-2cfa-43d6-bdb8-2547469606b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f85bb62037044ebae5a0a83194f9e9c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:43.844112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c293222b-2cfa-43d6-bdb8-2547469606b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f85bb62037044ebae5a0a83194f9e9c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:43.885922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c293222b-2cfa-43d6-bdb8-2547469606b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f85bb62037044ebae5a0a83194f9e9c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:43.887874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:43.889816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c293222b-2cfa-43d6-bdb8-2547469606b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f85bb62037044ebae5a0a83194f9e9c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:43.907249 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 962/1923] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:43 2022] DELETE /identity/v3/users/3f85bb62037044ebae5a0a83194f9e9c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:43.912271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9e5f6-553a-4920-9c1c-3dc0367bc562 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:43.912947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9e5f6-553a-4920-9c1c-3dc0367bc562 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:43.913434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9e5f6-553a-4920-9c1c-3dc0367bc562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:43.938700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1d263-a2eb-41ed-aae9-98b8c3e14b9d service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:43.978479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-02e9e5f6-553a-4920-9c1c-3dc0367bc562 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:43.986813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-02e9e5f6-553a-4920-9c1c-3dc0367bc562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:44.105865 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 963/1924] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:44.114493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2b8db-e77e-4eb1-b182-6d3704d6525c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:44.115462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2b8db-e77e-4eb1-b182-6d3704d6525c service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:44.115816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2b8db-e77e-4eb1-b182-6d3704d6525c service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:44.116147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2b8db-e77e-4eb1-b182-6d3704d6525c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:44.118211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a8f2b8db-e77e-4eb1-b182-6d3704d6525c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:44.208696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9e5f6-553a-4920-9c1c-3dc0367bc562 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:44.210312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-02e9e5f6-553a-4920-9c1c-3dc0367bc562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:44.213358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2b8db-e77e-4eb1-b182-6d3704d6525c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:44.215213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2b8db-e77e-4eb1-b182-6d3704d6525c service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:44.241358 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 962/1925] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:43 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:55:44.380244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9e5f6-553a-4920-9c1c-3dc0367bc562 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:44.385547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e952a-762d-4063-987e-0e8a329e8368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:44.385879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e952a-762d-4063-987e-0e8a329e8368 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:44.386140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e952a-762d-4063-987e-0e8a329e8368 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:44.386855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e952a-762d-4063-987e-0e8a329e8368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:44.408995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e952a-762d-4063-987e-0e8a329e8368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:44.416911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e952a-762d-4063-987e-0e8a329e8368 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:44.454363 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 964/1926] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:55:44.888614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2b8db-e77e-4eb1-b182-6d3704d6525c service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:44.890667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a8f2b8db-e77e-4eb1-b182-6d3704d6525c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:44.913778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2b8db-e77e-4eb1-b182-6d3704d6525c service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:44.929826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:44.941088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:44.942298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:44.942851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] PATH_INFO: `/v3/projects/28b6481e72f84322b4090cf6c2a20927` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:44.945983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] RBAC: Authorizing `identity:delete_project(project_id=28b6481e72f84322b4090cf6c2a20927)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:44.955457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:45.097261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28b6481e72f84322b4090cf6c2a20927', 'request_id': 'req-828e0500-1296-41e9-b068-5e5e8635231e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:45.110585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28b6481e72f84322b4090cf6c2a20927', 'request_id': 'req-828e0500-1296-41e9-b068-5e5e8635231e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:45.130991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28b6481e72f84322b4090cf6c2a20927', 'request_id': 'req-828e0500-1296-41e9-b068-5e5e8635231e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:45.155376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] The token cache is being invalidate because project 28b6481e72f84322b4090cf6c2a20927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:45.155672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:45.162371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:45.167150 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 963/1927] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:44 2022] DELETE /identity/v3/projects/28b6481e72f84322b4090cf6c2a20927 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:45.172252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e952a-762d-4063-987e-0e8a329e8368 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:45.194232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e952a-762d-4063-987e-0e8a329e8368 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:45.223273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:45.309879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:45.310808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:45.311272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:45.311721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] PATH_INFO: `/v3/users/77b94de1ecd64564bac67b21996158d9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:45.313420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b94de1ecd64564bac67b21996158d9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:45.315747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:45.431871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b94de1ecd64564bac67b21996158d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:45.447698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b94de1ecd64564bac67b21996158d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:45.471272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b94de1ecd64564bac67b21996158d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:45.504163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b94de1ecd64564bac67b21996158d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:45.505107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b94de1ecd64564bac67b21996158d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:45.518644 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 965/1928] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:45 2022] DELETE /identity/v3/users/77b94de1ecd64564bac67b21996158d9 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:46.018794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:46.020266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:46.037649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e0500-1296-41e9-b068-5e5e8635231e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:46.056227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:46.204829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:46.212282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:46.213215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:46.215284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] PATH_INFO: `/v3/projects/a6c380136f644611bfcb756f87feba71` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:46.216922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6c380136f644611bfcb756f87feba71)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:46.226564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:46.332789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6c380136f644611bfcb756f87feba71', 'request_id': 'req-da47a80e-3fa2-4eda-9c6c-7437422e3bee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.341689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6c380136f644611bfcb756f87feba71', 'request_id': 'req-da47a80e-3fa2-4eda-9c6c-7437422e3bee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.361125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6c380136f644611bfcb756f87feba71', 'request_id': 'req-da47a80e-3fa2-4eda-9c6c-7437422e3bee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.373196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:46.375294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:46.400297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] The token cache is being invalidate because project a6c380136f644611bfcb756f87feba71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:46.401370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.404574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74909f-9d76-4949-82e9-5288717c421c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:46.404972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.410693 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 964/1929] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:46 2022] DELETE /identity/v3/projects/a6c380136f644611bfcb756f87feba71 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:46.411637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:46.415825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:46.577132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47a80e-3fa2-4eda-9c6c-7437422e3bee admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:46.584486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae54ff-f9d5-4a60-8004-01e835faa600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:46.585518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae54ff-f9d5-4a60-8004-01e835faa600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:46.585991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae54ff-f9d5-4a60-8004-01e835faa600 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:46.586432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae54ff-f9d5-4a60-8004-01e835faa600 admin admin] PATH_INFO: `/v3/users/218942b27bdb401ca842ee6ef27670d2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:46.587875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae54ff-f9d5-4a60-8004-01e835faa600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=218942b27bdb401ca842ee6ef27670d2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:46.590519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae54ff-f9d5-4a60-8004-01e835faa600 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:46.623693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:46.624420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:46.624682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:46.624954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] PATH_INFO: `/v3/users/44f542f6dc164350b3ffa192616d0f95` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:46.636072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] RBAC: Authorizing `identity:delete_user(user_id=44f542f6dc164350b3ffa192616d0f95)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:46.648942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:46.673476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-adae54ff-f9d5-4a60-8004-01e835faa600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218942b27bdb401ca842ee6ef27670d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.676223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:46.689195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-adae54ff-f9d5-4a60-8004-01e835faa600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218942b27bdb401ca842ee6ef27670d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.710795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-adae54ff-f9d5-4a60-8004-01e835faa600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218942b27bdb401ca842ee6ef27670d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.712552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-adae54ff-f9d5-4a60-8004-01e835faa600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218942b27bdb401ca842ee6ef27670d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.713662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-adae54ff-f9d5-4a60-8004-01e835faa600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218942b27bdb401ca842ee6ef27670d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.736950 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 965/1930] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:46 2022] DELETE /identity/v3/users/218942b27bdb401ca842ee6ef27670d2 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:46.786885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:46.796290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44f542f6dc164350b3ffa192616d0f95'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.813443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44f542f6dc164350b3ffa192616d0f95'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.825511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44f542f6dc164350b3ffa192616d0f95'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.856751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44f542f6dc164350b3ffa192616d0f95'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.857544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44f542f6dc164350b3ffa192616d0f95'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:46.882758 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 966/1931] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:46 2022] DELETE /identity/v3/users/44f542f6dc164350b3ffa192616d0f95 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:46.885130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dbf93-fd00-4671-a7bd-64c12f52d0c6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:46.885471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dbf93-fd00-4671-a7bd-64c12f52d0c6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:46.885785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dbf93-fd00-4671-a7bd-64c12f52d0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:46.935630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-748dbf93-fd00-4671-a7bd-64c12f52d0c6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:46.940220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-748dbf93-fd00-4671-a7bd-64c12f52d0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:47.073448 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 966/1932] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:47.104131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:47.106948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:47.262324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f4011-4725-4618-bafb-07d0e413041a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:47.272066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce61e04-4108-47a4-be27-827cc4e6633a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:47.272815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce61e04-4108-47a4-be27-827cc4e6633a service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:47.273152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce61e04-4108-47a4-be27-827cc4e6633a service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:47.273483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce61e04-4108-47a4-be27-827cc4e6633a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:47.292564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce61e04-4108-47a4-be27-827cc4e6633a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:47.294027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce61e04-4108-47a4-be27-827cc4e6633a service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:47.320519 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 967/1933] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:47.844582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dbf93-fd00-4671-a7bd-64c12f52d0c6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:47.846287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-748dbf93-fd00-4671-a7bd-64c12f52d0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:47.871399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dbf93-fd00-4671-a7bd-64c12f52d0c6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:47.888520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:47.888520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:47.888520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:47.888520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] PATH_INFO: `/v3/projects/06a86e61472b4ad69a50b4308ff3a7b7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:47.896984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06a86e61472b4ad69a50b4308ff3a7b7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:47.899319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:48.018856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a86e61472b4ad69a50b4308ff3a7b7', 'request_id': 'req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:48.026937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a86e61472b4ad69a50b4308ff3a7b7', 'request_id': 'req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:48.039126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a86e61472b4ad69a50b4308ff3a7b7', 'request_id': 'req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:48.058512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] The token cache is being invalidate because project 06a86e61472b4ad69a50b4308ff3a7b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:48.059039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:48.060725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:48.063685 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 967/1934] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:47 2022] DELETE /identity/v3/projects/06a86e61472b4ad69a50b4308ff3a7b7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:48.070231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89b53d-a388-4ede-9849-303b050063d3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:48.070773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89b53d-a388-4ede-9849-303b050063d3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:48.070773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89b53d-a388-4ede-9849-303b050063d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:48.133394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:48.139981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:48.167430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8e89b53d-a388-4ede-9849-303b050063d3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:48.199476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8e89b53d-a388-4ede-9849-303b050063d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:48.322039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:48.353995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a2c19-69d1-4ed5-8440-3d395d6e4cf9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:48.369165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:48.370168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:48.373834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:48.374724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3 admin admin] PATH_INFO: `/v3/projects/022df6b28f2644038160415fe5fbf339` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:48.379974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=022df6b28f2644038160415fe5fbf339)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:48.383268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:48.441404 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 968/1935] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:48.453275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e197bd-cf78-45a8-8124-beaa920276d3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:48.455158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e197bd-cf78-45a8-8124-beaa920276d3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:48.455662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e197bd-cf78-45a8-8124-beaa920276d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:48.517426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '022df6b28f2644038160415fe5fbf339', 'request_id': 'req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:48.539153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '022df6b28f2644038160415fe5fbf339', 'request_id': 'req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:48.546215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-04e197bd-cf78-45a8-8124-beaa920276d3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:48.555707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-04e197bd-cf78-45a8-8124-beaa920276d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:48.565008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '022df6b28f2644038160415fe5fbf339', 'request_id': 'req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:48.598966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3 admin admin] The token cache is being invalidate because project 022df6b28f2644038160415fe5fbf339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:48.599680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:48.604100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecbe9320-1d72-43f0-9a0c-2109ba6833c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:48.606292 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 968/1936] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:48 2022] DELETE /identity/v3/projects/022df6b28f2644038160415fe5fbf339 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:48.614001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bf86c-f71c-45c6-ac2f-1e0e394f3596 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:48.614411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bf86c-f71c-45c6-ac2f-1e0e394f3596 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:48.615569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bf86c-f71c-45c6-ac2f-1e0e394f3596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:48.681546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7f2bf86c-f71c-45c6-ac2f-1e0e394f3596 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:48.685904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7f2bf86c-f71c-45c6-ac2f-1e0e394f3596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:48.693844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:48.707749 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 969/1937] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:48.712813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e197bd-cf78-45a8-8124-beaa920276d3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:48.731276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e197bd-cf78-45a8-8124-beaa920276d3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:48.739452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1e306-e158-462c-bf98-117fcb6a265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:48.740361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1e306-e158-462c-bf98-117fcb6a265c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:48.741630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1e306-e158-462c-bf98-117fcb6a265c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:48.742106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1e306-e158-462c-bf98-117fcb6a265c admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:48.742928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1e306-e158-462c-bf98-117fcb6a265c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:55:48.743373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1e306-e158-462c-bf98-117fcb6a265c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:48.746002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1e306-e158-462c-bf98-117fcb6a265c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:48.774346 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 970/1938] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:55:48.775792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1e306-e158-462c-bf98-117fcb6a265c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:48.792971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1e306-e158-462c-bf98-117fcb6a265c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:48.801877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b1ec1-ff0c-4077-b5c5-8df89a10ed21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:48.804693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b1ec1-ff0c-4077-b5c5-8df89a10ed21 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:48.807186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b1ec1-ff0c-4077-b5c5-8df89a10ed21 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:48.807486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b1ec1-ff0c-4077-b5c5-8df89a10ed21 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:48.808162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b1ec1-ff0c-4077-b5c5-8df89a10ed21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:48.811415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b1ec1-ff0c-4077-b5c5-8df89a10ed21 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:48.864549 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 971/1939] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:48 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:48.874474 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 969/1940] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:48.878754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd225e-f732-4881-9f20-1d6650e3bbe0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:48.879572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b1ec1-ff0c-4077-b5c5-8df89a10ed21 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:48.879802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd225e-f732-4881-9f20-1d6650e3bbe0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:48.880258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd225e-f732-4881-9f20-1d6650e3bbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:48.936266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-eddd225e-f732-4881-9f20-1d6650e3bbe0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:48.937777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-eddd225e-f732-4881-9f20-1d6650e3bbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:49.067306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b1ec1-ff0c-4077-b5c5-8df89a10ed21 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:49.081165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abc12b-905e-48c1-b9ca-97a616245bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:49.081784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc12b-905e-48c1-b9ca-97a616245bbd admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:49.082122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc12b-905e-48c1-b9ca-97a616245bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:49.082536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc12b-905e-48c1-b9ca-97a616245bbd admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:49.083191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abc12b-905e-48c1-b9ca-97a616245bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:49.086564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abc12b-905e-48c1-b9ca-97a616245bbd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:49.110261 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 970/1941] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:49.111959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd225e-f732-4881-9f20-1d6650e3bbe0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:49.132918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd225e-f732-4881-9f20-1d6650e3bbe0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:49.139424 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 972/1942] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:48 2022] POST /identity/v3/users => generated 293 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:49.141324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abc12b-905e-48c1-b9ca-97a616245bbd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:49.146179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30be802-4168-4074-aae4-5ed3a6c0c457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:49.146814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30be802-4168-4074-aae4-5ed3a6c0c457 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:49.147089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30be802-4168-4074-aae4-5ed3a6c0c457 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:49.147354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30be802-4168-4074-aae4-5ed3a6c0c457 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:49.148013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30be802-4168-4074-aae4-5ed3a6c0c457 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:55:49.148307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30be802-4168-4074-aae4-5ed3a6c0c457 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:49.149997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30be802-4168-4074-aae4-5ed3a6c0c457 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:49.162901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abc12b-905e-48c1-b9ca-97a616245bbd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:49.172721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f886a00-3f33-48b8-b115-86e628cc0e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:49.173514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f886a00-3f33-48b8-b115-86e628cc0e07 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:49.173903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f886a00-3f33-48b8-b115-86e628cc0e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:49.174259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f886a00-3f33-48b8-b115-86e628cc0e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:49.176143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f886a00-3f33-48b8-b115-86e628cc0e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:49.178012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f886a00-3f33-48b8-b115-86e628cc0e07 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:49.195268 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 971/1943] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:55:49.200129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30be802-4168-4074-aae4-5ed3a6c0c457 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:49.220057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30be802-4168-4074-aae4-5ed3a6c0c457 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:49.231632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d537f97-d9f6-4eed-b5f5-bb9191a4d947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:49.232258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d537f97-d9f6-4eed-b5f5-bb9191a4d947 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:49.232573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d537f97-d9f6-4eed-b5f5-bb9191a4d947 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:49.232840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d537f97-d9f6-4eed-b5f5-bb9191a4d947 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:49.233478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d537f97-d9f6-4eed-b5f5-bb9191a4d947 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:49.235550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d537f97-d9f6-4eed-b5f5-bb9191a4d947 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:49.240895 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 973/1944] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:55:49.242122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f886a00-3f33-48b8-b115-86e628cc0e07 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:49.242919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:49.266631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f886a00-3f33-48b8-b115-86e628cc0e07 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:49.284287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3c200-ff26-446a-a87d-583b59c87b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:49.285359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3c200-ff26-446a-a87d-583b59c87b2e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:49.285843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3c200-ff26-446a-a87d-583b59c87b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:49.286301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3c200-ff26-446a-a87d-583b59c87b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:49.287640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3c200-ff26-446a-a87d-583b59c87b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:49.289575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3c200-ff26-446a-a87d-583b59c87b2e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:49.313267 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 974/1945] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:49.318373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3c200-ff26-446a-a87d-583b59c87b2e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:49.323670 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 972/1946] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:49 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:55:49.327880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d537f97-d9f6-4eed-b5f5-bb9191a4d947 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:49.341675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3c200-ff26-446a-a87d-583b59c87b2e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:49.464706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d537f97-d9f6-4eed-b5f5-bb9191a4d947 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:49.478998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dd2f2-6916-40a3-a4c8-1b722bbd7098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:49.479707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dd2f2-6916-40a3-a4c8-1b722bbd7098 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:49.480042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dd2f2-6916-40a3-a4c8-1b722bbd7098 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:49.480519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dd2f2-6916-40a3-a4c8-1b722bbd7098 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:49.481117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50dd2f2-6916-40a3-a4c8-1b722bbd7098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:49.483192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50dd2f2-6916-40a3-a4c8-1b722bbd7098 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:49.486363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ff999-3267-41a3-9683-7aac15c4f50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:49.489731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ff999-3267-41a3-9683-7aac15c4f50e admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:49.490058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ff999-3267-41a3-9683-7aac15c4f50e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:49.490377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ff999-3267-41a3-9683-7aac15c4f50e admin admin] PATH_INFO: `/v3/projects/aa99ed809f344171923311bef4b8bc23/users/1e6b31adfd68421c8299ac53656490e6/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:49.535859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ff999-3267-41a3-9683-7aac15c4f50e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa99ed809f344171923311bef4b8bc23, user_id=1e6b31adfd68421c8299ac53656490e6, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:49.544550 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 973/1947] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:49 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:49.545737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dd2f2-6916-40a3-a4c8-1b722bbd7098 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:49.547248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ff999-3267-41a3-9683-7aac15c4f50e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:49.569506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d50dd2f2-6916-40a3-a4c8-1b722bbd7098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:49.588653 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 975/1948] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:49 2022] PUT /identity/v3/projects/aa99ed809f344171923311bef4b8bc23/users/1e6b31adfd68421c8299ac53656490e6/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:49.588653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ff999-3267-41a3-9683-7aac15c4f50e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:49.607693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dd2f2-6916-40a3-a4c8-1b722bbd7098 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:49.637924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:49.757073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:49.757999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:49.758413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:49.758815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] PATH_INFO: `/v3/projects/9a703d5afdeb467eab897527129a2d01` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:49.762128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ff999-3267-41a3-9683-7aac15c4f50e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:49.763158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a703d5afdeb467eab897527129a2d01)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:49.765289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:49.778439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6f8b1-b424-44fc-a582-a4be6cc3050c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:49.779305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6f8b1-b424-44fc-a582-a4be6cc3050c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:49.779714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6f8b1-b424-44fc-a582-a4be6cc3050c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:49.780135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6f8b1-b424-44fc-a582-a4be6cc3050c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:49.780968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6f8b1-b424-44fc-a582-a4be6cc3050c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:49.782746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6f8b1-b424-44fc-a582-a4be6cc3050c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:49.806724 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 976/1949] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:49.808399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec86ec-fff0-4c1a-82e2-b5850f722405 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:49.812291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec86ec-fff0-4c1a-82e2-b5850f722405 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:49.813069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec86ec-fff0-4c1a-82e2-b5850f722405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:49.874318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a703d5afdeb467eab897527129a2d01', 'request_id': 'req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:49.881295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-86ec86ec-fff0-4c1a-82e2-b5850f722405 None None] MFA Rules not processed for user `1e6b31adfd68421c8299ac53656490e6`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:49.882524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-86ec86ec-fff0-4c1a-82e2-b5850f722405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:49.886421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a703d5afdeb467eab897527129a2d01', 'request_id': 'req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:49.902821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a703d5afdeb467eab897527129a2d01', 'request_id': 'req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:49.946573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] The token cache is being invalidate because project 9a703d5afdeb467eab897527129a2d01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:49.946950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:49.949207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:49.952425 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 974/1950] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:49 2022] DELETE /identity/v3/projects/9a703d5afdeb467eab897527129a2d01 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:49.956315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:49.964547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:49.983516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:50.130561 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 977/1951] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:50.130561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec86ec-fff0-4c1a-82e2-b5850f722405 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:50.131292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-86ec86ec-fff0-4c1a-82e2-b5850f722405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:50.187037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900799a-cdf9-4e28-8bff-3ee2fc5ec4f8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:50.207664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5b67d-8d32-47f7-a68e-db0102c09e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:50.208358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5b67d-8d32-47f7-a68e-db0102c09e99 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:50.208620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5b67d-8d32-47f7-a68e-db0102c09e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:50.208889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5b67d-8d32-47f7-a68e-db0102c09e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:50.209711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5b67d-8d32-47f7-a68e-db0102c09e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:50.217555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5b67d-8d32-47f7-a68e-db0102c09e99 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:50.257700 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 975/1952] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:50.258865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5b67d-8d32-47f7-a68e-db0102c09e99 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:50.266619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-55f5b67d-8d32-47f7-a68e-db0102c09e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:50.303266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5b67d-8d32-47f7-a68e-db0102c09e99 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:50.320215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e10ea5-444d-4f48-bbf1-f1fc3b76be9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:50.321222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e10ea5-444d-4f48-bbf1-f1fc3b76be9b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:50.321496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e10ea5-444d-4f48-bbf1-f1fc3b76be9b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:50.321757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e10ea5-444d-4f48-bbf1-f1fc3b76be9b admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:50.322391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e10ea5-444d-4f48-bbf1-f1fc3b76be9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:50.324691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e10ea5-444d-4f48-bbf1-f1fc3b76be9b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:50.349371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec86ec-fff0-4c1a-82e2-b5850f722405 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:50.359501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:50.361638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:50.364320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:50.364754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] PATH_INFO: `/v3/users/e8ee40e4a8b7453e8f3b394f7cf81094` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:50.366789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ee40e4a8b7453e8f3b394f7cf81094)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:50.368001 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 976/1953] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:50 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:50.369032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e10ea5-444d-4f48-bbf1-f1fc3b76be9b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:50.380489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:50.465637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:50.470020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ee40e4a8b7453e8f3b394f7cf81094'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:50.481241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e10ea5-444d-4f48-bbf1-f1fc3b76be9b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:50.485037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ee40e4a8b7453e8f3b394f7cf81094'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:50.488409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc21cb9-a5ca-4b75-8f75-cdf1f99048dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:50.489161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc21cb9-a5ca-4b75-8f75-cdf1f99048dd admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:50.489587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc21cb9-a5ca-4b75-8f75-cdf1f99048dd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:50.489902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc21cb9-a5ca-4b75-8f75-cdf1f99048dd admin admin] PATH_INFO: `/v3/projects/922daf4916c843cfa61964471f91d4d1/users/d846054832cb45c2bb0b5b8af9e76051/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:50.497592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ee40e4a8b7453e8f3b394f7cf81094'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:50.532622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc21cb9-a5ca-4b75-8f75-cdf1f99048dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=922daf4916c843cfa61964471f91d4d1, user_id=d846054832cb45c2bb0b5b8af9e76051, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:50.539440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ee40e4a8b7453e8f3b394f7cf81094'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:50.539914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc21cb9-a5ca-4b75-8f75-cdf1f99048dd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:50.545156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ee40e4a8b7453e8f3b394f7cf81094'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:50.557395 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 978/1954] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:50 2022] DELETE /identity/v3/users/e8ee40e4a8b7453e8f3b394f7cf81094 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:50.558857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:50.560435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:50.561511 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 977/1955] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:50 2022] PUT /identity/v3/projects/922daf4916c843cfa61964471f91d4d1/users/d846054832cb45c2bb0b5b8af9e76051/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:50.573479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb06c9c-4851-41e0-8b38-0d2fc3bace46 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:50.573796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb06c9c-4851-41e0-8b38-0d2fc3bace46 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:50.574014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb06c9c-4851-41e0-8b38-0d2fc3bace46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:50.615291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8cb06c9c-4851-41e0-8b38-0d2fc3bace46 None None] MFA Rules not processed for user `d846054832cb45c2bb0b5b8af9e76051`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:50.617028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8cb06c9c-4851-41e0-8b38-0d2fc3bace46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:50.712246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42b480-de25-449e-9f2b-08a20beb807d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:50.723120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:50.728819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe67735-b26a-4cec-8b47-45d6248d0dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:50.729719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe67735-b26a-4cec-8b47-45d6248d0dcd admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:50.730457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe67735-b26a-4cec-8b47-45d6248d0dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:50.730864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe67735-b26a-4cec-8b47-45d6248d0dcd admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:50.731781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe67735-b26a-4cec-8b47-45d6248d0dcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:50.734068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe67735-b26a-4cec-8b47-45d6248d0dcd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:50.786277 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 979/1956] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:50 2022] POST /identity/v3/users => generated 298 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:50.800390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe67735-b26a-4cec-8b47-45d6248d0dcd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:50.823399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe67735-b26a-4cec-8b47-45d6248d0dcd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:50.829596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fa169-394b-496b-b1ae-e1648c606a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:50.833590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fa169-394b-496b-b1ae-e1648c606a10 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:50.836112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fa169-394b-496b-b1ae-e1648c606a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:50.836472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fa169-394b-496b-b1ae-e1648c606a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:50.837180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fa169-394b-496b-b1ae-e1648c606a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:50.844026 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 978/1957] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:50.845372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb06c9c-4851-41e0-8b38-0d2fc3bace46 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:50.850528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fa169-394b-496b-b1ae-e1648c606a10 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:50.851850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8cb06c9c-4851-41e0-8b38-0d2fc3bace46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:50.889852 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 980/1958] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:50.895300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fa169-394b-496b-b1ae-e1648c606a10 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:50.896873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb06c9c-4851-41e0-8b38-0d2fc3bace46 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:50.911130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fa169-394b-496b-b1ae-e1648c606a10 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:50.911377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b00b9a-61b7-4b32-814b-c3de6c371059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:50.912018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b00b9a-61b7-4b32-814b-c3de6c371059 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:50.913894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b00b9a-61b7-4b32-814b-c3de6c371059 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:50.914212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b00b9a-61b7-4b32-814b-c3de6c371059 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:50.916944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b00b9a-61b7-4b32-814b-c3de6c371059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:50.921471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5f522-334c-4390-865c-c32b304ad5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:50.922500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b00b9a-61b7-4b32-814b-c3de6c371059 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:50.924072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5f522-334c-4390-865c-c32b304ad5ac admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:50.924524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5f522-334c-4390-865c-c32b304ad5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:50.925273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5f522-334c-4390-865c-c32b304ad5ac admin admin] PATH_INFO: `/v3/projects/279b2d5d2f2647f69be35817b22cdb04/users/131e1f9e976d4a1784cf6b4e55cf87dd/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:50.951134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5f522-334c-4390-865c-c32b304ad5ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=279b2d5d2f2647f69be35817b22cdb04, user_id=131e1f9e976d4a1784cf6b4e55cf87dd, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:50.955148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5f522-334c-4390-865c-c32b304ad5ac admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:50.989599 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 979/1959] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:50 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:50.994323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b00b9a-61b7-4b32-814b-c3de6c371059 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:51.003247 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 981/1960] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:50 2022] PUT /identity/v3/projects/279b2d5d2f2647f69be35817b22cdb04/users/131e1f9e976d4a1784cf6b4e55cf87dd/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:51.007459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5f522-334c-4390-865c-c32b304ad5ac admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:51.075029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b00b9a-61b7-4b32-814b-c3de6c371059 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:51.081703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13653cc-4007-40b7-9c12-0448afa42560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:51.082612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13653cc-4007-40b7-9c12-0448afa42560 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.082982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13653cc-4007-40b7-9c12-0448afa42560 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.083599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13653cc-4007-40b7-9c12-0448afa42560 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.084326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13653cc-4007-40b7-9c12-0448afa42560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:51.086757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13653cc-4007-40b7-9c12-0448afa42560 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:51.117212 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 980/1961] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:50 2022] POST /identity/v3/users => generated 297 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:55:51.119108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6800243-2f92-4117-b3fd-6347491d3c2b None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.119489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6800243-2f92-4117-b3fd-6347491d3c2b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.120167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6800243-2f92-4117-b3fd-6347491d3c2b None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.121564 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 981/1962] 10.222.0.23 () {54 vars in 921 bytes} [Sat Aug 20 20:55:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:55:51.122590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6800243-2f92-4117-b3fd-6347491d3c2b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:51.138521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6800243-2f92-4117-b3fd-6347491d3c2b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:51.144757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299cb6e-0edb-4af7-9a81-7a553bbf3b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:51.145481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299cb6e-0edb-4af7-9a81-7a553bbf3b1a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.145850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299cb6e-0edb-4af7-9a81-7a553bbf3b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.146717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299cb6e-0edb-4af7-9a81-7a553bbf3b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.147728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299cb6e-0edb-4af7-9a81-7a553bbf3b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:51.149616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299cb6e-0edb-4af7-9a81-7a553bbf3b1a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:51.160583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5f522-334c-4390-865c-c32b304ad5ac admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:51.163166 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 982/1963] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:51.165812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b953cae-c9ee-449b-8970-c3a2339b1da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:51.166834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b953cae-c9ee-449b-8970-c3a2339b1da4 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.167103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b953cae-c9ee-449b-8970-c3a2339b1da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.167365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b953cae-c9ee-449b-8970-c3a2339b1da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.168000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b953cae-c9ee-449b-8970-c3a2339b1da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:51.168297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299cb6e-0edb-4af7-9a81-7a553bbf3b1a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:51.169837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b953cae-c9ee-449b-8970-c3a2339b1da4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:51.180456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299cb6e-0edb-4af7-9a81-7a553bbf3b1a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:51.188276 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 982/1964] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:51.189646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bd5db-97eb-444e-af74-63e6c81494c4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.189927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bd5db-97eb-444e-af74-63e6c81494c4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.190337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bd5db-97eb-444e-af74-63e6c81494c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.193616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8de9a-592d-420d-9f8e-d9b1958c7e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:51.194651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8de9a-592d-420d-9f8e-d9b1958c7e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.195502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8de9a-592d-420d-9f8e-d9b1958c7e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.196013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8de9a-592d-420d-9f8e-d9b1958c7e93 admin admin] PATH_INFO: `/v3/projects/9888e3606d5a48b29f1996f8e7917518/users/a8bce517d8d8401f9f2efefc295f5a30/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.219247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8de9a-592d-420d-9f8e-d9b1958c7e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9888e3606d5a48b29f1996f8e7917518, user_id=a8bce517d8d8401f9f2efefc295f5a30, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:51.223582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8de9a-592d-420d-9f8e-d9b1958c7e93 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:51.232641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b20bd5db-97eb-444e-af74-63e6c81494c4 None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:51.233886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b20bd5db-97eb-444e-af74-63e6c81494c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:51.241302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:51.251379 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 983/1965] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:51 2022] PUT /identity/v3/projects/9888e3606d5a48b29f1996f8e7917518/users/a8bce517d8d8401f9f2efefc295f5a30/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:51.257340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8de9a-592d-420d-9f8e-d9b1958c7e93 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:51.329312 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 983/1966] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 20:55:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:51.330585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20bd5db-97eb-444e-af74-63e6c81494c4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:51.346063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20bd5db-97eb-444e-af74-63e6c81494c4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:51.352047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4033e5-0092-4a11-80d3-7cea90ce35ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:51.352699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4033e5-0092-4a11-80d3-7cea90ce35ca admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.352972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4033e5-0092-4a11-80d3-7cea90ce35ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.353232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4033e5-0092-4a11-80d3-7cea90ce35ca admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.353870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4033e5-0092-4a11-80d3-7cea90ce35ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:51.354311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8de9a-592d-420d-9f8e-d9b1958c7e93 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:51.355718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4033e5-0092-4a11-80d3-7cea90ce35ca admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:51.359723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49497f-69ea-422a-bb69-883d7a407c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:51.361911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49497f-69ea-422a-bb69-883d7a407c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.362291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49497f-69ea-422a-bb69-883d7a407c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.362656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49497f-69ea-422a-bb69-883d7a407c13 admin admin] PATH_INFO: `/v3/projects/279b2d5d2f2647f69be35817b22cdb04/users/131e1f9e976d4a1784cf6b4e55cf87dd/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.365291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49497f-69ea-422a-bb69-883d7a407c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=279b2d5d2f2647f69be35817b22cdb04, user_id=131e1f9e976d4a1784cf6b4e55cf87dd, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:51.368483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49497f-69ea-422a-bb69-883d7a407c13 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:51.374966 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 984/1967] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:51.377507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4033e5-0092-4a11-80d3-7cea90ce35ca admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:51.383538 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 984/1968] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:51 2022] PUT /identity/v3/projects/279b2d5d2f2647f69be35817b22cdb04/users/131e1f9e976d4a1784cf6b4e55cf87dd/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:51.392547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4033e5-0092-4a11-80d3-7cea90ce35ca admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:51.394361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a65678-5249-4dac-94d2-f8bdcf561e76 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.394729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a65678-5249-4dac-94d2-f8bdcf561e76 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.395118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a65678-5249-4dac-94d2-f8bdcf561e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.433972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-38a65678-5249-4dac-94d2-f8bdcf561e76 None None] MFA Rules not processed for user `131e1f9e976d4a1784cf6b4e55cf87dd`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:51.435693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-38a65678-5249-4dac-94d2-f8bdcf561e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:51.445961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:51.490229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75676a2-6a07-4d83-80c1-c6855859abb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:51.491208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75676a2-6a07-4d83-80c1-c6855859abb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.491576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75676a2-6a07-4d83-80c1-c6855859abb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.491937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75676a2-6a07-4d83-80c1-c6855859abb1 admin admin] PATH_INFO: `/v3/projects/9888e3606d5a48b29f1996f8e7917518/users/a8bce517d8d8401f9f2efefc295f5a30/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.498371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75676a2-6a07-4d83-80c1-c6855859abb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9888e3606d5a48b29f1996f8e7917518, user_id=a8bce517d8d8401f9f2efefc295f5a30, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:51.501444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75676a2-6a07-4d83-80c1-c6855859abb1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:51.529471 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 985/1969] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:51 2022] PUT /identity/v3/projects/9888e3606d5a48b29f1996f8e7917518/users/a8bce517d8d8401f9f2efefc295f5a30/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:51.529471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75676a2-6a07-4d83-80c1-c6855859abb1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:51.529471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f75676a2-6a07-4d83-80c1-c6855859abb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:51.574009 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 985/1970] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:51.574009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcab30d-1b73-4d67-a7f1-814dee4a4bde None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.574009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcab30d-1b73-4d67-a7f1-814dee4a4bde None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.574009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcab30d-1b73-4d67-a7f1-814dee4a4bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.653748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bfcab30d-1b73-4d67-a7f1-814dee4a4bde None None] MFA Rules not processed for user `a8bce517d8d8401f9f2efefc295f5a30`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:51.654559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75676a2-6a07-4d83-80c1-c6855859abb1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:51.656674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bfcab30d-1b73-4d67-a7f1-814dee4a4bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:51.661548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5106218-9af7-4524-b25e-360b312752e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:51.662306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5106218-9af7-4524-b25e-360b312752e5 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.662690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5106218-9af7-4524-b25e-360b312752e5 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.663059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5106218-9af7-4524-b25e-360b312752e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.677431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5106218-9af7-4524-b25e-360b312752e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:51.679313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5106218-9af7-4524-b25e-360b312752e5 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:51.692323 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 986/1971] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:51.693455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5106218-9af7-4524-b25e-360b312752e5 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:51.696395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e5106218-9af7-4524-b25e-360b312752e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:51.718193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:51.791609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5106218-9af7-4524-b25e-360b312752e5 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:51.811538 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 986/1972] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:51.824060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133f185-8528-45d1-a30f-015097a5f268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:51.824983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133f185-8528-45d1-a30f-015097a5f268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.825551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcab30d-1b73-4d67-a7f1-814dee4a4bde None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:51.826015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133f185-8528-45d1-a30f-015097a5f268 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.826944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133f185-8528-45d1-a30f-015097a5f268 admin admin] PATH_INFO: `/v3/projects/5a6ddebe39bc4c32914dab26c3b7d437` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.831031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133f185-8528-45d1-a30f-015097a5f268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a6ddebe39bc4c32914dab26c3b7d437)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:51.838781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133f185-8528-45d1-a30f-015097a5f268 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:51.847266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcab30d-1b73-4d67-a7f1-814dee4a4bde None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:51.859789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aca3ff-575c-4ffb-b23a-8d5cafc605ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:51.860151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aca3ff-575c-4ffb-b23a-8d5cafc605ab service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:51.860438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aca3ff-575c-4ffb-b23a-8d5cafc605ab service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:51.860717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aca3ff-575c-4ffb-b23a-8d5cafc605ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:51.972022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e133f185-8528-45d1-a30f-015097a5f268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a6ddebe39bc4c32914dab26c3b7d437', 'request_id': 'req-e133f185-8528-45d1-a30f-015097a5f268'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:51.990944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e133f185-8528-45d1-a30f-015097a5f268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a6ddebe39bc4c32914dab26c3b7d437', 'request_id': 'req-e133f185-8528-45d1-a30f-015097a5f268'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:51.996678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aca3ff-575c-4ffb-b23a-8d5cafc605ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:51.998391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aca3ff-575c-4ffb-b23a-8d5cafc605ab service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:52.009268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e133f185-8528-45d1-a30f-015097a5f268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a6ddebe39bc4c32914dab26c3b7d437', 'request_id': 'req-e133f185-8528-45d1-a30f-015097a5f268'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:52.024746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e133f185-8528-45d1-a30f-015097a5f268 admin admin] The token cache is being invalidate because project 5a6ddebe39bc4c32914dab26c3b7d437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:52.025147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e133f185-8528-45d1-a30f-015097a5f268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:52.029810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e133f185-8528-45d1-a30f-015097a5f268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:52.031954 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 987/1973] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:51 2022] DELETE /identity/v3/projects/5a6ddebe39bc4c32914dab26c3b7d437 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:52.038450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea87c97-7648-42e2-bce7-56e796f59e58 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:52.038450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea87c97-7648-42e2-bce7-56e796f59e58 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:52.038450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea87c97-7648-42e2-bce7-56e796f59e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:52.047164 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 987/1974] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:55:52.063002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cea87c97-7648-42e2-bce7-56e796f59e58 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:52.064494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cea87c97-7648-42e2-bce7-56e796f59e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:52.150102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aca3ff-575c-4ffb-b23a-8d5cafc605ab service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:52.169279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aca3ff-575c-4ffb-b23a-8d5cafc605ab service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:52.196312 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 988/1975] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:52.206992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02fd4-26db-4d61-972e-487b35cd44c1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:52.207295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02fd4-26db-4d61-972e-487b35cd44c1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:52.211760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02fd4-26db-4d61-972e-487b35cd44c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:52.216996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:52.284223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1be02fd4-26db-4d61-972e-487b35cd44c1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:52.285483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1be02fd4-26db-4d61-972e-487b35cd44c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:52.305169 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 989/1976] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:52.309356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be02fd4-26db-4d61-972e-487b35cd44c1 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:52.330369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff6b4e-cc81-4000-81d6-b4f904a610b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:52.331013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff6b4e-cc81-4000-81d6-b4f904a610b6 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:52.331239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff6b4e-cc81-4000-81d6-b4f904a610b6 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:52.331424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff6b4e-cc81-4000-81d6-b4f904a610b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:52.333966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6fff6b4e-cc81-4000-81d6-b4f904a610b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:52.341308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be02fd4-26db-4d61-972e-487b35cd44c1 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:52.352874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765118c-c6e8-40f0-b3cf-04ec80ff3f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:52.354746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765118c-c6e8-40f0-b3cf-04ec80ff3f84 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:52.354992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765118c-c6e8-40f0-b3cf-04ec80ff3f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:52.355262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765118c-c6e8-40f0-b3cf-04ec80ff3f84 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:52.363000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765118c-c6e8-40f0-b3cf-04ec80ff3f84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:55:52.363358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765118c-c6e8-40f0-b3cf-04ec80ff3f84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:52.367836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765118c-c6e8-40f0-b3cf-04ec80ff3f84 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:52.407422 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 990/1977] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:55:52.408463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765118c-c6e8-40f0-b3cf-04ec80ff3f84 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:52.433274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765118c-c6e8-40f0-b3cf-04ec80ff3f84 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:52.440462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181885fd-6685-4a5b-8bea-1a896b222258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:52.442867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181885fd-6685-4a5b-8bea-1a896b222258 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:52.443415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181885fd-6685-4a5b-8bea-1a896b222258 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:52.443703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181885fd-6685-4a5b-8bea-1a896b222258 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:52.444434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181885fd-6685-4a5b-8bea-1a896b222258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:52.447249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181885fd-6685-4a5b-8bea-1a896b222258 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:52.486198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:52.496432 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 991/1978] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:52 2022] POST /identity/v3/projects => generated 375 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:52.500727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181885fd-6685-4a5b-8bea-1a896b222258 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:52.507651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff6b4e-cc81-4000-81d6-b4f904a610b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:52.509285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff6b4e-cc81-4000-81d6-b4f904a610b6 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:52.561537 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 988/1979] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:55:52.574185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff6b4e-cc81-4000-81d6-b4f904a610b6 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:52.575845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6fff6b4e-cc81-4000-81d6-b4f904a610b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:52.632997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff6b4e-cc81-4000-81d6-b4f904a610b6 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:52.711132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181885fd-6685-4a5b-8bea-1a896b222258 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:52.717011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc18f6d-dc06-4796-9df1-9b4576d42238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:52.717740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc18f6d-dc06-4796-9df1-9b4576d42238 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:52.718110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc18f6d-dc06-4796-9df1-9b4576d42238 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:52.718495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc18f6d-dc06-4796-9df1-9b4576d42238 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:52.719259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc18f6d-dc06-4796-9df1-9b4576d42238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:52.725994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc18f6d-dc06-4796-9df1-9b4576d42238 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:52.775717 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 992/1980] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:52 2022] POST /identity/v3/users => generated 288 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:52.777190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc18f6d-dc06-4796-9df1-9b4576d42238 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:52.812237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc18f6d-dc06-4796-9df1-9b4576d42238 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:52.833957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3813b-5370-4f4b-a552-cb800ef58ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:52.834791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3813b-5370-4f4b-a552-cb800ef58ebc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:52.835166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3813b-5370-4f4b-a552-cb800ef58ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:52.835989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:52.836396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3813b-5370-4f4b-a552-cb800ef58ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:52.837155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3813b-5370-4f4b-a552-cb800ef58ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:52.843100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:52.845164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3813b-5370-4f4b-a552-cb800ef58ebc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:52.846253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:52.846964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] PATH_INFO: `/v3/users/d846054832cb45c2bb0b5b8af9e76051` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:52.853676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d846054832cb45c2bb0b5b8af9e76051)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:52.855770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:52.873441 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 993/1981] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:52.880083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3813b-5370-4f4b-a552-cb800ef58ebc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:52.885497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-34e3813b-5370-4f4b-a552-cb800ef58ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:52.908658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3813b-5370-4f4b-a552-cb800ef58ebc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:52.915289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176814aa-cfa3-41ec-b620-392e482152c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:52.915984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176814aa-cfa3-41ec-b620-392e482152c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:52.916252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176814aa-cfa3-41ec-b620-392e482152c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:52.916510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176814aa-cfa3-41ec-b620-392e482152c9 admin admin] PATH_INFO: `/v3/users/1e6b31adfd68421c8299ac53656490e6` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:52.919433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176814aa-cfa3-41ec-b620-392e482152c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e6b31adfd68421c8299ac53656490e6)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:52.921532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176814aa-cfa3-41ec-b620-392e482152c9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:52.959030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd846054832cb45c2bb0b5b8af9e76051'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:52.976137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd846054832cb45c2bb0b5b8af9e76051'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:52.992689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:53.006149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd846054832cb45c2bb0b5b8af9e76051'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:53.006799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-176814aa-cfa3-41ec-b620-392e482152c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e6b31adfd68421c8299ac53656490e6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:53.007532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd846054832cb45c2bb0b5b8af9e76051'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:53.008408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd846054832cb45c2bb0b5b8af9e76051'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:53.022282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-176814aa-cfa3-41ec-b620-392e482152c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e6b31adfd68421c8299ac53656490e6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:53.042724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-176814aa-cfa3-41ec-b620-392e482152c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e6b31adfd68421c8299ac53656490e6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:53.066570 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 989/1982] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:52 2022] DELETE /identity/v3/users/d846054832cb45c2bb0b5b8af9e76051 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:53.066570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:53.066880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-176814aa-cfa3-41ec-b620-392e482152c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e6b31adfd68421c8299ac53656490e6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:53.072997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-176814aa-cfa3-41ec-b620-392e482152c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e6b31adfd68421c8299ac53656490e6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:53.075393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:53.090119 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 994/1983] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:52 2022] DELETE /identity/v3/users/1e6b31adfd68421c8299ac53656490e6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:53.090119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f742741-f75f-4560-80b8-7fee0e8cd76a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:53.090119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f742741-f75f-4560-80b8-7fee0e8cd76a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:53.090119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f742741-f75f-4560-80b8-7fee0e8cd76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:53.133622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8f742741-f75f-4560-80b8-7fee0e8cd76a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:53.134843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8f742741-f75f-4560-80b8-7fee0e8cd76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:53.196011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:53.221167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b6b95-b516-43a4-81cd-81e139862cb1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:53.229399 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 995/1984] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:53.230189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751c46e-a239-44d0-9494-8ab27dad7616 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:53.230574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751c46e-a239-44d0-9494-8ab27dad7616 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:53.230964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751c46e-a239-44d0-9494-8ab27dad7616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:53.241407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c8da2b-2976-4367-bc4a-6ae7db117b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:53.242046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c8da2b-2976-4367-bc4a-6ae7db117b15 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:53.242313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c8da2b-2976-4367-bc4a-6ae7db117b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:53.242571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c8da2b-2976-4367-bc4a-6ae7db117b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:53.243240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c8da2b-2976-4367-bc4a-6ae7db117b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:53.244928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c8da2b-2976-4367-bc4a-6ae7db117b15 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:53.266126 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 990/1985] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:53.269597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfad02c-22fd-4ef3-8981-908d42f43920 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:53.269941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfad02c-22fd-4ef3-8981-908d42f43920 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:53.270276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfad02c-22fd-4ef3-8981-908d42f43920 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:53.276711 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 991/1986] 10.222.0.23 () {54 vars in 921 bytes} [Sat Aug 20 20:55:53 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:55:53.277545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfad02c-22fd-4ef3-8981-908d42f43920 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:53.293227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4cfad02c-22fd-4ef3-8981-908d42f43920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:53.302861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7751c46e-a239-44d0-9494-8ab27dad7616 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:53.304704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7751c46e-a239-44d0-9494-8ab27dad7616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:53.319085 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 996/1987] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:53.320751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5b395-1cb7-4a34-8f3a-f9e4cf7ad326 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:53.321611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5b395-1cb7-4a34-8f3a-f9e4cf7ad326 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:53.321981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5b395-1cb7-4a34-8f3a-f9e4cf7ad326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:53.325236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfad02c-22fd-4ef3-8981-908d42f43920 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:53.350179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec720db-9f1f-460b-86cf-c03452df5c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:53.355011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec720db-9f1f-460b-86cf-c03452df5c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:53.355378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec720db-9f1f-460b-86cf-c03452df5c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:53.355571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec720db-9f1f-460b-86cf-c03452df5c43 admin admin] PATH_INFO: `/v3/projects/795b560b3af140a38757105cd17530c3/users/905d35c1df7248cfa32d5349f898e9c1/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:53.398269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f0c5b395-1cb7-4a34-8f3a-f9e4cf7ad326 None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:53.399474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f0c5b395-1cb7-4a34-8f3a-f9e4cf7ad326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:53.402485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec720db-9f1f-460b-86cf-c03452df5c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=795b560b3af140a38757105cd17530c3, user_id=905d35c1df7248cfa32d5349f898e9c1, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:53.406427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec720db-9f1f-460b-86cf-c03452df5c43 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:53.437270 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 992/1988] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:53 2022] PUT /identity/v3/projects/795b560b3af140a38757105cd17530c3/users/905d35c1df7248cfa32d5349f898e9c1/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:53.446690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec720db-9f1f-460b-86cf-c03452df5c43 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:53.457167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5ec720db-9f1f-460b-86cf-c03452df5c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:53.579204 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 997/1989] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 20:55:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:53.579422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec720db-9f1f-460b-86cf-c03452df5c43 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:53.580494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5b395-1cb7-4a34-8f3a-f9e4cf7ad326 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:53.588827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cdc15-dc73-47e6-bf38-4450e39a0142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:53.590045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cdc15-dc73-47e6-bf38-4450e39a0142 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:53.590316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cdc15-dc73-47e6-bf38-4450e39a0142 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:53.590575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cdc15-dc73-47e6-bf38-4450e39a0142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:53.674326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5b395-1cb7-4a34-8f3a-f9e4cf7ad326 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:53.684221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef24d8-7eb2-48dc-91cc-951f678bb10f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:53.685425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef24d8-7eb2-48dc-91cc-951f678bb10f service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:53.686136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef24d8-7eb2-48dc-91cc-951f678bb10f service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:53.687572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef24d8-7eb2-48dc-91cc-951f678bb10f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:53.715987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cdc15-dc73-47e6-bf38-4450e39a0142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:53.717608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cdc15-dc73-47e6-bf38-4450e39a0142 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:53.724025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:53.740487 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 993/1990] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:53.742484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cdc15-dc73-47e6-bf38-4450e39a0142 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:53.747998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3f1cdc15-dc73-47e6-bf38-4450e39a0142 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:53.780530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cdc15-dc73-47e6-bf38-4450e39a0142 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:53.792354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:53.793045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:53.793480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:53.793749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] PATH_INFO: `/v3/users/a3f7a1130dff44708fc03a7c5757efa7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:53.801698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3f7a1130dff44708fc03a7c5757efa7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:53.804633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:53.846320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef24d8-7eb2-48dc-91cc-951f678bb10f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:53.849930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef24d8-7eb2-48dc-91cc-951f678bb10f service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:53.870295 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 998/1991] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:53.872485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1001259-dc3f-4f61-9708-e494e0118161 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:53.872926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1001259-dc3f-4f61-9708-e494e0118161 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:53.873478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1001259-dc3f-4f61-9708-e494e0118161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:53.922865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3f7a1130dff44708fc03a7c5757efa7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:53.923425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b1001259-dc3f-4f61-9708-e494e0118161 None None] MFA Rules not processed for user `905d35c1df7248cfa32d5349f898e9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:53.926232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b1001259-dc3f-4f61-9708-e494e0118161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:53.931726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3f7a1130dff44708fc03a7c5757efa7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:53.951389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3f7a1130dff44708fc03a7c5757efa7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:53.952334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3f7a1130dff44708fc03a7c5757efa7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:53.953668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3f7a1130dff44708fc03a7c5757efa7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:53.966940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:53.977434 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 994/1992] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:53 2022] DELETE /identity/v3/users/a3f7a1130dff44708fc03a7c5757efa7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:53.978794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:53.980529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:54.059429 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 999/1993] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:54.064698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad76c9c-b16d-4a5c-8512-96a251b2a858 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:54.065558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad76c9c-b16d-4a5c-8512-96a251b2a858 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:54.066776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad76c9c-b16d-4a5c-8512-96a251b2a858 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:54.071507 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1000/1994] 10.222.0.23 () {54 vars in 921 bytes} [Sat Aug 20 20:55:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:55:54.072953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad76c9c-b16d-4a5c-8512-96a251b2a858 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:54.075106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5ad76c9c-b16d-4a5c-8512-96a251b2a858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:54.106728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c70ffa-3d65-493d-9ff2-0c14591a138b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:54.116142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005f53e-73a9-4cdf-b62c-f9e9368e9f1d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:54.117312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005f53e-73a9-4cdf-b62c-f9e9368e9f1d service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:54.131210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005f53e-73a9-4cdf-b62c-f9e9368e9f1d service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:54.132417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005f53e-73a9-4cdf-b62c-f9e9368e9f1d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:54.137699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0005f53e-73a9-4cdf-b62c-f9e9368e9f1d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:54.172664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad76c9c-b16d-4a5c-8512-96a251b2a858 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:54.173527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005f53e-73a9-4cdf-b62c-f9e9368e9f1d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:54.178624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9715490-acd3-4b1f-a00e-cbe2ff72749e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:54.179391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9715490-acd3-4b1f-a00e-cbe2ff72749e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:54.179694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005f53e-73a9-4cdf-b62c-f9e9368e9f1d service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:54.180011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9715490-acd3-4b1f-a00e-cbe2ff72749e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:54.180338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9715490-acd3-4b1f-a00e-cbe2ff72749e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:54.181102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9715490-acd3-4b1f-a00e-cbe2ff72749e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:54.183182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9715490-acd3-4b1f-a00e-cbe2ff72749e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:54.215044 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 995/1995] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:54.228297 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1001/1996] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:54 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:54.232708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475fd61-6b13-420a-b2ce-b2619f5d698c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:54.234179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475fd61-6b13-420a-b2ce-b2619f5d698c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:54.234287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9715490-acd3-4b1f-a00e-cbe2ff72749e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:54.234679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475fd61-6b13-420a-b2ce-b2619f5d698c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:54.294106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e475fd61-6b13-420a-b2ce-b2619f5d698c None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:54.299787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e475fd61-6b13-420a-b2ce-b2619f5d698c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:54.317137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9715490-acd3-4b1f-a00e-cbe2ff72749e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:54.324165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ff7e7-8ef1-4f5b-a165-03bf3d4b0872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:54.324871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ff7e7-8ef1-4f5b-a165-03bf3d4b0872 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:54.325144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ff7e7-8ef1-4f5b-a165-03bf3d4b0872 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:54.325412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ff7e7-8ef1-4f5b-a165-03bf3d4b0872 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:54.326060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ff7e7-8ef1-4f5b-a165-03bf3d4b0872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:54.329436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ff7e7-8ef1-4f5b-a165-03bf3d4b0872 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:54.371273 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1002/1997] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:54 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:54.372499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ff7e7-8ef1-4f5b-a165-03bf3d4b0872 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:54.392949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ff7e7-8ef1-4f5b-a165-03bf3d4b0872 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:54.411788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27f89a-58c4-4d99-a624-6f06a05fb2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:54.415116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27f89a-58c4-4d99-a624-6f06a05fb2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:54.415966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27f89a-58c4-4d99-a624-6f06a05fb2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:54.416671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27f89a-58c4-4d99-a624-6f06a05fb2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:54.418668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27f89a-58c4-4d99-a624-6f06a05fb2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:54.420143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27f89a-58c4-4d99-a624-6f06a05fb2e0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:54.442205 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1003/1998] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:54.443734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27f89a-58c4-4d99-a624-6f06a05fb2e0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:54.451810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:54.459993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27f89a-58c4-4d99-a624-6f06a05fb2e0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:54.466517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997c129-1e7e-48f9-a180-71631741f849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:54.467135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997c129-1e7e-48f9-a180-71631741f849 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:54.467512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997c129-1e7e-48f9-a180-71631741f849 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:54.467880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997c129-1e7e-48f9-a180-71631741f849 admin admin] PATH_INFO: `/v3/projects/b3157bb9ec164b84976e80a3d336bb28/users/4f107659c22d4e0b893adb4c53045635/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:54.478404 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 996/1999] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 20:55:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:54.509874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997c129-1e7e-48f9-a180-71631741f849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3157bb9ec164b84976e80a3d336bb28, user_id=4f107659c22d4e0b893adb4c53045635, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:54.510742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475fd61-6b13-420a-b2ce-b2619f5d698c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:54.514413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997c129-1e7e-48f9-a180-71631741f849 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:54.525844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475fd61-6b13-420a-b2ce-b2619f5d698c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:54.530374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:54.543962 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1004/2000] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:54 2022] PUT /identity/v3/projects/b3157bb9ec164b84976e80a3d336bb28/users/4f107659c22d4e0b893adb4c53045635/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:54.546360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997c129-1e7e-48f9-a180-71631741f849 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:54.639788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dad41-6ab3-4cc9-85eb-ffefc7f0e0e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:54.640464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dad41-6ab3-4cc9-85eb-ffefc7f0e0e2 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:54.640741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dad41-6ab3-4cc9-85eb-ffefc7f0e0e2 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:54.641016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dad41-6ab3-4cc9-85eb-ffefc7f0e0e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:54.659276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dad41-6ab3-4cc9-85eb-ffefc7f0e0e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:54.660840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dad41-6ab3-4cc9-85eb-ffefc7f0e0e2 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:54.670587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997c129-1e7e-48f9-a180-71631741f849 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:54.677305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267e58c-1b7c-43b0-a37e-bd49fc3c6c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:54.678560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267e58c-1b7c-43b0-a37e-bd49fc3c6c6e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:54.679160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267e58c-1b7c-43b0-a37e-bd49fc3c6c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:54.679592 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 997/2001] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:54.680497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267e58c-1b7c-43b0-a37e-bd49fc3c6c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:54.681471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9267e58c-1b7c-43b0-a37e-bd49fc3c6c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:54.683537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9267e58c-1b7c-43b0-a37e-bd49fc3c6c6e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:54.701432 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1005/2002] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:54.702490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dad41-6ab3-4cc9-85eb-ffefc7f0e0e2 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:54.735728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dad41-6ab3-4cc9-85eb-ffefc7f0e0e2 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:54.741697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c09d1-7bc3-46df-8ab5-9f309b85ac02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:54.742544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c09d1-7bc3-46df-8ab5-9f309b85ac02 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:54.742737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c09d1-7bc3-46df-8ab5-9f309b85ac02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:54.743045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c09d1-7bc3-46df-8ab5-9f309b85ac02 admin admin] PATH_INFO: `/v3/projects/b3157bb9ec164b84976e80a3d336bb28/users/4f107659c22d4e0b893adb4c53045635/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:54.746052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c09d1-7bc3-46df-8ab5-9f309b85ac02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3157bb9ec164b84976e80a3d336bb28, user_id=4f107659c22d4e0b893adb4c53045635, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:54.750233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c09d1-7bc3-46df-8ab5-9f309b85ac02 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:54.772502 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 998/2003] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:54 2022] PUT /identity/v3/projects/b3157bb9ec164b84976e80a3d336bb28/users/4f107659c22d4e0b893adb4c53045635/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:55:54.777370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08ed44-a45a-43f4-ac89-7c6a5cd68651 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:54.777760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08ed44-a45a-43f4-ac89-7c6a5cd68651 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:54.778136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08ed44-a45a-43f4-ac89-7c6a5cd68651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:54.785161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c09d1-7bc3-46df-8ab5-9f309b85ac02 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:54.788526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6f0c09d1-7bc3-46df-8ab5-9f309b85ac02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:54.891629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5f08ed44-a45a-43f4-ac89-7c6a5cd68651 None None] MFA Rules not processed for user `4f107659c22d4e0b893adb4c53045635`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:54.892956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5f08ed44-a45a-43f4-ac89-7c6a5cd68651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:54.924927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c09d1-7bc3-46df-8ab5-9f309b85ac02 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:54.933432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:54.935035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:54.935035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:54.936639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] PATH_INFO: `/v3/projects/922daf4916c843cfa61964471f91d4d1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:54.937835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=922daf4916c843cfa61964471f91d4d1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:54.940871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:55.008810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '922daf4916c843cfa61964471f91d4d1', 'request_id': 'req-c43ec0ef-350e-4522-9250-208a66c3c834'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.015587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '922daf4916c843cfa61964471f91d4d1', 'request_id': 'req-c43ec0ef-350e-4522-9250-208a66c3c834'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.021219 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1006/2004] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:55.022229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08ed44-a45a-43f4-ac89-7c6a5cd68651 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:55.029705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5f08ed44-a45a-43f4-ac89-7c6a5cd68651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:55.033458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '922daf4916c843cfa61964471f91d4d1', 'request_id': 'req-c43ec0ef-350e-4522-9250-208a66c3c834'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.045632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] The token cache is being invalidate because project 922daf4916c843cfa61964471f91d4d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:55.046027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.046436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08ed44-a45a-43f4-ac89-7c6a5cd68651 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:55.046668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.048150 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 999/2005] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:54 2022] DELETE /identity/v3/projects/922daf4916c843cfa61964471f91d4d1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:55.051621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:55.051621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:55.057940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:55.113158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:55.113807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:55.114089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:55.114493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] PATH_INFO: `/v3/projects/54cdd4799ffa49d5bcca23b8d5cbd2b2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:55.115665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54cdd4799ffa49d5bcca23b8d5cbd2b2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:55.117744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:55.141872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ec0ef-350e-4522-9250-208a66c3c834 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:55.146785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:55.150380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbcfb5b-e669-4a46-a192-95c0256b8d1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:55.151161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcfb5b-e669-4a46-a192-95c0256b8d1c service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:55.151627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcfb5b-e669-4a46-a192-95c0256b8d1c service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:55.151895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcfb5b-e669-4a46-a192-95c0256b8d1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:55.154490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2bbcfb5b-e669-4a46-a192-95c0256b8d1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:55.173337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54cdd4799ffa49d5bcca23b8d5cbd2b2', 'request_id': 'req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.183106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54cdd4799ffa49d5bcca23b8d5cbd2b2', 'request_id': 'req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.190605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54cdd4799ffa49d5bcca23b8d5cbd2b2', 'request_id': 'req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.205603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] The token cache is being invalidate because project 54cdd4799ffa49d5bcca23b8d5cbd2b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:55.205888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.206622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.208504 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1007/2006] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:55 2022] DELETE /identity/v3/projects/54cdd4799ffa49d5bcca23b8d5cbd2b2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:55.209376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:55.210739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:55.240714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbcfb5b-e669-4a46-a192-95c0256b8d1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:55.242429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbcfb5b-e669-4a46-a192-95c0256b8d1c service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:55.253467 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1000/2007] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:55.265762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9aea6-66b9-42a3-86cc-98b1d5d077b0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:55.266046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9aea6-66b9-42a3-86cc-98b1d5d077b0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:55.266314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9aea6-66b9-42a3-86cc-98b1d5d077b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:55.290994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822217ab-e6fe-44fb-a0d4-26bb4e34d6f5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:55.295713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:55.296451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:55.296772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:55.297506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] PATH_INFO: `/v3/users/a8bce517d8d8401f9f2efefc295f5a30` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:55.299132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8bce517d8d8401f9f2efefc295f5a30)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:55.301997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:55.312542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0df9aea6-66b9-42a3-86cc-98b1d5d077b0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:55.313887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0df9aea6-66b9-42a3-86cc-98b1d5d077b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:55.327073 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1001/2008] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:55.328307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9aea6-66b9-42a3-86cc-98b1d5d077b0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:55.329643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0df9aea6-66b9-42a3-86cc-98b1d5d077b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:55.377308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8bce517d8d8401f9f2efefc295f5a30'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.390011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8bce517d8d8401f9f2efefc295f5a30'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.399722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8bce517d8d8401f9f2efefc295f5a30'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.417102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8bce517d8d8401f9f2efefc295f5a30'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.418747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8bce517d8d8401f9f2efefc295f5a30'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.426772 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1008/2009] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:55 2022] DELETE /identity/v3/users/a8bce517d8d8401f9f2efefc295f5a30 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:55.428566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:55.429057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:55.447445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9aea6-66b9-42a3-86cc-98b1d5d077b0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:55.477787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:55.486912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:55.542216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86990fe-4949-4dc9-8940-f2b0ed38d114 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:55.550946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:55.552331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:55.552641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:55.552949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe admin admin] PATH_INFO: `/v3/projects/aa99ed809f344171923311bef4b8bc23` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:55.554412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa99ed809f344171923311bef4b8bc23)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:55.556498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:55.567415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fa097-d9ef-4e77-b70d-225576294522 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:55.569587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa097-d9ef-4e77-b70d-225576294522 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:55.571399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa097-d9ef-4e77-b70d-225576294522 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:55.571874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa097-d9ef-4e77-b70d-225576294522 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:55.575096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a98fa097-d9ef-4e77-b70d-225576294522 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:55.632280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa99ed809f344171923311bef4b8bc23', 'request_id': 'req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.649599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa99ed809f344171923311bef4b8bc23', 'request_id': 'req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.660796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa99ed809f344171923311bef4b8bc23', 'request_id': 'req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.681121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe admin admin] The token cache is being invalidate because project aa99ed809f344171923311bef4b8bc23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:55.681121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.681997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5088c37c-24a9-4bbf-b087-23ca2d0f3afe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:55.688581 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1009/2010] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:55 2022] DELETE /identity/v3/projects/aa99ed809f344171923311bef4b8bc23 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:55.691028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effb8dc-e0b9-47de-9ab8-2811eb339b89 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:55.691436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effb8dc-e0b9-47de-9ab8-2811eb339b89 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:55.691935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effb8dc-e0b9-47de-9ab8-2811eb339b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:55.718302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1effb8dc-e0b9-47de-9ab8-2811eb339b89 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:55.720332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1effb8dc-e0b9-47de-9ab8-2811eb339b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:55.740983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fa097-d9ef-4e77-b70d-225576294522 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:55.743626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fa097-d9ef-4e77-b70d-225576294522 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:55.755885 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1002/2011] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:55.756884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fa097-d9ef-4e77-b70d-225576294522 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:55.771323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fa097-d9ef-4e77-b70d-225576294522 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:55.871303 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1010/2012] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:55.871303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effb8dc-e0b9-47de-9ab8-2811eb339b89 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:55.871303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1effb8dc-e0b9-47de-9ab8-2811eb339b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:55.887302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effb8dc-e0b9-47de-9ab8-2811eb339b89 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:55.893742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:55.894608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:55.895095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:55.895432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] PATH_INFO: `/v3/projects/9888e3606d5a48b29f1996f8e7917518` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:55.898291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9888e3606d5a48b29f1996f8e7917518)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:55.899745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:55.906764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4cd8d-53fb-4c72-8463-1b62b98fd62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:55.908412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4cd8d-53fb-4c72-8463-1b62b98fd62b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:55.908841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4cd8d-53fb-4c72-8463-1b62b98fd62b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:55.909258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4cd8d-53fb-4c72-8463-1b62b98fd62b admin admin] PATH_INFO: `/v3/users/131e1f9e976d4a1784cf6b4e55cf87dd` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:55.910451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4cd8d-53fb-4c72-8463-1b62b98fd62b admin admin] RBAC: Authorizing `identity:delete_user(user_id=131e1f9e976d4a1784cf6b4e55cf87dd)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:55.912356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4cd8d-53fb-4c72-8463-1b62b98fd62b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:55.997662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9888e3606d5a48b29f1996f8e7917518', 'request_id': 'req-301134b0-4186-4302-9a7d-07a27e628dfb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.007914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9888e3606d5a48b29f1996f8e7917518', 'request_id': 'req-301134b0-4186-4302-9a7d-07a27e628dfb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.011262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51c4cd8d-53fb-4c72-8463-1b62b98fd62b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '131e1f9e976d4a1784cf6b4e55cf87dd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.011415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:56.027952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51c4cd8d-53fb-4c72-8463-1b62b98fd62b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '131e1f9e976d4a1784cf6b4e55cf87dd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.033677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9888e3606d5a48b29f1996f8e7917518', 'request_id': 'req-301134b0-4186-4302-9a7d-07a27e628dfb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.049756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:56.053154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51c4cd8d-53fb-4c72-8463-1b62b98fd62b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '131e1f9e976d4a1784cf6b4e55cf87dd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.054566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51c4cd8d-53fb-4c72-8463-1b62b98fd62b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '131e1f9e976d4a1784cf6b4e55cf87dd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.055089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] The token cache is being invalidate because project 9888e3606d5a48b29f1996f8e7917518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:56.055554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.056117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51c4cd8d-53fb-4c72-8463-1b62b98fd62b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '131e1f9e976d4a1784cf6b4e55cf87dd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.056855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.059012 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1011/2013] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:55 2022] DELETE /identity/v3/projects/9888e3606d5a48b29f1996f8e7917518 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:56.063150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:56.064989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:56.075940 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1003/2014] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:55 2022] DELETE /identity/v3/users/131e1f9e976d4a1784cf6b4e55cf87dd => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:56.077541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d62652-40f0-4db0-86b7-35813a8dfde8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.078019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d62652-40f0-4db0-86b7-35813a8dfde8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.078505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d62652-40f0-4db0-86b7-35813a8dfde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.108738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-50d62652-40f0-4db0-86b7-35813a8dfde8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:56.111189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-50d62652-40f0-4db0-86b7-35813a8dfde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:56.189645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301134b0-4186-4302-9a7d-07a27e628dfb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:56.247196 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1004/2015] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:56.247196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d62652-40f0-4db0-86b7-35813a8dfde8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:56.252419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-50d62652-40f0-4db0-86b7-35813a8dfde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:56.273764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ce279-4bca-45f4-a912-0d61fd12e2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:56.274381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ce279-4bca-45f4-a912-0d61fd12e2fc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.274654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ce279-4bca-45f4-a912-0d61fd12e2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.274920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ce279-4bca-45f4-a912-0d61fd12e2fc admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.275561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ce279-4bca-45f4-a912-0d61fd12e2fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:55:56.275863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ce279-4bca-45f4-a912-0d61fd12e2fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:56.277751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ce279-4bca-45f4-a912-0d61fd12e2fc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:56.290514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d62652-40f0-4db0-86b7-35813a8dfde8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:56.304496 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1012/2016] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:55:56.304496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f6d05-8eeb-4c21-b685-504bb9ee8e9c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.304496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f6d05-8eeb-4c21-b685-504bb9ee8e9c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.304496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f6d05-8eeb-4c21-b685-504bb9ee8e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.309620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:56.310786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.311254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.311726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] PATH_INFO: `/v3/users/905d35c1df7248cfa32d5349f898e9c1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.317890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=905d35c1df7248cfa32d5349f898e9c1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:56.321700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:56.326642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bc0f6d05-8eeb-4c21-b685-504bb9ee8e9c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:56.327927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bc0f6d05-8eeb-4c21-b685-504bb9ee8e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:56.340649 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1013/2017] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:56.340649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f6d05-8eeb-4c21-b685-504bb9ee8e9c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:56.351403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f6d05-8eeb-4c21-b685-504bb9ee8e9c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:56.356479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc540b7b-8f33-4aa0-b8b9-dbb98840a0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:56.357095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc540b7b-8f33-4aa0-b8b9-dbb98840a0b8 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.357431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc540b7b-8f33-4aa0-b8b9-dbb98840a0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.357760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc540b7b-8f33-4aa0-b8b9-dbb98840a0b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.358518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc540b7b-8f33-4aa0-b8b9-dbb98840a0b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:56.361098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc540b7b-8f33-4aa0-b8b9-dbb98840a0b8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:56.415440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '905d35c1df7248cfa32d5349f898e9c1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.419130 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1014/2018] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:56 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:56.432860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc540b7b-8f33-4aa0-b8b9-dbb98840a0b8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:56.434007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '905d35c1df7248cfa32d5349f898e9c1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.451676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '905d35c1df7248cfa32d5349f898e9c1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.452478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '905d35c1df7248cfa32d5349f898e9c1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.453524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '905d35c1df7248cfa32d5349f898e9c1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:56.472206 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1005/2019] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:56 2022] DELETE /identity/v3/users/905d35c1df7248cfa32d5349f898e9c1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:56.474285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:56.475436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:56.574946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:56.580734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:56.584252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970845a-3174-45fc-871c-b32b7b1da5c4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:56.590282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc540b7b-8f33-4aa0-b8b9-dbb98840a0b8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:56.596288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb194e09-3f6a-4fd8-a085-27e30bd7c313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:56.597219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb194e09-3f6a-4fd8-a085-27e30bd7c313 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.597604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb194e09-3f6a-4fd8-a085-27e30bd7c313 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.597880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb194e09-3f6a-4fd8-a085-27e30bd7c313 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.598517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb194e09-3f6a-4fd8-a085-27e30bd7c313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:56.600653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610d09f-8016-4b94-a7ae-7c899871d503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:56.602400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d09f-8016-4b94-a7ae-7c899871d503 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.603012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d09f-8016-4b94-a7ae-7c899871d503 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.603538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d09f-8016-4b94-a7ae-7c899871d503 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.605184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610d09f-8016-4b94-a7ae-7c899871d503 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:55:56.605305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb194e09-3f6a-4fd8-a085-27e30bd7c313 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:56.605698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610d09f-8016-4b94-a7ae-7c899871d503 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:56.609915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610d09f-8016-4b94-a7ae-7c899871d503 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:56.634927 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1015/2020] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:55:56.637967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd08df7-bc93-4adb-9e4d-720ab44175a5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.638381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd08df7-bc93-4adb-9e4d-720ab44175a5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.638942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd08df7-bc93-4adb-9e4d-720ab44175a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.647089 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1006/2021] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:56 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:56.648228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb194e09-3f6a-4fd8-a085-27e30bd7c313 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:56.650520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bb194e09-3f6a-4fd8-a085-27e30bd7c313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:56.667165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb194e09-3f6a-4fd8-a085-27e30bd7c313 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:56.672768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580055e9-5eda-4c86-be0d-0c7d7c18de06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:56.673650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580055e9-5eda-4c86-be0d-0c7d7c18de06 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.674080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580055e9-5eda-4c86-be0d-0c7d7c18de06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.674504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580055e9-5eda-4c86-be0d-0c7d7c18de06 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.675719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580055e9-5eda-4c86-be0d-0c7d7c18de06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:56.677851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580055e9-5eda-4c86-be0d-0c7d7c18de06 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:56.710092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9fd08df7-bc93-4adb-9e4d-720ab44175a5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:56.712027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9fd08df7-bc93-4adb-9e4d-720ab44175a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:56.715709 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1007/2022] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:56 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:56.717579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580055e9-5eda-4c86-be0d-0c7d7c18de06 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:56.797619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580055e9-5eda-4c86-be0d-0c7d7c18de06 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:56.803672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807206b-4779-4830-b8fe-bf85b248f3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:56.804410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807206b-4779-4830-b8fe-bf85b248f3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.804697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807206b-4779-4830-b8fe-bf85b248f3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.804953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807206b-4779-4830-b8fe-bf85b248f3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.805690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807206b-4779-4830-b8fe-bf85b248f3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:56.807693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807206b-4779-4830-b8fe-bf85b248f3c2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:56.819248 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1008/2023] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:56.824091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807206b-4779-4830-b8fe-bf85b248f3c2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:56.831555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807206b-4779-4830-b8fe-bf85b248f3c2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:56.837630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3671d-3c4b-4f29-be02-0270900b1736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:56.838339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3671d-3c4b-4f29-be02-0270900b1736 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.838623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3671d-3c4b-4f29-be02-0270900b1736 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.838900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3671d-3c4b-4f29-be02-0270900b1736 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.839758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d3671d-3c4b-4f29-be02-0270900b1736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:56.841843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d3671d-3c4b-4f29-be02-0270900b1736 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:56.886197 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1009/2024] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:56 2022] POST /identity/v3/users => generated 294 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:56.888328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3671d-3c4b-4f29-be02-0270900b1736 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:56.916019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3671d-3c4b-4f29-be02-0270900b1736 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:56.917358 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1016/2025] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:56.918273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd08df7-bc93-4adb-9e4d-720ab44175a5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:56.920331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9fd08df7-bc93-4adb-9e4d-720ab44175a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:56.929258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd4704-06c3-4727-b982-ab6cee1aef53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:56.929886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd4704-06c3-4727-b982-ab6cee1aef53 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.931147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd4704-06c3-4727-b982-ab6cee1aef53 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.931147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd4704-06c3-4727-b982-ab6cee1aef53 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.931147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd4704-06c3-4727-b982-ab6cee1aef53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:56.932955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd4704-06c3-4727-b982-ab6cee1aef53 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:56.951729 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1010/2026] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:56.952726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd4704-06c3-4727-b982-ab6cee1aef53 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:56.969807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd08df7-bc93-4adb-9e4d-720ab44175a5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:56.973649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd4704-06c3-4727-b982-ab6cee1aef53 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:56.981008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:56.983510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.984815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.985461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] PATH_INFO: `/v3/projects/279b2d5d2f2647f69be35817b22cdb04` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.985857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54de7d9-18fe-4397-92da-f459389d080e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:56.986565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54de7d9-18fe-4397-92da-f459389d080e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:56.986839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54de7d9-18fe-4397-92da-f459389d080e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:56.987097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54de7d9-18fe-4397-92da-f459389d080e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:56.987894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54de7d9-18fe-4397-92da-f459389d080e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:56.988491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=279b2d5d2f2647f69be35817b22cdb04)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:56.989535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54de7d9-18fe-4397-92da-f459389d080e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:57.002052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:57.007011 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1011/2027] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:57.008161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54de7d9-18fe-4397-92da-f459389d080e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:57.010203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b54de7d9-18fe-4397-92da-f459389d080e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:57.181231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54de7d9-18fe-4397-92da-f459389d080e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:57.192620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '279b2d5d2f2647f69be35817b22cdb04', 'request_id': 'req-2e082a97-9783-4052-ad72-953ceb697311'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:57.214417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '279b2d5d2f2647f69be35817b22cdb04', 'request_id': 'req-2e082a97-9783-4052-ad72-953ceb697311'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:57.250566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '279b2d5d2f2647f69be35817b22cdb04', 'request_id': 'req-2e082a97-9783-4052-ad72-953ceb697311'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:57.274830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:57.278712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] The token cache is being invalidate because project 279b2d5d2f2647f69be35817b22cdb04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:57.279286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:57.281187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:57.288233 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1017/2028] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:56 2022] DELETE /identity/v3/projects/279b2d5d2f2647f69be35817b22cdb04 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:57.290543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:57.294100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:57.308001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d03bc-73db-40fc-8f56-87cbf5aba170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:57.308917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d03bc-73db-40fc-8f56-87cbf5aba170 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:57.309658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d03bc-73db-40fc-8f56-87cbf5aba170 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:57.309876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d03bc-73db-40fc-8f56-87cbf5aba170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:57.312739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-051d03bc-73db-40fc-8f56-87cbf5aba170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:57.415237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d03bc-73db-40fc-8f56-87cbf5aba170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:57.416953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d03bc-73db-40fc-8f56-87cbf5aba170 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:57.428931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e082a97-9783-4052-ad72-953ceb697311 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:57.430161 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1012/2029] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:57.431545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d03bc-73db-40fc-8f56-87cbf5aba170 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:57.432934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-051d03bc-73db-40fc-8f56-87cbf5aba170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:57.440536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91677e-58bf-42f6-9bc3-d347e3abf838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:57.441812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91677e-58bf-42f6-9bc3-d347e3abf838 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:57.442445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91677e-58bf-42f6-9bc3-d347e3abf838 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:57.443012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91677e-58bf-42f6-9bc3-d347e3abf838 admin admin] PATH_INFO: `/v3/projects/6a97d94d1bda4de2899aca13952c2291/users/e7e70776606c41b28dd8092752a1619c/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:57.451724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d03bc-73db-40fc-8f56-87cbf5aba170 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:57.475406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60553c6-8fe0-4722-b5cc-70df707729fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:57.476326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60553c6-8fe0-4722-b5cc-70df707729fc admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:57.476881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60553c6-8fe0-4722-b5cc-70df707729fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:57.477320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60553c6-8fe0-4722-b5cc-70df707729fc admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:57.478363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60553c6-8fe0-4722-b5cc-70df707729fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:57.481216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60553c6-8fe0-4722-b5cc-70df707729fc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:57.509660 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1013/2030] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:57.511451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e412340-d916-42da-b76a-72e993b3187b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:57.512013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e412340-d916-42da-b76a-72e993b3187b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:57.512494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e412340-d916-42da-b76a-72e993b3187b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:57.515292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91677e-58bf-42f6-9bc3-d347e3abf838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a97d94d1bda4de2899aca13952c2291, user_id=e7e70776606c41b28dd8092752a1619c, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:57.521284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91677e-58bf-42f6-9bc3-d347e3abf838 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:57.537381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:57.541415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3e412340-d916-42da-b76a-72e993b3187b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:57.543456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3e412340-d916-42da-b76a-72e993b3187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:57.550568 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1018/2031] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:57 2022] PUT /identity/v3/projects/6a97d94d1bda4de2899aca13952c2291/users/e7e70776606c41b28dd8092752a1619c/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:57.552353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91677e-58bf-42f6-9bc3-d347e3abf838 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:57.677782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91677e-58bf-42f6-9bc3-d347e3abf838 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:57.685534 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1014/2032] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:57.687440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc5c8a-9299-4f3e-aea7-6b66c9c13517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:57.688598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc5c8a-9299-4f3e-aea7-6b66c9c13517 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:57.689403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc5c8a-9299-4f3e-aea7-6b66c9c13517 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:57.689509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc5c8a-9299-4f3e-aea7-6b66c9c13517 admin admin] PATH_INFO: `/v3/projects/924c0091213d43088504f8bfed54e7a3/users/a4bb63e20fc3465fb682998442d9b6b9/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:57.689611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874d347-3050-410e-919e-9f1e278a82d9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:57.689775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874d347-3050-410e-919e-9f1e278a82d9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:57.690044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874d347-3050-410e-919e-9f1e278a82d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:57.723144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc5c8a-9299-4f3e-aea7-6b66c9c13517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=924c0091213d43088504f8bfed54e7a3, user_id=a4bb63e20fc3465fb682998442d9b6b9, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:57.728016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7874d347-3050-410e-919e-9f1e278a82d9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:57.730256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7874d347-3050-410e-919e-9f1e278a82d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:57.731150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc5c8a-9299-4f3e-aea7-6b66c9c13517 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:57.745575 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1015/2033] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:57.747759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb465fcb-ee7c-41dd-81ef-d17abcdcaa0a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:57.748231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb465fcb-ee7c-41dd-81ef-d17abcdcaa0a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:57.748726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb465fcb-ee7c-41dd-81ef-d17abcdcaa0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:57.804474 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1019/2034] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:57 2022] PUT /identity/v3/projects/924c0091213d43088504f8bfed54e7a3/users/a4bb63e20fc3465fb682998442d9b6b9/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:57.804474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc5c8a-9299-4f3e-aea7-6b66c9c13517 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:57.804765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bb465fcb-ee7c-41dd-81ef-d17abcdcaa0a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:57.804765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bb465fcb-ee7c-41dd-81ef-d17abcdcaa0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:57.805656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-55cc5c8a-9299-4f3e-aea7-6b66c9c13517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:57.919871 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1016/2035] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:57.922021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1acbef-cf69-4ff0-bddb-e48fa8920777 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:57.922316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1acbef-cf69-4ff0-bddb-e48fa8920777 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:57.922585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1acbef-cf69-4ff0-bddb-e48fa8920777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:57.956917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:57.967455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0b1acbef-cf69-4ff0-bddb-e48fa8920777 None None] MFA Rules not processed for user `a4bb63e20fc3465fb682998442d9b6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:57.969820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0b1acbef-cf69-4ff0-bddb-e48fa8920777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:57.992483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc5c8a-9299-4f3e-aea7-6b66c9c13517 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:58.026116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee664b98-e462-4ab7-8c89-02b0b246641b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:58.026822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee664b98-e462-4ab7-8c89-02b0b246641b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:58.027153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee664b98-e462-4ab7-8c89-02b0b246641b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:58.027477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee664b98-e462-4ab7-8c89-02b0b246641b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:58.041550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee664b98-e462-4ab7-8c89-02b0b246641b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:58.043466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee664b98-e462-4ab7-8c89-02b0b246641b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:58.059243 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1020/2036] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:55:58.060460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee664b98-e462-4ab7-8c89-02b0b246641b service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:58.061914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ee664b98-e462-4ab7-8c89-02b0b246641b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:58.072951 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1017/2037] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:55:58.073960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1acbef-cf69-4ff0-bddb-e48fa8920777 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:58.076812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee664b98-e462-4ab7-8c89-02b0b246641b service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:58.082711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:58.083593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:58.083867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:58.084131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] PATH_INFO: `/v3/projects/795b560b3af140a38757105cd17530c3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:58.087429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] RBAC: Authorizing `identity:delete_project(project_id=795b560b3af140a38757105cd17530c3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:58.088183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1acbef-cf69-4ff0-bddb-e48fa8920777 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:58.090499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:58.093704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67285481-37a7-4151-babf-0e2adc9dd2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:58.094406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67285481-37a7-4151-babf-0e2adc9dd2fd admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:58.094677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67285481-37a7-4151-babf-0e2adc9dd2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:58.095067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67285481-37a7-4151-babf-0e2adc9dd2fd admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:58.096672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67285481-37a7-4151-babf-0e2adc9dd2fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:55:58.097197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67285481-37a7-4151-babf-0e2adc9dd2fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:58.099657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67285481-37a7-4151-babf-0e2adc9dd2fd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:58.117970 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1018/2038] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:55:58.117970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67285481-37a7-4151-babf-0e2adc9dd2fd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:58.129075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67285481-37a7-4151-babf-0e2adc9dd2fd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:58.134800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea6ab3-7f0c-4fc0-97db-7c32f1707053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:58.135702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea6ab3-7f0c-4fc0-97db-7c32f1707053 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:58.135990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea6ab3-7f0c-4fc0-97db-7c32f1707053 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:58.136242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea6ab3-7f0c-4fc0-97db-7c32f1707053 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:58.137044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea6ab3-7f0c-4fc0-97db-7c32f1707053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:58.139888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea6ab3-7f0c-4fc0-97db-7c32f1707053 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:58.162363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '795b560b3af140a38757105cd17530c3', 'request_id': 'req-f932fdbe-1df9-4b6c-b924-09123636aedf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:58.175954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '795b560b3af140a38757105cd17530c3', 'request_id': 'req-f932fdbe-1df9-4b6c-b924-09123636aedf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:58.188005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '795b560b3af140a38757105cd17530c3', 'request_id': 'req-f932fdbe-1df9-4b6c-b924-09123636aedf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:58.241092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] The token cache is being invalidate because project 795b560b3af140a38757105cd17530c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:58.241379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:58.241869 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1019/2039] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:58 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:58.243066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea6ab3-7f0c-4fc0-97db-7c32f1707053 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:58.243948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:58.247046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3eea6ab3-7f0c-4fc0-97db-7c32f1707053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:58.248862 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1021/2040] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:58 2022] DELETE /identity/v3/projects/795b560b3af140a38757105cd17530c3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:58.250523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:58.253246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:58.278751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:58.390181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea6ab3-7f0c-4fc0-97db-7c32f1707053 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:58.405164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be12f1-1432-40e8-93bb-0bf3713f19ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:58.405799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be12f1-1432-40e8-93bb-0bf3713f19ec admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:58.406064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be12f1-1432-40e8-93bb-0bf3713f19ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:58.406345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be12f1-1432-40e8-93bb-0bf3713f19ec admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:58.406967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be12f1-1432-40e8-93bb-0bf3713f19ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:58.412704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be12f1-1432-40e8-93bb-0bf3713f19ec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:58.420890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932fdbe-1df9-4b6c-b924-09123636aedf admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:58.437007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd7dd9-c221-4ef7-bf83-b8622f176450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:58.437851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd7dd9-c221-4ef7-bf83-b8622f176450 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:58.438200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd7dd9-c221-4ef7-bf83-b8622f176450 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:58.438413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd7dd9-c221-4ef7-bf83-b8622f176450 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:58.442445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd7dd9-c221-4ef7-bf83-b8622f176450 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:58.449283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd7dd9-c221-4ef7-bf83-b8622f176450 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:58.503449 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1020/2041] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:58 2022] POST /identity/v3/projects => generated 383 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:58.504753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be12f1-1432-40e8-93bb-0bf3713f19ec admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:58.513055 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1022/2042] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:58 2022] POST /identity/v3/users => generated 299 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:58.515819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e1be12f1-1432-40e8-93bb-0bf3713f19ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:58.518272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd7dd9-c221-4ef7-bf83-b8622f176450 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:58.543351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd7dd9-c221-4ef7-bf83-b8622f176450 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:58.628421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:58.686361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be12f1-1432-40e8-93bb-0bf3713f19ec admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:58.693605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:58.694661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:58.695344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:58.695611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] PATH_INFO: `/v3/users/4f107659c22d4e0b893adb4c53045635` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:58.696994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f107659c22d4e0b893adb4c53045635)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:58.698276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365b689-a477-4d40-9506-205abb8a4834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:58.699316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365b689-a477-4d40-9506-205abb8a4834 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:58.699435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:58.699856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365b689-a477-4d40-9506-205abb8a4834 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:58.700268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365b689-a477-4d40-9506-205abb8a4834 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:58.701060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365b689-a477-4d40-9506-205abb8a4834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:58.703134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365b689-a477-4d40-9506-205abb8a4834 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:58.746337 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1023/2043] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:58 2022] POST /identity/v3/users => generated 292 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:58.747434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365b689-a477-4d40-9506-205abb8a4834 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:58.764291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365b689-a477-4d40-9506-205abb8a4834 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:58.771802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f40921-309e-4f4a-a597-a60943a239aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:58.772804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f40921-309e-4f4a-a597-a60943a239aa admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:58.773286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f40921-309e-4f4a-a597-a60943a239aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:58.773951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f40921-309e-4f4a-a597-a60943a239aa admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:58.775028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f40921-309e-4f4a-a597-a60943a239aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:58.778215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f40921-309e-4f4a-a597-a60943a239aa admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:58.794068 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1024/2044] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:58.795187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f40921-309e-4f4a-a597-a60943a239aa admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:58.806650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f107659c22d4e0b893adb4c53045635'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:58.818577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f107659c22d4e0b893adb4c53045635'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:58.823767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f40921-309e-4f4a-a597-a60943a239aa admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:58.836684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbe7c2-cfee-4fc5-8aee-445c2e0c096e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:58.837382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbe7c2-cfee-4fc5-8aee-445c2e0c096e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:58.837854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbe7c2-cfee-4fc5-8aee-445c2e0c096e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:58.838163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbe7c2-cfee-4fc5-8aee-445c2e0c096e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:58.838785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbe7c2-cfee-4fc5-8aee-445c2e0c096e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:58.840940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbe7c2-cfee-4fc5-8aee-445c2e0c096e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:58.848196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f107659c22d4e0b893adb4c53045635'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:58.849144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f107659c22d4e0b893adb4c53045635'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:58.849896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f107659c22d4e0b893adb4c53045635'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:58.865623 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1025/2045] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:58.866490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbe7c2-cfee-4fc5-8aee-445c2e0c096e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:58.880079 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1021/2046] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:55:58 2022] DELETE /identity/v3/users/4f107659c22d4e0b893adb4c53045635 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:58.880726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-11bbe7c2-cfee-4fc5-8aee-445c2e0c096e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:58.885893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:58.887572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:58.907832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbe7c2-cfee-4fc5-8aee-445c2e0c096e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:59.006436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee97ab-d935-46dc-a190-90905b29e591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:59.007694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee97ab-d935-46dc-a190-90905b29e591 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:59.008254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee97ab-d935-46dc-a190-90905b29e591 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:59.008709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee97ab-d935-46dc-a190-90905b29e591 admin admin] PATH_INFO: `/v3/projects/bc204fb7a0614bf68a7ee5180d7b6d1f/users/e5a475895cd34b43a27713a5e90c24f9/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:59.018915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:59.021609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912259d-9502-4f2c-ad97-c16b3245559e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:59.028763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee97ab-d935-46dc-a190-90905b29e591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc204fb7a0614bf68a7ee5180d7b6d1f, user_id=e5a475895cd34b43a27713a5e90c24f9, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:59.029082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7eaf2-a82e-4ae1-b32a-7f68841d8011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:59.029961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7eaf2-a82e-4ae1-b32a-7f68841d8011 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:59.030374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7eaf2-a82e-4ae1-b32a-7f68841d8011 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:59.030695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7eaf2-a82e-4ae1-b32a-7f68841d8011 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:59.031542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7eaf2-a82e-4ae1-b32a-7f68841d8011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:59.033295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee97ab-d935-46dc-a190-90905b29e591 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:59.033900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7eaf2-a82e-4ae1-b32a-7f68841d8011 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:59.048036 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1022/2047] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:59.053685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7eaf2-a82e-4ae1-b32a-7f68841d8011 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:59.054203 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1026/2048] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:58 2022] PUT /identity/v3/projects/bc204fb7a0614bf68a7ee5180d7b6d1f/users/e5a475895cd34b43a27713a5e90c24f9/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:59.058749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee97ab-d935-46dc-a190-90905b29e591 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:59.072073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7eaf2-a82e-4ae1-b32a-7f68841d8011 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:59.170060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee97ab-d935-46dc-a190-90905b29e591 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:59.193221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7c506-ae9f-4624-b3ff-8accf9a1f2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:59.193428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7c506-ae9f-4624-b3ff-8accf9a1f2ad admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:59.193611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7c506-ae9f-4624-b3ff-8accf9a1f2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:59.193806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7c506-ae9f-4624-b3ff-8accf9a1f2ad admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:59.194464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7c506-ae9f-4624-b3ff-8accf9a1f2ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:59.198400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7c506-ae9f-4624-b3ff-8accf9a1f2ad admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:59.210167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a3b4d-2e1c-4341-9df4-c2a71d15bab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:59.210884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a3b4d-2e1c-4341-9df4-c2a71d15bab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:59.211156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a3b4d-2e1c-4341-9df4-c2a71d15bab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:59.211421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a3b4d-2e1c-4341-9df4-c2a71d15bab8 admin admin] PATH_INFO: `/v3/projects/addc32428670401f80dac2b6a5de18bf/users/c520b51166614d64bf90ca39841a049e/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:59.227532 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1027/2049] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:55:59.238131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7c506-ae9f-4624-b3ff-8accf9a1f2ad admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:59.245075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:59.256187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a3b4d-2e1c-4341-9df4-c2a71d15bab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=addc32428670401f80dac2b6a5de18bf, user_id=c520b51166614d64bf90ca39841a049e, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:59.266528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7c506-ae9f-4624-b3ff-8accf9a1f2ad admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:59.271438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a3b4d-2e1c-4341-9df4-c2a71d15bab8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:59.282152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d117eb-c236-4701-94a1-02e7d14bdc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:59.284418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d117eb-c236-4701-94a1-02e7d14bdc56 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:59.284803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d117eb-c236-4701-94a1-02e7d14bdc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:59.285051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d117eb-c236-4701-94a1-02e7d14bdc56 admin admin] PATH_INFO: `/v3/projects/bc204fb7a0614bf68a7ee5180d7b6d1f/users/e5a475895cd34b43a27713a5e90c24f9/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:59.287674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d117eb-c236-4701-94a1-02e7d14bdc56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc204fb7a0614bf68a7ee5180d7b6d1f, user_id=e5a475895cd34b43a27713a5e90c24f9, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:59.293675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d117eb-c236-4701-94a1-02e7d14bdc56 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:59.303990 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1023/2050] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:59 2022] PUT /identity/v3/projects/addc32428670401f80dac2b6a5de18bf/users/c520b51166614d64bf90ca39841a049e/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:59.316432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e356e-8782-4f44-8028-71724efe599e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:59.316763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e356e-8782-4f44-8028-71724efe599e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:59.317084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e356e-8782-4f44-8028-71724efe599e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:59.337326 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1028/2051] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:55:59 2022] PUT /identity/v3/projects/bc204fb7a0614bf68a7ee5180d7b6d1f/users/e5a475895cd34b43a27713a5e90c24f9/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:59.350639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ca862-707b-46fc-8ea6-a3dc6369e4a6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:59.350958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ca862-707b-46fc-8ea6-a3dc6369e4a6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:59.351283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ca862-707b-46fc-8ea6-a3dc6369e4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:59.386846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d91e356e-8782-4f44-8028-71724efe599e None None] MFA Rules not processed for user `c520b51166614d64bf90ca39841a049e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:59.389911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d91e356e-8782-4f44-8028-71724efe599e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:59.411899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5a9ca862-707b-46fc-8ea6-a3dc6369e4a6 None None] MFA Rules not processed for user `e5a475895cd34b43a27713a5e90c24f9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:55:59.414444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5a9ca862-707b-46fc-8ea6-a3dc6369e4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:59.539847 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1024/2052] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:59.540899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e356e-8782-4f44-8028-71724efe599e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:59.543947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d91e356e-8782-4f44-8028-71724efe599e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:59.553599 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1029/2053] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:55:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:55:59.554451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ca862-707b-46fc-8ea6-a3dc6369e4a6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:59.656326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:59.676068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ca862-707b-46fc-8ea6-a3dc6369e4a6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:59.682982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5433d-34c3-48e7-8b95-736d7da4c53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:59.683916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5433d-34c3-48e7-8b95-736d7da4c53c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:59.684291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5433d-34c3-48e7-8b95-736d7da4c53c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:59.684822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5433d-34c3-48e7-8b95-736d7da4c53c admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:59.686248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5433d-34c3-48e7-8b95-736d7da4c53c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:59.689288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5433d-34c3-48e7-8b95-736d7da4c53c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:59.704448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e356e-8782-4f44-8028-71724efe599e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:59.719009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:59.721853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:59.722200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:59.722450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] PATH_INFO: `/v3/projects/b3157bb9ec164b84976e80a3d336bb28` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:59.723863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3157bb9ec164b84976e80a3d336bb28)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:55:59.726917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:55:59.746767 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1030/2054] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:55:59 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:55:59.746767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5433d-34c3-48e7-8b95-736d7da4c53c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:59.748768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b7d5433d-34c3-48e7-8b95-736d7da4c53c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:59.826294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3157bb9ec164b84976e80a3d336bb28', 'request_id': 'req-717e0eba-cfd2-44a2-b904-1c08191d6804'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:59.836285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3157bb9ec164b84976e80a3d336bb28', 'request_id': 'req-717e0eba-cfd2-44a2-b904-1c08191d6804'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:59.849045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3157bb9ec164b84976e80a3d336bb28', 'request_id': 'req-717e0eba-cfd2-44a2-b904-1c08191d6804'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:59.862837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] The token cache is being invalidate because project b3157bb9ec164b84976e80a3d336bb28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:55:59.863218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:59.863968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:55:59.869466 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1025/2055] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:55:59 2022] DELETE /identity/v3/projects/b3157bb9ec164b84976e80a3d336bb28 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:55:59.870413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:55:59.870833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5433d-34c3-48e7-8b95-736d7da4c53c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:55:59.871966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:59.883461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:55:59.980544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25991f96-e3eb-43e7-a675-a75095f7bcd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:55:59.981178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25991f96-e3eb-43e7-a675-a75095f7bcd0 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:55:59.981450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25991f96-e3eb-43e7-a675-a75095f7bcd0 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:55:59.981708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25991f96-e3eb-43e7-a675-a75095f7bcd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:55:59.990773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-25991f96-e3eb-43e7-a675-a75095f7bcd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:55:59.994718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e0eba-cfd2-44a2-b904-1c08191d6804 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:00.010437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc642fb-34c3-4aa7-8e38-5ba509cf75a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:00.011190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc642fb-34c3-4aa7-8e38-5ba509cf75a1 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:00.011560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc642fb-34c3-4aa7-8e38-5ba509cf75a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:00.011915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc642fb-34c3-4aa7-8e38-5ba509cf75a1 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:00.012847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc642fb-34c3-4aa7-8e38-5ba509cf75a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:00.019633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc642fb-34c3-4aa7-8e38-5ba509cf75a1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:00.063640 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1026/2056] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:55:59 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:00.066258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d7ac3-2182-4d60-8b1a-fc5f320e3e7f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:00.066405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d7ac3-2182-4d60-8b1a-fc5f320e3e7f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:00.067762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d7ac3-2182-4d60-8b1a-fc5f320e3e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:00.132744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-068d7ac3-2182-4d60-8b1a-fc5f320e3e7f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:00.134137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-068d7ac3-2182-4d60-8b1a-fc5f320e3e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:00.138108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25991f96-e3eb-43e7-a675-a75095f7bcd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:00.140572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25991f96-e3eb-43e7-a675-a75095f7bcd0 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:00.143966 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1027/2057] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:00.150592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d7ac3-2182-4d60-8b1a-fc5f320e3e7f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:00.156588 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1031/2058] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:55:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:00.158029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff4fea-6481-4fa7-868f-c1d45d38a6fb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:00.158312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff4fea-6481-4fa7-868f-c1d45d38a6fb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:00.158587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff4fea-6481-4fa7-868f-c1d45d38a6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:00.167492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d7ac3-2182-4d60-8b1a-fc5f320e3e7f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:00.184979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fce512-939b-48e6-bfd9-6b09bfe11e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:00.187655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fce512-939b-48e6-bfd9-6b09bfe11e8d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:00.188860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fce512-939b-48e6-bfd9-6b09bfe11e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:00.192418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fce512-939b-48e6-bfd9-6b09bfe11e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:00.193705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fce512-939b-48e6-bfd9-6b09bfe11e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:00.201039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fce512-939b-48e6-bfd9-6b09bfe11e8d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:00.226947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-75ff4fea-6481-4fa7-868f-c1d45d38a6fb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:00.229225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-75ff4fea-6481-4fa7-868f-c1d45d38a6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:00.245595 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1028/2059] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:00.249963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fce512-939b-48e6-bfd9-6b09bfe11e8d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:00.279763 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1032/2060] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:00.287305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff4fea-6481-4fa7-868f-c1d45d38a6fb None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:00.287557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fce512-939b-48e6-bfd9-6b09bfe11e8d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:00.309129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdfae3-edcc-40de-9c0f-5d16c54dbca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:00.309884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdfae3-edcc-40de-9c0f-5d16c54dbca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:00.310102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdfae3-edcc-40de-9c0f-5d16c54dbca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:00.310332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdfae3-edcc-40de-9c0f-5d16c54dbca9 admin admin] PATH_INFO: `/v3/projects/60a0d8eb4f54460ebf5987d14b2108e2/users/3a678c3d6e8b4896b9d5839bd49c418a/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:00.321666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:00.332738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff4fea-6481-4fa7-868f-c1d45d38a6fb None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:00.354305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdfae3-edcc-40de-9c0f-5d16c54dbca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a0d8eb4f54460ebf5987d14b2108e2, user_id=3a678c3d6e8b4896b9d5839bd49c418a, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:00.355207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4c667-6f46-4b6e-a966-1d75b43bd4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:00.356196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4c667-6f46-4b6e-a966-1d75b43bd4cc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:00.356814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4c667-6f46-4b6e-a966-1d75b43bd4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:00.357225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4c667-6f46-4b6e-a966-1d75b43bd4cc admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:00.357833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdfae3-edcc-40de-9c0f-5d16c54dbca9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:00.358280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4c667-6f46-4b6e-a966-1d75b43bd4cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:00.358487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4c667-6f46-4b6e-a966-1d75b43bd4cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:00.360277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4c667-6f46-4b6e-a966-1d75b43bd4cc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:00.391655 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1029/2061] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:00 2022] PUT /identity/v3/projects/60a0d8eb4f54460ebf5987d14b2108e2/users/3a678c3d6e8b4896b9d5839bd49c418a/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:00.393008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdfae3-edcc-40de-9c0f-5d16c54dbca9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:00.396719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-23cdfae3-edcc-40de-9c0f-5d16c54dbca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:00.405914 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1033/2062] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:00.407128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4c667-6f46-4b6e-a966-1d75b43bd4cc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:00.422325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4c667-6f46-4b6e-a966-1d75b43bd4cc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:00.507418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a541c9-1cba-4ec9-a350-45f4141fb019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:00.508121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a541c9-1cba-4ec9-a350-45f4141fb019 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:00.508347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a541c9-1cba-4ec9-a350-45f4141fb019 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:00.508732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a541c9-1cba-4ec9-a350-45f4141fb019 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:00.509379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a541c9-1cba-4ec9-a350-45f4141fb019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:00.512557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a541c9-1cba-4ec9-a350-45f4141fb019 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:00.525333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdfae3-edcc-40de-9c0f-5d16c54dbca9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:00.525687 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1034/2063] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:00.526645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a541c9-1cba-4ec9-a350-45f4141fb019 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:00.533127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b9b82-d92a-4394-9271-10c529402bf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:00.533737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b9b82-d92a-4394-9271-10c529402bf5 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:00.534007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b9b82-d92a-4394-9271-10c529402bf5 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:00.534268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b9b82-d92a-4394-9271-10c529402bf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:00.536340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c47b9b82-d92a-4394-9271-10c529402bf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:00.543796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a541c9-1cba-4ec9-a350-45f4141fb019 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:00.551524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e8b4e-cb55-458a-af27-981ad6906783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:00.552148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e8b4e-cb55-458a-af27-981ad6906783 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:00.552444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e8b4e-cb55-458a-af27-981ad6906783 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:00.552716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e8b4e-cb55-458a-af27-981ad6906783 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:00.553526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e8b4e-cb55-458a-af27-981ad6906783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:00.556817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e8b4e-cb55-458a-af27-981ad6906783 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:00.597335 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1035/2064] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:00 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:00.597824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e8b4e-cb55-458a-af27-981ad6906783 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:00.626970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:00.637304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b9b82-d92a-4394-9271-10c529402bf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:00.639660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b9b82-d92a-4394-9271-10c529402bf5 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:00.663397 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1030/2065] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:00 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:00.664299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b9b82-d92a-4394-9271-10c529402bf5 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:00.673682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e8b4e-cb55-458a-af27-981ad6906783 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:00.686582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02788d-55e0-4045-b3ca-c3da783f67b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:00.688817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02788d-55e0-4045-b3ca-c3da783f67b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:00.689138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02788d-55e0-4045-b3ca-c3da783f67b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:00.690089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02788d-55e0-4045-b3ca-c3da783f67b7 admin admin] PATH_INFO: `/v3/projects/60a0d8eb4f54460ebf5987d14b2108e2/users/3a678c3d6e8b4896b9d5839bd49c418a/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:00.692654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02788d-55e0-4045-b3ca-c3da783f67b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a0d8eb4f54460ebf5987d14b2108e2, user_id=3a678c3d6e8b4896b9d5839bd49c418a, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:00.695968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02788d-55e0-4045-b3ca-c3da783f67b7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:00.704255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b9b82-d92a-4394-9271-10c529402bf5 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:00.712449 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1036/2066] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:00 2022] PUT /identity/v3/projects/60a0d8eb4f54460ebf5987d14b2108e2/users/3a678c3d6e8b4896b9d5839bd49c418a/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:00.726775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9eedd-aa8c-4b6c-9008-42fa0e5d87a1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:00.727413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9eedd-aa8c-4b6c-9008-42fa0e5d87a1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:00.727820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9eedd-aa8c-4b6c-9008-42fa0e5d87a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:00.792084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-baa9eedd-aa8c-4b6c-9008-42fa0e5d87a1 None None] MFA Rules not processed for user `3a678c3d6e8b4896b9d5839bd49c418a`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:00.793471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-baa9eedd-aa8c-4b6c-9008-42fa0e5d87a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:00.875543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b228df-2b2f-4c03-9180-faa514fcee3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:00.875543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b228df-2b2f-4c03-9180-faa514fcee3c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:00.875543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b228df-2b2f-4c03-9180-faa514fcee3c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:00.876299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b228df-2b2f-4c03-9180-faa514fcee3c admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:00.878244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b228df-2b2f-4c03-9180-faa514fcee3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:00.883200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b228df-2b2f-4c03-9180-faa514fcee3c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:00.920933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:00.927663 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1031/2067] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:00 2022] POST /identity/v3/users => generated 287 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:00.931193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b228df-2b2f-4c03-9180-faa514fcee3c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:00.946869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b228df-2b2f-4c03-9180-faa514fcee3c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:00.956208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6345db-dcf1-4963-839c-8d51bbb54854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:00.957308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6345db-dcf1-4963-839c-8d51bbb54854 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:00.958359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6345db-dcf1-4963-839c-8d51bbb54854 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:00.958729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6345db-dcf1-4963-839c-8d51bbb54854 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:00.959560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6345db-dcf1-4963-839c-8d51bbb54854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:00.962950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6345db-dcf1-4963-839c-8d51bbb54854 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:00.970217 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1037/2068] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:56:00.978130 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1032/2069] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:00.981837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9eedd-aa8c-4b6c-9008-42fa0e5d87a1 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:00.988296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6345db-dcf1-4963-839c-8d51bbb54854 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:00.997578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9eedd-aa8c-4b6c-9008-42fa0e5d87a1 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:01.003846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07f00b-6c5f-4b2d-b74c-23add8812b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:01.004753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07f00b-6c5f-4b2d-b74c-23add8812b04 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:01.005152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07f00b-6c5f-4b2d-b74c-23add8812b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:01.005667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07f00b-6c5f-4b2d-b74c-23add8812b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:01.006439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07f00b-6c5f-4b2d-b74c-23add8812b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:01.008940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07f00b-6c5f-4b2d-b74c-23add8812b04 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:01.021192 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1038/2070] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:01.025215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07f00b-6c5f-4b2d-b74c-23add8812b04 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:01.039491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07f00b-6c5f-4b2d-b74c-23add8812b04 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:01.046198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2994c3-4d26-4f23-923a-8a62302d13ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:01.047116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2994c3-4d26-4f23-923a-8a62302d13ae admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:01.047472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2994c3-4d26-4f23-923a-8a62302d13ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:01.047748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2994c3-4d26-4f23-923a-8a62302d13ae admin admin] PATH_INFO: `/v3/projects/e741faff80614dd3b7bb376b6c577f53/users/ecc575e9946b451490df7a2940378c99/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:01.068591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2994c3-4d26-4f23-923a-8a62302d13ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=e741faff80614dd3b7bb376b6c577f53, user_id=ecc575e9946b451490df7a2940378c99, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:01.073568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2994c3-4d26-4f23-923a-8a62302d13ae admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:01.097414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6345db-dcf1-4963-839c-8d51bbb54854 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:01.098047 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1039/2071] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:01 2022] PUT /identity/v3/projects/e741faff80614dd3b7bb376b6c577f53/users/ecc575e9946b451490df7a2940378c99/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:56:01.109481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e401f9e-a690-4630-aa49-cb5e96faa03f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:01.109772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e401f9e-a690-4630-aa49-cb5e96faa03f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:01.110044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e401f9e-a690-4630-aa49-cb5e96faa03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:01.150082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4e401f9e-a690-4630-aa49-cb5e96faa03f None None] MFA Rules not processed for user `ecc575e9946b451490df7a2940378c99`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:01.151886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4e401f9e-a690-4630-aa49-cb5e96faa03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:01.233090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ac814-d478-482a-8b96-337e3fd195a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:01.240814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ac814-d478-482a-8b96-337e3fd195a2 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:01.241216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ac814-d478-482a-8b96-337e3fd195a2 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:01.241491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ac814-d478-482a-8b96-337e3fd195a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:01.254245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:01.268131 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1040/2072] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:56:01.269100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e401f9e-a690-4630-aa49-cb5e96faa03f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:01.370464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ac814-d478-482a-8b96-337e3fd195a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:01.373034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ac814-d478-482a-8b96-337e3fd195a2 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:01.388453 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1033/2073] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:01.402371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e401f9e-a690-4630-aa49-cb5e96faa03f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:01.409198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc1d39-2592-41fa-b255-f707e06ef532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:01.411100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc1d39-2592-41fa-b255-f707e06ef532 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:01.411482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc1d39-2592-41fa-b255-f707e06ef532 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:01.411895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc1d39-2592-41fa-b255-f707e06ef532 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:01.413478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc1d39-2592-41fa-b255-f707e06ef532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:01.416274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc1d39-2592-41fa-b255-f707e06ef532 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:01.479134 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1041/2074] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:01 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:01.482327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ac814-d478-482a-8b96-337e3fd195a2 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:01.500239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ac814-d478-482a-8b96-337e3fd195a2 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:01.692597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c93cb-f4ca-4b32-bd21-ac8c8cddd4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:01.693275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c93cb-f4ca-4b32-bd21-ac8c8cddd4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:01.693497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c93cb-f4ca-4b32-bd21-ac8c8cddd4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:01.696259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c93cb-f4ca-4b32-bd21-ac8c8cddd4e8 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:01.696259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c93cb-f4ca-4b32-bd21-ac8c8cddd4e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:01.698404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c93cb-f4ca-4b32-bd21-ac8c8cddd4e8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:01.730855 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1034/2075] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:01 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:01.735469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc1d39-2592-41fa-b255-f707e06ef532 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:01.758852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc1d39-2592-41fa-b255-f707e06ef532 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:01.774312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf959f2-4b55-4594-91f2-0f3d91dd4cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:01.775391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf959f2-4b55-4594-91f2-0f3d91dd4cba admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:01.779732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf959f2-4b55-4594-91f2-0f3d91dd4cba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:01.780033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf959f2-4b55-4594-91f2-0f3d91dd4cba admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:01.780795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf959f2-4b55-4594-91f2-0f3d91dd4cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:01.782559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf959f2-4b55-4594-91f2-0f3d91dd4cba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:01.798790 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1042/2076] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:01.802691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c93cb-f4ca-4b32-bd21-ac8c8cddd4e8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:01.822576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c93cb-f4ca-4b32-bd21-ac8c8cddd4e8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:01.829549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5908c0-eab2-45c8-b3c1-dac4d1e12946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:01.840842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5908c0-eab2-45c8-b3c1-dac4d1e12946 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:01.841416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5908c0-eab2-45c8-b3c1-dac4d1e12946 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:01.842463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5908c0-eab2-45c8-b3c1-dac4d1e12946 admin admin] PATH_INFO: `/v3/projects/5f86c8a4563b46f98542eeaa49be7e46/users/10c860aed5c846aa8c96c27ed38272ea/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:01.884361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5908c0-eab2-45c8-b3c1-dac4d1e12946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f86c8a4563b46f98542eeaa49be7e46, user_id=10c860aed5c846aa8c96c27ed38272ea, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:01.887957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5908c0-eab2-45c8-b3c1-dac4d1e12946 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:01.921248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf959f2-4b55-4594-91f2-0f3d91dd4cba admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:01.921673 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1035/2077] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:01 2022] PUT /identity/v3/projects/5f86c8a4563b46f98542eeaa49be7e46/users/10c860aed5c846aa8c96c27ed38272ea/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:01.940763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf959f2-4b55-4594-91f2-0f3d91dd4cba admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:01.959835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:02.019404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5908c0-eab2-45c8-b3c1-dac4d1e12946 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:02.116312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe256395-cd7f-4118-9192-4261fa7968bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:02.117678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe256395-cd7f-4118-9192-4261fa7968bc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:02.118262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe256395-cd7f-4118-9192-4261fa7968bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:02.120307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe256395-cd7f-4118-9192-4261fa7968bc admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:02.121184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe256395-cd7f-4118-9192-4261fa7968bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:02.127377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe256395-cd7f-4118-9192-4261fa7968bc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:02.152510 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1043/2078] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:56:02.156400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe256395-cd7f-4118-9192-4261fa7968bc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:02.170119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe256395-cd7f-4118-9192-4261fa7968bc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:02.183595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5908c0-eab2-45c8-b3c1-dac4d1e12946 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:02.187535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753c69e-528e-4050-825a-60deebf4b41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:02.188520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe26d79-1956-4045-a58a-a65aca0e943a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:02.189248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe26d79-1956-4045-a58a-a65aca0e943a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:02.189534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe26d79-1956-4045-a58a-a65aca0e943a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:02.190190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe26d79-1956-4045-a58a-a65aca0e943a admin admin] PATH_INFO: `/v3/users/c520b51166614d64bf90ca39841a049e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:02.191000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe26d79-1956-4045-a58a-a65aca0e943a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c520b51166614d64bf90ca39841a049e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:02.192311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753c69e-528e-4050-825a-60deebf4b41f admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:02.193146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe26d79-1956-4045-a58a-a65aca0e943a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:02.193548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753c69e-528e-4050-825a-60deebf4b41f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:02.193893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753c69e-528e-4050-825a-60deebf4b41f admin admin] PATH_INFO: `/v3/projects/5f86c8a4563b46f98542eeaa49be7e46/users/10c860aed5c846aa8c96c27ed38272ea/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:02.198068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753c69e-528e-4050-825a-60deebf4b41f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f86c8a4563b46f98542eeaa49be7e46, user_id=10c860aed5c846aa8c96c27ed38272ea, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:02.207545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:02.216582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753c69e-528e-4050-825a-60deebf4b41f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:02.254106 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1044/2079] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:02 2022] PUT /identity/v3/projects/5f86c8a4563b46f98542eeaa49be7e46/users/10c860aed5c846aa8c96c27ed38272ea/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:56:02.256754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753c69e-528e-4050-825a-60deebf4b41f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:02.261160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1753c69e-528e-4050-825a-60deebf4b41f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:02.330826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1fe26d79-1956-4045-a58a-a65aca0e943a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c520b51166614d64bf90ca39841a049e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:02.377608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1fe26d79-1956-4045-a58a-a65aca0e943a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c520b51166614d64bf90ca39841a049e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:02.409460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1fe26d79-1956-4045-a58a-a65aca0e943a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c520b51166614d64bf90ca39841a049e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:02.412245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1fe26d79-1956-4045-a58a-a65aca0e943a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c520b51166614d64bf90ca39841a049e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:02.413533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1fe26d79-1956-4045-a58a-a65aca0e943a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c520b51166614d64bf90ca39841a049e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:02.438749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753c69e-528e-4050-825a-60deebf4b41f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:02.455680 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1036/2080] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:02 2022] DELETE /identity/v3/users/c520b51166614d64bf90ca39841a049e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:02.459895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca30c6-ca6c-4d64-8da1-aa6099ffb02b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:02.461155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca30c6-ca6c-4d64-8da1-aa6099ffb02b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:02.463484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca30c6-ca6c-4d64-8da1-aa6099ffb02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:02.470966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a480b2-f3d7-49d5-af83-6fbff8aae3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:02.471647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a480b2-f3d7-49d5-af83-6fbff8aae3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:02.471920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a480b2-f3d7-49d5-af83-6fbff8aae3db admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:02.472197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a480b2-f3d7-49d5-af83-6fbff8aae3db admin admin] PATH_INFO: `/v3/users/a4bb63e20fc3465fb682998442d9b6b9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:02.480150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a480b2-f3d7-49d5-af83-6fbff8aae3db admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4bb63e20fc3465fb682998442d9b6b9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:02.482145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a480b2-f3d7-49d5-af83-6fbff8aae3db admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:02.571131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-54ca30c6-ca6c-4d64-8da1-aa6099ffb02b None None] MFA Rules not processed for user `10c860aed5c846aa8c96c27ed38272ea`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:02.572495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-54ca30c6-ca6c-4d64-8da1-aa6099ffb02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:02.616486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8a480b2-f3d7-49d5-af83-6fbff8aae3db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4bb63e20fc3465fb682998442d9b6b9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:02.628794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8a480b2-f3d7-49d5-af83-6fbff8aae3db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4bb63e20fc3465fb682998442d9b6b9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:02.639030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8a480b2-f3d7-49d5-af83-6fbff8aae3db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4bb63e20fc3465fb682998442d9b6b9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:02.661338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8a480b2-f3d7-49d5-af83-6fbff8aae3db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4bb63e20fc3465fb682998442d9b6b9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:02.662402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8a480b2-f3d7-49d5-af83-6fbff8aae3db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4bb63e20fc3465fb682998442d9b6b9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:02.681130 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1045/2081] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:02 2022] DELETE /identity/v3/users/a4bb63e20fc3465fb682998442d9b6b9 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:02.683203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be089e17-d2c9-4ca3-b3af-566215c74845 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:02.684077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be089e17-d2c9-4ca3-b3af-566215c74845 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:02.690603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be089e17-d2c9-4ca3-b3af-566215c74845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:02.734524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-be089e17-d2c9-4ca3-b3af-566215c74845 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:02.736092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-be089e17-d2c9-4ca3-b3af-566215c74845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:02.740131 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1037/2082] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:02.741850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca30c6-ca6c-4d64-8da1-aa6099ffb02b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:02.743713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-54ca30c6-ca6c-4d64-8da1-aa6099ffb02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:02.771799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:02.902901 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1046/2083] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:02.904773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca30c6-ca6c-4d64-8da1-aa6099ffb02b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:02.904988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c98533-3850-4599-972f-2c86130cce1d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:02.905263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c98533-3850-4599-972f-2c86130cce1d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:02.905659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c98533-3850-4599-972f-2c86130cce1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:02.913061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:02.914051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:02.914417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:02.914786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] PATH_INFO: `/v3/users/b9ac6652707a4819a2b1229bd08ca524` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:02.916685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9ac6652707a4819a2b1229bd08ca524)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:02.920642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:02.934349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-13c98533-3850-4599-972f-2c86130cce1d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:02.936986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-13c98533-3850-4599-972f-2c86130cce1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:02.940825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:02.971678 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1047/2084] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:02.974152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c98533-3850-4599-972f-2c86130cce1d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:02.977637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-13c98533-3850-4599-972f-2c86130cce1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:03.007819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9ac6652707a4819a2b1229bd08ca524'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.023249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9ac6652707a4819a2b1229bd08ca524'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.043799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9ac6652707a4819a2b1229bd08ca524'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.044698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9ac6652707a4819a2b1229bd08ca524'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.045344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9ac6652707a4819a2b1229bd08ca524'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.070098 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1038/2085] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:02 2022] DELETE /identity/v3/users/b9ac6652707a4819a2b1229bd08ca524 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:03.071589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:03.073624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:03.112996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c98533-3850-4599-972f-2c86130cce1d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:03.192564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64b7af-c8a6-4f8a-98e1-eb0811906860 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:03.198487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891edfc-75e5-4245-8a38-1d123e27cf36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:03.199355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891edfc-75e5-4245-8a38-1d123e27cf36 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:03.199772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891edfc-75e5-4245-8a38-1d123e27cf36 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:03.200176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891edfc-75e5-4245-8a38-1d123e27cf36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:03.202910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3891edfc-75e5-4245-8a38-1d123e27cf36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:03.234458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920653da-f2d6-4229-adaf-41bab7a99da9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:03.235186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920653da-f2d6-4229-adaf-41bab7a99da9 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:03.235461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920653da-f2d6-4229-adaf-41bab7a99da9 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:03.236283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920653da-f2d6-4229-adaf-41bab7a99da9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:03.238514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-920653da-f2d6-4229-adaf-41bab7a99da9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:03.367303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:03.377384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891edfc-75e5-4245-8a38-1d123e27cf36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:03.378889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891edfc-75e5-4245-8a38-1d123e27cf36 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:03.396780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920653da-f2d6-4229-adaf-41bab7a99da9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:03.398633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920653da-f2d6-4229-adaf-41bab7a99da9 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:03.404358 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1039/2086] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:03.405361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891edfc-75e5-4245-8a38-1d123e27cf36 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:03.417473 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1048/2087] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:03.422567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891edfc-75e5-4245-8a38-1d123e27cf36 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:03.435200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ef8df-04cf-4de7-9c2a-d1cbd4eb4aae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:03.436259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ef8df-04cf-4de7-9c2a-d1cbd4eb4aae service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:03.436581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ef8df-04cf-4de7-9c2a-d1cbd4eb4aae service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:03.436847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ef8df-04cf-4de7-9c2a-d1cbd4eb4aae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:03.438673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d60ef8df-04cf-4de7-9c2a-d1cbd4eb4aae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:03.468585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ef8df-04cf-4de7-9c2a-d1cbd4eb4aae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:03.470037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ef8df-04cf-4de7-9c2a-d1cbd4eb4aae service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:03.493993 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1040/2088] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:03.524907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920653da-f2d6-4229-adaf-41bab7a99da9 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:03.526201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-920653da-f2d6-4229-adaf-41bab7a99da9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:03.583073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920653da-f2d6-4229-adaf-41bab7a99da9 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:03.599850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:03.600687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:03.601034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:03.601371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] PATH_INFO: `/v3/projects/7f2a18b59cac4d7385a897ef2695b826` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:03.605256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f2a18b59cac4d7385a897ef2695b826)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:03.607387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:03.704717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f2a18b59cac4d7385a897ef2695b826', 'request_id': 'req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.716958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f2a18b59cac4d7385a897ef2695b826', 'request_id': 'req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.730297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f2a18b59cac4d7385a897ef2695b826', 'request_id': 'req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.748748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] The token cache is being invalidate because project 7f2a18b59cac4d7385a897ef2695b826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:03.749326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.749721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.755142 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1049/2089] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:03 2022] DELETE /identity/v3/projects/7f2a18b59cac4d7385a897ef2695b826 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:03.755457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ef8df-04cf-4de7-9c2a-d1cbd4eb4aae service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:03.768139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ef8df-04cf-4de7-9c2a-d1cbd4eb4aae service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:03.804362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:03.869553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:03.870354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:03.870706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:03.871324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] PATH_INFO: `/v3/users/d83156b45fed412f8506ef2d8af25339` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:03.872613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d83156b45fed412f8506ef2d8af25339)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:03.876812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:03.943799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd83156b45fed412f8506ef2d8af25339'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.958130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd83156b45fed412f8506ef2d8af25339'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.991280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd83156b45fed412f8506ef2d8af25339'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.993848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd83156b45fed412f8506ef2d8af25339'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:03.994631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd83156b45fed412f8506ef2d8af25339'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.028604 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1041/2090] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:03 2022] DELETE /identity/v3/users/d83156b45fed412f8506ef2d8af25339 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:04.034671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:04.036245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:04.078674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:04.084093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:04.099381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f3d44-2ea2-4ca3-80c1-cd91e9f5cb8e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:04.258662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e81001-203f-4b97-9b7b-f64cf4f67721 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:04.274525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:04.275539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:04.275814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:04.276764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] PATH_INFO: `/v3/projects/924c0091213d43088504f8bfed54e7a3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:04.280527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:04.281785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] RBAC: Authorizing `identity:delete_project(project_id=924c0091213d43088504f8bfed54e7a3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:04.285428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:04.285964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:04.286379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] PATH_INFO: `/v3/projects/addc32428670401f80dac2b6a5de18bf` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:04.288744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=addc32428670401f80dac2b6a5de18bf)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:04.289853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:04.295350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:04.404653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '924c0091213d43088504f8bfed54e7a3', 'request_id': 'req-11a69b00-45fb-4d77-a38e-ec57fdde647c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.406685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:04.416364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '924c0091213d43088504f8bfed54e7a3', 'request_id': 'req-11a69b00-45fb-4d77-a38e-ec57fdde647c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.436172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '924c0091213d43088504f8bfed54e7a3', 'request_id': 'req-11a69b00-45fb-4d77-a38e-ec57fdde647c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.466247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] The token cache is being invalidate because project 924c0091213d43088504f8bfed54e7a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:04.466896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.469485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.469613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'addc32428670401f80dac2b6a5de18bf', 'request_id': 'req-44615ec2-7990-4227-8144-a388b62fe242'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.474328 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1042/2091] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:04 2022] DELETE /identity/v3/projects/924c0091213d43088504f8bfed54e7a3 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:04.476498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:04.480825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:04.488248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'addc32428670401f80dac2b6a5de18bf', 'request_id': 'req-44615ec2-7990-4227-8144-a388b62fe242'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.499224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'addc32428670401f80dac2b6a5de18bf', 'request_id': 'req-44615ec2-7990-4227-8144-a388b62fe242'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.500727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:04.525345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] The token cache is being invalidate because project addc32428670401f80dac2b6a5de18bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:04.531051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.531835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.534070 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1050/2092] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:04 2022] DELETE /identity/v3/projects/addc32428670401f80dac2b6a5de18bf => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:04.534978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:04.536935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:04.681937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a69b00-45fb-4d77-a38e-ec57fdde647c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:04.694353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:04.695390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:04.695661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:04.695924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] PATH_INFO: `/v3/projects/4019258611094cbf979dece7390ba297` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:04.699494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44615ec2-7990-4227-8144-a388b62fe242 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:04.701268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4019258611094cbf979dece7390ba297)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:04.703651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:04.714153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb0edc-ab4a-4682-8d5a-c6ad7e408546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:04.714835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb0edc-ab4a-4682-8d5a-c6ad7e408546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:04.715110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb0edc-ab4a-4682-8d5a-c6ad7e408546 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:04.715377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb0edc-ab4a-4682-8d5a-c6ad7e408546 admin admin] PATH_INFO: `/v3/users/e5a475895cd34b43a27713a5e90c24f9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:04.718263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb0edc-ab4a-4682-8d5a-c6ad7e408546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5a475895cd34b43a27713a5e90c24f9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:04.720301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb0edc-ab4a-4682-8d5a-c6ad7e408546 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:04.817186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4019258611094cbf979dece7390ba297', 'request_id': 'req-93d0862a-bca9-489f-8718-7849476cdff8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.839303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4019258611094cbf979dece7390ba297', 'request_id': 'req-93d0862a-bca9-489f-8718-7849476cdff8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.847103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-64bb0edc-ab4a-4682-8d5a-c6ad7e408546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5a475895cd34b43a27713a5e90c24f9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.864516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4019258611094cbf979dece7390ba297', 'request_id': 'req-93d0862a-bca9-489f-8718-7849476cdff8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.875222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-64bb0edc-ab4a-4682-8d5a-c6ad7e408546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5a475895cd34b43a27713a5e90c24f9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.890022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] The token cache is being invalidate because project 4019258611094cbf979dece7390ba297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:04.892949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.893549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.895362 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1043/2093] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:04 2022] DELETE /identity/v3/projects/4019258611094cbf979dece7390ba297 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:04.896090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-64bb0edc-ab4a-4682-8d5a-c6ad7e408546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5a475895cd34b43a27713a5e90c24f9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.902610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:04.909742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:04.928068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-64bb0edc-ab4a-4682-8d5a-c6ad7e408546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5a475895cd34b43a27713a5e90c24f9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.930904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-64bb0edc-ab4a-4682-8d5a-c6ad7e408546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5a475895cd34b43a27713a5e90c24f9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:04.946521 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1051/2094] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:04 2022] DELETE /identity/v3/users/e5a475895cd34b43a27713a5e90c24f9 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:04.946521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3d560-3dc4-4f0a-ba7e-72db0666588d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:04.946521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3d560-3dc4-4f0a-ba7e-72db0666588d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:04.946521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3d560-3dc4-4f0a-ba7e-72db0666588d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:05.002366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-73b3d560-3dc4-4f0a-ba7e-72db0666588d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:05.003237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-73b3d560-3dc4-4f0a-ba7e-72db0666588d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:05.057599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0862a-bca9-489f-8718-7849476cdff8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:05.091654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:05.178347 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1052/2095] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:05.180938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a68bb-077d-47ba-92e9-828709ffc0d2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:05.181203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a68bb-077d-47ba-92e9-828709ffc0d2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:05.181469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a68bb-077d-47ba-92e9-828709ffc0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:05.220961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:05.225028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:05.225323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:05.225556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] PATH_INFO: `/v3/users/3a678c3d6e8b4896b9d5839bd49c418a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:05.229138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a678c3d6e8b4896b9d5839bd49c418a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:05.236907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:05.241137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-791a68bb-077d-47ba-92e9-828709ffc0d2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:05.245205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-791a68bb-077d-47ba-92e9-828709ffc0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:05.264621 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1053/2096] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:05.270936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a68bb-077d-47ba-92e9-828709ffc0d2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:05.273930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-791a68bb-077d-47ba-92e9-828709ffc0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:05.299673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a68bb-077d-47ba-92e9-828709ffc0d2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:05.309139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464928c-4133-4e78-8188-8b935c7a41e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:05.310073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464928c-4133-4e78-8188-8b935c7a41e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:05.310567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464928c-4133-4e78-8188-8b935c7a41e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:05.311088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464928c-4133-4e78-8188-8b935c7a41e5 admin admin] PATH_INFO: `/v3/users/ecc575e9946b451490df7a2940378c99` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:05.312607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464928c-4133-4e78-8188-8b935c7a41e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecc575e9946b451490df7a2940378c99)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:05.315628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464928c-4133-4e78-8188-8b935c7a41e5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:05.317964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:05.349030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a678c3d6e8b4896b9d5839bd49c418a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.385506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a678c3d6e8b4896b9d5839bd49c418a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.416500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4464928c-4133-4e78-8188-8b935c7a41e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecc575e9946b451490df7a2940378c99'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.426432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a678c3d6e8b4896b9d5839bd49c418a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.427499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a678c3d6e8b4896b9d5839bd49c418a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.428761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a678c3d6e8b4896b9d5839bd49c418a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.433123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4464928c-4133-4e78-8188-8b935c7a41e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecc575e9946b451490df7a2940378c99'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.453230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4464928c-4133-4e78-8188-8b935c7a41e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecc575e9946b451490df7a2940378c99'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.463287 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1044/2097] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:04 2022] DELETE /identity/v3/users/3a678c3d6e8b4896b9d5839bd49c418a => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:05.463287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:05.465452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:05.486172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4464928c-4133-4e78-8188-8b935c7a41e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecc575e9946b451490df7a2940378c99'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.487409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4464928c-4133-4e78-8188-8b935c7a41e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecc575e9946b451490df7a2940378c99'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.504641 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1054/2098] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:05 2022] DELETE /identity/v3/users/ecc575e9946b451490df7a2940378c99 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:05.509428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d70d17-61ca-446b-81be-79ab84845027 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:05.509709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d70d17-61ca-446b-81be-79ab84845027 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:05.510436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d70d17-61ca-446b-81be-79ab84845027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:05.559452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-41d70d17-61ca-446b-81be-79ab84845027 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:05.566916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-41d70d17-61ca-446b-81be-79ab84845027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:05.574542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68bad3-7c65-4b31-962c-510e82067096 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:05.683089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:05.685209 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1055/2099] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:56:05.689768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d70d17-61ca-446b-81be-79ab84845027 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:05.691228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-41d70d17-61ca-446b-81be-79ab84845027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:05.706490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d70d17-61ca-446b-81be-79ab84845027 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:05.712784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:05.713491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:05.713848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:05.714118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] PATH_INFO: `/v3/projects/bc204fb7a0614bf68a7ee5180d7b6d1f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:05.715715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc204fb7a0614bf68a7ee5180d7b6d1f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:05.718076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:05.748080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c914259-a503-4b74-914a-3fc74ed764c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:05.748794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c914259-a503-4b74-914a-3fc74ed764c3 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:05.750003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c914259-a503-4b74-914a-3fc74ed764c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:05.750291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c914259-a503-4b74-914a-3fc74ed764c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:05.750955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c914259-a503-4b74-914a-3fc74ed764c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:05.751255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c914259-a503-4b74-914a-3fc74ed764c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:05.752995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c914259-a503-4b74-914a-3fc74ed764c3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:05.791530 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1045/2100] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:05.791530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c914259-a503-4b74-914a-3fc74ed764c3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:05.795640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5c914259-a503-4b74-914a-3fc74ed764c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:05.812737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc204fb7a0614bf68a7ee5180d7b6d1f', 'request_id': 'req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.830785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc204fb7a0614bf68a7ee5180d7b6d1f', 'request_id': 'req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.841566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc204fb7a0614bf68a7ee5180d7b6d1f', 'request_id': 'req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.862330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] The token cache is being invalidate because project bc204fb7a0614bf68a7ee5180d7b6d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:05.862997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.863967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:05.867868 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1056/2101] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:05 2022] DELETE /identity/v3/projects/bc204fb7a0614bf68a7ee5180d7b6d1f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:05.868832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:05.870925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:05.898327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:05.937435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c914259-a503-4b74-914a-3fc74ed764c3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:06.000770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a99a1-d1f9-4e1b-b114-d93bb3976d4a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:06.010360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90809f-0719-4b5a-a0e4-857897f69a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:06.011295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90809f-0719-4b5a-a0e4-857897f69a3d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:06.012636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90809f-0719-4b5a-a0e4-857897f69a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:06.014231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90809f-0719-4b5a-a0e4-857897f69a3d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:06.015137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90809f-0719-4b5a-a0e4-857897f69a3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:06.020558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90809f-0719-4b5a-a0e4-857897f69a3d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:06.061734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942737f1-bb49-4fc3-abfd-2bbd21ea6cd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:06.064478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942737f1-bb49-4fc3-abfd-2bbd21ea6cd4 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:06.064478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942737f1-bb49-4fc3-abfd-2bbd21ea6cd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:06.065086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942737f1-bb49-4fc3-abfd-2bbd21ea6cd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:06.067755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-942737f1-bb49-4fc3-abfd-2bbd21ea6cd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:06.092267 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1057/2102] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:05 2022] POST /identity/v3/projects => generated 373 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:06.093799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9ea94-33c0-47d0-bd18-c03f324b1fa8 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:06.094083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9ea94-33c0-47d0-bd18-c03f324b1fa8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:06.094451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9ea94-33c0-47d0-bd18-c03f324b1fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:06.112920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942737f1-bb49-4fc3-abfd-2bbd21ea6cd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:06.114484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942737f1-bb49-4fc3-abfd-2bbd21ea6cd4 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:06.155359 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1046/2103] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:06.157985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942737f1-bb49-4fc3-abfd-2bbd21ea6cd4 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:06.159093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9be9ea94-33c0-47d0-bd18-c03f324b1fa8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:06.161577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9be9ea94-33c0-47d0-bd18-c03f324b1fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:06.178190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942737f1-bb49-4fc3-abfd-2bbd21ea6cd4 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:06.369574 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1058/2104] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:06.370810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9ea94-33c0-47d0-bd18-c03f324b1fa8 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:06.377094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9be9ea94-33c0-47d0-bd18-c03f324b1fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:06.387714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8decf-3046-4c3f-a6a9-2f2160bebd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:06.389688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8decf-3046-4c3f-a6a9-2f2160bebd81 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:06.390359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8decf-3046-4c3f-a6a9-2f2160bebd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:06.390853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8decf-3046-4c3f-a6a9-2f2160bebd81 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:06.393131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8decf-3046-4c3f-a6a9-2f2160bebd81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:06.396038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8decf-3046-4c3f-a6a9-2f2160bebd81 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:06.413468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9ea94-33c0-47d0-bd18-c03f324b1fa8 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:06.425552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:06.426363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:06.427082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:06.427082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] PATH_INFO: `/v3/projects/60a0d8eb4f54460ebf5987d14b2108e2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:06.430785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=60a0d8eb4f54460ebf5987d14b2108e2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:06.435666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:06.453680 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1047/2105] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:06 2022] POST /identity/v3/users => generated 287 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:06.463270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276b5ff-44c1-40de-81ac-a79beaf9dad3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:06.463857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276b5ff-44c1-40de-81ac-a79beaf9dad3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:06.464135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276b5ff-44c1-40de-81ac-a79beaf9dad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:06.502186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1276b5ff-44c1-40de-81ac-a79beaf9dad3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:06.503440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1276b5ff-44c1-40de-81ac-a79beaf9dad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:06.535537 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1048/2106] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:06.536599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276b5ff-44c1-40de-81ac-a79beaf9dad3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:06.547651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276b5ff-44c1-40de-81ac-a79beaf9dad3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:06.552914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60a0d8eb4f54460ebf5987d14b2108e2', 'request_id': 'req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:06.571286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60a0d8eb4f54460ebf5987d14b2108e2', 'request_id': 'req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:06.581697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60a0d8eb4f54460ebf5987d14b2108e2', 'request_id': 'req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:06.582506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:06.611862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] The token cache is being invalidate because project 60a0d8eb4f54460ebf5987d14b2108e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:06.612746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:06.617168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:06.620485 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1059/2107] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:06 2022] DELETE /identity/v3/projects/60a0d8eb4f54460ebf5987d14b2108e2 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:06.622095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:06.624213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:06.669289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:06.704959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7467aaf-38d6-49ff-ba07-0e8609d02fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:06.705882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7467aaf-38d6-49ff-ba07-0e8609d02fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:06.707593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7467aaf-38d6-49ff-ba07-0e8609d02fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:06.707737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7467aaf-38d6-49ff-ba07-0e8609d02fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:06.724455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7467aaf-38d6-49ff-ba07-0e8609d02fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:06.745122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7467aaf-38d6-49ff-ba07-0e8609d02fd4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:06.747736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5593ad-f695-4dc1-ac4d-baff4bcdbe1e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:06.777628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da2b47-61f9-4e49-b895-b8608e0a6819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:06.778261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da2b47-61f9-4e49-b895-b8608e0a6819 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:06.778532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da2b47-61f9-4e49-b895-b8608e0a6819 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:06.778800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da2b47-61f9-4e49-b895-b8608e0a6819 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:06.780315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da2b47-61f9-4e49-b895-b8608e0a6819 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:06.780315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da2b47-61f9-4e49-b895-b8608e0a6819 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:06.781401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da2b47-61f9-4e49-b895-b8608e0a6819 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:06.791824 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1049/2108] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:06.795272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e7914-3991-48df-b14b-21d5de09b7bb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:06.795553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e7914-3991-48df-b14b-21d5de09b7bb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:06.795816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e7914-3991-48df-b14b-21d5de09b7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:06.807063 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1060/2109] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:06.808290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da2b47-61f9-4e49-b895-b8608e0a6819 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:06.810295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b0da2b47-61f9-4e49-b895-b8608e0a6819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:06.831419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da2b47-61f9-4e49-b895-b8608e0a6819 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:06.841705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494edf04-bb60-4d9a-8840-a2bbb7b7e999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:06.842747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494edf04-bb60-4d9a-8840-a2bbb7b7e999 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:06.843219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494edf04-bb60-4d9a-8840-a2bbb7b7e999 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:06.843677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494edf04-bb60-4d9a-8840-a2bbb7b7e999 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:06.844459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fe8e7914-3991-48df-b14b-21d5de09b7bb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:06.844916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494edf04-bb60-4d9a-8840-a2bbb7b7e999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:06.846585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fe8e7914-3991-48df-b14b-21d5de09b7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:06.847316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494edf04-bb60-4d9a-8840-a2bbb7b7e999 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:06.868171 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1050/2110] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:06.868171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e7914-3991-48df-b14b-21d5de09b7bb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:06.877465 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1061/2111] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:06.877465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e951efc-2226-4c7f-b94d-a8f18143df9c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:06.877465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e951efc-2226-4c7f-b94d-a8f18143df9c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:06.877465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e951efc-2226-4c7f-b94d-a8f18143df9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:06.879677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e7914-3991-48df-b14b-21d5de09b7bb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:06.885321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3dd5b-94f5-422b-9518-be43302a1775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:06.886190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3dd5b-94f5-422b-9518-be43302a1775 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:06.886902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3dd5b-94f5-422b-9518-be43302a1775 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:06.887419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3dd5b-94f5-422b-9518-be43302a1775 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:06.888259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3dd5b-94f5-422b-9518-be43302a1775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:06.891659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3dd5b-94f5-422b-9518-be43302a1775 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:06.929252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5e951efc-2226-4c7f-b94d-a8f18143df9c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:06.930954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5e951efc-2226-4c7f-b94d-a8f18143df9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:06.946338 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1051/2112] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:06 2022] POST /identity/v3/projects => generated 393 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:06.947634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3dd5b-94f5-422b-9518-be43302a1775 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:06.961366 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1062/2113] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:06.961366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e951efc-2226-4c7f-b94d-a8f18143df9c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:06.989410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e951efc-2226-4c7f-b94d-a8f18143df9c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:07.085574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3dd5b-94f5-422b-9518-be43302a1775 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:07.092265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc34da-23e2-4434-88c4-59e4f7e65885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:07.093020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc34da-23e2-4434-88c4-59e4f7e65885 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:07.093293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc34da-23e2-4434-88c4-59e4f7e65885 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:07.093563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc34da-23e2-4434-88c4-59e4f7e65885 admin admin] PATH_INFO: `/v3/projects/7b43087682a341af83e0b3d5fe8ed1b4/users/7df78853c45d47ba8afe445b80f7e786/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:07.125956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc34da-23e2-4434-88c4-59e4f7e65885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b43087682a341af83e0b3d5fe8ed1b4, user_id=7df78853c45d47ba8afe445b80f7e786, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:07.129871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc34da-23e2-4434-88c4-59e4f7e65885 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:07.133148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0dab8-5c24-4993-8411-1266fda4d893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:07.137258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0dab8-5c24-4993-8411-1266fda4d893 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:07.137258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0dab8-5c24-4993-8411-1266fda4d893 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:07.137258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0dab8-5c24-4993-8411-1266fda4d893 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:07.137920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0dab8-5c24-4993-8411-1266fda4d893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:07.141659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0dab8-5c24-4993-8411-1266fda4d893 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:07.166497 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1052/2114] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:06 2022] PUT /identity/v3/projects/7b43087682a341af83e0b3d5fe8ed1b4/users/7df78853c45d47ba8afe445b80f7e786/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:07.168018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc34da-23e2-4434-88c4-59e4f7e65885 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:07.185734 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1063/2115] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:06 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:07.188598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77432264-7198-424f-a734-36296d53d772 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:07.189043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77432264-7198-424f-a734-36296d53d772 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:07.191413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77432264-7198-424f-a734-36296d53d772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:07.250777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-77432264-7198-424f-a734-36296d53d772 None None] MFA Rules not processed for user `7df78853c45d47ba8afe445b80f7e786`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:07.255858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-77432264-7198-424f-a734-36296d53d772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:07.290348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc34da-23e2-4434-88c4-59e4f7e65885 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:07.301542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d50ce4-339a-499c-937f-f65711a6b006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:07.302700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d50ce4-339a-499c-937f-f65711a6b006 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:07.303168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d50ce4-339a-499c-937f-f65711a6b006 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:07.303628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d50ce4-339a-499c-937f-f65711a6b006 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:07.304498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d50ce4-339a-499c-937f-f65711a6b006 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:07.304992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d50ce4-339a-499c-937f-f65711a6b006 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:07.307296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d50ce4-339a-499c-937f-f65711a6b006 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:07.326320 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1053/2116] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:07.327499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d50ce4-339a-499c-937f-f65711a6b006 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:07.338077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:07.347598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d50ce4-339a-499c-937f-f65711a6b006 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:07.355793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884585d-2ea8-461c-9d94-3265c5957738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:07.356603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884585d-2ea8-461c-9d94-3265c5957738 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:07.357570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884585d-2ea8-461c-9d94-3265c5957738 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:07.360409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884585d-2ea8-461c-9d94-3265c5957738 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:07.360409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884585d-2ea8-461c-9d94-3265c5957738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:07.360969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884585d-2ea8-461c-9d94-3265c5957738 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:07.390325 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1054/2117] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:07.391247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884585d-2ea8-461c-9d94-3265c5957738 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:07.411438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884585d-2ea8-461c-9d94-3265c5957738 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:07.416728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005e5b3-b236-4f94-a305-d9cfbf0c6e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:07.417363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005e5b3-b236-4f94-a305-d9cfbf0c6e2b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:07.417637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005e5b3-b236-4f94-a305-d9cfbf0c6e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:07.417900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005e5b3-b236-4f94-a305-d9cfbf0c6e2b admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:07.418521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005e5b3-b236-4f94-a305-d9cfbf0c6e2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:07.420478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005e5b3-b236-4f94-a305-d9cfbf0c6e2b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:07.432943 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1064/2118] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:07.433985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77432264-7198-424f-a734-36296d53d772 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:07.459487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77432264-7198-424f-a734-36296d53d772 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:07.468403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc44ef-dadb-493c-bd59-ab5a7db9c9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:07.469355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc44ef-dadb-493c-bd59-ab5a7db9c9df admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:07.469581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc44ef-dadb-493c-bd59-ab5a7db9c9df admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:07.469908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc44ef-dadb-493c-bd59-ab5a7db9c9df admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:07.470744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc44ef-dadb-493c-bd59-ab5a7db9c9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:07.476879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc44ef-dadb-493c-bd59-ab5a7db9c9df admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:07.481321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:07.490103 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1055/2119] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:07 2022] POST /identity/v3/projects => generated 391 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:07.494744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005e5b3-b236-4f94-a305-d9cfbf0c6e2b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:07.498752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6005e5b3-b236-4f94-a305-d9cfbf0c6e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:07.511606 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1065/2120] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:07.512575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc44ef-dadb-493c-bd59-ab5a7db9c9df admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:07.539741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc44ef-dadb-493c-bd59-ab5a7db9c9df admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:07.670169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005e5b3-b236-4f94-a305-d9cfbf0c6e2b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:07.678254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:07.678841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8756e-4b0d-421c-af5f-d27ee5fe520e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:07.679237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:07.679457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8756e-4b0d-421c-af5f-d27ee5fe520e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:07.679726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8756e-4b0d-421c-af5f-d27ee5fe520e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:07.679929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:07.680059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8756e-4b0d-421c-af5f-d27ee5fe520e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:07.680427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] PATH_INFO: `/v3/projects/e741faff80614dd3b7bb376b6c577f53` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:07.680823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8756e-4b0d-421c-af5f-d27ee5fe520e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:07.682297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e741faff80614dd3b7bb376b6c577f53)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:07.684565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8756e-4b0d-421c-af5f-d27ee5fe520e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:07.700289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:07.753013 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1066/2121] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:07 2022] POST /identity/v3/projects => generated 371 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:07.754105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8756e-4b0d-421c-af5f-d27ee5fe520e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:07.842215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e741faff80614dd3b7bb376b6c577f53', 'request_id': 'req-fba3fb14-36bf-449b-9a20-e739663847fc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:07.846676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8756e-4b0d-421c-af5f-d27ee5fe520e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:07.876169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e741faff80614dd3b7bb376b6c577f53', 'request_id': 'req-fba3fb14-36bf-449b-9a20-e739663847fc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:07.895036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e741faff80614dd3b7bb376b6c577f53', 'request_id': 'req-fba3fb14-36bf-449b-9a20-e739663847fc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:07.916424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] The token cache is being invalidate because project e741faff80614dd3b7bb376b6c577f53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:07.916786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:07.917527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:07.922154 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1056/2122] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:07 2022] DELETE /identity/v3/projects/e741faff80614dd3b7bb376b6c577f53 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:07.922154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:07.922154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:07.947981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2a27e-9d6f-43a1-8f64-c863dd52fed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:07.949148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2a27e-9d6f-43a1-8f64-c863dd52fed1 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:07.949148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2a27e-9d6f-43a1-8f64-c863dd52fed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:07.949323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2a27e-9d6f-43a1-8f64-c863dd52fed1 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:07.949983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2a27e-9d6f-43a1-8f64-c863dd52fed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:07.953713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2a27e-9d6f-43a1-8f64-c863dd52fed1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:08.023745 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1067/2123] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:07 2022] POST /identity/v3/users => generated 296 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:08.025424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2a27e-9d6f-43a1-8f64-c863dd52fed1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:08.029029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-28c2a27e-9d6f-43a1-8f64-c863dd52fed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:08.064273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2a27e-9d6f-43a1-8f64-c863dd52fed1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:08.071595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53dceb-6fcf-4ec0-a9ef-30f9c4f255d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:08.072228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53dceb-6fcf-4ec0-a9ef-30f9c4f255d3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:08.072556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53dceb-6fcf-4ec0-a9ef-30f9c4f255d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:08.072827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53dceb-6fcf-4ec0-a9ef-30f9c4f255d3 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:08.073838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53dceb-6fcf-4ec0-a9ef-30f9c4f255d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:08.075877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53dceb-6fcf-4ec0-a9ef-30f9c4f255d3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:08.080478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3fb14-36bf-449b-9a20-e739663847fc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:08.095305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce319c0-b8e5-4173-b78b-5f87948f323f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:08.096283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce319c0-b8e5-4173-b78b-5f87948f323f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:08.096753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce319c0-b8e5-4173-b78b-5f87948f323f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:08.096753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce319c0-b8e5-4173-b78b-5f87948f323f admin admin] PATH_INFO: `/v3/projects/8ffe1d0b8b60449895d98a4df2c0affa/users/09a500b6be6e4ab0a0e89dd69e8a169b/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:08.101860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:08.114760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:08.128326 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1068/2124] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:08 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:08.129657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53dceb-6fcf-4ec0-a9ef-30f9c4f255d3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:08.133247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cf53dceb-6fcf-4ec0-a9ef-30f9c4f255d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:08.134448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce319c0-b8e5-4173-b78b-5f87948f323f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ffe1d0b8b60449895d98a4df2c0affa, user_id=09a500b6be6e4ab0a0e89dd69e8a169b, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:08.138768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce319c0-b8e5-4173-b78b-5f87948f323f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:08.154989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53dceb-6fcf-4ec0-a9ef-30f9c4f255d3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:08.162171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b5ee8-aa25-450a-84a9-2dc25317c9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:08.163446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b5ee8-aa25-450a-84a9-2dc25317c9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:08.163865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b5ee8-aa25-450a-84a9-2dc25317c9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:08.164192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b5ee8-aa25-450a-84a9-2dc25317c9ae admin admin] PATH_INFO: `/v3/users/10c860aed5c846aa8c96c27ed38272ea` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:08.181661 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1057/2125] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:07 2022] PUT /identity/v3/projects/8ffe1d0b8b60449895d98a4df2c0affa/users/09a500b6be6e4ab0a0e89dd69e8a169b/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:08.182947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce319c0-b8e5-4173-b78b-5f87948f323f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:08.187700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b5ee8-aa25-450a-84a9-2dc25317c9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=10c860aed5c846aa8c96c27ed38272ea)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:08.199890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2ce319c0-b8e5-4173-b78b-5f87948f323f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:08.213176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b5ee8-aa25-450a-84a9-2dc25317c9ae admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:08.367102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-638b5ee8-aa25-450a-84a9-2dc25317c9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10c860aed5c846aa8c96c27ed38272ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:08.384290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-638b5ee8-aa25-450a-84a9-2dc25317c9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10c860aed5c846aa8c96c27ed38272ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:08.395723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-638b5ee8-aa25-450a-84a9-2dc25317c9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10c860aed5c846aa8c96c27ed38272ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:08.399221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce319c0-b8e5-4173-b78b-5f87948f323f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:08.408103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b3c6a-781a-47c5-ad55-c21e6ec7a50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:08.408946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b3c6a-781a-47c5-ad55-c21e6ec7a50e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:08.409223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b3c6a-781a-47c5-ad55-c21e6ec7a50e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:08.409533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b3c6a-781a-47c5-ad55-c21e6ec7a50e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:08.410471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b3c6a-781a-47c5-ad55-c21e6ec7a50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:08.412663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b3c6a-781a-47c5-ad55-c21e6ec7a50e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:08.427287 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1058/2126] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:08.430574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-638b5ee8-aa25-450a-84a9-2dc25317c9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10c860aed5c846aa8c96c27ed38272ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:08.431932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-638b5ee8-aa25-450a-84a9-2dc25317c9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10c860aed5c846aa8c96c27ed38272ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:08.432336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b3c6a-781a-47c5-ad55-c21e6ec7a50e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:08.434188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1b6b3c6a-781a-47c5-ad55-c21e6ec7a50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:08.463470 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1069/2127] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:08 2022] DELETE /identity/v3/users/10c860aed5c846aa8c96c27ed38272ea => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:08.463470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0c810-a19c-4a30-9cf3-f28fd27e82bb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:08.463470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0c810-a19c-4a30-9cf3-f28fd27e82bb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:08.464798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0c810-a19c-4a30-9cf3-f28fd27e82bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:08.479235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b3c6a-781a-47c5-ad55-c21e6ec7a50e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:08.540597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fed0c810-a19c-4a30-9cf3-f28fd27e82bb None None] MFA Rules not processed for user `09a500b6be6e4ab0a0e89dd69e8a169b`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:08.542901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fed0c810-a19c-4a30-9cf3-f28fd27e82bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:08.674589 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1070/2128] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:08.675026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3990a0d-e497-4b07-a940-1cce954982b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:08.675858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3990a0d-e497-4b07-a940-1cce954982b6 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:08.676104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d86bc-300d-4f43-af02-6e9eb37a1803 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:08.676324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3990a0d-e497-4b07-a940-1cce954982b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:08.676569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d86bc-300d-4f43-af02-6e9eb37a1803 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:08.676855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3990a0d-e497-4b07-a940-1cce954982b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:08.677070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d86bc-300d-4f43-af02-6e9eb37a1803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:08.677810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3990a0d-e497-4b07-a940-1cce954982b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:08.679965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3990a0d-e497-4b07-a940-1cce954982b6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:08.709647 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1059/2129] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:08.710591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3990a0d-e497-4b07-a940-1cce954982b6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:08.712750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c3990a0d-e497-4b07-a940-1cce954982b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:08.717744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2d9d86bc-300d-4f43-af02-6e9eb37a1803 None None] MFA Rules not processed for user `e7e70776606c41b28dd8092752a1619c`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:08.720855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2d9d86bc-300d-4f43-af02-6e9eb37a1803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:08.725757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3990a0d-e497-4b07-a940-1cce954982b6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:08.732317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea8dd8-bcef-4341-b50d-0bfeb6dbc924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:08.733030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea8dd8-bcef-4341-b50d-0bfeb6dbc924 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:08.733295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea8dd8-bcef-4341-b50d-0bfeb6dbc924 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:08.733559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea8dd8-bcef-4341-b50d-0bfeb6dbc924 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:08.734201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea8dd8-bcef-4341-b50d-0bfeb6dbc924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:08.736366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea8dd8-bcef-4341-b50d-0bfeb6dbc924 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:08.755226 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1060/2130] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:08.759694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea8dd8-bcef-4341-b50d-0bfeb6dbc924 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:08.765451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a3ea8dd8-bcef-4341-b50d-0bfeb6dbc924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:08.792210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea8dd8-bcef-4341-b50d-0bfeb6dbc924 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:08.803184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabea85-b27c-4c0e-aaef-4a3be9d44aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:08.803957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabea85-b27c-4c0e-aaef-4a3be9d44aef admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:08.804232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabea85-b27c-4c0e-aaef-4a3be9d44aef admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:08.804532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabea85-b27c-4c0e-aaef-4a3be9d44aef admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:08.805172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabea85-b27c-4c0e-aaef-4a3be9d44aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:08.814224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabea85-b27c-4c0e-aaef-4a3be9d44aef admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:08.817255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:08.856146 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1071/2131] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:08.856146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d86bc-300d-4f43-af02-6e9eb37a1803 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:08.868157 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1061/2132] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:08 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:08.869144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabea85-b27c-4c0e-aaef-4a3be9d44aef admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:08.877447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d86bc-300d-4f43-af02-6e9eb37a1803 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:08.975450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:08.997362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10cef5-f952-4765-bc45-cf7f3503473b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:08.998096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10cef5-f952-4765-bc45-cf7f3503473b admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:08.999035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10cef5-f952-4765-bc45-cf7f3503473b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:08.999035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10cef5-f952-4765-bc45-cf7f3503473b admin admin] PATH_INFO: `/v3/projects/3b9527f28a9846ff84e2a965962e7329/users/a2893a64447f419c89ce450babc9fbe5/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:09.025546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabea85-b27c-4c0e-aaef-4a3be9d44aef admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:09.026266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10cef5-f952-4765-bc45-cf7f3503473b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b9527f28a9846ff84e2a965962e7329, user_id=a2893a64447f419c89ce450babc9fbe5, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:09.040310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be26bd-26f7-4e2b-92fd-4eef5742f33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:09.040310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be26bd-26f7-4e2b-92fd-4eef5742f33a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:09.040310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be26bd-26f7-4e2b-92fd-4eef5742f33a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:09.041649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be26bd-26f7-4e2b-92fd-4eef5742f33a admin admin] PATH_INFO: `/v3/projects/a8dc72ffe83845899ffcf918c3f79302/users/b3c65f1a7588466f8fdb1a614f98a69d/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:09.042687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10cef5-f952-4765-bc45-cf7f3503473b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:09.071127 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1072/2133] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:08 2022] PUT /identity/v3/projects/3b9527f28a9846ff84e2a965962e7329/users/a2893a64447f419c89ce450babc9fbe5/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:09.072557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10cef5-f952-4765-bc45-cf7f3503473b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:09.074774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-eb10cef5-f952-4765-bc45-cf7f3503473b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:09.077398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be26bd-26f7-4e2b-92fd-4eef5742f33a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8dc72ffe83845899ffcf918c3f79302, user_id=b3c65f1a7588466f8fdb1a614f98a69d, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:09.080276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be26bd-26f7-4e2b-92fd-4eef5742f33a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:09.132739 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1062/2134] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:08 2022] PUT /identity/v3/projects/a8dc72ffe83845899ffcf918c3f79302/users/b3c65f1a7588466f8fdb1a614f98a69d/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:09.134151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be26bd-26f7-4e2b-92fd-4eef5742f33a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:09.172496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10cef5-f952-4765-bc45-cf7f3503473b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:09.273797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be26bd-26f7-4e2b-92fd-4eef5742f33a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:09.279526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53459b-24bc-4306-823d-242bbd92d097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:09.280295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53459b-24bc-4306-823d-242bbd92d097 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:09.280609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53459b-24bc-4306-823d-242bbd92d097 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:09.280873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53459b-24bc-4306-823d-242bbd92d097 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:09.281806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53459b-24bc-4306-823d-242bbd92d097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:09.284292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53459b-24bc-4306-823d-242bbd92d097 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:09.298345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a5273-115c-41fd-92e9-14fd265aa628 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:09.298980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a5273-115c-41fd-92e9-14fd265aa628 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:09.299371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a5273-115c-41fd-92e9-14fd265aa628 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:09.299889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a5273-115c-41fd-92e9-14fd265aa628 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:09.314838 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1063/2135] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:09 2022] POST /identity/v3/users => generated 302 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:09.316126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53459b-24bc-4306-823d-242bbd92d097 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:09.318583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3a53459b-24bc-4306-823d-242bbd92d097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:09.318743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:09.336430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53459b-24bc-4306-823d-242bbd92d097 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:09.341894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:09.342636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:09.343002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:09.344575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32 admin admin] PATH_INFO: `/v3/projects/5f86c8a4563b46f98542eeaa49be7e46` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:09.346322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f86c8a4563b46f98542eeaa49be7e46)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:09.348533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:09.381621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a5273-115c-41fd-92e9-14fd265aa628 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:09.383480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a5273-115c-41fd-92e9-14fd265aa628 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:09.394703 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1073/2136] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:09.397435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a5273-115c-41fd-92e9-14fd265aa628 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:09.406812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f86c8a4563b46f98542eeaa49be7e46', 'request_id': 'req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:09.410800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a5273-115c-41fd-92e9-14fd265aa628 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:09.415075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f86c8a4563b46f98542eeaa49be7e46', 'request_id': 'req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:09.425403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f86c8a4563b46f98542eeaa49be7e46', 'request_id': 'req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:09.440305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32 admin admin] The token cache is being invalidate because project 5f86c8a4563b46f98542eeaa49be7e46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:09.440831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:09.442537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c7c36691-a81b-44a8-bc5c-e73b31d0aa32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:09.448812 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1064/2137] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:09 2022] DELETE /identity/v3/projects/5f86c8a4563b46f98542eeaa49be7e46 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:09.448812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5779f-cf17-4011-8a06-d7fc65c3776a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:09.448812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5779f-cf17-4011-8a06-d7fc65c3776a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:09.448812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5779f-cf17-4011-8a06-d7fc65c3776a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:09.486412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c8f5779f-cf17-4011-8a06-d7fc65c3776a None None] MFA Rules not processed for user `b3c65f1a7588466f8fdb1a614f98a69d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:09.486686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62729c-f58e-4521-8b8b-17762f447db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:09.487424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62729c-f58e-4521-8b8b-17762f447db8 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:09.487795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62729c-f58e-4521-8b8b-17762f447db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:09.488051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62729c-f58e-4521-8b8b-17762f447db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:09.489017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62729c-f58e-4521-8b8b-17762f447db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:09.490765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62729c-f58e-4521-8b8b-17762f447db8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:09.493896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c8f5779f-cf17-4011-8a06-d7fc65c3776a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:09.504446 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1074/2138] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:09.505376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62729c-f58e-4521-8b8b-17762f447db8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:09.507143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c62729c-f58e-4521-8b8b-17762f447db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:09.528570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62729c-f58e-4521-8b8b-17762f447db8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:09.532912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:09.536317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9400fa4-05f3-482f-86a2-797d053cdb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:09.537775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9400fa4-05f3-482f-86a2-797d053cdb25 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:09.537775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9400fa4-05f3-482f-86a2-797d053cdb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:09.538121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9400fa4-05f3-482f-86a2-797d053cdb25 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:09.539884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9400fa4-05f3-482f-86a2-797d053cdb25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:09.542467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9400fa4-05f3-482f-86a2-797d053cdb25 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:09.560570 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1075/2139] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:09.561750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9400fa4-05f3-482f-86a2-797d053cdb25 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:09.568509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e9400fa4-05f3-482f-86a2-797d053cdb25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:09.593181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9400fa4-05f3-482f-86a2-797d053cdb25 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:09.600989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:09.601729 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1065/2140] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:09.603053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1c691-a826-42b5-89fd-d03622e06d8e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:09.603346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1c691-a826-42b5-89fd-d03622e06d8e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:09.604044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1c691-a826-42b5-89fd-d03622e06d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:09.607076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:09.607636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:09.607917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] PATH_INFO: `/v3/users/1b96f59ee1304302bc8f1e0b728f1df3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:09.609668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b96f59ee1304302bc8f1e0b728f1df3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:09.612145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:09.631059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4dc1c691-a826-42b5-89fd-d03622e06d8e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:09.632399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4dc1c691-a826-42b5-89fd-d03622e06d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:09.648990 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1066/2141] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:09.649742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1c691-a826-42b5-89fd-d03622e06d8e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:09.651410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4dc1c691-a826-42b5-89fd-d03622e06d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:09.666531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1c691-a826-42b5-89fd-d03622e06d8e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:09.671651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc0d10-9c5d-4cba-9e3e-dfe638500bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:09.672779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc0d10-9c5d-4cba-9e3e-dfe638500bad admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:09.673146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc0d10-9c5d-4cba-9e3e-dfe638500bad admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:09.673421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc0d10-9c5d-4cba-9e3e-dfe638500bad admin admin] PATH_INFO: `/v3/projects/3b9527f28a9846ff84e2a965962e7329/users/a2893a64447f419c89ce450babc9fbe5/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:09.675692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc0d10-9c5d-4cba-9e3e-dfe638500bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b9527f28a9846ff84e2a965962e7329, user_id=a2893a64447f419c89ce450babc9fbe5, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:09.678888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc0d10-9c5d-4cba-9e3e-dfe638500bad admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:09.704247 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1067/2142] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:09 2022] PUT /identity/v3/projects/3b9527f28a9846ff84e2a965962e7329/users/a2893a64447f419c89ce450babc9fbe5/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:09.705966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc0d10-9c5d-4cba-9e3e-dfe638500bad admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:09.706084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b96f59ee1304302bc8f1e0b728f1df3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:09.725361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b96f59ee1304302bc8f1e0b728f1df3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:09.758575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b96f59ee1304302bc8f1e0b728f1df3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:09.805363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b96f59ee1304302bc8f1e0b728f1df3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:09.806065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b96f59ee1304302bc8f1e0b728f1df3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:09.835491 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1076/2143] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:09 2022] DELETE /identity/v3/users/1b96f59ee1304302bc8f1e0b728f1df3 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:09.836743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:09.839108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:09.845063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc0d10-9c5d-4cba-9e3e-dfe638500bad admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:09.859891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:09.963306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759bebf-f12a-4e78-80a4-4f199ca8c022 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:09.973916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f37b1-8821-4cd3-b1fd-77e40e1300d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:09.975396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f37b1-8821-4cd3-b1fd-77e40e1300d8 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:09.976313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f37b1-8821-4cd3-b1fd-77e40e1300d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:09.976627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f37b1-8821-4cd3-b1fd-77e40e1300d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:09.977359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f37b1-8821-4cd3-b1fd-77e40e1300d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:09.979839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f37b1-8821-4cd3-b1fd-77e40e1300d8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:09.997211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69ff06-e33d-49bc-b787-66c5f3675343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:09.997904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69ff06-e33d-49bc-b787-66c5f3675343 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:09.998190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69ff06-e33d-49bc-b787-66c5f3675343 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:09.998443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69ff06-e33d-49bc-b787-66c5f3675343 admin admin] PATH_INFO: `/v3/projects/a66ff272ec8b489987e463cd52b2d6ac/users/eaba83770cd145eb9fe0687476a4845d/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.024747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69ff06-e33d-49bc-b787-66c5f3675343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a66ff272ec8b489987e463cd52b2d6ac, user_id=eaba83770cd145eb9fe0687476a4845d, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:10.026175 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1077/2144] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:09 2022] POST /identity/v3/projects => generated 389 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:10.028184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3194e1-36c2-4738-8e8d-2be6216dfab7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.028490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3194e1-36c2-4738-8e8d-2be6216dfab7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.029181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3194e1-36c2-4738-8e8d-2be6216dfab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.031749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69ff06-e33d-49bc-b787-66c5f3675343 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:10.054145 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1068/2145] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:09 2022] PUT /identity/v3/projects/a66ff272ec8b489987e463cd52b2d6ac/users/eaba83770cd145eb9fe0687476a4845d/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:10.056107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f680aa4-4699-4579-81c1-761534bb3b86 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.056478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f680aa4-4699-4579-81c1-761534bb3b86 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.056834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f680aa4-4699-4579-81c1-761534bb3b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.070268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ac3194e1-36c2-4738-8e8d-2be6216dfab7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:10.072940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ac3194e1-36c2-4738-8e8d-2be6216dfab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:10.101276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6f680aa4-4699-4579-81c1-761534bb3b86 None None] MFA Rules not processed for user `a2893a64447f419c89ce450babc9fbe5`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:10.103372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6f680aa4-4699-4579-81c1-761534bb3b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:10.142633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:10.161801 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1078/2146] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:10.161801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9729c98-2375-4cf3-9e30-5f4689dc2118 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.161801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9729c98-2375-4cf3-9e30-5f4689dc2118 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.161801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9729c98-2375-4cf3-9e30-5f4689dc2118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.191642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d9729c98-2375-4cf3-9e30-5f4689dc2118 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:10.193397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d9729c98-2375-4cf3-9e30-5f4689dc2118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:10.213352 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1079/2147] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:10.214250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9729c98-2375-4cf3-9e30-5f4689dc2118 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:10.227414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9729c98-2375-4cf3-9e30-5f4689dc2118 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:10.232941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b3f52-62f9-402c-a56c-7505568cc153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:10.233600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b3f52-62f9-402c-a56c-7505568cc153 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.233869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b3f52-62f9-402c-a56c-7505568cc153 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.234118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b3f52-62f9-402c-a56c-7505568cc153 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.235869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b3f52-62f9-402c-a56c-7505568cc153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:10.242132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b3f52-62f9-402c-a56c-7505568cc153 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:10.242849 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1069/2148] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:10.242849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f680aa4-4699-4579-81c1-761534bb3b86 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:10.244271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6f680aa4-4699-4579-81c1-761534bb3b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:10.268765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f680aa4-4699-4579-81c1-761534bb3b86 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:10.278376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc13828-8c41-42b9-a216-e99491dc6416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:10.280630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc13828-8c41-42b9-a216-e99491dc6416 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.281027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc13828-8c41-42b9-a216-e99491dc6416 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.281559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc13828-8c41-42b9-a216-e99491dc6416 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.283308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc13828-8c41-42b9-a216-e99491dc6416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:10.285998 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1080/2149] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:10 2022] POST /identity/v3/users => generated 301 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:10.287855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc13828-8c41-42b9-a216-e99491dc6416 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:10.290358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b3f52-62f9-402c-a56c-7505568cc153 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:10.307665 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1070/2150] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:10.310462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b3f52-62f9-402c-a56c-7505568cc153 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:10.311454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc13828-8c41-42b9-a216-e99491dc6416 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:10.315497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fdc13828-8c41-42b9-a216-e99491dc6416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:10.321762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e75f94-e428-4b6c-b3e7-f60bff3f456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:10.322531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e75f94-e428-4b6c-b3e7-f60bff3f456b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.322895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e75f94-e428-4b6c-b3e7-f60bff3f456b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.323547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e75f94-e428-4b6c-b3e7-f60bff3f456b admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.325065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e75f94-e428-4b6c-b3e7-f60bff3f456b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:10.325743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e75f94-e428-4b6c-b3e7-f60bff3f456b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:10.328658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e75f94-e428-4b6c-b3e7-f60bff3f456b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:10.352364 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1081/2151] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:10.353478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e75f94-e428-4b6c-b3e7-f60bff3f456b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:10.379027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e75f94-e428-4b6c-b3e7-f60bff3f456b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:10.390150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37de1f-3cf8-4d15-bec6-1141d2cc7e05 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2893a64447f419c89ce450babc9fbe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9527f28a9846ff84e2a965962e7329', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:10.390917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37de1f-3cf8-4d15-bec6-1141d2cc7e05 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.391281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37de1f-3cf8-4d15-bec6-1141d2cc7e05 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.391652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37de1f-3cf8-4d15-bec6-1141d2cc7e05 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.392747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37de1f-3cf8-4d15-bec6-1141d2cc7e05 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:10.393152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37de1f-3cf8-4d15-bec6-1141d2cc7e05 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:10.396876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37de1f-3cf8-4d15-bec6-1141d2cc7e05 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:10.420404 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1082/2152] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:10.422060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37de1f-3cf8-4d15-bec6-1141d2cc7e05 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:10.441916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37de1f-3cf8-4d15-bec6-1141d2cc7e05 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:10.443234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc13828-8c41-42b9-a216-e99491dc6416 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:10.449348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d5479-2caa-40e0-9351-8028cdb9426d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:10.450107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d5479-2caa-40e0-9351-8028cdb9426d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.450474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d5479-2caa-40e0-9351-8028cdb9426d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.450833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d5479-2caa-40e0-9351-8028cdb9426d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.456412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446dc97-787e-41f1-8d24-43636b067696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:10.457262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446dc97-787e-41f1-8d24-43636b067696 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.458105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446dc97-787e-41f1-8d24-43636b067696 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.459834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446dc97-787e-41f1-8d24-43636b067696 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.460623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446dc97-787e-41f1-8d24-43636b067696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:10.463771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446dc97-787e-41f1-8d24-43636b067696 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:10.469806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d5479-2caa-40e0-9351-8028cdb9426d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:10.474965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d5479-2caa-40e0-9351-8028cdb9426d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:10.487640 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1083/2153] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:10.488618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446dc97-787e-41f1-8d24-43636b067696 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:10.506738 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1071/2154] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:10.507695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d5479-2caa-40e0-9351-8028cdb9426d service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:10.526947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446dc97-787e-41f1-8d24-43636b067696 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:10.541082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d5479-2caa-40e0-9351-8028cdb9426d service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:10.543735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18618b4e-7cef-46d7-a06e-c4ed263ee613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:10.544549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18618b4e-7cef-46d7-a06e-c4ed263ee613 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.549805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18618b4e-7cef-46d7-a06e-c4ed263ee613 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.551538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18618b4e-7cef-46d7-a06e-c4ed263ee613 admin admin] PATH_INFO: `/v3/projects/a66ff272ec8b489987e463cd52b2d6ac/users/eaba83770cd145eb9fe0687476a4845d/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.553955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18618b4e-7cef-46d7-a06e-c4ed263ee613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a66ff272ec8b489987e463cd52b2d6ac, user_id=eaba83770cd145eb9fe0687476a4845d, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:10.560755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18618b4e-7cef-46d7-a06e-c4ed263ee613 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:10.561691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6677ffd-2583-4a83-b110-6aedf17026cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:10.562509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6677ffd-2583-4a83-b110-6aedf17026cf admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.562870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6677ffd-2583-4a83-b110-6aedf17026cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.563222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6677ffd-2583-4a83-b110-6aedf17026cf admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.566185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6677ffd-2583-4a83-b110-6aedf17026cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:10.570468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6677ffd-2583-4a83-b110-6aedf17026cf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:10.599078 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1084/2155] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:10 2022] PUT /identity/v3/projects/a66ff272ec8b489987e463cd52b2d6ac/users/eaba83770cd145eb9fe0687476a4845d/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:10.600175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18618b4e-7cef-46d7-a06e-c4ed263ee613 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:10.624246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:10.647092 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1072/2156] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:10 2022] POST /identity/v3/projects => generated 391 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:10.647092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6677ffd-2583-4a83-b110-6aedf17026cf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:10.700849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18618b4e-7cef-46d7-a06e-c4ed263ee613 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:10.765580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6677ffd-2583-4a83-b110-6aedf17026cf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:10.777593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f5d46-b9b6-43fd-8032-717c99211032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:10.778266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f5d46-b9b6-43fd-8032-717c99211032 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.778532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f5d46-b9b6-43fd-8032-717c99211032 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.778787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f5d46-b9b6-43fd-8032-717c99211032 admin admin] PATH_INFO: `/v3/projects/e63011fca84e43a4b1001bbd082b1bef/users/7f9214cc422141219e0377590b78a31e/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.802362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3c8ff-b1c6-4b54-bea4-d86ae76fbbca tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2893a64447f419c89ce450babc9fbe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9527f28a9846ff84e2a965962e7329', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:10.802984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3c8ff-b1c6-4b54-bea4-d86ae76fbbca tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.803256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3c8ff-b1c6-4b54-bea4-d86ae76fbbca tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.803519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3c8ff-b1c6-4b54-bea4-d86ae76fbbca tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.804152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3c8ff-b1c6-4b54-bea4-d86ae76fbbca tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:10.806096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3c8ff-b1c6-4b54-bea4-d86ae76fbbca tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:10.810688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f5d46-b9b6-43fd-8032-717c99211032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e63011fca84e43a4b1001bbd082b1bef, user_id=7f9214cc422141219e0377590b78a31e, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:10.813522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f5d46-b9b6-43fd-8032-717c99211032 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:10.835675 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1073/2157] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:10 2022] PUT /identity/v3/projects/e63011fca84e43a4b1001bbd082b1bef/users/7f9214cc422141219e0377590b78a31e/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:10.837330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d657be8-29ca-4395-94e8-ccbc011f4dba None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.837747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d657be8-29ca-4395-94e8-ccbc011f4dba None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.838150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d657be8-29ca-4395-94e8-ccbc011f4dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.867114 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1085/2158] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:10 2022] POST /identity/v3/projects => generated 369 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:10.868214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3c8ff-b1c6-4b54-bea4-d86ae76fbbca tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:10.875426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4d657be8-29ca-4395-94e8-ccbc011f4dba None None] MFA Rules not processed for user `eaba83770cd145eb9fe0687476a4845d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:10.877360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4d657be8-29ca-4395-94e8-ccbc011f4dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:10.883442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:10.963290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3c8ff-b1c6-4b54-bea4-d86ae76fbbca tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:10.969390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b08fa6-087c-489c-805b-0d7bf123c0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:10.971363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b08fa6-087c-489c-805b-0d7bf123c0d4 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:10.971963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b08fa6-087c-489c-805b-0d7bf123c0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:10.973070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b08fa6-087c-489c-805b-0d7bf123c0d4 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:10.973789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b08fa6-087c-489c-805b-0d7bf123c0d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:10.975853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b08fa6-087c-489c-805b-0d7bf123c0d4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:10.978526 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1074/2159] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:10 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:10.979420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d657be8-29ca-4395-94e8-ccbc011f4dba None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:10.993629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d657be8-29ca-4395-94e8-ccbc011f4dba None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:11.001044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78319d5e-3122-4057-83d2-1a059716fff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:11.003118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78319d5e-3122-4057-83d2-1a059716fff0 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:11.003500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78319d5e-3122-4057-83d2-1a059716fff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:11.003878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78319d5e-3122-4057-83d2-1a059716fff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:11.004726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78319d5e-3122-4057-83d2-1a059716fff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:11.006588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78319d5e-3122-4057-83d2-1a059716fff0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:11.019744 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1086/2160] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:10 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:11.019744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b08fa6-087c-489c-805b-0d7bf123c0d4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:11.023813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-00b08fa6-087c-489c-805b-0d7bf123c0d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:11.024069 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1075/2161] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:11.025408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78319d5e-3122-4057-83d2-1a059716fff0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:11.039138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b08fa6-087c-489c-805b-0d7bf123c0d4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:11.044853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:11.045508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:11.045783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:11.046050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] PATH_INFO: `/v3/projects/3fca15c7755e4b9ca531cf6b3d266a03` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:11.048730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fca15c7755e4b9ca531cf6b3d266a03)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:11.050073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:11.119546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78319d5e-3122-4057-83d2-1a059716fff0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:11.125189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03c6ab-b404-47e6-a7bc-6ff6f7f1a320 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2893a64447f419c89ce450babc9fbe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9527f28a9846ff84e2a965962e7329', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:11.125978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03c6ab-b404-47e6-a7bc-6ff6f7f1a320 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:11.126560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03c6ab-b404-47e6-a7bc-6ff6f7f1a320 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:11.126696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03c6ab-b404-47e6-a7bc-6ff6f7f1a320 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:11.127290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03c6ab-b404-47e6-a7bc-6ff6f7f1a320 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:11.127743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03c6ab-b404-47e6-a7bc-6ff6f7f1a320 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:11.129519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03c6ab-b404-47e6-a7bc-6ff6f7f1a320 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:11.168801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:11.191797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fca15c7755e4b9ca531cf6b3d266a03', 'request_id': 'req-cf3d6d69-b3a2-430a-88b5-1a565be77067'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.206096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fca15c7755e4b9ca531cf6b3d266a03', 'request_id': 'req-cf3d6d69-b3a2-430a-88b5-1a565be77067'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.208274 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1076/2162] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:11.209982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03c6ab-b404-47e6-a7bc-6ff6f7f1a320 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:11.225418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fca15c7755e4b9ca531cf6b3d266a03', 'request_id': 'req-cf3d6d69-b3a2-430a-88b5-1a565be77067'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.251218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03c6ab-b404-47e6-a7bc-6ff6f7f1a320 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:11.253424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] The token cache is being invalidate because project 3fca15c7755e4b9ca531cf6b3d266a03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:11.253815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.254613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.257319 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1087/2163] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:11 2022] DELETE /identity/v3/projects/3fca15c7755e4b9ca531cf6b3d266a03 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:11.258277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:11.260296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:11.350663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:11.351313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:11.352401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:11.352818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] PATH_INFO: `/v3/users/09a500b6be6e4ab0a0e89dd69e8a169b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:11.354007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09a500b6be6e4ab0a0e89dd69e8a169b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:11.356760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:11.364993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d6d69-b3a2-430a-88b5-1a565be77067 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:11.379171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc22830-6922-41db-9313-87c1ac61d221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:11.379998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc22830-6922-41db-9313-87c1ac61d221 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:11.380636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc22830-6922-41db-9313-87c1ac61d221 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:11.381007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc22830-6922-41db-9313-87c1ac61d221 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:11.381848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc22830-6922-41db-9313-87c1ac61d221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:11.383732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc22830-6922-41db-9313-87c1ac61d221 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:11.401025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:11.404964 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1088/2164] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:11.404964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc22830-6922-41db-9313-87c1ac61d221 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:11.407012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5cc22830-6922-41db-9313-87c1ac61d221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:11.424431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc22830-6922-41db-9313-87c1ac61d221 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:11.427623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09a500b6be6e4ab0a0e89dd69e8a169b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.431232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770820c-d6f7-4705-ada7-886dc2bea47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:11.432360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770820c-d6f7-4705-ada7-886dc2bea47b admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:11.432663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770820c-d6f7-4705-ada7-886dc2bea47b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:11.433630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770820c-d6f7-4705-ada7-886dc2bea47b admin admin] PATH_INFO: `/v3/projects/e63011fca84e43a4b1001bbd082b1bef/users/7f9214cc422141219e0377590b78a31e/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:11.436055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770820c-d6f7-4705-ada7-886dc2bea47b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e63011fca84e43a4b1001bbd082b1bef, user_id=7f9214cc422141219e0377590b78a31e, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:11.440870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770820c-d6f7-4705-ada7-886dc2bea47b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:11.441181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09a500b6be6e4ab0a0e89dd69e8a169b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.458232 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1089/2165] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:11 2022] PUT /identity/v3/projects/e63011fca84e43a4b1001bbd082b1bef/users/7f9214cc422141219e0377590b78a31e/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:11.459440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770820c-d6f7-4705-ada7-886dc2bea47b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:11.459676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09a500b6be6e4ab0a0e89dd69e8a169b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.460614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09a500b6be6e4ab0a0e89dd69e8a169b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.461929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09a500b6be6e4ab0a0e89dd69e8a169b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.462523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9770820c-d6f7-4705-ada7-886dc2bea47b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:11.482177 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1077/2166] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:11 2022] DELETE /identity/v3/users/09a500b6be6e4ab0a0e89dd69e8a169b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:11.482177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:11.482741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:11.571288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448dd75-74c0-459c-9976-e151461bc3f6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:11.587867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:11.588759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:11.588877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:11.589092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] PATH_INFO: `/v3/users/429681ce41994ec197658f17573c6081` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:11.590285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] RBAC: Authorizing `identity:delete_user(user_id=429681ce41994ec197658f17573c6081)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:11.591836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770820c-d6f7-4705-ada7-886dc2bea47b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:11.592252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:11.663287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '429681ce41994ec197658f17573c6081'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.673240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:11.675966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '429681ce41994ec197658f17573c6081'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.698131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '429681ce41994ec197658f17573c6081'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.699584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '429681ce41994ec197658f17573c6081'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.701030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '429681ce41994ec197658f17573c6081'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:11.714088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c5048-95ee-40da-972a-3367b70c16cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:11.715078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c5048-95ee-40da-972a-3367b70c16cf service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:11.715363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c5048-95ee-40da-972a-3367b70c16cf service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:11.715666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c5048-95ee-40da-972a-3367b70c16cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:11.716615 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1078/2167] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:11 2022] DELETE /identity/v3/users/429681ce41994ec197658f17573c6081 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:11.718267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-837c5048-95ee-40da-972a-3367b70c16cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:11.720999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:11.722687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:11.758618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:11.844290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa883b34-92f2-4a46-983b-ced074d5015f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:11.865631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034b41a-dc8f-4f05-ba2d-5e039fb0a3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:11.866347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034b41a-dc8f-4f05-ba2d-5e039fb0a3fe admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:11.866625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034b41a-dc8f-4f05-ba2d-5e039fb0a3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:11.866893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034b41a-dc8f-4f05-ba2d-5e039fb0a3fe admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:11.867714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034b41a-dc8f-4f05-ba2d-5e039fb0a3fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:11.869559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034b41a-dc8f-4f05-ba2d-5e039fb0a3fe admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:11.870074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c5048-95ee-40da-972a-3367b70c16cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:11.872606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c5048-95ee-40da-972a-3367b70c16cf service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:11.886766 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1090/2168] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:11.888354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71125f-a048-4ed2-afee-845862a30cec None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:11.889704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71125f-a048-4ed2-afee-845862a30cec None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:11.890274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71125f-a048-4ed2-afee-845862a30cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:11.904610 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1079/2169] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:11.906569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b22d0-25fc-468e-8711-fb9179c54c81 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:11.907067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b22d0-25fc-468e-8711-fb9179c54c81 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:11.907682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b22d0-25fc-468e-8711-fb9179c54c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:11.949094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7a71125f-a048-4ed2-afee-845862a30cec None None] MFA Rules not processed for user `7f9214cc422141219e0377590b78a31e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:11.952202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ee0b22d0-25fc-468e-8711-fb9179c54c81 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:11.952340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7a71125f-a048-4ed2-afee-845862a30cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:11.962550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ee0b22d0-25fc-468e-8711-fb9179c54c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:11.985141 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1080/2170] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:11.987188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b22d0-25fc-468e-8711-fb9179c54c81 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:12.031760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b22d0-25fc-468e-8711-fb9179c54c81 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:12.039950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f97a67-90a7-41db-a9c3-566d4fefd6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:12.041700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f97a67-90a7-41db-a9c3-566d4fefd6cd admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:12.042010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f97a67-90a7-41db-a9c3-566d4fefd6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:12.042379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f97a67-90a7-41db-a9c3-566d4fefd6cd admin admin] PATH_INFO: `/v3/projects/328d24ff14ce405ca559512bf8cdea07/users/5ca1ee61eaf245089afd5a7a033608be/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:12.064154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f97a67-90a7-41db-a9c3-566d4fefd6cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=328d24ff14ce405ca559512bf8cdea07, user_id=5ca1ee61eaf245089afd5a7a033608be, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:12.068022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f97a67-90a7-41db-a9c3-566d4fefd6cd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:12.094772 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1091/2171] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:11 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:12.094772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4239383-3f3c-4451-bf52-fa980c740694 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:12.094772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4239383-3f3c-4451-bf52-fa980c740694 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:12.094772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4239383-3f3c-4451-bf52-fa980c740694 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:12.094772 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1092/2172] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:56:12.096302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4239383-3f3c-4451-bf52-fa980c740694 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:12.097947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e4239383-3f3c-4451-bf52-fa980c740694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:12.122973 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1081/2173] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:11 2022] PUT /identity/v3/projects/328d24ff14ce405ca559512bf8cdea07/users/5ca1ee61eaf245089afd5a7a033608be/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:12.126022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56174f81-f2c4-4f90-a72c-a71e2d959629 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:12.126690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56174f81-f2c4-4f90-a72c-a71e2d959629 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:12.127064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56174f81-f2c4-4f90-a72c-a71e2d959629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:12.156871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-56174f81-f2c4-4f90-a72c-a71e2d959629 None None] MFA Rules not processed for user `e7e70776606c41b28dd8092752a1619c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:12.158248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-56174f81-f2c4-4f90-a72c-a71e2d959629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:12.208410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4239383-3f3c-4451-bf52-fa980c740694 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:12.214958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c75004-71eb-4a43-af45-810bcdc47e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:12.215789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c75004-71eb-4a43-af45-810bcdc47e58 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:12.216201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c75004-71eb-4a43-af45-810bcdc47e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:12.216664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c75004-71eb-4a43-af45-810bcdc47e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:12.275147 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1082/2174] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:12.276117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56174f81-f2c4-4f90-a72c-a71e2d959629 None None] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:56:12.277494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-56174f81-f2c4-4f90-a72c-a71e2d959629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:12.284839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:12.412822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c75004-71eb-4a43-af45-810bcdc47e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:12.414471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c75004-71eb-4a43-af45-810bcdc47e58 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:12.435335 ncdv-master-1 devstack@keystone.service[104119]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-56174f81-f2c4-4f90-a72c-a71e2d959629 None None] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Aug 20 20:56:12.435645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56174f81-f2c4-4f90-a72c-a71e2d959629 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:12.443716 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1093/2175] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:12.446476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c75004-71eb-4a43-af45-810bcdc47e58 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:12.451388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f4c75004-71eb-4a43-af45-810bcdc47e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:12.512981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c75004-71eb-4a43-af45-810bcdc47e58 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:12.523182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:12.524276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886defb5-6f9c-4ad5-91ce-0323aa70ee96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:12.525213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886defb5-6f9c-4ad5-91ce-0323aa70ee96 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:12.526831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886defb5-6f9c-4ad5-91ce-0323aa70ee96 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:12.526979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886defb5-6f9c-4ad5-91ce-0323aa70ee96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:12.592160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56174f81-f2c4-4f90-a72c-a71e2d959629 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:12.604934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe2841-24ed-4945-a767-9cc9252e872e tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2893a64447f419c89ce450babc9fbe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9527f28a9846ff84e2a965962e7329', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:12.605582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe2841-24ed-4945-a767-9cc9252e872e tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:12.605853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe2841-24ed-4945-a767-9cc9252e872e tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:12.606114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe2841-24ed-4945-a767-9cc9252e872e tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] PATH_INFO: `/v3/projects/4899a4244fb34985aa5163e0311cfe6d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:12.611508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe2841-24ed-4945-a767-9cc9252e872e tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorizing `identity:get_project(project_id=4899a4244fb34985aa5163e0311cfe6d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:12.616694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe2841-24ed-4945-a767-9cc9252e872e tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:12.620318 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1083/2176] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:56:12 2022] GET /identity/v3/projects/4899a4244fb34985aa5163e0311cfe6d => generated 369 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:12.621646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332a25c-4edd-4488-9802-b2562d3ff44c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:12.621838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332a25c-4edd-4488-9802-b2562d3ff44c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:12.622102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332a25c-4edd-4488-9802-b2562d3ff44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:12.658309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886defb5-6f9c-4ad5-91ce-0323aa70ee96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:12.660142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886defb5-6f9c-4ad5-91ce-0323aa70ee96 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:12.684199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f332a25c-4edd-4488-9802-b2562d3ff44c None None] MFA Rules not processed for user `5ca1ee61eaf245089afd5a7a033608be`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:12.685494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f332a25c-4edd-4488-9802-b2562d3ff44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:12.687315 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1094/2177] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:12 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:12.688222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886defb5-6f9c-4ad5-91ce-0323aa70ee96 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:12.690291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-886defb5-6f9c-4ad5-91ce-0323aa70ee96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:12.720225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886defb5-6f9c-4ad5-91ce-0323aa70ee96 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:12.734673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d8884-4465-44d1-9582-c37f058e53cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:12.735222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d8884-4465-44d1-9582-c37f058e53cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:12.735604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d8884-4465-44d1-9582-c37f058e53cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:12.735838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d8884-4465-44d1-9582-c37f058e53cd admin admin] PATH_INFO: `/v3/projects/a466057405124a4cb8f212e31a08984f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:12.741390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d8884-4465-44d1-9582-c37f058e53cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a466057405124a4cb8f212e31a08984f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:12.743306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d8884-4465-44d1-9582-c37f058e53cd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:12.873712 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1084/2178] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:12.874934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332a25c-4edd-4488-9802-b2562d3ff44c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:12.883171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-145d8884-4465-44d1-9582-c37f058e53cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a466057405124a4cb8f212e31a08984f', 'request_id': 'req-145d8884-4465-44d1-9582-c37f058e53cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:12.900977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332a25c-4edd-4488-9802-b2562d3ff44c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:12.903125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-145d8884-4465-44d1-9582-c37f058e53cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a466057405124a4cb8f212e31a08984f', 'request_id': 'req-145d8884-4465-44d1-9582-c37f058e53cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:12.916266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-145d8884-4465-44d1-9582-c37f058e53cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a466057405124a4cb8f212e31a08984f', 'request_id': 'req-145d8884-4465-44d1-9582-c37f058e53cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:12.966874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-145d8884-4465-44d1-9582-c37f058e53cd admin admin] The token cache is being invalidate because project a466057405124a4cb8f212e31a08984f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:12.969580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-145d8884-4465-44d1-9582-c37f058e53cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:12.972839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-145d8884-4465-44d1-9582-c37f058e53cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:12.974866 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1095/2179] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:12 2022] DELETE /identity/v3/projects/a466057405124a4cb8f212e31a08984f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:12.982257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389055dd-6c2c-426d-8adc-eeeb3be0123a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:12.983499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389055dd-6c2c-426d-8adc-eeeb3be0123a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:12.983844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389055dd-6c2c-426d-8adc-eeeb3be0123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:13.083586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-389055dd-6c2c-426d-8adc-eeeb3be0123a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:13.088423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-389055dd-6c2c-426d-8adc-eeeb3be0123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:13.152197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e59885-adec-41a7-b524-f80614ec6f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:13.152925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e59885-adec-41a7-b524-f80614ec6f8f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:13.153273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e59885-adec-41a7-b524-f80614ec6f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:13.153678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e59885-adec-41a7-b524-f80614ec6f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:13.154325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e59885-adec-41a7-b524-f80614ec6f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:13.156817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:13.164702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e59885-adec-41a7-b524-f80614ec6f8f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:13.218758 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1085/2180] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:12 2022] POST /identity/v3/projects => generated 393 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:13.219952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e59885-adec-41a7-b524-f80614ec6f8f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:13.222002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a9e59885-adec-41a7-b524-f80614ec6f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:13.264902 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1096/2181] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:13.266219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389055dd-6c2c-426d-8adc-eeeb3be0123a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:13.269388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-389055dd-6c2c-426d-8adc-eeeb3be0123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:13.293139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:13.304774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389055dd-6c2c-426d-8adc-eeeb3be0123a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:13.321158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659dc79b-71a5-46e6-b86d-0714682653cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:13.322011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659dc79b-71a5-46e6-b86d-0714682653cf admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:13.322575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659dc79b-71a5-46e6-b86d-0714682653cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:13.323013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659dc79b-71a5-46e6-b86d-0714682653cf admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:13.323673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e59885-adec-41a7-b524-f80614ec6f8f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:13.324163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659dc79b-71a5-46e6-b86d-0714682653cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:13.326582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659dc79b-71a5-46e6-b86d-0714682653cf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:13.328679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:13.329449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:13.330678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:13.330971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] PATH_INFO: `/v3/projects/8ffe1d0b8b60449895d98a4df2c0affa` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:13.333122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ffe1d0b8b60449895d98a4df2c0affa)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:13.336296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:13.365321 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1097/2182] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:13 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:13.367402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183585c9-4b5d-4df1-b27a-2e5bcf6a32f2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:13.367857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183585c9-4b5d-4df1-b27a-2e5bcf6a32f2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:13.368356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183585c9-4b5d-4df1-b27a-2e5bcf6a32f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:13.393624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-183585c9-4b5d-4df1-b27a-2e5bcf6a32f2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:13.395592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-183585c9-4b5d-4df1-b27a-2e5bcf6a32f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:13.395796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ffe1d0b8b60449895d98a4df2c0affa', 'request_id': 'req-04cf0885-00f5-4f5f-b899-db2caf8b76b4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.403812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ffe1d0b8b60449895d98a4df2c0affa', 'request_id': 'req-04cf0885-00f5-4f5f-b899-db2caf8b76b4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.414874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ffe1d0b8b60449895d98a4df2c0affa', 'request_id': 'req-04cf0885-00f5-4f5f-b899-db2caf8b76b4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.429549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] The token cache is being invalidate because project 8ffe1d0b8b60449895d98a4df2c0affa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:13.435881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.436854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.442330 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1086/2183] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:13 2022] DELETE /identity/v3/projects/8ffe1d0b8b60449895d98a4df2c0affa => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:13.443425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:13.445925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:13.478297 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1098/2184] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:13.479841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82337be5-936a-4c59-a3ef-730d8a871a4b None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:13.480212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82337be5-936a-4c59-a3ef-730d8a871a4b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:13.480607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82337be5-936a-4c59-a3ef-730d8a871a4b None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:13.482779 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1099/2185] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:56:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:56:13.484333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82337be5-936a-4c59-a3ef-730d8a871a4b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:13.486108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-82337be5-936a-4c59-a3ef-730d8a871a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:13.502680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82337be5-936a-4c59-a3ef-730d8a871a4b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:13.507484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6612b0c-4901-45ee-9b6d-3853fc3b910d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:13.508194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6612b0c-4901-45ee-9b6d-3853fc3b910d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:13.508591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6612b0c-4901-45ee-9b6d-3853fc3b910d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:13.508945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6612b0c-4901-45ee-9b6d-3853fc3b910d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:13.509946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6612b0c-4901-45ee-9b6d-3853fc3b910d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:13.511427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6612b0c-4901-45ee-9b6d-3853fc3b910d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:13.526345 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1100/2186] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:13.526345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6612b0c-4901-45ee-9b6d-3853fc3b910d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:13.528025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e6612b0c-4901-45ee-9b6d-3853fc3b910d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:13.542234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6612b0c-4901-45ee-9b6d-3853fc3b910d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:13.548290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:13.549216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:13.549582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:13.550028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] PATH_INFO: `/v3/users/eaba83770cd145eb9fe0687476a4845d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:13.551311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaba83770cd145eb9fe0687476a4845d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:13.554501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:13.573735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf0885-00f5-4f5f-b899-db2caf8b76b4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:13.579957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:13.580630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:13.581055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:13.581325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] PATH_INFO: `/v3/users/c3adc70e73614d48be23e3542424ee1d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:13.582956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3adc70e73614d48be23e3542424ee1d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:13.585585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:13.618056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaba83770cd145eb9fe0687476a4845d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.628340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaba83770cd145eb9fe0687476a4845d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.643099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaba83770cd145eb9fe0687476a4845d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.653797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3adc70e73614d48be23e3542424ee1d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.666720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaba83770cd145eb9fe0687476a4845d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.667365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3adc70e73614d48be23e3542424ee1d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.669652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaba83770cd145eb9fe0687476a4845d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.685237 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1101/2187] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:13 2022] DELETE /identity/v3/users/eaba83770cd145eb9fe0687476a4845d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:13.688074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:13.690349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3adc70e73614d48be23e3542424ee1d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.690935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:13.691532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3adc70e73614d48be23e3542424ee1d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.692239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3adc70e73614d48be23e3542424ee1d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:13.698984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:13.716161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:13.721997 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1087/2188] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:13 2022] DELETE /identity/v3/users/c3adc70e73614d48be23e3542424ee1d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:13.723109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:56:13.727398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:13.777371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530ff32-2b4d-4e5c-92eb-304a85133b3a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:13.861516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc2ff2-14f5-4bed-a2b8-6a86b737939f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:13.862360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc2ff2-14f5-4bed-a2b8-6a86b737939f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:13.862634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc2ff2-14f5-4bed-a2b8-6a86b737939f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:13.862901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc2ff2-14f5-4bed-a2b8-6a86b737939f admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:13.865219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc2ff2-14f5-4bed-a2b8-6a86b737939f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:13.865527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc2ff2-14f5-4bed-a2b8-6a86b737939f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:13.867862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc2ff2-14f5-4bed-a2b8-6a86b737939f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:13.884118 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1102/2189] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:13.885372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc2ff2-14f5-4bed-a2b8-6a86b737939f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:13.887336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8dcc2ff2-14f5-4bed-a2b8-6a86b737939f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:13.902789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc2ff2-14f5-4bed-a2b8-6a86b737939f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:13.908078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e5463-4c0c-479b-9570-6a5bd0256e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:13.909053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e5463-4c0c-479b-9570-6a5bd0256e06 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:13.909389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e5463-4c0c-479b-9570-6a5bd0256e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:13.909794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e5463-4c0c-479b-9570-6a5bd0256e06 admin admin] PATH_INFO: `/v3/projects/46a47a6119924e719ea973fb6e234ac0/users/444b9a04d9af48e7b57a30a168bcef6c/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:13.926376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:13.928193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:13.929112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e5463-4c0c-479b-9570-6a5bd0256e06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a47a6119924e719ea973fb6e234ac0, user_id=444b9a04d9af48e7b57a30a168bcef6c, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:13.932415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e5463-4c0c-479b-9570-6a5bd0256e06 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:13.975410 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1103/2190] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:13 2022] PUT /identity/v3/projects/46a47a6119924e719ea973fb6e234ac0/users/444b9a04d9af48e7b57a30a168bcef6c/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:13.978305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a832a14-e5d3-4d97-a15b-9206c5fb6cf8 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:13.978305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a832a14-e5d3-4d97-a15b-9206c5fb6cf8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:13.979333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a832a14-e5d3-4d97-a15b-9206c5fb6cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:14.021370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6a832a14-e5d3-4d97-a15b-9206c5fb6cf8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:14.022583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6a832a14-e5d3-4d97-a15b-9206c5fb6cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:14.114252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd441ed7-c158-4f37-9894-342696b28c75 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:14.178105 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1104/2191] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:14.179034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a832a14-e5d3-4d97-a15b-9206c5fb6cf8 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:14.182776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6a832a14-e5d3-4d97-a15b-9206c5fb6cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:14.210022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a832a14-e5d3-4d97-a15b-9206c5fb6cf8 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:14.221978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ddeef-8e70-4211-9533-5fb8b83fc72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:14.234689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ddeef-8e70-4211-9533-5fb8b83fc72d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:14.234992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ddeef-8e70-4211-9533-5fb8b83fc72d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:14.235236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ddeef-8e70-4211-9533-5fb8b83fc72d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:14.238342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ddeef-8e70-4211-9533-5fb8b83fc72d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:14.246171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ddeef-8e70-4211-9533-5fb8b83fc72d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:14.289953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f5f09-8bd4-4eea-9fae-55f5bbacdfe9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2893a64447f419c89ce450babc9fbe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9527f28a9846ff84e2a965962e7329', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:14.290540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f5f09-8bd4-4eea-9fae-55f5bbacdfe9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:14.290866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f5f09-8bd4-4eea-9fae-55f5bbacdfe9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:14.291280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f5f09-8bd4-4eea-9fae-55f5bbacdfe9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] PATH_INFO: `/v3/projects/4899a4244fb34985aa5163e0311cfe6d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:14.294999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f5f09-8bd4-4eea-9fae-55f5bbacdfe9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorizing `identity:get_project(project_id=4899a4244fb34985aa5163e0311cfe6d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:14.302253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f5f09-8bd4-4eea-9fae-55f5bbacdfe9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:14.307423 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1088/2192] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:56:13 2022] GET /identity/v3/projects/4899a4244fb34985aa5163e0311cfe6d => generated 369 bytes in 584 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:14.308255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f5f09-8bd4-4eea-9fae-55f5bbacdfe9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:14.322408 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1105/2193] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:14 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:14.323635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ddeef-8e70-4211-9533-5fb8b83fc72d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:14.328939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ba2ddeef-8e70-4211-9533-5fb8b83fc72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:14.334062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f5f09-8bd4-4eea-9fae-55f5bbacdfe9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:14.360867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:14.439016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:14.444689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ddeef-8e70-4211-9533-5fb8b83fc72d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:14.456294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f14730-fc0e-4cdb-8121-57a54be833ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:14.457722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f14730-fc0e-4cdb-8121-57a54be833ed service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:14.458271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f14730-fc0e-4cdb-8121-57a54be833ed service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:14.461234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f14730-fc0e-4cdb-8121-57a54be833ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:14.494363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642fc76-cc30-4e3a-b85d-206ce9a2b544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:14.495690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642fc76-cc30-4e3a-b85d-206ce9a2b544 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:14.496356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642fc76-cc30-4e3a-b85d-206ce9a2b544 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:14.498083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642fc76-cc30-4e3a-b85d-206ce9a2b544 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:14.500305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642fc76-cc30-4e3a-b85d-206ce9a2b544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:14.505807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642fc76-cc30-4e3a-b85d-206ce9a2b544 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:14.536067 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1089/2194] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:14.537595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642fc76-cc30-4e3a-b85d-206ce9a2b544 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:14.544842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e642fc76-cc30-4e3a-b85d-206ce9a2b544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:14.598924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642fc76-cc30-4e3a-b85d-206ce9a2b544 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:14.604286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:14.604962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:14.605262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:14.605502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] PATH_INFO: `/v3/projects/a66ff272ec8b489987e463cd52b2d6ac` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:14.609073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a66ff272ec8b489987e463cd52b2d6ac)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:14.611089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:14.639972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f14730-fc0e-4cdb-8121-57a54be833ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:14.641782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f14730-fc0e-4cdb-8121-57a54be833ed service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:14.671878 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1106/2195] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:14.673261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f14730-fc0e-4cdb-8121-57a54be833ed service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:14.703185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f14730-fc0e-4cdb-8121-57a54be833ed service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:14.715636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a3ccb-0684-43bd-93b5-a80534fcd8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:14.716473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a3ccb-0684-43bd-93b5-a80534fcd8e4 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:14.716749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a3ccb-0684-43bd-93b5-a80534fcd8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:14.717012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a3ccb-0684-43bd-93b5-a80534fcd8e4 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:14.717868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a3ccb-0684-43bd-93b5-a80534fcd8e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:14.724051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a3ccb-0684-43bd-93b5-a80534fcd8e4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:14.749576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a66ff272ec8b489987e463cd52b2d6ac', 'request_id': 'req-b1606910-3ddf-4415-ac2f-aad4e3b99475'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:14.769737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a66ff272ec8b489987e463cd52b2d6ac', 'request_id': 'req-b1606910-3ddf-4415-ac2f-aad4e3b99475'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:14.795432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a66ff272ec8b489987e463cd52b2d6ac', 'request_id': 'req-b1606910-3ddf-4415-ac2f-aad4e3b99475'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:14.803730 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1107/2196] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:14 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:14.804829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a3ccb-0684-43bd-93b5-a80534fcd8e4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:14.827240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] The token cache is being invalidate because project a66ff272ec8b489987e463cd52b2d6ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:14.828119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:14.829286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:14.832191 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1090/2197] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:14 2022] DELETE /identity/v3/projects/a66ff272ec8b489987e463cd52b2d6ac => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:14.833242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:14.846155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:14.938813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a3ccb-0684-43bd-93b5-a80534fcd8e4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:14.960181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1606910-3ddf-4415-ac2f-aad4e3b99475 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:14.969861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f61a9-17c8-430d-a3bd-5aa93519bbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:14.970515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f61a9-17c8-430d-a3bd-5aa93519bbba admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:14.970784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f61a9-17c8-430d-a3bd-5aa93519bbba admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:14.971044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f61a9-17c8-430d-a3bd-5aa93519bbba admin admin] PATH_INFO: `/v3/users/b3c65f1a7588466f8fdb1a614f98a69d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:14.976665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f61a9-17c8-430d-a3bd-5aa93519bbba admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c65f1a7588466f8fdb1a614f98a69d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:14.980808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f61a9-17c8-430d-a3bd-5aa93519bbba admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:15.036988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:15.063704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a24620-d255-4f6a-abdd-c257aa115319 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2893a64447f419c89ce450babc9fbe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9527f28a9846ff84e2a965962e7329', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:15.064888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a24620-d255-4f6a-abdd-c257aa115319 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:15.065592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a24620-d255-4f6a-abdd-c257aa115319 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:15.066272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a24620-d255-4f6a-abdd-c257aa115319 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:15.067031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a24620-d255-4f6a-abdd-c257aa115319 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:15.067479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a24620-d255-4f6a-abdd-c257aa115319 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:15.069380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a24620-d255-4f6a-abdd-c257aa115319 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:15.083746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a13f61a9-17c8-430d-a3bd-5aa93519bbba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c65f1a7588466f8fdb1a614f98a69d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.086801 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1108/2198] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:15.087805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a24620-d255-4f6a-abdd-c257aa115319 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:15.089732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-83a24620-d255-4f6a-abdd-c257aa115319 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:15.093936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a13f61a9-17c8-430d-a3bd-5aa93519bbba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c65f1a7588466f8fdb1a614f98a69d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.101350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:15.105428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a24620-d255-4f6a-abdd-c257aa115319 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:15.111828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e529f61-3fa5-49da-b830-4857d205feb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:15.112734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e529f61-3fa5-49da-b830-4857d205feb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:15.113297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e529f61-3fa5-49da-b830-4857d205feb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:15.113755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e529f61-3fa5-49da-b830-4857d205feb9 admin admin] PATH_INFO: `/v3/projects/46a47a6119924e719ea973fb6e234ac0/users/444b9a04d9af48e7b57a30a168bcef6c/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:15.115948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e529f61-3fa5-49da-b830-4857d205feb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a47a6119924e719ea973fb6e234ac0, user_id=444b9a04d9af48e7b57a30a168bcef6c, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:15.118397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a13f61a9-17c8-430d-a3bd-5aa93519bbba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c65f1a7588466f8fdb1a614f98a69d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.119393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e529f61-3fa5-49da-b830-4857d205feb9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:15.119667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a13f61a9-17c8-430d-a3bd-5aa93519bbba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c65f1a7588466f8fdb1a614f98a69d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.120579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a13f61a9-17c8-430d-a3bd-5aa93519bbba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c65f1a7588466f8fdb1a614f98a69d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.137023 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1091/2200] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:14 2022] DELETE /identity/v3/users/b3c65f1a7588466f8fdb1a614f98a69d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:15.139094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2c145-e0f2-4b20-af9f-2ed08f3e344c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:15.139573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2c145-e0f2-4b20-af9f-2ed08f3e344c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:15.140171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2c145-e0f2-4b20-af9f-2ed08f3e344c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:15.142489 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1109/2200] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:15 2022] PUT /identity/v3/projects/46a47a6119924e719ea973fb6e234ac0/users/444b9a04d9af48e7b57a30a168bcef6c/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:15.143679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e529f61-3fa5-49da-b830-4857d205feb9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:15.146063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6e529f61-3fa5-49da-b830-4857d205feb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:15.182533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a9a2c145-e0f2-4b20-af9f-2ed08f3e344c None None] MFA Rules not processed for user `a3378bf4fa90489cb103378667f00399`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:15.187032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a9a2c145-e0f2-4b20-af9f-2ed08f3e344c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:15.257849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e529f61-3fa5-49da-b830-4857d205feb9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:15.264520 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1092/2201] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:15 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:15.266441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535e365-e790-44ce-bdcf-3508609a1655 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:15.266777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535e365-e790-44ce-bdcf-3508609a1655 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:15.267402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535e365-e790-44ce-bdcf-3508609a1655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:15.293289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b535e365-e790-44ce-bdcf-3508609a1655 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:15.294684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b535e365-e790-44ce-bdcf-3508609a1655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:15.332324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefb6cc-b912-4902-bf3f-c7897a089b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:15.333193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefb6cc-b912-4902-bf3f-c7897a089b3c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:15.333540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefb6cc-b912-4902-bf3f-c7897a089b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:15.333809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefb6cc-b912-4902-bf3f-c7897a089b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:15.334467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefb6cc-b912-4902-bf3f-c7897a089b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:15.336357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefb6cc-b912-4902-bf3f-c7897a089b3c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:15.352145 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1110/2202] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:15.352145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefb6cc-b912-4902-bf3f-c7897a089b3c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:15.354254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-deefb6cc-b912-4902-bf3f-c7897a089b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:15.384496 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1093/2203] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:15.391375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefb6cc-b912-4902-bf3f-c7897a089b3c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:15.393768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535e365-e790-44ce-bdcf-3508609a1655 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:15.395335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b535e365-e790-44ce-bdcf-3508609a1655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:15.409682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:15.410527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:15.410922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:15.411262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba admin admin] PATH_INFO: `/v3/projects/1e753a6065044fcd81165523768fe2f4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:15.413868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e753a6065044fcd81165523768fe2f4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:15.416474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:15.520747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e753a6065044fcd81165523768fe2f4', 'request_id': 'req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.533571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535e365-e790-44ce-bdcf-3508609a1655 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:15.535593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e753a6065044fcd81165523768fe2f4', 'request_id': 'req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.544536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e753a6065044fcd81165523768fe2f4', 'request_id': 'req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.546834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:15.558861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba admin admin] The token cache is being invalidate because project 1e753a6065044fcd81165523768fe2f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:15.559277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.560050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-eefa3dbd-2b8f-4088-aa24-957d09eb54ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.561999 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1111/2204] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:15 2022] DELETE /identity/v3/projects/1e753a6065044fcd81165523768fe2f4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:15.563585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf096a4d-125f-4c92-ba0f-8a770f3aeda4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:15.563997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf096a4d-125f-4c92-ba0f-8a770f3aeda4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:15.564875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf096a4d-125f-4c92-ba0f-8a770f3aeda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:15.588585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cf096a4d-125f-4c92-ba0f-8a770f3aeda4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:15.590344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cf096a4d-125f-4c92-ba0f-8a770f3aeda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:15.619408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:15.672791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568aa419-a4d2-48a8-9603-af906956c8df tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2893a64447f419c89ce450babc9fbe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9527f28a9846ff84e2a965962e7329', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:15.673653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568aa419-a4d2-48a8-9603-af906956c8df tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:15.673973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568aa419-a4d2-48a8-9603-af906956c8df tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:15.674251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568aa419-a4d2-48a8-9603-af906956c8df tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:15.674910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568aa419-a4d2-48a8-9603-af906956c8df tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:15.677551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568aa419-a4d2-48a8-9603-af906956c8df tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:15.723489 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1094/2205] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:15 2022] POST /identity/v3/users => generated 383 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:15.733679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88b947-7bcd-4452-92e9-5520465196ba None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:15.733966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88b947-7bcd-4452-92e9-5520465196ba None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:15.734242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88b947-7bcd-4452-92e9-5520465196ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:15.742298 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1112/2206] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:15.742298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf096a4d-125f-4c92-ba0f-8a770f3aeda4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:15.742298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cf096a4d-125f-4c92-ba0f-8a770f3aeda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:15.757019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf096a4d-125f-4c92-ba0f-8a770f3aeda4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:15.762725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabe989-54f3-413e-bf86-d44a6b0b1417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:15.763495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabe989-54f3-413e-bf86-d44a6b0b1417 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:15.763863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabe989-54f3-413e-bf86-d44a6b0b1417 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:15.764224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabe989-54f3-413e-bf86-d44a6b0b1417 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:15.765014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabe989-54f3-413e-bf86-d44a6b0b1417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:15.766806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabe989-54f3-413e-bf86-d44a6b0b1417 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:15.780577 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1113/2207] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:15.782492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d802c8b-e50e-47fa-89f0-6f5ce839a85e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:15.782933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d802c8b-e50e-47fa-89f0-6f5ce839a85e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:15.783580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d802c8b-e50e-47fa-89f0-6f5ce839a85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:15.807213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4d802c8b-e50e-47fa-89f0-6f5ce839a85e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:15.808807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4d802c8b-e50e-47fa-89f0-6f5ce839a85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:15.818462 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1114/2208] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:15.819599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d802c8b-e50e-47fa-89f0-6f5ce839a85e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:15.823189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4d802c8b-e50e-47fa-89f0-6f5ce839a85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:15.831731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7a88b947-7bcd-4452-92e9-5520465196ba None None] MFA Rules not processed for user `444b9a04d9af48e7b57a30a168bcef6c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:15.836161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7a88b947-7bcd-4452-92e9-5520465196ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:15.838590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d802c8b-e50e-47fa-89f0-6f5ce839a85e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:15.843641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:15.844470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:15.844891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:15.845200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] PATH_INFO: `/v3/users/bb0837c4526042d59310159de6b9383f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:15.846459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb0837c4526042d59310159de6b9383f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:15.848667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:15.943972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb0837c4526042d59310159de6b9383f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.958884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb0837c4526042d59310159de6b9383f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.971525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb0837c4526042d59310159de6b9383f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.988199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb0837c4526042d59310159de6b9383f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:15.989094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb0837c4526042d59310159de6b9383f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:16.000094 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1115/2209] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:15 2022] DELETE /identity/v3/users/bb0837c4526042d59310159de6b9383f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:16.001563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:16.003303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:16.020159 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1095/2210] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:15 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:16.022434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88b947-7bcd-4452-92e9-5520465196ba None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:16.035591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7a88b947-7bcd-4452-92e9-5520465196ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:16.068903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:16.089483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bbc72-35d2-4da8-8fe6-17af34fa8c39 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:16.095409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8642df4-895b-4a40-a5a6-8616eba70462 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2893a64447f419c89ce450babc9fbe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9527f28a9846ff84e2a965962e7329', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:16.097438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8642df4-895b-4a40-a5a6-8616eba70462 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:16.097438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8642df4-895b-4a40-a5a6-8616eba70462 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:16.097438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8642df4-895b-4a40-a5a6-8616eba70462 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:16.098213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8642df4-895b-4a40-a5a6-8616eba70462 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:16.098656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8642df4-895b-4a40-a5a6-8616eba70462 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:16.101843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8642df4-895b-4a40-a5a6-8616eba70462 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:16.133777 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1116/2211] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:16.136734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8642df4-895b-4a40-a5a6-8616eba70462 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:16.138976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b8642df4-895b-4a40-a5a6-8616eba70462 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:16.146089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88b947-7bcd-4452-92e9-5520465196ba None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:16.207425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666441b9-92f1-4f46-8bf9-9ba4e0ffece2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:16.208034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666441b9-92f1-4f46-8bf9-9ba4e0ffece2 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:16.208330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666441b9-92f1-4f46-8bf9-9ba4e0ffece2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:16.208579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666441b9-92f1-4f46-8bf9-9ba4e0ffece2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:16.210688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-666441b9-92f1-4f46-8bf9-9ba4e0ffece2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:16.226169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:16.270060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8642df4-895b-4a40-a5a6-8616eba70462 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:16.277842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e146b5-0a8f-4955-b557-4a2e0d5d1c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:16.278540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e146b5-0a8f-4955-b557-4a2e0d5d1c52 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:16.278811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e146b5-0a8f-4955-b557-4a2e0d5d1c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:16.279378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e146b5-0a8f-4955-b557-4a2e0d5d1c52 admin admin] PATH_INFO: `/v3/projects/7b964ec386f64872a3dc6ec6432e360c/users/6e8cfaf24b134d65994aaca75c8a2e27/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:16.297590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666441b9-92f1-4f46-8bf9-9ba4e0ffece2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:16.298268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e146b5-0a8f-4955-b557-4a2e0d5d1c52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b964ec386f64872a3dc6ec6432e360c, user_id=6e8cfaf24b134d65994aaca75c8a2e27, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:16.299553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666441b9-92f1-4f46-8bf9-9ba4e0ffece2 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:16.301265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e146b5-0a8f-4955-b557-4a2e0d5d1c52 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:16.313702 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1096/2212] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:16.315396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666441b9-92f1-4f46-8bf9-9ba4e0ffece2 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:16.321131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-666441b9-92f1-4f46-8bf9-9ba4e0ffece2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:16.328706 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1117/2213] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:16 2022] PUT /identity/v3/projects/7b964ec386f64872a3dc6ec6432e360c/users/6e8cfaf24b134d65994aaca75c8a2e27/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:16.330158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e146b5-0a8f-4955-b557-4a2e0d5d1c52 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:16.333097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-67e146b5-0a8f-4955-b557-4a2e0d5d1c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:16.361588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666441b9-92f1-4f46-8bf9-9ba4e0ffece2 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:16.491408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e146b5-0a8f-4955-b557-4a2e0d5d1c52 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:16.499999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91cfb6-de65-44f2-b48f-30192c5173f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:16.500651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91cfb6-de65-44f2-b48f-30192c5173f8 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:16.500935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91cfb6-de65-44f2-b48f-30192c5173f8 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:16.501625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91cfb6-de65-44f2-b48f-30192c5173f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:16.503983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2d91cfb6-de65-44f2-b48f-30192c5173f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:16.519023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271cdd6-7000-49b9-b6cb-29f66c69f14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:16.519669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271cdd6-7000-49b9-b6cb-29f66c69f14f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:16.519905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271cdd6-7000-49b9-b6cb-29f66c69f14f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:16.520172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271cdd6-7000-49b9-b6cb-29f66c69f14f admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:16.520935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271cdd6-7000-49b9-b6cb-29f66c69f14f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:16.521233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271cdd6-7000-49b9-b6cb-29f66c69f14f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:16.522861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271cdd6-7000-49b9-b6cb-29f66c69f14f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:16.557920 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1097/2214] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:16.557920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d922a-eab0-4045-82b4-42a99a08ecc8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:16.557920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d922a-eab0-4045-82b4-42a99a08ecc8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:16.557920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d922a-eab0-4045-82b4-42a99a08ecc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:16.611993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e52d922a-eab0-4045-82b4-42a99a08ecc8 None None] MFA Rules not processed for user `6e8cfaf24b134d65994aaca75c8a2e27`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:16.613283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e52d922a-eab0-4045-82b4-42a99a08ecc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:16.619072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91cfb6-de65-44f2-b48f-30192c5173f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:16.622548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91cfb6-de65-44f2-b48f-30192c5173f8 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:16.637595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:16.653107 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1118/2215] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:16 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:16.657393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aad073-2df3-4faf-b0d9-ac887a155301 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:16.659871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aad073-2df3-4faf-b0d9-ac887a155301 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:16.659871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aad073-2df3-4faf-b0d9-ac887a155301 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:16.668266 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1119/2216] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:56:16 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:56:16.675500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aad073-2df3-4faf-b0d9-ac887a155301 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:16.709415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aad073-2df3-4faf-b0d9-ac887a155301 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:16.715184 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1098/2217] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:16.716128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d922a-eab0-4045-82b4-42a99a08ecc8 None None] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:56:16.718094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e52d922a-eab0-4045-82b4-42a99a08ecc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:16.726737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ec9f2-d36e-447f-a5c0-bc616727bb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:16.728101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ec9f2-d36e-447f-a5c0-bc616727bb4f admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:16.728101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ec9f2-d36e-447f-a5c0-bc616727bb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:16.728101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ec9f2-d36e-447f-a5c0-bc616727bb4f admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:16.728631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ec9f2-d36e-447f-a5c0-bc616727bb4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:16.730559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ec9f2-d36e-447f-a5c0-bc616727bb4f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:16.740246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d922a-eab0-4045-82b4-42a99a08ecc8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:16.801677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:16.814184 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1120/2218] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:16 2022] POST /identity/v3/projects => generated 389 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:16.815674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ec9f2-d36e-447f-a5c0-bc616727bb4f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:16.821797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-430ec9f2-d36e-447f-a5c0-bc616727bb4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:16.850158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d922a-eab0-4045-82b4-42a99a08ecc8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:16.953534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ec9f2-d36e-447f-a5c0-bc616727bb4f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:16.965211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:16.965881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:16.966159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:16.968638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] PATH_INFO: `/v3/projects/ffc82a7c80da462980f6a6d9ca5f8303` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:16.970191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc82a7c80da462980f6a6d9ca5f8303)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:16.976737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:17.017743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd894cb-9150-4966-9fbb-bd3d612deae9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2893a64447f419c89ce450babc9fbe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9527f28a9846ff84e2a965962e7329', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:17.019036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd894cb-9150-4966-9fbb-bd3d612deae9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.019518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd894cb-9150-4966-9fbb-bd3d612deae9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.020108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd894cb-9150-4966-9fbb-bd3d612deae9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] PATH_INFO: `/v3/projects/4899a4244fb34985aa5163e0311cfe6d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.024061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd894cb-9150-4966-9fbb-bd3d612deae9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorizing `identity:get_project(project_id=4899a4244fb34985aa5163e0311cfe6d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:17.028408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd894cb-9150-4966-9fbb-bd3d612deae9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:17.030975 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1099/2219] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:56:16 2022] GET /identity/v3/projects/4899a4244fb34985aa5163e0311cfe6d => generated 369 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:17.032733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd894cb-9150-4966-9fbb-bd3d612deae9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:17.063382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd894cb-9150-4966-9fbb-bd3d612deae9 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:17.077176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20699f19-56f6-4807-8445-03c6f2b2a91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:17.078266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20699f19-56f6-4807-8445-03c6f2b2a91d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.078706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20699f19-56f6-4807-8445-03c6f2b2a91d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.078891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20699f19-56f6-4807-8445-03c6f2b2a91d admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.081476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20699f19-56f6-4807-8445-03c6f2b2a91d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:17.088021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20699f19-56f6-4807-8445-03c6f2b2a91d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:17.136714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc82a7c80da462980f6a6d9ca5f8303', 'request_id': 'req-4b8aecf6-656f-46c7-85bc-3bb642657407'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:17.152906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc82a7c80da462980f6a6d9ca5f8303', 'request_id': 'req-4b8aecf6-656f-46c7-85bc-3bb642657407'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:17.155589 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1100/2220] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:17 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:17.165956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc82a7c80da462980f6a6d9ca5f8303', 'request_id': 'req-4b8aecf6-656f-46c7-85bc-3bb642657407'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:17.166686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20699f19-56f6-4807-8445-03c6f2b2a91d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:17.208744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] The token cache is being invalidate because project ffc82a7c80da462980f6a6d9ca5f8303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:17.209050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:17.210484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:17.214837 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1121/2221] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:16 2022] DELETE /identity/v3/projects/ffc82a7c80da462980f6a6d9ca5f8303 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:17.215659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:17.218972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:17.355355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20699f19-56f6-4807-8445-03c6f2b2a91d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:17.362055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826d116-0eec-4df4-b9e7-cb8b07310664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:17.362825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826d116-0eec-4df4-b9e7-cb8b07310664 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.363209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826d116-0eec-4df4-b9e7-cb8b07310664 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.363483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826d116-0eec-4df4-b9e7-cb8b07310664 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.364193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826d116-0eec-4df4-b9e7-cb8b07310664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:17.366585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826d116-0eec-4df4-b9e7-cb8b07310664 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:17.405727 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1101/2222] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:17 2022] POST /identity/v3/users => generated 295 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:17.409779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd78ebd-9e2d-4e0d-8ff3-c4c27cd790e9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.411148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd78ebd-9e2d-4e0d-8ff3-c4c27cd790e9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.411632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd78ebd-9e2d-4e0d-8ff3-c4c27cd790e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.438573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ebd78ebd-9e2d-4e0d-8ff3-c4c27cd790e9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:17.439947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ebd78ebd-9e2d-4e0d-8ff3-c4c27cd790e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:17.450854 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1102/2223] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:17.451194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8aecf6-656f-46c7-85bc-3bb642657407 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:17.451636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd78ebd-9e2d-4e0d-8ff3-c4c27cd790e9 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:17.453065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ebd78ebd-9e2d-4e0d-8ff3-c4c27cd790e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:17.467767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83205a8a-d71b-4131-aacf-0a8a3f9867be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:17.468501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83205a8a-d71b-4131-aacf-0a8a3f9867be admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.468863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83205a8a-d71b-4131-aacf-0a8a3f9867be admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.469892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83205a8a-d71b-4131-aacf-0a8a3f9867be admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.470761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83205a8a-d71b-4131-aacf-0a8a3f9867be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:17.476824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83205a8a-d71b-4131-aacf-0a8a3f9867be admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:17.518491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:17.531587 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1122/2224] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:17 2022] POST /identity/v3/users => generated 294 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:17.531587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaed59-052a-4677-9cc0-9ac5644e09b6 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.531587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaed59-052a-4677-9cc0-9ac5644e09b6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.531587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaed59-052a-4677-9cc0-9ac5644e09b6 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.540582 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1123/2225] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:56:17 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:56:17.542375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:17.543105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaed59-052a-4677-9cc0-9ac5644e09b6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:17.556629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dddaed59-052a-4677-9cc0-9ac5644e09b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:17.579764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaed59-052a-4677-9cc0-9ac5644e09b6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:17.590475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbf8fe-cb60-4e5d-83bc-40c068888b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:17.591154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbf8fe-cb60-4e5d-83bc-40c068888b9a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.592010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbf8fe-cb60-4e5d-83bc-40c068888b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.592297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbf8fe-cb60-4e5d-83bc-40c068888b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.592985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bbf8fe-cb60-4e5d-83bc-40c068888b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:17.602338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bbf8fe-cb60-4e5d-83bc-40c068888b9a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:17.610979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd78ebd-9e2d-4e0d-8ff3-c4c27cd790e9 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:17.617858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db4519-622f-438b-9be8-11567354b7a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:17.618798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db4519-622f-438b-9be8-11567354b7a2 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.619224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db4519-622f-438b-9be8-11567354b7a2 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.619786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db4519-622f-438b-9be8-11567354b7a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.621801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f6db4519-622f-438b-9be8-11567354b7a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:17.653037 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1124/2226] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:17.655128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11733ed1-62f2-43ed-855c-1410197b827c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.655633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11733ed1-62f2-43ed-855c-1410197b827c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.656239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11733ed1-62f2-43ed-855c-1410197b827c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.693457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-11733ed1-62f2-43ed-855c-1410197b827c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:17.697091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-11733ed1-62f2-43ed-855c-1410197b827c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:17.713662 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1125/2227] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:17.719637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11733ed1-62f2-43ed-855c-1410197b827c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:17.723835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db4519-622f-438b-9be8-11567354b7a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:17.725563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db4519-622f-438b-9be8-11567354b7a2 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:17.748158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11733ed1-62f2-43ed-855c-1410197b827c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:17.749730 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1103/2228] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:17 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:17.750741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db4519-622f-438b-9be8-11567354b7a2 service nova] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:56:17.754521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f6db4519-622f-438b-9be8-11567354b7a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:17.756305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d7d5a-3443-4d17-9c80-f31a14720b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:17.756972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d7d5a-3443-4d17-9c80-f31a14720b33 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.757269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d7d5a-3443-4d17-9c80-f31a14720b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.757513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d7d5a-3443-4d17-9c80-f31a14720b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.760850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d7d5a-3443-4d17-9c80-f31a14720b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:17.761773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d7d5a-3443-4d17-9c80-f31a14720b33 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:17.794312 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1126/2229] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:17.795070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d7d5a-3443-4d17-9c80-f31a14720b33 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:17.810743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db4519-622f-438b-9be8-11567354b7a2 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:17.814323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f6db4519-622f-438b-9be8-11567354b7a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:17.821457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d7d5a-3443-4d17-9c80-f31a14720b33 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:17.825899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84273275-7df2-40b8-9dec-df367bdd68a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:17.826523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84273275-7df2-40b8-9dec-df367bdd68a8 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.826798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84273275-7df2-40b8-9dec-df367bdd68a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.827069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84273275-7df2-40b8-9dec-df367bdd68a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.827715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84273275-7df2-40b8-9dec-df367bdd68a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:17.829824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84273275-7df2-40b8-9dec-df367bdd68a8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:17.842586 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1127/2230] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:17.843612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84273275-7df2-40b8-9dec-df367bdd68a8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:17.858842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84273275-7df2-40b8-9dec-df367bdd68a8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:17.865573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc69ae-b472-4fba-824c-18801be197bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:17.867114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc69ae-b472-4fba-824c-18801be197bf admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.867404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc69ae-b472-4fba-824c-18801be197bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.867677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc69ae-b472-4fba-824c-18801be197bf admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.868336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc69ae-b472-4fba-824c-18801be197bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:17.868820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc69ae-b472-4fba-824c-18801be197bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:17.870826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc69ae-b472-4fba-824c-18801be197bf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:17.896463 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1128/2231] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:17.897849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc69ae-b472-4fba-824c-18801be197bf admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:17.918794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc69ae-b472-4fba-824c-18801be197bf admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:17.925817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17f71c-d3dd-4bcd-8340-1af64e4f316d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:17.926508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17f71c-d3dd-4bcd-8340-1af64e4f316d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.926797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17f71c-d3dd-4bcd-8340-1af64e4f316d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.927081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17f71c-d3dd-4bcd-8340-1af64e4f316d admin admin] PATH_INFO: `/v3/projects/441d89fde86f4e43ad40d4b9d62d7406/users/adb9e7f031ae467e8ba70a64fa3cf51b/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.935488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db4519-622f-438b-9be8-11567354b7a2 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:17.943653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915daad5-7885-4418-9968-e0d27c59d57f tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2893a64447f419c89ce450babc9fbe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9527f28a9846ff84e2a965962e7329', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:17.946139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915daad5-7885-4418-9968-e0d27c59d57f tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:17.946584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915daad5-7885-4418-9968-e0d27c59d57f tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:17.947257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915daad5-7885-4418-9968-e0d27c59d57f tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] PATH_INFO: `/v3/projects/4899a4244fb34985aa5163e0311cfe6d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:17.949008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915daad5-7885-4418-9968-e0d27c59d57f tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorizing `identity:get_project(project_id=4899a4244fb34985aa5163e0311cfe6d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:17.951738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915daad5-7885-4418-9968-e0d27c59d57f tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:17.956116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17f71c-d3dd-4bcd-8340-1af64e4f316d admin admin] RBAC: Authorizing `identity:create_grant(project_id=441d89fde86f4e43ad40d4b9d62d7406, user_id=adb9e7f031ae467e8ba70a64fa3cf51b, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:17.965688 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1104/2232] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:56:17 2022] GET /identity/v3/projects/4899a4244fb34985aa5163e0311cfe6d => generated 369 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:17.967311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915daad5-7885-4418-9968-e0d27c59d57f tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:17.975619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-915daad5-7885-4418-9968-e0d27c59d57f tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:17.977041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17f71c-d3dd-4bcd-8340-1af64e4f316d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:18.009385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915daad5-7885-4418-9968-e0d27c59d57f tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:18.017187 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1129/2233] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:17 2022] PUT /identity/v3/projects/441d89fde86f4e43ad40d4b9d62d7406/users/adb9e7f031ae467e8ba70a64fa3cf51b/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:18.020326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17f71c-d3dd-4bcd-8340-1af64e4f316d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:18.082048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:18.082392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:18.082685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:18.082930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] PATH_INFO: `/v3/users/5ca1ee61eaf245089afd5a7a033608be` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:18.084780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca1ee61eaf245089afd5a7a033608be)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:18.089113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:18.132147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:18.190993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17f71c-d3dd-4bcd-8340-1af64e4f316d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:18.201689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca1ee61eaf245089afd5a7a033608be'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.205492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588f064-8ea4-40aa-a81a-3d5ad6cc62a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:18.206478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588f064-8ea4-40aa-a81a-3d5ad6cc62a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:18.206751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588f064-8ea4-40aa-a81a-3d5ad6cc62a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:18.207064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588f064-8ea4-40aa-a81a-3d5ad6cc62a5 admin admin] PATH_INFO: `/v3/projects/a89c81af4c64435cafa250b1d93e2c6c/users/5bbd3d48bdce4bfa8af26db3d0dffbbd/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:18.228565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588f064-8ea4-40aa-a81a-3d5ad6cc62a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89c81af4c64435cafa250b1d93e2c6c, user_id=5bbd3d48bdce4bfa8af26db3d0dffbbd, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:18.229204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca1ee61eaf245089afd5a7a033608be'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.239186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588f064-8ea4-40aa-a81a-3d5ad6cc62a5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:18.257558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca1ee61eaf245089afd5a7a033608be'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.258355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca1ee61eaf245089afd5a7a033608be'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.259056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca1ee61eaf245089afd5a7a033608be'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.268634 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1130/2234] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:18 2022] PUT /identity/v3/projects/a89c81af4c64435cafa250b1d93e2c6c/users/5bbd3d48bdce4bfa8af26db3d0dffbbd/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:18.269598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588f064-8ea4-40aa-a81a-3d5ad6cc62a5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:18.279296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0588f064-8ea4-40aa-a81a-3d5ad6cc62a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:18.289405 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1105/2235] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:17 2022] DELETE /identity/v3/users/5ca1ee61eaf245089afd5a7a033608be => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:18.290268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:18.291808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:18.387024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588f064-8ea4-40aa-a81a-3d5ad6cc62a5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:18.409372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:18.446052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43231b20-4afe-42ec-84c3-45e16cd157de admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:18.460980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ed191-a01f-4fca-82d9-993d05cb658c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:18.461921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ed191-a01f-4fca-82d9-993d05cb658c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:18.462430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ed191-a01f-4fca-82d9-993d05cb658c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:18.465673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ed191-a01f-4fca-82d9-993d05cb658c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:18.467055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ed191-a01f-4fca-82d9-993d05cb658c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:18.472029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ed191-a01f-4fca-82d9-993d05cb658c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:18.497868 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1106/2236] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:18.500518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ed191-a01f-4fca-82d9-993d05cb658c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:18.515735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ef5ed191-a01f-4fca-82d9-993d05cb658c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:18.517614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a3ca4-4dcd-4c09-845e-8ef6253683b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:18.518237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a3ca4-4dcd-4c09-845e-8ef6253683b4 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:18.518506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a3ca4-4dcd-4c09-845e-8ef6253683b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:18.518766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a3ca4-4dcd-4c09-845e-8ef6253683b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:18.519651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a3ca4-4dcd-4c09-845e-8ef6253683b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:18.522494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a3ca4-4dcd-4c09-845e-8ef6253683b4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:18.568434 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1131/2237] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:18 2022] POST /identity/v3/projects => generated 391 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:18.568434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a3ca4-4dcd-4c09-845e-8ef6253683b4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:18.571456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-756a3ca4-4dcd-4c09-845e-8ef6253683b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:18.634803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ed191-a01f-4fca-82d9-993d05cb658c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:18.680824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:18.701663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a3ca4-4dcd-4c09-845e-8ef6253683b4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:18.707530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2df41-ca48-48a7-aeeb-780fa537c351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:18.708356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2df41-ca48-48a7-aeeb-780fa537c351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:18.708784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2df41-ca48-48a7-aeeb-780fa537c351 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:18.709292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2df41-ca48-48a7-aeeb-780fa537c351 admin admin] PATH_INFO: `/v3/projects/a8dc72ffe83845899ffcf918c3f79302` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:18.710787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2df41-ca48-48a7-aeeb-780fa537c351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8dc72ffe83845899ffcf918c3f79302)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:18.713092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2df41-ca48-48a7-aeeb-780fa537c351 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:18.719363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc74b2-6fe0-4276-96f3-f0bb937c6815 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2893a64447f419c89ce450babc9fbe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9527f28a9846ff84e2a965962e7329', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:18.720087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc74b2-6fe0-4276-96f3-f0bb937c6815 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:18.720664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc74b2-6fe0-4276-96f3-f0bb937c6815 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:18.720934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc74b2-6fe0-4276-96f3-f0bb937c6815 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] PATH_INFO: `/v3/users/2f56f0efaec04576bd5316c90ebc6d32` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:18.748630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc74b2-6fe0-4276-96f3-f0bb937c6815 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorizing `identity:delete_user(user_id=2f56f0efaec04576bd5316c90ebc6d32)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:18.750747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc74b2-6fe0-4276-96f3-f0bb937c6815 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:18.790377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0bf2df41-ca48-48a7-aeeb-780fa537c351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8dc72ffe83845899ffcf918c3f79302', 'request_id': 'req-0bf2df41-ca48-48a7-aeeb-780fa537c351'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.801223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0bf2df41-ca48-48a7-aeeb-780fa537c351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8dc72ffe83845899ffcf918c3f79302', 'request_id': 'req-0bf2df41-ca48-48a7-aeeb-780fa537c351'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.834622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cdfc74b2-6fe0-4276-96f3-f0bb937c6815 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f56f0efaec04576bd5316c90ebc6d32'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.841921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0bf2df41-ca48-48a7-aeeb-780fa537c351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8dc72ffe83845899ffcf918c3f79302', 'request_id': 'req-0bf2df41-ca48-48a7-aeeb-780fa537c351'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.851153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cdfc74b2-6fe0-4276-96f3-f0bb937c6815 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f56f0efaec04576bd5316c90ebc6d32'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.869926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0bf2df41-ca48-48a7-aeeb-780fa537c351 admin admin] The token cache is being invalidate because project a8dc72ffe83845899ffcf918c3f79302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:18.870951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0bf2df41-ca48-48a7-aeeb-780fa537c351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.873151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0bf2df41-ca48-48a7-aeeb-780fa537c351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.879829 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1132/2238] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:18 2022] DELETE /identity/v3/projects/a8dc72ffe83845899ffcf918c3f79302 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:18.880272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cdfc74b2-6fe0-4276-96f3-f0bb937c6815 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f56f0efaec04576bd5316c90ebc6d32'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.880272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cdfc74b2-6fe0-4276-96f3-f0bb937c6815 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f56f0efaec04576bd5316c90ebc6d32'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.880495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720c6ea-9ed0-4b8b-a33a-adfd6864710e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:18.880495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720c6ea-9ed0-4b8b-a33a-adfd6864710e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:18.880661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cdfc74b2-6fe0-4276-96f3-f0bb937c6815 tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f56f0efaec04576bd5316c90ebc6d32'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:18.880755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720c6ea-9ed0-4b8b-a33a-adfd6864710e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:18.902668 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1107/2239] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:18 2022] DELETE /identity/v3/users/2f56f0efaec04576bd5316c90ebc6d32 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:18.904149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46325e9-4406-40a5-8b62-43de470e517d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:18.904477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46325e9-4406-40a5-8b62-43de470e517d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:18.904747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46325e9-4406-40a5-8b62-43de470e517d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:18.933319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d46325e9-4406-40a5-8b62-43de470e517d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:18.937905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d46325e9-4406-40a5-8b62-43de470e517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:18.943644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5720c6ea-9ed0-4b8b-a33a-adfd6864710e None None] MFA Rules not processed for user `5bbd3d48bdce4bfa8af26db3d0dffbbd`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:18.945099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5720c6ea-9ed0-4b8b-a33a-adfd6864710e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:18.965092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:19.072508 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1133/2240] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:19.073437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720c6ea-9ed0-4b8b-a33a-adfd6864710e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:19.081810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5720c6ea-9ed0-4b8b-a33a-adfd6864710e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:19.085292 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1108/2241] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:19.086995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46325e9-4406-40a5-8b62-43de470e517d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:19.089495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d46325e9-4406-40a5-8b62-43de470e517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:19.119269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46325e9-4406-40a5-8b62-43de470e517d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:19.126028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd1c47-9ae7-481c-8a7c-7b7d5c61dcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:19.127406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd1c47-9ae7-481c-8a7c-7b7d5c61dcbb admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:19.127749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd1c47-9ae7-481c-8a7c-7b7d5c61dcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:19.128075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd1c47-9ae7-481c-8a7c-7b7d5c61dcbb admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:19.129005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd1c47-9ae7-481c-8a7c-7b7d5c61dcbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:19.131626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd1c47-9ae7-481c-8a7c-7b7d5c61dcbb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:19.147240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720c6ea-9ed0-4b8b-a33a-adfd6864710e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:19.163418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b31a0-5390-4c94-bbf8-383f304cca97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:19.164245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b31a0-5390-4c94-bbf8-383f304cca97 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:19.164547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b31a0-5390-4c94-bbf8-383f304cca97 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:19.164816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b31a0-5390-4c94-bbf8-383f304cca97 admin admin] PATH_INFO: `/v3/projects/441d89fde86f4e43ad40d4b9d62d7406/users/adb9e7f031ae467e8ba70a64fa3cf51b/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:19.168404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b31a0-5390-4c94-bbf8-383f304cca97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=441d89fde86f4e43ad40d4b9d62d7406, user_id=adb9e7f031ae467e8ba70a64fa3cf51b, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:19.171862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b31a0-5390-4c94-bbf8-383f304cca97 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:19.195457 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1109/2242] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:19 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:19.204464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd1c47-9ae7-481c-8a7c-7b7d5c61dcbb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:19.205225 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1134/2243] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:19 2022] PUT /identity/v3/projects/441d89fde86f4e43ad40d4b9d62d7406/users/adb9e7f031ae467e8ba70a64fa3cf51b/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:19.205225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b31a0-5390-4c94-bbf8-383f304cca97 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:19.205367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8bdd1c47-9ae7-481c-8a7c-7b7d5c61dcbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:19.205454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7a4b31a0-5390-4c94-bbf8-383f304cca97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:19.242378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd1c47-9ae7-481c-8a7c-7b7d5c61dcbb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:19.317232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:19.374034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b31a0-5390-4c94-bbf8-383f304cca97 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:19.383623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2893a64447f419c89ce450babc9fbe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9527f28a9846ff84e2a965962e7329', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:19.384305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:19.384619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:19.384888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] PATH_INFO: `/v3/projects/4899a4244fb34985aa5163e0311cfe6d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:19.386080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4899a4244fb34985aa5163e0311cfe6d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:19.388243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:19.389320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:19.390181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:19.390566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:19.390933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] PATH_INFO: `/v3/users/7f9214cc422141219e0377590b78a31e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:19.392650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f9214cc422141219e0377590b78a31e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:19.395474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:19.488583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4899a4244fb34985aa5163e0311cfe6d', 'request_id': 'req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:19.489791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f9214cc422141219e0377590b78a31e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:19.499161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4899a4244fb34985aa5163e0311cfe6d', 'request_id': 'req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:19.504235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f9214cc422141219e0377590b78a31e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:19.507024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4899a4244fb34985aa5163e0311cfe6d', 'request_id': 'req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:19.521361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] The token cache is being invalidate because project 4899a4244fb34985aa5163e0311cfe6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:19.522017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:19.522305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:19.523899 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1135/2244] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:19 2022] DELETE /identity/v3/projects/4899a4244fb34985aa5163e0311cfe6d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:19.529583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:19.530624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f9214cc422141219e0377590b78a31e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:19.531054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:19.531503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f9214cc422141219e0377590b78a31e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:19.532230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f9214cc422141219e0377590b78a31e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:19.551179 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1110/2245] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:19 2022] DELETE /identity/v3/users/7f9214cc422141219e0377590b78a31e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:19.551179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:19.559633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:19.600630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:19.675160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24675363-7033-47c4-9634-3b793dbe0f2d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:19.688136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6145e-fbe7-48f4-bf89-35dae4c66878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:19.688969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6145e-fbe7-48f4-bf89-35dae4c66878 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:19.689403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6145e-fbe7-48f4-bf89-35dae4c66878 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:19.689781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6145e-fbe7-48f4-bf89-35dae4c66878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:19.693282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9cf6145e-fbe7-48f4-bf89-35dae4c66878 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:19.722822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec2eb4-9cbb-45bd-adf3-6274c4d4414a tempest-QuotasAdminTestV236-72580814 tempest-QuotasAdminTestV236-72580814-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:19.814629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa34190-a14a-496a-ac91-3346c1064eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:19.815512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa34190-a14a-496a-ac91-3346c1064eda admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:19.817170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa34190-a14a-496a-ac91-3346c1064eda admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:19.817518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa34190-a14a-496a-ac91-3346c1064eda admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:19.820048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa34190-a14a-496a-ac91-3346c1064eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:19.827093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa34190-a14a-496a-ac91-3346c1064eda admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:19.861379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6145e-fbe7-48f4-bf89-35dae4c66878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:19.863444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6145e-fbe7-48f4-bf89-35dae4c66878 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:19.904579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:19.919250 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1111/2246] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:19.919613 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1136/2247] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:19 2022] POST /identity/v3/projects => generated 389 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:19.923032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa34190-a14a-496a-ac91-3346c1064eda admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:19.923352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a993ac1-d771-4d5a-985b-0232b9ed3179 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:19.923977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a993ac1-d771-4d5a-985b-0232b9ed3179 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:19.924581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a993ac1-d771-4d5a-985b-0232b9ed3179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:19.931733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1aa34190-a14a-496a-ac91-3346c1064eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:20.003023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7a993ac1-d771-4d5a-985b-0232b9ed3179 None None] MFA Rules not processed for user `adb9e7f031ae467e8ba70a64fa3cf51b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:20.004466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7a993ac1-d771-4d5a-985b-0232b9ed3179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:20.131120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa34190-a14a-496a-ac91-3346c1064eda admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:20.145645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02c94a-da73-4144-8c41-c5ca1781a45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:20.146202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02c94a-da73-4144-8c41-c5ca1781a45e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:20.146472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02c94a-da73-4144-8c41-c5ca1781a45e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:20.146733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02c94a-da73-4144-8c41-c5ca1781a45e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:20.147398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02c94a-da73-4144-8c41-c5ca1781a45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:20.151644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02c94a-da73-4144-8c41-c5ca1781a45e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:20.188190 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1137/2248] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:20.189223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02c94a-da73-4144-8c41-c5ca1781a45e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:20.195457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5d02c94a-da73-4144-8c41-c5ca1781a45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:20.216948 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1112/2249] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:20.216948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a993ac1-d771-4d5a-985b-0232b9ed3179 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:20.228318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7a993ac1-d771-4d5a-985b-0232b9ed3179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:20.246590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02c94a-da73-4144-8c41-c5ca1781a45e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:20.256402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:20.257077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:20.258159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:20.258428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] PATH_INFO: `/v3/users/7df78853c45d47ba8afe445b80f7e786` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:20.261515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=7df78853c45d47ba8afe445b80f7e786)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:20.264032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a993ac1-d771-4d5a-985b-0232b9ed3179 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:20.266406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:20.278030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fa2ff-772d-4384-963e-0ae7bd5f4879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:20.278664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fa2ff-772d-4384-963e-0ae7bd5f4879 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:20.278899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fa2ff-772d-4384-963e-0ae7bd5f4879 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:20.279165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fa2ff-772d-4384-963e-0ae7bd5f4879 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:20.280412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fa2ff-772d-4384-963e-0ae7bd5f4879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:20.283909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fa2ff-772d-4384-963e-0ae7bd5f4879 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:20.338949 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1113/2250] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:20 2022] POST /identity/v3/users => generated 301 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:20.340870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fa2ff-772d-4384-963e-0ae7bd5f4879 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:20.352811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fa2ff-772d-4384-963e-0ae7bd5f4879 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:20.358829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254f4dc-6a58-416d-890f-20fc39d752bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:20.359469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254f4dc-6a58-416d-890f-20fc39d752bf admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:20.359742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254f4dc-6a58-416d-890f-20fc39d752bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:20.360106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254f4dc-6a58-416d-890f-20fc39d752bf admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:20.360889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254f4dc-6a58-416d-890f-20fc39d752bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:20.362699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254f4dc-6a58-416d-890f-20fc39d752bf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:20.380203 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1114/2251] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:20.380203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254f4dc-6a58-416d-890f-20fc39d752bf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:20.381415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7df78853c45d47ba8afe445b80f7e786'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:20.381677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c254f4dc-6a58-416d-890f-20fc39d752bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:20.401983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7df78853c45d47ba8afe445b80f7e786'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:20.406613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:20.425712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7df78853c45d47ba8afe445b80f7e786'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:20.463671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7df78853c45d47ba8afe445b80f7e786'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:20.465276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7df78853c45d47ba8afe445b80f7e786'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:20.478935 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1138/2252] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:20 2022] DELETE /identity/v3/users/7df78853c45d47ba8afe445b80f7e786 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:20.479940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:20.481569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:20.486433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254f4dc-6a58-416d-890f-20fc39d752bf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:20.609942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:20.623077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7c3d6-fb8d-4bca-82c1-e80a82cc02ea admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:20.632317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:20.633114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:20.633391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:20.633658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] PATH_INFO: `/v3/projects/e63011fca84e43a4b1001bbd082b1bef` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:20.634906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e63011fca84e43a4b1001bbd082b1bef)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:20.637170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:20.641381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74520919-7b69-48a2-a110-500baa613748 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:20.642357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74520919-7b69-48a2-a110-500baa613748 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:20.642931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74520919-7b69-48a2-a110-500baa613748 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:20.643449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74520919-7b69-48a2-a110-500baa613748 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:20.647302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-74520919-7b69-48a2-a110-500baa613748 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:20.709517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e63011fca84e43a4b1001bbd082b1bef', 'request_id': 'req-85329446-62ee-49cc-8e5b-3aff28c0e041'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:20.718214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e63011fca84e43a4b1001bbd082b1bef', 'request_id': 'req-85329446-62ee-49cc-8e5b-3aff28c0e041'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:20.725006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e63011fca84e43a4b1001bbd082b1bef', 'request_id': 'req-85329446-62ee-49cc-8e5b-3aff28c0e041'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:20.737681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74520919-7b69-48a2-a110-500baa613748 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:20.739309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74520919-7b69-48a2-a110-500baa613748 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:20.742427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] The token cache is being invalidate because project e63011fca84e43a4b1001bbd082b1bef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:20.742708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:20.743688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:20.748513 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1139/2253] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:20 2022] DELETE /identity/v3/projects/e63011fca84e43a4b1001bbd082b1bef => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:20.749401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:20.750893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:20.756839 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1115/2254] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:20.758181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74520919-7b69-48a2-a110-500baa613748 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:20.760218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-74520919-7b69-48a2-a110-500baa613748 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:20.783625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74520919-7b69-48a2-a110-500baa613748 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:20.841361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85329446-62ee-49cc-8e5b-3aff28c0e041 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:20.847489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984f794-6893-45c9-8a66-567be975e5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:20.848425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984f794-6893-45c9-8a66-567be975e5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:20.848771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984f794-6893-45c9-8a66-567be975e5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:20.849130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984f794-6893-45c9-8a66-567be975e5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:20.849858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984f794-6893-45c9-8a66-567be975e5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:20.851569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984f794-6893-45c9-8a66-567be975e5a2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:20.867583 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1140/2255] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:20.867583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a6e14-d51f-4217-bac7-8126c47a3d4f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:20.867583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a6e14-d51f-4217-bac7-8126c47a3d4f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:20.867583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a6e14-d51f-4217-bac7-8126c47a3d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:20.873708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:20.897625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-540a6e14-d51f-4217-bac7-8126c47a3d4f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:20.898252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f98f0-3eb0-4dce-b069-f12b31ba590f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:20.899482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f98f0-3eb0-4dce-b069-f12b31ba590f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:20.900062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-540a6e14-d51f-4217-bac7-8126c47a3d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:20.900465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f98f0-3eb0-4dce-b069-f12b31ba590f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:20.900910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f98f0-3eb0-4dce-b069-f12b31ba590f admin admin] PATH_INFO: `/v3/projects/ec819c2e3517424fb5b00cd5fbca5f35/users/63d2977c98ed4acc8d53d8cfbed6f0c4/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:20.910616 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1141/2256] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:20.911732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a6e14-d51f-4217-bac7-8126c47a3d4f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:20.913671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-540a6e14-d51f-4217-bac7-8126c47a3d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:20.932182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a6e14-d51f-4217-bac7-8126c47a3d4f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:20.944357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:20.945635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:20.946095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:20.946404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f98f0-3eb0-4dce-b069-f12b31ba590f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec819c2e3517424fb5b00cd5fbca5f35, user_id=63d2977c98ed4acc8d53d8cfbed6f0c4, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:20.946886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] PATH_INFO: `/v3/projects/328d24ff14ce405ca559512bf8cdea07` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:20.948773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=328d24ff14ce405ca559512bf8cdea07)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:20.951448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f98f0-3eb0-4dce-b069-f12b31ba590f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:20.951817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:20.981106 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1116/2257] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:20 2022] PUT /identity/v3/projects/ec819c2e3517424fb5b00cd5fbca5f35/users/63d2977c98ed4acc8d53d8cfbed6f0c4/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:20.981106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c95b0-4fe6-4b55-8758-f8334e4564fb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:20.981106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c95b0-4fe6-4b55-8758-f8334e4564fb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:20.981106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c95b0-4fe6-4b55-8758-f8334e4564fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:21.001254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f97c95b0-4fe6-4b55-8758-f8334e4564fb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:21.002602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f97c95b0-4fe6-4b55-8758-f8334e4564fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:21.086212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '328d24ff14ce405ca559512bf8cdea07', 'request_id': 'req-f8e2b491-2068-40e5-acc5-7842558d76d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:21.110331 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1117/2258] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:21.114352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c95b0-4fe6-4b55-8758-f8334e4564fb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:21.115186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '328d24ff14ce405ca559512bf8cdea07', 'request_id': 'req-f8e2b491-2068-40e5-acc5-7842558d76d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:21.130443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '328d24ff14ce405ca559512bf8cdea07', 'request_id': 'req-f8e2b491-2068-40e5-acc5-7842558d76d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:21.134400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c95b0-4fe6-4b55-8758-f8334e4564fb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:21.143451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:21.150504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] The token cache is being invalidate because project 328d24ff14ce405ca559512bf8cdea07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:21.150739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:21.151484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:21.153407 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1142/2259] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:20 2022] DELETE /identity/v3/projects/328d24ff14ce405ca559512bf8cdea07 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:21.154304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:21.156665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:21.259371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3dbda-424f-441f-8100-86b31f0bc61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:21.259371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3dbda-424f-441f-8100-86b31f0bc61a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:21.259683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3dbda-424f-441f-8100-86b31f0bc61a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:21.259998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3dbda-424f-441f-8100-86b31f0bc61a admin admin] PATH_INFO: `/v3/projects/8dd70c651ee3425891a3b1f92ae2e946/users/327556af0d174529a6bbf2a90b376b02/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:21.288123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2b491-2068-40e5-acc5-7842558d76d9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:21.297315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dca14-1f64-40d4-b288-5973aa4c7632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:21.298253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dca14-1f64-40d4-b288-5973aa4c7632 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:21.298535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dca14-1f64-40d4-b288-5973aa4c7632 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:21.298807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dca14-1f64-40d4-b288-5973aa4c7632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:21.300740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0d8dca14-1f64-40d4-b288-5973aa4c7632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:21.308489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3dbda-424f-441f-8100-86b31f0bc61a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd70c651ee3425891a3b1f92ae2e946, user_id=327556af0d174529a6bbf2a90b376b02, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:21.316432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3dbda-424f-441f-8100-86b31f0bc61a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:21.325234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dca14-1f64-40d4-b288-5973aa4c7632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:21.327066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dca14-1f64-40d4-b288-5973aa4c7632 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:21.341012 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1118/2260] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:21 2022] PUT /identity/v3/projects/8dd70c651ee3425891a3b1f92ae2e946/users/327556af0d174529a6bbf2a90b376b02/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:21.345473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95b3fb-e474-4bda-b80d-2cacb6902ffa None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:21.348240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95b3fb-e474-4bda-b80d-2cacb6902ffa None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:21.348586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95b3fb-e474-4bda-b80d-2cacb6902ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:21.355172 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1143/2261] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:21.356484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51004-1930-4fe9-b30c-b7c9ea555d72 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:21.356767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51004-1930-4fe9-b30c-b7c9ea555d72 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:21.357035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51004-1930-4fe9-b30c-b7c9ea555d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:21.386443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-92b51004-1930-4fe9-b30c-b7c9ea555d72 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:21.387387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-92b51004-1930-4fe9-b30c-b7c9ea555d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:21.431952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9c95b3fb-e474-4bda-b80d-2cacb6902ffa None None] MFA Rules not processed for user `63d2977c98ed4acc8d53d8cfbed6f0c4`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:21.434158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9c95b3fb-e474-4bda-b80d-2cacb6902ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:21.512709 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1144/2262] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:21.513744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b51004-1930-4fe9-b30c-b7c9ea555d72 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:21.515916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-92b51004-1930-4fe9-b30c-b7c9ea555d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:21.536833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b51004-1930-4fe9-b30c-b7c9ea555d72 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:21.543878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:21.544683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:21.545153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:21.545539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] PATH_INFO: `/v3/users/444b9a04d9af48e7b57a30a168bcef6c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:21.547201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=444b9a04d9af48e7b57a30a168bcef6c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:21.549731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:21.569390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:21.594990 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1119/2263] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:21.596470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95b3fb-e474-4bda-b80d-2cacb6902ffa None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:21.599715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9c95b3fb-e474-4bda-b80d-2cacb6902ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:21.629660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95b3fb-e474-4bda-b80d-2cacb6902ffa None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:21.644952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '444b9a04d9af48e7b57a30a168bcef6c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:21.648745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f355e1-027c-44ac-ab30-390b657a2927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:21.649652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f355e1-027c-44ac-ab30-390b657a2927 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:21.650392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f355e1-027c-44ac-ab30-390b657a2927 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:21.651584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f355e1-027c-44ac-ab30-390b657a2927 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:21.652442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f355e1-027c-44ac-ab30-390b657a2927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:21.654639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f355e1-027c-44ac-ab30-390b657a2927 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:21.670286 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1120/2264] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:21.671657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f355e1-027c-44ac-ab30-390b657a2927 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:21.676386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '444b9a04d9af48e7b57a30a168bcef6c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:21.687094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '444b9a04d9af48e7b57a30a168bcef6c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:21.700942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f355e1-027c-44ac-ab30-390b657a2927 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:21.733561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '444b9a04d9af48e7b57a30a168bcef6c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:21.735903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f08d5-a667-4a78-a834-8285bb561a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:21.736206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '444b9a04d9af48e7b57a30a168bcef6c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:21.736889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f08d5-a667-4a78-a834-8285bb561a93 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:21.737306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f08d5-a667-4a78-a834-8285bb561a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:21.737698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f08d5-a667-4a78-a834-8285bb561a93 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:21.738485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f08d5-a667-4a78-a834-8285bb561a93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:21.748352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f08d5-a667-4a78-a834-8285bb561a93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:21.750933 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1145/2265] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:21 2022] DELETE /identity/v3/users/444b9a04d9af48e7b57a30a168bcef6c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:21.751854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:21.755059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:21.755262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f08d5-a667-4a78-a834-8285bb561a93 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:21.779054 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1121/2266] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:21.779054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f08d5-a667-4a78-a834-8285bb561a93 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:21.785524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-720f08d5-a667-4a78-a834-8285bb561a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:21.802467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f08d5-a667-4a78-a834-8285bb561a93 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:21.855936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9e64e-772c-47f9-815b-4037766db1c1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:21.862756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da7566-e511-4376-b29f-a5b072244916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:21.863774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da7566-e511-4376-b29f-a5b072244916 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:21.864416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da7566-e511-4376-b29f-a5b072244916 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:21.864416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da7566-e511-4376-b29f-a5b072244916 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:21.865330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da7566-e511-4376-b29f-a5b072244916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:21.868539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da7566-e511-4376-b29f-a5b072244916 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:21.891607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:21.899914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2219619-9425-4b2d-a7a8-01b4c761ba9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:21.900741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2219619-9425-4b2d-a7a8-01b4c761ba9e admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:21.901507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2219619-9425-4b2d-a7a8-01b4c761ba9e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:21.901507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2219619-9425-4b2d-a7a8-01b4c761ba9e admin admin] PATH_INFO: `/v3/projects/8dd70c651ee3425891a3b1f92ae2e946/users/327556af0d174529a6bbf2a90b376b02/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:21.905847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2219619-9425-4b2d-a7a8-01b4c761ba9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd70c651ee3425891a3b1f92ae2e946, user_id=327556af0d174529a6bbf2a90b376b02, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:21.913259 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1146/2267] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:21 2022] POST /identity/v3/projects => generated 393 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:21.914156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da7566-e511-4376-b29f-a5b072244916 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:21.920948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2219619-9425-4b2d-a7a8-01b4c761ba9e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:21.927106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-68da7566-e511-4376-b29f-a5b072244916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:21.947123 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1122/2268] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:21 2022] PUT /identity/v3/projects/8dd70c651ee3425891a3b1f92ae2e946/users/327556af0d174529a6bbf2a90b376b02/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:21.947123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2219619-9425-4b2d-a7a8-01b4c761ba9e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:21.947123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d2219619-9425-4b2d-a7a8-01b4c761ba9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:22.040575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2219619-9425-4b2d-a7a8-01b4c761ba9e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:22.051473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61befa-829f-4b99-8ef2-7d7e7dd7b54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:22.052342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61befa-829f-4b99-8ef2-7d7e7dd7b54f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:22.052725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61befa-829f-4b99-8ef2-7d7e7dd7b54f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:22.053063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61befa-829f-4b99-8ef2-7d7e7dd7b54f admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:22.053896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61befa-829f-4b99-8ef2-7d7e7dd7b54f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:22.055932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61befa-829f-4b99-8ef2-7d7e7dd7b54f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:22.069399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da7566-e511-4376-b29f-a5b072244916 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:22.088598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:22.092643 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1123/2269] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:21 2022] POST /identity/v3/projects => generated 399 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:22.093972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61befa-829f-4b99-8ef2-7d7e7dd7b54f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:22.167737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef294ff-776f-4d03-8d3b-74dea65d47ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:22.168647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef294ff-776f-4d03-8d3b-74dea65d47ee service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:22.168980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef294ff-776f-4d03-8d3b-74dea65d47ee service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:22.169341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef294ff-776f-4d03-8d3b-74dea65d47ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:22.172891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3ef294ff-776f-4d03-8d3b-74dea65d47ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:22.181689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61befa-829f-4b99-8ef2-7d7e7dd7b54f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:22.187084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51cb6b-a206-4a5a-9fd1-c8b588c2d98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:22.188076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51cb6b-a206-4a5a-9fd1-c8b588c2d98d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:22.188656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51cb6b-a206-4a5a-9fd1-c8b588c2d98d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:22.189293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51cb6b-a206-4a5a-9fd1-c8b588c2d98d admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:22.190260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51cb6b-a206-4a5a-9fd1-c8b588c2d98d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:22.192473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51cb6b-a206-4a5a-9fd1-c8b588c2d98d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:22.224349 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1124/2270] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:22 2022] POST /identity/v3/users => generated 303 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:22.238012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2535087-d4e2-472d-b46f-2a4da88f7474 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:22.238298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2535087-d4e2-472d-b46f-2a4da88f7474 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:22.238569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2535087-d4e2-472d-b46f-2a4da88f7474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:22.273081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef294ff-776f-4d03-8d3b-74dea65d47ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:22.275596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef294ff-776f-4d03-8d3b-74dea65d47ee service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:22.279746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e2535087-d4e2-472d-b46f-2a4da88f7474 None None] MFA Rules not processed for user `327556af0d174529a6bbf2a90b376b02`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:22.281728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e2535087-d4e2-472d-b46f-2a4da88f7474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:22.294105 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1147/2271] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:22.304772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef294ff-776f-4d03-8d3b-74dea65d47ee service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:22.344491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef294ff-776f-4d03-8d3b-74dea65d47ee service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:22.353214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b99c6-6a6d-4329-8386-352e24fee4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:22.355380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b99c6-6a6d-4329-8386-352e24fee4df admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:22.355911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b99c6-6a6d-4329-8386-352e24fee4df admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:22.356351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b99c6-6a6d-4329-8386-352e24fee4df admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:22.357477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b99c6-6a6d-4329-8386-352e24fee4df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:22.360823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b99c6-6a6d-4329-8386-352e24fee4df admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:22.365624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:22.409660 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1125/2272] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:22 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:22.409660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2535087-d4e2-472d-b46f-2a4da88f7474 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:22.419650 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1148/2273] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:22 2022] POST /identity/v3/users => generated 300 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:22.420632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b99c6-6a6d-4329-8386-352e24fee4df admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:22.432828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2535087-d4e2-472d-b46f-2a4da88f7474 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:22.436643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b99c6-6a6d-4329-8386-352e24fee4df admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:22.440433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf39f5-aba5-4a79-8507-aebee179984d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:22.441398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf39f5-aba5-4a79-8507-aebee179984d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:22.442312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf39f5-aba5-4a79-8507-aebee179984d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:22.442858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf39f5-aba5-4a79-8507-aebee179984d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:22.443657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf39f5-aba5-4a79-8507-aebee179984d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:22.446341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf39f5-aba5-4a79-8507-aebee179984d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:22.448212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674046a-1036-4016-8329-065f22c8811d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:22.448869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674046a-1036-4016-8329-065f22c8811d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:22.449139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674046a-1036-4016-8329-065f22c8811d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:22.449417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674046a-1036-4016-8329-065f22c8811d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:22.450780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674046a-1036-4016-8329-065f22c8811d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:22.452578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674046a-1036-4016-8329-065f22c8811d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:22.465774 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1126/2274] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:22.467032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf39f5-aba5-4a79-8507-aebee179984d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:22.473955 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1149/2275] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:22.475046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674046a-1036-4016-8329-065f22c8811d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:22.507698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674046a-1036-4016-8329-065f22c8811d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:22.517482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56052aa0-70c8-45e0-9d65-b02ec1e3d08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:22.518343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56052aa0-70c8-45e0-9d65-b02ec1e3d08d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:22.518622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56052aa0-70c8-45e0-9d65-b02ec1e3d08d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:22.522256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56052aa0-70c8-45e0-9d65-b02ec1e3d08d admin admin] PATH_INFO: `/v3/projects/faa12a42f1bc41d9a89ea4ef6fc229a6/users/f2fde96348ba419fa50ae7c4ef031acc/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:22.548196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56052aa0-70c8-45e0-9d65-b02ec1e3d08d admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa12a42f1bc41d9a89ea4ef6fc229a6, user_id=f2fde96348ba419fa50ae7c4ef031acc, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:22.551472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56052aa0-70c8-45e0-9d65-b02ec1e3d08d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:22.571640 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1150/2276] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:22 2022] PUT /identity/v3/projects/faa12a42f1bc41d9a89ea4ef6fc229a6/users/f2fde96348ba419fa50ae7c4ef031acc/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:22.573462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56052aa0-70c8-45e0-9d65-b02ec1e3d08d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:22.581900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf39f5-aba5-4a79-8507-aebee179984d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:22.675632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56052aa0-70c8-45e0-9d65-b02ec1e3d08d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:22.681847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf43ff-d111-4540-afa0-5be7caff4382 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:22.682566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf43ff-d111-4540-afa0-5be7caff4382 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:22.682993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf43ff-d111-4540-afa0-5be7caff4382 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:22.683128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf43ff-d111-4540-afa0-5be7caff4382 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:22.692908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b37e0-122b-4f24-b4ec-e2d01c3fac78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:22.693978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b37e0-122b-4f24-b4ec-e2d01c3fac78 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:22.694629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b37e0-122b-4f24-b4ec-e2d01c3fac78 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:22.695620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b37e0-122b-4f24-b4ec-e2d01c3fac78 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:22.696547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b37e0-122b-4f24-b4ec-e2d01c3fac78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:22.699225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b37e0-122b-4f24-b4ec-e2d01c3fac78 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:22.701700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:22.719053 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1151/2277] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:22.722250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b37e0-122b-4f24-b4ec-e2d01c3fac78 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:22.738688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b37e0-122b-4f24-b4ec-e2d01c3fac78 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:22.746192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba00bbc-1e8d-4abd-99b5-df7b191c6ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:22.748301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba00bbc-1e8d-4abd-99b5-df7b191c6ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:22.749502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba00bbc-1e8d-4abd-99b5-df7b191c6ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:22.750310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba00bbc-1e8d-4abd-99b5-df7b191c6ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:22.752942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba00bbc-1e8d-4abd-99b5-df7b191c6ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:22.755118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba00bbc-1e8d-4abd-99b5-df7b191c6ad8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:22.758318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf43ff-d111-4540-afa0-5be7caff4382 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:22.760434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf43ff-d111-4540-afa0-5be7caff4382 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:22.772879 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1152/2278] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:22.775083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba00bbc-1e8d-4abd-99b5-df7b191c6ad8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:22.785357 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1127/2279] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:22 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:22.786371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf43ff-d111-4540-afa0-5be7caff4382 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:22.794630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba00bbc-1e8d-4abd-99b5-df7b191c6ad8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:22.801971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cceeed-551e-4319-a693-4a3fd7da792a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:22.803192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cceeed-551e-4319-a693-4a3fd7da792a admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:22.803887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cceeed-551e-4319-a693-4a3fd7da792a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:22.804356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cceeed-551e-4319-a693-4a3fd7da792a admin admin] PATH_INFO: `/v3/projects/3a9fb306b25c4e61944be6e48b4bbc70/users/c759473a06f147e1bed7c15e245e01fc/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:22.812852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf43ff-d111-4540-afa0-5be7caff4382 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:22.830984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cceeed-551e-4319-a693-4a3fd7da792a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a9fb306b25c4e61944be6e48b4bbc70, user_id=c759473a06f147e1bed7c15e245e01fc, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:22.839168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cceeed-551e-4319-a693-4a3fd7da792a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:22.840637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dce26-6bfd-4aa6-8235-13f1ec99a2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:22.841548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dce26-6bfd-4aa6-8235-13f1ec99a2cf admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:22.841776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dce26-6bfd-4aa6-8235-13f1ec99a2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:22.842076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dce26-6bfd-4aa6-8235-13f1ec99a2cf admin admin] PATH_INFO: `/v3/projects/faa12a42f1bc41d9a89ea4ef6fc229a6/users/f2fde96348ba419fa50ae7c4ef031acc/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:22.861783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57dce26-6bfd-4aa6-8235-13f1ec99a2cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa12a42f1bc41d9a89ea4ef6fc229a6, user_id=f2fde96348ba419fa50ae7c4ef031acc, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:22.874301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57dce26-6bfd-4aa6-8235-13f1ec99a2cf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:22.887493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:22.899033 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1128/2281] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:22 2022] PUT /identity/v3/projects/faa12a42f1bc41d9a89ea4ef6fc229a6/users/f2fde96348ba419fa50ae7c4ef031acc/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:22.899629 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1153/2281] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:22 2022] PUT /identity/v3/projects/3a9fb306b25c4e61944be6e48b4bbc70/users/c759473a06f147e1bed7c15e245e01fc/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:22.900780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dce26-6bfd-4aa6-8235-13f1ec99a2cf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:22.905141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b57dce26-6bfd-4aa6-8235-13f1ec99a2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:22.906613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cceeed-551e-4319-a693-4a3fd7da792a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:23.036256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dce26-6bfd-4aa6-8235-13f1ec99a2cf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:23.051479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ac71a-cb79-4cbc-889e-56c786d80962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:23.052255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ac71a-cb79-4cbc-889e-56c786d80962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:23.052600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ac71a-cb79-4cbc-889e-56c786d80962 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:23.052873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ac71a-cb79-4cbc-889e-56c786d80962 admin admin] PATH_INFO: `/v3/projects/7b43087682a341af83e0b3d5fe8ed1b4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:23.056632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ac71a-cb79-4cbc-889e-56c786d80962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b43087682a341af83e0b3d5fe8ed1b4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:23.058599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ac71a-cb79-4cbc-889e-56c786d80962 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:23.074435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cceeed-551e-4319-a693-4a3fd7da792a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:23.168561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2e5ac71a-cb79-4cbc-889e-56c786d80962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b43087682a341af83e0b3d5fe8ed1b4', 'request_id': 'req-2e5ac71a-cb79-4cbc-889e-56c786d80962'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.175690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2e5ac71a-cb79-4cbc-889e-56c786d80962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b43087682a341af83e0b3d5fe8ed1b4', 'request_id': 'req-2e5ac71a-cb79-4cbc-889e-56c786d80962'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.188023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2e5ac71a-cb79-4cbc-889e-56c786d80962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b43087682a341af83e0b3d5fe8ed1b4', 'request_id': 'req-2e5ac71a-cb79-4cbc-889e-56c786d80962'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.206277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2e5ac71a-cb79-4cbc-889e-56c786d80962 admin admin] The token cache is being invalidate because project 7b43087682a341af83e0b3d5fe8ed1b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:23.206833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2e5ac71a-cb79-4cbc-889e-56c786d80962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.207869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2e5ac71a-cb79-4cbc-889e-56c786d80962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.209630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1aa08a-30fe-4de5-aecf-a817390873b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:23.210349 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1129/2282] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:22 2022] DELETE /identity/v3/projects/7b43087682a341af83e0b3d5fe8ed1b4 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:23.210485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1aa08a-30fe-4de5-aecf-a817390873b1 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:23.210628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1aa08a-30fe-4de5-aecf-a817390873b1 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:23.210894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1aa08a-30fe-4de5-aecf-a817390873b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:23.212012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15439e-2923-4b83-b757-5e7b070aace8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:23.212553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15439e-2923-4b83-b757-5e7b070aace8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:23.213108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15439e-2923-4b83-b757-5e7b070aace8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:23.213683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4a1aa08a-30fe-4de5-aecf-a817390873b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:23.247827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9a15439e-2923-4b83-b757-5e7b070aace8 None None] MFA Rules not processed for user `c759473a06f147e1bed7c15e245e01fc`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:23.249433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9a15439e-2923-4b83-b757-5e7b070aace8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:23.288830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:23.293489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1aa08a-30fe-4de5-aecf-a817390873b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:23.294948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1aa08a-30fe-4de5-aecf-a817390873b1 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:23.311028 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1154/2283] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:23.316794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb637b-d373-4ea1-a6d8-cc73dd9ae856 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:23.317079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb637b-d373-4ea1-a6d8-cc73dd9ae856 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:23.317341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb637b-d373-4ea1-a6d8-cc73dd9ae856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:23.363144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b6fb637b-d373-4ea1-a6d8-cc73dd9ae856 None None] MFA Rules not processed for user `f2fde96348ba419fa50ae7c4ef031acc`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:23.364344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b6fb637b-d373-4ea1-a6d8-cc73dd9ae856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:23.443013 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1130/2284] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:23.447062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15439e-2923-4b83-b757-5e7b070aace8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:23.450601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9a15439e-2923-4b83-b757-5e7b070aace8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:23.492145 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1155/2285] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:23 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:23.493352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb637b-d373-4ea1-a6d8-cc73dd9ae856 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:23.494957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b6fb637b-d373-4ea1-a6d8-cc73dd9ae856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:23.505205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:23.599315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15439e-2923-4b83-b757-5e7b070aace8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:23.605916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:23.606704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:23.607022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:23.607472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] PATH_INFO: `/v3/projects/46a47a6119924e719ea973fb6e234ac0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:23.608619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=46a47a6119924e719ea973fb6e234ac0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:23.611031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:23.627267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb637b-d373-4ea1-a6d8-cc73dd9ae856 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:23.633377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6ea24-7eb9-418c-bf6c-d5dd3778fef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:23.634126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6ea24-7eb9-418c-bf6c-d5dd3778fef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:23.634493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6ea24-7eb9-418c-bf6c-d5dd3778fef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:23.637190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6ea24-7eb9-418c-bf6c-d5dd3778fef0 admin admin] PATH_INFO: `/v3/users/a2893a64447f419c89ce450babc9fbe5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:23.641814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6ea24-7eb9-418c-bf6c-d5dd3778fef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2893a64447f419c89ce450babc9fbe5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:23.645641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6ea24-7eb9-418c-bf6c-d5dd3778fef0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:23.684886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46a47a6119924e719ea973fb6e234ac0', 'request_id': 'req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.710727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46a47a6119924e719ea973fb6e234ac0', 'request_id': 'req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.728120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46a47a6119924e719ea973fb6e234ac0', 'request_id': 'req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.745415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24f6ea24-7eb9-418c-bf6c-d5dd3778fef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2893a64447f419c89ce450babc9fbe5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.765364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24f6ea24-7eb9-418c-bf6c-d5dd3778fef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2893a64447f419c89ce450babc9fbe5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.769100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] The token cache is being invalidate because project 46a47a6119924e719ea973fb6e234ac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:23.769628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.770385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.779425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24f6ea24-7eb9-418c-bf6c-d5dd3778fef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2893a64447f419c89ce450babc9fbe5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.781027 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1131/2286] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:23 2022] DELETE /identity/v3/projects/46a47a6119924e719ea973fb6e234ac0 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:23.781027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:23.781027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:23.793529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24f6ea24-7eb9-418c-bf6c-d5dd3778fef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2893a64447f419c89ce450babc9fbe5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.794640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24f6ea24-7eb9-418c-bf6c-d5dd3778fef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2893a64447f419c89ce450babc9fbe5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:23.797239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:23.810993 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1156/2287] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:23 2022] DELETE /identity/v3/users/a2893a64447f419c89ce450babc9fbe5 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:23.810993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6425b8c-8841-4b8e-91ef-82eab613b731 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:23.810993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6425b8c-8841-4b8e-91ef-82eab613b731 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:23.810993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6425b8c-8841-4b8e-91ef-82eab613b731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:23.859076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f6425b8c-8841-4b8e-91ef-82eab613b731 None None] MFA Rules not processed for user `63d2977c98ed4acc8d53d8cfbed6f0c4`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:23.862040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f6425b8c-8841-4b8e-91ef-82eab613b731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:23.932216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a2b39-e95a-4ee8-97cd-4bb4c9b529dd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:23.950127 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1157/2288] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:23.950127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d7f45-93ce-4f43-b902-fd085a33ee2c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:23.950127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d7f45-93ce-4f43-b902-fd085a33ee2c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:23.950127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d7f45-93ce-4f43-b902-fd085a33ee2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:23.984459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bf0d7f45-93ce-4f43-b902-fd085a33ee2c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:23.987170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bf0d7f45-93ce-4f43-b902-fd085a33ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.014781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:24.025157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397238f-443c-419c-958f-4b4e7dba39d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:24.028127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397238f-443c-419c-958f-4b4e7dba39d7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.028543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397238f-443c-419c-958f-4b4e7dba39d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.029825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397238f-443c-419c-958f-4b4e7dba39d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.030393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397238f-443c-419c-958f-4b4e7dba39d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:24.034546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397238f-443c-419c-958f-4b4e7dba39d7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:24.068846 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1158/2289] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:24.070171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d7f45-93ce-4f43-b902-fd085a33ee2c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:24.071500 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1132/2290] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:23 2022] POST /identity/v3/projects => generated 397 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:24.071806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bf0d7f45-93ce-4f43-b902-fd085a33ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.072824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397238f-443c-419c-958f-4b4e7dba39d7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:24.076070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e397238f-443c-419c-958f-4b4e7dba39d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.085917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d7f45-93ce-4f43-b902-fd085a33ee2c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:24.181871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a58f6-2ed5-4b69-bd36-ad7793c49717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:24.182597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a58f6-2ed5-4b69-bd36-ad7793c49717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.182940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a58f6-2ed5-4b69-bd36-ad7793c49717 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.183269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a58f6-2ed5-4b69-bd36-ad7793c49717 admin admin] PATH_INFO: `/v3/users/5bbd3d48bdce4bfa8af26db3d0dffbbd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.184536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a58f6-2ed5-4b69-bd36-ad7793c49717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bbd3d48bdce4bfa8af26db3d0dffbbd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:24.187178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a58f6-2ed5-4b69-bd36-ad7793c49717 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:24.188196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397238f-443c-419c-958f-4b4e7dba39d7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:24.197046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8708c2-3c66-4336-ab76-3ca3382a2300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:24.198242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8708c2-3c66-4336-ab76-3ca3382a2300 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.198592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8708c2-3c66-4336-ab76-3ca3382a2300 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.198931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8708c2-3c66-4336-ab76-3ca3382a2300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.200980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cf8708c2-3c66-4336-ab76-3ca3382a2300 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.264996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fc3a58f6-2ed5-4b69-bd36-ad7793c49717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bbd3d48bdce4bfa8af26db3d0dffbbd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.267861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:24.283961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fc3a58f6-2ed5-4b69-bd36-ad7793c49717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bbd3d48bdce4bfa8af26db3d0dffbbd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.304121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fc3a58f6-2ed5-4b69-bd36-ad7793c49717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bbd3d48bdce4bfa8af26db3d0dffbbd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.330377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fc3a58f6-2ed5-4b69-bd36-ad7793c49717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bbd3d48bdce4bfa8af26db3d0dffbbd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.330949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8708c2-3c66-4336-ab76-3ca3382a2300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:24.332459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fc3a58f6-2ed5-4b69-bd36-ad7793c49717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bbd3d48bdce4bfa8af26db3d0dffbbd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.333524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8708c2-3c66-4336-ab76-3ca3382a2300 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:24.344461 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1159/2291] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:24 2022] DELETE /identity/v3/users/5bbd3d48bdce4bfa8af26db3d0dffbbd => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:24.344461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef173af4-cb2e-420e-947f-d3650cfa5c54 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.344461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef173af4-cb2e-420e-947f-d3650cfa5c54 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.345362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef173af4-cb2e-420e-947f-d3650cfa5c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.349942 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1133/2292] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:24.352841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8708c2-3c66-4336-ab76-3ca3382a2300 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:24.354345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cf8708c2-3c66-4336-ab76-3ca3382a2300 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.388472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8708c2-3c66-4336-ab76-3ca3382a2300 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:24.397849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ef173af4-cb2e-420e-947f-d3650cfa5c54 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:24.399304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ef173af4-cb2e-420e-947f-d3650cfa5c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.478861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d3f3a-c254-4c96-8834-77276f09c592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:24.480767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d3f3a-c254-4c96-8834-77276f09c592 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.481443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d3f3a-c254-4c96-8834-77276f09c592 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.481922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d3f3a-c254-4c96-8834-77276f09c592 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.482767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d3f3a-c254-4c96-8834-77276f09c592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:24.485704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d3f3a-c254-4c96-8834-77276f09c592 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:24.521592 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1134/2293] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:24 2022] POST /identity/v3/users => generated 305 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:24.523566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20349308-2205-4e92-9aed-31591850f662 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.524099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20349308-2205-4e92-9aed-31591850f662 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.524328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20349308-2205-4e92-9aed-31591850f662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.527038 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1160/2294] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:24.527983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef173af4-cb2e-420e-947f-d3650cfa5c54 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:24.530484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ef173af4-cb2e-420e-947f-d3650cfa5c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.541541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:24.543013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef173af4-cb2e-420e-947f-d3650cfa5c54 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:24.548451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-20349308-2205-4e92-9aed-31591850f662 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:24.548959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:24.549605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.549879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.550144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] PATH_INFO: `/v3/projects/3b9527f28a9846ff84e2a965962e7329` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.550774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-20349308-2205-4e92-9aed-31591850f662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.551302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b9527f28a9846ff84e2a965962e7329)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:24.553281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:24.560100 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1135/2295] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:24.560736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20349308-2205-4e92-9aed-31591850f662 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:24.562197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-20349308-2205-4e92-9aed-31591850f662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.574999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20349308-2205-4e92-9aed-31591850f662 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:24.580226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:24.580402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.580631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.580896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] PATH_INFO: `/v3/users/6e8cfaf24b134d65994aaca75c8a2e27` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.582210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8cfaf24b134d65994aaca75c8a2e27)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:24.584268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:24.615840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b9527f28a9846ff84e2a965962e7329', 'request_id': 'req-b7f83798-7439-4e27-a159-757f423b39d2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.625880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b9527f28a9846ff84e2a965962e7329', 'request_id': 'req-b7f83798-7439-4e27-a159-757f423b39d2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.632184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b9527f28a9846ff84e2a965962e7329', 'request_id': 'req-b7f83798-7439-4e27-a159-757f423b39d2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.644078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] The token cache is being invalidate because project 3b9527f28a9846ff84e2a965962e7329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:24.644584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.645251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.646711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8cfaf24b134d65994aaca75c8a2e27'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.652062 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1161/2296] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:24 2022] DELETE /identity/v3/projects/3b9527f28a9846ff84e2a965962e7329 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:24.652062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:24.653103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.656935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8cfaf24b134d65994aaca75c8a2e27'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.674763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8cfaf24b134d65994aaca75c8a2e27'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.675635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8cfaf24b134d65994aaca75c8a2e27'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.676494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8cfaf24b134d65994aaca75c8a2e27'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:24.690923 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1136/2297] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:24 2022] DELETE /identity/v3/users/6e8cfaf24b134d65994aaca75c8a2e27 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:24.694482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:24.696263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.722050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:24.746590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f83798-7439-4e27-a159-757f423b39d2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:24.790166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefeefdc-66dd-4e70-91db-d01a3053532e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:24.797110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1ed5f-6dc2-44a6-8602-d8a242d4d907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:24.797852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1ed5f-6dc2-44a6-8602-d8a242d4d907 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.798525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1ed5f-6dc2-44a6-8602-d8a242d4d907 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.798830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1ed5f-6dc2-44a6-8602-d8a242d4d907 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.801091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1ed5f-6dc2-44a6-8602-d8a242d4d907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:24.801467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1ed5f-6dc2-44a6-8602-d8a242d4d907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:24.803118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1ed5f-6dc2-44a6-8602-d8a242d4d907 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:24.823536 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1137/2298] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:24.824492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1ed5f-6dc2-44a6-8602-d8a242d4d907 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:24.833293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-89a1ed5f-6dc2-44a6-8602-d8a242d4d907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.839921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196418de-d754-43d7-a251-d8a6ceec14e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:24.841362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196418de-d754-43d7-a251-d8a6ceec14e6 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.841633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196418de-d754-43d7-a251-d8a6ceec14e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.841899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196418de-d754-43d7-a251-d8a6ceec14e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.842949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196418de-d754-43d7-a251-d8a6ceec14e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:24.846893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196418de-d754-43d7-a251-d8a6ceec14e6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:24.868759 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1162/2299] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:24.871594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff4df4-2db8-477a-b97b-6c886d2d4aec None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.871877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff4df4-2db8-477a-b97b-6c886d2d4aec None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.872146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff4df4-2db8-477a-b97b-6c886d2d4aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.898382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-81ff4df4-2db8-477a-b97b-6c886d2d4aec None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:24.899781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-81ff4df4-2db8-477a-b97b-6c886d2d4aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.911601 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1163/2300] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:24.914515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a82f6-2270-4c82-9420-dbab14925ecc None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.915600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a82f6-2270-4c82-9420-dbab14925ecc None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.917119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a82f6-2270-4c82-9420-dbab14925ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.938416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1ed5f-6dc2-44a6-8602-d8a242d4d907 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:24.942917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53751e88-ab4f-458c-8e67-7e1c96b7baa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:24.943255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7d7a82f6-2270-4c82-9420-dbab14925ecc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:24.943661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53751e88-ab4f-458c-8e67-7e1c96b7baa8 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.943936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53751e88-ab4f-458c-8e67-7e1c96b7baa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.944204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53751e88-ab4f-458c-8e67-7e1c96b7baa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.946388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7d7a82f6-2270-4c82-9420-dbab14925ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.946542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-53751e88-ab4f-458c-8e67-7e1c96b7baa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:24.961562 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1164/2301] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:24.961562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a82f6-2270-4c82-9420-dbab14925ecc None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:24.961804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53751e88-ab4f-458c-8e67-7e1c96b7baa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:24.962582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53751e88-ab4f-458c-8e67-7e1c96b7baa8 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:24.971442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a82f6-2270-4c82-9420-dbab14925ecc None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:24.976112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a5899-a45b-46ac-adbc-3922fa92ce20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:24.976835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a5899-a45b-46ac-adbc-3922fa92ce20 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:24.977232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a5899-a45b-46ac-adbc-3922fa92ce20 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:24.977511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a5899-a45b-46ac-adbc-3922fa92ce20 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:24.978893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a5899-a45b-46ac-adbc-3922fa92ce20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:24.981483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a5899-a45b-46ac-adbc-3922fa92ce20 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:24.984512 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1138/2302] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:24.988148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53751e88-ab4f-458c-8e67-7e1c96b7baa8 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:24.991717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-53751e88-ab4f-458c-8e67-7e1c96b7baa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:25.010464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53751e88-ab4f-458c-8e67-7e1c96b7baa8 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:25.030252 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1165/2303] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:24 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:25.030252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109321a0-5268-4507-8fbb-e88e992b950f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.030252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109321a0-5268-4507-8fbb-e88e992b950f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.030252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109321a0-5268-4507-8fbb-e88e992b950f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.036627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ebf6e-2730-496c-a89d-9300943f6665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:25.037529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ebf6e-2730-496c-a89d-9300943f6665 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.037827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ebf6e-2730-496c-a89d-9300943f6665 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.038107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ebf6e-2730-496c-a89d-9300943f6665 admin admin] PATH_INFO: `/v3/projects/a1d90d46eb0f45688c3dc536a480c559/users/7e79b0ad260746de853b8fd77a90fa28/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.043476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:25.058463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-109321a0-5268-4507-8fbb-e88e992b950f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:25.060269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-109321a0-5268-4507-8fbb-e88e992b950f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:25.063961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ebf6e-2730-496c-a89d-9300943f6665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d90d46eb0f45688c3dc536a480c559, user_id=7e79b0ad260746de853b8fd77a90fa28, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:25.068074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ebf6e-2730-496c-a89d-9300943f6665 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:25.090462 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1139/2304] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:24 2022] PUT /identity/v3/projects/a1d90d46eb0f45688c3dc536a480c559/users/7e79b0ad260746de853b8fd77a90fa28/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:25.091813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ebf6e-2730-496c-a89d-9300943f6665 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:25.094459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-414ebf6e-2730-496c-a89d-9300943f6665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:25.149251 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1166/2305] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:25.150988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109321a0-5268-4507-8fbb-e88e992b950f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:25.163539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109321a0-5268-4507-8fbb-e88e992b950f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:25.181123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ebf6e-2730-496c-a89d-9300943f6665 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:25.187190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fee493-055e-4993-892d-d46d5960d228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:25.188086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fee493-055e-4993-892d-d46d5960d228 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.189130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fee493-055e-4993-892d-d46d5960d228 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.189498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fee493-055e-4993-892d-d46d5960d228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.242598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:25.249856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ea5ea-deba-4398-b40c-2a6f501c484d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:25.250630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ea5ea-deba-4398-b40c-2a6f501c484d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.252609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ea5ea-deba-4398-b40c-2a6f501c484d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.253482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ea5ea-deba-4398-b40c-2a6f501c484d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.254884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ea5ea-deba-4398-b40c-2a6f501c484d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:25.257629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ea5ea-deba-4398-b40c-2a6f501c484d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:25.295438 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1167/2306] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:25 2022] POST /identity/v3/users => generated 291 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:25.296803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ea5ea-deba-4398-b40c-2a6f501c484d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:25.299209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fee493-055e-4993-892d-d46d5960d228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:25.301170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fee493-055e-4993-892d-d46d5960d228 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:25.314150 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1140/2307] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:25.315698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fee493-055e-4993-892d-d46d5960d228 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:25.318587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ea5ea-deba-4398-b40c-2a6f501c484d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:25.324815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a07012-52dd-4870-a64a-e5987d122a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:25.326548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a07012-52dd-4870-a64a-e5987d122a20 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.326828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a07012-52dd-4870-a64a-e5987d122a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.327147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a07012-52dd-4870-a64a-e5987d122a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.327868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a07012-52dd-4870-a64a-e5987d122a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:25.329769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a07012-52dd-4870-a64a-e5987d122a20 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:25.330349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fee493-055e-4993-892d-d46d5960d228 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:25.337886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b435674-7fbf-4888-a304-69db3990f14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:25.338764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b435674-7fbf-4888-a304-69db3990f14d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.339473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b435674-7fbf-4888-a304-69db3990f14d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.339880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b435674-7fbf-4888-a304-69db3990f14d admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.340729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b435674-7fbf-4888-a304-69db3990f14d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:25.341253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b435674-7fbf-4888-a304-69db3990f14d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:25.343442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b435674-7fbf-4888-a304-69db3990f14d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:25.349140 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1168/2308] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:25.351167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a07012-52dd-4870-a64a-e5987d122a20 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:25.367392 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1141/2309] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:25.367392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b435674-7fbf-4888-a304-69db3990f14d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:25.367747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a07012-52dd-4870-a64a-e5987d122a20 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:25.373575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cc6fb-d8c1-4712-9936-34edef08be8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:25.374426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cc6fb-d8c1-4712-9936-34edef08be8c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.374711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cc6fb-d8c1-4712-9936-34edef08be8c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.374980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cc6fb-d8c1-4712-9936-34edef08be8c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.375637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cc6fb-d8c1-4712-9936-34edef08be8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:25.377555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cc6fb-d8c1-4712-9936-34edef08be8c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:25.386604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b435674-7fbf-4888-a304-69db3990f14d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:25.392510 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1169/2310] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:25.392510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cc6fb-d8c1-4712-9936-34edef08be8c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:25.397642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d53929-6569-4584-b562-7a08525f2452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:25.399528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d53929-6569-4584-b562-7a08525f2452 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.399915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d53929-6569-4584-b562-7a08525f2452 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.400257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d53929-6569-4584-b562-7a08525f2452 admin admin] PATH_INFO: `/v3/projects/a1d90d46eb0f45688c3dc536a480c559/users/7e79b0ad260746de853b8fd77a90fa28/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.403481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d53929-6569-4584-b562-7a08525f2452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d90d46eb0f45688c3dc536a480c559, user_id=7e79b0ad260746de853b8fd77a90fa28, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:25.403853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cc6fb-d8c1-4712-9936-34edef08be8c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:25.407506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d53929-6569-4584-b562-7a08525f2452 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:25.410182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd51eb-c448-4425-b3ab-6f92b17f5ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:25.410869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd51eb-c448-4425-b3ab-6f92b17f5ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.411504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd51eb-c448-4425-b3ab-6f92b17f5ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.411841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd51eb-c448-4425-b3ab-6f92b17f5ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.412596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd51eb-c448-4425-b3ab-6f92b17f5ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:25.414601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd51eb-c448-4425-b3ab-6f92b17f5ec9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:25.433245 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1142/2311] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:25 2022] PUT /identity/v3/projects/a1d90d46eb0f45688c3dc536a480c559/users/7e79b0ad260746de853b8fd77a90fa28/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:25.434193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d53929-6569-4584-b562-7a08525f2452 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:25.468908 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1170/2312] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:25 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:25.471425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59571a43-35b8-4dcf-b334-1ba10a95b76d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.473126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59571a43-35b8-4dcf-b334-1ba10a95b76d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.473913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59571a43-35b8-4dcf-b334-1ba10a95b76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.524584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-59571a43-35b8-4dcf-b334-1ba10a95b76d None None] MFA Rules not processed for user `7e79b0ad260746de853b8fd77a90fa28`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:25.526408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-59571a43-35b8-4dcf-b334-1ba10a95b76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:25.535219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d53929-6569-4584-b562-7a08525f2452 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:25.557152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:25.633475 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1171/2313] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:25 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:25.634458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59571a43-35b8-4dcf-b334-1ba10a95b76d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:25.696198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b131a-e1b4-4b66-a136-e20e438f07af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:25.696912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b131a-e1b4-4b66-a136-e20e438f07af admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.697182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b131a-e1b4-4b66-a136-e20e438f07af admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.697411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b131a-e1b4-4b66-a136-e20e438f07af admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.698059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b131a-e1b4-4b66-a136-e20e438f07af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:25.699997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b131a-e1b4-4b66-a136-e20e438f07af admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:25.716934 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1143/2314] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:25.717874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b131a-e1b4-4b66-a136-e20e438f07af admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:25.719778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9c1b131a-e1b4-4b66-a136-e20e438f07af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:25.749177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59571a43-35b8-4dcf-b334-1ba10a95b76d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:25.757436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d39d4e-ecec-4662-86e7-7388fd85f3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:25.758401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d39d4e-ecec-4662-86e7-7388fd85f3eb admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.758745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d39d4e-ecec-4662-86e7-7388fd85f3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.759080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d39d4e-ecec-4662-86e7-7388fd85f3eb admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.760405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d39d4e-ecec-4662-86e7-7388fd85f3eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:25.763035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d39d4e-ecec-4662-86e7-7388fd85f3eb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:25.792741 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1172/2315] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:25 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:25.792741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d39d4e-ecec-4662-86e7-7388fd85f3eb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:25.802220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-09d39d4e-ecec-4662-86e7-7388fd85f3eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:25.823571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d39d4e-ecec-4662-86e7-7388fd85f3eb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:25.833432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:25.834234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.834577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.834970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] PATH_INFO: `/v3/projects/a89c81af4c64435cafa250b1d93e2c6c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.837032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89c81af4c64435cafa250b1d93e2c6c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:25.839553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:25.843652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b131a-e1b4-4b66-a136-e20e438f07af admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:25.854929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8ac22-b1a2-482c-a4cb-5becd7e66ab1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:25.855996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8ac22-b1a2-482c-a4cb-5becd7e66ab1 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:25.856460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8ac22-b1a2-482c-a4cb-5becd7e66ab1 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:25.857185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:25.857413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8ac22-b1a2-482c-a4cb-5becd7e66ab1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:25.859881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-80f8ac22-b1a2-482c-a4cb-5becd7e66ab1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:25.921571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89c81af4c64435cafa250b1d93e2c6c', 'request_id': 'req-23426587-ba18-4b87-ad00-3547acac3b11'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:25.932562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89c81af4c64435cafa250b1d93e2c6c', 'request_id': 'req-23426587-ba18-4b87-ad00-3547acac3b11'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:25.933849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8ac22-b1a2-482c-a4cb-5becd7e66ab1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:25.935944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8ac22-b1a2-482c-a4cb-5becd7e66ab1 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:25.941004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89c81af4c64435cafa250b1d93e2c6c', 'request_id': 'req-23426587-ba18-4b87-ad00-3547acac3b11'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:25.966171 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1144/2316] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:25 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:25.968787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8ac22-b1a2-482c-a4cb-5becd7e66ab1 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:25.983132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] The token cache is being invalidate because project a89c81af4c64435cafa250b1d93e2c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:25.983975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8ac22-b1a2-482c-a4cb-5becd7e66ab1 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:25.985199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:25.991944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:25.996460 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1173/2317] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:25 2022] DELETE /identity/v3/projects/a89c81af4c64435cafa250b1d93e2c6c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:25.998634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:26.001436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:26.082299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:26.099117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdebef-a60d-44d2-b285-d91d7ec44228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:26.100008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdebef-a60d-44d2-b285-d91d7ec44228 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:26.100189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdebef-a60d-44d2-b285-d91d7ec44228 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:26.100470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdebef-a60d-44d2-b285-d91d7ec44228 admin admin] PATH_INFO: `/v3/projects/61de886fca884a25a484d8d0f9e6662f/users/1762f75263ad4e2f84b265522e76764d/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:26.116849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23426587-ba18-4b87-ad00-3547acac3b11 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:26.119473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdebef-a60d-44d2-b285-d91d7ec44228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61de886fca884a25a484d8d0f9e6662f, user_id=1762f75263ad4e2f84b265522e76764d, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:26.122221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9809b4c-9f7d-4bcd-97a0-d20f4fd3dd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:26.123427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9809b4c-9f7d-4bcd-97a0-d20f4fd3dd7f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:26.123427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9809b4c-9f7d-4bcd-97a0-d20f4fd3dd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:26.123641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdebef-a60d-44d2-b285-d91d7ec44228 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:26.123720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9809b4c-9f7d-4bcd-97a0-d20f4fd3dd7f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:26.124012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9809b4c-9f7d-4bcd-97a0-d20f4fd3dd7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:26.126183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9809b4c-9f7d-4bcd-97a0-d20f4fd3dd7f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:26.140918 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1174/2318] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:26.141902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9809b4c-9f7d-4bcd-97a0-d20f4fd3dd7f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:26.143639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a9809b4c-9f7d-4bcd-97a0-d20f4fd3dd7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:26.152928 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1145/2319] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:25 2022] PUT /identity/v3/projects/61de886fca884a25a484d8d0f9e6662f/users/1762f75263ad4e2f84b265522e76764d/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:26.152928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdebef-a60d-44d2-b285-d91d7ec44228 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:26.152928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-11cdebef-a60d-44d2-b285-d91d7ec44228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:26.157998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9809b4c-9f7d-4bcd-97a0-d20f4fd3dd7f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:26.233290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:26.234774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:26.235377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:26.236183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] PATH_INFO: `/v3/projects/7b964ec386f64872a3dc6ec6432e360c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:26.238214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b964ec386f64872a3dc6ec6432e360c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:26.242144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:26.258686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdebef-a60d-44d2-b285-d91d7ec44228 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:26.266046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:26.267147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:26.268065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:26.268576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] PATH_INFO: `/v3/users/327556af0d174529a6bbf2a90b376b02` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:26.270056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=327556af0d174529a6bbf2a90b376b02)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:26.273428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:26.295082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:26.311372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b964ec386f64872a3dc6ec6432e360c', 'request_id': 'req-7ba4a491-b5b8-45b1-9204-dc217b5cef37'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.330396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b964ec386f64872a3dc6ec6432e360c', 'request_id': 'req-7ba4a491-b5b8-45b1-9204-dc217b5cef37'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.345477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b964ec386f64872a3dc6ec6432e360c', 'request_id': 'req-7ba4a491-b5b8-45b1-9204-dc217b5cef37'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.354457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '327556af0d174529a6bbf2a90b376b02'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.361229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] The token cache is being invalidate because project 7b964ec386f64872a3dc6ec6432e360c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:26.361555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.363085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.364932 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1175/2320] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:26 2022] DELETE /identity/v3/projects/7b964ec386f64872a3dc6ec6432e360c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:26.366656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '327556af0d174529a6bbf2a90b376b02'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.368862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:26.372128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:26.389200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '327556af0d174529a6bbf2a90b376b02'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.389934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '327556af0d174529a6bbf2a90b376b02'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.390566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '327556af0d174529a6bbf2a90b376b02'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.409656 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1146/2321] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:26 2022] DELETE /identity/v3/users/327556af0d174529a6bbf2a90b376b02 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:26.410605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:26.412312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:26.478866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:26.491158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4a491-b5b8-45b1-9204-dc217b5cef37 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:26.498158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973e6f8-b087-439f-b346-6772f98d50a3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:26.507036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1c819-637b-4a31-b90f-d265266d0e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:26.507870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1c819-637b-4a31-b90f-d265266d0e41 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:26.508141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1c819-637b-4a31-b90f-d265266d0e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:26.508439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1c819-637b-4a31-b90f-d265266d0e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:26.509156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1c819-637b-4a31-b90f-d265266d0e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:26.511285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1c819-637b-4a31-b90f-d265266d0e41 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:26.524235 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1147/2322] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:26.525699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e0d51-be5d-484f-b614-032d74695056 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:26.525979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e0d51-be5d-484f-b614-032d74695056 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:26.526251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e0d51-be5d-484f-b614-032d74695056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:26.548810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5a2e0d51-be5d-484f-b614-032d74695056 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:26.550041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5a2e0d51-be5d-484f-b614-032d74695056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:26.559585 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1148/2323] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:26.563748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e0d51-be5d-484f-b614-032d74695056 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:26.566241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5a2e0d51-be5d-484f-b614-032d74695056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:26.579852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25177f45-9750-4f99-8c5c-ba4b8942c779 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:26.580772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25177f45-9750-4f99-8c5c-ba4b8942c779 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:26.581049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25177f45-9750-4f99-8c5c-ba4b8942c779 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:26.581307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25177f45-9750-4f99-8c5c-ba4b8942c779 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:26.582591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e0d51-be5d-484f-b614-032d74695056 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:26.583817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-25177f45-9750-4f99-8c5c-ba4b8942c779 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:26.589043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:26.589694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:26.589970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:26.590237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] PATH_INFO: `/v3/users/adb9e7f031ae467e8ba70a64fa3cf51b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:26.591511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adb9e7f031ae467e8ba70a64fa3cf51b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:26.593750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:26.668090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adb9e7f031ae467e8ba70a64fa3cf51b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.684152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adb9e7f031ae467e8ba70a64fa3cf51b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.701869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25177f45-9750-4f99-8c5c-ba4b8942c779 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:26.705425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25177f45-9750-4f99-8c5c-ba4b8942c779 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:26.713132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adb9e7f031ae467e8ba70a64fa3cf51b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.713988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adb9e7f031ae467e8ba70a64fa3cf51b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.715270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:26.715553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adb9e7f031ae467e8ba70a64fa3cf51b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:26.742221 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1149/2324] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:26 2022] DELETE /identity/v3/users/adb9e7f031ae467e8ba70a64fa3cf51b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:26.742221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:26.742818 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1176/2325] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:26 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:26.742962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:26.743085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25177f45-9750-4f99-8c5c-ba4b8942c779 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:26.747411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-25177f45-9750-4f99-8c5c-ba4b8942c779 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:26.917990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3943240-48b4-4a1a-bcb9-3dde5f11e960 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:26.933425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25177f45-9750-4f99-8c5c-ba4b8942c779 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:26.942699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bf216-37aa-4bcc-86f4-39a39b66521d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:26.947644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bf216-37aa-4bcc-86f4-39a39b66521d admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:26.950272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bf216-37aa-4bcc-86f4-39a39b66521d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:26.951154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bf216-37aa-4bcc-86f4-39a39b66521d admin admin] PATH_INFO: `/v3/projects/3f7229da1351452f8faa09c632d41b77/users/d7ca457b2b5140bfb3944fc6ff48ed9e/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:27.003943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bf216-37aa-4bcc-86f4-39a39b66521d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f7229da1351452f8faa09c632d41b77, user_id=d7ca457b2b5140bfb3944fc6ff48ed9e, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:27.009264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bf216-37aa-4bcc-86f4-39a39b66521d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:27.038210 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1150/2326] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:26 2022] PUT /identity/v3/projects/3f7229da1351452f8faa09c632d41b77/users/d7ca457b2b5140bfb3944fc6ff48ed9e/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:27.055829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1680c-c43c-4354-9e21-4ae410ddada3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:27.056115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1680c-c43c-4354-9e21-4ae410ddada3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:27.056410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1680c-c43c-4354-9e21-4ae410ddada3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:27.071721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c6f04-0bb3-4c5f-849c-81e0a9c60aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:27.072412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c6f04-0bb3-4c5f-849c-81e0a9c60aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:27.072691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c6f04-0bb3-4c5f-849c-81e0a9c60aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:27.073233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c6f04-0bb3-4c5f-849c-81e0a9c60aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:27.074916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1f9c6f04-0bb3-4c5f-849c-81e0a9c60aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:27.091582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c6f04-0bb3-4c5f-849c-81e0a9c60aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:27.093914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2ce1680c-c43c-4354-9e21-4ae410ddada3 None None] MFA Rules not processed for user `d7ca457b2b5140bfb3944fc6ff48ed9e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:27.095628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c6f04-0bb3-4c5f-849c-81e0a9c60aa0 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:27.097108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2ce1680c-c43c-4354-9e21-4ae410ddada3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:27.124495 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1177/2327] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:27.139428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:27.170644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c6f04-0bb3-4c5f-849c-81e0a9c60aa0 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:27.172391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1f9c6f04-0bb3-4c5f-849c-81e0a9c60aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:27.194203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c6f04-0bb3-4c5f-849c-81e0a9c60aa0 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:27.234085 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1151/2328] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:56:27.235064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1680c-c43c-4354-9e21-4ae410ddada3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:27.288428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:27.289327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:27.289607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:27.289877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] PATH_INFO: `/v3/users/c759473a06f147e1bed7c15e245e01fc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:27.291276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c759473a06f147e1bed7c15e245e01fc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:27.293537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:27.378321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1680c-c43c-4354-9e21-4ae410ddada3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:27.387898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c759473a06f147e1bed7c15e245e01fc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:27.392075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf67c62-2b85-4119-bdf4-e25a971a99b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:27.393020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf67c62-2b85-4119-bdf4-e25a971a99b8 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:27.393682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf67c62-2b85-4119-bdf4-e25a971a99b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:27.394171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf67c62-2b85-4119-bdf4-e25a971a99b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:27.394786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf67c62-2b85-4119-bdf4-e25a971a99b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:27.400340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf67c62-2b85-4119-bdf4-e25a971a99b8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:27.409460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c759473a06f147e1bed7c15e245e01fc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:27.419632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:27.426597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c759473a06f147e1bed7c15e245e01fc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:27.458818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c759473a06f147e1bed7c15e245e01fc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:27.459614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c759473a06f147e1bed7c15e245e01fc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:27.469864 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1178/2329] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:27 2022] DELETE /identity/v3/users/c759473a06f147e1bed7c15e245e01fc => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:56:27.470932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:27.472733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:27.488477 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1152/2330] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:27 2022] POST /identity/v3/projects => generated 375 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:27.489654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9fd35-2ddd-4b38-a080-0b509ae2401e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:27.489928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9fd35-2ddd-4b38-a080-0b509ae2401e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:27.490188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9fd35-2ddd-4b38-a080-0b509ae2401e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:27.551852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-71b9fd35-2ddd-4b38-a080-0b509ae2401e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:27.554327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-71b9fd35-2ddd-4b38-a080-0b509ae2401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:27.578242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fba0e5-7f13-4557-9845-74d532ff26d4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:27.678241 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1153/2331] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:27.681516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9fd35-2ddd-4b38-a080-0b509ae2401e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:27.683203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-71b9fd35-2ddd-4b38-a080-0b509ae2401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:27.704235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:27.705260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:27.705705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:27.706686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] PATH_INFO: `/v3/projects/8dd70c651ee3425891a3b1f92ae2e946` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:27.709375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd70c651ee3425891a3b1f92ae2e946)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:27.712167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9fd35-2ddd-4b38-a080-0b509ae2401e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:27.712481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:27.720991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fcd411-0dfe-43fd-b6e6-6f0af1017dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:27.722682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcd411-0dfe-43fd-b6e6-6f0af1017dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:27.723094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcd411-0dfe-43fd-b6e6-6f0af1017dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:27.723455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcd411-0dfe-43fd-b6e6-6f0af1017dc3 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:27.724234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fcd411-0dfe-43fd-b6e6-6f0af1017dc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:27.727055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fcd411-0dfe-43fd-b6e6-6f0af1017dc3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:27.791394 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1154/2332] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:27 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:27.807870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fcd411-0dfe-43fd-b6e6-6f0af1017dc3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:27.812175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-79fcd411-0dfe-43fd-b6e6-6f0af1017dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:27.830527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd70c651ee3425891a3b1f92ae2e946', 'request_id': 'req-649a3e3a-8757-4157-95c2-75fb987f95cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:27.847712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd70c651ee3425891a3b1f92ae2e946', 'request_id': 'req-649a3e3a-8757-4157-95c2-75fb987f95cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:27.862060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd70c651ee3425891a3b1f92ae2e946', 'request_id': 'req-649a3e3a-8757-4157-95c2-75fb987f95cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:27.881866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] The token cache is being invalidate because project 8dd70c651ee3425891a3b1f92ae2e946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:27.882133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:27.885211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:27.891768 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1179/2333] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:27 2022] DELETE /identity/v3/projects/8dd70c651ee3425891a3b1f92ae2e946 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:27.893386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:27.895541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:27.896045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:27.917657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:27.939597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fcd411-0dfe-43fd-b6e6-6f0af1017dc3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:28.011624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a3e3a-8757-4157-95c2-75fb987f95cd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:28.013440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1bad-6b0b-4637-b908-0d295329b4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:28.014357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1bad-6b0b-4637-b908-0d295329b4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.014718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1bad-6b0b-4637-b908-0d295329b4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.015108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1bad-6b0b-4637-b908-0d295329b4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.015927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a1bad-6b0b-4637-b908-0d295329b4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:28.019206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a1bad-6b0b-4637-b908-0d295329b4e4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:28.038424 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1180/2334] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:28.043165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1bad-6b0b-4637-b908-0d295329b4e4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:28.046523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8f8a1bad-6b0b-4637-b908-0d295329b4e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:28.052097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf63e57-5a4f-453d-aaed-ed4d4e69d8a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:28.054639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf63e57-5a4f-453d-aaed-ed4d4e69d8a7 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.054974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf63e57-5a4f-453d-aaed-ed4d4e69d8a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.055166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf63e57-5a4f-453d-aaed-ed4d4e69d8a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.065272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9cf63e57-5a4f-453d-aaed-ed4d4e69d8a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:28.073564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1bad-6b0b-4637-b908-0d295329b4e4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:28.078750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:28.079774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.080155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.080515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] PATH_INFO: `/v3/projects/441d89fde86f4e43ad40d4b9d62d7406` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.082097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] RBAC: Authorizing `identity:delete_project(project_id=441d89fde86f4e43ad40d4b9d62d7406)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:28.084229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:28.098695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf63e57-5a4f-453d-aaed-ed4d4e69d8a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:28.100287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf63e57-5a4f-453d-aaed-ed4d4e69d8a7 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:28.121438 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1155/2335] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:28.123335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dd790-7914-4a63-91d3-86004b0ee7c1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.123617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dd790-7914-4a63-91d3-86004b0ee7c1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.123884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dd790-7914-4a63-91d3-86004b0ee7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.174788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6b2dd790-7914-4a63-91d3-86004b0ee7c1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:28.177645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6b2dd790-7914-4a63-91d3-86004b0ee7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:28.189464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '441d89fde86f4e43ad40d4b9d62d7406', 'request_id': 'req-68a5b294-299c-458b-ac8c-dbaef66d064a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:28.205843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '441d89fde86f4e43ad40d4b9d62d7406', 'request_id': 'req-68a5b294-299c-458b-ac8c-dbaef66d064a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:28.214659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '441d89fde86f4e43ad40d4b9d62d7406', 'request_id': 'req-68a5b294-299c-458b-ac8c-dbaef66d064a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:28.232196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] The token cache is being invalidate because project 441d89fde86f4e43ad40d4b9d62d7406 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:28.232592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:28.237943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:28.241505 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1181/2336] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:28 2022] DELETE /identity/v3/projects/441d89fde86f4e43ad40d4b9d62d7406 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:28.242394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:28.245458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:28.316991 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1156/2337] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:28.318515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bee85-5540-4d6f-85e9-b581f571c7c0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.318851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bee85-5540-4d6f-85e9-b581f571c7c0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.319121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bee85-5540-4d6f-85e9-b581f571c7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.364238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4c1bee85-5540-4d6f-85e9-b581f571c7c0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:28.365494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4c1bee85-5540-4d6f-85e9-b581f571c7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:28.381608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5b294-299c-458b-ac8c-dbaef66d064a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:28.405534 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1157/2338] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:28.405534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475e854-3ad9-44fc-83da-0407d86b21f5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.405534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475e854-3ad9-44fc-83da-0407d86b21f5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.405534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475e854-3ad9-44fc-83da-0407d86b21f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.414599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa7d02-e412-416e-ae47-613d045cb740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:28.415263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa7d02-e412-416e-ae47-613d045cb740 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.415537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa7d02-e412-416e-ae47-613d045cb740 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.416106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa7d02-e412-416e-ae47-613d045cb740 admin admin] PATH_INFO: `/v3/projects/aebea2bc737942fc899aa04e350f2936/users/51c6feb9eaf0447dbce898b2629c05a2/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.439141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4475e854-3ad9-44fc-83da-0407d86b21f5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:28.441953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4475e854-3ad9-44fc-83da-0407d86b21f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:28.449600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa7d02-e412-416e-ae47-613d045cb740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aebea2bc737942fc899aa04e350f2936, user_id=51c6feb9eaf0447dbce898b2629c05a2, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:28.458167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa7d02-e412-416e-ae47-613d045cb740 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:28.460420 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1158/2339] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:28.463490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b98a0b-7985-4e94-a5ac-a7b033a1bb91 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.463859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b98a0b-7985-4e94-a5ac-a7b033a1bb91 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.464293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b98a0b-7985-4e94-a5ac-a7b033a1bb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.487524 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1182/2340] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:28 2022] PUT /identity/v3/projects/aebea2bc737942fc899aa04e350f2936/users/51c6feb9eaf0447dbce898b2629c05a2/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:28.488594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa7d02-e412-416e-ae47-613d045cb740 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:28.523813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-96b98a0b-7985-4e94-a5ac-a7b033a1bb91 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:28.528663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-96b98a0b-7985-4e94-a5ac-a7b033a1bb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:28.530411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:28.627802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa7d02-e412-416e-ae47-613d045cb740 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:28.635302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7765c3d-c115-42ca-914d-f47fec5cfaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:28.637622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7765c3d-c115-42ca-914d-f47fec5cfaeb admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.638595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7765c3d-c115-42ca-914d-f47fec5cfaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.639402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7765c3d-c115-42ca-914d-f47fec5cfaeb admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.644117 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1159/2341] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:28.644117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b98a0b-7985-4e94-a5ac-a7b033a1bb91 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:28.645923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7765c3d-c115-42ca-914d-f47fec5cfaeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:28.646770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7765c3d-c115-42ca-914d-f47fec5cfaeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:28.649265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7765c3d-c115-42ca-914d-f47fec5cfaeb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:28.668518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b98a0b-7985-4e94-a5ac-a7b033a1bb91 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:28.675086 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1183/2342] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:28.675086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7765c3d-c115-42ca-914d-f47fec5cfaeb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:28.681121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12cc4a-3770-4245-b7ba-50c5a53ba414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:28.682231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12cc4a-3770-4245-b7ba-50c5a53ba414 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.682766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12cc4a-3770-4245-b7ba-50c5a53ba414 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.683502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12cc4a-3770-4245-b7ba-50c5a53ba414 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.684486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12cc4a-3770-4245-b7ba-50c5a53ba414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:28.686500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12cc4a-3770-4245-b7ba-50c5a53ba414 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:28.691051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7765c3d-c115-42ca-914d-f47fec5cfaeb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:28.705920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d665500-dce5-4281-a245-144c09974d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:28.707758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d665500-dce5-4281-a245-144c09974d4e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.709237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d665500-dce5-4281-a245-144c09974d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.709741 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1160/2343] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:28.710247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d665500-dce5-4281-a245-144c09974d4e admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.711275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12cc4a-3770-4245-b7ba-50c5a53ba414 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:28.711880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d665500-dce5-4281-a245-144c09974d4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:28.712244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d665500-dce5-4281-a245-144c09974d4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:28.715412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d665500-dce5-4281-a245-144c09974d4e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:28.727782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12cc4a-3770-4245-b7ba-50c5a53ba414 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:28.731899 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1184/2344] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:28.737337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb80cb-4477-402e-a3b0-de119758e48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:28.737942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d665500-dce5-4281-a245-144c09974d4e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:28.738155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb80cb-4477-402e-a3b0-de119758e48d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.738524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb80cb-4477-402e-a3b0-de119758e48d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.738897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb80cb-4477-402e-a3b0-de119758e48d admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.739733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb80cb-4477-402e-a3b0-de119758e48d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:28.741856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb80cb-4477-402e-a3b0-de119758e48d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:28.748630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:28.755543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d665500-dce5-4281-a245-144c09974d4e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:28.765528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a156cfe-bc0b-4c0f-a90e-7f8b33996ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:28.765528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a156cfe-bc0b-4c0f-a90e-7f8b33996ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.765833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a156cfe-bc0b-4c0f-a90e-7f8b33996ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.765914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a156cfe-bc0b-4c0f-a90e-7f8b33996ec4 admin admin] PATH_INFO: `/v3/projects/aebea2bc737942fc899aa04e350f2936/users/51c6feb9eaf0447dbce898b2629c05a2/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.770215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a156cfe-bc0b-4c0f-a90e-7f8b33996ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aebea2bc737942fc899aa04e350f2936, user_id=51c6feb9eaf0447dbce898b2629c05a2, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:28.773971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a156cfe-bc0b-4c0f-a90e-7f8b33996ec4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:28.785923 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1161/2345] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:28 2022] POST /identity/v3/projects => generated 377 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:28.787116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb80cb-4477-402e-a3b0-de119758e48d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:28.806502 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1185/2346] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:28 2022] PUT /identity/v3/projects/aebea2bc737942fc899aa04e350f2936/users/51c6feb9eaf0447dbce898b2629c05a2/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:28.808152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a156cfe-bc0b-4c0f-a90e-7f8b33996ec4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:28.894290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a156cfe-bc0b-4c0f-a90e-7f8b33996ec4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:28.899816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26dc08f-1837-4133-935e-3e44dd2e2065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:28.900609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dc08f-1837-4133-935e-3e44dd2e2065 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.900993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dc08f-1837-4133-935e-3e44dd2e2065 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.901349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dc08f-1837-4133-935e-3e44dd2e2065 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.902150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26dc08f-1837-4133-935e-3e44dd2e2065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:28.904868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26dc08f-1837-4133-935e-3e44dd2e2065 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:28.908243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb80cb-4477-402e-a3b0-de119758e48d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:28.914243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69dc3d-1117-4b98-ab90-afaf0350cf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:28.915544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69dc3d-1117-4b98-ab90-afaf0350cf09 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.915902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69dc3d-1117-4b98-ab90-afaf0350cf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.916247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69dc3d-1117-4b98-ab90-afaf0350cf09 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.917451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69dc3d-1117-4b98-ab90-afaf0350cf09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:28.922963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69dc3d-1117-4b98-ab90-afaf0350cf09 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:28.935296 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1186/2347] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:28 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:28.936923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de227f-2049-4e16-b9db-dc0d0888f0d0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:28.937300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de227f-2049-4e16-b9db-dc0d0888f0d0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:28.937671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de227f-2049-4e16-b9db-dc0d0888f0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:28.983937 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1162/2348] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:28 2022] POST /identity/v3/projects => generated 373 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:28.984874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69dc3d-1117-4b98-ab90-afaf0350cf09 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:28.989905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a7de227f-2049-4e16-b9db-dc0d0888f0d0 None None] MFA Rules not processed for user `51c6feb9eaf0447dbce898b2629c05a2`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:28.990993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a7de227f-2049-4e16-b9db-dc0d0888f0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:29.125347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:29.132752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69dc3d-1117-4b98-ab90-afaf0350cf09 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:29.137338 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1187/2349] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:29.138317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de227f-2049-4e16-b9db-dc0d0888f0d0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:29.145356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412d661-79cb-4c72-a1c3-20b76457768a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:29.146030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d661-79cb-4c72-a1c3-20b76457768a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:29.146327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d661-79cb-4c72-a1c3-20b76457768a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:29.146895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d661-79cb-4c72-a1c3-20b76457768a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:29.147551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412d661-79cb-4c72-a1c3-20b76457768a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:29.149249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412d661-79cb-4c72-a1c3-20b76457768a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:29.164664 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1163/2350] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:29.164664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412d661-79cb-4c72-a1c3-20b76457768a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:29.166682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de227f-2049-4e16-b9db-dc0d0888f0d0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:29.180349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eebca2-badb-4f80-9cc8-91fe188b357a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:29.181040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eebca2-badb-4f80-9cc8-91fe188b357a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:29.181311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eebca2-badb-4f80-9cc8-91fe188b357a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:29.181594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eebca2-badb-4f80-9cc8-91fe188b357a admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:29.182231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eebca2-badb-4f80-9cc8-91fe188b357a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:29.184561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eebca2-badb-4f80-9cc8-91fe188b357a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:29.192920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412d661-79cb-4c72-a1c3-20b76457768a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:29.205209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e01fe-f814-469e-9e5c-1affdeb313bb tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c6feb9eaf0447dbce898b2629c05a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aebea2bc737942fc899aa04e350f2936', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:29.205835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e01fe-f814-469e-9e5c-1affdeb313bb tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:29.206112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e01fe-f814-469e-9e5c-1affdeb313bb tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:29.206951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e01fe-f814-469e-9e5c-1affdeb313bb tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:29.207230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e01fe-f814-469e-9e5c-1affdeb313bb tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:29.207363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e01fe-f814-469e-9e5c-1affdeb313bb tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:29.211437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e01fe-f814-469e-9e5c-1affdeb313bb tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:29.236700 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1188/2351] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:29 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:29.236700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eebca2-badb-4f80-9cc8-91fe188b357a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:29.243014 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1164/2352] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:29.246351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e01fe-f814-469e-9e5c-1affdeb313bb tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:29.251327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eebca2-badb-4f80-9cc8-91fe188b357a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:29.264838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e01fe-f814-469e-9e5c-1affdeb313bb tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:29.266915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9e53d-ef6d-423b-b3a6-7462765bd607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:29.267774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9e53d-ef6d-423b-b3a6-7462765bd607 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:29.268055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9e53d-ef6d-423b-b3a6-7462765bd607 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:29.268321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9e53d-ef6d-423b-b3a6-7462765bd607 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:29.268995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9e53d-ef6d-423b-b3a6-7462765bd607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:29.271445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9e53d-ef6d-423b-b3a6-7462765bd607 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:29.281735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273be9f4-11bd-4bc7-a3f6-7bec4f24ab4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:29.282370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273be9f4-11bd-4bc7-a3f6-7bec4f24ab4d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:29.282820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273be9f4-11bd-4bc7-a3f6-7bec4f24ab4d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:29.283460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273be9f4-11bd-4bc7-a3f6-7bec4f24ab4d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:29.284210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273be9f4-11bd-4bc7-a3f6-7bec4f24ab4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:29.288840 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1189/2353] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:29.290339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273be9f4-11bd-4bc7-a3f6-7bec4f24ab4d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:29.290463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9e53d-ef6d-423b-b3a6-7462765bd607 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:29.302041 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1165/2354] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:29.306009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273be9f4-11bd-4bc7-a3f6-7bec4f24ab4d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:29.310462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9e53d-ef6d-423b-b3a6-7462765bd607 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:29.318040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273be9f4-11bd-4bc7-a3f6-7bec4f24ab4d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:29.325056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c8660-00b7-4671-924d-df86be8d3fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:29.326621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c8660-00b7-4671-924d-df86be8d3fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:29.326621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c8660-00b7-4671-924d-df86be8d3fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:29.326917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c232648-95d3-409d-b84d-8ee4545bb069 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c6feb9eaf0447dbce898b2629c05a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aebea2bc737942fc899aa04e350f2936', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:29.327199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c8660-00b7-4671-924d-df86be8d3fd3 admin admin] PATH_INFO: `/v3/projects/5ccd8386725c4a328bbe4ff346d8aa52/users/0f7a20d91493486789c123453ac5586b/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:29.327552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c232648-95d3-409d-b84d-8ee4545bb069 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:29.327849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c232648-95d3-409d-b84d-8ee4545bb069 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:29.328101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c232648-95d3-409d-b84d-8ee4545bb069 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:29.329016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c232648-95d3-409d-b84d-8ee4545bb069 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:29.331052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c232648-95d3-409d-b84d-8ee4545bb069 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:29.361594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c8660-00b7-4671-924d-df86be8d3fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ccd8386725c4a328bbe4ff346d8aa52, user_id=0f7a20d91493486789c123453ac5586b, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:29.364866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c8660-00b7-4671-924d-df86be8d3fd3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:29.385307 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1166/2355] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:29 2022] PUT /identity/v3/projects/5ccd8386725c4a328bbe4ff346d8aa52/users/0f7a20d91493486789c123453ac5586b/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:29.386286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c8660-00b7-4671-924d-df86be8d3fd3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:29.401824 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1190/2356] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:29 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:29.403176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359cb3f-df77-41e0-85a5-9157e7da4d6a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:29.403460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359cb3f-df77-41e0-85a5-9157e7da4d6a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:29.403729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359cb3f-df77-41e0-85a5-9157e7da4d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:29.432862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e359cb3f-df77-41e0-85a5-9157e7da4d6a None None] MFA Rules not processed for user `0f7a20d91493486789c123453ac5586b`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:29.434091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e359cb3f-df77-41e0-85a5-9157e7da4d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:29.452110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:29.542981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c8660-00b7-4671-924d-df86be8d3fd3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:29.545570 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1191/2357] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:29.546575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359cb3f-df77-41e0-85a5-9157e7da4d6a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:29.660869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8f202-fa26-4bcc-9527-d3234067a3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:29.661498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8f202-fa26-4bcc-9527-d3234067a3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:29.661773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8f202-fa26-4bcc-9527-d3234067a3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:29.662040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8f202-fa26-4bcc-9527-d3234067a3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:29.664837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8f202-fa26-4bcc-9527-d3234067a3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:29.666464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8f202-fa26-4bcc-9527-d3234067a3c1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:29.679461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359cb3f-df77-41e0-85a5-9157e7da4d6a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:29.686785 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1167/2358] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:29.687669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8f202-fa26-4bcc-9527-d3234067a3c1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:29.691835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9841501-c51b-4340-9e61-47ca60cd7c00 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c6feb9eaf0447dbce898b2629c05a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aebea2bc737942fc899aa04e350f2936', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:29.692489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9841501-c51b-4340-9e61-47ca60cd7c00 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:29.692769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9841501-c51b-4340-9e61-47ca60cd7c00 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:29.693087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9841501-c51b-4340-9e61-47ca60cd7c00 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:29.693748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9841501-c51b-4340-9e61-47ca60cd7c00 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:29.694051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9841501-c51b-4340-9e61-47ca60cd7c00 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:29.696024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9841501-c51b-4340-9e61-47ca60cd7c00 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:29.696950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-30f8f202-fa26-4bcc-9527-d3234067a3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:29.735588 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1192/2359] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:29.737800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8f202-fa26-4bcc-9527-d3234067a3c1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:29.739149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9841501-c51b-4340-9e61-47ca60cd7c00 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:29.751460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:29.752226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:29.752531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:29.752800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] PATH_INFO: `/v3/projects/3a9fb306b25c4e61944be6e48b4bbc70` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:29.755197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a9fb306b25c4e61944be6e48b4bbc70)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:29.760188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:29.764150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9841501-c51b-4340-9e61-47ca60cd7c00 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:29.777327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f3025-1fcb-4a53-8efc-810043c45042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:29.777669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f3025-1fcb-4a53-8efc-810043c45042 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:29.777940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f3025-1fcb-4a53-8efc-810043c45042 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:29.778506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f3025-1fcb-4a53-8efc-810043c45042 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:29.778829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f3025-1fcb-4a53-8efc-810043c45042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:29.780794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f3025-1fcb-4a53-8efc-810043c45042 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:29.850612 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1193/2360] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:29 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:29.853700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f3025-1fcb-4a53-8efc-810043c45042 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:29.870841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a9fb306b25c4e61944be6e48b4bbc70', 'request_id': 'req-aeb32e84-c741-412d-91b5-24ad6b33ad88'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:29.876841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:29.886460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a9fb306b25c4e61944be6e48b4bbc70', 'request_id': 'req-aeb32e84-c741-412d-91b5-24ad6b33ad88'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:29.905030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a9fb306b25c4e61944be6e48b4bbc70', 'request_id': 'req-aeb32e84-c741-412d-91b5-24ad6b33ad88'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:29.923819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] The token cache is being invalidate because project 3a9fb306b25c4e61944be6e48b4bbc70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:29.925397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:29.927657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:29.934076 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1168/2361] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:29 2022] DELETE /identity/v3/projects/3a9fb306b25c4e61944be6e48b4bbc70 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:29.937033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:29.940285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:29.998395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f3025-1fcb-4a53-8efc-810043c45042 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:30.009847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0313bb2-a88a-4b35-9db9-0cba72f06369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:30.011074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0313bb2-a88a-4b35-9db9-0cba72f06369 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:30.011473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0313bb2-a88a-4b35-9db9-0cba72f06369 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:30.011805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0313bb2-a88a-4b35-9db9-0cba72f06369 admin admin] PATH_INFO: `/v3/projects/edddaed1c4774c3c815a69383b963535/users/861c813d6f5a42e3adbbe1e74ef1873a/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:30.042487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0313bb2-a88a-4b35-9db9-0cba72f06369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edddaed1c4774c3c815a69383b963535, user_id=861c813d6f5a42e3adbbe1e74ef1873a, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:30.045626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0313bb2-a88a-4b35-9db9-0cba72f06369 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:30.054151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:30.089281 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1194/2362] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:29 2022] PUT /identity/v3/projects/edddaed1c4774c3c815a69383b963535/users/861c813d6f5a42e3adbbe1e74ef1873a/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:30.089281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0313bb2-a88a-4b35-9db9-0cba72f06369 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:30.089281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a0313bb2-a88a-4b35-9db9-0cba72f06369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:30.123247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb32e84-c741-412d-91b5-24ad6b33ad88 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:30.239753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0313bb2-a88a-4b35-9db9-0cba72f06369 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:30.245287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48f7e4-e9ca-464b-8081-cff0de4a9a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:30.246230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48f7e4-e9ca-464b-8081-cff0de4a9a3a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:30.246980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48f7e4-e9ca-464b-8081-cff0de4a9a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:30.247329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48f7e4-e9ca-464b-8081-cff0de4a9a3a admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:30.249526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48f7e4-e9ca-464b-8081-cff0de4a9a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:30.252219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48f7e4-e9ca-464b-8081-cff0de4a9a3a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:30.282136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7c816-e36c-44a4-a6ab-d6f1ea05bcc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:30.283297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7c816-e36c-44a4-a6ab-d6f1ea05bcc5 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:30.283727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7c816-e36c-44a4-a6ab-d6f1ea05bcc5 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:30.284144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7c816-e36c-44a4-a6ab-d6f1ea05bcc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:30.286236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a3d7c816-e36c-44a4-a6ab-d6f1ea05bcc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:30.287216 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1195/2363] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:30 2022] POST /identity/v3/users => generated 295 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:30.288251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48f7e4-e9ca-464b-8081-cff0de4a9a3a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:30.294565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fe48f7e4-e9ca-464b-8081-cff0de4a9a3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:30.317841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48f7e4-e9ca-464b-8081-cff0de4a9a3a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:30.327197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:30.327821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:30.328108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:30.334075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] PATH_INFO: `/v3/users/7e79b0ad260746de853b8fd77a90fa28` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:30.336736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e79b0ad260746de853b8fd77a90fa28)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:30.339330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:30.470925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7c816-e36c-44a4-a6ab-d6f1ea05bcc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:30.472593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7c816-e36c-44a4-a6ab-d6f1ea05bcc5 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:30.483949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e79b0ad260746de853b8fd77a90fa28'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:30.499094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e79b0ad260746de853b8fd77a90fa28'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:30.511966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e79b0ad260746de853b8fd77a90fa28'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:30.523364 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1169/2364] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:30.527687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09138956-6588-4e51-849c-baacee2cb681 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:30.527976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09138956-6588-4e51-849c-baacee2cb681 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:30.533084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09138956-6588-4e51-849c-baacee2cb681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:30.568958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:30.578560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e79b0ad260746de853b8fd77a90fa28'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:30.581707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e79b0ad260746de853b8fd77a90fa28'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:30.593571 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1196/2365] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:30 2022] DELETE /identity/v3/users/7e79b0ad260746de853b8fd77a90fa28 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:30.594676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:30.599861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:30.625095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-09138956-6588-4e51-849c-baacee2cb681 None None] MFA Rules not processed for user `861c813d6f5a42e3adbbe1e74ef1873a`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:30.628601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-09138956-6588-4e51-849c-baacee2cb681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:30.792932 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1170/2366] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:30.795589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e6b18-be04-4e4e-8cbb-e7dffbc6c813 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:30.796176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e6b18-be04-4e4e-8cbb-e7dffbc6c813 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:30.796660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e6b18-be04-4e4e-8cbb-e7dffbc6c813 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:30.804233 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1171/2367] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:56:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:56:30.805462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e6b18-be04-4e4e-8cbb-e7dffbc6c813 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:30.808529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3e7e6b18-be04-4e4e-8cbb-e7dffbc6c813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:30.824685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e6625-3209-4661-a598-5b5143e5e6a9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:30.832489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e6b18-be04-4e4e-8cbb-e7dffbc6c813 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:30.837824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af4a93-9627-4407-9143-163b2bccee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:30.840973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af4a93-9627-4407-9143-163b2bccee61 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:30.841246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af4a93-9627-4407-9143-163b2bccee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:30.841521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af4a93-9627-4407-9143-163b2bccee61 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:30.841874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc4ed5-13a1-402c-8731-d42f11163356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:30.842212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af4a93-9627-4407-9143-163b2bccee61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:30.842841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc4ed5-13a1-402c-8731-d42f11163356 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:30.843286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc4ed5-13a1-402c-8731-d42f11163356 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:30.843698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc4ed5-13a1-402c-8731-d42f11163356 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:30.844896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc4ed5-13a1-402c-8731-d42f11163356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:30.847858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc4ed5-13a1-402c-8731-d42f11163356 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:30.849693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af4a93-9627-4407-9143-163b2bccee61 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:30.854200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:30.876487 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1197/2368] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:30.877349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af4a93-9627-4407-9143-163b2bccee61 admin admin] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:56:30.879128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-71af4a93-9627-4407-9143-163b2bccee61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:30.904609 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1172/2369] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:30 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:30.905583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc4ed5-13a1-402c-8731-d42f11163356 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:31.014065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af4a93-9627-4407-9143-163b2bccee61 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:31.016881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-71af4a93-9627-4407-9143-163b2bccee61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:31.106917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc4ed5-13a1-402c-8731-d42f11163356 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:31.121147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36dd16-35b9-4d52-9d87-6709e48c037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:31.121838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36dd16-35b9-4d52-9d87-6709e48c037a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:31.122231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36dd16-35b9-4d52-9d87-6709e48c037a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:31.122503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36dd16-35b9-4d52-9d87-6709e48c037a admin admin] PATH_INFO: `/v3/projects/48933280997c4d4c89db090da903cd9d/users/80a2eb3197d8498a99450b7a6e78d4df/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:31.125148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af4a93-9627-4407-9143-163b2bccee61 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:31.179313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36dd16-35b9-4d52-9d87-6709e48c037a admin admin] RBAC: Authorizing `identity:create_grant(project_id=48933280997c4d4c89db090da903cd9d, user_id=80a2eb3197d8498a99450b7a6e78d4df, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:31.182378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36dd16-35b9-4d52-9d87-6709e48c037a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:31.190498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:31.216995 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1173/2370] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:30 2022] PUT /identity/v3/projects/48933280997c4d4c89db090da903cd9d/users/80a2eb3197d8498a99450b7a6e78d4df/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:31.218742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36dd16-35b9-4d52-9d87-6709e48c037a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:31.253740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7f0e2-f884-4213-b721-af991bad4ec6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c6feb9eaf0447dbce898b2629c05a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aebea2bc737942fc899aa04e350f2936', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:31.254539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7f0e2-f884-4213-b721-af991bad4ec6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:31.254813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7f0e2-f884-4213-b721-af991bad4ec6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:31.255076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7f0e2-f884-4213-b721-af991bad4ec6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] PATH_INFO: `/v3/projects/386f25e9b7774734a1cc5e5899668848` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:31.257117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7f0e2-f884-4213-b721-af991bad4ec6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorizing `identity:get_project(project_id=386f25e9b7774734a1cc5e5899668848)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:31.259787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7f0e2-f884-4213-b721-af991bad4ec6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:31.269203 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1198/2371] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:56:30 2022] GET /identity/v3/projects/386f25e9b7774734a1cc5e5899668848 => generated 371 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:31.275972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7f0e2-f884-4213-b721-af991bad4ec6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:31.317681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7f0e2-f884-4213-b721-af991bad4ec6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:31.372153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36dd16-35b9-4d52-9d87-6709e48c037a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:31.395879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b42ae5-04c2-4d12-a773-acf6680e9a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:31.396692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b42ae5-04c2-4d12-a773-acf6680e9a16 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:31.397037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b42ae5-04c2-4d12-a773-acf6680e9a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:31.398233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b42ae5-04c2-4d12-a773-acf6680e9a16 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:31.399007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b42ae5-04c2-4d12-a773-acf6680e9a16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:31.401195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b42ae5-04c2-4d12-a773-acf6680e9a16 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:31.445113 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1174/2372] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:31 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:31.446572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b42ae5-04c2-4d12-a773-acf6680e9a16 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:31.474502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b42ae5-04c2-4d12-a773-acf6680e9a16 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:31.483745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ae59d-19c6-4ebc-8380-629d4ae80a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:31.486744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ae59d-19c6-4ebc-8380-629d4ae80a26 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:31.487267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ae59d-19c6-4ebc-8380-629d4ae80a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:31.487807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ae59d-19c6-4ebc-8380-629d4ae80a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:31.488733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ae59d-19c6-4ebc-8380-629d4ae80a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:31.491606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ae59d-19c6-4ebc-8380-629d4ae80a26 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:31.509310 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1175/2373] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:31.513305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ae59d-19c6-4ebc-8380-629d4ae80a26 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:31.522304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d124cf-0c17-4575-a584-f32177612ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:31.522968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d124cf-0c17-4575-a584-f32177612ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:31.523246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d124cf-0c17-4575-a584-f32177612ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:31.524332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d124cf-0c17-4575-a584-f32177612ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:31.524332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d124cf-0c17-4575-a584-f32177612ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:31.525934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d124cf-0c17-4575-a584-f32177612ae5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:31.529419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ae59d-19c6-4ebc-8380-629d4ae80a26 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:31.536087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa914e1-2b9f-4074-b277-2e56674793b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:31.537126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa914e1-2b9f-4074-b277-2e56674793b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:31.537891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa914e1-2b9f-4074-b277-2e56674793b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:31.538370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa914e1-2b9f-4074-b277-2e56674793b8 admin admin] PATH_INFO: `/v3/projects/cd9c82d4785244778d540e0dd1f358b6/users/5b9243cca176412d8ad64e6a610c2547/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:31.539737 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1199/2374] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:31.541314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddca67f-1deb-4060-a69a-1d77af449517 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:31.541594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddca67f-1deb-4060-a69a-1d77af449517 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:31.541855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddca67f-1deb-4060-a69a-1d77af449517 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:31.548602 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1200/2375] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:56:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:56:31.549672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddca67f-1deb-4060-a69a-1d77af449517 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:31.563184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa914e1-2b9f-4074-b277-2e56674793b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd9c82d4785244778d540e0dd1f358b6, user_id=5b9243cca176412d8ad64e6a610c2547, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:31.564216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddca67f-1deb-4060-a69a-1d77af449517 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:31.566859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa914e1-2b9f-4074-b277-2e56674793b8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:31.569618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d169a5-fde7-457f-80f6-85787a7b1a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:31.570787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d169a5-fde7-457f-80f6-85787a7b1a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:31.571126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d169a5-fde7-457f-80f6-85787a7b1a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:31.571682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d169a5-fde7-457f-80f6-85787a7b1a2d admin admin] PATH_INFO: `/v3/projects/48933280997c4d4c89db090da903cd9d/users/80a2eb3197d8498a99450b7a6e78d4df/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:31.574287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d169a5-fde7-457f-80f6-85787a7b1a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=48933280997c4d4c89db090da903cd9d, user_id=80a2eb3197d8498a99450b7a6e78d4df, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:31.577793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d169a5-fde7-457f-80f6-85787a7b1a2d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:31.592903 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1201/2376] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:31 2022] PUT /identity/v3/projects/48933280997c4d4c89db090da903cd9d/users/80a2eb3197d8498a99450b7a6e78d4df/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:31.594166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d169a5-fde7-457f-80f6-85787a7b1a2d admin admin] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:56:31.611054 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1176/2377] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:31 2022] PUT /identity/v3/projects/cd9c82d4785244778d540e0dd1f358b6/users/5b9243cca176412d8ad64e6a610c2547/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:56:31.613227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757e0da-6567-43c2-b9eb-af983193babf None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:31.613805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757e0da-6567-43c2-b9eb-af983193babf None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:31.614280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757e0da-6567-43c2-b9eb-af983193babf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:31.660758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3757e0da-6567-43c2-b9eb-af983193babf None None] MFA Rules not processed for user `80a2eb3197d8498a99450b7a6e78d4df`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:31.663099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3757e0da-6567-43c2-b9eb-af983193babf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:31.685446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:31.761568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d169a5-fde7-457f-80f6-85787a7b1a2d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:31.784913 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1177/2378] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:31.784913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757e0da-6567-43c2-b9eb-af983193babf None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:31.885377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757e0da-6567-43c2-b9eb-af983193babf None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:31.885998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:31.910956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d7574-85ff-4df3-b0cc-98cfb0055c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:31.911517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d7574-85ff-4df3-b0cc-98cfb0055c30 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:31.911769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d7574-85ff-4df3-b0cc-98cfb0055c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:31.911940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d169a5-fde7-457f-80f6-85787a7b1a2d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:31.912331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d7574-85ff-4df3-b0cc-98cfb0055c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:31.913114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d7574-85ff-4df3-b0cc-98cfb0055c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:31.916047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d7574-85ff-4df3-b0cc-98cfb0055c30 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:31.921722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d5731-aa50-4fa5-863b-031be32d011b tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c6feb9eaf0447dbce898b2629c05a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aebea2bc737942fc899aa04e350f2936', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:31.922427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d5731-aa50-4fa5-863b-031be32d011b tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:31.922784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d5731-aa50-4fa5-863b-031be32d011b tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:31.923225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d5731-aa50-4fa5-863b-031be32d011b tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] PATH_INFO: `/v3/projects/386f25e9b7774734a1cc5e5899668848` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:31.925678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d5731-aa50-4fa5-863b-031be32d011b tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorizing `identity:get_project(project_id=386f25e9b7774734a1cc5e5899668848)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:31.930819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d5731-aa50-4fa5-863b-031be32d011b tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:31.934723 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1202/2379] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:56:31 2022] GET /identity/v3/projects/386f25e9b7774734a1cc5e5899668848 => generated 371 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:31.938371 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1178/2380] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:31.942128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d5731-aa50-4fa5-863b-031be32d011b tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:31.944099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d7574-85ff-4df3-b0cc-98cfb0055c30 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:31.946160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7e4d5731-aa50-4fa5-863b-031be32d011b tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:31.978918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d5731-aa50-4fa5-863b-031be32d011b tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:31.979393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d7574-85ff-4df3-b0cc-98cfb0055c30 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:31.986089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66be15-4c5e-4d99-8dc8-d098850971ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:31.986420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da985a16-6e8b-4e92-b196-875ffa37c408 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:31.986785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66be15-4c5e-4d99-8dc8-d098850971ca admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:31.987014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da985a16-6e8b-4e92-b196-875ffa37c408 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:31.987214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66be15-4c5e-4d99-8dc8-d098850971ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:31.987336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da985a16-6e8b-4e92-b196-875ffa37c408 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:31.987490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66be15-4c5e-4d99-8dc8-d098850971ca admin admin] PATH_INFO: `/v3/projects/cd9c82d4785244778d540e0dd1f358b6/users/5b9243cca176412d8ad64e6a610c2547/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:31.987587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da985a16-6e8b-4e92-b196-875ffa37c408 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:31.992201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec66be15-4c5e-4d99-8dc8-d098850971ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd9c82d4785244778d540e0dd1f358b6, user_id=5b9243cca176412d8ad64e6a610c2547, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:31.995117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec66be15-4c5e-4d99-8dc8-d098850971ca admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:32.005621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da985a16-6e8b-4e92-b196-875ffa37c408 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:32.005621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da985a16-6e8b-4e92-b196-875ffa37c408 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:32.016188 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1179/2381] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:31 2022] PUT /identity/v3/projects/cd9c82d4785244778d540e0dd1f358b6/users/5b9243cca176412d8ad64e6a610c2547/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:32.018094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66be15-4c5e-4d99-8dc8-d098850971ca admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:32.020574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ec66be15-4c5e-4d99-8dc8-d098850971ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:32.029670 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1203/2382] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:32.031651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da985a16-6e8b-4e92-b196-875ffa37c408 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:32.059311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da985a16-6e8b-4e92-b196-875ffa37c408 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:32.142646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66be15-4c5e-4d99-8dc8-d098850971ca admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:32.167183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:32.168144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:32.168319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:32.169235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] PATH_INFO: `/v3/projects/a1d90d46eb0f45688c3dc536a480c559` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:32.171604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1d90d46eb0f45688c3dc536a480c559)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:32.177394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:32.183844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1bb6f-9813-4e36-9b3d-92c053450692 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c6feb9eaf0447dbce898b2629c05a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aebea2bc737942fc899aa04e350f2936', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:32.185007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1bb6f-9813-4e36-9b3d-92c053450692 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:32.185535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1bb6f-9813-4e36-9b3d-92c053450692 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:32.186166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1bb6f-9813-4e36-9b3d-92c053450692 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:32.187072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1bb6f-9813-4e36-9b3d-92c053450692 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:32.187578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1bb6f-9813-4e36-9b3d-92c053450692 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:32.190853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1bb6f-9813-4e36-9b3d-92c053450692 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:32.218345 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1204/2383] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:32.228110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6917bf-04ae-4a5b-be7f-b3e7d3dcf7ab None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:32.228110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6917bf-04ae-4a5b-be7f-b3e7d3dcf7ab None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:32.228110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6917bf-04ae-4a5b-be7f-b3e7d3dcf7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:32.270150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1d6917bf-04ae-4a5b-be7f-b3e7d3dcf7ab None None] MFA Rules not processed for user `5b9243cca176412d8ad64e6a610c2547`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:32.271744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1d6917bf-04ae-4a5b-be7f-b3e7d3dcf7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:32.299245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:32.334727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1d90d46eb0f45688c3dc536a480c559', 'request_id': 'req-155c38ef-3c0d-4d01-b1cc-8221db3058e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:32.346168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1d90d46eb0f45688c3dc536a480c559', 'request_id': 'req-155c38ef-3c0d-4d01-b1cc-8221db3058e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:32.361910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1d90d46eb0f45688c3dc536a480c559', 'request_id': 'req-155c38ef-3c0d-4d01-b1cc-8221db3058e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:32.392753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] The token cache is being invalidate because project a1d90d46eb0f45688c3dc536a480c559 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:32.393060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:32.398257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:32.401381 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1180/2384] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:32 2022] DELETE /identity/v3/projects/a1d90d46eb0f45688c3dc536a480c559 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:32.403806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:32.411574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:32.463720 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1205/2385] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:32.465161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ee7f0-8d2b-4d27-af44-2ff8f0b61191 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:32.465514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ee7f0-8d2b-4d27-af44-2ff8f0b61191 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:32.467056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ee7f0-8d2b-4d27-af44-2ff8f0b61191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:32.509297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0d5ee7f0-8d2b-4d27-af44-2ff8f0b61191 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:32.510672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0d5ee7f0-8d2b-4d27-af44-2ff8f0b61191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:32.625464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c38ef-3c0d-4d01-b1cc-8221db3058e6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:32.634052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:32.638444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc2e35-6044-44da-b4ec-141ec63eb92e tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c6feb9eaf0447dbce898b2629c05a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aebea2bc737942fc899aa04e350f2936', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:32.640284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc2e35-6044-44da-b4ec-141ec63eb92e tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:32.640811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc2e35-6044-44da-b4ec-141ec63eb92e tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:32.641646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc2e35-6044-44da-b4ec-141ec63eb92e tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:32.643238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc2e35-6044-44da-b4ec-141ec63eb92e tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:32.650845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc2e35-6044-44da-b4ec-141ec63eb92e tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:32.656308 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1206/2386] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:32.657452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ee7f0-8d2b-4d27-af44-2ff8f0b61191 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:32.659718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0d5ee7f0-8d2b-4d27-af44-2ff8f0b61191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:32.689472 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1181/2387] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:32 2022] POST /identity/v3/users => generated 383 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:32.689472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bb642-6566-4179-b255-b143943d600a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:32.689472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bb642-6566-4179-b255-b143943d600a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:32.689889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bb642-6566-4179-b255-b143943d600a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:32.726587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1a1bb642-6566-4179-b255-b143943d600a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:32.729466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1a1bb642-6566-4179-b255-b143943d600a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:32.744567 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1182/2388] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:32.744567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bb642-6566-4179-b255-b143943d600a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:32.760648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bb642-6566-4179-b255-b143943d600a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:32.770172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c783a0-fbe9-4430-9ac8-2ac56a4abb73 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c6feb9eaf0447dbce898b2629c05a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aebea2bc737942fc899aa04e350f2936', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:32.770850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c783a0-fbe9-4430-9ac8-2ac56a4abb73 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:32.771121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c783a0-fbe9-4430-9ac8-2ac56a4abb73 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:32.771382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c783a0-fbe9-4430-9ac8-2ac56a4abb73 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:32.772045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c783a0-fbe9-4430-9ac8-2ac56a4abb73 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:32.772344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c783a0-fbe9-4430-9ac8-2ac56a4abb73 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:32.776069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c783a0-fbe9-4430-9ac8-2ac56a4abb73 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:32.805082 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1183/2389] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:32.805082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c783a0-fbe9-4430-9ac8-2ac56a4abb73 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:32.818163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b7c783a0-fbe9-4430-9ac8-2ac56a4abb73 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:32.861383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ee7f0-8d2b-4d27-af44-2ff8f0b61191 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:32.865297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c783a0-fbe9-4430-9ac8-2ac56a4abb73 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:32.870777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac96bc1-2981-4ec8-bda5-00390f011739 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:32.871407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac96bc1-2981-4ec8-bda5-00390f011739 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:32.871677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac96bc1-2981-4ec8-bda5-00390f011739 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:32.871951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac96bc1-2981-4ec8-bda5-00390f011739 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:32.875581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677b3a0-656f-469f-8f40-0fefdfdc9bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:32.891086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677b3a0-656f-469f-8f40-0fefdfdc9bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:32.891865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677b3a0-656f-469f-8f40-0fefdfdc9bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:32.893303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677b3a0-656f-469f-8f40-0fefdfdc9bc5 admin admin] PATH_INFO: `/v3/users/0f7a20d91493486789c123453ac5586b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:32.894978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677b3a0-656f-469f-8f40-0fefdfdc9bc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f7a20d91493486789c123453ac5586b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:32.900666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677b3a0-656f-469f-8f40-0fefdfdc9bc5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:32.990277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac96bc1-2981-4ec8-bda5-00390f011739 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:32.992262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac96bc1-2981-4ec8-bda5-00390f011739 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:33.009730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2677b3a0-656f-469f-8f40-0fefdfdc9bc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f7a20d91493486789c123453ac5586b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:33.024798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2677b3a0-656f-469f-8f40-0fefdfdc9bc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f7a20d91493486789c123453ac5586b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:33.036041 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1207/2390] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:33.038374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac96bc1-2981-4ec8-bda5-00390f011739 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:33.057422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2677b3a0-656f-469f-8f40-0fefdfdc9bc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f7a20d91493486789c123453ac5586b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:33.058411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2677b3a0-656f-469f-8f40-0fefdfdc9bc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f7a20d91493486789c123453ac5586b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:33.059722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2677b3a0-656f-469f-8f40-0fefdfdc9bc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f7a20d91493486789c123453ac5586b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:33.080252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac96bc1-2981-4ec8-bda5-00390f011739 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:33.080721 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1184/2391] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:32 2022] DELETE /identity/v3/users/0f7a20d91493486789c123453ac5586b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:33.082216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaf3dc-658f-463c-9b64-82ffbd1376e2 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:33.082586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaf3dc-658f-463c-9b64-82ffbd1376e2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:33.082949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaf3dc-658f-463c-9b64-82ffbd1376e2 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:33.090433 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1185/2392] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:56:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:56:33.093191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaf3dc-658f-463c-9b64-82ffbd1376e2 None None] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:56:33.095114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dddaf3dc-658f-463c-9b64-82ffbd1376e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:33.108487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=80,keystone:DELETE=15,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:33.268006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9d3ce-8003-4438-9acb-6c7eed181376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:33.268674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9d3ce-8003-4438-9acb-6c7eed181376 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:33.268950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9d3ce-8003-4438-9acb-6c7eed181376 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:33.269219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9d3ce-8003-4438-9acb-6c7eed181376 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:33.270052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9d3ce-8003-4438-9acb-6c7eed181376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:33.270355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9d3ce-8003-4438-9acb-6c7eed181376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:33.272353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9d3ce-8003-4438-9acb-6c7eed181376 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:33.291522 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1208/2393] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:33.291522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4c822-929e-46e9-91e2-7d158d8073cd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:33.291522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4c822-929e-46e9-91e2-7d158d8073cd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:33.291522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4c822-929e-46e9-91e2-7d158d8073cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:33.305599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaf3dc-658f-463c-9b64-82ffbd1376e2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:33.308865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dddaf3dc-658f-463c-9b64-82ffbd1376e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:33.316737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-27c4c822-929e-46e9-91e2-7d158d8073cd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:33.318101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-27c4c822-929e-46e9-91e2-7d158d8073cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:33.330440 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1209/2394] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:33.331336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4c822-929e-46e9-91e2-7d158d8073cd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:33.334828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-27c4c822-929e-46e9-91e2-7d158d8073cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:33.373075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4c822-929e-46e9-91e2-7d158d8073cd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:33.386960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053255a-2ac7-4d76-85f3-ed2d110ed424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:33.387694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053255a-2ac7-4d76-85f3-ed2d110ed424 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:33.388071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053255a-2ac7-4d76-85f3-ed2d110ed424 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:33.388607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053255a-2ac7-4d76-85f3-ed2d110ed424 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:33.389508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053255a-2ac7-4d76-85f3-ed2d110ed424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:33.392447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053255a-2ac7-4d76-85f3-ed2d110ed424 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:33.434905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:33.452229 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1210/2395] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:33 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:33.453973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053255a-2ac7-4d76-85f3-ed2d110ed424 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:33.456673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaf3dc-658f-463c-9b64-82ffbd1376e2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:33.456842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1053255a-2ac7-4d76-85f3-ed2d110ed424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:33.539272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053255a-2ac7-4d76-85f3-ed2d110ed424 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:33.543231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1cdfc-f71a-4bac-8ab4-f3781c4cadb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:33.544026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1cdfc-f71a-4bac-8ab4-f3781c4cadb5 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:33.544341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1cdfc-f71a-4bac-8ab4-f3781c4cadb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:33.545208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1cdfc-f71a-4bac-8ab4-f3781c4cadb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:33.547331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8774ab4-572a-48fc-9485-70278a58b555 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c6feb9eaf0447dbce898b2629c05a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aebea2bc737942fc899aa04e350f2936', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:33.548580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8774ab4-572a-48fc-9485-70278a58b555 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:33.549101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8774ab4-572a-48fc-9485-70278a58b555 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:33.549562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8774ab4-572a-48fc-9485-70278a58b555 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] PATH_INFO: `/v3/projects/386f25e9b7774734a1cc5e5899668848` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:33.551068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8774ab4-572a-48fc-9485-70278a58b555 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorizing `identity:get_project(project_id=386f25e9b7774734a1cc5e5899668848)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:33.554764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8774ab4-572a-48fc-9485-70278a58b555 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:33.557111 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1186/2396] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:56:33 2022] GET /identity/v3/projects/386f25e9b7774734a1cc5e5899668848 => generated 371 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:33.558564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8774ab4-572a-48fc-9485-70278a58b555 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:33.561113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b8774ab4-572a-48fc-9485-70278a58b555 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:33.638322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1cdfc-f71a-4bac-8ab4-f3781c4cadb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:33.639363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:33.642608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1cdfc-f71a-4bac-8ab4-f3781c4cadb5 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:33.658797 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1211/2397] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:33.660231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1cdfc-f71a-4bac-8ab4-f3781c4cadb5 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:33.672986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1cdfc-f71a-4bac-8ab4-f3781c4cadb5 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:33.677498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4b666-6e24-4758-966e-7382cfc145b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:33.678122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4b666-6e24-4758-966e-7382cfc145b6 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:33.678397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4b666-6e24-4758-966e-7382cfc145b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:33.678890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4b666-6e24-4758-966e-7382cfc145b6 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:33.679444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8774ab4-572a-48fc-9485-70278a58b555 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:33.679785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4b666-6e24-4758-966e-7382cfc145b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:33.682216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4b666-6e24-4758-966e-7382cfc145b6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:33.695984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4fb27-77ea-43b3-9972-63dfac540e4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:33.695984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4fb27-77ea-43b3-9972-63dfac540e4f service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:33.696340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4fb27-77ea-43b3-9972-63dfac540e4f service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:33.696340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4fb27-77ea-43b3-9972-63dfac540e4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:33.698137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4ff4fb27-77ea-43b3-9972-63dfac540e4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:33.739679 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1212/2398] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:33 2022] POST /identity/v3/users => generated 286 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:33.744019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4b666-6e24-4758-966e-7382cfc145b6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:33.765224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4b666-6e24-4758-966e-7382cfc145b6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:33.781575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf770b-bb4f-4677-8d37-68e9f85aa105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:33.782273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf770b-bb4f-4677-8d37-68e9f85aa105 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:33.782784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf770b-bb4f-4677-8d37-68e9f85aa105 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:33.782784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf770b-bb4f-4677-8d37-68e9f85aa105 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:33.788079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf770b-bb4f-4677-8d37-68e9f85aa105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:33.790283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf770b-bb4f-4677-8d37-68e9f85aa105 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:33.814898 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1213/2399] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:33.815612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4fb27-77ea-43b3-9972-63dfac540e4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:33.818960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf770b-bb4f-4677-8d37-68e9f85aa105 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:33.821369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4fb27-77ea-43b3-9972-63dfac540e4f service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:33.841259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf770b-bb4f-4677-8d37-68e9f85aa105 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:33.843783 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1187/2400] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:33.853888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b4599-800a-40ff-9044-abf803c8627d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:33.855212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b4599-800a-40ff-9044-abf803c8627d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:33.855490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b4599-800a-40ff-9044-abf803c8627d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:33.855757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b4599-800a-40ff-9044-abf803c8627d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:33.857108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b4599-800a-40ff-9044-abf803c8627d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:33.863981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b4599-800a-40ff-9044-abf803c8627d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:33.910499 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1214/2401] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:33.913115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d829873-2d13-4e80-b6a3-5b07b73d8f31 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:33.913964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d829873-2d13-4e80-b6a3-5b07b73d8f31 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:33.914468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d829873-2d13-4e80-b6a3-5b07b73d8f31 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:33.915527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b4599-800a-40ff-9044-abf803c8627d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:33.917138 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1188/2402] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:56:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:56:33.924037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d829873-2d13-4e80-b6a3-5b07b73d8f31 None None] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:56:33.941153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b4599-800a-40ff-9044-abf803c8627d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:33.954256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee5d2d-04bf-48c1-9577-141eb7685801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:33.955111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee5d2d-04bf-48c1-9577-141eb7685801 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:33.955385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee5d2d-04bf-48c1-9577-141eb7685801 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:33.955813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee5d2d-04bf-48c1-9577-141eb7685801 admin admin] PATH_INFO: `/v3/projects/ea18513aca0a4723b3ae3043624a52da/users/8afc45f50dc54b84a8abf3b6e168426b/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:33.961520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d829873-2d13-4e80-b6a3-5b07b73d8f31 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:33.987922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee5d2d-04bf-48c1-9577-141eb7685801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea18513aca0a4723b3ae3043624a52da, user_id=8afc45f50dc54b84a8abf3b6e168426b, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:33.988128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d829873-2d13-4e80-b6a3-5b07b73d8f31 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:33.991626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee5d2d-04bf-48c1-9577-141eb7685801 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:33.999105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e128c3-5754-4aa7-949a-69707f06be99 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c6feb9eaf0447dbce898b2629c05a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aebea2bc737942fc899aa04e350f2936', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:34.000088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e128c3-5754-4aa7-949a-69707f06be99 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:34.000530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e128c3-5754-4aa7-949a-69707f06be99 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:34.001016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e128c3-5754-4aa7-949a-69707f06be99 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] PATH_INFO: `/v3/projects/386f25e9b7774734a1cc5e5899668848` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:34.004585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e128c3-5754-4aa7-949a-69707f06be99 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorizing `identity:get_project(project_id=386f25e9b7774734a1cc5e5899668848)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:34.011253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e128c3-5754-4aa7-949a-69707f06be99 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:34.016037 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1189/2403] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:56:33 2022] GET /identity/v3/projects/386f25e9b7774734a1cc5e5899668848 => generated 371 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:56:34.040846 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1215/2404] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:33 2022] PUT /identity/v3/projects/ea18513aca0a4723b3ae3043624a52da/users/8afc45f50dc54b84a8abf3b6e168426b/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:56:34.046326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ef47d-6e5f-43fb-a59d-638c3d388fad None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:34.047268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ef47d-6e5f-43fb-a59d-638c3d388fad None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:34.048027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ef47d-6e5f-43fb-a59d-638c3d388fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:34.076058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee5d2d-04bf-48c1-9577-141eb7685801 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:34.095380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bf2ef47d-6e5f-43fb-a59d-638c3d388fad None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:34.097822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bf2ef47d-6e5f-43fb-a59d-638c3d388fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:34.239307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:34.265052 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1190/2405] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:34.293826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee5d2d-04bf-48c1-9577-141eb7685801 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:34.298121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ef47d-6e5f-43fb-a59d-638c3d388fad None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:34.307703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c6feb9eaf0447dbce898b2629c05a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aebea2bc737942fc899aa04e350f2936', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:34.312069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:34.313534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:34.317856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] PATH_INFO: `/v3/users/dbd2ef2fcafb4858bcc4aa108f6f762c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:34.369392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorizing `identity:delete_user(user_id=dbd2ef2fcafb4858bcc4aa108f6f762c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:34.372652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:34.556322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbd2ef2fcafb4858bcc4aa108f6f762c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:34.568707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ef47d-6e5f-43fb-a59d-638c3d388fad None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:34.580062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbd2ef2fcafb4858bcc4aa108f6f762c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:34.581112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cf6ed-2b5d-49fe-abfa-f4f32ac261ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:34.583364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cf6ed-2b5d-49fe-abfa-f4f32ac261ce service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:34.583710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cf6ed-2b5d-49fe-abfa-f4f32ac261ce service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:34.584211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cf6ed-2b5d-49fe-abfa-f4f32ac261ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:34.599369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbd2ef2fcafb4858bcc4aa108f6f762c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:34.607582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8cf6ed-2b5d-49fe-abfa-f4f32ac261ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:34.609536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8cf6ed-2b5d-49fe-abfa-f4f32ac261ce service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:34.629830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbd2ef2fcafb4858bcc4aa108f6f762c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:34.641209 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1191/2406] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:56:34.642866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbd2ef2fcafb4858bcc4aa108f6f762c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:34.643834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cf6ed-2b5d-49fe-abfa-f4f32ac261ce service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:34.657317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4f8cf6ed-2b5d-49fe-abfa-f4f32ac261ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:34.665643 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1216/2407] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:34 2022] DELETE /identity/v3/users/dbd2ef2fcafb4858bcc4aa108f6f762c => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:34.669492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:34.673020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cf6ed-2b5d-49fe-abfa-f4f32ac261ce service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:34.674177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:34.687416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:34.756855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:34.760326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:34.760326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:34.760326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] PATH_INFO: `/v3/projects/5ccd8386725c4a328bbe4ff346d8aa52` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:34.762431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ccd8386725c4a328bbe4ff346d8aa52)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:34.765473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:34.811978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc3fbb-03e7-4ff2-a949-c1a70f6db759 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:34.828194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:34.831106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:34.831106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:34.831106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] PATH_INFO: `/v3/users/861c813d6f5a42e3adbbe1e74ef1873a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:34.835484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=861c813d6f5a42e3adbbe1e74ef1873a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:34.839556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:34.844141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ccd8386725c4a328bbe4ff346d8aa52', 'request_id': 'req-c7ca89d3-16ac-4e0f-9d58-352989570dca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:34.853904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ccd8386725c4a328bbe4ff346d8aa52', 'request_id': 'req-c7ca89d3-16ac-4e0f-9d58-352989570dca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:34.868002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ccd8386725c4a328bbe4ff346d8aa52', 'request_id': 'req-c7ca89d3-16ac-4e0f-9d58-352989570dca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:34.897098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] The token cache is being invalidate because project 5ccd8386725c4a328bbe4ff346d8aa52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:34.898906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:34.900675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:34.906242 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1192/2408] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:34 2022] DELETE /identity/v3/projects/5ccd8386725c4a328bbe4ff346d8aa52 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:34.907690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:34.910956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:34.965585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '861c813d6f5a42e3adbbe1e74ef1873a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:34.982610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '861c813d6f5a42e3adbbe1e74ef1873a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:34.984789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:34.994257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '861c813d6f5a42e3adbbe1e74ef1873a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.014058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '861c813d6f5a42e3adbbe1e74ef1873a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.020928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '861c813d6f5a42e3adbbe1e74ef1873a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.038944 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1217/2409] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:34 2022] DELETE /identity/v3/users/861c813d6f5a42e3adbbe1e74ef1873a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:35.039080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca89d3-16ac-4e0f-9d58-352989570dca admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:35.040282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:35.044444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:35.117183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:35.128821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82864efc-037b-45e5-93b0-b82e74f7b0d6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c6feb9eaf0447dbce898b2629c05a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aebea2bc737942fc899aa04e350f2936', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:35.130178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82864efc-037b-45e5-93b0-b82e74f7b0d6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:35.130456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82864efc-037b-45e5-93b0-b82e74f7b0d6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:35.130941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82864efc-037b-45e5-93b0-b82e74f7b0d6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] PATH_INFO: `/v3/projects/386f25e9b7774734a1cc5e5899668848` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:35.132434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82864efc-037b-45e5-93b0-b82e74f7b0d6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorizing `identity:delete_project(project_id=386f25e9b7774734a1cc5e5899668848)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:35.136800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82864efc-037b-45e5-93b0-b82e74f7b0d6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:35.192690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48743884-e667-4733-8fbc-f41ca8f324e9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:35.210699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82864efc-037b-45e5-93b0-b82e74f7b0d6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '386f25e9b7774734a1cc5e5899668848', 'request_id': 'req-82864efc-037b-45e5-93b0-b82e74f7b0d6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.224587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82864efc-037b-45e5-93b0-b82e74f7b0d6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '386f25e9b7774734a1cc5e5899668848', 'request_id': 'req-82864efc-037b-45e5-93b0-b82e74f7b0d6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.248453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82864efc-037b-45e5-93b0-b82e74f7b0d6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '386f25e9b7774734a1cc5e5899668848', 'request_id': 'req-82864efc-037b-45e5-93b0-b82e74f7b0d6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.281904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82864efc-037b-45e5-93b0-b82e74f7b0d6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] The token cache is being invalidate because project 386f25e9b7774734a1cc5e5899668848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:35.281904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82864efc-037b-45e5-93b0-b82e74f7b0d6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.284562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82864efc-037b-45e5-93b0-b82e74f7b0d6 tempest-QuotasAdminTestV257-1791068103 tempest-QuotasAdminTestV257-1791068103-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.294091 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1193/2410] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:34 2022] DELETE /identity/v3/projects/386f25e9b7774734a1cc5e5899668848 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:35.294091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccb599-bcd2-4700-b266-d663f8fb342e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:35.294529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccb599-bcd2-4700-b266-d663f8fb342e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:35.296038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccb599-bcd2-4700-b266-d663f8fb342e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:35.309875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:35.310528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:35.310706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:35.310975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] PATH_INFO: `/v3/users/80a2eb3197d8498a99450b7a6e78d4df` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:35.313572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80a2eb3197d8498a99450b7a6e78d4df)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:35.320176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:35.340011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f2ccb599-bcd2-4700-b266-d663f8fb342e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:35.343769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f2ccb599-bcd2-4700-b266-d663f8fb342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:35.365123 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1194/2411] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:35.366482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccb599-bcd2-4700-b266-d663f8fb342e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:35.379362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f2ccb599-bcd2-4700-b266-d663f8fb342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:35.406464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccb599-bcd2-4700-b266-d663f8fb342e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:35.414147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80a2eb3197d8498a99450b7a6e78d4df'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.424212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09770a-9b81-40cc-8692-251bbef8e45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:35.424700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80a2eb3197d8498a99450b7a6e78d4df'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.424897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09770a-9b81-40cc-8692-251bbef8e45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:35.425169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09770a-9b81-40cc-8692-251bbef8e45b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:35.425525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09770a-9b81-40cc-8692-251bbef8e45b admin admin] PATH_INFO: `/v3/users/d7ca457b2b5140bfb3944fc6ff48ed9e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:35.426829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09770a-9b81-40cc-8692-251bbef8e45b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7ca457b2b5140bfb3944fc6ff48ed9e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:35.429638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09770a-9b81-40cc-8692-251bbef8e45b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:35.436690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80a2eb3197d8498a99450b7a6e78d4df'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.455519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80a2eb3197d8498a99450b7a6e78d4df'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.458368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80a2eb3197d8498a99450b7a6e78d4df'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.461539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:35.469236 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1218/2412] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:35 2022] DELETE /identity/v3/users/80a2eb3197d8498a99450b7a6e78d4df => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:35.470655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:35.475232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:35.523520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0d09770a-9b81-40cc-8692-251bbef8e45b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7ca457b2b5140bfb3944fc6ff48ed9e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.555939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0d09770a-9b81-40cc-8692-251bbef8e45b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7ca457b2b5140bfb3944fc6ff48ed9e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.579096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0d09770a-9b81-40cc-8692-251bbef8e45b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7ca457b2b5140bfb3944fc6ff48ed9e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.582887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0d09770a-9b81-40cc-8692-251bbef8e45b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7ca457b2b5140bfb3944fc6ff48ed9e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.583550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0d09770a-9b81-40cc-8692-251bbef8e45b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7ca457b2b5140bfb3944fc6ff48ed9e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:35.600236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f610262-9d93-4c9a-8ec5-e50dc1b66d75 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:35.606610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2593d-89c1-4ab2-b61a-0461cfe588df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:35.610145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2593d-89c1-4ab2-b61a-0461cfe588df service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:35.610145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2593d-89c1-4ab2-b61a-0461cfe588df service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:35.610145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2593d-89c1-4ab2-b61a-0461cfe588df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:35.610496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-38e2593d-89c1-4ab2-b61a-0461cfe588df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:35.626843 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1195/2413] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:35 2022] DELETE /identity/v3/users/d7ca457b2b5140bfb3944fc6ff48ed9e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:35.629175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1055ca5-baae-40cd-b346-0cc5679b38e7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:35.629569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1055ca5-baae-40cd-b346-0cc5679b38e7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:35.629901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1055ca5-baae-40cd-b346-0cc5679b38e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:35.663230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c1055ca5-baae-40cd-b346-0cc5679b38e7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:35.664292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c1055ca5-baae-40cd-b346-0cc5679b38e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:35.723708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2593d-89c1-4ab2-b61a-0461cfe588df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:35.725531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2593d-89c1-4ab2-b61a-0461cfe588df service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:35.744226 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1219/2414] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:35.760012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:35.789156 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1196/2415] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:35.808575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2593d-89c1-4ab2-b61a-0461cfe588df service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:35.811655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-38e2593d-89c1-4ab2-b61a-0461cfe588df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:35.838561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2593d-89c1-4ab2-b61a-0461cfe588df service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:35.914180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72358e2c-be90-46d5-8510-1185a1657a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:35.915054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72358e2c-be90-46d5-8510-1185a1657a5d service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:35.915379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72358e2c-be90-46d5-8510-1185a1657a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:35.915719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72358e2c-be90-46d5-8510-1185a1657a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:35.930405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72358e2c-be90-46d5-8510-1185a1657a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:35.931917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72358e2c-be90-46d5-8510-1185a1657a5d service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:35.944209 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1220/2416] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:36.694469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1055ca5-baae-40cd-b346-0cc5679b38e7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:36.697310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c1055ca5-baae-40cd-b346-0cc5679b38e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:36.719158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1055ca5-baae-40cd-b346-0cc5679b38e7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:36.734510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415af7c-ef17-44ca-91bb-612501cae0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:36.735313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415af7c-ef17-44ca-91bb-612501cae0b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:36.735617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415af7c-ef17-44ca-91bb-612501cae0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:36.735880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415af7c-ef17-44ca-91bb-612501cae0b7 admin admin] PATH_INFO: `/v3/projects/48933280997c4d4c89db090da903cd9d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:36.738211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415af7c-ef17-44ca-91bb-612501cae0b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48933280997c4d4c89db090da903cd9d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:36.740418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415af7c-ef17-44ca-91bb-612501cae0b7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:36.839670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4415af7c-ef17-44ca-91bb-612501cae0b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48933280997c4d4c89db090da903cd9d', 'request_id': 'req-4415af7c-ef17-44ca-91bb-612501cae0b7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:36.849625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4415af7c-ef17-44ca-91bb-612501cae0b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48933280997c4d4c89db090da903cd9d', 'request_id': 'req-4415af7c-ef17-44ca-91bb-612501cae0b7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:36.867255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4415af7c-ef17-44ca-91bb-612501cae0b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48933280997c4d4c89db090da903cd9d', 'request_id': 'req-4415af7c-ef17-44ca-91bb-612501cae0b7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:36.877237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72358e2c-be90-46d5-8510-1185a1657a5d service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:36.879089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-72358e2c-be90-46d5-8510-1185a1657a5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:36.898605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4415af7c-ef17-44ca-91bb-612501cae0b7 admin admin] The token cache is being invalidate because project 48933280997c4d4c89db090da903cd9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:36.898876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4415af7c-ef17-44ca-91bb-612501cae0b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:36.899608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4415af7c-ef17-44ca-91bb-612501cae0b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:36.901750 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1197/2417] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:36 2022] DELETE /identity/v3/projects/48933280997c4d4c89db090da903cd9d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:36.906523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1e0f0-0859-4aed-9255-c97b344f1bf2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:36.906894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1e0f0-0859-4aed-9255-c97b344f1bf2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:36.907107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1e0f0-0859-4aed-9255-c97b344f1bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:36.913031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72358e2c-be90-46d5-8510-1185a1657a5d service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:36.951181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ffa1e0f0-0859-4aed-9255-c97b344f1bf2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:36.955848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ffa1e0f0-0859-4aed-9255-c97b344f1bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:37.047720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:37.070350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf8f35-f518-4cab-bec7-395a32445297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:37.070923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf8f35-f518-4cab-bec7-395a32445297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:37.071231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf8f35-f518-4cab-bec7-395a32445297 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:37.071449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf8f35-f518-4cab-bec7-395a32445297 admin admin] PATH_INFO: `/v3/projects/3f7229da1351452f8faa09c632d41b77` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:37.073095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf8f35-f518-4cab-bec7-395a32445297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f7229da1351452f8faa09c632d41b77)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:37.075014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf8f35-f518-4cab-bec7-395a32445297 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:37.098717 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1198/2418] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:37.100496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1e0f0-0859-4aed-9255-c97b344f1bf2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:37.102726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ffa1e0f0-0859-4aed-9255-c97b344f1bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:37.116593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1e0f0-0859-4aed-9255-c97b344f1bf2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:37.125961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:37.127766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:37.128256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:37.128806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] PATH_INFO: `/v3/projects/edddaed1c4774c3c815a69383b963535` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:37.132931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=edddaed1c4774c3c815a69383b963535)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:37.137061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:37.195019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-55cf8f35-f518-4cab-bec7-395a32445297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f7229da1351452f8faa09c632d41b77', 'request_id': 'req-55cf8f35-f518-4cab-bec7-395a32445297'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.208463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-55cf8f35-f518-4cab-bec7-395a32445297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f7229da1351452f8faa09c632d41b77', 'request_id': 'req-55cf8f35-f518-4cab-bec7-395a32445297'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.214766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edddaed1c4774c3c815a69383b963535', 'request_id': 'req-ba7c21b7-d620-4799-8388-a7cb73d82c3a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.220309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-55cf8f35-f518-4cab-bec7-395a32445297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f7229da1351452f8faa09c632d41b77', 'request_id': 'req-55cf8f35-f518-4cab-bec7-395a32445297'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.223357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edddaed1c4774c3c815a69383b963535', 'request_id': 'req-ba7c21b7-d620-4799-8388-a7cb73d82c3a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.235950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edddaed1c4774c3c815a69383b963535', 'request_id': 'req-ba7c21b7-d620-4799-8388-a7cb73d82c3a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.239739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-55cf8f35-f518-4cab-bec7-395a32445297 admin admin] The token cache is being invalidate because project 3f7229da1351452f8faa09c632d41b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:37.240039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-55cf8f35-f518-4cab-bec7-395a32445297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.240822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-55cf8f35-f518-4cab-bec7-395a32445297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.245764 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1221/2419] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:36 2022] DELETE /identity/v3/projects/3f7229da1351452f8faa09c632d41b77 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:37.248530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c8657-89bb-4474-a0ab-1ea91eea1455 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:37.248806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c8657-89bb-4474-a0ab-1ea91eea1455 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:37.249670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:37.250188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c8657-89bb-4474-a0ab-1ea91eea1455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:37.260000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] The token cache is being invalidate because project edddaed1c4774c3c815a69383b963535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:37.260754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.263666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.268633 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1199/2420] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:37 2022] DELETE /identity/v3/projects/edddaed1c4774c3c815a69383b963535 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:37.270065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:37.274239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:37.294215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ba8c8657-89bb-4474-a0ab-1ea91eea1455 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:37.295450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ba8c8657-89bb-4474-a0ab-1ea91eea1455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:37.373541 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1222/2421] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:37.374537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c8657-89bb-4474-a0ab-1ea91eea1455 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:37.376139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c21b7-d620-4799-8388-a7cb73d82c3a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:37.376841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ba8c8657-89bb-4474-a0ab-1ea91eea1455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:37.381857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:37.382763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:37.383478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:37.383820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] PATH_INFO: `/v3/users/51c6feb9eaf0447dbce898b2629c05a2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:37.385735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c6feb9eaf0447dbce898b2629c05a2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:37.388627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:37.390677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c8657-89bb-4474-a0ab-1ea91eea1455 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:37.396237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:37.396938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:37.409625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:37.410077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] PATH_INFO: `/v3/users/5b9243cca176412d8ad64e6a610c2547` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:37.413650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b9243cca176412d8ad64e6a610c2547)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:37.416010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:37.466250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c6feb9eaf0447dbce898b2629c05a2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.479664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c6feb9eaf0447dbce898b2629c05a2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.485039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b9243cca176412d8ad64e6a610c2547'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.498104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b9243cca176412d8ad64e6a610c2547'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.504041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c6feb9eaf0447dbce898b2629c05a2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.504946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c6feb9eaf0447dbce898b2629c05a2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.505766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c6feb9eaf0447dbce898b2629c05a2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.514132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b9243cca176412d8ad64e6a610c2547'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.534256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:37.538804 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1200/2422] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:37 2022] DELETE /identity/v3/users/51c6feb9eaf0447dbce898b2629c05a2 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:37.541177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b9243cca176412d8ad64e6a610c2547'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.543414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b9243cca176412d8ad64e6a610c2547'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:37.549370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:37.551564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:37.559477 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1223/2423] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:37 2022] DELETE /identity/v3/users/5b9243cca176412d8ad64e6a610c2547 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:37.702136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6d0df-32b0-4032-8aec-18df58cf5f50 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:37.729352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:37.805524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3faed-e33c-4308-b6d6-2a53b7682fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:37.809137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3faed-e33c-4308-b6d6-2a53b7682fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:37.809530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3faed-e33c-4308-b6d6-2a53b7682fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:37.809903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3faed-e33c-4308-b6d6-2a53b7682fe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:37.811659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3faed-e33c-4308-b6d6-2a53b7682fe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:37.813760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3faed-e33c-4308-b6d6-2a53b7682fe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:37.816906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3faed-e33c-4308-b6d6-2a53b7682fe6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:37.838838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:37.840017 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1201/2424] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:37.840432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:37.871244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab219a05-aee7-4248-905e-20300eb63377 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:37.878538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1923576-8369-490d-bc73-d04e00f567c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:37.879113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1923576-8369-490d-bc73-d04e00f567c1 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:37.879539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1923576-8369-490d-bc73-d04e00f567c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:37.879818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1923576-8369-490d-bc73-d04e00f567c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:37.880486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1923576-8369-490d-bc73-d04e00f567c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:37.883173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1923576-8369-490d-bc73-d04e00f567c1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:37.952339 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1224/2425] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:37 2022] POST /identity/v3/projects => generated 395 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:37.953879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3faed-e33c-4308-b6d6-2a53b7682fe6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:37.979974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3faed-e33c-4308-b6d6-2a53b7682fe6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:38.155159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38dc52-41a8-4464-becd-e1b2aece4d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:38.156207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38dc52-41a8-4464-becd-e1b2aece4d06 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:38.156207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38dc52-41a8-4464-becd-e1b2aece4d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:38.156207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38dc52-41a8-4464-becd-e1b2aece4d06 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:38.157349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38dc52-41a8-4464-becd-e1b2aece4d06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:38.164937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38dc52-41a8-4464-becd-e1b2aece4d06 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:38.205144 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1202/2426] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:37 2022] POST /identity/v3/users => generated 298 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:38.209464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1923576-8369-490d-bc73-d04e00f567c1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:38.224765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1923576-8369-490d-bc73-d04e00f567c1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:38.230755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b48d2-5db1-4a64-8675-e0005a8ff7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:38.232554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b48d2-5db1-4a64-8675-e0005a8ff7ff admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:38.233069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b48d2-5db1-4a64-8675-e0005a8ff7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:38.233526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b48d2-5db1-4a64-8675-e0005a8ff7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:38.234427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b48d2-5db1-4a64-8675-e0005a8ff7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:38.240533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b48d2-5db1-4a64-8675-e0005a8ff7ff admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:38.260762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38dc52-41a8-4464-becd-e1b2aece4d06 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:38.261530 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1225/2427] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:38.273535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38dc52-41a8-4464-becd-e1b2aece4d06 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:38.279893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926502c-1d1c-4287-9a26-f7454aad0a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:38.280879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926502c-1d1c-4287-9a26-f7454aad0a83 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:38.281367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926502c-1d1c-4287-9a26-f7454aad0a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:38.281894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926502c-1d1c-4287-9a26-f7454aad0a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:38.283099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926502c-1d1c-4287-9a26-f7454aad0a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:38.285633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926502c-1d1c-4287-9a26-f7454aad0a83 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:38.305139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b48d2-5db1-4a64-8675-e0005a8ff7ff admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:38.306760 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1203/2428] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:38.320174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b48d2-5db1-4a64-8675-e0005a8ff7ff admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:38.326361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c77c3-3920-456d-b482-eb8da8d86e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:38.327466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c77c3-3920-456d-b482-eb8da8d86e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:38.327923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c77c3-3920-456d-b482-eb8da8d86e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:38.328397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c77c3-3920-456d-b482-eb8da8d86e7b admin admin] PATH_INFO: `/v3/projects/c8bfb40c6e1e496fb1a3703ddb50e8bb/users/7c276ebf6fe14a9c8f50bd1d1f44fb58/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:38.354136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c77c3-3920-456d-b482-eb8da8d86e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8bfb40c6e1e496fb1a3703ddb50e8bb, user_id=7c276ebf6fe14a9c8f50bd1d1f44fb58, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:38.358712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c77c3-3920-456d-b482-eb8da8d86e7b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:38.384217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926502c-1d1c-4287-9a26-f7454aad0a83 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:38.384739 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1226/2429] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:38 2022] PUT /identity/v3/projects/c8bfb40c6e1e496fb1a3703ddb50e8bb/users/7c276ebf6fe14a9c8f50bd1d1f44fb58/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:38.386685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8926502c-1d1c-4287-9a26-f7454aad0a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:38.392782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45383f-a27f-4070-87cf-e35d18524a1c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:38.397802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45383f-a27f-4070-87cf-e35d18524a1c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:38.398701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45383f-a27f-4070-87cf-e35d18524a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:38.412745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926502c-1d1c-4287-9a26-f7454aad0a83 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:38.446303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1c45383f-a27f-4070-87cf-e35d18524a1c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:38.448422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1c45383f-a27f-4070-87cf-e35d18524a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:38.520462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:38.521269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:38.521608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:38.521980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0 admin admin] PATH_INFO: `/v3/projects/aebea2bc737942fc899aa04e350f2936` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:38.524034 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1227/2430] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:38.534926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aebea2bc737942fc899aa04e350f2936)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:38.535549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45383f-a27f-4070-87cf-e35d18524a1c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:38.537037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1c45383f-a27f-4070-87cf-e35d18524a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:38.537814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:38.609801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:38.620258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aebea2bc737942fc899aa04e350f2936', 'request_id': 'req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:38.629196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45383f-a27f-4070-87cf-e35d18524a1c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:38.630305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aebea2bc737942fc899aa04e350f2936', 'request_id': 'req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:38.644970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aebea2bc737942fc899aa04e350f2936', 'request_id': 'req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:38.662477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0 admin admin] The token cache is being invalidate because project aebea2bc737942fc899aa04e350f2936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:38.663313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:38.664094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ea0a3d26-3979-44b7-aab7-0dc13f29d2a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:38.672634 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1204/2431] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:38 2022] DELETE /identity/v3/projects/aebea2bc737942fc899aa04e350f2936 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:38.675128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f4cee-79c0-4257-a090-3a255a525543 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:38.675370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f4cee-79c0-4257-a090-3a255a525543 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:38.676026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f4cee-79c0-4257-a090-3a255a525543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:38.698319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ddfd5a-e724-470e-8ddb-8dbb410224bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:38.699030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ddfd5a-e724-470e-8ddb-8dbb410224bf service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:38.699361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ddfd5a-e724-470e-8ddb-8dbb410224bf service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:38.699627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ddfd5a-e724-470e-8ddb-8dbb410224bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:38.700034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-109f4cee-79c0-4257-a090-3a255a525543 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:38.701533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-00ddfd5a-e724-470e-8ddb-8dbb410224bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:38.702914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-109f4cee-79c0-4257-a090-3a255a525543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:38.711884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:38.787829 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1205/2432] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:56:38.799166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7ff08-55af-4d4e-a977-4007eea4eb84 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:38.799445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7ff08-55af-4d4e-a977-4007eea4eb84 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:38.799712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7ff08-55af-4d4e-a977-4007eea4eb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:38.803757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ddfd5a-e724-470e-8ddb-8dbb410224bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:38.805407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ddfd5a-e724-470e-8ddb-8dbb410224bf service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:38.828287 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1228/2433] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:38.844921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-19a7ff08-55af-4d4e-a977-4007eea4eb84 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:38.846253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-19a7ff08-55af-4d4e-a977-4007eea4eb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:38.879175 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1206/2434] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:38.881443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ddfd5a-e724-470e-8ddb-8dbb410224bf service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:38.899839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ddfd5a-e724-470e-8ddb-8dbb410224bf service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:38.941300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884d0d3-3979-4fc9-a465-45b5d5219b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:38.942040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884d0d3-3979-4fc9-a465-45b5d5219b74 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:38.942373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884d0d3-3979-4fc9-a465-45b5d5219b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:38.942645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884d0d3-3979-4fc9-a465-45b5d5219b74 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:38.943409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884d0d3-3979-4fc9-a465-45b5d5219b74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:38.943951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884d0d3-3979-4fc9-a465-45b5d5219b74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:38.945871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884d0d3-3979-4fc9-a465-45b5d5219b74 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:38.976536 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1229/2435] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:38.981480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7ff08-55af-4d4e-a977-4007eea4eb84 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:38.999717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7ff08-55af-4d4e-a977-4007eea4eb84 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:39.014515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ccadb-1603-4088-9261-b1e882aeb86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:39.017042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ccadb-1603-4088-9261-b1e882aeb86a admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:39.019639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ccadb-1603-4088-9261-b1e882aeb86a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:39.020150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ccadb-1603-4088-9261-b1e882aeb86a admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:39.024275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ccadb-1603-4088-9261-b1e882aeb86a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:39.036583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ccadb-1603-4088-9261-b1e882aeb86a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:39.116419 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1207/2436] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:38 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:39.117796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884d0d3-3979-4fc9-a465-45b5d5219b74 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:39.134440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ccadb-1603-4088-9261-b1e882aeb86a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:39.136677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b48ccadb-1603-4088-9261-b1e882aeb86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:39.136817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884d0d3-3979-4fc9-a465-45b5d5219b74 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:39.276471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df949d86-f603-4563-95bd-a6d927665047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:39.277317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df949d86-f603-4563-95bd-a6d927665047 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:39.277597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df949d86-f603-4563-95bd-a6d927665047 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:39.277870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df949d86-f603-4563-95bd-a6d927665047 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:39.278507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df949d86-f603-4563-95bd-a6d927665047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:39.281494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df949d86-f603-4563-95bd-a6d927665047 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:39.307739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ccadb-1603-4088-9261-b1e882aeb86a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:39.321155 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1230/2437] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:39 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:39.322302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a2342-db67-482d-a4ba-08b7fb31d327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:39.322529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df949d86-f603-4563-95bd-a6d927665047 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:39.323359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a2342-db67-482d-a4ba-08b7fb31d327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:39.328918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a2342-db67-482d-a4ba-08b7fb31d327 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:39.329801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a2342-db67-482d-a4ba-08b7fb31d327 admin admin] PATH_INFO: `/v3/projects/cd9c82d4785244778d540e0dd1f358b6` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:39.330817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a2342-db67-482d-a4ba-08b7fb31d327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd9c82d4785244778d540e0dd1f358b6)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:39.332901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a2342-db67-482d-a4ba-08b7fb31d327 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:39.336271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df949d86-f603-4563-95bd-a6d927665047 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:39.342718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d9c236-4ad7-4036-b5c2-c2edf7672c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:39.343343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9c236-4ad7-4036-b5c2-c2edf7672c25 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:39.343960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9c236-4ad7-4036-b5c2-c2edf7672c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:39.344401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9c236-4ad7-4036-b5c2-c2edf7672c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:39.345130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d9c236-4ad7-4036-b5c2-c2edf7672c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:39.347842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d9c236-4ad7-4036-b5c2-c2edf7672c25 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:39.366568 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1231/2438] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:39.367683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d9c236-4ad7-4036-b5c2-c2edf7672c25 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:39.381702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d9c236-4ad7-4036-b5c2-c2edf7672c25 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:39.391665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd5d6d-cc03-426b-9cb4-d1991b0c70f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:39.392728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd5d6d-cc03-426b-9cb4-d1991b0c70f7 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:39.393104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd5d6d-cc03-426b-9cb4-d1991b0c70f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:39.393529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd5d6d-cc03-426b-9cb4-d1991b0c70f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:39.394411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd5d6d-cc03-426b-9cb4-d1991b0c70f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:39.398348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd5d6d-cc03-426b-9cb4-d1991b0c70f7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:39.405557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:39.418731 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1232/2439] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:39.419920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd5d6d-cc03-426b-9cb4-d1991b0c70f7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:39.425619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8a5a2342-db67-482d-a4ba-08b7fb31d327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd9c82d4785244778d540e0dd1f358b6', 'request_id': 'req-8a5a2342-db67-482d-a4ba-08b7fb31d327'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:39.433351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8a5a2342-db67-482d-a4ba-08b7fb31d327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd9c82d4785244778d540e0dd1f358b6', 'request_id': 'req-8a5a2342-db67-482d-a4ba-08b7fb31d327'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:39.433779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd5d6d-cc03-426b-9cb4-d1991b0c70f7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:39.447603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8a5a2342-db67-482d-a4ba-08b7fb31d327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd9c82d4785244778d540e0dd1f358b6', 'request_id': 'req-8a5a2342-db67-482d-a4ba-08b7fb31d327'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:39.477022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8a5a2342-db67-482d-a4ba-08b7fb31d327 admin admin] The token cache is being invalidate because project cd9c82d4785244778d540e0dd1f358b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:39.481951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8a5a2342-db67-482d-a4ba-08b7fb31d327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:39.485236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8a5a2342-db67-482d-a4ba-08b7fb31d327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:39.489271 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1208/2440] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:39 2022] DELETE /identity/v3/projects/cd9c82d4785244778d540e0dd1f358b6 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:56:39.496040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480dfed-6f43-4076-8c55-2b01fda2ea72 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:39.496350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480dfed-6f43-4076-8c55-2b01fda2ea72 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:39.497513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480dfed-6f43-4076-8c55-2b01fda2ea72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:39.547868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d480dfed-6f43-4076-8c55-2b01fda2ea72 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:39.548134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:39.550389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d480dfed-6f43-4076-8c55-2b01fda2ea72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:39.561373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240c68d-7146-4853-a271-9d6694c1f3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:39.562669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240c68d-7146-4853-a271-9d6694c1f3d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:39.563203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240c68d-7146-4853-a271-9d6694c1f3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:39.565044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240c68d-7146-4853-a271-9d6694c1f3d5 admin admin] PATH_INFO: `/v3/projects/91d0220829a84c93a58eea44b8e07fa0/users/8cc1474bc2a1450eac2d15554d5fcd11/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:39.577936 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1209/2441] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:39.580184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff53d63-7116-4214-b5ab-05ba970ca750 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:39.580660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff53d63-7116-4214-b5ab-05ba970ca750 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:39.580948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff53d63-7116-4214-b5ab-05ba970ca750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:39.592901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240c68d-7146-4853-a271-9d6694c1f3d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d0220829a84c93a58eea44b8e07fa0, user_id=8cc1474bc2a1450eac2d15554d5fcd11, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:39.595809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240c68d-7146-4853-a271-9d6694c1f3d5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:39.621268 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1233/2442] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:39 2022] PUT /identity/v3/projects/91d0220829a84c93a58eea44b8e07fa0/users/8cc1474bc2a1450eac2d15554d5fcd11/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:39.634557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4ea54-7c5c-4960-bda4-ae10fc3e2dee None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:39.634769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4ea54-7c5c-4960-bda4-ae10fc3e2dee None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:39.635023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4ea54-7c5c-4960-bda4-ae10fc3e2dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:39.645228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-dff53d63-7116-4214-b5ab-05ba970ca750 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:39.647054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dff53d63-7116-4214-b5ab-05ba970ca750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:39.698495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-24d4ea54-7c5c-4960-bda4-ae10fc3e2dee None None] MFA Rules not processed for user `8cc1474bc2a1450eac2d15554d5fcd11`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:39.699746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-24d4ea54-7c5c-4960-bda4-ae10fc3e2dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:39.761166 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1210/2443] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:39.765502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff53d63-7116-4214-b5ab-05ba970ca750 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:39.781170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff53d63-7116-4214-b5ab-05ba970ca750 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:39.816802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9638c7-5dbb-4370-99a9-757b9bfdafae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:39.817755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9638c7-5dbb-4370-99a9-757b9bfdafae admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:39.818036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9638c7-5dbb-4370-99a9-757b9bfdafae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:39.818303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9638c7-5dbb-4370-99a9-757b9bfdafae admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:39.819142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9638c7-5dbb-4370-99a9-757b9bfdafae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:39.819457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9638c7-5dbb-4370-99a9-757b9bfdafae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:39.822964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9638c7-5dbb-4370-99a9-757b9bfdafae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:39.854177 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1211/2444] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:56:39.861072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9638c7-5dbb-4370-99a9-757b9bfdafae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:39.871528 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1234/2445] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:39.877231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4ea54-7c5c-4960-bda4-ae10fc3e2dee None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:39.880849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-24d4ea54-7c5c-4960-bda4-ae10fc3e2dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:39.887203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9638c7-5dbb-4370-99a9-757b9bfdafae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:39.903152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6118ac6-50f3-4b16-9208-5c7d1965a34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:39.903957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6118ac6-50f3-4b16-9208-5c7d1965a34d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:39.904244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6118ac6-50f3-4b16-9208-5c7d1965a34d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:39.904547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6118ac6-50f3-4b16-9208-5c7d1965a34d admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:39.905190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6118ac6-50f3-4b16-9208-5c7d1965a34d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:39.907129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4ea54-7c5c-4960-bda4-ae10fc3e2dee None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:39.907616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6118ac6-50f3-4b16-9208-5c7d1965a34d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:39.913945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0889953-066e-4664-95a2-43a965611c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:39.914582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0889953-066e-4664-95a2-43a965611c1e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:39.914854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0889953-066e-4664-95a2-43a965611c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:39.915119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0889953-066e-4664-95a2-43a965611c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:39.915767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0889953-066e-4664-95a2-43a965611c1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:39.918099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0889953-066e-4664-95a2-43a965611c1e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:39.972926 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1212/2446] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:39 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:39.977177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6118ac6-50f3-4b16-9208-5c7d1965a34d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:39.988335 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1235/2447] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:39 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:39.992463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0889953-066e-4664-95a2-43a965611c1e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:40.101825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6118ac6-50f3-4b16-9208-5c7d1965a34d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:40.107092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf232be5-a65f-40d5-93d5-4e6ebe20f7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:40.107771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf232be5-a65f-40d5-93d5-4e6ebe20f7ec admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.108051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf232be5-a65f-40d5-93d5-4e6ebe20f7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.108320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf232be5-a65f-40d5-93d5-4e6ebe20f7ec admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.109148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf232be5-a65f-40d5-93d5-4e6ebe20f7ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:40.109324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0889953-066e-4664-95a2-43a965611c1e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:40.111749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf232be5-a65f-40d5-93d5-4e6ebe20f7ec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:40.117116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756ab73-af8e-4612-a456-4eb491c89b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:40.124729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756ab73-af8e-4612-a456-4eb491c89b89 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.124729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756ab73-af8e-4612-a456-4eb491c89b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.124729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756ab73-af8e-4612-a456-4eb491c89b89 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.126084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756ab73-af8e-4612-a456-4eb491c89b89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:40.129670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756ab73-af8e-4612-a456-4eb491c89b89 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:40.148424 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1213/2448] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:39 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:40.152768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf232be5-a65f-40d5-93d5-4e6ebe20f7ec admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:40.164692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf232be5-a65f-40d5-93d5-4e6ebe20f7ec admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:40.167794 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1236/2449] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:39 2022] POST /identity/v3/users => generated 294 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:56:40.172352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756ab73-af8e-4612-a456-4eb491c89b89 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:40.173175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cebb82-1288-4c16-817c-ceebc046a62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:40.173852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cebb82-1288-4c16-817c-ceebc046a62a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.174267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cebb82-1288-4c16-817c-ceebc046a62a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.174588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cebb82-1288-4c16-817c-ceebc046a62a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.175304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cebb82-1288-4c16-817c-ceebc046a62a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:40.177062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cebb82-1288-4c16-817c-ceebc046a62a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:40.196155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756ab73-af8e-4612-a456-4eb491c89b89 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:40.202497 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1214/2450] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:40.208357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cebb82-1288-4c16-817c-ceebc046a62a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:40.214054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc513497-2c4e-467e-bb72-0792dd6fa7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:40.217529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc513497-2c4e-467e-bb72-0792dd6fa7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.218714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc513497-2c4e-467e-bb72-0792dd6fa7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.220082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc513497-2c4e-467e-bb72-0792dd6fa7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.220833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc513497-2c4e-467e-bb72-0792dd6fa7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:40.228753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cebb82-1288-4c16-817c-ceebc046a62a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:40.239897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcdd80-8621-4601-a52c-21b8bbff8bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:40.240717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcdd80-8621-4601-a52c-21b8bbff8bda admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.240998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcdd80-8621-4601-a52c-21b8bbff8bda admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.241269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcdd80-8621-4601-a52c-21b8bbff8bda admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.241923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcdd80-8621-4601-a52c-21b8bbff8bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:40.242223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc513497-2c4e-467e-bb72-0792dd6fa7b3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:40.243560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcdd80-8621-4601-a52c-21b8bbff8bda admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:40.251994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:40.272427 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1215/2451] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:56:40.275813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcdd80-8621-4601-a52c-21b8bbff8bda admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:40.284932 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1237/2452] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:40.286040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc513497-2c4e-467e-bb72-0792dd6fa7b3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:40.298377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc513497-2c4e-467e-bb72-0792dd6fa7b3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:40.306625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14204f3-a92f-4d35-b83e-e87415d9dc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:40.307301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14204f3-a92f-4d35-b83e-e87415d9dc80 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.307575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14204f3-a92f-4d35-b83e-e87415d9dc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.307841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14204f3-a92f-4d35-b83e-e87415d9dc80 admin admin] PATH_INFO: `/v3/projects/9b7b920f28ee4956b735595591863fbb/users/5ee9f4636b2c455ea4fe357955d687f8/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.322548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcdd80-8621-4601-a52c-21b8bbff8bda admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:40.334802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14204f3-a92f-4d35-b83e-e87415d9dc80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b7b920f28ee4956b735595591863fbb, user_id=5ee9f4636b2c455ea4fe357955d687f8, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:40.337383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b4fad-127a-4779-82ac-1402cb923eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:40.337847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14204f3-a92f-4d35-b83e-e87415d9dc80 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:40.338125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b4fad-127a-4779-82ac-1402cb923eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.338404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b4fad-127a-4779-82ac-1402cb923eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.338676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b4fad-127a-4779-82ac-1402cb923eaf admin admin] PATH_INFO: `/v3/projects/9fb4816f6ef44d4bbec4f9888c452349/users/54fd42f9614649a0af60b72656419432/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.365100 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1238/2453] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:40 2022] PUT /identity/v3/projects/9b7b920f28ee4956b735595591863fbb/users/5ee9f4636b2c455ea4fe357955d687f8/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:40.367114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14204f3-a92f-4d35-b83e-e87415d9dc80 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:40.377520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:40.383441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b4fad-127a-4779-82ac-1402cb923eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb4816f6ef44d4bbec4f9888c452349, user_id=54fd42f9614649a0af60b72656419432, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:40.388679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b4fad-127a-4779-82ac-1402cb923eaf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:40.448108 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1216/2454] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:40 2022] PUT /identity/v3/projects/9fb4816f6ef44d4bbec4f9888c452349/users/54fd42f9614649a0af60b72656419432/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:40.451924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec5f42-3eaa-4ef2-9c13-e47c3c6c75a1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.452212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec5f42-3eaa-4ef2-9c13-e47c3c6c75a1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.453423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec5f42-3eaa-4ef2-9c13-e47c3c6c75a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.526806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14204f3-a92f-4d35-b83e-e87415d9dc80 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:40.543607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-84ec5f42-3eaa-4ef2-9c13-e47c3c6c75a1 None None] MFA Rules not processed for user `54fd42f9614649a0af60b72656419432`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:40.544782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-84ec5f42-3eaa-4ef2-9c13-e47c3c6c75a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:40.666143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04cac3-a8ad-4cba-a39e-6f52eb756077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:40.666679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04cac3-a8ad-4cba-a39e-6f52eb756077 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.667365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04cac3-a8ad-4cba-a39e-6f52eb756077 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.667644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04cac3-a8ad-4cba-a39e-6f52eb756077 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.668465 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1217/2455] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:40.671870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04cac3-a8ad-4cba-a39e-6f52eb756077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:40.672843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec5f42-3eaa-4ef2-9c13-e47c3c6c75a1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:40.673761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04cac3-a8ad-4cba-a39e-6f52eb756077 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:40.685400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec5f42-3eaa-4ef2-9c13-e47c3c6c75a1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:40.690681 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1239/2456] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:40.691571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb619b0-df6a-416b-bd39-16a568b31ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:40.692952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb619b0-df6a-416b-bd39-16a568b31ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.693652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb619b0-df6a-416b-bd39-16a568b31ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.695163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04cac3-a8ad-4cba-a39e-6f52eb756077 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:40.695674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb619b0-df6a-416b-bd39-16a568b31ef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.697161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb619b0-df6a-416b-bd39-16a568b31ef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:40.701732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb619b0-df6a-416b-bd39-16a568b31ef2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:40.711340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04cac3-a8ad-4cba-a39e-6f52eb756077 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:40.719828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840822eb-c089-49ef-9329-3e8e24ee20c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:40.720641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840822eb-c089-49ef-9329-3e8e24ee20c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.721215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840822eb-c089-49ef-9329-3e8e24ee20c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.721547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840822eb-c089-49ef-9329-3e8e24ee20c0 admin admin] PATH_INFO: `/v3/projects/9b7b920f28ee4956b735595591863fbb/users/5ee9f4636b2c455ea4fe357955d687f8/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.724462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840822eb-c089-49ef-9329-3e8e24ee20c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b7b920f28ee4956b735595591863fbb, user_id=5ee9f4636b2c455ea4fe357955d687f8, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:40.727379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840822eb-c089-49ef-9329-3e8e24ee20c0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:40.739159 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1218/2457] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:40 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:56:40.740239 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1240/2458] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:40 2022] PUT /identity/v3/projects/9b7b920f28ee4956b735595591863fbb/users/5ee9f4636b2c455ea4fe357955d687f8/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:40.752777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb619b0-df6a-416b-bd39-16a568b31ef2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:40.754329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8f094-66c2-4872-af90-7b71aaf81fad None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.754607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8f094-66c2-4872-af90-7b71aaf81fad None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.754873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8f094-66c2-4872-af90-7b71aaf81fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.827837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-56f8f094-66c2-4872-af90-7b71aaf81fad None None] MFA Rules not processed for user `5ee9f4636b2c455ea4fe357955d687f8`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:40.829274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-56f8f094-66c2-4872-af90-7b71aaf81fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:40.838023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb619b0-df6a-416b-bd39-16a568b31ef2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:40.861387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd151a66-41c3-47c0-aba5-91c2fac80a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:40.861387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd151a66-41c3-47c0-aba5-91c2fac80a9e admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.861387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd151a66-41c3-47c0-aba5-91c2fac80a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.861387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd151a66-41c3-47c0-aba5-91c2fac80a9e admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.862221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd151a66-41c3-47c0-aba5-91c2fac80a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:40.862221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd151a66-41c3-47c0-aba5-91c2fac80a9e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:40.907536 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1219/2459] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:40 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:40.909763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d56f8-e289-4c46-9d1c-e20432e7277b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:40.910351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d56f8-e289-4c46-9d1c-e20432e7277b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:40.910806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d56f8-e289-4c46-9d1c-e20432e7277b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:40.975819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-102d56f8-e289-4c46-9d1c-e20432e7277b None None] MFA Rules not processed for user `1762f75263ad4e2f84b265522e76764d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:40.980654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-102d56f8-e289-4c46-9d1c-e20432e7277b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:41.049202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:41.106690 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1241/2460] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:56:41.111947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8f094-66c2-4872-af90-7b71aaf81fad None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:41.164343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8f094-66c2-4872-af90-7b71aaf81fad None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:41.187558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb728966-b849-49c4-865d-d9eb36221fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:41.190035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb728966-b849-49c4-865d-d9eb36221fec admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:41.190320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb728966-b849-49c4-865d-d9eb36221fec admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:41.190598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb728966-b849-49c4-865d-d9eb36221fec admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:41.191271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb728966-b849-49c4-865d-d9eb36221fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:41.192973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb728966-b849-49c4-865d-d9eb36221fec admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:41.240247 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1242/2461] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:41.241802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c34fa-122f-4a3b-9d38-b0098fd8f8fb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:41.242083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c34fa-122f-4a3b-9d38-b0098fd8f8fb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:41.244488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c34fa-122f-4a3b-9d38-b0098fd8f8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:41.262927 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1220/2462] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:41.264119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d56f8-e289-4c46-9d1c-e20432e7277b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:41.279347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:41.292546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d56f8-e289-4c46-9d1c-e20432e7277b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:41.293294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d54c34fa-122f-4a3b-9d38-b0098fd8f8fb None None] MFA Rules not processed for user `8cc1474bc2a1450eac2d15554d5fcd11`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:41.295025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d54c34fa-122f-4a3b-9d38-b0098fd8f8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:41.305061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0613f-840a-4dad-92f2-91115f89d4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:41.305742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0613f-840a-4dad-92f2-91115f89d4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:41.306013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0613f-840a-4dad-92f2-91115f89d4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:41.306276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0613f-840a-4dad-92f2-91115f89d4fe admin admin] PATH_INFO: `/v3/projects/be0d4924ff204a80a88e600e12b339f3/users/265aae0286fa47ce9470778f92910fdd/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:41.355164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0613f-840a-4dad-92f2-91115f89d4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0d4924ff204a80a88e600e12b339f3, user_id=265aae0286fa47ce9470778f92910fdd, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:41.361143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0613f-840a-4dad-92f2-91115f89d4fe admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:41.396405 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1221/2463] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:41 2022] PUT /identity/v3/projects/be0d4924ff204a80a88e600e12b339f3/users/265aae0286fa47ce9470778f92910fdd/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:41.405710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0613f-840a-4dad-92f2-91115f89d4fe admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:41.411785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-06a0613f-840a-4dad-92f2-91115f89d4fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:41.515344 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1243/2464] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:41.521490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c34fa-122f-4a3b-9d38-b0098fd8f8fb None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:41.550287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c34fa-122f-4a3b-9d38-b0098fd8f8fb None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:41.630206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0613f-840a-4dad-92f2-91115f89d4fe admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:41.644647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7720e-2921-49bb-8987-c44e6560683e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:41.645645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7720e-2921-49bb-8987-c44e6560683e admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:41.645925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7720e-2921-49bb-8987-c44e6560683e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:41.646196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7720e-2921-49bb-8987-c44e6560683e admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:41.646838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7720e-2921-49bb-8987-c44e6560683e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:41.648836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7720e-2921-49bb-8987-c44e6560683e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:41.690748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65b3bc-394d-4979-a619-ddbb11c09525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:41.691488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65b3bc-394d-4979-a619-ddbb11c09525 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:41.691901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65b3bc-394d-4979-a619-ddbb11c09525 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:41.692241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65b3bc-394d-4979-a619-ddbb11c09525 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:41.696772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b65b3bc-394d-4979-a619-ddbb11c09525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:41.698959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b65b3bc-394d-4979-a619-ddbb11c09525 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:41.723661 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1244/2465] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:41.733914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65b3bc-394d-4979-a619-ddbb11c09525 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:41.742525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7b65b3bc-394d-4979-a619-ddbb11c09525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:41.744460 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1222/2466] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:41 2022] POST /identity/v3/projects => generated 385 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:41.745424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7720e-2921-49bb-8987-c44e6560683e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:41.884903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7720e-2921-49bb-8987-c44e6560683e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:41.899115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2e5f5-cb9a-4f5e-84a4-cdc97b714135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:41.901814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2e5f5-cb9a-4f5e-84a4-cdc97b714135 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:41.902421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2e5f5-cb9a-4f5e-84a4-cdc97b714135 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:41.905242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2e5f5-cb9a-4f5e-84a4-cdc97b714135 admin admin] PATH_INFO: `/v3/projects/be0d4924ff204a80a88e600e12b339f3/users/265aae0286fa47ce9470778f92910fdd/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:41.915444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2e5f5-cb9a-4f5e-84a4-cdc97b714135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0d4924ff204a80a88e600e12b339f3, user_id=265aae0286fa47ce9470778f92910fdd, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:41.918975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2e5f5-cb9a-4f5e-84a4-cdc97b714135 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:41.919475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65b3bc-394d-4979-a619-ddbb11c09525 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:41.964318 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1223/2467] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:41 2022] PUT /identity/v3/projects/be0d4924ff204a80a88e600e12b339f3/users/265aae0286fa47ce9470778f92910fdd/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:41.965382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2e5f5-cb9a-4f5e-84a4-cdc97b714135 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:42.013165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:42.059954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:42.109445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef95e65-6b84-4b12-8073-7129e7d3c003 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:42.110376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef95e65-6b84-4b12-8073-7129e7d3c003 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:42.110505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef95e65-6b84-4b12-8073-7129e7d3c003 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:42.110780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef95e65-6b84-4b12-8073-7129e7d3c003 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:42.130988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2e5f5-cb9a-4f5e-84a4-cdc97b714135 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:42.146066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28d4ad-75bc-45f6-b6e7-3560368605ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:42.147081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28d4ad-75bc-45f6-b6e7-3560368605ca admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:42.147486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28d4ad-75bc-45f6-b6e7-3560368605ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:42.148150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28d4ad-75bc-45f6-b6e7-3560368605ca admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:42.148975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28d4ad-75bc-45f6-b6e7-3560368605ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:42.153150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28d4ad-75bc-45f6-b6e7-3560368605ca admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:42.202556 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1224/2468] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:41 2022] POST /identity/v3/users => generated 299 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:42.204155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034d945-7efb-4ae9-849a-c6a30a9c6c1c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:42.205751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034d945-7efb-4ae9-849a-c6a30a9c6c1c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:42.206420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034d945-7efb-4ae9-849a-c6a30a9c6c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:42.279457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c034d945-7efb-4ae9-849a-c6a30a9c6c1c None None] MFA Rules not processed for user `265aae0286fa47ce9470778f92910fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:42.280512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef95e65-6b84-4b12-8073-7129e7d3c003 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:42.280773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c034d945-7efb-4ae9-849a-c6a30a9c6c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:42.287538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef95e65-6b84-4b12-8073-7129e7d3c003 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:42.318632 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1245/2469] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:42.322774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef95e65-6b84-4b12-8073-7129e7d3c003 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:42.361969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef95e65-6b84-4b12-8073-7129e7d3c003 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:42.384524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb08c3-5c9e-4186-89b0-2b0a2630a9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:42.387340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb08c3-5c9e-4186-89b0-2b0a2630a9dc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:42.387625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb08c3-5c9e-4186-89b0-2b0a2630a9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:42.389302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb08c3-5c9e-4186-89b0-2b0a2630a9dc admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:42.391216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb08c3-5c9e-4186-89b0-2b0a2630a9dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:42.396365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb08c3-5c9e-4186-89b0-2b0a2630a9dc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:42.423483 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1246/2470] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:42.427667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb08c3-5c9e-4186-89b0-2b0a2630a9dc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:42.442762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb08c3-5c9e-4186-89b0-2b0a2630a9dc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:42.455970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bb2b9-d2be-4faf-930e-71f6884123a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:42.458887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bb2b9-d2be-4faf-930e-71f6884123a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:42.459270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bb2b9-d2be-4faf-930e-71f6884123a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:42.461748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bb2b9-d2be-4faf-930e-71f6884123a4 admin admin] PATH_INFO: `/v3/projects/71141fb46193431085ee6d268fc4ce5d/users/396027dc8be44f91805a44d83be35e29/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:42.475009 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1225/2471] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:42.513093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bb2b9-d2be-4faf-930e-71f6884123a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71141fb46193431085ee6d268fc4ce5d, user_id=396027dc8be44f91805a44d83be35e29, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:42.522807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bb2b9-d2be-4faf-930e-71f6884123a4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:42.558207 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1247/2472] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:42 2022] PUT /identity/v3/projects/71141fb46193431085ee6d268fc4ce5d/users/396027dc8be44f91805a44d83be35e29/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:42.562591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034d945-7efb-4ae9-849a-c6a30a9c6c1c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:42.579607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034d945-7efb-4ae9-849a-c6a30a9c6c1c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:42.745119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acce87a-dbb9-4b0d-9109-d184657d09f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:42.745763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acce87a-dbb9-4b0d-9109-d184657d09f9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:42.746042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acce87a-dbb9-4b0d-9109-d184657d09f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:42.746312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acce87a-dbb9-4b0d-9109-d184657d09f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:42.746971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acce87a-dbb9-4b0d-9109-d184657d09f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:42.748797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acce87a-dbb9-4b0d-9109-d184657d09f9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:42.775336 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1226/2473] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:42.780177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bb2b9-d2be-4faf-930e-71f6884123a4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:42.798606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bb2b9-d2be-4faf-930e-71f6884123a4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:42.811714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e041734-6fc1-4ea9-8044-d19cc9b6802d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:42.812657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e041734-6fc1-4ea9-8044-d19cc9b6802d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:42.813021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e041734-6fc1-4ea9-8044-d19cc9b6802d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:42.813298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e041734-6fc1-4ea9-8044-d19cc9b6802d admin admin] PATH_INFO: `/v3/projects/71141fb46193431085ee6d268fc4ce5d/users/396027dc8be44f91805a44d83be35e29/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:42.821379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e041734-6fc1-4ea9-8044-d19cc9b6802d admin admin] RBAC: Authorizing `identity:create_grant(project_id=71141fb46193431085ee6d268fc4ce5d, user_id=396027dc8be44f91805a44d83be35e29, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:42.829407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e041734-6fc1-4ea9-8044-d19cc9b6802d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:42.864494 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1248/2474] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:42 2022] PUT /identity/v3/projects/71141fb46193431085ee6d268fc4ce5d/users/396027dc8be44f91805a44d83be35e29/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:43.439472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acce87a-dbb9-4b0d-9109-d184657d09f9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:43.441792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3acce87a-dbb9-4b0d-9109-d184657d09f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:43.500480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:43.618111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acce87a-dbb9-4b0d-9109-d184657d09f9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:43.770352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce84338-74f9-4fe1-b173-8a83ab9203a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:43.770352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce84338-74f9-4fe1-b173-8a83ab9203a9 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:43.770352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce84338-74f9-4fe1-b173-8a83ab9203a9 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:43.770682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce84338-74f9-4fe1-b173-8a83ab9203a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:43.806131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0a4e7-b963-415c-a832-0c2781e1a547 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:43.806572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0a4e7-b963-415c-a832-0c2781e1a547 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:43.806686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0a4e7-b963-415c-a832-0c2781e1a547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:43.872272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-90e0a4e7-b963-415c-a832-0c2781e1a547 None None] MFA Rules not processed for user `8cc1474bc2a1450eac2d15554d5fcd11`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:43.873303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-90e0a4e7-b963-415c-a832-0c2781e1a547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:43.987156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce84338-74f9-4fe1-b173-8a83ab9203a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:43.989728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce84338-74f9-4fe1-b173-8a83ab9203a9 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:44.045071 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1227/2475] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:44.085142 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1249/2476] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:46.290961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce84338-74f9-4fe1-b173-8a83ab9203a9 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:46.516008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:46.530973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce84338-74f9-4fe1-b173-8a83ab9203a9 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:46.550407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:46.551384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:46.551834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:46.553060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] PATH_INFO: `/v3/users/54fd42f9614649a0af60b72656419432` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:46.557286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] RBAC: Authorizing `identity:delete_user(user_id=54fd42f9614649a0af60b72656419432)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:46.559657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:46.692960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54fd42f9614649a0af60b72656419432'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:46.706040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54fd42f9614649a0af60b72656419432'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:46.727088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54fd42f9614649a0af60b72656419432'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:46.729018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54fd42f9614649a0af60b72656419432'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:46.731786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54fd42f9614649a0af60b72656419432'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:46.773207 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1228/2477] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:46 2022] DELETE /identity/v3/users/54fd42f9614649a0af60b72656419432 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:46.775920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d459f50-5fc8-4855-a8a8-f0dfe210b85a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:46.779600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d459f50-5fc8-4855-a8a8-f0dfe210b85a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:46.780114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d459f50-5fc8-4855-a8a8-f0dfe210b85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:46.837916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6d459f50-5fc8-4855-a8a8-f0dfe210b85a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:46.839644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6d459f50-5fc8-4855-a8a8-f0dfe210b85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:46.926556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:47.001420 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1250/2478] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:47.083951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:47.086491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:47.238001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb44c-6dca-4dc1-9945-2fb3ef6ae03c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:47.255252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42f6c8-a522-4bc6-964c-16d5a2cc5f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:47.255931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42f6c8-a522-4bc6-964c-16d5a2cc5f4c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:47.256157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42f6c8-a522-4bc6-964c-16d5a2cc5f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:47.256442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42f6c8-a522-4bc6-964c-16d5a2cc5f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:47.287129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42f6c8-a522-4bc6-964c-16d5a2cc5f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:47.288609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42f6c8-a522-4bc6-964c-16d5a2cc5f4c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:47.312549 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1229/2479] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:48.948531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b0937-18f4-44ae-b8d5-c9845d863e88 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:48.949732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b0937-18f4-44ae-b8d5-c9845d863e88 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:48.950224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b0937-18f4-44ae-b8d5-c9845d863e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:48.994098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-541b0937-18f4-44ae-b8d5-c9845d863e88 None None] MFA Rules not processed for user `8afc45f50dc54b84a8abf3b6e168426b`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:48.995621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-541b0937-18f4-44ae-b8d5-c9845d863e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:49.094362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42f6c8-a522-4bc6-964c-16d5a2cc5f4c service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:49.097104 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1251/2480] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:49.098351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6c42f6c8-a522-4bc6-964c-16d5a2cc5f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:49.115495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42f6c8-a522-4bc6-964c-16d5a2cc5f4c service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:49.131893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94a85e-d23b-46bb-afbc-64138918230a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:49.132598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94a85e-d23b-46bb-afbc-64138918230a admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:49.132874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94a85e-d23b-46bb-afbc-64138918230a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:49.133136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94a85e-d23b-46bb-afbc-64138918230a admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:49.134087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94a85e-d23b-46bb-afbc-64138918230a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:49.136072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94a85e-d23b-46bb-afbc-64138918230a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:49.191615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b0937-18f4-44ae-b8d5-c9845d863e88 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:49.202513 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1230/2481] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:49 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:49.206138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b0937-18f4-44ae-b8d5-c9845d863e88 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:49.279011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79aeff-5917-4b5a-8276-fe42a1014ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:49.279702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79aeff-5917-4b5a-8276-fe42a1014ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:49.280038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79aeff-5917-4b5a-8276-fe42a1014ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:49.280313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79aeff-5917-4b5a-8276-fe42a1014ec2 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:49.280982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79aeff-5917-4b5a-8276-fe42a1014ec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:49.283069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79aeff-5917-4b5a-8276-fe42a1014ec2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:49.346482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94a85e-d23b-46bb-afbc-64138918230a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:49.361207 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1252/2482] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:49 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:49.368769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94a85e-d23b-46bb-afbc-64138918230a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:49.378555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402539e0-cfd1-4b4f-ac3f-4fa2ac5c63eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:49.379615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402539e0-cfd1-4b4f-ac3f-4fa2ac5c63eb admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:49.380305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402539e0-cfd1-4b4f-ac3f-4fa2ac5c63eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:49.380885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402539e0-cfd1-4b4f-ac3f-4fa2ac5c63eb admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:49.382256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402539e0-cfd1-4b4f-ac3f-4fa2ac5c63eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:49.385351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402539e0-cfd1-4b4f-ac3f-4fa2ac5c63eb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:49.408161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79aeff-5917-4b5a-8276-fe42a1014ec2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:49.422215 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1231/2483] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:49.423170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79aeff-5917-4b5a-8276-fe42a1014ec2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:49.430501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51daa07-06d2-4efd-91e9-7dda48510031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:49.432871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51daa07-06d2-4efd-91e9-7dda48510031 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:49.433346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51daa07-06d2-4efd-91e9-7dda48510031 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:49.434482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51daa07-06d2-4efd-91e9-7dda48510031 admin admin] PATH_INFO: `/v3/projects/d5241d90bd18473790afe7eec33ec9d2/users/0802ea75a81747e68e9bfa7d11ef50dc/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:49.458346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51daa07-06d2-4efd-91e9-7dda48510031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5241d90bd18473790afe7eec33ec9d2, user_id=0802ea75a81747e68e9bfa7d11ef50dc, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:49.464769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51daa07-06d2-4efd-91e9-7dda48510031 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:49.505632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402539e0-cfd1-4b4f-ac3f-4fa2ac5c63eb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:49.509056 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1253/2484] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:49 2022] PUT /identity/v3/projects/d5241d90bd18473790afe7eec33ec9d2/users/0802ea75a81747e68e9bfa7d11ef50dc/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:49.518995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402539e0-cfd1-4b4f-ac3f-4fa2ac5c63eb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:49.595442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:49.619606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2156cd8-3e87-4572-8410-d36c508cc411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:49.620236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2156cd8-3e87-4572-8410-d36c508cc411 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:49.620542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2156cd8-3e87-4572-8410-d36c508cc411 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:49.620814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2156cd8-3e87-4572-8410-d36c508cc411 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:49.621459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2156cd8-3e87-4572-8410-d36c508cc411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:49.625679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2156cd8-3e87-4572-8410-d36c508cc411 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:49.650818 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1232/2485] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:49.655445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51daa07-06d2-4efd-91e9-7dda48510031 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:49.671294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51daa07-06d2-4efd-91e9-7dda48510031 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:49.685629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1969997-666c-424f-bcc0-f94e32db1a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:49.686650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1969997-666c-424f-bcc0-f94e32db1a62 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:49.686943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1969997-666c-424f-bcc0-f94e32db1a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:49.689345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1969997-666c-424f-bcc0-f94e32db1a62 admin admin] PATH_INFO: `/v3/projects/d5241d90bd18473790afe7eec33ec9d2/users/0802ea75a81747e68e9bfa7d11ef50dc/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:49.698942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1969997-666c-424f-bcc0-f94e32db1a62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5241d90bd18473790afe7eec33ec9d2, user_id=0802ea75a81747e68e9bfa7d11ef50dc, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:49.705411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1969997-666c-424f-bcc0-f94e32db1a62 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:49.732557 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1254/2486] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:49 2022] PUT /identity/v3/projects/d5241d90bd18473790afe7eec33ec9d2/users/0802ea75a81747e68e9bfa7d11ef50dc/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:50.109923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2156cd8-3e87-4572-8410-d36c508cc411 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:50.111873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f2156cd8-3e87-4572-8410-d36c508cc411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:50.319385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1969997-666c-424f-bcc0-f94e32db1a62 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:50.321506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d1969997-666c-424f-bcc0-f94e32db1a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:50.330659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2156cd8-3e87-4572-8410-d36c508cc411 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:50.378432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:50.474334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1969997-666c-424f-bcc0-f94e32db1a62 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:50.478865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d611b3-21d2-4aee-875c-ff2ae823b258 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:50.479440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d611b3-21d2-4aee-875c-ff2ae823b258 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:50.479716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d611b3-21d2-4aee-875c-ff2ae823b258 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:50.479980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d611b3-21d2-4aee-875c-ff2ae823b258 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:50.481853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-86d611b3-21d2-4aee-875c-ff2ae823b258 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:50.493021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:50.493753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:50.494168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:50.494455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] PATH_INFO: `/v3/projects/9fb4816f6ef44d4bbec4f9888c452349` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:50.495640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb4816f6ef44d4bbec4f9888c452349)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:50.497737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:50.583863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d611b3-21d2-4aee-875c-ff2ae823b258 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:50.586815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d611b3-21d2-4aee-875c-ff2ae823b258 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:50.605339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb4816f6ef44d4bbec4f9888c452349', 'request_id': 'req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:50.615545 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1233/2487] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:50 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:50.638399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c8f7d-82e7-4f8b-99a1-856a621563f8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:50.638823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c8f7d-82e7-4f8b-99a1-856a621563f8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:50.639606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb4816f6ef44d4bbec4f9888c452349', 'request_id': 'req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:50.639836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c8f7d-82e7-4f8b-99a1-856a621563f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:50.653792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb4816f6ef44d4bbec4f9888c452349', 'request_id': 'req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:50.671558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9b7c8f7d-82e7-4f8b-99a1-856a621563f8 None None] MFA Rules not processed for user `63d2977c98ed4acc8d53d8cfbed6f0c4`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:50.673536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9b7c8f7d-82e7-4f8b-99a1-856a621563f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:50.681420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:50.685971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] The token cache is being invalidate because project 9fb4816f6ef44d4bbec4f9888c452349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:50.686268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:50.687008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:50.692002 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1255/2488] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:50 2022] DELETE /identity/v3/projects/9fb4816f6ef44d4bbec4f9888c452349 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:56:50.693264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:50.695068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:50.787674 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1234/2489] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:56:50.842270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2e2ed-040a-4e50-ba32-6cfff9a4f361 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:50.855715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:50.856415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:50.856694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:50.856954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] PATH_INFO: `/v3/users/265aae0286fa47ce9470778f92910fdd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:50.862629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=265aae0286fa47ce9470778f92910fdd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:50.865506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:50.947553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '265aae0286fa47ce9470778f92910fdd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:50.986100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '265aae0286fa47ce9470778f92910fdd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:50.987798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:51.003189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '265aae0286fa47ce9470778f92910fdd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:51.025679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '265aae0286fa47ce9470778f92910fdd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:51.026677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '265aae0286fa47ce9470778f92910fdd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:51.065606 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1256/2490] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:50 2022] DELETE /identity/v3/users/265aae0286fa47ce9470778f92910fdd => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:56:52.584977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f3ff0-8f03-48ef-9b4e-be46bd1c4910 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:52.585486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f3ff0-8f03-48ef-9b4e-be46bd1c4910 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:52.585583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f3ff0-8f03-48ef-9b4e-be46bd1c4910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:52.672326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-710f3ff0-8f03-48ef-9b4e-be46bd1c4910 None None] MFA Rules not processed for user `7c276ebf6fe14a9c8f50bd1d1f44fb58`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:52.673608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-710f3ff0-8f03-48ef-9b4e-be46bd1c4910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:52.755598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:52.758930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:52.847177 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1235/2491] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:52 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:52.852198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f3ff0-8f03-48ef-9b4e-be46bd1c4910 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:52.853760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-710f3ff0-8f03-48ef-9b4e-be46bd1c4910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:52.981684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f3ff0-8f03-48ef-9b4e-be46bd1c4910 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:52.988646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620107f-9350-4766-8c7c-74124272a7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:52.989500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620107f-9350-4766-8c7c-74124272a7e7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:52.990000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620107f-9350-4766-8c7c-74124272a7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:52.990440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620107f-9350-4766-8c7c-74124272a7e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:52.991262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620107f-9350-4766-8c7c-74124272a7e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:52.993500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620107f-9350-4766-8c7c-74124272a7e7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:52.996687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90568490-6511-4ede-a19a-ae6e0dd19e33 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:53.004709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8840b-20f4-43c8-a850-164ac9e3b677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:53.005373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8840b-20f4-43c8-a850-164ac9e3b677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:53.005643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8840b-20f4-43c8-a850-164ac9e3b677 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:53.005936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8840b-20f4-43c8-a850-164ac9e3b677 admin admin] PATH_INFO: `/v3/projects/be0d4924ff204a80a88e600e12b339f3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:53.007560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8840b-20f4-43c8-a850-164ac9e3b677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0d4924ff204a80a88e600e12b339f3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:53.009730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8840b-20f4-43c8-a850-164ac9e3b677 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:53.038446 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1236/2492] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:52 2022] POST /identity/v3/projects => generated 397 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:56:53.040003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620107f-9350-4766-8c7c-74124272a7e7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:53.093946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:53.101840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f9c8840b-20f4-43c8-a850-164ac9e3b677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0d4924ff204a80a88e600e12b339f3', 'request_id': 'req-f9c8840b-20f4-43c8-a850-164ac9e3b677'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:53.119725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f9c8840b-20f4-43c8-a850-164ac9e3b677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0d4924ff204a80a88e600e12b339f3', 'request_id': 'req-f9c8840b-20f4-43c8-a850-164ac9e3b677'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:53.136743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f9c8840b-20f4-43c8-a850-164ac9e3b677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0d4924ff204a80a88e600e12b339f3', 'request_id': 'req-f9c8840b-20f4-43c8-a850-164ac9e3b677'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:53.159224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620107f-9350-4766-8c7c-74124272a7e7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:53.160417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f9c8840b-20f4-43c8-a850-164ac9e3b677 admin admin] The token cache is being invalidate because project be0d4924ff204a80a88e600e12b339f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:53.163248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f9c8840b-20f4-43c8-a850-164ac9e3b677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:53.168651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f9c8840b-20f4-43c8-a850-164ac9e3b677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:53.170586 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1257/2493] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:52 2022] DELETE /identity/v3/projects/be0d4924ff204a80a88e600e12b339f3 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:53.183612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e54da-3b1c-432b-9ac4-daeca51b4669 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:53.185056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e54da-3b1c-432b-9ac4-daeca51b4669 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:53.185569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e54da-3b1c-432b-9ac4-daeca51b4669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:53.251713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e2b92-d80f-4a67-a2c1-665a269f5340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:53.253453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e2b92-d80f-4a67-a2c1-665a269f5340 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:53.253604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e2b92-d80f-4a67-a2c1-665a269f5340 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:53.253790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e2b92-d80f-4a67-a2c1-665a269f5340 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:53.256065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e2b92-d80f-4a67-a2c1-665a269f5340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:53.258111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a74e54da-3b1c-432b-9ac4-daeca51b4669 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:53.262152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e2b92-d80f-4a67-a2c1-665a269f5340 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:53.264416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a74e54da-3b1c-432b-9ac4-daeca51b4669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:53.312225 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1237/2494] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:53 2022] POST /identity/v3/users => generated 305 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:53.313827 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1258/2495] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:53.318068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e2b92-d80f-4a67-a2c1-665a269f5340 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:53.321651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-425e2b92-d80f-4a67-a2c1-665a269f5340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:53.321848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2553be-edbe-48f3-afd0-ec0db6ed395d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:53.322104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2553be-edbe-48f3-afd0-ec0db6ed395d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:53.322421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2553be-edbe-48f3-afd0-ec0db6ed395d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:53.338237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e2b92-d80f-4a67-a2c1-665a269f5340 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:53.344242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d1a81-ca2c-412a-82bb-47c6cd12ef25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:53.344989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d1a81-ca2c-412a-82bb-47c6cd12ef25 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:53.345317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d1a81-ca2c-412a-82bb-47c6cd12ef25 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:53.345589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d1a81-ca2c-412a-82bb-47c6cd12ef25 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:53.346297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d1a81-ca2c-412a-82bb-47c6cd12ef25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:53.347988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d1a81-ca2c-412a-82bb-47c6cd12ef25 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:53.368301 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1238/2496] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:53.369844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d1a81-ca2c-412a-82bb-47c6cd12ef25 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:53.375270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ac2553be-edbe-48f3-afd0-ec0db6ed395d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:53.377326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ac2553be-edbe-48f3-afd0-ec0db6ed395d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:53.384409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d1a81-ca2c-412a-82bb-47c6cd12ef25 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:53.392626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b8e78-1750-4c82-bb5a-89e1fc1a825b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:53.393539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b8e78-1750-4c82-bb5a-89e1fc1a825b admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:53.393839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b8e78-1750-4c82-bb5a-89e1fc1a825b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:53.394318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b8e78-1750-4c82-bb5a-89e1fc1a825b admin admin] PATH_INFO: `/v3/projects/57f3b0c432c4400f9c2a9a456b37dbf2/users/ae7d0d25bbf24edda377a9cc2e5a59e6/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:53.414976 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1259/2497] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:56:53.417304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b8e78-1750-4c82-bb5a-89e1fc1a825b admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f3b0c432c4400f9c2a9a456b37dbf2, user_id=ae7d0d25bbf24edda377a9cc2e5a59e6, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:53.420295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2553be-edbe-48f3-afd0-ec0db6ed395d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:53.426167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b8e78-1750-4c82-bb5a-89e1fc1a825b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:53.441742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2553be-edbe-48f3-afd0-ec0db6ed395d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:53.463982 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1239/2498] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:53 2022] PUT /identity/v3/projects/57f3b0c432c4400f9c2a9a456b37dbf2/users/ae7d0d25bbf24edda377a9cc2e5a59e6/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:53.466646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b8e78-1750-4c82-bb5a-89e1fc1a825b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:53.585092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:53.619144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de5ef9-6857-4157-a3b5-97551af216b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:53.619948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de5ef9-6857-4157-a3b5-97551af216b6 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:53.620905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de5ef9-6857-4157-a3b5-97551af216b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:53.622105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de5ef9-6857-4157-a3b5-97551af216b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:53.623690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de5ef9-6857-4157-a3b5-97551af216b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:53.624468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de5ef9-6857-4157-a3b5-97551af216b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:53.630570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de5ef9-6857-4157-a3b5-97551af216b6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:53.649560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b8e78-1750-4c82-bb5a-89e1fc1a825b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:53.661608 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1260/2499] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:56:53.662311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35480f50-6e07-4ad0-95a1-0e3a2046d635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:53.663340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35480f50-6e07-4ad0-95a1-0e3a2046d635 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:53.663868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35480f50-6e07-4ad0-95a1-0e3a2046d635 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:53.664460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35480f50-6e07-4ad0-95a1-0e3a2046d635 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:53.665259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35480f50-6e07-4ad0-95a1-0e3a2046d635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:53.666453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de5ef9-6857-4157-a3b5-97551af216b6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:53.675607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35480f50-6e07-4ad0-95a1-0e3a2046d635 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:53.711113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de5ef9-6857-4157-a3b5-97551af216b6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:53.713514 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1240/2500] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:53.719154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c54910-1dc4-44ac-8597-afd3bd6086bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:53.719434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35480f50-6e07-4ad0-95a1-0e3a2046d635 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:53.721059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c54910-1dc4-44ac-8597-afd3bd6086bf admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:53.721442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c54910-1dc4-44ac-8597-afd3bd6086bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:53.724883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c54910-1dc4-44ac-8597-afd3bd6086bf admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:53.726276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c54910-1dc4-44ac-8597-afd3bd6086bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:53.734913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c54910-1dc4-44ac-8597-afd3bd6086bf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:53.751862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35480f50-6e07-4ad0-95a1-0e3a2046d635 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:53.774334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28f848-b4ae-4cf0-b114-0b386183e345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:53.776052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28f848-b4ae-4cf0-b114-0b386183e345 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:53.776363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28f848-b4ae-4cf0-b114-0b386183e345 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:53.778047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28f848-b4ae-4cf0-b114-0b386183e345 admin admin] PATH_INFO: `/v3/projects/57f3b0c432c4400f9c2a9a456b37dbf2/users/ae7d0d25bbf24edda377a9cc2e5a59e6/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:53.782551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28f848-b4ae-4cf0-b114-0b386183e345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f3b0c432c4400f9c2a9a456b37dbf2, user_id=ae7d0d25bbf24edda377a9cc2e5a59e6, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:53.787753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28f848-b4ae-4cf0-b114-0b386183e345 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:53.827441 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1261/2501] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:53 2022] POST /identity/v3/projects => generated 393 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:53.829323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c54910-1dc4-44ac-8597-afd3bd6086bf admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:53.838953 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1241/2502] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:53 2022] PUT /identity/v3/projects/57f3b0c432c4400f9c2a9a456b37dbf2/users/ae7d0d25bbf24edda377a9cc2e5a59e6/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:53.979717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c54910-1dc4-44ac-8597-afd3bd6086bf admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:53.989667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ac43c-4b84-46c7-b538-0ba50de51ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:53.990651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ac43c-4b84-46c7-b538-0ba50de51ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:53.991098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ac43c-4b84-46c7-b538-0ba50de51ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:53.991670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ac43c-4b84-46c7-b538-0ba50de51ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:53.992634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ac43c-4b84-46c7-b538-0ba50de51ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:53.994931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ac43c-4b84-46c7-b538-0ba50de51ee5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:54.059177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28f848-b4ae-4cf0-b114-0b386183e345 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:54.064120 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1262/2503] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:53 2022] POST /identity/v3/users => generated 297 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:54.160387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:54.294397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28f848-b4ae-4cf0-b114-0b386183e345 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:54.308255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6fb93-a7d0-4e81-bcc3-dec4f06bec8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:54.309047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6fb93-a7d0-4e81-bcc3-dec4f06bec8f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:54.309383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6fb93-a7d0-4e81-bcc3-dec4f06bec8f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:54.309714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6fb93-a7d0-4e81-bcc3-dec4f06bec8f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:54.311773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6fb93-a7d0-4e81-bcc3-dec4f06bec8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:54.313777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6fb93-a7d0-4e81-bcc3-dec4f06bec8f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:54.331540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ac43c-4b84-46c7-b538-0ba50de51ee5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:54.338991 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1242/2504] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:54.345400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ac43c-4b84-46c7-b538-0ba50de51ee5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:54.357542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d293fc-351c-4e48-b83e-67ae82786094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:54.358556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d293fc-351c-4e48-b83e-67ae82786094 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:54.359094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d293fc-351c-4e48-b83e-67ae82786094 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:54.359360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d293fc-351c-4e48-b83e-67ae82786094 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:54.362265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d293fc-351c-4e48-b83e-67ae82786094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:54.366795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d293fc-351c-4e48-b83e-67ae82786094 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:54.386656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6fb93-a7d0-4e81-bcc3-dec4f06bec8f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:54.398163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6fb93-a7d0-4e81-bcc3-dec4f06bec8f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:54.398732 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1263/2505] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:56:54.404255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36084da-fda1-4ecb-866a-1ecbf6360ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:54.410507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36084da-fda1-4ecb-866a-1ecbf6360ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:54.410825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36084da-fda1-4ecb-866a-1ecbf6360ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:54.411642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36084da-fda1-4ecb-866a-1ecbf6360ea3 admin admin] PATH_INFO: `/v3/projects/e125ea31f2874f2ca485eb10e89b4302/users/b4db2b43914946b5bc46d8962cfe33a5/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:54.435933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36084da-fda1-4ecb-866a-1ecbf6360ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e125ea31f2874f2ca485eb10e89b4302, user_id=b4db2b43914946b5bc46d8962cfe33a5, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:54.438840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36084da-fda1-4ecb-866a-1ecbf6360ea3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:54.471993 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1243/2506] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:54 2022] PUT /identity/v3/projects/e125ea31f2874f2ca485eb10e89b4302/users/b4db2b43914946b5bc46d8962cfe33a5/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:54.472393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e35c7a-1d34-43c6-8748-22de05d746a3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:54.472680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e35c7a-1d34-43c6-8748-22de05d746a3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:54.472947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e35c7a-1d34-43c6-8748-22de05d746a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:54.541232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-64e35c7a-1d34-43c6-8748-22de05d746a3 None None] MFA Rules not processed for user `b4db2b43914946b5bc46d8962cfe33a5`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:54.542656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-64e35c7a-1d34-43c6-8748-22de05d746a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:54.700133 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1264/2507] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:54.705364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36084da-fda1-4ecb-866a-1ecbf6360ea3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:54.889475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36084da-fda1-4ecb-866a-1ecbf6360ea3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:54.899170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e1de5-b1f0-4a1b-8da1-e02aff134097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:54.900288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e1de5-b1f0-4a1b-8da1-e02aff134097 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:54.900724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e1de5-b1f0-4a1b-8da1-e02aff134097 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:54.900993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e1de5-b1f0-4a1b-8da1-e02aff134097 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:54.902813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e1de5-b1f0-4a1b-8da1-e02aff134097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:54.906739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e1de5-b1f0-4a1b-8da1-e02aff134097 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:54.979461 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1244/2508] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:56:54 2022] POST /identity/v3/projects => generated 391 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:56:54.983403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e35c7a-1d34-43c6-8748-22de05d746a3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:54.996770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e35c7a-1d34-43c6-8748-22de05d746a3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:55.028570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:55.065505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df31e8-8358-4912-9dd7-0697db0ed091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:55.066414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df31e8-8358-4912-9dd7-0697db0ed091 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:55.066845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df31e8-8358-4912-9dd7-0697db0ed091 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:55.067265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df31e8-8358-4912-9dd7-0697db0ed091 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:55.068044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df31e8-8358-4912-9dd7-0697db0ed091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:55.070547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df31e8-8358-4912-9dd7-0697db0ed091 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:55.097770 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1265/2509] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:56:54 2022] POST /identity/v3/users => generated 302 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:56:55.102261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e1de5-b1f0-4a1b-8da1-e02aff134097 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:55.117376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e1de5-b1f0-4a1b-8da1-e02aff134097 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:55.125293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78363813-5771-45a1-863a-5d113f66fb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:55.126318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78363813-5771-45a1-863a-5d113f66fb18 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:55.126763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78363813-5771-45a1-863a-5d113f66fb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:55.127394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78363813-5771-45a1-863a-5d113f66fb18 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:55.128226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78363813-5771-45a1-863a-5d113f66fb18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:55.132475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78363813-5771-45a1-863a-5d113f66fb18 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:55.148020 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1245/2510] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:56:55.151890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df31e8-8358-4912-9dd7-0697db0ed091 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:55.164577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df31e8-8358-4912-9dd7-0697db0ed091 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:55.170143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137c89f-9cea-4e81-84e5-390f5bb7a1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:55.171262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137c89f-9cea-4e81-84e5-390f5bb7a1ff admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:55.171730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137c89f-9cea-4e81-84e5-390f5bb7a1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:55.172180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137c89f-9cea-4e81-84e5-390f5bb7a1ff admin admin] PATH_INFO: `/v3/projects/8f995117c5a845deabceff48acd9071e/users/75ccc21e7d4f4a1eb8ea5cee01e97928/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:55.194264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137c89f-9cea-4e81-84e5-390f5bb7a1ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f995117c5a845deabceff48acd9071e, user_id=75ccc21e7d4f4a1eb8ea5cee01e97928, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:55.197777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137c89f-9cea-4e81-84e5-390f5bb7a1ff admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:55.218151 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1266/2511] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:55 2022] PUT /identity/v3/projects/8f995117c5a845deabceff48acd9071e/users/75ccc21e7d4f4a1eb8ea5cee01e97928/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:55.221501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78363813-5771-45a1-863a-5d113f66fb18 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:55.261485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78363813-5771-45a1-863a-5d113f66fb18 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:55.435694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d98d4-0f53-4d52-8f82-5cd9f8af3484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:55.437303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d98d4-0f53-4d52-8f82-5cd9f8af3484 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:55.437781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d98d4-0f53-4d52-8f82-5cd9f8af3484 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:55.438514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d98d4-0f53-4d52-8f82-5cd9f8af3484 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:55.440577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d98d4-0f53-4d52-8f82-5cd9f8af3484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:55.445309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d98d4-0f53-4d52-8f82-5cd9f8af3484 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:55.449471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:55.469877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137c89f-9cea-4e81-84e5-390f5bb7a1ff admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:55.471859 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1246/2512] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:56:55.483991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137c89f-9cea-4e81-84e5-390f5bb7a1ff admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:55.495667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af5b66-ce1c-4906-a45d-82cb93fce262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:55.496358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af5b66-ce1c-4906-a45d-82cb93fce262 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:55.496802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af5b66-ce1c-4906-a45d-82cb93fce262 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:55.500636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af5b66-ce1c-4906-a45d-82cb93fce262 admin admin] PATH_INFO: `/v3/projects/8f995117c5a845deabceff48acd9071e/users/75ccc21e7d4f4a1eb8ea5cee01e97928/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:55.502969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31af5b66-ce1c-4906-a45d-82cb93fce262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f995117c5a845deabceff48acd9071e, user_id=75ccc21e7d4f4a1eb8ea5cee01e97928, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:55.507265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31af5b66-ce1c-4906-a45d-82cb93fce262 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:55.545665 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1267/2513] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:56:55 2022] PUT /identity/v3/projects/8f995117c5a845deabceff48acd9071e/users/75ccc21e7d4f4a1eb8ea5cee01e97928/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:55.551686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa5d64-4ccc-4bdc-a1ce-88ea3ed6d63e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:55.553388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa5d64-4ccc-4bdc-a1ce-88ea3ed6d63e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:55.553892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa5d64-4ccc-4bdc-a1ce-88ea3ed6d63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:55.662028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f1fa5d64-4ccc-4bdc-a1ce-88ea3ed6d63e None None] MFA Rules not processed for user `75ccc21e7d4f4a1eb8ea5cee01e97928`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:55.664234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f1fa5d64-4ccc-4bdc-a1ce-88ea3ed6d63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:55.765384 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1247/2514] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:55 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:55.775316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af5b66-ce1c-4906-a45d-82cb93fce262 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:55.777309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-31af5b66-ce1c-4906-a45d-82cb93fce262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:55.907650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af5b66-ce1c-4906-a45d-82cb93fce262 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:55.913379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba4cb3-d712-4b5a-9ce7-fdbfea559421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:55.914080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba4cb3-d712-4b5a-9ce7-fdbfea559421 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:55.914354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba4cb3-d712-4b5a-9ce7-fdbfea559421 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:55.914698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba4cb3-d712-4b5a-9ce7-fdbfea559421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:55.923939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba4cb3-d712-4b5a-9ce7-fdbfea559421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:55.925671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba4cb3-d712-4b5a-9ce7-fdbfea559421 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:55.937413 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1268/2515] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:55 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:56.334038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa5d64-4ccc-4bdc-a1ce-88ea3ed6d63e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:56.500411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa5d64-4ccc-4bdc-a1ce-88ea3ed6d63e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:56.513992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:56.514676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:56.514947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:56.515214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] PATH_INFO: `/v3/users/b4db2b43914946b5bc46d8962cfe33a5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:56.516676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4db2b43914946b5bc46d8962cfe33a5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:56.519218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:56.626459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4db2b43914946b5bc46d8962cfe33a5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:56.643600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4db2b43914946b5bc46d8962cfe33a5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:56.684520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4db2b43914946b5bc46d8962cfe33a5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:56.685363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4db2b43914946b5bc46d8962cfe33a5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:56.696228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4db2b43914946b5bc46d8962cfe33a5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:56.734426 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1248/2516] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:56 2022] DELETE /identity/v3/users/b4db2b43914946b5bc46d8962cfe33a5 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:56.741651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25892bde-2e38-4520-9d70-97f9c6a40fc7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:56.742729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25892bde-2e38-4520-9d70-97f9c6a40fc7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:56.743178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25892bde-2e38-4520-9d70-97f9c6a40fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:56.785443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-25892bde-2e38-4520-9d70-97f9c6a40fc7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:56.786675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-25892bde-2e38-4520-9d70-97f9c6a40fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:56.904156 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1269/2517] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:56:56.921480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:56.923121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:56.928974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:57.012055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451350b6-f1ba-41d6-a147-b7421f2f7cae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:57.016331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ab304-ea53-49de-965c-0023c5f09b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:57.016985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ab304-ea53-49de-965c-0023c5f09b6b service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:57.017258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ab304-ea53-49de-965c-0023c5f09b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:57.017522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ab304-ea53-49de-965c-0023c5f09b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:57.028203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ab304-ea53-49de-965c-0023c5f09b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:57.029859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ab304-ea53-49de-965c-0023c5f09b6b service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:57.041176 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1249/2518] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:56:57.367726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25892bde-2e38-4520-9d70-97f9c6a40fc7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:57.369177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-25892bde-2e38-4520-9d70-97f9c6a40fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:57.381966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25892bde-2e38-4520-9d70-97f9c6a40fc7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:57.386474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:57.387124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:57.387395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:57.387658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] PATH_INFO: `/v3/projects/e125ea31f2874f2ca485eb10e89b4302` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:57.388893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e125ea31f2874f2ca485eb10e89b4302)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:57.391253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:57.392666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:57.458109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e125ea31f2874f2ca485eb10e89b4302', 'request_id': 'req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:57.470113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e125ea31f2874f2ca485eb10e89b4302', 'request_id': 'req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:57.477473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e125ea31f2874f2ca485eb10e89b4302', 'request_id': 'req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:57.494003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] The token cache is being invalidate because project e125ea31f2874f2ca485eb10e89b4302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:57.494447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:57.495327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:57.499300 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1270/2519] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:57 2022] DELETE /identity/v3/projects/e125ea31f2874f2ca485eb10e89b4302 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:57.504935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ab304-ea53-49de-965c-0023c5f09b6b service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:57.520159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ab304-ea53-49de-965c-0023c5f09b6b service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:57.624054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40df2ca-45ad-4fb6-bfaf-36f18c935478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:57.624878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40df2ca-45ad-4fb6-bfaf-36f18c935478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:57.625221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40df2ca-45ad-4fb6-bfaf-36f18c935478 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:57.625439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40df2ca-45ad-4fb6-bfaf-36f18c935478 admin admin] PATH_INFO: `/v3/users/75ccc21e7d4f4a1eb8ea5cee01e97928` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:57.626641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40df2ca-45ad-4fb6-bfaf-36f18c935478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75ccc21e7d4f4a1eb8ea5cee01e97928)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:57.628904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40df2ca-45ad-4fb6-bfaf-36f18c935478 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:57.716238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b40df2ca-45ad-4fb6-bfaf-36f18c935478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75ccc21e7d4f4a1eb8ea5cee01e97928'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:57.727415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b40df2ca-45ad-4fb6-bfaf-36f18c935478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75ccc21e7d4f4a1eb8ea5cee01e97928'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:57.762337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b40df2ca-45ad-4fb6-bfaf-36f18c935478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75ccc21e7d4f4a1eb8ea5cee01e97928'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:57.763689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b40df2ca-45ad-4fb6-bfaf-36f18c935478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75ccc21e7d4f4a1eb8ea5cee01e97928'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:57.764569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b40df2ca-45ad-4fb6-bfaf-36f18c935478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75ccc21e7d4f4a1eb8ea5cee01e97928'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:57.816338 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1250/2520] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:56:57 2022] DELETE /identity/v3/users/75ccc21e7d4f4a1eb8ea5cee01e97928 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:59.252966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:59.256824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:59.290682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff7f5b-69b0-4a12-b9d6-e2a99f798834 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:59.475509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469a62d-2265-44a7-b9ca-baa53fb9d668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:59.476315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469a62d-2265-44a7-b9ca-baa53fb9d668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:59.476607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469a62d-2265-44a7-b9ca-baa53fb9d668 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:59.476849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469a62d-2265-44a7-b9ca-baa53fb9d668 admin admin] PATH_INFO: `/v3/projects/8f995117c5a845deabceff48acd9071e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:59.479449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469a62d-2265-44a7-b9ca-baa53fb9d668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f995117c5a845deabceff48acd9071e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:59.484530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469a62d-2265-44a7-b9ca-baa53fb9d668 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:56:59.603116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5469a62d-2265-44a7-b9ca-baa53fb9d668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f995117c5a845deabceff48acd9071e', 'request_id': 'req-5469a62d-2265-44a7-b9ca-baa53fb9d668'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:59.619293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5469a62d-2265-44a7-b9ca-baa53fb9d668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f995117c5a845deabceff48acd9071e', 'request_id': 'req-5469a62d-2265-44a7-b9ca-baa53fb9d668'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:59.635654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5469a62d-2265-44a7-b9ca-baa53fb9d668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f995117c5a845deabceff48acd9071e', 'request_id': 'req-5469a62d-2265-44a7-b9ca-baa53fb9d668'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:59.659324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5469a62d-2265-44a7-b9ca-baa53fb9d668 admin admin] The token cache is being invalidate because project 8f995117c5a845deabceff48acd9071e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:56:59.659615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5469a62d-2265-44a7-b9ca-baa53fb9d668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:59.660305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5469a62d-2265-44a7-b9ca-baa53fb9d668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:56:59.661894 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1271/2521] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:56:59 2022] DELETE /identity/v3/projects/8f995117c5a845deabceff48acd9071e => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:56:59.667145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9433d3a-8bac-405d-8dac-79b313925c84 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:59.668428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9433d3a-8bac-405d-8dac-79b313925c84 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:59.668954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9433d3a-8bac-405d-8dac-79b313925c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:59.718153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f9433d3a-8bac-405d-8dac-79b313925c84 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:59.719516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f9433d3a-8bac-405d-8dac-79b313925c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:59.767332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:56:59.829698 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1251/2522] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:59.837130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586db5c-aad7-43eb-b167-50803a4c044e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:59.837519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586db5c-aad7-43eb-b167-50803a4c044e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:59.837670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586db5c-aad7-43eb-b167-50803a4c044e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:59.926856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8586db5c-aad7-43eb-b167-50803a4c044e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:56:59.928133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8586db5c-aad7-43eb-b167-50803a4c044e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:56:59.937888 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1272/2523] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:56:59.942108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9433d3a-8bac-405d-8dac-79b313925c84 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:56:59.958347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9433d3a-8bac-405d-8dac-79b313925c84 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:56:59.975590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ed16b-ee3e-40a1-9683-dfdefae72ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:56:59.976225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ed16b-ee3e-40a1-9683-dfdefae72ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:56:59.976568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ed16b-ee3e-40a1-9683-dfdefae72ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:56:59.976835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ed16b-ee3e-40a1-9683-dfdefae72ca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:56:59.977487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ed16b-ee3e-40a1-9683-dfdefae72ca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:56:59.977854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ed16b-ee3e-40a1-9683-dfdefae72ca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:56:59.979377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ed16b-ee3e-40a1-9683-dfdefae72ca1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:00.006738 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1252/2524] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:57:00.011320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586db5c-aad7-43eb-b167-50803a4c044e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:00.025392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586db5c-aad7-43eb-b167-50803a4c044e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:00.031911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216e457-94f7-4c91-a9f1-84b1a8fb309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:00.033303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216e457-94f7-4c91-a9f1-84b1a8fb309b admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:00.033733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216e457-94f7-4c91-a9f1-84b1a8fb309b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:00.034316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216e457-94f7-4c91-a9f1-84b1a8fb309b admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:00.035100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216e457-94f7-4c91-a9f1-84b1a8fb309b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:00.038011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216e457-94f7-4c91-a9f1-84b1a8fb309b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:00.055543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:00.092142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ed16b-ee3e-40a1-9683-dfdefae72ca1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:00.098795 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1273/2525] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:57:00 2022] POST /identity/v3/projects => generated 389 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:57:00.113859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ed16b-ee3e-40a1-9683-dfdefae72ca1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:00.238213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284bdc5-44b7-4332-8ec0-3935e361658e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:00.239235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284bdc5-44b7-4332-8ec0-3935e361658e admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:00.239656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284bdc5-44b7-4332-8ec0-3935e361658e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:00.241422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284bdc5-44b7-4332-8ec0-3935e361658e admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:00.242206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284bdc5-44b7-4332-8ec0-3935e361658e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:00.244642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284bdc5-44b7-4332-8ec0-3935e361658e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:00.315595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216e457-94f7-4c91-a9f1-84b1a8fb309b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:00.318556 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1253/2526] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:57:00 2022] POST /identity/v3/users => generated 295 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:57:00.339862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216e457-94f7-4c91-a9f1-84b1a8fb309b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:00.353198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a7b41-9986-4094-afbd-5360dbe64c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:00.354235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a7b41-9986-4094-afbd-5360dbe64c3e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:00.354667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a7b41-9986-4094-afbd-5360dbe64c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:00.355895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a7b41-9986-4094-afbd-5360dbe64c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:00.357796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a7b41-9986-4094-afbd-5360dbe64c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:00.360415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a7b41-9986-4094-afbd-5360dbe64c3e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:00.385633 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1274/2527] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:57:00.390732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284bdc5-44b7-4332-8ec0-3935e361658e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:00.404154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284bdc5-44b7-4332-8ec0-3935e361658e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:00.413387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8b782-eb6f-407f-b155-43956f225a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:00.414027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8b782-eb6f-407f-b155-43956f225a41 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:00.414301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8b782-eb6f-407f-b155-43956f225a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:00.414568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8b782-eb6f-407f-b155-43956f225a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:00.415224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8b782-eb6f-407f-b155-43956f225a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:00.417072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8b782-eb6f-407f-b155-43956f225a41 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:00.432901 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1254/2528] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:57:00.437172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a7b41-9986-4094-afbd-5360dbe64c3e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:00.464471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a7b41-9986-4094-afbd-5360dbe64c3e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:00.476572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d1017-5e70-4027-9dc5-17fd096de10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:00.478991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d1017-5e70-4027-9dc5-17fd096de10c admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:00.480326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d1017-5e70-4027-9dc5-17fd096de10c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:00.480811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d1017-5e70-4027-9dc5-17fd096de10c admin admin] PATH_INFO: `/v3/projects/6f09ec34654e4996893d91d215b5c3d8/users/19d39dd3a41744b7ab28c162d6748113/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:00.521491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d1017-5e70-4027-9dc5-17fd096de10c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f09ec34654e4996893d91d215b5c3d8, user_id=19d39dd3a41744b7ab28c162d6748113, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:00.524581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d1017-5e70-4027-9dc5-17fd096de10c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:00.554146 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1275/2529] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:57:00 2022] PUT /identity/v3/projects/6f09ec34654e4996893d91d215b5c3d8/users/19d39dd3a41744b7ab28c162d6748113/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:00.566787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871c627-ad6d-40d1-a0a3-52572f5a08ab None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:00.567195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871c627-ad6d-40d1-a0a3-52572f5a08ab None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:00.568080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871c627-ad6d-40d1-a0a3-52572f5a08ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:00.619613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a871c627-ad6d-40d1-a0a3-52572f5a08ab None None] MFA Rules not processed for user `19d39dd3a41744b7ab28c162d6748113`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:00.621013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a871c627-ad6d-40d1-a0a3-52572f5a08ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:00.735944 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1255/2530] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:00.740189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d1017-5e70-4027-9dc5-17fd096de10c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:00.844931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d1017-5e70-4027-9dc5-17fd096de10c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:00.850639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fbdeb-8914-41ef-83d7-3a7e581ab890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:00.851458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fbdeb-8914-41ef-83d7-3a7e581ab890 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:00.851808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fbdeb-8914-41ef-83d7-3a7e581ab890 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:00.852091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fbdeb-8914-41ef-83d7-3a7e581ab890 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:00.852824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46fbdeb-8914-41ef-83d7-3a7e581ab890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:00.855463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46fbdeb-8914-41ef-83d7-3a7e581ab890 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:00.898109 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1276/2531] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:57:00 2022] POST /identity/v3/projects => generated 389 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:57:00.901947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871c627-ad6d-40d1-a0a3-52572f5a08ab None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:00.914562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871c627-ad6d-40d1-a0a3-52572f5a08ab None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:00.941821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:01.008767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae67f72-87b1-4d3a-92d2-60efaf947db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:01.009234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae67f72-87b1-4d3a-92d2-60efaf947db2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:01.009538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae67f72-87b1-4d3a-92d2-60efaf947db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:01.010408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae67f72-87b1-4d3a-92d2-60efaf947db2 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:01.010631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae67f72-87b1-4d3a-92d2-60efaf947db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:01.012592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae67f72-87b1-4d3a-92d2-60efaf947db2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:01.038390 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1256/2532] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:57:00 2022] POST /identity/v3/users => generated 301 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:57:01.042477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fbdeb-8914-41ef-83d7-3a7e581ab890 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:01.054209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fbdeb-8914-41ef-83d7-3a7e581ab890 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:01.060009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e2e55-f7f2-4f69-8ff3-4d253daf7285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:01.060664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e2e55-f7f2-4f69-8ff3-4d253daf7285 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:01.061135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e2e55-f7f2-4f69-8ff3-4d253daf7285 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:01.061510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e2e55-f7f2-4f69-8ff3-4d253daf7285 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:01.062165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e2e55-f7f2-4f69-8ff3-4d253daf7285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:01.064183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e2e55-f7f2-4f69-8ff3-4d253daf7285 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:01.080309 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1277/2533] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:57:01.081634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae67f72-87b1-4d3a-92d2-60efaf947db2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:01.095187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae67f72-87b1-4d3a-92d2-60efaf947db2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:01.101778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a0c24-ddb2-4fd9-bf9a-9dbbc4c0e148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:01.102791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a0c24-ddb2-4fd9-bf9a-9dbbc4c0e148 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:01.103073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a0c24-ddb2-4fd9-bf9a-9dbbc4c0e148 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:01.103743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a0c24-ddb2-4fd9-bf9a-9dbbc4c0e148 admin admin] PATH_INFO: `/v3/projects/f3fd3099748c4f06847a34a463bced28/users/468cf602681141a793eccac6e443b1a7/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:01.127799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a0c24-ddb2-4fd9-bf9a-9dbbc4c0e148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3fd3099748c4f06847a34a463bced28, user_id=468cf602681141a793eccac6e443b1a7, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:01.131379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a0c24-ddb2-4fd9-bf9a-9dbbc4c0e148 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:01.152751 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1257/2534] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:57:01 2022] PUT /identity/v3/projects/f3fd3099748c4f06847a34a463bced28/users/468cf602681141a793eccac6e443b1a7/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:57:01.156898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e2e55-f7f2-4f69-8ff3-4d253daf7285 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:01.169613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e2e55-f7f2-4f69-8ff3-4d253daf7285 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:01.267420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2ac69-527f-4311-82f7-876e76dfc3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:01.268413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2ac69-527f-4311-82f7-876e76dfc3cd admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:01.269148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2ac69-527f-4311-82f7-876e76dfc3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:01.269566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2ac69-527f-4311-82f7-876e76dfc3cd admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:01.270474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2ac69-527f-4311-82f7-876e76dfc3cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:01.272737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2ac69-527f-4311-82f7-876e76dfc3cd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:01.292086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a0c24-ddb2-4fd9-bf9a-9dbbc4c0e148 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:01.297842 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1278/2535] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:57:01.308878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a0c24-ddb2-4fd9-bf9a-9dbbc4c0e148 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:01.325251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16091b53-4af9-4259-88b1-9dca881cf14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:01.326110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16091b53-4af9-4259-88b1-9dca881cf14d admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:01.326447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16091b53-4af9-4259-88b1-9dca881cf14d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:01.326780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16091b53-4af9-4259-88b1-9dca881cf14d admin admin] PATH_INFO: `/v3/projects/f3fd3099748c4f06847a34a463bced28/users/468cf602681141a793eccac6e443b1a7/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:01.330134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16091b53-4af9-4259-88b1-9dca881cf14d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3fd3099748c4f06847a34a463bced28, user_id=468cf602681141a793eccac6e443b1a7, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:01.333153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16091b53-4af9-4259-88b1-9dca881cf14d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:01.357024 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1258/2536] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:57:01 2022] PUT /identity/v3/projects/f3fd3099748c4f06847a34a463bced28/users/468cf602681141a793eccac6e443b1a7/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:57:01.358470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf907c-2990-4b8f-bd7a-1cb16317a1b1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:01.358934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf907c-2990-4b8f-bd7a-1cb16317a1b1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:01.359068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf907c-2990-4b8f-bd7a-1cb16317a1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:01.412798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d6cf907c-2990-4b8f-bd7a-1cb16317a1b1 None None] MFA Rules not processed for user `468cf602681141a793eccac6e443b1a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:01.414302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d6cf907c-2990-4b8f-bd7a-1cb16317a1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:01.434123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:01.521006 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1279/2537] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:01 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:01.532604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16091b53-4af9-4259-88b1-9dca881cf14d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:01.541707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-16091b53-4af9-4259-88b1-9dca881cf14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:01.803024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16091b53-4af9-4259-88b1-9dca881cf14d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:01.821381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9810c8f-ae52-44b7-8ec0-d524a38a03ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:01.822154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9810c8f-ae52-44b7-8ec0-d524a38a03ca service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:01.822430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9810c8f-ae52-44b7-8ec0-d524a38a03ca service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:01.822702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9810c8f-ae52-44b7-8ec0-d524a38a03ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:02.017638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9810c8f-ae52-44b7-8ec0-d524a38a03ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:02.019557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9810c8f-ae52-44b7-8ec0-d524a38a03ca service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:02.052391 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1259/2538] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:05.089877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2456bb-f812-406a-94a7-bc911dfc8a83 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:05.090953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2456bb-f812-406a-94a7-bc911dfc8a83 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:05.091374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2456bb-f812-406a-94a7-bc911dfc8a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:05.138928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6e2456bb-f812-406a-94a7-bc911dfc8a83 None None] MFA Rules not processed for user `396027dc8be44f91805a44d83be35e29`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:05.140200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6e2456bb-f812-406a-94a7-bc911dfc8a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:05.311626 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1280/2539] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:05.329319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9810c8f-ae52-44b7-8ec0-d524a38a03ca service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:05.343301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9810c8f-ae52-44b7-8ec0-d524a38a03ca service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:05.362325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b71236-68f1-4062-8c0d-0f6ba340ecd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:05.363243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b71236-68f1-4062-8c0d-0f6ba340ecd4 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:05.363697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b71236-68f1-4062-8c0d-0f6ba340ecd4 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:05.364175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b71236-68f1-4062-8c0d-0f6ba340ecd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:05.369083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c1b71236-68f1-4062-8c0d-0f6ba340ecd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:05.391998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2456bb-f812-406a-94a7-bc911dfc8a83 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:05.393096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:05.395751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6e2456bb-f812-406a-94a7-bc911dfc8a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:05.437147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2456bb-f812-406a-94a7-bc911dfc8a83 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:05.451221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced30d4f-c484-4b62-9659-42c76c71179d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:05.452260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced30d4f-c484-4b62-9659-42c76c71179d service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:05.452797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced30d4f-c484-4b62-9659-42c76c71179d service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:05.453547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced30d4f-c484-4b62-9659-42c76c71179d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:05.492217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced30d4f-c484-4b62-9659-42c76c71179d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:05.494250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced30d4f-c484-4b62-9659-42c76c71179d service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:05.534122 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1281/2540] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:05 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:05.547203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b71236-68f1-4062-8c0d-0f6ba340ecd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:05.549181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b71236-68f1-4062-8c0d-0f6ba340ecd4 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:05.580698 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1260/2541] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:05.922324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce876624-13be-4a56-87df-48b4157613bf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:05.922585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce876624-13be-4a56-87df-48b4157613bf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:05.922870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce876624-13be-4a56-87df-48b4157613bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:05.982995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ce876624-13be-4a56-87df-48b4157613bf None None] MFA Rules not processed for user `0802ea75a81747e68e9bfa7d11ef50dc`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:05.984350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ce876624-13be-4a56-87df-48b4157613bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:06.162954 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1282/2542] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:06.166450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2797226-2c0b-4166-8421-3e16758a38b6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:06.166728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2797226-2c0b-4166-8421-3e16758a38b6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:06.167017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2797226-2c0b-4166-8421-3e16758a38b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:06.253061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b2797226-2c0b-4166-8421-3e16758a38b6 None None] MFA Rules not processed for user `8afc45f50dc54b84a8abf3b6e168426b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:06.254763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b2797226-2c0b-4166-8421-3e16758a38b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:06.414955 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1261/2543] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:06.590402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce876624-13be-4a56-87df-48b4157613bf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:06.633881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce876624-13be-4a56-87df-48b4157613bf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:06.654884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf1e83-18f8-4f7b-98a1-4bdce5e07ea1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:06.656643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf1e83-18f8-4f7b-98a1-4bdce5e07ea1 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:06.666033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf1e83-18f8-4f7b-98a1-4bdce5e07ea1 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:06.666666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf1e83-18f8-4f7b-98a1-4bdce5e07ea1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:06.670181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3ecf1e83-18f8-4f7b-98a1-4bdce5e07ea1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:06.701893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:06.708525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf1e83-18f8-4f7b-98a1-4bdce5e07ea1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:06.713122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf1e83-18f8-4f7b-98a1-4bdce5e07ea1 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:06.762855 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1283/2544] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:09.926960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd5c0d-0955-4eff-9c5c-e4c8928eb36f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:09.927958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd5c0d-0955-4eff-9c5c-e4c8928eb36f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:09.929585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd5c0d-0955-4eff-9c5c-e4c8928eb36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:09.990823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2cdd5c0d-0955-4eff-9c5c-e4c8928eb36f None None] MFA Rules not processed for user `8cc1474bc2a1450eac2d15554d5fcd11`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:09.993106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2cdd5c0d-0955-4eff-9c5c-e4c8928eb36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:10.193918 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1262/2545] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:10.832422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee62bea-ce6b-4536-ab21-48349ebd86b5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:10.832994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee62bea-ce6b-4536-ab21-48349ebd86b5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:10.832994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee62bea-ce6b-4536-ab21-48349ebd86b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:10.872615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1ee62bea-ce6b-4536-ab21-48349ebd86b5 None None] MFA Rules not processed for user `1762f75263ad4e2f84b265522e76764d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:10.873836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1ee62bea-ce6b-4536-ab21-48349ebd86b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:10.888909 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1284/2546] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:12.511009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674ab3f-042c-4b7e-b7ab-720408fa928c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:12.511684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674ab3f-042c-4b7e-b7ab-720408fa928c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:12.511684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674ab3f-042c-4b7e-b7ab-720408fa928c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:12.620045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b674ab3f-042c-4b7e-b7ab-720408fa928c None None] MFA Rules not processed for user `ae7d0d25bbf24edda377a9cc2e5a59e6`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:12.621649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b674ab3f-042c-4b7e-b7ab-720408fa928c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:12.751830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:12.799240 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1263/2547] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:12 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:12.809782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cefbc0-8821-4b2e-b940-7f672d588267 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:12.810307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cefbc0-8821-4b2e-b940-7f672d588267 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:12.810307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cefbc0-8821-4b2e-b940-7f672d588267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:12.858166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e1cefbc0-8821-4b2e-b940-7f672d588267 None None] MFA Rules not processed for user `7c276ebf6fe14a9c8f50bd1d1f44fb58`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:12.860120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e1cefbc0-8821-4b2e-b940-7f672d588267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:12.960855 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1285/2548] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:12 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:14.512310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674ab3f-042c-4b7e-b7ab-720408fa928c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:14.531718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674ab3f-042c-4b7e-b7ab-720408fa928c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:14.538426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11d446-4cea-4e8d-9d07-12d68e969cf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:14.539216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11d446-4cea-4e8d-9d07-12d68e969cf2 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:14.539522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11d446-4cea-4e8d-9d07-12d68e969cf2 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:14.539882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11d446-4cea-4e8d-9d07-12d68e969cf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:14.542027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0a11d446-4cea-4e8d-9d07-12d68e969cf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:14.555998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11d446-4cea-4e8d-9d07-12d68e969cf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:14.557994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11d446-4cea-4e8d-9d07-12d68e969cf2 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:14.582538 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1264/2549] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:14 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:20.612077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cefbc0-8821-4b2e-b940-7f672d588267 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:20.704535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cefbc0-8821-4b2e-b940-7f672d588267 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:20.710181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:20.710836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:20.711107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:20.711371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] PATH_INFO: `/v3/users/19d39dd3a41744b7ab28c162d6748113` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:20.712516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d39dd3a41744b7ab28c162d6748113)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:20.714451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:20.770855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d39dd3a41744b7ab28c162d6748113'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:20.781323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d39dd3a41744b7ab28c162d6748113'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:20.794349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d39dd3a41744b7ab28c162d6748113'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:20.807965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d39dd3a41744b7ab28c162d6748113'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:20.808983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d39dd3a41744b7ab28c162d6748113'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:20.815274 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1286/2550] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:57:20 2022] DELETE /identity/v3/users/19d39dd3a41744b7ab28c162d6748113 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:20.820316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdf933-eacd-4e13-a244-1d1af468c2a7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:20.820850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdf933-eacd-4e13-a244-1d1af468c2a7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:20.820946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdf933-eacd-4e13-a244-1d1af468c2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:20.841223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-efbdf933-eacd-4e13-a244-1d1af468c2a7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:20.843047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-efbdf933-eacd-4e13-a244-1d1af468c2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:20.920639 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1265/2551] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:20.935284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:20.939848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:20.964545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:21.017786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd834d1-d8a4-4a76-98d7-040985fc7517 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:21.021649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529d760-65bb-4ac8-a47c-df085f8f3482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:21.022250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529d760-65bb-4ac8-a47c-df085f8f3482 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:21.022520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529d760-65bb-4ac8-a47c-df085f8f3482 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:21.022784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529d760-65bb-4ac8-a47c-df085f8f3482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:21.032276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529d760-65bb-4ac8-a47c-df085f8f3482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:21.033785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529d760-65bb-4ac8-a47c-df085f8f3482 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:21.043544 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1287/2552] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:21.376743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdf933-eacd-4e13-a244-1d1af468c2a7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:21.378495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-efbdf933-eacd-4e13-a244-1d1af468c2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:21.392330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdf933-eacd-4e13-a244-1d1af468c2a7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:21.399752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:21.400823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:21.401311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:21.401807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] PATH_INFO: `/v3/projects/6f09ec34654e4996893d91d215b5c3d8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:21.403461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f09ec34654e4996893d91d215b5c3d8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:21.407174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:21.423415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529d760-65bb-4ac8-a47c-df085f8f3482 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:21.425101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f529d760-65bb-4ac8-a47c-df085f8f3482 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:21.464677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f09ec34654e4996893d91d215b5c3d8', 'request_id': 'req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:21.471336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f09ec34654e4996893d91d215b5c3d8', 'request_id': 'req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:21.479684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f09ec34654e4996893d91d215b5c3d8', 'request_id': 'req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:21.493705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] The token cache is being invalidate because project 6f09ec34654e4996893d91d215b5c3d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:57:21.494016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:21.495627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:21.498023 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1266/2553] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:57:21 2022] DELETE /identity/v3/projects/6f09ec34654e4996893d91d215b5c3d8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:21.502531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:21.505487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:21.512588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529d760-65bb-4ac8-a47c-df085f8f3482 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:21.564107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:21.582420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e9dd5-6d86-4e07-a4fa-be9d5c64ffff admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:21.588720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:21.589764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:21.590651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:21.590931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] PATH_INFO: `/v3/users/468cf602681141a793eccac6e443b1a7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:21.592093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] RBAC: Authorizing `identity:delete_user(user_id=468cf602681141a793eccac6e443b1a7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:21.594300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:21.595196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7e48b-3986-4fb5-9197-79c9eb41ff8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:21.596065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7e48b-3986-4fb5-9197-79c9eb41ff8e service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:21.596336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7e48b-3986-4fb5-9197-79c9eb41ff8e service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:21.596675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7e48b-3986-4fb5-9197-79c9eb41ff8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:21.601599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-82e7e48b-3986-4fb5-9197-79c9eb41ff8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:21.652868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468cf602681141a793eccac6e443b1a7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:21.661346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468cf602681141a793eccac6e443b1a7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:21.674329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468cf602681141a793eccac6e443b1a7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:21.675034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468cf602681141a793eccac6e443b1a7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:21.675763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468cf602681141a793eccac6e443b1a7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:21.676242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:21.700891 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1267/2554] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:57:21 2022] DELETE /identity/v3/users/468cf602681141a793eccac6e443b1a7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:57:21.701366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7e48b-3986-4fb5-9197-79c9eb41ff8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:21.703219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7e48b-3986-4fb5-9197-79c9eb41ff8e service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:21.716278 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1288/2555] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:57:22.056117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:22.058794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:22.208663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4b1aa-4476-452e-b108-26b9e8ecfddc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:22.216077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9dced-b2ab-4538-91f9-c8931f94b280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:22.217202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9dced-b2ab-4538-91f9-c8931f94b280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:22.217686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9dced-b2ab-4538-91f9-c8931f94b280 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:22.219001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9dced-b2ab-4538-91f9-c8931f94b280 admin admin] PATH_INFO: `/v3/projects/f3fd3099748c4f06847a34a463bced28` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:22.221195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9dced-b2ab-4538-91f9-c8931f94b280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3fd3099748c4f06847a34a463bced28)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:22.243589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9dced-b2ab-4538-91f9-c8931f94b280 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:22.319093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bec9dced-b2ab-4538-91f9-c8931f94b280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3fd3099748c4f06847a34a463bced28', 'request_id': 'req-bec9dced-b2ab-4538-91f9-c8931f94b280'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:22.333984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bec9dced-b2ab-4538-91f9-c8931f94b280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3fd3099748c4f06847a34a463bced28', 'request_id': 'req-bec9dced-b2ab-4538-91f9-c8931f94b280'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:22.355659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bec9dced-b2ab-4538-91f9-c8931f94b280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3fd3099748c4f06847a34a463bced28', 'request_id': 'req-bec9dced-b2ab-4538-91f9-c8931f94b280'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:22.383373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bec9dced-b2ab-4538-91f9-c8931f94b280 admin admin] The token cache is being invalidate because project f3fd3099748c4f06847a34a463bced28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:57:22.384287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bec9dced-b2ab-4538-91f9-c8931f94b280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:22.385352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bec9dced-b2ab-4538-91f9-c8931f94b280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:22.388424 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1268/2556] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:57:22 2022] DELETE /identity/v3/projects/f3fd3099748c4f06847a34a463bced28 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:22.392461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b0bb3-dafe-492f-a609-5cff2c10021a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:22.392743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b0bb3-dafe-492f-a609-5cff2c10021a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:22.393006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b0bb3-dafe-492f-a609-5cff2c10021a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:22.434498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-827b0bb3-dafe-492f-a609-5cff2c10021a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:22.436884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-827b0bb3-dafe-492f-a609-5cff2c10021a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:22.549874 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1289/2557] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:22.558257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c222d9-e607-4cb4-bfd4-e0621e3104d0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:22.560805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c222d9-e607-4cb4-bfd4-e0621e3104d0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:22.562176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c222d9-e607-4cb4-bfd4-e0621e3104d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:22.576784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:22.619095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d3c222d9-e607-4cb4-bfd4-e0621e3104d0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:22.620486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d3c222d9-e607-4cb4-bfd4-e0621e3104d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:22.629728 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1269/2558] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:22.634255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b0bb3-dafe-492f-a609-5cff2c10021a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:22.652266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b0bb3-dafe-492f-a609-5cff2c10021a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:22.659995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985808af-054a-442b-9d0b-ab21c1f7e79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:22.660685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985808af-054a-442b-9d0b-ab21c1f7e79f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:22.660954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985808af-054a-442b-9d0b-ab21c1f7e79f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:22.661270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985808af-054a-442b-9d0b-ab21c1f7e79f admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:22.661979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985808af-054a-442b-9d0b-ab21c1f7e79f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:57:22.662272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985808af-054a-442b-9d0b-ab21c1f7e79f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:22.663882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985808af-054a-442b-9d0b-ab21c1f7e79f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:22.678292 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1290/2559] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:57:22.682529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c222d9-e607-4cb4-bfd4-e0621e3104d0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:22.693721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c222d9-e607-4cb4-bfd4-e0621e3104d0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:22.710803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8564117-e2f2-43a5-9e22-96b555fad9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:22.711989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8564117-e2f2-43a5-9e22-96b555fad9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:22.712647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8564117-e2f2-43a5-9e22-96b555fad9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:22.713317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8564117-e2f2-43a5-9e22-96b555fad9b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:22.714152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8564117-e2f2-43a5-9e22-96b555fad9b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:22.716155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8564117-e2f2-43a5-9e22-96b555fad9b8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:22.762031 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1270/2560] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:57:22 2022] POST /identity/v3/projects => generated 395 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:57:22.767819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985808af-054a-442b-9d0b-ab21c1f7e79f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:22.779825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985808af-054a-442b-9d0b-ab21c1f7e79f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:22.858245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670343c-b66c-4567-9f25-5ff35eef2434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:22.859048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670343c-b66c-4567-9f25-5ff35eef2434 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:22.859489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670343c-b66c-4567-9f25-5ff35eef2434 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:22.859770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670343c-b66c-4567-9f25-5ff35eef2434 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:22.861612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670343c-b66c-4567-9f25-5ff35eef2434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:22.863586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670343c-b66c-4567-9f25-5ff35eef2434 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:22.893840 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1291/2561] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:57:22 2022] POST /identity/v3/users => generated 298 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:57:22.898898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8564117-e2f2-43a5-9e22-96b555fad9b8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:22.912338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8564117-e2f2-43a5-9e22-96b555fad9b8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:22.927065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb57de5-5fda-4d3f-9eb4-ee2c2de2f61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:22.930479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb57de5-5fda-4d3f-9eb4-ee2c2de2f61a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:22.931223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb57de5-5fda-4d3f-9eb4-ee2c2de2f61a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:22.931223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb57de5-5fda-4d3f-9eb4-ee2c2de2f61a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:22.932032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb57de5-5fda-4d3f-9eb4-ee2c2de2f61a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:22.933776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb57de5-5fda-4d3f-9eb4-ee2c2de2f61a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:22.955200 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1271/2562] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:57:22.959547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670343c-b66c-4567-9f25-5ff35eef2434 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:22.978936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670343c-b66c-4567-9f25-5ff35eef2434 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:22.990244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf18a69-b687-48ea-8971-9642a7212d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:22.990879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf18a69-b687-48ea-8971-9642a7212d19 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:22.991149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf18a69-b687-48ea-8971-9642a7212d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:22.991412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf18a69-b687-48ea-8971-9642a7212d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:22.992551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf18a69-b687-48ea-8971-9642a7212d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:22.994767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf18a69-b687-48ea-8971-9642a7212d19 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:23.014251 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1292/2563] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:57:23.015713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb57de5-5fda-4d3f-9eb4-ee2c2de2f61a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:23.027134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb57de5-5fda-4d3f-9eb4-ee2c2de2f61a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:23.033616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844c107-5d99-4607-934e-a8af0d37e91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:23.034604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844c107-5d99-4607-934e-a8af0d37e91e admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:23.034937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844c107-5d99-4607-934e-a8af0d37e91e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:23.035265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844c107-5d99-4607-934e-a8af0d37e91e admin admin] PATH_INFO: `/v3/projects/55a14ddedf134193990792b64303f623/users/0827c37c91704226a0091adedd1d6e89/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:23.065106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844c107-5d99-4607-934e-a8af0d37e91e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55a14ddedf134193990792b64303f623, user_id=0827c37c91704226a0091adedd1d6e89, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:23.070692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844c107-5d99-4607-934e-a8af0d37e91e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:23.104434 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1272/2564] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:57:23 2022] PUT /identity/v3/projects/55a14ddedf134193990792b64303f623/users/0827c37c91704226a0091adedd1d6e89/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:23.116120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a9ebf-97e8-474e-92eb-05051638bf81 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:23.116428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a9ebf-97e8-474e-92eb-05051638bf81 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:23.116736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a9ebf-97e8-474e-92eb-05051638bf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:23.168940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c83a9ebf-97e8-474e-92eb-05051638bf81 None None] MFA Rules not processed for user `0827c37c91704226a0091adedd1d6e89`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:23.170312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c83a9ebf-97e8-474e-92eb-05051638bf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:23.220511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:23.248612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844c107-5d99-4607-934e-a8af0d37e91e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:23.250692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8844c107-5d99-4607-934e-a8af0d37e91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:23.252587 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1293/2565] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:23 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:23.257091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a9ebf-97e8-474e-92eb-05051638bf81 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:23.324055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a9ebf-97e8-474e-92eb-05051638bf81 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:23.328881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f89d57-5982-4e59-94d7-9755ab3a12f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:23.329477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f89d57-5982-4e59-94d7-9755ab3a12f0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:23.329747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f89d57-5982-4e59-94d7-9755ab3a12f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:23.330013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f89d57-5982-4e59-94d7-9755ab3a12f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:23.330646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f89d57-5982-4e59-94d7-9755ab3a12f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:23.332609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f89d57-5982-4e59-94d7-9755ab3a12f0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:23.333148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844c107-5d99-4607-934e-a8af0d37e91e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:23.338402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855780e-a042-4b32-b49d-77656c6ec1b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:23.339007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855780e-a042-4b32-b49d-77656c6ec1b3 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:23.339277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855780e-a042-4b32-b49d-77656c6ec1b3 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:23.339541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855780e-a042-4b32-b49d-77656c6ec1b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:23.341360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2855780e-a042-4b32-b49d-77656c6ec1b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:23.353866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855780e-a042-4b32-b49d-77656c6ec1b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:23.355733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855780e-a042-4b32-b49d-77656c6ec1b3 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:23.360486 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1294/2566] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:57:23 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:57:23.364766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f89d57-5982-4e59-94d7-9755ab3a12f0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:23.367381 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1273/2567] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:23.437445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f89d57-5982-4e59-94d7-9755ab3a12f0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:23.442263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37696b-0d2f-45ec-8e05-a78694f26ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:23.442873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37696b-0d2f-45ec-8e05-a78694f26ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:23.443140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37696b-0d2f-45ec-8e05-a78694f26ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:23.443403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37696b-0d2f-45ec-8e05-a78694f26ce9 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:23.444135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37696b-0d2f-45ec-8e05-a78694f26ce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:23.446415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37696b-0d2f-45ec-8e05-a78694f26ce9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:23.468422 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1295/2568] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:57:23 2022] POST /identity/v3/users => generated 303 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:57:23.472419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855780e-a042-4b32-b49d-77656c6ec1b3 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:23.483578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855780e-a042-4b32-b49d-77656c6ec1b3 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:23.490285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deae1e7-d91a-4b23-bdc7-b4d5865f8bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:23.490922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deae1e7-d91a-4b23-bdc7-b4d5865f8bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:23.491191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deae1e7-d91a-4b23-bdc7-b4d5865f8bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:23.491447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deae1e7-d91a-4b23-bdc7-b4d5865f8bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:23.492089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deae1e7-d91a-4b23-bdc7-b4d5865f8bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:23.493802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deae1e7-d91a-4b23-bdc7-b4d5865f8bd5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:23.505575 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1274/2569] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:57:23.510193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37696b-0d2f-45ec-8e05-a78694f26ce9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:23.521863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37696b-0d2f-45ec-8e05-a78694f26ce9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:23.527516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4de0d-5035-4a00-96fd-bec7d1e56671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:23.528465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4de0d-5035-4a00-96fd-bec7d1e56671 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:23.528736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4de0d-5035-4a00-96fd-bec7d1e56671 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:23.529079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4de0d-5035-4a00-96fd-bec7d1e56671 admin admin] PATH_INFO: `/v3/projects/531e1065a7794e7696b7774405265674/users/a1ce0dc2349f4b1f8fdbe8800ce22190/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:23.544840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4de0d-5035-4a00-96fd-bec7d1e56671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=531e1065a7794e7696b7774405265674, user_id=a1ce0dc2349f4b1f8fdbe8800ce22190, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:23.547624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4de0d-5035-4a00-96fd-bec7d1e56671 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:23.566471 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1296/2570] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:57:23 2022] PUT /identity/v3/projects/531e1065a7794e7696b7774405265674/users/a1ce0dc2349f4b1f8fdbe8800ce22190/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:57:23.571005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deae1e7-d91a-4b23-bdc7-b4d5865f8bd5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:23.583756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deae1e7-d91a-4b23-bdc7-b4d5865f8bd5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:23.655380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97ccba-96e7-464c-a363-a9eb15e5175f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:23.656687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ccba-96e7-464c-a363-a9eb15e5175f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:23.657089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ccba-96e7-464c-a363-a9eb15e5175f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:23.657430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ccba-96e7-464c-a363-a9eb15e5175f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:23.658274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97ccba-96e7-464c-a363-a9eb15e5175f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:23.660612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97ccba-96e7-464c-a363-a9eb15e5175f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:23.674477 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1275/2571] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:57:23.675529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4de0d-5035-4a00-96fd-bec7d1e56671 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:23.687188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4de0d-5035-4a00-96fd-bec7d1e56671 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:23.695466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961f04c-a317-4d74-a825-dc6847a63260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:23.696149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961f04c-a317-4d74-a825-dc6847a63260 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:23.696438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961f04c-a317-4d74-a825-dc6847a63260 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:23.696838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961f04c-a317-4d74-a825-dc6847a63260 admin admin] PATH_INFO: `/v3/projects/531e1065a7794e7696b7774405265674/users/a1ce0dc2349f4b1f8fdbe8800ce22190/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:23.699071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961f04c-a317-4d74-a825-dc6847a63260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=531e1065a7794e7696b7774405265674, user_id=a1ce0dc2349f4b1f8fdbe8800ce22190, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:23.701564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961f04c-a317-4d74-a825-dc6847a63260 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:23.713238 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1297/2572] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:57:23 2022] PUT /identity/v3/projects/531e1065a7794e7696b7774405265674/users/a1ce0dc2349f4b1f8fdbe8800ce22190/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:23.724824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab290cc9-6ae1-40e6-848f-64437ac270c8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:23.725104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab290cc9-6ae1-40e6-848f-64437ac270c8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:23.725374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab290cc9-6ae1-40e6-848f-64437ac270c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:23.756201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:23.760051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ab290cc9-6ae1-40e6-848f-64437ac270c8 None None] MFA Rules not processed for user `a1ce0dc2349f4b1f8fdbe8800ce22190`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:23.762390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ab290cc9-6ae1-40e6-848f-64437ac270c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:23.836918 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1276/2573] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:23 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:57:23.847650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961f04c-a317-4d74-a825-dc6847a63260 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:23.849553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7961f04c-a317-4d74-a825-dc6847a63260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:23.872826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:23.927746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961f04c-a317-4d74-a825-dc6847a63260 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:23.934494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e40f8-7b70-4f27-8e83-e9f7918321d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:23.935100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e40f8-7b70-4f27-8e83-e9f7918321d9 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:23.935364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e40f8-7b70-4f27-8e83-e9f7918321d9 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:23.935624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e40f8-7b70-4f27-8e83-e9f7918321d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:23.946173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e40f8-7b70-4f27-8e83-e9f7918321d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:23.947758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e40f8-7b70-4f27-8e83-e9f7918321d9 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:23.957642 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1298/2574] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:23 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:24.732507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab290cc9-6ae1-40e6-848f-64437ac270c8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:24.745455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab290cc9-6ae1-40e6-848f-64437ac270c8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:24.756224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa2839-f1e3-4745-87b5-d3169ae7c316 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:24.756884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa2839-f1e3-4745-87b5-d3169ae7c316 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:24.757151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa2839-f1e3-4745-87b5-d3169ae7c316 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:24.757419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa2839-f1e3-4745-87b5-d3169ae7c316 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:24.818529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa2839-f1e3-4745-87b5-d3169ae7c316 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:24.820315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa2839-f1e3-4745-87b5-d3169ae7c316 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:24.840826 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1277/2575] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:24 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:24.849387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e40f8-7b70-4f27-8e83-e9f7918321d9 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:24.919692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e40f8-7b70-4f27-8e83-e9f7918321d9 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:24.924638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:24.925284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:24.925552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:24.925813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] PATH_INFO: `/v3/users/0827c37c91704226a0091adedd1d6e89` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:24.927060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0827c37c91704226a0091adedd1d6e89)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:24.929646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:24.988672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0827c37c91704226a0091adedd1d6e89'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:24.997576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0827c37c91704226a0091adedd1d6e89'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.007397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0827c37c91704226a0091adedd1d6e89'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.022510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0827c37c91704226a0091adedd1d6e89'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.023365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0827c37c91704226a0091adedd1d6e89'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.029924 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1299/2576] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:57:24 2022] DELETE /identity/v3/users/0827c37c91704226a0091adedd1d6e89 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:25.038238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ab034-95b7-4d4c-8322-4ae12ec6c8fc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:25.038617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ab034-95b7-4d4c-8322-4ae12ec6c8fc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:25.038982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ab034-95b7-4d4c-8322-4ae12ec6c8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:25.059115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-335ab034-95b7-4d4c-8322-4ae12ec6c8fc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:25.060503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-335ab034-95b7-4d4c-8322-4ae12ec6c8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:25.121299 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1278/2577] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:25.131132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:25.132608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:25.160610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:25.191646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf871e5-40ad-4d5f-bd35-985edd4a05cc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:25.196272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368b06f-c702-46ed-b7e8-0be0fab1f79d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:25.196931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368b06f-c702-46ed-b7e8-0be0fab1f79d service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:25.197200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368b06f-c702-46ed-b7e8-0be0fab1f79d service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:25.197486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368b06f-c702-46ed-b7e8-0be0fab1f79d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:25.207359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368b06f-c702-46ed-b7e8-0be0fab1f79d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:25.208882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368b06f-c702-46ed-b7e8-0be0fab1f79d service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:25.219080 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1300/2578] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:25.606292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ab034-95b7-4d4c-8322-4ae12ec6c8fc None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:25.607843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-335ab034-95b7-4d4c-8322-4ae12ec6c8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:25.621266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ab034-95b7-4d4c-8322-4ae12ec6c8fc None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:25.628228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:25.629056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:25.629463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:25.629733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] PATH_INFO: `/v3/projects/55a14ddedf134193990792b64303f623` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:25.638047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55a14ddedf134193990792b64303f623)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:25.640115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:25.679689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:25.695298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55a14ddedf134193990792b64303f623', 'request_id': 'req-374e0db4-493b-4b83-9ac5-7c98cc7c5393'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.701940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55a14ddedf134193990792b64303f623', 'request_id': 'req-374e0db4-493b-4b83-9ac5-7c98cc7c5393'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.711600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55a14ddedf134193990792b64303f623', 'request_id': 'req-374e0db4-493b-4b83-9ac5-7c98cc7c5393'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.723194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] The token cache is being invalidate because project 55a14ddedf134193990792b64303f623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:57:25.723467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.724089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.725817 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1279/2579] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:57:25 2022] DELETE /identity/v3/projects/55a14ddedf134193990792b64303f623 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:25.729945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368b06f-c702-46ed-b7e8-0be0fab1f79d service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:25.744399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368b06f-c702-46ed-b7e8-0be0fab1f79d service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:25.851552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea137ad5-0e48-4d9c-ad0f-6b56534efc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:25.852250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea137ad5-0e48-4d9c-ad0f-6b56534efc9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:25.852509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea137ad5-0e48-4d9c-ad0f-6b56534efc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:25.852775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea137ad5-0e48-4d9c-ad0f-6b56534efc9c admin admin] PATH_INFO: `/v3/users/a1ce0dc2349f4b1f8fdbe8800ce22190` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:25.853885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea137ad5-0e48-4d9c-ad0f-6b56534efc9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ce0dc2349f4b1f8fdbe8800ce22190)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:25.855796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea137ad5-0e48-4d9c-ad0f-6b56534efc9c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:25.913719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea137ad5-0e48-4d9c-ad0f-6b56534efc9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ce0dc2349f4b1f8fdbe8800ce22190'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.923639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea137ad5-0e48-4d9c-ad0f-6b56534efc9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ce0dc2349f4b1f8fdbe8800ce22190'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.932831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea137ad5-0e48-4d9c-ad0f-6b56534efc9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ce0dc2349f4b1f8fdbe8800ce22190'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.949854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea137ad5-0e48-4d9c-ad0f-6b56534efc9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ce0dc2349f4b1f8fdbe8800ce22190'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.950646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea137ad5-0e48-4d9c-ad0f-6b56534efc9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ce0dc2349f4b1f8fdbe8800ce22190'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:25.957842 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1301/2580] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:57:25 2022] DELETE /identity/v3/users/a1ce0dc2349f4b1f8fdbe8800ce22190 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:57:26.380931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:26.383257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:26.403219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e0db4-493b-4b83-9ac5-7c98cc7c5393 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:26.422717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce3c8f-6332-40a9-9563-36071c5f430d None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:26.423232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce3c8f-6332-40a9-9563-36071c5f430d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:26.423854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce3c8f-6332-40a9-9563-36071c5f430d None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:26.426085 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1302/2581] 10.222.0.23 () {54 vars in 921 bytes} [Sat Aug 20 20:57:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:57:26.442444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6fea1-a1fe-4c48-b8f0-ba613b99e281 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:26.442444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6fea1-a1fe-4c48-b8f0-ba613b99e281 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:26.442444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6fea1-a1fe-4c48-b8f0-ba613b99e281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:26.481374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f9a6fea1-a1fe-4c48-b8f0-ba613b99e281 None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:26.482699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f9a6fea1-a1fe-4c48-b8f0-ba613b99e281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:26.517124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:26.517596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:26.517937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:26.518296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] PATH_INFO: `/v3/projects/531e1065a7794e7696b7774405265674` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:26.519651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=531e1065a7794e7696b7774405265674)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:26.522094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:26.560753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:26.583912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '531e1065a7794e7696b7774405265674', 'request_id': 'req-12be2d29-621d-4bf2-a04c-936005bb30c5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:26.594988 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1303/2582] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:57:26.603600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '531e1065a7794e7696b7774405265674', 'request_id': 'req-12be2d29-621d-4bf2-a04c-936005bb30c5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:26.614515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '531e1065a7794e7696b7774405265674', 'request_id': 'req-12be2d29-621d-4bf2-a04c-936005bb30c5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:26.628866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] The token cache is being invalidate because project 531e1065a7794e7696b7774405265674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:57:26.629163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:26.630227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:26.632418 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1280/2583] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:57:26 2022] DELETE /identity/v3/projects/531e1065a7794e7696b7774405265674 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:26.636889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdef95b-da89-4491-815b-c4bfd8434046 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:26.637306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdef95b-da89-4491-815b-c4bfd8434046 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:26.637654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdef95b-da89-4491-815b-c4bfd8434046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:26.642045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:26.644468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:26.660738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3cdef95b-da89-4491-815b-c4bfd8434046 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:26.662679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3cdef95b-da89-4491-815b-c4bfd8434046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:26.736401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be2d29-621d-4bf2-a04c-936005bb30c5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:26.740544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248dc059-92c1-44f9-91a9-5d43bee2e3db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:26.741250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248dc059-92c1-44f9-91a9-5d43bee2e3db service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:26.741713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248dc059-92c1-44f9-91a9-5d43bee2e3db service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:26.742029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248dc059-92c1-44f9-91a9-5d43bee2e3db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:26.743997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-248dc059-92c1-44f9-91a9-5d43bee2e3db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:26.749646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:26.761157 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1304/2584] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:26.770391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812303d-ccc7-46d0-bca3-ece1dacb3cbf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:26.770667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812303d-ccc7-46d0-bca3-ece1dacb3cbf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:26.771081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812303d-ccc7-46d0-bca3-ece1dacb3cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:26.795984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f812303d-ccc7-46d0-bca3-ece1dacb3cbf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:26.797937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f812303d-ccc7-46d0-bca3-ece1dacb3cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:26.808093 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1305/2585] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:26.812316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812303d-ccc7-46d0-bca3-ece1dacb3cbf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:26.823862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812303d-ccc7-46d0-bca3-ece1dacb3cbf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:26.828803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5babe0c4-7a7d-4d2a-839a-216d23a9933c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:26.829490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5babe0c4-7a7d-4d2a-839a-216d23a9933c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:26.829818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5babe0c4-7a7d-4d2a-839a-216d23a9933c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:26.830144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5babe0c4-7a7d-4d2a-839a-216d23a9933c admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:26.830985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5babe0c4-7a7d-4d2a-839a-216d23a9933c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:57:26.831347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5babe0c4-7a7d-4d2a-839a-216d23a9933c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:26.833200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5babe0c4-7a7d-4d2a-839a-216d23a9933c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:26.838151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248dc059-92c1-44f9-91a9-5d43bee2e3db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:26.839604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248dc059-92c1-44f9-91a9-5d43bee2e3db service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:26.847137 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1306/2586] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:57:26.851332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5babe0c4-7a7d-4d2a-839a-216d23a9933c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:26.854892 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1281/2587] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:26.862524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5babe0c4-7a7d-4d2a-839a-216d23a9933c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:26.878552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f9058-ac08-4a6b-9a6f-5f0cfe5c421a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:26.879178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f9058-ac08-4a6b-9a6f-5f0cfe5c421a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:26.879444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f9058-ac08-4a6b-9a6f-5f0cfe5c421a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:26.879703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f9058-ac08-4a6b-9a6f-5f0cfe5c421a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:26.880322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f9058-ac08-4a6b-9a6f-5f0cfe5c421a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:26.882590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb2fa2-8032-4b86-97ef-650e72073913 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:26.882717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f9058-ac08-4a6b-9a6f-5f0cfe5c421a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:26.882962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb2fa2-8032-4b86-97ef-650e72073913 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:26.883139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb2fa2-8032-4b86-97ef-650e72073913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:26.921166 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1307/2588] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:57:26 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:57:26.924792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a3fb2fa2-8032-4b86-97ef-650e72073913 None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:26.925435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f9058-ac08-4a6b-9a6f-5f0cfe5c421a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:26.926019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a3fb2fa2-8032-4b86-97ef-650e72073913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:26.998448 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1282/2589] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:27.011301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb2fa2-8032-4b86-97ef-650e72073913 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:27.011556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f9058-ac08-4a6b-9a6f-5f0cfe5c421a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:27.021661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb2fa2-8032-4b86-97ef-650e72073913 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:27.024652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466da5d6-d63e-455a-b9b4-64e48676afd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:27.025504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466da5d6-d63e-455a-b9b4-64e48676afd4 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:27.025781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466da5d6-d63e-455a-b9b4-64e48676afd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:27.026044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466da5d6-d63e-455a-b9b4-64e48676afd4 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:27.026997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466da5d6-d63e-455a-b9b4-64e48676afd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:27.029850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466da5d6-d63e-455a-b9b4-64e48676afd4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:27.032911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e99f50-e838-4962-b149-167d787070df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:27.033542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e99f50-e838-4962-b149-167d787070df service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:27.033864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e99f50-e838-4962-b149-167d787070df service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:27.034092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e99f50-e838-4962-b149-167d787070df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:27.044703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e99f50-e838-4962-b149-167d787070df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:27.047134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e99f50-e838-4962-b149-167d787070df service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:27.056591 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1283/2590] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:27.064631 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1308/2591] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:57:26 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:57:27.065800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e99f50-e838-4962-b149-167d787070df service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:27.077474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e99f50-e838-4962-b149-167d787070df service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:27.082863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecdc10-3c56-4f36-b0e2-9ec03e3ad902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:27.083237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecdc10-3c56-4f36-b0e2-9ec03e3ad902 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:27.083694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecdc10-3c56-4f36-b0e2-9ec03e3ad902 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:27.084138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecdc10-3c56-4f36-b0e2-9ec03e3ad902 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:27.084889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecdc10-3c56-4f36-b0e2-9ec03e3ad902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:27.086781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecdc10-3c56-4f36-b0e2-9ec03e3ad902 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:27.098044 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1284/2592] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:57:27.102427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466da5d6-d63e-455a-b9b4-64e48676afd4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:27.116176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466da5d6-d63e-455a-b9b4-64e48676afd4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:27.122331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a54aa1-0ba6-4ffc-af7c-ebd1c23ef404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:27.123017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a54aa1-0ba6-4ffc-af7c-ebd1c23ef404 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:27.123352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a54aa1-0ba6-4ffc-af7c-ebd1c23ef404 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:27.123698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a54aa1-0ba6-4ffc-af7c-ebd1c23ef404 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:27.126190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a54aa1-0ba6-4ffc-af7c-ebd1c23ef404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:27.126352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a54aa1-0ba6-4ffc-af7c-ebd1c23ef404 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:27.152712 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1309/2593] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:57:27.157007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecdc10-3c56-4f36-b0e2-9ec03e3ad902 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:27.170067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecdc10-3c56-4f36-b0e2-9ec03e3ad902 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:27.176509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c689a-3972-4c39-9f42-6927ea04f404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:27.177188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c689a-3972-4c39-9f42-6927ea04f404 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:27.177440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c689a-3972-4c39-9f42-6927ea04f404 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:27.177706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c689a-3972-4c39-9f42-6927ea04f404 admin admin] PATH_INFO: `/v3/projects/5e852b41b6d643e7a85f8f8a660ffaf7/users/e5f95390cf634275a05255be32cbdacc/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:27.206516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c689a-3972-4c39-9f42-6927ea04f404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e852b41b6d643e7a85f8f8a660ffaf7, user_id=e5f95390cf634275a05255be32cbdacc, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:27.209258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c689a-3972-4c39-9f42-6927ea04f404 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:27.236407 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1285/2594] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:57:27 2022] PUT /identity/v3/projects/5e852b41b6d643e7a85f8f8a660ffaf7/users/e5f95390cf634275a05255be32cbdacc/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:27.308886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e195c-05e1-4ccb-8d35-cd1979cd5683 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:27.309510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e195c-05e1-4ccb-8d35-cd1979cd5683 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:27.309782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e195c-05e1-4ccb-8d35-cd1979cd5683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:27.364830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-774e195c-05e1-4ccb-8d35-cd1979cd5683 None None] MFA Rules not processed for user `e5f95390cf634275a05255be32cbdacc`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:27.366203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-774e195c-05e1-4ccb-8d35-cd1979cd5683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:27.370770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:27.470869 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1310/2595] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:27.715406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c689a-3972-4c39-9f42-6927ea04f404 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:27.719698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ac3c689a-3972-4c39-9f42-6927ea04f404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:27.844536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:27.873787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c689a-3972-4c39-9f42-6927ea04f404 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:27.883901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfe40b-ad66-4e46-bdd6-62aa182cebab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:27.884892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfe40b-ad66-4e46-bdd6-62aa182cebab service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:27.885336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfe40b-ad66-4e46-bdd6-62aa182cebab service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:27.885740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfe40b-ad66-4e46-bdd6-62aa182cebab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:27.899063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfe40b-ad66-4e46-bdd6-62aa182cebab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:27.900934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfe40b-ad66-4e46-bdd6-62aa182cebab service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:27.915766 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1286/2596] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:28.209048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ae843-6b2c-42f5-8050-55be9b3f1f7f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:28.209978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ae843-6b2c-42f5-8050-55be9b3f1f7f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:28.210251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ae843-6b2c-42f5-8050-55be9b3f1f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:28.233104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-036ae843-6b2c-42f5-8050-55be9b3f1f7f None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:28.234506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-036ae843-6b2c-42f5-8050-55be9b3f1f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:28.294398 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1311/2597] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 20:57:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:28.335688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfe40b-ad66-4e46-bdd6-62aa182cebab service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:28.337184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b7cfe40b-ad66-4e46-bdd6-62aa182cebab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:28.413475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfe40b-ad66-4e46-bdd6-62aa182cebab service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:28.418867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd0727-2f97-480b-9058-e2dcfceccacb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:28.419491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd0727-2f97-480b-9058-e2dcfceccacb service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:28.419752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd0727-2f97-480b-9058-e2dcfceccacb service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:28.420026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd0727-2f97-480b-9058-e2dcfceccacb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:28.428616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd0727-2f97-480b-9058-e2dcfceccacb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:28.430230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd0727-2f97-480b-9058-e2dcfceccacb service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:28.440622 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1287/2598] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 20:57:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 20:57:28.483263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ae843-6b2c-42f5-8050-55be9b3f1f7f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:28.502627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ae843-6b2c-42f5-8050-55be9b3f1f7f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:28.509735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171ef13-b2e0-4e99-a71f-bf2c4861be5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:28.510492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171ef13-b2e0-4e99-a71f-bf2c4861be5b service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:28.511042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171ef13-b2e0-4e99-a71f-bf2c4861be5b service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:28.511190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171ef13-b2e0-4e99-a71f-bf2c4861be5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:28.514288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a171ef13-b2e0-4e99-a71f-bf2c4861be5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:28.598566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b7941-8f13-4841-adad-a1d173afdb1c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:28.598953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b7941-8f13-4841-adad-a1d173afdb1c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:28.599397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b7941-8f13-4841-adad-a1d173afdb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:28.604946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171ef13-b2e0-4e99-a71f-bf2c4861be5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:28.606768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171ef13-b2e0-4e99-a71f-bf2c4861be5b service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:28.618480 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1312/2599] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:28.624061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-874b7941-8f13-4841-adad-a1d173afdb1c None None] MFA Rules not processed for user `e5f95390cf634275a05255be32cbdacc`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:28.626136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-874b7941-8f13-4841-adad-a1d173afdb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:28.634016 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1288/2600] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:28.952002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee867346-f201-4eab-b79e-7453ba062bc9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:28.952468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee867346-f201-4eab-b79e-7453ba062bc9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:28.952854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee867346-f201-4eab-b79e-7453ba062bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:28.974589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ee867346-f201-4eab-b79e-7453ba062bc9 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:28.976178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ee867346-f201-4eab-b79e-7453ba062bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:28.984694 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1313/2601] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 20:57:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:28.998162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b7941-8f13-4841-adad-a1d173afdb1c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:29.016534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b7941-8f13-4841-adad-a1d173afdb1c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:29.026962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad1be7-4184-4e06-84f8-a865e7e543f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:29.028238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad1be7-4184-4e06-84f8-a865e7e543f5 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:29.028897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad1be7-4184-4e06-84f8-a865e7e543f5 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:29.030348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad1be7-4184-4e06-84f8-a865e7e543f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:29.059012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad1be7-4184-4e06-84f8-a865e7e543f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:29.061188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad1be7-4184-4e06-84f8-a865e7e543f5 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:29.078159 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1289/2602] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 20:57:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 20:57:29.229780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee867346-f201-4eab-b79e-7453ba062bc9 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:29.280957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:29.307076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee867346-f201-4eab-b79e-7453ba062bc9 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:29.312514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:29.313478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:29.313902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:29.314339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] PATH_INFO: `/v3/users/e5f95390cf634275a05255be32cbdacc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:29.315785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5f95390cf634275a05255be32cbdacc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:29.317886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:29.385446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5f95390cf634275a05255be32cbdacc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:29.394743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5f95390cf634275a05255be32cbdacc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:29.408137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5f95390cf634275a05255be32cbdacc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:29.424903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5f95390cf634275a05255be32cbdacc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:29.425664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5f95390cf634275a05255be32cbdacc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:29.432548 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1314/2603] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:57:29 2022] DELETE /identity/v3/users/e5f95390cf634275a05255be32cbdacc => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:29.437161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148de4c-6a89-4759-925e-57d3acf449f3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:29.437548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148de4c-6a89-4759-925e-57d3acf449f3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:29.437855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148de4c-6a89-4759-925e-57d3acf449f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:29.458079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7148de4c-6a89-4759-925e-57d3acf449f3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:29.459292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7148de4c-6a89-4759-925e-57d3acf449f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:29.520616 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1290/2604] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:29.534127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:29.535679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:29.595912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b6807-a430-4624-bfd4-77c45dcb60e0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:29.599556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbf78d-efea-4fce-ba89-61385dd68d5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:29.600274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbf78d-efea-4fce-ba89-61385dd68d5d service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:29.600683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbf78d-efea-4fce-ba89-61385dd68d5d service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:29.600967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbf78d-efea-4fce-ba89-61385dd68d5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:29.625300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbf78d-efea-4fce-ba89-61385dd68d5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:29.627250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbf78d-efea-4fce-ba89-61385dd68d5d service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:29.645812 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1315/2605] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:57:29.934794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148de4c-6a89-4759-925e-57d3acf449f3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:29.936225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7148de4c-6a89-4759-925e-57d3acf449f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:29.952228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148de4c-6a89-4759-925e-57d3acf449f3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:29.957966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:29.958620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:29.958896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:29.959163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1 admin admin] PATH_INFO: `/v3/projects/5e852b41b6d643e7a85f8f8a660ffaf7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:29.974384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e852b41b6d643e7a85f8f8a660ffaf7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:29.975941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:30.026027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:30.031007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e852b41b6d643e7a85f8f8a660ffaf7', 'request_id': 'req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:30.038088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e852b41b6d643e7a85f8f8a660ffaf7', 'request_id': 'req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:30.049931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e852b41b6d643e7a85f8f8a660ffaf7', 'request_id': 'req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:30.066594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1 admin admin] The token cache is being invalidate because project 5e852b41b6d643e7a85f8f8a660ffaf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:57:30.067197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:30.067875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e1a2b69d-c80b-4e26-b4a1-4c27f76ff1d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:30.069836 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1291/2606] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:57:29 2022] DELETE /identity/v3/projects/5e852b41b6d643e7a85f8f8a660ffaf7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:30.075353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558659d0-1f85-43f4-ad81-8cb440273473 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:30.075638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558659d0-1f85-43f4-ad81-8cb440273473 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:30.075907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558659d0-1f85-43f4-ad81-8cb440273473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:30.099254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-558659d0-1f85-43f4-ad81-8cb440273473 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:30.100500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-558659d0-1f85-43f4-ad81-8cb440273473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:30.174018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:30.186604 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1316/2607] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:30.194162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea1c97-ff56-456f-b567-72ab9d42f335 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:30.194737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea1c97-ff56-456f-b567-72ab9d42f335 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:30.195089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea1c97-ff56-456f-b567-72ab9d42f335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:30.228692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1fea1c97-ff56-456f-b567-72ab9d42f335 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:30.230278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1fea1c97-ff56-456f-b567-72ab9d42f335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:30.241770 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1292/2608] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:57:30.246051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558659d0-1f85-43f4-ad81-8cb440273473 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:30.258353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558659d0-1f85-43f4-ad81-8cb440273473 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:30.265195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac24af-1af8-4ff9-888c-1e619d01de20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:30.266813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac24af-1af8-4ff9-888c-1e619d01de20 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:30.267326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac24af-1af8-4ff9-888c-1e619d01de20 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:30.267769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac24af-1af8-4ff9-888c-1e619d01de20 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:30.268744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac24af-1af8-4ff9-888c-1e619d01de20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:57:30.269555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac24af-1af8-4ff9-888c-1e619d01de20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:30.271443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac24af-1af8-4ff9-888c-1e619d01de20 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:30.287980 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1317/2609] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:57:30.289145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea1c97-ff56-456f-b567-72ab9d42f335 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:30.298992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea1c97-ff56-456f-b567-72ab9d42f335 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:30.303609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba0b62-a2d3-46b1-8866-923f2bef3a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:30.304206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba0b62-a2d3-46b1-8866-923f2bef3a63 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:30.304512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba0b62-a2d3-46b1-8866-923f2bef3a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:30.304772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba0b62-a2d3-46b1-8866-923f2bef3a63 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:30.305396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba0b62-a2d3-46b1-8866-923f2bef3a63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:30.307276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba0b62-a2d3-46b1-8866-923f2bef3a63 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:30.338667 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1293/2610] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:57:30 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:57:30.342971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac24af-1af8-4ff9-888c-1e619d01de20 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:30.353614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac24af-1af8-4ff9-888c-1e619d01de20 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:30.410851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0cd6c-4b9a-4fc4-af37-a656f70c52b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:30.411461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0cd6c-4b9a-4fc4-af37-a656f70c52b2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:30.411728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0cd6c-4b9a-4fc4-af37-a656f70c52b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:30.411987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0cd6c-4b9a-4fc4-af37-a656f70c52b2 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:30.412639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0cd6c-4b9a-4fc4-af37-a656f70c52b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:30.414549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0cd6c-4b9a-4fc4-af37-a656f70c52b2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:30.441499 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1318/2611] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:57:30 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:57:30.442971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba0b62-a2d3-46b1-8866-923f2bef3a63 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:30.454551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba0b62-a2d3-46b1-8866-923f2bef3a63 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:30.460080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489e66d-ccd3-4696-856a-10787066f9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:30.460721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489e66d-ccd3-4696-856a-10787066f9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:30.460994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489e66d-ccd3-4696-856a-10787066f9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:30.461258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489e66d-ccd3-4696-856a-10787066f9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:30.461927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489e66d-ccd3-4696-856a-10787066f9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:30.463527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489e66d-ccd3-4696-856a-10787066f9a8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:30.477685 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1294/2612] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:57:30.479046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0cd6c-4b9a-4fc4-af37-a656f70c52b2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:30.489771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0cd6c-4b9a-4fc4-af37-a656f70c52b2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:30.495961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5ba84-3e20-4b9a-a633-d3361e3c2f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:30.496152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5ba84-3e20-4b9a-a633-d3361e3c2f03 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:30.496478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5ba84-3e20-4b9a-a633-d3361e3c2f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:30.496750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5ba84-3e20-4b9a-a633-d3361e3c2f03 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:30.497406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5ba84-3e20-4b9a-a633-d3361e3c2f03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:30.499006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5ba84-3e20-4b9a-a633-d3361e3c2f03 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:30.510651 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1319/2613] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:57:30.514810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489e66d-ccd3-4696-856a-10787066f9a8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:30.525084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489e66d-ccd3-4696-856a-10787066f9a8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:30.530029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6317510-2676-4140-bfd9-14fe047d1b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:30.530752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6317510-2676-4140-bfd9-14fe047d1b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:30.531020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6317510-2676-4140-bfd9-14fe047d1b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:30.531286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6317510-2676-4140-bfd9-14fe047d1b21 admin admin] PATH_INFO: `/v3/projects/8ac46d9ab1ca4815a59b43a819066e85/users/22489a6c355a49028ca16a394b27876f/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:30.546541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6317510-2676-4140-bfd9-14fe047d1b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac46d9ab1ca4815a59b43a819066e85, user_id=22489a6c355a49028ca16a394b27876f, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:30.549259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6317510-2676-4140-bfd9-14fe047d1b21 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:30.565213 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1295/2614] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:57:30 2022] PUT /identity/v3/projects/8ac46d9ab1ca4815a59b43a819066e85/users/22489a6c355a49028ca16a394b27876f/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:30.570807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afb479-deb2-4ba4-aab9-4c11f9706885 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:30.571079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afb479-deb2-4ba4-aab9-4c11f9706885 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:30.571338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afb479-deb2-4ba4-aab9-4c11f9706885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:30.595781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-16afb479-deb2-4ba4-aab9-4c11f9706885 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:30.597038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-16afb479-deb2-4ba4-aab9-4c11f9706885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:30.615700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6317510-2676-4140-bfd9-14fe047d1b21 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:30.618535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b6317510-2676-4140-bfd9-14fe047d1b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:30.666112 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1320/2615] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:30.676671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afb479-deb2-4ba4-aab9-4c11f9706885 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:30.678195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-16afb479-deb2-4ba4-aab9-4c11f9706885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:30.744216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6317510-2676-4140-bfd9-14fe047d1b21 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:30.751153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4fade-5d9a-42fb-b1c8-fa84e1b4ea26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:30.752396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4fade-5d9a-42fb-b1c8-fa84e1b4ea26 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:30.752832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4fade-5d9a-42fb-b1c8-fa84e1b4ea26 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:30.753112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4fade-5d9a-42fb-b1c8-fa84e1b4ea26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:30.753705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afb479-deb2-4ba4-aab9-4c11f9706885 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:30.755353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-49d4fade-5d9a-42fb-b1c8-fa84e1b4ea26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:30.758708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062c06f-1935-4d91-92c4-0e00c9ec9a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:30.759591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062c06f-1935-4d91-92c4-0e00c9ec9a03 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:30.759867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062c06f-1935-4d91-92c4-0e00c9ec9a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:30.760136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062c06f-1935-4d91-92c4-0e00c9ec9a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:30.771206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062c06f-1935-4d91-92c4-0e00c9ec9a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:30.772774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062c06f-1935-4d91-92c4-0e00c9ec9a03 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:30.784503 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1321/2616] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:57:30.845742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4fade-5d9a-42fb-b1c8-fa84e1b4ea26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:30.847697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4fade-5d9a-42fb-b1c8-fa84e1b4ea26 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:30.854569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:30.861018 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1296/2617] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:30 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:33.269663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dbdd0-aa5c-4cad-b9dd-ba86c98fe278 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:33.270207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dbdd0-aa5c-4cad-b9dd-ba86c98fe278 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:33.270378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dbdd0-aa5c-4cad-b9dd-ba86c98fe278 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:33.273021 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1322/2618] 10.222.0.23 () {54 vars in 921 bytes} [Sat Aug 20 20:57:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:57:33.279462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f1757-f173-4fd2-b9f4-7d4ae7170b04 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:33.279974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f1757-f173-4fd2-b9f4-7d4ae7170b04 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:33.279974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f1757-f173-4fd2-b9f4-7d4ae7170b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:33.300730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-269f1757-f173-4fd2-b9f4-7d4ae7170b04 None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:33.302125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-269f1757-f173-4fd2-b9f4-7d4ae7170b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:33.316127 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1297/2619] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 20:57:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:33.333503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dbdd0-aa5c-4cad-b9dd-ba86c98fe278 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:33.335876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6b7dbdd0-aa5c-4cad-b9dd-ba86c98fe278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:33.350740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dbdd0-aa5c-4cad-b9dd-ba86c98fe278 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:33.354277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018ffb9-24ca-48ba-aedd-f7bde43a2ccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:33.355120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018ffb9-24ca-48ba-aedd-f7bde43a2ccd service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:33.355566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018ffb9-24ca-48ba-aedd-f7bde43a2ccd service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:33.356150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018ffb9-24ca-48ba-aedd-f7bde43a2ccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:33.365336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:33.368723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018ffb9-24ca-48ba-aedd-f7bde43a2ccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:33.370627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018ffb9-24ca-48ba-aedd-f7bde43a2ccd service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:33.388112 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1323/2620] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:33.422707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b1008-aa4e-41e1-9f0e-f7964f055b46 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:33.422925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b1008-aa4e-41e1-9f0e-f7964f055b46 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:33.423200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b1008-aa4e-41e1-9f0e-f7964f055b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:33.474139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-dd5b1008-aa4e-41e1-9f0e-f7964f055b46 None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:33.475379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dd5b1008-aa4e-41e1-9f0e-f7964f055b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:33.489796 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1298/2621] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 20:57:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:33.504498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018ffb9-24ca-48ba-aedd-f7bde43a2ccd service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:33.514172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018ffb9-24ca-48ba-aedd-f7bde43a2ccd service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:33.518344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4089de-0b36-4e76-8c4a-fabb95107d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:33.519099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4089de-0b36-4e76-8c4a-fabb95107d94 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:33.519443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4089de-0b36-4e76-8c4a-fabb95107d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:33.519777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4089de-0b36-4e76-8c4a-fabb95107d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:33.529128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4089de-0b36-4e76-8c4a-fabb95107d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:33.530818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4089de-0b36-4e76-8c4a-fabb95107d94 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:33.549954 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1324/2622] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:35.475996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b1008-aa4e-41e1-9f0e-f7964f055b46 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:35.477496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dd5b1008-aa4e-41e1-9f0e-f7964f055b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:35.502181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b1008-aa4e-41e1-9f0e-f7964f055b46 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:35.507885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:35.508642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:35.509001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:35.509342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] PATH_INFO: `/v3/users/63d2977c98ed4acc8d53d8cfbed6f0c4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:35.510714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63d2977c98ed4acc8d53d8cfbed6f0c4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:35.512962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:35.607485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63d2977c98ed4acc8d53d8cfbed6f0c4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:35.621814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63d2977c98ed4acc8d53d8cfbed6f0c4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:35.637808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63d2977c98ed4acc8d53d8cfbed6f0c4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:35.638808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63d2977c98ed4acc8d53d8cfbed6f0c4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:35.639498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63d2977c98ed4acc8d53d8cfbed6f0c4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:35.658002 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1299/2623] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:57:35 2022] DELETE /identity/v3/users/63d2977c98ed4acc8d53d8cfbed6f0c4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:35.659503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ad521-670f-4421-b079-a90d2dbeb9e3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:35.660311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ad521-670f-4421-b079-a90d2dbeb9e3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:35.660615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ad521-670f-4421-b079-a90d2dbeb9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:35.731980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fa7ad521-670f-4421-b079-a90d2dbeb9e3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:35.733358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fa7ad521-670f-4421-b079-a90d2dbeb9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:35.922826 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1325/2624] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:57:35.949001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:35.950857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:36.142572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167fe38-0fb3-44fd-b9b1-04162f3ba367 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:36.153386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833694d5-ebd7-4600-a32a-a1e659236675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:36.154097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833694d5-ebd7-4600-a32a-a1e659236675 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:36.154402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833694d5-ebd7-4600-a32a-a1e659236675 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:36.154670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833694d5-ebd7-4600-a32a-a1e659236675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:36.180652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833694d5-ebd7-4600-a32a-a1e659236675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:36.184290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833694d5-ebd7-4600-a32a-a1e659236675 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:36.204304 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1300/2625] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:37.435560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb96bae-082e-4b78-9700-3c3d6be6c42e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:37.436175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb96bae-082e-4b78-9700-3c3d6be6c42e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:37.436175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb96bae-082e-4b78-9700-3c3d6be6c42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:37.533538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833694d5-ebd7-4600-a32a-a1e659236675 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:37.536358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-833694d5-ebd7-4600-a32a-a1e659236675 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:37.544521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ccb96bae-082e-4b78-9700-3c3d6be6c42e None None] MFA Rules not processed for user `22489a6c355a49028ca16a394b27876f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:37.546263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ccb96bae-082e-4b78-9700-3c3d6be6c42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:37.558242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833694d5-ebd7-4600-a32a-a1e659236675 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:37.566202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:37.567267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:37.568353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:37.569042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] PATH_INFO: `/v3/projects/ec819c2e3517424fb5b00cd5fbca5f35` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:37.572781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec819c2e3517424fb5b00cd5fbca5f35)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:37.574786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:37.705302 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1326/2626] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:37.711726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2096ec-a4d2-4972-9e0c-5519d3ad305f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:37.712097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2096ec-a4d2-4972-9e0c-5519d3ad305f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:37.712467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2096ec-a4d2-4972-9e0c-5519d3ad305f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:37.716186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec819c2e3517424fb5b00cd5fbca5f35', 'request_id': 'req-8ef43083-069e-424f-b3ff-b0f6e3f02969'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:37.732010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec819c2e3517424fb5b00cd5fbca5f35', 'request_id': 'req-8ef43083-069e-424f-b3ff-b0f6e3f02969'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:37.737952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:37.738374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ca2096ec-a4d2-4972-9e0c-5519d3ad305f None None] MFA Rules not processed for user `22489a6c355a49028ca16a394b27876f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:37.740244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ca2096ec-a4d2-4972-9e0c-5519d3ad305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:37.751384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec819c2e3517424fb5b00cd5fbca5f35', 'request_id': 'req-8ef43083-069e-424f-b3ff-b0f6e3f02969'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:37.772224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] The token cache is being invalidate because project ec819c2e3517424fb5b00cd5fbca5f35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:57:37.772224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:37.774248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:37.783272 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1301/2627] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:57:37 2022] DELETE /identity/v3/projects/ec819c2e3517424fb5b00cd5fbca5f35 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:37.784286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:37.787877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:37.878909 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1327/2628] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:37.897305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2096ec-a4d2-4972-9e0c-5519d3ad305f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:37.899215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ca2096ec-a4d2-4972-9e0c-5519d3ad305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:37.942856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:37.952109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef43083-069e-424f-b3ff-b0f6e3f02969 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:37.963093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:37.963758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:37.964027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:37.964290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] PATH_INFO: `/v3/users/f2fde96348ba419fa50ae7c4ef031acc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:37.965501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2fde96348ba419fa50ae7c4ef031acc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:37.967456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:38.058760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2fde96348ba419fa50ae7c4ef031acc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:38.066886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2096ec-a4d2-4972-9e0c-5519d3ad305f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:38.076559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2fde96348ba419fa50ae7c4ef031acc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:38.080337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bc048-d0ce-4322-8eb4-af36c255f40b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:38.081236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bc048-d0ce-4322-8eb4-af36c255f40b service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:38.082360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bc048-d0ce-4322-8eb4-af36c255f40b service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:38.082778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bc048-d0ce-4322-8eb4-af36c255f40b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:38.084978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c98bc048-d0ce-4322-8eb4-af36c255f40b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:38.101120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2fde96348ba419fa50ae7c4ef031acc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:38.103156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2fde96348ba419fa50ae7c4ef031acc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:38.104758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2fde96348ba419fa50ae7c4ef031acc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:38.118330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98bc048-d0ce-4322-8eb4-af36c255f40b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:38.120230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98bc048-d0ce-4322-8eb4-af36c255f40b service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:38.129029 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1302/2629] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:57:37 2022] DELETE /identity/v3/users/f2fde96348ba419fa50ae7c4ef031acc => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:38.151970 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1328/2630] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:37 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:38.727607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:38.729834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:38.888611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecc050-9674-471c-958e-e455acdade63 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:38.894914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:38.895556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:38.896160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:38.896160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d admin admin] PATH_INFO: `/v3/projects/faa12a42f1bc41d9a89ea4ef6fc229a6` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:38.899138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa12a42f1bc41d9a89ea4ef6fc229a6)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:38.901249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:38.908606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:38.966536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa12a42f1bc41d9a89ea4ef6fc229a6', 'request_id': 'req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:38.977189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa12a42f1bc41d9a89ea4ef6fc229a6', 'request_id': 'req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:38.990245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa12a42f1bc41d9a89ea4ef6fc229a6', 'request_id': 'req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:39.003968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d admin admin] The token cache is being invalidate because project faa12a42f1bc41d9a89ea4ef6fc229a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:57:39.004261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:39.004965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-162ffd9b-8d3b-4036-88a4-2dc9ab2ab89d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:57:39.012744 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1303/2631] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:57:38 2022] DELETE /identity/v3/projects/faa12a42f1bc41d9a89ea4ef6fc229a6 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:39.014440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3897df-68b0-4455-a6c4-87b967415b7a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:39.015153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3897df-68b0-4455-a6c4-87b967415b7a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:39.015431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3897df-68b0-4455-a6c4-87b967415b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:39.040810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bd3897df-68b0-4455-a6c4-87b967415b7a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:39.042155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bd3897df-68b0-4455-a6c4-87b967415b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:39.120524 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1329/2632] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:39.129886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f8f66-9b46-491d-81de-8229b7732c5e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:39.130226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f8f66-9b46-491d-81de-8229b7732c5e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:39.130555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f8f66-9b46-491d-81de-8229b7732c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:39.160022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fc6f8f66-9b46-491d-81de-8229b7732c5e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:39.161390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fc6f8f66-9b46-491d-81de-8229b7732c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:39.173351 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1304/2633] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:39.175092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3897df-68b0-4455-a6c4-87b967415b7a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:39.184984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3897df-68b0-4455-a6c4-87b967415b7a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:39.190078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab1835-dada-41ca-a4f8-8645f080ceef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:39.190693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab1835-dada-41ca-a4f8-8645f080ceef admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:39.191185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab1835-dada-41ca-a4f8-8645f080ceef admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:39.191453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab1835-dada-41ca-a4f8-8645f080ceef admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:39.192100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab1835-dada-41ca-a4f8-8645f080ceef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:57:39.192417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab1835-dada-41ca-a4f8-8645f080ceef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:39.194018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab1835-dada-41ca-a4f8-8645f080ceef admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:39.207509 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1330/2634] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:57:39.212413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f8f66-9b46-491d-81de-8229b7732c5e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:39.229138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f8f66-9b46-491d-81de-8229b7732c5e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:39.238597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ca3ee-7f8b-4c38-9d8e-34a3e95372ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:39.238920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ca3ee-7f8b-4c38-9d8e-34a3e95372ba admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:39.239198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ca3ee-7f8b-4c38-9d8e-34a3e95372ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:39.239463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ca3ee-7f8b-4c38-9d8e-34a3e95372ba admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:39.240087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ca3ee-7f8b-4c38-9d8e-34a3e95372ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:39.242033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ca3ee-7f8b-4c38-9d8e-34a3e95372ba admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:39.294530 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1305/2635] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:57:39 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:57:39.298776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab1835-dada-41ca-a4f8-8645f080ceef admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:39.316747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab1835-dada-41ca-a4f8-8645f080ceef admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:39.410641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dae473-36da-4d2e-9a00-c642a9b0242d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:39.411306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dae473-36da-4d2e-9a00-c642a9b0242d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:39.411579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dae473-36da-4d2e-9a00-c642a9b0242d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:39.411841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dae473-36da-4d2e-9a00-c642a9b0242d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:39.412541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dae473-36da-4d2e-9a00-c642a9b0242d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:39.414683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dae473-36da-4d2e-9a00-c642a9b0242d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:39.448266 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1331/2636] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:57:39 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:57:39.449579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ca3ee-7f8b-4c38-9d8e-34a3e95372ba admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:39.466519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ca3ee-7f8b-4c38-9d8e-34a3e95372ba admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:39.472869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247365f-b02e-4967-8753-201d56c55c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:39.473550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247365f-b02e-4967-8753-201d56c55c00 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:39.473891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247365f-b02e-4967-8753-201d56c55c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:39.474224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247365f-b02e-4967-8753-201d56c55c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:39.474990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247365f-b02e-4967-8753-201d56c55c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:39.476703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247365f-b02e-4967-8753-201d56c55c00 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:39.493500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dae473-36da-4d2e-9a00-c642a9b0242d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:39.493936 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1306/2637] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:57:39.506897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dae473-36da-4d2e-9a00-c642a9b0242d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:39.513328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7d69e-de21-4725-bfc9-514bfb1c345c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:39.514218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7d69e-de21-4725-bfc9-514bfb1c345c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:39.514598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7d69e-de21-4725-bfc9-514bfb1c345c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:39.515095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7d69e-de21-4725-bfc9-514bfb1c345c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:39.515868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7d69e-de21-4725-bfc9-514bfb1c345c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:39.519190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7d69e-de21-4725-bfc9-514bfb1c345c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:39.536977 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1332/2638] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:57:39.538302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247365f-b02e-4967-8753-201d56c55c00 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:39.551712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247365f-b02e-4967-8753-201d56c55c00 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:39.565769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed68a68-78d5-4b52-8e5d-1f2098cb005b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:39.570196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed68a68-78d5-4b52-8e5d-1f2098cb005b admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:39.570971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed68a68-78d5-4b52-8e5d-1f2098cb005b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:39.570971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed68a68-78d5-4b52-8e5d-1f2098cb005b admin admin] PATH_INFO: `/v3/projects/44af76b79ce54fe584fe6f6a9680a3a4/users/b91bab791d584fde8ce3f5c9c8b45ed4/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:39.622307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed68a68-78d5-4b52-8e5d-1f2098cb005b admin admin] RBAC: Authorizing `identity:create_grant(project_id=44af76b79ce54fe584fe6f6a9680a3a4, user_id=b91bab791d584fde8ce3f5c9c8b45ed4, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:39.625661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed68a68-78d5-4b52-8e5d-1f2098cb005b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:39.649445 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1307/2639] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:57:39 2022] PUT /identity/v3/projects/44af76b79ce54fe584fe6f6a9680a3a4/users/b91bab791d584fde8ce3f5c9c8b45ed4/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:39.654162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a801ab-cd88-4edf-842c-a28a69f7feea None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:39.660591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a801ab-cd88-4edf-842c-a28a69f7feea None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:39.660960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a801ab-cd88-4edf-842c-a28a69f7feea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:39.703262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-48a801ab-cd88-4edf-842c-a28a69f7feea None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:39.704911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-48a801ab-cd88-4edf-842c-a28a69f7feea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:39.713295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:39.796928 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1333/2640] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:39.797023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed68a68-78d5-4b52-8e5d-1f2098cb005b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:39.798983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-aed68a68-78d5-4b52-8e5d-1f2098cb005b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:39.892873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed68a68-78d5-4b52-8e5d-1f2098cb005b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:39.905791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87f571-9a8e-4969-bf35-5a4f8bab6dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:39.907606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87f571-9a8e-4969-bf35-5a4f8bab6dae service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:39.907862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87f571-9a8e-4969-bf35-5a4f8bab6dae service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:39.908157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87f571-9a8e-4969-bf35-5a4f8bab6dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:39.921079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87f571-9a8e-4969-bf35-5a4f8bab6dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:39.922988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87f571-9a8e-4969-bf35-5a4f8bab6dae service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:39.947691 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1308/2641] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:57:45.103139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a801ab-cd88-4edf-842c-a28a69f7feea None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:45.104672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-48a801ab-cd88-4edf-842c-a28a69f7feea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:45.211948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a801ab-cd88-4edf-842c-a28a69f7feea None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:45.217855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747111c-2ff7-4a6b-9b88-7383581daddf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:45.218488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747111c-2ff7-4a6b-9b88-7383581daddf service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:45.218841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747111c-2ff7-4a6b-9b88-7383581daddf service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:45.219228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747111c-2ff7-4a6b-9b88-7383581daddf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:45.221182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2747111c-2ff7-4a6b-9b88-7383581daddf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:45.234312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747111c-2ff7-4a6b-9b88-7383581daddf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:45.236019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747111c-2ff7-4a6b-9b88-7383581daddf service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:45.258195 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1334/2642] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:47.303620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2d15c-5a05-43c0-9b5f-7f7b57eb9da3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:47.304145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2d15c-5a05-43c0-9b5f-7f7b57eb9da3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:47.304145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2d15c-5a05-43c0-9b5f-7f7b57eb9da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:47.329571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c5b2d15c-5a05-43c0-9b5f-7f7b57eb9da3 None None] MFA Rules not processed for user `8afc45f50dc54b84a8abf3b6e168426b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:47.331059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c5b2d15c-5a05-43c0-9b5f-7f7b57eb9da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:47.368626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:47.400668 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1309/2643] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:50.738046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bdcfe-0bf6-4e58-8d86-32ea5ccaa8ff None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:50.738556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bdcfe-0bf6-4e58-8d86-32ea5ccaa8ff None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:50.738556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bdcfe-0bf6-4e58-8d86-32ea5ccaa8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:50.784831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-992bdcfe-0bf6-4e58-8d86-32ea5ccaa8ff None None] MFA Rules not processed for user `b91bab791d584fde8ce3f5c9c8b45ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:50.786225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-992bdcfe-0bf6-4e58-8d86-32ea5ccaa8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:50.934878 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1335/2644] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:50.939092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2d15c-5a05-43c0-9b5f-7f7b57eb9da3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:50.954859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2d15c-5a05-43c0-9b5f-7f7b57eb9da3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:50.961563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378a4bf-60f5-47df-83bb-ca7d60480f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:50.962463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378a4bf-60f5-47df-83bb-ca7d60480f60 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:50.962962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378a4bf-60f5-47df-83bb-ca7d60480f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:50.963419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378a4bf-60f5-47df-83bb-ca7d60480f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:50.964306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378a4bf-60f5-47df-83bb-ca7d60480f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:50.968009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378a4bf-60f5-47df-83bb-ca7d60480f60 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:51.008738 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1310/2645] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:57:50 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:57:51.013400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bdcfe-0bf6-4e58-8d86-32ea5ccaa8ff None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:51.028327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bdcfe-0bf6-4e58-8d86-32ea5ccaa8ff None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:51.073816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:51.124751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ec289-5c7b-4f49-9344-e64a789989ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:51.125190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ec289-5c7b-4f49-9344-e64a789989ce admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:51.125468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ec289-5c7b-4f49-9344-e64a789989ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:51.125733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ec289-5c7b-4f49-9344-e64a789989ce admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:51.126477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ec289-5c7b-4f49-9344-e64a789989ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:51.135363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ec289-5c7b-4f49-9344-e64a789989ce admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:51.187825 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1336/2646] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:57:51 2022] POST /identity/v3/users => generated 299 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:57:51.193021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378a4bf-60f5-47df-83bb-ca7d60480f60 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:51.208209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378a4bf-60f5-47df-83bb-ca7d60480f60 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:51.213400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16abf6-84ce-4fae-9d1b-27b84720561e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:51.214022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16abf6-84ce-4fae-9d1b-27b84720561e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:51.214296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16abf6-84ce-4fae-9d1b-27b84720561e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:51.214563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16abf6-84ce-4fae-9d1b-27b84720561e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:51.215252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16abf6-84ce-4fae-9d1b-27b84720561e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:51.217611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16abf6-84ce-4fae-9d1b-27b84720561e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:51.231983 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1311/2647] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:57:51.236788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ec289-5c7b-4f49-9344-e64a789989ce admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:51.255298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ec289-5c7b-4f49-9344-e64a789989ce admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:51.261817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b325c16-7a28-4645-ad22-4527f9634111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:51.262573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b325c16-7a28-4645-ad22-4527f9634111 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:51.262845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b325c16-7a28-4645-ad22-4527f9634111 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:51.263115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b325c16-7a28-4645-ad22-4527f9634111 admin admin] PATH_INFO: `/v3/projects/8dd2126e33bd4f6c96b8faa511f70215/users/3c3972a48f7a473895fee3b574d22316/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:51.303320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b325c16-7a28-4645-ad22-4527f9634111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd2126e33bd4f6c96b8faa511f70215, user_id=3c3972a48f7a473895fee3b574d22316, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:51.310952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b325c16-7a28-4645-ad22-4527f9634111 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:51.348137 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1337/2648] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:57:51 2022] PUT /identity/v3/projects/8dd2126e33bd4f6c96b8faa511f70215/users/3c3972a48f7a473895fee3b574d22316/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:51.351920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16abf6-84ce-4fae-9d1b-27b84720561e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:51.369693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16abf6-84ce-4fae-9d1b-27b84720561e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:51.483631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0943248-8d60-4aa5-932e-bf57d321fe52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:51.484798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0943248-8d60-4aa5-932e-bf57d321fe52 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:51.486704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0943248-8d60-4aa5-932e-bf57d321fe52 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:51.487185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0943248-8d60-4aa5-932e-bf57d321fe52 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:51.488321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0943248-8d60-4aa5-932e-bf57d321fe52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:51.490355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0943248-8d60-4aa5-932e-bf57d321fe52 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:51.505246 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1312/2649] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:57:51.509372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b325c16-7a28-4645-ad22-4527f9634111 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:51.535922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b325c16-7a28-4645-ad22-4527f9634111 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:51.547920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da747cbd-22f6-4e45-8c24-38233607665e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:51.548676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da747cbd-22f6-4e45-8c24-38233607665e admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:51.548951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da747cbd-22f6-4e45-8c24-38233607665e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:51.549355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da747cbd-22f6-4e45-8c24-38233607665e admin admin] PATH_INFO: `/v3/projects/8dd2126e33bd4f6c96b8faa511f70215/users/3c3972a48f7a473895fee3b574d22316/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:51.551302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da747cbd-22f6-4e45-8c24-38233607665e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd2126e33bd4f6c96b8faa511f70215, user_id=3c3972a48f7a473895fee3b574d22316, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:51.554262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da747cbd-22f6-4e45-8c24-38233607665e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:51.569131 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1338/2650] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:57:51 2022] PUT /identity/v3/projects/8dd2126e33bd4f6c96b8faa511f70215/users/3c3972a48f7a473895fee3b574d22316/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:57:52.586024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0943248-8d60-4aa5-932e-bf57d321fe52 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:52.586024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d0943248-8d60-4aa5-932e-bf57d321fe52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:52.658915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5fb76-2c22-4125-bfae-84d1ebc93532 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:52.659421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5fb76-2c22-4125-bfae-84d1ebc93532 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:52.659947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5fb76-2c22-4125-bfae-84d1ebc93532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:52.700900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a2b5fb76-2c22-4125-bfae-84d1ebc93532 None None] MFA Rules not processed for user `7c276ebf6fe14a9c8f50bd1d1f44fb58`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:57:52.702906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a2b5fb76-2c22-4125-bfae-84d1ebc93532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:52.762860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0943248-8d60-4aa5-932e-bf57d321fe52 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:52.818456 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1339/2651] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:57:52 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:57:52.818456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5fb76-2c22-4125-bfae-84d1ebc93532 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:57:52.854274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:57:52.902895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c3021-a5e3-4b96-98ce-a596d2fde450 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:52.903584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c3021-a5e3-4b96-98ce-a596d2fde450 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:52.903947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c3021-a5e3-4b96-98ce-a596d2fde450 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:52.907485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c3021-a5e3-4b96-98ce-a596d2fde450 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:52.911711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6f0c3021-a5e3-4b96-98ce-a596d2fde450 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:57:52.938991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c3021-a5e3-4b96-98ce-a596d2fde450 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:52.941518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c3021-a5e3-4b96-98ce-a596d2fde450 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:52.965060 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1313/2652] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:57:52.994807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5fb76-2c22-4125-bfae-84d1ebc93532 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:57:53.017151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b402bc2-93dd-4aae-b98f-ebb1e023b5e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:57:53.018035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b402bc2-93dd-4aae-b98f-ebb1e023b5e9 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:57:53.018473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b402bc2-93dd-4aae-b98f-ebb1e023b5e9 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:57:53.018879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b402bc2-93dd-4aae-b98f-ebb1e023b5e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:57:53.039778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b402bc2-93dd-4aae-b98f-ebb1e023b5e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:57:53.041415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b402bc2-93dd-4aae-b98f-ebb1e023b5e9 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:57:53.071557 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1340/2653] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:57:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:58:00.438757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3929f4d-f3c1-483e-8e49-b91354d20cc0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:00.439318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3929f4d-f3c1-483e-8e49-b91354d20cc0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:00.439318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3929f4d-f3c1-483e-8e49-b91354d20cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:00.491661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c3929f4d-f3c1-483e-8e49-b91354d20cc0 None None] MFA Rules not processed for user `3c3972a48f7a473895fee3b574d22316`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:00.493240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c3929f4d-f3c1-483e-8e49-b91354d20cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:00.595268 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1314/2654] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:00.605122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b402bc2-93dd-4aae-b98f-ebb1e023b5e9 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:00.607320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8b402bc2-93dd-4aae-b98f-ebb1e023b5e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:00.627945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:00.676593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b402bc2-93dd-4aae-b98f-ebb1e023b5e9 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:00.680532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7633c2-6dd3-4394-8019-27be1e749a6a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:00.681243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7633c2-6dd3-4394-8019-27be1e749a6a service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:00.681582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7633c2-6dd3-4394-8019-27be1e749a6a service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:00.681913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7633c2-6dd3-4394-8019-27be1e749a6a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:00.692842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7633c2-6dd3-4394-8019-27be1e749a6a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:00.694650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7633c2-6dd3-4394-8019-27be1e749a6a service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:00.707969 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1341/2655] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:58:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:58:08.784295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3929f4d-f3c1-483e-8e49-b91354d20cc0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:08.786764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c3929f4d-f3c1-483e-8e49-b91354d20cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:08.927943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3929f4d-f3c1-483e-8e49-b91354d20cc0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:08.932992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dea65-3167-45c2-b62b-e7fd24fa24b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:08.933634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dea65-3167-45c2-b62b-e7fd24fa24b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:08.934073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dea65-3167-45c2-b62b-e7fd24fa24b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:08.934499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dea65-3167-45c2-b62b-e7fd24fa24b5 admin admin] PATH_INFO: `/v3/users/e7e70776606c41b28dd8092752a1619c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:08.935733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dea65-3167-45c2-b62b-e7fd24fa24b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e70776606c41b28dd8092752a1619c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:08.937787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dea65-3167-45c2-b62b-e7fd24fa24b5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:09.028041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-183dea65-3167-45c2-b62b-e7fd24fa24b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e70776606c41b28dd8092752a1619c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:09.042520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-183dea65-3167-45c2-b62b-e7fd24fa24b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e70776606c41b28dd8092752a1619c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:09.051988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:09.063629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-183dea65-3167-45c2-b62b-e7fd24fa24b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e70776606c41b28dd8092752a1619c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:09.064814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-183dea65-3167-45c2-b62b-e7fd24fa24b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e70776606c41b28dd8092752a1619c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:09.078383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-183dea65-3167-45c2-b62b-e7fd24fa24b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e70776606c41b28dd8092752a1619c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:09.104144 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1315/2656] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:58:08 2022] DELETE /identity/v3/users/e7e70776606c41b28dd8092752a1619c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:09.446920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7633c2-6dd3-4394-8019-27be1e749a6a service glance] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:09.479909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7633c2-6dd3-4394-8019-27be1e749a6a service glance] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:09.568009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836a849-8d09-4eed-aca1-30ad8ced0760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:09.569943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836a849-8d09-4eed-aca1-30ad8ced0760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:09.570459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836a849-8d09-4eed-aca1-30ad8ced0760 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:09.570952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836a849-8d09-4eed-aca1-30ad8ced0760 admin admin] PATH_INFO: `/v3/projects/6a97d94d1bda4de2899aca13952c2291` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:09.573093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836a849-8d09-4eed-aca1-30ad8ced0760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a97d94d1bda4de2899aca13952c2291)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:09.575523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836a849-8d09-4eed-aca1-30ad8ced0760 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:09.745475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2836a849-8d09-4eed-aca1-30ad8ced0760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a97d94d1bda4de2899aca13952c2291', 'request_id': 'req-2836a849-8d09-4eed-aca1-30ad8ced0760'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:09.767270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2836a849-8d09-4eed-aca1-30ad8ced0760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a97d94d1bda4de2899aca13952c2291', 'request_id': 'req-2836a849-8d09-4eed-aca1-30ad8ced0760'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:09.787539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2836a849-8d09-4eed-aca1-30ad8ced0760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a97d94d1bda4de2899aca13952c2291', 'request_id': 'req-2836a849-8d09-4eed-aca1-30ad8ced0760'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:09.813903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2836a849-8d09-4eed-aca1-30ad8ced0760 admin admin] The token cache is being invalidate because project 6a97d94d1bda4de2899aca13952c2291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:58:09.814384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2836a849-8d09-4eed-aca1-30ad8ced0760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:09.817405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2836a849-8d09-4eed-aca1-30ad8ced0760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:09.826745 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1342/2657] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:58:09 2022] DELETE /identity/v3/projects/6a97d94d1bda4de2899aca13952c2291 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:09.829380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cfc49-3516-408a-a60a-a7809de05d8e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:09.829906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cfc49-3516-408a-a60a-a7809de05d8e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:09.829993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cfc49-3516-408a-a60a-a7809de05d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:09.895896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-da9cfc49-3516-408a-a60a-a7809de05d8e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:09.902757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-da9cfc49-3516-408a-a60a-a7809de05d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:10.038195 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1316/2658] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:10.046111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc7fcf-62b7-4ea8-bf8c-5add923df905 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:10.046574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc7fcf-62b7-4ea8-bf8c-5add923df905 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:10.047030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc7fcf-62b7-4ea8-bf8c-5add923df905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:10.079770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-aedc7fcf-62b7-4ea8-bf8c-5add923df905 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:10.081380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aedc7fcf-62b7-4ea8-bf8c-5add923df905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:10.091527 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1343/2659] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:10.096483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cfc49-3516-408a-a60a-a7809de05d8e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:10.109605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cfc49-3516-408a-a60a-a7809de05d8e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:10.124524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24133a8-192e-40c6-b0f3-ab38a8ba01dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:10.126723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24133a8-192e-40c6-b0f3-ab38a8ba01dd admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:10.127104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24133a8-192e-40c6-b0f3-ab38a8ba01dd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:10.129803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24133a8-192e-40c6-b0f3-ab38a8ba01dd admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:10.130628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24133a8-192e-40c6-b0f3-ab38a8ba01dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:58:10.131040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24133a8-192e-40c6-b0f3-ab38a8ba01dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:10.136711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24133a8-192e-40c6-b0f3-ab38a8ba01dd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:10.153846 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1317/2660] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:10.158176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc7fcf-62b7-4ea8-bf8c-5add923df905 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:10.169648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc7fcf-62b7-4ea8-bf8c-5add923df905 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:10.181013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59ec42-d9a5-4b23-85f4-8e192c9adbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:10.182685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59ec42-d9a5-4b23-85f4-8e192c9adbd3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:10.183142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59ec42-d9a5-4b23-85f4-8e192c9adbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:10.186472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59ec42-d9a5-4b23-85f4-8e192c9adbd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:10.187963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59ec42-d9a5-4b23-85f4-8e192c9adbd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:10.190692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59ec42-d9a5-4b23-85f4-8e192c9adbd3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:10.244422 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1344/2661] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:58:10 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:10.247925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24133a8-192e-40c6-b0f3-ab38a8ba01dd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:10.262780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24133a8-192e-40c6-b0f3-ab38a8ba01dd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:10.385683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7e218-8fbe-4673-aae0-fe529cf31da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:10.385948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7e218-8fbe-4673-aae0-fe529cf31da4 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:10.386053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7e218-8fbe-4673-aae0-fe529cf31da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:10.386325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7e218-8fbe-4673-aae0-fe529cf31da4 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:10.387103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7e218-8fbe-4673-aae0-fe529cf31da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:10.389684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7e218-8fbe-4673-aae0-fe529cf31da4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:10.438298 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1318/2662] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:58:10 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:58:10.443360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59ec42-d9a5-4b23-85f4-8e192c9adbd3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:10.474720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59ec42-d9a5-4b23-85f4-8e192c9adbd3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:10.486744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca29f6c-0efc-4b4b-833b-db3309d584a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:10.487464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca29f6c-0efc-4b4b-833b-db3309d584a5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:10.487742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca29f6c-0efc-4b4b-833b-db3309d584a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:10.488016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca29f6c-0efc-4b4b-833b-db3309d584a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:10.490442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca29f6c-0efc-4b4b-833b-db3309d584a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:10.491848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca29f6c-0efc-4b4b-833b-db3309d584a5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:10.520617 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1345/2663] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:58:10.521208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7e218-8fbe-4673-aae0-fe529cf31da4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:10.536104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7e218-8fbe-4673-aae0-fe529cf31da4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:10.551830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fce88c-37e5-46fc-9123-83466955029a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:10.553026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fce88c-37e5-46fc-9123-83466955029a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:10.553450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fce88c-37e5-46fc-9123-83466955029a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:10.553914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fce88c-37e5-46fc-9123-83466955029a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:10.554776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fce88c-37e5-46fc-9123-83466955029a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:10.556917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fce88c-37e5-46fc-9123-83466955029a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:10.584078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca29f6c-0efc-4b4b-833b-db3309d584a5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:10.585454 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1319/2664] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:10.599405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca29f6c-0efc-4b4b-833b-db3309d584a5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:10.617251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd11d07-cf66-46cf-b5bc-9a35e34d4c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:10.618269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd11d07-cf66-46cf-b5bc-9a35e34d4c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:10.620233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd11d07-cf66-46cf-b5bc-9a35e34d4c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:10.623995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd11d07-cf66-46cf-b5bc-9a35e34d4c57 admin admin] PATH_INFO: `/v3/projects/1e4a377c3b9444c6aa448e3799d3cdb2/users/2281296acde947168cb5c5d97d2c5b6e/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:10.632802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:10.681271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd11d07-cf66-46cf-b5bc-9a35e34d4c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4a377c3b9444c6aa448e3799d3cdb2, user_id=2281296acde947168cb5c5d97d2c5b6e, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:10.685696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd11d07-cf66-46cf-b5bc-9a35e34d4c57 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:10.735324 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1346/2665] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:58:10 2022] PUT /identity/v3/projects/1e4a377c3b9444c6aa448e3799d3cdb2/users/2281296acde947168cb5c5d97d2c5b6e/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:10.736730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0ea69-d5ec-4151-b2b3-eb4669952643 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:10.737227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0ea69-d5ec-4151-b2b3-eb4669952643 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:10.737635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0ea69-d5ec-4151-b2b3-eb4669952643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:10.796448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0cc0ea69-d5ec-4151-b2b3-eb4669952643 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:10.796448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0cc0ea69-d5ec-4151-b2b3-eb4669952643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:10.907964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:10.954717 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1320/2666] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:10.997767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd11d07-cf66-46cf-b5bc-9a35e34d4c57 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:11.000085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3dd11d07-cf66-46cf-b5bc-9a35e34d4c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:11.162767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd11d07-cf66-46cf-b5bc-9a35e34d4c57 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:11.166812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973f9a6-4630-400f-a4aa-c6231e694e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:11.167487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973f9a6-4630-400f-a4aa-c6231e694e57 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:11.167821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973f9a6-4630-400f-a4aa-c6231e694e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:11.168144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973f9a6-4630-400f-a4aa-c6231e694e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:11.178239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973f9a6-4630-400f-a4aa-c6231e694e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:11.179844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973f9a6-4630-400f-a4aa-c6231e694e57 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:11.199573 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1347/2667] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:58:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:58:13.080362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e441d26-3ecc-496a-b7b1-335f8edf5f2f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:13.081038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e441d26-3ecc-496a-b7b1-335f8edf5f2f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:13.081038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e441d26-3ecc-496a-b7b1-335f8edf5f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:13.152957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0e441d26-3ecc-496a-b7b1-335f8edf5f2f None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:13.163194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0e441d26-3ecc-496a-b7b1-335f8edf5f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:13.383144 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1321/2668] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 20:58:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:13.397345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973f9a6-4630-400f-a4aa-c6231e694e57 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:13.399400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8973f9a6-4630-400f-a4aa-c6231e694e57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:13.598108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973f9a6-4630-400f-a4aa-c6231e694e57 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:13.606663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ab38e-512d-4c46-802e-8db3330acf21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:13.607765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ab38e-512d-4c46-802e-8db3330acf21 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:13.608210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ab38e-512d-4c46-802e-8db3330acf21 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:13.608724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ab38e-512d-4c46-802e-8db3330acf21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:13.621403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ab38e-512d-4c46-802e-8db3330acf21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:13.626127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ab38e-512d-4c46-802e-8db3330acf21 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:13.650431 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1348/2669] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 20:58:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 20:58:23.356270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=32 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:23.635930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=64,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:25.136963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e441d26-3ecc-496a-b7b1-335f8edf5f2f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:25.138925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0e441d26-3ecc-496a-b7b1-335f8edf5f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:25.167305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e441d26-3ecc-496a-b7b1-335f8edf5f2f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:25.176118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:25.177710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:25.178227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:25.178759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] PATH_INFO: `/v3/users/a3378bf4fa90489cb103378667f00399` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:25.180388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3378bf4fa90489cb103378667f00399)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:25.183109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:25.323877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3378bf4fa90489cb103378667f00399'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:25.343786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3378bf4fa90489cb103378667f00399'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:25.386979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3378bf4fa90489cb103378667f00399'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:25.389951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3378bf4fa90489cb103378667f00399'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:25.390640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3378bf4fa90489cb103378667f00399'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:25.429461 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1322/2670] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:58:25 2022] DELETE /identity/v3/users/a3378bf4fa90489cb103378667f00399 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:25.971363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ab38e-512d-4c46-802e-8db3330acf21 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:26.008344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ab38e-512d-4c46-802e-8db3330acf21 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:26.138243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:26.138756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:26.139091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:26.139299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] PATH_INFO: `/v3/projects/42cbcac0c2d54eaea98f071074003560` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:26.140593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=42cbcac0c2d54eaea98f071074003560)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:26.151136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:26.235133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42cbcac0c2d54eaea98f071074003560', 'request_id': 'req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:26.246712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42cbcac0c2d54eaea98f071074003560', 'request_id': 'req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:26.257386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42cbcac0c2d54eaea98f071074003560', 'request_id': 'req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:26.282604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] The token cache is being invalidate because project 42cbcac0c2d54eaea98f071074003560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:58:26.283124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:26.284526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:26.286838 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1349/2671] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:58:25 2022] DELETE /identity/v3/projects/42cbcac0c2d54eaea98f071074003560 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:26.290632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:26.292715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:26.315575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:26.317049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:26.323259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40910a-9de8-49f5-a0c4-821fdb5c80f7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:26.427349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b0d99-b3d9-4df9-9794-260922fab434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:26.430288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b0d99-b3d9-4df9-9794-260922fab434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:26.430918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b0d99-b3d9-4df9-9794-260922fab434 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:26.431652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b0d99-b3d9-4df9-9794-260922fab434 admin admin] PATH_INFO: `/v3/users/112c5395a1b243e684a5ec3db9990e9a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:26.434072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b0d99-b3d9-4df9-9794-260922fab434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=112c5395a1b243e684a5ec3db9990e9a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:26.439290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b0d99-b3d9-4df9-9794-260922fab434 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:26.457582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f558d57-d75a-405e-9a8d-c4c527bdcc5c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:26.472834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4be58-77d8-4e4d-801f-b148951603f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:26.473641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4be58-77d8-4e4d-801f-b148951603f9 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:26.473915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4be58-77d8-4e4d-801f-b148951603f9 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:26.474282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4be58-77d8-4e4d-801f-b148951603f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:26.476940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0ec4be58-77d8-4e4d-801f-b148951603f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:26.525185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:26.597898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4be58-77d8-4e4d-801f-b148951603f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:26.599752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4be58-77d8-4e4d-801f-b148951603f9 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:26.611943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-124b0d99-b3d9-4df9-9794-260922fab434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '112c5395a1b243e684a5ec3db9990e9a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:26.632339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-124b0d99-b3d9-4df9-9794-260922fab434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '112c5395a1b243e684a5ec3db9990e9a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:26.657729 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1350/2672] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:58:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:58:26.661737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-124b0d99-b3d9-4df9-9794-260922fab434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '112c5395a1b243e684a5ec3db9990e9a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:26.662804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-124b0d99-b3d9-4df9-9794-260922fab434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '112c5395a1b243e684a5ec3db9990e9a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:26.663609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-124b0d99-b3d9-4df9-9794-260922fab434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '112c5395a1b243e684a5ec3db9990e9a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:26.688284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:26.732222 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1323/2673] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:58:26 2022] DELETE /identity/v3/users/112c5395a1b243e684a5ec3db9990e9a => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:27.076226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4be58-77d8-4e4d-801f-b148951603f9 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:27.077700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0ec4be58-77d8-4e4d-801f-b148951603f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:27.102833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4be58-77d8-4e4d-801f-b148951603f9 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:27.341103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:27.343493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:27.343781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:27.344054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3 admin admin] PATH_INFO: `/v3/projects/ecc2ab17545442f68aa78857a0140871` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:27.346982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecc2ab17545442f68aa78857a0140871)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:27.351840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:27.481091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecc2ab17545442f68aa78857a0140871', 'request_id': 'req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:27.490796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecc2ab17545442f68aa78857a0140871', 'request_id': 'req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:27.499069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecc2ab17545442f68aa78857a0140871', 'request_id': 'req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:27.512547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3 admin admin] The token cache is being invalidate because project ecc2ab17545442f68aa78857a0140871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:58:27.513072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:27.513951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7c1d5e02-7bc8-4d35-bff4-c4b0eac003e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:27.515893 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1351/2674] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:58:27 2022] DELETE /identity/v3/projects/ecc2ab17545442f68aa78857a0140871 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:27.526271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b666bd-8817-42eb-bc4a-d8f3d83c1d57 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:27.526855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b666bd-8817-42eb-bc4a-d8f3d83c1d57 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:27.527009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b666bd-8817-42eb-bc4a-d8f3d83c1d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:27.548225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-68b666bd-8817-42eb-bc4a-d8f3d83c1d57 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:27.549836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-68b666bd-8817-42eb-bc4a-d8f3d83c1d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:27.631186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03936002-76b6-487e-b88c-5c18f84b6fcf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:27.631607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03936002-76b6-487e-b88c-5c18f84b6fcf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:27.632060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03936002-76b6-487e-b88c-5c18f84b6fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:27.636185 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1324/2675] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:27.665420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-03936002-76b6-487e-b88c-5c18f84b6fcf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:27.666713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-03936002-76b6-487e-b88c-5c18f84b6fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:27.680846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b666bd-8817-42eb-bc4a-d8f3d83c1d57 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:27.696669 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1352/2676] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:27.700020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b666bd-8817-42eb-bc4a-d8f3d83c1d57 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:27.706586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650092f-0d8a-4b04-8393-6560aa94c635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:27.707514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650092f-0d8a-4b04-8393-6560aa94c635 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:27.707998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650092f-0d8a-4b04-8393-6560aa94c635 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:27.708643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650092f-0d8a-4b04-8393-6560aa94c635 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:27.709517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650092f-0d8a-4b04-8393-6560aa94c635 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:58:27.710555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650092f-0d8a-4b04-8393-6560aa94c635 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:27.717077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650092f-0d8a-4b04-8393-6560aa94c635 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:27.746251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03936002-76b6-487e-b88c-5c18f84b6fcf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:27.753599 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1325/2677] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:27.758258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03936002-76b6-487e-b88c-5c18f84b6fcf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:27.763936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c291f5a-5a7d-486f-a968-dc647c1df8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:27.765194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c291f5a-5a7d-486f-a968-dc647c1df8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:27.765687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c291f5a-5a7d-486f-a968-dc647c1df8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:27.766278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c291f5a-5a7d-486f-a968-dc647c1df8c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:27.767139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c291f5a-5a7d-486f-a968-dc647c1df8c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:27.774335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c291f5a-5a7d-486f-a968-dc647c1df8c4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:27.815827 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1353/2678] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:58:27 2022] POST /identity/v3/projects => generated 393 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:27.817395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650092f-0d8a-4b04-8393-6560aa94c635 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:27.834933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650092f-0d8a-4b04-8393-6560aa94c635 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:27.981408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30645f84-e49c-4d61-aedd-bc18ab4db23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:27.982710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30645f84-e49c-4d61-aedd-bc18ab4db23f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:27.983479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30645f84-e49c-4d61-aedd-bc18ab4db23f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:27.984206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30645f84-e49c-4d61-aedd-bc18ab4db23f admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:27.985260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30645f84-e49c-4d61-aedd-bc18ab4db23f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:27.987684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30645f84-e49c-4d61-aedd-bc18ab4db23f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:28.041916 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1326/2679] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:58:27 2022] POST /identity/v3/users => generated 297 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:28.044874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c291f5a-5a7d-486f-a968-dc647c1df8c4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:28.060981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c291f5a-5a7d-486f-a968-dc647c1df8c4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:28.071635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c87dec-545f-433e-94a0-881323e89841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:28.072363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c87dec-545f-433e-94a0-881323e89841 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:28.074447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c87dec-545f-433e-94a0-881323e89841 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:28.074837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c87dec-545f-433e-94a0-881323e89841 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:28.075645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c87dec-545f-433e-94a0-881323e89841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:28.078356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c87dec-545f-433e-94a0-881323e89841 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:28.105927 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1354/2680] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:28.107260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30645f84-e49c-4d61-aedd-bc18ab4db23f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:28.177256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30645f84-e49c-4d61-aedd-bc18ab4db23f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:28.205343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd108c-8a84-4535-b0bb-a25f37119f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:28.206150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd108c-8a84-4535-b0bb-a25f37119f9b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:28.206431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd108c-8a84-4535-b0bb-a25f37119f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:28.206708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd108c-8a84-4535-b0bb-a25f37119f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:28.207382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd108c-8a84-4535-b0bb-a25f37119f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:28.210425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd108c-8a84-4535-b0bb-a25f37119f9b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:28.226873 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1327/2681] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:28.231220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c87dec-545f-433e-94a0-881323e89841 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:28.244216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c87dec-545f-433e-94a0-881323e89841 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:28.273414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ffdcf-03c4-4480-8d60-debd25fb9c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:28.274485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ffdcf-03c4-4480-8d60-debd25fb9c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:28.274485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ffdcf-03c4-4480-8d60-debd25fb9c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:28.274629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ffdcf-03c4-4480-8d60-debd25fb9c9d admin admin] PATH_INFO: `/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1/users/a84396aff9d14cb38bde06259b8059d9/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:28.297085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:28.305749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ffdcf-03c4-4480-8d60-debd25fb9c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=231b58fd6a8f410aa1e9fa8cf59ecaf1, user_id=a84396aff9d14cb38bde06259b8059d9, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:28.321738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ffdcf-03c4-4480-8d60-debd25fb9c9d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:28.356824 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1355/2682] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:58:28 2022] PUT /identity/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1/users/a84396aff9d14cb38bde06259b8059d9/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:28.367786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1584da5-4519-4913-bd3c-1c3313c410f9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:28.368159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1584da5-4519-4913-bd3c-1c3313c410f9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:28.368523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1584da5-4519-4913-bd3c-1c3313c410f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:28.451053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b1584da5-4519-4913-bd3c-1c3313c410f9 None None] MFA Rules not processed for user `a84396aff9d14cb38bde06259b8059d9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:28.453301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b1584da5-4519-4913-bd3c-1c3313c410f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:28.515557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:28.549572 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1328/2683] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:28.554069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ffdcf-03c4-4480-8d60-debd25fb9c9d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:28.644850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ffdcf-03c4-4480-8d60-debd25fb9c9d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:28.650954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751d1a3-ca21-47a1-8c99-88f7b37612e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:28.651581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751d1a3-ca21-47a1-8c99-88f7b37612e6 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:28.651852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751d1a3-ca21-47a1-8c99-88f7b37612e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:28.652254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751d1a3-ca21-47a1-8c99-88f7b37612e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:28.652931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751d1a3-ca21-47a1-8c99-88f7b37612e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:28.655913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751d1a3-ca21-47a1-8c99-88f7b37612e6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:28.727408 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1356/2684] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:58:28 2022] POST /identity/v3/projects => generated 393 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:28.727874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1584da5-4519-4913-bd3c-1c3313c410f9 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:28.743496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1584da5-4519-4913-bd3c-1c3313c410f9 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:28.936790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4314270-d1b1-4590-b568-081032f17abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:28.938729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4314270-d1b1-4590-b568-081032f17abb admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:28.939830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4314270-d1b1-4590-b568-081032f17abb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:28.940478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4314270-d1b1-4590-b568-081032f17abb admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:28.942777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4314270-d1b1-4590-b568-081032f17abb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:28.947827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4314270-d1b1-4590-b568-081032f17abb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:29.021195 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1329/2685] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:58:28 2022] POST /identity/v3/users => generated 303 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:58:29.025402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751d1a3-ca21-47a1-8c99-88f7b37612e6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:29.037919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751d1a3-ca21-47a1-8c99-88f7b37612e6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:29.065186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b040f3-fb2a-4680-b0f3-8d5b26c1af4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:29.065832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b040f3-fb2a-4680-b0f3-8d5b26c1af4a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:29.066103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b040f3-fb2a-4680-b0f3-8d5b26c1af4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:29.066876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b040f3-fb2a-4680-b0f3-8d5b26c1af4a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:29.067538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b040f3-fb2a-4680-b0f3-8d5b26c1af4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:29.069603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b040f3-fb2a-4680-b0f3-8d5b26c1af4a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:29.109794 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1357/2686] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:29.113670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4314270-d1b1-4590-b568-081032f17abb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:29.126134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4314270-d1b1-4590-b568-081032f17abb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:29.137732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c28698-173e-473a-8f2f-e8d7d6a3ce76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:29.138449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c28698-173e-473a-8f2f-e8d7d6a3ce76 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:29.138746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c28698-173e-473a-8f2f-e8d7d6a3ce76 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:29.138987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c28698-173e-473a-8f2f-e8d7d6a3ce76 admin admin] PATH_INFO: `/v3/projects/60c65e7e1fae4105a1949520942bed79/users/b367d61269fd4fe6a4cd11eb20d0b4e7/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:29.166055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c28698-173e-473a-8f2f-e8d7d6a3ce76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c65e7e1fae4105a1949520942bed79, user_id=b367d61269fd4fe6a4cd11eb20d0b4e7, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:29.171025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c28698-173e-473a-8f2f-e8d7d6a3ce76 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:29.201280 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1330/2687] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:58:29 2022] PUT /identity/v3/projects/60c65e7e1fae4105a1949520942bed79/users/b367d61269fd4fe6a4cd11eb20d0b4e7/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:29.205891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b040f3-fb2a-4680-b0f3-8d5b26c1af4a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:29.218627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b040f3-fb2a-4680-b0f3-8d5b26c1af4a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:29.336592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb34e6-d4eb-44c2-b789-8e10f90a6c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:29.337426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb34e6-d4eb-44c2-b789-8e10f90a6c4b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:29.337733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb34e6-d4eb-44c2-b789-8e10f90a6c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:29.338135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb34e6-d4eb-44c2-b789-8e10f90a6c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:29.340289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb34e6-d4eb-44c2-b789-8e10f90a6c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:29.342790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb34e6-d4eb-44c2-b789-8e10f90a6c4b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:29.371712 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1358/2688] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:29.376354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c28698-173e-473a-8f2f-e8d7d6a3ce76 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:29.392938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c28698-173e-473a-8f2f-e8d7d6a3ce76 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:29.401005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3ccd3-01ef-4d3c-a571-9e04901f3db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:29.402115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3ccd3-01ef-4d3c-a571-9e04901f3db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:29.402615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3ccd3-01ef-4d3c-a571-9e04901f3db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:29.402933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3ccd3-01ef-4d3c-a571-9e04901f3db0 admin admin] PATH_INFO: `/v3/projects/60c65e7e1fae4105a1949520942bed79/users/b367d61269fd4fe6a4cd11eb20d0b4e7/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:29.405759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3ccd3-01ef-4d3c-a571-9e04901f3db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c65e7e1fae4105a1949520942bed79, user_id=b367d61269fd4fe6a4cd11eb20d0b4e7, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:29.409095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3ccd3-01ef-4d3c-a571-9e04901f3db0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:29.457505 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1331/2689] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:58:29 2022] PUT /identity/v3/projects/60c65e7e1fae4105a1949520942bed79/users/b367d61269fd4fe6a4cd11eb20d0b4e7/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:29.468317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665844c-5d3f-4389-8131-322cfad4e790 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:29.468756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665844c-5d3f-4389-8131-322cfad4e790 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:29.469619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665844c-5d3f-4389-8131-322cfad4e790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:29.518455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f665844c-5d3f-4389-8131-322cfad4e790 None None] MFA Rules not processed for user `b367d61269fd4fe6a4cd11eb20d0b4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:29.520314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f665844c-5d3f-4389-8131-322cfad4e790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:29.627781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:29.719988 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1359/2690] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:29 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:29.721867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3ccd3-01ef-4d3c-a571-9e04901f3db0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:29.889508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3ccd3-01ef-4d3c-a571-9e04901f3db0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:29.919162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88262c-f058-4022-9f89-844e392f110f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:29.920527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88262c-f058-4022-9f89-844e392f110f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:29.920527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88262c-f058-4022-9f89-844e392f110f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:29.920949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88262c-f058-4022-9f89-844e392f110f admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:29.921721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88262c-f058-4022-9f89-844e392f110f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:29.923726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88262c-f058-4022-9f89-844e392f110f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:29.965848 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1332/2691] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:58:29 2022] POST /identity/v3/projects => generated 393 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:29.969940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665844c-5d3f-4389-8131-322cfad4e790 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:29.987811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665844c-5d3f-4389-8131-322cfad4e790 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:30.090464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4a7c5-9262-44c1-b22d-bbadf78cd531 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:30.090976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4a7c5-9262-44c1-b22d-bbadf78cd531 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:30.091383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4a7c5-9262-44c1-b22d-bbadf78cd531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:30.093508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512edddb-46e5-4320-b6bc-a6f39e8b67d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:30.094321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512edddb-46e5-4320-b6bc-a6f39e8b67d2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:30.094743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512edddb-46e5-4320-b6bc-a6f39e8b67d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:30.095638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512edddb-46e5-4320-b6bc-a6f39e8b67d2 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:30.098925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512edddb-46e5-4320-b6bc-a6f39e8b67d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:30.102121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512edddb-46e5-4320-b6bc-a6f39e8b67d2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:30.141539 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1360/2692] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:58:29 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:30.146260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512edddb-46e5-4320-b6bc-a6f39e8b67d2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:30.163778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512edddb-46e5-4320-b6bc-a6f39e8b67d2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:30.170575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196864c-9fa4-4e60-a36e-f34cc20884fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:30.171369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196864c-9fa4-4e60-a36e-f34cc20884fa admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:30.171631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196864c-9fa4-4e60-a36e-f34cc20884fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:30.171977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196864c-9fa4-4e60-a36e-f34cc20884fa admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:30.172665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196864c-9fa4-4e60-a36e-f34cc20884fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:30.175411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196864c-9fa4-4e60-a36e-f34cc20884fa admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:30.179489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6db4a7c5-9262-44c1-b22d-bbadf78cd531 None None] MFA Rules not processed for user `1762f75263ad4e2f84b265522e76764d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:30.180772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6db4a7c5-9262-44c1-b22d-bbadf78cd531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:30.195980 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1361/2693] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:30.202904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196864c-9fa4-4e60-a36e-f34cc20884fa admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:30.223272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196864c-9fa4-4e60-a36e-f34cc20884fa admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:30.238396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610bfd1-a4ba-41fc-bfe5-cc827a0154a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:30.239521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610bfd1-a4ba-41fc-bfe5-cc827a0154a2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:30.239859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610bfd1-a4ba-41fc-bfe5-cc827a0154a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:30.241517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610bfd1-a4ba-41fc-bfe5-cc827a0154a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:30.242247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610bfd1-a4ba-41fc-bfe5-cc827a0154a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:30.259777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610bfd1-a4ba-41fc-bfe5-cc827a0154a2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:30.267167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:30.306040 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1362/2694] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:30.313475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610bfd1-a4ba-41fc-bfe5-cc827a0154a2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:30.335456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610bfd1-a4ba-41fc-bfe5-cc827a0154a2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:30.350822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d529b8-ca26-455b-8bcb-31a2d1398ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:30.351636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d529b8-ca26-455b-8bcb-31a2d1398ae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:30.352022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d529b8-ca26-455b-8bcb-31a2d1398ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:30.352363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d529b8-ca26-455b-8bcb-31a2d1398ae6 admin admin] PATH_INFO: `/v3/projects/4469f359d98a4647bcafdc6809ec80d1/users/8fde9bb5c5b1408dbf815745c6092eeb/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:30.367130 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1333/2695] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:30.385811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d529b8-ca26-455b-8bcb-31a2d1398ae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4469f359d98a4647bcafdc6809ec80d1, user_id=8fde9bb5c5b1408dbf815745c6092eeb, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:30.388732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d529b8-ca26-455b-8bcb-31a2d1398ae6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:30.416132 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1363/2696] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:58:30 2022] PUT /identity/v3/projects/4469f359d98a4647bcafdc6809ec80d1/users/8fde9bb5c5b1408dbf815745c6092eeb/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:30.430246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2d3bf-f0f7-4542-940d-d1a892397c80 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:30.430924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2d3bf-f0f7-4542-940d-d1a892397c80 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:30.431565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2d3bf-f0f7-4542-940d-d1a892397c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:30.501008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-22b2d3bf-f0f7-4542-940d-d1a892397c80 None None] MFA Rules not processed for user `8fde9bb5c5b1408dbf815745c6092eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:30.502308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-22b2d3bf-f0f7-4542-940d-d1a892397c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:30.604765 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1334/2697] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:30.676964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6e147-8427-44cd-93dc-ffab497b9059 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:30.677358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6e147-8427-44cd-93dc-ffab497b9059 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:30.677715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6e147-8427-44cd-93dc-ffab497b9059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:30.720126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-55b6e147-8427-44cd-93dc-ffab497b9059 None None] MFA Rules not processed for user `22489a6c355a49028ca16a394b27876f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:30.722257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-55b6e147-8427-44cd-93dc-ffab497b9059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:30.837335 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1364/2698] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:30 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:31.441443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2d3bf-f0f7-4542-940d-d1a892397c80 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:31.443229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-22b2d3bf-f0f7-4542-940d-d1a892397c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:31.517080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e74c7-bc3f-40b9-a857-3ba0e3d23803 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:31.517633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e74c7-bc3f-40b9-a857-3ba0e3d23803 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:31.517892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e74c7-bc3f-40b9-a857-3ba0e3d23803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:31.564626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ef0e74c7-bc3f-40b9-a857-3ba0e3d23803 None None] MFA Rules not processed for user `2281296acde947168cb5c5d97d2c5b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:31.566548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ef0e74c7-bc3f-40b9-a857-3ba0e3d23803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:31.600485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:31.631923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2d3bf-f0f7-4542-940d-d1a892397c80 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:31.646125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e4635-fb01-4a3c-a8c4-9ea18dbf565a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:31.646987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e4635-fb01-4a3c-a8c4-9ea18dbf565a service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:31.647321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e4635-fb01-4a3c-a8c4-9ea18dbf565a service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:31.647650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e4635-fb01-4a3c-a8c4-9ea18dbf565a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:31.724603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:31.731644 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1365/2699] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:31.733452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04929ff-9c5c-4e53-9007-65066ba99e50 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:31.733798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04929ff-9c5c-4e53-9007-65066ba99e50 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:31.734130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04929ff-9c5c-4e53-9007-65066ba99e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:31.737802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e4635-fb01-4a3c-a8c4-9ea18dbf565a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:31.739833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e4635-fb01-4a3c-a8c4-9ea18dbf565a service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:31.764467 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1335/2700] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:58:31 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:58:31.773633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b04929ff-9c5c-4e53-9007-65066ba99e50 None None] MFA Rules not processed for user `2281296acde947168cb5c5d97d2c5b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:31.775895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b04929ff-9c5c-4e53-9007-65066ba99e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:31.792692 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1366/2701] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:33.028993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e4635-fb01-4a3c-a8c4-9ea18dbf565a service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:33.043557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e4635-fb01-4a3c-a8c4-9ea18dbf565a service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:33.074672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1f7ac-f832-424d-a7cd-17fb3a9b6bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:33.075438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1f7ac-f832-424d-a7cd-17fb3a9b6bd2 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:33.075868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1f7ac-f832-424d-a7cd-17fb3a9b6bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:33.107061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1f7ac-f832-424d-a7cd-17fb3a9b6bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:33.138769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1f7ac-f832-424d-a7cd-17fb3a9b6bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:33.143738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1f7ac-f832-424d-a7cd-17fb3a9b6bd2 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:33.146474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f388b-ac5c-4af9-83df-5775c263131c None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:33.146970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f388b-ac5c-4af9-83df-5775c263131c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:33.147164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f388b-ac5c-4af9-83df-5775c263131c None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:33.150469 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1367/2702] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:58:33.159745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f388b-ac5c-4af9-83df-5775c263131c None None] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:33.162185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8b5f388b-ac5c-4af9-83df-5775c263131c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:33.166360 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1336/2703] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:58:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:58:33.207450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f388b-ac5c-4af9-83df-5775c263131c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:33.253548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f388b-ac5c-4af9-83df-5775c263131c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:33.267283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421352ed-da28-4701-bfbd-fb3c735277a1 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b367d61269fd4fe6a4cd11eb20d0b4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60c65e7e1fae4105a1949520942bed79', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:33.268108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421352ed-da28-4701-bfbd-fb3c735277a1 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:33.268401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421352ed-da28-4701-bfbd-fb3c735277a1 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:33.268679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421352ed-da28-4701-bfbd-fb3c735277a1 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] PATH_INFO: `/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:33.270446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421352ed-da28-4701-bfbd-fb3c735277a1 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: Authorizing `identity:get_project(project_id=231b58fd6a8f410aa1e9fa8cf59ecaf1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:33.273090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421352ed-da28-4701-bfbd-fb3c735277a1 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:33.275189 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1368/2704] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:33 2022] GET /identity/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1 => generated 393 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:34.545471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e54b-a3fe-4f8f-8296-8b9068be85f6 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:34.545935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e54b-a3fe-4f8f-8296-8b9068be85f6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:34.546142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e54b-a3fe-4f8f-8296-8b9068be85f6 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:34.548501 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1337/2705] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:58:34.555477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421352ed-da28-4701-bfbd-fb3c735277a1 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:34.585973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421352ed-da28-4701-bfbd-fb3c735277a1 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:34.614154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421352ed-da28-4701-bfbd-fb3c735277a1 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:34.627865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d711325-f723-49d6-a6e5-d0322a59876b tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b367d61269fd4fe6a4cd11eb20d0b4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60c65e7e1fae4105a1949520942bed79', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:34.628647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d711325-f723-49d6-a6e5-d0322a59876b tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:34.628923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d711325-f723-49d6-a6e5-d0322a59876b tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:34.629188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d711325-f723-49d6-a6e5-d0322a59876b tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] PATH_INFO: `/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:34.630308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d711325-f723-49d6-a6e5-d0322a59876b tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: Authorizing `identity:get_project(project_id=231b58fd6a8f410aa1e9fa8cf59ecaf1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:34.632623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d711325-f723-49d6-a6e5-d0322a59876b tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:34.634585 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1369/2706] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:34 2022] GET /identity/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1 => generated 393 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:34.722088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e54b-a3fe-4f8f-8296-8b9068be85f6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:34.741910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e54b-a3fe-4f8f-8296-8b9068be85f6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:34.748512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc7068-da80-4909-8c45-8f314dd4f1b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:34.749139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc7068-da80-4909-8c45-8f314dd4f1b6 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:34.749409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc7068-da80-4909-8c45-8f314dd4f1b6 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:34.749671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc7068-da80-4909-8c45-8f314dd4f1b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:34.761087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a8fc7068-da80-4909-8c45-8f314dd4f1b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:34.857466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc7068-da80-4909-8c45-8f314dd4f1b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:34.859557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc7068-da80-4909-8c45-8f314dd4f1b6 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:34.887085 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1338/2707] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:58:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:58:35.278121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a4672-320c-4f89-b1a4-1c83bb0ee2f5 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:35.278914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a4672-320c-4f89-b1a4-1c83bb0ee2f5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:35.279420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a4672-320c-4f89-b1a4-1c83bb0ee2f5 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:35.281618 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1370/2708] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:58:35.291334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc7068-da80-4909-8c45-8f314dd4f1b6 service nova] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:35.326077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc7068-da80-4909-8c45-8f314dd4f1b6 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:35.381209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc7068-da80-4909-8c45-8f314dd4f1b6 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:35.401386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f497a-eb78-461e-abaf-ad9d6041f667 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b367d61269fd4fe6a4cd11eb20d0b4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60c65e7e1fae4105a1949520942bed79', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:35.402346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f497a-eb78-461e-abaf-ad9d6041f667 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:35.402817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f497a-eb78-461e-abaf-ad9d6041f667 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:35.403386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f497a-eb78-461e-abaf-ad9d6041f667 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] PATH_INFO: `/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:35.405769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f497a-eb78-461e-abaf-ad9d6041f667 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: Authorizing `identity:get_project(project_id=231b58fd6a8f410aa1e9fa8cf59ecaf1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:35.413614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f497a-eb78-461e-abaf-ad9d6041f667 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:35.415815 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1339/2709] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:35 2022] GET /identity/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1 => generated 393 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:36.562789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a4672-320c-4f89-b1a4-1c83bb0ee2f5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:36.581089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a4672-320c-4f89-b1a4-1c83bb0ee2f5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:36.588784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a9ac4-451a-4100-a271-e3e35983c17e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:36.589748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a9ac4-451a-4100-a271-e3e35983c17e service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:36.590164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a9ac4-451a-4100-a271-e3e35983c17e service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:36.590569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a9ac4-451a-4100-a271-e3e35983c17e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:36.660738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a9ac4-451a-4100-a271-e3e35983c17e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:36.662623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a9ac4-451a-4100-a271-e3e35983c17e service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:36.686997 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1371/2710] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:58:36 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:58:36.842499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d842d-1761-44b7-9bc5-cac0f44f3534 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:36.843212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d842d-1761-44b7-9bc5-cac0f44f3534 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:36.843212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d842d-1761-44b7-9bc5-cac0f44f3534 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:36.844621 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1340/2711] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:58:36.854004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a9ac4-451a-4100-a271-e3e35983c17e service nova] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:36.868628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a9ac4-451a-4100-a271-e3e35983c17e service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:36.881409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a9ac4-451a-4100-a271-e3e35983c17e service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:36.887423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20850810-b7b4-4c70-8420-56158467b613 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b367d61269fd4fe6a4cd11eb20d0b4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60c65e7e1fae4105a1949520942bed79', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:36.888656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20850810-b7b4-4c70-8420-56158467b613 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:36.889631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20850810-b7b4-4c70-8420-56158467b613 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:36.889910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20850810-b7b4-4c70-8420-56158467b613 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] PATH_INFO: `/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:36.892571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20850810-b7b4-4c70-8420-56158467b613 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: Authorizing `identity:get_project(project_id=231b58fd6a8f410aa1e9fa8cf59ecaf1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:36.895057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20850810-b7b4-4c70-8420-56158467b613 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:36.902378 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1372/2712] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:36 2022] GET /identity/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1 => generated 393 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:36.960946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96946be2-fac7-4d4d-b1c4-c4f5c66cf936 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:36.961188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96946be2-fac7-4d4d-b1c4-c4f5c66cf936 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:36.961459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96946be2-fac7-4d4d-b1c4-c4f5c66cf936 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:36.962832 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1341/2713] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:58:36.968424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20850810-b7b4-4c70-8420-56158467b613 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:36.988130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20850810-b7b4-4c70-8420-56158467b613 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:36.999037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20850810-b7b4-4c70-8420-56158467b613 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:37.004100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d4024-33f7-407f-8754-24ded16d9a4c tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b367d61269fd4fe6a4cd11eb20d0b4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60c65e7e1fae4105a1949520942bed79', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:37.004996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d4024-33f7-407f-8754-24ded16d9a4c tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:37.005450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d4024-33f7-407f-8754-24ded16d9a4c tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:37.005868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d4024-33f7-407f-8754-24ded16d9a4c tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] PATH_INFO: `/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:37.007152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d4024-33f7-407f-8754-24ded16d9a4c tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: Authorizing `identity:get_project(project_id=231b58fd6a8f410aa1e9fa8cf59ecaf1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:37.009238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d4024-33f7-407f-8754-24ded16d9a4c tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:37.010914 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1373/2714] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:36 2022] GET /identity/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1 => generated 393 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:37.320314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15029b3e-8c80-4b8e-b7f8-597a03dd7e60 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:37.320845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15029b3e-8c80-4b8e-b7f8-597a03dd7e60 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:37.321258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15029b3e-8c80-4b8e-b7f8-597a03dd7e60 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:37.323531 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1342/2715] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:58:37.330226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d4024-33f7-407f-8754-24ded16d9a4c tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:37.362357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d4024-33f7-407f-8754-24ded16d9a4c tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:37.375989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d4024-33f7-407f-8754-24ded16d9a4c tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:37.385198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fcd9c-baa8-4c7c-9e99-d50424fe61b8 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b367d61269fd4fe6a4cd11eb20d0b4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60c65e7e1fae4105a1949520942bed79', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:37.386304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fcd9c-baa8-4c7c-9e99-d50424fe61b8 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:37.387528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fcd9c-baa8-4c7c-9e99-d50424fe61b8 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:37.387878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fcd9c-baa8-4c7c-9e99-d50424fe61b8 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] PATH_INFO: `/v3/projects/4469f359d98a4647bcafdc6809ec80d1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:37.389327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fcd9c-baa8-4c7c-9e99-d50424fe61b8 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: Authorizing `identity:get_project(project_id=4469f359d98a4647bcafdc6809ec80d1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:37.393179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fcd9c-baa8-4c7c-9e99-d50424fe61b8 tempest-FlavorsAccessNegativeTestJSON-333234287 tempest-FlavorsAccessNegativeTestJSON-333234287-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:37.395598 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1374/2716] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:37 2022] GET /identity/v3/projects/4469f359d98a4647bcafdc6809ec80d1 => generated 393 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:37.523644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15029b3e-8c80-4b8e-b7f8-597a03dd7e60 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:37.648858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15029b3e-8c80-4b8e-b7f8-597a03dd7e60 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:37.659521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:37.660505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:37.660966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:37.661418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] PATH_INFO: `/v3/users/a84396aff9d14cb38bde06259b8059d9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:37.662830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a84396aff9d14cb38bde06259b8059d9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:37.665842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:37.757198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a84396aff9d14cb38bde06259b8059d9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:37.768236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a84396aff9d14cb38bde06259b8059d9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:37.782612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:37.784252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a84396aff9d14cb38bde06259b8059d9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:37.785411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a84396aff9d14cb38bde06259b8059d9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:37.787010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a84396aff9d14cb38bde06259b8059d9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:37.808720 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1343/2717] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:58:37 2022] DELETE /identity/v3/users/a84396aff9d14cb38bde06259b8059d9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:37.814327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ec8ea-f12a-4c76-b79c-243d8b11d950 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:37.815260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ec8ea-f12a-4c76-b79c-243d8b11d950 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:37.815710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ec8ea-f12a-4c76-b79c-243d8b11d950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:37.838016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5d2ec8ea-f12a-4c76-b79c-243d8b11d950 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:37.839634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5d2ec8ea-f12a-4c76-b79c-243d8b11d950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:37.922739 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1375/2718] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:37.962487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:37.969760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:38.101548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7356b9-19bc-49b7-a673-e70dfc6bbd8b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:38.122434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62d193-df19-4f58-a1b1-f9b13932cd2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:38.124315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62d193-df19-4f58-a1b1-f9b13932cd2b service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:38.124793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62d193-df19-4f58-a1b1-f9b13932cd2b service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:38.125242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62d193-df19-4f58-a1b1-f9b13932cd2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:38.146405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62d193-df19-4f58-a1b1-f9b13932cd2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:38.149238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62d193-df19-4f58-a1b1-f9b13932cd2b service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:38.163654 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1344/2719] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:58:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:58:38.276474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e96194-3065-4edc-beb8-dc93a630664a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:38.276675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e96194-3065-4edc-beb8-dc93a630664a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:38.276971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e96194-3065-4edc-beb8-dc93a630664a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:38.319684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d9e96194-3065-4edc-beb8-dc93a630664a None None] MFA Rules not processed for user `b91bab791d584fde8ce3f5c9c8b45ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:38.321069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d9e96194-3065-4edc-beb8-dc93a630664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:38.335753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:38.403071 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1376/2720] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:38.763315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc08303-562b-4801-8b2d-34fa12955488 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:38.763668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc08303-562b-4801-8b2d-34fa12955488 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:38.764001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc08303-562b-4801-8b2d-34fa12955488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:38.937411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2fc08303-562b-4801-8b2d-34fa12955488 None None] MFA Rules not processed for user `8afc45f50dc54b84a8abf3b6e168426b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:38.938884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2fc08303-562b-4801-8b2d-34fa12955488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:39.029108 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1345/2721] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:39.375893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e96194-3065-4edc-beb8-dc93a630664a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:39.377541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d9e96194-3065-4edc-beb8-dc93a630664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:39.396430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e96194-3065-4edc-beb8-dc93a630664a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:39.407055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:39.408601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:39.409467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:39.410757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] PATH_INFO: `/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:39.412459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=231b58fd6a8f410aa1e9fa8cf59ecaf1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:39.414663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:39.492109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231b58fd6a8f410aa1e9fa8cf59ecaf1', 'request_id': 'req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:39.503193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231b58fd6a8f410aa1e9fa8cf59ecaf1', 'request_id': 'req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:39.534768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231b58fd6a8f410aa1e9fa8cf59ecaf1', 'request_id': 'req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:39.555653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] The token cache is being invalidate because project 231b58fd6a8f410aa1e9fa8cf59ecaf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:58:39.555863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:39.556683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:39.562453 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1377/2722] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:58:39 2022] DELETE /identity/v3/projects/231b58fd6a8f410aa1e9fa8cf59ecaf1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:39.563747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc08303-562b-4801-8b2d-34fa12955488 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:39.586432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc08303-562b-4801-8b2d-34fa12955488 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:39.651930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:39.652990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:39.653478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:39.654213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] PATH_INFO: `/v3/users/b367d61269fd4fe6a4cd11eb20d0b4e7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:39.656406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b367d61269fd4fe6a4cd11eb20d0b4e7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:39.658401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:39.715880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:39.729096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b367d61269fd4fe6a4cd11eb20d0b4e7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:39.746274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b367d61269fd4fe6a4cd11eb20d0b4e7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:39.760778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b367d61269fd4fe6a4cd11eb20d0b4e7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:39.762616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b367d61269fd4fe6a4cd11eb20d0b4e7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:39.763309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b367d61269fd4fe6a4cd11eb20d0b4e7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:39.780253 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1346/2723] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:58:39 2022] DELETE /identity/v3/users/b367d61269fd4fe6a4cd11eb20d0b4e7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:58:40.402515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:40.404732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:40.432012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e3fb7-d56a-4f6d-a0b6-0436a2d10942 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:40.513341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:40.514108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:40.514501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:40.514774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] PATH_INFO: `/v3/projects/60c65e7e1fae4105a1949520942bed79` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:40.517865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60c65e7e1fae4105a1949520942bed79)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:40.520234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:40.571591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:40.579800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60c65e7e1fae4105a1949520942bed79', 'request_id': 'req-60292c21-8eee-467e-ad2b-5dcd821c9fb3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:40.588677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60c65e7e1fae4105a1949520942bed79', 'request_id': 'req-60292c21-8eee-467e-ad2b-5dcd821c9fb3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:40.596141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60c65e7e1fae4105a1949520942bed79', 'request_id': 'req-60292c21-8eee-467e-ad2b-5dcd821c9fb3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:40.611494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] The token cache is being invalidate because project 60c65e7e1fae4105a1949520942bed79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:58:40.612243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:40.613264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:40.615259 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1378/2724] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:58:40 2022] DELETE /identity/v3/projects/60c65e7e1fae4105a1949520942bed79 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:40.619408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:40.621302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:40.635463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86b12b-28d2-466e-9c0a-cd12f618780a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:40.716172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e9b5f-6521-4931-bfdb-b75de0369a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:40.717187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e9b5f-6521-4931-bfdb-b75de0369a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:40.717694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e9b5f-6521-4931-bfdb-b75de0369a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:40.718976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e9b5f-6521-4931-bfdb-b75de0369a29 admin admin] PATH_INFO: `/v3/users/8fde9bb5c5b1408dbf815745c6092eeb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:40.720475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e9b5f-6521-4931-bfdb-b75de0369a29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fde9bb5c5b1408dbf815745c6092eeb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:40.722729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e9b5f-6521-4931-bfdb-b75de0369a29 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:40.792196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f11e9b5f-6521-4931-bfdb-b75de0369a29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fde9bb5c5b1408dbf815745c6092eeb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:40.805496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f11e9b5f-6521-4931-bfdb-b75de0369a29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fde9bb5c5b1408dbf815745c6092eeb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:40.829938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f11e9b5f-6521-4931-bfdb-b75de0369a29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fde9bb5c5b1408dbf815745c6092eeb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:40.831046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f11e9b5f-6521-4931-bfdb-b75de0369a29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fde9bb5c5b1408dbf815745c6092eeb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:40.832029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f11e9b5f-6521-4931-bfdb-b75de0369a29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fde9bb5c5b1408dbf815745c6092eeb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:40.854862 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1347/2725] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:58:40 2022] DELETE /identity/v3/users/8fde9bb5c5b1408dbf815745c6092eeb => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:41.855091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:41.857326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:41.878851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60292c21-8eee-467e-ad2b-5dcd821c9fb3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:42.083018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4996e5-672b-4b3e-b436-c7bb3022eaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:42.083936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4996e5-672b-4b3e-b436-c7bb3022eaac admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:42.084366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4996e5-672b-4b3e-b436-c7bb3022eaac admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:42.084873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4996e5-672b-4b3e-b436-c7bb3022eaac admin admin] PATH_INFO: `/v3/projects/4469f359d98a4647bcafdc6809ec80d1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:42.087693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4996e5-672b-4b3e-b436-c7bb3022eaac admin admin] RBAC: Authorizing `identity:delete_project(project_id=4469f359d98a4647bcafdc6809ec80d1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:42.090117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4996e5-672b-4b3e-b436-c7bb3022eaac admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:42.223968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cf4996e5-672b-4b3e-b436-c7bb3022eaac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4469f359d98a4647bcafdc6809ec80d1', 'request_id': 'req-cf4996e5-672b-4b3e-b436-c7bb3022eaac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:42.240279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cf4996e5-672b-4b3e-b436-c7bb3022eaac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4469f359d98a4647bcafdc6809ec80d1', 'request_id': 'req-cf4996e5-672b-4b3e-b436-c7bb3022eaac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:42.258248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cf4996e5-672b-4b3e-b436-c7bb3022eaac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4469f359d98a4647bcafdc6809ec80d1', 'request_id': 'req-cf4996e5-672b-4b3e-b436-c7bb3022eaac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:42.285261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cf4996e5-672b-4b3e-b436-c7bb3022eaac admin admin] The token cache is being invalidate because project 4469f359d98a4647bcafdc6809ec80d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:58:42.285554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cf4996e5-672b-4b3e-b436-c7bb3022eaac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:42.286375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cf4996e5-672b-4b3e-b436-c7bb3022eaac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:42.292428 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1379/2726] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:58:41 2022] DELETE /identity/v3/projects/4469f359d98a4647bcafdc6809ec80d1 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:42.295273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0784ab3-c010-4105-a8ba-639a4e30fde4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:42.296103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0784ab3-c010-4105-a8ba-639a4e30fde4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:42.296416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0784ab3-c010-4105-a8ba-639a4e30fde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:42.343779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e0784ab3-c010-4105-a8ba-639a4e30fde4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:42.348646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e0784ab3-c010-4105-a8ba-639a4e30fde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:42.449119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:42.516264 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1348/2727] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:42.530375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa8b0d-483d-4e0b-9f96-3b7cb259828a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:42.530895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa8b0d-483d-4e0b-9f96-3b7cb259828a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:42.531235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa8b0d-483d-4e0b-9f96-3b7cb259828a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:42.586785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5daa8b0d-483d-4e0b-9f96-3b7cb259828a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:42.588393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5daa8b0d-483d-4e0b-9f96-3b7cb259828a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:42.608228 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1380/2728] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:42.610431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0784ab3-c010-4105-a8ba-639a4e30fde4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:42.623597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0784ab3-c010-4105-a8ba-639a4e30fde4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:42.630277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8d98f-2b4d-442c-b6fd-993a837af7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:42.631165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8d98f-2b4d-442c-b6fd-993a837af7ea admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:42.631603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8d98f-2b4d-442c-b6fd-993a837af7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:42.632033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8d98f-2b4d-442c-b6fd-993a837af7ea admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:42.632873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8d98f-2b4d-442c-b6fd-993a837af7ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:58:42.633337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8d98f-2b4d-442c-b6fd-993a837af7ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:42.635287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8d98f-2b4d-442c-b6fd-993a837af7ea admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:42.654895 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1349/2729] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:42.659116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa8b0d-483d-4e0b-9f96-3b7cb259828a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:42.674091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa8b0d-483d-4e0b-9f96-3b7cb259828a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:42.684002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4441f-b7ef-463f-821e-48afa59f8927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:42.685751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4441f-b7ef-463f-821e-48afa59f8927 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:42.686210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4441f-b7ef-463f-821e-48afa59f8927 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:42.686701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4441f-b7ef-463f-821e-48afa59f8927 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:42.689187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4441f-b7ef-463f-821e-48afa59f8927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:42.692766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4441f-b7ef-463f-821e-48afa59f8927 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:42.748579 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1381/2730] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:58:42 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:42.753184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8d98f-2b4d-442c-b6fd-993a837af7ea admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:42.768663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8d98f-2b4d-442c-b6fd-993a837af7ea admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:42.897989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2ee6e-b112-40c1-a07e-42098df22136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:42.898615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2ee6e-b112-40c1-a07e-42098df22136 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:42.898902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2ee6e-b112-40c1-a07e-42098df22136 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:42.899154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2ee6e-b112-40c1-a07e-42098df22136 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:42.899782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2ee6e-b112-40c1-a07e-42098df22136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:42.901832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2ee6e-b112-40c1-a07e-42098df22136 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:42.945008 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1350/2731] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:58:42 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:58:42.947967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4441f-b7ef-463f-821e-48afa59f8927 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:42.962600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4441f-b7ef-463f-821e-48afa59f8927 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:42.970767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ad0ba-0fc8-48c4-b04f-283358e4a6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:42.971397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ad0ba-0fc8-48c4-b04f-283358e4a6ca admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:42.971679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ad0ba-0fc8-48c4-b04f-283358e4a6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:42.972593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ad0ba-0fc8-48c4-b04f-283358e4a6ca admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:42.975105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ad0ba-0fc8-48c4-b04f-283358e4a6ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:42.976934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ad0ba-0fc8-48c4-b04f-283358e4a6ca admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:43.005732 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1382/2732] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:43.006947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2ee6e-b112-40c1-a07e-42098df22136 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:43.019879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2ee6e-b112-40c1-a07e-42098df22136 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:43.024544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe6d01-9aa3-442d-ab5b-bb6684100002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:43.025456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe6d01-9aa3-442d-ab5b-bb6684100002 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:43.025670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe6d01-9aa3-442d-ab5b-bb6684100002 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:43.026049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe6d01-9aa3-442d-ab5b-bb6684100002 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:43.026541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe6d01-9aa3-442d-ab5b-bb6684100002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:43.028958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe6d01-9aa3-442d-ab5b-bb6684100002 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:43.050489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ad0ba-0fc8-48c4-b04f-283358e4a6ca admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:43.057743 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1351/2733] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:43.062074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ad0ba-0fc8-48c4-b04f-283358e4a6ca admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:43.067719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db3535-0c95-4469-bece-0384a06aac9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:43.068981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db3535-0c95-4469-bece-0384a06aac9e admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:43.070069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db3535-0c95-4469-bece-0384a06aac9e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:43.070344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db3535-0c95-4469-bece-0384a06aac9e admin admin] PATH_INFO: `/v3/projects/617f326c60cc4e08aff6e39a7da18258/users/a0d378fc380044178ccee83de30400d4/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:43.106740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db3535-0c95-4469-bece-0384a06aac9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=617f326c60cc4e08aff6e39a7da18258, user_id=a0d378fc380044178ccee83de30400d4, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:43.109957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db3535-0c95-4469-bece-0384a06aac9e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:43.113769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:43.138448 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1383/2734] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:58:43 2022] PUT /identity/v3/projects/617f326c60cc4e08aff6e39a7da18258/users/a0d378fc380044178ccee83de30400d4/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:43.155994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ada9b0-c2ed-4ed1-8b1a-76dd3adb88f1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:43.156285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ada9b0-c2ed-4ed1-8b1a-76dd3adb88f1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:43.156595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ada9b0-c2ed-4ed1-8b1a-76dd3adb88f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:43.228592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c8ada9b0-c2ed-4ed1-8b1a-76dd3adb88f1 None None] MFA Rules not processed for user `a0d378fc380044178ccee83de30400d4`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:43.230199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c8ada9b0-c2ed-4ed1-8b1a-76dd3adb88f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:43.330325 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1352/2735] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:43.334797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db3535-0c95-4469-bece-0384a06aac9e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:43.473993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db3535-0c95-4469-bece-0384a06aac9e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:43.479959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243fe16-3216-4ebb-ac60-76c0aae3eb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:43.480928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243fe16-3216-4ebb-ac60-76c0aae3eb4e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:43.481287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243fe16-3216-4ebb-ac60-76c0aae3eb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:43.481802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243fe16-3216-4ebb-ac60-76c0aae3eb4e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:43.483246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243fe16-3216-4ebb-ac60-76c0aae3eb4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:43.486053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243fe16-3216-4ebb-ac60-76c0aae3eb4e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:43.537689 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1384/2736] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:58:43 2022] POST /identity/v3/projects => generated 383 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:58:43.542523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ada9b0-c2ed-4ed1-8b1a-76dd3adb88f1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:43.560145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ada9b0-c2ed-4ed1-8b1a-76dd3adb88f1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:43.596711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:43.648252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8383f-c406-4ddd-b6eb-4312e3af0a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:43.649259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8383f-c406-4ddd-b6eb-4312e3af0a38 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:43.649535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8383f-c406-4ddd-b6eb-4312e3af0a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:43.649802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8383f-c406-4ddd-b6eb-4312e3af0a38 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:43.650431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8383f-c406-4ddd-b6eb-4312e3af0a38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:43.652315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8383f-c406-4ddd-b6eb-4312e3af0a38 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:43.688454 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1353/2737] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:58:43 2022] POST /identity/v3/users => generated 298 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:43.691215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243fe16-3216-4ebb-ac60-76c0aae3eb4e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:43.703854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243fe16-3216-4ebb-ac60-76c0aae3eb4e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:43.709040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f23de-4359-4e01-a4cc-380a34df7d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:43.709834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f23de-4359-4e01-a4cc-380a34df7d08 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:43.710115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f23de-4359-4e01-a4cc-380a34df7d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:43.710547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f23de-4359-4e01-a4cc-380a34df7d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:43.711200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f23de-4359-4e01-a4cc-380a34df7d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:43.712848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f23de-4359-4e01-a4cc-380a34df7d08 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:43.725331 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1385/2738] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:43.729524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8383f-c406-4ddd-b6eb-4312e3af0a38 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:43.744963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8383f-c406-4ddd-b6eb-4312e3af0a38 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:43.750935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46becf2-4021-4d27-87c0-433ac3b17d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:43.752029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46becf2-4021-4d27-87c0-433ac3b17d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:43.752553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46becf2-4021-4d27-87c0-433ac3b17d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:43.753037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46becf2-4021-4d27-87c0-433ac3b17d97 admin admin] PATH_INFO: `/v3/projects/8e25ccab5db24e88bb50233b759e37ff/users/6c461590591b4985b337c2931bcda169/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:43.771140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46becf2-4021-4d27-87c0-433ac3b17d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e25ccab5db24e88bb50233b759e37ff, user_id=6c461590591b4985b337c2931bcda169, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:43.774591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46becf2-4021-4d27-87c0-433ac3b17d97 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:43.792773 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1354/2739] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:58:43 2022] PUT /identity/v3/projects/8e25ccab5db24e88bb50233b759e37ff/users/6c461590591b4985b337c2931bcda169/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:43.796651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f23de-4359-4e01-a4cc-380a34df7d08 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:43.819056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f23de-4359-4e01-a4cc-380a34df7d08 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:43.918105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03bc7f-2deb-4325-80de-e367723a656a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:43.918501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03bc7f-2deb-4325-80de-e367723a656a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:43.919382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03bc7f-2deb-4325-80de-e367723a656a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:43.919607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03bc7f-2deb-4325-80de-e367723a656a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:43.920467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03bc7f-2deb-4325-80de-e367723a656a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:43.923850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03bc7f-2deb-4325-80de-e367723a656a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:43.942094 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1386/2740] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:43.946348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46becf2-4021-4d27-87c0-433ac3b17d97 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:43.971690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46becf2-4021-4d27-87c0-433ac3b17d97 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:43.979168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528108f4-4685-4e9f-8fb1-2945f1fda181 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:43.981387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528108f4-4685-4e9f-8fb1-2945f1fda181 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:43.982039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528108f4-4685-4e9f-8fb1-2945f1fda181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:43.988197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2f021-319d-4829-b16d-c4033d6ab85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:43.989995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2f021-319d-4829-b16d-c4033d6ab85f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:43.992198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2f021-319d-4829-b16d-c4033d6ab85f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:43.992732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2f021-319d-4829-b16d-c4033d6ab85f admin admin] PATH_INFO: `/v3/projects/8e25ccab5db24e88bb50233b759e37ff/users/6c461590591b4985b337c2931bcda169/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:44.005368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2f021-319d-4829-b16d-c4033d6ab85f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e25ccab5db24e88bb50233b759e37ff, user_id=6c461590591b4985b337c2931bcda169, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:44.008205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2f021-319d-4829-b16d-c4033d6ab85f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:44.028534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-528108f4-4685-4e9f-8fb1-2945f1fda181 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:44.035872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-528108f4-4685-4e9f-8fb1-2945f1fda181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:44.056407 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1355/2741] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:58:43 2022] PUT /identity/v3/projects/8e25ccab5db24e88bb50233b759e37ff/users/6c461590591b4985b337c2931bcda169/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:44.058240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5261885-9dfc-4ff7-afd5-ed702ed59db1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:44.058522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5261885-9dfc-4ff7-afd5-ed702ed59db1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:44.058790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5261885-9dfc-4ff7-afd5-ed702ed59db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:44.127459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e5261885-9dfc-4ff7-afd5-ed702ed59db1 None None] MFA Rules not processed for user `6c461590591b4985b337c2931bcda169`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:44.129120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e5261885-9dfc-4ff7-afd5-ed702ed59db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:44.133402 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1387/2742] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 20:58:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:58:44.141371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528108f4-4685-4e9f-8fb1-2945f1fda181 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:44.143827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-528108f4-4685-4e9f-8fb1-2945f1fda181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:44.162988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:44.254283 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1356/2743] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:44.262111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528108f4-4685-4e9f-8fb1-2945f1fda181 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:44.268955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3a7da-9d57-454e-931e-4c50f0f5bbaf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:44.269822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3a7da-9d57-454e-931e-4c50f0f5bbaf service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:44.270016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3a7da-9d57-454e-931e-4c50f0f5bbaf service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:44.270251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3a7da-9d57-454e-931e-4c50f0f5bbaf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:44.280573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3a7da-9d57-454e-931e-4c50f0f5bbaf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:44.282454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3a7da-9d57-454e-931e-4c50f0f5bbaf service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:44.293433 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1388/2744] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 20:58:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 20:58:45.238186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5261885-9dfc-4ff7-afd5-ed702ed59db1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:45.239969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e5261885-9dfc-4ff7-afd5-ed702ed59db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:45.421672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:45.425796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5261885-9dfc-4ff7-afd5-ed702ed59db1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:45.437932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580f2fd-5689-4197-a8d3-149fc4e076ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:45.439019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580f2fd-5689-4197-a8d3-149fc4e076ff service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:45.439443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580f2fd-5689-4197-a8d3-149fc4e076ff service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:45.439986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580f2fd-5689-4197-a8d3-149fc4e076ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:45.471850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580f2fd-5689-4197-a8d3-149fc4e076ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:45.473682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580f2fd-5689-4197-a8d3-149fc4e076ff service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:45.511725 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1357/2745] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:58:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:58:47.857568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3a7da-9d57-454e-931e-4c50f0f5bbaf service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:47.994606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3a7da-9d57-454e-931e-4c50f0f5bbaf service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:48.018750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:48.019493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:48.019768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:48.020120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] PATH_INFO: `/v3/users/a0d378fc380044178ccee83de30400d4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:48.022888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d378fc380044178ccee83de30400d4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:48.025279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:48.169611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d378fc380044178ccee83de30400d4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:48.185873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d378fc380044178ccee83de30400d4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:48.197019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d378fc380044178ccee83de30400d4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:48.226484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d378fc380044178ccee83de30400d4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:48.227629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d378fc380044178ccee83de30400d4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:48.239726 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1389/2746] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:58:47 2022] DELETE /identity/v3/users/a0d378fc380044178ccee83de30400d4 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:48.245084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d64727-8f47-48d3-a153-34296b25b0fd None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:48.245626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d64727-8f47-48d3-a153-34296b25b0fd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:48.245626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d64727-8f47-48d3-a153-34296b25b0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:48.270290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a0d64727-8f47-48d3-a153-34296b25b0fd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:48.271513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a0d64727-8f47-48d3-a153-34296b25b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:48.447678 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1358/2747] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:48.508831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:48.510757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:48.556695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:48.620828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569c53f-3b61-400e-837a-12204ec11cbd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:48.628250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83cab3-9467-468e-9cbd-3764439623a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:48.629353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83cab3-9467-468e-9cbd-3764439623a5 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:48.629684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83cab3-9467-468e-9cbd-3764439623a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:48.632981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83cab3-9467-468e-9cbd-3764439623a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:48.657092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83cab3-9467-468e-9cbd-3764439623a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:48.659169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83cab3-9467-468e-9cbd-3764439623a5 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:48.677098 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1390/2748] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:58:49.332900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d64727-8f47-48d3-a153-34296b25b0fd None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:49.334946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a0d64727-8f47-48d3-a153-34296b25b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:49.361003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d64727-8f47-48d3-a153-34296b25b0fd None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:49.377051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22009474-85df-4692-a803-20dd4b29745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:49.379700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22009474-85df-4692-a803-20dd4b29745b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:49.380564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22009474-85df-4692-a803-20dd4b29745b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:49.380784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22009474-85df-4692-a803-20dd4b29745b admin admin] PATH_INFO: `/v3/projects/617f326c60cc4e08aff6e39a7da18258` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:49.386171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22009474-85df-4692-a803-20dd4b29745b admin admin] RBAC: Authorizing `identity:delete_project(project_id=617f326c60cc4e08aff6e39a7da18258)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:49.388817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22009474-85df-4692-a803-20dd4b29745b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:49.465530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22009474-85df-4692-a803-20dd4b29745b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '617f326c60cc4e08aff6e39a7da18258', 'request_id': 'req-22009474-85df-4692-a803-20dd4b29745b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:49.488108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22009474-85df-4692-a803-20dd4b29745b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '617f326c60cc4e08aff6e39a7da18258', 'request_id': 'req-22009474-85df-4692-a803-20dd4b29745b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:49.505751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22009474-85df-4692-a803-20dd4b29745b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '617f326c60cc4e08aff6e39a7da18258', 'request_id': 'req-22009474-85df-4692-a803-20dd4b29745b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:49.525614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22009474-85df-4692-a803-20dd4b29745b admin admin] The token cache is being invalidate because project 617f326c60cc4e08aff6e39a7da18258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:58:49.526191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22009474-85df-4692-a803-20dd4b29745b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:49.526965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22009474-85df-4692-a803-20dd4b29745b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:49.531975 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1359/2749] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:58:49 2022] DELETE /identity/v3/projects/617f326c60cc4e08aff6e39a7da18258 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:49.533142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83cab3-9467-468e-9cbd-3764439623a5 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:49.550191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83cab3-9467-468e-9cbd-3764439623a5 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:49.636885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:49.637523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:49.637901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:49.648997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] PATH_INFO: `/v3/users/6c461590591b4985b337c2931bcda169` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:49.648997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c461590591b4985b337c2931bcda169)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:49.648997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:49.737252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c461590591b4985b337c2931bcda169'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:49.747654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c461590591b4985b337c2931bcda169'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:49.757081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c461590591b4985b337c2931bcda169'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:49.773768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c461590591b4985b337c2931bcda169'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:49.774650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c461590591b4985b337c2931bcda169'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:49.782716 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1391/2750] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:58:49 2022] DELETE /identity/v3/users/6c461590591b4985b337c2931bcda169 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:49.920804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf6a32-60fa-462e-9ebf-f79be4d3d78a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:49.921411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf6a32-60fa-462e-9ebf-f79be4d3d78a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:49.921928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf6a32-60fa-462e-9ebf-f79be4d3d78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:49.943237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2acf6a32-60fa-462e-9ebf-f79be4d3d78a None None] MFA Rules not processed for user `7c276ebf6fe14a9c8f50bd1d1f44fb58`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:49.945203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2acf6a32-60fa-462e-9ebf-f79be4d3d78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:50.030304 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1360/2751] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:49 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:50.421201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:50.422957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:50.482395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:50.518947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a3149-8d68-4511-9302-c7240b068afe admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:50.523584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce6b12-865e-48a7-92af-8196d2ea3f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:50.524311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce6b12-865e-48a7-92af-8196d2ea3f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:50.524612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce6b12-865e-48a7-92af-8196d2ea3f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:50.524877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce6b12-865e-48a7-92af-8196d2ea3f4d admin admin] PATH_INFO: `/v3/projects/8e25ccab5db24e88bb50233b759e37ff` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:50.526016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce6b12-865e-48a7-92af-8196d2ea3f4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e25ccab5db24e88bb50233b759e37ff)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:50.527934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce6b12-865e-48a7-92af-8196d2ea3f4d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:50.575168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4dce6b12-865e-48a7-92af-8196d2ea3f4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e25ccab5db24e88bb50233b759e37ff', 'request_id': 'req-4dce6b12-865e-48a7-92af-8196d2ea3f4d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:50.585799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4dce6b12-865e-48a7-92af-8196d2ea3f4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e25ccab5db24e88bb50233b759e37ff', 'request_id': 'req-4dce6b12-865e-48a7-92af-8196d2ea3f4d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:50.593932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4dce6b12-865e-48a7-92af-8196d2ea3f4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e25ccab5db24e88bb50233b759e37ff', 'request_id': 'req-4dce6b12-865e-48a7-92af-8196d2ea3f4d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:50.608312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4dce6b12-865e-48a7-92af-8196d2ea3f4d admin admin] The token cache is being invalidate because project 8e25ccab5db24e88bb50233b759e37ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:58:50.608650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4dce6b12-865e-48a7-92af-8196d2ea3f4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:50.609413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4dce6b12-865e-48a7-92af-8196d2ea3f4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:50.611058 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1392/2752] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:58:50 2022] DELETE /identity/v3/projects/8e25ccab5db24e88bb50233b759e37ff => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:50.616962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b38182-d872-4894-b5f0-0109c3eda22a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:50.617466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b38182-d872-4894-b5f0-0109c3eda22a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:50.617466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b38182-d872-4894-b5f0-0109c3eda22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:50.639397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c8b38182-d872-4894-b5f0-0109c3eda22a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:50.640912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c8b38182-d872-4894-b5f0-0109c3eda22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:50.654393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:50.702791 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1361/2753] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:50.710679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2614e31-4b89-4124-9322-d1e0329ce482 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:50.711055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2614e31-4b89-4124-9322-d1e0329ce482 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:50.711380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2614e31-4b89-4124-9322-d1e0329ce482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:50.732007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a2614e31-4b89-4124-9322-d1e0329ce482 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:50.733707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a2614e31-4b89-4124-9322-d1e0329ce482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:50.741641 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1393/2754] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:50.746141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b38182-d872-4894-b5f0-0109c3eda22a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:50.758836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b38182-d872-4894-b5f0-0109c3eda22a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:50.764762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b70258-606d-4973-9739-b9f336773ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:50.765376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b70258-606d-4973-9739-b9f336773ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:50.765652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b70258-606d-4973-9739-b9f336773ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:50.765910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b70258-606d-4973-9739-b9f336773ba7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:50.766734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b70258-606d-4973-9739-b9f336773ba7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:58:50.767033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b70258-606d-4973-9739-b9f336773ba7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:50.768676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b70258-606d-4973-9739-b9f336773ba7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:50.784899 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1362/2755] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:50.790528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2614e31-4b89-4124-9322-d1e0329ce482 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:50.807856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2614e31-4b89-4124-9322-d1e0329ce482 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:50.812705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe15f9-d6bf-472d-a2d5-a77dd013a174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:50.813308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe15f9-d6bf-472d-a2d5-a77dd013a174 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:50.813571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe15f9-d6bf-472d-a2d5-a77dd013a174 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:50.813829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe15f9-d6bf-472d-a2d5-a77dd013a174 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:50.814575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe15f9-d6bf-472d-a2d5-a77dd013a174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:50.816685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe15f9-d6bf-472d-a2d5-a77dd013a174 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:50.854738 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1394/2756] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:58:50 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:50.855909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b70258-606d-4973-9739-b9f336773ba7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:50.867661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b70258-606d-4973-9739-b9f336773ba7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:50.979351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20549d2f-4075-4a4a-a2b1-eebd96e517d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:50.980326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20549d2f-4075-4a4a-a2b1-eebd96e517d2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:50.980668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20549d2f-4075-4a4a-a2b1-eebd96e517d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:50.980935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20549d2f-4075-4a4a-a2b1-eebd96e517d2 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:50.981679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20549d2f-4075-4a4a-a2b1-eebd96e517d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:50.983957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20549d2f-4075-4a4a-a2b1-eebd96e517d2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:51.020050 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1363/2757] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:58:50 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:51.021397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe15f9-d6bf-472d-a2d5-a77dd013a174 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:51.035182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe15f9-d6bf-472d-a2d5-a77dd013a174 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:51.050916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612a591-ee0c-478b-a3f6-d149980420ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:51.051758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612a591-ee0c-478b-a3f6-d149980420ed admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:51.052204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612a591-ee0c-478b-a3f6-d149980420ed admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:51.054068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612a591-ee0c-478b-a3f6-d149980420ed admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:51.054068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612a591-ee0c-478b-a3f6-d149980420ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:51.055622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612a591-ee0c-478b-a3f6-d149980420ed admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:51.108439 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1395/2758] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:51.111418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20549d2f-4075-4a4a-a2b1-eebd96e517d2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:51.133331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20549d2f-4075-4a4a-a2b1-eebd96e517d2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:51.147173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a658691-fc9b-417b-a332-ca3f71615783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:51.148042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a658691-fc9b-417b-a332-ca3f71615783 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:51.148520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a658691-fc9b-417b-a332-ca3f71615783 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:51.148935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a658691-fc9b-417b-a332-ca3f71615783 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:51.149747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a658691-fc9b-417b-a332-ca3f71615783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:51.151376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a658691-fc9b-417b-a332-ca3f71615783 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:51.182947 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1364/2759] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:51.187475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612a591-ee0c-478b-a3f6-d149980420ed admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:51.201708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612a591-ee0c-478b-a3f6-d149980420ed admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:51.210276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330007e1-7a07-47d1-8b4e-34360b77678a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:51.210982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330007e1-7a07-47d1-8b4e-34360b77678a admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:51.211319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330007e1-7a07-47d1-8b4e-34360b77678a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:51.211495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330007e1-7a07-47d1-8b4e-34360b77678a admin admin] PATH_INFO: `/v3/projects/046b7723ad834dba8b76eca7953e43e0/users/8c1247bbaa144442a16011bff76b73b5/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:51.235514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330007e1-7a07-47d1-8b4e-34360b77678a admin admin] RBAC: Authorizing `identity:create_grant(project_id=046b7723ad834dba8b76eca7953e43e0, user_id=8c1247bbaa144442a16011bff76b73b5, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:51.238553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330007e1-7a07-47d1-8b4e-34360b77678a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:51.264500 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1396/2760] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:58:51 2022] PUT /identity/v3/projects/046b7723ad834dba8b76eca7953e43e0/users/8c1247bbaa144442a16011bff76b73b5/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:51.276142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ebc2d-0ebc-48b2-a5aa-470e9686ffb2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:51.276565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ebc2d-0ebc-48b2-a5aa-470e9686ffb2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:51.282025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ebc2d-0ebc-48b2-a5aa-470e9686ffb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:51.355110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9f6ebc2d-0ebc-48b2-a5aa-470e9686ffb2 None None] MFA Rules not processed for user `8c1247bbaa144442a16011bff76b73b5`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:51.357466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9f6ebc2d-0ebc-48b2-a5aa-470e9686ffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:51.593156 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1365/2761] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:51.598262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330007e1-7a07-47d1-8b4e-34360b77678a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:51.685213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330007e1-7a07-47d1-8b4e-34360b77678a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:51.696200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb6d44-a796-47be-baf4-6c44d958bdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:51.708474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb6d44-a796-47be-baf4-6c44d958bdf9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:51.708474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb6d44-a796-47be-baf4-6c44d958bdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:51.708474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb6d44-a796-47be-baf4-6c44d958bdf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:51.708474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb6d44-a796-47be-baf4-6c44d958bdf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:51.709060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb6d44-a796-47be-baf4-6c44d958bdf9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:51.741441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:51.772870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ebc2d-0ebc-48b2-a5aa-470e9686ffb2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:51.779241 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1397/2762] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:58:51 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:51.785789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ebc2d-0ebc-48b2-a5aa-470e9686ffb2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:51.801256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:51.927788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a15fe-40c2-4764-a17b-1740f65e101f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:51.928497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a15fe-40c2-4764-a17b-1740f65e101f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:51.928860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a15fe-40c2-4764-a17b-1740f65e101f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:51.929160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a15fe-40c2-4764-a17b-1740f65e101f admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:51.929875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a15fe-40c2-4764-a17b-1740f65e101f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:51.931736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a15fe-40c2-4764-a17b-1740f65e101f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:51.993923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb6d44-a796-47be-baf4-6c44d958bdf9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:52.000386 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1366/2763] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:58:51 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:52.010557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb6d44-a796-47be-baf4-6c44d958bdf9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:52.025928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8a246-2af1-4469-898f-7ae933c6bb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:52.026562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8a246-2af1-4469-898f-7ae933c6bb1d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:52.026924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8a246-2af1-4469-898f-7ae933c6bb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:52.027230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8a246-2af1-4469-898f-7ae933c6bb1d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:52.027862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8a246-2af1-4469-898f-7ae933c6bb1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:52.029654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8a246-2af1-4469-898f-7ae933c6bb1d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:52.056989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a15fe-40c2-4764-a17b-1740f65e101f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:52.068391 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1398/2764] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:58:52.071360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a15fe-40c2-4764-a17b-1740f65e101f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:52.087938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525200a8-80f7-46aa-96b7-1b54e7019377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:52.088791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525200a8-80f7-46aa-96b7-1b54e7019377 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:52.089060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525200a8-80f7-46aa-96b7-1b54e7019377 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:52.089326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525200a8-80f7-46aa-96b7-1b54e7019377 admin admin] PATH_INFO: `/v3/projects/30f195a934da4dfbae503a7314123542/users/790a2ba7ac9d40798d1fef9fa33e0fe7/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:52.121048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525200a8-80f7-46aa-96b7-1b54e7019377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f195a934da4dfbae503a7314123542, user_id=790a2ba7ac9d40798d1fef9fa33e0fe7, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:52.124007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525200a8-80f7-46aa-96b7-1b54e7019377 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:52.153074 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1367/2765] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:58:52 2022] PUT /identity/v3/projects/30f195a934da4dfbae503a7314123542/users/790a2ba7ac9d40798d1fef9fa33e0fe7/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:52.157573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8a246-2af1-4469-898f-7ae933c6bb1d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:52.168035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8a246-2af1-4469-898f-7ae933c6bb1d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:52.244464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee713faf-5deb-4781-bd7b-a88cdea4d08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:52.245395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee713faf-5deb-4781-bd7b-a88cdea4d08c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:52.245395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee713faf-5deb-4781-bd7b-a88cdea4d08c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:52.245591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee713faf-5deb-4781-bd7b-a88cdea4d08c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:52.246692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee713faf-5deb-4781-bd7b-a88cdea4d08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:52.250527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee713faf-5deb-4781-bd7b-a88cdea4d08c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:52.266280 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1399/2766] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:58:52.270808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525200a8-80f7-46aa-96b7-1b54e7019377 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:52.289897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525200a8-80f7-46aa-96b7-1b54e7019377 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:52.320581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050262c-4e15-4171-8c04-5cd7cd535ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:52.321271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050262c-4e15-4171-8c04-5cd7cd535ad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:52.321543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050262c-4e15-4171-8c04-5cd7cd535ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:52.322252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050262c-4e15-4171-8c04-5cd7cd535ad9 admin admin] PATH_INFO: `/v3/projects/30f195a934da4dfbae503a7314123542/users/790a2ba7ac9d40798d1fef9fa33e0fe7/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:52.327515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050262c-4e15-4171-8c04-5cd7cd535ad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f195a934da4dfbae503a7314123542, user_id=790a2ba7ac9d40798d1fef9fa33e0fe7, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:52.330392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050262c-4e15-4171-8c04-5cd7cd535ad9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:52.344427 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1368/2767] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:58:52 2022] PUT /identity/v3/projects/30f195a934da4dfbae503a7314123542/users/790a2ba7ac9d40798d1fef9fa33e0fe7/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:52.355616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b45e1e-88a7-4ba3-b8f3-86b8a6e84169 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:52.355904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b45e1e-88a7-4ba3-b8f3-86b8a6e84169 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:52.357990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b45e1e-88a7-4ba3-b8f3-86b8a6e84169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:52.389538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-33b45e1e-88a7-4ba3-b8f3-86b8a6e84169 None None] MFA Rules not processed for user `790a2ba7ac9d40798d1fef9fa33e0fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:58:52.390909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-33b45e1e-88a7-4ba3-b8f3-86b8a6e84169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:52.475128 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1400/2768] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:58:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:58:52.483193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050262c-4e15-4171-8c04-5cd7cd535ad9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:52.510580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050262c-4e15-4171-8c04-5cd7cd535ad9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:52.516240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4c324-bb47-4a15-be67-16c76b6fc767 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:52.516917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4c324-bb47-4a15-be67-16c76b6fc767 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:52.517204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4c324-bb47-4a15-be67-16c76b6fc767 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:52.517965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4c324-bb47-4a15-be67-16c76b6fc767 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:52.518632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4c324-bb47-4a15-be67-16c76b6fc767 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:58:52.518936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4c324-bb47-4a15-be67-16c76b6fc767 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:52.520576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4c324-bb47-4a15-be67-16c76b6fc767 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:52.544716 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1369/2769] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:52.547097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b45e1e-88a7-4ba3-b8f3-86b8a6e84169 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:52.557334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b45e1e-88a7-4ba3-b8f3-86b8a6e84169 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:52.562169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac347d-8a04-4b7d-a931-af0df8c1f193 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:52.562787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac347d-8a04-4b7d-a931-af0df8c1f193 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:52.563054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac347d-8a04-4b7d-a931-af0df8c1f193 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:52.563316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac347d-8a04-4b7d-a931-af0df8c1f193 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:52.563934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac347d-8a04-4b7d-a931-af0df8c1f193 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:52.565842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac347d-8a04-4b7d-a931-af0df8c1f193 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:52.612085 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1401/2770] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:58:52 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:52.616671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4c324-bb47-4a15-be67-16c76b6fc767 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:52.628247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4c324-bb47-4a15-be67-16c76b6fc767 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:52.690869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fb835-a0a3-4f7f-8fbb-1a191be484f6 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:52.691629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fb835-a0a3-4f7f-8fbb-1a191be484f6 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:52.692130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fb835-a0a3-4f7f-8fbb-1a191be484f6 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:52.692475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fb835-a0a3-4f7f-8fbb-1a191be484f6 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:52.693174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fb835-a0a3-4f7f-8fbb-1a191be484f6 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:58:52.693551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fb835-a0a3-4f7f-8fbb-1a191be484f6 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:52.695808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fb835-a0a3-4f7f-8fbb-1a191be484f6 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:52.712161 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1370/2771] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:52.795101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac347d-8a04-4b7d-a931-af0df8c1f193 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:52.797284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-50ac347d-8a04-4b7d-a931-af0df8c1f193 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:52.881879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:52.932940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac347d-8a04-4b7d-a931-af0df8c1f193 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:52.945960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d738fc-1c95-4447-a507-ab7df370b82f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:52.946633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d738fc-1c95-4447-a507-ab7df370b82f service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:52.946975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d738fc-1c95-4447-a507-ab7df370b82f service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:52.947324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d738fc-1c95-4447-a507-ab7df370b82f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:52.973986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d738fc-1c95-4447-a507-ab7df370b82f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:52.979976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d738fc-1c95-4447-a507-ab7df370b82f service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:53.002592 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1402/2772] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:58:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:58:53.024950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784fa2a-3728-48ca-84f3-51380797d8aa None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:53.025570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784fa2a-3728-48ca-84f3-51380797d8aa None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:53.025570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784fa2a-3728-48ca-84f3-51380797d8aa None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:53.027465 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1371/2773] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:58:53.034066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d738fc-1c95-4447-a507-ab7df370b82f service nova] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:53.052317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d738fc-1c95-4447-a507-ab7df370b82f service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:53.078235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d738fc-1c95-4447-a507-ab7df370b82f service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:53.091296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a95af-8a86-4352-a0f3-ded89a39f51f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:53.092550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a95af-8a86-4352-a0f3-ded89a39f51f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:53.093116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a95af-8a86-4352-a0f3-ded89a39f51f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:53.093454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a95af-8a86-4352-a0f3-ded89a39f51f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects/f8b6230ff0a24ca59c0fa0f34e66e74d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:53.099186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a95af-8a86-4352-a0f3-ded89a39f51f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:get_project(project_id=f8b6230ff0a24ca59c0fa0f34e66e74d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:53.103353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a95af-8a86-4352-a0f3-ded89a39f51f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:53.108801 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1403/2774] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:53 2022] GET /identity/v3/projects/f8b6230ff0a24ca59c0fa0f34e66e74d => generated 371 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:53.655242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ad1a2-6f41-4246-a659-a99595ec0d16 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:53.655776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ad1a2-6f41-4246-a659-a99595ec0d16 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:53.657683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ad1a2-6f41-4246-a659-a99595ec0d16 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:53.660224 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1372/2775] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 20:58:53.665799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a95af-8a86-4352-a0f3-ded89a39f51f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:53.669664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-593a95af-8a86-4352-a0f3-ded89a39f51f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:53.716769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a95af-8a86-4352-a0f3-ded89a39f51f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:53.751053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a95af-8a86-4352-a0f3-ded89a39f51f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:53.770782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f266e7-1b2f-4f9b-88e9-56390d2995a5 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:53.771858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f266e7-1b2f-4f9b-88e9-56390d2995a5 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:53.772411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f266e7-1b2f-4f9b-88e9-56390d2995a5 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:53.772903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f266e7-1b2f-4f9b-88e9-56390d2995a5 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects/f8b6230ff0a24ca59c0fa0f34e66e74d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:53.774531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f266e7-1b2f-4f9b-88e9-56390d2995a5 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:get_project(project_id=f8b6230ff0a24ca59c0fa0f34e66e74d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:53.777008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f266e7-1b2f-4f9b-88e9-56390d2995a5 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:53.779588 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1404/2776] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:53 2022] GET /identity/v3/projects/f8b6230ff0a24ca59c0fa0f34e66e74d => generated 371 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:54.552801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ade2b-75fc-4482-8f7c-5ab7d3517bfb None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:54.553403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ade2b-75fc-4482-8f7c-5ab7d3517bfb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:54.553403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ade2b-75fc-4482-8f7c-5ab7d3517bfb None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:54.554725 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1373/2777] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:58:54.571920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f266e7-1b2f-4f9b-88e9-56390d2995a5 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:54.606133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f266e7-1b2f-4f9b-88e9-56390d2995a5 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:54.635914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f266e7-1b2f-4f9b-88e9-56390d2995a5 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:54.656313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99aa06-f0c7-4176-9d75-ddd46540bf3d tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:54.658299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99aa06-f0c7-4176-9d75-ddd46540bf3d tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:54.659795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99aa06-f0c7-4176-9d75-ddd46540bf3d tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:54.661501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99aa06-f0c7-4176-9d75-ddd46540bf3d tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects/f8b6230ff0a24ca59c0fa0f34e66e74d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:54.665516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99aa06-f0c7-4176-9d75-ddd46540bf3d tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:get_project(project_id=f8b6230ff0a24ca59c0fa0f34e66e74d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:54.674980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99aa06-f0c7-4176-9d75-ddd46540bf3d tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:54.679153 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1405/2778] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:54 2022] GET /identity/v3/projects/f8b6230ff0a24ca59c0fa0f34e66e74d => generated 371 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:54.713769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ade2b-75fc-4482-8f7c-5ab7d3517bfb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:54.739629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ade2b-75fc-4482-8f7c-5ab7d3517bfb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:54.762266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:54.766048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:54.772790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:54.773073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects/f8b6230ff0a24ca59c0fa0f34e66e74d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:54.777060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f8b6230ff0a24ca59c0fa0f34e66e74d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:54.779230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:55.029138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:55.057418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b6230ff0a24ca59c0fa0f34e66e74d', 'request_id': 'req-56db096d-02ac-48c7-8195-50d152f604b3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:55.115590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b6230ff0a24ca59c0fa0f34e66e74d', 'request_id': 'req-56db096d-02ac-48c7-8195-50d152f604b3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:55.151294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b6230ff0a24ca59c0fa0f34e66e74d', 'request_id': 'req-56db096d-02ac-48c7-8195-50d152f604b3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:55.232693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] The token cache is being invalidate because project f8b6230ff0a24ca59c0fa0f34e66e74d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:58:55.233333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:55.234351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:58:55.249868 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1374/2779] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:58:54 2022] DELETE /identity/v3/projects/f8b6230ff0a24ca59c0fa0f34e66e74d => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:58:55.943717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9548abf-f581-4f8f-a20e-b38742db5b52 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:55.944748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9548abf-f581-4f8f-a20e-b38742db5b52 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:55.944748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9548abf-f581-4f8f-a20e-b38742db5b52 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:55.946724 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1406/2780] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:58:55.953125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:55.954724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:56.168545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:56.181730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:56.345363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db096d-02ac-48c7-8195-50d152f604b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:56.357988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed954e98-1a9f-4905-b658-daef15bcc554 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:56.358870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed954e98-1a9f-4905-b658-daef15bcc554 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:56.359136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed954e98-1a9f-4905-b658-daef15bcc554 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:56.359468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed954e98-1a9f-4905-b658-daef15bcc554 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects/046b7723ad834dba8b76eca7953e43e0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:56.361350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed954e98-1a9f-4905-b658-daef15bcc554 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:get_project(project_id=046b7723ad834dba8b76eca7953e43e0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:56.363323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed954e98-1a9f-4905-b658-daef15bcc554 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:56.365635 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1375/2781] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:55 2022] GET /identity/v3/projects/046b7723ad834dba8b76eca7953e43e0 => generated 373 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:56.398262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9548abf-f581-4f8f-a20e-b38742db5b52 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:56.432220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9548abf-f581-4f8f-a20e-b38742db5b52 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:56.439946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9339df-689d-4560-bc56-cacc1122a717 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:56.440938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9339df-689d-4560-bc56-cacc1122a717 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:56.441376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9339df-689d-4560-bc56-cacc1122a717 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:56.441863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9339df-689d-4560-bc56-cacc1122a717 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:56.442760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9339df-689d-4560-bc56-cacc1122a717 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:58:56.443295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9339df-689d-4560-bc56-cacc1122a717 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:56.446335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9339df-689d-4560-bc56-cacc1122a717 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:56.467745 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1407/2782] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:56.468898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed954e98-1a9f-4905-b658-daef15bcc554 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:56.491587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed954e98-1a9f-4905-b658-daef15bcc554 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:56.519946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc3d47-fb91-40f9-977a-bc91b00de0b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:56.522191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc3d47-fb91-40f9-977a-bc91b00de0b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:56.522473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc3d47-fb91-40f9-977a-bc91b00de0b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:56.522736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc3d47-fb91-40f9-977a-bc91b00de0b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:56.525118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc3d47-fb91-40f9-977a-bc91b00de0b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:56.534758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc3d47-fb91-40f9-977a-bc91b00de0b3 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:56.604465 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1376/2783] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:58:56 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:58:56.606068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9339df-689d-4560-bc56-cacc1122a717 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:56.629676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9339df-689d-4560-bc56-cacc1122a717 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:56.763610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa760b5d-9740-49bb-97e7-3fd857aa1f03 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:56.764931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa760b5d-9740-49bb-97e7-3fd857aa1f03 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:56.765867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa760b5d-9740-49bb-97e7-3fd857aa1f03 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:56.766754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa760b5d-9740-49bb-97e7-3fd857aa1f03 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:56.768671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa760b5d-9740-49bb-97e7-3fd857aa1f03 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:58:56.769808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa760b5d-9740-49bb-97e7-3fd857aa1f03 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:56.774792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa760b5d-9740-49bb-97e7-3fd857aa1f03 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:56.802001 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1408/2784] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:57.380081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c40a73-b4d6-414a-b26f-c377a6f52c06 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:57.380617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c40a73-b4d6-414a-b26f-c377a6f52c06 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:57.380833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c40a73-b4d6-414a-b26f-c377a6f52c06 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:57.382814 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1377/2785] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:58:57.389064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa760b5d-9740-49bb-97e7-3fd857aa1f03 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:57.390906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fa760b5d-9740-49bb-97e7-3fd857aa1f03 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:58:57.487544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa760b5d-9740-49bb-97e7-3fd857aa1f03 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:57.501916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa760b5d-9740-49bb-97e7-3fd857aa1f03 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:57.517069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126e2ac-e4f2-4c58-b00d-6abbd8354b23 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:57.518051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126e2ac-e4f2-4c58-b00d-6abbd8354b23 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:57.518733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126e2ac-e4f2-4c58-b00d-6abbd8354b23 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:57.519282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126e2ac-e4f2-4c58-b00d-6abbd8354b23 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects/71b8fa7b693d4dae94ebfd6f2930b995` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:57.520832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126e2ac-e4f2-4c58-b00d-6abbd8354b23 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:get_project(project_id=71b8fa7b693d4dae94ebfd6f2930b995)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:57.523442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126e2ac-e4f2-4c58-b00d-6abbd8354b23 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:57.527899 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1409/2786] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:57 2022] GET /identity/v3/projects/71b8fa7b693d4dae94ebfd6f2930b995 => generated 369 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:58:58.418612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f15858-53f4-4a55-ba26-5f519b64fddd None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:58.419177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f15858-53f4-4a55-ba26-5f519b64fddd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:58.419366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f15858-53f4-4a55-ba26-5f519b64fddd None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:58.430792 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1378/2787] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:58:58.443554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126e2ac-e4f2-4c58-b00d-6abbd8354b23 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:58.466609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126e2ac-e4f2-4c58-b00d-6abbd8354b23 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:58.486566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126e2ac-e4f2-4c58-b00d-6abbd8354b23 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:58.508285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76338106-db82-493d-b4d4-639dacbcd20f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:58.511467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76338106-db82-493d-b4d4-639dacbcd20f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:58.512073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76338106-db82-493d-b4d4-639dacbcd20f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:58.512192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76338106-db82-493d-b4d4-639dacbcd20f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects/71b8fa7b693d4dae94ebfd6f2930b995` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:58.517884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76338106-db82-493d-b4d4-639dacbcd20f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:get_project(project_id=71b8fa7b693d4dae94ebfd6f2930b995)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:58.520030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76338106-db82-493d-b4d4-639dacbcd20f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:58.526041 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1410/2788] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:58 2022] GET /identity/v3/projects/71b8fa7b693d4dae94ebfd6f2930b995 => generated 369 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:58.590662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f15858-53f4-4a55-ba26-5f519b64fddd None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:58.622923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f15858-53f4-4a55-ba26-5f519b64fddd None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:58.638138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011eb6c7-aa6a-4907-806b-78df92b83026 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:58.652500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011eb6c7-aa6a-4907-806b-78df92b83026 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:58.656899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011eb6c7-aa6a-4907-806b-78df92b83026 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:58.657171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011eb6c7-aa6a-4907-806b-78df92b83026 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:58.657947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011eb6c7-aa6a-4907-806b-78df92b83026 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:58:58.658255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011eb6c7-aa6a-4907-806b-78df92b83026 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:58.659886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011eb6c7-aa6a-4907-806b-78df92b83026 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:58.722403 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1379/2789] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:58.723252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76338106-db82-493d-b4d4-639dacbcd20f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:58.729861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:58:58.737959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76338106-db82-493d-b4d4-639dacbcd20f tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:58.758086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f4401-1609-4ec1-a8c1-d897a1119754 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:58.760700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f4401-1609-4ec1-a8c1-d897a1119754 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:58.761238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f4401-1609-4ec1-a8c1-d897a1119754 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:58.761756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f4401-1609-4ec1-a8c1-d897a1119754 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:58.765223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f4401-1609-4ec1-a8c1-d897a1119754 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:58.785345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f4401-1609-4ec1-a8c1-d897a1119754 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:58.885946 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1411/2790] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:58:58 2022] POST /identity/v3/users => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:58:58.890408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011eb6c7-aa6a-4907-806b-78df92b83026 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:58.915742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011eb6c7-aa6a-4907-806b-78df92b83026 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:58.936786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa6471-2856-4c34-9d82-d2609cb932cd tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:58.940273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa6471-2856-4c34-9d82-d2609cb932cd tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:58.940955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa6471-2856-4c34-9d82-d2609cb932cd tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:58.941381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa6471-2856-4c34-9d82-d2609cb932cd tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:58.942213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa6471-2856-4c34-9d82-d2609cb932cd tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:58:58.945523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa6471-2856-4c34-9d82-d2609cb932cd tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:58.950114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa6471-2856-4c34-9d82-d2609cb932cd tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:58.996969 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1380/2791] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:59.239200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bffe9-bf99-46e7-ac79-edd1309e76c8 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:59.240062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bffe9-bf99-46e7-ac79-edd1309e76c8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:59.241383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bffe9-bf99-46e7-ac79-edd1309e76c8 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:59.243182 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1412/2792] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:58:59.255456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa6471-2856-4c34-9d82-d2609cb932cd tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:59.295219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa6471-2856-4c34-9d82-d2609cb932cd tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:59.307157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa6471-2856-4c34-9d82-d2609cb932cd tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:59.317005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371410d-eda6-4b31-b249-a8d1ca5af268 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:59.317005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371410d-eda6-4b31-b249-a8d1ca5af268 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:59.317518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371410d-eda6-4b31-b249-a8d1ca5af268 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:59.317518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371410d-eda6-4b31-b249-a8d1ca5af268 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects/71b8fa7b693d4dae94ebfd6f2930b995` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:59.318754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371410d-eda6-4b31-b249-a8d1ca5af268 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:get_project(project_id=71b8fa7b693d4dae94ebfd6f2930b995)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:59.322579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371410d-eda6-4b31-b249-a8d1ca5af268 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:59.325236 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1381/2793] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:59 2022] GET /identity/v3/projects/71b8fa7b693d4dae94ebfd6f2930b995 => generated 369 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:58:59.839471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9a606-559a-461a-9b4d-d8dc0fabd841 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:59.840024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9a606-559a-461a-9b4d-d8dc0fabd841 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:59.840135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9a606-559a-461a-9b4d-d8dc0fabd841 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:59.841544 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1413/2794] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:58:59.846387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371410d-eda6-4b31-b249-a8d1ca5af268 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:58:59.859906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371410d-eda6-4b31-b249-a8d1ca5af268 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:59.868977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371410d-eda6-4b31-b249-a8d1ca5af268 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:59.874104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b1292-7853-40ca-9f34-e733f0d34ceb tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:59.874728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b1292-7853-40ca-9f34-e733f0d34ceb tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:59.874999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b1292-7853-40ca-9f34-e733f0d34ceb tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:59.875261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b1292-7853-40ca-9f34-e733f0d34ceb tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects/71b8fa7b693d4dae94ebfd6f2930b995` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:59.876579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b1292-7853-40ca-9f34-e733f0d34ceb tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:get_project(project_id=71b8fa7b693d4dae94ebfd6f2930b995)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:59.878655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b1292-7853-40ca-9f34-e733f0d34ceb tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:58:59.880600 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1382/2795] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:58:59 2022] GET /identity/v3/projects/71b8fa7b693d4dae94ebfd6f2930b995 => generated 369 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:58:59.915724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9a606-559a-461a-9b4d-d8dc0fabd841 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:58:59.930761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9a606-559a-461a-9b4d-d8dc0fabd841 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:58:59.936621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542faaf1-25d4-4d91-b2e2-79e94d43de57 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:58:59.937239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542faaf1-25d4-4d91-b2e2-79e94d43de57 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:58:59.937743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542faaf1-25d4-4d91-b2e2-79e94d43de57 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:58:59.938009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542faaf1-25d4-4d91-b2e2-79e94d43de57 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/users/e5d93c146e114c2592dfc072c22037d0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:58:59.974425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542faaf1-25d4-4d91-b2e2-79e94d43de57 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:delete_user(user_id=e5d93c146e114c2592dfc072c22037d0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:58:59.977118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542faaf1-25d4-4d91-b2e2-79e94d43de57 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:00.044308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-542faaf1-25d4-4d91-b2e2-79e94d43de57 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5d93c146e114c2592dfc072c22037d0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:00.057508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-542faaf1-25d4-4d91-b2e2-79e94d43de57 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5d93c146e114c2592dfc072c22037d0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:00.068199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-542faaf1-25d4-4d91-b2e2-79e94d43de57 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5d93c146e114c2592dfc072c22037d0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:00.094749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-542faaf1-25d4-4d91-b2e2-79e94d43de57 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5d93c146e114c2592dfc072c22037d0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:00.095677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-542faaf1-25d4-4d91-b2e2-79e94d43de57 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5d93c146e114c2592dfc072c22037d0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:00.103223 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1414/2796] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:58:59 2022] DELETE /identity/v3/users/e5d93c146e114c2592dfc072c22037d0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:00.108182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b1292-7853-40ca-9f34-e733f0d34ceb tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:00.110557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0b4b1292-7853-40ca-9f34-e733f0d34ceb tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:00.119678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:00.126876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b1292-7853-40ca-9f34-e733f0d34ceb tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:00.235564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:00.236254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:00.236549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:00.237062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects/71b8fa7b693d4dae94ebfd6f2930b995` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:00.239454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:delete_project(project_id=71b8fa7b693d4dae94ebfd6f2930b995)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:00.243256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:00.311139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b8fa7b693d4dae94ebfd6f2930b995', 'request_id': 'req-b659528a-aa01-4d8f-bb89-195d6584ac9b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:00.318312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b8fa7b693d4dae94ebfd6f2930b995', 'request_id': 'req-b659528a-aa01-4d8f-bb89-195d6584ac9b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:00.327027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b8fa7b693d4dae94ebfd6f2930b995', 'request_id': 'req-b659528a-aa01-4d8f-bb89-195d6584ac9b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:00.338810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] The token cache is being invalidate because project 71b8fa7b693d4dae94ebfd6f2930b995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:59:00.339108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:00.339775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:00.341833 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1383/2797] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:59:00 2022] DELETE /identity/v3/projects/71b8fa7b693d4dae94ebfd6f2930b995 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:00.560804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef40d0-d01f-4ad0-9d78-0e901ee64016 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:00.561068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef40d0-d01f-4ad0-9d78-0e901ee64016 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:00.561365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef40d0-d01f-4ad0-9d78-0e901ee64016 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:00.562737 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1415/2798] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:59:00.567682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:59:00.569629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:00.647685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:00.648953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:00.677293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:00.715489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659528a-aa01-4d8f-bb89-195d6584ac9b tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:00.721639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458256f-4be1-466a-8af2-7f49d328f4f9 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:00.722253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458256f-4be1-466a-8af2-7f49d328f4f9 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:00.722735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458256f-4be1-466a-8af2-7f49d328f4f9 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:00.723222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458256f-4be1-466a-8af2-7f49d328f4f9 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects/046b7723ad834dba8b76eca7953e43e0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:00.724973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458256f-4be1-466a-8af2-7f49d328f4f9 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:get_project(project_id=046b7723ad834dba8b76eca7953e43e0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:00.727171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458256f-4be1-466a-8af2-7f49d328f4f9 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:00.729063 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1384/2799] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:59:00 2022] GET /identity/v3/projects/046b7723ad834dba8b76eca7953e43e0 => generated 373 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:59:00.761678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2313601-fd73-49b4-aa68-9dd2871b9ad0 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:00.762034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2313601-fd73-49b4-aa68-9dd2871b9ad0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:00.762398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2313601-fd73-49b4-aa68-9dd2871b9ad0 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:00.764365 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1416/2800] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:59:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:59:00.769196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458256f-4be1-466a-8af2-7f49d328f4f9 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:59:00.782778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458256f-4be1-466a-8af2-7f49d328f4f9 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:00.791955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458256f-4be1-466a-8af2-7f49d328f4f9 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:00.796906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b35ccc-91a5-4fff-b6da-7e489f6bfe74 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:00.797525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b35ccc-91a5-4fff-b6da-7e489f6bfe74 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:00.797912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b35ccc-91a5-4fff-b6da-7e489f6bfe74 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:00.798185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b35ccc-91a5-4fff-b6da-7e489f6bfe74 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects/046b7723ad834dba8b76eca7953e43e0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:00.799309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b35ccc-91a5-4fff-b6da-7e489f6bfe74 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:get_project(project_id=046b7723ad834dba8b76eca7953e43e0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:00.801382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b35ccc-91a5-4fff-b6da-7e489f6bfe74 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:00.803021 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1385/2801] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:59:00 2022] GET /identity/v3/projects/046b7723ad834dba8b76eca7953e43e0 => generated 373 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:59:01.157719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9519d31-435a-48a1-a53f-9f22f862a2c2 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:01.158343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9519d31-435a-48a1-a53f-9f22f862a2c2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:01.158669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9519d31-435a-48a1-a53f-9f22f862a2c2 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:01.160444 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1417/2802] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 20:59:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 20:59:01.165722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b35ccc-91a5-4fff-b6da-7e489f6bfe74 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 20:59:01.184612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b35ccc-91a5-4fff-b6da-7e489f6bfe74 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:01.198754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b35ccc-91a5-4fff-b6da-7e489f6bfe74 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:01.204745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df78e8-6c79-46f2-bfd1-63f6189785e0 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '790a2ba7ac9d40798d1fef9fa33e0fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30f195a934da4dfbae503a7314123542', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:01.205657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df78e8-6c79-46f2-bfd1-63f6189785e0 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:01.206206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df78e8-6c79-46f2-bfd1-63f6189785e0 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:01.206569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df78e8-6c79-46f2-bfd1-63f6189785e0 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] PATH_INFO: `/v3/projects/046b7723ad834dba8b76eca7953e43e0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:01.207915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df78e8-6c79-46f2-bfd1-63f6189785e0 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorizing `identity:get_project(project_id=046b7723ad834dba8b76eca7953e43e0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:01.210101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df78e8-6c79-46f2-bfd1-63f6189785e0 tempest-QuotasAdminTestJSON-682706813 tempest-QuotasAdminTestJSON-682706813-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:01.211923 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1386/2803] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 20:59:01 2022] GET /identity/v3/projects/046b7723ad834dba8b76eca7953e43e0 => generated 373 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:59:01.549389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9519d31-435a-48a1-a53f-9f22f862a2c2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:01.552085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c9519d31-435a-48a1-a53f-9f22f862a2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:01.636648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9519d31-435a-48a1-a53f-9f22f862a2c2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:01.642418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:01.643095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:01.643410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:01.643623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] PATH_INFO: `/v3/users/8c1247bbaa144442a16011bff76b73b5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:01.645105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c1247bbaa144442a16011bff76b73b5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:01.647130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:01.699532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c1247bbaa144442a16011bff76b73b5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:01.702047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=79,keystone:INSERT=4,keystone:DELETE=17 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:01.712362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c1247bbaa144442a16011bff76b73b5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:01.720728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c1247bbaa144442a16011bff76b73b5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:01.733236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c1247bbaa144442a16011bff76b73b5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:01.733930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c1247bbaa144442a16011bff76b73b5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:01.740008 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1418/2804] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:59:01 2022] DELETE /identity/v3/users/8c1247bbaa144442a16011bff76b73b5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:01.744329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389eb221-2183-40f5-b148-0949a5d6f558 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:01.744726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389eb221-2183-40f5-b148-0949a5d6f558 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:01.745083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389eb221-2183-40f5-b148-0949a5d6f558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:01.763983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-389eb221-2183-40f5-b148-0949a5d6f558 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:01.765316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-389eb221-2183-40f5-b148-0949a5d6f558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:01.829047 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1387/2805] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:01.839243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:01.840895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:01.931711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674422e8-9f6c-4d32-a266-333851a01d0e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:01.966920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e52f1-b62f-4654-9ea9-86cdf598a32a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:01.969253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e52f1-b62f-4654-9ea9-86cdf598a32a service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:01.970466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e52f1-b62f-4654-9ea9-86cdf598a32a service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:01.970906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e52f1-b62f-4654-9ea9-86cdf598a32a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:01.989519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e52f1-b62f-4654-9ea9-86cdf598a32a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:01.990979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e52f1-b62f-4654-9ea9-86cdf598a32a service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:02.003963 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1419/2806] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 20:59:02.584940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389eb221-2183-40f5-b148-0949a5d6f558 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:02.586231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-389eb221-2183-40f5-b148-0949a5d6f558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:02.599990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389eb221-2183-40f5-b148-0949a5d6f558 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:02.605679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c9125-6cd8-43bd-b55a-584dc0e131b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:02.606698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c9125-6cd8-43bd-b55a-584dc0e131b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:02.606975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c9125-6cd8-43bd-b55a-584dc0e131b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:02.607231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c9125-6cd8-43bd-b55a-584dc0e131b1 admin admin] PATH_INFO: `/v3/projects/046b7723ad834dba8b76eca7953e43e0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:02.609103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c9125-6cd8-43bd-b55a-584dc0e131b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=046b7723ad834dba8b76eca7953e43e0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:02.620717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c9125-6cd8-43bd-b55a-584dc0e131b1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:02.719984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-414c9125-6cd8-43bd-b55a-584dc0e131b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '046b7723ad834dba8b76eca7953e43e0', 'request_id': 'req-414c9125-6cd8-43bd-b55a-584dc0e131b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:02.733324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-414c9125-6cd8-43bd-b55a-584dc0e131b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '046b7723ad834dba8b76eca7953e43e0', 'request_id': 'req-414c9125-6cd8-43bd-b55a-584dc0e131b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:02.745007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-414c9125-6cd8-43bd-b55a-584dc0e131b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '046b7723ad834dba8b76eca7953e43e0', 'request_id': 'req-414c9125-6cd8-43bd-b55a-584dc0e131b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:02.769581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-414c9125-6cd8-43bd-b55a-584dc0e131b1 admin admin] The token cache is being invalidate because project 046b7723ad834dba8b76eca7953e43e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:59:02.770345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-414c9125-6cd8-43bd-b55a-584dc0e131b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:02.771999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-414c9125-6cd8-43bd-b55a-584dc0e131b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:02.778487 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1388/2807] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:59:02 2022] DELETE /identity/v3/projects/046b7723ad834dba8b76eca7953e43e0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:02.780761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e52f1-b62f-4654-9ea9-86cdf598a32a service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:02.794388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e52f1-b62f-4654-9ea9-86cdf598a32a service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:02.902418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:02.903043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:02.903665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:02.904063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] PATH_INFO: `/v3/users/790a2ba7ac9d40798d1fef9fa33e0fe7` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:02.905668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=790a2ba7ac9d40798d1fef9fa33e0fe7)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:02.910813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:03.010590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '790a2ba7ac9d40798d1fef9fa33e0fe7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:03.020882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '790a2ba7ac9d40798d1fef9fa33e0fe7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:03.033097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '790a2ba7ac9d40798d1fef9fa33e0fe7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:03.040984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:03.052888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '790a2ba7ac9d40798d1fef9fa33e0fe7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:03.067580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '790a2ba7ac9d40798d1fef9fa33e0fe7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:03.082237 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1420/2808] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:59:02 2022] DELETE /identity/v3/users/790a2ba7ac9d40798d1fef9fa33e0fe7 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:03.193299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f31f1-950f-4390-a6cf-f02e98643a1f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:03.193612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f31f1-950f-4390-a6cf-f02e98643a1f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:03.194461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f31f1-950f-4390-a6cf-f02e98643a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:03.219416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3d4f31f1-950f-4390-a6cf-f02e98643a1f None None] MFA Rules not processed for user `22489a6c355a49028ca16a394b27876f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:03.220685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3d4f31f1-950f-4390-a6cf-f02e98643a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:03.298824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:03.357372 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1389/2809] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:03 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:04.011373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:04.013712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:04.102576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7259ef-b7e2-4d89-8278-9cb04e0cc400 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:04.107841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090a69b-5cf8-4960-b2d7-11347d408e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:04.108520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090a69b-5cf8-4960-b2d7-11347d408e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:04.108869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090a69b-5cf8-4960-b2d7-11347d408e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:04.109470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090a69b-5cf8-4960-b2d7-11347d408e51 admin admin] PATH_INFO: `/v3/projects/30f195a934da4dfbae503a7314123542` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:04.110264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090a69b-5cf8-4960-b2d7-11347d408e51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f195a934da4dfbae503a7314123542)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:04.112171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090a69b-5cf8-4960-b2d7-11347d408e51 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:04.160645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6090a69b-5cf8-4960-b2d7-11347d408e51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f195a934da4dfbae503a7314123542', 'request_id': 'req-6090a69b-5cf8-4960-b2d7-11347d408e51'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:04.171883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6090a69b-5cf8-4960-b2d7-11347d408e51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f195a934da4dfbae503a7314123542', 'request_id': 'req-6090a69b-5cf8-4960-b2d7-11347d408e51'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:04.178275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6090a69b-5cf8-4960-b2d7-11347d408e51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f195a934da4dfbae503a7314123542', 'request_id': 'req-6090a69b-5cf8-4960-b2d7-11347d408e51'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:04.192542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6090a69b-5cf8-4960-b2d7-11347d408e51 admin admin] The token cache is being invalidate because project 30f195a934da4dfbae503a7314123542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:59:04.193122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6090a69b-5cf8-4960-b2d7-11347d408e51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:04.193971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6090a69b-5cf8-4960-b2d7-11347d408e51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:04.196752 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1421/2810] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:59:04 2022] DELETE /identity/v3/projects/30f195a934da4dfbae503a7314123542 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:04.202724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205221d-30e5-4176-ab6c-546a03a99215 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:04.203285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205221d-30e5-4176-ab6c-546a03a99215 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:04.204211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205221d-30e5-4176-ab6c-546a03a99215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:04.233842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6205221d-30e5-4176-ab6c-546a03a99215 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:04.235453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6205221d-30e5-4176-ab6c-546a03a99215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:04.313016 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1390/2811] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:04.314130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6227d91-287e-4836-ac94-c9bddd2549fe None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:04.314433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6227d91-287e-4836-ac94-c9bddd2549fe None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:04.314701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6227d91-287e-4836-ac94-c9bddd2549fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:04.338671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b6227d91-287e-4836-ac94-c9bddd2549fe None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:04.340099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b6227d91-287e-4836-ac94-c9bddd2549fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:04.348992 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1422/2812] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:59:04.353438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205221d-30e5-4176-ab6c-546a03a99215 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:04.364360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205221d-30e5-4176-ab6c-546a03a99215 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:04.369504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96eefb-a309-4310-8fd3-95ecfeb0b625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:04.370105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96eefb-a309-4310-8fd3-95ecfeb0b625 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:04.370375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96eefb-a309-4310-8fd3-95ecfeb0b625 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:04.370634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96eefb-a309-4310-8fd3-95ecfeb0b625 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:04.371277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96eefb-a309-4310-8fd3-95ecfeb0b625 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:59:04.371604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96eefb-a309-4310-8fd3-95ecfeb0b625 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:04.373661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96eefb-a309-4310-8fd3-95ecfeb0b625 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:04.387613 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1391/2813] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:59:04.395090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6227d91-287e-4836-ac94-c9bddd2549fe None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:04.410811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6227d91-287e-4836-ac94-c9bddd2549fe None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:04.416651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fff83-8549-4620-83e3-85a593742543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:04.417717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fff83-8549-4620-83e3-85a593742543 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:04.418156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fff83-8549-4620-83e3-85a593742543 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:04.419039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fff83-8549-4620-83e3-85a593742543 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:04.419252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fff83-8549-4620-83e3-85a593742543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:04.421217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fff83-8549-4620-83e3-85a593742543 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:04.476447 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1423/2814] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:59:04 2022] POST /identity/v3/projects => generated 375 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:04.477318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96eefb-a309-4310-8fd3-95ecfeb0b625 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:04.495222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96eefb-a309-4310-8fd3-95ecfeb0b625 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:04.581929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c196c-7749-4e01-9000-821c2f035875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:04.583170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c196c-7749-4e01-9000-821c2f035875 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:04.583506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c196c-7749-4e01-9000-821c2f035875 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:04.583712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c196c-7749-4e01-9000-821c2f035875 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:04.584342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c196c-7749-4e01-9000-821c2f035875 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:04.589196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c196c-7749-4e01-9000-821c2f035875 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:04.615134 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1392/2815] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:59:04 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:04.618954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fff83-8549-4620-83e3-85a593742543 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:04.630838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fff83-8549-4620-83e3-85a593742543 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:04.635911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae870aa-b1c7-4297-be09-cfc129527351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:04.636660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae870aa-b1c7-4297-be09-cfc129527351 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:04.636957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae870aa-b1c7-4297-be09-cfc129527351 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:04.637344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae870aa-b1c7-4297-be09-cfc129527351 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:04.638049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae870aa-b1c7-4297-be09-cfc129527351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:04.639750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae870aa-b1c7-4297-be09-cfc129527351 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:04.651505 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1424/2816] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:04.655844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c196c-7749-4e01-9000-821c2f035875 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:04.681456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c196c-7749-4e01-9000-821c2f035875 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:04.690766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f18405-50e8-4a85-acd4-403cf295b97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:04.691392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f18405-50e8-4a85-acd4-403cf295b97c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:04.691757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f18405-50e8-4a85-acd4-403cf295b97c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:04.691986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f18405-50e8-4a85-acd4-403cf295b97c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:04.692594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f18405-50e8-4a85-acd4-403cf295b97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:04.695012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f18405-50e8-4a85-acd4-403cf295b97c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:04.716421 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1393/2817] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:59:04.717845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae870aa-b1c7-4297-be09-cfc129527351 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:04.733227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae870aa-b1c7-4297-be09-cfc129527351 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:04.739819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb31235-660c-4354-90dc-0a4683ea1f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:04.740742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb31235-660c-4354-90dc-0a4683ea1f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:04.741243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb31235-660c-4354-90dc-0a4683ea1f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:04.741732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb31235-660c-4354-90dc-0a4683ea1f34 admin admin] PATH_INFO: `/v3/projects/7f66f844c2ea41dd8912a4a22d7c9730/users/2517214c35344f3b9b60ec480909b79c/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:04.919109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb31235-660c-4354-90dc-0a4683ea1f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f66f844c2ea41dd8912a4a22d7c9730, user_id=2517214c35344f3b9b60ec480909b79c, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:04.922495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb31235-660c-4354-90dc-0a4683ea1f34 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:04.939875 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1425/2818] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:59:04 2022] PUT /identity/v3/projects/7f66f844c2ea41dd8912a4a22d7c9730/users/2517214c35344f3b9b60ec480909b79c/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:04.953250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf7ff9-cac9-44f2-aed3-d5630a1ee876 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:04.953560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf7ff9-cac9-44f2-aed3-d5630a1ee876 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:04.953825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf7ff9-cac9-44f2-aed3-d5630a1ee876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:04.987015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-40bf7ff9-cac9-44f2-aed3-d5630a1ee876 None None] MFA Rules not processed for user `2517214c35344f3b9b60ec480909b79c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:04.988224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-40bf7ff9-cac9-44f2-aed3-d5630a1ee876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:05.036648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:05.071664 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1394/2819] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:59:05.076324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb31235-660c-4354-90dc-0a4683ea1f34 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:05.089501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:05.141531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb31235-660c-4354-90dc-0a4683ea1f34 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:05.146719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7756cb2-1c9c-4bf7-aa05-acbe4e4da555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:05.147331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7756cb2-1c9c-4bf7-aa05-acbe4e4da555 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:05.147605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7756cb2-1c9c-4bf7-aa05-acbe4e4da555 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:05.147874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7756cb2-1c9c-4bf7-aa05-acbe4e4da555 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:05.148550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7756cb2-1c9c-4bf7-aa05-acbe4e4da555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:05.150446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7756cb2-1c9c-4bf7-aa05-acbe4e4da555 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:05.178929 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1426/2820] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:59:05 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:05.183258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf7ff9-cac9-44f2-aed3-d5630a1ee876 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:05.194696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf7ff9-cac9-44f2-aed3-d5630a1ee876 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:05.258756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8316524-dedc-40db-82f1-324ca0829565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:05.259477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8316524-dedc-40db-82f1-324ca0829565 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:05.259726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8316524-dedc-40db-82f1-324ca0829565 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:05.260172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8316524-dedc-40db-82f1-324ca0829565 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:05.261314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8316524-dedc-40db-82f1-324ca0829565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:05.265107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8316524-dedc-40db-82f1-324ca0829565 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:05.290328 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1395/2821] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:59:05 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:05.295472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7756cb2-1c9c-4bf7-aa05-acbe4e4da555 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:05.306505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7756cb2-1c9c-4bf7-aa05-acbe4e4da555 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:05.311259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe96341-d93d-4ad6-8c9f-3e051707c473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:05.312802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe96341-d93d-4ad6-8c9f-3e051707c473 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:05.313407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe96341-d93d-4ad6-8c9f-3e051707c473 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:05.313875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe96341-d93d-4ad6-8c9f-3e051707c473 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:05.314880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe96341-d93d-4ad6-8c9f-3e051707c473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:05.316998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe96341-d93d-4ad6-8c9f-3e051707c473 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:05.339070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8316524-dedc-40db-82f1-324ca0829565 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:05.344608 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1427/2822] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:05.353819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8316524-dedc-40db-82f1-324ca0829565 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:05.360059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69805e5-31f3-465d-9c6d-18bddd8776c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:05.361033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69805e5-31f3-465d-9c6d-18bddd8776c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:05.361306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69805e5-31f3-465d-9c6d-18bddd8776c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:05.361573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69805e5-31f3-465d-9c6d-18bddd8776c4 admin admin] PATH_INFO: `/v3/projects/f84b00c0ede244e092e45bc6659dc119/users/99597ed0ca7d4f5fadb126ac7d800968/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:05.389907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69805e5-31f3-465d-9c6d-18bddd8776c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84b00c0ede244e092e45bc6659dc119, user_id=99597ed0ca7d4f5fadb126ac7d800968, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:05.393142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69805e5-31f3-465d-9c6d-18bddd8776c4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:05.411222 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1396/2823] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:59:05 2022] PUT /identity/v3/projects/f84b00c0ede244e092e45bc6659dc119/users/99597ed0ca7d4f5fadb126ac7d800968/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:05.415625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe96341-d93d-4ad6-8c9f-3e051707c473 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:05.441764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe96341-d93d-4ad6-8c9f-3e051707c473 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:05.533545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa146b-3147-4d83-a3cc-65a101cd1aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:05.534188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa146b-3147-4d83-a3cc-65a101cd1aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:05.534460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa146b-3147-4d83-a3cc-65a101cd1aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:05.534723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa146b-3147-4d83-a3cc-65a101cd1aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:05.535508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa146b-3147-4d83-a3cc-65a101cd1aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:05.537286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa146b-3147-4d83-a3cc-65a101cd1aa8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:05.549429 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1428/2824] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:05.553502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69805e5-31f3-465d-9c6d-18bddd8776c4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:05.567264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69805e5-31f3-465d-9c6d-18bddd8776c4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:05.574334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c828b9-8bd4-4ef5-a8a2-5ebbbc9e5ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:05.574334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c828b9-8bd4-4ef5-a8a2-5ebbbc9e5ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:05.574334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c828b9-8bd4-4ef5-a8a2-5ebbbc9e5ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:05.574760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c828b9-8bd4-4ef5-a8a2-5ebbbc9e5ea2 admin admin] PATH_INFO: `/v3/projects/f84b00c0ede244e092e45bc6659dc119/users/99597ed0ca7d4f5fadb126ac7d800968/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:05.576216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c828b9-8bd4-4ef5-a8a2-5ebbbc9e5ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84b00c0ede244e092e45bc6659dc119, user_id=99597ed0ca7d4f5fadb126ac7d800968, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:05.578945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c828b9-8bd4-4ef5-a8a2-5ebbbc9e5ea2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:05.599637 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1397/2825] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:59:05 2022] PUT /identity/v3/projects/f84b00c0ede244e092e45bc6659dc119/users/99597ed0ca7d4f5fadb126ac7d800968/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:59:05.610081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42599b-6f97-4a37-baed-c362a7dd0ac9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:05.610387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42599b-6f97-4a37-baed-c362a7dd0ac9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:05.610675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42599b-6f97-4a37-baed-c362a7dd0ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:05.642633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4f42599b-6f97-4a37-baed-c362a7dd0ac9 None None] MFA Rules not processed for user `99597ed0ca7d4f5fadb126ac7d800968`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:05.644080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4f42599b-6f97-4a37-baed-c362a7dd0ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:05.722552 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1429/2826] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:05.735468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874f70-9ff0-4173-ac81-967197dd70ff None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:05.735834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874f70-9ff0-4173-ac81-967197dd70ff None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:05.736196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874f70-9ff0-4173-ac81-967197dd70ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:05.784580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a8874f70-9ff0-4173-ac81-967197dd70ff None None] MFA Rules not processed for user `2517214c35344f3b9b60ec480909b79c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:05.785811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a8874f70-9ff0-4173-ac81-967197dd70ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:05.894606 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1398/2827] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:05.905800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42599b-6f97-4a37-baed-c362a7dd0ac9 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:05.907415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4f42599b-6f97-4a37-baed-c362a7dd0ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:05.923439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:06.035669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42599b-6f97-4a37-baed-c362a7dd0ac9 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:06.044552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff435ee-704c-4046-b31f-32a6751f0f31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:06.045459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff435ee-704c-4046-b31f-32a6751f0f31 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:06.046423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff435ee-704c-4046-b31f-32a6751f0f31 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:06.046705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff435ee-704c-4046-b31f-32a6751f0f31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:06.057718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff435ee-704c-4046-b31f-32a6751f0f31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:06.059868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff435ee-704c-4046-b31f-32a6751f0f31 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:06.074690 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1430/2828] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:14.116205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8874f70-9ff0-4173-ac81-967197dd70ff None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:14.118240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a8874f70-9ff0-4173-ac81-967197dd70ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:14.213615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:14.291857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8874f70-9ff0-4173-ac81-967197dd70ff None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:14.298043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:14.298820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:14.299188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:14.299544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] PATH_INFO: `/v3/users/8cc1474bc2a1450eac2d15554d5fcd11` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:14.300994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cc1474bc2a1450eac2d15554d5fcd11)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:14.303609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:14.382908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cc1474bc2a1450eac2d15554d5fcd11'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:14.393993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cc1474bc2a1450eac2d15554d5fcd11'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:14.412779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cc1474bc2a1450eac2d15554d5fcd11'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:14.413613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cc1474bc2a1450eac2d15554d5fcd11'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:14.414479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cc1474bc2a1450eac2d15554d5fcd11'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:14.430760 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1399/2829] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:59:14 2022] DELETE /identity/v3/users/8cc1474bc2a1450eac2d15554d5fcd11 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:14.435304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fde789-5d2e-4439-ab54-27c6ce461822 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:14.435841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fde789-5d2e-4439-ab54-27c6ce461822 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:14.435841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fde789-5d2e-4439-ab54-27c6ce461822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:14.457203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-27fde789-5d2e-4439-ab54-27c6ce461822 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:14.458315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-27fde789-5d2e-4439-ab54-27c6ce461822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:14.546388 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1431/2830] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:14.604968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:14.607719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:14.676486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f639e3-3f3c-4ee6-af2b-40f89d5f3166 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:14.682565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14a637-29e8-46f9-a6d8-b64552e78349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:14.683359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14a637-29e8-46f9-a6d8-b64552e78349 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:14.686231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14a637-29e8-46f9-a6d8-b64552e78349 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:14.686231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14a637-29e8-46f9-a6d8-b64552e78349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:14.699602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14a637-29e8-46f9-a6d8-b64552e78349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:14.702019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14a637-29e8-46f9-a6d8-b64552e78349 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:14.713588 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1400/2831] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:15.482330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fde789-5d2e-4439-ab54-27c6ce461822 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:15.484000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-27fde789-5d2e-4439-ab54-27c6ce461822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:15.502646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fde789-5d2e-4439-ab54-27c6ce461822 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:15.520220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:15.521218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:15.522443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:15.523041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] PATH_INFO: `/v3/projects/91d0220829a84c93a58eea44b8e07fa0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:15.525113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d0220829a84c93a58eea44b8e07fa0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:15.527189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:15.617021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d0220829a84c93a58eea44b8e07fa0', 'request_id': 'req-cab4fa88-255b-49e3-b707-33c0a523af30'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:15.641356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d0220829a84c93a58eea44b8e07fa0', 'request_id': 'req-cab4fa88-255b-49e3-b707-33c0a523af30'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:15.652806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d0220829a84c93a58eea44b8e07fa0', 'request_id': 'req-cab4fa88-255b-49e3-b707-33c0a523af30'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:15.695757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] The token cache is being invalidate because project 91d0220829a84c93a58eea44b8e07fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:59:15.696232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:15.697872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:15.707327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14a637-29e8-46f9-a6d8-b64552e78349 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:15.709491 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1432/2832] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:59:15 2022] DELETE /identity/v3/projects/91d0220829a84c93a58eea44b8e07fa0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:15.729555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14a637-29e8-46f9-a6d8-b64552e78349 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:15.857541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:15.872826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:15.873481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:15.873752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:15.874006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] PATH_INFO: `/v3/users/5ee9f4636b2c455ea4fe357955d687f8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:15.875461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ee9f4636b2c455ea4fe357955d687f8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:15.877780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:15.991774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ee9f4636b2c455ea4fe357955d687f8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:16.015717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ee9f4636b2c455ea4fe357955d687f8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:16.059461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ee9f4636b2c455ea4fe357955d687f8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:16.061248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ee9f4636b2c455ea4fe357955d687f8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:16.064106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ee9f4636b2c455ea4fe357955d687f8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:16.100049 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1401/2833] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:59:15 2022] DELETE /identity/v3/users/5ee9f4636b2c455ea4fe357955d687f8 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:16.953412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:16.957233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:17.055950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:17.073046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:17.074781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:17.215362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4fa88-255b-49e3-b707-33c0a523af30 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:17.300491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a569c90-78f3-466e-86ff-f56b95ed8394 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:17.307477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:17.308395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:17.309380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:17.309380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84 admin admin] PATH_INFO: `/v3/projects/9b7b920f28ee4956b735595591863fbb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:17.311544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b7b920f28ee4956b735595591863fbb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:17.314640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:17.355987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70add6b5-a0ad-436d-8bca-234b074c08a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:17.357481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70add6b5-a0ad-436d-8bca-234b074c08a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:17.357481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70add6b5-a0ad-436d-8bca-234b074c08a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:17.357481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70add6b5-a0ad-436d-8bca-234b074c08a2 admin admin] PATH_INFO: `/v3/users/2281296acde947168cb5c5d97d2c5b6e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:17.367803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70add6b5-a0ad-436d-8bca-234b074c08a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2281296acde947168cb5c5d97d2c5b6e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:17.373683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:17.375190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70add6b5-a0ad-436d-8bca-234b074c08a2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:17.452135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b7b920f28ee4956b735595591863fbb', 'request_id': 'req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:17.468331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b7b920f28ee4956b735595591863fbb', 'request_id': 'req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:17.487861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b7b920f28ee4956b735595591863fbb', 'request_id': 'req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:17.501014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-70add6b5-a0ad-436d-8bca-234b074c08a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2281296acde947168cb5c5d97d2c5b6e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:17.512689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84 admin admin] The token cache is being invalidate because project 9b7b920f28ee4956b735595591863fbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:59:17.513009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:17.513842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a87bacf6-1f60-46d0-9f82-6ac3d8e2dc84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:17.518912 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1402/2834] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:59:17 2022] DELETE /identity/v3/projects/9b7b920f28ee4956b735595591863fbb => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:17.525369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c47a9-d522-4a5e-a337-b40d2a344dca None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:17.525755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c47a9-d522-4a5e-a337-b40d2a344dca None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:17.525968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c47a9-d522-4a5e-a337-b40d2a344dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:17.528721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-70add6b5-a0ad-436d-8bca-234b074c08a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2281296acde947168cb5c5d97d2c5b6e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:17.549759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-70add6b5-a0ad-436d-8bca-234b074c08a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2281296acde947168cb5c5d97d2c5b6e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:17.587334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-70add6b5-a0ad-436d-8bca-234b074c08a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2281296acde947168cb5c5d97d2c5b6e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:17.587995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-70add6b5-a0ad-436d-8bca-234b074c08a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2281296acde947168cb5c5d97d2c5b6e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:17.610162 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1433/2835] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:59:16 2022] DELETE /identity/v3/users/2281296acde947168cb5c5d97d2c5b6e => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:17.650295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-972c47a9-d522-4a5e-a337-b40d2a344dca None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:17.651746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-972c47a9-d522-4a5e-a337-b40d2a344dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:17.846751 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1403/2836] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:59:17.855135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f75c19-bd6b-421a-adb3-e2e9d80dc0cc None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:17.856280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f75c19-bd6b-421a-adb3-e2e9d80dc0cc None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:17.856918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f75c19-bd6b-421a-adb3-e2e9d80dc0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:17.897525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a7f75c19-bd6b-421a-adb3-e2e9d80dc0cc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:17.899249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a7f75c19-bd6b-421a-adb3-e2e9d80dc0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:17.920665 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1434/2837] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:17.925943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c47a9-d522-4a5e-a337-b40d2a344dca None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:17.944888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c47a9-d522-4a5e-a337-b40d2a344dca None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:17.956755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba2ab6-5ff2-493b-a9b5-1c7593860db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:17.957390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba2ab6-5ff2-493b-a9b5-1c7593860db7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:17.957665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba2ab6-5ff2-493b-a9b5-1c7593860db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:17.957930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba2ab6-5ff2-493b-a9b5-1c7593860db7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:17.958588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba2ab6-5ff2-493b-a9b5-1c7593860db7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:59:17.958909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba2ab6-5ff2-493b-a9b5-1c7593860db7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:17.962476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba2ab6-5ff2-493b-a9b5-1c7593860db7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:17.994362 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1404/2838] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:59:17.998803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f75c19-bd6b-421a-adb3-e2e9d80dc0cc None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:18.014820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f75c19-bd6b-421a-adb3-e2e9d80dc0cc None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:18.030813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac463a-867c-4d78-9102-302a802236f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:18.031455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac463a-867c-4d78-9102-302a802236f8 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:18.031724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac463a-867c-4d78-9102-302a802236f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:18.031986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac463a-867c-4d78-9102-302a802236f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:18.032649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac463a-867c-4d78-9102-302a802236f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:18.034712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac463a-867c-4d78-9102-302a802236f8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:18.093087 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1435/2839] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:59:17 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:18.095120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba2ab6-5ff2-493b-a9b5-1c7593860db7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:18.114394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba2ab6-5ff2-493b-a9b5-1c7593860db7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:18.178965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac463a-867c-4d78-9102-302a802236f8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:18.180938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2eac463a-867c-4d78-9102-302a802236f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:18.197320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:18.212207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313dd5c4-960d-4946-8b85-5f124d0468d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:18.220460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313dd5c4-960d-4946-8b85-5f124d0468d8 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:18.221681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313dd5c4-960d-4946-8b85-5f124d0468d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:18.222181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313dd5c4-960d-4946-8b85-5f124d0468d8 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:18.224825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313dd5c4-960d-4946-8b85-5f124d0468d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:18.227075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313dd5c4-960d-4946-8b85-5f124d0468d8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:18.273466 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1405/2840] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:59:18 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:18.276297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313dd5c4-960d-4946-8b85-5f124d0468d8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:18.310298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313dd5c4-960d-4946-8b85-5f124d0468d8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:18.314172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac463a-867c-4d78-9102-302a802236f8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:18.321649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11c565-f378-4205-8513-46b3f0004530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:18.322285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11c565-f378-4205-8513-46b3f0004530 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:18.322570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11c565-f378-4205-8513-46b3f0004530 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:18.324158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11c565-f378-4205-8513-46b3f0004530 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:18.325358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11c565-f378-4205-8513-46b3f0004530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:18.326986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11c565-f378-4205-8513-46b3f0004530 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:18.328968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc110e-ac15-4876-8404-80756f5ae928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:18.329626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc110e-ac15-4876-8404-80756f5ae928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:18.330041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc110e-ac15-4876-8404-80756f5ae928 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:18.330140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc110e-ac15-4876-8404-80756f5ae928 admin admin] PATH_INFO: `/v3/projects/1e4a377c3b9444c6aa448e3799d3cdb2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:18.332609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc110e-ac15-4876-8404-80756f5ae928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e4a377c3b9444c6aa448e3799d3cdb2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:18.334907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc110e-ac15-4876-8404-80756f5ae928 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:18.361764 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1406/2841] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:18.364584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11c565-f378-4205-8513-46b3f0004530 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:18.380877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:18.395820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11c565-f378-4205-8513-46b3f0004530 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:18.410830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff818de-9252-40e6-b97b-1d1144a081ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:18.412964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff818de-9252-40e6-b97b-1d1144a081ff admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:18.413255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff818de-9252-40e6-b97b-1d1144a081ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:18.413527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff818de-9252-40e6-b97b-1d1144a081ff admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:18.415325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff818de-9252-40e6-b97b-1d1144a081ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:18.419135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff818de-9252-40e6-b97b-1d1144a081ff admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:18.432636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-51cc110e-ac15-4876-8404-80756f5ae928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e4a377c3b9444c6aa448e3799d3cdb2', 'request_id': 'req-51cc110e-ac15-4876-8404-80756f5ae928'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:18.450594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-51cc110e-ac15-4876-8404-80756f5ae928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e4a377c3b9444c6aa448e3799d3cdb2', 'request_id': 'req-51cc110e-ac15-4876-8404-80756f5ae928'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:18.457598 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1407/2842] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:18.461508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-51cc110e-ac15-4876-8404-80756f5ae928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e4a377c3b9444c6aa448e3799d3cdb2', 'request_id': 'req-51cc110e-ac15-4876-8404-80756f5ae928'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:18.461675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff818de-9252-40e6-b97b-1d1144a081ff admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:18.482179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-51cc110e-ac15-4876-8404-80756f5ae928 admin admin] The token cache is being invalidate because project 1e4a377c3b9444c6aa448e3799d3cdb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:59:18.482659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-51cc110e-ac15-4876-8404-80756f5ae928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:18.483500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-51cc110e-ac15-4876-8404-80756f5ae928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:18.484486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff818de-9252-40e6-b97b-1d1144a081ff admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:18.485132 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1436/2843] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:59:18 2022] DELETE /identity/v3/projects/1e4a377c3b9444c6aa448e3799d3cdb2 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:59:18.490047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72a6c3-7fe4-4bc3-b45f-332060469cdd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:18.490442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72a6c3-7fe4-4bc3-b45f-332060469cdd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:18.491606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72a6c3-7fe4-4bc3-b45f-332060469cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:18.531029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-db72a6c3-7fe4-4bc3-b45f-332060469cdd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:18.533629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-db72a6c3-7fe4-4bc3-b45f-332060469cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:18.619796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f627e83-8f43-4bcf-be72-eb5d3b0c7de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:18.620905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f627e83-8f43-4bcf-be72-eb5d3b0c7de1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:18.621332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f627e83-8f43-4bcf-be72-eb5d3b0c7de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:18.621838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f627e83-8f43-4bcf-be72-eb5d3b0c7de1 admin admin] PATH_INFO: `/v3/projects/b4c402c5e1ba421290728f887fa30b9a/users/0a583f93e79843448f5eb36b844b9eac/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:18.627730 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1437/2844] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:18.640986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f627e83-8f43-4bcf-be72-eb5d3b0c7de1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c402c5e1ba421290728f887fa30b9a, user_id=0a583f93e79843448f5eb36b844b9eac, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:18.641370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af56add-fddc-4cc4-bc0e-4c8ca3410235 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:18.641744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af56add-fddc-4cc4-bc0e-4c8ca3410235 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:18.642079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af56add-fddc-4cc4-bc0e-4c8ca3410235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:18.644501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f627e83-8f43-4bcf-be72-eb5d3b0c7de1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:18.681626 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1408/2845] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:59:18 2022] PUT /identity/v3/projects/b4c402c5e1ba421290728f887fa30b9a/users/0a583f93e79843448f5eb36b844b9eac/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:59:18.693372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33539860-8065-4c96-9af6-21b035b5c134 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:18.693958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33539860-8065-4c96-9af6-21b035b5c134 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:18.694921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33539860-8065-4c96-9af6-21b035b5c134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:18.709815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2af56add-fddc-4cc4-bc0e-4c8ca3410235 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:18.711479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2af56add-fddc-4cc4-bc0e-4c8ca3410235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:18.754656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-33539860-8065-4c96-9af6-21b035b5c134 None None] MFA Rules not processed for user `0a583f93e79843448f5eb36b844b9eac`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:18.756319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-33539860-8065-4c96-9af6-21b035b5c134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:18.761128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:18.811662 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1438/2846] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:18.815401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af56add-fddc-4cc4-bc0e-4c8ca3410235 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:18.831115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af56add-fddc-4cc4-bc0e-4c8ca3410235 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:18.837937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d6047-47a5-4069-a660-9614cb66d49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:18.838842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d6047-47a5-4069-a660-9614cb66d49e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:18.839306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d6047-47a5-4069-a660-9614cb66d49e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:18.839792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d6047-47a5-4069-a660-9614cb66d49e admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:18.840857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d6047-47a5-4069-a660-9614cb66d49e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:59:18.841351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d6047-47a5-4069-a660-9614cb66d49e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:18.842915 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1409/2847] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:59:18.843775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d6047-47a5-4069-a660-9614cb66d49e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:18.843933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33539860-8065-4c96-9af6-21b035b5c134 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:18.846316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-33539860-8065-4c96-9af6-21b035b5c134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:18.866054 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1439/2848] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:59:18.867992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d6047-47a5-4069-a660-9614cb66d49e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:18.873342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33539860-8065-4c96-9af6-21b035b5c134 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:18.880866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23857c07-2140-4d17-8e42-d8faafa18b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:18.881509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23857c07-2140-4d17-8e42-d8faafa18b66 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:18.881779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23857c07-2140-4d17-8e42-d8faafa18b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:18.882038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23857c07-2140-4d17-8e42-d8faafa18b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:18.882463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d6047-47a5-4069-a660-9614cb66d49e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:18.883013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23857c07-2140-4d17-8e42-d8faafa18b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:18.885617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23857c07-2140-4d17-8e42-d8faafa18b66 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:18.888597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5cfaa-6a4c-4030-9586-a297d84ee8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:18.889583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5cfaa-6a4c-4030-9586-a297d84ee8dc admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:18.890085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5cfaa-6a4c-4030-9586-a297d84ee8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:18.890552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5cfaa-6a4c-4030-9586-a297d84ee8dc admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:18.892105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5cfaa-6a4c-4030-9586-a297d84ee8dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:18.894301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5cfaa-6a4c-4030-9586-a297d84ee8dc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:18.941954 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1440/2849] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:59:18 2022] POST /identity/v3/projects => generated 393 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:18.943213 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1410/2850] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:59:18 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:18.943486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5cfaa-6a4c-4030-9586-a297d84ee8dc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:18.948161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23857c07-2140-4d17-8e42-d8faafa18b66 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:19.039806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5cfaa-6a4c-4030-9586-a297d84ee8dc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:19.063710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:19.122883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23857c07-2140-4d17-8e42-d8faafa18b66 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:19.139819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2e499-72b7-453d-92af-3eeb00c667a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:19.140346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2e499-72b7-453d-92af-3eeb00c667a4 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:19.140646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2e499-72b7-453d-92af-3eeb00c667a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:19.141375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2e499-72b7-453d-92af-3eeb00c667a4 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:19.141692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2e499-72b7-453d-92af-3eeb00c667a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:19.143987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2e499-72b7-453d-92af-3eeb00c667a4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:19.186899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9402cd4-0b02-48d4-b60e-6a651ab218ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:19.187275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9402cd4-0b02-48d4-b60e-6a651ab218ec admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:19.187443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9402cd4-0b02-48d4-b60e-6a651ab218ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:19.187678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9402cd4-0b02-48d4-b60e-6a651ab218ec admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:19.188925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9402cd4-0b02-48d4-b60e-6a651ab218ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:19.191823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9402cd4-0b02-48d4-b60e-6a651ab218ec admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:19.194033 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1411/2851] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:59:18 2022] POST /identity/v3/users => generated 294 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:19.195553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2e499-72b7-453d-92af-3eeb00c667a4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:19.216810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2e499-72b7-453d-92af-3eeb00c667a4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:19.222357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06725e2d-8d47-4807-bbb6-04e84c6f95bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:19.223155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06725e2d-8d47-4807-bbb6-04e84c6f95bd admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:19.224522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06725e2d-8d47-4807-bbb6-04e84c6f95bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:19.224522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06725e2d-8d47-4807-bbb6-04e84c6f95bd admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:19.228406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06725e2d-8d47-4807-bbb6-04e84c6f95bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:19.228406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06725e2d-8d47-4807-bbb6-04e84c6f95bd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:19.242801 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1412/2852] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:19.243844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06725e2d-8d47-4807-bbb6-04e84c6f95bd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:19.244194 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1441/2853] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:59:18 2022] POST /identity/v3/users => generated 297 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:19.250418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9402cd4-0b02-48d4-b60e-6a651ab218ec admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:19.265936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06725e2d-8d47-4807-bbb6-04e84c6f95bd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:19.272158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c883ec2-ce25-411d-99ff-a541c6f5f476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:19.273107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c883ec2-ce25-411d-99ff-a541c6f5f476 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:19.273816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c883ec2-ce25-411d-99ff-a541c6f5f476 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:19.274268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c883ec2-ce25-411d-99ff-a541c6f5f476 admin admin] PATH_INFO: `/v3/projects/5d5c9b69be95428ca0e839de0877bf36/users/3896e4fc6cb5405882082d0eb22f7e23/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:19.274446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9402cd4-0b02-48d4-b60e-6a651ab218ec admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:19.284721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07675a42-29b6-4d27-b669-af5c4af9479c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:19.285255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07675a42-29b6-4d27-b669-af5c4af9479c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:19.285523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07675a42-29b6-4d27-b669-af5c4af9479c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:19.285786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07675a42-29b6-4d27-b669-af5c4af9479c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:19.286681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07675a42-29b6-4d27-b669-af5c4af9479c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:19.288280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07675a42-29b6-4d27-b669-af5c4af9479c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:19.295844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c883ec2-ce25-411d-99ff-a541c6f5f476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d5c9b69be95428ca0e839de0877bf36, user_id=3896e4fc6cb5405882082d0eb22f7e23, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:19.298804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c883ec2-ce25-411d-99ff-a541c6f5f476 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:19.319693 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1442/2854] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:19.321445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07675a42-29b6-4d27-b669-af5c4af9479c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:19.344132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07675a42-29b6-4d27-b669-af5c4af9479c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:19.355039 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1413/2855] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:59:19 2022] PUT /identity/v3/projects/5d5c9b69be95428ca0e839de0877bf36/users/3896e4fc6cb5405882082d0eb22f7e23/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:19.360710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c883ec2-ce25-411d-99ff-a541c6f5f476 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:19.382536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:19.468524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405a1d5-5fc9-4b16-8632-c33de8acef9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:19.469170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405a1d5-5fc9-4b16-8632-c33de8acef9d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:19.469448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405a1d5-5fc9-4b16-8632-c33de8acef9d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:19.469717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405a1d5-5fc9-4b16-8632-c33de8acef9d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:19.470417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405a1d5-5fc9-4b16-8632-c33de8acef9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:19.472063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405a1d5-5fc9-4b16-8632-c33de8acef9d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:19.486356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c883ec2-ce25-411d-99ff-a541c6f5f476 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:19.491912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81081a-e782-401e-86c6-6ffb0c63b6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:19.492824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81081a-e782-401e-86c6-6ffb0c63b6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:19.493308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81081a-e782-401e-86c6-6ffb0c63b6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:19.493828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81081a-e782-401e-86c6-6ffb0c63b6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:19.494851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81081a-e782-401e-86c6-6ffb0c63b6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:19.496927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81081a-e782-401e-86c6-6ffb0c63b6d8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:19.516054 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1443/2856] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:19.517260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405a1d5-5fc9-4b16-8632-c33de8acef9d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:19.519680 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1414/2857] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:19.520816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81081a-e782-401e-86c6-6ffb0c63b6d8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:19.540926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405a1d5-5fc9-4b16-8632-c33de8acef9d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:19.546202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81081a-e782-401e-86c6-6ffb0c63b6d8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:19.551462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495b1ea-c2da-4ab8-a040-3237b826acb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:19.552221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495b1ea-c2da-4ab8-a040-3237b826acb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:19.552498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495b1ea-c2da-4ab8-a040-3237b826acb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:19.552765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495b1ea-c2da-4ab8-a040-3237b826acb8 admin admin] PATH_INFO: `/v3/projects/ce96fc00ce214713947c7e949c7a7192/users/811bc5f0350147f18454a1ac0127e47b/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:19.560779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6d7e5-591a-4a2c-909b-5209e6697ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:19.561496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6d7e5-591a-4a2c-909b-5209e6697ed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:19.562267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6d7e5-591a-4a2c-909b-5209e6697ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:19.562267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6d7e5-591a-4a2c-909b-5209e6697ed5 admin admin] PATH_INFO: `/v3/projects/5d5c9b69be95428ca0e839de0877bf36/users/3896e4fc6cb5405882082d0eb22f7e23/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:19.576850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6d7e5-591a-4a2c-909b-5209e6697ed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d5c9b69be95428ca0e839de0877bf36, user_id=3896e4fc6cb5405882082d0eb22f7e23, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:19.580307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6d7e5-591a-4a2c-909b-5209e6697ed5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:19.583707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495b1ea-c2da-4ab8-a040-3237b826acb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce96fc00ce214713947c7e949c7a7192, user_id=811bc5f0350147f18454a1ac0127e47b, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:19.588274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495b1ea-c2da-4ab8-a040-3237b826acb8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:19.595025 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1415/2858] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:59:19 2022] PUT /identity/v3/projects/5d5c9b69be95428ca0e839de0877bf36/users/3896e4fc6cb5405882082d0eb22f7e23/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:19.610451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841a2cc-1982-4871-a530-a85d160ccf25 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:19.610732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841a2cc-1982-4871-a530-a85d160ccf25 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:19.611111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841a2cc-1982-4871-a530-a85d160ccf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:19.613536 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1444/2859] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:59:19 2022] PUT /identity/v3/projects/ce96fc00ce214713947c7e949c7a7192/users/811bc5f0350147f18454a1ac0127e47b/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:19.619539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0963f56-ba4c-4a99-ac6e-de6949770dab None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:19.619811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0963f56-ba4c-4a99-ac6e-de6949770dab None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:19.620114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0963f56-ba4c-4a99-ac6e-de6949770dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:19.654526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f0963f56-ba4c-4a99-ac6e-de6949770dab None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:19.656728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f0963f56-ba4c-4a99-ac6e-de6949770dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:19.657801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5841a2cc-1982-4871-a530-a85d160ccf25 None None] MFA Rules not processed for user `3896e4fc6cb5405882082d0eb22f7e23`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:19.665719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5841a2cc-1982-4871-a530-a85d160ccf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:19.782454 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1445/2860] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:19.798623 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1416/2861] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:19.812133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841a2cc-1982-4871-a530-a85d160ccf25 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:19.812237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf06b7-2d96-400b-afb2-7f67ba0dd54a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:19.812844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf06b7-2d96-400b-afb2-7f67ba0dd54a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:19.813116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf06b7-2d96-400b-afb2-7f67ba0dd54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:19.815357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5841a2cc-1982-4871-a530-a85d160ccf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:19.820979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:19.855184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2faf06b7-2d96-400b-afb2-7f67ba0dd54a None None] MFA Rules not processed for user `0a583f93e79843448f5eb36b844b9eac`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:19.856606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2faf06b7-2d96-400b-afb2-7f67ba0dd54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:19.963891 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1446/2862] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:19.986782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf06b7-2d96-400b-afb2-7f67ba0dd54a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:19.989114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2faf06b7-2d96-400b-afb2-7f67ba0dd54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:20.000016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841a2cc-1982-4871-a530-a85d160ccf25 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:20.003739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252494c3-5924-4897-a323-1d5d8708a8b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:20.004351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252494c3-5924-4897-a323-1d5d8708a8b6 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:20.004651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252494c3-5924-4897-a323-1d5d8708a8b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:20.004918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252494c3-5924-4897-a323-1d5d8708a8b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:20.014727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252494c3-5924-4897-a323-1d5d8708a8b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:20.016240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252494c3-5924-4897-a323-1d5d8708a8b6 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:20.028396 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1417/2863] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:20.070519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:20.115412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf06b7-2d96-400b-afb2-7f67ba0dd54a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:20.126037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eead7c-e0d9-428a-9794-addea2bcaec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:20.126777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eead7c-e0d9-428a-9794-addea2bcaec2 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:20.127184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eead7c-e0d9-428a-9794-addea2bcaec2 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:20.127533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eead7c-e0d9-428a-9794-addea2bcaec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:20.137592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eead7c-e0d9-428a-9794-addea2bcaec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:20.139678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eead7c-e0d9-428a-9794-addea2bcaec2 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:20.162858 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1447/2864] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:29.279896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403df565-1043-426e-8abf-b1c070d2b871 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:29.280471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403df565-1043-426e-8abf-b1c070d2b871 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:29.281152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403df565-1043-426e-8abf-b1c070d2b871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:29.316325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-403df565-1043-426e-8abf-b1c070d2b871 None None] MFA Rules not processed for user `811bc5f0350147f18454a1ac0127e47b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:29.318080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-403df565-1043-426e-8abf-b1c070d2b871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:29.493061 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1418/2865] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:29.494204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656f7dc-9a5e-4c0f-9d39-83d3f4873988 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:29.494679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656f7dc-9a5e-4c0f-9d39-83d3f4873988 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:29.494835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656f7dc-9a5e-4c0f-9d39-83d3f4873988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:29.530586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5656f7dc-9a5e-4c0f-9d39-83d3f4873988 None None] MFA Rules not processed for user `811bc5f0350147f18454a1ac0127e47b`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:29.532184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5656f7dc-9a5e-4c0f-9d39-83d3f4873988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:29.545419 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1448/2866] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:29.729728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403df565-1043-426e-8abf-b1c070d2b871 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:29.732252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-403df565-1043-426e-8abf-b1c070d2b871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:29.760573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403df565-1043-426e-8abf-b1c070d2b871 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:29.772407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd582e-22c5-4ecc-b58a-130444761cb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:29.773403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd582e-22c5-4ecc-b58a-130444761cb2 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:29.773543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd582e-22c5-4ecc-b58a-130444761cb2 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:29.773728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd582e-22c5-4ecc-b58a-130444761cb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:29.787102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd582e-22c5-4ecc-b58a-130444761cb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:29.791005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd582e-22c5-4ecc-b58a-130444761cb2 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:29.822486 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1419/2867] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:29 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:30.677844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656f7dc-9a5e-4c0f-9d39-83d3f4873988 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:30.679031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5656f7dc-9a5e-4c0f-9d39-83d3f4873988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:30.702135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656f7dc-9a5e-4c0f-9d39-83d3f4873988 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:30.713103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:30.714091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:30.716767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:30.716957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] PATH_INFO: `/v3/users/1762f75263ad4e2f84b265522e76764d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:30.718706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1762f75263ad4e2f84b265522e76764d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:30.720683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:30.809428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1762f75263ad4e2f84b265522e76764d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:30.819589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1762f75263ad4e2f84b265522e76764d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:30.828152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1762f75263ad4e2f84b265522e76764d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:30.843013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1762f75263ad4e2f84b265522e76764d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:30.843839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1762f75263ad4e2f84b265522e76764d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:30.851825 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1449/2868] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:59:30 2022] DELETE /identity/v3/users/1762f75263ad4e2f84b265522e76764d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:31.077211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556095f8-3dad-4537-8f72-c4f00a00254f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:31.077699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556095f8-3dad-4537-8f72-c4f00a00254f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:31.077699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556095f8-3dad-4537-8f72-c4f00a00254f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:31.130210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-556095f8-3dad-4537-8f72-c4f00a00254f None None] MFA Rules not processed for user `811bc5f0350147f18454a1ac0127e47b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:31.131514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-556095f8-3dad-4537-8f72-c4f00a00254f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:31.280493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:31.282596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:31.286368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:31.314504 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1420/2869] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:31.462997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac23839-13f3-45c2-85d6-fbd374c9f3c3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:31.477053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:31.477840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:31.478119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:31.478452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] PATH_INFO: `/v3/projects/61de886fca884a25a484d8d0f9e6662f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:31.481987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61de886fca884a25a484d8d0f9e6662f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:31.484139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:31.570543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:31.587167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61de886fca884a25a484d8d0f9e6662f', 'request_id': 'req-abe4d9c4-e788-46ef-a30e-d833e4d019e9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:31.605145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61de886fca884a25a484d8d0f9e6662f', 'request_id': 'req-abe4d9c4-e788-46ef-a30e-d833e4d019e9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:31.613749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61de886fca884a25a484d8d0f9e6662f', 'request_id': 'req-abe4d9c4-e788-46ef-a30e-d833e4d019e9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:31.637376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] The token cache is being invalidate because project 61de886fca884a25a484d8d0f9e6662f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:59:31.637723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:31.643003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:31.645477 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1450/2870] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:59:31 2022] DELETE /identity/v3/projects/61de886fca884a25a484d8d0f9e6662f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:59:31.649693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556095f8-3dad-4537-8f72-c4f00a00254f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:31.666424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556095f8-3dad-4537-8f72-c4f00a00254f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:31.828588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64108b15-5d94-4c2d-a513-534bb3fc8d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:31.829860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108b15-5d94-4c2d-a513-534bb3fc8d3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:31.830340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108b15-5d94-4c2d-a513-534bb3fc8d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:31.830831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108b15-5d94-4c2d-a513-534bb3fc8d3c admin admin] PATH_INFO: `/v3/users/396027dc8be44f91805a44d83be35e29` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:31.834083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64108b15-5d94-4c2d-a513-534bb3fc8d3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=396027dc8be44f91805a44d83be35e29)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:31.837845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64108b15-5d94-4c2d-a513-534bb3fc8d3c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:31.947391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64108b15-5d94-4c2d-a513-534bb3fc8d3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '396027dc8be44f91805a44d83be35e29'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:31.964839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64108b15-5d94-4c2d-a513-534bb3fc8d3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '396027dc8be44f91805a44d83be35e29'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:31.986962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64108b15-5d94-4c2d-a513-534bb3fc8d3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '396027dc8be44f91805a44d83be35e29'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:31.987799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64108b15-5d94-4c2d-a513-534bb3fc8d3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '396027dc8be44f91805a44d83be35e29'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:31.988457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64108b15-5d94-4c2d-a513-534bb3fc8d3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '396027dc8be44f91805a44d83be35e29'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:32.006936 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1421/2871] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:59:31 2022] DELETE /identity/v3/users/396027dc8be44f91805a44d83be35e29 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:32.536561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:32.538573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:32.592092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4d9c4-e788-46ef-a30e-d833e4d019e9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:32.706701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc31124-9fb5-4a01-a468-eaabbb7b9713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:32.707626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc31124-9fb5-4a01-a468-eaabbb7b9713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:32.708154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc31124-9fb5-4a01-a468-eaabbb7b9713 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:32.710032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc31124-9fb5-4a01-a468-eaabbb7b9713 admin admin] PATH_INFO: `/v3/projects/71141fb46193431085ee6d268fc4ce5d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:32.712495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc31124-9fb5-4a01-a468-eaabbb7b9713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71141fb46193431085ee6d268fc4ce5d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:32.715178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc31124-9fb5-4a01-a468-eaabbb7b9713 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:32.833388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9bc31124-9fb5-4a01-a468-eaabbb7b9713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71141fb46193431085ee6d268fc4ce5d', 'request_id': 'req-9bc31124-9fb5-4a01-a468-eaabbb7b9713'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:32.860489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9bc31124-9fb5-4a01-a468-eaabbb7b9713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71141fb46193431085ee6d268fc4ce5d', 'request_id': 'req-9bc31124-9fb5-4a01-a468-eaabbb7b9713'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:32.867695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9bc31124-9fb5-4a01-a468-eaabbb7b9713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71141fb46193431085ee6d268fc4ce5d', 'request_id': 'req-9bc31124-9fb5-4a01-a468-eaabbb7b9713'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:32.879597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9bc31124-9fb5-4a01-a468-eaabbb7b9713 admin admin] The token cache is being invalidate because project 71141fb46193431085ee6d268fc4ce5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:59:32.880202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9bc31124-9fb5-4a01-a468-eaabbb7b9713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:32.881034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9bc31124-9fb5-4a01-a468-eaabbb7b9713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:32.888096 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1451/2872] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:59:32 2022] DELETE /identity/v3/projects/71141fb46193431085ee6d268fc4ce5d => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:32.889936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e017bb-b661-4911-803d-33894d156426 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:32.890440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e017bb-b661-4911-803d-33894d156426 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:32.890573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e017bb-b661-4911-803d-33894d156426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:32.919041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a9e017bb-b661-4911-803d-33894d156426 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:32.919623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a9e017bb-b661-4911-803d-33894d156426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:33.065819 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1422/2873] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:33.074693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357d5eb-0b84-4a30-9525-348ce2ca5d52 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:33.074693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357d5eb-0b84-4a30-9525-348ce2ca5d52 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:33.075017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357d5eb-0b84-4a30-9525-348ce2ca5d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:33.112265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0357d5eb-0b84-4a30-9525-348ce2ca5d52 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:33.117151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0357d5eb-0b84-4a30-9525-348ce2ca5d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:33.148038 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1452/2874] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:33.153043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e017bb-b661-4911-803d-33894d156426 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:33.166820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e017bb-b661-4911-803d-33894d156426 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:33.176303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd478d94-f873-4ad6-9aa3-bd708f2af6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:33.176657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd478d94-f873-4ad6-9aa3-bd708f2af6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:33.176978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd478d94-f873-4ad6-9aa3-bd708f2af6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:33.177240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd478d94-f873-4ad6-9aa3-bd708f2af6d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:33.177889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd478d94-f873-4ad6-9aa3-bd708f2af6d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:59:33.178529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd478d94-f873-4ad6-9aa3-bd708f2af6d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:33.180477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd478d94-f873-4ad6-9aa3-bd708f2af6d0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:33.193162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:33.204780 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1423/2875] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:59:33.207140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357d5eb-0b84-4a30-9525-348ce2ca5d52 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:33.221760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357d5eb-0b84-4a30-9525-348ce2ca5d52 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:33.235791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a92e6-f4da-4abf-a286-9e7873b98c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:33.236717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a92e6-f4da-4abf-a286-9e7873b98c3a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:33.237362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a92e6-f4da-4abf-a286-9e7873b98c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:33.237683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a92e6-f4da-4abf-a286-9e7873b98c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:33.238495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a92e6-f4da-4abf-a286-9e7873b98c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:33.245282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a92e6-f4da-4abf-a286-9e7873b98c3a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:33.326050 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1453/2876] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:59:33 2022] POST /identity/v3/projects => generated 397 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:33.329915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd478d94-f873-4ad6-9aa3-bd708f2af6d0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:33.345254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd478d94-f873-4ad6-9aa3-bd708f2af6d0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:33.467589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486f973-5612-4601-b327-c8a40d033857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:33.468663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486f973-5612-4601-b327-c8a40d033857 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:33.469145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486f973-5612-4601-b327-c8a40d033857 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:33.471474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486f973-5612-4601-b327-c8a40d033857 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:33.473139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486f973-5612-4601-b327-c8a40d033857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:33.475642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486f973-5612-4601-b327-c8a40d033857 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:33.528677 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1424/2877] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:59:33 2022] POST /identity/v3/users => generated 299 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:33.532479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a92e6-f4da-4abf-a286-9e7873b98c3a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:33.569079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a92e6-f4da-4abf-a286-9e7873b98c3a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:33.605248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755624ad-e079-420d-9cb3-c7508fe21546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:33.608767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755624ad-e079-420d-9cb3-c7508fe21546 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:33.609624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755624ad-e079-420d-9cb3-c7508fe21546 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:33.610429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755624ad-e079-420d-9cb3-c7508fe21546 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:33.611534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755624ad-e079-420d-9cb3-c7508fe21546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:33.615115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755624ad-e079-420d-9cb3-c7508fe21546 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:33.655683 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1454/2878] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:33.660629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486f973-5612-4601-b327-c8a40d033857 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:33.671483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486f973-5612-4601-b327-c8a40d033857 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:33.678288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731246e0-3a04-4d47-992d-bbe8c690d6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:33.679365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731246e0-3a04-4d47-992d-bbe8c690d6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:33.679849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731246e0-3a04-4d47-992d-bbe8c690d6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:33.680297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731246e0-3a04-4d47-992d-bbe8c690d6f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:33.681247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731246e0-3a04-4d47-992d-bbe8c690d6f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:33.683095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731246e0-3a04-4d47-992d-bbe8c690d6f7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:33.709766 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1425/2879] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:33.712425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755624ad-e079-420d-9cb3-c7508fe21546 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:33.747532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755624ad-e079-420d-9cb3-c7508fe21546 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:33.757564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd67b5-2a52-48d5-a73d-b47eb3e127aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:33.758647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd67b5-2a52-48d5-a73d-b47eb3e127aa admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:33.759168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd67b5-2a52-48d5-a73d-b47eb3e127aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:33.759653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd67b5-2a52-48d5-a73d-b47eb3e127aa admin admin] PATH_INFO: `/v3/projects/3339546f8f5e4572946f823f9b69b308/users/37506d5c69f64974ad817a22fefb6db5/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:33.822793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd67b5-2a52-48d5-a73d-b47eb3e127aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3339546f8f5e4572946f823f9b69b308, user_id=37506d5c69f64974ad817a22fefb6db5, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:33.832011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd67b5-2a52-48d5-a73d-b47eb3e127aa admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:33.834657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:33.894319 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1455/2880] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:59:33 2022] PUT /identity/v3/projects/3339546f8f5e4572946f823f9b69b308/users/37506d5c69f64974ad817a22fefb6db5/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:33.896297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcb9f3-d24c-4ad3-a7e0-8366f7cc3057 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:33.897208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcb9f3-d24c-4ad3-a7e0-8366f7cc3057 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:33.897712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcb9f3-d24c-4ad3-a7e0-8366f7cc3057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:33.969965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fbdcb9f3-d24c-4ad3-a7e0-8366f7cc3057 None None] MFA Rules not processed for user `37506d5c69f64974ad817a22fefb6db5`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:33.971935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fbdcb9f3-d24c-4ad3-a7e0-8366f7cc3057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:34.101105 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1426/2881] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:33 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:59:34.120405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd67b5-2a52-48d5-a73d-b47eb3e127aa admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:34.122734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-22dd67b5-2a52-48d5-a73d-b47eb3e127aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:34.270220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd67b5-2a52-48d5-a73d-b47eb3e127aa admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:34.280949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d298e-6433-49a4-99a2-0dd9f083c864 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:34.281739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d298e-6433-49a4-99a2-0dd9f083c864 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:34.282106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d298e-6433-49a4-99a2-0dd9f083c864 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:34.282463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d298e-6433-49a4-99a2-0dd9f083c864 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:34.293767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d298e-6433-49a4-99a2-0dd9f083c864 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:34.295548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d298e-6433-49a4-99a2-0dd9f083c864 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:34.307575 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1456/2882] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:34 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:34.395392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60139e-56d7-4dfc-99b0-97e5a905dab8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:34.396025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60139e-56d7-4dfc-99b0-97e5a905dab8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:34.396560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60139e-56d7-4dfc-99b0-97e5a905dab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:34.429076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ba60139e-56d7-4dfc-99b0-97e5a905dab8 None None] MFA Rules not processed for user `37506d5c69f64974ad817a22fefb6db5`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:34.431163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ba60139e-56d7-4dfc-99b0-97e5a905dab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:34.444665 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1427/2883] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:34 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:35.060231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d298e-6433-49a4-99a2-0dd9f083c864 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:35.160814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d298e-6433-49a4-99a2-0dd9f083c864 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:35.165555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:35.166186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:35.166506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:35.166772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] PATH_INFO: `/v3/users/37506d5c69f64974ad817a22fefb6db5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:35.167880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37506d5c69f64974ad817a22fefb6db5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:35.170256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:35.229796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37506d5c69f64974ad817a22fefb6db5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:35.242377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37506d5c69f64974ad817a22fefb6db5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:35.251734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37506d5c69f64974ad817a22fefb6db5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:35.283415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37506d5c69f64974ad817a22fefb6db5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:35.284477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37506d5c69f64974ad817a22fefb6db5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:35.286461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:35.292929 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1457/2884] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:59:35 2022] DELETE /identity/v3/users/37506d5c69f64974ad817a22fefb6db5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:59:35.297544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169f8e1-7b5f-4341-8623-78202e0bf23a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:35.298502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169f8e1-7b5f-4341-8623-78202e0bf23a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:35.298797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169f8e1-7b5f-4341-8623-78202e0bf23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:35.325740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5169f8e1-7b5f-4341-8623-78202e0bf23a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:35.327197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5169f8e1-7b5f-4341-8623-78202e0bf23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:35.418549 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1428/2885] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:35.441490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:35.442937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:35.534311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bced7-cc9b-4601-a562-05acb6438875 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:35.538586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230ea62-8bae-4658-a143-06abffad1436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:35.539804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230ea62-8bae-4658-a143-06abffad1436 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:35.540309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230ea62-8bae-4658-a143-06abffad1436 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:35.541087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230ea62-8bae-4658-a143-06abffad1436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:35.553144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230ea62-8bae-4658-a143-06abffad1436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:35.555713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230ea62-8bae-4658-a143-06abffad1436 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:35.576946 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1458/2886] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:36.618497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169f8e1-7b5f-4341-8623-78202e0bf23a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:36.620607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5169f8e1-7b5f-4341-8623-78202e0bf23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:36.629501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:36.637226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169f8e1-7b5f-4341-8623-78202e0bf23a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:36.652760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7432d-47bb-4163-b839-15e38fd1351f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:36.653622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7432d-47bb-4163-b839-15e38fd1351f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:36.654739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7432d-47bb-4163-b839-15e38fd1351f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:36.655163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7432d-47bb-4163-b839-15e38fd1351f admin admin] PATH_INFO: `/v3/projects/3339546f8f5e4572946f823f9b69b308` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:36.657585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7432d-47bb-4163-b839-15e38fd1351f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3339546f8f5e4572946f823f9b69b308)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:36.659912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7432d-47bb-4163-b839-15e38fd1351f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:36.742052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6bb7432d-47bb-4163-b839-15e38fd1351f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3339546f8f5e4572946f823f9b69b308', 'request_id': 'req-6bb7432d-47bb-4163-b839-15e38fd1351f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:36.749972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6bb7432d-47bb-4163-b839-15e38fd1351f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3339546f8f5e4572946f823f9b69b308', 'request_id': 'req-6bb7432d-47bb-4163-b839-15e38fd1351f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:36.761070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6bb7432d-47bb-4163-b839-15e38fd1351f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3339546f8f5e4572946f823f9b69b308', 'request_id': 'req-6bb7432d-47bb-4163-b839-15e38fd1351f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:36.777937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6bb7432d-47bb-4163-b839-15e38fd1351f admin admin] The token cache is being invalidate because project 3339546f8f5e4572946f823f9b69b308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:59:36.778630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6bb7432d-47bb-4163-b839-15e38fd1351f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:36.780344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6bb7432d-47bb-4163-b839-15e38fd1351f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:36.782641 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1429/2887] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:59:36 2022] DELETE /identity/v3/projects/3339546f8f5e4572946f823f9b69b308 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:36.787456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3771517-d10e-40ef-b22c-f590c752e6b0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:36.787963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3771517-d10e-40ef-b22c-f590c752e6b0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:36.788046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3771517-d10e-40ef-b22c-f590c752e6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:36.814250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b3771517-d10e-40ef-b22c-f590c752e6b0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:36.815631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b3771517-d10e-40ef-b22c-f590c752e6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:36.902567 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1459/2888] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:36.911420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc7527-9f11-4c13-934e-4791a4071a8d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:36.911724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc7527-9f11-4c13-934e-4791a4071a8d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:36.916887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc7527-9f11-4c13-934e-4791a4071a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:36.944061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8bfc7527-9f11-4c13-934e-4791a4071a8d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:36.945740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8bfc7527-9f11-4c13-934e-4791a4071a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:36.956237 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1430/2889] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:36.961416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3771517-d10e-40ef-b22c-f590c752e6b0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:36.976077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3771517-d10e-40ef-b22c-f590c752e6b0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:36.977678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc7527-9f11-4c13-934e-4791a4071a8d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:36.979414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8bfc7527-9f11-4c13-934e-4791a4071a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:36.982000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55d491-09af-4e2c-9a6a-673b10035366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:36.982621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55d491-09af-4e2c-9a6a-673b10035366 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:36.982955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55d491-09af-4e2c-9a6a-673b10035366 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:36.983140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55d491-09af-4e2c-9a6a-673b10035366 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:36.983891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55d491-09af-4e2c-9a6a-673b10035366 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:59:36.984188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55d491-09af-4e2c-9a6a-673b10035366 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:36.986756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55d491-09af-4e2c-9a6a-673b10035366 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:37.001486 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1460/2890] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 20:59:37.006103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55d491-09af-4e2c-9a6a-673b10035366 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:37.023378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55d491-09af-4e2c-9a6a-673b10035366 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:37.031516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8f885-1574-411c-9b16-826723bb8f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:37.032202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8f885-1574-411c-9b16-826723bb8f3a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:37.032597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8f885-1574-411c-9b16-826723bb8f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:37.032921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8f885-1574-411c-9b16-826723bb8f3a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:37.033564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8f885-1574-411c-9b16-826723bb8f3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:37.035486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8f885-1574-411c-9b16-826723bb8f3a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:37.065471 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1461/2891] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:59:37 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:37.069938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8f885-1574-411c-9b16-826723bb8f3a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:37.082264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc7527-9f11-4c13-934e-4791a4071a8d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:37.160887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8f885-1574-411c-9b16-826723bb8f3a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:37.167096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d9988-231c-493d-a2d7-3e1c80f248f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:37.167763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d9988-231c-493d-a2d7-3e1c80f248f9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:37.168221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d9988-231c-493d-a2d7-3e1c80f248f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:37.169147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d9988-231c-493d-a2d7-3e1c80f248f9 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:37.169147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d9988-231c-493d-a2d7-3e1c80f248f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:37.171602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d9988-231c-493d-a2d7-3e1c80f248f9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:37.184052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:37.188940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907a32a-1d85-4179-8cfe-1dd6d64c081c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:37.189645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907a32a-1d85-4179-8cfe-1dd6d64c081c service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:37.189922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907a32a-1d85-4179-8cfe-1dd6d64c081c service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:37.190262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907a32a-1d85-4179-8cfe-1dd6d64c081c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:37.192288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f907a32a-1d85-4179-8cfe-1dd6d64c081c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:37.210530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:37.216226 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1462/2892] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:59:37 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:59:37.216843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d9988-231c-493d-a2d7-3e1c80f248f9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:37.221839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-170d9988-231c-493d-a2d7-3e1c80f248f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:37.250660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d9988-231c-493d-a2d7-3e1c80f248f9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:37.267782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713d0b8-e561-41d1-83d1-862f67f0199a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:37.268650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713d0b8-e561-41d1-83d1-862f67f0199a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:37.269053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713d0b8-e561-41d1-83d1-862f67f0199a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:37.269320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713d0b8-e561-41d1-83d1-862f67f0199a admin admin] PATH_INFO: `/v3/users/22489a6c355a49028ca16a394b27876f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:37.270885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713d0b8-e561-41d1-83d1-862f67f0199a admin admin] RBAC: Authorizing `identity:delete_user(user_id=22489a6c355a49028ca16a394b27876f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:37.273085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713d0b8-e561-41d1-83d1-862f67f0199a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:37.282686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907a32a-1d85-4179-8cfe-1dd6d64c081c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:37.284721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907a32a-1d85-4179-8cfe-1dd6d64c081c service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:37.298602 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1431/2893] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:37.299327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907a32a-1d85-4179-8cfe-1dd6d64c081c service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:37.314999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907a32a-1d85-4179-8cfe-1dd6d64c081c service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:37.321474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df05a3-c476-41e3-a4f4-7a3b10be2440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:37.322444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df05a3-c476-41e3-a4f4-7a3b10be2440 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:37.322790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df05a3-c476-41e3-a4f4-7a3b10be2440 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:37.323766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df05a3-c476-41e3-a4f4-7a3b10be2440 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:37.324776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df05a3-c476-41e3-a4f4-7a3b10be2440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:37.326881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df05a3-c476-41e3-a4f4-7a3b10be2440 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:37.336614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5713d0b8-e561-41d1-83d1-862f67f0199a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22489a6c355a49028ca16a394b27876f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:37.344414 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1432/2894] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:37.347006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df05a3-c476-41e3-a4f4-7a3b10be2440 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:37.350517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5713d0b8-e561-41d1-83d1-862f67f0199a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22489a6c355a49028ca16a394b27876f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:37.361042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df05a3-c476-41e3-a4f4-7a3b10be2440 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:37.361220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5713d0b8-e561-41d1-83d1-862f67f0199a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22489a6c355a49028ca16a394b27876f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:37.367339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff27b71-ac03-4e04-af4a-e9e986935245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:37.368300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff27b71-ac03-4e04-af4a-e9e986935245 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:37.369125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff27b71-ac03-4e04-af4a-e9e986935245 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:37.369990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff27b71-ac03-4e04-af4a-e9e986935245 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:37.370488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff27b71-ac03-4e04-af4a-e9e986935245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:37.372941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff27b71-ac03-4e04-af4a-e9e986935245 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:37.379533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5713d0b8-e561-41d1-83d1-862f67f0199a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22489a6c355a49028ca16a394b27876f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:37.380287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5713d0b8-e561-41d1-83d1-862f67f0199a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22489a6c355a49028ca16a394b27876f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:37.386233 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1433/2895] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:37.389110 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1463/2896] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:59:37 2022] DELETE /identity/v3/users/22489a6c355a49028ca16a394b27876f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:37.397579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff27b71-ac03-4e04-af4a-e9e986935245 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:37.399660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-eff27b71-ac03-4e04-af4a-e9e986935245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:37.416286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff27b71-ac03-4e04-af4a-e9e986935245 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:37.496275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274754e-4731-4e23-bb7f-9f15d5030666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:37.497852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274754e-4731-4e23-bb7f-9f15d5030666 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:37.498104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274754e-4731-4e23-bb7f-9f15d5030666 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:37.498495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274754e-4731-4e23-bb7f-9f15d5030666 admin admin] PATH_INFO: `/v3/projects/c9247efa9b4f4d629272ddad62fd50e9/users/08ec4ea53cf84dec83652bf484ab577c/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:37.524546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274754e-4731-4e23-bb7f-9f15d5030666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9247efa9b4f4d629272ddad62fd50e9, user_id=08ec4ea53cf84dec83652bf484ab577c, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:37.525941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274754e-4731-4e23-bb7f-9f15d5030666 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:37.559741 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1434/2897] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:59:37 2022] PUT /identity/v3/projects/c9247efa9b4f4d629272ddad62fd50e9/users/08ec4ea53cf84dec83652bf484ab577c/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:37.561433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fd148-5cf0-4321-ad38-695809e066fa None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:37.561835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fd148-5cf0-4321-ad38-695809e066fa None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:37.562233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fd148-5cf0-4321-ad38-695809e066fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:37.597873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7c5fd148-5cf0-4321-ad38-695809e066fa None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:37.604098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c5fd148-5cf0-4321-ad38-695809e066fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:37.720029 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1464/2898] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:37.779077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fd148-5cf0-4321-ad38-695809e066fa None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:37.779560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274754e-4731-4e23-bb7f-9f15d5030666 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:37.781661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c5fd148-5cf0-4321-ad38-695809e066fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:37.784074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e274754e-4731-4e23-bb7f-9f15d5030666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:37.811059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274754e-4731-4e23-bb7f-9f15d5030666 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:37.847181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b0476-d0c4-4eb1-ae58-491144e3d27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:37.848704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b0476-d0c4-4eb1-ae58-491144e3d27a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:37.851860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b0476-d0c4-4eb1-ae58-491144e3d27a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:37.853016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b0476-d0c4-4eb1-ae58-491144e3d27a admin admin] PATH_INFO: `/v3/projects/8ac46d9ab1ca4815a59b43a819066e85` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:37.856245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b0476-d0c4-4eb1-ae58-491144e3d27a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac46d9ab1ca4815a59b43a819066e85)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:37.875376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b0476-d0c4-4eb1-ae58-491144e3d27a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:37.952122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fd148-5cf0-4321-ad38-695809e066fa None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:37.967773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-582b0476-d0c4-4eb1-ae58-491144e3d27a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac46d9ab1ca4815a59b43a819066e85', 'request_id': 'req-582b0476-d0c4-4eb1-ae58-491144e3d27a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:37.979175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-582b0476-d0c4-4eb1-ae58-491144e3d27a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac46d9ab1ca4815a59b43a819066e85', 'request_id': 'req-582b0476-d0c4-4eb1-ae58-491144e3d27a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:37.989763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:37.990621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-582b0476-d0c4-4eb1-ae58-491144e3d27a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac46d9ab1ca4815a59b43a819066e85', 'request_id': 'req-582b0476-d0c4-4eb1-ae58-491144e3d27a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:37.996001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:38.006724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-582b0476-d0c4-4eb1-ae58-491144e3d27a admin admin] The token cache is being invalidate because project 8ac46d9ab1ca4815a59b43a819066e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:59:38.007105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-582b0476-d0c4-4eb1-ae58-491144e3d27a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:38.007970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-582b0476-d0c4-4eb1-ae58-491144e3d27a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:38.009936 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1435/2899] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:59:37 2022] DELETE /identity/v3/projects/8ac46d9ab1ca4815a59b43a819066e85 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:38.014590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d290c-50ad-4e7c-bd50-b6ea1276f956 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:38.014899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d290c-50ad-4e7c-bd50-b6ea1276f956 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:38.015166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d290c-50ad-4e7c-bd50-b6ea1276f956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:38.025547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc65e52-cfdc-4da9-96c3-9376ebabb665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:38.026152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc65e52-cfdc-4da9-96c3-9376ebabb665 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:38.026416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc65e52-cfdc-4da9-96c3-9376ebabb665 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:38.026674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc65e52-cfdc-4da9-96c3-9376ebabb665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:38.030237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fbc65e52-cfdc-4da9-96c3-9376ebabb665 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:38.037884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-562d290c-50ad-4e7c-bd50-b6ea1276f956 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:38.039151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-562d290c-50ad-4e7c-bd50-b6ea1276f956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:38.107208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc65e52-cfdc-4da9-96c3-9376ebabb665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:38.108744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc65e52-cfdc-4da9-96c3-9376ebabb665 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:38.118945 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1436/2900] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:38.121814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fee7cb-e413-4846-9361-7bda620d4fbb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:38.122098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fee7cb-e413-4846-9361-7bda620d4fbb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:38.122368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fee7cb-e413-4846-9361-7bda620d4fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:38.124750 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1465/2901] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:38.147803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d3fee7cb-e413-4846-9361-7bda620d4fbb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:38.149319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d3fee7cb-e413-4846-9361-7bda620d4fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:38.161636 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1437/2902] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:38.165906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc65e52-cfdc-4da9-96c3-9376ebabb665 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:38.176114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc65e52-cfdc-4da9-96c3-9376ebabb665 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:38.181406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354964a-2e07-4ee9-8c87-eb0c8b2c3e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:38.182033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354964a-2e07-4ee9-8c87-eb0c8b2c3e78 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:38.182305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354964a-2e07-4ee9-8c87-eb0c8b2c3e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:38.182962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354964a-2e07-4ee9-8c87-eb0c8b2c3e78 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:38.183782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354964a-2e07-4ee9-8c87-eb0c8b2c3e78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:59:38.184056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354964a-2e07-4ee9-8c87-eb0c8b2c3e78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:38.186326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354964a-2e07-4ee9-8c87-eb0c8b2c3e78 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:38.206056 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1466/2903] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:59:38.210284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fee7cb-e413-4846-9361-7bda620d4fbb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:38.226327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fee7cb-e413-4846-9361-7bda620d4fbb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:38.242224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b058b5d-4b51-4958-a080-c720c29a19ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:38.243134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b058b5d-4b51-4958-a080-c720c29a19ae admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:38.243334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b058b5d-4b51-4958-a080-c720c29a19ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:38.243827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b058b5d-4b51-4958-a080-c720c29a19ae admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:38.244872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b058b5d-4b51-4958-a080-c720c29a19ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:38.247299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b058b5d-4b51-4958-a080-c720c29a19ae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:38.278490 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1438/2904] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:59:38 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:59:38.282688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354964a-2e07-4ee9-8c87-eb0c8b2c3e78 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:38.293311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354964a-2e07-4ee9-8c87-eb0c8b2c3e78 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:38.358589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b6807-63c5-42ff-9a51-abc5e2cf1419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:38.359281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b6807-63c5-42ff-9a51-abc5e2cf1419 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:38.359623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b6807-63c5-42ff-9a51-abc5e2cf1419 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:38.360112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b6807-63c5-42ff-9a51-abc5e2cf1419 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:38.360803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b6807-63c5-42ff-9a51-abc5e2cf1419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:38.362805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b6807-63c5-42ff-9a51-abc5e2cf1419 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:38.393663 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1467/2905] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:59:38 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:38.398875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b058b5d-4b51-4958-a080-c720c29a19ae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:38.412824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b058b5d-4b51-4958-a080-c720c29a19ae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:38.418601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cb413-49ad-40a6-bee7-0e61a5714fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:38.419546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cb413-49ad-40a6-bee7-0e61a5714fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:38.419886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cb413-49ad-40a6-bee7-0e61a5714fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:38.420249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cb413-49ad-40a6-bee7-0e61a5714fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:38.421724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cb413-49ad-40a6-bee7-0e61a5714fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:38.423894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cb413-49ad-40a6-bee7-0e61a5714fd9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:38.441937 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1439/2906] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:59:38.447157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b6807-63c5-42ff-9a51-abc5e2cf1419 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:38.459457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b6807-63c5-42ff-9a51-abc5e2cf1419 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:38.464198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca526a3-4c0c-461d-af7d-1c910b84fc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:38.464940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca526a3-4c0c-461d-af7d-1c910b84fc92 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:38.465244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca526a3-4c0c-461d-af7d-1c910b84fc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:38.465641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca526a3-4c0c-461d-af7d-1c910b84fc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:38.466289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca526a3-4c0c-461d-af7d-1c910b84fc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:38.469102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca526a3-4c0c-461d-af7d-1c910b84fc92 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:38.480265 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1468/2907] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:59:38.485084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cb413-49ad-40a6-bee7-0e61a5714fd9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:38.497657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cb413-49ad-40a6-bee7-0e61a5714fd9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:38.503175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b925ad8-724d-4048-b9a9-25f50d184845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:38.510079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b925ad8-724d-4048-b9a9-25f50d184845 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:38.510370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b925ad8-724d-4048-b9a9-25f50d184845 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:38.510646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b925ad8-724d-4048-b9a9-25f50d184845 admin admin] PATH_INFO: `/v3/projects/c8a0819e11dd459796712d37c8ed0f49/users/303ece907c2a44b2afe950f039df2c37/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:38.533078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b925ad8-724d-4048-b9a9-25f50d184845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8a0819e11dd459796712d37c8ed0f49, user_id=303ece907c2a44b2afe950f039df2c37, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:38.537350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b925ad8-724d-4048-b9a9-25f50d184845 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:38.566023 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1440/2908] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:59:38 2022] PUT /identity/v3/projects/c8a0819e11dd459796712d37c8ed0f49/users/303ece907c2a44b2afe950f039df2c37/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:59:38.572737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca7ab7-e553-4f0f-92f9-89f1e3c1c399 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:38.573277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca7ab7-e553-4f0f-92f9-89f1e3c1c399 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:38.573750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca7ab7-e553-4f0f-92f9-89f1e3c1c399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:38.602920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8cca7ab7-e553-4f0f-92f9-89f1e3c1c399 None None] MFA Rules not processed for user `303ece907c2a44b2afe950f039df2c37`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:38.604222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8cca7ab7-e553-4f0f-92f9-89f1e3c1c399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:38.644046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:38.682933 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1469/2909] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:38.693931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b925ad8-724d-4048-b9a9-25f50d184845 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:38.696408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3b925ad8-724d-4048-b9a9-25f50d184845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:38.774747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b925ad8-724d-4048-b9a9-25f50d184845 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:38.782099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1e2d5-ad18-48d5-9f73-54732889fe1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:38.782725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1e2d5-ad18-48d5-9f73-54732889fe1f service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:38.782980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1e2d5-ad18-48d5-9f73-54732889fe1f service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:38.783305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1e2d5-ad18-48d5-9f73-54732889fe1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:38.794508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1e2d5-ad18-48d5-9f73-54732889fe1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:38.796625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1e2d5-ad18-48d5-9f73-54732889fe1f service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:38.807869 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1441/2910] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:39.167308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca7ab7-e553-4f0f-92f9-89f1e3c1c399 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:39.274377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca7ab7-e553-4f0f-92f9-89f1e3c1c399 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:39.281220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:39.281907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:39.283145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:39.283871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] PATH_INFO: `/v3/users/303ece907c2a44b2afe950f039df2c37` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:39.287649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=303ece907c2a44b2afe950f039df2c37)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:39.290116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:39.358690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303ece907c2a44b2afe950f039df2c37'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:39.372598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303ece907c2a44b2afe950f039df2c37'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:39.380766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303ece907c2a44b2afe950f039df2c37'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:39.397839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303ece907c2a44b2afe950f039df2c37'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:39.399112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303ece907c2a44b2afe950f039df2c37'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:39.405644 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1470/2911] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:59:39 2022] DELETE /identity/v3/users/303ece907c2a44b2afe950f039df2c37 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:39.413835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebe153-049b-494b-b988-4b721dd7e988 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:39.416604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebe153-049b-494b-b988-4b721dd7e988 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:39.417938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebe153-049b-494b-b988-4b721dd7e988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:39.456835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2bebe153-049b-494b-b988-4b721dd7e988 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:39.459042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2bebe153-049b-494b-b988-4b721dd7e988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:39.472273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:39.589220 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1442/2912] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:39.594220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:39.595594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:39.696979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:39.701327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd65ab-8d42-44e9-9348-ba20b6847554 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:39.708697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd68504-8a02-43b3-8351-7a5ccf74d428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:39.709775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd68504-8a02-43b3-8351-7a5ccf74d428 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:39.710233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd68504-8a02-43b3-8351-7a5ccf74d428 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:39.711033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd68504-8a02-43b3-8351-7a5ccf74d428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:39.725108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd68504-8a02-43b3-8351-7a5ccf74d428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:39.727668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd68504-8a02-43b3-8351-7a5ccf74d428 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:39.740020 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1471/2913] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:40.537000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebe153-049b-494b-b988-4b721dd7e988 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:40.538750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2bebe153-049b-494b-b988-4b721dd7e988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:40.572916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebe153-049b-494b-b988-4b721dd7e988 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:40.588168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30a671-6ca3-4c4c-8727-0418bffee76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:40.588810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30a671-6ca3-4c4c-8727-0418bffee76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:40.589088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30a671-6ca3-4c4c-8727-0418bffee76e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:40.589400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30a671-6ca3-4c4c-8727-0418bffee76e admin admin] PATH_INFO: `/v3/projects/c8a0819e11dd459796712d37c8ed0f49` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:40.590770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30a671-6ca3-4c4c-8727-0418bffee76e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8a0819e11dd459796712d37c8ed0f49)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:40.592783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30a671-6ca3-4c4c-8727-0418bffee76e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:40.661392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5b30a671-6ca3-4c4c-8727-0418bffee76e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8a0819e11dd459796712d37c8ed0f49', 'request_id': 'req-5b30a671-6ca3-4c4c-8727-0418bffee76e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:40.669280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5b30a671-6ca3-4c4c-8727-0418bffee76e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8a0819e11dd459796712d37c8ed0f49', 'request_id': 'req-5b30a671-6ca3-4c4c-8727-0418bffee76e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:40.678754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5b30a671-6ca3-4c4c-8727-0418bffee76e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8a0819e11dd459796712d37c8ed0f49', 'request_id': 'req-5b30a671-6ca3-4c4c-8727-0418bffee76e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:40.692646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5b30a671-6ca3-4c4c-8727-0418bffee76e admin admin] The token cache is being invalidate because project c8a0819e11dd459796712d37c8ed0f49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:59:40.692957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5b30a671-6ca3-4c4c-8727-0418bffee76e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:40.693593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5b30a671-6ca3-4c4c-8727-0418bffee76e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:40.696463 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1443/2914] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:59:40 2022] DELETE /identity/v3/projects/c8a0819e11dd459796712d37c8ed0f49 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:40.701217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b23e-de8f-4d59-976b-b547c347d7fb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:40.702198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b23e-de8f-4d59-976b-b547c347d7fb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:40.702430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b23e-de8f-4d59-976b-b547c347d7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:40.753569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b220b23e-de8f-4d59-976b-b547c347d7fb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:40.757296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b220b23e-de8f-4d59-976b-b547c347d7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:40.845178 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1472/2915] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 20:59:40.847529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d767a4-fb8c-4c90-8d79-06b9d4192725 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:40.847812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d767a4-fb8c-4c90-8d79-06b9d4192725 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:40.848076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d767a4-fb8c-4c90-8d79-06b9d4192725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:40.909686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-03d767a4-fb8c-4c90-8d79-06b9d4192725 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:40.911572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-03d767a4-fb8c-4c90-8d79-06b9d4192725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:40.923892 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1444/2916] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:40.928333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220b23e-de8f-4d59-976b-b547c347d7fb None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:40.939246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220b23e-de8f-4d59-976b-b547c347d7fb None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:40.944252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c096421-ecce-4794-ae57-aa5ae5cb13dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:40.944908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c096421-ecce-4794-ae57-aa5ae5cb13dc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:40.945170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c096421-ecce-4794-ae57-aa5ae5cb13dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:40.945422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c096421-ecce-4794-ae57-aa5ae5cb13dc admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:40.946201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c096421-ecce-4794-ae57-aa5ae5cb13dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:59:40.946377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c096421-ecce-4794-ae57-aa5ae5cb13dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:40.948101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c096421-ecce-4794-ae57-aa5ae5cb13dc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:40.974497 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1473/2917] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:59:40.978997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d767a4-fb8c-4c90-8d79-06b9d4192725 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:40.992649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d767a4-fb8c-4c90-8d79-06b9d4192725 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:41.000513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b248fe2-743c-4b4a-9170-b285d313f2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:41.001261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248fe2-743c-4b4a-9170-b285d313f2be admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:41.001637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248fe2-743c-4b4a-9170-b285d313f2be admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:41.002000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248fe2-743c-4b4a-9170-b285d313f2be admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:41.002886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b248fe2-743c-4b4a-9170-b285d313f2be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:41.005178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b248fe2-743c-4b4a-9170-b285d313f2be admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:41.052295 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1445/2918] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:59:40 2022] POST /identity/v3/projects => generated 389 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:41.053677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c096421-ecce-4794-ae57-aa5ae5cb13dc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:41.064852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c096421-ecce-4794-ae57-aa5ae5cb13dc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:41.134106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a477fd-d22c-45ed-8f97-74c6a77b103a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:41.135042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a477fd-d22c-45ed-8f97-74c6a77b103a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:41.135383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a477fd-d22c-45ed-8f97-74c6a77b103a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:41.135719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a477fd-d22c-45ed-8f97-74c6a77b103a admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:41.136720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a477fd-d22c-45ed-8f97-74c6a77b103a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:41.138884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a477fd-d22c-45ed-8f97-74c6a77b103a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:41.170478 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1474/2919] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:59:41 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:41.174818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b248fe2-743c-4b4a-9170-b285d313f2be admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:41.185759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b248fe2-743c-4b4a-9170-b285d313f2be admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:41.190351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352430e-4ca0-4cd2-b2b8-7fa410e7d84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:41.190961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352430e-4ca0-4cd2-b2b8-7fa410e7d84a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:41.191231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352430e-4ca0-4cd2-b2b8-7fa410e7d84a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:41.191496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352430e-4ca0-4cd2-b2b8-7fa410e7d84a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:41.192135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352430e-4ca0-4cd2-b2b8-7fa410e7d84a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:41.193753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352430e-4ca0-4cd2-b2b8-7fa410e7d84a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:41.210618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a477fd-d22c-45ed-8f97-74c6a77b103a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:41.211057 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1446/2920] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:41.221494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a477fd-d22c-45ed-8f97-74c6a77b103a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:41.238000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74d906-f09d-42ba-a9fc-29489751ba40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:41.238728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74d906-f09d-42ba-a9fc-29489751ba40 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:41.239000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74d906-f09d-42ba-a9fc-29489751ba40 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:41.239818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74d906-f09d-42ba-a9fc-29489751ba40 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:41.240463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74d906-f09d-42ba-a9fc-29489751ba40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:41.242570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74d906-f09d-42ba-a9fc-29489751ba40 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:41.265247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352430e-4ca0-4cd2-b2b8-7fa410e7d84a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:41.270683 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1475/2921] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:41.277915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352430e-4ca0-4cd2-b2b8-7fa410e7d84a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:41.285255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c11522-e4e7-4ea0-8fd4-8d5063de2b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:41.286299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c11522-e4e7-4ea0-8fd4-8d5063de2b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:41.286669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c11522-e4e7-4ea0-8fd4-8d5063de2b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:41.286965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c11522-e4e7-4ea0-8fd4-8d5063de2b9d admin admin] PATH_INFO: `/v3/projects/9848ccbab06c45ae822845161f1a4f91/users/df6f322961b4407cbba93e930784df3a/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:41.306764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c11522-e4e7-4ea0-8fd4-8d5063de2b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9848ccbab06c45ae822845161f1a4f91, user_id=df6f322961b4407cbba93e930784df3a, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:41.310834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c11522-e4e7-4ea0-8fd4-8d5063de2b9d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:41.328927 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1447/2922] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:59:41 2022] PUT /identity/v3/projects/9848ccbab06c45ae822845161f1a4f91/users/df6f322961b4407cbba93e930784df3a/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:59:41.334433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aba9e-77a0-4101-ab41-04bc8780e23e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:41.334779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aba9e-77a0-4101-ab41-04bc8780e23e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:41.335105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aba9e-77a0-4101-ab41-04bc8780e23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:41.359991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ea1aba9e-77a0-4101-ab41-04bc8780e23e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:41.361556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ea1aba9e-77a0-4101-ab41-04bc8780e23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:41.454736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:41.475362 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1476/2923] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:41.483596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c11522-e4e7-4ea0-8fd4-8d5063de2b9d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:41.486899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-54c11522-e4e7-4ea0-8fd4-8d5063de2b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:41.548152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:41.583416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c11522-e4e7-4ea0-8fd4-8d5063de2b9d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:41.597632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1165389-90f3-4334-a76a-b5853c1a3660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:41.599465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1165389-90f3-4334-a76a-b5853c1a3660 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:41.599841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1165389-90f3-4334-a76a-b5853c1a3660 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:41.600234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1165389-90f3-4334-a76a-b5853c1a3660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:41.625485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1165389-90f3-4334-a76a-b5853c1a3660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:41.627330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1165389-90f3-4334-a76a-b5853c1a3660 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:41.644335 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1448/2924] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:42.997369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aba9e-77a0-4101-ab41-04bc8780e23e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:43.000336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ea1aba9e-77a0-4101-ab41-04bc8780e23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:43.016484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aba9e-77a0-4101-ab41-04bc8780e23e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:43.022069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:43.022764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:43.023036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:43.023302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] PATH_INFO: `/v3/users/0a583f93e79843448f5eb36b844b9eac` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:43.028012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a583f93e79843448f5eb36b844b9eac)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:43.030159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:43.096885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a583f93e79843448f5eb36b844b9eac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:43.110793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a583f93e79843448f5eb36b844b9eac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:43.136838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a583f93e79843448f5eb36b844b9eac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:43.175710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a583f93e79843448f5eb36b844b9eac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:43.176270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a583f93e79843448f5eb36b844b9eac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:43.187014 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1477/2925] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:59:42 2022] DELETE /identity/v3/users/0a583f93e79843448f5eb36b844b9eac => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:43.189546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cec1f-fe4a-4522-90b9-e0074549fb1a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:43.190493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cec1f-fe4a-4522-90b9-e0074549fb1a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:43.190937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cec1f-fe4a-4522-90b9-e0074549fb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:43.229665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-851cec1f-fe4a-4522-90b9-e0074549fb1a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:43.231193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-851cec1f-fe4a-4522-90b9-e0074549fb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:43.412111 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1449/2926] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:43.423378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:43.425024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:43.555129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bb1e3-57fc-44e1-bed0-c9f52091ac7d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:43.558976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730247f-f94c-4166-a397-0683698a61ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:43.559754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730247f-f94c-4166-a397-0683698a61ff service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:43.560124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730247f-f94c-4166-a397-0683698a61ff service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:43.560527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730247f-f94c-4166-a397-0683698a61ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:43.580630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730247f-f94c-4166-a397-0683698a61ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:43.583554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730247f-f94c-4166-a397-0683698a61ff service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:43.610870 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1478/2927] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:43.933309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cec1f-fe4a-4522-90b9-e0074549fb1a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:43.935945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-851cec1f-fe4a-4522-90b9-e0074549fb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:43.956294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cec1f-fe4a-4522-90b9-e0074549fb1a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:43.970929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:43.971892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:43.972213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:43.973811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] PATH_INFO: `/v3/projects/b4c402c5e1ba421290728f887fa30b9a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:43.975343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4c402c5e1ba421290728f887fa30b9a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:43.980943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:44.084165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4c402c5e1ba421290728f887fa30b9a', 'request_id': 'req-f0ca3f0d-17a6-4de0-90cd-8b670900f041'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:44.094968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4c402c5e1ba421290728f887fa30b9a', 'request_id': 'req-f0ca3f0d-17a6-4de0-90cd-8b670900f041'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:44.120647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4c402c5e1ba421290728f887fa30b9a', 'request_id': 'req-f0ca3f0d-17a6-4de0-90cd-8b670900f041'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:44.138317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] The token cache is being invalidate because project b4c402c5e1ba421290728f887fa30b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:59:44.141036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:44.142479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:44.147770 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1450/2928] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:59:43 2022] DELETE /identity/v3/projects/b4c402c5e1ba421290728f887fa30b9a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:44.149112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730247f-f94c-4166-a397-0683698a61ff service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:44.175172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730247f-f94c-4166-a397-0683698a61ff service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:44.307301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5de3e-093f-4b42-85b8-be50328d1a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:44.307951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5de3e-093f-4b42-85b8-be50328d1a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:44.308260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5de3e-093f-4b42-85b8-be50328d1a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:44.308607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5de3e-093f-4b42-85b8-be50328d1a64 admin admin] PATH_INFO: `/v3/users/3896e4fc6cb5405882082d0eb22f7e23` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:44.309924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5de3e-093f-4b42-85b8-be50328d1a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3896e4fc6cb5405882082d0eb22f7e23)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:44.311843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5de3e-093f-4b42-85b8-be50328d1a64 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:44.330734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:44.427945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e4a5de3e-093f-4b42-85b8-be50328d1a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3896e4fc6cb5405882082d0eb22f7e23'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:44.447980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e4a5de3e-093f-4b42-85b8-be50328d1a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3896e4fc6cb5405882082d0eb22f7e23'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:44.484082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e4a5de3e-093f-4b42-85b8-be50328d1a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3896e4fc6cb5405882082d0eb22f7e23'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:44.522546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e4a5de3e-093f-4b42-85b8-be50328d1a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3896e4fc6cb5405882082d0eb22f7e23'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:44.523296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e4a5de3e-093f-4b42-85b8-be50328d1a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3896e4fc6cb5405882082d0eb22f7e23'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:44.531775 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1479/2929] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 20:59:44 2022] DELETE /identity/v3/users/3896e4fc6cb5405882082d0eb22f7e23 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 20:59:45.053557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:45.055574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:45.071796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca3f0d-17a6-4de0-90cd-8b670900f041 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:45.150853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507fcd7-126f-4c79-9f3e-4dd5a5434705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:45.151779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507fcd7-126f-4c79-9f3e-4dd5a5434705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:45.152157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507fcd7-126f-4c79-9f3e-4dd5a5434705 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:45.154755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507fcd7-126f-4c79-9f3e-4dd5a5434705 admin admin] PATH_INFO: `/v3/projects/5d5c9b69be95428ca0e839de0877bf36` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:45.156291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507fcd7-126f-4c79-9f3e-4dd5a5434705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d5c9b69be95428ca0e839de0877bf36)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:45.158489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507fcd7-126f-4c79-9f3e-4dd5a5434705 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:45.161805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:45.221667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7507fcd7-126f-4c79-9f3e-4dd5a5434705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d5c9b69be95428ca0e839de0877bf36', 'request_id': 'req-7507fcd7-126f-4c79-9f3e-4dd5a5434705'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:45.241167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7507fcd7-126f-4c79-9f3e-4dd5a5434705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d5c9b69be95428ca0e839de0877bf36', 'request_id': 'req-7507fcd7-126f-4c79-9f3e-4dd5a5434705'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:45.266323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7507fcd7-126f-4c79-9f3e-4dd5a5434705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d5c9b69be95428ca0e839de0877bf36', 'request_id': 'req-7507fcd7-126f-4c79-9f3e-4dd5a5434705'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:45.284480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7507fcd7-126f-4c79-9f3e-4dd5a5434705 admin admin] The token cache is being invalidate because project 5d5c9b69be95428ca0e839de0877bf36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 20:59:45.284789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7507fcd7-126f-4c79-9f3e-4dd5a5434705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:45.285454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7507fcd7-126f-4c79-9f3e-4dd5a5434705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 20:59:45.293459 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1451/2930] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 20:59:45 2022] DELETE /identity/v3/projects/5d5c9b69be95428ca0e839de0877bf36 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:45.295015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ed909-6d5e-47eb-bc71-8ba6a13dd46b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:45.295525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ed909-6d5e-47eb-bc71-8ba6a13dd46b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:45.295525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ed909-6d5e-47eb-bc71-8ba6a13dd46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:45.319065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-680ed909-6d5e-47eb-bc71-8ba6a13dd46b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:45.321148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-680ed909-6d5e-47eb-bc71-8ba6a13dd46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:45.421260 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1480/2931] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:45.430338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e368-c41d-46d9-ada0-57656508b0e7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:45.430615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e368-c41d-46d9-ada0-57656508b0e7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:45.431307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e368-c41d-46d9-ada0-57656508b0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:45.462247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4ed9e368-c41d-46d9-ada0-57656508b0e7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:45.463991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4ed9e368-c41d-46d9-ada0-57656508b0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:45.485123 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1452/2932] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:45.485510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ed909-6d5e-47eb-bc71-8ba6a13dd46b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:45.503061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ed909-6d5e-47eb-bc71-8ba6a13dd46b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:45.508354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff95ff7-09da-4ffd-aa8d-76adac19fd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:45.509183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff95ff7-09da-4ffd-aa8d-76adac19fd91 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:45.509551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff95ff7-09da-4ffd-aa8d-76adac19fd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:45.509905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff95ff7-09da-4ffd-aa8d-76adac19fd91 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:45.510661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff95ff7-09da-4ffd-aa8d-76adac19fd91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 20:59:45.511049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff95ff7-09da-4ffd-aa8d-76adac19fd91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:45.512962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff95ff7-09da-4ffd-aa8d-76adac19fd91 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:45.537942 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1481/2933] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 20:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 20:59:45.538899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9e368-c41d-46d9-ada0-57656508b0e7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:45.551659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9e368-c41d-46d9-ada0-57656508b0e7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:45.559006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f922bc7-c53b-4b7d-a814-0501c3f85787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:45.559800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922bc7-c53b-4b7d-a814-0501c3f85787 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:45.560068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922bc7-c53b-4b7d-a814-0501c3f85787 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:45.560555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922bc7-c53b-4b7d-a814-0501c3f85787 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:45.561418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f922bc7-c53b-4b7d-a814-0501c3f85787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:45.585894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f922bc7-c53b-4b7d-a814-0501c3f85787 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:45.654605 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1453/2934] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 20:59:45 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 20:59:45.659039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff95ff7-09da-4ffd-aa8d-76adac19fd91 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:45.710198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff95ff7-09da-4ffd-aa8d-76adac19fd91 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:45.798181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bade491-1114-4f87-84ab-c612454933bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:45.799395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bade491-1114-4f87-84ab-c612454933bc admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:45.799395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bade491-1114-4f87-84ab-c612454933bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:45.799548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bade491-1114-4f87-84ab-c612454933bc admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:45.800169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bade491-1114-4f87-84ab-c612454933bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:45.802610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bade491-1114-4f87-84ab-c612454933bc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:45.838752 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1482/2935] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 20:59:45 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 20:59:45.839754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f922bc7-c53b-4b7d-a814-0501c3f85787 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:45.856903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f922bc7-c53b-4b7d-a814-0501c3f85787 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:45.874273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f88543-edcb-41bf-a69d-69a0f6fba21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:45.875347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88543-edcb-41bf-a69d-69a0f6fba21b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:45.875626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88543-edcb-41bf-a69d-69a0f6fba21b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:45.878589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88543-edcb-41bf-a69d-69a0f6fba21b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:45.879498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f88543-edcb-41bf-a69d-69a0f6fba21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:45.886307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f88543-edcb-41bf-a69d-69a0f6fba21b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:45.924425 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1454/2936] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 20:59:45.928782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bade491-1114-4f87-84ab-c612454933bc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:45.946216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bade491-1114-4f87-84ab-c612454933bc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:45.963421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052f672-b9f3-43c3-a1ba-94d6df826f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:45.964247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052f672-b9f3-43c3-a1ba-94d6df826f85 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:45.964552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052f672-b9f3-43c3-a1ba-94d6df826f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:45.964857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052f672-b9f3-43c3-a1ba-94d6df826f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:45.965955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052f672-b9f3-43c3-a1ba-94d6df826f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:45.967906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052f672-b9f3-43c3-a1ba-94d6df826f85 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:45.977384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:46.005420 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1483/2937] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 20:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 20:59:46.006702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f88543-edcb-41bf-a69d-69a0f6fba21b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:46.026612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f88543-edcb-41bf-a69d-69a0f6fba21b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:46.037737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ddd0a-00a7-495c-9541-29f7ca99f73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:46.038519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ddd0a-00a7-495c-9541-29f7ca99f73a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:46.038778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ddd0a-00a7-495c-9541-29f7ca99f73a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:46.039045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ddd0a-00a7-495c-9541-29f7ca99f73a admin admin] PATH_INFO: `/v3/projects/7ef75542615640b1ab4d97a6f0708540/users/b2ba80bfab064d71a80521c91ccf13c8/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:46.104248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ddd0a-00a7-495c-9541-29f7ca99f73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef75542615640b1ab4d97a6f0708540, user_id=b2ba80bfab064d71a80521c91ccf13c8, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:46.118123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ddd0a-00a7-495c-9541-29f7ca99f73a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:46.139387 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1455/2938] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 20:59:46 2022] PUT /identity/v3/projects/7ef75542615640b1ab4d97a6f0708540/users/b2ba80bfab064d71a80521c91ccf13c8/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 20:59:46.141027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79576de7-0b66-4083-a874-5b728dd49480 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:46.141408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79576de7-0b66-4083-a874-5b728dd49480 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:46.141773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79576de7-0b66-4083-a874-5b728dd49480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:46.187341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-79576de7-0b66-4083-a874-5b728dd49480 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:46.189325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-79576de7-0b66-4083-a874-5b728dd49480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:46.291779 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1484/2939] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:46.301656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ddd0a-00a7-495c-9541-29f7ca99f73a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:46.303485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fe3ddd0a-00a7-495c-9541-29f7ca99f73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:46.376869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ddd0a-00a7-495c-9541-29f7ca99f73a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:46.380683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c48158-9653-4683-a0c2-a8e4c0d8e548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:46.381675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c48158-9653-4683-a0c2-a8e4c0d8e548 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:46.382114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c48158-9653-4683-a0c2-a8e4c0d8e548 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:46.382541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c48158-9653-4683-a0c2-a8e4c0d8e548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:46.394983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c48158-9653-4683-a0c2-a8e4c0d8e548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:46.396539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c48158-9653-4683-a0c2-a8e4c0d8e548 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:46.409548 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1456/2940] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:46.793911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59037cbb-23c9-4bff-b19c-4d8fdc217919 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:46.794460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59037cbb-23c9-4bff-b19c-4d8fdc217919 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:46.794460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59037cbb-23c9-4bff-b19c-4d8fdc217919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:46.845386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-59037cbb-23c9-4bff-b19c-4d8fdc217919 None None] MFA Rules not processed for user `08ec4ea53cf84dec83652bf484ab577c`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:46.847769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-59037cbb-23c9-4bff-b19c-4d8fdc217919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:46.950758 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1485/2941] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:46.961390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40903156-d61f-4553-8fe2-ceffa63eceeb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:46.961745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40903156-d61f-4553-8fe2-ceffa63eceeb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:46.962072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40903156-d61f-4553-8fe2-ceffa63eceeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:46.994549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-40903156-d61f-4553-8fe2-ceffa63eceeb None None] MFA Rules not processed for user `08ec4ea53cf84dec83652bf484ab577c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:46.996622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-40903156-d61f-4553-8fe2-ceffa63eceeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:47.025172 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1457/2942] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:47.038253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59037cbb-23c9-4bff-b19c-4d8fdc217919 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:47.041150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-59037cbb-23c9-4bff-b19c-4d8fdc217919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:47.195006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59037cbb-23c9-4bff-b19c-4d8fdc217919 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:47.204192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ce757-919d-421b-b294-7fcfc4b6e9a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:47.205136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ce757-919d-421b-b294-7fcfc4b6e9a3 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:47.205136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ce757-919d-421b-b294-7fcfc4b6e9a3 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:47.205465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ce757-919d-421b-b294-7fcfc4b6e9a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:47.217653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ce757-919d-421b-b294-7fcfc4b6e9a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:47.219904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ce757-919d-421b-b294-7fcfc4b6e9a3 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:47.243952 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1486/2943] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:49.836065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40903156-d61f-4553-8fe2-ceffa63eceeb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:49.877196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40903156-d61f-4553-8fe2-ceffa63eceeb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:49.897151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac5403-1316-4d82-9452-c54d4ecf2519 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:49.898358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac5403-1316-4d82-9452-c54d4ecf2519 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:49.898963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac5403-1316-4d82-9452-c54d4ecf2519 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:49.899535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac5403-1316-4d82-9452-c54d4ecf2519 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:49.905702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-12ac5403-1316-4d82-9452-c54d4ecf2519 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:49.976441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:50.122417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac5403-1316-4d82-9452-c54d4ecf2519 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:50.124096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac5403-1316-4d82-9452-c54d4ecf2519 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:50.135798 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1458/2944] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:50.314205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ce757-919d-421b-b294-7fcfc4b6e9a3 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:50.330229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ce757-919d-421b-b294-7fcfc4b6e9a3 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:50.334584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0c1ea-f16e-4706-9ce5-9e12e8a13473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:50.335574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0c1ea-f16e-4706-9ce5-9e12e8a13473 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:50.335870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0c1ea-f16e-4706-9ce5-9e12e8a13473 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:50.336319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0c1ea-f16e-4706-9ce5-9e12e8a13473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:50.338369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f9f0c1ea-f16e-4706-9ce5-9e12e8a13473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:50.358597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0c1ea-f16e-4706-9ce5-9e12e8a13473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:50.360361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0c1ea-f16e-4706-9ce5-9e12e8a13473 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:50.377581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 20:59:50.380909 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1487/2945] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:52.566218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac5403-1316-4d82-9452-c54d4ecf2519 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:52.582192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac5403-1316-4d82-9452-c54d4ecf2519 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:52.595397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d808ee-2426-4729-a39e-7b006ea6e791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:52.597625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d808ee-2426-4729-a39e-7b006ea6e791 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:52.600037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d808ee-2426-4729-a39e-7b006ea6e791 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:52.600620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d808ee-2426-4729-a39e-7b006ea6e791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:52.610781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c5d808ee-2426-4729-a39e-7b006ea6e791 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:52.666505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9cff1-042d-441b-b949-68c16daa3ecf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:52.667856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9cff1-042d-441b-b949-68c16daa3ecf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:52.668433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9cff1-042d-441b-b949-68c16daa3ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:52.679862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d808ee-2426-4729-a39e-7b006ea6e791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:52.687645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d808ee-2426-4729-a39e-7b006ea6e791 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:52.729497 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1459/2946] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:52.781501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bab9cff1-042d-441b-b949-68c16daa3ecf None None] MFA Rules not processed for user `df6f322961b4407cbba93e930784df3a`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:52.784664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bab9cff1-042d-441b-b949-68c16daa3ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:53.003175 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1488/2947] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:53.255932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d808ee-2426-4729-a39e-7b006ea6e791 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:53.262843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c5d808ee-2426-4729-a39e-7b006ea6e791 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:53.285918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d808ee-2426-4729-a39e-7b006ea6e791 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:53.297841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b2c09-d8ad-4f41-a09e-d557205ade01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:53.298630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b2c09-d8ad-4f41-a09e-d557205ade01 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:53.299379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b2c09-d8ad-4f41-a09e-d557205ade01 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:53.299686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b2c09-d8ad-4f41-a09e-d557205ade01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:53.310030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b2c09-d8ad-4f41-a09e-d557205ade01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:53.312024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b2c09-d8ad-4f41-a09e-d557205ade01 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:53.325629 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1460/2948] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 20:59:58.913959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9f158-550a-4edf-b3b7-70b0c151b5fd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:58.914518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9f158-550a-4edf-b3b7-70b0c151b5fd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:58.914518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9f158-550a-4edf-b3b7-70b0c151b5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:58.948484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b7f9f158-550a-4edf-b3b7-70b0c151b5fd None None] MFA Rules not processed for user `b2ba80bfab064d71a80521c91ccf13c8`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 20:59:58.949965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b7f9f158-550a-4edf-b3b7-70b0c151b5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 20:59:59.086779 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1489/2949] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 20:59:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 20:59:59.289560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b2c09-d8ad-4f41-a09e-d557205ade01 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 20:59:59.316352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b2c09-d8ad-4f41-a09e-d557205ade01 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 20:59:59.327471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22b538-a4a1-474a-91bd-5e71bda40bbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 20:59:59.328092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22b538-a4a1-474a-91bd-5e71bda40bbf service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 20:59:59.328361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22b538-a4a1-474a-91bd-5e71bda40bbf service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 20:59:59.328656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22b538-a4a1-474a-91bd-5e71bda40bbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 20:59:59.339890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22b538-a4a1-474a-91bd-5e71bda40bbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 20:59:59.341894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22b538-a4a1-474a-91bd-5e71bda40bbf service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 20:59:59.354138 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1461/2950] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 20:59:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:00.807120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9f158-550a-4edf-b3b7-70b0c151b5fd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:00.808601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b7f9f158-550a-4edf-b3b7-70b0c151b5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:00.821427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9f158-550a-4edf-b3b7-70b0c151b5fd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:00.827347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58435740-d02d-4f9e-a1c9-55d825a0e265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:00.828202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58435740-d02d-4f9e-a1c9-55d825a0e265 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:00.828781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58435740-d02d-4f9e-a1c9-55d825a0e265 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:00.829214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58435740-d02d-4f9e-a1c9-55d825a0e265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:00.831140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-58435740-d02d-4f9e-a1c9-55d825a0e265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:00.841682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58435740-d02d-4f9e-a1c9-55d825a0e265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:00.843464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58435740-d02d-4f9e-a1c9-55d825a0e265 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:00.852585 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1490/2951] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:09.349511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=37 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:10.825052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22b538-a4a1-474a-91bd-5e71bda40bbf service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:10.840958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22b538-a4a1-474a-91bd-5e71bda40bbf service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:10.844998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68fa75d-3705-4b28-bcd6-6a337d3aef87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:10.846431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fa75d-3705-4b28-bcd6-6a337d3aef87 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:10.846888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fa75d-3705-4b28-bcd6-6a337d3aef87 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:10.847324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fa75d-3705-4b28-bcd6-6a337d3aef87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:10.848995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=70 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:10.850697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a68fa75d-3705-4b28-bcd6-6a337d3aef87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:10.872732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68fa75d-3705-4b28-bcd6-6a337d3aef87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:10.883404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68fa75d-3705-4b28-bcd6-6a337d3aef87 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:10.905291 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1462/2952] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:13.144072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58435740-d02d-4f9e-a1c9-55d825a0e265 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:13.148456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-58435740-d02d-4f9e-a1c9-55d825a0e265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:13.178044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58435740-d02d-4f9e-a1c9-55d825a0e265 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:13.184581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:13.185685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:13.186017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:13.186345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] PATH_INFO: `/v3/users/df6f322961b4407cbba93e930784df3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:13.188600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df6f322961b4407cbba93e930784df3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:13.192709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:13.276561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df6f322961b4407cbba93e930784df3a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:13.286891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df6f322961b4407cbba93e930784df3a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:13.295621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df6f322961b4407cbba93e930784df3a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:13.316263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df6f322961b4407cbba93e930784df3a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:13.317170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df6f322961b4407cbba93e930784df3a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:13.325310 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1491/2953] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:00:13 2022] DELETE /identity/v3/users/df6f322961b4407cbba93e930784df3a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:13.574744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68fa75d-3705-4b28-bcd6-6a337d3aef87 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:13.594714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68fa75d-3705-4b28-bcd6-6a337d3aef87 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:13.672441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a25681-f8a2-4c4a-94b6-e70237046ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:13.673489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a25681-f8a2-4c4a-94b6-e70237046ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:13.673845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a25681-f8a2-4c4a-94b6-e70237046ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:13.674254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a25681-f8a2-4c4a-94b6-e70237046ebc admin admin] PATH_INFO: `/v3/projects/9848ccbab06c45ae822845161f1a4f91` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:13.675882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a25681-f8a2-4c4a-94b6-e70237046ebc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9848ccbab06c45ae822845161f1a4f91)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:13.678087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a25681-f8a2-4c4a-94b6-e70237046ebc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:13.743537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-62a25681-f8a2-4c4a-94b6-e70237046ebc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9848ccbab06c45ae822845161f1a4f91', 'request_id': 'req-62a25681-f8a2-4c4a-94b6-e70237046ebc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:13.772230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:13.773830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:13.791841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-62a25681-f8a2-4c4a-94b6-e70237046ebc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9848ccbab06c45ae822845161f1a4f91', 'request_id': 'req-62a25681-f8a2-4c4a-94b6-e70237046ebc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:13.798007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c6785-3361-4ff8-9888-1f4928b55624 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:13.817949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-62a25681-f8a2-4c4a-94b6-e70237046ebc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9848ccbab06c45ae822845161f1a4f91', 'request_id': 'req-62a25681-f8a2-4c4a-94b6-e70237046ebc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:13.845634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-62a25681-f8a2-4c4a-94b6-e70237046ebc admin admin] The token cache is being invalidate because project 9848ccbab06c45ae822845161f1a4f91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:00:13.848300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-62a25681-f8a2-4c4a-94b6-e70237046ebc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:13.849125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-62a25681-f8a2-4c4a-94b6-e70237046ebc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:13.856756 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1463/2954] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:00:13 2022] DELETE /identity/v3/projects/9848ccbab06c45ae822845161f1a4f91 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:13.862649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f5da8-a86d-48b1-a07b-16399ad14395 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:13.862924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f5da8-a86d-48b1-a07b-16399ad14395 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:13.863188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f5da8-a86d-48b1-a07b-16399ad14395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:13.925928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1f1f5da8-a86d-48b1-a07b-16399ad14395 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:13.937846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1f1f5da8-a86d-48b1-a07b-16399ad14395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:13.970194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:13.971326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:13.971928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:13.972615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] PATH_INFO: `/v3/users/b2ba80bfab064d71a80521c91ccf13c8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:13.974895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ba80bfab064d71a80521c91ccf13c8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:13.977545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.016188 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1464/2955] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:14.025510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb0f50-75fe-42c9-977b-08a6c3dcd6fc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.026053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb0f50-75fe-42c9-977b-08a6c3dcd6fc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.027407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb0f50-75fe-42c9-977b-08a6c3dcd6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.051082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-43fb0f50-75fe-42c9-977b-08a6c3dcd6fc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:14.053231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-43fb0f50-75fe-42c9-977b-08a6c3dcd6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:14.063920 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1465/2956] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:14.066662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb0f50-75fe-42c9-977b-08a6c3dcd6fc None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.077539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb0f50-75fe-42c9-977b-08a6c3dcd6fc None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.082619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edbf6c-c44a-4562-badb-3f7d446f2a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.083255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edbf6c-c44a-4562-badb-3f7d446f2a74 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.083594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edbf6c-c44a-4562-badb-3f7d446f2a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.083864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edbf6c-c44a-4562-badb-3f7d446f2a74 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.084683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edbf6c-c44a-4562-badb-3f7d446f2a74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:00:14.085147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edbf6c-c44a-4562-badb-3f7d446f2a74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.086732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edbf6c-c44a-4562-badb-3f7d446f2a74 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.088875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ba80bfab064d71a80521c91ccf13c8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:14.096123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:14.102515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ba80bfab064d71a80521c91ccf13c8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:14.108075 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1466/2957] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:00:14.111260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edbf6c-c44a-4562-badb-3f7d446f2a74 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.116602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ba80bfab064d71a80521c91ccf13c8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:14.127568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:14.128005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edbf6c-c44a-4562-badb-3f7d446f2a74 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.133753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ba80bfab064d71a80521c91ccf13c8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:14.134692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ba80bfab064d71a80521c91ccf13c8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:14.135326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b874ef-6382-41ee-b563-615e51383a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.136066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b874ef-6382-41ee-b563-615e51383a75 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.136572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b874ef-6382-41ee-b563-615e51383a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.137122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b874ef-6382-41ee-b563-615e51383a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.138576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b874ef-6382-41ee-b563-615e51383a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.140656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b874ef-6382-41ee-b563-615e51383a75 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.154954 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1492/2958] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:00:13 2022] DELETE /identity/v3/users/b2ba80bfab064d71a80521c91ccf13c8 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:14.183330 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1467/2959] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:00:14 2022] POST /identity/v3/projects => generated 391 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:00:14.188171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.189635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:14.273234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520c7b5-5816-4711-baa0-792a053b2c4d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.331286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88372b30-0ea0-46be-8232-e5f21912ca9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.331942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88372b30-0ea0-46be-8232-e5f21912ca9a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.332146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88372b30-0ea0-46be-8232-e5f21912ca9a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.332445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88372b30-0ea0-46be-8232-e5f21912ca9a admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.333104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88372b30-0ea0-46be-8232-e5f21912ca9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.335044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88372b30-0ea0-46be-8232-e5f21912ca9a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.351267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b874ef-6382-41ee-b563-615e51383a75 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.353117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-79b874ef-6382-41ee-b563-615e51383a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:14.364228 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1493/2960] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:00:14 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:00:14.366369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b874ef-6382-41ee-b563-615e51383a75 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.369115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88372b30-0ea0-46be-8232-e5f21912ca9a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.371277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0243c-0386-48ca-85db-28347dede4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.371990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0243c-0386-48ca-85db-28347dede4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.372257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0243c-0386-48ca-85db-28347dede4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.372551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0243c-0386-48ca-85db-28347dede4ee admin admin] PATH_INFO: `/v3/projects/7ef75542615640b1ab4d97a6f0708540` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.373677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0243c-0386-48ca-85db-28347dede4ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ef75542615640b1ab4d97a6f0708540)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.375678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0243c-0386-48ca-85db-28347dede4ee admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.381381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88372b30-0ea0-46be-8232-e5f21912ca9a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.387840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a44e3-9243-4c05-a791-6c1034fa2e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.388826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a44e3-9243-4c05-a791-6c1034fa2e25 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.389103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a44e3-9243-4c05-a791-6c1034fa2e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.389377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a44e3-9243-4c05-a791-6c1034fa2e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.390035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a44e3-9243-4c05-a791-6c1034fa2e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.391598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a44e3-9243-4c05-a791-6c1034fa2e25 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.405592 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1494/2961] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:00:14.406717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a44e3-9243-4c05-a791-6c1034fa2e25 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.416937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a44e3-9243-4c05-a791-6c1034fa2e25 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.430275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-66a0243c-0386-48ca-85db-28347dede4ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ef75542615640b1ab4d97a6f0708540', 'request_id': 'req-66a0243c-0386-48ca-85db-28347dede4ee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:14.437611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-66a0243c-0386-48ca-85db-28347dede4ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ef75542615640b1ab4d97a6f0708540', 'request_id': 'req-66a0243c-0386-48ca-85db-28347dede4ee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:14.445667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-66a0243c-0386-48ca-85db-28347dede4ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ef75542615640b1ab4d97a6f0708540', 'request_id': 'req-66a0243c-0386-48ca-85db-28347dede4ee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:14.456848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-66a0243c-0386-48ca-85db-28347dede4ee admin admin] The token cache is being invalidate because project 7ef75542615640b1ab4d97a6f0708540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:00:14.457130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-66a0243c-0386-48ca-85db-28347dede4ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:14.457762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-66a0243c-0386-48ca-85db-28347dede4ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:14.459273 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1468/2962] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:00:14 2022] DELETE /identity/v3/projects/7ef75542615640b1ab4d97a6f0708540 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:00:14.464531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417e806-ade4-4197-aa35-e21403c00d64 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.465062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417e806-ade4-4197-aa35-e21403c00d64 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.465344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417e806-ade4-4197-aa35-e21403c00d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.474644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535b808-db91-4e02-86b3-c26381afb150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.475253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535b808-db91-4e02-86b3-c26381afb150 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.475527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535b808-db91-4e02-86b3-c26381afb150 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.475788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535b808-db91-4e02-86b3-c26381afb150 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.476454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535b808-db91-4e02-86b3-c26381afb150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.478036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535b808-db91-4e02-86b3-c26381afb150 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.485187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8417e806-ade4-4197-aa35-e21403c00d64 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:14.486414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8417e806-ade4-4197-aa35-e21403c00d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:14.487841 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1495/2963] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:00:14.491980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535b808-db91-4e02-86b3-c26381afb150 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.493674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a535b808-db91-4e02-86b3-c26381afb150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:14.494675 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1469/2964] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:00:14.503088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef5042-2b96-435b-919f-11b68a23d51a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.503359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef5042-2b96-435b-919f-11b68a23d51a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.503655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef5042-2b96-435b-919f-11b68a23d51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.507135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535b808-db91-4e02-86b3-c26381afb150 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.512349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e363575-1550-45b4-876b-a502d3b855e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.513088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e363575-1550-45b4-876b-a502d3b855e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.513359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e363575-1550-45b4-876b-a502d3b855e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.513620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e363575-1550-45b4-876b-a502d3b855e6 admin admin] PATH_INFO: `/v3/projects/1471ca0ad0914fab8ba53f9b6f6b95a4/users/3e574b1c33bf4726942002eb8d4dd7dc/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.522885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f8ef5042-2b96-435b-919f-11b68a23d51a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:14.524166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f8ef5042-2b96-435b-919f-11b68a23d51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:14.525365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:14.530554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e363575-1550-45b4-876b-a502d3b855e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1471ca0ad0914fab8ba53f9b6f6b95a4, user_id=3e574b1c33bf4726942002eb8d4dd7dc, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.533628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e363575-1550-45b4-876b-a502d3b855e6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.533750 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1470/2965] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:14.537899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef5042-2b96-435b-919f-11b68a23d51a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.547531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef5042-2b96-435b-919f-11b68a23d51a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.548355 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1496/2966] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:00:14 2022] PUT /identity/v3/projects/1471ca0ad0914fab8ba53f9b6f6b95a4/users/3e574b1c33bf4726942002eb8d4dd7dc/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:00:14.552855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ddb10-c0fc-4205-82e6-2d2100bda1be None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.553137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ddb10-c0fc-4205-82e6-2d2100bda1be None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.553402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ddb10-c0fc-4205-82e6-2d2100bda1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.575086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-623ddb10-c0fc-4205-82e6-2d2100bda1be None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:14.576189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-623ddb10-c0fc-4205-82e6-2d2100bda1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:14.605787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e2f64-c5d7-44ef-9b7a-89d081b3bbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.606789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e2f64-c5d7-44ef-9b7a-89d081b3bbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.607231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e2f64-c5d7-44ef-9b7a-89d081b3bbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.607630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e2f64-c5d7-44ef-9b7a-89d081b3bbb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.608589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e2f64-c5d7-44ef-9b7a-89d081b3bbb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:00:14.609039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e2f64-c5d7-44ef-9b7a-89d081b3bbb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.613235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e2f64-c5d7-44ef-9b7a-89d081b3bbb5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.632039 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1471/2967] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:00:14.636679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e2f64-c5d7-44ef-9b7a-89d081b3bbb5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.640566 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1497/2968] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:00:14.647219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ddb10-c0fc-4205-82e6-2d2100bda1be None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.648692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-623ddb10-c0fc-4205-82e6-2d2100bda1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:14.650284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e2f64-c5d7-44ef-9b7a-89d081b3bbb5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.656198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8218b0e-70d7-497c-a7c8-70b8da1d721c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.657100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8218b0e-70d7-497c-a7c8-70b8da1d721c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.657376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8218b0e-70d7-497c-a7c8-70b8da1d721c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.657648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8218b0e-70d7-497c-a7c8-70b8da1d721c admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.658272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8218b0e-70d7-497c-a7c8-70b8da1d721c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.660542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8218b0e-70d7-497c-a7c8-70b8da1d721c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.692592 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1472/2969] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:00:14 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:00:14.693733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8218b0e-70d7-497c-a7c8-70b8da1d721c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.713631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ddb10-c0fc-4205-82e6-2d2100bda1be None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.713847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:14.763991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8218b0e-70d7-497c-a7c8-70b8da1d721c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.768100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df98f5-e911-468e-adf1-81b3c740da04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.768738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df98f5-e911-468e-adf1-81b3c740da04 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.768889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c09bd3-7f67-433e-b99f-c537c79981e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.769034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df98f5-e911-468e-adf1-81b3c740da04 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.769250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df98f5-e911-468e-adf1-81b3c740da04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.769501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c09bd3-7f67-433e-b99f-c537c79981e2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.769779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c09bd3-7f67-433e-b99f-c537c79981e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.770045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c09bd3-7f67-433e-b99f-c537c79981e2 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.770674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c09bd3-7f67-433e-b99f-c537c79981e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.772672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c09bd3-7f67-433e-b99f-c537c79981e2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.779452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df98f5-e911-468e-adf1-81b3c740da04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.780360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df98f5-e911-468e-adf1-81b3c740da04 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.791595 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1498/2970] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:14.796256 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1473/2971] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:00:14 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:00:14.800400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df98f5-e911-468e-adf1-81b3c740da04 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.810291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df98f5-e911-468e-adf1-81b3c740da04 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.815261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138910a-111d-450a-ab34-361d99f1bf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.815889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138910a-111d-450a-ab34-361d99f1bf67 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.816157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138910a-111d-450a-ab34-361d99f1bf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.816504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138910a-111d-450a-ab34-361d99f1bf67 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.817108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138910a-111d-450a-ab34-361d99f1bf67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.818687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138910a-111d-450a-ab34-361d99f1bf67 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.829845 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1499/2972] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:00:14.834242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c09bd3-7f67-433e-b99f-c537c79981e2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.844801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c09bd3-7f67-433e-b99f-c537c79981e2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.849625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629203c-9ae8-47f5-a194-a09f24e10ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.850232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629203c-9ae8-47f5-a194-a09f24e10ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.850514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629203c-9ae8-47f5-a194-a09f24e10ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.850778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629203c-9ae8-47f5-a194-a09f24e10ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.851446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629203c-9ae8-47f5-a194-a09f24e10ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.853081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629203c-9ae8-47f5-a194-a09f24e10ca0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.864408 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1474/2973] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:00:14.868505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138910a-111d-450a-ab34-361d99f1bf67 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:14.879468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138910a-111d-450a-ab34-361d99f1bf67 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:14.884979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5ba0e-d97d-4791-b1a2-b58d801e9702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:14.885638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5ba0e-d97d-4791-b1a2-b58d801e9702 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.885901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5ba0e-d97d-4791-b1a2-b58d801e9702 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.886273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5ba0e-d97d-4791-b1a2-b58d801e9702 admin admin] PATH_INFO: `/v3/projects/8bd147414c5a49fbb03c7d1b8f008f9e/users/d4fdfe895ccf4b619bdb6229b96fddb9/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.902632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5ba0e-d97d-4791-b1a2-b58d801e9702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd147414c5a49fbb03c7d1b8f008f9e, user_id=d4fdfe895ccf4b619bdb6229b96fddb9, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:14.905408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5ba0e-d97d-4791-b1a2-b58d801e9702 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:14.923516 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1500/2974] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:00:14 2022] PUT /identity/v3/projects/8bd147414c5a49fbb03c7d1b8f008f9e/users/d4fdfe895ccf4b619bdb6229b96fddb9/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:14.929105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944c593-7e50-4fb8-b39d-601dcf056878 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:14.929532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944c593-7e50-4fb8-b39d-601dcf056878 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:14.929949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944c593-7e50-4fb8-b39d-601dcf056878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:14.951283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2944c593-7e50-4fb8-b39d-601dcf056878 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:14.954997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2944c593-7e50-4fb8-b39d-601dcf056878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:15.047541 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1475/2975] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:15.057609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5ba0e-d97d-4791-b1a2-b58d801e9702 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:15.076472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:15.143059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5ba0e-d97d-4791-b1a2-b58d801e9702 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:15.146846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bae090-ee8b-4d3c-9773-d6e938bd35cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:15.147608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bae090-ee8b-4d3c-9773-d6e938bd35cd service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:15.147954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bae090-ee8b-4d3c-9773-d6e938bd35cd service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:15.148395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bae090-ee8b-4d3c-9773-d6e938bd35cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:15.160627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bae090-ee8b-4d3c-9773-d6e938bd35cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:15.162233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bae090-ee8b-4d3c-9773-d6e938bd35cd service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:15.172909 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1501/2976] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:20.909069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81893-3184-48a8-b83d-fb705bee229a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:20.909597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81893-3184-48a8-b83d-fb705bee229a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:20.909597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81893-3184-48a8-b83d-fb705bee229a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:20.956948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e5f81893-3184-48a8-b83d-fb705bee229a None None] MFA Rules not processed for user `d4fdfe895ccf4b619bdb6229b96fddb9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:20.958787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e5f81893-3184-48a8-b83d-fb705bee229a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:21.071993 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1476/2977] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:21.079406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bae090-ee8b-4d3c-9773-d6e938bd35cd service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:21.080934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-96bae090-ee8b-4d3c-9773-d6e938bd35cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:21.181497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bae090-ee8b-4d3c-9773-d6e938bd35cd service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:21.188151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1262a-907f-43fc-9738-184bd4a5c307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:21.188981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1262a-907f-43fc-9738-184bd4a5c307 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:21.189349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1262a-907f-43fc-9738-184bd4a5c307 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:21.189711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1262a-907f-43fc-9738-184bd4a5c307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:21.206844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1262a-907f-43fc-9738-184bd4a5c307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:21.208709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1262a-907f-43fc-9738-184bd4a5c307 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:21.235344 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1502/2978] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:21 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:23.723563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e89a07-6cdc-486d-90da-e65f071c7c21 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:23.724150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e89a07-6cdc-486d-90da-e65f071c7c21 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:23.724150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e89a07-6cdc-486d-90da-e65f071c7c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:23.803939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-42e89a07-6cdc-486d-90da-e65f071c7c21 None None] MFA Rules not processed for user `3e574b1c33bf4726942002eb8d4dd7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:23.805202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-42e89a07-6cdc-486d-90da-e65f071c7c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:23.837247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:23.971481 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1477/2979] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:23.983800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bb435-b57d-4ac1-82b8-a60c9039bcb0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:23.984362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bb435-b57d-4ac1-82b8-a60c9039bcb0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:23.984362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bb435-b57d-4ac1-82b8-a60c9039bcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:24.036052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2d5bb435-b57d-4ac1-82b8-a60c9039bcb0 None None] MFA Rules not processed for user `3e574b1c33bf4726942002eb8d4dd7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:24.037467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2d5bb435-b57d-4ac1-82b8-a60c9039bcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:24.068614 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1503/2980] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:24.115850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e89a07-6cdc-486d-90da-e65f071c7c21 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:24.144044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e89a07-6cdc-486d-90da-e65f071c7c21 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:24.168665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcee2d1-0d7c-4980-95cd-b7f68331f529 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:24.169387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcee2d1-0d7c-4980-95cd-b7f68331f529 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:24.169666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcee2d1-0d7c-4980-95cd-b7f68331f529 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:24.169936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcee2d1-0d7c-4980-95cd-b7f68331f529 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:24.200105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcee2d1-0d7c-4980-95cd-b7f68331f529 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:24.201827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcee2d1-0d7c-4980-95cd-b7f68331f529 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:24.228844 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1478/2981] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:25.295043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bb435-b57d-4ac1-82b8-a60c9039bcb0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:25.296731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2d5bb435-b57d-4ac1-82b8-a60c9039bcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:25.487974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bb435-b57d-4ac1-82b8-a60c9039bcb0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:25.494186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4ee9c-1c85-4362-bb46-dfd3ddf198a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:25.495067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4ee9c-1c85-4362-bb46-dfd3ddf198a2 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:25.495659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4ee9c-1c85-4362-bb46-dfd3ddf198a2 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:25.496168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4ee9c-1c85-4362-bb46-dfd3ddf198a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:25.498217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-86f4ee9c-1c85-4362-bb46-dfd3ddf198a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:25.518949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4ee9c-1c85-4362-bb46-dfd3ddf198a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:25.521349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4ee9c-1c85-4362-bb46-dfd3ddf198a2 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:25.563315 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1504/2982] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:27.729939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcee2d1-0d7c-4980-95cd-b7f68331f529 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:27.759307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcee2d1-0d7c-4980-95cd-b7f68331f529 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:27.769715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708b4ca-c5f7-4e56-990f-ef13a1a32c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:27.770492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708b4ca-c5f7-4e56-990f-ef13a1a32c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:27.771086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708b4ca-c5f7-4e56-990f-ef13a1a32c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:27.771515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708b4ca-c5f7-4e56-990f-ef13a1a32c19 admin admin] PATH_INFO: `/v3/users/d4fdfe895ccf4b619bdb6229b96fddb9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:27.773983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708b4ca-c5f7-4e56-990f-ef13a1a32c19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4fdfe895ccf4b619bdb6229b96fddb9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:27.776184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708b4ca-c5f7-4e56-990f-ef13a1a32c19 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:27.880665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b708b4ca-c5f7-4e56-990f-ef13a1a32c19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4fdfe895ccf4b619bdb6229b96fddb9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:27.892449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b708b4ca-c5f7-4e56-990f-ef13a1a32c19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4fdfe895ccf4b619bdb6229b96fddb9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:27.910654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b708b4ca-c5f7-4e56-990f-ef13a1a32c19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4fdfe895ccf4b619bdb6229b96fddb9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:27.912308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b708b4ca-c5f7-4e56-990f-ef13a1a32c19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4fdfe895ccf4b619bdb6229b96fddb9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:27.913056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b708b4ca-c5f7-4e56-990f-ef13a1a32c19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4fdfe895ccf4b619bdb6229b96fddb9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:27.972464 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1479/2983] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:00:27 2022] DELETE /identity/v3/users/d4fdfe895ccf4b619bdb6229b96fddb9 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:28.221998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4ee9c-1c85-4362-bb46-dfd3ddf198a2 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:28.240925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4ee9c-1c85-4362-bb46-dfd3ddf198a2 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:28.306694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:28.385358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600ffaf-dde0-4396-b706-11d9f326c0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:28.387120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600ffaf-dde0-4396-b706-11d9f326c0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:28.387704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600ffaf-dde0-4396-b706-11d9f326c0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:28.388177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600ffaf-dde0-4396-b706-11d9f326c0f8 admin admin] PATH_INFO: `/v3/projects/8bd147414c5a49fbb03c7d1b8f008f9e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:28.389706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600ffaf-dde0-4396-b706-11d9f326c0f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bd147414c5a49fbb03c7d1b8f008f9e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:28.392651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600ffaf-dde0-4396-b706-11d9f326c0f8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:28.480487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4600ffaf-dde0-4396-b706-11d9f326c0f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bd147414c5a49fbb03c7d1b8f008f9e', 'request_id': 'req-4600ffaf-dde0-4396-b706-11d9f326c0f8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:28.491482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4600ffaf-dde0-4396-b706-11d9f326c0f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bd147414c5a49fbb03c7d1b8f008f9e', 'request_id': 'req-4600ffaf-dde0-4396-b706-11d9f326c0f8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:28.500947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4600ffaf-dde0-4396-b706-11d9f326c0f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bd147414c5a49fbb03c7d1b8f008f9e', 'request_id': 'req-4600ffaf-dde0-4396-b706-11d9f326c0f8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:28.515534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4600ffaf-dde0-4396-b706-11d9f326c0f8 admin admin] The token cache is being invalidate because project 8bd147414c5a49fbb03c7d1b8f008f9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:00:28.516038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4600ffaf-dde0-4396-b706-11d9f326c0f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:28.516846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4600ffaf-dde0-4396-b706-11d9f326c0f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:28.519465 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1505/2984] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:00:28 2022] DELETE /identity/v3/projects/8bd147414c5a49fbb03c7d1b8f008f9e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:00:28.524680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732c0ae-76d6-419c-9a6c-c0660f7f2be6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:28.524963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732c0ae-76d6-419c-9a6c-c0660f7f2be6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:28.525233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732c0ae-76d6-419c-9a6c-c0660f7f2be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:28.549825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c732c0ae-76d6-419c-9a6c-c0660f7f2be6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:28.551052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c732c0ae-76d6-419c-9a6c-c0660f7f2be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:28.619349 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1480/2985] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:28.628466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6529-601f-4801-ac60-eb6c14e1de6e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:28.628745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6529-601f-4801-ac60-eb6c14e1de6e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:28.629282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6529-601f-4801-ac60-eb6c14e1de6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:28.670048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8ddb6529-601f-4801-ac60-eb6c14e1de6e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:28.673415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8ddb6529-601f-4801-ac60-eb6c14e1de6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:28.695371 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1506/2986] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:28.700622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732c0ae-76d6-419c-9a6c-c0660f7f2be6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:28.715335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732c0ae-76d6-419c-9a6c-c0660f7f2be6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:28.737107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a3d2e-dc32-4547-95ea-4f13006816db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:28.739035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a3d2e-dc32-4547-95ea-4f13006816db admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:28.739752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a3d2e-dc32-4547-95ea-4f13006816db admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:28.741391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a3d2e-dc32-4547-95ea-4f13006816db admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:28.742499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a3d2e-dc32-4547-95ea-4f13006816db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:00:28.743064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a3d2e-dc32-4547-95ea-4f13006816db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:28.745426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a3d2e-dc32-4547-95ea-4f13006816db admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:28.780904 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1481/2987] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:00:28.789138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb6529-601f-4801-ac60-eb6c14e1de6e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:28.806880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb6529-601f-4801-ac60-eb6c14e1de6e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:28.826554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f33f13-4979-4887-9403-3d38519d0a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:28.826554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f33f13-4979-4887-9403-3d38519d0a50 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:28.826554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f33f13-4979-4887-9403-3d38519d0a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:28.827029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f33f13-4979-4887-9403-3d38519d0a50 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:28.827380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f33f13-4979-4887-9403-3d38519d0a50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:28.829371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f33f13-4979-4887-9403-3d38519d0a50 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:28.903564 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1507/2988] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:00:28 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:00:28.913624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a3d2e-dc32-4547-95ea-4f13006816db admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:28.919478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:28.929254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a3d2e-dc32-4547-95ea-4f13006816db admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:29.017782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcf3fd-3a0b-42a5-b019-09efc0b1e435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:29.018761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcf3fd-3a0b-42a5-b019-09efc0b1e435 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:29.019043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcf3fd-3a0b-42a5-b019-09efc0b1e435 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:29.019311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcf3fd-3a0b-42a5-b019-09efc0b1e435 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:29.019979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dcf3fd-3a0b-42a5-b019-09efc0b1e435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:29.021978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dcf3fd-3a0b-42a5-b019-09efc0b1e435 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:29.049230 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1482/2989] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:00:28 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:00:29.053430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f33f13-4979-4887-9403-3d38519d0a50 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:29.066770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f33f13-4979-4887-9403-3d38519d0a50 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:29.072815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a746d-b828-4549-b5f5-ef3f760e2af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:29.074179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a746d-b828-4549-b5f5-ef3f760e2af6 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:29.074932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a746d-b828-4549-b5f5-ef3f760e2af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:29.075557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a746d-b828-4549-b5f5-ef3f760e2af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:29.076603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a746d-b828-4549-b5f5-ef3f760e2af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:29.078677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a746d-b828-4549-b5f5-ef3f760e2af6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:29.092925 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1508/2990] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:00:29.097428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcf3fd-3a0b-42a5-b019-09efc0b1e435 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:29.110146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcf3fd-3a0b-42a5-b019-09efc0b1e435 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:29.115749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b948b68-fdf2-4dd4-866c-d4bf901b89ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:29.116622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b948b68-fdf2-4dd4-866c-d4bf901b89ae admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:29.117054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b948b68-fdf2-4dd4-866c-d4bf901b89ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:29.117586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b948b68-fdf2-4dd4-866c-d4bf901b89ae admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:29.118385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b948b68-fdf2-4dd4-866c-d4bf901b89ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:29.120622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b948b68-fdf2-4dd4-866c-d4bf901b89ae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:29.134510 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1483/2991] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:00:29.139263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a746d-b828-4549-b5f5-ef3f760e2af6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:29.156656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a746d-b828-4549-b5f5-ef3f760e2af6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:29.167587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be51a9f-f522-4e04-84ba-5bb7a0a69914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:29.168825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be51a9f-f522-4e04-84ba-5bb7a0a69914 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:29.170035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be51a9f-f522-4e04-84ba-5bb7a0a69914 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:29.170597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be51a9f-f522-4e04-84ba-5bb7a0a69914 admin admin] PATH_INFO: `/v3/projects/e9e8ba133a1b43b2a7a6085abb10d914/users/92bb237651114d398df9efc4214fc071/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:29.195133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be51a9f-f522-4e04-84ba-5bb7a0a69914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9e8ba133a1b43b2a7a6085abb10d914, user_id=92bb237651114d398df9efc4214fc071, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:29.198066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be51a9f-f522-4e04-84ba-5bb7a0a69914 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:29.218594 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1509/2992] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:00:29 2022] PUT /identity/v3/projects/e9e8ba133a1b43b2a7a6085abb10d914/users/92bb237651114d398df9efc4214fc071/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:29.230825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0e9cd-72f1-4099-a451-a496e26055b6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:29.231140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0e9cd-72f1-4099-a451-a496e26055b6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:29.231464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0e9cd-72f1-4099-a451-a496e26055b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:29.265978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8ac0e9cd-72f1-4099-a451-a496e26055b6 None None] MFA Rules not processed for user `92bb237651114d398df9efc4214fc071`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:29.267311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8ac0e9cd-72f1-4099-a451-a496e26055b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:29.342775 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1484/2993] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:00:29.355213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be51a9f-f522-4e04-84ba-5bb7a0a69914 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:29.357111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6be51a9f-f522-4e04-84ba-5bb7a0a69914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:29.443374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:29.499547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be51a9f-f522-4e04-84ba-5bb7a0a69914 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:29.514044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd372822-d8ef-4236-9ad3-2abf3f670933 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:29.516324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd372822-d8ef-4236-9ad3-2abf3f670933 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:29.516699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd372822-d8ef-4236-9ad3-2abf3f670933 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:29.517399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd372822-d8ef-4236-9ad3-2abf3f670933 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:29.527976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd372822-d8ef-4236-9ad3-2abf3f670933 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:29.529630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd372822-d8ef-4236-9ad3-2abf3f670933 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:29.547738 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1510/2994] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:29.667551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0e9cd-72f1-4099-a451-a496e26055b6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:29.805492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0e9cd-72f1-4099-a451-a496e26055b6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:29.812208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:29.812923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:29.813397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:29.813860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] PATH_INFO: `/v3/users/92bb237651114d398df9efc4214fc071` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:29.815828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=92bb237651114d398df9efc4214fc071)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:29.818043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:29.833202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:29.880198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92bb237651114d398df9efc4214fc071'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:29.889899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92bb237651114d398df9efc4214fc071'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:29.908137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92bb237651114d398df9efc4214fc071'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:29.908976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92bb237651114d398df9efc4214fc071'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:29.909583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92bb237651114d398df9efc4214fc071'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:29.926773 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1485/2995] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:00:29 2022] DELETE /identity/v3/users/92bb237651114d398df9efc4214fc071 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:29.931900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3052b-7fa5-45d0-a4b0-6ae6e74dd554 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:29.932522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3052b-7fa5-45d0-a4b0-6ae6e74dd554 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:29.932966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3052b-7fa5-45d0-a4b0-6ae6e74dd554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:29.963077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b0a3052b-7fa5-45d0-a4b0-6ae6e74dd554 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:29.964956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b0a3052b-7fa5-45d0-a4b0-6ae6e74dd554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:29.988274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:29.990458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:30.076674 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1511/2996] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:30.087942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3052b-7fa5-45d0-a4b0-6ae6e74dd554 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:30.093045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b0a3052b-7fa5-45d0-a4b0-6ae6e74dd554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:30.102823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854a5c0-2263-4372-b27a-ced92f9b94bf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:30.108304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:30.109052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:30.109394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:30.109671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] PATH_INFO: `/v3/users/2517214c35344f3b9b60ec480909b79c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:30.111036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2517214c35344f3b9b60ec480909b79c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:30.113078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:30.158565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3052b-7fa5-45d0-a4b0-6ae6e74dd554 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:30.162983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cb7db-362a-4791-81e3-b3ae9cdcf00f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:30.163594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cb7db-362a-4791-81e3-b3ae9cdcf00f service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:30.164238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cb7db-362a-4791-81e3-b3ae9cdcf00f service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:30.164541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cb7db-362a-4791-81e3-b3ae9cdcf00f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:30.173828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2517214c35344f3b9b60ec480909b79c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:30.182180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cb7db-362a-4791-81e3-b3ae9cdcf00f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:30.184004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cb7db-362a-4791-81e3-b3ae9cdcf00f service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:30.188617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2517214c35344f3b9b60ec480909b79c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:30.198634 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1512/2997] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:30.216154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2517214c35344f3b9b60ec480909b79c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:30.217364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2517214c35344f3b9b60ec480909b79c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:30.218062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2517214c35344f3b9b60ec480909b79c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:30.244017 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1486/2998] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:00:29 2022] DELETE /identity/v3/users/2517214c35344f3b9b60ec480909b79c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:30.249290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff5648-5f4d-487f-8160-3be5cbf96aa9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:30.249823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff5648-5f4d-487f-8160-3be5cbf96aa9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:30.249976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff5648-5f4d-487f-8160-3be5cbf96aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:30.273325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-aaff5648-5f4d-487f-8160-3be5cbf96aa9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:30.274588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aaff5648-5f4d-487f-8160-3be5cbf96aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:30.350812 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1513/2999] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:00:30.367343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:30.368912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:30.381205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:30.434989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0ec94-26ef-4c2d-abd5-9c9c7e68b014 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:30.445470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c5cae-f6e9-4bb2-8779-24156aeca5a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:30.446109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c5cae-f6e9-4bb2-8779-24156aeca5a8 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:30.446361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c5cae-f6e9-4bb2-8779-24156aeca5a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:30.446639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c5cae-f6e9-4bb2-8779-24156aeca5a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:30.457940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c5cae-f6e9-4bb2-8779-24156aeca5a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:30.459399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c5cae-f6e9-4bb2-8779-24156aeca5a8 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:30.471069 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1487/3000] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:30.621862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39d441-66a6-466f-9a37-44e61fd276ba None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:30.621862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39d441-66a6-466f-9a37-44e61fd276ba None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:30.621862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39d441-66a6-466f-9a37-44e61fd276ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:30.649864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5f39d441-66a6-466f-9a37-44e61fd276ba None None] MFA Rules not processed for user `7c276ebf6fe14a9c8f50bd1d1f44fb58`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:30.651069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5f39d441-66a6-466f-9a37-44e61fd276ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:30.668803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:30.735240 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1514/3001] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:30 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:30.768888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c5cae-f6e9-4bb2-8779-24156aeca5a8 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:30.774472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-742c5cae-f6e9-4bb2-8779-24156aeca5a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:30.806215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c5cae-f6e9-4bb2-8779-24156aeca5a8 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:30.825620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:30.826760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:30.827528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:30.828011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4 admin admin] PATH_INFO: `/v3/projects/e9e8ba133a1b43b2a7a6085abb10d914` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:30.829853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9e8ba133a1b43b2a7a6085abb10d914)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:30.833382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:30.974322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9e8ba133a1b43b2a7a6085abb10d914', 'request_id': 'req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:30.984788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9e8ba133a1b43b2a7a6085abb10d914', 'request_id': 'req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.012647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9e8ba133a1b43b2a7a6085abb10d914', 'request_id': 'req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.036945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4 admin admin] The token cache is being invalidate because project e9e8ba133a1b43b2a7a6085abb10d914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:00:31.037565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.041132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec251373-c1c9-412a-8b64-8b7d0c9b91a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.046308 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1488/3002] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:00:30 2022] DELETE /identity/v3/projects/e9e8ba133a1b43b2a7a6085abb10d914 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:31.050788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8be2d-471d-4a32-a09e-e26dd3fee079 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:31.051062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8be2d-471d-4a32-a09e-e26dd3fee079 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:31.051322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8be2d-471d-4a32-a09e-e26dd3fee079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:31.105480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b0c8be2d-471d-4a32-a09e-e26dd3fee079 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:31.106985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b0c8be2d-471d-4a32-a09e-e26dd3fee079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:31.235049 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1515/3003] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:31.238184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c119fd-90e3-4165-b4f6-d86e02140046 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:31.240862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c119fd-90e3-4165-b4f6-d86e02140046 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:31.241153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c119fd-90e3-4165-b4f6-d86e02140046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:31.269409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-08c119fd-90e3-4165-b4f6-d86e02140046 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:31.270894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-08c119fd-90e3-4165-b4f6-d86e02140046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:31.294161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8be2d-471d-4a32-a09e-e26dd3fee079 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:31.295361 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1489/3004] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:00:31.313640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8be2d-471d-4a32-a09e-e26dd3fee079 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:31.322313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc74cf-932e-4926-a266-74b0e83034f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:31.324031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc74cf-932e-4926-a266-74b0e83034f5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:31.324656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc74cf-932e-4926-a266-74b0e83034f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:31.325761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc74cf-932e-4926-a266-74b0e83034f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:31.326573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc74cf-932e-4926-a266-74b0e83034f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:00:31.327063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc74cf-932e-4926-a266-74b0e83034f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:31.328971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc74cf-932e-4926-a266-74b0e83034f5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:31.340920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c119fd-90e3-4165-b4f6-d86e02140046 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:31.342926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-08c119fd-90e3-4165-b4f6-d86e02140046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:31.355054 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1516/3005] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:00:31.360315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc74cf-932e-4926-a266-74b0e83034f5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:31.376306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c119fd-90e3-4165-b4f6-d86e02140046 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:31.391747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc74cf-932e-4926-a266-74b0e83034f5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:31.402130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:31.402682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:31.403676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:31.403890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] PATH_INFO: `/v3/projects/7f66f844c2ea41dd8912a4a22d7c9730` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:31.405404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f66f844c2ea41dd8912a4a22d7c9730)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:31.408971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14650d8f-aa73-45f3-94df-410e2703772b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:31.409663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:31.409804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14650d8f-aa73-45f3-94df-410e2703772b admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:31.410062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14650d8f-aa73-45f3-94df-410e2703772b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:31.410386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14650d8f-aa73-45f3-94df-410e2703772b admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:31.411432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14650d8f-aa73-45f3-94df-410e2703772b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:31.417835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14650d8f-aa73-45f3-94df-410e2703772b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:31.479399 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1517/3006] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:00:31 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:00:31.480908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14650d8f-aa73-45f3-94df-410e2703772b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:31.499252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f66f844c2ea41dd8912a4a22d7c9730', 'request_id': 'req-8cee2251-7be8-44cb-9219-3b98cf7f0b64'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.508825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f66f844c2ea41dd8912a4a22d7c9730', 'request_id': 'req-8cee2251-7be8-44cb-9219-3b98cf7f0b64'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.519108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f66f844c2ea41dd8912a4a22d7c9730', 'request_id': 'req-8cee2251-7be8-44cb-9219-3b98cf7f0b64'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.532832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] The token cache is being invalidate because project 7f66f844c2ea41dd8912a4a22d7c9730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:00:31.533393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.534435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.539591 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1490/3007] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:00:31 2022] DELETE /identity/v3/projects/7f66f844c2ea41dd8912a4a22d7c9730 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:31.540911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:31.542816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:31.557087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:31.592108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14650d8f-aa73-45f3-94df-410e2703772b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:31.599401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd21a04-8e51-4b2f-98ca-2dbbb1d602c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:31.600019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd21a04-8e51-4b2f-98ca-2dbbb1d602c3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:31.600634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd21a04-8e51-4b2f-98ca-2dbbb1d602c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:31.601293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd21a04-8e51-4b2f-98ca-2dbbb1d602c3 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:31.601926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd21a04-8e51-4b2f-98ca-2dbbb1d602c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:31.604346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd21a04-8e51-4b2f-98ca-2dbbb1d602c3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:31.618643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee2251-7be8-44cb-9219-3b98cf7f0b64 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:31.624606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabca16c-2e51-48b7-9a0d-78db90e13b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:31.625458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabca16c-2e51-48b7-9a0d-78db90e13b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:31.625912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabca16c-2e51-48b7-9a0d-78db90e13b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:31.626599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabca16c-2e51-48b7-9a0d-78db90e13b6e admin admin] PATH_INFO: `/v3/users/99597ed0ca7d4f5fadb126ac7d800968` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:31.632755 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1518/3008] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:00:31 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:00:31.633355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabca16c-2e51-48b7-9a0d-78db90e13b6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=99597ed0ca7d4f5fadb126ac7d800968)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:31.635605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabca16c-2e51-48b7-9a0d-78db90e13b6e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:31.637378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd21a04-8e51-4b2f-98ca-2dbbb1d602c3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:31.640299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ffd21a04-8e51-4b2f-98ca-2dbbb1d602c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:31.655376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd21a04-8e51-4b2f-98ca-2dbbb1d602c3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:31.664356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973f509-982c-42e7-80b8-00a45ab02a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:31.664997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973f509-982c-42e7-80b8-00a45ab02a07 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:31.665289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973f509-982c-42e7-80b8-00a45ab02a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:31.665564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973f509-982c-42e7-80b8-00a45ab02a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:31.666242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973f509-982c-42e7-80b8-00a45ab02a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:31.668475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973f509-982c-42e7-80b8-00a45ab02a07 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:31.684236 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1519/3009] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:00:31.685319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973f509-982c-42e7-80b8-00a45ab02a07 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:31.696585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973f509-982c-42e7-80b8-00a45ab02a07 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:31.697206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aabca16c-2e51-48b7-9a0d-78db90e13b6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99597ed0ca7d4f5fadb126ac7d800968'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.702359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e462c4-5b26-4a72-bc83-963cec15eadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:31.703197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e462c4-5b26-4a72-bc83-963cec15eadc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:31.703541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e462c4-5b26-4a72-bc83-963cec15eadc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:31.703851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e462c4-5b26-4a72-bc83-963cec15eadc admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:31.704633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e462c4-5b26-4a72-bc83-963cec15eadc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:31.706504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e462c4-5b26-4a72-bc83-963cec15eadc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:31.707993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aabca16c-2e51-48b7-9a0d-78db90e13b6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99597ed0ca7d4f5fadb126ac7d800968'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.709422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:31.723109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aabca16c-2e51-48b7-9a0d-78db90e13b6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99597ed0ca7d4f5fadb126ac7d800968'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.724161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aabca16c-2e51-48b7-9a0d-78db90e13b6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99597ed0ca7d4f5fadb126ac7d800968'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.724835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-aabca16c-2e51-48b7-9a0d-78db90e13b6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99597ed0ca7d4f5fadb126ac7d800968'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:31.726249 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1520/3010] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:00:31.726936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e462c4-5b26-4a72-bc83-963cec15eadc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:31.728702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c8e462c4-5b26-4a72-bc83-963cec15eadc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:31.740641 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1491/3011] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:00:31 2022] DELETE /identity/v3/users/99597ed0ca7d4f5fadb126ac7d800968 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:31.743290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e462c4-5b26-4a72-bc83-963cec15eadc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:31.818293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13e7f8-091b-4975-9547-28eaea48acd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:31.819036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13e7f8-091b-4975-9547-28eaea48acd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:31.819308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13e7f8-091b-4975-9547-28eaea48acd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:31.819944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13e7f8-091b-4975-9547-28eaea48acd2 admin admin] PATH_INFO: `/v3/projects/940ad92c29e64ac893be0ab2c12afe2e/users/a211978552be446dbf3992f8df096477/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:31.842487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13e7f8-091b-4975-9547-28eaea48acd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=940ad92c29e64ac893be0ab2c12afe2e, user_id=a211978552be446dbf3992f8df096477, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:31.845469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13e7f8-091b-4975-9547-28eaea48acd2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:31.862710 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1521/3012] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:00:31 2022] PUT /identity/v3/projects/940ad92c29e64ac893be0ab2c12afe2e/users/a211978552be446dbf3992f8df096477/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:31.868767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cf73c-7ab1-40ae-8df5-852dc487dde9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:31.869289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cf73c-7ab1-40ae-8df5-852dc487dde9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:31.869423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cf73c-7ab1-40ae-8df5-852dc487dde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:31.893473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-538cf73c-7ab1-40ae-8df5-852dc487dde9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:31.894703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-538cf73c-7ab1-40ae-8df5-852dc487dde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:31.978865 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1492/3013] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:00:31.989530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13e7f8-091b-4975-9547-28eaea48acd2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:31.991458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2b13e7f8-091b-4975-9547-28eaea48acd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:32.064493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13e7f8-091b-4975-9547-28eaea48acd2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:32.068289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36498e6-92ac-4ba0-b18e-34e68d201944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:32.068928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36498e6-92ac-4ba0-b18e-34e68d201944 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:32.069200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36498e6-92ac-4ba0-b18e-34e68d201944 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:32.069461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36498e6-92ac-4ba0-b18e-34e68d201944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:32.077257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cf73c-7ab1-40ae-8df5-852dc487dde9 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:32.078968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-538cf73c-7ab1-40ae-8df5-852dc487dde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:32.080698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36498e6-92ac-4ba0-b18e-34e68d201944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:32.082297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36498e6-92ac-4ba0-b18e-34e68d201944 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:32.095345 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1522/3014] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:32.101537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cf73c-7ab1-40ae-8df5-852dc487dde9 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:32.108052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05cb51-45db-4311-b5b6-7f51beb929de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:32.108875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05cb51-45db-4311-b5b6-7f51beb929de admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:32.109222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05cb51-45db-4311-b5b6-7f51beb929de admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:32.109558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05cb51-45db-4311-b5b6-7f51beb929de admin admin] PATH_INFO: `/v3/projects/f84b00c0ede244e092e45bc6659dc119` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:32.111185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05cb51-45db-4311-b5b6-7f51beb929de admin admin] RBAC: Authorizing `identity:delete_project(project_id=f84b00c0ede244e092e45bc6659dc119)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:32.113858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05cb51-45db-4311-b5b6-7f51beb929de admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:32.165440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:32.187172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ad05cb51-45db-4311-b5b6-7f51beb929de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f84b00c0ede244e092e45bc6659dc119', 'request_id': 'req-ad05cb51-45db-4311-b5b6-7f51beb929de'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:32.195160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ad05cb51-45db-4311-b5b6-7f51beb929de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f84b00c0ede244e092e45bc6659dc119', 'request_id': 'req-ad05cb51-45db-4311-b5b6-7f51beb929de'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:32.209062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ad05cb51-45db-4311-b5b6-7f51beb929de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f84b00c0ede244e092e45bc6659dc119', 'request_id': 'req-ad05cb51-45db-4311-b5b6-7f51beb929de'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:32.227192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ad05cb51-45db-4311-b5b6-7f51beb929de admin admin] The token cache is being invalidate because project f84b00c0ede244e092e45bc6659dc119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:00:32.227737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ad05cb51-45db-4311-b5b6-7f51beb929de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:32.229594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ad05cb51-45db-4311-b5b6-7f51beb929de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:32.235220 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1493/3015] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:00:32 2022] DELETE /identity/v3/projects/f84b00c0ede244e092e45bc6659dc119 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:00:32.238019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b45b4c-eea1-4c51-873d-7c62a3bd09d1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:32.238314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b45b4c-eea1-4c51-873d-7c62a3bd09d1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:32.238540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b45b4c-eea1-4c51-873d-7c62a3bd09d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:32.279018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-31b45b4c-eea1-4c51-873d-7c62a3bd09d1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:32.281762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-31b45b4c-eea1-4c51-873d-7c62a3bd09d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:32.379047 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1523/3016] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:32.381342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf9009-287f-4d2c-bdb7-c6e4cf39070e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:32.381621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf9009-287f-4d2c-bdb7-c6e4cf39070e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:32.381913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf9009-287f-4d2c-bdb7-c6e4cf39070e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:32.430138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-adaf9009-287f-4d2c-bdb7-c6e4cf39070e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:32.431499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-adaf9009-287f-4d2c-bdb7-c6e4cf39070e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:32.446799 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1494/3017] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:32.452075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b45b4c-eea1-4c51-873d-7c62a3bd09d1 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:32.465556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b45b4c-eea1-4c51-873d-7c62a3bd09d1 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:32.473507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83784d23-3167-4866-bc3e-9254e39ae2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:32.475154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83784d23-3167-4866-bc3e-9254e39ae2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:32.475488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83784d23-3167-4866-bc3e-9254e39ae2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:32.475828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83784d23-3167-4866-bc3e-9254e39ae2d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:32.477296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83784d23-3167-4866-bc3e-9254e39ae2d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:00:32.479907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83784d23-3167-4866-bc3e-9254e39ae2d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:32.482683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83784d23-3167-4866-bc3e-9254e39ae2d1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:32.500459 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1524/3018] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:00:32.504429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf9009-287f-4d2c-bdb7-c6e4cf39070e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:32.516274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf9009-287f-4d2c-bdb7-c6e4cf39070e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:32.522178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a3b3-91df-4034-9557-c6591337e8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:32.523260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a3b3-91df-4034-9557-c6591337e8ed admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:32.523535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a3b3-91df-4034-9557-c6591337e8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:32.523800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a3b3-91df-4034-9557-c6591337e8ed admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:32.524458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51a3b3-91df-4034-9557-c6591337e8ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:32.526679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51a3b3-91df-4034-9557-c6591337e8ed admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:32.614443 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1495/3019] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:00:32 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:00:32.626763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83784d23-3167-4866-bc3e-9254e39ae2d1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:32.659453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83784d23-3167-4866-bc3e-9254e39ae2d1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:32.681697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:32.782605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1dd35-2cbf-4191-8b18-df559fe8d349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:32.786272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1dd35-2cbf-4191-8b18-df559fe8d349 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:32.786543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1dd35-2cbf-4191-8b18-df559fe8d349 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:32.786810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1dd35-2cbf-4191-8b18-df559fe8d349 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:32.790836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1dd35-2cbf-4191-8b18-df559fe8d349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:32.797681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1dd35-2cbf-4191-8b18-df559fe8d349 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:32.830760 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1525/3020] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:00:32 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:00:32.834197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a3b3-91df-4034-9557-c6591337e8ed admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:32.850843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a3b3-91df-4034-9557-c6591337e8ed admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:32.860195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6587ac2-a8e1-4176-a501-5f7ff1ebb65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:32.861268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6587ac2-a8e1-4176-a501-5f7ff1ebb65a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:32.862020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6587ac2-a8e1-4176-a501-5f7ff1ebb65a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:32.862513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6587ac2-a8e1-4176-a501-5f7ff1ebb65a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:32.863578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6587ac2-a8e1-4176-a501-5f7ff1ebb65a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:32.865860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6587ac2-a8e1-4176-a501-5f7ff1ebb65a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:32.883767 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1496/3021] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:00:32.888721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1dd35-2cbf-4191-8b18-df559fe8d349 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:32.901655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1dd35-2cbf-4191-8b18-df559fe8d349 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:32.917056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca80da9-b831-4865-bb96-01a49cd18160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:32.917849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca80da9-b831-4865-bb96-01a49cd18160 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:32.918747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca80da9-b831-4865-bb96-01a49cd18160 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:32.919110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca80da9-b831-4865-bb96-01a49cd18160 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:32.919857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca80da9-b831-4865-bb96-01a49cd18160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:32.923113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca80da9-b831-4865-bb96-01a49cd18160 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:32.939947 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1526/3022] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:00:32.944495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6587ac2-a8e1-4176-a501-5f7ff1ebb65a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:32.962279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6587ac2-a8e1-4176-a501-5f7ff1ebb65a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:32.975679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427839e3-c87b-42e6-a8a9-1efd4bfa68b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:32.976460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427839e3-c87b-42e6-a8a9-1efd4bfa68b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:32.976729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427839e3-c87b-42e6-a8a9-1efd4bfa68b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:32.977252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427839e3-c87b-42e6-a8a9-1efd4bfa68b1 admin admin] PATH_INFO: `/v3/projects/56abed6a790140b08939484f98f2db8a/users/334fcd5a53b04e21b107fafebcfa98cf/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:33.000982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427839e3-c87b-42e6-a8a9-1efd4bfa68b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56abed6a790140b08939484f98f2db8a, user_id=334fcd5a53b04e21b107fafebcfa98cf, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:33.005949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427839e3-c87b-42e6-a8a9-1efd4bfa68b1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:33.040848 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1497/3023] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:00:32 2022] PUT /identity/v3/projects/56abed6a790140b08939484f98f2db8a/users/334fcd5a53b04e21b107fafebcfa98cf/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:33.058768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21977b26-da33-46f6-9d70-6b59d6c8d3a0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:33.059051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21977b26-da33-46f6-9d70-6b59d6c8d3a0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:33.059772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21977b26-da33-46f6-9d70-6b59d6c8d3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:33.116738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-21977b26-da33-46f6-9d70-6b59d6c8d3a0 None None] MFA Rules not processed for user `334fcd5a53b04e21b107fafebcfa98cf`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:33.118302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-21977b26-da33-46f6-9d70-6b59d6c8d3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:33.321087 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1527/3024] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:33.325437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427839e3-c87b-42e6-a8a9-1efd4bfa68b1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:33.523289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427839e3-c87b-42e6-a8a9-1efd4bfa68b1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:33.536108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6c10a-2ec5-4965-83e6-252a3fd6461c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:33.537170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6c10a-2ec5-4965-83e6-252a3fd6461c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:33.537469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6c10a-2ec5-4965-83e6-252a3fd6461c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:33.537701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6c10a-2ec5-4965-83e6-252a3fd6461c admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:33.542854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6c10a-2ec5-4965-83e6-252a3fd6461c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:33.547424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6c10a-2ec5-4965-83e6-252a3fd6461c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:33.625181 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1498/3025] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:00:33 2022] POST /identity/v3/projects => generated 383 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:00:33.625628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21977b26-da33-46f6-9d70-6b59d6c8d3a0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:33.668129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21977b26-da33-46f6-9d70-6b59d6c8d3a0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:33.850504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295950c-7004-44d7-9864-9a1139f715bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:33.853141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295950c-7004-44d7-9864-9a1139f715bf admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:33.853476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295950c-7004-44d7-9864-9a1139f715bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:33.853748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295950c-7004-44d7-9864-9a1139f715bf admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:33.854414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295950c-7004-44d7-9864-9a1139f715bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:33.857483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295950c-7004-44d7-9864-9a1139f715bf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:33.892403 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1528/3026] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:00:33 2022] POST /identity/v3/users => generated 298 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:00:33.893780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6c10a-2ec5-4965-83e6-252a3fd6461c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:33.908191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6c10a-2ec5-4965-83e6-252a3fd6461c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:33.914810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912162e-dc5b-460a-8496-5d1c8ea71354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:33.916005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912162e-dc5b-460a-8496-5d1c8ea71354 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:33.916548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912162e-dc5b-460a-8496-5d1c8ea71354 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:33.917151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912162e-dc5b-460a-8496-5d1c8ea71354 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:33.918111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912162e-dc5b-460a-8496-5d1c8ea71354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:33.920021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912162e-dc5b-460a-8496-5d1c8ea71354 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:33.932139 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1499/3027] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:00:33.936672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295950c-7004-44d7-9864-9a1139f715bf admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:33.955053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295950c-7004-44d7-9864-9a1139f715bf admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:33.966400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd258a0a-ef81-492e-bb0b-ff556f867709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:33.966400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd258a0a-ef81-492e-bb0b-ff556f867709 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:33.966913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd258a0a-ef81-492e-bb0b-ff556f867709 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:33.967263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd258a0a-ef81-492e-bb0b-ff556f867709 admin admin] PATH_INFO: `/v3/projects/6e1c22c3b00f4178a869cd2664770221/users/cfbee39bc79b4e82add22264be359ecd/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:33.972275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:33.993322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd258a0a-ef81-492e-bb0b-ff556f867709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1c22c3b00f4178a869cd2664770221, user_id=cfbee39bc79b4e82add22264be359ecd, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:33.998021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd258a0a-ef81-492e-bb0b-ff556f867709 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:34.015213 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1529/3028] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:00:33 2022] PUT /identity/v3/projects/6e1c22c3b00f4178a869cd2664770221/users/cfbee39bc79b4e82add22264be359ecd/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:00:34.019419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912162e-dc5b-460a-8496-5d1c8ea71354 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:34.032915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912162e-dc5b-460a-8496-5d1c8ea71354 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:34.076357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:34.141958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82568992-713c-489b-8d1a-19c708f6b57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:34.142713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82568992-713c-489b-8d1a-19c708f6b57c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:34.143296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82568992-713c-489b-8d1a-19c708f6b57c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:34.143568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82568992-713c-489b-8d1a-19c708f6b57c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:34.144291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82568992-713c-489b-8d1a-19c708f6b57c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:34.148757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82568992-713c-489b-8d1a-19c708f6b57c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:34.165225 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1500/3029] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:00:34.170644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd258a0a-ef81-492e-bb0b-ff556f867709 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:34.189582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd258a0a-ef81-492e-bb0b-ff556f867709 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:34.196854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075d4de-4a8e-423b-8573-5f3e4e48609e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:34.199436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075d4de-4a8e-423b-8573-5f3e4e48609e admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:34.199436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075d4de-4a8e-423b-8573-5f3e4e48609e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:34.199436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075d4de-4a8e-423b-8573-5f3e4e48609e admin admin] PATH_INFO: `/v3/projects/6e1c22c3b00f4178a869cd2664770221/users/cfbee39bc79b4e82add22264be359ecd/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:34.204531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075d4de-4a8e-423b-8573-5f3e4e48609e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1c22c3b00f4178a869cd2664770221, user_id=cfbee39bc79b4e82add22264be359ecd, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:34.208338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075d4de-4a8e-423b-8573-5f3e4e48609e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:34.234937 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1530/3030] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:00:34 2022] PUT /identity/v3/projects/6e1c22c3b00f4178a869cd2664770221/users/cfbee39bc79b4e82add22264be359ecd/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:34.246947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54607a-b9eb-4db9-b4a7-058d6c9e0fd7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:34.247288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54607a-b9eb-4db9-b4a7-058d6c9e0fd7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:34.247670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54607a-b9eb-4db9-b4a7-058d6c9e0fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:34.296045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ae54607a-b9eb-4db9-b4a7-058d6c9e0fd7 None None] MFA Rules not processed for user `cfbee39bc79b4e82add22264be359ecd`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:34.297685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ae54607a-b9eb-4db9-b4a7-058d6c9e0fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:34.405249 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1501/3031] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:34.407540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2bc8c-cace-4fce-8559-270e1ceb8421 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:34.407820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2bc8c-cace-4fce-8559-270e1ceb8421 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:34.408092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2bc8c-cace-4fce-8559-270e1ceb8421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:34.429989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a3d2bc8c-cace-4fce-8559-270e1ceb8421 None None] MFA Rules not processed for user `334fcd5a53b04e21b107fafebcfa98cf`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:34.431500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a3d2bc8c-cace-4fce-8559-270e1ceb8421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:34.498059 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1531/3032] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:34.505659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54607a-b9eb-4db9-b4a7-058d6c9e0fd7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:34.507218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ae54607a-b9eb-4db9-b4a7-058d6c9e0fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:34.709621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54607a-b9eb-4db9-b4a7-058d6c9e0fd7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:34.739069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30402c-5d76-4fce-8b96-bcba6e51c541 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:34.739730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30402c-5d76-4fce-8b96-bcba6e51c541 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:34.740750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30402c-5d76-4fce-8b96-bcba6e51c541 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:34.741069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30402c-5d76-4fce-8b96-bcba6e51c541 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:34.758609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30402c-5d76-4fce-8b96-bcba6e51c541 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:34.762282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30402c-5d76-4fce-8b96-bcba6e51c541 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:34.779395 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1502/3033] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:37.174812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2bc8c-cace-4fce-8559-270e1ceb8421 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:37.197019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2bc8c-cace-4fce-8559-270e1ceb8421 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:37.204463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafb9d4-f287-424e-9bc1-f1728ad799ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:37.206949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb9d4-f287-424e-9bc1-f1728ad799ee service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:37.207313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb9d4-f287-424e-9bc1-f1728ad799ee service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:37.207643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb9d4-f287-424e-9bc1-f1728ad799ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:37.209787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8cafb9d4-f287-424e-9bc1-f1728ad799ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:37.235149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafb9d4-f287-424e-9bc1-f1728ad799ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:37.237444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafb9d4-f287-424e-9bc1-f1728ad799ee service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:37.260624 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1532/3034] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:40.387417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddf781-27f4-4ea5-9584-d52ab25dc28e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:40.387865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddf781-27f4-4ea5-9584-d52ab25dc28e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:40.388914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddf781-27f4-4ea5-9584-d52ab25dc28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:40.446029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ccddf781-27f4-4ea5-9584-d52ab25dc28e None None] MFA Rules not processed for user `a211978552be446dbf3992f8df096477`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:40.448729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ccddf781-27f4-4ea5-9584-d52ab25dc28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:40.482568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:40.618303 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1503/3035] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:40.628414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f78e3-c4fb-4aa4-bb6d-10769053fcec None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:40.628921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f78e3-c4fb-4aa4-bb6d-10769053fcec None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:40.628921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f78e3-c4fb-4aa4-bb6d-10769053fcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:40.672284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c22f78e3-c4fb-4aa4-bb6d-10769053fcec None None] MFA Rules not processed for user `a211978552be446dbf3992f8df096477`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:40.673637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c22f78e3-c4fb-4aa4-bb6d-10769053fcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:40.693816 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1533/3036] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:40.794388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccddf781-27f4-4ea5-9584-d52ab25dc28e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:40.844554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccddf781-27f4-4ea5-9584-d52ab25dc28e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:40.851205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e246fb5-eb81-488c-9873-54e44402a492 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:40.851859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e246fb5-eb81-488c-9873-54e44402a492 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:40.852082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e246fb5-eb81-488c-9873-54e44402a492 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:40.852343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e246fb5-eb81-488c-9873-54e44402a492 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:40.864924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e246fb5-eb81-488c-9873-54e44402a492 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:40.867299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e246fb5-eb81-488c-9873-54e44402a492 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:40.880188 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1504/3037] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:42.078912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f78e3-c4fb-4aa4-bb6d-10769053fcec None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:42.080359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c22f78e3-c4fb-4aa4-bb6d-10769053fcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:42.209885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f78e3-c4fb-4aa4-bb6d-10769053fcec None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:42.221725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1c4c4-93c8-4234-8a98-a22661850fe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:42.222435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1c4c4-93c8-4234-8a98-a22661850fe8 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:42.222760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1c4c4-93c8-4234-8a98-a22661850fe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:42.225410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1c4c4-93c8-4234-8a98-a22661850fe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:42.233501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cdd1c4c4-93c8-4234-8a98-a22661850fe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:42.269532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1c4c4-93c8-4234-8a98-a22661850fe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:42.271932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1c4c4-93c8-4234-8a98-a22661850fe8 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:42.297757 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1534/3038] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:43.491548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e246fb5-eb81-488c-9873-54e44402a492 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:43.537786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e246fb5-eb81-488c-9873-54e44402a492 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:43.560058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4b6ad-53c9-442c-9b64-a3d3f82b468e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:43.561703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4b6ad-53c9-442c-9b64-a3d3f82b468e service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:43.562521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4b6ad-53c9-442c-9b64-a3d3f82b468e service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:43.563695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4b6ad-53c9-442c-9b64-a3d3f82b468e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:43.570692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e0e4b6ad-53c9-442c-9b64-a3d3f82b468e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:43.606527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4b6ad-53c9-442c-9b64-a3d3f82b468e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:43.615169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4b6ad-53c9-442c-9b64-a3d3f82b468e service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:43.668728 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1505/3039] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:52.286603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=71,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:53.120860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1c4c4-93c8-4234-8a98-a22661850fe8 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:53.141716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1c4c4-93c8-4234-8a98-a22661850fe8 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:53.158051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7554470-0337-4ece-ac3b-7ecee173a5b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:53.158896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7554470-0337-4ece-ac3b-7ecee173a5b8 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:53.159170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7554470-0337-4ece-ac3b-7ecee173a5b8 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:53.159434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7554470-0337-4ece-ac3b-7ecee173a5b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:53.161695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b7554470-0337-4ece-ac3b-7ecee173a5b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:53.182957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7554470-0337-4ece-ac3b-7ecee173a5b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:53.185329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7554470-0337-4ece-ac3b-7ecee173a5b8 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:53.217118 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1535/3040] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:53.646443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=30 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:55.756996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91025b-c0a2-4127-a6d0-1ea0e83e6777 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:55.758180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91025b-c0a2-4127-a6d0-1ea0e83e6777 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:55.758671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91025b-c0a2-4127-a6d0-1ea0e83e6777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:55.785887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0a91025b-c0a2-4127-a6d0-1ea0e83e6777 None None] MFA Rules not processed for user `08ec4ea53cf84dec83652bf484ab577c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:55.787772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0a91025b-c0a2-4127-a6d0-1ea0e83e6777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:55.868565 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1506/3041] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:55.895422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7554470-0337-4ece-ac3b-7ecee173a5b8 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:55.897263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b7554470-0337-4ece-ac3b-7ecee173a5b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:56.021810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7554470-0337-4ece-ac3b-7ecee173a5b8 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:56.032842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cfaad-45e9-45b1-9b9a-1e8df2a6357b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:56.034428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cfaad-45e9-45b1-9b9a-1e8df2a6357b service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:56.034713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cfaad-45e9-45b1-9b9a-1e8df2a6357b service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:56.034986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cfaad-45e9-45b1-9b9a-1e8df2a6357b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:56.037362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-668cfaad-45e9-45b1-9b9a-1e8df2a6357b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:56.055858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cfaad-45e9-45b1-9b9a-1e8df2a6357b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:56.059364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cfaad-45e9-45b1-9b9a-1e8df2a6357b service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:56.079992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91025b-c0a2-4127-a6d0-1ea0e83e6777 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:56.081457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0a91025b-c0a2-4127-a6d0-1ea0e83e6777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:56.084672 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1536/3042] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:56.234257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91025b-c0a2-4127-a6d0-1ea0e83e6777 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:56.261100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:56.266253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:56.266651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:56.267091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] PATH_INFO: `/v3/users/3e574b1c33bf4726942002eb8d4dd7dc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:56.270319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e574b1c33bf4726942002eb8d4dd7dc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:56.275582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:56.344979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e574b1c33bf4726942002eb8d4dd7dc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:56.360504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e574b1c33bf4726942002eb8d4dd7dc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:56.378865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e574b1c33bf4726942002eb8d4dd7dc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:56.379879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e574b1c33bf4726942002eb8d4dd7dc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:56.380667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e574b1c33bf4726942002eb8d4dd7dc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:56.414249 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1507/3043] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:00:56 2022] DELETE /identity/v3/users/3e574b1c33bf4726942002eb8d4dd7dc => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:56.559026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cfaad-45e9-45b1-9b9a-1e8df2a6357b service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:56.576604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cfaad-45e9-45b1-9b9a-1e8df2a6357b service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:56.644069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:56.646074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:56.712397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82cfaa-1a23-4653-a662-3e193c459d7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:56.713620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82cfaa-1a23-4653-a662-3e193c459d7e service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:56.715385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82cfaa-1a23-4653-a662-3e193c459d7e service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:56.715817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82cfaa-1a23-4653-a662-3e193c459d7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:56.732014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-db82cfaa-1a23-4653-a662-3e193c459d7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:56.735150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:56.762971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82cfaa-1a23-4653-a662-3e193c459d7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:56.765468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82cfaa-1a23-4653-a662-3e193c459d7e service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:56.776427 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1537/3044] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:56.780111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa075d-9412-4bcd-8bfe-51fce1d515d5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:56.785100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6c487-32ad-4ef3-b165-4d8b9528de60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:56.785808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6c487-32ad-4ef3-b165-4d8b9528de60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:56.786142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6c487-32ad-4ef3-b165-4d8b9528de60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:56.786570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6c487-32ad-4ef3-b165-4d8b9528de60 admin admin] PATH_INFO: `/v3/projects/1471ca0ad0914fab8ba53f9b6f6b95a4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:56.787766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6c487-32ad-4ef3-b165-4d8b9528de60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1471ca0ad0914fab8ba53f9b6f6b95a4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:56.789782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6c487-32ad-4ef3-b165-4d8b9528de60 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:56.846839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-48c6c487-32ad-4ef3-b165-4d8b9528de60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1471ca0ad0914fab8ba53f9b6f6b95a4', 'request_id': 'req-48c6c487-32ad-4ef3-b165-4d8b9528de60'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:56.855379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-48c6c487-32ad-4ef3-b165-4d8b9528de60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1471ca0ad0914fab8ba53f9b6f6b95a4', 'request_id': 'req-48c6c487-32ad-4ef3-b165-4d8b9528de60'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:56.871271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-48c6c487-32ad-4ef3-b165-4d8b9528de60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1471ca0ad0914fab8ba53f9b6f6b95a4', 'request_id': 'req-48c6c487-32ad-4ef3-b165-4d8b9528de60'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:56.883359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-48c6c487-32ad-4ef3-b165-4d8b9528de60 admin admin] The token cache is being invalidate because project 1471ca0ad0914fab8ba53f9b6f6b95a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:00:56.883715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-48c6c487-32ad-4ef3-b165-4d8b9528de60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:56.884551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-48c6c487-32ad-4ef3-b165-4d8b9528de60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:00:56.886606 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1508/3045] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:00:56 2022] DELETE /identity/v3/projects/1471ca0ad0914fab8ba53f9b6f6b95a4 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:56.891396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112050bb-0707-46d9-9b46-e1648ea2cb04 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:56.891880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112050bb-0707-46d9-9b46-e1648ea2cb04 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:56.892424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112050bb-0707-46d9-9b46-e1648ea2cb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:56.915593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-112050bb-0707-46d9-9b46-e1648ea2cb04 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:56.916926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-112050bb-0707-46d9-9b46-e1648ea2cb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:56.998446 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1538/3046] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:00:57.000275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8c7a2-7b76-40fe-bc3e-aa8dcd6fa4f4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:57.000579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8c7a2-7b76-40fe-bc3e-aa8dcd6fa4f4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:57.000845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8c7a2-7b76-40fe-bc3e-aa8dcd6fa4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:57.052137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8cb8c7a2-7b76-40fe-bc3e-aa8dcd6fa4f4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:57.053678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8cb8c7a2-7b76-40fe-bc3e-aa8dcd6fa4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:57.077393 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1509/3047] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:57.084222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112050bb-0707-46d9-9b46-e1648ea2cb04 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:57.124091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112050bb-0707-46d9-9b46-e1648ea2cb04 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:57.168468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f828c-5fa8-4109-9222-5f6d0b8e79ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:57.169104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f828c-5fa8-4109-9222-5f6d0b8e79ab admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:57.169380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f828c-5fa8-4109-9222-5f6d0b8e79ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:57.169649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f828c-5fa8-4109-9222-5f6d0b8e79ab admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:57.170312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f828c-5fa8-4109-9222-5f6d0b8e79ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:00:57.170599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f828c-5fa8-4109-9222-5f6d0b8e79ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:57.179339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8c7a2-7b76-40fe-bc3e-aa8dcd6fa4f4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:57.181403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8cb8c7a2-7b76-40fe-bc3e-aa8dcd6fa4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:57.187382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f828c-5fa8-4109-9222-5f6d0b8e79ab admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:57.224118 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1539/3048] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:00:57.225634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f828c-5fa8-4109-9222-5f6d0b8e79ab admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:57.238565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f828c-5fa8-4109-9222-5f6d0b8e79ab admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:57.247880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0935ee0-fbb8-4fcd-8bed-54473b8669bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:57.249499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0935ee0-fbb8-4fcd-8bed-54473b8669bc admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:57.250788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0935ee0-fbb8-4fcd-8bed-54473b8669bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:57.251239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0935ee0-fbb8-4fcd-8bed-54473b8669bc admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:57.252180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0935ee0-fbb8-4fcd-8bed-54473b8669bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:57.255065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0935ee0-fbb8-4fcd-8bed-54473b8669bc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:57.261724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:57.299792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8c7a2-7b76-40fe-bc3e-aa8dcd6fa4f4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:57.308259 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1540/3049] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:00:57 2022] POST /identity/v3/projects => generated 395 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:00:57.309265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0935ee0-fbb8-4fcd-8bed-54473b8669bc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:57.315461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad232ef7-dbce-43b5-bd14-b78eb1937a03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:57.316653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad232ef7-dbce-43b5-bd14-b78eb1937a03 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:57.316867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad232ef7-dbce-43b5-bd14-b78eb1937a03 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:57.317110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad232ef7-dbce-43b5-bd14-b78eb1937a03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:57.322264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ad232ef7-dbce-43b5-bd14-b78eb1937a03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:57.425349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:57.439852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0935ee0-fbb8-4fcd-8bed-54473b8669bc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:57.446642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1586e9a-5bd1-4469-b4ec-37dd7325d23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:57.447275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1586e9a-5bd1-4469-b4ec-37dd7325d23d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:57.447570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1586e9a-5bd1-4469-b4ec-37dd7325d23d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:57.447885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1586e9a-5bd1-4469-b4ec-37dd7325d23d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:57.449619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1586e9a-5bd1-4469-b4ec-37dd7325d23d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:57.451600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1586e9a-5bd1-4469-b4ec-37dd7325d23d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:57.466979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad232ef7-dbce-43b5-bd14-b78eb1937a03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:57.468672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad232ef7-dbce-43b5-bd14-b78eb1937a03 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:57.483042 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1541/3050] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:00:57 2022] POST /identity/v3/users => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:00:57.488852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1586e9a-5bd1-4469-b4ec-37dd7325d23d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:57.491992 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1510/3051] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:57.501735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1586e9a-5bd1-4469-b4ec-37dd7325d23d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:57.511588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87138c-46a2-4dd5-b77d-16ee4ef548e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:57.512219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87138c-46a2-4dd5-b77d-16ee4ef548e8 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:57.512578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87138c-46a2-4dd5-b77d-16ee4ef548e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:57.513009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87138c-46a2-4dd5-b77d-16ee4ef548e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:57.513676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87138c-46a2-4dd5-b77d-16ee4ef548e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:57.515809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87138c-46a2-4dd5-b77d-16ee4ef548e8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:57.535534 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1542/3052] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:00:57.535661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad232ef7-dbce-43b5-bd14-b78eb1937a03 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:57.548612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad232ef7-dbce-43b5-bd14-b78eb1937a03 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:57.563557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a628fb-4f5f-4acb-9056-a9192cecfc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:57.564181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a628fb-4f5f-4acb-9056-a9192cecfc08 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:57.564475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a628fb-4f5f-4acb-9056-a9192cecfc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:57.564734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a628fb-4f5f-4acb-9056-a9192cecfc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:57.565377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a628fb-4f5f-4acb-9056-a9192cecfc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:57.566963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a628fb-4f5f-4acb-9056-a9192cecfc08 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:57.585745 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1511/3053] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:00:57.589977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87138c-46a2-4dd5-b77d-16ee4ef548e8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:57.602562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87138c-46a2-4dd5-b77d-16ee4ef548e8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:57.609219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfbe4c-214c-4ffa-b174-bd099d5746d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:57.610432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfbe4c-214c-4ffa-b174-bd099d5746d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:57.610876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfbe4c-214c-4ffa-b174-bd099d5746d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:57.611378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfbe4c-214c-4ffa-b174-bd099d5746d9 admin admin] PATH_INFO: `/v3/projects/e6aba867732b442d898a52a733253a29/users/0762310138c74902a6ece83b71b123b3/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:57.631369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfbe4c-214c-4ffa-b174-bd099d5746d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6aba867732b442d898a52a733253a29, user_id=0762310138c74902a6ece83b71b123b3, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:57.634500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfbe4c-214c-4ffa-b174-bd099d5746d9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:57.667138 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1543/3054] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:00:57 2022] PUT /identity/v3/projects/e6aba867732b442d898a52a733253a29/users/0762310138c74902a6ece83b71b123b3/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:00:57.672136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823bee2-1cfd-4458-a8f9-63844cdb8d33 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:57.672472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823bee2-1cfd-4458-a8f9-63844cdb8d33 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:57.672741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823bee2-1cfd-4458-a8f9-63844cdb8d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:57.713553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f823bee2-1cfd-4458-a8f9-63844cdb8d33 None None] MFA Rules not processed for user `0762310138c74902a6ece83b71b123b3`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:57.715432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f823bee2-1cfd-4458-a8f9-63844cdb8d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:57.830293 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1512/3055] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:57 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:57.840194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82256808-a300-40a5-b199-563e920060b8 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:57.840592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82256808-a300-40a5-b199-563e920060b8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:57.841086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82256808-a300-40a5-b199-563e920060b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:57.874196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-82256808-a300-40a5-b199-563e920060b8 None None] MFA Rules not processed for user `0762310138c74902a6ece83b71b123b3`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:57.876329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-82256808-a300-40a5-b199-563e920060b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:57.899731 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1544/3056] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:57 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:58.378076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823bee2-1cfd-4458-a8f9-63844cdb8d33 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:00:58.544716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823bee2-1cfd-4458-a8f9-63844cdb8d33 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:00:58.564549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a461d68-2d10-41ce-a6e6-9cc2302ab4ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:00:58.565384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a461d68-2d10-41ce-a6e6-9cc2302ab4ff service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:58.565662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a461d68-2d10-41ce-a6e6-9cc2302ab4ff service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:58.566052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a461d68-2d10-41ce-a6e6-9cc2302ab4ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:58.587801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a461d68-2d10-41ce-a6e6-9cc2302ab4ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:00:58.590693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a461d68-2d10-41ce-a6e6-9cc2302ab4ff service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:00:58.615474 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1513/3057] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:00:58 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:00:58.654904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326809f6-6210-425b-b841-61846103689c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:58.656203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326809f6-6210-425b-b841-61846103689c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:58.657122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326809f6-6210-425b-b841-61846103689c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:58.707872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-326809f6-6210-425b-b841-61846103689c None None] MFA Rules not processed for user `0762310138c74902a6ece83b71b123b3`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:58.712584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-326809f6-6210-425b-b841-61846103689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:58.731443 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1545/3058] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:58 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:58.924697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce119948-99d7-4c6e-9120-7615da827770 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:58.925238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce119948-99d7-4c6e-9120-7615da827770 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:58.925238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce119948-99d7-4c6e-9120-7615da827770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:58.954815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840726b-bae0-4eef-b238-987e9ef42c8d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:58.955086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840726b-bae0-4eef-b238-987e9ef42c8d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:58.955356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840726b-bae0-4eef-b238-987e9ef42c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:58.956579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ce119948-99d7-4c6e-9120-7615da827770 None None] MFA Rules not processed for user `334fcd5a53b04e21b107fafebcfa98cf`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:58.957790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ce119948-99d7-4c6e-9120-7615da827770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:59.006058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4840726b-bae0-4eef-b238-987e9ef42c8d None None] MFA Rules not processed for user `0762310138c74902a6ece83b71b123b3`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:59.009032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4840726b-bae0-4eef-b238-987e9ef42c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:59.023280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:00:59.032991 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1546/3059] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:58 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:59.046557 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1514/3060] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:59.133968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c055db-174b-4ee8-8a8b-e781b3a51e08 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:59.134298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c055db-174b-4ee8-8a8b-e781b3a51e08 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:59.134942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c055db-174b-4ee8-8a8b-e781b3a51e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:59.208304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-82c055db-174b-4ee8-8a8b-e781b3a51e08 None None] MFA Rules not processed for user `0762310138c74902a6ece83b71b123b3`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:59.215845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-82c055db-174b-4ee8-8a8b-e781b3a51e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:59.242377 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1547/3061] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:59 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:00:59.803725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9a9c3-bde9-4bae-9491-3aef3b025aec None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:00:59.804116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9a9c3-bde9-4bae-9491-3aef3b025aec None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:00:59.804520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9a9c3-bde9-4bae-9491-3aef3b025aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:00:59.854759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4ae9a9c3-bde9-4bae-9491-3aef3b025aec None None] MFA Rules not processed for user `0762310138c74902a6ece83b71b123b3`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:00:59.856180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4ae9a9c3-bde9-4bae-9491-3aef3b025aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:00:59.882603 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1515/3062] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:00:59 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:00.248646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c055db-174b-4ee8-8a8b-e781b3a51e08 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:00.411928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c055db-174b-4ee8-8a8b-e781b3a51e08 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:00.424242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:00.424940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:00.425215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:00.425481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] PATH_INFO: `/v3/users/0762310138c74902a6ece83b71b123b3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:00.428362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0762310138c74902a6ece83b71b123b3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:00.430315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:00.461709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:00.534145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0762310138c74902a6ece83b71b123b3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:00.551788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0762310138c74902a6ece83b71b123b3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:00.560427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0762310138c74902a6ece83b71b123b3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:00.592448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0762310138c74902a6ece83b71b123b3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:00.594306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0762310138c74902a6ece83b71b123b3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:00.610633 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1548/3063] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:01:00 2022] DELETE /identity/v3/users/0762310138c74902a6ece83b71b123b3 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:00.615163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785be68-c826-433d-af9a-6c8097dd17cd None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:00.615662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785be68-c826-433d-af9a-6c8097dd17cd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:00.615662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785be68-c826-433d-af9a-6c8097dd17cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:00.665240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3785be68-c826-433d-af9a-6c8097dd17cd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:00.670260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3785be68-c826-433d-af9a-6c8097dd17cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:00.810165 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1516/3064] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:01:00.847335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:00.849338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:01.131485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabb3b8-5f0e-49b1-9b99-0267bd46d1c8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:01.149844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02232749-7972-429e-bce2-616da2032b75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:01.152932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02232749-7972-429e-bce2-616da2032b75 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:01.153208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02232749-7972-429e-bce2-616da2032b75 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:01.153476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02232749-7972-429e-bce2-616da2032b75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:01.183107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02232749-7972-429e-bce2-616da2032b75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:01.184546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02232749-7972-429e-bce2-616da2032b75 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:01.210435 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1549/3065] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:01:03.259364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785be68-c826-433d-af9a-6c8097dd17cd None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:03.261899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3785be68-c826-433d-af9a-6c8097dd17cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:03.303487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785be68-c826-433d-af9a-6c8097dd17cd None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:03.323176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aeaa94-cc64-42ca-b7df-1ee99048c543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:03.324673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeaa94-cc64-42ca-b7df-1ee99048c543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:03.325172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeaa94-cc64-42ca-b7df-1ee99048c543 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:03.329332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeaa94-cc64-42ca-b7df-1ee99048c543 admin admin] PATH_INFO: `/v3/projects/e6aba867732b442d898a52a733253a29` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:03.341934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aeaa94-cc64-42ca-b7df-1ee99048c543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6aba867732b442d898a52a733253a29)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:03.344318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aeaa94-cc64-42ca-b7df-1ee99048c543 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:03.467396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b0aeaa94-cc64-42ca-b7df-1ee99048c543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6aba867732b442d898a52a733253a29', 'request_id': 'req-b0aeaa94-cc64-42ca-b7df-1ee99048c543'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:03.488724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b0aeaa94-cc64-42ca-b7df-1ee99048c543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6aba867732b442d898a52a733253a29', 'request_id': 'req-b0aeaa94-cc64-42ca-b7df-1ee99048c543'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:03.516891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b0aeaa94-cc64-42ca-b7df-1ee99048c543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6aba867732b442d898a52a733253a29', 'request_id': 'req-b0aeaa94-cc64-42ca-b7df-1ee99048c543'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:03.545753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b0aeaa94-cc64-42ca-b7df-1ee99048c543 admin admin] The token cache is being invalidate because project e6aba867732b442d898a52a733253a29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:01:03.546041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b0aeaa94-cc64-42ca-b7df-1ee99048c543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:03.546897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b0aeaa94-cc64-42ca-b7df-1ee99048c543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:03.549178 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1517/3066] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:01:03 2022] DELETE /identity/v3/projects/e6aba867732b442d898a52a733253a29 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:03.554498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb804d-186f-41dd-b78e-2dce18a2817e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:03.555402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb804d-186f-41dd-b78e-2dce18a2817e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:03.555637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb804d-186f-41dd-b78e-2dce18a2817e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:03.630541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cbeb804d-186f-41dd-b78e-2dce18a2817e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:03.631780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cbeb804d-186f-41dd-b78e-2dce18a2817e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:03.781530 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1550/3067] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:03.789866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920845c-2744-4f4d-b761-2068d5d163a4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:03.790118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920845c-2744-4f4d-b761-2068d5d163a4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:03.790600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920845c-2744-4f4d-b761-2068d5d163a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:03.828024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1920845c-2744-4f4d-b761-2068d5d163a4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:03.831516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1920845c-2744-4f4d-b761-2068d5d163a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:03.856680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb804d-186f-41dd-b78e-2dce18a2817e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:03.870872 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1518/3068] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:03.872314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb804d-186f-41dd-b78e-2dce18a2817e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:03.877780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e63d81-2f42-4862-943c-8ea4b6aa4e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:03.878410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e63d81-2f42-4862-943c-8ea4b6aa4e23 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:03.878679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e63d81-2f42-4862-943c-8ea4b6aa4e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:03.878944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e63d81-2f42-4862-943c-8ea4b6aa4e23 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:03.880228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e63d81-2f42-4862-943c-8ea4b6aa4e23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:01:03.880561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e63d81-2f42-4862-943c-8ea4b6aa4e23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:03.882273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e63d81-2f42-4862-943c-8ea4b6aa4e23 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:03.901556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920845c-2744-4f4d-b761-2068d5d163a4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:03.908460 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1551/3069] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:01:03.921423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920845c-2744-4f4d-b761-2068d5d163a4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:03.924936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62891f11-8e38-4285-8e3b-66240483949a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:03.925347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62891f11-8e38-4285-8e3b-66240483949a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:03.925709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62891f11-8e38-4285-8e3b-66240483949a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:03.937373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0f4d0-86d6-4d34-b94e-12b7209d9815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:03.939629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0f4d0-86d6-4d34-b94e-12b7209d9815 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:03.939802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0f4d0-86d6-4d34-b94e-12b7209d9815 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:03.941599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0f4d0-86d6-4d34-b94e-12b7209d9815 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:03.942899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0f4d0-86d6-4d34-b94e-12b7209d9815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:03.948593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0f4d0-86d6-4d34-b94e-12b7209d9815 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:04.002158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-62891f11-8e38-4285-8e3b-66240483949a None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:04.003857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-62891f11-8e38-4285-8e3b-66240483949a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:04.020197 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1519/3070] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:01:03 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:01:04.024396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0f4d0-86d6-4d34-b94e-12b7209d9815 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:04.037189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:04.132838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:04.161767 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1552/3071] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:01:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:04.173070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0f4d0-86d6-4d34-b94e-12b7209d9815 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:04.190279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143126b7-b1f3-46e6-ae46-97cbef3c80b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:04.190914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143126b7-b1f3-46e6-ae46-97cbef3c80b6 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:04.191325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143126b7-b1f3-46e6-ae46-97cbef3c80b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:04.191593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143126b7-b1f3-46e6-ae46-97cbef3c80b6 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:04.192222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143126b7-b1f3-46e6-ae46-97cbef3c80b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:04.194178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143126b7-b1f3-46e6-ae46-97cbef3c80b6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:04.215607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62891f11-8e38-4285-8e3b-66240483949a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:04.217035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-62891f11-8e38-4285-8e3b-66240483949a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:04.256573 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1520/3072] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:01:04 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:01:04.257534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143126b7-b1f3-46e6-ae46-97cbef3c80b6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:04.285861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143126b7-b1f3-46e6-ae46-97cbef3c80b6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:04.300041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0266d7e-9924-407c-a9c8-d21930032f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:04.301143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0266d7e-9924-407c-a9c8-d21930032f8c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:04.301343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0266d7e-9924-407c-a9c8-d21930032f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:04.301611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0266d7e-9924-407c-a9c8-d21930032f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:04.302261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0266d7e-9924-407c-a9c8-d21930032f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:04.309250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0266d7e-9924-407c-a9c8-d21930032f8c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:04.342012 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1521/3073] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:01:04.343300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0266d7e-9924-407c-a9c8-d21930032f8c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:04.359040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0266d7e-9924-407c-a9c8-d21930032f8c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:04.364611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9484e-061f-4643-8581-9d3d0882570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:04.365241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9484e-061f-4643-8581-9d3d0882570c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:04.365767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9484e-061f-4643-8581-9d3d0882570c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:04.366062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9484e-061f-4643-8581-9d3d0882570c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:04.366844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9484e-061f-4643-8581-9d3d0882570c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:04.368712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9484e-061f-4643-8581-9d3d0882570c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:04.392826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62891f11-8e38-4285-8e3b-66240483949a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:04.401680 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1522/3074] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:01:04.406404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9484e-061f-4643-8581-9d3d0882570c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:04.421923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9484e-061f-4643-8581-9d3d0882570c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:04.454076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f50cd-21b9-47eb-a8e9-73dbf7f7136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:04.455081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f50cd-21b9-47eb-a8e9-73dbf7f7136c admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:04.473406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f50cd-21b9-47eb-a8e9-73dbf7f7136c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:04.473775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f50cd-21b9-47eb-a8e9-73dbf7f7136c admin admin] PATH_INFO: `/v3/projects/b1584c72d057406ea00af9d0277fa47f/users/bbc87613dd6349babc15334a66bee64f/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:04.500027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f50cd-21b9-47eb-a8e9-73dbf7f7136c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1584c72d057406ea00af9d0277fa47f, user_id=bbc87613dd6349babc15334a66bee64f, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:04.504480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f50cd-21b9-47eb-a8e9-73dbf7f7136c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:04.557493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded5802-cf01-45f1-a1f6-26246f83305a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:04.558202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded5802-cf01-45f1-a1f6-26246f83305a service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:04.558496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded5802-cf01-45f1-a1f6-26246f83305a service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:04.558684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded5802-cf01-45f1-a1f6-26246f83305a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:04.559645 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1523/3075] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:01:04 2022] PUT /identity/v3/projects/b1584c72d057406ea00af9d0277fa47f/users/bbc87613dd6349babc15334a66bee64f/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:04.567659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b0b98-aca7-484a-9b27-4a80e1418161 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:04.569610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b0b98-aca7-484a-9b27-4a80e1418161 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:04.571454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b0b98-aca7-484a-9b27-4a80e1418161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:04.626977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-437b0b98-aca7-484a-9b27-4a80e1418161 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:04.629193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-437b0b98-aca7-484a-9b27-4a80e1418161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:04.665421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded5802-cf01-45f1-a1f6-26246f83305a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:04.667810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded5802-cf01-45f1-a1f6-26246f83305a service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:04.696599 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1553/3076] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:01:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:01:04.802350 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1524/3077] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:04.855960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded5802-cf01-45f1-a1f6-26246f83305a service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:04.859547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bded5802-cf01-45f1-a1f6-26246f83305a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:04.876041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:04.992251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded5802-cf01-45f1-a1f6-26246f83305a service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:05.085116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254bc31-3deb-4617-b0a1-1df2415d8811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:05.089569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254bc31-3deb-4617-b0a1-1df2415d8811 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:05.100994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254bc31-3deb-4617-b0a1-1df2415d8811 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:05.101990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254bc31-3deb-4617-b0a1-1df2415d8811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:05.125197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8254bc31-3deb-4617-b0a1-1df2415d8811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:05.127272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8254bc31-3deb-4617-b0a1-1df2415d8811 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:05.140228 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1554/3078] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:01:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:01:14.776142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=59,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:15.134546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=45 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:15.870154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a223a93-625c-4005-9978-171b259170ae None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:15.871051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a223a93-625c-4005-9978-171b259170ae None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:15.871542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a223a93-625c-4005-9978-171b259170ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:15.907134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8a223a93-625c-4005-9978-171b259170ae None None] MFA Rules not processed for user `811bc5f0350147f18454a1ac0127e47b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:15.910285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a223a93-625c-4005-9978-171b259170ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:16.111701 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1525/3079] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:18.062990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254bc31-3deb-4617-b0a1-1df2415d8811 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:18.064621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8254bc31-3deb-4617-b0a1-1df2415d8811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:18.293139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254bc31-3deb-4617-b0a1-1df2415d8811 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:18.307806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0ed10-9cf5-41fb-8389-6360abff9813 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:18.308864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0ed10-9cf5-41fb-8389-6360abff9813 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:18.309413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0ed10-9cf5-41fb-8389-6360abff9813 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:18.309901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0ed10-9cf5-41fb-8389-6360abff9813 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:18.312984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-adf0ed10-9cf5-41fb-8389-6360abff9813 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:18.334527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0ed10-9cf5-41fb-8389-6360abff9813 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:18.337774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0ed10-9cf5-41fb-8389-6360abff9813 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:18.361538 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1555/3080] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:01:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:01:23.735108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1091ac1-e087-4b72-b0b9-ab95e7848467 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:23.736734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1091ac1-e087-4b72-b0b9-ab95e7848467 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:23.737007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1091ac1-e087-4b72-b0b9-ab95e7848467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:23.791656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a1091ac1-e087-4b72-b0b9-ab95e7848467 None None] MFA Rules not processed for user `bbc87613dd6349babc15334a66bee64f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:23.793062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a1091ac1-e087-4b72-b0b9-ab95e7848467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:23.909861 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1526/3081] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:23.924404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0ed10-9cf5-41fb-8389-6360abff9813 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:23.926848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-adf0ed10-9cf5-41fb-8389-6360abff9813 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:24.147444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0ed10-9cf5-41fb-8389-6360abff9813 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:24.154979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7b893-1088-4d02-b14e-864d91450c53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:24.156562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7b893-1088-4d02-b14e-864d91450c53 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:24.157064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7b893-1088-4d02-b14e-864d91450c53 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:24.157968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7b893-1088-4d02-b14e-864d91450c53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:24.170901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7b893-1088-4d02-b14e-864d91450c53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:24.173002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7b893-1088-4d02-b14e-864d91450c53 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:24.199760 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1556/3082] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:01:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:01:24.604827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d32254-1a29-4127-bc5e-632886294831 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:24.605110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d32254-1a29-4127-bc5e-632886294831 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:24.605390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d32254-1a29-4127-bc5e-632886294831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:24.640103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f8d32254-1a29-4127-bc5e-632886294831 None None] MFA Rules not processed for user `8afc45f50dc54b84a8abf3b6e168426b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:24.641823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f8d32254-1a29-4127-bc5e-632886294831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:24.726026 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1527/3083] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:28.524248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7b893-1088-4d02-b14e-864d91450c53 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:28.525076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2eb7b893-1088-4d02-b14e-864d91450c53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:28.554766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7b893-1088-4d02-b14e-864d91450c53 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:28.581209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:28.582144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:28.582628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:28.582950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] PATH_INFO: `/v3/users/b91bab791d584fde8ce3f5c9c8b45ed4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:28.584166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b91bab791d584fde8ce3f5c9c8b45ed4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:28.586235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:28.645875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b91bab791d584fde8ce3f5c9c8b45ed4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:28.654427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b91bab791d584fde8ce3f5c9c8b45ed4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:28.661987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b91bab791d584fde8ce3f5c9c8b45ed4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:28.667499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:28.676183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b91bab791d584fde8ce3f5c9c8b45ed4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:28.677224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b91bab791d584fde8ce3f5c9c8b45ed4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:28.688612 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1557/3084] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:01:28 2022] DELETE /identity/v3/users/b91bab791d584fde8ce3f5c9c8b45ed4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:28.888316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0217f6-7ab0-4aab-9e82-96c7232a62d3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:28.888937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0217f6-7ab0-4aab-9e82-96c7232a62d3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:28.889133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0217f6-7ab0-4aab-9e82-96c7232a62d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:28.914106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5e0217f6-7ab0-4aab-9e82-96c7232a62d3 None None] MFA Rules not processed for user `334fcd5a53b04e21b107fafebcfa98cf`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:28.915601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5e0217f6-7ab0-4aab-9e82-96c7232a62d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:28.927434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:28.930220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:28.957729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:28.985453 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1528/3085] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:01:29.039739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415de9b-490d-455d-b9f4-4bc4b1087d6a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:29.054333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:29.055086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:29.055454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:29.055738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] PATH_INFO: `/v3/projects/44af76b79ce54fe584fe6f6a9680a3a4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:29.056985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44af76b79ce54fe584fe6f6a9680a3a4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:29.058907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:29.110550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44af76b79ce54fe584fe6f6a9680a3a4', 'request_id': 'req-95a99279-cf40-4642-ac17-5133c54096f9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.119001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44af76b79ce54fe584fe6f6a9680a3a4', 'request_id': 'req-95a99279-cf40-4642-ac17-5133c54096f9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.125146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44af76b79ce54fe584fe6f6a9680a3a4', 'request_id': 'req-95a99279-cf40-4642-ac17-5133c54096f9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.136856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] The token cache is being invalidate because project 44af76b79ce54fe584fe6f6a9680a3a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:01:29.137136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.137763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.139254 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1558/3086] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:01:28 2022] DELETE /identity/v3/projects/44af76b79ce54fe584fe6f6a9680a3a4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:01:29.143289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0217f6-7ab0-4aab-9e82-96c7232a62d3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:29.154853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0217f6-7ab0-4aab-9e82-96c7232a62d3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:29.217416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d8ef7-66f7-41c7-ba40-51dbba6fa4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:29.218013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d8ef7-66f7-41c7-ba40-51dbba6fa4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:29.218455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d8ef7-66f7-41c7-ba40-51dbba6fa4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:29.218800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d8ef7-66f7-41c7-ba40-51dbba6fa4a2 admin admin] PATH_INFO: `/v3/users/3c3972a48f7a473895fee3b574d22316` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:29.220038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d8ef7-66f7-41c7-ba40-51dbba6fa4a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c3972a48f7a473895fee3b574d22316)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:29.222103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d8ef7-66f7-41c7-ba40-51dbba6fa4a2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:29.282185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f73d8ef7-66f7-41c7-ba40-51dbba6fa4a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c3972a48f7a473895fee3b574d22316'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.290686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f73d8ef7-66f7-41c7-ba40-51dbba6fa4a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c3972a48f7a473895fee3b574d22316'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.304007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f73d8ef7-66f7-41c7-ba40-51dbba6fa4a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c3972a48f7a473895fee3b574d22316'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.304801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f73d8ef7-66f7-41c7-ba40-51dbba6fa4a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c3972a48f7a473895fee3b574d22316'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.305437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f73d8ef7-66f7-41c7-ba40-51dbba6fa4a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c3972a48f7a473895fee3b574d22316'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.319019 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1529/3087] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:01:29 2022] DELETE /identity/v3/users/3c3972a48f7a473895fee3b574d22316 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:01:29.516257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:29.518329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:29.534244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a99279-cf40-4642-ac17-5133c54096f9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:29.621092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351783b-afdf-46c2-816d-2facb99aea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:29.622019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351783b-afdf-46c2-816d-2facb99aea86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:29.622386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351783b-afdf-46c2-816d-2facb99aea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:29.622780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351783b-afdf-46c2-816d-2facb99aea86 admin admin] PATH_INFO: `/v3/projects/8dd2126e33bd4f6c96b8faa511f70215` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:29.624191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351783b-afdf-46c2-816d-2facb99aea86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd2126e33bd4f6c96b8faa511f70215)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:29.626292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351783b-afdf-46c2-816d-2facb99aea86 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:29.657525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:29.697441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f351783b-afdf-46c2-816d-2facb99aea86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd2126e33bd4f6c96b8faa511f70215', 'request_id': 'req-f351783b-afdf-46c2-816d-2facb99aea86'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.708053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f351783b-afdf-46c2-816d-2facb99aea86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd2126e33bd4f6c96b8faa511f70215', 'request_id': 'req-f351783b-afdf-46c2-816d-2facb99aea86'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.718390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f351783b-afdf-46c2-816d-2facb99aea86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd2126e33bd4f6c96b8faa511f70215', 'request_id': 'req-f351783b-afdf-46c2-816d-2facb99aea86'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.736701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f351783b-afdf-46c2-816d-2facb99aea86 admin admin] The token cache is being invalidate because project 8dd2126e33bd4f6c96b8faa511f70215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:01:29.738577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f351783b-afdf-46c2-816d-2facb99aea86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.740079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f351783b-afdf-46c2-816d-2facb99aea86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:29.750324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f3944-fba1-415d-85cf-d70f20de08d4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:29.750620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f3944-fba1-415d-85cf-d70f20de08d4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:29.750887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f3944-fba1-415d-85cf-d70f20de08d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:29.754570 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1559/3088] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:01:29 2022] DELETE /identity/v3/projects/8dd2126e33bd4f6c96b8faa511f70215 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:29.781077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-750f3944-fba1-415d-85cf-d70f20de08d4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:29.782323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-750f3944-fba1-415d-85cf-d70f20de08d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:29.878118 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1530/3089] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:29.888563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8281b2-18d9-4bbf-9f8b-95dc1ad9b924 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:29.888805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8281b2-18d9-4bbf-9f8b-95dc1ad9b924 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:29.889402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8281b2-18d9-4bbf-9f8b-95dc1ad9b924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:29.913331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8d8281b2-18d9-4bbf-9f8b-95dc1ad9b924 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:29.914533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8d8281b2-18d9-4bbf-9f8b-95dc1ad9b924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:29.925293 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1560/3090] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:29.930363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f3944-fba1-415d-85cf-d70f20de08d4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:29.937023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:29.941886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f3944-fba1-415d-85cf-d70f20de08d4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:29.948751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca598eb2-316d-4a59-8dc3-1ac12394f84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:29.949692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca598eb2-316d-4a59-8dc3-1ac12394f84c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:29.950382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca598eb2-316d-4a59-8dc3-1ac12394f84c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:29.950769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca598eb2-316d-4a59-8dc3-1ac12394f84c admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:29.951506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca598eb2-316d-4a59-8dc3-1ac12394f84c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:01:29.952128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca598eb2-316d-4a59-8dc3-1ac12394f84c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:29.954059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca598eb2-316d-4a59-8dc3-1ac12394f84c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:29.968946 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1531/3091] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:01:29.973687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8281b2-18d9-4bbf-9f8b-95dc1ad9b924 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:29.986130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8281b2-18d9-4bbf-9f8b-95dc1ad9b924 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:29.992621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69645c-dd8a-40bb-b557-8863a5a9da82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:29.994284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69645c-dd8a-40bb-b557-8863a5a9da82 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:29.994480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69645c-dd8a-40bb-b557-8863a5a9da82 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:29.994605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69645c-dd8a-40bb-b557-8863a5a9da82 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:29.995464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69645c-dd8a-40bb-b557-8863a5a9da82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:29.997461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69645c-dd8a-40bb-b557-8863a5a9da82 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:30.053072 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1561/3092] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:01:29 2022] POST /identity/v3/projects => generated 401 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:01:30.055131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca598eb2-316d-4a59-8dc3-1ac12394f84c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:30.075002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca598eb2-316d-4a59-8dc3-1ac12394f84c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:30.148254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7eb68-01ed-48d2-b8fa-d852ff6be29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:30.149302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7eb68-01ed-48d2-b8fa-d852ff6be29d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:30.149749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7eb68-01ed-48d2-b8fa-d852ff6be29d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:30.150402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7eb68-01ed-48d2-b8fa-d852ff6be29d admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:30.151013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7eb68-01ed-48d2-b8fa-d852ff6be29d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:30.153277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7eb68-01ed-48d2-b8fa-d852ff6be29d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:30.180079 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1532/3093] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:01:30 2022] POST /identity/v3/users => generated 301 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:01:30.187844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69645c-dd8a-40bb-b557-8863a5a9da82 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:30.202098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69645c-dd8a-40bb-b557-8863a5a9da82 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:30.207625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959fcf3-1157-4639-9756-6774a562d2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:30.208851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959fcf3-1157-4639-9756-6774a562d2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:30.209996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959fcf3-1157-4639-9756-6774a562d2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:30.210618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959fcf3-1157-4639-9756-6774a562d2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:30.212791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959fcf3-1157-4639-9756-6774a562d2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:30.214843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959fcf3-1157-4639-9756-6774a562d2c0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:30.227338 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1562/3094] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:01:30.232299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7eb68-01ed-48d2-b8fa-d852ff6be29d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:30.249382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7eb68-01ed-48d2-b8fa-d852ff6be29d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:30.283732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c2567-2784-4ae4-8f5e-a0d429d06a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:30.285135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c2567-2784-4ae4-8f5e-a0d429d06a7a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:30.285379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c2567-2784-4ae4-8f5e-a0d429d06a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:30.285648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c2567-2784-4ae4-8f5e-a0d429d06a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:30.286355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c2567-2784-4ae4-8f5e-a0d429d06a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:30.288542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c2567-2784-4ae4-8f5e-a0d429d06a7a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:30.305080 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1533/3095] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:01:30.306661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959fcf3-1157-4639-9756-6774a562d2c0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:30.318807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959fcf3-1157-4639-9756-6774a562d2c0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:30.327648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd3f55-ff02-4a64-a768-86f78d2cbca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:30.328925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3f55-ff02-4a64-a768-86f78d2cbca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:30.329321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3f55-ff02-4a64-a768-86f78d2cbca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:30.329594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3f55-ff02-4a64-a768-86f78d2cbca6 admin admin] PATH_INFO: `/v3/projects/33c233de25d54990bcd727f693138ac7/users/cd713888346d420eb6d191bb88ae1456/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:30.356801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd3f55-ff02-4a64-a768-86f78d2cbca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33c233de25d54990bcd727f693138ac7, user_id=cd713888346d420eb6d191bb88ae1456, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:30.361250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd3f55-ff02-4a64-a768-86f78d2cbca6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:30.403752 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1563/3096] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:01:30 2022] PUT /identity/v3/projects/33c233de25d54990bcd727f693138ac7/users/cd713888346d420eb6d191bb88ae1456/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:01:30.409391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d3bbc-c0a6-4050-a3ea-43a5b245cce0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:30.410077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d3bbc-c0a6-4050-a3ea-43a5b245cce0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:30.410077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d3bbc-c0a6-4050-a3ea-43a5b245cce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:30.442610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3f7d3bbc-c0a6-4050-a3ea-43a5b245cce0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:30.443914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3f7d3bbc-c0a6-4050-a3ea-43a5b245cce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:30.528664 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1534/3097] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:30.535480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd3f55-ff02-4a64-a768-86f78d2cbca6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:30.539280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e1dd3f55-ff02-4a64-a768-86f78d2cbca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:30.642481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd3f55-ff02-4a64-a768-86f78d2cbca6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:30.648112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25e889-b68e-4a9f-bc2b-6ab01b0a8fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:30.655046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25e889-b68e-4a9f-bc2b-6ab01b0a8fca service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:30.655046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25e889-b68e-4a9f-bc2b-6ab01b0a8fca service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:30.655046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25e889-b68e-4a9f-bc2b-6ab01b0a8fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:30.670992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25e889-b68e-4a9f-bc2b-6ab01b0a8fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:30.672852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25e889-b68e-4a9f-bc2b-6ab01b0a8fca service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:30.688001 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1564/3098] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:01:30.941725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc07d7d-4cb8-4526-a085-ce2ea2d797bf None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:30.942634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc07d7d-4cb8-4526-a085-ce2ea2d797bf None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:30.943132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc07d7d-4cb8-4526-a085-ce2ea2d797bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:30.973311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9cc07d7d-4cb8-4526-a085-ce2ea2d797bf None None] MFA Rules not processed for user `bbc87613dd6349babc15334a66bee64f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:30.976557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9cc07d7d-4cb8-4526-a085-ce2ea2d797bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:31.063226 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1535/3099] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:32.374510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25e889-b68e-4a9f-bc2b-6ab01b0a8fca service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:32.376360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5b25e889-b68e-4a9f-bc2b-6ab01b0a8fca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:32.393387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25e889-b68e-4a9f-bc2b-6ab01b0a8fca service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:32.410626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee08596-2535-4f08-b96e-e79ca989f955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:32.411277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee08596-2535-4f08-b96e-e79ca989f955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:32.411548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee08596-2535-4f08-b96e-e79ca989f955 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:32.411823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee08596-2535-4f08-b96e-e79ca989f955 admin admin] PATH_INFO: `/v3/users/a211978552be446dbf3992f8df096477` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:32.413052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee08596-2535-4f08-b96e-e79ca989f955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a211978552be446dbf3992f8df096477)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:32.414987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee08596-2535-4f08-b96e-e79ca989f955 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:32.529168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:32.539440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6ee08596-2535-4f08-b96e-e79ca989f955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a211978552be446dbf3992f8df096477'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:32.552321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6ee08596-2535-4f08-b96e-e79ca989f955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a211978552be446dbf3992f8df096477'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:32.568940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6ee08596-2535-4f08-b96e-e79ca989f955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a211978552be446dbf3992f8df096477'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:32.631720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6ee08596-2535-4f08-b96e-e79ca989f955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a211978552be446dbf3992f8df096477'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:32.632550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6ee08596-2535-4f08-b96e-e79ca989f955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a211978552be446dbf3992f8df096477'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:32.649282 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1565/3100] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:01:32 2022] DELETE /identity/v3/users/a211978552be446dbf3992f8df096477 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:33.382803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc07d7d-4cb8-4526-a085-ce2ea2d797bf None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:33.396487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc07d7d-4cb8-4526-a085-ce2ea2d797bf None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:33.411652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:33.539849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb700e-0d06-4340-8af9-5bde119f0a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:33.540589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb700e-0d06-4340-8af9-5bde119f0a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:33.541459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb700e-0d06-4340-8af9-5bde119f0a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:33.542390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb700e-0d06-4340-8af9-5bde119f0a4e admin admin] PATH_INFO: `/v3/projects/940ad92c29e64ac893be0ab2c12afe2e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:33.545443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb700e-0d06-4340-8af9-5bde119f0a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=940ad92c29e64ac893be0ab2c12afe2e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:33.550824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb700e-0d06-4340-8af9-5bde119f0a4e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:33.664217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03cb700e-0d06-4340-8af9-5bde119f0a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '940ad92c29e64ac893be0ab2c12afe2e', 'request_id': 'req-03cb700e-0d06-4340-8af9-5bde119f0a4e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:33.700190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03cb700e-0d06-4340-8af9-5bde119f0a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '940ad92c29e64ac893be0ab2c12afe2e', 'request_id': 'req-03cb700e-0d06-4340-8af9-5bde119f0a4e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:33.716136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03cb700e-0d06-4340-8af9-5bde119f0a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '940ad92c29e64ac893be0ab2c12afe2e', 'request_id': 'req-03cb700e-0d06-4340-8af9-5bde119f0a4e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:33.737849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03cb700e-0d06-4340-8af9-5bde119f0a4e admin admin] The token cache is being invalidate because project 940ad92c29e64ac893be0ab2c12afe2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:01:33.737849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03cb700e-0d06-4340-8af9-5bde119f0a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:33.739360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03cb700e-0d06-4340-8af9-5bde119f0a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:33.743684 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1536/3101] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:01:33 2022] DELETE /identity/v3/projects/940ad92c29e64ac893be0ab2c12afe2e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:33.749060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2417804-8f4d-4766-afe1-7e5864b7a51c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:33.749634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2417804-8f4d-4766-afe1-7e5864b7a51c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:33.749634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2417804-8f4d-4766-afe1-7e5864b7a51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:33.787730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e2417804-8f4d-4766-afe1-7e5864b7a51c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:33.788994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e2417804-8f4d-4766-afe1-7e5864b7a51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:33.880655 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1566/3102] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:33.888882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21732c-773b-4a17-ad39-d7b1ed0bdd2a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:33.889167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21732c-773b-4a17-ad39-d7b1ed0bdd2a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:33.889441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21732c-773b-4a17-ad39-d7b1ed0bdd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:33.932710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ea21732c-773b-4a17-ad39-d7b1ed0bdd2a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:33.934071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ea21732c-773b-4a17-ad39-d7b1ed0bdd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:33.945483 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1537/3103] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:33.952111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2417804-8f4d-4766-afe1-7e5864b7a51c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:33.983493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2417804-8f4d-4766-afe1-7e5864b7a51c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:34.009375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36f7f7-1677-4bb4-a0ba-61e159da7405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:34.010995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36f7f7-1677-4bb4-a0ba-61e159da7405 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:34.011507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36f7f7-1677-4bb4-a0ba-61e159da7405 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:34.012396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36f7f7-1677-4bb4-a0ba-61e159da7405 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:34.014426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36f7f7-1677-4bb4-a0ba-61e159da7405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:01:34.015075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36f7f7-1677-4bb4-a0ba-61e159da7405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:34.017453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36f7f7-1677-4bb4-a0ba-61e159da7405 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:34.036309 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1567/3104] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:01:34.041555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21732c-773b-4a17-ad39-d7b1ed0bdd2a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:34.055178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21732c-773b-4a17-ad39-d7b1ed0bdd2a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:34.061868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89f085-f25b-4aaa-9ae6-1e371363b73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:34.062106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89f085-f25b-4aaa-9ae6-1e371363b73b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:34.062769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89f085-f25b-4aaa-9ae6-1e371363b73b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:34.063163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89f085-f25b-4aaa-9ae6-1e371363b73b admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:34.064015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89f085-f25b-4aaa-9ae6-1e371363b73b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:34.066172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89f085-f25b-4aaa-9ae6-1e371363b73b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:34.113453 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1538/3105] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:01:34 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:01:34.114508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36f7f7-1677-4bb4-a0ba-61e159da7405 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:34.127085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36f7f7-1677-4bb4-a0ba-61e159da7405 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:34.209138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb54ee7-cd28-4383-8875-c4256de69d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:34.209772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb54ee7-cd28-4383-8875-c4256de69d4c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:34.210132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb54ee7-cd28-4383-8875-c4256de69d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:34.210345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb54ee7-cd28-4383-8875-c4256de69d4c admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:34.211406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb54ee7-cd28-4383-8875-c4256de69d4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:34.214318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb54ee7-cd28-4383-8875-c4256de69d4c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:34.243812 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1568/3106] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:01:34 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:01:34.248033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89f085-f25b-4aaa-9ae6-1e371363b73b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:34.259562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89f085-f25b-4aaa-9ae6-1e371363b73b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:34.265741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bf483-03cc-4310-8ad1-6a1f433b16fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:34.266858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf483-03cc-4310-8ad1-6a1f433b16fe admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:34.266858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf483-03cc-4310-8ad1-6a1f433b16fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:34.266858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf483-03cc-4310-8ad1-6a1f433b16fe admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:34.267629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bf483-03cc-4310-8ad1-6a1f433b16fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:34.269409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bf483-03cc-4310-8ad1-6a1f433b16fe admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:34.308224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb54ee7-cd28-4383-8875-c4256de69d4c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:34.309104 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1539/3107] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:01:34.321213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb54ee7-cd28-4383-8875-c4256de69d4c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:34.326984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71fc03-855c-43b9-bd1b-8bf8187d5d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:34.328059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71fc03-855c-43b9-bd1b-8bf8187d5d09 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:34.328059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71fc03-855c-43b9-bd1b-8bf8187d5d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:34.328430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71fc03-855c-43b9-bd1b-8bf8187d5d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:34.334027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71fc03-855c-43b9-bd1b-8bf8187d5d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:34.335592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71fc03-855c-43b9-bd1b-8bf8187d5d09 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:34.365241 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1569/3108] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:01:34.369891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bf483-03cc-4310-8ad1-6a1f433b16fe admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:34.386512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bf483-03cc-4310-8ad1-6a1f433b16fe admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:34.395736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98101f6-fadb-4de0-ad2f-b730b2b05428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:34.396901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98101f6-fadb-4de0-ad2f-b730b2b05428 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:34.397386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98101f6-fadb-4de0-ad2f-b730b2b05428 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:34.397863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98101f6-fadb-4de0-ad2f-b730b2b05428 admin admin] PATH_INFO: `/v3/projects/53860884cef54be49cbf6d2e9aca37bb/users/01f695eccd4a43ba877c2d5d2a492284/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:34.453207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98101f6-fadb-4de0-ad2f-b730b2b05428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53860884cef54be49cbf6d2e9aca37bb, user_id=01f695eccd4a43ba877c2d5d2a492284, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:34.466120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98101f6-fadb-4de0-ad2f-b730b2b05428 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:34.503912 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1540/3109] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:01:34 2022] PUT /identity/v3/projects/53860884cef54be49cbf6d2e9aca37bb/users/01f695eccd4a43ba877c2d5d2a492284/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:34.505251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2a47a-4f0f-4a0d-8deb-bd316829c138 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:34.506306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2a47a-4f0f-4a0d-8deb-bd316829c138 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:34.506581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2a47a-4f0f-4a0d-8deb-bd316829c138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:34.569133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f2c2a47a-4f0f-4a0d-8deb-bd316829c138 None None] MFA Rules not processed for user `01f695eccd4a43ba877c2d5d2a492284`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:34.570666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f2c2a47a-4f0f-4a0d-8deb-bd316829c138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:34.604066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:34.717347 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1570/3110] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:34.742466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c79d39-77aa-4125-b3cd-7739dfd3124a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:34.743524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c79d39-77aa-4125-b3cd-7739dfd3124a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:34.743524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c79d39-77aa-4125-b3cd-7739dfd3124a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:34.805063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-01c79d39-77aa-4125-b3cd-7739dfd3124a None None] MFA Rules not processed for user `01f695eccd4a43ba877c2d5d2a492284`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:34.806318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-01c79d39-77aa-4125-b3cd-7739dfd3124a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:34.830644 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1541/3111] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:35.013060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2a47a-4f0f-4a0d-8deb-bd316829c138 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:35.017350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f2c2a47a-4f0f-4a0d-8deb-bd316829c138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:35.102113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2a47a-4f0f-4a0d-8deb-bd316829c138 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:35.109526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642104d-6ab3-4dfc-9128-597bf557e024 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:35.110675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642104d-6ab3-4dfc-9128-597bf557e024 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:35.111109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642104d-6ab3-4dfc-9128-597bf557e024 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:35.111545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642104d-6ab3-4dfc-9128-597bf557e024 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:35.138024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642104d-6ab3-4dfc-9128-597bf557e024 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:35.141344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642104d-6ab3-4dfc-9128-597bf557e024 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:35.153702 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1571/3112] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:01:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:01:42.155422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031b224-8b48-4048-96d7-fe8c897e4ad8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:42.156031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031b224-8b48-4048-96d7-fe8c897e4ad8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:42.156949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031b224-8b48-4048-96d7-fe8c897e4ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:42.213620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9031b224-8b48-4048-96d7-fe8c897e4ad8 None None] MFA Rules not processed for user `cd713888346d420eb6d191bb88ae1456`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:42.215957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9031b224-8b48-4048-96d7-fe8c897e4ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:42.261189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:42.373378 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1542/3113] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:42 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:42.760112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642104d-6ab3-4dfc-9128-597bf557e024 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:42.773090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642104d-6ab3-4dfc-9128-597bf557e024 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:42.779749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eccfb3-bb6b-41da-9f77-bdfe839d5468 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:42.780744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eccfb3-bb6b-41da-9f77-bdfe839d5468 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:42.781134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eccfb3-bb6b-41da-9f77-bdfe839d5468 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:42.781620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eccfb3-bb6b-41da-9f77-bdfe839d5468 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:42.792926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eccfb3-bb6b-41da-9f77-bdfe839d5468 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:42.795346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eccfb3-bb6b-41da-9f77-bdfe839d5468 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:42.808010 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1572/3114] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:01:42 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:01:48.125689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031b224-8b48-4048-96d7-fe8c897e4ad8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:48.329976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031b224-8b48-4048-96d7-fe8c897e4ad8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:48.339409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:48.340104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:48.340608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:48.341631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] PATH_INFO: `/v3/users/01f695eccd4a43ba877c2d5d2a492284` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:48.349001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f695eccd4a43ba877c2d5d2a492284)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:48.351293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:48.454286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f695eccd4a43ba877c2d5d2a492284'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:48.472499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f695eccd4a43ba877c2d5d2a492284'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:48.490885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f695eccd4a43ba877c2d5d2a492284'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:48.492234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f695eccd4a43ba877c2d5d2a492284'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:48.493091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f695eccd4a43ba877c2d5d2a492284'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:48.512993 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1543/3115] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:01:48 2022] DELETE /identity/v3/users/01f695eccd4a43ba877c2d5d2a492284 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:48.517742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1e3ac-2dc9-41c6-be5f-0fd809d51b75 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:48.518277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1e3ac-2dc9-41c6-be5f-0fd809d51b75 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:48.518277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1e3ac-2dc9-41c6-be5f-0fd809d51b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:48.540941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-84c1e3ac-2dc9-41c6-be5f-0fd809d51b75 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:48.542386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-84c1e3ac-2dc9-41c6-be5f-0fd809d51b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:48.620224 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1573/3116] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:01:48.631503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:48.641061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:48.733004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:48.765035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8349ce8-3bea-43c7-bff8-ec80ad6c1d69 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:48.772272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d68fd-00d6-43f0-82e7-4378ac4ef445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:48.773138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d68fd-00d6-43f0-82e7-4378ac4ef445 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:48.773488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d68fd-00d6-43f0-82e7-4378ac4ef445 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:48.773882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d68fd-00d6-43f0-82e7-4378ac4ef445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:48.785356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d68fd-00d6-43f0-82e7-4378ac4ef445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:48.786978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d68fd-00d6-43f0-82e7-4378ac4ef445 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:48.802441 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1544/3117] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:01:49.001475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1e3ac-2dc9-41c6-be5f-0fd809d51b75 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:49.003984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-84c1e3ac-2dc9-41c6-be5f-0fd809d51b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:49.038523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1e3ac-2dc9-41c6-be5f-0fd809d51b75 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:49.062515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf4ba1-24fe-4444-8e7f-fb4c75397fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:49.066301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf4ba1-24fe-4444-8e7f-fb4c75397fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:49.067999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf4ba1-24fe-4444-8e7f-fb4c75397fde admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:49.068837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf4ba1-24fe-4444-8e7f-fb4c75397fde admin admin] PATH_INFO: `/v3/users/cd713888346d420eb6d191bb88ae1456` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:49.074000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf4ba1-24fe-4444-8e7f-fb4c75397fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd713888346d420eb6d191bb88ae1456)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:49.077392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf4ba1-24fe-4444-8e7f-fb4c75397fde admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:49.131150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:49.180695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f3cf4ba1-24fe-4444-8e7f-fb4c75397fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd713888346d420eb6d191bb88ae1456'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.204918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f3cf4ba1-24fe-4444-8e7f-fb4c75397fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd713888346d420eb6d191bb88ae1456'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.223368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f3cf4ba1-24fe-4444-8e7f-fb4c75397fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd713888346d420eb6d191bb88ae1456'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.247671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f3cf4ba1-24fe-4444-8e7f-fb4c75397fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd713888346d420eb6d191bb88ae1456'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.248605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f3cf4ba1-24fe-4444-8e7f-fb4c75397fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd713888346d420eb6d191bb88ae1456'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.262456 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1574/3118] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:01:49 2022] DELETE /identity/v3/users/cd713888346d420eb6d191bb88ae1456 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:49.435011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d68fd-00d6-43f0-82e7-4378ac4ef445 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:49.437355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4b0d68fd-00d6-43f0-82e7-4378ac4ef445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:49.481912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d68fd-00d6-43f0-82e7-4378ac4ef445 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:49.564936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:49.565936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:49.566111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:49.566915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] PATH_INFO: `/v3/projects/53860884cef54be49cbf6d2e9aca37bb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:49.568949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53860884cef54be49cbf6d2e9aca37bb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:49.573114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:49.638528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53860884cef54be49cbf6d2e9aca37bb', 'request_id': 'req-44f7bcdf-e441-4a94-bef1-946c89625ba8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.645411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53860884cef54be49cbf6d2e9aca37bb', 'request_id': 'req-44f7bcdf-e441-4a94-bef1-946c89625ba8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.654762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53860884cef54be49cbf6d2e9aca37bb', 'request_id': 'req-44f7bcdf-e441-4a94-bef1-946c89625ba8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.667550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] The token cache is being invalidate because project 53860884cef54be49cbf6d2e9aca37bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:01:49.668336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.669214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.672331 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1545/3119] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:01:49 2022] DELETE /identity/v3/projects/53860884cef54be49cbf6d2e9aca37bb => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:49.687290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a5796-82ec-4f69-aa04-4e04c5ebebb5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:49.691073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a5796-82ec-4f69-aa04-4e04c5ebebb5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:49.693670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a5796-82ec-4f69-aa04-4e04c5ebebb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:49.707833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:49.709713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:49.725201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dd7a5796-82ec-4f69-aa04-4e04c5ebebb5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:49.727664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dd7a5796-82ec-4f69-aa04-4e04c5ebebb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:49.803201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7bcdf-e441-4a94-bef1-946c89625ba8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:49.809099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac89203-6130-41ff-bba8-106e2cd151e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:49.809769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac89203-6130-41ff-bba8-106e2cd151e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:49.810032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac89203-6130-41ff-bba8-106e2cd151e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:49.810468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac89203-6130-41ff-bba8-106e2cd151e4 admin admin] PATH_INFO: `/v3/projects/33c233de25d54990bcd727f693138ac7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:49.811661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac89203-6130-41ff-bba8-106e2cd151e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33c233de25d54990bcd727f693138ac7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:49.813670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac89203-6130-41ff-bba8-106e2cd151e4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:49.821169 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1575/3120] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:49.829510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b2b9f-5b4c-4490-9f10-25dca9dbced2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:49.829871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b2b9f-5b4c-4490-9f10-25dca9dbced2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:49.830202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b2b9f-5b4c-4490-9f10-25dca9dbced2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:49.842306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:49.858802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5a8b2b9f-5b4c-4490-9f10-25dca9dbced2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:49.860095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5a8b2b9f-5b4c-4490-9f10-25dca9dbced2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:49.870855 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1576/3121] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:49.877723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b2b9f-5b4c-4490-9f10-25dca9dbced2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:49.894517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b2b9f-5b4c-4490-9f10-25dca9dbced2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:49.904744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1ac89203-6130-41ff-bba8-106e2cd151e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33c233de25d54990bcd727f693138ac7', 'request_id': 'req-1ac89203-6130-41ff-bba8-106e2cd151e4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.913540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1ac89203-6130-41ff-bba8-106e2cd151e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33c233de25d54990bcd727f693138ac7', 'request_id': 'req-1ac89203-6130-41ff-bba8-106e2cd151e4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.923808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1ac89203-6130-41ff-bba8-106e2cd151e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33c233de25d54990bcd727f693138ac7', 'request_id': 'req-1ac89203-6130-41ff-bba8-106e2cd151e4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.937203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1ac89203-6130-41ff-bba8-106e2cd151e4 admin admin] The token cache is being invalidate because project 33c233de25d54990bcd727f693138ac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:01:49.937622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1ac89203-6130-41ff-bba8-106e2cd151e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.938401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1ac89203-6130-41ff-bba8-106e2cd151e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:49.940332 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1546/3122] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:01:49 2022] DELETE /identity/v3/projects/33c233de25d54990bcd727f693138ac7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:49.946236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975790f-8c15-4f42-a910-a13c074aac28 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:49.946600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975790f-8c15-4f42-a910-a13c074aac28 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:49.947424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975790f-8c15-4f42-a910-a13c074aac28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:49.973116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b81b04-6262-40c8-a645-e7562928e8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:49.974113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b81b04-6262-40c8-a645-e7562928e8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:49.974454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b81b04-6262-40c8-a645-e7562928e8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:49.974807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b81b04-6262-40c8-a645-e7562928e8f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:49.975537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b81b04-6262-40c8-a645-e7562928e8f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:01:49.975867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b81b04-6262-40c8-a645-e7562928e8f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:49.977589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b81b04-6262-40c8-a645-e7562928e8f9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:49.978475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6975790f-8c15-4f42-a910-a13c074aac28 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:49.981641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6975790f-8c15-4f42-a910-a13c074aac28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:49.992426 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1547/3123] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:50.000408 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1577/3124] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:01:50.000747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c7905-2a16-4a04-ac8f-b28658905806 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.000924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c7905-2a16-4a04-ac8f-b28658905806 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.001747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b81b04-6262-40c8-a645-e7562928e8f9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:50.001892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c7905-2a16-4a04-ac8f-b28658905806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.003839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b2b81b04-6262-40c8-a645-e7562928e8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:50.018181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b81b04-6262-40c8-a645-e7562928e8f9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:50.023164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4a2c-93fa-4a3b-afa5-bfc1063e467c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:50.023824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4a2c-93fa-4a3b-afa5-bfc1063e467c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.024091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4a2c-93fa-4a3b-afa5-bfc1063e467c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.024349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4a2c-93fa-4a3b-afa5-bfc1063e467c admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.025116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c4a2c-93fa-4a3b-afa5-bfc1063e467c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:50.027140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c4a2c-93fa-4a3b-afa5-bfc1063e467c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:50.027999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ec5c7905-2a16-4a04-ac8f-b28658905806 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:50.029239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ec5c7905-2a16-4a04-ac8f-b28658905806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:50.037200 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1548/3125] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:01:50.041658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c7905-2a16-4a04-ac8f-b28658905806 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:50.051798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c7905-2a16-4a04-ac8f-b28658905806 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:50.059646 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1578/3126] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:01:50 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:01:50.059826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98086ff3-91f7-4851-bc65-d52ec39a2135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:50.059826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98086ff3-91f7-4851-bc65-d52ec39a2135 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.059826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98086ff3-91f7-4851-bc65-d52ec39a2135 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.059826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98086ff3-91f7-4851-bc65-d52ec39a2135 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.060244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98086ff3-91f7-4851-bc65-d52ec39a2135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:01:50.060244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98086ff3-91f7-4851-bc65-d52ec39a2135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:50.060684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4a2c-93fa-4a3b-afa5-bfc1063e467c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:50.060827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98086ff3-91f7-4851-bc65-d52ec39a2135 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:50.067210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:50.074659 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1549/3127] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:01:50.079377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98086ff3-91f7-4851-bc65-d52ec39a2135 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:50.095677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98086ff3-91f7-4851-bc65-d52ec39a2135 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:50.136679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4a2c-93fa-4a3b-afa5-bfc1063e467c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:50.142516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040e50f-a0a1-4910-8a96-efbc435c4ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:50.143140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040e50f-a0a1-4910-8a96-efbc435c4ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.143415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040e50f-a0a1-4910-8a96-efbc435c4ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.143828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040e50f-a0a1-4910-8a96-efbc435c4ce7 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.144641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040e50f-a0a1-4910-8a96-efbc435c4ce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:50.146710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040e50f-a0a1-4910-8a96-efbc435c4ce7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:50.167688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd6dfc-eed4-40b1-853a-31148fdb5047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:50.168346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd6dfc-eed4-40b1-853a-31148fdb5047 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.168636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd6dfc-eed4-40b1-853a-31148fdb5047 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.168948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd6dfc-eed4-40b1-853a-31148fdb5047 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.184803 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1579/3128] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:01:50 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:01:50.185031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd6dfc-eed4-40b1-853a-31148fdb5047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:50.186516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040e50f-a0a1-4910-8a96-efbc435c4ce7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:50.191135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd6dfc-eed4-40b1-853a-31148fdb5047 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:50.205601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040e50f-a0a1-4910-8a96-efbc435c4ce7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:50.221038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a60b41-253d-4508-a4b5-a6adb700690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:50.221905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a60b41-253d-4508-a4b5-a6adb700690a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.222611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a60b41-253d-4508-a4b5-a6adb700690a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.222611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a60b41-253d-4508-a4b5-a6adb700690a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.229069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a60b41-253d-4508-a4b5-a6adb700690a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:50.229069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a60b41-253d-4508-a4b5-a6adb700690a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:50.244478 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1580/3129] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:01:50.249027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a60b41-253d-4508-a4b5-a6adb700690a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:50.263751 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1550/3130] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:01:50 2022] POST /identity/v3/projects => generated 395 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:01:50.265220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd6dfc-eed4-40b1-853a-31148fdb5047 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:50.274095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a60b41-253d-4508-a4b5-a6adb700690a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:50.350046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fb2cb-637c-4858-a66a-d5781e7a164a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:50.350921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb2cb-637c-4858-a66a-d5781e7a164a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.351296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb2cb-637c-4858-a66a-d5781e7a164a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.351927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb2cb-637c-4858-a66a-d5781e7a164a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.352527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fb2cb-637c-4858-a66a-d5781e7a164a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:50.354776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fb2cb-637c-4858-a66a-d5781e7a164a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:50.361726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd6dfc-eed4-40b1-853a-31148fdb5047 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:50.370086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add9ca3-6187-4921-acb1-15d38f5264df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:50.370853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add9ca3-6187-4921-acb1-15d38f5264df admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.371820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add9ca3-6187-4921-acb1-15d38f5264df admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.372182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add9ca3-6187-4921-acb1-15d38f5264df admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.373603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add9ca3-6187-4921-acb1-15d38f5264df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:50.375888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add9ca3-6187-4921-acb1-15d38f5264df admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:50.394471 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1581/3131] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:01:50.394471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fb2cb-637c-4858-a66a-d5781e7a164a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:50.394966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:50.408995 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1551/3132] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:01:50 2022] POST /identity/v3/users => generated 298 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:01:50.413272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add9ca3-6187-4921-acb1-15d38f5264df admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:50.420577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fb2cb-637c-4858-a66a-d5781e7a164a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:50.427280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add9ca3-6187-4921-acb1-15d38f5264df admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:50.455273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7025e2-9eaa-4d28-9428-219f6f8b249a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:50.455919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d06c64-8567-4167-a8d6-07db42789ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:50.456066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7025e2-9eaa-4d28-9428-219f6f8b249a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.456230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7025e2-9eaa-4d28-9428-219f6f8b249a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.456528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7025e2-9eaa-4d28-9428-219f6f8b249a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.456673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d06c64-8567-4167-a8d6-07db42789ad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.456903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d06c64-8567-4167-a8d6-07db42789ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.457171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d06c64-8567-4167-a8d6-07db42789ad9 admin admin] PATH_INFO: `/v3/projects/76984aa1ee96476db1708d912aca50da/users/a91a81826380481d912fbf590bae364a/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.457286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7025e2-9eaa-4d28-9428-219f6f8b249a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:50.461752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7025e2-9eaa-4d28-9428-219f6f8b249a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:50.475959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d06c64-8567-4167-a8d6-07db42789ad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76984aa1ee96476db1708d912aca50da, user_id=a91a81826380481d912fbf590bae364a, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:50.479019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d06c64-8567-4167-a8d6-07db42789ad9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:50.483488 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1552/3133] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:01:50.489617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7025e2-9eaa-4d28-9428-219f6f8b249a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:50.497126 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1582/3134] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:01:50 2022] PUT /identity/v3/projects/76984aa1ee96476db1708d912aca50da/users/a91a81826380481d912fbf590bae364a/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:01:50.502067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4d1fd-0ad6-4924-bb7a-42e6f12480c0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.502415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4d1fd-0ad6-4924-bb7a-42e6f12480c0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.502776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4d1fd-0ad6-4924-bb7a-42e6f12480c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.503138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7025e2-9eaa-4d28-9428-219f6f8b249a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:50.522451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e2f4d1fd-0ad6-4924-bb7a-42e6f12480c0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:50.523821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e2f4d1fd-0ad6-4924-bb7a-42e6f12480c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:50.592965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:50.594376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3b8f1-0ed1-45e6-9f23-2ea19e215f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:50.594586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3b8f1-0ed1-45e6-9f23-2ea19e215f53 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.594928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3b8f1-0ed1-45e6-9f23-2ea19e215f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.595229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3b8f1-0ed1-45e6-9f23-2ea19e215f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.596057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3b8f1-0ed1-45e6-9f23-2ea19e215f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:50.599602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3b8f1-0ed1-45e6-9f23-2ea19e215f53 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:50.615386 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1553/3135] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:01:50.619052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3b8f1-0ed1-45e6-9f23-2ea19e215f53 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:50.631002 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1583/3136] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:01:50.637637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3b8f1-0ed1-45e6-9f23-2ea19e215f53 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:50.644176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4d1fd-0ad6-4924-bb7a-42e6f12480c0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:50.644857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167e41a-2c41-4dee-8508-e2c73ff1f347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:50.646706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167e41a-2c41-4dee-8508-e2c73ff1f347 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.646984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167e41a-2c41-4dee-8508-e2c73ff1f347 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.647259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167e41a-2c41-4dee-8508-e2c73ff1f347 admin admin] PATH_INFO: `/v3/projects/e56dbf16b7ff4d43ade0ae2bf8d92bcc/users/45a676f646114328b4e78d1618481ebd/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.647400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e2f4d1fd-0ad6-4924-bb7a-42e6f12480c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:50.670056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167e41a-2c41-4dee-8508-e2c73ff1f347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e56dbf16b7ff4d43ade0ae2bf8d92bcc, user_id=45a676f646114328b4e78d1618481ebd, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:50.675415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167e41a-2c41-4dee-8508-e2c73ff1f347 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:50.700868 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1554/3137] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:01:50 2022] PUT /identity/v3/projects/e56dbf16b7ff4d43ade0ae2bf8d92bcc/users/45a676f646114328b4e78d1618481ebd/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:01:50.712669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7c23d-465a-4fec-8663-40622515a36e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.713125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7c23d-465a-4fec-8663-40622515a36e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.713478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7c23d-465a-4fec-8663-40622515a36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.724935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4d1fd-0ad6-4924-bb7a-42e6f12480c0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:50.756042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c2e7c23d-465a-4fec-8663-40622515a36e None None] MFA Rules not processed for user `45a676f646114328b4e78d1618481ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:50.757332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c2e7c23d-465a-4fec-8663-40622515a36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:50.809798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e130be4-4546-4031-8a98-049d8aeef55c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:50.810943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e130be4-4546-4031-8a98-049d8aeef55c service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.811402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e130be4-4546-4031-8a98-049d8aeef55c service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.811673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e130be4-4546-4031-8a98-049d8aeef55c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.853609 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1555/3138] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:01:50.883403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7c23d-465a-4fec-8663-40622515a36e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:50.886787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c2e7c23d-465a-4fec-8663-40622515a36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:50.888986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e130be4-4546-4031-8a98-049d8aeef55c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:50.890439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e130be4-4546-4031-8a98-049d8aeef55c service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:50.901872 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1584/3139] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:01:50.939349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:50.971584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7c23d-465a-4fec-8663-40622515a36e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:50.977296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def5fab-9095-4ae0-bc8f-90c265ab7f45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:50.977920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def5fab-9095-4ae0-bc8f-90c265ab7f45 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:50.978255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def5fab-9095-4ae0-bc8f-90c265ab7f45 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:50.978531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def5fab-9095-4ae0-bc8f-90c265ab7f45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:50.987653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def5fab-9095-4ae0-bc8f-90c265ab7f45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:50.989296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def5fab-9095-4ae0-bc8f-90c265ab7f45 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:50.998593 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1556/3140] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:01:50 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:01:51.351879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e130be4-4546-4031-8a98-049d8aeef55c service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:51.362994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e130be4-4546-4031-8a98-049d8aeef55c service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:51.375502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:51.376466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:51.376937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:51.377374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] PATH_INFO: `/v3/users/45a676f646114328b4e78d1618481ebd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:51.379484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45a676f646114328b4e78d1618481ebd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:51.381874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:51.452581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45a676f646114328b4e78d1618481ebd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:51.471664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45a676f646114328b4e78d1618481ebd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:51.475195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:51.489521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45a676f646114328b4e78d1618481ebd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:51.529384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45a676f646114328b4e78d1618481ebd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:51.530194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45a676f646114328b4e78d1618481ebd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:51.537858 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1585/3141] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:01:51 2022] DELETE /identity/v3/users/45a676f646114328b4e78d1618481ebd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:01:51.543653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceef0d-4cdd-4318-98a6-40b023304a06 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:51.544540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceef0d-4cdd-4318-98a6-40b023304a06 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:51.544969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceef0d-4cdd-4318-98a6-40b023304a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:51.571891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-01ceef0d-4cdd-4318-98a6-40b023304a06 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:51.573315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-01ceef0d-4cdd-4318-98a6-40b023304a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:51.649985 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1557/3142] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:51.676819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:51.678252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:51.748125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5765622-d617-402c-a138-39a531327f59 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:51.753449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bedaa-658e-463e-9f16-1d39e13c99cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:51.754389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bedaa-658e-463e-9f16-1d39e13c99cc service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:51.754797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bedaa-658e-463e-9f16-1d39e13c99cc service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:51.755066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bedaa-658e-463e-9f16-1d39e13c99cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:51.765887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bedaa-658e-463e-9f16-1d39e13c99cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:51.767598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bedaa-658e-463e-9f16-1d39e13c99cc service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:51.778997 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1586/3143] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:01:52.411438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceef0d-4cdd-4318-98a6-40b023304a06 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:52.413431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-01ceef0d-4cdd-4318-98a6-40b023304a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:52.440316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceef0d-4cdd-4318-98a6-40b023304a06 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:52.451453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a753a-2680-4c31-aee5-bab7eff1444b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:52.453268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a753a-2680-4c31-aee5-bab7eff1444b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:52.453268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a753a-2680-4c31-aee5-bab7eff1444b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:52.453268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a753a-2680-4c31-aee5-bab7eff1444b admin admin] PATH_INFO: `/v3/projects/e56dbf16b7ff4d43ade0ae2bf8d92bcc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:52.455208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a753a-2680-4c31-aee5-bab7eff1444b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e56dbf16b7ff4d43ade0ae2bf8d92bcc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:52.462116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a753a-2680-4c31-aee5-bab7eff1444b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:52.560867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bf2a753a-2680-4c31-aee5-bab7eff1444b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e56dbf16b7ff4d43ade0ae2bf8d92bcc', 'request_id': 'req-bf2a753a-2680-4c31-aee5-bab7eff1444b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:52.576226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bf2a753a-2680-4c31-aee5-bab7eff1444b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e56dbf16b7ff4d43ade0ae2bf8d92bcc', 'request_id': 'req-bf2a753a-2680-4c31-aee5-bab7eff1444b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:52.596033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bf2a753a-2680-4c31-aee5-bab7eff1444b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e56dbf16b7ff4d43ade0ae2bf8d92bcc', 'request_id': 'req-bf2a753a-2680-4c31-aee5-bab7eff1444b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:52.613906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bf2a753a-2680-4c31-aee5-bab7eff1444b admin admin] The token cache is being invalidate because project e56dbf16b7ff4d43ade0ae2bf8d92bcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:01:52.614475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bf2a753a-2680-4c31-aee5-bab7eff1444b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:52.615857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bf2a753a-2680-4c31-aee5-bab7eff1444b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:01:52.620086 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1558/3144] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:01:52 2022] DELETE /identity/v3/projects/e56dbf16b7ff4d43ade0ae2bf8d92bcc => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:52.625668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd25580-34c0-4ca2-aeac-076d6270d15d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:52.626420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd25580-34c0-4ca2-aeac-076d6270d15d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:52.626420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd25580-34c0-4ca2-aeac-076d6270d15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:52.663690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fcd25580-34c0-4ca2-aeac-076d6270d15d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:52.665074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fcd25580-34c0-4ca2-aeac-076d6270d15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:52.735637 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1587/3145] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:52.744154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6139d5-ecd2-40a3-ae21-fe0329971255 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:52.744828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6139d5-ecd2-40a3-ae21-fe0329971255 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:52.745319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6139d5-ecd2-40a3-ae21-fe0329971255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:52.774942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ea6139d5-ecd2-40a3-ae21-fe0329971255 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:52.776483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ea6139d5-ecd2-40a3-ae21-fe0329971255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:52.794862 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1559/3146] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:52.795898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd25580-34c0-4ca2-aeac-076d6270d15d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:52.820679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd25580-34c0-4ca2-aeac-076d6270d15d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:52.829556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b527a1-2866-47a8-abcd-ca13f6e53bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:52.830594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b527a1-2866-47a8-abcd-ca13f6e53bff admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:52.831088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b527a1-2866-47a8-abcd-ca13f6e53bff admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:52.831570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b527a1-2866-47a8-abcd-ca13f6e53bff admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:52.832500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b527a1-2866-47a8-abcd-ca13f6e53bff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:01:52.833019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b527a1-2866-47a8-abcd-ca13f6e53bff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:52.834873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b527a1-2866-47a8-abcd-ca13f6e53bff admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:52.850152 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1588/3147] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:01:52.855025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6139d5-ecd2-40a3-ae21-fe0329971255 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:52.884317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6139d5-ecd2-40a3-ae21-fe0329971255 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:52.891011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fad794-8459-46eb-97bd-286850a1ba92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:52.891693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fad794-8459-46eb-97bd-286850a1ba92 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:52.892849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fad794-8459-46eb-97bd-286850a1ba92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:52.893903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fad794-8459-46eb-97bd-286850a1ba92 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:52.895991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fad794-8459-46eb-97bd-286850a1ba92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:52.898994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fad794-8459-46eb-97bd-286850a1ba92 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:52.970778 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1560/3148] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:01:52 2022] POST /identity/v3/projects => generated 405 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:01:52.971296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b527a1-2866-47a8-abcd-ca13f6e53bff admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:52.987970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b527a1-2866-47a8-abcd-ca13f6e53bff admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:53.158875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc20ea-61a4-411d-8d4c-8215c9c6c65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:53.159506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc20ea-61a4-411d-8d4c-8215c9c6c65e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:53.159782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc20ea-61a4-411d-8d4c-8215c9c6c65e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:53.160048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc20ea-61a4-411d-8d4c-8215c9c6c65e admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:53.160718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc20ea-61a4-411d-8d4c-8215c9c6c65e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:53.162680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc20ea-61a4-411d-8d4c-8215c9c6c65e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:53.198944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:53.210168 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1589/3149] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:01:52 2022] POST /identity/v3/users => generated 303 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:01:53.214244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fad794-8459-46eb-97bd-286850a1ba92 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:53.228871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fad794-8459-46eb-97bd-286850a1ba92 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:53.236014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52676c-0afd-4c99-a7dc-ada31ceb2650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:53.237269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52676c-0afd-4c99-a7dc-ada31ceb2650 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:53.237841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52676c-0afd-4c99-a7dc-ada31ceb2650 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:53.238297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52676c-0afd-4c99-a7dc-ada31ceb2650 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:53.239498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52676c-0afd-4c99-a7dc-ada31ceb2650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:53.250202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52676c-0afd-4c99-a7dc-ada31ceb2650 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:53.266883 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1561/3150] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:01:53.268087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc20ea-61a4-411d-8d4c-8215c9c6c65e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:53.284318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc20ea-61a4-411d-8d4c-8215c9c6c65e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:53.296660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90265a9-6c96-401f-87a7-855f79fb3f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:53.297683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90265a9-6c96-401f-87a7-855f79fb3f94 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:53.298140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90265a9-6c96-401f-87a7-855f79fb3f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:53.298566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90265a9-6c96-401f-87a7-855f79fb3f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:53.299376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90265a9-6c96-401f-87a7-855f79fb3f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:53.301043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90265a9-6c96-401f-87a7-855f79fb3f94 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:53.332257 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1590/3151] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:01:53.336382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52676c-0afd-4c99-a7dc-ada31ceb2650 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:53.353390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52676c-0afd-4c99-a7dc-ada31ceb2650 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:53.369887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315d0c2-b0d5-4f9f-98e0-4fe539ccdb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:53.370931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315d0c2-b0d5-4f9f-98e0-4fe539ccdb44 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:53.371436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315d0c2-b0d5-4f9f-98e0-4fe539ccdb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:53.371855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315d0c2-b0d5-4f9f-98e0-4fe539ccdb44 admin admin] PATH_INFO: `/v3/projects/e5a52e36f01640f6a5dc79c6f33ec34c/users/c2a94bf67baa4bf6920f2655451048b8/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:53.391570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:01:53.403676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315d0c2-b0d5-4f9f-98e0-4fe539ccdb44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5a52e36f01640f6a5dc79c6f33ec34c, user_id=c2a94bf67baa4bf6920f2655451048b8, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:53.407068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315d0c2-b0d5-4f9f-98e0-4fe539ccdb44 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:53.449427 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1562/3152] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:01:53 2022] PUT /identity/v3/projects/e5a52e36f01640f6a5dc79c6f33ec34c/users/c2a94bf67baa4bf6920f2655451048b8/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:01:53.454143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73963108-8fec-40ad-8f42-dd62e96187e5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:53.454701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73963108-8fec-40ad-8f42-dd62e96187e5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:53.455193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73963108-8fec-40ad-8f42-dd62e96187e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:53.486147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-73963108-8fec-40ad-8f42-dd62e96187e5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:53.488098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-73963108-8fec-40ad-8f42-dd62e96187e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:53.607037 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1591/3153] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:01:53.622947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315d0c2-b0d5-4f9f-98e0-4fe539ccdb44 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:01:53.634465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6315d0c2-b0d5-4f9f-98e0-4fe539ccdb44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:53.763825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315d0c2-b0d5-4f9f-98e0-4fe539ccdb44 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:01:53.768135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11483cd-9f20-427a-9167-d83d3f8f00d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:01:53.768795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11483cd-9f20-427a-9167-d83d3f8f00d2 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:53.769637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11483cd-9f20-427a-9167-d83d3f8f00d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:53.769762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11483cd-9f20-427a-9167-d83d3f8f00d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:53.780795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11483cd-9f20-427a-9167-d83d3f8f00d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:01:53.782545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11483cd-9f20-427a-9167-d83d3f8f00d2 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:01:53.798068 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1563/3154] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:01:53.857294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7d92d-1e4d-4846-b2ab-65bf7f1bd3eb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:01:53.857888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7d92d-1e4d-4846-b2ab-65bf7f1bd3eb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:01:53.857888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7d92d-1e4d-4846-b2ab-65bf7f1bd3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:01:53.881131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-09a7d92d-1e4d-4846-b2ab-65bf7f1bd3eb None None] MFA Rules not processed for user `334fcd5a53b04e21b107fafebcfa98cf`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:01:53.882730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-09a7d92d-1e4d-4846-b2ab-65bf7f1bd3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:01:53.959050 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1592/3155] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:00.028419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4ada9-d08c-4d09-9651-dff2af4192ff None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:00.029358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4ada9-d08c-4d09-9651-dff2af4192ff None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:00.029866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4ada9-d08c-4d09-9651-dff2af4192ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:00.068087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-84d4ada9-d08c-4d09-9651-dff2af4192ff None None] MFA Rules not processed for user `08ec4ea53cf84dec83652bf484ab577c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:00.069395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-84d4ada9-d08c-4d09-9651-dff2af4192ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:00.219603 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1564/3156] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:01.721500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bea6f0-ddaf-43cf-aa11-62a788165ac1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:01.722397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bea6f0-ddaf-43cf-aa11-62a788165ac1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:01.722936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bea6f0-ddaf-43cf-aa11-62a788165ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:01.788856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d5bea6f0-ddaf-43cf-aa11-62a788165ac1 None None] MFA Rules not processed for user `a91a81826380481d912fbf590bae364a`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:01.790816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d5bea6f0-ddaf-43cf-aa11-62a788165ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:01.959781 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1593/3157] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:01.961898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f18b7b-d3d0-4517-8f81-bb1053242e2d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:01.962985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f18b7b-d3d0-4517-8f81-bb1053242e2d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:01.963159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f18b7b-d3d0-4517-8f81-bb1053242e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:01.989920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e1f18b7b-d3d0-4517-8f81-bb1053242e2d None None] MFA Rules not processed for user `a91a81826380481d912fbf590bae364a`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:01.992053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e1f18b7b-d3d0-4517-8f81-bb1053242e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:02.034356 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1565/3158] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:02.274186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bea6f0-ddaf-43cf-aa11-62a788165ac1 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:02.276325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d5bea6f0-ddaf-43cf-aa11-62a788165ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:02.399355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:02.554017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bea6f0-ddaf-43cf-aa11-62a788165ac1 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:02.565357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b586f-1977-4e57-9522-bdeda695491e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:02.566588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b586f-1977-4e57-9522-bdeda695491e service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:02.566588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b586f-1977-4e57-9522-bdeda695491e service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:02.566588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b586f-1977-4e57-9522-bdeda695491e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:02.584112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b586f-1977-4e57-9522-bdeda695491e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:02.591273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b586f-1977-4e57-9522-bdeda695491e service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:02.614710 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1594/3159] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:02:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:02:06.565321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef9476-c0de-41f1-b08e-6af89a947648 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:06.565845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef9476-c0de-41f1-b08e-6af89a947648 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:06.566101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef9476-c0de-41f1-b08e-6af89a947648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:06.657655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-34ef9476-c0de-41f1-b08e-6af89a947648 None None] MFA Rules not processed for user `c2a94bf67baa4bf6920f2655451048b8`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:06.658983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-34ef9476-c0de-41f1-b08e-6af89a947648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:06.821365 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1566/3160] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:06 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:06.831513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b586f-1977-4e57-9522-bdeda695491e service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:06.833877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8a6b586f-1977-4e57-9522-bdeda695491e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:06.867250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b586f-1977-4e57-9522-bdeda695491e service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:06.877037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b1c9c-c0a0-4c9c-9024-3c07ddb45fcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:06.878194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b1c9c-c0a0-4c9c-9024-3c07ddb45fcd service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:06.878466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b1c9c-c0a0-4c9c-9024-3c07ddb45fcd service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:06.878755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b1c9c-c0a0-4c9c-9024-3c07ddb45fcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:06.890438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b1c9c-c0a0-4c9c-9024-3c07ddb45fcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:06.892328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b1c9c-c0a0-4c9c-9024-3c07ddb45fcd service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:06.906185 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1595/3161] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:02:06 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:02:06.969328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef9476-c0de-41f1-b08e-6af89a947648 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:06.972642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-34ef9476-c0de-41f1-b08e-6af89a947648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:07.017305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:07.096404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef9476-c0de-41f1-b08e-6af89a947648 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:07.101297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d16b3-9264-4567-afe0-f5a49d4c3bb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:07.102828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d16b3-9264-4567-afe0-f5a49d4c3bb5 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:07.103609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d16b3-9264-4567-afe0-f5a49d4c3bb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:07.104083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d16b3-9264-4567-afe0-f5a49d4c3bb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:07.107228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2d2d16b3-9264-4567-afe0-f5a49d4c3bb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:07.217533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d16b3-9264-4567-afe0-f5a49d4c3bb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:07.221482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d16b3-9264-4567-afe0-f5a49d4c3bb5 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:07.237837 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1567/3162] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:02:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:02:14.481167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b96d18-ef50-45f4-afca-550a8c32f606 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:14.481809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b96d18-ef50-45f4-afca-550a8c32f606 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:14.481809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b96d18-ef50-45f4-afca-550a8c32f606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:14.502210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-77b96d18-ef50-45f4-afca-550a8c32f606 None None] MFA Rules not processed for user `334fcd5a53b04e21b107fafebcfa98cf`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:14.503442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-77b96d18-ef50-45f4-afca-550a8c32f606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:14.512411 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1596/3163] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:14.750510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d16b3-9264-4567-afe0-f5a49d4c3bb5 service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:14.765223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d16b3-9264-4567-afe0-f5a49d4c3bb5 service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:14.771266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122beb9-40d0-4599-9a29-397a83a5cad2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:14.772081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122beb9-40d0-4599-9a29-397a83a5cad2 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:14.772466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122beb9-40d0-4599-9a29-397a83a5cad2 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:14.772738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122beb9-40d0-4599-9a29-397a83a5cad2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:14.775674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7122beb9-40d0-4599-9a29-397a83a5cad2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:14.861958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122beb9-40d0-4599-9a29-397a83a5cad2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:14.863974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122beb9-40d0-4599-9a29-397a83a5cad2 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:14.876409 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1568/3164] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:02:14 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:02:21.616044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b96d18-ef50-45f4-afca-550a8c32f606 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:21.628808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b96d18-ef50-45f4-afca-550a8c32f606 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:21.638702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f0f79-6f01-4ea0-ad79-df68ea702008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:21.639530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0f79-6f01-4ea0-ad79-df68ea702008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:21.639896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0f79-6f01-4ea0-ad79-df68ea702008 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:21.640268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0f79-6f01-4ea0-ad79-df68ea702008 admin admin] PATH_INFO: `/v3/users/c2a94bf67baa4bf6920f2655451048b8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:21.641764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f0f79-6f01-4ea0-ad79-df68ea702008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2a94bf67baa4bf6920f2655451048b8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:21.644213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f0f79-6f01-4ea0-ad79-df68ea702008 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:21.706164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-105f0f79-6f01-4ea0-ad79-df68ea702008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2a94bf67baa4bf6920f2655451048b8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:21.717005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-105f0f79-6f01-4ea0-ad79-df68ea702008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2a94bf67baa4bf6920f2655451048b8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:21.724824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-105f0f79-6f01-4ea0-ad79-df68ea702008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2a94bf67baa4bf6920f2655451048b8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:21.739741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-105f0f79-6f01-4ea0-ad79-df68ea702008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2a94bf67baa4bf6920f2655451048b8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:21.740509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-105f0f79-6f01-4ea0-ad79-df68ea702008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2a94bf67baa4bf6920f2655451048b8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:21.749458 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1597/3165] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:02:21 2022] DELETE /identity/v3/users/c2a94bf67baa4bf6920f2655451048b8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:02:22.177122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122beb9-40d0-4599-9a29-397a83a5cad2 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:22.196043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122beb9-40d0-4599-9a29-397a83a5cad2 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:22.293451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:22.309854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1269e-a928-4825-b0a7-26775274820d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:22.311718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1269e-a928-4825-b0a7-26775274820d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:22.312643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1269e-a928-4825-b0a7-26775274820d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:22.313390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1269e-a928-4825-b0a7-26775274820d admin admin] PATH_INFO: `/v3/projects/e5a52e36f01640f6a5dc79c6f33ec34c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:22.315083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1269e-a928-4825-b0a7-26775274820d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5a52e36f01640f6a5dc79c6f33ec34c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:22.317781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1269e-a928-4825-b0a7-26775274820d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:22.389505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2cb1269e-a928-4825-b0a7-26775274820d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5a52e36f01640f6a5dc79c6f33ec34c', 'request_id': 'req-2cb1269e-a928-4825-b0a7-26775274820d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:22.398598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2cb1269e-a928-4825-b0a7-26775274820d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5a52e36f01640f6a5dc79c6f33ec34c', 'request_id': 'req-2cb1269e-a928-4825-b0a7-26775274820d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:22.414169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2cb1269e-a928-4825-b0a7-26775274820d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5a52e36f01640f6a5dc79c6f33ec34c', 'request_id': 'req-2cb1269e-a928-4825-b0a7-26775274820d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:22.429390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2cb1269e-a928-4825-b0a7-26775274820d admin admin] The token cache is being invalidate because project e5a52e36f01640f6a5dc79c6f33ec34c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:02:22.429765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2cb1269e-a928-4825-b0a7-26775274820d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:22.430531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2cb1269e-a928-4825-b0a7-26775274820d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:22.432885 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1569/3166] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:02:22 2022] DELETE /identity/v3/projects/e5a52e36f01640f6a5dc79c6f33ec34c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:02:22.438043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b6c59-e01e-42d3-bade-8949f8b04575 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:22.438318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b6c59-e01e-42d3-bade-8949f8b04575 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:22.439667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b6c59-e01e-42d3-bade-8949f8b04575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:22.471195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f31b6c59-e01e-42d3-bade-8949f8b04575 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:22.473013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f31b6c59-e01e-42d3-bade-8949f8b04575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:22.560599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:22.584681 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1598/3167] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:22.593766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1c0f-5bf9-4ae4-83ca-970ff8c93472 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:22.594822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1c0f-5bf9-4ae4-83ca-970ff8c93472 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:22.595320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1c0f-5bf9-4ae4-83ca-970ff8c93472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:22.627340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-527b1c0f-5bf9-4ae4-83ca-970ff8c93472 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:22.628892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-527b1c0f-5bf9-4ae4-83ca-970ff8c93472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:22.643280 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1570/3168] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:22.646213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b6c59-e01e-42d3-bade-8949f8b04575 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:22.657404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b6c59-e01e-42d3-bade-8949f8b04575 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:22.663755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad22b3b-efb2-4525-9f57-9faef91945e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:22.664593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad22b3b-efb2-4525-9f57-9faef91945e5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:22.665356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad22b3b-efb2-4525-9f57-9faef91945e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:22.666079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad22b3b-efb2-4525-9f57-9faef91945e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:22.667376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad22b3b-efb2-4525-9f57-9faef91945e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:02:22.668173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad22b3b-efb2-4525-9f57-9faef91945e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:22.671804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad22b3b-efb2-4525-9f57-9faef91945e5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:22.689868 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1599/3169] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:02:22.696545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b1c0f-5bf9-4ae4-83ca-970ff8c93472 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:22.709442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b1c0f-5bf9-4ae4-83ca-970ff8c93472 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:22.716284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61824598-eb3f-403b-b0b1-9b905e47ddcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:22.717064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61824598-eb3f-403b-b0b1-9b905e47ddcf admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:22.717339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61824598-eb3f-403b-b0b1-9b905e47ddcf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:22.717725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61824598-eb3f-403b-b0b1-9b905e47ddcf admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:22.718480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61824598-eb3f-403b-b0b1-9b905e47ddcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:22.720539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61824598-eb3f-403b-b0b1-9b905e47ddcf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:22.751672 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1571/3170] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:02:22 2022] POST /identity/v3/projects => generated 361 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:02:22.755971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad22b3b-efb2-4525-9f57-9faef91945e5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:22.767855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad22b3b-efb2-4525-9f57-9faef91945e5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:22.832843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85555ae7-a738-4045-9d22-b4e0fb4f497d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:22.833461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85555ae7-a738-4045-9d22-b4e0fb4f497d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:22.833759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85555ae7-a738-4045-9d22-b4e0fb4f497d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:22.833997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85555ae7-a738-4045-9d22-b4e0fb4f497d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:22.834632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85555ae7-a738-4045-9d22-b4e0fb4f497d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:22.836586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85555ae7-a738-4045-9d22-b4e0fb4f497d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:22.867028 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1600/3171] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:02:22 2022] POST /identity/v3/users => generated 281 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:02:22.871379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61824598-eb3f-403b-b0b1-9b905e47ddcf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:22.885146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61824598-eb3f-403b-b0b1-9b905e47ddcf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:22.893391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfefff-3b26-46cc-954d-7cb64247e9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:22.894015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfefff-3b26-46cc-954d-7cb64247e9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:22.894298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfefff-3b26-46cc-954d-7cb64247e9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:22.894998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfefff-3b26-46cc-954d-7cb64247e9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:22.895711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfefff-3b26-46cc-954d-7cb64247e9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:22.897361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfefff-3b26-46cc-954d-7cb64247e9a3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:22.912060 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1572/3172] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:02:22.913610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85555ae7-a738-4045-9d22-b4e0fb4f497d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:22.924960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85555ae7-a738-4045-9d22-b4e0fb4f497d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:22.935422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c275ae0-3f4b-4cc2-bab0-3aaede58d231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:22.936393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c275ae0-3f4b-4cc2-bab0-3aaede58d231 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:22.936752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c275ae0-3f4b-4cc2-bab0-3aaede58d231 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:22.937102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c275ae0-3f4b-4cc2-bab0-3aaede58d231 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:22.937766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c275ae0-3f4b-4cc2-bab0-3aaede58d231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:22.939693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c275ae0-3f4b-4cc2-bab0-3aaede58d231 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:22.959056 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1601/3173] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:02:22.964538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfefff-3b26-46cc-954d-7cb64247e9a3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:22.981397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfefff-3b26-46cc-954d-7cb64247e9a3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:22.989133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92873a69-d71d-42d8-9021-86a05398e7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:22.990199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92873a69-d71d-42d8-9021-86a05398e7a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:22.990672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92873a69-d71d-42d8-9021-86a05398e7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:22.991579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92873a69-d71d-42d8-9021-86a05398e7a6 admin admin] PATH_INFO: `/v3/projects/cfedaf15fd3e46b0af653a64ca4e6b77/users/8a2701ab6df941b8a657e7a9c9be37b5/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:23.015721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92873a69-d71d-42d8-9021-86a05398e7a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfedaf15fd3e46b0af653a64ca4e6b77, user_id=8a2701ab6df941b8a657e7a9c9be37b5, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:23.019011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92873a69-d71d-42d8-9021-86a05398e7a6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:23.038923 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1573/3174] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:02:22 2022] PUT /identity/v3/projects/cfedaf15fd3e46b0af653a64ca4e6b77/users/8a2701ab6df941b8a657e7a9c9be37b5/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:02:23.049896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d125220-41fd-4c20-945d-4f05c0fc2175 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:23.050181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d125220-41fd-4c20-945d-4f05c0fc2175 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:23.050477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d125220-41fd-4c20-945d-4f05c0fc2175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:23.092113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5d125220-41fd-4c20-945d-4f05c0fc2175 None None] MFA Rules not processed for user `8a2701ab6df941b8a657e7a9c9be37b5`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:23.093751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5d125220-41fd-4c20-945d-4f05c0fc2175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:23.201874 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1602/3175] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:23.212873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92873a69-d71d-42d8-9021-86a05398e7a6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:23.214603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-92873a69-d71d-42d8-9021-86a05398e7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:23.311825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92873a69-d71d-42d8-9021-86a05398e7a6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:23.318379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ec48d-86a3-47b3-a6f5-5b543a515c96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:23.319139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ec48d-86a3-47b3-a6f5-5b543a515c96 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:23.319487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ec48d-86a3-47b3-a6f5-5b543a515c96 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:23.319753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ec48d-86a3-47b3-a6f5-5b543a515c96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:23.329973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ec48d-86a3-47b3-a6f5-5b543a515c96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:23.331682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ec48d-86a3-47b3-a6f5-5b543a515c96 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:23.344077 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1574/3176] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:02:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:02:23.398842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d125220-41fd-4c20-945d-4f05c0fc2175 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:23.458036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:23.474604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d125220-41fd-4c20-945d-4f05c0fc2175 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:23.480553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:23.481465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:23.481917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:23.482283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] PATH_INFO: `/v3/users/8a2701ab6df941b8a657e7a9c9be37b5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:23.484073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a2701ab6df941b8a657e7a9c9be37b5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:23.486072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:23.542952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a2701ab6df941b8a657e7a9c9be37b5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:23.554648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a2701ab6df941b8a657e7a9c9be37b5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:23.566244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a2701ab6df941b8a657e7a9c9be37b5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:23.583186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a2701ab6df941b8a657e7a9c9be37b5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:23.584886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a2701ab6df941b8a657e7a9c9be37b5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:23.601878 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1603/3177] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:02:23 2022] DELETE /identity/v3/users/8a2701ab6df941b8a657e7a9c9be37b5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:02:23.606452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa59dc-4a82-4b8c-88a0-d049a7eddbd8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:23.607057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa59dc-4a82-4b8c-88a0-d049a7eddbd8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:23.607667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa59dc-4a82-4b8c-88a0-d049a7eddbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:23.628464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:23.640796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-57fa59dc-4a82-4b8c-88a0-d049a7eddbd8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:23.648972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-57fa59dc-4a82-4b8c-88a0-d049a7eddbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:23.746224 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1575/3178] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:23.756570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:23.758160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:23.827452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf2b07-caca-4f7c-bcae-cb8b7d3b6a53 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:23.831226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcfb79-46fa-4532-bb3f-c412236b238f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:23.831835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcfb79-46fa-4532-bb3f-c412236b238f service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:23.832135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcfb79-46fa-4532-bb3f-c412236b238f service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:23.832422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcfb79-46fa-4532-bb3f-c412236b238f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:23.841987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfcfb79-46fa-4532-bb3f-c412236b238f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:23.843430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfcfb79-46fa-4532-bb3f-c412236b238f service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:23.853957 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1604/3179] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:02:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:02:24.207639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa59dc-4a82-4b8c-88a0-d049a7eddbd8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:24.209347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-57fa59dc-4a82-4b8c-88a0-d049a7eddbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:24.223154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa59dc-4a82-4b8c-88a0-d049a7eddbd8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:24.229883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75be160-9b00-4f94-bf47-8c9c8e2996bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:24.230535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75be160-9b00-4f94-bf47-8c9c8e2996bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:24.231504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75be160-9b00-4f94-bf47-8c9c8e2996bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:24.231504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75be160-9b00-4f94-bf47-8c9c8e2996bc admin admin] PATH_INFO: `/v3/projects/cfedaf15fd3e46b0af653a64ca4e6b77` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:24.232566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75be160-9b00-4f94-bf47-8c9c8e2996bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfedaf15fd3e46b0af653a64ca4e6b77)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:24.234615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75be160-9b00-4f94-bf47-8c9c8e2996bc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:24.288139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e75be160-9b00-4f94-bf47-8c9c8e2996bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfedaf15fd3e46b0af653a64ca4e6b77', 'request_id': 'req-e75be160-9b00-4f94-bf47-8c9c8e2996bc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:24.294912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e75be160-9b00-4f94-bf47-8c9c8e2996bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfedaf15fd3e46b0af653a64ca4e6b77', 'request_id': 'req-e75be160-9b00-4f94-bf47-8c9c8e2996bc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:24.304953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e75be160-9b00-4f94-bf47-8c9c8e2996bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfedaf15fd3e46b0af653a64ca4e6b77', 'request_id': 'req-e75be160-9b00-4f94-bf47-8c9c8e2996bc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:24.317984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e75be160-9b00-4f94-bf47-8c9c8e2996bc admin admin] The token cache is being invalidate because project cfedaf15fd3e46b0af653a64ca4e6b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:02:24.318274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e75be160-9b00-4f94-bf47-8c9c8e2996bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:24.319056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e75be160-9b00-4f94-bf47-8c9c8e2996bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:24.321010 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1576/3180] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:02:24 2022] DELETE /identity/v3/projects/cfedaf15fd3e46b0af653a64ca4e6b77 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:02:24.327122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba69de4-c89d-42b2-81a0-bbbef279b139 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:24.327402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba69de4-c89d-42b2-81a0-bbbef279b139 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:24.327670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba69de4-c89d-42b2-81a0-bbbef279b139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:24.349916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4ba69de4-c89d-42b2-81a0-bbbef279b139 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:24.351210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4ba69de4-c89d-42b2-81a0-bbbef279b139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:24.424833 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1605/3181] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:02:24.435030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03c814-4bb7-40c6-9e74-685759b85f32 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:24.435314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03c814-4bb7-40c6-9e74-685759b85f32 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:24.435642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03c814-4bb7-40c6-9e74-685759b85f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:24.459369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4f03c814-4bb7-40c6-9e74-685759b85f32 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:24.460926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4f03c814-4bb7-40c6-9e74-685759b85f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:24.476172 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1577/3182] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:02:24.482563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba69de4-c89d-42b2-81a0-bbbef279b139 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:24.494571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba69de4-c89d-42b2-81a0-bbbef279b139 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:24.500636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f628a-6944-4b5b-aa25-601059179751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:24.501314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f628a-6944-4b5b-aa25-601059179751 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:24.501592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f628a-6944-4b5b-aa25-601059179751 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:24.501858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f628a-6944-4b5b-aa25-601059179751 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:24.502505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f628a-6944-4b5b-aa25-601059179751 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:02:24.502799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f628a-6944-4b5b-aa25-601059179751 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:24.504607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f628a-6944-4b5b-aa25-601059179751 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:24.516549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:24.525491 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1606/3183] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:02:24.528806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03c814-4bb7-40c6-9e74-685759b85f32 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:24.537212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03c814-4bb7-40c6-9e74-685759b85f32 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:24.543482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1e5f1-d883-462e-8f6f-8cfa8ec97196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:24.544158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1e5f1-d883-462e-8f6f-8cfa8ec97196 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:24.544587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1e5f1-d883-462e-8f6f-8cfa8ec97196 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:24.544854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1e5f1-d883-462e-8f6f-8cfa8ec97196 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:24.545513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1e5f1-d883-462e-8f6f-8cfa8ec97196 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:24.547415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1e5f1-d883-462e-8f6f-8cfa8ec97196 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:24.579271 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1578/3184] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:02:24 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:02:24.583672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f628a-6944-4b5b-aa25-601059179751 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:24.594545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f628a-6944-4b5b-aa25-601059179751 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:24.663235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119896c5-56be-4ac1-bab8-f05e1f28231f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:24.663898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119896c5-56be-4ac1-bab8-f05e1f28231f admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:24.664102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119896c5-56be-4ac1-bab8-f05e1f28231f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:24.664389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119896c5-56be-4ac1-bab8-f05e1f28231f admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:24.665069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119896c5-56be-4ac1-bab8-f05e1f28231f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:24.667013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119896c5-56be-4ac1-bab8-f05e1f28231f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:24.690599 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1607/3185] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:02:24 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:02:24.694754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1e5f1-d883-462e-8f6f-8cfa8ec97196 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:24.707963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1e5f1-d883-462e-8f6f-8cfa8ec97196 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:24.713323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d760ab-9069-4efa-a119-ab14455f21c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:24.713926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d760ab-9069-4efa-a119-ab14455f21c8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:24.714197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d760ab-9069-4efa-a119-ab14455f21c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:24.714459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d760ab-9069-4efa-a119-ab14455f21c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:24.715177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d760ab-9069-4efa-a119-ab14455f21c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:24.716985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d760ab-9069-4efa-a119-ab14455f21c8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:24.727936 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1579/3186] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:02:24.732855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119896c5-56be-4ac1-bab8-f05e1f28231f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:24.744292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119896c5-56be-4ac1-bab8-f05e1f28231f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:24.748888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec5600-e5cd-49d7-a278-f1ab881d2065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:24.749490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec5600-e5cd-49d7-a278-f1ab881d2065 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:24.749791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec5600-e5cd-49d7-a278-f1ab881d2065 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:24.750025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec5600-e5cd-49d7-a278-f1ab881d2065 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:24.750791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec5600-e5cd-49d7-a278-f1ab881d2065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:24.752391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec5600-e5cd-49d7-a278-f1ab881d2065 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:24.768439 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1608/3187] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:02:24.772546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d760ab-9069-4efa-a119-ab14455f21c8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:24.789428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d760ab-9069-4efa-a119-ab14455f21c8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:24.797148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382825df-a77a-45cd-bb6b-39cb756c1f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:24.798181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382825df-a77a-45cd-bb6b-39cb756c1f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:24.798589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382825df-a77a-45cd-bb6b-39cb756c1f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:24.799016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382825df-a77a-45cd-bb6b-39cb756c1f93 admin admin] PATH_INFO: `/v3/projects/bb300cc2384b4c63a23d57242f6704cb/users/815db4215e8b4b1984e199a40ed107c1/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:24.822886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382825df-a77a-45cd-bb6b-39cb756c1f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb300cc2384b4c63a23d57242f6704cb, user_id=815db4215e8b4b1984e199a40ed107c1, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:24.825763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382825df-a77a-45cd-bb6b-39cb756c1f93 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:24.843392 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1580/3188] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:02:24 2022] PUT /identity/v3/projects/bb300cc2384b4c63a23d57242f6704cb/users/815db4215e8b4b1984e199a40ed107c1/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:02:24.849829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1868b-e386-46d9-897c-3c41d8455132 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:24.850255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1868b-e386-46d9-897c-3c41d8455132 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:24.850530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1868b-e386-46d9-897c-3c41d8455132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:24.870777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-16c1868b-e386-46d9-897c-3c41d8455132 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:24.872325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-16c1868b-e386-46d9-897c-3c41d8455132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:24.968585 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1609/3189] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:24.977080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382825df-a77a-45cd-bb6b-39cb756c1f93 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:24.980592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-382825df-a77a-45cd-bb6b-39cb756c1f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:25.023670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:25.060492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382825df-a77a-45cd-bb6b-39cb756c1f93 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:25.065401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07daf77c-98ea-401b-a659-c2fcb94eafef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:25.067320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daf77c-98ea-401b-a659-c2fcb94eafef service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:25.067539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daf77c-98ea-401b-a659-c2fcb94eafef service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:25.067785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daf77c-98ea-401b-a659-c2fcb94eafef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:25.082558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07daf77c-98ea-401b-a659-c2fcb94eafef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:25.084361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07daf77c-98ea-401b-a659-c2fcb94eafef service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:25.098171 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1581/3190] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:02:28.280683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c63785-cff6-4004-8196-7af914a9dc4b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:28.281541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c63785-cff6-4004-8196-7af914a9dc4b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:28.281854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c63785-cff6-4004-8196-7af914a9dc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:28.323762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-58c63785-cff6-4004-8196-7af914a9dc4b None None] MFA Rules not processed for user `334fcd5a53b04e21b107fafebcfa98cf`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:28.325897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-58c63785-cff6-4004-8196-7af914a9dc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:28.418050 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1610/3191] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:31.364353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc75ad-4eb2-4f66-9b00-c56b4f9cfa7b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:31.365132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc75ad-4eb2-4f66-9b00-c56b4f9cfa7b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:31.365132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc75ad-4eb2-4f66-9b00-c56b4f9cfa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:31.409999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e5cc75ad-4eb2-4f66-9b00-c56b4f9cfa7b None None] MFA Rules not processed for user `815db4215e8b4b1984e199a40ed107c1`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:31.411447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e5cc75ad-4eb2-4f66-9b00-c56b4f9cfa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:31.533689 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1582/3192] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:31.541450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c63785-cff6-4004-8196-7af914a9dc4b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:31.553461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c63785-cff6-4004-8196-7af914a9dc4b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:31.558041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff2942-0818-45e6-91d1-d0de3796cd18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:31.558673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff2942-0818-45e6-91d1-d0de3796cd18 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:31.558945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff2942-0818-45e6-91d1-d0de3796cd18 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:31.559207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff2942-0818-45e6-91d1-d0de3796cd18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:31.570699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff2942-0818-45e6-91d1-d0de3796cd18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:31.572405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff2942-0818-45e6-91d1-d0de3796cd18 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:31.583074 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1611/3193] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:02:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:02:33.554568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f97bcb-8390-4d8d-8626-ee3b1833703f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:33.555682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f97bcb-8390-4d8d-8626-ee3b1833703f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:33.556789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f97bcb-8390-4d8d-8626-ee3b1833703f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:33.589067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-37f97bcb-8390-4d8d-8626-ee3b1833703f None None] MFA Rules not processed for user `815db4215e8b4b1984e199a40ed107c1`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:33.590742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-37f97bcb-8390-4d8d-8626-ee3b1833703f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:33.607649 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1583/3194] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:40.462466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff2942-0818-45e6-91d1-d0de3796cd18 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:40.465948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b2ff2942-0818-45e6-91d1-d0de3796cd18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:40.490020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff2942-0818-45e6-91d1-d0de3796cd18 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:40.512782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325cdcf-7c6b-425f-b470-afcbef43edd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:40.514815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325cdcf-7c6b-425f-b470-afcbef43edd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:40.515177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325cdcf-7c6b-425f-b470-afcbef43edd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:40.515550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325cdcf-7c6b-425f-b470-afcbef43edd7 admin admin] PATH_INFO: `/v3/users/08ec4ea53cf84dec83652bf484ab577c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:40.519260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325cdcf-7c6b-425f-b470-afcbef43edd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08ec4ea53cf84dec83652bf484ab577c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:40.521832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325cdcf-7c6b-425f-b470-afcbef43edd7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:40.528901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:40.624189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3325cdcf-7c6b-425f-b470-afcbef43edd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08ec4ea53cf84dec83652bf484ab577c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:40.649580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3325cdcf-7c6b-425f-b470-afcbef43edd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08ec4ea53cf84dec83652bf484ab577c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:40.683314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3325cdcf-7c6b-425f-b470-afcbef43edd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08ec4ea53cf84dec83652bf484ab577c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:40.733353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3325cdcf-7c6b-425f-b470-afcbef43edd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08ec4ea53cf84dec83652bf484ab577c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:40.736147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3325cdcf-7c6b-425f-b470-afcbef43edd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08ec4ea53cf84dec83652bf484ab577c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:40.751148 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1612/3195] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:02:40 2022] DELETE /identity/v3/users/08ec4ea53cf84dec83652bf484ab577c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:02:41.019556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f97bcb-8390-4d8d-8626-ee3b1833703f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:41.026202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f97bcb-8390-4d8d-8626-ee3b1833703f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:41.136550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:41.139628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:41.140010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:41.140823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d admin admin] PATH_INFO: `/v3/projects/c9247efa9b4f4d629272ddad62fd50e9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:41.145257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9247efa9b4f4d629272ddad62fd50e9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:41.149311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:41.239154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9247efa9b4f4d629272ddad62fd50e9', 'request_id': 'req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:41.248149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9247efa9b4f4d629272ddad62fd50e9', 'request_id': 'req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:41.251175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:41.262520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9247efa9b4f4d629272ddad62fd50e9', 'request_id': 'req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:41.278433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d admin admin] The token cache is being invalidate because project c9247efa9b4f4d629272ddad62fd50e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:02:41.278802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:41.279433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8eefcb3d-ab40-4de1-84b4-c7a54cf5a03d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:41.281559 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1584/3196] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:02:41 2022] DELETE /identity/v3/projects/c9247efa9b4f4d629272ddad62fd50e9 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:02:41.287024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c7256-1d46-4487-be21-00e7e8e41f25 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:41.287390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c7256-1d46-4487-be21-00e7e8e41f25 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:41.287626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c7256-1d46-4487-be21-00e7e8e41f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:41.318419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8f7c7256-1d46-4487-be21-00e7e8e41f25 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:41.320560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8f7c7256-1d46-4487-be21-00e7e8e41f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:41.444007 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1613/3197] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:41.453325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316809e8-c786-41ad-9956-a4673eb33406 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:41.454312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316809e8-c786-41ad-9956-a4673eb33406 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:41.454586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316809e8-c786-41ad-9956-a4673eb33406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:41.487605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-316809e8-c786-41ad-9956-a4673eb33406 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:41.489089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-316809e8-c786-41ad-9956-a4673eb33406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:41.509915 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1585/3198] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:41.515662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c7256-1d46-4487-be21-00e7e8e41f25 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:41.531195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c7256-1d46-4487-be21-00e7e8e41f25 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:41.541862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ca8db-a2bc-4ddc-969a-f91928e3d46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:41.542586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ca8db-a2bc-4ddc-969a-f91928e3d46d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:41.542744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ca8db-a2bc-4ddc-969a-f91928e3d46d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:41.544395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ca8db-a2bc-4ddc-969a-f91928e3d46d admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:41.545386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ca8db-a2bc-4ddc-969a-f91928e3d46d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:02:41.545782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ca8db-a2bc-4ddc-969a-f91928e3d46d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:41.547871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ca8db-a2bc-4ddc-969a-f91928e3d46d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:41.562312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316809e8-c786-41ad-9956-a4673eb33406 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:41.565033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-316809e8-c786-41ad-9956-a4673eb33406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:41.568125 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1614/3199] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:02:41.573084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ca8db-a2bc-4ddc-969a-f91928e3d46d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:41.585803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ca8db-a2bc-4ddc-969a-f91928e3d46d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:41.596783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca6759-fdfe-41db-b132-335e34a7a568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:41.597546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca6759-fdfe-41db-b132-335e34a7a568 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:41.598623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca6759-fdfe-41db-b132-335e34a7a568 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:41.598962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca6759-fdfe-41db-b132-335e34a7a568 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:41.599729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca6759-fdfe-41db-b132-335e34a7a568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:41.600567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316809e8-c786-41ad-9956-a4673eb33406 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:41.601905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca6759-fdfe-41db-b132-335e34a7a568 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:41.609077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:41.610568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:41.611005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:41.611958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] PATH_INFO: `/v3/users/a91a81826380481d912fbf590bae364a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:41.613402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a91a81826380481d912fbf590bae364a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:41.615719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:41.667129 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1615/3200] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:02:41 2022] POST /identity/v3/projects => generated 399 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:02:41.671257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca6759-fdfe-41db-b132-335e34a7a568 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:41.705936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a91a81826380481d912fbf590bae364a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:41.722192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a91a81826380481d912fbf590bae364a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:41.745055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a91a81826380481d912fbf590bae364a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:41.745901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a91a81826380481d912fbf590bae364a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:41.747072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a91a81826380481d912fbf590bae364a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:41.759522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:41.776751 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1586/3201] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:02:41 2022] DELETE /identity/v3/users/a91a81826380481d912fbf590bae364a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:02:41.794708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca6759-fdfe-41db-b132-335e34a7a568 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:41.897021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd5ef9-4be4-4449-aaf6-4857232d403b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:41.897778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd5ef9-4be4-4449-aaf6-4857232d403b admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:41.898064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd5ef9-4be4-4449-aaf6-4857232d403b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:41.898360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd5ef9-4be4-4449-aaf6-4857232d403b admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:41.899239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd5ef9-4be4-4449-aaf6-4857232d403b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:41.901745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd5ef9-4be4-4449-aaf6-4857232d403b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:41.938012 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1616/3202] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:02:41 2022] POST /identity/v3/users => generated 300 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:02:41.939992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:41.941906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:41.958660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997db33-9690-4bfe-80f4-fba5ff1008d2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:41.965476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf18540-df1b-44a4-bdf1-5719ae5b2877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:41.966422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf18540-df1b-44a4-bdf1-5719ae5b2877 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:41.966699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf18540-df1b-44a4-bdf1-5719ae5b2877 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:41.966946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf18540-df1b-44a4-bdf1-5719ae5b2877 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:41.967592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf18540-df1b-44a4-bdf1-5719ae5b2877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:41.969566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf18540-df1b-44a4-bdf1-5719ae5b2877 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:41.985456 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1587/3203] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:02:41.990077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd5ef9-4be4-4449-aaf6-4857232d403b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:42.008243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf18540-df1b-44a4-bdf1-5719ae5b2877 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:42.009593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd5ef9-4be4-4449-aaf6-4857232d403b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:42.010573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-faf18540-df1b-44a4-bdf1-5719ae5b2877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:42.015759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fbdd0-47ae-4d77-98ef-31eb13afdb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:42.016679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fbdd0-47ae-4d77-98ef-31eb13afdb26 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.017063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fbdd0-47ae-4d77-98ef-31eb13afdb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.017510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fbdd0-47ae-4d77-98ef-31eb13afdb26 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:42.018403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fbdd0-47ae-4d77-98ef-31eb13afdb26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:42.020320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fbdd0-47ae-4d77-98ef-31eb13afdb26 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:42.035810 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1617/3204] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:02:42.041192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf18540-df1b-44a4-bdf1-5719ae5b2877 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:42.043162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fbdd0-47ae-4d77-98ef-31eb13afdb26 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:42.050533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1b77b-10e1-4d8f-a502-230e73dd3825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:42.051735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1b77b-10e1-4d8f-a502-230e73dd3825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.052242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1b77b-10e1-4d8f-a502-230e73dd3825 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.052748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1b77b-10e1-4d8f-a502-230e73dd3825 admin admin] PATH_INFO: `/v3/projects/76984aa1ee96476db1708d912aca50da` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:42.054475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1b77b-10e1-4d8f-a502-230e73dd3825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76984aa1ee96476db1708d912aca50da)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:42.057512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1b77b-10e1-4d8f-a502-230e73dd3825 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:42.084311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fbdd0-47ae-4d77-98ef-31eb13afdb26 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:42.108424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b04240-a7ab-45bd-b80a-abd6bd2e8424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:42.109178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b04240-a7ab-45bd-b80a-abd6bd2e8424 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.109509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b04240-a7ab-45bd-b80a-abd6bd2e8424 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.109890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b04240-a7ab-45bd-b80a-abd6bd2e8424 admin admin] PATH_INFO: `/v3/projects/d41c4b12a9a142c993e5b9bcb0055318/users/5004bd5cf2be403195939791113e31d8/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:42.122207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ccd1b77b-10e1-4d8f-a502-230e73dd3825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76984aa1ee96476db1708d912aca50da', 'request_id': 'req-ccd1b77b-10e1-4d8f-a502-230e73dd3825'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:42.130459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b04240-a7ab-45bd-b80a-abd6bd2e8424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d41c4b12a9a142c993e5b9bcb0055318, user_id=5004bd5cf2be403195939791113e31d8, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:42.130688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ccd1b77b-10e1-4d8f-a502-230e73dd3825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76984aa1ee96476db1708d912aca50da', 'request_id': 'req-ccd1b77b-10e1-4d8f-a502-230e73dd3825'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:42.133580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b04240-a7ab-45bd-b80a-abd6bd2e8424 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:42.145173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ccd1b77b-10e1-4d8f-a502-230e73dd3825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76984aa1ee96476db1708d912aca50da', 'request_id': 'req-ccd1b77b-10e1-4d8f-a502-230e73dd3825'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:42.153878 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1618/3205] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:02:42 2022] PUT /identity/v3/projects/d41c4b12a9a142c993e5b9bcb0055318/users/5004bd5cf2be403195939791113e31d8/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:02:42.159223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b2b7f-884b-4a35-afce-720d9ae6b557 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.159393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b2b7f-884b-4a35-afce-720d9ae6b557 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.159703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b2b7f-884b-4a35-afce-720d9ae6b557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:42.165550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ccd1b77b-10e1-4d8f-a502-230e73dd3825 admin admin] The token cache is being invalidate because project 76984aa1ee96476db1708d912aca50da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:02:42.166056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ccd1b77b-10e1-4d8f-a502-230e73dd3825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:42.166974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ccd1b77b-10e1-4d8f-a502-230e73dd3825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:02:42.170056 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1588/3206] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:02:42 2022] DELETE /identity/v3/projects/76984aa1ee96476db1708d912aca50da => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:02:42.176676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b6c3f-bb17-48fa-a18f-ca3dd83b77e9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.178010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b6c3f-bb17-48fa-a18f-ca3dd83b77e9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.178505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b6c3f-bb17-48fa-a18f-ca3dd83b77e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:42.193698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-363b2b7f-884b-4a35-afce-720d9ae6b557 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:42.195417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-363b2b7f-884b-4a35-afce-720d9ae6b557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:42.204728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-790b6c3f-bb17-48fa-a18f-ca3dd83b77e9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:42.206384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-790b6c3f-bb17-48fa-a18f-ca3dd83b77e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:42.240912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=3,keystone:DELETE=20,keystone:SELECT=77 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:42.328565 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1619/3207] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:42.340432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b2b7f-884b-4a35-afce-720d9ae6b557 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:42.342747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-363b2b7f-884b-4a35-afce-720d9ae6b557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:42.344760 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1589/3208] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:42.346651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6261e5-6c50-4d4c-9f01-ef9c61f476d7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.346651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6261e5-6c50-4d4c-9f01-ef9c61f476d7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.347309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6261e5-6c50-4d4c-9f01-ef9c61f476d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:42.374240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0f6261e5-6c50-4d4c-9f01-ef9c61f476d7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:42.376838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0f6261e5-6c50-4d4c-9f01-ef9c61f476d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:42.394903 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1590/3209] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:42.400104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6261e5-6c50-4d4c-9f01-ef9c61f476d7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:42.417092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6261e5-6c50-4d4c-9f01-ef9c61f476d7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:42.428083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d19079-8df7-4c3c-9856-dffc8ac680c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:42.429130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d19079-8df7-4c3c-9856-dffc8ac680c8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.429461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d19079-8df7-4c3c-9856-dffc8ac680c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.429728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d19079-8df7-4c3c-9856-dffc8ac680c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:42.430447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d19079-8df7-4c3c-9856-dffc8ac680c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:02:42.430856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d19079-8df7-4c3c-9856-dffc8ac680c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:42.432563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d19079-8df7-4c3c-9856-dffc8ac680c8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:42.435246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b2b7f-884b-4a35-afce-720d9ae6b557 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:42.443127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f2389-e3be-41ad-88e2-c036551931c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:42.444397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f2389-e3be-41ad-88e2-c036551931c1 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.444999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f2389-e3be-41ad-88e2-c036551931c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.445689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f2389-e3be-41ad-88e2-c036551931c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:42.450461 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1591/3210] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:02:42.454653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d19079-8df7-4c3c-9856-dffc8ac680c8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:42.460423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:42.469334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f2389-e3be-41ad-88e2-c036551931c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:42.471243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f2389-e3be-41ad-88e2-c036551931c1 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:42.476221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d19079-8df7-4c3c-9856-dffc8ac680c8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:42.483110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7e857-3868-440e-9cdc-f846932f3607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:42.483371 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1620/3211] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:02:42.484258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7e857-3868-440e-9cdc-f846932f3607 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.484510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7e857-3868-440e-9cdc-f846932f3607 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.485788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7e857-3868-440e-9cdc-f846932f3607 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:42.486371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7e857-3868-440e-9cdc-f846932f3607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:42.488617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7e857-3868-440e-9cdc-f846932f3607 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:42.560499 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1592/3212] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:02:42 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:02:42.565418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f2389-e3be-41ad-88e2-c036551931c1 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:42.577611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f2389-e3be-41ad-88e2-c036551931c1 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:42.687263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151ff7d-1cb7-423a-86a2-21e84ae2ba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:42.688542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151ff7d-1cb7-423a-86a2-21e84ae2ba7a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.688942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151ff7d-1cb7-423a-86a2-21e84ae2ba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.689110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151ff7d-1cb7-423a-86a2-21e84ae2ba7a admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:42.689751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151ff7d-1cb7-423a-86a2-21e84ae2ba7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:42.691859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151ff7d-1cb7-423a-86a2-21e84ae2ba7a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:42.731938 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1621/3213] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:02:42 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:02:42.736740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7e857-3868-440e-9cdc-f846932f3607 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:42.754606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7e857-3868-440e-9cdc-f846932f3607 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:42.764893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cf523-07f0-4353-82a0-f4a1147f7d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:42.765740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cf523-07f0-4353-82a0-f4a1147f7d29 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.766330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cf523-07f0-4353-82a0-f4a1147f7d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.766552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cf523-07f0-4353-82a0-f4a1147f7d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:42.767365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073cf523-07f0-4353-82a0-f4a1147f7d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:42.769803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073cf523-07f0-4353-82a0-f4a1147f7d29 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:42.784825 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1593/3214] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:02:42.790074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151ff7d-1cb7-423a-86a2-21e84ae2ba7a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:42.806159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151ff7d-1cb7-423a-86a2-21e84ae2ba7a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:42.813339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c713a-2366-4faf-8204-085b8d8a56bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:42.814229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c713a-2366-4faf-8204-085b8d8a56bb admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.814609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c713a-2366-4faf-8204-085b8d8a56bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.814976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c713a-2366-4faf-8204-085b8d8a56bb admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:42.816084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c713a-2366-4faf-8204-085b8d8a56bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:42.818698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c713a-2366-4faf-8204-085b8d8a56bb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:42.840635 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1622/3215] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:02:42.843099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cf523-07f0-4353-82a0-f4a1147f7d29 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:42.859108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cf523-07f0-4353-82a0-f4a1147f7d29 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:42.866113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b28c5-03c9-4dde-8205-6c3c6ae94060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:42.867186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b28c5-03c9-4dde-8205-6c3c6ae94060 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.867944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b28c5-03c9-4dde-8205-6c3c6ae94060 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.868215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b28c5-03c9-4dde-8205-6c3c6ae94060 admin admin] PATH_INFO: `/v3/projects/93eac1ec0c394b66ac622cb9669e92fe/users/896d2aaaf6cc42a69affa6342f65973a/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:42.919569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b28c5-03c9-4dde-8205-6c3c6ae94060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93eac1ec0c394b66ac622cb9669e92fe, user_id=896d2aaaf6cc42a69affa6342f65973a, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:42.929090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b28c5-03c9-4dde-8205-6c3c6ae94060 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:42.952452 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1594/3216] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:02:42 2022] PUT /identity/v3/projects/93eac1ec0c394b66ac622cb9669e92fe/users/896d2aaaf6cc42a69affa6342f65973a/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:02:42.963573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaada182-697a-4198-a65e-6535737f0824 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:42.965389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaada182-697a-4198-a65e-6535737f0824 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:42.967321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaada182-697a-4198-a65e-6535737f0824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:43.020245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-aaada182-697a-4198-a65e-6535737f0824 None None] MFA Rules not processed for user `896d2aaaf6cc42a69affa6342f65973a`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:43.022384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aaada182-697a-4198-a65e-6535737f0824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:43.164609 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1623/3217] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:43.168080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c60cf8-f507-408b-b5e7-dc4d3071231c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:43.169638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c60cf8-f507-408b-b5e7-dc4d3071231c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:43.170530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c60cf8-f507-408b-b5e7-dc4d3071231c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:43.222248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-58c60cf8-f507-408b-b5e7-dc4d3071231c None None] MFA Rules not processed for user `896d2aaaf6cc42a69affa6342f65973a`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:43.225862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-58c60cf8-f507-408b-b5e7-dc4d3071231c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:43.248424 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1595/3218] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:43.263560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaada182-697a-4198-a65e-6535737f0824 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:43.265380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aaada182-697a-4198-a65e-6535737f0824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:43.425081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:43.452243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaada182-697a-4198-a65e-6535737f0824 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:43.469263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4ee2d-3052-422b-a2f3-4894768b3506 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:43.470029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4ee2d-3052-422b-a2f3-4894768b3506 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:43.470222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4ee2d-3052-422b-a2f3-4894768b3506 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:43.472637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4ee2d-3052-422b-a2f3-4894768b3506 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:43.488926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4ee2d-3052-422b-a2f3-4894768b3506 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:43.491805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4ee2d-3052-422b-a2f3-4894768b3506 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:43.505183 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1624/3219] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:02:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:02:43.968204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a76861-c596-4e78-bc3e-a677e4f403d7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:43.969077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a76861-c596-4e78-bc3e-a677e4f403d7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:43.969803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a76861-c596-4e78-bc3e-a677e4f403d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:44.004080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d9a76861-c596-4e78-bc3e-a677e4f403d7 None None] MFA Rules not processed for user `896d2aaaf6cc42a69affa6342f65973a`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:44.005726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d9a76861-c596-4e78-bc3e-a677e4f403d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:44.021253 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1596/3220] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:51.173896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4ee2d-3052-422b-a2f3-4894768b3506 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:51.175364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-88c4ee2d-3052-422b-a2f3-4894768b3506 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:51.387847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4ee2d-3052-422b-a2f3-4894768b3506 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:51.393721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11671bd5-d362-4031-b7d3-5963a22bcaf9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:51.394511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11671bd5-d362-4031-b7d3-5963a22bcaf9 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:51.394892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11671bd5-d362-4031-b7d3-5963a22bcaf9 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:51.395550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11671bd5-d362-4031-b7d3-5963a22bcaf9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:51.397997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-11671bd5-d362-4031-b7d3-5963a22bcaf9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:51.412779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11671bd5-d362-4031-b7d3-5963a22bcaf9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:51.421561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11671bd5-d362-4031-b7d3-5963a22bcaf9 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:51.455079 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1625/3221] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:02:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:02:53.994024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=57,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:02:54.007845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4adfbc-7f29-4ffa-98b5-651c5dbbe917 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:54.008763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4adfbc-7f29-4ffa-98b5-651c5dbbe917 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:54.010014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4adfbc-7f29-4ffa-98b5-651c5dbbe917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:54.041824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-aa4adfbc-7f29-4ffa-98b5-651c5dbbe917 None None] MFA Rules not processed for user `5004bd5cf2be403195939791113e31d8`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:54.043173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-aa4adfbc-7f29-4ffa-98b5-651c5dbbe917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:54.124475 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1597/3222] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:54 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:54.127286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad0f36-eab0-482a-b3b1-8ca7868384ef None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:54.127782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad0f36-eab0-482a-b3b1-8ca7868384ef None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:54.127782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad0f36-eab0-482a-b3b1-8ca7868384ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:54.147161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-01ad0f36-eab0-482a-b3b1-8ca7868384ef None None] MFA Rules not processed for user `5004bd5cf2be403195939791113e31d8`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:02:54.148395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-01ad0f36-eab0-482a-b3b1-8ca7868384ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:54.155574 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1626/3223] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:02:54 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:02:54.169923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4adfbc-7f29-4ffa-98b5-651c5dbbe917 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:54.171610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-aa4adfbc-7f29-4ffa-98b5-651c5dbbe917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:54.205288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4adfbc-7f29-4ffa-98b5-651c5dbbe917 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:54.211527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d1607-d2fe-452e-a75c-76432421c10e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:54.212543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d1607-d2fe-452e-a75c-76432421c10e service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:54.212859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d1607-d2fe-452e-a75c-76432421c10e service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:54.213695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d1607-d2fe-452e-a75c-76432421c10e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:54.228682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d1607-d2fe-452e-a75c-76432421c10e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:54.230748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d1607-d2fe-452e-a75c-76432421c10e service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:54.246048 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1598/3224] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:02:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:02:56.426465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad0f36-eab0-482a-b3b1-8ca7868384ef None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:02:56.428703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-01ad0f36-eab0-482a-b3b1-8ca7868384ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:56.458311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad0f36-eab0-482a-b3b1-8ca7868384ef None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:02:56.471714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0f9cf-7ff3-4a4b-8619-4cbc16113fae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:02:56.474270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0f9cf-7ff3-4a4b-8619-4cbc16113fae service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:02:56.474985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0f9cf-7ff3-4a4b-8619-4cbc16113fae service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:02:56.476653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0f9cf-7ff3-4a4b-8619-4cbc16113fae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:02:56.480913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4eb0f9cf-7ff3-4a4b-8619-4cbc16113fae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:02:56.505048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0f9cf-7ff3-4a4b-8619-4cbc16113fae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:02:56.509883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0f9cf-7ff3-4a4b-8619-4cbc16113fae service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:02:56.528951 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1627/3225] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:02:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:03:04.236797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=38 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:06.524110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=56 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:19.423232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d1607-d2fe-452e-a75c-76432421c10e service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:19.424815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-092d1607-d2fe-452e-a75c-76432421c10e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:19.498265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d1607-d2fe-452e-a75c-76432421c10e service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:19.503486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:19.504280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:19.504606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:19.505046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] PATH_INFO: `/v3/users/334fcd5a53b04e21b107fafebcfa98cf` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:19.506238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] RBAC: Authorizing `identity:delete_user(user_id=334fcd5a53b04e21b107fafebcfa98cf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:19.508206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:19.561245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '334fcd5a53b04e21b107fafebcfa98cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:19.569999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '334fcd5a53b04e21b107fafebcfa98cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:19.585493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '334fcd5a53b04e21b107fafebcfa98cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:19.586471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '334fcd5a53b04e21b107fafebcfa98cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:19.587361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '334fcd5a53b04e21b107fafebcfa98cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:19.601359 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1599/3226] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:03:19 2022] DELETE /identity/v3/users/334fcd5a53b04e21b107fafebcfa98cf => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:19.607512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6975d-744c-4a16-9ed1-df21a0f40308 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:19.608415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6975d-744c-4a16-9ed1-df21a0f40308 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:19.608662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6975d-744c-4a16-9ed1-df21a0f40308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:19.630718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-35f6975d-744c-4a16-9ed1-df21a0f40308 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:19.631950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-35f6975d-744c-4a16-9ed1-df21a0f40308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:19.701132 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1628/3227] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:19.711305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:19.713621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:19.778877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a121ab5-6ea1-4d39-a32b-ca117ae60640 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:19.782814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ab2d3-26fe-4b8a-8e66-5520a02cc95a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:19.783431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ab2d3-26fe-4b8a-8e66-5520a02cc95a service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:19.783722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ab2d3-26fe-4b8a-8e66-5520a02cc95a service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:19.783985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ab2d3-26fe-4b8a-8e66-5520a02cc95a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:19.794465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ab2d3-26fe-4b8a-8e66-5520a02cc95a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:19.795935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ab2d3-26fe-4b8a-8e66-5520a02cc95a service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:19.811617 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1600/3228] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:03:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:03:20.009810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6975d-744c-4a16-9ed1-df21a0f40308 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:20.011535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-35f6975d-744c-4a16-9ed1-df21a0f40308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:20.027442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6975d-744c-4a16-9ed1-df21a0f40308 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:20.032755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:20.033574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:20.033985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:20.034542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] PATH_INFO: `/v3/projects/56abed6a790140b08939484f98f2db8a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:20.035836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=56abed6a790140b08939484f98f2db8a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:20.038334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:20.093389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56abed6a790140b08939484f98f2db8a', 'request_id': 'req-5315eee8-2e9e-4e95-8af2-f977baaebc7a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.102828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56abed6a790140b08939484f98f2db8a', 'request_id': 'req-5315eee8-2e9e-4e95-8af2-f977baaebc7a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.109534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56abed6a790140b08939484f98f2db8a', 'request_id': 'req-5315eee8-2e9e-4e95-8af2-f977baaebc7a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.126551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] The token cache is being invalidate because project 56abed6a790140b08939484f98f2db8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:03:20.126843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.127492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.132652 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1629/3229] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:03:20 2022] DELETE /identity/v3/projects/56abed6a790140b08939484f98f2db8a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:20.133647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ab2d3-26fe-4b8a-8e66-5520a02cc95a service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:20.144362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ab2d3-26fe-4b8a-8e66-5520a02cc95a service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:20.167792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:20.220146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d7331-5c16-4048-9c11-92dd23b2f2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:20.221139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d7331-5c16-4048-9c11-92dd23b2f2e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:20.221599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d7331-5c16-4048-9c11-92dd23b2f2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:20.221874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d7331-5c16-4048-9c11-92dd23b2f2e5 admin admin] PATH_INFO: `/v3/users/cfbee39bc79b4e82add22264be359ecd` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:20.223053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d7331-5c16-4048-9c11-92dd23b2f2e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfbee39bc79b4e82add22264be359ecd)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:20.225410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d7331-5c16-4048-9c11-92dd23b2f2e5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:20.319281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-230d7331-5c16-4048-9c11-92dd23b2f2e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfbee39bc79b4e82add22264be359ecd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.334329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-230d7331-5c16-4048-9c11-92dd23b2f2e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfbee39bc79b4e82add22264be359ecd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.361255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-230d7331-5c16-4048-9c11-92dd23b2f2e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfbee39bc79b4e82add22264be359ecd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.362725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-230d7331-5c16-4048-9c11-92dd23b2f2e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfbee39bc79b4e82add22264be359ecd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.364629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-230d7331-5c16-4048-9c11-92dd23b2f2e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfbee39bc79b4e82add22264be359ecd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.381712 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1601/3230] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:03:20 2022] DELETE /identity/v3/users/cfbee39bc79b4e82add22264be359ecd => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:20.689505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:20.691061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:20.705996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315eee8-2e9e-4e95-8af2-f977baaebc7a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:20.765153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3163ee0-9778-4b98-aa07-476b3cb71070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:20.765799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3163ee0-9778-4b98-aa07-476b3cb71070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:20.766075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3163ee0-9778-4b98-aa07-476b3cb71070 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:20.766514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3163ee0-9778-4b98-aa07-476b3cb71070 admin admin] PATH_INFO: `/v3/projects/6e1c22c3b00f4178a869cd2664770221` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:20.767895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3163ee0-9778-4b98-aa07-476b3cb71070 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e1c22c3b00f4178a869cd2664770221)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:20.769925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3163ee0-9778-4b98-aa07-476b3cb71070 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:20.806711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:20.825318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d3163ee0-9778-4b98-aa07-476b3cb71070 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e1c22c3b00f4178a869cd2664770221', 'request_id': 'req-d3163ee0-9778-4b98-aa07-476b3cb71070'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.834229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d3163ee0-9778-4b98-aa07-476b3cb71070 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e1c22c3b00f4178a869cd2664770221', 'request_id': 'req-d3163ee0-9778-4b98-aa07-476b3cb71070'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.840217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d3163ee0-9778-4b98-aa07-476b3cb71070 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e1c22c3b00f4178a869cd2664770221', 'request_id': 'req-d3163ee0-9778-4b98-aa07-476b3cb71070'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.851516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d3163ee0-9778-4b98-aa07-476b3cb71070 admin admin] The token cache is being invalidate because project 6e1c22c3b00f4178a869cd2664770221 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:03:20.851799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d3163ee0-9778-4b98-aa07-476b3cb71070 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.852629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d3163ee0-9778-4b98-aa07-476b3cb71070 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:20.854246 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1630/3231] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:03:20 2022] DELETE /identity/v3/projects/6e1c22c3b00f4178a869cd2664770221 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:20.859428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d77208-3286-4772-b979-fefadabaa4b5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:20.859899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d77208-3286-4772-b979-fefadabaa4b5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:20.859970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d77208-3286-4772-b979-fefadabaa4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:20.918637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-72d77208-3286-4772-b979-fefadabaa4b5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:20.919944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-72d77208-3286-4772-b979-fefadabaa4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:20.995229 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1602/3232] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:21.003346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7f252-4af5-489b-8eac-58e309c0c412 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.003791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7f252-4af5-489b-8eac-58e309c0c412 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.004070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7f252-4af5-489b-8eac-58e309c0c412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.024928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-81c7f252-4af5-489b-8eac-58e309c0c412 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:21.026328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-81c7f252-4af5-489b-8eac-58e309c0c412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:21.034212 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1631/3233] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:21.038616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d77208-3286-4772-b979-fefadabaa4b5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:21.048514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d77208-3286-4772-b979-fefadabaa4b5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:21.053326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec805b1-57fd-4e87-9a12-8426e9cfba17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:21.053919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec805b1-57fd-4e87-9a12-8426e9cfba17 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.054186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec805b1-57fd-4e87-9a12-8426e9cfba17 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.054444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec805b1-57fd-4e87-9a12-8426e9cfba17 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.055074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec805b1-57fd-4e87-9a12-8426e9cfba17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:03:21.055367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec805b1-57fd-4e87-9a12-8426e9cfba17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:21.057029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec805b1-57fd-4e87-9a12-8426e9cfba17 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:21.074086 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1603/3234] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:03:21.075258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7f252-4af5-489b-8eac-58e309c0c412 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:21.085117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7f252-4af5-489b-8eac-58e309c0c412 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:21.091139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0cb01-cbb5-4dff-a33f-dd030df9743f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:21.091734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0cb01-cbb5-4dff-a33f-dd030df9743f admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.092000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0cb01-cbb5-4dff-a33f-dd030df9743f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.092256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0cb01-cbb5-4dff-a33f-dd030df9743f admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.092918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0cb01-cbb5-4dff-a33f-dd030df9743f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:21.094816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0cb01-cbb5-4dff-a33f-dd030df9743f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:21.124550 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1632/3235] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:03:21 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:03:21.128727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec805b1-57fd-4e87-9a12-8426e9cfba17 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:21.139489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec805b1-57fd-4e87-9a12-8426e9cfba17 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:21.149106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:21.205164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0a451-492c-408f-b3e8-a32310b820ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:21.205784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a451-492c-408f-b3e8-a32310b820ba admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.206071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a451-492c-408f-b3e8-a32310b820ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.206335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a451-492c-408f-b3e8-a32310b820ba admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.207742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0a451-492c-408f-b3e8-a32310b820ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:21.208965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0a451-492c-408f-b3e8-a32310b820ba admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:21.245871 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1604/3236] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:03:21 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:03:21.252246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0cb01-cbb5-4dff-a33f-dd030df9743f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:21.270601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0cb01-cbb5-4dff-a33f-dd030df9743f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:21.276917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31783d3d-40c1-4820-8a57-ade486e22a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:21.277555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31783d3d-40c1-4820-8a57-ade486e22a64 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.277861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31783d3d-40c1-4820-8a57-ade486e22a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.278085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31783d3d-40c1-4820-8a57-ade486e22a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.278958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31783d3d-40c1-4820-8a57-ade486e22a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:21.280593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31783d3d-40c1-4820-8a57-ade486e22a64 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:21.301818 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1633/3237] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:03:21.307112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0a451-492c-408f-b3e8-a32310b820ba admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:21.318324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0a451-492c-408f-b3e8-a32310b820ba admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:21.325169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9dcfd-56d1-4a7b-af73-6d012f1c79ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:21.325855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9dcfd-56d1-4a7b-af73-6d012f1c79ac admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.326347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9dcfd-56d1-4a7b-af73-6d012f1c79ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.326347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9dcfd-56d1-4a7b-af73-6d012f1c79ac admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.327197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9dcfd-56d1-4a7b-af73-6d012f1c79ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:21.330108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9dcfd-56d1-4a7b-af73-6d012f1c79ac admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:21.342482 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1605/3238] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:03:21.346953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31783d3d-40c1-4820-8a57-ade486e22a64 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:21.358866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31783d3d-40c1-4820-8a57-ade486e22a64 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:21.363519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081f3e7-c18f-4394-b492-0aea0b2cd4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:21.364202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081f3e7-c18f-4394-b492-0aea0b2cd4db admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.364675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081f3e7-c18f-4394-b492-0aea0b2cd4db admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.365207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081f3e7-c18f-4394-b492-0aea0b2cd4db admin admin] PATH_INFO: `/v3/projects/a857554be43f4954af10882c91310b5c/users/d3228428aab04272b20887439f2f133c/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.382335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081f3e7-c18f-4394-b492-0aea0b2cd4db admin admin] RBAC: Authorizing `identity:create_grant(project_id=a857554be43f4954af10882c91310b5c, user_id=d3228428aab04272b20887439f2f133c, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:21.391785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081f3e7-c18f-4394-b492-0aea0b2cd4db admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:21.407665 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1634/3239] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:03:21 2022] PUT /identity/v3/projects/a857554be43f4954af10882c91310b5c/users/d3228428aab04272b20887439f2f133c/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:21.419507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b40ce-f48e-4cf6-a37f-53ce72c84c6c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.419779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b40ce-f48e-4cf6-a37f-53ce72c84c6c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.420009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b40ce-f48e-4cf6-a37f-53ce72c84c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.449147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-613b40ce-f48e-4cf6-a37f-53ce72c84c6c None None] MFA Rules not processed for user `d3228428aab04272b20887439f2f133c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:21.450338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-613b40ce-f48e-4cf6-a37f-53ce72c84c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:21.521957 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1606/3240] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:21.526797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081f3e7-c18f-4394-b492-0aea0b2cd4db admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:21.593750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081f3e7-c18f-4394-b492-0aea0b2cd4db admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:21.598844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6caff0-6636-4986-a41b-32a9343c41cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:21.599456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6caff0-6636-4986-a41b-32a9343c41cd admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.599725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6caff0-6636-4986-a41b-32a9343c41cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.600035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6caff0-6636-4986-a41b-32a9343c41cd admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.600690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6caff0-6636-4986-a41b-32a9343c41cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:21.602591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6caff0-6636-4986-a41b-32a9343c41cd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:21.632535 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1635/3241] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:03:21 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:03:21.637039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b40ce-f48e-4cf6-a37f-53ce72c84c6c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:21.648108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b40ce-f48e-4cf6-a37f-53ce72c84c6c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:21.710691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a646cec-46c1-420d-9d9f-97f04039298d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:21.711227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a646cec-46c1-420d-9d9f-97f04039298d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.711503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a646cec-46c1-420d-9d9f-97f04039298d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.711773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a646cec-46c1-420d-9d9f-97f04039298d admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.712503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a646cec-46c1-420d-9d9f-97f04039298d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:21.714630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a646cec-46c1-420d-9d9f-97f04039298d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:21.745998 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1607/3242] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:03:21 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:03:21.747227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6caff0-6636-4986-a41b-32a9343c41cd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:21.758820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6caff0-6636-4986-a41b-32a9343c41cd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:21.764495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bea1e8-9412-4717-842a-91a2c7f78f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:21.765102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bea1e8-9412-4717-842a-91a2c7f78f26 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.765374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bea1e8-9412-4717-842a-91a2c7f78f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.765711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bea1e8-9412-4717-842a-91a2c7f78f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.766743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bea1e8-9412-4717-842a-91a2c7f78f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:21.769719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bea1e8-9412-4717-842a-91a2c7f78f26 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:21.784590 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1636/3243] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:03:21.788991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a646cec-46c1-420d-9d9f-97f04039298d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:21.799949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a646cec-46c1-420d-9d9f-97f04039298d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:21.805139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719fb7a-0598-4d04-90ea-4ebfedc86192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:21.805823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719fb7a-0598-4d04-90ea-4ebfedc86192 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.806126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719fb7a-0598-4d04-90ea-4ebfedc86192 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.806791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719fb7a-0598-4d04-90ea-4ebfedc86192 admin admin] PATH_INFO: `/v3/projects/2f1b22e0473e42a6af85174b5b2804d8/users/7d830b299ee0421f9dde99fca4efb7db/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.816527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:21.825260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719fb7a-0598-4d04-90ea-4ebfedc86192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1b22e0473e42a6af85174b5b2804d8, user_id=7d830b299ee0421f9dde99fca4efb7db, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:21.828202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719fb7a-0598-4d04-90ea-4ebfedc86192 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:21.842149 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1608/3244] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:03:21 2022] PUT /identity/v3/projects/2f1b22e0473e42a6af85174b5b2804d8/users/7d830b299ee0421f9dde99fca4efb7db/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:21.846264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bea1e8-9412-4717-842a-91a2c7f78f26 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:21.857027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bea1e8-9412-4717-842a-91a2c7f78f26 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:21.877788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:21.918538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ccdd6-6b80-415b-9f34-88aa22ecc913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:21.919151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ccdd6-6b80-415b-9f34-88aa22ecc913 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.919423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ccdd6-6b80-415b-9f34-88aa22ecc913 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.919707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ccdd6-6b80-415b-9f34-88aa22ecc913 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.920345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ccdd6-6b80-415b-9f34-88aa22ecc913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:21.921983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ccdd6-6b80-415b-9f34-88aa22ecc913 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:21.935364 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1637/3245] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:03:21.936537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719fb7a-0598-4d04-90ea-4ebfedc86192 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:21.947452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719fb7a-0598-4d04-90ea-4ebfedc86192 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:21.952766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1d539-0ee8-48a9-9667-5a1acf4b7840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:21.953562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1d539-0ee8-48a9-9667-5a1acf4b7840 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.953835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1d539-0ee8-48a9-9667-5a1acf4b7840 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.954099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1d539-0ee8-48a9-9667-5a1acf4b7840 admin admin] PATH_INFO: `/v3/projects/2f1b22e0473e42a6af85174b5b2804d8/users/7d830b299ee0421f9dde99fca4efb7db/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:21.956255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1d539-0ee8-48a9-9667-5a1acf4b7840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1b22e0473e42a6af85174b5b2804d8, user_id=7d830b299ee0421f9dde99fca4efb7db, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:21.958915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1d539-0ee8-48a9-9667-5a1acf4b7840 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:21.973323 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1609/3246] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:03:21 2022] PUT /identity/v3/projects/2f1b22e0473e42a6af85174b5b2804d8/users/7d830b299ee0421f9dde99fca4efb7db/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:21.983924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c1498-b74d-4498-994c-9e7c7bb0db3a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:21.984198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c1498-b74d-4498-994c-9e7c7bb0db3a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:21.984495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c1498-b74d-4498-994c-9e7c7bb0db3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:22.013209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d23c1498-b74d-4498-994c-9e7c7bb0db3a None None] MFA Rules not processed for user `7d830b299ee0421f9dde99fca4efb7db`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:22.014431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d23c1498-b74d-4498-994c-9e7c7bb0db3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:22.090065 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1638/3247] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:21 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:22.101287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1d539-0ee8-48a9-9667-5a1acf4b7840 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:22.103070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9ef1d539-0ee8-48a9-9667-5a1acf4b7840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:22.208199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1d539-0ee8-48a9-9667-5a1acf4b7840 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:22.215237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b31b34-6a2d-4e97-b974-fcd941b075f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:22.215820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b31b34-6a2d-4e97-b974-fcd941b075f8 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:22.216089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b31b34-6a2d-4e97-b974-fcd941b075f8 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:22.216423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b31b34-6a2d-4e97-b974-fcd941b075f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:22.226104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b31b34-6a2d-4e97-b974-fcd941b075f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:22.227719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b31b34-6a2d-4e97-b974-fcd941b075f8 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:22.241409 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1610/3248] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:03:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:03:22.428588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c1498-b74d-4498-994c-9e7c7bb0db3a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:22.520440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c1498-b74d-4498-994c-9e7c7bb0db3a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:22.526141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:22.526868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:22.527341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:22.527586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] PATH_INFO: `/v3/users/d3228428aab04272b20887439f2f133c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:22.528838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3228428aab04272b20887439f2f133c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:22.530775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:22.609893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3228428aab04272b20887439f2f133c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:22.616940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:22.642987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3228428aab04272b20887439f2f133c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:22.664066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3228428aab04272b20887439f2f133c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:22.694915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3228428aab04272b20887439f2f133c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:22.696884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3228428aab04272b20887439f2f133c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:22.707442 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1639/3249] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:03:22 2022] DELETE /identity/v3/users/d3228428aab04272b20887439f2f133c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:22.712562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d8cbc-857a-4629-9f76-266e9b994fe5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:22.713044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d8cbc-857a-4629-9f76-266e9b994fe5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:22.713531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d8cbc-857a-4629-9f76-266e9b994fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:22.743950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-018d8cbc-857a-4629-9f76-266e9b994fe5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:22.745561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-018d8cbc-857a-4629-9f76-266e9b994fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:22.828354 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1611/3250] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:22.836996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:22.838674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:22.909018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57a39e-f244-4051-a2d9-3d40be21f024 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:22.912920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505bd90-f346-4bf3-bfdd-0fe8059f4ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:22.913533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505bd90-f346-4bf3-bfdd-0fe8059f4ceb service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:22.913801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505bd90-f346-4bf3-bfdd-0fe8059f4ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:22.914061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505bd90-f346-4bf3-bfdd-0fe8059f4ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:22.936749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505bd90-f346-4bf3-bfdd-0fe8059f4ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:22.938494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505bd90-f346-4bf3-bfdd-0fe8059f4ceb service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:22.953677 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1640/3251] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:03:23.629522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d8cbc-857a-4629-9f76-266e9b994fe5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:23.631839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-018d8cbc-857a-4629-9f76-266e9b994fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:23.648573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d8cbc-857a-4629-9f76-266e9b994fe5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:23.654579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:23.655363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:23.655640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:23.655908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] PATH_INFO: `/v3/projects/a857554be43f4954af10882c91310b5c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:23.657113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a857554be43f4954af10882c91310b5c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:23.659307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:23.712646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:23.735726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a857554be43f4954af10882c91310b5c', 'request_id': 'req-8b7dc1fc-d331-43c0-8852-150848cec8b3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:23.772661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a857554be43f4954af10882c91310b5c', 'request_id': 'req-8b7dc1fc-d331-43c0-8852-150848cec8b3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:23.785160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a857554be43f4954af10882c91310b5c', 'request_id': 'req-8b7dc1fc-d331-43c0-8852-150848cec8b3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:23.804949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] The token cache is being invalidate because project a857554be43f4954af10882c91310b5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:03:23.805335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:23.806135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:23.809294 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1612/3252] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:03:23 2022] DELETE /identity/v3/projects/a857554be43f4954af10882c91310b5c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:23.814095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505bd90-f346-4bf3-bfdd-0fe8059f4ceb service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:23.829621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505bd90-f346-4bf3-bfdd-0fe8059f4ceb service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:23.939251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f547542-6efd-496c-8b80-22abd501f779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:23.939966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f547542-6efd-496c-8b80-22abd501f779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:23.940187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f547542-6efd-496c-8b80-22abd501f779 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:23.940478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f547542-6efd-496c-8b80-22abd501f779 admin admin] PATH_INFO: `/v3/users/7d830b299ee0421f9dde99fca4efb7db` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:23.941705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f547542-6efd-496c-8b80-22abd501f779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d830b299ee0421f9dde99fca4efb7db)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:23.944240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f547542-6efd-496c-8b80-22abd501f779 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:24.022285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2f547542-6efd-496c-8b80-22abd501f779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d830b299ee0421f9dde99fca4efb7db'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:24.035422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2f547542-6efd-496c-8b80-22abd501f779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d830b299ee0421f9dde99fca4efb7db'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:24.047712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2f547542-6efd-496c-8b80-22abd501f779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d830b299ee0421f9dde99fca4efb7db'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:24.054350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:24.072736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2f547542-6efd-496c-8b80-22abd501f779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d830b299ee0421f9dde99fca4efb7db'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:24.074377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2f547542-6efd-496c-8b80-22abd501f779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d830b299ee0421f9dde99fca4efb7db'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:24.083684 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1641/3253] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:03:23 2022] DELETE /identity/v3/users/7d830b299ee0421f9dde99fca4efb7db => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:03:24.808879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:24.810938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:24.836216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7dc1fc-d331-43c0-8852-150848cec8b3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:24.935917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:24.936613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:24.936883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:24.937142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef admin admin] PATH_INFO: `/v3/projects/2f1b22e0473e42a6af85174b5b2804d8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:24.938345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f1b22e0473e42a6af85174b5b2804d8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:24.940572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:25.042566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f1b22e0473e42a6af85174b5b2804d8', 'request_id': 'req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:25.051668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f1b22e0473e42a6af85174b5b2804d8', 'request_id': 'req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:25.064395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f1b22e0473e42a6af85174b5b2804d8', 'request_id': 'req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:25.083422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef admin admin] The token cache is being invalidate because project 2f1b22e0473e42a6af85174b5b2804d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:03:25.084581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:25.085331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-957351e6-4bf2-46c6-80c2-c36f9d4c22ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:25.087786 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1613/3254] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:03:24 2022] DELETE /identity/v3/projects/2f1b22e0473e42a6af85174b5b2804d8 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:25.093721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416de050-f38e-4a8d-815d-9a4f9531d133 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:25.094254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416de050-f38e-4a8d-815d-9a4f9531d133 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:25.094254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416de050-f38e-4a8d-815d-9a4f9531d133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:25.134121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-416de050-f38e-4a8d-815d-9a4f9531d133 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:25.136121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-416de050-f38e-4a8d-815d-9a4f9531d133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:25.248050 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1642/3255] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:25.255905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b34811-7bf8-471c-8941-a6ee3b27a5eb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:25.256180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b34811-7bf8-471c-8941-a6ee3b27a5eb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:25.256479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b34811-7bf8-471c-8941-a6ee3b27a5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:25.279010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d1b34811-7bf8-471c-8941-a6ee3b27a5eb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:25.280699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d1b34811-7bf8-471c-8941-a6ee3b27a5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:25.298067 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1614/3256] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:25.314388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416de050-f38e-4a8d-815d-9a4f9531d133 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:25.327215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416de050-f38e-4a8d-815d-9a4f9531d133 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:25.334597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1177b-1c3e-4ae7-86cf-8ee2b59f23d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:25.335222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1177b-1c3e-4ae7-86cf-8ee2b59f23d0 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:25.335489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1177b-1c3e-4ae7-86cf-8ee2b59f23d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:25.335843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1177b-1c3e-4ae7-86cf-8ee2b59f23d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:25.336400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1177b-1c3e-4ae7-86cf-8ee2b59f23d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:03:25.336715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1177b-1c3e-4ae7-86cf-8ee2b59f23d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:25.338648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1177b-1c3e-4ae7-86cf-8ee2b59f23d0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:25.359348 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1643/3257] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:03:25.359711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b34811-7bf8-471c-8941-a6ee3b27a5eb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:25.373034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b34811-7bf8-471c-8941-a6ee3b27a5eb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:25.386525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2c5c1-be82-4642-96ab-b8475e8b540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:25.387209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c5c1-be82-4642-96ab-b8475e8b540d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:25.387613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c5c1-be82-4642-96ab-b8475e8b540d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:25.387896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c5c1-be82-4642-96ab-b8475e8b540d admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:25.388719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2c5c1-be82-4642-96ab-b8475e8b540d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:25.390631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2c5c1-be82-4642-96ab-b8475e8b540d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:25.449211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1177b-1c3e-4ae7-86cf-8ee2b59f23d0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:25.456488 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1615/3258] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:03:25 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:03:25.464218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1177b-1c3e-4ae7-86cf-8ee2b59f23d0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:25.629842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a1094-2ebd-474f-bc92-156b1033dc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:25.630918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a1094-2ebd-474f-bc92-156b1033dc33 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:25.631365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a1094-2ebd-474f-bc92-156b1033dc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:25.631793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a1094-2ebd-474f-bc92-156b1033dc33 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:25.634524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a1094-2ebd-474f-bc92-156b1033dc33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:25.637477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a1094-2ebd-474f-bc92-156b1033dc33 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:25.676628 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1644/3259] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:03:25 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:03:25.677793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2c5c1-be82-4642-96ab-b8475e8b540d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:25.697855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2c5c1-be82-4642-96ab-b8475e8b540d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:25.714780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2cb5a-e06e-4b42-bca0-8f8e4357f98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:25.715780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2cb5a-e06e-4b42-bca0-8f8e4357f98a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:25.716226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2cb5a-e06e-4b42-bca0-8f8e4357f98a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:25.716703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2cb5a-e06e-4b42-bca0-8f8e4357f98a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:25.718926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2cb5a-e06e-4b42-bca0-8f8e4357f98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:25.723287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2cb5a-e06e-4b42-bca0-8f8e4357f98a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:25.755119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a1094-2ebd-474f-bc92-156b1033dc33 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:25.757465 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1616/3260] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:03:25.771037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a1094-2ebd-474f-bc92-156b1033dc33 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:25.781800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efc0a3-bd04-4ad7-b63d-f305ff75aed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:25.782428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efc0a3-bd04-4ad7-b63d-f305ff75aed3 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:25.782692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efc0a3-bd04-4ad7-b63d-f305ff75aed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:25.782955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efc0a3-bd04-4ad7-b63d-f305ff75aed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:25.783696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efc0a3-bd04-4ad7-b63d-f305ff75aed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:25.785332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efc0a3-bd04-4ad7-b63d-f305ff75aed3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:25.808570 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1645/3261] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:03:25.811175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2cb5a-e06e-4b42-bca0-8f8e4357f98a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:25.826236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2cb5a-e06e-4b42-bca0-8f8e4357f98a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:25.835714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddbdeb-1a67-46fb-82bd-803423ca98b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:25.836495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddbdeb-1a67-46fb-82bd-803423ca98b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:25.836769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddbdeb-1a67-46fb-82bd-803423ca98b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:25.837562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddbdeb-1a67-46fb-82bd-803423ca98b9 admin admin] PATH_INFO: `/v3/projects/cfc2d104d5b443b2b975dd7bb314049d/users/ed3aa4ba3b7146b9a286d4093945770e/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:25.841701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:25.862521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddbdeb-1a67-46fb-82bd-803423ca98b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc2d104d5b443b2b975dd7bb314049d, user_id=ed3aa4ba3b7146b9a286d4093945770e, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:25.867287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddbdeb-1a67-46fb-82bd-803423ca98b9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:25.902328 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1617/3262] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:03:25 2022] PUT /identity/v3/projects/cfc2d104d5b443b2b975dd7bb314049d/users/ed3aa4ba3b7146b9a286d4093945770e/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:25.915402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701a0e3-3a27-4c72-87df-543b51f7a254 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:25.915686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701a0e3-3a27-4c72-87df-543b51f7a254 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:25.916331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701a0e3-3a27-4c72-87df-543b51f7a254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:25.974349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e701a0e3-3a27-4c72-87df-543b51f7a254 None None] MFA Rules not processed for user `ed3aa4ba3b7146b9a286d4093945770e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:25.977882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e701a0e3-3a27-4c72-87df-543b51f7a254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:26.028650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:26.063750 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1646/3263] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:26.076839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddbdeb-1a67-46fb-82bd-803423ca98b9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:26.078975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-08ddbdeb-1a67-46fb-82bd-803423ca98b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:26.185692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddbdeb-1a67-46fb-82bd-803423ca98b9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:26.200135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3d0fc-4433-404a-9890-147a4c7f4784 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:26.200933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3d0fc-4433-404a-9890-147a4c7f4784 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:26.201269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3d0fc-4433-404a-9890-147a4c7f4784 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:26.201601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3d0fc-4433-404a-9890-147a4c7f4784 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:26.211268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3d0fc-4433-404a-9890-147a4c7f4784 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:26.215490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3d0fc-4433-404a-9890-147a4c7f4784 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:26.238730 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1618/3264] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:03:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:03:27.077335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b5098-4e7d-49ac-ad49-7e13d5bf61dd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:27.078369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b5098-4e7d-49ac-ad49-7e13d5bf61dd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:27.079136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b5098-4e7d-49ac-ad49-7e13d5bf61dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:27.114252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-694b5098-4e7d-49ac-ad49-7e13d5bf61dd None None] MFA Rules not processed for user `815db4215e8b4b1984e199a40ed107c1`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:27.115574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-694b5098-4e7d-49ac-ad49-7e13d5bf61dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:27.288422 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1647/3265] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:27.386143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3d0fc-4433-404a-9890-147a4c7f4784 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:27.509503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3d0fc-4433-404a-9890-147a4c7f4784 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:27.521515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:27.524697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:27.526624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:27.527155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] PATH_INFO: `/v3/users/ed3aa4ba3b7146b9a286d4093945770e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:27.533785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed3aa4ba3b7146b9a286d4093945770e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:27.536014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:27.603001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed3aa4ba3b7146b9a286d4093945770e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:27.615359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed3aa4ba3b7146b9a286d4093945770e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:27.638954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:27.642984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed3aa4ba3b7146b9a286d4093945770e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:27.644819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed3aa4ba3b7146b9a286d4093945770e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:27.645547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed3aa4ba3b7146b9a286d4093945770e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:27.692779 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1619/3266] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:03:27 2022] DELETE /identity/v3/users/ed3aa4ba3b7146b9a286d4093945770e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:27.699347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fd785-6430-4be9-b306-8b1801f614f5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:27.699347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fd785-6430-4be9-b306-8b1801f614f5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:27.699347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fd785-6430-4be9-b306-8b1801f614f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:27.745087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5e9fd785-6430-4be9-b306-8b1801f614f5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:27.746420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5e9fd785-6430-4be9-b306-8b1801f614f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:27.870932 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1648/3267] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:27.878272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:27.880120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:27.948261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c0eb8-bd4f-4618-bd64-bcaa89591929 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:27.957873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb883b-35b9-471e-8d23-6a887890d7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:27.958641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb883b-35b9-471e-8d23-6a887890d7df service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:27.960204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb883b-35b9-471e-8d23-6a887890d7df service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:27.960820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb883b-35b9-471e-8d23-6a887890d7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:27.973973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb883b-35b9-471e-8d23-6a887890d7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:27.975591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb883b-35b9-471e-8d23-6a887890d7df service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:28.004055 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1620/3268] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:03:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:03:28.240163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fd785-6430-4be9-b306-8b1801f614f5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:28.243781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5e9fd785-6430-4be9-b306-8b1801f614f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:28.265511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fd785-6430-4be9-b306-8b1801f614f5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:28.301846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:28.302504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:28.303178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:28.303688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] PATH_INFO: `/v3/users/5004bd5cf2be403195939791113e31d8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:28.308193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5004bd5cf2be403195939791113e31d8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:28.322905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:28.474055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5004bd5cf2be403195939791113e31d8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:28.494287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5004bd5cf2be403195939791113e31d8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:28.514138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5004bd5cf2be403195939791113e31d8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:28.534616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5004bd5cf2be403195939791113e31d8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:28.535725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5004bd5cf2be403195939791113e31d8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:28.545941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:28.556831 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1649/3269] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:03:28 2022] DELETE /identity/v3/users/5004bd5cf2be403195939791113e31d8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:28.699540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb883b-35b9-471e-8d23-6a887890d7df service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:28.701747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-86bb883b-35b9-471e-8d23-6a887890d7df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:28.756206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb883b-35b9-471e-8d23-6a887890d7df service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:28.943692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20553f65-6c05-457f-b3fe-a41f9153ed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:28.944383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553f65-6c05-457f-b3fe-a41f9153ed33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:28.944810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553f65-6c05-457f-b3fe-a41f9153ed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:28.945076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553f65-6c05-457f-b3fe-a41f9153ed33 admin admin] PATH_INFO: `/v3/projects/cfc2d104d5b443b2b975dd7bb314049d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:28.947472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20553f65-6c05-457f-b3fe-a41f9153ed33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfc2d104d5b443b2b975dd7bb314049d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:28.949443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20553f65-6c05-457f-b3fe-a41f9153ed33 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:29.057867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20553f65-6c05-457f-b3fe-a41f9153ed33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfc2d104d5b443b2b975dd7bb314049d', 'request_id': 'req-20553f65-6c05-457f-b3fe-a41f9153ed33'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:29.089943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20553f65-6c05-457f-b3fe-a41f9153ed33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfc2d104d5b443b2b975dd7bb314049d', 'request_id': 'req-20553f65-6c05-457f-b3fe-a41f9153ed33'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:29.127506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20553f65-6c05-457f-b3fe-a41f9153ed33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfc2d104d5b443b2b975dd7bb314049d', 'request_id': 'req-20553f65-6c05-457f-b3fe-a41f9153ed33'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:29.155518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20553f65-6c05-457f-b3fe-a41f9153ed33 admin admin] The token cache is being invalidate because project cfc2d104d5b443b2b975dd7bb314049d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:03:29.155816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20553f65-6c05-457f-b3fe-a41f9153ed33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:29.156700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20553f65-6c05-457f-b3fe-a41f9153ed33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:29.158721 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1621/3270] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:03:28 2022] DELETE /identity/v3/projects/cfc2d104d5b443b2b975dd7bb314049d => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:03:29.163508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:29.165266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:29.167483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229c3f6-84c9-4ead-8f45-d05aa55d710a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:29.167797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229c3f6-84c9-4ead-8f45-d05aa55d710a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:29.170850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229c3f6-84c9-4ead-8f45-d05aa55d710a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:29.192417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579dd60-10d9-4459-ad4a-7b4d1d76290b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:29.206679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6229c3f6-84c9-4ead-8f45-d05aa55d710a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:29.208413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6229c3f6-84c9-4ead-8f45-d05aa55d710a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:29.244618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:29.318774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13075b-4e8c-4307-b228-a6d4abfa2948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:29.325344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13075b-4e8c-4307-b228-a6d4abfa2948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:29.325834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13075b-4e8c-4307-b228-a6d4abfa2948 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:29.326402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13075b-4e8c-4307-b228-a6d4abfa2948 admin admin] PATH_INFO: `/v3/projects/d41c4b12a9a142c993e5b9bcb0055318` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:29.327861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13075b-4e8c-4307-b228-a6d4abfa2948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d41c4b12a9a142c993e5b9bcb0055318)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:29.334597 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1622/3271] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:29.350175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13075b-4e8c-4307-b228-a6d4abfa2948 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:29.355277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3299edf-48b4-45e7-9f88-d2732166aeb0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:29.355413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3299edf-48b4-45e7-9f88-d2732166aeb0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:29.355695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3299edf-48b4-45e7-9f88-d2732166aeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:29.394862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f3299edf-48b4-45e7-9f88-d2732166aeb0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:29.397070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f3299edf-48b4-45e7-9f88-d2732166aeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:29.408572 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1623/3272] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:29.416201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3299edf-48b4-45e7-9f88-d2732166aeb0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:29.435891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3299edf-48b4-45e7-9f88-d2732166aeb0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:29.444974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aaae58-6b43-4272-b94c-5f71197b7a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:29.447727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aaae58-6b43-4272-b94c-5f71197b7a45 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:29.460187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aaae58-6b43-4272-b94c-5f71197b7a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:29.460637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aaae58-6b43-4272-b94c-5f71197b7a45 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:29.461343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aaae58-6b43-4272-b94c-5f71197b7a45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:03:29.464038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aaae58-6b43-4272-b94c-5f71197b7a45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:29.467450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aaae58-6b43-4272-b94c-5f71197b7a45 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:29.491305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ca13075b-4e8c-4307-b228-a6d4abfa2948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd41c4b12a9a142c993e5b9bcb0055318', 'request_id': 'req-ca13075b-4e8c-4307-b228-a6d4abfa2948'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:29.507696 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1624/3273] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:03:29.508854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aaae58-6b43-4272-b94c-5f71197b7a45 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:29.516883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ca13075b-4e8c-4307-b228-a6d4abfa2948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd41c4b12a9a142c993e5b9bcb0055318', 'request_id': 'req-ca13075b-4e8c-4307-b228-a6d4abfa2948'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:29.526128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ca13075b-4e8c-4307-b228-a6d4abfa2948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd41c4b12a9a142c993e5b9bcb0055318', 'request_id': 'req-ca13075b-4e8c-4307-b228-a6d4abfa2948'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:29.538644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aaae58-6b43-4272-b94c-5f71197b7a45 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:29.551372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ca13075b-4e8c-4307-b228-a6d4abfa2948 admin admin] The token cache is being invalidate because project d41c4b12a9a142c993e5b9bcb0055318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:03:29.551663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ca13075b-4e8c-4307-b228-a6d4abfa2948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:29.552530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ca13075b-4e8c-4307-b228-a6d4abfa2948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:29.564296 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1650/3274] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:03:29 2022] DELETE /identity/v3/projects/d41c4b12a9a142c993e5b9bcb0055318 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:03:29.570144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89d60a-7408-4c6e-907a-e67f19e2b81d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:29.577732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89d60a-7408-4c6e-907a-e67f19e2b81d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:29.578427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89d60a-7408-4c6e-907a-e67f19e2b81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:29.622940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4a89d60a-7408-4c6e-907a-e67f19e2b81d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:29.625158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4a89d60a-7408-4c6e-907a-e67f19e2b81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:29.652154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74df202-1495-4f5e-91d3-d503923a6a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:29.652573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74df202-1495-4f5e-91d3-d503923a6a55 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:29.652896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74df202-1495-4f5e-91d3-d503923a6a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:29.653107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74df202-1495-4f5e-91d3-d503923a6a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:29.653735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74df202-1495-4f5e-91d3-d503923a6a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:29.655780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74df202-1495-4f5e-91d3-d503923a6a55 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:29.692686 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1625/3275] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:03:29 2022] POST /identity/v3/projects => generated 397 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:03:29.703080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74df202-1495-4f5e-91d3-d503923a6a55 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:29.706016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e74df202-1495-4f5e-91d3-d503923a6a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:29.776454 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1651/3276] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:03:29.790044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399e26a-d932-431a-a581-3e57f696c831 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:29.790335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399e26a-d932-431a-a581-3e57f696c831 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:29.790603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399e26a-d932-431a-a581-3e57f696c831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:29.817842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74df202-1495-4f5e-91d3-d503923a6a55 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:29.825181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7399e26a-d932-431a-a581-3e57f696c831 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:29.826847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7399e26a-d932-431a-a581-3e57f696c831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:29.827329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e07a68-ae78-47d5-af33-325513d8d627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:29.828841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e07a68-ae78-47d5-af33-325513d8d627 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:29.829208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e07a68-ae78-47d5-af33-325513d8d627 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:29.829551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e07a68-ae78-47d5-af33-325513d8d627 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:29.830529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e07a68-ae78-47d5-af33-325513d8d627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:29.833520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e07a68-ae78-47d5-af33-325513d8d627 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:29.843577 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1652/3277] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:29.845582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399e26a-d932-431a-a581-3e57f696c831 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:29.860011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399e26a-d932-431a-a581-3e57f696c831 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:29.862221 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1626/3278] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:03:29 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:03:29.867117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e07a68-ae78-47d5-af33-325513d8d627 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:29.868536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d50708-b3e0-47c1-862a-2628c6747678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:29.869189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d50708-b3e0-47c1-862a-2628c6747678 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:29.870017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d50708-b3e0-47c1-862a-2628c6747678 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:29.870320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d50708-b3e0-47c1-862a-2628c6747678 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:29.871087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d50708-b3e0-47c1-862a-2628c6747678 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:03:29.871393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d50708-b3e0-47c1-862a-2628c6747678 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:29.873335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d50708-b3e0-47c1-862a-2628c6747678 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:29.887593 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1653/3279] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:03:29.887716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e07a68-ae78-47d5-af33-325513d8d627 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:29.892263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d50708-b3e0-47c1-862a-2628c6747678 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:29.896338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13064725-256c-42f4-850f-851780ccda87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:29.897064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13064725-256c-42f4-850f-851780ccda87 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:29.897336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13064725-256c-42f4-850f-851780ccda87 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:29.897596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13064725-256c-42f4-850f-851780ccda87 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:29.898330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13064725-256c-42f4-850f-851780ccda87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:29.899903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13064725-256c-42f4-850f-851780ccda87 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:29.905886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d50708-b3e0-47c1-862a-2628c6747678 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:29.911390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79155a8e-5823-4b65-baf2-ec7714395181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:29.912134 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1627/3280] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:03:29.912262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79155a8e-5823-4b65-baf2-ec7714395181 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:29.912494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79155a8e-5823-4b65-baf2-ec7714395181 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:29.912808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79155a8e-5823-4b65-baf2-ec7714395181 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:29.913486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79155a8e-5823-4b65-baf2-ec7714395181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:29.915473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79155a8e-5823-4b65-baf2-ec7714395181 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:29.915992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13064725-256c-42f4-850f-851780ccda87 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:29.925647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:29.928688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:29.933802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13064725-256c-42f4-850f-851780ccda87 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:29.946337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4b58a-a99f-430d-a1de-dbd24fdc997e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:29.950376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4b58a-a99f-430d-a1de-dbd24fdc997e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:29.950569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4b58a-a99f-430d-a1de-dbd24fdc997e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:29.950860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4b58a-a99f-430d-a1de-dbd24fdc997e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:29.951662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4b58a-a99f-430d-a1de-dbd24fdc997e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:29.953711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4b58a-a99f-430d-a1de-dbd24fdc997e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:29.962704 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1654/3281] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:03:29 2022] POST /identity/v3/projects => generated 387 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:03:29.967803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79155a8e-5823-4b65-baf2-ec7714395181 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:29.971696 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1628/3282] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:03:29.972559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4b58a-a99f-430d-a1de-dbd24fdc997e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:29.984926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4b58a-a99f-430d-a1de-dbd24fdc997e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:30.094252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79155a8e-5823-4b65-baf2-ec7714395181 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:30.094965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ffdf8-a399-4a63-a9d7-557c24db877c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:30.095715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ffdf8-a399-4a63-a9d7-557c24db877c admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:30.095988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ffdf8-a399-4a63-a9d7-557c24db877c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:30.096256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ffdf8-a399-4a63-a9d7-557c24db877c admin admin] PATH_INFO: `/v3/projects/3381148916fe4893a3ab0edc998a6847/users/bdb15c8c1f3d4bc38552b33ba1f891ca/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:30.108806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937a703-cefb-4bd2-bb9d-8189f7468944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:30.113268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937a703-cefb-4bd2-bb9d-8189f7468944 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:30.113685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937a703-cefb-4bd2-bb9d-8189f7468944 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:30.114049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937a703-cefb-4bd2-bb9d-8189f7468944 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:30.114939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937a703-cefb-4bd2-bb9d-8189f7468944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:30.121165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937a703-cefb-4bd2-bb9d-8189f7468944 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:30.127907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ffdf8-a399-4a63-a9d7-557c24db877c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3381148916fe4893a3ab0edc998a6847, user_id=bdb15c8c1f3d4bc38552b33ba1f891ca, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:30.137790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ffdf8-a399-4a63-a9d7-557c24db877c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:30.158496 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1655/3283] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:03:29 2022] POST /identity/v3/users => generated 294 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:03:30.166256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937a703-cefb-4bd2-bb9d-8189f7468944 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:30.166516 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1629/3284] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:03:29 2022] PUT /identity/v3/projects/3381148916fe4893a3ab0edc998a6847/users/bdb15c8c1f3d4bc38552b33ba1f891ca/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:30.173611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e1020-23ed-41d0-9119-a091fbc5cf32 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:30.174258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e1020-23ed-41d0-9119-a091fbc5cf32 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:30.174770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e1020-23ed-41d0-9119-a091fbc5cf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:30.178602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937a703-cefb-4bd2-bb9d-8189f7468944 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:30.203492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-207e1020-23ed-41d0-9119-a091fbc5cf32 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:30.204898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-207e1020-23ed-41d0-9119-a091fbc5cf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:30.264352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea54b6-8d13-41f7-a09b-f8f2a3707b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:30.265124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea54b6-8d13-41f7-a09b-f8f2a3707b2a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:30.265394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea54b6-8d13-41f7-a09b-f8f2a3707b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:30.265654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea54b6-8d13-41f7-a09b-f8f2a3707b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:30.266330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea54b6-8d13-41f7-a09b-f8f2a3707b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:30.268491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea54b6-8d13-41f7-a09b-f8f2a3707b2a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:30.292450 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1656/3285] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:03:30.301367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea54b6-8d13-41f7-a09b-f8f2a3707b2a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:30.313225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea54b6-8d13-41f7-a09b-f8f2a3707b2a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:30.317123 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1630/3286] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:30.324219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a222da8-f40d-4e73-97b7-909ef5a70edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:30.325487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a222da8-f40d-4e73-97b7-909ef5a70edd admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:30.326062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a222da8-f40d-4e73-97b7-909ef5a70edd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:30.326539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a222da8-f40d-4e73-97b7-909ef5a70edd admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:30.327483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a222da8-f40d-4e73-97b7-909ef5a70edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:30.327595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e1020-23ed-41d0-9119-a091fbc5cf32 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:30.329776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a222da8-f40d-4e73-97b7-909ef5a70edd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:30.330106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-207e1020-23ed-41d0-9119-a091fbc5cf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:30.348359 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1657/3287] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:03:30.351170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a222da8-f40d-4e73-97b7-909ef5a70edd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:30.364213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a222da8-f40d-4e73-97b7-909ef5a70edd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:30.370169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689f483-f65e-4cdc-8b69-803bc7670ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:30.371046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689f483-f65e-4cdc-8b69-803bc7670ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:30.371463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689f483-f65e-4cdc-8b69-803bc7670ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:30.371732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689f483-f65e-4cdc-8b69-803bc7670ce9 admin admin] PATH_INFO: `/v3/projects/d16b4d4dafe34c55aa270a0b3dd149bd/users/d28741c49cf9427fa5b670e2c979acc3/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:30.390536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689f483-f65e-4cdc-8b69-803bc7670ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d16b4d4dafe34c55aa270a0b3dd149bd, user_id=d28741c49cf9427fa5b670e2c979acc3, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:30.393357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689f483-f65e-4cdc-8b69-803bc7670ce9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:30.414170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e1020-23ed-41d0-9119-a091fbc5cf32 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:30.414383 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1658/3288] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:03:30 2022] PUT /identity/v3/projects/d16b4d4dafe34c55aa270a0b3dd149bd/users/d28741c49cf9427fa5b670e2c979acc3/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:30.420364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2c30-3140-4144-9568-3ce92be6f968 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:30.420677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2c30-3140-4144-9568-3ce92be6f968 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:30.421128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2c30-3140-4144-9568-3ce92be6f968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:30.464780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c93e2c30-3140-4144-9568-3ce92be6f968 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:30.466017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c93e2c30-3140-4144-9568-3ce92be6f968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:30.469817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:30.504093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504b98f-8ec7-4d6d-97f6-1bf63d819a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:30.505086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504b98f-8ec7-4d6d-97f6-1bf63d819a32 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:30.505716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504b98f-8ec7-4d6d-97f6-1bf63d819a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:30.505716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504b98f-8ec7-4d6d-97f6-1bf63d819a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:30.509891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:30.565851 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1659/3289] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:30.577523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e2c30-3140-4144-9568-3ce92be6f968 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:30.579309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c93e2c30-3140-4144-9568-3ce92be6f968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:30.598035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e2c30-3140-4144-9568-3ce92be6f968 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:30.602565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e037b5-d648-4a61-96f0-5a99b60babd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:30.603307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e037b5-d648-4a61-96f0-5a99b60babd5 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:30.603522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e037b5-d648-4a61-96f0-5a99b60babd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:30.604654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e037b5-d648-4a61-96f0-5a99b60babd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:30.618958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e037b5-d648-4a61-96f0-5a99b60babd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:30.620952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e037b5-d648-4a61-96f0-5a99b60babd5 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:30.621426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504b98f-8ec7-4d6d-97f6-1bf63d819a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:30.623252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504b98f-8ec7-4d6d-97f6-1bf63d819a32 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:30.647394 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1631/3290] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:03:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:03:30.665287 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1660/3291] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:03:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:03:32.243984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c32f9c-2a39-440e-9e44-ea32746d3042 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:32.244897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c32f9c-2a39-440e-9e44-ea32746d3042 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:32.245583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c32f9c-2a39-440e-9e44-ea32746d3042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:32.298471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-39c32f9c-2a39-440e-9e44-ea32746d3042 None None] MFA Rules not processed for user `d28741c49cf9427fa5b670e2c979acc3`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:32.301001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-39c32f9c-2a39-440e-9e44-ea32746d3042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:32.465278 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1632/3292] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:32.476667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88757374-b4a7-410c-919c-5f55048aedb0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:32.477223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88757374-b4a7-410c-919c-5f55048aedb0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:32.477295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88757374-b4a7-410c-919c-5f55048aedb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:32.517720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-88757374-b4a7-410c-919c-5f55048aedb0 None None] MFA Rules not processed for user `d28741c49cf9427fa5b670e2c979acc3`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:32.518992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-88757374-b4a7-410c-919c-5f55048aedb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:32.539157 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1661/3293] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:32.550020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2978b59-7649-4054-a78f-76d0c18b55b9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:32.550339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2978b59-7649-4054-a78f-76d0c18b55b9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:32.550607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2978b59-7649-4054-a78f-76d0c18b55b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:32.601108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e2978b59-7649-4054-a78f-76d0c18b55b9 None None] MFA Rules not processed for user `d28741c49cf9427fa5b670e2c979acc3`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:32.602606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e2978b59-7649-4054-a78f-76d0c18b55b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:32.623119 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1633/3294] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:32.751865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88757374-b4a7-410c-919c-5f55048aedb0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:32.753740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-88757374-b4a7-410c-919c-5f55048aedb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:32.875900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88757374-b4a7-410c-919c-5f55048aedb0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:32.883072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedb5aa-83ff-4ff5-85c5-27d1db59aa0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:32.884063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedb5aa-83ff-4ff5-85c5-27d1db59aa0b service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:32.884577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedb5aa-83ff-4ff5-85c5-27d1db59aa0b service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:32.885182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedb5aa-83ff-4ff5-85c5-27d1db59aa0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:32.895088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedb5aa-83ff-4ff5-85c5-27d1db59aa0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:32.897116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedb5aa-83ff-4ff5-85c5-27d1db59aa0b service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:32.910032 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1662/3295] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:03:32 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:03:34.810635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41779e-f748-4fff-88aa-c1bcbbd53e9d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:34.811091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41779e-f748-4fff-88aa-c1bcbbd53e9d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:34.814258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41779e-f748-4fff-88aa-c1bcbbd53e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:34.898763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4f41779e-f748-4fff-88aa-c1bcbbd53e9d None None] MFA Rules not processed for user `d28741c49cf9427fa5b670e2c979acc3`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:34.905661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4f41779e-f748-4fff-88aa-c1bcbbd53e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:34.938879 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1634/3296] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:36.767022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076437d-71af-4e42-97a8-4131cee76e39 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:36.767899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076437d-71af-4e42-97a8-4131cee76e39 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:36.768308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076437d-71af-4e42-97a8-4131cee76e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:36.853660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b076437d-71af-4e42-97a8-4131cee76e39 None None] MFA Rules not processed for user `d28741c49cf9427fa5b670e2c979acc3`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:36.865876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b076437d-71af-4e42-97a8-4131cee76e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:36.906405 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1663/3297] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:37.543565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc34cd-0a2c-439f-b631-65cd451b1915 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:37.544059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc34cd-0a2c-439f-b631-65cd451b1915 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:37.544141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc34cd-0a2c-439f-b631-65cd451b1915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:37.598258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-edcc34cd-0a2c-439f-b631-65cd451b1915 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:37.599600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-edcc34cd-0a2c-439f-b631-65cd451b1915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:37.763718 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1635/3298] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:03:37.866059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e619bba-e175-4634-ac81-76815fbf5f8f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:37.866620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e619bba-e175-4634-ac81-76815fbf5f8f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:37.866620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e619bba-e175-4634-ac81-76815fbf5f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:37.913554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7e619bba-e175-4634-ac81-76815fbf5f8f None None] MFA Rules not processed for user `bdb15c8c1f3d4bc38552b33ba1f891ca`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:37.915354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7e619bba-e175-4634-ac81-76815fbf5f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:37.993250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc34cd-0a2c-439f-b631-65cd451b1915 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:37.995210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-edcc34cd-0a2c-439f-b631-65cd451b1915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:38.012628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:38.035256 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1664/3299] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:38.046779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8da99f-8e64-4f57-ba91-21b713f4124c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:38.047174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8da99f-8e64-4f57-ba91-21b713f4124c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:38.047670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8da99f-8e64-4f57-ba91-21b713f4124c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:38.073025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7a8da99f-8e64-4f57-ba91-21b713f4124c None None] MFA Rules not processed for user `bdb15c8c1f3d4bc38552b33ba1f891ca`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:38.074347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7a8da99f-8e64-4f57-ba91-21b713f4124c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:38.091153 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1665/3300] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:38 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:38.134540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc34cd-0a2c-439f-b631-65cd451b1915 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:38.153684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943e69c-e162-4875-9c08-db2f469618e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:38.155094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943e69c-e162-4875-9c08-db2f469618e7 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:38.155375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943e69c-e162-4875-9c08-db2f469618e7 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:38.155643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943e69c-e162-4875-9c08-db2f469618e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:38.166533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7943e69c-e162-4875-9c08-db2f469618e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:38.168536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7943e69c-e162-4875-9c08-db2f469618e7 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:38.179869 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1636/3301] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:03:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:03:39.218325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8da99f-8e64-4f57-ba91-21b713f4124c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:39.240633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8da99f-8e64-4f57-ba91-21b713f4124c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:39.247499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972122f-da7a-4ccc-96ca-92e8d57164b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:39.248485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972122f-da7a-4ccc-96ca-92e8d57164b7 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:39.248982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972122f-da7a-4ccc-96ca-92e8d57164b7 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:39.249448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972122f-da7a-4ccc-96ca-92e8d57164b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:39.262889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972122f-da7a-4ccc-96ca-92e8d57164b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:39.283289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972122f-da7a-4ccc-96ca-92e8d57164b7 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:39.309021 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1666/3302] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:03:39 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:03:48.174947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=30 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:49.135160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943e69c-e162-4875-9c08-db2f469618e7 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:49.137107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7943e69c-e162-4875-9c08-db2f469618e7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:49.164590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943e69c-e162-4875-9c08-db2f469618e7 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:49.170274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:49.171110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:49.171538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:49.172030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] PATH_INFO: `/v3/users/896d2aaaf6cc42a69affa6342f65973a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:49.173367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] RBAC: Authorizing `identity:delete_user(user_id=896d2aaaf6cc42a69affa6342f65973a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:49.175671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:49.289839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:49.332392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '896d2aaaf6cc42a69affa6342f65973a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:49.344721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '896d2aaaf6cc42a69affa6342f65973a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:49.373486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '896d2aaaf6cc42a69affa6342f65973a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:49.374158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '896d2aaaf6cc42a69affa6342f65973a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:49.375227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '896d2aaaf6cc42a69affa6342f65973a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:49.397401 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1637/3303] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:03:49 2022] DELETE /identity/v3/users/896d2aaaf6cc42a69affa6342f65973a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:49.402711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43492805-5568-47ff-a92a-08b25170b269 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:49.403020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43492805-5568-47ff-a92a-08b25170b269 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:49.403634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43492805-5568-47ff-a92a-08b25170b269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:49.436995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-43492805-5568-47ff-a92a-08b25170b269 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:49.441359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-43492805-5568-47ff-a92a-08b25170b269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:49.600347 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1667/3304] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:49.610163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:49.612115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:49.708035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886db5a1-1247-4b39-9072-5c594341369b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:49.712638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ce55d-2b8d-4566-b2e8-2daa5af741da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:49.714898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ce55d-2b8d-4566-b2e8-2daa5af741da service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:49.715256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ce55d-2b8d-4566-b2e8-2daa5af741da service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:49.715609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ce55d-2b8d-4566-b2e8-2daa5af741da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:49.728408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ce55d-2b8d-4566-b2e8-2daa5af741da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:49.730525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ce55d-2b8d-4566-b2e8-2daa5af741da service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:49.748529 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1638/3305] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:03:50.466735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43492805-5568-47ff-a92a-08b25170b269 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:50.468199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-43492805-5568-47ff-a92a-08b25170b269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:50.487933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43492805-5568-47ff-a92a-08b25170b269 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:50.497044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04572682-cef3-40e5-9c03-1053cc6ecc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:50.499279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04572682-cef3-40e5-9c03-1053cc6ecc52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:50.500860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04572682-cef3-40e5-9c03-1053cc6ecc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:50.501446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04572682-cef3-40e5-9c03-1053cc6ecc52 admin admin] PATH_INFO: `/v3/projects/93eac1ec0c394b66ac622cb9669e92fe` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:50.506262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04572682-cef3-40e5-9c03-1053cc6ecc52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93eac1ec0c394b66ac622cb9669e92fe)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:50.512335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04572682-cef3-40e5-9c03-1053cc6ecc52 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:50.633988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-04572682-cef3-40e5-9c03-1053cc6ecc52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93eac1ec0c394b66ac622cb9669e92fe', 'request_id': 'req-04572682-cef3-40e5-9c03-1053cc6ecc52'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:50.655871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-04572682-cef3-40e5-9c03-1053cc6ecc52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93eac1ec0c394b66ac622cb9669e92fe', 'request_id': 'req-04572682-cef3-40e5-9c03-1053cc6ecc52'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:50.662438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-04572682-cef3-40e5-9c03-1053cc6ecc52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93eac1ec0c394b66ac622cb9669e92fe', 'request_id': 'req-04572682-cef3-40e5-9c03-1053cc6ecc52'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:50.696694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-04572682-cef3-40e5-9c03-1053cc6ecc52 admin admin] The token cache is being invalidate because project 93eac1ec0c394b66ac622cb9669e92fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:03:50.697047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-04572682-cef3-40e5-9c03-1053cc6ecc52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:50.697959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-04572682-cef3-40e5-9c03-1053cc6ecc52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:03:50.699682 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1668/3306] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:03:50 2022] DELETE /identity/v3/projects/93eac1ec0c394b66ac622cb9669e92fe => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:50.704950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff3c13-567f-4852-be73-fa7bad79084b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:50.705482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff3c13-567f-4852-be73-fa7bad79084b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:50.705561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff3c13-567f-4852-be73-fa7bad79084b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:50.767515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-77ff3c13-567f-4852-be73-fa7bad79084b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:50.768975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-77ff3c13-567f-4852-be73-fa7bad79084b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:50.987872 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1639/3307] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:50.990059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1673fa-a9fa-4951-b705-181f50ea8838 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:50.990565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1673fa-a9fa-4951-b705-181f50ea8838 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:50.991023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1673fa-a9fa-4951-b705-181f50ea8838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:51.028117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8b1673fa-a9fa-4951-b705-181f50ea8838 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:51.031939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8b1673fa-a9fa-4951-b705-181f50ea8838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:51.055958 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1669/3308] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:51.060631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff3c13-567f-4852-be73-fa7bad79084b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:51.086057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff3c13-567f-4852-be73-fa7bad79084b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:51.107479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a4e39-e7fc-4c95-ae05-66b11f892376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:51.108105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a4e39-e7fc-4c95-ae05-66b11f892376 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:51.108403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a4e39-e7fc-4c95-ae05-66b11f892376 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:51.108667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a4e39-e7fc-4c95-ae05-66b11f892376 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:51.109316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a4e39-e7fc-4c95-ae05-66b11f892376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:03:51.109595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a4e39-e7fc-4c95-ae05-66b11f892376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:51.111215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a4e39-e7fc-4c95-ae05-66b11f892376 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:51.161880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1673fa-a9fa-4951-b705-181f50ea8838 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:51.170594 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1640/3309] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:03:51.180914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1673fa-a9fa-4951-b705-181f50ea8838 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:51.194722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5a3c3-89ae-4e86-aaf2-77771f123dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:51.196189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a3c3-89ae-4e86-aaf2-77771f123dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:51.198287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a3c3-89ae-4e86-aaf2-77771f123dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:51.198716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a3c3-89ae-4e86-aaf2-77771f123dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:51.199504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5a3c3-89ae-4e86-aaf2-77771f123dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:51.204295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5a3c3-89ae-4e86-aaf2-77771f123dc3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:51.265885 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1670/3310] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:03:51 2022] POST /identity/v3/projects => generated 403 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:03:51.269486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a4e39-e7fc-4c95-ae05-66b11f892376 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:51.276734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:51.282460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a4e39-e7fc-4c95-ae05-66b11f892376 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:51.425373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4927acc-d2bb-46f8-be16-86b2b0d3ccfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:51.425999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4927acc-d2bb-46f8-be16-86b2b0d3ccfa admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:51.426262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4927acc-d2bb-46f8-be16-86b2b0d3ccfa admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:51.426534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4927acc-d2bb-46f8-be16-86b2b0d3ccfa admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:51.427153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4927acc-d2bb-46f8-be16-86b2b0d3ccfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:51.429123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4927acc-d2bb-46f8-be16-86b2b0d3ccfa admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:51.474834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5a3c3-89ae-4e86-aaf2-77771f123dc3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:51.482295 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1641/3311] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:03:51 2022] POST /identity/v3/users => generated 302 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:03:51.489325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5a3c3-89ae-4e86-aaf2-77771f123dc3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:51.499326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa3011-a2a0-42e7-8503-48d515676ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:51.499888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa3011-a2a0-42e7-8503-48d515676ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:51.500135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa3011-a2a0-42e7-8503-48d515676ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:51.500412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa3011-a2a0-42e7-8503-48d515676ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:51.508219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa3011-a2a0-42e7-8503-48d515676ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:51.510615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa3011-a2a0-42e7-8503-48d515676ec7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:51.533596 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1671/3312] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:03:51.537986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4927acc-d2bb-46f8-be16-86b2b0d3ccfa admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:51.549759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4927acc-d2bb-46f8-be16-86b2b0d3ccfa admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:51.555431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08bdd4-6c8b-47f5-9385-93a9ef789572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:51.556928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08bdd4-6c8b-47f5-9385-93a9ef789572 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:51.557414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08bdd4-6c8b-47f5-9385-93a9ef789572 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:51.557904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08bdd4-6c8b-47f5-9385-93a9ef789572 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:51.559250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08bdd4-6c8b-47f5-9385-93a9ef789572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:51.561169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08bdd4-6c8b-47f5-9385-93a9ef789572 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:51.575410 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1642/3313] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:03:51.579725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa3011-a2a0-42e7-8503-48d515676ec7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:51.595355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa3011-a2a0-42e7-8503-48d515676ec7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:51.614626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876d37-d11f-4fd8-8cd6-00b12f5013ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:51.615316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876d37-d11f-4fd8-8cd6-00b12f5013ff admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:51.615893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876d37-d11f-4fd8-8cd6-00b12f5013ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:51.616169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876d37-d11f-4fd8-8cd6-00b12f5013ff admin admin] PATH_INFO: `/v3/projects/7cc44d3240bd4a6390a63c00a6f45006/users/f6e104705be14b02a1f3c7a2bebf2aff/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:51.651014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c876d37-d11f-4fd8-8cd6-00b12f5013ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc44d3240bd4a6390a63c00a6f45006, user_id=f6e104705be14b02a1f3c7a2bebf2aff, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:51.653955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c876d37-d11f-4fd8-8cd6-00b12f5013ff admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:51.682531 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1672/3314] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:03:51 2022] PUT /identity/v3/projects/7cc44d3240bd4a6390a63c00a6f45006/users/f6e104705be14b02a1f3c7a2bebf2aff/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:03:51.687633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa86248-c9cb-4de1-bac3-704533c1e5d8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:51.688168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa86248-c9cb-4de1-bac3-704533c1e5d8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:51.688623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa86248-c9cb-4de1-bac3-704533c1e5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:51.724126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9aa86248-c9cb-4de1-bac3-704533c1e5d8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:03:51.726393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9aa86248-c9cb-4de1-bac3-704533c1e5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:51.748831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876d37-d11f-4fd8-8cd6-00b12f5013ff admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:51.750970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6c876d37-d11f-4fd8-8cd6-00b12f5013ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:51.803858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:03:51.905368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876d37-d11f-4fd8-8cd6-00b12f5013ff admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:51.923836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f07e45-839f-4f13-aad4-acae3c9dca22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:51.924815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f07e45-839f-4f13-aad4-acae3c9dca22 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:51.925105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f07e45-839f-4f13-aad4-acae3c9dca22 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:51.925377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f07e45-839f-4f13-aad4-acae3c9dca22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:51.930214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a4f07e45-839f-4f13-aad4-acae3c9dca22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:51.994129 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1643/3315] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:03:52.051137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa86248-c9cb-4de1-bac3-704533c1e5d8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:03:52.052625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9aa86248-c9cb-4de1-bac3-704533c1e5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:03:52.097657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f07e45-839f-4f13-aad4-acae3c9dca22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:52.099287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f07e45-839f-4f13-aad4-acae3c9dca22 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:52.128981 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1673/3316] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:03:51 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:03:52.270312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa86248-c9cb-4de1-bac3-704533c1e5d8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:03:52.276062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac54b18-b786-40dc-9459-6a40fcd6d239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:03:52.276857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac54b18-b786-40dc-9459-6a40fcd6d239 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:03:52.277158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac54b18-b786-40dc-9459-6a40fcd6d239 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:03:52.277467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac54b18-b786-40dc-9459-6a40fcd6d239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:03:52.291252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac54b18-b786-40dc-9459-6a40fcd6d239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:03:52.293214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac54b18-b786-40dc-9459-6a40fcd6d239 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:03:52.307075 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1644/3317] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:04:02.116049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=41 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:02.300865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:03.508651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c9191-3413-44b1-9f00-70a5c0f654bc None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:03.509109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c9191-3413-44b1-9f00-70a5c0f654bc None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:03.509274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c9191-3413-44b1-9f00-70a5c0f654bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:03.552485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-452c9191-3413-44b1-9f00-70a5c0f654bc None None] MFA Rules not processed for user `f6e104705be14b02a1f3c7a2bebf2aff`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:03.553831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-452c9191-3413-44b1-9f00-70a5c0f654bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:03.654203 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1674/3318] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:03 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:03.658667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac54b18-b786-40dc-9459-6a40fcd6d239 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:03.677594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac54b18-b786-40dc-9459-6a40fcd6d239 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:03.684072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d8940-1a8c-48a5-a610-270e008cadbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:03.684896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d8940-1a8c-48a5-a610-270e008cadbe admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:03.686010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d8940-1a8c-48a5-a610-270e008cadbe admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:03.686411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d8940-1a8c-48a5-a610-270e008cadbe admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:03.687064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d8940-1a8c-48a5-a610-270e008cadbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:03.689405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d8940-1a8c-48a5-a610-270e008cadbe admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:03.731431 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1645/3319] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:04:03 2022] POST /identity/v3/projects => generated 401 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:04:03.735341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c9191-3413-44b1-9f00-70a5c0f654bc None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:03.747191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c9191-3413-44b1-9f00-70a5c0f654bc None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:03.814249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d61606-f76d-460e-90b1-64c9c74e7331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:03.814953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d61606-f76d-460e-90b1-64c9c74e7331 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:03.815470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d61606-f76d-460e-90b1-64c9c74e7331 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:03.815652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d61606-f76d-460e-90b1-64c9c74e7331 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:03.816447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d61606-f76d-460e-90b1-64c9c74e7331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:03.818792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d61606-f76d-460e-90b1-64c9c74e7331 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:03.845927 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1675/3320] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:04:03 2022] POST /identity/v3/users => generated 301 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:04:03.850495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d8940-1a8c-48a5-a610-270e008cadbe admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:03.862871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d8940-1a8c-48a5-a610-270e008cadbe admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:03.869195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529a0f9-8101-45b3-9b02-dbad92b80d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:03.869995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529a0f9-8101-45b3-9b02-dbad92b80d7b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:03.870411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529a0f9-8101-45b3-9b02-dbad92b80d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:03.870719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529a0f9-8101-45b3-9b02-dbad92b80d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:03.871382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529a0f9-8101-45b3-9b02-dbad92b80d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:03.873044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529a0f9-8101-45b3-9b02-dbad92b80d7b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:03.887837 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1646/3321] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:04:03.889124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d61606-f76d-460e-90b1-64c9c74e7331 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:03.899861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d61606-f76d-460e-90b1-64c9c74e7331 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:03.905194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fb304-fc46-45ab-b2ab-4424e0a4acb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:03.906546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fb304-fc46-45ab-b2ab-4424e0a4acb6 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:03.906546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fb304-fc46-45ab-b2ab-4424e0a4acb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:03.906925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fb304-fc46-45ab-b2ab-4424e0a4acb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:03.907651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fb304-fc46-45ab-b2ab-4424e0a4acb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:03.909863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fb304-fc46-45ab-b2ab-4424e0a4acb6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:03.924848 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1676/3322] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:04:03.926435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529a0f9-8101-45b3-9b02-dbad92b80d7b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:03.938002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529a0f9-8101-45b3-9b02-dbad92b80d7b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:03.943094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2269cb-9980-47c5-aaa9-1d8c6f10be3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:03.943812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2269cb-9980-47c5-aaa9-1d8c6f10be3a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:03.944089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2269cb-9980-47c5-aaa9-1d8c6f10be3a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:03.944358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2269cb-9980-47c5-aaa9-1d8c6f10be3a admin admin] PATH_INFO: `/v3/projects/cfc2ad0194bf418b8ff85ca6fb0a0fc9/users/fbd13a56acd246e4ab7ff547b9d848e0/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:03.962901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2269cb-9980-47c5-aaa9-1d8c6f10be3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc2ad0194bf418b8ff85ca6fb0a0fc9, user_id=fbd13a56acd246e4ab7ff547b9d848e0, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:03.965842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2269cb-9980-47c5-aaa9-1d8c6f10be3a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:03.984266 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1647/3323] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:04:03 2022] PUT /identity/v3/projects/cfc2ad0194bf418b8ff85ca6fb0a0fc9/users/fbd13a56acd246e4ab7ff547b9d848e0/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:09.405902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9fe46-0f7c-4a65-a6f2-2b7a86ddee16 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:09.406809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9fe46-0f7c-4a65-a6f2-2b7a86ddee16 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:09.407239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9fe46-0f7c-4a65-a6f2-2b7a86ddee16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:09.444674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-16e9fe46-0f7c-4a65-a6f2-2b7a86ddee16 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:09.447983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-16e9fe46-0f7c-4a65-a6f2-2b7a86ddee16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:09.547146 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1677/3324] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:04:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:09.578412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2269cb-9980-47c5-aaa9-1d8c6f10be3a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:09.580300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0e2269cb-9980-47c5-aaa9-1d8c6f10be3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:09.758870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2269cb-9980-47c5-aaa9-1d8c6f10be3a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:09.771924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edbf92-2b1c-4f14-9455-7b356a656bcd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:09.772586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edbf92-2b1c-4f14-9455-7b356a656bcd service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:09.773384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edbf92-2b1c-4f14-9455-7b356a656bcd service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:09.773701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edbf92-2b1c-4f14-9455-7b356a656bcd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:09.802511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edbf92-2b1c-4f14-9455-7b356a656bcd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:09.802511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edbf92-2b1c-4f14-9455-7b356a656bcd service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:09.826650 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1648/3325] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:04:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:04:11.177274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee84b39-ec66-4db3-a967-afaee57822c5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:11.177830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee84b39-ec66-4db3-a967-afaee57822c5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:11.177830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee84b39-ec66-4db3-a967-afaee57822c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:11.208804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:11.214743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7ee84b39-ec66-4db3-a967-afaee57822c5 None None] MFA Rules not processed for user `fbd13a56acd246e4ab7ff547b9d848e0`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:11.217227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7ee84b39-ec66-4db3-a967-afaee57822c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:11.365202 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1678/3326] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:11 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:11.375457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdd7db-a110-4629-b58a-dce4ffe00ea0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:11.376314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdd7db-a110-4629-b58a-dce4ffe00ea0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:11.376821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdd7db-a110-4629-b58a-dce4ffe00ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:11.413719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9ffdd7db-a110-4629-b58a-dce4ffe00ea0 None None] MFA Rules not processed for user `f6e104705be14b02a1f3c7a2bebf2aff`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:11.415011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9ffdd7db-a110-4629-b58a-dce4ffe00ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:11.528427 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1649/3327] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:11 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:11.650072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee84b39-ec66-4db3-a967-afaee57822c5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:11.651602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7ee84b39-ec66-4db3-a967-afaee57822c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:11.820068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee84b39-ec66-4db3-a967-afaee57822c5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:11.830887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddde0d-99ba-4c87-97f8-b6f9e5d566bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:11.831835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddde0d-99ba-4c87-97f8-b6f9e5d566bb service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:11.832714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddde0d-99ba-4c87-97f8-b6f9e5d566bb service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:11.833189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddde0d-99ba-4c87-97f8-b6f9e5d566bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:11.845214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbddde0d-99ba-4c87-97f8-b6f9e5d566bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:11.848791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbddde0d-99ba-4c87-97f8-b6f9e5d566bb service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:11.867827 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1679/3328] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:04:11 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:04:12.672570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0b0ac-120e-4b8b-bf8a-37c09567c666 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:12.673116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0b0ac-120e-4b8b-bf8a-37c09567c666 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:12.673116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0b0ac-120e-4b8b-bf8a-37c09567c666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:12.707669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c0d0b0ac-120e-4b8b-bf8a-37c09567c666 None None] MFA Rules not processed for user `bdb15c8c1f3d4bc38552b33ba1f891ca`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:12.709716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c0d0b0ac-120e-4b8b-bf8a-37c09567c666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:12.763651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:12.818348 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1650/3329] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:12 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:17.978386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddde0d-99ba-4c87-97f8-b6f9e5d566bb service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:17.981948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cbddde0d-99ba-4c87-97f8-b6f9e5d566bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:18.180008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddde0d-99ba-4c87-97f8-b6f9e5d566bb service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:18.190459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f35bb-6668-4f9f-bca7-d91d8e68502b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:18.191123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f35bb-6668-4f9f-bca7-d91d8e68502b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:18.191400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f35bb-6668-4f9f-bca7-d91d8e68502b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:18.191668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f35bb-6668-4f9f-bca7-d91d8e68502b admin admin] PATH_INFO: `/v3/users/815db4215e8b4b1984e199a40ed107c1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:18.196725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f35bb-6668-4f9f-bca7-d91d8e68502b admin admin] RBAC: Authorizing `identity:delete_user(user_id=815db4215e8b4b1984e199a40ed107c1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:18.199114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f35bb-6668-4f9f-bca7-d91d8e68502b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:18.307887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:18.350070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f03f35bb-6668-4f9f-bca7-d91d8e68502b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '815db4215e8b4b1984e199a40ed107c1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:18.376496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f03f35bb-6668-4f9f-bca7-d91d8e68502b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '815db4215e8b4b1984e199a40ed107c1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:18.396298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f03f35bb-6668-4f9f-bca7-d91d8e68502b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '815db4215e8b4b1984e199a40ed107c1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:18.424875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f03f35bb-6668-4f9f-bca7-d91d8e68502b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '815db4215e8b4b1984e199a40ed107c1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:18.425783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f03f35bb-6668-4f9f-bca7-d91d8e68502b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '815db4215e8b4b1984e199a40ed107c1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:18.443822 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1680/3330] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:04:17 2022] DELETE /identity/v3/users/815db4215e8b4b1984e199a40ed107c1 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:18.745659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0b0ac-120e-4b8b-bf8a-37c09567c666 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:18.776582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0b0ac-120e-4b8b-bf8a-37c09567c666 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:18.958153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2c540-5178-459a-8f3b-7c145e44f762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:18.958895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2c540-5178-459a-8f3b-7c145e44f762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:18.959176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2c540-5178-459a-8f3b-7c145e44f762 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:18.959457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2c540-5178-459a-8f3b-7c145e44f762 admin admin] PATH_INFO: `/v3/projects/bb300cc2384b4c63a23d57242f6704cb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:18.962684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2c540-5178-459a-8f3b-7c145e44f762 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb300cc2384b4c63a23d57242f6704cb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:18.965077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2c540-5178-459a-8f3b-7c145e44f762 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:19.056545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5d2c540-5178-459a-8f3b-7c145e44f762 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb300cc2384b4c63a23d57242f6704cb', 'request_id': 'req-c5d2c540-5178-459a-8f3b-7c145e44f762'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:19.071926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5d2c540-5178-459a-8f3b-7c145e44f762 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb300cc2384b4c63a23d57242f6704cb', 'request_id': 'req-c5d2c540-5178-459a-8f3b-7c145e44f762'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:19.085030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5d2c540-5178-459a-8f3b-7c145e44f762 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb300cc2384b4c63a23d57242f6704cb', 'request_id': 'req-c5d2c540-5178-459a-8f3b-7c145e44f762'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:19.118517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5d2c540-5178-459a-8f3b-7c145e44f762 admin admin] The token cache is being invalidate because project bb300cc2384b4c63a23d57242f6704cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:04:19.119099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5d2c540-5178-459a-8f3b-7c145e44f762 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:19.119980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5d2c540-5178-459a-8f3b-7c145e44f762 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:19.122067 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1651/3331] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:04:18 2022] DELETE /identity/v3/projects/bb300cc2384b4c63a23d57242f6704cb => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:19.129053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53198fb2-adc6-4ed4-9a4a-ab466948854b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:19.130769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53198fb2-adc6-4ed4-9a4a-ab466948854b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:19.131246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53198fb2-adc6-4ed4-9a4a-ab466948854b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:19.196330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-53198fb2-adc6-4ed4-9a4a-ab466948854b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:19.200461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-53198fb2-adc6-4ed4-9a4a-ab466948854b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:19.374660 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1681/3332] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:19.382831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fee0e-e78e-43d0-8a2d-b4c2b5b2682b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:19.384095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fee0e-e78e-43d0-8a2d-b4c2b5b2682b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:19.384332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fee0e-e78e-43d0-8a2d-b4c2b5b2682b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:19.420036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9f0fee0e-e78e-43d0-8a2d-b4c2b5b2682b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:19.422597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9f0fee0e-e78e-43d0-8a2d-b4c2b5b2682b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:19.440805 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1652/3333] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:19.444874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53198fb2-adc6-4ed4-9a4a-ab466948854b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:19.458534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53198fb2-adc6-4ed4-9a4a-ab466948854b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:19.473152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdac54-a56a-40ad-8884-4a0ec29e42d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:19.474897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdac54-a56a-40ad-8884-4a0ec29e42d2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:19.475174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdac54-a56a-40ad-8884-4a0ec29e42d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:19.476686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdac54-a56a-40ad-8884-4a0ec29e42d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:19.477380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdac54-a56a-40ad-8884-4a0ec29e42d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:04:19.477790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdac54-a56a-40ad-8884-4a0ec29e42d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:19.483392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdac54-a56a-40ad-8884-4a0ec29e42d2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:19.517141 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1682/3334] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:04:19.524091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fee0e-e78e-43d0-8a2d-b4c2b5b2682b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:19.537480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fee0e-e78e-43d0-8a2d-b4c2b5b2682b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:19.562475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2aa6dd-8da1-40e8-991c-dc31dff4d0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:19.563114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2aa6dd-8da1-40e8-991c-dc31dff4d0de admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:19.563389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2aa6dd-8da1-40e8-991c-dc31dff4d0de admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:19.563654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2aa6dd-8da1-40e8-991c-dc31dff4d0de admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:19.564285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2aa6dd-8da1-40e8-991c-dc31dff4d0de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:19.575172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2aa6dd-8da1-40e8-991c-dc31dff4d0de admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:19.679625 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1653/3335] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:04:19 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:04:19.683741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdac54-a56a-40ad-8884-4a0ec29e42d2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:19.698611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdac54-a56a-40ad-8884-4a0ec29e42d2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:19.819713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacff4f-0071-4721-87e8-65666a9c9654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:19.822009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacff4f-0071-4721-87e8-65666a9c9654 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:19.822009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacff4f-0071-4721-87e8-65666a9c9654 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:19.822009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacff4f-0071-4721-87e8-65666a9c9654 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:19.822511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacff4f-0071-4721-87e8-65666a9c9654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:19.824776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacff4f-0071-4721-87e8-65666a9c9654 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:19.866003 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1683/3336] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:04:19 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:04:19.867299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2aa6dd-8da1-40e8-991c-dc31dff4d0de admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:19.879905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2aa6dd-8da1-40e8-991c-dc31dff4d0de admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:19.885196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a526129-18ea-432b-97b7-f850e4242c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:19.886104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a526129-18ea-432b-97b7-f850e4242c49 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:19.886562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a526129-18ea-432b-97b7-f850e4242c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:19.887036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a526129-18ea-432b-97b7-f850e4242c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:19.887910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a526129-18ea-432b-97b7-f850e4242c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:19.890801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a526129-18ea-432b-97b7-f850e4242c49 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:19.917629 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1654/3337] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:04:19.923483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacff4f-0071-4721-87e8-65666a9c9654 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:19.941537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacff4f-0071-4721-87e8-65666a9c9654 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:19.949777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1142fe0-7623-4e14-b74f-fb53b63114df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:19.950408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1142fe0-7623-4e14-b74f-fb53b63114df admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:19.950713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1142fe0-7623-4e14-b74f-fb53b63114df admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:19.951112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1142fe0-7623-4e14-b74f-fb53b63114df admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:19.951776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1142fe0-7623-4e14-b74f-fb53b63114df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:19.954256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1142fe0-7623-4e14-b74f-fb53b63114df admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:19.970326 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1684/3338] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:04:19.974865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a526129-18ea-432b-97b7-f850e4242c49 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:19.992103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a526129-18ea-432b-97b7-f850e4242c49 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:20.006163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a04ef-a8f2-47ab-9ef5-c7a26ea18447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:20.007201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a04ef-a8f2-47ab-9ef5-c7a26ea18447 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:20.007810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a04ef-a8f2-47ab-9ef5-c7a26ea18447 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:20.008663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a04ef-a8f2-47ab-9ef5-c7a26ea18447 admin admin] PATH_INFO: `/v3/projects/a4f9c633b6d54ac1869108eadb004a87/users/b20aea7088dc4e4fadb737c906cf7ea2/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:20.046593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a04ef-a8f2-47ab-9ef5-c7a26ea18447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f9c633b6d54ac1869108eadb004a87, user_id=b20aea7088dc4e4fadb737c906cf7ea2, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:20.051875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a04ef-a8f2-47ab-9ef5-c7a26ea18447 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:20.088702 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1655/3339] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:04:19 2022] PUT /identity/v3/projects/a4f9c633b6d54ac1869108eadb004a87/users/b20aea7088dc4e4fadb737c906cf7ea2/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:20.094573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dc6d4-a57b-4eef-a86c-a9b45220ae3b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:20.095231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dc6d4-a57b-4eef-a86c-a9b45220ae3b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:20.095481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dc6d4-a57b-4eef-a86c-a9b45220ae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:20.123355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-089dc6d4-a57b-4eef-a86c-a9b45220ae3b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:20.124888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-089dc6d4-a57b-4eef-a86c-a9b45220ae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:20.135478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:20.232503 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1685/3340] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:04:20.244871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a04ef-a8f2-47ab-9ef5-c7a26ea18447 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:20.246807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a6a04ef-a8f2-47ab-9ef5-c7a26ea18447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:20.260107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:20.342553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a04ef-a8f2-47ab-9ef5-c7a26ea18447 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:20.353548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33a822-167a-4db9-a5f3-2822b693c296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:20.354190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33a822-167a-4db9-a5f3-2822b693c296 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:20.354485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33a822-167a-4db9-a5f3-2822b693c296 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:20.354750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33a822-167a-4db9-a5f3-2822b693c296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:20.377820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33a822-167a-4db9-a5f3-2822b693c296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:20.380319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33a822-167a-4db9-a5f3-2822b693c296 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:20.396464 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1656/3341] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:04:28.417871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089dc6d4-a57b-4eef-a86c-a9b45220ae3b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:28.420011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-089dc6d4-a57b-4eef-a86c-a9b45220ae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:28.508138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089dc6d4-a57b-4eef-a86c-a9b45220ae3b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:28.514805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52862c2c-3a10-428f-ada2-3a3e73c5d59e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:28.515649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52862c2c-3a10-428f-ada2-3a3e73c5d59e service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:28.516022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52862c2c-3a10-428f-ada2-3a3e73c5d59e service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:28.516413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52862c2c-3a10-428f-ada2-3a3e73c5d59e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:28.518388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-52862c2c-3a10-428f-ada2-3a3e73c5d59e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:28.587331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52862c2c-3a10-428f-ada2-3a3e73c5d59e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:28.589287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52862c2c-3a10-428f-ada2-3a3e73c5d59e service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:28.601854 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1686/3342] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:04:28 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:04:28.946844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311696f9-5a0d-4545-bc9f-07fe6a8e42f6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:28.947941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311696f9-5a0d-4545-bc9f-07fe6a8e42f6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:28.948221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311696f9-5a0d-4545-bc9f-07fe6a8e42f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:28.992943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-311696f9-5a0d-4545-bc9f-07fe6a8e42f6 None None] MFA Rules not processed for user `b20aea7088dc4e4fadb737c906cf7ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:28.992943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-311696f9-5a0d-4545-bc9f-07fe6a8e42f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:29.082296 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1657/3343] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:29.098139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db18ba-cfa1-4e26-b298-da5df3f42a11 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:29.098481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db18ba-cfa1-4e26-b298-da5df3f42a11 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:29.098859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db18ba-cfa1-4e26-b298-da5df3f42a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:29.119589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-73db18ba-cfa1-4e26-b298-da5df3f42a11 None None] MFA Rules not processed for user `b20aea7088dc4e4fadb737c906cf7ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:29.120866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-73db18ba-cfa1-4e26-b298-da5df3f42a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:29.127695 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1687/3344] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:29.139031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311696f9-5a0d-4545-bc9f-07fe6a8e42f6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:29.154701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311696f9-5a0d-4545-bc9f-07fe6a8e42f6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:29.161272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd8433-1fb5-4b10-95bb-b791f192bbfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:29.162188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd8433-1fb5-4b10-95bb-b791f192bbfa service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:29.162611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd8433-1fb5-4b10-95bb-b791f192bbfa service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:29.162880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd8433-1fb5-4b10-95bb-b791f192bbfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:29.174134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd8433-1fb5-4b10-95bb-b791f192bbfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:29.175881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd8433-1fb5-4b10-95bb-b791f192bbfa service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:29.187598 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1658/3345] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:04:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:04:39.114280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=80 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:39.181450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=60 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:40.791371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665ccd0-e5f0-4728-93fe-b78c5510c4f2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:40.791977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665ccd0-e5f0-4728-93fe-b78c5510c4f2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:40.791977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665ccd0-e5f0-4728-93fe-b78c5510c4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:40.817736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b665ccd0-e5f0-4728-93fe-b78c5510c4f2 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:40.822576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b665ccd0-e5f0-4728-93fe-b78c5510c4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:40.925045 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1688/3346] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:04:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:40.952067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd8433-1fb5-4b10-95bb-b791f192bbfa service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:40.954166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-87bd8433-1fb5-4b10-95bb-b791f192bbfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:41.045853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd8433-1fb5-4b10-95bb-b791f192bbfa service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:41.051338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc63e99-f4a8-40c0-bc0a-6ad9a317c5a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:41.052028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc63e99-f4a8-40c0-bc0a-6ad9a317c5a5 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:41.052385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc63e99-f4a8-40c0-bc0a-6ad9a317c5a5 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:41.052731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc63e99-f4a8-40c0-bc0a-6ad9a317c5a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:41.066153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc63e99-f4a8-40c0-bc0a-6ad9a317c5a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:41.068053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc63e99-f4a8-40c0-bc0a-6ad9a317c5a5 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:41.080098 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1659/3347] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:04:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:04:48.580032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665ccd0-e5f0-4728-93fe-b78c5510c4f2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:48.581649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b665ccd0-e5f0-4728-93fe-b78c5510c4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:48.613763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665ccd0-e5f0-4728-93fe-b78c5510c4f2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:48.622277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:48.623264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:48.623712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:48.624220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] PATH_INFO: `/v3/users/f6e104705be14b02a1f3c7a2bebf2aff` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:48.625886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e104705be14b02a1f3c7a2bebf2aff)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:48.628016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:48.717245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e104705be14b02a1f3c7a2bebf2aff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:48.729740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e104705be14b02a1f3c7a2bebf2aff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:48.742278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e104705be14b02a1f3c7a2bebf2aff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:48.759798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e104705be14b02a1f3c7a2bebf2aff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:48.760913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e104705be14b02a1f3c7a2bebf2aff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:48.772147 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1689/3348] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:04:48 2022] DELETE /identity/v3/users/f6e104705be14b02a1f3c7a2bebf2aff => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:49.050310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc63e99-f4a8-40c0-bc0a-6ad9a317c5a5 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:49.066121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc63e99-f4a8-40c0-bc0a-6ad9a317c5a5 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:49.234576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24a189-1294-4914-a9f0-fa1093b8d687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:49.236534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24a189-1294-4914-a9f0-fa1093b8d687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:49.238722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24a189-1294-4914-a9f0-fa1093b8d687 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:49.239108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24a189-1294-4914-a9f0-fa1093b8d687 admin admin] PATH_INFO: `/v3/projects/7cc44d3240bd4a6390a63c00a6f45006` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:49.241277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24a189-1294-4914-a9f0-fa1093b8d687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cc44d3240bd4a6390a63c00a6f45006)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:49.244138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24a189-1294-4914-a9f0-fa1093b8d687 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:49.334691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c24a189-1294-4914-a9f0-fa1093b8d687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cc44d3240bd4a6390a63c00a6f45006', 'request_id': 'req-4c24a189-1294-4914-a9f0-fa1093b8d687'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:49.343672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c24a189-1294-4914-a9f0-fa1093b8d687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cc44d3240bd4a6390a63c00a6f45006', 'request_id': 'req-4c24a189-1294-4914-a9f0-fa1093b8d687'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:49.354610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c24a189-1294-4914-a9f0-fa1093b8d687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cc44d3240bd4a6390a63c00a6f45006', 'request_id': 'req-4c24a189-1294-4914-a9f0-fa1093b8d687'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:49.377502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c24a189-1294-4914-a9f0-fa1093b8d687 admin admin] The token cache is being invalidate because project 7cc44d3240bd4a6390a63c00a6f45006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:04:49.377502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c24a189-1294-4914-a9f0-fa1093b8d687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:49.377502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c24a189-1294-4914-a9f0-fa1093b8d687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:49.379157 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1660/3349] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:04:49 2022] DELETE /identity/v3/projects/7cc44d3240bd4a6390a63c00a6f45006 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:49.383788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:49.385853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:49.400702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee072e-851d-4e30-96eb-664d1ae80ee5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:49.484651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb468794-b64e-4215-9c15-a1b5253c3e89 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:49.485133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb468794-b64e-4215-9c15-a1b5253c3e89 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:49.485548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb468794-b64e-4215-9c15-a1b5253c3e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:49.519796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cb468794-b64e-4215-9c15-a1b5253c3e89 None None] MFA Rules not processed for user `b20aea7088dc4e4fadb737c906cf7ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:49.526694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cb468794-b64e-4215-9c15-a1b5253c3e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:49.540466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd9707-d8ef-4e47-9a33-feca63604da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:49.541993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd9707-d8ef-4e47-9a33-feca63604da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:49.542499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd9707-d8ef-4e47-9a33-feca63604da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:49.543561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd9707-d8ef-4e47-9a33-feca63604da3 admin admin] PATH_INFO: `/v3/users/fbd13a56acd246e4ab7ff547b9d848e0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:49.544780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd9707-d8ef-4e47-9a33-feca63604da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbd13a56acd246e4ab7ff547b9d848e0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:49.547311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd9707-d8ef-4e47-9a33-feca63604da3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:49.556863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:49.584148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:49.628657 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1661/3350] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:49.638583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4dd9707-d8ef-4e47-9a33-feca63604da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbd13a56acd246e4ab7ff547b9d848e0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:49.661033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4dd9707-d8ef-4e47-9a33-feca63604da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbd13a56acd246e4ab7ff547b9d848e0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:49.684145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4dd9707-d8ef-4e47-9a33-feca63604da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbd13a56acd246e4ab7ff547b9d848e0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:49.720211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4dd9707-d8ef-4e47-9a33-feca63604da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbd13a56acd246e4ab7ff547b9d848e0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:49.721072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4dd9707-d8ef-4e47-9a33-feca63604da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbd13a56acd246e4ab7ff547b9d848e0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:49.733427 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1690/3351] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:04:49 2022] DELETE /identity/v3/users/fbd13a56acd246e4ab7ff547b9d848e0 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:50.067546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb468794-b64e-4215-9c15-a1b5253c3e89 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:50.070213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cb468794-b64e-4215-9c15-a1b5253c3e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:50.111265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb468794-b64e-4215-9c15-a1b5253c3e89 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:50.193050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:50.194363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:50.194745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:50.195120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f admin admin] PATH_INFO: `/v3/projects/cfc2ad0194bf418b8ff85ca6fb0a0fc9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:50.197859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfc2ad0194bf418b8ff85ca6fb0a0fc9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:50.202054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:50.324281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfc2ad0194bf418b8ff85ca6fb0a0fc9', 'request_id': 'req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:50.336857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfc2ad0194bf418b8ff85ca6fb0a0fc9', 'request_id': 'req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:50.355685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfc2ad0194bf418b8ff85ca6fb0a0fc9', 'request_id': 'req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:50.373644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f admin admin] The token cache is being invalidate because project cfc2ad0194bf418b8ff85ca6fb0a0fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:04:50.378418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:50.382942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0a30bb80-b07d-4bed-b9b4-bd6f1c4ab46f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:50.392916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa51e4a-c116-45e1-a056-5d218bb692f5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:50.393202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa51e4a-c116-45e1-a056-5d218bb692f5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:50.393475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa51e4a-c116-45e1-a056-5d218bb692f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:50.396596 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1662/3352] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:04:50 2022] DELETE /identity/v3/projects/cfc2ad0194bf418b8ff85ca6fb0a0fc9 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:50.416365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6aa51e4a-c116-45e1-a056-5d218bb692f5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:50.417680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6aa51e4a-c116-45e1-a056-5d218bb692f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:50.550496 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1691/3353] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:50.560876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea351a-56d1-42cb-b437-19db21c24a61 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:50.561228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea351a-56d1-42cb-b437-19db21c24a61 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:50.561588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea351a-56d1-42cb-b437-19db21c24a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:50.595979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0cea351a-56d1-42cb-b437-19db21c24a61 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:50.597583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0cea351a-56d1-42cb-b437-19db21c24a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:50.623174 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1663/3354] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:50.627884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa51e4a-c116-45e1-a056-5d218bb692f5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:50.639737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa51e4a-c116-45e1-a056-5d218bb692f5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:50.665090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7cbf7-39f8-4506-af44-fa3d4c686f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:50.666526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7cbf7-39f8-4506-af44-fa3d4c686f4a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:50.667068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7cbf7-39f8-4506-af44-fa3d4c686f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:50.667517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7cbf7-39f8-4506-af44-fa3d4c686f4a admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:50.668328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7cbf7-39f8-4506-af44-fa3d4c686f4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:04:50.668808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7cbf7-39f8-4506-af44-fa3d4c686f4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:50.671162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7cbf7-39f8-4506-af44-fa3d4c686f4a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:50.686586 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1692/3355] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:04:50.694435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea351a-56d1-42cb-b437-19db21c24a61 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:50.707333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea351a-56d1-42cb-b437-19db21c24a61 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:50.713085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732a109-4ffe-4926-bdbe-2ff9bd429674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:50.714255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732a109-4ffe-4926-bdbe-2ff9bd429674 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:50.714533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732a109-4ffe-4926-bdbe-2ff9bd429674 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:50.714803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732a109-4ffe-4926-bdbe-2ff9bd429674 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:50.715426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732a109-4ffe-4926-bdbe-2ff9bd429674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:50.717713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732a109-4ffe-4926-bdbe-2ff9bd429674 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:50.789825 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1664/3356] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:04:50 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:04:50.790018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7cbf7-39f8-4506-af44-fa3d4c686f4a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:50.805995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7cbf7-39f8-4506-af44-fa3d4c686f4a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:51.026208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247887cf-22a8-45b2-adb6-c642cfaafa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:51.027231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247887cf-22a8-45b2-adb6-c642cfaafa8d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:51.027694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247887cf-22a8-45b2-adb6-c642cfaafa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:51.028155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247887cf-22a8-45b2-adb6-c642cfaafa8d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:51.042672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247887cf-22a8-45b2-adb6-c642cfaafa8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:51.068676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247887cf-22a8-45b2-adb6-c642cfaafa8d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:51.144667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732a109-4ffe-4926-bdbe-2ff9bd429674 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:51.146228 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1693/3357] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:04:50 2022] POST /identity/v3/users => generated 284 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:04:51.159387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732a109-4ffe-4926-bdbe-2ff9bd429674 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:51.182637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916765c-6c6a-4392-bc8e-cfec5ee91d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:51.183529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916765c-6c6a-4392-bc8e-cfec5ee91d78 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:51.184426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916765c-6c6a-4392-bc8e-cfec5ee91d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:51.184426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916765c-6c6a-4392-bc8e-cfec5ee91d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:51.185080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916765c-6c6a-4392-bc8e-cfec5ee91d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:51.189027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916765c-6c6a-4392-bc8e-cfec5ee91d78 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:51.218607 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1665/3358] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:04:51.222373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247887cf-22a8-45b2-adb6-c642cfaafa8d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:51.254420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247887cf-22a8-45b2-adb6-c642cfaafa8d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:51.273378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba519c-d431-44a2-a440-a27408f24c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:51.274939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba519c-d431-44a2-a440-a27408f24c34 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:51.275321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba519c-d431-44a2-a440-a27408f24c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:51.275885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba519c-d431-44a2-a440-a27408f24c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:51.286848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba519c-d431-44a2-a440-a27408f24c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:51.288754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba519c-d431-44a2-a440-a27408f24c34 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:51.318698 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1694/3359] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:04:51.324424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916765c-6c6a-4392-bc8e-cfec5ee91d78 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:51.337862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916765c-6c6a-4392-bc8e-cfec5ee91d78 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:51.357553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353e8d8-ab22-41d6-bda4-2449cc5e3a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:51.358246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353e8d8-ab22-41d6-bda4-2449cc5e3a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:51.358525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353e8d8-ab22-41d6-bda4-2449cc5e3a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:51.358796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353e8d8-ab22-41d6-bda4-2449cc5e3a51 admin admin] PATH_INFO: `/v3/projects/8415ca2a42f64925b08aa22c86c50300/users/615c47a0e578418894061d9cfd3d5b3e/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:51.368967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba519c-d431-44a2-a440-a27408f24c34 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:51.370586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-15ba519c-d431-44a2-a440-a27408f24c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:51.376001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:51.381045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:51.418902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353e8d8-ab22-41d6-bda4-2449cc5e3a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8415ca2a42f64925b08aa22c86c50300, user_id=615c47a0e578418894061d9cfd3d5b3e, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:51.425094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353e8d8-ab22-41d6-bda4-2449cc5e3a51 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:51.467646 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1666/3360] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:04:51 2022] PUT /identity/v3/projects/8415ca2a42f64925b08aa22c86c50300/users/615c47a0e578418894061d9cfd3d5b3e/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:51.547387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353e8d8-ab22-41d6-bda4-2449cc5e3a51 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:51.594471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba519c-d431-44a2-a440-a27408f24c34 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:51.692927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353e8d8-ab22-41d6-bda4-2449cc5e3a51 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:51.700045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558ce7d-7b4c-4725-bcbc-3a1378de207c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:51.700791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558ce7d-7b4c-4725-bcbc-3a1378de207c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:51.701155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558ce7d-7b4c-4725-bcbc-3a1378de207c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:51.701493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558ce7d-7b4c-4725-bcbc-3a1378de207c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:51.703801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e558ce7d-7b4c-4725-bcbc-3a1378de207c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:51.735244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c761024-1821-4ab8-aef2-6a07a27d41d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:51.737268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c761024-1821-4ab8-aef2-6a07a27d41d6 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:51.739260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c761024-1821-4ab8-aef2-6a07a27d41d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:51.739745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c761024-1821-4ab8-aef2-6a07a27d41d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:51.745134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c761024-1821-4ab8-aef2-6a07a27d41d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:51.817130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558ce7d-7b4c-4725-bcbc-3a1378de207c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:51.819689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558ce7d-7b4c-4725-bcbc-3a1378de207c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:51.844927 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1667/3361] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:04:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:04:51.847525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1094d-60c8-4bc5-8dc0-e0e6ce6c31b1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:51.849806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1094d-60c8-4bc5-8dc0-e0e6ce6c31b1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:51.849806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1094d-60c8-4bc5-8dc0-e0e6ce6c31b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:51.935602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-72f1094d-60c8-4bc5-8dc0-e0e6ce6c31b1 None None] MFA Rules not processed for user `615c47a0e578418894061d9cfd3d5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:51.937473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-72f1094d-60c8-4bc5-8dc0-e0e6ce6c31b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:51.977180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c761024-1821-4ab8-aef2-6a07a27d41d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:51.981051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c761024-1821-4ab8-aef2-6a07a27d41d6 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:52.011369 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1695/3362] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:04:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:04:52.105915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c761024-1821-4ab8-aef2-6a07a27d41d6 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:52.113338 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1668/3363] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:52.275729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c761024-1821-4ab8-aef2-6a07a27d41d6 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:52.282289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b1c45-6f3d-4767-b955-099d6f5d8241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:52.283570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b1c45-6f3d-4767-b955-099d6f5d8241 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:52.284086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b1c45-6f3d-4767-b955-099d6f5d8241 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:52.284573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b1c45-6f3d-4767-b955-099d6f5d8241 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:52.285400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b1c45-6f3d-4767-b955-099d6f5d8241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:52.288105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b1c45-6f3d-4767-b955-099d6f5d8241 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:52.295428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:52.352655 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1696/3364] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:04:52 2022] POST /identity/v3/projects => generated 369 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:04:52.366836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1094d-60c8-4bc5-8dc0-e0e6ce6c31b1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:52.395570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1094d-60c8-4bc5-8dc0-e0e6ce6c31b1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:52.453199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:52.576601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b453b-b9d9-4afc-8513-2cfff505f4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:52.577442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b453b-b9d9-4afc-8513-2cfff505f4fc admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:52.578098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b453b-b9d9-4afc-8513-2cfff505f4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:52.578662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b453b-b9d9-4afc-8513-2cfff505f4fc admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:52.579989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b453b-b9d9-4afc-8513-2cfff505f4fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:52.582123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b453b-b9d9-4afc-8513-2cfff505f4fc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:52.680835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b1c45-6f3d-4767-b955-099d6f5d8241 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:52.681176 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1669/3365] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:04:52 2022] POST /identity/v3/users => generated 291 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:04:52.698306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b1c45-6f3d-4767-b955-099d6f5d8241 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:52.714450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed8bce-8c75-40f7-b713-614df7a3f1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:52.715260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed8bce-8c75-40f7-b713-614df7a3f1fe admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:52.715595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed8bce-8c75-40f7-b713-614df7a3f1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:52.715934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed8bce-8c75-40f7-b713-614df7a3f1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:52.716667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed8bce-8c75-40f7-b713-614df7a3f1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:52.721554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed8bce-8c75-40f7-b713-614df7a3f1fe admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:52.746918 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1697/3366] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:04:52.751574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b453b-b9d9-4afc-8513-2cfff505f4fc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:52.767234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b453b-b9d9-4afc-8513-2cfff505f4fc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:52.778751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bdedc-ecae-441a-984c-aa32026e57b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:52.779965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bdedc-ecae-441a-984c-aa32026e57b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:52.781784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bdedc-ecae-441a-984c-aa32026e57b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:52.781784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bdedc-ecae-441a-984c-aa32026e57b8 admin admin] PATH_INFO: `/v3/projects/ae946db7ceac4d608cfbb8c8e99efb7b/users/8fd1ba5a738e4c64861097a06a1dbd44/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:52.843691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bdedc-ecae-441a-984c-aa32026e57b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae946db7ceac4d608cfbb8c8e99efb7b, user_id=8fd1ba5a738e4c64861097a06a1dbd44, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:52.851370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bdedc-ecae-441a-984c-aa32026e57b8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:52.883270 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1670/3367] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:04:52 2022] PUT /identity/v3/projects/ae946db7ceac4d608cfbb8c8e99efb7b/users/8fd1ba5a738e4c64861097a06a1dbd44/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:52.887913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed8bce-8c75-40f7-b713-614df7a3f1fe admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:52.905388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed8bce-8c75-40f7-b713-614df7a3f1fe admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:53.030203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23ce15-b95d-4610-99ad-4d6246691216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:53.030723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23ce15-b95d-4610-99ad-4d6246691216 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:53.030990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23ce15-b95d-4610-99ad-4d6246691216 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:53.031251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23ce15-b95d-4610-99ad-4d6246691216 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:53.031888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23ce15-b95d-4610-99ad-4d6246691216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:53.033709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23ce15-b95d-4610-99ad-4d6246691216 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:53.051582 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1698/3368] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:04:53.055710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bdedc-ecae-441a-984c-aa32026e57b8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:53.078598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bdedc-ecae-441a-984c-aa32026e57b8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:53.096039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c28bb9-3aa4-424c-8c14-c181a2e8ca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:53.097003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c28bb9-3aa4-424c-8c14-c181a2e8ca06 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:53.097274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c28bb9-3aa4-424c-8c14-c181a2e8ca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:53.097535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c28bb9-3aa4-424c-8c14-c181a2e8ca06 admin admin] PATH_INFO: `/v3/projects/ae946db7ceac4d608cfbb8c8e99efb7b/users/8fd1ba5a738e4c64861097a06a1dbd44/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:53.099963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c28bb9-3aa4-424c-8c14-c181a2e8ca06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae946db7ceac4d608cfbb8c8e99efb7b, user_id=8fd1ba5a738e4c64861097a06a1dbd44, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:53.103033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c28bb9-3aa4-424c-8c14-c181a2e8ca06 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:53.116855 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1671/3369] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:04:53 2022] PUT /identity/v3/projects/ae946db7ceac4d608cfbb8c8e99efb7b/users/8fd1ba5a738e4c64861097a06a1dbd44/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:53.127438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b97fd-7584-4f72-a28a-b0f8600c46d0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:53.127915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b97fd-7584-4f72-a28a-b0f8600c46d0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:53.128357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b97fd-7584-4f72-a28a-b0f8600c46d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:53.163397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-916b97fd-7584-4f72-a28a-b0f8600c46d0 None None] MFA Rules not processed for user `8fd1ba5a738e4c64861097a06a1dbd44`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:53.165484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-916b97fd-7584-4f72-a28a-b0f8600c46d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:53.298417 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1699/3370] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:53.304640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c28bb9-3aa4-424c-8c14-c181a2e8ca06 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:53.322594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c28bb9-3aa4-424c-8c14-c181a2e8ca06 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:53.335057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba7109-9e2e-4672-9251-e4fd9c3012c6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd1ba5a738e4c64861097a06a1dbd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:53.339756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba7109-9e2e-4672-9251-e4fd9c3012c6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:53.340341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba7109-9e2e-4672-9251-e4fd9c3012c6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:53.340846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba7109-9e2e-4672-9251-e4fd9c3012c6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] PATH_INFO: `/v3/policies` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:53.342172 ncdv-master-1 devstack@keystone.service[104119]: WARNING oslo_log.versionutils [None req-8cba7109-9e2e-4672-9251-e4fd9c3012c6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Aug 20 21:04:53.345845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba7109-9e2e-4672-9251-e4fd9c3012c6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:53.347402 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-8cba7109-9e2e-4672-9251-e4fd9c3012c6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:04:53.347402 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:04:53.347402 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:04:53.348833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba7109-9e2e-4672-9251-e4fd9c3012c6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:53.408022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b97fd-7584-4f72-a28a-b0f8600c46d0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:53.408150 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1672/3371] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:04:53 2022] POST /identity/v3/policies => generated 230 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:04:53.420294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b97fd-7584-4f72-a28a-b0f8600c46d0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:53.434427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b1236-52fe-4d4c-9b05-23e2635599f6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd1ba5a738e4c64861097a06a1dbd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:53.435339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1236-52fe-4d4c-9b05-23e2635599f6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:53.435806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1236-52fe-4d4c-9b05-23e2635599f6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:53.436254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1236-52fe-4d4c-9b05-23e2635599f6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] PATH_INFO: `/v3/policies/af053d44e6424524a34de9d5d87baedc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:53.437416 ncdv-master-1 devstack@keystone.service[104118]: WARNING oslo_log.versionutils [None req-309b1236-52fe-4d4c-9b05-23e2635599f6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Deprecated: identity:update_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Aug 20 21:04:53.439856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b1236-52fe-4d4c-9b05-23e2635599f6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorizing `identity:update_policy(policy_id=af053d44e6424524a34de9d5d87baedc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:53.443191 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-309b1236-52fe-4d4c-9b05-23e2635599f6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:04:53.443191 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:04:53.443191 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:04:53.448006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b1236-52fe-4d4c-9b05-23e2635599f6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:53.493288 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1700/3372] 10.222.0.23 () {60 vars in 1252 bytes} [Sat Aug 20 21:04:53 2022] PATCH /identity/v3/policies/af053d44e6424524a34de9d5d87baedc => generated 237 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:04:53.497070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba7109-9e2e-4672-9251-e4fd9c3012c6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:53.510708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba7109-9e2e-4672-9251-e4fd9c3012c6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:53.519614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843576f8-7c2c-470d-9d54-d8fcaa2885d2 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd1ba5a738e4c64861097a06a1dbd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:53.520411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843576f8-7c2c-470d-9d54-d8fcaa2885d2 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:53.520692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843576f8-7c2c-470d-9d54-d8fcaa2885d2 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:53.520958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843576f8-7c2c-470d-9d54-d8fcaa2885d2 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] PATH_INFO: `/v3/policies/af053d44e6424524a34de9d5d87baedc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:53.521755 ncdv-master-1 devstack@keystone.service[104119]: WARNING oslo_log.versionutils [None req-843576f8-7c2c-470d-9d54-d8fcaa2885d2 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Deprecated: identity:get_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Aug 20 21:04:53.522350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843576f8-7c2c-470d-9d54-d8fcaa2885d2 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorizing `identity:get_policy(policy_id=af053d44e6424524a34de9d5d87baedc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:53.523583 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-843576f8-7c2c-470d-9d54-d8fcaa2885d2 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_policy": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:04:53.523583 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:04:53.523583 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:04:53.524688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843576f8-7c2c-470d-9d54-d8fcaa2885d2 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:53.542319 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1673/3373] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:04:53 2022] GET /identity/v3/policies/af053d44e6424524a34de9d5d87baedc => generated 237 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:04:53.543722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b1236-52fe-4d4c-9b05-23e2635599f6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:53.565547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b1236-52fe-4d4c-9b05-23e2635599f6 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:53.578999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa012e-4d25-434e-9cad-b097dc1d2261 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd1ba5a738e4c64861097a06a1dbd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:53.579820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa012e-4d25-434e-9cad-b097dc1d2261 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:53.580159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa012e-4d25-434e-9cad-b097dc1d2261 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:53.580569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa012e-4d25-434e-9cad-b097dc1d2261 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] PATH_INFO: `/v3/policies/af053d44e6424524a34de9d5d87baedc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:53.581457 ncdv-master-1 devstack@keystone.service[104118]: WARNING oslo_log.versionutils [None req-85fa012e-4d25-434e-9cad-b097dc1d2261 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Aug 20 21:04:53.582126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa012e-4d25-434e-9cad-b097dc1d2261 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=af053d44e6424524a34de9d5d87baedc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:53.583872 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-85fa012e-4d25-434e-9cad-b097dc1d2261 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:04:53.583872 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:04:53.583872 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:04:53.584961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa012e-4d25-434e-9cad-b097dc1d2261 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:53.624408 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1701/3374] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:04:53 2022] DELETE /identity/v3/policies/af053d44e6424524a34de9d5d87baedc => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:53.624699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843576f8-7c2c-470d-9d54-d8fcaa2885d2 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:53.644223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843576f8-7c2c-470d-9d54-d8fcaa2885d2 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:53.652846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28005344-91e3-4e30-b258-6cedfc7a6b7c tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd1ba5a738e4c64861097a06a1dbd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:53.654072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28005344-91e3-4e30-b258-6cedfc7a6b7c tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:53.654848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28005344-91e3-4e30-b258-6cedfc7a6b7c tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:53.655128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28005344-91e3-4e30-b258-6cedfc7a6b7c tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] PATH_INFO: `/v3/policies` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:53.656352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28005344-91e3-4e30-b258-6cedfc7a6b7c tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:53.659645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28005344-91e3-4e30-b258-6cedfc7a6b7c tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:53.692298 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1674/3375] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:04:53 2022] POST /identity/v3/policies => generated 228 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:04:53.696659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa012e-4d25-434e-9cad-b097dc1d2261 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:53.715765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa012e-4d25-434e-9cad-b097dc1d2261 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:53.734506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f1e20-0947-4010-89a6-c14763a52488 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd1ba5a738e4c64861097a06a1dbd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:53.735153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f1e20-0947-4010-89a6-c14763a52488 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:53.735432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f1e20-0947-4010-89a6-c14763a52488 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:53.735702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f1e20-0947-4010-89a6-c14763a52488 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] PATH_INFO: `/v3/policies` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:53.736661 ncdv-master-1 devstack@keystone.service[104118]: WARNING oslo_log.versionutils [None req-bd8f1e20-0947-4010-89a6-c14763a52488 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Aug 20 21:04:53.737140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f1e20-0947-4010-89a6-c14763a52488 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:53.738582 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-bd8f1e20-0947-4010-89a6-c14763a52488 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:04:53.738582 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:04:53.738582 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:04:53.739481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f1e20-0947-4010-89a6-c14763a52488 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:53.775779 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1702/3376] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:04:53 2022] POST /identity/v3/policies => generated 230 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:04:53.780783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28005344-91e3-4e30-b258-6cedfc7a6b7c tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:53.794923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28005344-91e3-4e30-b258-6cedfc7a6b7c tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:53.808074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816712d-0b24-43da-95ef-b43b50e4c15a tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd1ba5a738e4c64861097a06a1dbd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:53.810003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816712d-0b24-43da-95ef-b43b50e4c15a tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:53.810552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816712d-0b24-43da-95ef-b43b50e4c15a tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:53.811162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816712d-0b24-43da-95ef-b43b50e4c15a tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] PATH_INFO: `/v3/policies` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:53.812775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816712d-0b24-43da-95ef-b43b50e4c15a tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:53.815571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816712d-0b24-43da-95ef-b43b50e4c15a tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:53.862269 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1675/3377] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:04:53 2022] POST /identity/v3/policies => generated 229 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:04:53.869146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f1e20-0947-4010-89a6-c14763a52488 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:53.885935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f1e20-0947-4010-89a6-c14763a52488 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:53.893258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d03ee9-afbf-4103-8a6e-f7be49d60671 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd1ba5a738e4c64861097a06a1dbd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:53.894268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d03ee9-afbf-4103-8a6e-f7be49d60671 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:53.895049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d03ee9-afbf-4103-8a6e-f7be49d60671 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:53.895573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d03ee9-afbf-4103-8a6e-f7be49d60671 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] PATH_INFO: `/v3/policies` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:53.896617 ncdv-master-1 devstack@keystone.service[104118]: WARNING oslo_log.versionutils [None req-17d03ee9-afbf-4103-8a6e-f7be49d60671 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Deprecated: identity:list_policies of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Aug 20 21:04:53.897364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d03ee9-afbf-4103-8a6e-f7be49d60671 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorizing `identity:list_policies()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:53.900345 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-17d03ee9-afbf-4103-8a6e-f7be49d60671 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_policies": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:04:53.900345 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:04:53.900345 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:04:53.902201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d03ee9-afbf-4103-8a6e-f7be49d60671 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:53.914617 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1703/3378] 10.222.0.23 () {58 vars in 1131 bytes} [Sat Aug 20 21:04:53 2022] GET /identity/v3/policies => generated 763 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:04:53.918265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816712d-0b24-43da-95ef-b43b50e4c15a tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:53.931198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816712d-0b24-43da-95ef-b43b50e4c15a tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:53.936363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c72dbf-f3eb-4ec4-a99a-6dc5e875b1de tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd1ba5a738e4c64861097a06a1dbd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:53.937013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c72dbf-f3eb-4ec4-a99a-6dc5e875b1de tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:53.937313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c72dbf-f3eb-4ec4-a99a-6dc5e875b1de tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:53.937552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c72dbf-f3eb-4ec4-a99a-6dc5e875b1de tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] PATH_INFO: `/v3/policies/72bd81f4660f41b8a18ea2f4b8b12cdd` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:53.938386 ncdv-master-1 devstack@keystone.service[104119]: WARNING oslo_log.versionutils [None req-38c72dbf-f3eb-4ec4-a99a-6dc5e875b1de tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Aug 20 21:04:53.938940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c72dbf-f3eb-4ec4-a99a-6dc5e875b1de tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=72bd81f4660f41b8a18ea2f4b8b12cdd)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:53.940225 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-38c72dbf-f3eb-4ec4-a99a-6dc5e875b1de tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:04:53.940225 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:04:53.940225 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:04:53.941168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c72dbf-f3eb-4ec4-a99a-6dc5e875b1de tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:53.967602 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1676/3379] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:04:53 2022] DELETE /identity/v3/policies/72bd81f4660f41b8a18ea2f4b8b12cdd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:53.969589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d03ee9-afbf-4103-8a6e-f7be49d60671 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:53.983270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:53.999099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d03ee9-afbf-4103-8a6e-f7be49d60671 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:54.017228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520e2b6-6778-495c-8964-6cb34bca0ea3 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd1ba5a738e4c64861097a06a1dbd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:54.020363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520e2b6-6778-495c-8964-6cb34bca0ea3 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:54.020363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520e2b6-6778-495c-8964-6cb34bca0ea3 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:54.020363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520e2b6-6778-495c-8964-6cb34bca0ea3 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] PATH_INFO: `/v3/policies/8cc7e9249f7841d99b1e128a78e61439` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:54.020363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520e2b6-6778-495c-8964-6cb34bca0ea3 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=8cc7e9249f7841d99b1e128a78e61439)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:54.030872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520e2b6-6778-495c-8964-6cb34bca0ea3 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:54.067419 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1704/3380] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:04:53 2022] DELETE /identity/v3/policies/8cc7e9249f7841d99b1e128a78e61439 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:54.077041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c72dbf-f3eb-4ec4-a99a-6dc5e875b1de tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:54.091606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c72dbf-f3eb-4ec4-a99a-6dc5e875b1de tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:54.103171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb81fc5-9ef3-48b7-87ae-e9c0ffc0baeb tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd1ba5a738e4c64861097a06a1dbd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:54.105097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb81fc5-9ef3-48b7-87ae-e9c0ffc0baeb tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:54.105949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb81fc5-9ef3-48b7-87ae-e9c0ffc0baeb tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:54.107437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb81fc5-9ef3-48b7-87ae-e9c0ffc0baeb tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] PATH_INFO: `/v3/policies/2e47599a78f24fe3a56fc046353d9fa7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:54.109254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb81fc5-9ef3-48b7-87ae-e9c0ffc0baeb tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=2e47599a78f24fe3a56fc046353d9fa7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:54.111713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb81fc5-9ef3-48b7-87ae-e9c0ffc0baeb tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:54.154099 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1677/3381] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:04:54 2022] DELETE /identity/v3/policies/2e47599a78f24fe3a56fc046353d9fa7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:54.155460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520e2b6-6778-495c-8964-6cb34bca0ea3 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:54.323658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520e2b6-6778-495c-8964-6cb34bca0ea3 tempest-PoliciesTestJSON-1407540685 tempest-PoliciesTestJSON-1407540685-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:54.339174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:54.340190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:54.340335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:54.340779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] PATH_INFO: `/v3/users/615c47a0e578418894061d9cfd3d5b3e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:54.349306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=615c47a0e578418894061d9cfd3d5b3e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:54.352278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:54.468286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '615c47a0e578418894061d9cfd3d5b3e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:54.486122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '615c47a0e578418894061d9cfd3d5b3e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:54.497197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '615c47a0e578418894061d9cfd3d5b3e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:54.521034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '615c47a0e578418894061d9cfd3d5b3e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:54.528867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '615c47a0e578418894061d9cfd3d5b3e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:54.539096 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1705/3382] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:04:54 2022] DELETE /identity/v3/users/615c47a0e578418894061d9cfd3d5b3e => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:54.541599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40d86a-4a2d-49cf-9de0-255ae1ad44c4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:54.542095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40d86a-4a2d-49cf-9de0-255ae1ad44c4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:54.542095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40d86a-4a2d-49cf-9de0-255ae1ad44c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:54.585092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bb40d86a-4a2d-49cf-9de0-255ae1ad44c4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:54.586013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bb40d86a-4a2d-49cf-9de0-255ae1ad44c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:54.683890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:INSERT=8,keystone:DELETE=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:54.726535 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1678/3383] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:54.765786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:54.767486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:54.887372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde30367-dbc8-4416-9456-fa94ab1e7f5f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:54.901353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e94187-3ac6-4740-8a49-fc1962149400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:54.904010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e94187-3ac6-4740-8a49-fc1962149400 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:54.904551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e94187-3ac6-4740-8a49-fc1962149400 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:54.905027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e94187-3ac6-4740-8a49-fc1962149400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:54.930715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e94187-3ac6-4740-8a49-fc1962149400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:54.935838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e94187-3ac6-4740-8a49-fc1962149400 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:54.970501 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1706/3384] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:04:57.440523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40d86a-4a2d-49cf-9de0-255ae1ad44c4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:57.444825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bb40d86a-4a2d-49cf-9de0-255ae1ad44c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:57.494203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40d86a-4a2d-49cf-9de0-255ae1ad44c4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:57.543661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:57.552550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:57.555361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:57.556027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] PATH_INFO: `/v3/projects/8415ca2a42f64925b08aa22c86c50300` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:57.564817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] RBAC: Authorizing `identity:delete_project(project_id=8415ca2a42f64925b08aa22c86c50300)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:57.577445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:57.691883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8415ca2a42f64925b08aa22c86c50300', 'request_id': 'req-68356c3e-c5ca-4f56-850c-389c788614be'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:57.707703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8415ca2a42f64925b08aa22c86c50300', 'request_id': 'req-68356c3e-c5ca-4f56-850c-389c788614be'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:57.726127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8415ca2a42f64925b08aa22c86c50300', 'request_id': 'req-68356c3e-c5ca-4f56-850c-389c788614be'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:57.739781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] The token cache is being invalidate because project 8415ca2a42f64925b08aa22c86c50300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:04:57.740513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:57.741380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:57.746213 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1679/3385] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:04:57 2022] DELETE /identity/v3/projects/8415ca2a42f64925b08aa22c86c50300 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:57.747424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e94187-3ac6-4740-8a49-fc1962149400 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:57.758643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e94187-3ac6-4740-8a49-fc1962149400 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:57.791984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:57.971427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0db8e-032c-405e-b286-c79edfa8cdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:57.972230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0db8e-032c-405e-b286-c79edfa8cdc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:57.972539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0db8e-032c-405e-b286-c79edfa8cdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:57.972812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0db8e-032c-405e-b286-c79edfa8cdc6 admin admin] PATH_INFO: `/v3/users/8fd1ba5a738e4c64861097a06a1dbd44` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:57.982476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0db8e-032c-405e-b286-c79edfa8cdc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd1ba5a738e4c64861097a06a1dbd44)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:57.996491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0db8e-032c-405e-b286-c79edfa8cdc6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:58.144649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6bf0db8e-032c-405e-b286-c79edfa8cdc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd1ba5a738e4c64861097a06a1dbd44'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:58.166115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6bf0db8e-032c-405e-b286-c79edfa8cdc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd1ba5a738e4c64861097a06a1dbd44'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:58.181296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6bf0db8e-032c-405e-b286-c79edfa8cdc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd1ba5a738e4c64861097a06a1dbd44'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:58.312757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6bf0db8e-032c-405e-b286-c79edfa8cdc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd1ba5a738e4c64861097a06a1dbd44'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:58.313723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6bf0db8e-032c-405e-b286-c79edfa8cdc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd1ba5a738e4c64861097a06a1dbd44'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:58.370241 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1707/3386] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:04:57 2022] DELETE /identity/v3/users/8fd1ba5a738e4c64861097a06a1dbd44 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:59.440148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:59.442392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:59.457114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68356c3e-c5ca-4f56-850c-389c788614be admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:59.557192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478867bb-235d-4df6-a197-0b1b87de1cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:59.557913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478867bb-235d-4df6-a197-0b1b87de1cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:59.558255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478867bb-235d-4df6-a197-0b1b87de1cae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:59.558593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478867bb-235d-4df6-a197-0b1b87de1cae admin admin] PATH_INFO: `/v3/projects/ae946db7ceac4d608cfbb8c8e99efb7b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:59.559966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478867bb-235d-4df6-a197-0b1b87de1cae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae946db7ceac4d608cfbb8c8e99efb7b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:59.561961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478867bb-235d-4df6-a197-0b1b87de1cae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:59.630708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-478867bb-235d-4df6-a197-0b1b87de1cae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'request_id': 'req-478867bb-235d-4df6-a197-0b1b87de1cae'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:59.645517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-478867bb-235d-4df6-a197-0b1b87de1cae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'request_id': 'req-478867bb-235d-4df6-a197-0b1b87de1cae'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:59.662222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-478867bb-235d-4df6-a197-0b1b87de1cae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae946db7ceac4d608cfbb8c8e99efb7b', 'request_id': 'req-478867bb-235d-4df6-a197-0b1b87de1cae'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:59.677881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:04:59.684165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-478867bb-235d-4df6-a197-0b1b87de1cae admin admin] The token cache is being invalidate because project ae946db7ceac4d608cfbb8c8e99efb7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:04:59.684608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-478867bb-235d-4df6-a197-0b1b87de1cae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:59.685436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-478867bb-235d-4df6-a197-0b1b87de1cae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:04:59.691626 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1680/3387] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:04:59 2022] DELETE /identity/v3/projects/ae946db7ceac4d608cfbb8c8e99efb7b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:04:59.692920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20568cd3-c54c-4777-8bf4-efe7b4a02909 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:59.693752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20568cd3-c54c-4777-8bf4-efe7b4a02909 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:59.694361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20568cd3-c54c-4777-8bf4-efe7b4a02909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:59.715609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-20568cd3-c54c-4777-8bf4-efe7b4a02909 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:59.717140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-20568cd3-c54c-4777-8bf4-efe7b4a02909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:59.779666 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1708/3388] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:04:59.787417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433a60d-f1c3-4944-a0ff-a42cd290ac75 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:59.787902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433a60d-f1c3-4944-a0ff-a42cd290ac75 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:59.788259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433a60d-f1c3-4944-a0ff-a42cd290ac75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:59.812654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5433a60d-f1c3-4944-a0ff-a42cd290ac75 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:04:59.814005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5433a60d-f1c3-4944-a0ff-a42cd290ac75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:04:59.827659 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1681/3389] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:04:59.828052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20568cd3-c54c-4777-8bf4-efe7b4a02909 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:59.841295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20568cd3-c54c-4777-8bf4-efe7b4a02909 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:59.852800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f30031-aa03-45b3-8d46-379bddda4f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:59.853530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f30031-aa03-45b3-8d46-379bddda4f9c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:59.853918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f30031-aa03-45b3-8d46-379bddda4f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:59.854282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f30031-aa03-45b3-8d46-379bddda4f9c admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:59.855041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f30031-aa03-45b3-8d46-379bddda4f9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:04:59.855438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f30031-aa03-45b3-8d46-379bddda4f9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:59.857218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f30031-aa03-45b3-8d46-379bddda4f9c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:59.874400 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1709/3390] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:04:59.881514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433a60d-f1c3-4944-a0ff-a42cd290ac75 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:59.892456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433a60d-f1c3-4944-a0ff-a42cd290ac75 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:59.897555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f81f4b-da8b-416d-9087-264ad67e437e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:04:59.898320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f81f4b-da8b-416d-9087-264ad67e437e admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:04:59.898695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f81f4b-da8b-416d-9087-264ad67e437e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:04:59.899037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f81f4b-da8b-416d-9087-264ad67e437e admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:04:59.899689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f81f4b-da8b-416d-9087-264ad67e437e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:04:59.901860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f81f4b-da8b-416d-9087-264ad67e437e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:04:59.940413 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1682/3391] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:04:59 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:04:59.942553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f30031-aa03-45b3-8d46-379bddda4f9c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:04:59.953713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f30031-aa03-45b3-8d46-379bddda4f9c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:04:59.967943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:00.023171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd91004-671d-45f6-b7de-b49a23f1c1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:00.024067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd91004-671d-45f6-b7de-b49a23f1c1ed admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:00.024483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd91004-671d-45f6-b7de-b49a23f1c1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:00.024893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd91004-671d-45f6-b7de-b49a23f1c1ed admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:00.025503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd91004-671d-45f6-b7de-b49a23f1c1ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:00.027417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd91004-671d-45f6-b7de-b49a23f1c1ed admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:00.052023 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1710/3392] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:04:59 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:00.056316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f81f4b-da8b-416d-9087-264ad67e437e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:00.067309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f81f4b-da8b-416d-9087-264ad67e437e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:00.072690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1105-cd67-4e88-997e-c1eb63cd85ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:00.073399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1105-cd67-4e88-997e-c1eb63cd85ae admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:00.073688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1105-cd67-4e88-997e-c1eb63cd85ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:00.074106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1105-cd67-4e88-997e-c1eb63cd85ae admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:00.074789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff1105-cd67-4e88-997e-c1eb63cd85ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:00.076410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff1105-cd67-4e88-997e-c1eb63cd85ae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:00.090031 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1683/3393] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:00.091213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd91004-671d-45f6-b7de-b49a23f1c1ed admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:00.101629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd91004-671d-45f6-b7de-b49a23f1c1ed admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:00.106734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225fe79-6bff-419b-a542-c728c37f6986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:00.107346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225fe79-6bff-419b-a542-c728c37f6986 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:00.107634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225fe79-6bff-419b-a542-c728c37f6986 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:00.107896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225fe79-6bff-419b-a542-c728c37f6986 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:00.108565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225fe79-6bff-419b-a542-c728c37f6986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:00.110269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225fe79-6bff-419b-a542-c728c37f6986 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:00.121428 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1711/3394] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:05:00.127711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1105-cd67-4e88-997e-c1eb63cd85ae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:00.139356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1105-cd67-4e88-997e-c1eb63cd85ae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:00.144840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c9c0d-dda3-4dcb-a122-da9086cc26db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:00.145505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c9c0d-dda3-4dcb-a122-da9086cc26db admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:00.145772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c9c0d-dda3-4dcb-a122-da9086cc26db admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:00.146032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c9c0d-dda3-4dcb-a122-da9086cc26db admin admin] PATH_INFO: `/v3/projects/ce260c2340f84129a95a2435c3605b18/users/5ffa280e0b8e4141828fa7e306ce5ff1/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:00.162626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c9c0d-dda3-4dcb-a122-da9086cc26db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce260c2340f84129a95a2435c3605b18, user_id=5ffa280e0b8e4141828fa7e306ce5ff1, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:00.165673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c9c0d-dda3-4dcb-a122-da9086cc26db admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:00.181425 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1684/3395] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:00 2022] PUT /identity/v3/projects/ce260c2340f84129a95a2435c3605b18/users/5ffa280e0b8e4141828fa7e306ce5ff1/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:00.191933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa22473-6d04-4a8a-b828-127107b99d34 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:00.192245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa22473-6d04-4a8a-b828-127107b99d34 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:00.192547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa22473-6d04-4a8a-b828-127107b99d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:00.222423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-baa22473-6d04-4a8a-b828-127107b99d34 None None] MFA Rules not processed for user `5ffa280e0b8e4141828fa7e306ce5ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:00.223788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-baa22473-6d04-4a8a-b828-127107b99d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:00.302140 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1712/3396] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:00.308937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c9c0d-dda3-4dcb-a122-da9086cc26db admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:00.380359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c9c0d-dda3-4dcb-a122-da9086cc26db admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:00.385904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb5305-6c66-4253-9f42-be4640b8d816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:00.386507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb5305-6c66-4253-9f42-be4640b8d816 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:00.386781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb5305-6c66-4253-9f42-be4640b8d816 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:00.387045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb5305-6c66-4253-9f42-be4640b8d816 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:00.387679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb5305-6c66-4253-9f42-be4640b8d816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:00.389622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb5305-6c66-4253-9f42-be4640b8d816 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:00.422078 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1685/3397] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:00 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:00.428687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa22473-6d04-4a8a-b828-127107b99d34 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:00.440785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa22473-6d04-4a8a-b828-127107b99d34 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:00.499339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a9a3b-270e-4aaa-bf8a-4c62720e465c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:00.499970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a9a3b-270e-4aaa-bf8a-4c62720e465c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:00.500261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a9a3b-270e-4aaa-bf8a-4c62720e465c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:00.500579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a9a3b-270e-4aaa-bf8a-4c62720e465c admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:00.501172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a9a3b-270e-4aaa-bf8a-4c62720e465c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:00.503111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a9a3b-270e-4aaa-bf8a-4c62720e465c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:00.531802 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1713/3398] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:00 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:00.533121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb5305-6c66-4253-9f42-be4640b8d816 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:00.544252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb5305-6c66-4253-9f42-be4640b8d816 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:00.549493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c1e7d-1d09-4803-b6de-c82ff54e8474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:00.550085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1e7d-1d09-4803-b6de-c82ff54e8474 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:00.550346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1e7d-1d09-4803-b6de-c82ff54e8474 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:00.550632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1e7d-1d09-4803-b6de-c82ff54e8474 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:00.551289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c1e7d-1d09-4803-b6de-c82ff54e8474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:00.552915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c1e7d-1d09-4803-b6de-c82ff54e8474 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:00.564310 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1686/3399] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:00.568607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a9a3b-270e-4aaa-bf8a-4c62720e465c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:00.579288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a9a3b-270e-4aaa-bf8a-4c62720e465c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:00.584073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47a994-d3bf-4c35-8e2c-7fd10b0e727d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:00.584901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47a994-d3bf-4c35-8e2c-7fd10b0e727d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:00.585177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47a994-d3bf-4c35-8e2c-7fd10b0e727d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:00.585454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47a994-d3bf-4c35-8e2c-7fd10b0e727d admin admin] PATH_INFO: `/v3/projects/7d06965396174bd0922ebceeb387cfbe/users/99481615eafb4cbba88546ecb09a3020/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:00.596888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:00.601307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47a994-d3bf-4c35-8e2c-7fd10b0e727d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d06965396174bd0922ebceeb387cfbe, user_id=99481615eafb4cbba88546ecb09a3020, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:00.604973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47a994-d3bf-4c35-8e2c-7fd10b0e727d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:00.619865 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1714/3400] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:00 2022] PUT /identity/v3/projects/7d06965396174bd0922ebceeb387cfbe/users/99481615eafb4cbba88546ecb09a3020/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:00.624001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c1e7d-1d09-4803-b6de-c82ff54e8474 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:00.635334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c1e7d-1d09-4803-b6de-c82ff54e8474 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:00.700984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13297557-7f19-4e83-ba69-5fbff23d32f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:00.701364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13297557-7f19-4e83-ba69-5fbff23d32f5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:00.701642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13297557-7f19-4e83-ba69-5fbff23d32f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:00.701907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13297557-7f19-4e83-ba69-5fbff23d32f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:00.702772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13297557-7f19-4e83-ba69-5fbff23d32f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:00.705126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13297557-7f19-4e83-ba69-5fbff23d32f5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:00.716797 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1687/3401] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:00.720871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47a994-d3bf-4c35-8e2c-7fd10b0e727d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:00.732682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47a994-d3bf-4c35-8e2c-7fd10b0e727d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:00.743922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459f548-85c3-460c-89f5-dd365049a585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:00.744725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459f548-85c3-460c-89f5-dd365049a585 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:00.744993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459f548-85c3-460c-89f5-dd365049a585 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:00.745278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459f548-85c3-460c-89f5-dd365049a585 admin admin] PATH_INFO: `/v3/projects/7d06965396174bd0922ebceeb387cfbe/users/99481615eafb4cbba88546ecb09a3020/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:00.747612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459f548-85c3-460c-89f5-dd365049a585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d06965396174bd0922ebceeb387cfbe, user_id=99481615eafb4cbba88546ecb09a3020, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:00.750430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459f548-85c3-460c-89f5-dd365049a585 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:00.762412 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1715/3402] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:00 2022] PUT /identity/v3/projects/7d06965396174bd0922ebceeb387cfbe/users/99481615eafb4cbba88546ecb09a3020/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:00.773039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53880f5a-71c3-41e5-9366-1e4276ca9326 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:00.773422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53880f5a-71c3-41e5-9366-1e4276ca9326 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:00.773876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53880f5a-71c3-41e5-9366-1e4276ca9326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:00.790774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:00.807323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-53880f5a-71c3-41e5-9366-1e4276ca9326 None None] MFA Rules not processed for user `99481615eafb4cbba88546ecb09a3020`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:00.808667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-53880f5a-71c3-41e5-9366-1e4276ca9326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:00.888073 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1688/3403] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:00.893139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459f548-85c3-460c-89f5-dd365049a585 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:00.907572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459f548-85c3-460c-89f5-dd365049a585 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:00.916548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc942d27-5942-443f-92fa-29ef80609a7d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:00.917548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc942d27-5942-443f-92fa-29ef80609a7d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:00.917992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc942d27-5942-443f-92fa-29ef80609a7d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:00.918416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc942d27-5942-443f-92fa-29ef80609a7d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:00.919309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc942d27-5942-443f-92fa-29ef80609a7d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:00.931807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc942d27-5942-443f-92fa-29ef80609a7d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:00.981249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53880f5a-71c3-41e5-9366-1e4276ca9326 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:00.988338 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1716/3404] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:05:00 2022] POST /identity/v3/projects => generated 329 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:00.995885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53880f5a-71c3-41e5-9366-1e4276ca9326 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:01.085022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c5425-955c-43df-b87e-03d21345847c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:01.085593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c5425-955c-43df-b87e-03d21345847c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:01.085846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c5425-955c-43df-b87e-03d21345847c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:01.086411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c5425-955c-43df-b87e-03d21345847c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:01.087391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c5425-955c-43df-b87e-03d21345847c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:01.089689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c5425-955c-43df-b87e-03d21345847c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:01.141716 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1689/3405] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:00 2022] POST /identity/v3/users => generated 416 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:01.146475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc942d27-5942-443f-92fa-29ef80609a7d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:01.151369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c2cc9-149d-4f0e-a586-2c4ff88e46f5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:01.151659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c2cc9-149d-4f0e-a586-2c4ff88e46f5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:01.151891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c2cc9-149d-4f0e-a586-2c4ff88e46f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:01.159828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc942d27-5942-443f-92fa-29ef80609a7d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:01.169010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b9cd9-b9f3-4aaf-ad62-fe86998845bb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:01.170897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b9cd9-b9f3-4aaf-ad62-fe86998845bb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:01.171373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b9cd9-b9f3-4aaf-ad62-fe86998845bb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:01.172152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b9cd9-b9f3-4aaf-ad62-fe86998845bb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:01.173067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b9cd9-b9f3-4aaf-ad62-fe86998845bb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:01.174829 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-d44b9cd9-b9f3-4aaf-ad62-fe86998845bb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:05:01.174829 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:05:01.174829 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:05:01.178968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b9cd9-b9f3-4aaf-ad62-fe86998845bb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:01.205726 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1717/3406] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:05:01 2022] POST /identity/v3/roles => generated 243 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:01.213801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b9cd9-b9f3-4aaf-ad62-fe86998845bb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:01.222033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-399c2cc9-149d-4f0e-a586-2c4ff88e46f5 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:01.227129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-399c2cc9-149d-4f0e-a586-2c4ff88e46f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:01.242144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b9cd9-b9f3-4aaf-ad62-fe86998845bb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:01.251098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd729672-5ced-42b2-a1a4-da1e2403c425 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:01.251704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd729672-5ced-42b2-a1a4-da1e2403c425 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:01.251982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd729672-5ced-42b2-a1a4-da1e2403c425 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:01.252257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd729672-5ced-42b2-a1a4-da1e2403c425 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:01.253285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd729672-5ced-42b2-a1a4-da1e2403c425 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:01.254944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd729672-5ced-42b2-a1a4-da1e2403c425 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:01.280776 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1718/3407] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:05:01 2022] POST /identity/v3/roles => generated 246 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:01.288842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd729672-5ced-42b2-a1a4-da1e2403c425 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:01.304494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd729672-5ced-42b2-a1a4-da1e2403c425 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:01.310081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd52504-bd81-4da2-aad7-e7bd4889fcc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:01.310883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd52504-bd81-4da2-aad7-e7bd4889fcc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:01.311251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd52504-bd81-4da2-aad7-e7bd4889fcc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:01.311617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd52504-bd81-4da2-aad7-e7bd4889fcc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/4cabc095076a465ba1babf2966c87248/users/90f075f25d824395966597209a44a26e/roles/2f742ee0054e4427af8b04aaeab25c63` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:01.339067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd52504-bd81-4da2-aad7-e7bd4889fcc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_grant(project_id=4cabc095076a465ba1babf2966c87248, user_id=90f075f25d824395966597209a44a26e, role_id=2f742ee0054e4427af8b04aaeab25c63)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:01.342114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd52504-bd81-4da2-aad7-e7bd4889fcc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:01.345719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:01.366279 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1690/3409] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:01.366675 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1719/3409] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:01 2022] PUT /identity/v3/projects/4cabc095076a465ba1babf2966c87248/users/90f075f25d824395966597209a44a26e/roles/2f742ee0054e4427af8b04aaeab25c63 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:01.367643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c2cc9-149d-4f0e-a586-2c4ff88e46f5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:01.373279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd52504-bd81-4da2-aad7-e7bd4889fcc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:01.375223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6cd52504-bd81-4da2-aad7-e7bd4889fcc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:01.396466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c2cc9-149d-4f0e-a586-2c4ff88e46f5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:01.504506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a9c95-bb4c-4dd1-af40-a1e578be95e6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:01.504506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a9c95-bb4c-4dd1-af40-a1e578be95e6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:01.504506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a9c95-bb4c-4dd1-af40-a1e578be95e6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:01.504936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a9c95-bb4c-4dd1-af40-a1e578be95e6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/4cabc095076a465ba1babf2966c87248/users/90f075f25d824395966597209a44a26e/roles/b406cd8db8bf4826a35716473b0ea6f0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:01.504936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a9c95-bb4c-4dd1-af40-a1e578be95e6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_grant(project_id=4cabc095076a465ba1babf2966c87248, user_id=90f075f25d824395966597209a44a26e, role_id=b406cd8db8bf4826a35716473b0ea6f0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:01.505715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a9c95-bb4c-4dd1-af40-a1e578be95e6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:01.524256 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1691/3410] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:01 2022] PUT /identity/v3/projects/4cabc095076a465ba1babf2966c87248/users/90f075f25d824395966597209a44a26e/roles/b406cd8db8bf4826a35716473b0ea6f0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:01.528343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a9c95-bb4c-4dd1-af40-a1e578be95e6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:01.533724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd52504-bd81-4da2-aad7-e7bd4889fcc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:01.595149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:01.605904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a9c95-bb4c-4dd1-af40-a1e578be95e6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:01.611546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b36509-6da8-4d71-bbaa-c41345e139ea tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:01.612216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b36509-6da8-4d71-bbaa-c41345e139ea tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:01.612541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b36509-6da8-4d71-bbaa-c41345e139ea tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:01.613157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b36509-6da8-4d71-bbaa-c41345e139ea tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:01.614009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b36509-6da8-4d71-bbaa-c41345e139ea tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:01.615154 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-11b36509-6da8-4d71-bbaa-c41345e139ea tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:05:01.615154 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:05:01.615154 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:05:01.616235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b36509-6da8-4d71-bbaa-c41345e139ea tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:01.617094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93a11d-7c55-4999-bc4c-aeb23d306bba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:01.618368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93a11d-7c55-4999-bc4c-aeb23d306bba service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:01.618696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93a11d-7c55-4999-bc4c-aeb23d306bba service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:01.619020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93a11d-7c55-4999-bc4c-aeb23d306bba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:01.672289 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1692/3411] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:01 2022] GET /identity/v3/users => generated 8647 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:05:01.678410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39e916-f023-4493-9e4e-b96c126c0540 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:01.678694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39e916-f023-4493-9e4e-b96c126c0540 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:01.679165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39e916-f023-4493-9e4e-b96c126c0540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:01.697319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93a11d-7c55-4999-bc4c-aeb23d306bba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:01.699229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93a11d-7c55-4999-bc4c-aeb23d306bba service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:01.710314 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1720/3412] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:05:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:05:01.757096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9b39e916-f023-4493-9e4e-b96c126c0540 None None] MFA Rules not processed for user `90f075f25d824395966597209a44a26e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:01.761722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9b39e916-f023-4493-9e4e-b96c126c0540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:01.904638 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1693/3413] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:01.907257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28fb09-a8ad-4487-8b85-e2957cb18777 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:01.908210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28fb09-a8ad-4487-8b85-e2957cb18777 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:01.908676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28fb09-a8ad-4487-8b85-e2957cb18777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:01.937412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ea28fb09-a8ad-4487-8b85-e2957cb18777 None None] MFA Rules not processed for user `90f075f25d824395966597209a44a26e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:01.938494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ea28fb09-a8ad-4487-8b85-e2957cb18777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:01.963415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39e916-f023-4493-9e4e-b96c126c0540 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:01.971342 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1721/3414] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:01.978698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39e916-f023-4493-9e4e-b96c126c0540 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:01.988860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64518f4b-f08d-49fe-88ee-316686972bf9 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f075f25d824395966597209a44a26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cabc095076a465ba1babf2966c87248', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-793566250', 'tempest-DelegatedRole-563074194'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:01.990729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64518f4b-f08d-49fe-88ee-316686972bf9 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:01.991176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64518f4b-f08d-49fe-88ee-316686972bf9 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:01.991486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64518f4b-f08d-49fe-88ee-316686972bf9 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:01.992158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64518f4b-f08d-49fe-88ee-316686972bf9 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] RBAC: Authorizing `identity:create_trust()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:01.993718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64518f4b-f08d-49fe-88ee-316686972bf9 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:02.005747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:02.284964 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1694/3415] 10.222.0.23 () {60 vars in 1174 bytes} [Sat Aug 20 21:05:01 2022] POST /identity/v3/OS-TRUST/trusts => generated 833 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:02.289324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28fb09-a8ad-4487-8b85-e2957cb18777 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:02.313853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28fb09-a8ad-4487-8b85-e2957cb18777 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:02.329747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23afdc6-a87c-4644-a28f-855b6a86e5eb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:02.331095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23afdc6-a87c-4644-a28f-855b6a86e5eb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:02.331379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23afdc6-a87c-4644-a28f-855b6a86e5eb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:02.331649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23afdc6-a87c-4644-a28f-855b6a86e5eb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:02.332285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23afdc6-a87c-4644-a28f-855b6a86e5eb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:list_trusts()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:02.333355 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-f23afdc6-a87c-4644-a28f-855b6a86e5eb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_trusts": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:05:02.333355 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:05:02.333355 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:05:02.338548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23afdc6-a87c-4644-a28f-855b6a86e5eb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:02.359898 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1722/3416] 10.222.0.23 () {58 vars in 1155 bytes} [Sat Aug 20 21:05:02 2022] GET /identity/v3/OS-TRUST/trusts/ => generated 562 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:02.364599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64518f4b-f08d-49fe-88ee-316686972bf9 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:02.378136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64518f4b-f08d-49fe-88ee-316686972bf9 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:02.385328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e3cfc-0745-4e31-beac-7562c1669102 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f075f25d824395966597209a44a26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cabc095076a465ba1babf2966c87248', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-793566250', 'tempest-DelegatedRole-563074194'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:02.387122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e3cfc-0745-4e31-beac-7562c1669102 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:02.387607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e3cfc-0745-4e31-beac-7562c1669102 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:02.388074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e3cfc-0745-4e31-beac-7562c1669102 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] PATH_INFO: `/v3/OS-TRUST/trusts/01217f39934f41f18f9c5d31c82ff097` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:02.402770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e3cfc-0745-4e31-beac-7562c1669102 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] RBAC: Authorizing `identity:delete_trust(trust_id=01217f39934f41f18f9c5d31c82ff097)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:02.404781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e3cfc-0745-4e31-beac-7562c1669102 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:02.447949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4e9e3cfc-0745-4e31-beac-7562c1669102 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '01217f39934f41f18f9c5d31c82ff097', 'request_id': 'req-4e9e3cfc-0745-4e31-beac-7562c1669102'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:02.449715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4e9e3cfc-0745-4e31-beac-7562c1669102 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '01217f39934f41f18f9c5d31c82ff097', 'request_id': 'req-4e9e3cfc-0745-4e31-beac-7562c1669102'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:02.450429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4e9e3cfc-0745-4e31-beac-7562c1669102 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '01217f39934f41f18f9c5d31c82ff097', 'request_id': 'req-4e9e3cfc-0745-4e31-beac-7562c1669102'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:02.474278 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1695/3417] 10.222.0.23 () {58 vars in 1254 bytes} [Sat Aug 20 21:05:02 2022] DELETE /identity/v3/OS-TRUST/trusts/01217f39934f41f18f9c5d31c82ff097 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:02.489461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23afdc6-a87c-4644-a28f-855b6a86e5eb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:02.491183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f23afdc6-a87c-4644-a28f-855b6a86e5eb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:02.512431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23afdc6-a87c-4644-a28f-855b6a86e5eb tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:02.527110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1a3a2-2c51-47fa-9f43-51052cce0d0a tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f075f25d824395966597209a44a26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cabc095076a465ba1babf2966c87248', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-793566250', 'tempest-DelegatedRole-563074194'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:02.527781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1a3a2-2c51-47fa-9f43-51052cce0d0a tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:02.528416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1a3a2-2c51-47fa-9f43-51052cce0d0a tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:02.528700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1a3a2-2c51-47fa-9f43-51052cce0d0a tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] PATH_INFO: `/v3/OS-TRUST/trusts/01217f39934f41f18f9c5d31c82ff097` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-87f1a3a2-2c51-47fa-9f43-51052cce0d0a tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] Could not find trust: 01217f39934f41f18f9c5d31c82ff097.: keystone.exception.TrustNotFound: Could not find trust: 01217f39934f41f18f9c5d31c82ff097. Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Aug 20 21:05:02.567546 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Aug 20 21:05:02.568761 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Aug 20 21:05:02.568761 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Aug 20 21:05:02.568761 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Aug 20 21:05:02.568761 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Aug 20 21:05:02.568761 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Aug 20 21:05:02.568761 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:02.568761 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:02.568761 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Aug 20 21:05:02.568761 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Aug 20 21:05:02.568761 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Aug 20 21:05:02.568761 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Aug 20 21:05:02.568761 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 01217f39934f41f18f9c5d31c82ff097. Aug 20 21:05:02.568761 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:02.577498 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1723/3418] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:05:02 2022] GET /identity/v3/OS-TRUST/trusts/01217f39934f41f18f9c5d31c82ff097 => generated 111 bytes in 88 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:05:02.582625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e3cfc-0745-4e31-beac-7562c1669102 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:02.583280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4e9e3cfc-0745-4e31-beac-7562c1669102 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:02.600548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e3cfc-0745-4e31-beac-7562c1669102 tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:02.606344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c11682-d0cf-4381-82ce-41d061356db9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:02.606989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c11682-d0cf-4381-82ce-41d061356db9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:02.607261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c11682-d0cf-4381-82ce-41d061356db9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:02.607525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c11682-d0cf-4381-82ce-41d061356db9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles/b406cd8db8bf4826a35716473b0ea6f0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:02.609052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c11682-d0cf-4381-82ce-41d061356db9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_role(role_id=b406cd8db8bf4826a35716473b0ea6f0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:02.610155 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-62c11682-d0cf-4381-82ce-41d061356db9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:05:02.610155 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:05:02.610155 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:05:02.611050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c11682-d0cf-4381-82ce-41d061356db9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:02.669606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-62c11682-d0cf-4381-82ce-41d061356db9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invalidating the token cache because role b406cd8db8bf4826a35716473b0ea6f0 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:02.670655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-62c11682-d0cf-4381-82ce-41d061356db9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:02.671348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-62c11682-d0cf-4381-82ce-41d061356db9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:02.676287 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1696/3419] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:02 2022] DELETE /identity/v3/roles/b406cd8db8bf4826a35716473b0ea6f0 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:02.677785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1a3a2-2c51-47fa-9f43-51052cce0d0a tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:02.680385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-87f1a3a2-2c51-47fa-9f43-51052cce0d0a tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:02.708811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1a3a2-2c51-47fa-9f43-51052cce0d0a tempest-TrustsV3TestJSON-1419901824 tempest-user-1529183704] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:02.820128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97e853-8281-4bf8-9791-3965c32bb481 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:02.821590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97e853-8281-4bf8-9791-3965c32bb481 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:02.822176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97e853-8281-4bf8-9791-3965c32bb481 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:02.822681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97e853-8281-4bf8-9791-3965c32bb481 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles/2f742ee0054e4427af8b04aaeab25c63` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:02.827289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97e853-8281-4bf8-9791-3965c32bb481 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_role(role_id=2f742ee0054e4427af8b04aaeab25c63)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:02.832867 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-af97e853-8281-4bf8-9791-3965c32bb481 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:05:02.832867 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:05:02.832867 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:05:02.834742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97e853-8281-4bf8-9791-3965c32bb481 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:02.899495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-af97e853-8281-4bf8-9791-3965c32bb481 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invalidating the token cache because role 2f742ee0054e4427af8b04aaeab25c63 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:02.900656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-af97e853-8281-4bf8-9791-3965c32bb481 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:02.902303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-af97e853-8281-4bf8-9791-3965c32bb481 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:02.905465 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1724/3420] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:02 2022] DELETE /identity/v3/roles/2f742ee0054e4427af8b04aaeab25c63 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:02.909589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c11682-d0cf-4381-82ce-41d061356db9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:02.935539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c11682-d0cf-4381-82ce-41d061356db9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:03.034788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:03.035761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:03.036419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:03.037466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users/90f075f25d824395966597209a44a26e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:03.039433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_user(user_id=90f075f25d824395966597209a44a26e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:03.044317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:03.066315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:03.096625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839dd46-e985-43c1-9436-234a991d8063 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:03.097538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839dd46-e985-43c1-9436-234a991d8063 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:03.098020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839dd46-e985-43c1-9436-234a991d8063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:03.114536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90f075f25d824395966597209a44a26e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:03.126826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90f075f25d824395966597209a44a26e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:03.128771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4839dd46-e985-43c1-9436-234a991d8063 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:03.130240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4839dd46-e985-43c1-9436-234a991d8063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:03.142750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90f075f25d824395966597209a44a26e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:03.143481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90f075f25d824395966597209a44a26e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:03.144220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90f075f25d824395966597209a44a26e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:03.160126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:03.160455 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1697/3421] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:02 2022] DELETE /identity/v3/users/90f075f25d824395966597209a44a26e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:03.170075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:03.171614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:03.219966 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1725/3422] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:03.245522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839dd46-e985-43c1-9436-234a991d8063 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:03.247201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4839dd46-e985-43c1-9436-234a991d8063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:03.301219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07f755-6c8b-4042-a99c-ea4caccaf29a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:03.308089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:03.308982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:03.309253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:03.309515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/4cabc095076a465ba1babf2966c87248` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:03.311383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4cabc095076a465ba1babf2966c87248)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:03.314650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:03.376351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839dd46-e985-43c1-9436-234a991d8063 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:03.416086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cabc095076a465ba1babf2966c87248', 'request_id': 'req-b76c7d41-34c1-4be1-a44f-5415afff1e2b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:03.423777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cabc095076a465ba1babf2966c87248', 'request_id': 'req-b76c7d41-34c1-4be1-a44f-5415afff1e2b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:03.435953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cabc095076a465ba1babf2966c87248', 'request_id': 'req-b76c7d41-34c1-4be1-a44f-5415afff1e2b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:03.450642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] The token cache is being invalidate because project 4cabc095076a465ba1babf2966c87248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:03.450931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:03.451735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:03.453505 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1698/3423] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:03 2022] DELETE /identity/v3/projects/4cabc095076a465ba1babf2966c87248 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:03.459037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:03.460682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:03.475744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a285abf-f85a-4619-bcbe-39e85845e2c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:03.476399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a285abf-f85a-4619-bcbe-39e85845e2c2 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:03.476744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a285abf-f85a-4619-bcbe-39e85845e2c2 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:03.477032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a285abf-f85a-4619-bcbe-39e85845e2c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:03.479100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5a285abf-f85a-4619-bcbe-39e85845e2c2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:03.494674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:03.553320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c7d41-34c1-4be1-a44f-5415afff1e2b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:03.557903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a285abf-f85a-4619-bcbe-39e85845e2c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:03.559909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a285abf-f85a-4619-bcbe-39e85845e2c2 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:03.560088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815a311-8414-4d2f-b7a5-84a2c822fb06 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:03.561076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815a311-8414-4d2f-b7a5-84a2c822fb06 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:03.561503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815a311-8414-4d2f-b7a5-84a2c822fb06 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:03.562112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815a311-8414-4d2f-b7a5-84a2c822fb06 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:03.562778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815a311-8414-4d2f-b7a5-84a2c822fb06 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:03.565066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815a311-8414-4d2f-b7a5-84a2c822fb06 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:03.571014 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1726/3424] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:05:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:05:03.613387 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1699/3425] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:05:03 2022] POST /identity/v3/projects => generated 329 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:03.620194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a285abf-f85a-4619-bcbe-39e85845e2c2 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:03.646945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a285abf-f85a-4619-bcbe-39e85845e2c2 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:03.678681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:03.757418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b5923-94cb-4785-83cc-1c0d8dea9ddc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:03.758692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b5923-94cb-4785-83cc-1c0d8dea9ddc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:03.759042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b5923-94cb-4785-83cc-1c0d8dea9ddc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:03.759285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b5923-94cb-4785-83cc-1c0d8dea9ddc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:03.769394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b5923-94cb-4785-83cc-1c0d8dea9ddc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:03.771609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b5923-94cb-4785-83cc-1c0d8dea9ddc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:03.806069 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1727/3426] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:03 2022] POST /identity/v3/users => generated 416 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:03.810527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815a311-8414-4d2f-b7a5-84a2c822fb06 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:03.826698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815a311-8414-4d2f-b7a5-84a2c822fb06 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:03.838746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f3402-df4e-459d-a48c-61db06c1499a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:03.839402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f3402-df4e-459d-a48c-61db06c1499a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:03.839862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f3402-df4e-459d-a48c-61db06c1499a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:03.840141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f3402-df4e-459d-a48c-61db06c1499a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:03.840974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f3402-df4e-459d-a48c-61db06c1499a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:03.842240 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-546f3402-df4e-459d-a48c-61db06c1499a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:05:03.842240 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:05:03.842240 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:05:03.843213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f3402-df4e-459d-a48c-61db06c1499a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:03.873116 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1700/3427] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:05:03 2022] POST /identity/v3/roles => generated 243 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:03.874521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b5923-94cb-4785-83cc-1c0d8dea9ddc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:03.889573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b5923-94cb-4785-83cc-1c0d8dea9ddc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:03.895144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee869c3-edac-487e-90fb-3dbf410d95d8 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:03.895915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee869c3-edac-487e-90fb-3dbf410d95d8 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:03.896261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee869c3-edac-487e-90fb-3dbf410d95d8 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:03.896755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee869c3-edac-487e-90fb-3dbf410d95d8 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:03.897551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee869c3-edac-487e-90fb-3dbf410d95d8 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:03.899594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee869c3-edac-487e-90fb-3dbf410d95d8 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:03.932579 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1728/3428] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:05:03 2022] POST /identity/v3/roles => generated 247 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:03.936951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f3402-df4e-459d-a48c-61db06c1499a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:03.974141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f3402-df4e-459d-a48c-61db06c1499a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:03.984798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8d456-812f-47f3-b722-ee4dffd83530 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:03.986590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8d456-812f-47f3-b722-ee4dffd83530 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:03.987314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8d456-812f-47f3-b722-ee4dffd83530 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:03.987314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8d456-812f-47f3-b722-ee4dffd83530 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/0205e02728374f65b8c049cbe7f76e16/users/6801dcc2a64d4bc4acfd32b7e73c0513/roles/0fe2845ecc834c67a2280a8b2e92b4cc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:04.019710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8d456-812f-47f3-b722-ee4dffd83530 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_grant(project_id=0205e02728374f65b8c049cbe7f76e16, user_id=6801dcc2a64d4bc4acfd32b7e73c0513, role_id=0fe2845ecc834c67a2280a8b2e92b4cc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:04.023294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8d456-812f-47f3-b722-ee4dffd83530 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:04.053888 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1701/3429] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:03 2022] PUT /identity/v3/projects/0205e02728374f65b8c049cbe7f76e16/users/6801dcc2a64d4bc4acfd32b7e73c0513/roles/0fe2845ecc834c67a2280a8b2e92b4cc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:04.057768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee869c3-edac-487e-90fb-3dbf410d95d8 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:04.073190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee869c3-edac-487e-90fb-3dbf410d95d8 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:04.164588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28832dee-8ef1-4ddd-b031-6477f49b98ae None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:04.164890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28832dee-8ef1-4ddd-b031-6477f49b98ae None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:04.165161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28832dee-8ef1-4ddd-b031-6477f49b98ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:04.200604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-28832dee-8ef1-4ddd-b031-6477f49b98ae None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:04.201836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-28832dee-8ef1-4ddd-b031-6477f49b98ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:04.205007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e2e41-bb8e-4a08-b9d8-8e149d1cd6bf tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:04.206664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2e41-bb8e-4a08-b9d8-8e149d1cd6bf tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:04.206961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2e41-bb8e-4a08-b9d8-8e149d1cd6bf tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:04.207386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2e41-bb8e-4a08-b9d8-8e149d1cd6bf tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/0205e02728374f65b8c049cbe7f76e16/users/6801dcc2a64d4bc4acfd32b7e73c0513/roles/5592daac868644839c7bd512d73831c4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:04.210023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e2e41-bb8e-4a08-b9d8-8e149d1cd6bf tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_grant(project_id=0205e02728374f65b8c049cbe7f76e16, user_id=6801dcc2a64d4bc4acfd32b7e73c0513, role_id=5592daac868644839c7bd512d73831c4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:04.216023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e2e41-bb8e-4a08-b9d8-8e149d1cd6bf tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:04.248860 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1729/3430] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:04 2022] PUT /identity/v3/projects/0205e02728374f65b8c049cbe7f76e16/users/6801dcc2a64d4bc4acfd32b7e73c0513/roles/5592daac868644839c7bd512d73831c4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:04.251429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e2e41-bb8e-4a08-b9d8-8e149d1cd6bf tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:04.301199 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1702/3431] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:04.318868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28832dee-8ef1-4ddd-b031-6477f49b98ae None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:04.321165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-28832dee-8ef1-4ddd-b031-6477f49b98ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:04.348426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e2e41-bb8e-4a08-b9d8-8e149d1cd6bf tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:04.354560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f8c58-1bfe-4814-9294-13093b420746 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:04.355352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8c58-1bfe-4814-9294-13093b420746 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:04.355728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8c58-1bfe-4814-9294-13093b420746 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:04.356100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8c58-1bfe-4814-9294-13093b420746 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:04.357450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f8c58-1bfe-4814-9294-13093b420746 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:04.358747 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-530f8c58-1bfe-4814-9294-13093b420746 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:05:04.358747 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:05:04.358747 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:05:04.360311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f8c58-1bfe-4814-9294-13093b420746 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:04.407078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28832dee-8ef1-4ddd-b031-6477f49b98ae None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:04.416616 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1730/3432] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:04 2022] GET /identity/v3/users => generated 8647 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:05:04.421608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:04.423333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4e7b-85c1-4e81-a9bb-6c0acf757039 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:04.423927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4e7b-85c1-4e81-a9bb-6c0acf757039 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:04.424260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4e7b-85c1-4e81-a9bb-6c0acf757039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:04.476026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-020e4e7b-85c1-4e81-a9bb-6c0acf757039 None None] MFA Rules not processed for user `6801dcc2a64d4bc4acfd32b7e73c0513`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:04.481175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-020e4e7b-85c1-4e81-a9bb-6c0acf757039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:04.490532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:04.497341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b5159-f8ec-4555-a3c4-592fa4709fec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:04.497976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b5159-f8ec-4555-a3c4-592fa4709fec service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:04.498277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b5159-f8ec-4555-a3c4-592fa4709fec service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:04.498573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b5159-f8ec-4555-a3c4-592fa4709fec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:04.571524 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1731/3433] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:04.573048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3444-23f1-4349-9166-8efdc450d9ad None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:04.573335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3444-23f1-4349-9166-8efdc450d9ad None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:04.573604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3444-23f1-4349-9166-8efdc450d9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:04.581678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b5159-f8ec-4555-a3c4-592fa4709fec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:04.585470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b5159-f8ec-4555-a3c4-592fa4709fec service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:04.593750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-74ff3444-23f1-4349-9166-8efdc450d9ad None None] MFA Rules not processed for user `6801dcc2a64d4bc4acfd32b7e73c0513`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:04.595030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-74ff3444-23f1-4349-9166-8efdc450d9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:04.599639 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1703/3434] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:05:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:05:04.603587 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1732/3435] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:04.607635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b5159-f8ec-4555-a3c4-592fa4709fec service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:04.619993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b5159-f8ec-4555-a3c4-592fa4709fec service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:04.626094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3e5c9-da3a-43bf-a8ca-173314987c80 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6801dcc2a64d4bc4acfd32b7e73c0513', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0205e02728374f65b8c049cbe7f76e16', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-409121486', 'tempest-NotDelegatedRole-2085263922'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:04.626717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3e5c9-da3a-43bf-a8ca-173314987c80 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:04.626984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3e5c9-da3a-43bf-a8ca-173314987c80 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:04.627246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3e5c9-da3a-43bf-a8ca-173314987c80 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:04.627920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3e5c9-da3a-43bf-a8ca-173314987c80 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] RBAC: Authorizing `identity:create_trust()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:04.629245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3e5c9-da3a-43bf-a8ca-173314987c80 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:04.715707 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1704/3436] 10.222.0.23 () {60 vars in 1174 bytes} [Sat Aug 20 21:05:04 2022] POST /identity/v3/OS-TRUST/trusts => generated 833 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:04.717031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff3444-23f1-4349-9166-8efdc450d9ad None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:04.732217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff3444-23f1-4349-9166-8efdc450d9ad None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:04.736836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc5bdc-14f1-492d-9900-918fdfd8b62c tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6801dcc2a64d4bc4acfd32b7e73c0513', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0205e02728374f65b8c049cbe7f76e16', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-409121486', 'tempest-NotDelegatedRole-2085263922'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:04.737686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc5bdc-14f1-492d-9900-918fdfd8b62c tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:04.738084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc5bdc-14f1-492d-9900-918fdfd8b62c tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:04.738511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc5bdc-14f1-492d-9900-918fdfd8b62c tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:04.739237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc5bdc-14f1-492d-9900-918fdfd8b62c tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] RBAC: Authorizing `identity:list_trusts_for_trustor()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:04.740843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc5bdc-14f1-492d-9900-918fdfd8b62c tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:04.751640 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1733/3437] 10.222.0.23 () {58 vars in 1252 bytes} [Sat Aug 20 21:05:04 2022] GET /identity/v3/OS-TRUST/trusts/?trustor_user_id=6801dcc2a64d4bc4acfd32b7e73c0513 => generated 611 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:04.755804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3e5c9-da3a-43bf-a8ca-173314987c80 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:04.781265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3e5c9-da3a-43bf-a8ca-173314987c80 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:04.791138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6801dcc2a64d4bc4acfd32b7e73c0513', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0205e02728374f65b8c049cbe7f76e16', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-409121486', 'tempest-NotDelegatedRole-2085263922'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:04.791829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:04.792113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:04.792392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] PATH_INFO: `/v3/OS-TRUST/trusts/d6903ee147154e3eb429741c8e59dabd` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:04.808117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] RBAC: Authorizing `identity:delete_trust(trust_id=d6903ee147154e3eb429741c8e59dabd)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:04.811819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:04.853146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'd6903ee147154e3eb429741c8e59dabd', 'request_id': 'req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:04.854164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'd6903ee147154e3eb429741c8e59dabd', 'request_id': 'req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:04.854763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'd6903ee147154e3eb429741c8e59dabd', 'request_id': 'req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:04.875660 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1705/3438] 10.222.0.23 () {58 vars in 1254 bytes} [Sat Aug 20 21:05:04 2022] DELETE /identity/v3/OS-TRUST/trusts/d6903ee147154e3eb429741c8e59dabd => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:04.880956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc5bdc-14f1-492d-9900-918fdfd8b62c tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:04.889913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fbfc5bdc-14f1-492d-9900-918fdfd8b62c tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:04.920116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc5bdc-14f1-492d-9900-918fdfd8b62c tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:04.924682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9302bbd-b917-4fc3-a30b-ff3e1552f381 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6801dcc2a64d4bc4acfd32b7e73c0513', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0205e02728374f65b8c049cbe7f76e16', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-409121486', 'tempest-NotDelegatedRole-2085263922'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:04.925456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9302bbd-b917-4fc3-a30b-ff3e1552f381 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:04.925858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9302bbd-b917-4fc3-a30b-ff3e1552f381 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:04.926353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9302bbd-b917-4fc3-a30b-ff3e1552f381 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] PATH_INFO: `/v3/OS-TRUST/trusts/d6903ee147154e3eb429741c8e59dabd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-b9302bbd-b917-4fc3-a30b-ff3e1552f381 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Could not find trust: d6903ee147154e3eb429741c8e59dabd.: keystone.exception.TrustNotFound: Could not find trust: d6903ee147154e3eb429741c8e59dabd. Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Aug 20 21:05:04.937552 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Aug 20 21:05:04.938662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Aug 20 21:05:04.938662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Aug 20 21:05:04.938662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Aug 20 21:05:04.938662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Aug 20 21:05:04.938662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Aug 20 21:05:04.938662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:04.938662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:04.938662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Aug 20 21:05:04.938662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Aug 20 21:05:04.938662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Aug 20 21:05:04.938662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Aug 20 21:05:04.938662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: d6903ee147154e3eb429741c8e59dabd. Aug 20 21:05:04.938662 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:04.941346 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1734/3439] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:05:04 2022] GET /identity/v3/OS-TRUST/trusts/d6903ee147154e3eb429741c8e59dabd => generated 111 bytes in 61 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:05:04.945822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:04.949464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:04.976401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcb347-3bc7-4c10-98d7-bcb6f2cd620a tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:04.983785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3dcf2-a26a-43b6-94ed-1af6a46d6acd tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:04.984800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3dcf2-a26a-43b6-94ed-1af6a46d6acd tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:04.985149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3dcf2-a26a-43b6-94ed-1af6a46d6acd tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:04.985517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3dcf2-a26a-43b6-94ed-1af6a46d6acd tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles/5592daac868644839c7bd512d73831c4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:04.987879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3dcf2-a26a-43b6-94ed-1af6a46d6acd tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_role(role_id=5592daac868644839c7bd512d73831c4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:04.990190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3dcf2-a26a-43b6-94ed-1af6a46d6acd tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:04.993795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:05.078223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-49f3dcf2-a26a-43b6-94ed-1af6a46d6acd tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invalidating the token cache because role 5592daac868644839c7bd512d73831c4 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:05.079713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-49f3dcf2-a26a-43b6-94ed-1af6a46d6acd tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.080448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-49f3dcf2-a26a-43b6-94ed-1af6a46d6acd tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.082540 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1706/3440] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:04 2022] DELETE /identity/v3/roles/5592daac868644839c7bd512d73831c4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:05.086784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9302bbd-b917-4fc3-a30b-ff3e1552f381 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:05.088670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b9302bbd-b917-4fc3-a30b-ff3e1552f381 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:05.113619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9302bbd-b917-4fc3-a30b-ff3e1552f381 tempest-TrustsV3TestJSON-1677420359 tempest-user-1602297818] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:05.263272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17583c-a10e-46cf-b924-e7129e423fc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:05.263919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17583c-a10e-46cf-b924-e7129e423fc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:05.264176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17583c-a10e-46cf-b924-e7129e423fc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:05.264535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17583c-a10e-46cf-b924-e7129e423fc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles/0fe2845ecc834c67a2280a8b2e92b4cc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:05.265928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17583c-a10e-46cf-b924-e7129e423fc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_role(role_id=0fe2845ecc834c67a2280a8b2e92b4cc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:05.267521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17583c-a10e-46cf-b924-e7129e423fc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:05.312801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2f17583c-a10e-46cf-b924-e7129e423fc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invalidating the token cache because role 0fe2845ecc834c67a2280a8b2e92b4cc has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:05.313523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2f17583c-a10e-46cf-b924-e7129e423fc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.314416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2f17583c-a10e-46cf-b924-e7129e423fc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.316809 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1735/3441] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:05 2022] DELETE /identity/v3/roles/0fe2845ecc834c67a2280a8b2e92b4cc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:05.325159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3dcf2-a26a-43b6-94ed-1af6a46d6acd tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:05.339534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3dcf2-a26a-43b6-94ed-1af6a46d6acd tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:05.417404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:05.418310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:05.418658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:05.418999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users/6801dcc2a64d4bc4acfd32b7e73c0513` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:05.420389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_user(user_id=6801dcc2a64d4bc4acfd32b7e73c0513)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:05.422451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:05.477615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6801dcc2a64d4bc4acfd32b7e73c0513'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.491981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6801dcc2a64d4bc4acfd32b7e73c0513'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.507710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6801dcc2a64d4bc4acfd32b7e73c0513'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.508860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6801dcc2a64d4bc4acfd32b7e73c0513'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.509466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6801dcc2a64d4bc4acfd32b7e73c0513'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.528475 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1707/3442] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:05 2022] DELETE /identity/v3/users/6801dcc2a64d4bc4acfd32b7e73c0513 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:05.530257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17583c-a10e-46cf-b924-e7129e423fc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:05.531940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2f17583c-a10e-46cf-b924-e7129e423fc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:05.555274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17583c-a10e-46cf-b924-e7129e423fc3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:05.624320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:05.648419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:05.649548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:05.650019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:05.650510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/0205e02728374f65b8c049cbe7f76e16` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:05.652395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0205e02728374f65b8c049cbe7f76e16)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:05.654579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:05.739252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0205e02728374f65b8c049cbe7f76e16', 'request_id': 'req-37c0967e-70f8-4371-a8a1-3fff639bbcac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.755164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0205e02728374f65b8c049cbe7f76e16', 'request_id': 'req-37c0967e-70f8-4371-a8a1-3fff639bbcac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.764440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0205e02728374f65b8c049cbe7f76e16', 'request_id': 'req-37c0967e-70f8-4371-a8a1-3fff639bbcac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.790958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] The token cache is being invalidate because project 0205e02728374f65b8c049cbe7f76e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:05.791382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.792192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:05.794466 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1736/3443] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:05 2022] DELETE /identity/v3/projects/0205e02728374f65b8c049cbe7f76e16 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:05.799580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:05.801044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:05.821293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9158ecc-e10b-49d7-acbf-91b622083fa0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:05.893169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:05.910234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bf006-68db-41a7-8b0a-5ef14281b1da tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:05.911881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bf006-68db-41a7-8b0a-5ef14281b1da tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:05.912262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bf006-68db-41a7-8b0a-5ef14281b1da tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:05.912654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bf006-68db-41a7-8b0a-5ef14281b1da tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:05.913512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bf006-68db-41a7-8b0a-5ef14281b1da tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:05.917224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bf006-68db-41a7-8b0a-5ef14281b1da tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:05.972647 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1708/3444] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:05:05 2022] POST /identity/v3/projects => generated 329 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:05.977013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:05.978521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:05.993686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0967e-70f8-4371-a8a1-3fff639bbcac tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:06.080389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9386df7-65c7-4fad-bc7d-9b5383e98642 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:06.081118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9386df7-65c7-4fad-bc7d-9b5383e98642 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:06.081620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9386df7-65c7-4fad-bc7d-9b5383e98642 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:06.081960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9386df7-65c7-4fad-bc7d-9b5383e98642 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:06.082676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9386df7-65c7-4fad-bc7d-9b5383e98642 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:06.085882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9386df7-65c7-4fad-bc7d-9b5383e98642 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:06.134662 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1737/3445] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:05 2022] POST /identity/v3/users => generated 416 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:06.136175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bf006-68db-41a7-8b0a-5ef14281b1da tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:06.154285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bf006-68db-41a7-8b0a-5ef14281b1da tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:06.163308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f3c84-06b0-4b25-a6c3-519cce55f88b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:06.164189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f3c84-06b0-4b25-a6c3-519cce55f88b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:06.164689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f3c84-06b0-4b25-a6c3-519cce55f88b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:06.165333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f3c84-06b0-4b25-a6c3-519cce55f88b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:06.166289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f3c84-06b0-4b25-a6c3-519cce55f88b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:06.169828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f3c84-06b0-4b25-a6c3-519cce55f88b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:06.207525 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1709/3446] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:05:06 2022] POST /identity/v3/roles => generated 243 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:06.208157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9386df7-65c7-4fad-bc7d-9b5383e98642 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:06.224886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9386df7-65c7-4fad-bc7d-9b5383e98642 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:06.235031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f58a91-d142-4526-835d-91815490bfb7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:06.235673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f58a91-d142-4526-835d-91815490bfb7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:06.235953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f58a91-d142-4526-835d-91815490bfb7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:06.236225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f58a91-d142-4526-835d-91815490bfb7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:06.236934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f58a91-d142-4526-835d-91815490bfb7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:06.238930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f58a91-d142-4526-835d-91815490bfb7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:06.260412 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1738/3447] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:05:06 2022] POST /identity/v3/roles => generated 245 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:06.261663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f3c84-06b0-4b25-a6c3-519cce55f88b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:06.273163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f3c84-06b0-4b25-a6c3-519cce55f88b tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:06.288094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45c1f4-fa02-416d-bba6-d82ac03571a5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:06.290318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45c1f4-fa02-416d-bba6-d82ac03571a5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:06.290848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45c1f4-fa02-416d-bba6-d82ac03571a5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:06.291835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45c1f4-fa02-416d-bba6-d82ac03571a5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/cba455d8677748599f47eeda75c6229a/users/e6f22f3a8eee4806b8677ac4cfbaeaf3/roles/0113f9ec53594039947350846a41d87f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:06.314096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45c1f4-fa02-416d-bba6-d82ac03571a5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_grant(project_id=cba455d8677748599f47eeda75c6229a, user_id=e6f22f3a8eee4806b8677ac4cfbaeaf3, role_id=0113f9ec53594039947350846a41d87f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:06.319353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45c1f4-fa02-416d-bba6-d82ac03571a5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:06.355663 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1710/3448] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:06 2022] PUT /identity/v3/projects/cba455d8677748599f47eeda75c6229a/users/e6f22f3a8eee4806b8677ac4cfbaeaf3/roles/0113f9ec53594039947350846a41d87f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:06.360699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f58a91-d142-4526-835d-91815490bfb7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:06.380501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f58a91-d142-4526-835d-91815490bfb7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:06.485907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:06.525579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a5f51-2936-4208-b1f4-4a01be955d45 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:06.527654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a5f51-2936-4208-b1f4-4a01be955d45 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:06.527930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a5f51-2936-4208-b1f4-4a01be955d45 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:06.528206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a5f51-2936-4208-b1f4-4a01be955d45 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/cba455d8677748599f47eeda75c6229a/users/e6f22f3a8eee4806b8677ac4cfbaeaf3/roles/54eabd2b955c4057a61b17e511a6bd9e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:06.531072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a5f51-2936-4208-b1f4-4a01be955d45 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_grant(project_id=cba455d8677748599f47eeda75c6229a, user_id=e6f22f3a8eee4806b8677ac4cfbaeaf3, role_id=54eabd2b955c4057a61b17e511a6bd9e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:06.533808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a5f51-2936-4208-b1f4-4a01be955d45 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:06.557323 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1739/3449] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:06 2022] PUT /identity/v3/projects/cba455d8677748599f47eeda75c6229a/users/e6f22f3a8eee4806b8677ac4cfbaeaf3/roles/54eabd2b955c4057a61b17e511a6bd9e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:06.561416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45c1f4-fa02-416d-bba6-d82ac03571a5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:06.574656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45c1f4-fa02-416d-bba6-d82ac03571a5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:06.666012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2bb20-f870-4549-b309-0ac36510bf72 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:06.666652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2bb20-f870-4549-b309-0ac36510bf72 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:06.666990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2bb20-f870-4549-b309-0ac36510bf72 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:06.667264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2bb20-f870-4549-b309-0ac36510bf72 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:06.667932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2bb20-f870-4549-b309-0ac36510bf72 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:06.670093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2bb20-f870-4549-b309-0ac36510bf72 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:06.712205 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1711/3450] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:06 2022] GET /identity/v3/users => generated 8647 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:05:06.716935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bea744-2e4f-47db-8115-7c27011c8cb8 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:06.717355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bea744-2e4f-47db-8115-7c27011c8cb8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:06.717697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bea744-2e4f-47db-8115-7c27011c8cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:06.795776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-08bea744-2e4f-47db-8115-7c27011c8cb8 None None] MFA Rules not processed for user `e6f22f3a8eee4806b8677ac4cfbaeaf3`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:06.797473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-08bea744-2e4f-47db-8115-7c27011c8cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:06.861921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2bb20-f870-4549-b309-0ac36510bf72 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:06.864963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-08a2bb20-f870-4549-b309-0ac36510bf72 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:06.918960 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1740/3451] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:06 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:06.921037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab4207-2a28-4d38-998c-1f55c65764f7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:06.921313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab4207-2a28-4d38-998c-1f55c65764f7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:06.921699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab4207-2a28-4d38-998c-1f55c65764f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:06.942542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cfab4207-2a28-4d38-998c-1f55c65764f7 None None] MFA Rules not processed for user `e6f22f3a8eee4806b8677ac4cfbaeaf3`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:06.943759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cfab4207-2a28-4d38-998c-1f55c65764f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:06.953919 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1741/3452] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:06 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:06.956927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab4207-2a28-4d38-998c-1f55c65764f7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:06.961870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2bb20-f870-4549-b309-0ac36510bf72 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:06.969219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdd229-466e-49c9-a1ca-74aae4466d5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:06.972645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab4207-2a28-4d38-998c-1f55c65764f7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:06.985176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdd229-466e-49c9-a1ca-74aae4466d5d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:06.985520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdd229-466e-49c9-a1ca-74aae4466d5d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:06.985814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdd229-466e-49c9-a1ca-74aae4466d5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:06.987612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1535dcf-a107-42b9-92c2-20ebd97d2eba tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6f22f3a8eee4806b8677ac4cfbaeaf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cba455d8677748599f47eeda75c6229a', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-995472406', 'tempest-NotDelegatedRole-90579454'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:06.987915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-57bdd229-466e-49c9-a1ca-74aae4466d5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:06.988243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1535dcf-a107-42b9-92c2-20ebd97d2eba tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:06.988560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1535dcf-a107-42b9-92c2-20ebd97d2eba tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:06.988820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1535dcf-a107-42b9-92c2-20ebd97d2eba tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:06.989495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1535dcf-a107-42b9-92c2-20ebd97d2eba tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorizing `identity:create_trust()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:06.990929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1535dcf-a107-42b9-92c2-20ebd97d2eba tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:07.014851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bdd229-466e-49c9-a1ca-74aae4466d5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:07.016807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bdd229-466e-49c9-a1ca-74aae4466d5d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:07.036200 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1712/3453] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:05:07.183081 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1742/3454] 10.222.0.23 () {60 vars in 1174 bytes} [Sat Aug 20 21:05:06 2022] POST /identity/v3/OS-TRUST/trusts => generated 858 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:07.189180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdd229-466e-49c9-a1ca-74aae4466d5d service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:07.203081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdd229-466e-49c9-a1ca-74aae4466d5d service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:07.208296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2dc46-7afa-43b3-8a38-178be5877533 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6f22f3a8eee4806b8677ac4cfbaeaf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cba455d8677748599f47eeda75c6229a', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-995472406', 'tempest-NotDelegatedRole-90579454'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:07.209018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2dc46-7afa-43b3-8a38-178be5877533 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:07.209285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2dc46-7afa-43b3-8a38-178be5877533 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:07.209550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2dc46-7afa-43b3-8a38-178be5877533 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] PATH_INFO: `/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:07.222901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2dc46-7afa-43b3-8a38-178be5877533 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorizing `identity:get_trust(trust_id=b1104eb9110441d6af57e68c81d80b21)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:07.224054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2dc46-7afa-43b3-8a38-178be5877533 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:07.241425 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1713/3455] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:05:07 2022] GET /identity/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21 => generated 858 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:05:07.245953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1535dcf-a107-42b9-92c2-20ebd97d2eba tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:07.259234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1535dcf-a107-42b9-92c2-20ebd97d2eba tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:07.265022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46312da1-18da-47bc-9b7a-d45403152801 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6f22f3a8eee4806b8677ac4cfbaeaf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cba455d8677748599f47eeda75c6229a', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-995472406', 'tempest-NotDelegatedRole-90579454'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:07.265892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46312da1-18da-47bc-9b7a-d45403152801 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:07.266166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46312da1-18da-47bc-9b7a-d45403152801 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:07.266436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46312da1-18da-47bc-9b7a-d45403152801 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] PATH_INFO: `/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:07.275809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46312da1-18da-47bc-9b7a-d45403152801 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=b1104eb9110441d6af57e68c81d80b21)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:07.277345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46312da1-18da-47bc-9b7a-d45403152801 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:07.290193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2dc46-7afa-43b3-8a38-178be5877533 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:07.298343 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1743/3456] 10.222.0.23 () {58 vars in 1269 bytes} [Sat Aug 20 21:05:07 2022] GET /identity/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21/roles => generated 370 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:07.302345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2dc46-7afa-43b3-8a38-178be5877533 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:07.306546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e5820-3918-4320-bc88-d22a2e48b021 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6f22f3a8eee4806b8677ac4cfbaeaf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cba455d8677748599f47eeda75c6229a', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-995472406', 'tempest-NotDelegatedRole-90579454'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:07.307196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e5820-3918-4320-bc88-d22a2e48b021 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:07.307466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e5820-3918-4320-bc88-d22a2e48b021 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:07.307730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e5820-3918-4320-bc88-d22a2e48b021 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] PATH_INFO: `/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21/roles/0113f9ec53594039947350846a41d87f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:07.314632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e5820-3918-4320-bc88-d22a2e48b021 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorizing `identity:get_role_for_trust(trust_id=b1104eb9110441d6af57e68c81d80b21, role_id=0113f9ec53594039947350846a41d87f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:07.316406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e5820-3918-4320-bc88-d22a2e48b021 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:07.318860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:07.329329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46312da1-18da-47bc-9b7a-d45403152801 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:07.340095 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1714/3457] 10.222.0.23 () {58 vars in 1368 bytes} [Sat Aug 20 21:05:07 2022] GET /identity/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21/roles/0113f9ec53594039947350846a41d87f => generated 243 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:07.340738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46312da1-18da-47bc-9b7a-d45403152801 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:07.346634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff9d25-dc9e-4cac-8fa4-2a9d05646a73 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6f22f3a8eee4806b8677ac4cfbaeaf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cba455d8677748599f47eeda75c6229a', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-995472406', 'tempest-NotDelegatedRole-90579454'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:07.347750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff9d25-dc9e-4cac-8fa4-2a9d05646a73 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] REQUEST_METHOD: `HEAD` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:07.349002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff9d25-dc9e-4cac-8fa4-2a9d05646a73 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:07.349436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff9d25-dc9e-4cac-8fa4-2a9d05646a73 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] PATH_INFO: `/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21/roles/0113f9ec53594039947350846a41d87f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:07.356466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff9d25-dc9e-4cac-8fa4-2a9d05646a73 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorizing `identity:get_role_for_trust(trust_id=b1104eb9110441d6af57e68c81d80b21, role_id=0113f9ec53594039947350846a41d87f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:07.358310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff9d25-dc9e-4cac-8fa4-2a9d05646a73 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:07.369539 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1744/3458] 10.222.0.23 () {58 vars in 1369 bytes} [Sat Aug 20 21:05:07 2022] HEAD /identity/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21/roles/0113f9ec53594039947350846a41d87f => generated 0 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Aug 20 21:05:07.371277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e5820-3918-4320-bc88-d22a2e48b021 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:07.380997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e5820-3918-4320-bc88-d22a2e48b021 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:07.386023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091fced-126e-4f7c-95d7-8ad972e813dc tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6f22f3a8eee4806b8677ac4cfbaeaf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cba455d8677748599f47eeda75c6229a', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-995472406', 'tempest-NotDelegatedRole-90579454'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:07.386791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091fced-126e-4f7c-95d7-8ad972e813dc tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:07.386791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091fced-126e-4f7c-95d7-8ad972e813dc tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:07.386791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091fced-126e-4f7c-95d7-8ad972e813dc tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] PATH_INFO: `/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21/roles/54eabd2b955c4057a61b17e511a6bd9e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:07.393280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091fced-126e-4f7c-95d7-8ad972e813dc tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorizing `identity:get_role_for_trust(trust_id=b1104eb9110441d6af57e68c81d80b21, role_id=54eabd2b955c4057a61b17e511a6bd9e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:07.394712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091fced-126e-4f7c-95d7-8ad972e813dc tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-5091fced-126e-4f7c-95d7-8ad972e813dc tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Could not find role: 54eabd2b955c4057a61b17e511a6bd9e.: keystone.exception.RoleNotFound: Could not find role: 54eabd2b955c4057a61b17e511a6bd9e. Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 21:05:07.405201 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 54eabd2b955c4057a61b17e511a6bd9e. Aug 20 21:05:07.406133 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:05:07.407885 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1715/3459] 10.222.0.23 () {58 vars in 1368 bytes} [Sat Aug 20 21:05:07 2022] GET /identity/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21/roles/54eabd2b955c4057a61b17e511a6bd9e => generated 110 bytes in 37 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:05:07.410597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff9d25-dc9e-4cac-8fa4-2a9d05646a73 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:07.421178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff9d25-dc9e-4cac-8fa4-2a9d05646a73 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:07.427540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925778b-3b8b-49ab-93fd-fc937caf4383 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6f22f3a8eee4806b8677ac4cfbaeaf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cba455d8677748599f47eeda75c6229a', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-995472406', 'tempest-NotDelegatedRole-90579454'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:07.428154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925778b-3b8b-49ab-93fd-fc937caf4383 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] REQUEST_METHOD: `HEAD` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:07.428461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925778b-3b8b-49ab-93fd-fc937caf4383 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:07.428722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925778b-3b8b-49ab-93fd-fc937caf4383 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] PATH_INFO: `/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21/roles/54eabd2b955c4057a61b17e511a6bd9e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:07.436067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925778b-3b8b-49ab-93fd-fc937caf4383 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorizing `identity:get_role_for_trust(trust_id=b1104eb9110441d6af57e68c81d80b21, role_id=54eabd2b955c4057a61b17e511a6bd9e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:07.437632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925778b-3b8b-49ab-93fd-fc937caf4383 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:07.443007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-5925778b-3b8b-49ab-93fd-fc937caf4383 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Could not find role: 54eabd2b955c4057a61b17e511a6bd9e.: keystone.exception.RoleNotFound: Could not find role: 54eabd2b955c4057a61b17e511a6bd9e. Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 21:05:07.452603 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 54eabd2b955c4057a61b17e511a6bd9e. Aug 20 21:05:07.455348 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:07.458870 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1745/3460] 10.222.0.23 () {58 vars in 1369 bytes} [Sat Aug 20 21:05:07 2022] HEAD /identity/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21/roles/54eabd2b955c4057a61b17e511a6bd9e => generated 0 bytes in 45 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Aug 20 21:05:07.459948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091fced-126e-4f7c-95d7-8ad972e813dc tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:07.474623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091fced-126e-4f7c-95d7-8ad972e813dc tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:07.480771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487a642-358c-413c-a74d-684e77043cde tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6f22f3a8eee4806b8677ac4cfbaeaf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cba455d8677748599f47eeda75c6229a', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-995472406', 'tempest-NotDelegatedRole-90579454'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:07.481420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487a642-358c-413c-a74d-684e77043cde tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:07.481695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487a642-358c-413c-a74d-684e77043cde tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:07.482093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487a642-358c-413c-a74d-684e77043cde tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] PATH_INFO: `/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:07.491422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487a642-358c-413c-a74d-684e77043cde tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorizing `identity:delete_trust(trust_id=b1104eb9110441d6af57e68c81d80b21)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:07.492974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487a642-358c-413c-a74d-684e77043cde tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:07.519470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c487a642-358c-413c-a74d-684e77043cde tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'b1104eb9110441d6af57e68c81d80b21', 'request_id': 'req-c487a642-358c-413c-a74d-684e77043cde'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:07.524469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c487a642-358c-413c-a74d-684e77043cde tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'b1104eb9110441d6af57e68c81d80b21', 'request_id': 'req-c487a642-358c-413c-a74d-684e77043cde'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:07.524469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c487a642-358c-413c-a74d-684e77043cde tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'b1104eb9110441d6af57e68c81d80b21', 'request_id': 'req-c487a642-358c-413c-a74d-684e77043cde'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:07.552210 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1716/3461] 10.222.0.23 () {58 vars in 1254 bytes} [Sat Aug 20 21:05:07 2022] DELETE /identity/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:07.560792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925778b-3b8b-49ab-93fd-fc937caf4383 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:07.563757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5925778b-3b8b-49ab-93fd-fc937caf4383 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:07.585309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925778b-3b8b-49ab-93fd-fc937caf4383 tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:07.591638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46216e73-81e0-41ce-920f-751b7e31f83e tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6f22f3a8eee4806b8677ac4cfbaeaf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cba455d8677748599f47eeda75c6229a', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-995472406', 'tempest-NotDelegatedRole-90579454'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:07.592536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46216e73-81e0-41ce-920f-751b7e31f83e tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:07.592808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46216e73-81e0-41ce-920f-751b7e31f83e tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:07.593066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46216e73-81e0-41ce-920f-751b7e31f83e tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] PATH_INFO: `/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-46216e73-81e0-41ce-920f-751b7e31f83e tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Could not find trust: b1104eb9110441d6af57e68c81d80b21.: keystone.exception.TrustNotFound: Could not find trust: b1104eb9110441d6af57e68c81d80b21. Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Aug 20 21:05:07.611240 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Aug 20 21:05:07.612173 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Aug 20 21:05:07.612173 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Aug 20 21:05:07.612173 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Aug 20 21:05:07.612173 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Aug 20 21:05:07.612173 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Aug 20 21:05:07.612173 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:07.612173 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:07.612173 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Aug 20 21:05:07.612173 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Aug 20 21:05:07.612173 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Aug 20 21:05:07.612173 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Aug 20 21:05:07.612173 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: b1104eb9110441d6af57e68c81d80b21. Aug 20 21:05:07.612173 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:07.615898 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1746/3462] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:05:07 2022] GET /identity/v3/OS-TRUST/trusts/b1104eb9110441d6af57e68c81d80b21 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:05:07.620597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487a642-358c-413c-a74d-684e77043cde tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:07.623776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c487a642-358c-413c-a74d-684e77043cde tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:07.655750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487a642-358c-413c-a74d-684e77043cde tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:07.664709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d38a7-e577-4f90-8795-137f3769074c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:07.665497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d38a7-e577-4f90-8795-137f3769074c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:07.665833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d38a7-e577-4f90-8795-137f3769074c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:07.666280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d38a7-e577-4f90-8795-137f3769074c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles/54eabd2b955c4057a61b17e511a6bd9e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:07.668031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d38a7-e577-4f90-8795-137f3769074c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_role(role_id=54eabd2b955c4057a61b17e511a6bd9e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:07.669703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d38a7-e577-4f90-8795-137f3769074c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:07.735115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a3d38a7-e577-4f90-8795-137f3769074c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invalidating the token cache because role 54eabd2b955c4057a61b17e511a6bd9e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:07.735540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a3d38a7-e577-4f90-8795-137f3769074c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:07.736218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a3d38a7-e577-4f90-8795-137f3769074c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:07.738288 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1717/3463] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:07 2022] DELETE /identity/v3/roles/54eabd2b955c4057a61b17e511a6bd9e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:07.742879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46216e73-81e0-41ce-920f-751b7e31f83e tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:07.746157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-46216e73-81e0-41ce-920f-751b7e31f83e tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:07.764693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46216e73-81e0-41ce-920f-751b7e31f83e tempest-TrustsV3TestJSON-1588477709 tempest-user-1033392276] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:07.827831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea8734-a445-4ecc-8b2f-966c184cd324 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:07.828512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea8734-a445-4ecc-8b2f-966c184cd324 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:07.828791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea8734-a445-4ecc-8b2f-966c184cd324 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:07.829057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea8734-a445-4ecc-8b2f-966c184cd324 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles/0113f9ec53594039947350846a41d87f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:07.830457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea8734-a445-4ecc-8b2f-966c184cd324 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_role(role_id=0113f9ec53594039947350846a41d87f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:07.832069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea8734-a445-4ecc-8b2f-966c184cd324 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:07.872051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4aea8734-a445-4ecc-8b2f-966c184cd324 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invalidating the token cache because role 0113f9ec53594039947350846a41d87f has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:07.872392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4aea8734-a445-4ecc-8b2f-966c184cd324 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:07.873029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4aea8734-a445-4ecc-8b2f-966c184cd324 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:07.875145 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1747/3464] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:07 2022] DELETE /identity/v3/roles/0113f9ec53594039947350846a41d87f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:07.879365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d38a7-e577-4f90-8795-137f3769074c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:07.897108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d38a7-e577-4f90-8795-137f3769074c tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:07.966023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:07.966661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:07.966937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:07.967203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users/e6f22f3a8eee4806b8677ac4cfbaeaf3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:07.968324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_user(user_id=e6f22f3a8eee4806b8677ac4cfbaeaf3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:07.970310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:08.018034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f22f3a8eee4806b8677ac4cfbaeaf3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.027395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f22f3a8eee4806b8677ac4cfbaeaf3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.033777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=13 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:08.046991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f22f3a8eee4806b8677ac4cfbaeaf3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.047993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f22f3a8eee4806b8677ac4cfbaeaf3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.048930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f22f3a8eee4806b8677ac4cfbaeaf3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.081155 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1718/3465] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:07 2022] DELETE /identity/v3/users/e6f22f3a8eee4806b8677ac4cfbaeaf3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:08.085522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea8734-a445-4ecc-8b2f-966c184cd324 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:08.087501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4aea8734-a445-4ecc-8b2f-966c184cd324 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:08.101963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea8734-a445-4ecc-8b2f-966c184cd324 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:08.201475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:08.204906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:08.207195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:08.223738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:08.224210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:08.224535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/cba455d8677748599f47eeda75c6229a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:08.227433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cba455d8677748599f47eeda75c6229a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:08.234465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:08.298737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba455d8677748599f47eeda75c6229a', 'request_id': 'req-a4e659c0-b05c-44b6-959e-777176fabea7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.308848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba455d8677748599f47eeda75c6229a', 'request_id': 'req-a4e659c0-b05c-44b6-959e-777176fabea7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.310559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:08.321181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba455d8677748599f47eeda75c6229a', 'request_id': 'req-a4e659c0-b05c-44b6-959e-777176fabea7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.332244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825239b-4276-4821-8810-28615a71eca4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:08.343304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] The token cache is being invalidate because project cba455d8677748599f47eeda75c6229a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:08.343684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.344444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.346403 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1748/3466] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:08 2022] DELETE /identity/v3/projects/cba455d8677748599f47eeda75c6229a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:08.354254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:08.355804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:08.438580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:08.439123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:08.439418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:08.439670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] PATH_INFO: `/v3/users/bdb15c8c1f3d4bc38552b33ba1f891ca` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:08.440850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdb15c8c1f3d4bc38552b33ba1f891ca)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:08.442792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:08.452480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e659c0-b05c-44b6-959e-777176fabea7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:08.460679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7b550-9a24-4020-8ef6-bf784bfeed6d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:08.462367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7b550-9a24-4020-8ef6-bf784bfeed6d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:08.462833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7b550-9a24-4020-8ef6-bf784bfeed6d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:08.463128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7b550-9a24-4020-8ef6-bf784bfeed6d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:08.463940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7b550-9a24-4020-8ef6-bf784bfeed6d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:08.467578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7b550-9a24-4020-8ef6-bf784bfeed6d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:08.514520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdb15c8c1f3d4bc38552b33ba1f891ca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.517142 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1749/3467] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:05:08 2022] POST /identity/v3/projects => generated 329 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:08.521807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7b550-9a24-4020-8ef6-bf784bfeed6d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:08.528404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdb15c8c1f3d4bc38552b33ba1f891ca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.558008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdb15c8c1f3d4bc38552b33ba1f891ca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.558788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdb15c8c1f3d4bc38552b33ba1f891ca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.559523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdb15c8c1f3d4bc38552b33ba1f891ca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:08.573972 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1719/3468] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:08 2022] DELETE /identity/v3/users/bdb15c8c1f3d4bc38552b33ba1f891ca => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:08.622259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7b550-9a24-4020-8ef6-bf784bfeed6d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:08.713236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336495e-24ba-4b65-b740-d8b590602bd1 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:08.714140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336495e-24ba-4b65-b740-d8b590602bd1 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:08.714564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336495e-24ba-4b65-b740-d8b590602bd1 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:08.715013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336495e-24ba-4b65-b740-d8b590602bd1 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:08.715812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336495e-24ba-4b65-b740-d8b590602bd1 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:08.718494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336495e-24ba-4b65-b740-d8b590602bd1 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:08.770987 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1750/3469] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:08 2022] POST /identity/v3/users => generated 413 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:08.773057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:08.774760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:08.788048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:08.803934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44367f12-23e1-41a6-8a4f-58da64b9740e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:08.810717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209901e0-03b5-40c6-b0b8-53a871192d85 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:08.812203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209901e0-03b5-40c6-b0b8-53a871192d85 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:08.812694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209901e0-03b5-40c6-b0b8-53a871192d85 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:08.813026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209901e0-03b5-40c6-b0b8-53a871192d85 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:08.814040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209901e0-03b5-40c6-b0b8-53a871192d85 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:08.815926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209901e0-03b5-40c6-b0b8-53a871192d85 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:08.841357 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1720/3470] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:05:08 2022] POST /identity/v3/roles => generated 244 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:08.846562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336495e-24ba-4b65-b740-d8b590602bd1 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:08.866108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336495e-24ba-4b65-b740-d8b590602bd1 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:08.878458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209901e0-03b5-40c6-b0b8-53a871192d85 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:08.879697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-209901e0-03b5-40c6-b0b8-53a871192d85 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:08.898132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1d6a2-0790-4478-88ed-356c1d2e2e60 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:08.900074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1d6a2-0790-4478-88ed-356c1d2e2e60 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:08.900074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1d6a2-0790-4478-88ed-356c1d2e2e60 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:08.900074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1d6a2-0790-4478-88ed-356c1d2e2e60 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:08.901782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1d6a2-0790-4478-88ed-356c1d2e2e60 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:08.903541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1d6a2-0790-4478-88ed-356c1d2e2e60 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:08.907406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:08.928936 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1751/3471] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:05:08 2022] POST /identity/v3/roles => generated 246 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:08.933270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1d6a2-0790-4478-88ed-356c1d2e2e60 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:08.966294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1d6a2-0790-4478-88ed-356c1d2e2e60 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:08.996831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da6ae1-0dea-46a0-95b4-c7577f163587 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:08.998220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6ae1-0dea-46a0-95b4-c7577f163587 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:08.998747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6ae1-0dea-46a0-95b4-c7577f163587 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:08.999175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6ae1-0dea-46a0-95b4-c7577f163587 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/9fedb70c317e452ca0475ace9de369c0/users/29b9ba554172470383d8241107eb8651/roles/aa7d5638250d4706b8ae3c88f565f449` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.025591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da6ae1-0dea-46a0-95b4-c7577f163587 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_grant(project_id=9fedb70c317e452ca0475ace9de369c0, user_id=29b9ba554172470383d8241107eb8651, role_id=aa7d5638250d4706b8ae3c88f565f449)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:09.027563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209901e0-03b5-40c6-b0b8-53a871192d85 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:09.028554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da6ae1-0dea-46a0-95b4-c7577f163587 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:09.036709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b5ef9-437a-49bb-9f86-4b198ae03303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:09.037498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b5ef9-437a-49bb-9f86-4b198ae03303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.037783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b5ef9-437a-49bb-9f86-4b198ae03303 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.038069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b5ef9-437a-49bb-9f86-4b198ae03303 admin admin] PATH_INFO: `/v3/projects/3381148916fe4893a3ab0edc998a6847` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.043071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b5ef9-437a-49bb-9f86-4b198ae03303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3381148916fe4893a3ab0edc998a6847)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:09.045175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b5ef9-437a-49bb-9f86-4b198ae03303 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:09.053723 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1752/3472] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:08 2022] PUT /identity/v3/projects/9fedb70c317e452ca0475ace9de369c0/users/29b9ba554172470383d8241107eb8651/roles/aa7d5638250d4706b8ae3c88f565f449 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:09.058296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da6ae1-0dea-46a0-95b4-c7577f163587 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:09.103022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-480b5ef9-437a-49bb-9f86-4b198ae03303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3381148916fe4893a3ab0edc998a6847', 'request_id': 'req-480b5ef9-437a-49bb-9f86-4b198ae03303'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:09.109541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-480b5ef9-437a-49bb-9f86-4b198ae03303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3381148916fe4893a3ab0edc998a6847', 'request_id': 'req-480b5ef9-437a-49bb-9f86-4b198ae03303'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:09.120679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-480b5ef9-437a-49bb-9f86-4b198ae03303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3381148916fe4893a3ab0edc998a6847', 'request_id': 'req-480b5ef9-437a-49bb-9f86-4b198ae03303'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:09.133687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-480b5ef9-437a-49bb-9f86-4b198ae03303 admin admin] The token cache is being invalidate because project 3381148916fe4893a3ab0edc998a6847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:09.134112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-480b5ef9-437a-49bb-9f86-4b198ae03303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:09.134992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-480b5ef9-437a-49bb-9f86-4b198ae03303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:09.135197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da6ae1-0dea-46a0-95b4-c7577f163587 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:09.137864 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1721/3473] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:08 2022] DELETE /identity/v3/projects/3381148916fe4893a3ab0edc998a6847 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:09.142530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b94f0f-5bdd-4296-a0a2-4fb98dd22ca6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.142804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b94f0f-5bdd-4296-a0a2-4fb98dd22ca6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.143068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b94f0f-5bdd-4296-a0a2-4fb98dd22ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.163419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-25b94f0f-5bdd-4296-a0a2-4fb98dd22ca6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:09.165283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-25b94f0f-5bdd-4296-a0a2-4fb98dd22ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:09.218176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcb565-5bc5-41a1-a494-069a0965e8b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:09.218859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcb565-5bc5-41a1-a494-069a0965e8b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.219134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcb565-5bc5-41a1-a494-069a0965e8b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.219406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcb565-5bc5-41a1-a494-069a0965e8b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/9fedb70c317e452ca0475ace9de369c0/users/29b9ba554172470383d8241107eb8651/roles/0a900e13aa1548cf8b67df63ee2cbf71` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.222439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcb565-5bc5-41a1-a494-069a0965e8b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_grant(project_id=9fedb70c317e452ca0475ace9de369c0, user_id=29b9ba554172470383d8241107eb8651, role_id=0a900e13aa1548cf8b67df63ee2cbf71)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:09.225458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcb565-5bc5-41a1-a494-069a0965e8b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:09.243681 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1722/3474] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:09.244023 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1753/3475] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:09 2022] PUT /identity/v3/projects/9fedb70c317e452ca0475ace9de369c0/users/29b9ba554172470383d8241107eb8651/roles/0a900e13aa1548cf8b67df63ee2cbf71 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:09.248145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b94f0f-5bdd-4296-a0a2-4fb98dd22ca6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:09.249896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-25b94f0f-5bdd-4296-a0a2-4fb98dd22ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:09.254679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311e3a5-c70b-4d45-9d02-19a154322702 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.254967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311e3a5-c70b-4d45-9d02-19a154322702 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.255287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311e3a5-c70b-4d45-9d02-19a154322702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.263869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b94f0f-5bdd-4296-a0a2-4fb98dd22ca6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:09.282820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8311e3a5-c70b-4d45-9d02-19a154322702 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:09.284322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8311e3a5-c70b-4d45-9d02-19a154322702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:09.289007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:09.340022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f418826-b189-470d-9882-7f06edd558b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:09.340750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f418826-b189-470d-9882-7f06edd558b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.341351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f418826-b189-470d-9882-7f06edd558b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.341351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f418826-b189-470d-9882-7f06edd558b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.342232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f418826-b189-470d-9882-7f06edd558b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:09.344184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f418826-b189-470d-9882-7f06edd558b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:09.374625 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1754/3476] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:09.375259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311e3a5-c70b-4d45-9d02-19a154322702 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:09.378531 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1723/3477] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:09 2022] GET /identity/v3/users => generated 8354 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:05:09.378661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8311e3a5-c70b-4d45-9d02-19a154322702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:09.378766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f418826-b189-470d-9882-7f06edd558b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:09.392703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f418826-b189-470d-9882-7f06edd558b9 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:09.393126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311e3a5-c70b-4d45-9d02-19a154322702 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:09.401120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e4059-cd04-48fb-83ea-5d72c66e9cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:09.401501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:09.402046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e4059-cd04-48fb-83ea-5d72c66e9cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.402324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e4059-cd04-48fb-83ea-5d72c66e9cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.402429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.402596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e4059-cd04-48fb-83ea-5d72c66e9cf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.402690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.402857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] PATH_INFO: `/v3/users/d28741c49cf9427fa5b670e2c979acc3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.403361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e4059-cd04-48fb-83ea-5d72c66e9cf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:05:09.403660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e4059-cd04-48fb-83ea-5d72c66e9cf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:09.404018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28741c49cf9427fa5b670e2c979acc3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:09.405307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e4059-cd04-48fb-83ea-5d72c66e9cf7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:09.406118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:09.411209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:09.421774 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1724/3478] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:09.425234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b6f90-c04e-40d9-a3ce-aa2d07a7d3c2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.425234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b6f90-c04e-40d9-a3ce-aa2d07a7d3c2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.425234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b6f90-c04e-40d9-a3ce-aa2d07a7d3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.470112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ea8b6f90-c04e-40d9-a3ce-aa2d07a7d3c2 None None] MFA Rules not processed for user `29b9ba554172470383d8241107eb8651`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:09.471737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ea8b6f90-c04e-40d9-a3ce-aa2d07a7d3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:09.484357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28741c49cf9427fa5b670e2c979acc3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:09.495699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28741c49cf9427fa5b670e2c979acc3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:09.503680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28741c49cf9427fa5b670e2c979acc3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:09.521730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28741c49cf9427fa5b670e2c979acc3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:09.522528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28741c49cf9427fa5b670e2c979acc3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:09.529622 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1755/3479] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:09 2022] DELETE /identity/v3/users/d28741c49cf9427fa5b670e2c979acc3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:09.532741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:09.532741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:09.565127 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1725/3480] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:09.566335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b6f90-c04e-40d9-a3ce-aa2d07a7d3c2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:09.568339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ea8b6f90-c04e-40d9-a3ce-aa2d07a7d3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:09.594866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b6f90-c04e-40d9-a3ce-aa2d07a7d3c2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:09.613915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96714859-cbe0-437a-b121-a35443de1e23 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:09.620706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efc1e7-9b14-4744-95d6-a103c361ee11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:09.621597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efc1e7-9b14-4744-95d6-a103c361ee11 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.622104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efc1e7-9b14-4744-95d6-a103c361ee11 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.622565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efc1e7-9b14-4744-95d6-a103c361ee11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.624951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-93efc1e7-9b14-4744-95d6-a103c361ee11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:09.663550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1342ff7-0564-4cd5-a523-52c653183f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:09.664641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1342ff7-0564-4cd5-a523-52c653183f48 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.665115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1342ff7-0564-4cd5-a523-52c653183f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.665438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1342ff7-0564-4cd5-a523-52c653183f48 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.666127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1342ff7-0564-4cd5-a523-52c653183f48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:09.669531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1342ff7-0564-4cd5-a523-52c653183f48 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:09.710944 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1726/3481] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:09 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:09.712308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da0293-8012-46a3-8fb9-80754c2309e3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.712614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da0293-8012-46a3-8fb9-80754c2309e3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.712885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da0293-8012-46a3-8fb9-80754c2309e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.723092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efc1e7-9b14-4744-95d6-a103c361ee11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:09.724963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efc1e7-9b14-4744-95d6-a103c361ee11 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:09.737873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-88da0293-8012-46a3-8fb9-80754c2309e3 None None] MFA Rules not processed for user `29b9ba554172470383d8241107eb8651`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:09.739244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-88da0293-8012-46a3-8fb9-80754c2309e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:09.746382 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1756/3482] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:09 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:09.747473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efc1e7-9b14-4744-95d6-a103c361ee11 service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:09.755270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:09.773297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efc1e7-9b14-4744-95d6-a103c361ee11 service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:09.828956 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1727/3483] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:09.830278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da0293-8012-46a3-8fb9-80754c2309e3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:09.831803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-88da0293-8012-46a3-8fb9-80754c2309e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:09.832899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:09.864906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae849c-ffdd-4e1a-82fd-de28220c6940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:09.865611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae849c-ffdd-4e1a-82fd-de28220c6940 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.865882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae849c-ffdd-4e1a-82fd-de28220c6940 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.866163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae849c-ffdd-4e1a-82fd-de28220c6940 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.866754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae849c-ffdd-4e1a-82fd-de28220c6940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:09.868899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae849c-ffdd-4e1a-82fd-de28220c6940 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:09.901419 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1757/3484] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:09 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:09.902557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae849c-ffdd-4e1a-82fd-de28220c6940 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:09.916814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da0293-8012-46a3-8fb9-80754c2309e3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:09.917395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae849c-ffdd-4e1a-82fd-de28220c6940 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:09.922504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffdaa8-0acc-47cf-a715-b1f086514444 tempest-TrustsV3TestJSON-1806530054 tempest-user-152848166] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b9ba554172470383d8241107eb8651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fedb70c317e452ca0475ace9de369c0', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-128772085', 'tempest-DelegatedRole-1330382774'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:09.922637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:09.923124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffdaa8-0acc-47cf-a715-b1f086514444 tempest-TrustsV3TestJSON-1806530054 tempest-user-152848166] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.923248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:09.923451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.923552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffdaa8-0acc-47cf-a715-b1f086514444 tempest-TrustsV3TestJSON-1806530054 tempest-user-152848166] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:09.923702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] PATH_INFO: `/v3/projects/d16b4d4dafe34c55aa270a0b3dd149bd` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.923795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffdaa8-0acc-47cf-a715-b1f086514444 tempest-TrustsV3TestJSON-1806530054 tempest-user-152848166] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:09.924698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffdaa8-0acc-47cf-a715-b1f086514444 tempest-TrustsV3TestJSON-1806530054 tempest-user-152848166] RBAC: Authorizing `identity:create_trust()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:09.925053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d16b4d4dafe34c55aa270a0b3dd149bd)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:09.925911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffdaa8-0acc-47cf-a715-b1f086514444 tempest-TrustsV3TestJSON-1806530054 tempest-user-152848166] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:09.926955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:09.986450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd16b4d4dafe34c55aa270a0b3dd149bd', 'request_id': 'req-59b49296-fa95-4615-b726-00015d13a8f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:09.995318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd16b4d4dafe34c55aa270a0b3dd149bd', 'request_id': 'req-59b49296-fa95-4615-b726-00015d13a8f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:10.012291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd16b4d4dafe34c55aa270a0b3dd149bd', 'request_id': 'req-59b49296-fa95-4615-b726-00015d13a8f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-edffdaa8-0acc-47cf-a715-b1f086514444 tempest-TrustsV3TestJSON-1806530054 tempest-user-152848166] Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 44, in parse_isotime Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return iso8601.parse_date(timestr) Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/iso8601/iso8601.py", line 127, in parse_date Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise ParseError(f"Unable to parse date string {datestring!r}") Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application iso8601.iso8601.ParseError: Unable to parse date string 'bad.123Z' Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 346, in parse_expiration_date Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application expiration_time = timeutils.parse_isotime(expiration_date) Aug 20 21:05:10.020532 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 46, in parse_isotime Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise ValueError(str(e)) Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ValueError: Unable to parse date string 'bad.123Z' Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 296, in post Aug 20 21:05:10.021574 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application trust['expires_at'] = self._parse_expiration_date( Aug 20 21:05:10.022753 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 127, in _parse_expiration_date Aug 20 21:05:10.022753 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return utils.parse_expiration_date(expiration_date) Aug 20 21:05:10.022753 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 348, in parse_expiration_date Aug 20 21:05:10.022753 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ValidationTimeStampError() Aug 20 21:05:10.022753 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 20 21:05:10.022753 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:10.022753 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1758/3485] 10.222.0.23 () {60 vars in 1174 bytes} [Sat Aug 20 21:05:09 2022] POST /identity/v3/OS-TRUST/trusts => generated 227 bytes in 120 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 20 21:05:10.023247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffdaa8-0acc-47cf-a715-b1f086514444 tempest-TrustsV3TestJSON-1806530054 tempest-user-152848166] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:10.046902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] The token cache is being invalidate because project d16b4d4dafe34c55aa270a0b3dd149bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:10.047197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:10.050599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:10.051018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffdaa8-0acc-47cf-a715-b1f086514444 tempest-TrustsV3TestJSON-1806530054 tempest-user-152848166] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:10.056737 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1728/3486] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:09 2022] DELETE /identity/v3/projects/d16b4d4dafe34c55aa270a0b3dd149bd => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:10.059653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:10.063703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:10.143203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:10.149180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bdd8d2-4665-42e7-b2cd-0498bcbc1a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:10.151442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bdd8d2-4665-42e7-b2cd-0498bcbc1a14 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:10.151783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bdd8d2-4665-42e7-b2cd-0498bcbc1a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:10.152643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bdd8d2-4665-42e7-b2cd-0498bcbc1a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:10.153360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bdd8d2-4665-42e7-b2cd-0498bcbc1a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:10.156078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bdd8d2-4665-42e7-b2cd-0498bcbc1a14 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:10.167552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b49296-fa95-4615-b726-00015d13a8f2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:10.174201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d15b0-e769-4cb7-929d-1148f5750fa6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:10.175006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d15b0-e769-4cb7-929d-1148f5750fa6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:10.175347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d15b0-e769-4cb7-929d-1148f5750fa6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:10.175686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d15b0-e769-4cb7-929d-1148f5750fa6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles/0a900e13aa1548cf8b67df63ee2cbf71` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:10.177884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d15b0-e769-4cb7-929d-1148f5750fa6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_role(role_id=0a900e13aa1548cf8b67df63ee2cbf71)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:10.182226 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1759/3487] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:10 2022] GET /identity/v3/roles => generated 1892 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:10.182403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d15b0-e769-4cb7-929d-1148f5750fa6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:10.187005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f2fbe-70b4-439e-8c55-b0a0caafdf9d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:10.188142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f2fbe-70b4-439e-8c55-b0a0caafdf9d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:10.188410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f2fbe-70b4-439e-8c55-b0a0caafdf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:10.233293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bb3f2fbe-70b4-439e-8c55-b0a0caafdf9d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:10.234743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bb3f2fbe-70b4-439e-8c55-b0a0caafdf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:10.244942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2d4d15b0-e769-4cb7-929d-1148f5750fa6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invalidating the token cache because role 0a900e13aa1548cf8b67df63ee2cbf71 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:10.245500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2d4d15b0-e769-4cb7-929d-1148f5750fa6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:10.246392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2d4d15b0-e769-4cb7-929d-1148f5750fa6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:10.248925 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1729/3488] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:10 2022] DELETE /identity/v3/roles/0a900e13aa1548cf8b67df63ee2cbf71 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:10.249991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d15b0-e769-4cb7-929d-1148f5750fa6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:10.251611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2d4d15b0-e769-4cb7-929d-1148f5750fa6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:10.252659 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1760/3489] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:10.254035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f2fbe-70b4-439e-8c55-b0a0caafdf9d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:10.274779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f2fbe-70b4-439e-8c55-b0a0caafdf9d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:10.375037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d15b0-e769-4cb7-929d-1148f5750fa6 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:10.388298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237ff67-6082-48f7-b8d3-57b0c3e2b45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:10.389003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237ff67-6082-48f7-b8d3-57b0c3e2b45c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:10.390629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237ff67-6082-48f7-b8d3-57b0c3e2b45c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:10.391035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237ff67-6082-48f7-b8d3-57b0c3e2b45c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:10.391700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237ff67-6082-48f7-b8d3-57b0c3e2b45c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:10.393393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237ff67-6082-48f7-b8d3-57b0c3e2b45c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:10.395036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:10.406051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9fe7b-99e8-4ecc-8e38-9cde0b7f1534 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:10.406950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9fe7b-99e8-4ecc-8e38-9cde0b7f1534 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:10.407439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9fe7b-99e8-4ecc-8e38-9cde0b7f1534 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:10.407723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9fe7b-99e8-4ecc-8e38-9cde0b7f1534 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles/aa7d5638250d4706b8ae3c88f565f449` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:10.410669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9fe7b-99e8-4ecc-8e38-9cde0b7f1534 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_role(role_id=aa7d5638250d4706b8ae3c88f565f449)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:10.412474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9fe7b-99e8-4ecc-8e38-9cde0b7f1534 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:10.444877 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1730/3490] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:10 2022] GET /identity/v3/roles => generated 1655 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:10.447886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b2fda-c309-4d76-9c0b-ef59b197576f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:10.448167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b2fda-c309-4d76-9c0b-ef59b197576f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:10.455967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b2fda-c309-4d76-9c0b-ef59b197576f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:10.465665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9fa9fe7b-99e8-4ecc-8e38-9cde0b7f1534 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invalidating the token cache because role aa7d5638250d4706b8ae3c88f565f449 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:10.466172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9fa9fe7b-99e8-4ecc-8e38-9cde0b7f1534 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:10.466955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9fa9fe7b-99e8-4ecc-8e38-9cde0b7f1534 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:10.472592 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1761/3491] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:10 2022] DELETE /identity/v3/roles/aa7d5638250d4706b8ae3c88f565f449 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:10.472592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9fe7b-99e8-4ecc-8e38-9cde0b7f1534 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:10.472592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9fa9fe7b-99e8-4ecc-8e38-9cde0b7f1534 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:10.508027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-028b2fda-c309-4d76-9c0b-ef59b197576f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:10.509991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-028b2fda-c309-4d76-9c0b-ef59b197576f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:10.583409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9fe7b-99e8-4ecc-8e38-9cde0b7f1534 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:10.590561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38860994-a86e-4996-a4e6-606c7b424a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:10.591331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38860994-a86e-4996-a4e6-606c7b424a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:10.591818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38860994-a86e-4996-a4e6-606c7b424a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:10.592156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38860994-a86e-4996-a4e6-606c7b424a54 admin admin] PATH_INFO: `/v3/projects/c58f159460cd4bcd8327c5b5ed84b1fe/users/c8b2c80165614ad58df38edc24b47715/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:10.609525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38860994-a86e-4996-a4e6-606c7b424a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58f159460cd4bcd8327c5b5ed84b1fe, user_id=c8b2c80165614ad58df38edc24b47715, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:10.614509 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1731/3492] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:10.615525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b2fda-c309-4d76-9c0b-ef59b197576f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:10.616903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-028b2fda-c309-4d76-9c0b-ef59b197576f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:10.617541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38860994-a86e-4996-a4e6-606c7b424a54 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:10.635250 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1762/3493] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:10 2022] PUT /identity/v3/projects/c58f159460cd4bcd8327c5b5ed84b1fe/users/c8b2c80165614ad58df38edc24b47715/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:10.636190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38860994-a86e-4996-a4e6-606c7b424a54 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:10.671748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:10.688693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b2fda-c309-4d76-9c0b-ef59b197576f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:10.715697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38860994-a86e-4996-a4e6-606c7b424a54 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:10.721163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327479d-5ba7-4fe6-8e3d-db05c0f57ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:10.721505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327479d-5ba7-4fe6-8e3d-db05c0f57ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:10.721781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327479d-5ba7-4fe6-8e3d-db05c0f57ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:10.722045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327479d-5ba7-4fe6-8e3d-db05c0f57ac2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:10.722738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327479d-5ba7-4fe6-8e3d-db05c0f57ac2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:05:10.722981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327479d-5ba7-4fe6-8e3d-db05c0f57ac2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:10.724681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327479d-5ba7-4fe6-8e3d-db05c0f57ac2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:10.743061 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1763/3494] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:10.743061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767930c7-ea8c-498c-98e1-c371a33623ed None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:10.743061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767930c7-ea8c-498c-98e1-c371a33623ed None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:10.743061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767930c7-ea8c-498c-98e1-c371a33623ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:10.757508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:10.758540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:10.758880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:10.759655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users/29b9ba554172470383d8241107eb8651` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:10.760570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_user(user_id=29b9ba554172470383d8241107eb8651)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:10.762675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:10.762989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-767930c7-ea8c-498c-98e1-c371a33623ed None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:10.764937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-767930c7-ea8c-498c-98e1-c371a33623ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:10.775273 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1764/3495] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:10.779016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767930c7-ea8c-498c-98e1-c371a33623ed None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:10.790013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767930c7-ea8c-498c-98e1-c371a33623ed None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:10.794629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed2c62-38f8-4759-803c-3c9fd5785f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:10.795272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed2c62-38f8-4759-803c-3c9fd5785f03 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:10.795851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed2c62-38f8-4759-803c-3c9fd5785f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:10.795851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed2c62-38f8-4759-803c-3c9fd5785f03 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:10.796441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed2c62-38f8-4759-803c-3c9fd5785f03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:10.798456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed2c62-38f8-4759-803c-3c9fd5785f03 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:10.816662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29b9ba554172470383d8241107eb8651'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:10.827002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29b9ba554172470383d8241107eb8651'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:10.833729 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1765/3496] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:10 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:10.833729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed2c62-38f8-4759-803c-3c9fd5785f03 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:10.833729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-91ed2c62-38f8-4759-803c-3c9fd5785f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:10.847713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29b9ba554172470383d8241107eb8651'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:10.848459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:10.848579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29b9ba554172470383d8241107eb8651'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:10.849126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29b9ba554172470383d8241107eb8651'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:10.864667 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1732/3497] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:10 2022] DELETE /identity/v3/users/29b9ba554172470383d8241107eb8651 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:10.864667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:10.864667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:10.902781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed2c62-38f8-4759-803c-3c9fd5785f03 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:10.955354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c97e1f-fe6e-42e4-a044-ec8d8c442bee tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:10.963417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8642d-502e-4bf2-98a4-6d2a411f26cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:10.963647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd9787-a6aa-4c0f-a7b7-6f86bcad9145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:10.964194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8642d-502e-4bf2-98a4-6d2a411f26cd service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:10.964573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8642d-502e-4bf2-98a4-6d2a411f26cd service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:10.964702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd9787-a6aa-4c0f-a7b7-6f86bcad9145 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:10.964961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8642d-502e-4bf2-98a4-6d2a411f26cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:10.965069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd9787-a6aa-4c0f-a7b7-6f86bcad9145 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:10.965703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd9787-a6aa-4c0f-a7b7-6f86bcad9145 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:10.966541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd9787-a6aa-4c0f-a7b7-6f86bcad9145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:10.966984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4cb8642d-502e-4bf2-98a4-6d2a411f26cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:10.969504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd9787-a6aa-4c0f-a7b7-6f86bcad9145 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:10.983985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8642d-502e-4bf2-98a4-6d2a411f26cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:10.988689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8642d-502e-4bf2-98a4-6d2a411f26cd service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:11.012898 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1766/3498] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:11.012898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8642d-502e-4bf2-98a4-6d2a411f26cd service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:11.012898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4cb8642d-502e-4bf2-98a4-6d2a411f26cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:11.013300 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1733/3499] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:10 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:11.019305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd9787-a6aa-4c0f-a7b7-6f86bcad9145 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:11.032227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd9787-a6aa-4c0f-a7b7-6f86bcad9145 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:11.038829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da15f1-37b6-4671-9b75-469be025a647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:11.040728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da15f1-37b6-4671-9b75-469be025a647 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.041009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da15f1-37b6-4671-9b75-469be025a647 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.041413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da15f1-37b6-4671-9b75-469be025a647 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.042166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da15f1-37b6-4671-9b75-469be025a647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:11.044544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da15f1-37b6-4671-9b75-469be025a647 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:11.046190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:11.059723 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1734/3500] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:11.061034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18edfa5-be43-4faf-a987-0a0a1ea37b48 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.061323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18edfa5-be43-4faf-a987-0a0a1ea37b48 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.061595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18edfa5-be43-4faf-a987-0a0a1ea37b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.084220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a18edfa5-be43-4faf-a987-0a0a1ea37b48 None None] MFA Rules not processed for user `bbc87613dd6349babc15334a66bee64f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:11.085407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a18edfa5-be43-4faf-a987-0a0a1ea37b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:11.120923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8642d-502e-4bf2-98a4-6d2a411f26cd service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:11.126919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:11.127808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.128181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.129362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/9fedb70c317e452ca0475ace9de369c0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.131954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9fedb70c317e452ca0475ace9de369c0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:11.135371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:11.153547 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1735/3501] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:11.153547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18edfa5-be43-4faf-a987-0a0a1ea37b48 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:11.186849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18edfa5-be43-4faf-a987-0a0a1ea37b48 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:11.209967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fedb70c317e452ca0475ace9de369c0', 'request_id': 'req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:11.224509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fedb70c317e452ca0475ace9de369c0', 'request_id': 'req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:11.231572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fedb70c317e452ca0475ace9de369c0', 'request_id': 'req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:11.240874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:11.246339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] The token cache is being invalidate because project 9fedb70c317e452ca0475ace9de369c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:11.246882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:11.247940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:11.249909 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1767/3502] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:11 2022] DELETE /identity/v3/projects/9fedb70c317e452ca0475ace9de369c0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:11.255401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:11.259979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:11.298151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45dba1-8069-44f0-907e-bb5fe11ccf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:11.298873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45dba1-8069-44f0-907e-bb5fe11ccf13 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.299239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45dba1-8069-44f0-907e-bb5fe11ccf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.300067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45dba1-8069-44f0-907e-bb5fe11ccf13 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.300830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45dba1-8069-44f0-907e-bb5fe11ccf13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:11.302603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45dba1-8069-44f0-907e-bb5fe11ccf13 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:11.314951 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1736/3503] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:11.318982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45dba1-8069-44f0-907e-bb5fe11ccf13 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:11.321434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3c45dba1-8069-44f0-907e-bb5fe11ccf13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:11.334553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45dba1-8069-44f0-907e-bb5fe11ccf13 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:11.339332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e6c7d-bc5e-4921-aed8-6329e77f9dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:11.340066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e6c7d-bc5e-4921-aed8-6329e77f9dfb admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.340480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e6c7d-bc5e-4921-aed8-6329e77f9dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.340851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e6c7d-bc5e-4921-aed8-6329e77f9dfb admin admin] PATH_INFO: `/v3/projects/ac3507f6a20140959a61eac7fa4cf2c5/users/647583dffce54234bcf9276d4a61f80e/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.341152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e1e3a-07c5-405c-9577-6bb45c124fc7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:11.346061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacda97-a5fc-49a4-bf8f-c9787b8d1f4a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:11.346663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacda97-a5fc-49a4-bf8f-c9787b8d1f4a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.346936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacda97-a5fc-49a4-bf8f-c9787b8d1f4a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.347335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacda97-a5fc-49a4-bf8f-c9787b8d1f4a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.347952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacda97-a5fc-49a4-bf8f-c9787b8d1f4a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:11.349915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacda97-a5fc-49a4-bf8f-c9787b8d1f4a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:11.359029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e6c7d-bc5e-4921-aed8-6329e77f9dfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3507f6a20140959a61eac7fa4cf2c5, user_id=647583dffce54234bcf9276d4a61f80e, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:11.361869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e6c7d-bc5e-4921-aed8-6329e77f9dfb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:11.381444 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1737/3504] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:11 2022] PUT /identity/v3/projects/ac3507f6a20140959a61eac7fa4cf2c5/users/647583dffce54234bcf9276d4a61f80e/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:11.383134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c70a8-ba46-4e68-9e32-e370d38acdb0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.383651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c70a8-ba46-4e68-9e32-e370d38acdb0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.384056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c70a8-ba46-4e68-9e32-e370d38acdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.387883 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1768/3505] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:05:11 2022] POST /identity/v3/projects => generated 329 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:11.393299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacda97-a5fc-49a4-bf8f-c9787b8d1f4a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:11.418236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-179c70a8-ba46-4e68-9e32-e370d38acdb0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:11.419634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-179c70a8-ba46-4e68-9e32-e370d38acdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:11.475637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacda97-a5fc-49a4-bf8f-c9787b8d1f4a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:11.486235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a526e-9a4e-4f34-a941-5c256b52343d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:11.486873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a526e-9a4e-4f34-a941-5c256b52343d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.487131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a526e-9a4e-4f34-a941-5c256b52343d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.487398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a526e-9a4e-4f34-a941-5c256b52343d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.488304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a526e-9a4e-4f34-a941-5c256b52343d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:11.490992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a526e-9a4e-4f34-a941-5c256b52343d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:11.523968 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1769/3506] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:11 2022] POST /identity/v3/users => generated 416 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:11.525243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a526e-9a4e-4f34-a941-5c256b52343d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:11.529701 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1738/3507] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:11.539373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a526e-9a4e-4f34-a941-5c256b52343d tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:11.545289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b633c-46ba-4e8f-981b-0d2a3899e0fa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:11.546114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b633c-46ba-4e8f-981b-0d2a3899e0fa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.546462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b633c-46ba-4e8f-981b-0d2a3899e0fa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.546813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b633c-46ba-4e8f-981b-0d2a3899e0fa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.548268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b633c-46ba-4e8f-981b-0d2a3899e0fa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:11.550263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b633c-46ba-4e8f-981b-0d2a3899e0fa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:11.574233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c70a8-ba46-4e68-9e32-e370d38acdb0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:11.576514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-179c70a8-ba46-4e68-9e32-e370d38acdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:11.592202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:11.599570 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1770/3508] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:05:11 2022] POST /identity/v3/roles => generated 242 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:11.603773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b633c-46ba-4e8f-981b-0d2a3899e0fa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:11.632033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b633c-46ba-4e8f-981b-0d2a3899e0fa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:11.640683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4f7c2-f04f-4c6b-9a69-ce5ba35610f7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:11.641417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4f7c2-f04f-4c6b-9a69-ce5ba35610f7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.641702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4f7c2-f04f-4c6b-9a69-ce5ba35610f7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.641981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4f7c2-f04f-4c6b-9a69-ce5ba35610f7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.643294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4f7c2-f04f-4c6b-9a69-ce5ba35610f7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:11.648002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4f7c2-f04f-4c6b-9a69-ce5ba35610f7 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:11.651374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c70a8-ba46-4e68-9e32-e370d38acdb0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:11.658711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4f046-0f76-4b0f-8ed5-5aacb0d93cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:11.659376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4f046-0f76-4b0f-8ed5-5aacb0d93cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.659650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4f046-0f76-4b0f-8ed5-5aacb0d93cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.659962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4f046-0f76-4b0f-8ed5-5aacb0d93cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.678493 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1771/3509] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:05:11 2022] POST /identity/v3/roles => generated 247 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:11.680403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f3563-99ca-4c73-9406-a8b992aec781 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.683533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f3563-99ca-4c73-9406-a8b992aec781 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.683814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f3563-99ca-4c73-9406-a8b992aec781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.687726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4f046-0f76-4b0f-8ed5-5aacb0d93cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:11.690169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4f046-0f76-4b0f-8ed5-5aacb0d93cb4 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:11.712503 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1739/3510] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:11.715077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4f046-0f76-4b0f-8ed5-5aacb0d93cb4 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:11.717020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-de5f3563-99ca-4c73-9406-a8b992aec781 None None] MFA Rules not processed for user `b20aea7088dc4e4fadb737c906cf7ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:11.718280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-de5f3563-99ca-4c73-9406-a8b992aec781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:11.732916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4f046-0f76-4b0f-8ed5-5aacb0d93cb4 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:11.740300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1dda4-6108-4c11-a5e3-217de433395f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:11.741336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1dda4-6108-4c11-a5e3-217de433395f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.741764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1dda4-6108-4c11-a5e3-217de433395f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.742787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1dda4-6108-4c11-a5e3-217de433395f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/ed322fa3f340471ca755df35190850e1/users/e892b6baadd54dc29f1b30ccd9f82b15/roles/a22255977a0e42bf959fa3e1bbe3bc81` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.768219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1dda4-6108-4c11-a5e3-217de433395f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_grant(project_id=ed322fa3f340471ca755df35190850e1, user_id=e892b6baadd54dc29f1b30ccd9f82b15, role_id=a22255977a0e42bf959fa3e1bbe3bc81)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:11.772390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1dda4-6108-4c11-a5e3-217de433395f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:11.780931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:11.795310 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1772/3511] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:11.798003 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1740/3512] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:11 2022] PUT /identity/v3/projects/ed322fa3f340471ca755df35190850e1/users/e892b6baadd54dc29f1b30ccd9f82b15/roles/a22255977a0e42bf959fa3e1bbe3bc81 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:11.801796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f3563-99ca-4c73-9406-a8b992aec781 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:11.817151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f3563-99ca-4c73-9406-a8b992aec781 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:11.897343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f8fea-9c2d-430e-90b5-f0136760b8e0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:11.899010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f8fea-9c2d-430e-90b5-f0136760b8e0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:11.899307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f8fea-9c2d-430e-90b5-f0136760b8e0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:11.899580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f8fea-9c2d-430e-90b5-f0136760b8e0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/ed322fa3f340471ca755df35190850e1/users/e892b6baadd54dc29f1b30ccd9f82b15/roles/e50d2f8c21ac40fb9c01cde8b5d2b8b5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:11.902281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108f8fea-9c2d-430e-90b5-f0136760b8e0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_grant(project_id=ed322fa3f340471ca755df35190850e1, user_id=e892b6baadd54dc29f1b30ccd9f82b15, role_id=e50d2f8c21ac40fb9c01cde8b5d2b8b5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:11.905196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108f8fea-9c2d-430e-90b5-f0136760b8e0 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:11.921792 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1773/3513] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:11 2022] PUT /identity/v3/projects/ed322fa3f340471ca755df35190850e1/users/e892b6baadd54dc29f1b30ccd9f82b15/roles/e50d2f8c21ac40fb9c01cde8b5d2b8b5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:11.925740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1dda4-6108-4c11-a5e3-217de433395f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:11.951880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1dda4-6108-4c11-a5e3-217de433395f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:12.099920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc3cf1-4046-4ce4-824c-20c57ff7fac5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:12.100970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc3cf1-4046-4ce4-824c-20c57ff7fac5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:12.102094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc3cf1-4046-4ce4-824c-20c57ff7fac5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:12.103114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc3cf1-4046-4ce4-824c-20c57ff7fac5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:12.105768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc3cf1-4046-4ce4-824c-20c57ff7fac5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:12.110125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc3cf1-4046-4ce4-824c-20c57ff7fac5 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:12.208649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1331f5b-d462-40de-97f3-d1adb76eba85 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:12.208924 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1741/3514] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:11 2022] GET /identity/v3/users => generated 8640 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:12.209056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1331f5b-d462-40de-97f3-d1adb76eba85 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:12.209213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1331f5b-d462-40de-97f3-d1adb76eba85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:12.306943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c1331f5b-d462-40de-97f3-d1adb76eba85 None None] MFA Rules not processed for user `e892b6baadd54dc29f1b30ccd9f82b15`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:12.310778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c1331f5b-d462-40de-97f3-d1adb76eba85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:12.510160 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1774/3515] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:12.518090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a5434-e2c7-43f1-b571-3ddc68583df4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:12.518591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a5434-e2c7-43f1-b571-3ddc68583df4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:12.519004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a5434-e2c7-43f1-b571-3ddc68583df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:12.567700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-611a5434-e2c7-43f1-b571-3ddc68583df4 None None] MFA Rules not processed for user `e892b6baadd54dc29f1b30ccd9f82b15`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:12.572521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-611a5434-e2c7-43f1-b571-3ddc68583df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:12.597998 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1742/3516] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:12.598450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1331f5b-d462-40de-97f3-d1adb76eba85 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:12.612466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1331f5b-d462-40de-97f3-d1adb76eba85 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:12.624558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff32842-9622-4eb1-9c2a-e92ecc548d85 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e892b6baadd54dc29f1b30ccd9f82b15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed322fa3f340471ca755df35190850e1', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1550119992', 'tempest-DelegatedRole-33718875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:12.624938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff32842-9622-4eb1-9c2a-e92ecc548d85 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:12.625179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff32842-9622-4eb1-9c2a-e92ecc548d85 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:12.625488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff32842-9622-4eb1-9c2a-e92ecc548d85 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:12.626126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff32842-9622-4eb1-9c2a-e92ecc548d85 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorizing `identity:create_trust()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:12.632548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff32842-9622-4eb1-9c2a-e92ecc548d85 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:12.769162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:12.793782 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1775/3517] 10.222.0.23 () {60 vars in 1174 bytes} [Sat Aug 20 21:05:12 2022] POST /identity/v3/OS-TRUST/trusts => generated 832 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:12.798411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a5434-e2c7-43f1-b571-3ddc68583df4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:12.809602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a5434-e2c7-43f1-b571-3ddc68583df4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:12.823988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d570d-1a6d-4ef2-afe4-57b3aea9e5ea tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e892b6baadd54dc29f1b30ccd9f82b15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed322fa3f340471ca755df35190850e1', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1550119992', 'tempest-DelegatedRole-33718875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:12.824734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d570d-1a6d-4ef2-afe4-57b3aea9e5ea tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:12.825013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d570d-1a6d-4ef2-afe4-57b3aea9e5ea tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:12.825335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d570d-1a6d-4ef2-afe4-57b3aea9e5ea tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] PATH_INFO: `/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:12.841185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d570d-1a6d-4ef2-afe4-57b3aea9e5ea tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorizing `identity:get_trust(trust_id=1f3c1cdfcf4a41ac8f8a63a932a5fea9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:12.846108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d570d-1a6d-4ef2-afe4-57b3aea9e5ea tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:12.869946 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1743/3518] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:05:12 2022] GET /identity/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9 => generated 832 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:12.873843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff32842-9622-4eb1-9c2a-e92ecc548d85 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:12.887817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff32842-9622-4eb1-9c2a-e92ecc548d85 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:12.901551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bf93e-902c-4339-af72-5fec31f51765 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e892b6baadd54dc29f1b30ccd9f82b15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed322fa3f340471ca755df35190850e1', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1550119992', 'tempest-DelegatedRole-33718875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:12.903909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bf93e-902c-4339-af72-5fec31f51765 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:12.904226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bf93e-902c-4339-af72-5fec31f51765 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:12.905688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bf93e-902c-4339-af72-5fec31f51765 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] PATH_INFO: `/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:12.921573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bf93e-902c-4339-af72-5fec31f51765 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=1f3c1cdfcf4a41ac8f8a63a932a5fea9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:12.925720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bf93e-902c-4339-af72-5fec31f51765 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:12.940601 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1776/3519] 10.222.0.23 () {58 vars in 1269 bytes} [Sat Aug 20 21:05:12 2022] GET /identity/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9/roles => generated 369 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:12.944530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d570d-1a6d-4ef2-afe4-57b3aea9e5ea tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:12.957175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d570d-1a6d-4ef2-afe4-57b3aea9e5ea tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:12.963992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfe578-2a60-4067-a7f0-ef568836fba7 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e892b6baadd54dc29f1b30ccd9f82b15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed322fa3f340471ca755df35190850e1', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1550119992', 'tempest-DelegatedRole-33718875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:12.965415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfe578-2a60-4067-a7f0-ef568836fba7 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:12.965922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfe578-2a60-4067-a7f0-ef568836fba7 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:12.966230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfe578-2a60-4067-a7f0-ef568836fba7 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] PATH_INFO: `/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9/roles/a22255977a0e42bf959fa3e1bbe3bc81` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:12.973123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cfe578-2a60-4067-a7f0-ef568836fba7 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorizing `identity:get_role_for_trust(trust_id=1f3c1cdfcf4a41ac8f8a63a932a5fea9, role_id=a22255977a0e42bf959fa3e1bbe3bc81)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:12.975121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cfe578-2a60-4067-a7f0-ef568836fba7 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:12.984496 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1744/3520] 10.222.0.23 () {58 vars in 1368 bytes} [Sat Aug 20 21:05:12 2022] GET /identity/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9/roles/a22255977a0e42bf959fa3e1bbe3bc81 => generated 242 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:12.988062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bf93e-902c-4339-af72-5fec31f51765 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:13.000246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bf93e-902c-4339-af72-5fec31f51765 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:13.006569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440742ab-7085-4312-aa72-047f55da87b9 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e892b6baadd54dc29f1b30ccd9f82b15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed322fa3f340471ca755df35190850e1', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1550119992', 'tempest-DelegatedRole-33718875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:13.007779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440742ab-7085-4312-aa72-047f55da87b9 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] REQUEST_METHOD: `HEAD` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:13.008055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440742ab-7085-4312-aa72-047f55da87b9 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:13.008320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440742ab-7085-4312-aa72-047f55da87b9 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] PATH_INFO: `/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9/roles/a22255977a0e42bf959fa3e1bbe3bc81` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:13.018066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440742ab-7085-4312-aa72-047f55da87b9 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorizing `identity:get_role_for_trust(trust_id=1f3c1cdfcf4a41ac8f8a63a932a5fea9, role_id=a22255977a0e42bf959fa3e1bbe3bc81)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:13.019675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440742ab-7085-4312-aa72-047f55da87b9 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:13.033496 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1777/3521] 10.222.0.23 () {58 vars in 1369 bytes} [Sat Aug 20 21:05:12 2022] HEAD /identity/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9/roles/a22255977a0e42bf959fa3e1bbe3bc81 => generated 0 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Aug 20 21:05:13.039402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfe578-2a60-4067-a7f0-ef568836fba7 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:13.053764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfe578-2a60-4067-a7f0-ef568836fba7 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:13.060207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a8013-8ab2-409b-ba24-b1a5039d40f8 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e892b6baadd54dc29f1b30ccd9f82b15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed322fa3f340471ca755df35190850e1', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1550119992', 'tempest-DelegatedRole-33718875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:13.061314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a8013-8ab2-409b-ba24-b1a5039d40f8 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:13.061413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a8013-8ab2-409b-ba24-b1a5039d40f8 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:13.061676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a8013-8ab2-409b-ba24-b1a5039d40f8 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] PATH_INFO: `/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9/roles/e50d2f8c21ac40fb9c01cde8b5d2b8b5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:13.070118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a8013-8ab2-409b-ba24-b1a5039d40f8 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorizing `identity:get_role_for_trust(trust_id=1f3c1cdfcf4a41ac8f8a63a932a5fea9, role_id=e50d2f8c21ac40fb9c01cde8b5d2b8b5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:13.071872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a8013-8ab2-409b-ba24-b1a5039d40f8 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-373a8013-8ab2-409b-ba24-b1a5039d40f8 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Could not find role: e50d2f8c21ac40fb9c01cde8b5d2b8b5.: keystone.exception.RoleNotFound: Could not find role: e50d2f8c21ac40fb9c01cde8b5d2b8b5. Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 21:05:13.087459 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: e50d2f8c21ac40fb9c01cde8b5d2b8b5. Aug 20 21:05:13.088482 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:05:13.089249 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1745/3522] 10.222.0.23 () {58 vars in 1368 bytes} [Sat Aug 20 21:05:13 2022] GET /identity/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9/roles/e50d2f8c21ac40fb9c01cde8b5d2b8b5 => generated 110 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:05:13.093393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440742ab-7085-4312-aa72-047f55da87b9 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:13.106075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440742ab-7085-4312-aa72-047f55da87b9 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:13.111313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976f03d-a6a7-4d19-842b-d4319ceb314d tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e892b6baadd54dc29f1b30ccd9f82b15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed322fa3f340471ca755df35190850e1', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1550119992', 'tempest-DelegatedRole-33718875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:13.112607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976f03d-a6a7-4d19-842b-d4319ceb314d tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] REQUEST_METHOD: `HEAD` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:13.113478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976f03d-a6a7-4d19-842b-d4319ceb314d tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:13.113904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976f03d-a6a7-4d19-842b-d4319ceb314d tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] PATH_INFO: `/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9/roles/e50d2f8c21ac40fb9c01cde8b5d2b8b5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:13.123225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976f03d-a6a7-4d19-842b-d4319ceb314d tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorizing `identity:get_role_for_trust(trust_id=1f3c1cdfcf4a41ac8f8a63a932a5fea9, role_id=e50d2f8c21ac40fb9c01cde8b5d2b8b5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:13.126307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976f03d-a6a7-4d19-842b-d4319ceb314d tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-5976f03d-a6a7-4d19-842b-d4319ceb314d tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Could not find role: e50d2f8c21ac40fb9c01cde8b5d2b8b5.: keystone.exception.RoleNotFound: Could not find role: e50d2f8c21ac40fb9c01cde8b5d2b8b5. Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 21:05:13.138687 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: e50d2f8c21ac40fb9c01cde8b5d2b8b5. Aug 20 21:05:13.139700 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:13.140405 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1778/3523] 10.222.0.23 () {58 vars in 1369 bytes} [Sat Aug 20 21:05:13 2022] HEAD /identity/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9/roles/e50d2f8c21ac40fb9c01cde8b5d2b8b5 => generated 0 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:05:13.147424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a8013-8ab2-409b-ba24-b1a5039d40f8 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:13.161761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a8013-8ab2-409b-ba24-b1a5039d40f8 tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:13.180026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07755716-338a-46e9-ac96-5ca5b6aeafed tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e892b6baadd54dc29f1b30ccd9f82b15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed322fa3f340471ca755df35190850e1', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1550119992', 'tempest-DelegatedRole-33718875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:13.180951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07755716-338a-46e9-ac96-5ca5b6aeafed tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:13.181220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07755716-338a-46e9-ac96-5ca5b6aeafed tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:13.181531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07755716-338a-46e9-ac96-5ca5b6aeafed tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] PATH_INFO: `/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:13.208361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07755716-338a-46e9-ac96-5ca5b6aeafed tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorizing `identity:delete_trust(trust_id=1f3c1cdfcf4a41ac8f8a63a932a5fea9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:13.212883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07755716-338a-46e9-ac96-5ca5b6aeafed tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:13.296061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:13.309181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-07755716-338a-46e9-ac96-5ca5b6aeafed tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '1f3c1cdfcf4a41ac8f8a63a932a5fea9', 'request_id': 'req-07755716-338a-46e9-ac96-5ca5b6aeafed'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:13.309999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-07755716-338a-46e9-ac96-5ca5b6aeafed tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '1f3c1cdfcf4a41ac8f8a63a932a5fea9', 'request_id': 'req-07755716-338a-46e9-ac96-5ca5b6aeafed'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:13.310582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-07755716-338a-46e9-ac96-5ca5b6aeafed tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '1f3c1cdfcf4a41ac8f8a63a932a5fea9', 'request_id': 'req-07755716-338a-46e9-ac96-5ca5b6aeafed'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:13.360902 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1746/3524] 10.222.0.23 () {58 vars in 1254 bytes} [Sat Aug 20 21:05:13 2022] DELETE /identity/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:13.365238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976f03d-a6a7-4d19-842b-d4319ceb314d tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:13.367324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5976f03d-a6a7-4d19-842b-d4319ceb314d tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:13.403008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976f03d-a6a7-4d19-842b-d4319ceb314d tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:13.419654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69212f7-6915-44b5-822a-e1a916535a2e tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e892b6baadd54dc29f1b30ccd9f82b15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed322fa3f340471ca755df35190850e1', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1550119992', 'tempest-DelegatedRole-33718875'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:13.420506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69212f7-6915-44b5-822a-e1a916535a2e tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:13.420941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69212f7-6915-44b5-822a-e1a916535a2e tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:13.421433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69212f7-6915-44b5-822a-e1a916535a2e tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] PATH_INFO: `/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:13.430918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07755716-338a-46e9-ac96-5ca5b6aeafed tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:13.434313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-07755716-338a-46e9-ac96-5ca5b6aeafed tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-d69212f7-6915-44b5-822a-e1a916535a2e tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Could not find trust: 1f3c1cdfcf4a41ac8f8a63a932a5fea9.: keystone.exception.TrustNotFound: Could not find trust: 1f3c1cdfcf4a41ac8f8a63a932a5fea9. Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Aug 20 21:05:13.455628 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Aug 20 21:05:13.456793 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Aug 20 21:05:13.456793 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Aug 20 21:05:13.456793 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Aug 20 21:05:13.456793 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Aug 20 21:05:13.456793 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Aug 20 21:05:13.456793 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:13.456793 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:13.456793 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Aug 20 21:05:13.456793 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Aug 20 21:05:13.456793 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Aug 20 21:05:13.456793 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Aug 20 21:05:13.456793 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 1f3c1cdfcf4a41ac8f8a63a932a5fea9. Aug 20 21:05:13.456793 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:13.460742 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1779/3525] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:05:13 2022] GET /identity/v3/OS-TRUST/trusts/1f3c1cdfcf4a41ac8f8a63a932a5fea9 => generated 111 bytes in 96 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Aug 20 21:05:13.469644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69212f7-6915-44b5-822a-e1a916535a2e tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:13.479398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d69212f7-6915-44b5-822a-e1a916535a2e tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:13.500444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69212f7-6915-44b5-822a-e1a916535a2e tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:13.516648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d5410-a5fa-4d5a-8acc-990904d31a21 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:13.517449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d5410-a5fa-4d5a-8acc-990904d31a21 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:13.517923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d5410-a5fa-4d5a-8acc-990904d31a21 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:13.518937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d5410-a5fa-4d5a-8acc-990904d31a21 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles/e50d2f8c21ac40fb9c01cde8b5d2b8b5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:13.520644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d5410-a5fa-4d5a-8acc-990904d31a21 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_role(role_id=e50d2f8c21ac40fb9c01cde8b5d2b8b5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:13.522460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d5410-a5fa-4d5a-8acc-990904d31a21 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:13.631472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07755716-338a-46e9-ac96-5ca5b6aeafed tempest-TrustsV3TestJSON-1224259456 tempest-user-1541667984] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:13.634929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0a5d5410-a5fa-4d5a-8acc-990904d31a21 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invalidating the token cache because role e50d2f8c21ac40fb9c01cde8b5d2b8b5 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:13.647331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0a5d5410-a5fa-4d5a-8acc-990904d31a21 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:13.648849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f57523-7243-4a54-8cf2-9207c12716fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:13.649342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0a5d5410-a5fa-4d5a-8acc-990904d31a21 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:13.649488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f57523-7243-4a54-8cf2-9207c12716fb service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:13.649769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f57523-7243-4a54-8cf2-9207c12716fb service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:13.650075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f57523-7243-4a54-8cf2-9207c12716fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:13.654106 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1780/3526] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:13 2022] DELETE /identity/v3/roles/e50d2f8c21ac40fb9c01cde8b5d2b8b5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:13.657419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c5f57523-7243-4a54-8cf2-9207c12716fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:13.658616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d5410-a5fa-4d5a-8acc-990904d31a21 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:13.660218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0a5d5410-a5fa-4d5a-8acc-990904d31a21 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:13.687935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f57523-7243-4a54-8cf2-9207c12716fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:13.691431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f57523-7243-4a54-8cf2-9207c12716fb service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:13.723821 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1747/3527] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:05:13.774741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d5410-a5fa-4d5a-8acc-990904d31a21 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:13.795522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253f407-c651-4a7c-97cb-6211b31324a3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:13.796226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253f407-c651-4a7c-97cb-6211b31324a3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:13.796522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253f407-c651-4a7c-97cb-6211b31324a3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:13.796880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253f407-c651-4a7c-97cb-6211b31324a3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles/a22255977a0e42bf959fa3e1bbe3bc81` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:13.804199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c253f407-c651-4a7c-97cb-6211b31324a3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_role(role_id=a22255977a0e42bf959fa3e1bbe3bc81)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:13.806565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f57523-7243-4a54-8cf2-9207c12716fb service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:13.808047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c5f57523-7243-4a54-8cf2-9207c12716fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:13.837220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c253f407-c651-4a7c-97cb-6211b31324a3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:13.862340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f57523-7243-4a54-8cf2-9207c12716fb service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:13.933081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c253f407-c651-4a7c-97cb-6211b31324a3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invalidating the token cache because role a22255977a0e42bf959fa3e1bbe3bc81 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:13.933414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c253f407-c651-4a7c-97cb-6211b31324a3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:13.934131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c253f407-c651-4a7c-97cb-6211b31324a3 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:13.942280 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1781/3528] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:13 2022] DELETE /identity/v3/roles/a22255977a0e42bf959fa3e1bbe3bc81 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:13.942481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5de75-2396-4c98-86ab-f4b2041e0fad None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:13.943280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5de75-2396-4c98-86ab-f4b2041e0fad None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:13.943557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5de75-2396-4c98-86ab-f4b2041e0fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:13.989862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dfc5de75-2396-4c98-86ab-f4b2041e0fad None None] MFA Rules not processed for user `b20aea7088dc4e4fadb737c906cf7ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:13.993920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dfc5de75-2396-4c98-86ab-f4b2041e0fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:14.015381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17400a-b135-40c6-abcb-8ac8fb0c14fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:14.017335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17400a-b135-40c6-abcb-8ac8fb0c14fe service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:14.017929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17400a-b135-40c6-abcb-8ac8fb0c14fe service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:14.017929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17400a-b135-40c6-abcb-8ac8fb0c14fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:14.022677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3d17400a-b135-40c6-abcb-8ac8fb0c14fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:14.033790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:14.050160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17400a-b135-40c6-abcb-8ac8fb0c14fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:14.051651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17400a-b135-40c6-abcb-8ac8fb0c14fe service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:14.078317 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1748/3529] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:14.093174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17400a-b135-40c6-abcb-8ac8fb0c14fe service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:14.099787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3d17400a-b135-40c6-abcb-8ac8fb0c14fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:14.146336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17400a-b135-40c6-abcb-8ac8fb0c14fe service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:14.179849 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1782/3530] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:14.359649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:14.361041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:14.361729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:14.362496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users/e892b6baadd54dc29f1b30ccd9f82b15` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:14.364637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_user(user_id=e892b6baadd54dc29f1b30ccd9f82b15)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:14.367703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:14.392707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:14.462219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e892b6baadd54dc29f1b30ccd9f82b15'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:14.476333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e892b6baadd54dc29f1b30ccd9f82b15'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:14.503444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e892b6baadd54dc29f1b30ccd9f82b15'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:14.504362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e892b6baadd54dc29f1b30ccd9f82b15'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:14.509024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e892b6baadd54dc29f1b30ccd9f82b15'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:14.533415 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1749/3531] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:14 2022] DELETE /identity/v3/users/e892b6baadd54dc29f1b30ccd9f82b15 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:14.534273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5de75-2396-4c98-86ab-f4b2041e0fad None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:14.554338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5de75-2396-4c98-86ab-f4b2041e0fad None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:14.714381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:14.716004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:14.717142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:14.717647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/ed322fa3f340471ca755df35190850e1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:14.719083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ed322fa3f340471ca755df35190850e1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:14.721704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:14.792040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed322fa3f340471ca755df35190850e1', 'request_id': 'req-256d5a17-ac2a-45bb-863c-6c06e3018562'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:14.803891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed322fa3f340471ca755df35190850e1', 'request_id': 'req-256d5a17-ac2a-45bb-863c-6c06e3018562'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:14.817061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed322fa3f340471ca755df35190850e1', 'request_id': 'req-256d5a17-ac2a-45bb-863c-6c06e3018562'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:14.833403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] The token cache is being invalidate because project ed322fa3f340471ca755df35190850e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:14.833702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:14.834434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:14.842695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:14.843528 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1783/3532] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:14 2022] DELETE /identity/v3/projects/ed322fa3f340471ca755df35190850e1 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:14.844231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:14.915960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0789e11-1448-47d3-ba2f-24308d897cb4 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:15.049814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda428ba-6a51-4aca-ac88-078bc0e72058 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:15.050734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda428ba-6a51-4aca-ac88-078bc0e72058 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:15.051057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda428ba-6a51-4aca-ac88-078bc0e72058 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:15.051308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda428ba-6a51-4aca-ac88-078bc0e72058 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:15.051913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda428ba-6a51-4aca-ac88-078bc0e72058 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:15.054184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda428ba-6a51-4aca-ac88-078bc0e72058 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:15.122865 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1750/3533] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:05:14 2022] POST /identity/v3/projects => generated 328 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:15.125242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:15.127250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:15.152030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d5a17-ac2a-45bb-863c-6c06e3018562 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:15.265735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:15.274736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871fbd72-68ef-4ac0-a1e0-f88be9de87bc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:15.275354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fbd72-68ef-4ac0-a1e0-f88be9de87bc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:15.275631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fbd72-68ef-4ac0-a1e0-f88be9de87bc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:15.275901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fbd72-68ef-4ac0-a1e0-f88be9de87bc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:15.276583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871fbd72-68ef-4ac0-a1e0-f88be9de87bc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:15.279116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871fbd72-68ef-4ac0-a1e0-f88be9de87bc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:15.308718 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1784/3534] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:15 2022] POST /identity/v3/users => generated 416 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:15.312274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda428ba-6a51-4aca-ac88-078bc0e72058 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:15.328956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda428ba-6a51-4aca-ac88-078bc0e72058 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:15.335635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd8c0a-918e-4064-b5d3-32a4a715c55a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:15.336459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd8c0a-918e-4064-b5d3-32a4a715c55a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:15.336680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd8c0a-918e-4064-b5d3-32a4a715c55a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:15.336950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd8c0a-918e-4064-b5d3-32a4a715c55a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:15.337818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd8c0a-918e-4064-b5d3-32a4a715c55a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:15.339797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd8c0a-918e-4064-b5d3-32a4a715c55a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:15.386032 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1751/3535] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:05:15 2022] POST /identity/v3/roles => generated 244 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:15.390471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871fbd72-68ef-4ac0-a1e0-f88be9de87bc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:15.412991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871fbd72-68ef-4ac0-a1e0-f88be9de87bc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:15.425377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee20a3-0c91-4b1f-aff5-2eede291df28 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:15.425817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee20a3-0c91-4b1f-aff5-2eede291df28 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:15.426070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee20a3-0c91-4b1f-aff5-2eede291df28 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:15.426343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee20a3-0c91-4b1f-aff5-2eede291df28 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:15.427780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee20a3-0c91-4b1f-aff5-2eede291df28 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:15.445714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee20a3-0c91-4b1f-aff5-2eede291df28 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:15.483123 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1785/3536] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:05:15 2022] POST /identity/v3/roles => generated 247 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:15.484977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd8c0a-918e-4064-b5d3-32a4a715c55a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:15.519169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd8c0a-918e-4064-b5d3-32a4a715c55a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:15.530467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7cb86-479b-4b98-8eac-c3ff76c0798e tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:15.531183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7cb86-479b-4b98-8eac-c3ff76c0798e tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:15.531462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7cb86-479b-4b98-8eac-c3ff76c0798e tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:15.531733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7cb86-479b-4b98-8eac-c3ff76c0798e tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/01c09a03601a45aaad31c04a3c589d09/users/99f13b8480954e4aa35ed35ec18dd591/roles/6e3be0eec33c493ea9531d3fc19f3ac3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:15.554010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7cb86-479b-4b98-8eac-c3ff76c0798e tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_grant(project_id=01c09a03601a45aaad31c04a3c589d09, user_id=99f13b8480954e4aa35ed35ec18dd591, role_id=6e3be0eec33c493ea9531d3fc19f3ac3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:15.557220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7cb86-479b-4b98-8eac-c3ff76c0798e tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:15.574465 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1752/3537] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:15 2022] PUT /identity/v3/projects/01c09a03601a45aaad31c04a3c589d09/users/99f13b8480954e4aa35ed35ec18dd591/roles/6e3be0eec33c493ea9531d3fc19f3ac3 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:15.578398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee20a3-0c91-4b1f-aff5-2eede291df28 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:15.595996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee20a3-0c91-4b1f-aff5-2eede291df28 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:15.697314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329b42b-e275-4179-bb08-13f5acf90cfa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:15.698597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329b42b-e275-4179-bb08-13f5acf90cfa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:15.698597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329b42b-e275-4179-bb08-13f5acf90cfa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:15.698597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329b42b-e275-4179-bb08-13f5acf90cfa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/01c09a03601a45aaad31c04a3c589d09/users/99f13b8480954e4aa35ed35ec18dd591/roles/b44bd28fbc4040d0815cb2332416b774` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:15.701302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329b42b-e275-4179-bb08-13f5acf90cfa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:create_grant(project_id=01c09a03601a45aaad31c04a3c589d09, user_id=99f13b8480954e4aa35ed35ec18dd591, role_id=b44bd28fbc4040d0815cb2332416b774)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:15.704239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329b42b-e275-4179-bb08-13f5acf90cfa tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:15.741093 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1786/3538] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:15 2022] PUT /identity/v3/projects/01c09a03601a45aaad31c04a3c589d09/users/99f13b8480954e4aa35ed35ec18dd591/roles/b44bd28fbc4040d0815cb2332416b774 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:15.743427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7cb86-479b-4b98-8eac-c3ff76c0798e tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:15.761948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7cb86-479b-4b98-8eac-c3ff76c0798e tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:15.875389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:15.948312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c4458-a37c-4cce-a837-c217ac1c62fc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:15.950419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c4458-a37c-4cce-a837-c217ac1c62fc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:15.951272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c4458-a37c-4cce-a837-c217ac1c62fc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:15.951765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c4458-a37c-4cce-a837-c217ac1c62fc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:15.952845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c4458-a37c-4cce-a837-c217ac1c62fc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:15.965216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c4458-a37c-4cce-a837-c217ac1c62fc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:16.087364 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1753/3539] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:15 2022] GET /identity/v3/users => generated 8640 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:16.093279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9604a6-6af6-420f-843a-4505ecf43b28 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:16.093604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9604a6-6af6-420f-843a-4505ecf43b28 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:16.093929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9604a6-6af6-420f-843a-4505ecf43b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:16.222826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ef9604a6-6af6-420f-843a-4505ecf43b28 None None] MFA Rules not processed for user `99f13b8480954e4aa35ed35ec18dd591`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:16.232316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ef9604a6-6af6-420f-843a-4505ecf43b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:16.468224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482c257-4365-4c96-a46f-cface6a71885 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:16.468804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482c257-4365-4c96-a46f-cface6a71885 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:16.468804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482c257-4365-4c96-a46f-cface6a71885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:16.481726 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1787/3540] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:16.507759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0482c257-4365-4c96-a46f-cface6a71885 None None] MFA Rules not processed for user `99f13b8480954e4aa35ed35ec18dd591`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:16.511678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0482c257-4365-4c96-a46f-cface6a71885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:16.533161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9604a6-6af6-420f-843a-4505ecf43b28 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:16.540916 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1754/3541] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:16.547501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9604a6-6af6-420f-843a-4505ecf43b28 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:16.558941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06d7af-9063-4a1d-b08c-e806c5518f99 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f13b8480954e4aa35ed35ec18dd591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01c09a03601a45aaad31c04a3c589d09', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1055341923', 'tempest-NotDelegatedRole-1341946189'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:16.559733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06d7af-9063-4a1d-b08c-e806c5518f99 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:16.561948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06d7af-9063-4a1d-b08c-e806c5518f99 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:16.562335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06d7af-9063-4a1d-b08c-e806c5518f99 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:16.563280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06d7af-9063-4a1d-b08c-e806c5518f99 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorizing `identity:create_trust()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:16.564970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06d7af-9063-4a1d-b08c-e806c5518f99 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:16.763000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:16.848968 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1788/3542] 10.222.0.23 () {60 vars in 1174 bytes} [Sat Aug 20 21:05:16 2022] POST /identity/v3/OS-TRUST/trusts => generated 835 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:16.853517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482c257-4365-4c96-a46f-cface6a71885 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:16.869362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482c257-4365-4c96-a46f-cface6a71885 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:16.879713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca703c6-387e-4752-8a1d-d4f446f8bbf0 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f13b8480954e4aa35ed35ec18dd591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01c09a03601a45aaad31c04a3c589d09', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1055341923', 'tempest-NotDelegatedRole-1341946189'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:16.880603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca703c6-387e-4752-8a1d-d4f446f8bbf0 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:16.884947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca703c6-387e-4752-8a1d-d4f446f8bbf0 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:16.885348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca703c6-387e-4752-8a1d-d4f446f8bbf0 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] PATH_INFO: `/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:16.900657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca703c6-387e-4752-8a1d-d4f446f8bbf0 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorizing `identity:get_trust(trust_id=2c563655cd364d5e84a295c3167b296b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:16.902157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca703c6-387e-4752-8a1d-d4f446f8bbf0 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:16.922152 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1755/3543] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:05:16 2022] GET /identity/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b => generated 835 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:16.929815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06d7af-9063-4a1d-b08c-e806c5518f99 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:16.947957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06d7af-9063-4a1d-b08c-e806c5518f99 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:16.960429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db1604-5368-405b-8a7e-33655f32178e tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f13b8480954e4aa35ed35ec18dd591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01c09a03601a45aaad31c04a3c589d09', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1055341923', 'tempest-NotDelegatedRole-1341946189'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:16.961780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db1604-5368-405b-8a7e-33655f32178e tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:16.962298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db1604-5368-405b-8a7e-33655f32178e tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:16.963307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db1604-5368-405b-8a7e-33655f32178e tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] PATH_INFO: `/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:16.977918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db1604-5368-405b-8a7e-33655f32178e tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=2c563655cd364d5e84a295c3167b296b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:16.983456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db1604-5368-405b-8a7e-33655f32178e tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:17.001528 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1789/3544] 10.222.0.23 () {58 vars in 1269 bytes} [Sat Aug 20 21:05:16 2022] GET /identity/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b/roles => generated 371 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:17.006921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca703c6-387e-4752-8a1d-d4f446f8bbf0 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:17.020993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca703c6-387e-4752-8a1d-d4f446f8bbf0 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:17.033244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320675f9-6377-4b26-8757-dadca53baf42 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f13b8480954e4aa35ed35ec18dd591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01c09a03601a45aaad31c04a3c589d09', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1055341923', 'tempest-NotDelegatedRole-1341946189'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:17.034371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320675f9-6377-4b26-8757-dadca53baf42 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:17.034890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320675f9-6377-4b26-8757-dadca53baf42 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:17.035418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320675f9-6377-4b26-8757-dadca53baf42 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] PATH_INFO: `/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b/roles/6e3be0eec33c493ea9531d3fc19f3ac3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:17.044573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320675f9-6377-4b26-8757-dadca53baf42 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorizing `identity:get_role_for_trust(trust_id=2c563655cd364d5e84a295c3167b296b, role_id=6e3be0eec33c493ea9531d3fc19f3ac3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:17.047033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320675f9-6377-4b26-8757-dadca53baf42 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:17.062978 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1756/3545] 10.222.0.23 () {58 vars in 1368 bytes} [Sat Aug 20 21:05:17 2022] GET /identity/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b/roles/6e3be0eec33c493ea9531d3fc19f3ac3 => generated 244 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:17.063896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db1604-5368-405b-8a7e-33655f32178e tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:17.076318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db1604-5368-405b-8a7e-33655f32178e tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:17.081911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfd286-1c13-48c0-b13d-b35c5b525ffb tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f13b8480954e4aa35ed35ec18dd591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01c09a03601a45aaad31c04a3c589d09', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1055341923', 'tempest-NotDelegatedRole-1341946189'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:17.082946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfd286-1c13-48c0-b13d-b35c5b525ffb tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] REQUEST_METHOD: `HEAD` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:17.083476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfd286-1c13-48c0-b13d-b35c5b525ffb tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:17.083951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfd286-1c13-48c0-b13d-b35c5b525ffb tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] PATH_INFO: `/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b/roles/6e3be0eec33c493ea9531d3fc19f3ac3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:17.091016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfd286-1c13-48c0-b13d-b35c5b525ffb tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorizing `identity:get_role_for_trust(trust_id=2c563655cd364d5e84a295c3167b296b, role_id=6e3be0eec33c493ea9531d3fc19f3ac3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:17.092930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfd286-1c13-48c0-b13d-b35c5b525ffb tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:17.105533 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1790/3546] 10.222.0.23 () {58 vars in 1369 bytes} [Sat Aug 20 21:05:17 2022] HEAD /identity/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b/roles/6e3be0eec33c493ea9531d3fc19f3ac3 => generated 0 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:17.106911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320675f9-6377-4b26-8757-dadca53baf42 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:17.118199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320675f9-6377-4b26-8757-dadca53baf42 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:17.125210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006813a6-3a37-4c6b-b348-013a57d624a2 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f13b8480954e4aa35ed35ec18dd591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01c09a03601a45aaad31c04a3c589d09', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1055341923', 'tempest-NotDelegatedRole-1341946189'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:17.126743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006813a6-3a37-4c6b-b348-013a57d624a2 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:17.128988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006813a6-3a37-4c6b-b348-013a57d624a2 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:17.129959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006813a6-3a37-4c6b-b348-013a57d624a2 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] PATH_INFO: `/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b/roles/b44bd28fbc4040d0815cb2332416b774` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:17.137807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006813a6-3a37-4c6b-b348-013a57d624a2 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorizing `identity:get_role_for_trust(trust_id=2c563655cd364d5e84a295c3167b296b, role_id=b44bd28fbc4040d0815cb2332416b774)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:17.140155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006813a6-3a37-4c6b-b348-013a57d624a2 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-006813a6-3a37-4c6b-b348-013a57d624a2 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Could not find role: b44bd28fbc4040d0815cb2332416b774.: keystone.exception.RoleNotFound: Could not find role: b44bd28fbc4040d0815cb2332416b774. Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 21:05:17.152789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: b44bd28fbc4040d0815cb2332416b774. Aug 20 21:05:17.153753 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:05:17.159885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfd286-1c13-48c0-b13d-b35c5b525ffb tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:17.160615 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1757/3547] 10.222.0.23 () {58 vars in 1368 bytes} [Sat Aug 20 21:05:17 2022] GET /identity/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b/roles/b44bd28fbc4040d0815cb2332416b774 => generated 110 bytes in 53 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Aug 20 21:05:17.171695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfd286-1c13-48c0-b13d-b35c5b525ffb tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:17.176345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8969df-235d-4573-b086-eb9ed05814bf tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f13b8480954e4aa35ed35ec18dd591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01c09a03601a45aaad31c04a3c589d09', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1055341923', 'tempest-NotDelegatedRole-1341946189'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:17.177294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8969df-235d-4573-b086-eb9ed05814bf tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] REQUEST_METHOD: `HEAD` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:17.177776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8969df-235d-4573-b086-eb9ed05814bf tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:17.178272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8969df-235d-4573-b086-eb9ed05814bf tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] PATH_INFO: `/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b/roles/b44bd28fbc4040d0815cb2332416b774` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:17.186161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8969df-235d-4573-b086-eb9ed05814bf tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorizing `identity:get_role_for_trust(trust_id=2c563655cd364d5e84a295c3167b296b, role_id=b44bd28fbc4040d0815cb2332416b774)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:17.194879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8969df-235d-4573-b086-eb9ed05814bf tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-5e8969df-235d-4573-b086-eb9ed05814bf tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Could not find role: b44bd28fbc4040d0815cb2332416b774.: keystone.exception.RoleNotFound: Could not find role: b44bd28fbc4040d0815cb2332416b774. Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 21:05:17.216209 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: b44bd28fbc4040d0815cb2332416b774. Aug 20 21:05:17.217235 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:17.222429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006813a6-3a37-4c6b-b348-013a57d624a2 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:17.222772 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1791/3548] 10.222.0.23 () {58 vars in 1369 bytes} [Sat Aug 20 21:05:17 2022] HEAD /identity/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b/roles/b44bd28fbc4040d0815cb2332416b774 => generated 0 bytes in 58 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:05:17.247416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006813a6-3a37-4c6b-b348-013a57d624a2 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:17.252826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a1965-7331-4b30-8c55-db9bd845c7a6 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f13b8480954e4aa35ed35ec18dd591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01c09a03601a45aaad31c04a3c589d09', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1055341923', 'tempest-NotDelegatedRole-1341946189'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:17.253518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a1965-7331-4b30-8c55-db9bd845c7a6 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:17.254144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a1965-7331-4b30-8c55-db9bd845c7a6 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:17.254144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a1965-7331-4b30-8c55-db9bd845c7a6 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] PATH_INFO: `/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:17.263328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a1965-7331-4b30-8c55-db9bd845c7a6 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorizing `identity:delete_trust(trust_id=2c563655cd364d5e84a295c3167b296b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:17.266273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a1965-7331-4b30-8c55-db9bd845c7a6 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:17.290759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-910a1965-7331-4b30-8c55-db9bd845c7a6 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '2c563655cd364d5e84a295c3167b296b', 'request_id': 'req-910a1965-7331-4b30-8c55-db9bd845c7a6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:17.291376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-910a1965-7331-4b30-8c55-db9bd845c7a6 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '2c563655cd364d5e84a295c3167b296b', 'request_id': 'req-910a1965-7331-4b30-8c55-db9bd845c7a6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:17.292909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-910a1965-7331-4b30-8c55-db9bd845c7a6 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '2c563655cd364d5e84a295c3167b296b', 'request_id': 'req-910a1965-7331-4b30-8c55-db9bd845c7a6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:17.330081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8969df-235d-4573-b086-eb9ed05814bf tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:17.330771 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1758/3549] 10.222.0.23 () {58 vars in 1254 bytes} [Sat Aug 20 21:05:17 2022] DELETE /identity/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:17.331842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5e8969df-235d-4573-b086-eb9ed05814bf tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:17.348891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8969df-235d-4573-b086-eb9ed05814bf tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:17.362969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aec1c-d93c-49fc-b60c-d0905fbb3088 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f13b8480954e4aa35ed35ec18dd591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01c09a03601a45aaad31c04a3c589d09', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1055341923', 'tempest-NotDelegatedRole-1341946189'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:17.363121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aec1c-d93c-49fc-b60c-d0905fbb3088 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:17.363397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aec1c-d93c-49fc-b60c-d0905fbb3088 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:17.363663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aec1c-d93c-49fc-b60c-d0905fbb3088 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] PATH_INFO: `/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-8c9aec1c-d93c-49fc-b60c-d0905fbb3088 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Could not find trust: 2c563655cd364d5e84a295c3167b296b.: keystone.exception.TrustNotFound: Could not find trust: 2c563655cd364d5e84a295c3167b296b. Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Aug 20 21:05:17.388930 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Aug 20 21:05:17.390317 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Aug 20 21:05:17.390317 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Aug 20 21:05:17.390317 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Aug 20 21:05:17.390317 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Aug 20 21:05:17.390317 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Aug 20 21:05:17.390317 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:17.390317 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:17.390317 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Aug 20 21:05:17.390317 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Aug 20 21:05:17.390317 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Aug 20 21:05:17.390317 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Aug 20 21:05:17.390317 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 2c563655cd364d5e84a295c3167b296b. Aug 20 21:05:17.390317 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:17.396898 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1792/3550] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:05:17 2022] GET /identity/v3/OS-TRUST/trusts/2c563655cd364d5e84a295c3167b296b => generated 111 bytes in 67 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:05:17.401815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a1965-7331-4b30-8c55-db9bd845c7a6 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:17.406176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-910a1965-7331-4b30-8c55-db9bd845c7a6 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:17.460469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a1965-7331-4b30-8c55-db9bd845c7a6 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:17.484463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acd6b7-8dfe-4dd3-8974-4201f79e1bcc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:17.485609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acd6b7-8dfe-4dd3-8974-4201f79e1bcc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:17.486183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acd6b7-8dfe-4dd3-8974-4201f79e1bcc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:17.486548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acd6b7-8dfe-4dd3-8974-4201f79e1bcc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles/b44bd28fbc4040d0815cb2332416b774` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:17.490210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58acd6b7-8dfe-4dd3-8974-4201f79e1bcc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_role(role_id=b44bd28fbc4040d0815cb2332416b774)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:17.492366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58acd6b7-8dfe-4dd3-8974-4201f79e1bcc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:17.607068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-58acd6b7-8dfe-4dd3-8974-4201f79e1bcc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invalidating the token cache because role b44bd28fbc4040d0815cb2332416b774 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:17.607475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-58acd6b7-8dfe-4dd3-8974-4201f79e1bcc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:17.614686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-58acd6b7-8dfe-4dd3-8974-4201f79e1bcc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:17.619917 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1759/3551] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:17 2022] DELETE /identity/v3/roles/b44bd28fbc4040d0815cb2332416b774 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:17.625149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aec1c-d93c-49fc-b60c-d0905fbb3088 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:17.627429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8c9aec1c-d93c-49fc-b60c-d0905fbb3088 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:17.668233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aec1c-d93c-49fc-b60c-d0905fbb3088 tempest-TrustsV3TestJSON-296687929 tempest-user-1793466316] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:17.788826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb293e6f-b9f7-4c37-a30b-7885f9228c2f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:17.789676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb293e6f-b9f7-4c37-a30b-7885f9228c2f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:17.789956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb293e6f-b9f7-4c37-a30b-7885f9228c2f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:17.790230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb293e6f-b9f7-4c37-a30b-7885f9228c2f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/roles/6e3be0eec33c493ea9531d3fc19f3ac3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:17.793780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb293e6f-b9f7-4c37-a30b-7885f9228c2f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_role(role_id=6e3be0eec33c493ea9531d3fc19f3ac3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:17.798470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb293e6f-b9f7-4c37-a30b-7885f9228c2f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:17.904978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cb293e6f-b9f7-4c37-a30b-7885f9228c2f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invalidating the token cache because role 6e3be0eec33c493ea9531d3fc19f3ac3 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:17.905354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cb293e6f-b9f7-4c37-a30b-7885f9228c2f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:17.907132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cb293e6f-b9f7-4c37-a30b-7885f9228c2f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:17.912855 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1793/3552] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:17 2022] DELETE /identity/v3/roles/6e3be0eec33c493ea9531d3fc19f3ac3 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:17.918205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acd6b7-8dfe-4dd3-8974-4201f79e1bcc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:17.966999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acd6b7-8dfe-4dd3-8974-4201f79e1bcc tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:18.077196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:18.115309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:18.116977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:18.117563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:18.118408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/users/99f13b8480954e4aa35ed35ec18dd591` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:18.123222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_user(user_id=99f13b8480954e4aa35ed35ec18dd591)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:18.130019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:18.278241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f13b8480954e4aa35ed35ec18dd591'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:18.294318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f13b8480954e4aa35ed35ec18dd591'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:18.331932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f13b8480954e4aa35ed35ec18dd591'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:18.337383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f13b8480954e4aa35ed35ec18dd591'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:18.338862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f13b8480954e4aa35ed35ec18dd591'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:18.370096 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1760/3553] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:17 2022] DELETE /identity/v3/users/99f13b8480954e4aa35ed35ec18dd591 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:18.375408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb293e6f-b9f7-4c37-a30b-7885f9228c2f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:18.377969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cb293e6f-b9f7-4c37-a30b-7885f9228c2f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:18.417986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb293e6f-b9f7-4c37-a30b-7885f9228c2f tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:18.505264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:18.560426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67454e-e1f3-45b7-8f63-5ef21451479a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99481615eafb4cbba88546ecb09a3020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d06965396174bd0922ebceeb387cfbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:18.563211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67454e-e1f3-45b7-8f63-5ef21451479a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:18.564344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67454e-e1f3-45b7-8f63-5ef21451479a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:18.565002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67454e-e1f3-45b7-8f63-5ef21451479a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] PATH_INFO: `/v3/projects/01c09a03601a45aaad31c04a3c589d09` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:18.569079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67454e-e1f3-45b7-8f63-5ef21451479a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorizing `identity:delete_project(project_id=01c09a03601a45aaad31c04a3c589d09)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:18.575445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67454e-e1f3-45b7-8f63-5ef21451479a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:18.694498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9c67454e-e1f3-45b7-8f63-5ef21451479a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01c09a03601a45aaad31c04a3c589d09', 'request_id': 'req-9c67454e-e1f3-45b7-8f63-5ef21451479a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:18.715627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9c67454e-e1f3-45b7-8f63-5ef21451479a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01c09a03601a45aaad31c04a3c589d09', 'request_id': 'req-9c67454e-e1f3-45b7-8f63-5ef21451479a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:18.724192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9c67454e-e1f3-45b7-8f63-5ef21451479a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01c09a03601a45aaad31c04a3c589d09', 'request_id': 'req-9c67454e-e1f3-45b7-8f63-5ef21451479a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:18.755618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9c67454e-e1f3-45b7-8f63-5ef21451479a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] The token cache is being invalidate because project 01c09a03601a45aaad31c04a3c589d09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:18.755939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9c67454e-e1f3-45b7-8f63-5ef21451479a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:18.757487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9c67454e-e1f3-45b7-8f63-5ef21451479a tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:18.765167 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1794/3554] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:18 2022] DELETE /identity/v3/projects/01c09a03601a45aaad31c04a3c589d09 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:18.766590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:18.769213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:18.892313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd6873-35fa-4812-bc5d-e6297d23d017 tempest-TrustsV3TestJSON-673311334 tempest-TrustsV3TestJSON-673311334-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:19.045204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:19.046499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:19.047441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:19.048473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] PATH_INFO: `/v3/users/5ffa280e0b8e4141828fa7e306ce5ff1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:19.052263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ffa280e0b8e4141828fa7e306ce5ff1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:19.055919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:19.132343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:19.215837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ffa280e0b8e4141828fa7e306ce5ff1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:19.230449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ffa280e0b8e4141828fa7e306ce5ff1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:19.260166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ffa280e0b8e4141828fa7e306ce5ff1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:19.261700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ffa280e0b8e4141828fa7e306ce5ff1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:19.265982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ffa280e0b8e4141828fa7e306ce5ff1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:19.302678 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1761/3555] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:18 2022] DELETE /identity/v3/users/5ffa280e0b8e4141828fa7e306ce5ff1 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:19.307380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb502091-287f-4384-a1ab-b6d2593b6054 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:19.307886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb502091-287f-4384-a1ab-b6d2593b6054 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:19.307886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb502091-287f-4384-a1ab-b6d2593b6054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:19.336276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cb502091-287f-4384-a1ab-b6d2593b6054 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:19.337755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cb502091-287f-4384-a1ab-b6d2593b6054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:19.454831 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1795/3556] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:19.480817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:19.483591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:19.634916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee450ad-ab72-418a-b315-c90d87cf3580 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:19.644511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96fa8a-58d5-4b2d-8b57-2f9df686b4d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:19.646965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96fa8a-58d5-4b2d-8b57-2f9df686b4d1 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:19.647235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96fa8a-58d5-4b2d-8b57-2f9df686b4d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:19.647650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96fa8a-58d5-4b2d-8b57-2f9df686b4d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:19.670958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96fa8a-58d5-4b2d-8b57-2f9df686b4d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:19.673772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96fa8a-58d5-4b2d-8b57-2f9df686b4d1 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:19.708158 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1762/3557] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:20.978433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb502091-287f-4384-a1ab-b6d2593b6054 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:20.999453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb502091-287f-4384-a1ab-b6d2593b6054 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:21.016789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d969b-5c0d-4d4b-8c8f-bb3a30e89acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:21.017704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d969b-5c0d-4d4b-8c8f-bb3a30e89acf service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:21.018189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d969b-5c0d-4d4b-8c8f-bb3a30e89acf service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:21.018888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d969b-5c0d-4d4b-8c8f-bb3a30e89acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:21.026286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e59d969b-5c0d-4d4b-8c8f-bb3a30e89acf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:21.049023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d969b-5c0d-4d4b-8c8f-bb3a30e89acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:21.051192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d969b-5c0d-4d4b-8c8f-bb3a30e89acf service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:21.068243 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1796/3558] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:21.541486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96fa8a-58d5-4b2d-8b57-2f9df686b4d1 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:21.543882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7a96fa8a-58d5-4b2d-8b57-2f9df686b4d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:21.576015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96fa8a-58d5-4b2d-8b57-2f9df686b4d1 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:21.603469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0f82d-fb05-4018-9862-35a362980818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:21.603469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0f82d-fb05-4018-9862-35a362980818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:21.603469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0f82d-fb05-4018-9862-35a362980818 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:21.603469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0f82d-fb05-4018-9862-35a362980818 admin admin] PATH_INFO: `/v3/projects/ce260c2340f84129a95a2435c3605b18` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:21.603790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0f82d-fb05-4018-9862-35a362980818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce260c2340f84129a95a2435c3605b18)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:21.603790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0f82d-fb05-4018-9862-35a362980818 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:21.693266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-90e0f82d-fb05-4018-9862-35a362980818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce260c2340f84129a95a2435c3605b18', 'request_id': 'req-90e0f82d-fb05-4018-9862-35a362980818'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:21.701993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-90e0f82d-fb05-4018-9862-35a362980818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce260c2340f84129a95a2435c3605b18', 'request_id': 'req-90e0f82d-fb05-4018-9862-35a362980818'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:21.718795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-90e0f82d-fb05-4018-9862-35a362980818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce260c2340f84129a95a2435c3605b18', 'request_id': 'req-90e0f82d-fb05-4018-9862-35a362980818'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:21.761191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-90e0f82d-fb05-4018-9862-35a362980818 admin admin] The token cache is being invalidate because project ce260c2340f84129a95a2435c3605b18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:21.761758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-90e0f82d-fb05-4018-9862-35a362980818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:21.763307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-90e0f82d-fb05-4018-9862-35a362980818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:21.770651 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1763/3559] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:21 2022] DELETE /identity/v3/projects/ce260c2340f84129a95a2435c3605b18 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:21.774943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d969b-5c0d-4d4b-8c8f-bb3a30e89acf service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:21.788604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d969b-5c0d-4d4b-8c8f-bb3a30e89acf service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:21.923858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:21.942701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c555f4-a977-468c-a831-ba428b1f0ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:21.944534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c555f4-a977-468c-a831-ba428b1f0ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:21.946234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c555f4-a977-468c-a831-ba428b1f0ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:21.956180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c555f4-a977-468c-a831-ba428b1f0ba5 admin admin] PATH_INFO: `/v3/users/99481615eafb4cbba88546ecb09a3020` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:21.958119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c555f4-a977-468c-a831-ba428b1f0ba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99481615eafb4cbba88546ecb09a3020)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:21.960292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c555f4-a977-468c-a831-ba428b1f0ba5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:22.096144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5c555f4-a977-468c-a831-ba428b1f0ba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99481615eafb4cbba88546ecb09a3020'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:22.114744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb399aa-447c-4a24-9b35-7c5e8d715243 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:22.115027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb399aa-447c-4a24-9b35-7c5e8d715243 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:22.115296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb399aa-447c-4a24-9b35-7c5e8d715243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:22.121412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5c555f4-a977-468c-a831-ba428b1f0ba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99481615eafb4cbba88546ecb09a3020'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:22.150928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5c555f4-a977-468c-a831-ba428b1f0ba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99481615eafb4cbba88546ecb09a3020'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:22.161235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0bb399aa-447c-4a24-9b35-7c5e8d715243 None None] MFA Rules not processed for user `647583dffce54234bcf9276d4a61f80e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:22.163289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0bb399aa-447c-4a24-9b35-7c5e8d715243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:22.189212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:22.217721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5c555f4-a977-468c-a831-ba428b1f0ba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99481615eafb4cbba88546ecb09a3020'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:22.221139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5c555f4-a977-468c-a831-ba428b1f0ba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99481615eafb4cbba88546ecb09a3020'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:22.237745 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1797/3560] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:21 2022] DELETE /identity/v3/users/99481615eafb4cbba88546ecb09a3020 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:22.396364 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1764/3561] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:22.401342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a9fa7-3813-40a2-b8a4-270124b132a1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:22.402023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a9fa7-3813-40a2-b8a4-270124b132a1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:22.402521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a9fa7-3813-40a2-b8a4-270124b132a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:22.439109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6e4a9fa7-3813-40a2-b8a4-270124b132a1 None None] MFA Rules not processed for user `647583dffce54234bcf9276d4a61f80e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:22.443637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6e4a9fa7-3813-40a2-b8a4-270124b132a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:22.587281 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1798/3562] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:22.596939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb399aa-447c-4a24-9b35-7c5e8d715243 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:22.599148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0bb399aa-447c-4a24-9b35-7c5e8d715243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:22.785056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb399aa-447c-4a24-9b35-7c5e8d715243 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:22.894072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395abb2e-d2e4-4883-9f20-d3719e6fae9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:22.894706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395abb2e-d2e4-4883-9f20-d3719e6fae9a service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:22.895141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395abb2e-d2e4-4883-9f20-d3719e6fae9a service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:22.895416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395abb2e-d2e4-4883-9f20-d3719e6fae9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:22.906231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395abb2e-d2e4-4883-9f20-d3719e6fae9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:22.907857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395abb2e-d2e4-4883-9f20-d3719e6fae9a service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:22.921807 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1765/3563] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:23.029415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a9fa7-3813-40a2-b8a4-270124b132a1 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:23.030935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6e4a9fa7-3813-40a2-b8a4-270124b132a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:23.153899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a9fa7-3813-40a2-b8a4-270124b132a1 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:23.159686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:23.160438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:23.160810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:23.161146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7 admin admin] PATH_INFO: `/v3/projects/7d06965396174bd0922ebceeb387cfbe` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:23.162347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d06965396174bd0922ebceeb387cfbe)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:23.165270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:23.200540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:23.262780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d06965396174bd0922ebceeb387cfbe', 'request_id': 'req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:23.276763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d06965396174bd0922ebceeb387cfbe', 'request_id': 'req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:23.284317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d06965396174bd0922ebceeb387cfbe', 'request_id': 'req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:23.300232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7 admin admin] The token cache is being invalidate because project 7d06965396174bd0922ebceeb387cfbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:23.300406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:23.301136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3b7dfbbc-b650-4a9f-8da5-57750f8879a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:23.302933 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1799/3564] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:23 2022] DELETE /identity/v3/projects/7d06965396174bd0922ebceeb387cfbe => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:23.307913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9024dde-bf68-4513-96bb-162a816a0d4f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:23.308341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9024dde-bf68-4513-96bb-162a816a0d4f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:23.308799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9024dde-bf68-4513-96bb-162a816a0d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:23.330525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab29de0-35db-41bc-a2e4-894102b75a1b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:23.330839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab29de0-35db-41bc-a2e4-894102b75a1b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:23.335511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab29de0-35db-41bc-a2e4-894102b75a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:23.356627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c9024dde-bf68-4513-96bb-162a816a0d4f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:23.358703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c9024dde-bf68-4513-96bb-162a816a0d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:23.415497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:23.419750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4ab29de0-35db-41bc-a2e4-894102b75a1b None None] MFA Rules not processed for user `c8b2c80165614ad58df38edc24b47715`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:23.422018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4ab29de0-35db-41bc-a2e4-894102b75a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:23.473670 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1766/3565] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:23.476222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d7b8b-0eb5-4723-b537-d8fa525c09a0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:23.478609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d7b8b-0eb5-4723-b537-d8fa525c09a0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:23.478998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d7b8b-0eb5-4723-b537-d8fa525c09a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:23.532634 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1800/3566] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:23.549323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab29de0-35db-41bc-a2e4-894102b75a1b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:23.551165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4ab29de0-35db-41bc-a2e4-894102b75a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:23.554050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5a4d7b8b-0eb5-4723-b537-d8fa525c09a0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:23.557860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5a4d7b8b-0eb5-4723-b537-d8fa525c09a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:23.585915 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1767/3567] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:23.588003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d7b8b-0eb5-4723-b537-d8fa525c09a0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:23.608332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d7b8b-0eb5-4723-b537-d8fa525c09a0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:23.635241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1b05d-d099-4c43-a90c-2227b1bf374e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:23.635974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1b05d-d099-4c43-a90c-2227b1bf374e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:23.636297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1b05d-d099-4c43-a90c-2227b1bf374e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:23.636646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1b05d-d099-4c43-a90c-2227b1bf374e admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:23.638112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1b05d-d099-4c43-a90c-2227b1bf374e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:05:23.638599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1b05d-d099-4c43-a90c-2227b1bf374e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:23.640525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1b05d-d099-4c43-a90c-2227b1bf374e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:23.675170 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1768/3568] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:05:23.679468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1b05d-d099-4c43-a90c-2227b1bf374e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:23.693682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1b05d-d099-4c43-a90c-2227b1bf374e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:23.699296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8bc1d-4084-43f2-bf0d-d7a04a6f4061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:23.699980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8bc1d-4084-43f2-bf0d-d7a04a6f4061 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:23.700315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8bc1d-4084-43f2-bf0d-d7a04a6f4061 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:23.701134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8bc1d-4084-43f2-bf0d-d7a04a6f4061 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:23.701372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8bc1d-4084-43f2-bf0d-d7a04a6f4061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:23.703627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8bc1d-4084-43f2-bf0d-d7a04a6f4061 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:23.721170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab29de0-35db-41bc-a2e4-894102b75a1b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:23.737679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6105a-9eb5-4d4a-857c-1589ae4e0e91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:23.739081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6105a-9eb5-4d4a-857c-1589ae4e0e91 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:23.739369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6105a-9eb5-4d4a-857c-1589ae4e0e91 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:23.742731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6105a-9eb5-4d4a-857c-1589ae4e0e91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:23.774293 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1769/3569] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:23 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:23.776029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8bc1d-4084-43f2-bf0d-d7a04a6f4061 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:23.782183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6105a-9eb5-4d4a-857c-1589ae4e0e91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:23.784795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6105a-9eb5-4d4a-857c-1589ae4e0e91 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:23.821434 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1801/3570] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:23.873818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8bc1d-4084-43f2-bf0d-d7a04a6f4061 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:23.883801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27fbf5-6885-4009-90cd-2cb036f9e54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:23.884646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27fbf5-6885-4009-90cd-2cb036f9e54d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:23.884988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27fbf5-6885-4009-90cd-2cb036f9e54d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:23.885256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27fbf5-6885-4009-90cd-2cb036f9e54d admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:23.887436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27fbf5-6885-4009-90cd-2cb036f9e54d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:23.894864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27fbf5-6885-4009-90cd-2cb036f9e54d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:23.928426 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1770/3571] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:23 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:23.931157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6105a-9eb5-4d4a-857c-1589ae4e0e91 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:23.944759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6105a-9eb5-4d4a-857c-1589ae4e0e91 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:23.950279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234e36e-48ee-4c6d-9f5e-6b1598535b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:23.951492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234e36e-48ee-4c6d-9f5e-6b1598535b42 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:23.951771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234e36e-48ee-4c6d-9f5e-6b1598535b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:23.952087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234e36e-48ee-4c6d-9f5e-6b1598535b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:23.952799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234e36e-48ee-4c6d-9f5e-6b1598535b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:23.957526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234e36e-48ee-4c6d-9f5e-6b1598535b42 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:23.989036 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1802/3572] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:23.990335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27fbf5-6885-4009-90cd-2cb036f9e54d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:24.003197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27fbf5-6885-4009-90cd-2cb036f9e54d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:24.039356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea4dec-b6e6-41c8-a2bc-f37c4d9fc8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:24.040508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4dec-b6e6-41c8-a2bc-f37c4d9fc8dc admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:24.040962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4dec-b6e6-41c8-a2bc-f37c4d9fc8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:24.041396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4dec-b6e6-41c8-a2bc-f37c4d9fc8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:24.043463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea4dec-b6e6-41c8-a2bc-f37c4d9fc8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:24.047119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea4dec-b6e6-41c8-a2bc-f37c4d9fc8dc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:24.077806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234e36e-48ee-4c6d-9f5e-6b1598535b42 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:24.084930 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1771/3573] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:24.089200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234e36e-48ee-4c6d-9f5e-6b1598535b42 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:24.095207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723caeb-dd1d-4399-8524-b654b305931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:24.095884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723caeb-dd1d-4399-8524-b654b305931d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:24.096153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723caeb-dd1d-4399-8524-b654b305931d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:24.096644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723caeb-dd1d-4399-8524-b654b305931d admin admin] PATH_INFO: `/v3/projects/46e18bb6ade449f0b34f1350b23ffde5/users/a3f45acdcf924c6092f325cba9b8f971/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:24.112157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:24.128136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723caeb-dd1d-4399-8524-b654b305931d admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e18bb6ade449f0b34f1350b23ffde5, user_id=a3f45acdcf924c6092f325cba9b8f971, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:24.137424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723caeb-dd1d-4399-8524-b654b305931d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:24.172483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b8708-be69-4201-876f-94b8d5d4d979 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:24.172767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b8708-be69-4201-876f-94b8d5d4d979 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:24.175432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b8708-be69-4201-876f-94b8d5d4d979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:24.179925 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1803/3574] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:24 2022] PUT /identity/v3/projects/46e18bb6ade449f0b34f1350b23ffde5/users/a3f45acdcf924c6092f325cba9b8f971/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:24.216726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6b4b8708-be69-4201-876f-94b8d5d4d979 None None] MFA Rules not processed for user `a3f45acdcf924c6092f325cba9b8f971`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:24.219472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6b4b8708-be69-4201-876f-94b8d5d4d979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:24.406401 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1772/3575] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:24.408109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723caeb-dd1d-4399-8524-b654b305931d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:24.653061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723caeb-dd1d-4399-8524-b654b305931d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:24.671112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f062100-c02e-45c5-aad1-35783b4bd93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:24.671804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f062100-c02e-45c5-aad1-35783b4bd93e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:24.672084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f062100-c02e-45c5-aad1-35783b4bd93e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:24.672394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f062100-c02e-45c5-aad1-35783b4bd93e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:24.673270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f062100-c02e-45c5-aad1-35783b4bd93e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:24.679445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f062100-c02e-45c5-aad1-35783b4bd93e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:24.749080 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1804/3576] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:24 2022] POST /identity/v3/projects => generated 371 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:24.749282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b8708-be69-4201-876f-94b8d5d4d979 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:24.754817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:24.763839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b8708-be69-4201-876f-94b8d5d4d979 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:24.867295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95eedf-afa2-4ab0-a5fc-34b383607466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:24.868025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95eedf-afa2-4ab0-a5fc-34b383607466 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:24.868302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95eedf-afa2-4ab0-a5fc-34b383607466 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:24.868592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95eedf-afa2-4ab0-a5fc-34b383607466 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:24.869224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95eedf-afa2-4ab0-a5fc-34b383607466 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:24.871146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95eedf-afa2-4ab0-a5fc-34b383607466 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:24.913893 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1773/3577] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:24 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:24.920323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f062100-c02e-45c5-aad1-35783b4bd93e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:24.956116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f062100-c02e-45c5-aad1-35783b4bd93e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:24.977899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980a476-2867-45e1-9f87-972f2b91d179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:24.978541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980a476-2867-45e1-9f87-972f2b91d179 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:24.978815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980a476-2867-45e1-9f87-972f2b91d179 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:24.979084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980a476-2867-45e1-9f87-972f2b91d179 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:24.979734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980a476-2867-45e1-9f87-972f2b91d179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:24.982352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980a476-2867-45e1-9f87-972f2b91d179 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:25.010278 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1805/3578] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:25.014968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95eedf-afa2-4ab0-a5fc-34b383607466 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:25.032672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95eedf-afa2-4ab0-a5fc-34b383607466 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:25.041243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29fe1ec-2855-4210-9952-ad5c0c6ae907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:25.042164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fe1ec-2855-4210-9952-ad5c0c6ae907 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:25.042440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fe1ec-2855-4210-9952-ad5c0c6ae907 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:25.042709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fe1ec-2855-4210-9952-ad5c0c6ae907 admin admin] PATH_INFO: `/v3/projects/6deb361b7941467fb7165dc2cac092e1/users/e90b44e3397f4d71aec446ec33db92b8/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:25.082175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29fe1ec-2855-4210-9952-ad5c0c6ae907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6deb361b7941467fb7165dc2cac092e1, user_id=e90b44e3397f4d71aec446ec33db92b8, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:25.084935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29fe1ec-2855-4210-9952-ad5c0c6ae907 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:25.108919 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1774/3579] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:25 2022] PUT /identity/v3/projects/6deb361b7941467fb7165dc2cac092e1/users/e90b44e3397f4d71aec446ec33db92b8/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:25.112940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980a476-2867-45e1-9f87-972f2b91d179 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:25.130327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980a476-2867-45e1-9f87-972f2b91d179 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:25.274948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a41ce6-3df0-4ce1-b63d-bc721218bf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:25.276267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a41ce6-3df0-4ce1-b63d-bc721218bf2a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:25.276918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a41ce6-3df0-4ce1-b63d-bc721218bf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:25.277367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a41ce6-3df0-4ce1-b63d-bc721218bf2a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:25.278484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a41ce6-3df0-4ce1-b63d-bc721218bf2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:25.281168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a41ce6-3df0-4ce1-b63d-bc721218bf2a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:25.314119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29fe1ec-2855-4210-9952-ad5c0c6ae907 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:25.314399 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1806/3580] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:25.331244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29fe1ec-2855-4210-9952-ad5c0c6ae907 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:25.356558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee7747-965a-4781-8c2d-e47d0e7f7ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:25.358878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee7747-965a-4781-8c2d-e47d0e7f7ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:25.359788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee7747-965a-4781-8c2d-e47d0e7f7ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:25.360012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee7747-965a-4781-8c2d-e47d0e7f7ba5 admin admin] PATH_INFO: `/v3/projects/6deb361b7941467fb7165dc2cac092e1/users/e90b44e3397f4d71aec446ec33db92b8/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:25.366725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee7747-965a-4781-8c2d-e47d0e7f7ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6deb361b7941467fb7165dc2cac092e1, user_id=e90b44e3397f4d71aec446ec33db92b8, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:25.376792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee7747-965a-4781-8c2d-e47d0e7f7ba5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:25.393212 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1775/3581] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:25 2022] PUT /identity/v3/projects/6deb361b7941467fb7165dc2cac092e1/users/e90b44e3397f4d71aec446ec33db92b8/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:25.408091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c19984-667e-4189-a7a3-9001347f97e5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:25.408514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c19984-667e-4189-a7a3-9001347f97e5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:25.409099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c19984-667e-4189-a7a3-9001347f97e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:25.454806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-07c19984-667e-4189-a7a3-9001347f97e5 None None] MFA Rules not processed for user `e90b44e3397f4d71aec446ec33db92b8`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:25.456808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-07c19984-667e-4189-a7a3-9001347f97e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:25.634939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:25.720882 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1807/3582] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:25.726812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee7747-965a-4781-8c2d-e47d0e7f7ba5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:25.743409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee7747-965a-4781-8c2d-e47d0e7f7ba5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:25.756153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eaa0b-fa9e-4eb5-be8d-4b65b3079651 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90b44e3397f4d71aec446ec33db92b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6deb361b7941467fb7165dc2cac092e1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:25.758385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eaa0b-fa9e-4eb5-be8d-4b65b3079651 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:25.758385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eaa0b-fa9e-4eb5-be8d-4b65b3079651 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:25.758385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eaa0b-fa9e-4eb5-be8d-4b65b3079651 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:25.761035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719eaa0b-fa9e-4eb5-be8d-4b65b3079651 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:25.763193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719eaa0b-fa9e-4eb5-be8d-4b65b3079651 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:25.827342 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1776/3583] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:25 2022] POST /identity/v3/projects => generated 358 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:25.831680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c19984-667e-4189-a7a3-9001347f97e5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:25.844851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c19984-667e-4189-a7a3-9001347f97e5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:25.961174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285cac4-fddc-46d7-adbe-7b01c8849e7b tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90b44e3397f4d71aec446ec33db92b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6deb361b7941467fb7165dc2cac092e1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:25.962043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285cac4-fddc-46d7-adbe-7b01c8849e7b tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:25.962418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285cac4-fddc-46d7-adbe-7b01c8849e7b tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:25.962790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285cac4-fddc-46d7-adbe-7b01c8849e7b tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:25.963535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285cac4-fddc-46d7-adbe-7b01c8849e7b tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:25.965673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285cac4-fddc-46d7-adbe-7b01c8849e7b tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:26.026743 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1808/3584] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:25 2022] POST /identity/v3/users => generated 371 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:26.031095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eaa0b-fa9e-4eb5-be8d-4b65b3079651 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:26.046881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eaa0b-fa9e-4eb5-be8d-4b65b3079651 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:26.065003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c3deb-c43f-488e-835c-ed61f8ed8940 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90b44e3397f4d71aec446ec33db92b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6deb361b7941467fb7165dc2cac092e1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:26.067992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c3deb-c43f-488e-835c-ed61f8ed8940 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:26.069953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c3deb-c43f-488e-835c-ed61f8ed8940 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:26.071818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c3deb-c43f-488e-835c-ed61f8ed8940 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:26.073913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c3deb-c43f-488e-835c-ed61f8ed8940 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:05:26.077106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c3deb-c43f-488e-835c-ed61f8ed8940 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:26.081966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c3deb-c43f-488e-835c-ed61f8ed8940 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:26.202507 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1777/3585] 10.222.0.23 () {58 vars in 1157 bytes} [Sat Aug 20 21:05:26 2022] GET /identity/v3/users?domain_id=default => generated 8316 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:26.204988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285cac4-fddc-46d7-adbe-7b01c8849e7b tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:26.232679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285cac4-fddc-46d7-adbe-7b01c8849e7b tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:26.247155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90b44e3397f4d71aec446ec33db92b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6deb361b7941467fb7165dc2cac092e1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:26.249462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:26.249837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:26.250212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] PATH_INFO: `/v3/users/91db9028f1444e1181949b98a040f254` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:26.289433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorizing `identity:update_user(user_id=91db9028f1444e1181949b98a040f254)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:26.290813 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:05:26.290813 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:05:26.290813 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:05:26.293482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:26.361759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '91db9028f1444e1181949b98a040f254'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:26.379649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invalidating the token cache because user 91db9028f1444e1181949b98a040f254 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:26.380095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:26.381003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:26.386276 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1809/3586] 10.222.0.23 () {60 vars in 1244 bytes} [Sat Aug 20 21:05:26 2022] PATCH /identity/v3/users/91db9028f1444e1181949b98a040f254 => generated 485 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:26.388248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd059c1-8d23-455b-bd25-4fab8d0910b2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:26.389390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd059c1-8d23-455b-bd25-4fab8d0910b2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:26.389783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd059c1-8d23-455b-bd25-4fab8d0910b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:26.420223 ncdv-master-1 devstack@keystone.service[104119]: WARNING keystone.auth.plugins.core [None req-5bd059c1-8d23-455b-bd25-4fab8d0910b2 None None] User is disabled: 91db9028f1444e1181949b98a040f254: AssertionError: User is disabled: 91db9028f1444e1181949b98a040f254 Aug 20 21:05:26.431016 ncdv-master-1 devstack@keystone.service[104119]: WARNING keystone.server.flask.application [None req-5bd059c1-8d23-455b-bd25-4fab8d0910b2 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.23: keystone.exception.Unauthorized: The request you have made requires authentication. Aug 20 21:05:26.432906 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1778/3587] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:26 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 46 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Aug 20 21:05:26.437121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:26.439257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:26.462403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b5463-f9f4-49b8-a859-86d25e029d45 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:26.475483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b265f34-e360-4345-96cb-6dcc1f73d806 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90b44e3397f4d71aec446ec33db92b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6deb361b7941467fb7165dc2cac092e1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:26.476425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b265f34-e360-4345-96cb-6dcc1f73d806 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:26.476810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b265f34-e360-4345-96cb-6dcc1f73d806 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:26.477196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b265f34-e360-4345-96cb-6dcc1f73d806 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] PATH_INFO: `/v3/users/91db9028f1444e1181949b98a040f254` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:26.514100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b265f34-e360-4345-96cb-6dcc1f73d806 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorizing `identity:delete_user(user_id=91db9028f1444e1181949b98a040f254)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:26.516284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b265f34-e360-4345-96cb-6dcc1f73d806 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:26.631400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b265f34-e360-4345-96cb-6dcc1f73d806 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91db9028f1444e1181949b98a040f254'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:26.650072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b265f34-e360-4345-96cb-6dcc1f73d806 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91db9028f1444e1181949b98a040f254'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:26.676662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b265f34-e360-4345-96cb-6dcc1f73d806 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91db9028f1444e1181949b98a040f254'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:26.712469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b265f34-e360-4345-96cb-6dcc1f73d806 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91db9028f1444e1181949b98a040f254'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:26.713620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b265f34-e360-4345-96cb-6dcc1f73d806 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91db9028f1444e1181949b98a040f254'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:26.715437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=83,keystone:INSERT=5,keystone:UPDATE=1,keystone:DELETE=11 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:26.729541 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1810/3588] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:26 2022] DELETE /identity/v3/users/91db9028f1444e1181949b98a040f254 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:26.731113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd059c1-8d23-455b-bd25-4fab8d0910b2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:26.733242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5bd059c1-8d23-455b-bd25-4fab8d0910b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:26.872418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd059c1-8d23-455b-bd25-4fab8d0910b2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:26.893354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90b44e3397f4d71aec446ec33db92b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6deb361b7941467fb7165dc2cac092e1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:26.894128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:26.895853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:26.896428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] PATH_INFO: `/v3/projects/cd1eab1328804c0c9763c516da06b00b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:26.899178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cd1eab1328804c0c9763c516da06b00b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:26.904812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:26.943156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:26.997890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd1eab1328804c0c9763c516da06b00b', 'request_id': 'req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:27.012312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd1eab1328804c0c9763c516da06b00b', 'request_id': 'req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:27.029957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd1eab1328804c0c9763c516da06b00b', 'request_id': 'req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:27.059978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] The token cache is being invalidate because project cd1eab1328804c0c9763c516da06b00b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:27.065199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:27.071730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:27.079097 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1779/3589] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:26 2022] DELETE /identity/v3/projects/cd1eab1328804c0c9763c516da06b00b => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:27.084846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b265f34-e360-4345-96cb-6dcc1f73d806 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:27.116607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b265f34-e360-4345-96cb-6dcc1f73d806 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:27.332785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17b8a2-8d06-41a4-ba82-c0b18940732f tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90b44e3397f4d71aec446ec33db92b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6deb361b7941467fb7165dc2cac092e1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:27.333519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17b8a2-8d06-41a4-ba82-c0b18940732f tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:27.333689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17b8a2-8d06-41a4-ba82-c0b18940732f tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:27.334783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17b8a2-8d06-41a4-ba82-c0b18940732f tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:27.335446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17b8a2-8d06-41a4-ba82-c0b18940732f tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:27.340673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17b8a2-8d06-41a4-ba82-c0b18940732f tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-fb17b8a2-8d06-41a4-ba82-c0b18940732f tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Could not find domain: 0797ff9adc5246e4bd4dec9fce0a9e9b.: keystone.exception.DomainNotFound: Could not find domain: 0797ff9adc5246e4bd4dec9fce0a9e9b. Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 0797ff9adc5246e4bd4dec9fce0a9e9b. Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:27.390804 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:05:27.392011 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 987, in create_user Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_domain(domain_id) Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:05:27.393186 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:05:27.395664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:05:27.395664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:05:27.395664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 0797ff9adc5246e4bd4dec9fce0a9e9b. Aug 20 21:05:27.395664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:05:27.397095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:27.398688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:27.408626 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1811/3590] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:27 2022] POST /identity/v3/users => generated 112 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Aug 20 21:05:27.540140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5283d5-4df1-4e3e-b56c-a565f0c51a13 tempest-UsersNegativeTest-1592966368 tempest-UsersNegativeTest-1592966368-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:27.549431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:27.550115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:27.550391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:27.550661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] PATH_INFO: `/v3/users/a3f45acdcf924c6092f325cba9b8f971` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:27.553554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3f45acdcf924c6092f325cba9b8f971)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:27.556901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:27.642231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3f45acdcf924c6092f325cba9b8f971'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:27.657454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3f45acdcf924c6092f325cba9b8f971'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:27.681316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3f45acdcf924c6092f325cba9b8f971'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:27.682083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3f45acdcf924c6092f325cba9b8f971'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:27.682748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3f45acdcf924c6092f325cba9b8f971'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:27.699481 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1780/3591] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:27 2022] DELETE /identity/v3/users/a3f45acdcf924c6092f325cba9b8f971 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:27.707344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d2740-0b52-48e8-9065-0795d786d6b7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:27.707755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d2740-0b52-48e8-9065-0795d786d6b7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:27.708135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d2740-0b52-48e8-9065-0795d786d6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:27.743159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-960d2740-0b52-48e8-9065-0795d786d6b7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:27.745085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-960d2740-0b52-48e8-9065-0795d786d6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:27.906792 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1812/3592] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:27.952455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:27.955044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:28.056751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:28.156982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98703e6-bd20-472a-8172-9887185c563b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:28.167438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a35069-c66c-41bc-b120-1d5dc32cde2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:28.168086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a35069-c66c-41bc-b120-1d5dc32cde2b service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:28.168358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a35069-c66c-41bc-b120-1d5dc32cde2b service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:28.168646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a35069-c66c-41bc-b120-1d5dc32cde2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:28.204515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a35069-c66c-41bc-b120-1d5dc32cde2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:28.206047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a35069-c66c-41bc-b120-1d5dc32cde2b service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:28.248718 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1781/3593] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:29.215972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d2740-0b52-48e8-9065-0795d786d6b7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:29.217653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-960d2740-0b52-48e8-9065-0795d786d6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:29.237367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d2740-0b52-48e8-9065-0795d786d6b7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:29.252335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:29.254557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:29.255186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:29.256559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] PATH_INFO: `/v3/projects/46e18bb6ade449f0b34f1350b23ffde5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:29.260938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46e18bb6ade449f0b34f1350b23ffde5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:29.266742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:29.375882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e18bb6ade449f0b34f1350b23ffde5', 'request_id': 'req-698a166d-0e85-4129-8c22-bdcc93e31d29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:29.396259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e18bb6ade449f0b34f1350b23ffde5', 'request_id': 'req-698a166d-0e85-4129-8c22-bdcc93e31d29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:29.416748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e18bb6ade449f0b34f1350b23ffde5', 'request_id': 'req-698a166d-0e85-4129-8c22-bdcc93e31d29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:29.453791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] The token cache is being invalidate because project 46e18bb6ade449f0b34f1350b23ffde5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:29.454419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:29.456670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:29.459783 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1813/3594] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:29 2022] DELETE /identity/v3/projects/46e18bb6ade449f0b34f1350b23ffde5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:29.465056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a35069-c66c-41bc-b120-1d5dc32cde2b service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:29.482778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a35069-c66c-41bc-b120-1d5dc32cde2b service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:29.656595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c788d7-4023-4264-8756-fe48f51d4447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:29.657373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c788d7-4023-4264-8756-fe48f51d4447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:29.657706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c788d7-4023-4264-8756-fe48f51d4447 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:29.658037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c788d7-4023-4264-8756-fe48f51d4447 admin admin] PATH_INFO: `/v3/users/e90b44e3397f4d71aec446ec33db92b8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:29.660189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c788d7-4023-4264-8756-fe48f51d4447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e90b44e3397f4d71aec446ec33db92b8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:29.664845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c788d7-4023-4264-8756-fe48f51d4447 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:29.783227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-38c788d7-4023-4264-8756-fe48f51d4447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e90b44e3397f4d71aec446ec33db92b8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:29.804955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-38c788d7-4023-4264-8756-fe48f51d4447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e90b44e3397f4d71aec446ec33db92b8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:29.832509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-38c788d7-4023-4264-8756-fe48f51d4447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e90b44e3397f4d71aec446ec33db92b8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:29.833039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-38c788d7-4023-4264-8756-fe48f51d4447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e90b44e3397f4d71aec446ec33db92b8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:29.833947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-38c788d7-4023-4264-8756-fe48f51d4447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e90b44e3397f4d71aec446ec33db92b8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:29.866383 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1782/3595] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:29 2022] DELETE /identity/v3/users/e90b44e3397f4d71aec446ec33db92b8 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:30.807628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:30.809185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:30.823744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a166d-0e85-4129-8c22-bdcc93e31d29 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:30.859701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:30.907068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:30.907810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:30.907966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:30.908236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f admin admin] PATH_INFO: `/v3/projects/6deb361b7941467fb7165dc2cac092e1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:30.909402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6deb361b7941467fb7165dc2cac092e1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:30.911326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:30.967538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6deb361b7941467fb7165dc2cac092e1', 'request_id': 'req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:30.977107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6deb361b7941467fb7165dc2cac092e1', 'request_id': 'req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:30.983921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6deb361b7941467fb7165dc2cac092e1', 'request_id': 'req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:30.997926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f admin admin] The token cache is being invalidate because project 6deb361b7941467fb7165dc2cac092e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:30.998207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:30.998856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-577e34da-5b9d-43c4-ab5b-5e0b96b7c81f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:31.003936 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1814/3596] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:30 2022] DELETE /identity/v3/projects/6deb361b7941467fb7165dc2cac092e1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:31.006382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d157f-a1b5-44b2-be1c-a45698ff49ac None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.006905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d157f-a1b5-44b2-be1c-a45698ff49ac None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.007033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d157f-a1b5-44b2-be1c-a45698ff49ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.027366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fa3d157f-a1b5-44b2-be1c-a45698ff49ac None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:31.028496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fa3d157f-a1b5-44b2-be1c-a45698ff49ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:31.074802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:31.097240 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1783/3597] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:31.105261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2cdab-95b1-4e87-bbea-fb2f82e1fea4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.105760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2cdab-95b1-4e87-bbea-fb2f82e1fea4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.107520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2cdab-95b1-4e87-bbea-fb2f82e1fea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.141391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-73c2cdab-95b1-4e87-bbea-fb2f82e1fea4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:31.145129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-73c2cdab-95b1-4e87-bbea-fb2f82e1fea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:31.163791 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1815/3598] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:31.168218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d157f-a1b5-44b2-be1c-a45698ff49ac None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:31.178304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d157f-a1b5-44b2-be1c-a45698ff49ac None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:31.183231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732d685-255d-4cc4-bb2f-81c0a0f33ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:31.184046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732d685-255d-4cc4-bb2f-81c0a0f33ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.184314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732d685-255d-4cc4-bb2f-81c0a0f33ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.184611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732d685-255d-4cc4-bb2f-81c0a0f33ed9 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.185250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732d685-255d-4cc4-bb2f-81c0a0f33ed9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:05:31.185667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732d685-255d-4cc4-bb2f-81c0a0f33ed9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:31.187269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732d685-255d-4cc4-bb2f-81c0a0f33ed9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:31.200340 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1784/3599] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:31.204985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2cdab-95b1-4e87-bbea-fb2f82e1fea4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:31.214568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2cdab-95b1-4e87-bbea-fb2f82e1fea4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:31.221466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1886312-75d4-45fd-a47f-1029785e8c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:31.222219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1886312-75d4-45fd-a47f-1029785e8c5d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.222471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1886312-75d4-45fd-a47f-1029785e8c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.223715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1886312-75d4-45fd-a47f-1029785e8c5d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.230206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1886312-75d4-45fd-a47f-1029785e8c5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:31.232576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1886312-75d4-45fd-a47f-1029785e8c5d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:31.268219 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1816/3600] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:31 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:31.272497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732d685-255d-4cc4-bb2f-81c0a0f33ed9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:31.283084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732d685-255d-4cc4-bb2f-81c0a0f33ed9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:31.345310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615c10b-5ac2-4c17-8975-6eb285b21c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:31.346049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615c10b-5ac2-4c17-8975-6eb285b21c1f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.346386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615c10b-5ac2-4c17-8975-6eb285b21c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.346720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615c10b-5ac2-4c17-8975-6eb285b21c1f admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.347410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615c10b-5ac2-4c17-8975-6eb285b21c1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:31.349475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615c10b-5ac2-4c17-8975-6eb285b21c1f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:31.375233 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1785/3601] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:31 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:31.376493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1886312-75d4-45fd-a47f-1029785e8c5d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:31.387445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1886312-75d4-45fd-a47f-1029785e8c5d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:31.392096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa19daa-49b7-43c5-a0a7-ce823d73de17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:31.392745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa19daa-49b7-43c5-a0a7-ce823d73de17 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.393016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa19daa-49b7-43c5-a0a7-ce823d73de17 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.393263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa19daa-49b7-43c5-a0a7-ce823d73de17 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.393916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa19daa-49b7-43c5-a0a7-ce823d73de17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:31.395511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa19daa-49b7-43c5-a0a7-ce823d73de17 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:31.409716 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1817/3602] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:31.412009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615c10b-5ac2-4c17-8975-6eb285b21c1f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:31.422328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615c10b-5ac2-4c17-8975-6eb285b21c1f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:31.427068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d51159-4946-4248-9860-980e99252811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:31.427665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d51159-4946-4248-9860-980e99252811 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.427937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d51159-4946-4248-9860-980e99252811 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.428197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d51159-4946-4248-9860-980e99252811 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.428947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d51159-4946-4248-9860-980e99252811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:31.430576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d51159-4946-4248-9860-980e99252811 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:31.448474 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1786/3603] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:31.450305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa19daa-49b7-43c5-a0a7-ce823d73de17 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:31.464184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa19daa-49b7-43c5-a0a7-ce823d73de17 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:31.469476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a020b12-9039-4782-b292-38c709971995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:31.470306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a020b12-9039-4782-b292-38c709971995 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.470644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a020b12-9039-4782-b292-38c709971995 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.471013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a020b12-9039-4782-b292-38c709971995 admin admin] PATH_INFO: `/v3/projects/521028ff5c904d6e95ca46651b383701/users/bec72d9712dc467795d7633cc28f6c4c/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.490847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a020b12-9039-4782-b292-38c709971995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=521028ff5c904d6e95ca46651b383701, user_id=bec72d9712dc467795d7633cc28f6c4c, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:31.494285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a020b12-9039-4782-b292-38c709971995 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:31.514305 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1818/3604] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:31 2022] PUT /identity/v3/projects/521028ff5c904d6e95ca46651b383701/users/bec72d9712dc467795d7633cc28f6c4c/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:31.525243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee82913-6ec7-440b-bd96-cac6b33d9172 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.525522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee82913-6ec7-440b-bd96-cac6b33d9172 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.525792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee82913-6ec7-440b-bd96-cac6b33d9172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.562132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1ee82913-6ec7-440b-bd96-cac6b33d9172 None None] MFA Rules not processed for user `bec72d9712dc467795d7633cc28f6c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:31.563367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1ee82913-6ec7-440b-bd96-cac6b33d9172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:31.639790 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1787/3605] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:31.644987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a020b12-9039-4782-b292-38c709971995 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:31.659333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a020b12-9039-4782-b292-38c709971995 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:31.665292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40aa25-0149-466d-ba7d-848703cec18e tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec72d9712dc467795d7633cc28f6c4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '521028ff5c904d6e95ca46651b383701', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:31.665810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40aa25-0149-466d-ba7d-848703cec18e tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.666143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40aa25-0149-466d-ba7d-848703cec18e tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.666391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40aa25-0149-466d-ba7d-848703cec18e tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] PATH_INFO: `/v3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.668978 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1819/3606] 10.222.0.23 () {58 vars in 1104 bytes} [Sat Aug 20 21:05:31 2022] GET /identity/v3 => generated 255 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:31.673730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee82913-6ec7-440b-bd96-cac6b33d9172 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:31.686574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee82913-6ec7-440b-bd96-cac6b33d9172 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:31.691354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae9fbb-b550-414a-a92c-ad9ce831583d tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec72d9712dc467795d7633cc28f6c4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '521028ff5c904d6e95ca46651b383701', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:31.691981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae9fbb-b550-414a-a92c-ad9ce831583d tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.692699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae9fbb-b550-414a-a92c-ad9ce831583d tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.692699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae9fbb-b550-414a-a92c-ad9ce831583d tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] PATH_INFO: `/v3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.694289 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1788/3607] 10.222.0.23 () {58 vars in 1104 bytes} [Sat Aug 20 21:05:31 2022] GET /identity/v3 => generated 255 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:31.713494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40aa25-0149-466d-ba7d-848703cec18e tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:31.723515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40aa25-0149-466d-ba7d-848703cec18e tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:31.729633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c3062-88a0-426f-aa8e-fcdded85f5d4 tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec72d9712dc467795d7633cc28f6c4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '521028ff5c904d6e95ca46651b383701', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:31.730814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c3062-88a0-426f-aa8e-fcdded85f5d4 tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.731265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c3062-88a0-426f-aa8e-fcdded85f5d4 tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.731683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c3062-88a0-426f-aa8e-fcdded85f5d4 tempest-TestApiDiscovery-105153694 tempest-TestApiDiscovery-105153694-project] PATH_INFO: `/v3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.733630 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1820/3608] 10.222.0.23 () {58 vars in 1104 bytes} [Sat Aug 20 21:05:31 2022] GET /identity/v3 => generated 255 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:31.739030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52483b-6286-4884-bae5-11d91880aaeb None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.739522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52483b-6286-4884-bae5-11d91880aaeb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.739953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52483b-6286-4884-bae5-11d91880aaeb None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.742296 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1789/3609] 10.222.0.23 () {56 vars in 895 bytes} [Sat Aug 20 21:05:31 2022] GET /identity/ => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:05:31.748307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af2dd0-bd80-40d6-ad76-6c40ec908849 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.748921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af2dd0-bd80-40d6-ad76-6c40ec908849 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.749405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af2dd0-bd80-40d6-ad76-6c40ec908849 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.751080 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1821/3610] 10.222.0.23 () {56 vars in 895 bytes} [Sat Aug 20 21:05:31 2022] GET /identity/ => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:05:31.756525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52483b-6286-4884-bae5-11d91880aaeb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:31.801097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:31.830645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52483b-6286-4884-bae5-11d91880aaeb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:31.835870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:31.836881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.837329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.837622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] PATH_INFO: `/v3/users/bec72d9712dc467795d7633cc28f6c4c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.838974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec72d9712dc467795d7633cc28f6c4c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:31.840980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:31.905042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec72d9712dc467795d7633cc28f6c4c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:31.914744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec72d9712dc467795d7633cc28f6c4c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:31.927447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec72d9712dc467795d7633cc28f6c4c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:31.928136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec72d9712dc467795d7633cc28f6c4c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:31.928758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec72d9712dc467795d7633cc28f6c4c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:31.942460 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1790/3611] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:31 2022] DELETE /identity/v3/users/bec72d9712dc467795d7633cc28f6c4c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:31.947683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7b47c-059e-41a5-8982-9ce798d0ca2b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:31.948209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7b47c-059e-41a5-8982-9ce798d0ca2b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:31.948209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7b47c-059e-41a5-8982-9ce798d0ca2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:31.969134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-80f7b47c-059e-41a5-8982-9ce798d0ca2b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:31.971430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-80f7b47c-059e-41a5-8982-9ce798d0ca2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:32.015387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:32.065251 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1822/3612] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:32.083938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:32.086971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:32.168966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f017f87-56a1-4ab6-adc7-1cab923e1a73 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:32.173620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94867b9f-ee13-4df7-9c9e-d6daa9596ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:32.174369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94867b9f-ee13-4df7-9c9e-d6daa9596ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:32.174877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94867b9f-ee13-4df7-9c9e-d6daa9596ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:32.175229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94867b9f-ee13-4df7-9c9e-d6daa9596ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:32.187303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94867b9f-ee13-4df7-9c9e-d6daa9596ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:32.189162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94867b9f-ee13-4df7-9c9e-d6daa9596ca6 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:32.208556 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1791/3613] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:32.818267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7b47c-059e-41a5-8982-9ce798d0ca2b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:32.824207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-80f7b47c-059e-41a5-8982-9ce798d0ca2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:32.841526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7b47c-059e-41a5-8982-9ce798d0ca2b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:32.850002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f30dc-594f-4a31-9e43-73e78460f79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:32.851348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f30dc-594f-4a31-9e43-73e78460f79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:32.851913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f30dc-594f-4a31-9e43-73e78460f79d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:32.852523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f30dc-594f-4a31-9e43-73e78460f79d admin admin] PATH_INFO: `/v3/projects/521028ff5c904d6e95ca46651b383701` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:32.854491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f30dc-594f-4a31-9e43-73e78460f79d admin admin] RBAC: Authorizing `identity:delete_project(project_id=521028ff5c904d6e95ca46651b383701)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:32.859396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f30dc-594f-4a31-9e43-73e78460f79d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:32.932810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ae7f30dc-594f-4a31-9e43-73e78460f79d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '521028ff5c904d6e95ca46651b383701', 'request_id': 'req-ae7f30dc-594f-4a31-9e43-73e78460f79d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:32.943795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ae7f30dc-594f-4a31-9e43-73e78460f79d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '521028ff5c904d6e95ca46651b383701', 'request_id': 'req-ae7f30dc-594f-4a31-9e43-73e78460f79d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:32.950962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ae7f30dc-594f-4a31-9e43-73e78460f79d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '521028ff5c904d6e95ca46651b383701', 'request_id': 'req-ae7f30dc-594f-4a31-9e43-73e78460f79d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:32.964124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ae7f30dc-594f-4a31-9e43-73e78460f79d admin admin] The token cache is being invalidate because project 521028ff5c904d6e95ca46651b383701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:32.964712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ae7f30dc-594f-4a31-9e43-73e78460f79d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:32.965654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ae7f30dc-594f-4a31-9e43-73e78460f79d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:32.967450 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1823/3614] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:32 2022] DELETE /identity/v3/projects/521028ff5c904d6e95ca46651b383701 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:32.982271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b37a1-4864-459a-b594-b206a47fcb9d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:32.982271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b37a1-4864-459a-b594-b206a47fcb9d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:32.982271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b37a1-4864-459a-b594-b206a47fcb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:32.999762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-403b37a1-4864-459a-b594-b206a47fcb9d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:33.001066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-403b37a1-4864-459a-b594-b206a47fcb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:33.140801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:33.161338 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1792/3615] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:33.169634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5d4ba-0093-44f4-a2c5-7957bb1876ba None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.169917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5d4ba-0093-44f4-a2c5-7957bb1876ba None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.170182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5d4ba-0093-44f4-a2c5-7957bb1876ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.190821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b6a5d4ba-0093-44f4-a2c5-7957bb1876ba None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:33.191961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b6a5d4ba-0093-44f4-a2c5-7957bb1876ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:33.200364 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1824/3616] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:33.204756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b37a1-4864-459a-b594-b206a47fcb9d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:33.216546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b37a1-4864-459a-b594-b206a47fcb9d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:33.222818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eb1d3-61a1-4bc2-b776-0787d3a728a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:33.229123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb1d3-61a1-4bc2-b776-0787d3a728a8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.229123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb1d3-61a1-4bc2-b776-0787d3a728a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.229123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb1d3-61a1-4bc2-b776-0787d3a728a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.229123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828eb1d3-61a1-4bc2-b776-0787d3a728a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:05:33.229123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828eb1d3-61a1-4bc2-b776-0787d3a728a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:33.229123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828eb1d3-61a1-4bc2-b776-0787d3a728a8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:33.241109 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1793/3617] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:05:33.246458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5d4ba-0093-44f4-a2c5-7957bb1876ba None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:33.257717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5d4ba-0093-44f4-a2c5-7957bb1876ba None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:33.263570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa08ee-1bbd-42a5-8f40-b221ff5208ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:33.264624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa08ee-1bbd-42a5-8f40-b221ff5208ab admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.265076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa08ee-1bbd-42a5-8f40-b221ff5208ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.265447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa08ee-1bbd-42a5-8f40-b221ff5208ab admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.266258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa08ee-1bbd-42a5-8f40-b221ff5208ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:33.268959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa08ee-1bbd-42a5-8f40-b221ff5208ab admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:33.330164 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1825/3618] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:33 2022] POST /identity/v3/projects => generated 361 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:33.334049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eb1d3-61a1-4bc2-b776-0787d3a728a8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:33.347231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eb1d3-61a1-4bc2-b776-0787d3a728a8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:33.465173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d9da4-fc85-4020-8adf-1cdd37074642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:33.465882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d9da4-fc85-4020-8adf-1cdd37074642 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.466225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d9da4-fc85-4020-8adf-1cdd37074642 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.466636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d9da4-fc85-4020-8adf-1cdd37074642 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.467299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d9da4-fc85-4020-8adf-1cdd37074642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:33.469452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d9da4-fc85-4020-8adf-1cdd37074642 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:33.498446 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1794/3619] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:33 2022] POST /identity/v3/users => generated 281 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:33.499286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa08ee-1bbd-42a5-8f40-b221ff5208ab admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:33.512402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa08ee-1bbd-42a5-8f40-b221ff5208ab admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:33.517956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab80e3b-ed5d-4775-8aff-e2b91b6c87a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:33.518707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab80e3b-ed5d-4775-8aff-e2b91b6c87a0 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.519074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab80e3b-ed5d-4775-8aff-e2b91b6c87a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.519432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab80e3b-ed5d-4775-8aff-e2b91b6c87a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.520168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab80e3b-ed5d-4775-8aff-e2b91b6c87a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:33.521897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab80e3b-ed5d-4775-8aff-e2b91b6c87a0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:33.533554 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1826/3620] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:05:33.537616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d9da4-fc85-4020-8adf-1cdd37074642 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:33.548809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d9da4-fc85-4020-8adf-1cdd37074642 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:33.554360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d307b48-4d29-455f-ad36-14d95bea241f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:33.555477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d307b48-4d29-455f-ad36-14d95bea241f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.555747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d307b48-4d29-455f-ad36-14d95bea241f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.556028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d307b48-4d29-455f-ad36-14d95bea241f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.556693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d307b48-4d29-455f-ad36-14d95bea241f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:33.562197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d307b48-4d29-455f-ad36-14d95bea241f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:33.579239 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1795/3621] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:05:33.583511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab80e3b-ed5d-4775-8aff-e2b91b6c87a0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:33.594129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab80e3b-ed5d-4775-8aff-e2b91b6c87a0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:33.598914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1962a1-2731-43dc-bc51-1a96e9034c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:33.599656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1962a1-2731-43dc-bc51-1a96e9034c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.599997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1962a1-2731-43dc-bc51-1a96e9034c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.600336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1962a1-2731-43dc-bc51-1a96e9034c95 admin admin] PATH_INFO: `/v3/projects/630a74681a0e4e14bc22ee0e731d4905/users/86b38e21c5bf4d84a6f425b86e456cc5/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.633869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1962a1-2731-43dc-bc51-1a96e9034c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=630a74681a0e4e14bc22ee0e731d4905, user_id=86b38e21c5bf4d84a6f425b86e456cc5, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:33.636714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1962a1-2731-43dc-bc51-1a96e9034c95 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:33.654336 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1827/3622] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:33 2022] PUT /identity/v3/projects/630a74681a0e4e14bc22ee0e731d4905/users/86b38e21c5bf4d84a6f425b86e456cc5/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:33.664958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5d9e1-387f-4147-980f-26df38b2346d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.665179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5d9e1-387f-4147-980f-26df38b2346d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.665433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5d9e1-387f-4147-980f-26df38b2346d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.693199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a5c5d9e1-387f-4147-980f-26df38b2346d None None] MFA Rules not processed for user `86b38e21c5bf4d84a6f425b86e456cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:33.694346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a5c5d9e1-387f-4147-980f-26df38b2346d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:33.782163 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1796/3623] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:33.787617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0e936-2d57-4520-81f7-d4ae57ede62a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.787899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0e936-2d57-4520-81f7-d4ae57ede62a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.788170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0e936-2d57-4520-81f7-d4ae57ede62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.806274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cbe0e936-2d57-4520-81f7-d4ae57ede62a None None] MFA Rules not processed for user `86b38e21c5bf4d84a6f425b86e456cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:33.807466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cbe0e936-2d57-4520-81f7-d4ae57ede62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:33.811721 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1828/3624] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:33 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 25 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:05:33.817289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f363b3-d9ff-4e6e-9166-5edf46261724 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.817635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f363b3-d9ff-4e6e-9166-5edf46261724 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.817969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f363b3-d9ff-4e6e-9166-5edf46261724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.837855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-42f363b3-d9ff-4e6e-9166-5edf46261724 None None] MFA Rules not processed for user `86b38e21c5bf4d84a6f425b86e456cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:33.839035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-42f363b3-d9ff-4e6e-9166-5edf46261724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:33.842285 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1797/3625] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:33 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:05:33.846590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0e936-2d57-4520-81f7-d4ae57ede62a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:33.858497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0e936-2d57-4520-81f7-d4ae57ede62a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:33.863330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94f769-8117-4aff-aa18-4bc7c232a59c tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b38e21c5bf4d84a6f425b86e456cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '630a74681a0e4e14bc22ee0e731d4905', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:33.864017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94f769-8117-4aff-aa18-4bc7c232a59c tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] REQUEST_METHOD: `HEAD` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.864362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94f769-8117-4aff-aa18-4bc7c232a59c tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.864729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94f769-8117-4aff-aa18-4bc7c232a59c tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.872200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94f769-8117-4aff-aa18-4bc7c232a59c tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: Authorizing `identity:check_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:33.874232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94f769-8117-4aff-aa18-4bc7c232a59c tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:33.882166 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1829/3626] 10.222.0.23 () {56 vars in 1264 bytes} [Sat Aug 20 21:05:33 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 358 bytes (0 switches on core 0) Aug 20 21:05:33.886154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f363b3-d9ff-4e6e-9166-5edf46261724 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:33.898861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f363b3-d9ff-4e6e-9166-5edf46261724 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:33.904029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9bb84-da46-48f8-b761-a293101cb107 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b38e21c5bf4d84a6f425b86e456cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '630a74681a0e4e14bc22ee0e731d4905', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:33.904753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9bb84-da46-48f8-b761-a293101cb107 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.905105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9bb84-da46-48f8-b761-a293101cb107 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.905451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9bb84-da46-48f8-b761-a293101cb107 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.912585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9bb84-da46-48f8-b761-a293101cb107 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:33.914919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9bb84-da46-48f8-b761-a293101cb107 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:33.922869 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1798/3627] 10.222.0.23 () {56 vars in 1263 bytes} [Sat Aug 20 21:05:33 2022] GET /identity/v3/auth/tokens => generated 346 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 21:05:33.926585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94f769-8117-4aff-aa18-4bc7c232a59c tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:33.937956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94f769-8117-4aff-aa18-4bc7c232a59c tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:33.942300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426afe95-54ca-41a9-b456-b88867224ae1 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b38e21c5bf4d84a6f425b86e456cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '630a74681a0e4e14bc22ee0e731d4905', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:33.942971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426afe95-54ca-41a9-b456-b88867224ae1 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.943426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426afe95-54ca-41a9-b456-b88867224ae1 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.943701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426afe95-54ca-41a9-b456-b88867224ae1 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.951253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426afe95-54ca-41a9-b456-b88867224ae1 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:33.953014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426afe95-54ca-41a9-b456-b88867224ae1 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:33.969461 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1830/3628] 10.222.0.23 () {56 vars in 1266 bytes} [Sat Aug 20 21:05:33 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:33.973526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9bb84-da46-48f8-b761-a293101cb107 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:33.986552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9bb84-da46-48f8-b761-a293101cb107 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:33.994874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97249d16-278e-41e3-9a01-687c297aeccc tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b38e21c5bf4d84a6f425b86e456cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '630a74681a0e4e14bc22ee0e731d4905', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:33.995583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97249d16-278e-41e3-9a01-687c297aeccc tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] REQUEST_METHOD: `HEAD` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:33.995866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97249d16-278e-41e3-9a01-687c297aeccc tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:33.996138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97249d16-278e-41e3-9a01-687c297aeccc tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:33.998464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-97249d16-278e-41e3-9a01-687c297aeccc tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-97249d16-278e-41e3-9a01-687c297aeccc tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 283, in get Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:check_token') Aug 20 21:05:34.038957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.check_revocation(token) Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Aug 20 21:05:34.040048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Aug 20 21:05:34.042143 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:34.042143 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:34.042143 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Aug 20 21:05:34.042143 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Aug 20 21:05:34.042143 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Aug 20 21:05:34.042143 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:05:34.042143 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1799/3629] 10.222.0.23 () {56 vars in 1264 bytes} [Sat Aug 20 21:05:33 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 68 msecs (HTTP/1.1 404) 5 headers in 183 bytes (0 switches on core 0) Aug 20 21:05:34.046575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a2338-7aee-47bb-b712-67c855b52c2c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:34.046893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a2338-7aee-47bb-b712-67c855b52c2c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:34.047174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a2338-7aee-47bb-b712-67c855b52c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:34.080700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-312a2338-7aee-47bb-b712-67c855b52c2c None None] MFA Rules not processed for user `86b38e21c5bf4d84a6f425b86e456cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:34.082229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-312a2338-7aee-47bb-b712-67c855b52c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:34.086054 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1831/3630] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:34 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 20 21:05:34.090794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97249d16-278e-41e3-9a01-687c297aeccc tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:34.105538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97249d16-278e-41e3-9a01-687c297aeccc tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:34.110909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958eb01-63db-48bb-831b-c88399e4dec7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b38e21c5bf4d84a6f425b86e456cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '630a74681a0e4e14bc22ee0e731d4905', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:34.111532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958eb01-63db-48bb-831b-c88399e4dec7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:34.111808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958eb01-63db-48bb-831b-c88399e4dec7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:34.112071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958eb01-63db-48bb-831b-c88399e4dec7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:34.123401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958eb01-63db-48bb-831b-c88399e4dec7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:34.126933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958eb01-63db-48bb-831b-c88399e4dec7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:34.135276 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1800/3631] 10.222.0.23 () {56 vars in 1263 bytes} [Sat Aug 20 21:05:34 2022] GET /identity/v3/auth/tokens => generated 346 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 21:05:34.139304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a2338-7aee-47bb-b712-67c855b52c2c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:34.157682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a2338-7aee-47bb-b712-67c855b52c2c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:34.163947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede869c-d4dd-47af-9310-270ed84ac5e0 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b38e21c5bf4d84a6f425b86e456cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '630a74681a0e4e14bc22ee0e731d4905', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:34.164927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede869c-d4dd-47af-9310-270ed84ac5e0 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:34.165373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede869c-d4dd-47af-9310-270ed84ac5e0 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:34.165777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede869c-d4dd-47af-9310-270ed84ac5e0 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:34.174982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede869c-d4dd-47af-9310-270ed84ac5e0 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:34.176997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede869c-d4dd-47af-9310-270ed84ac5e0 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:34.180845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:34.207001 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1832/3632] 10.222.0.23 () {56 vars in 1266 bytes} [Sat Aug 20 21:05:34 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:34.215224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958eb01-63db-48bb-831b-c88399e4dec7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:34.225049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958eb01-63db-48bb-831b-c88399e4dec7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:34.239327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8c81b-1108-4f43-b821-6259297d53a7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b38e21c5bf4d84a6f425b86e456cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '630a74681a0e4e14bc22ee0e731d4905', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:34.240174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8c81b-1108-4f43-b821-6259297d53a7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:34.241195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8c81b-1108-4f43-b821-6259297d53a7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:34.241647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8c81b-1108-4f43-b821-6259297d53a7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:34.257461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6fe8c81b-1108-4f43-b821-6259297d53a7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-6fe8c81b-1108-4f43-b821-6259297d53a7 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 20 21:05:34.271985 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.check_revocation(token) Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Aug 20 21:05:34.273215 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Aug 20 21:05:34.274382 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:05:34.274382 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:05:34.274382 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Aug 20 21:05:34.274382 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Aug 20 21:05:34.274382 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Aug 20 21:05:34.274382 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:05:34.279161 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1801/3633] 10.222.0.23 () {56 vars in 1263 bytes} [Sat Aug 20 21:05:34 2022] GET /identity/v3/auth/tokens => generated 80 bytes in 66 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 21:05:34.280643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede869c-d4dd-47af-9310-270ed84ac5e0 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:34.375546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede869c-d4dd-47af-9310-270ed84ac5e0 tempest-TokensV3Test-1998053215 tempest-TokensV3Test-1998053215-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:34.381073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:34.382005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:34.382574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:34.382841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] PATH_INFO: `/v3/users/86b38e21c5bf4d84a6f425b86e456cc5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:34.384277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86b38e21c5bf4d84a6f425b86e456cc5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:34.386627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:34.441597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86b38e21c5bf4d84a6f425b86e456cc5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:34.450584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86b38e21c5bf4d84a6f425b86e456cc5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:34.457980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86b38e21c5bf4d84a6f425b86e456cc5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:34.474243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86b38e21c5bf4d84a6f425b86e456cc5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:34.475557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86b38e21c5bf4d84a6f425b86e456cc5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:34.484222 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1833/3634] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:34 2022] DELETE /identity/v3/users/86b38e21c5bf4d84a6f425b86e456cc5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:34.489122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f2981-2d16-4136-ab3a-e0030803f7ad None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:34.489360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f2981-2d16-4136-ab3a-e0030803f7ad None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:34.489895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f2981-2d16-4136-ab3a-e0030803f7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:34.499420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:34.511474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b72f2981-2d16-4136-ab3a-e0030803f7ad None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:34.512832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b72f2981-2d16-4136-ab3a-e0030803f7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:34.593199 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1802/3635] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:34.599656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:34.601245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:34.666474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac26b1-83c5-48d7-a433-7a6d9f1204e4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:34.670306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71267ff5-3010-4fe6-a65e-7f9583eb4bbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:34.670919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71267ff5-3010-4fe6-a65e-7f9583eb4bbc service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:34.671194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71267ff5-3010-4fe6-a65e-7f9583eb4bbc service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:34.671463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71267ff5-3010-4fe6-a65e-7f9583eb4bbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:34.681497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71267ff5-3010-4fe6-a65e-7f9583eb4bbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:34.683285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71267ff5-3010-4fe6-a65e-7f9583eb4bbc service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:34.693437 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1834/3636] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:35.034568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f2981-2d16-4136-ab3a-e0030803f7ad None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:35.036266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b72f2981-2d16-4136-ab3a-e0030803f7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:35.059113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f2981-2d16-4136-ab3a-e0030803f7ad None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:35.065928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:35.067222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:35.067495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:35.068596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4 admin admin] PATH_INFO: `/v3/projects/630a74681a0e4e14bc22ee0e731d4905` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:35.069741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=630a74681a0e4e14bc22ee0e731d4905)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:35.073108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:35.136622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '630a74681a0e4e14bc22ee0e731d4905', 'request_id': 'req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:35.144333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '630a74681a0e4e14bc22ee0e731d4905', 'request_id': 'req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:35.154946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '630a74681a0e4e14bc22ee0e731d4905', 'request_id': 'req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:35.168004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4 admin admin] The token cache is being invalidate because project 630a74681a0e4e14bc22ee0e731d4905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:35.168294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:35.168980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-730fd11c-a5c1-44c8-b5cf-1f40bff9d9d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:35.170517 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1803/3637] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:35 2022] DELETE /identity/v3/projects/630a74681a0e4e14bc22ee0e731d4905 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:35.179351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f16f87-bf84-4b0d-b782-ba86aa50da71 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:35.179960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f16f87-bf84-4b0d-b782-ba86aa50da71 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:35.179960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f16f87-bf84-4b0d-b782-ba86aa50da71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:35.203733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a2f16f87-bf84-4b0d-b782-ba86aa50da71 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:35.204992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a2f16f87-bf84-4b0d-b782-ba86aa50da71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:35.215060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:35.282997 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1835/3638] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:35.291728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb0def-43bb-4272-adbd-ef2a26dc1fc1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:35.292724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb0def-43bb-4272-adbd-ef2a26dc1fc1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:35.293268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb0def-43bb-4272-adbd-ef2a26dc1fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:35.327712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b2eb0def-43bb-4272-adbd-ef2a26dc1fc1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:35.328998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b2eb0def-43bb-4272-adbd-ef2a26dc1fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:35.337797 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1804/3639] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:35.341949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f16f87-bf84-4b0d-b782-ba86aa50da71 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:35.352482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f16f87-bf84-4b0d-b782-ba86aa50da71 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:35.358124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ba037-c2f3-4a29-abc9-af9f28073484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:35.358601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ba037-c2f3-4a29-abc9-af9f28073484 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:35.358871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ba037-c2f3-4a29-abc9-af9f28073484 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:35.359144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ba037-c2f3-4a29-abc9-af9f28073484 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:35.359801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ba037-c2f3-4a29-abc9-af9f28073484 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:05:35.361893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ba037-c2f3-4a29-abc9-af9f28073484 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:35.363583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ba037-c2f3-4a29-abc9-af9f28073484 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:35.380444 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1836/3640] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:35.382596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb0def-43bb-4272-adbd-ef2a26dc1fc1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:35.393478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb0def-43bb-4272-adbd-ef2a26dc1fc1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:35.400850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84dc8e-2842-4778-9099-dedaf7f7a4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:35.401478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84dc8e-2842-4778-9099-dedaf7f7a4e4 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:35.401746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84dc8e-2842-4778-9099-dedaf7f7a4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:35.402077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84dc8e-2842-4778-9099-dedaf7f7a4e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:35.402803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84dc8e-2842-4778-9099-dedaf7f7a4e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:35.404809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84dc8e-2842-4778-9099-dedaf7f7a4e4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:35.444071 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1805/3641] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:35 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:35.449249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ba037-c2f3-4a29-abc9-af9f28073484 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:35.461302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ba037-c2f3-4a29-abc9-af9f28073484 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:35.528360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de1ecf-04f9-4722-9930-c27e49c792ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:35.529204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de1ecf-04f9-4722-9930-c27e49c792ad admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:35.529204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de1ecf-04f9-4722-9930-c27e49c792ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:35.529432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de1ecf-04f9-4722-9930-c27e49c792ad admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:35.530147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de1ecf-04f9-4722-9930-c27e49c792ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:35.533006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de1ecf-04f9-4722-9930-c27e49c792ad admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:35.600861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84dc8e-2842-4778-9099-dedaf7f7a4e4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:35.609520 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1837/3642] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:35 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:35.613237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84dc8e-2842-4778-9099-dedaf7f7a4e4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:35.618822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb080c0-ffa7-4344-ae1d-028c0feb2d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:35.619703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb080c0-ffa7-4344-ae1d-028c0feb2d68 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:35.620154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb080c0-ffa7-4344-ae1d-028c0feb2d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:35.620659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb080c0-ffa7-4344-ae1d-028c0feb2d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:35.621499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb080c0-ffa7-4344-ae1d-028c0feb2d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:35.623131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb080c0-ffa7-4344-ae1d-028c0feb2d68 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:35.634664 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1806/3643] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:05:35.638750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de1ecf-04f9-4722-9930-c27e49c792ad admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:35.653314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de1ecf-04f9-4722-9930-c27e49c792ad admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:35.658456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780444a-4d78-4fd0-9308-097472fb2bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:35.659071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780444a-4d78-4fd0-9308-097472fb2bfa admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:35.659345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780444a-4d78-4fd0-9308-097472fb2bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:35.659614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780444a-4d78-4fd0-9308-097472fb2bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:35.660248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780444a-4d78-4fd0-9308-097472fb2bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:35.661893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780444a-4d78-4fd0-9308-097472fb2bfa admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:35.673359 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1838/3644] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:35.677655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb080c0-ffa7-4344-ae1d-028c0feb2d68 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:35.688406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb080c0-ffa7-4344-ae1d-028c0feb2d68 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:35.693363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3735795-50e9-40d9-b5f0-e2d4a05a4e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:35.694103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3735795-50e9-40d9-b5f0-e2d4a05a4e1b admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:35.694461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3735795-50e9-40d9-b5f0-e2d4a05a4e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:35.694833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3735795-50e9-40d9-b5f0-e2d4a05a4e1b admin admin] PATH_INFO: `/v3/projects/89048540216d4d389d016047d81a8a39/users/d6933fe5e49249c1a1fd34100cfadcae/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:35.711537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3735795-50e9-40d9-b5f0-e2d4a05a4e1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=89048540216d4d389d016047d81a8a39, user_id=d6933fe5e49249c1a1fd34100cfadcae, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:35.714343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3735795-50e9-40d9-b5f0-e2d4a05a4e1b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:35.731891 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1807/3645] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:35 2022] PUT /identity/v3/projects/89048540216d4d389d016047d81a8a39/users/d6933fe5e49249c1a1fd34100cfadcae/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:35.743493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08496b-c06b-405a-b0a9-ca1a5275a4f2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:35.743761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08496b-c06b-405a-b0a9-ca1a5275a4f2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:35.744021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08496b-c06b-405a-b0a9-ca1a5275a4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:35.771945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0b08496b-c06b-405a-b0a9-ca1a5275a4f2 None None] MFA Rules not processed for user `d6933fe5e49249c1a1fd34100cfadcae`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:35.773242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0b08496b-c06b-405a-b0a9-ca1a5275a4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:35.879037 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1839/3646] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:35.883924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3735795-50e9-40d9-b5f0-e2d4a05a4e1b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:35.915851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:35.976671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3735795-50e9-40d9-b5f0-e2d4a05a4e1b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:35.985072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec48fd-6390-4355-b3d5-d7b2d934b120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:35.993011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec48fd-6390-4355-b3d5-d7b2d934b120 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:35.993386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec48fd-6390-4355-b3d5-d7b2d934b120 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:35.993828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec48fd-6390-4355-b3d5-d7b2d934b120 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:35.994914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec48fd-6390-4355-b3d5-d7b2d934b120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:35.997138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec48fd-6390-4355-b3d5-d7b2d934b120 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:36.043566 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1808/3647] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:35 2022] POST /identity/v3/projects => generated 367 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:36.047684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08496b-c06b-405a-b0a9-ca1a5275a4f2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:36.060486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08496b-c06b-405a-b0a9-ca1a5275a4f2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:36.066191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:36.127002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b161dd-324b-440a-a01a-91656920bd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:36.127896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b161dd-324b-440a-a01a-91656920bd63 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:36.128324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b161dd-324b-440a-a01a-91656920bd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:36.128795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b161dd-324b-440a-a01a-91656920bd63 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:36.129579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b161dd-324b-440a-a01a-91656920bd63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:36.131665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b161dd-324b-440a-a01a-91656920bd63 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:36.163770 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1840/3648] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:36 2022] POST /identity/v3/users => generated 284 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:36.167385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec48fd-6390-4355-b3d5-d7b2d934b120 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:36.186701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec48fd-6390-4355-b3d5-d7b2d934b120 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:36.191580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f6c7f-fdd1-455f-a96e-171d9ceb8a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:36.192282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f6c7f-fdd1-455f-a96e-171d9ceb8a22 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:36.192647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f6c7f-fdd1-455f-a96e-171d9ceb8a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:36.192981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f6c7f-fdd1-455f-a96e-171d9ceb8a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:36.193683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f6c7f-fdd1-455f-a96e-171d9ceb8a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:36.196613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f6c7f-fdd1-455f-a96e-171d9ceb8a22 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:36.208562 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1809/3649] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:36.213564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b161dd-324b-440a-a01a-91656920bd63 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:36.230087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b161dd-324b-440a-a01a-91656920bd63 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:36.233888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f5844-360b-40fd-8dc1-b7652dd9ec1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:36.234635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f5844-360b-40fd-8dc1-b7652dd9ec1b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:36.235096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f5844-360b-40fd-8dc1-b7652dd9ec1b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:36.235754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f5844-360b-40fd-8dc1-b7652dd9ec1b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:36.237010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f5844-360b-40fd-8dc1-b7652dd9ec1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:36.238728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f5844-360b-40fd-8dc1-b7652dd9ec1b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:36.250155 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1841/3650] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:36.254417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f6c7f-fdd1-455f-a96e-171d9ceb8a22 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:36.265204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f6c7f-fdd1-455f-a96e-171d9ceb8a22 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:36.271597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43966422-cd78-47e0-8b17-ea1aa8dc790d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:36.272677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43966422-cd78-47e0-8b17-ea1aa8dc790d admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:36.273440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43966422-cd78-47e0-8b17-ea1aa8dc790d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:36.273926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43966422-cd78-47e0-8b17-ea1aa8dc790d admin admin] PATH_INFO: `/v3/projects/31927230d2d049ae80da03d3ee8c84a1/users/53f850e77ad544ad8eaba00845b3fa98/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:36.297794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43966422-cd78-47e0-8b17-ea1aa8dc790d admin admin] RBAC: Authorizing `identity:create_grant(project_id=31927230d2d049ae80da03d3ee8c84a1, user_id=53f850e77ad544ad8eaba00845b3fa98, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:36.302533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43966422-cd78-47e0-8b17-ea1aa8dc790d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:36.330325 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1810/3651] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:36 2022] PUT /identity/v3/projects/31927230d2d049ae80da03d3ee8c84a1/users/53f850e77ad544ad8eaba00845b3fa98/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:36.343429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01c9a5-e0ae-4fcb-925a-1125f01db08c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:36.343708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01c9a5-e0ae-4fcb-925a-1125f01db08c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:36.343973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01c9a5-e0ae-4fcb-925a-1125f01db08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:36.385116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8f01c9a5-e0ae-4fcb-925a-1125f01db08c None None] MFA Rules not processed for user `53f850e77ad544ad8eaba00845b3fa98`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:36.386977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8f01c9a5-e0ae-4fcb-925a-1125f01db08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:36.631760 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1842/3652] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:36.641930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43966422-cd78-47e0-8b17-ea1aa8dc790d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:36.644181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-43966422-cd78-47e0-8b17-ea1aa8dc790d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:36.754580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43966422-cd78-47e0-8b17-ea1aa8dc790d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:36.761824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060c7f3-3e7d-4c93-8188-c22d0d4972fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:36.762135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060c7f3-3e7d-4c93-8188-c22d0d4972fb service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:36.762422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060c7f3-3e7d-4c93-8188-c22d0d4972fb service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:36.762683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060c7f3-3e7d-4c93-8188-c22d0d4972fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:36.827269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060c7f3-3e7d-4c93-8188-c22d0d4972fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:36.828770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060c7f3-3e7d-4c93-8188-c22d0d4972fb service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:36.837863 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1811/3653] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:36.983396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01c9a5-e0ae-4fcb-925a-1125f01db08c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:36.998220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01c9a5-e0ae-4fcb-925a-1125f01db08c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:37.007346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929962c-e0d0-4ac1-b067-5120339d799d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:37.008566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929962c-e0d0-4ac1-b067-5120339d799d service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:37.008862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929962c-e0d0-4ac1-b067-5120339d799d service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:37.009606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929962c-e0d0-4ac1-b067-5120339d799d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:37.021732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929962c-e0d0-4ac1-b067-5120339d799d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:37.024596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929962c-e0d0-4ac1-b067-5120339d799d service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:37.044762 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1843/3654] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:37.504956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618d656-058f-447f-8d8c-ac76d1a62669 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:37.505417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618d656-058f-447f-8d8c-ac76d1a62669 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:37.505580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618d656-058f-447f-8d8c-ac76d1a62669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:37.534036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a618d656-058f-447f-8d8c-ac76d1a62669 None None] MFA Rules not processed for user `b20aea7088dc4e4fadb737c906cf7ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:37.535434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a618d656-058f-447f-8d8c-ac76d1a62669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:37.540341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:37.623595 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1812/3655] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:38.423989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929962c-e0d0-4ac1-b067-5120339d799d service glance] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:38.496715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929962c-e0d0-4ac1-b067-5120339d799d service glance] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:38.504136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad55e22-b57b-4039-b2dd-5a458b422329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:38.504825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad55e22-b57b-4039-b2dd-5a458b422329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:38.505092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad55e22-b57b-4039-b2dd-5a458b422329 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:38.505389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad55e22-b57b-4039-b2dd-5a458b422329 admin admin] PATH_INFO: `/v3/users/d6933fe5e49249c1a1fd34100cfadcae` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:38.506853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad55e22-b57b-4039-b2dd-5a458b422329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6933fe5e49249c1a1fd34100cfadcae)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:38.509588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad55e22-b57b-4039-b2dd-5a458b422329 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:38.515074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:38.592990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ad55e22-b57b-4039-b2dd-5a458b422329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6933fe5e49249c1a1fd34100cfadcae'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:38.611947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ad55e22-b57b-4039-b2dd-5a458b422329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6933fe5e49249c1a1fd34100cfadcae'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:38.629085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ad55e22-b57b-4039-b2dd-5a458b422329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6933fe5e49249c1a1fd34100cfadcae'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:38.653766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ad55e22-b57b-4039-b2dd-5a458b422329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6933fe5e49249c1a1fd34100cfadcae'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:38.654617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ad55e22-b57b-4039-b2dd-5a458b422329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6933fe5e49249c1a1fd34100cfadcae'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:38.659550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618d656-058f-447f-8d8c-ac76d1a62669 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:38.661567 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1844/3656] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:38 2022] DELETE /identity/v3/users/d6933fe5e49249c1a1fd34100cfadcae => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:38.661847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a618d656-058f-447f-8d8c-ac76d1a62669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:38.666160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c902ae-fc6b-4205-967d-4df59cbc50c5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:38.666437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c902ae-fc6b-4205-967d-4df59cbc50c5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:38.666703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c902ae-fc6b-4205-967d-4df59cbc50c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:38.689816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f2c902ae-fc6b-4205-967d-4df59cbc50c5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:38.691119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f2c902ae-fc6b-4205-967d-4df59cbc50c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:38.777199 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1845/3657] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:38.788073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618d656-058f-447f-8d8c-ac76d1a62669 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:38.789029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c902ae-fc6b-4205-967d-4df59cbc50c5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:38.790772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f2c902ae-fc6b-4205-967d-4df59cbc50c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:38.863478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce386eea-cda7-4291-afb8-314e73caa7ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:38.864098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce386eea-cda7-4291-afb8-314e73caa7ae service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:38.865392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce386eea-cda7-4291-afb8-314e73caa7ae service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:38.865680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce386eea-cda7-4291-afb8-314e73caa7ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:38.868125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ce386eea-cda7-4291-afb8-314e73caa7ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:38.874350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c902ae-fc6b-4205-967d-4df59cbc50c5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:38.880332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665ef5d-1d96-4d7e-b289-7d4bf085b0ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:38.881295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665ef5d-1d96-4d7e-b289-7d4bf085b0ee service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:38.881999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665ef5d-1d96-4d7e-b289-7d4bf085b0ee service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:38.882585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce386eea-cda7-4291-afb8-314e73caa7ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:38.884002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665ef5d-1d96-4d7e-b289-7d4bf085b0ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:38.884581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce386eea-cda7-4291-afb8-314e73caa7ae service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:38.900587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665ef5d-1d96-4d7e-b289-7d4bf085b0ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:38.901937 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1813/3658] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:05:38.902048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665ef5d-1d96-4d7e-b289-7d4bf085b0ee service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:38.916443 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1846/3659] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:05:39.474402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce386eea-cda7-4291-afb8-314e73caa7ae service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:39.475937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ce386eea-cda7-4291-afb8-314e73caa7ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:39.491158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce386eea-cda7-4291-afb8-314e73caa7ae service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:39.499768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:39.500466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:39.500741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:39.501002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] PATH_INFO: `/v3/projects/89048540216d4d389d016047d81a8a39` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:39.502751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89048540216d4d389d016047d81a8a39)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:39.504098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:39.580547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:39.592458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89048540216d4d389d016047d81a8a39', 'request_id': 'req-83b5dc8b-626c-4665-8b7f-e86f061d61b7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:39.600690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89048540216d4d389d016047d81a8a39', 'request_id': 'req-83b5dc8b-626c-4665-8b7f-e86f061d61b7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:39.626850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89048540216d4d389d016047d81a8a39', 'request_id': 'req-83b5dc8b-626c-4665-8b7f-e86f061d61b7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:39.655454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] The token cache is being invalidate because project 89048540216d4d389d016047d81a8a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:39.655454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:39.655454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:39.655454 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1814/3660] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:39 2022] DELETE /identity/v3/projects/89048540216d4d389d016047d81a8a39 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:39.659125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665ef5d-1d96-4d7e-b289-7d4bf085b0ee service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:39.678906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665ef5d-1d96-4d7e-b289-7d4bf085b0ee service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:39.711450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:39.746588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d278e-fa88-4c34-822c-1695a5f2e553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:39.746588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d278e-fa88-4c34-822c-1695a5f2e553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:39.746913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d278e-fa88-4c34-822c-1695a5f2e553 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:39.746913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d278e-fa88-4c34-822c-1695a5f2e553 admin admin] PATH_INFO: `/v3/users/53f850e77ad544ad8eaba00845b3fa98` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:39.748074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d278e-fa88-4c34-822c-1695a5f2e553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53f850e77ad544ad8eaba00845b3fa98)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:39.750255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d278e-fa88-4c34-822c-1695a5f2e553 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:39.810610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a75d278e-fa88-4c34-822c-1695a5f2e553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53f850e77ad544ad8eaba00845b3fa98'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:39.823858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a75d278e-fa88-4c34-822c-1695a5f2e553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53f850e77ad544ad8eaba00845b3fa98'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:39.832674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a75d278e-fa88-4c34-822c-1695a5f2e553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53f850e77ad544ad8eaba00845b3fa98'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:39.846636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a75d278e-fa88-4c34-822c-1695a5f2e553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53f850e77ad544ad8eaba00845b3fa98'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:39.847347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a75d278e-fa88-4c34-822c-1695a5f2e553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53f850e77ad544ad8eaba00845b3fa98'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:39.853334 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1847/3661] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:39 2022] DELETE /identity/v3/users/53f850e77ad544ad8eaba00845b3fa98 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:40.279050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:40.281505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:40.302040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5dc8b-626c-4665-8b7f-e86f061d61b7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:40.385148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279db299-5c78-4283-a45e-32bc0869601f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:40.386158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279db299-5c78-4283-a45e-32bc0869601f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:40.386584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279db299-5c78-4283-a45e-32bc0869601f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:40.387439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279db299-5c78-4283-a45e-32bc0869601f admin admin] PATH_INFO: `/v3/projects/31927230d2d049ae80da03d3ee8c84a1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:40.388357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279db299-5c78-4283-a45e-32bc0869601f admin admin] RBAC: Authorizing `identity:delete_project(project_id=31927230d2d049ae80da03d3ee8c84a1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:40.390345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279db299-5c78-4283-a45e-32bc0869601f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:40.454704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-279db299-5c78-4283-a45e-32bc0869601f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31927230d2d049ae80da03d3ee8c84a1', 'request_id': 'req-279db299-5c78-4283-a45e-32bc0869601f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:40.462714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-279db299-5c78-4283-a45e-32bc0869601f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31927230d2d049ae80da03d3ee8c84a1', 'request_id': 'req-279db299-5c78-4283-a45e-32bc0869601f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:40.473759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-279db299-5c78-4283-a45e-32bc0869601f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31927230d2d049ae80da03d3ee8c84a1', 'request_id': 'req-279db299-5c78-4283-a45e-32bc0869601f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:40.490396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-279db299-5c78-4283-a45e-32bc0869601f admin admin] The token cache is being invalidate because project 31927230d2d049ae80da03d3ee8c84a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:40.490691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-279db299-5c78-4283-a45e-32bc0869601f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:40.491198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-279db299-5c78-4283-a45e-32bc0869601f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:40.493667 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1815/3662] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:40 2022] DELETE /identity/v3/projects/31927230d2d049ae80da03d3ee8c84a1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:40.500351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43754d14-c27c-4723-8e63-cd9f5997e487 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:40.501286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43754d14-c27c-4723-8e63-cd9f5997e487 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:40.501788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43754d14-c27c-4723-8e63-cd9f5997e487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:40.548153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-43754d14-c27c-4723-8e63-cd9f5997e487 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:40.549909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-43754d14-c27c-4723-8e63-cd9f5997e487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:40.654217 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1848/3663] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:40.671380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dbaed-6ad5-4a21-8f4c-a30d49782478 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:40.671380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dbaed-6ad5-4a21-8f4c-a30d49782478 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:40.671380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dbaed-6ad5-4a21-8f4c-a30d49782478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:40.692600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b03dbaed-6ad5-4a21-8f4c-a30d49782478 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:40.693826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b03dbaed-6ad5-4a21-8f4c-a30d49782478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:40.704805 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1816/3664] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:40.709300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43754d14-c27c-4723-8e63-cd9f5997e487 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:40.722443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43754d14-c27c-4723-8e63-cd9f5997e487 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:40.735567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa53f27-3bf3-4b90-9d36-5548f2f568af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:40.736494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa53f27-3bf3-4b90-9d36-5548f2f568af admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:40.738201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa53f27-3bf3-4b90-9d36-5548f2f568af admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:40.738657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa53f27-3bf3-4b90-9d36-5548f2f568af admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:40.739446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa53f27-3bf3-4b90-9d36-5548f2f568af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:05:40.739898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa53f27-3bf3-4b90-9d36-5548f2f568af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:40.741924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa53f27-3bf3-4b90-9d36-5548f2f568af admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:40.755249 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1849/3665] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:40.759383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dbaed-6ad5-4a21-8f4c-a30d49782478 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:40.769423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dbaed-6ad5-4a21-8f4c-a30d49782478 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:40.774988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e34ec-9a48-46d8-8801-8a9270b37caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:40.775604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e34ec-9a48-46d8-8801-8a9270b37caf admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:40.775872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e34ec-9a48-46d8-8801-8a9270b37caf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:40.776162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e34ec-9a48-46d8-8801-8a9270b37caf admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:40.776858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e34ec-9a48-46d8-8801-8a9270b37caf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:40.778779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e34ec-9a48-46d8-8801-8a9270b37caf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:40.809912 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1817/3666] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:40 2022] POST /identity/v3/projects => generated 411 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:40.814505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa53f27-3bf3-4b90-9d36-5548f2f568af admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:40.828080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa53f27-3bf3-4b90-9d36-5548f2f568af admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:40.859155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:40.905243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa826665-cbc7-4751-9bdd-7a791b957fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:40.906081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa826665-cbc7-4751-9bdd-7a791b957fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:40.906462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa826665-cbc7-4751-9bdd-7a791b957fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:40.906815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa826665-cbc7-4751-9bdd-7a791b957fe5 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:40.907655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa826665-cbc7-4751-9bdd-7a791b957fe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:40.910168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa826665-cbc7-4751-9bdd-7a791b957fe5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:40.948102 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1850/3667] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:40 2022] POST /identity/v3/users => generated 306 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:40.949422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e34ec-9a48-46d8-8801-8a9270b37caf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:40.963018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e34ec-9a48-46d8-8801-8a9270b37caf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:40.969234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f416f0-1894-4c6f-bd5f-a07da40b385e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:40.969423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f416f0-1894-4c6f-bd5f-a07da40b385e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:40.969565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f416f0-1894-4c6f-bd5f-a07da40b385e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:40.969835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f416f0-1894-4c6f-bd5f-a07da40b385e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:40.970626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f416f0-1894-4c6f-bd5f-a07da40b385e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:40.972447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f416f0-1894-4c6f-bd5f-a07da40b385e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:40.988105 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1818/3668] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:05:40.989346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa826665-cbc7-4751-9bdd-7a791b957fe5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:41.006303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa826665-cbc7-4751-9bdd-7a791b957fe5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:41.013860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b25c7-46fe-44c7-8b20-bdf473188c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:41.014762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b25c7-46fe-44c7-8b20-bdf473188c3a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:41.015183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b25c7-46fe-44c7-8b20-bdf473188c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:41.015595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b25c7-46fe-44c7-8b20-bdf473188c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:41.016632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b25c7-46fe-44c7-8b20-bdf473188c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:41.019571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b25c7-46fe-44c7-8b20-bdf473188c3a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:41.035571 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1851/3669] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:41.040183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f416f0-1894-4c6f-bd5f-a07da40b385e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:41.071938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f416f0-1894-4c6f-bd5f-a07da40b385e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:41.087577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82d031-c822-4eb6-a242-92ccccdedfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:41.088514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82d031-c822-4eb6-a242-92ccccdedfd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:41.089055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82d031-c822-4eb6-a242-92ccccdedfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:41.089482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82d031-c822-4eb6-a242-92ccccdedfd9 admin admin] PATH_INFO: `/v3/projects/b6421999029c4ce8834418433989f481/users/5bf09f8c22dc418cba097b5807e2ad04/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:41.110626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82d031-c822-4eb6-a242-92ccccdedfd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6421999029c4ce8834418433989f481, user_id=5bf09f8c22dc418cba097b5807e2ad04, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:41.113925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82d031-c822-4eb6-a242-92ccccdedfd9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:41.118421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:41.161799 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1819/3670] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:41 2022] PUT /identity/v3/projects/b6421999029c4ce8834418433989f481/users/5bf09f8c22dc418cba097b5807e2ad04/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:41.175568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87887a03-e1fd-4fe1-9f9f-4a49a8317547 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:41.175835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87887a03-e1fd-4fe1-9f9f-4a49a8317547 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:41.176102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87887a03-e1fd-4fe1-9f9f-4a49a8317547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:41.217463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-87887a03-e1fd-4fe1-9f9f-4a49a8317547 None None] MFA Rules not processed for user `5bf09f8c22dc418cba097b5807e2ad04`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:41.218692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-87887a03-e1fd-4fe1-9f9f-4a49a8317547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:41.316445 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1852/3671] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:41 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:41.320323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82d031-c822-4eb6-a242-92ccccdedfd9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:41.390398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82d031-c822-4eb6-a242-92ccccdedfd9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:41.396812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd27bfe-e0c4-4284-ad75-053de3667fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:41.397411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd27bfe-e0c4-4284-ad75-053de3667fec admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:41.397700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd27bfe-e0c4-4284-ad75-053de3667fec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:41.398157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd27bfe-e0c4-4284-ad75-053de3667fec admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:41.398907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd27bfe-e0c4-4284-ad75-053de3667fec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:41.400846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd27bfe-e0c4-4284-ad75-053de3667fec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:41.437375 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1820/3672] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:41 2022] POST /identity/v3/projects => generated 409 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:41.443542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87887a03-e1fd-4fe1-9f9f-4a49a8317547 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:41.458940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87887a03-e1fd-4fe1-9f9f-4a49a8317547 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:41.522661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc7921-fdea-41d6-a994-3cbf0c483936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:41.523421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc7921-fdea-41d6-a994-3cbf0c483936 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:41.523738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc7921-fdea-41d6-a994-3cbf0c483936 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:41.523934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc7921-fdea-41d6-a994-3cbf0c483936 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:41.524614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc7921-fdea-41d6-a994-3cbf0c483936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:41.526622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc7921-fdea-41d6-a994-3cbf0c483936 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:41.552255 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1853/3673] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:41 2022] POST /identity/v3/users => generated 311 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:41.556642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd27bfe-e0c4-4284-ad75-053de3667fec admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:41.568283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd27bfe-e0c4-4284-ad75-053de3667fec admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:41.574232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38adfd8b-575e-40b1-8d9e-e1d432a97763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:41.574951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adfd8b-575e-40b1-8d9e-e1d432a97763 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:41.576606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adfd8b-575e-40b1-8d9e-e1d432a97763 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:41.576606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adfd8b-575e-40b1-8d9e-e1d432a97763 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:41.576606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38adfd8b-575e-40b1-8d9e-e1d432a97763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:41.578214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38adfd8b-575e-40b1-8d9e-e1d432a97763 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:41.591487 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1821/3674] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:41.595434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc7921-fdea-41d6-a994-3cbf0c483936 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:41.609261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc7921-fdea-41d6-a994-3cbf0c483936 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:41.614575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fef6b-fd8b-48d0-9820-eab3743ea168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:41.615340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fef6b-fd8b-48d0-9820-eab3743ea168 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:41.615676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fef6b-fd8b-48d0-9820-eab3743ea168 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:41.616012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fef6b-fd8b-48d0-9820-eab3743ea168 admin admin] PATH_INFO: `/v3/projects/5527d8c657b64b4d917ff56f7b898f83/users/17d4d0a1df47497ab41b41b59a5175e2/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:41.633206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018fef6b-fd8b-48d0-9820-eab3743ea168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5527d8c657b64b4d917ff56f7b898f83, user_id=17d4d0a1df47497ab41b41b59a5175e2, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:41.636071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018fef6b-fd8b-48d0-9820-eab3743ea168 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:41.656436 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1854/3675] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:41 2022] PUT /identity/v3/projects/5527d8c657b64b4d917ff56f7b898f83/users/17d4d0a1df47497ab41b41b59a5175e2/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:41.659787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38adfd8b-575e-40b1-8d9e-e1d432a97763 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:41.671346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38adfd8b-575e-40b1-8d9e-e1d432a97763 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:41.765391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b92fc-87b5-4a68-a69c-badceb93e924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:41.765954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b92fc-87b5-4a68-a69c-badceb93e924 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:41.766232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b92fc-87b5-4a68-a69c-badceb93e924 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:41.766498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b92fc-87b5-4a68-a69c-badceb93e924 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:41.767147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b92fc-87b5-4a68-a69c-badceb93e924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:41.769251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b92fc-87b5-4a68-a69c-badceb93e924 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:41.785653 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1822/3676] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:41.787257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fef6b-fd8b-48d0-9820-eab3743ea168 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:41.810558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fef6b-fd8b-48d0-9820-eab3743ea168 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:41.818604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfed2b8-ba22-4e2b-b0ce-cfed7b9087d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:41.818604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfed2b8-ba22-4e2b-b0ce-cfed7b9087d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:41.818604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfed2b8-ba22-4e2b-b0ce-cfed7b9087d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:41.818989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfed2b8-ba22-4e2b-b0ce-cfed7b9087d1 admin admin] PATH_INFO: `/v3/projects/5527d8c657b64b4d917ff56f7b898f83/users/17d4d0a1df47497ab41b41b59a5175e2/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:41.818989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfed2b8-ba22-4e2b-b0ce-cfed7b9087d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5527d8c657b64b4d917ff56f7b898f83, user_id=17d4d0a1df47497ab41b41b59a5175e2, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:41.823994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfed2b8-ba22-4e2b-b0ce-cfed7b9087d1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:41.827881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:41.841486 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1855/3677] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:41 2022] PUT /identity/v3/projects/5527d8c657b64b4d917ff56f7b898f83/users/17d4d0a1df47497ab41b41b59a5175e2/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:41.852316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b37085-4cbc-4b99-8c07-efc6fe35280a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:41.852786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b37085-4cbc-4b99-8c07-efc6fe35280a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:41.853059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b37085-4cbc-4b99-8c07-efc6fe35280a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:41.887149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e4b37085-4cbc-4b99-8c07-efc6fe35280a None None] MFA Rules not processed for user `17d4d0a1df47497ab41b41b59a5175e2`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:41.888420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e4b37085-4cbc-4b99-8c07-efc6fe35280a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:41.962285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:41.987785 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1823/3678] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:41 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:41.999407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfed2b8-ba22-4e2b-b0ce-cfed7b9087d1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:42.001442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7bfed2b8-ba22-4e2b-b0ce-cfed7b9087d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:42.068633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfed2b8-ba22-4e2b-b0ce-cfed7b9087d1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:42.072876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfeca0-2948-40d3-86d3-42c779b8559b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:42.073508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfeca0-2948-40d3-86d3-42c779b8559b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:42.073771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfeca0-2948-40d3-86d3-42c779b8559b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:42.074046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfeca0-2948-40d3-86d3-42c779b8559b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:42.084184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfeca0-2948-40d3-86d3-42c779b8559b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:42.085811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfeca0-2948-40d3-86d3-42c779b8559b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:42.096956 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1856/3679] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:41 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:45.269434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b37085-4cbc-4b99-8c07-efc6fe35280a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:45.398871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b37085-4cbc-4b99-8c07-efc6fe35280a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:45.404806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:45.405475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:45.405741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:45.406006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] PATH_INFO: `/v3/users/5bf09f8c22dc418cba097b5807e2ad04` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:45.407265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bf09f8c22dc418cba097b5807e2ad04)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:45.409332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:45.486585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bf09f8c22dc418cba097b5807e2ad04'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:45.496598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bf09f8c22dc418cba097b5807e2ad04'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:45.519434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bf09f8c22dc418cba097b5807e2ad04'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:45.520228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bf09f8c22dc418cba097b5807e2ad04'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:45.521537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bf09f8c22dc418cba097b5807e2ad04'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:45.548636 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1824/3680] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:45 2022] DELETE /identity/v3/users/5bf09f8c22dc418cba097b5807e2ad04 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:45.553519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147afa3e-ba88-4c28-926b-81ef766cf03b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:45.554143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147afa3e-ba88-4c28-926b-81ef766cf03b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:45.554143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147afa3e-ba88-4c28-926b-81ef766cf03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:45.594186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-147afa3e-ba88-4c28-926b-81ef766cf03b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:45.596714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-147afa3e-ba88-4c28-926b-81ef766cf03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:45.674004 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1857/3681] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:45.724285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:45.726057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:45.811930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac7162-0a34-4b24-8072-5b523ad93325 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:45.816295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca87da-2c27-4ebb-a303-64e29518f467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:45.817620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca87da-2c27-4ebb-a303-64e29518f467 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:45.818051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca87da-2c27-4ebb-a303-64e29518f467 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:45.818469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca87da-2c27-4ebb-a303-64e29518f467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:45.830412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca87da-2c27-4ebb-a303-64e29518f467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:45.832239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca87da-2c27-4ebb-a303-64e29518f467 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:45.846019 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1825/3682] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:05:47.107423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147afa3e-ba88-4c28-926b-81ef766cf03b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:47.110258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-147afa3e-ba88-4c28-926b-81ef766cf03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:47.128771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147afa3e-ba88-4c28-926b-81ef766cf03b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:47.137707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:47.138799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:47.139221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:47.139663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] PATH_INFO: `/v3/projects/b6421999029c4ce8834418433989f481` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:47.141281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6421999029c4ce8834418433989f481)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:47.145599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:47.224528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6421999029c4ce8834418433989f481', 'request_id': 'req-46b68402-c75d-4f00-9e6e-32473cc29cba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.251681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6421999029c4ce8834418433989f481', 'request_id': 'req-46b68402-c75d-4f00-9e6e-32473cc29cba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.259586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6421999029c4ce8834418433989f481', 'request_id': 'req-46b68402-c75d-4f00-9e6e-32473cc29cba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.289590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] The token cache is being invalidate because project b6421999029c4ce8834418433989f481 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:47.291660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.293410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.297909 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1858/3683] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:47 2022] DELETE /identity/v3/projects/b6421999029c4ce8834418433989f481 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:47.301729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca87da-2c27-4ebb-a303-64e29518f467 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:47.320174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca87da-2c27-4ebb-a303-64e29518f467 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:47.367230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:47.387351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:47.389543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:47.434938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:47.495732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:47.496591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:47.496865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:47.497136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] PATH_INFO: `/v3/users/17d4d0a1df47497ab41b41b59a5175e2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:47.498747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] RBAC: Authorizing `identity:delete_user(user_id=17d4d0a1df47497ab41b41b59a5175e2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:47.500852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:47.522989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b68402-c75d-4f00-9e6e-32473cc29cba admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:47.541888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:47.543175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:47.543175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:47.543375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] PATH_INFO: `/v3/users/c8b2c80165614ad58df38edc24b47715` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:47.544912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b2c80165614ad58df38edc24b47715)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:47.547338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:47.572297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17d4d0a1df47497ab41b41b59a5175e2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.588191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17d4d0a1df47497ab41b41b59a5175e2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.627317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17d4d0a1df47497ab41b41b59a5175e2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.628732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17d4d0a1df47497ab41b41b59a5175e2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.630110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17d4d0a1df47497ab41b41b59a5175e2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.647405 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1826/3684] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:47 2022] DELETE /identity/v3/users/17d4d0a1df47497ab41b41b59a5175e2 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:47.690939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b2c80165614ad58df38edc24b47715'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.701080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b2c80165614ad58df38edc24b47715'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.709740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b2c80165614ad58df38edc24b47715'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.722907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b2c80165614ad58df38edc24b47715'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.723604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b2c80165614ad58df38edc24b47715'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:47.732724 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1859/3685] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:47 2022] DELETE /identity/v3/users/c8b2c80165614ad58df38edc24b47715 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:47.974042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:47.975506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:48.039653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:48.041167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:48.076284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304d923-85de-4e31-8e82-74462a9812be admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:48.127633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:48.136572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05195815-5726-4197-999b-13455145c6d8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:48.143218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:48.144561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:48.144841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:48.145106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29 admin admin] PATH_INFO: `/v3/projects/c58f159460cd4bcd8327c5b5ed84b1fe` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:48.146794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c58f159460cd4bcd8327c5b5ed84b1fe)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:48.149499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:48.160418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:48.161371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:48.161863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:48.162312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55 admin admin] PATH_INFO: `/v3/projects/5527d8c657b64b4d917ff56f7b898f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:48.163812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5527d8c657b64b4d917ff56f7b898f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:48.165967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:48.203830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:48.212316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c58f159460cd4bcd8327c5b5ed84b1fe', 'request_id': 'req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:48.232748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5527d8c657b64b4d917ff56f7b898f83', 'request_id': 'req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:48.236205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c58f159460cd4bcd8327c5b5ed84b1fe', 'request_id': 'req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:48.243498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5527d8c657b64b4d917ff56f7b898f83', 'request_id': 'req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:48.255367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5527d8c657b64b4d917ff56f7b898f83', 'request_id': 'req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:48.272270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55 admin admin] The token cache is being invalidate because project 5527d8c657b64b4d917ff56f7b898f83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:48.273349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:48.274146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9a3f9a61-3ba0-4188-9e9a-cc6374ce5e55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:48.279359 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1827/3686] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:47 2022] DELETE /identity/v3/projects/5527d8c657b64b4d917ff56f7b898f83 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:48.281298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041a455-495f-490f-b1ad-3793e70d3c1c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:48.281580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041a455-495f-490f-b1ad-3793e70d3c1c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:48.281849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041a455-495f-490f-b1ad-3793e70d3c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:48.282566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c58f159460cd4bcd8327c5b5ed84b1fe', 'request_id': 'req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:48.305694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29 admin admin] The token cache is being invalidate because project c58f159460cd4bcd8327c5b5ed84b1fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:48.305999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:48.306835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fe9f436e-b7ac-47cf-a53e-3996e5e98c29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:48.309354 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1860/3687] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:48 2022] DELETE /identity/v3/projects/c58f159460cd4bcd8327c5b5ed84b1fe => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:48.315673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1516445-d67c-4b2e-9312-812c4ec419f2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:48.316060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1516445-d67c-4b2e-9312-812c4ec419f2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:48.316179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8041a455-495f-490f-b1ad-3793e70d3c1c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:48.316680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1516445-d67c-4b2e-9312-812c4ec419f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:48.317581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8041a455-495f-490f-b1ad-3793e70d3c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:48.354497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e1516445-d67c-4b2e-9312-812c4ec419f2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:48.356177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e1516445-d67c-4b2e-9312-812c4ec419f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:48.400577 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1828/3688] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:48.409985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87afacd-7f13-4989-b017-6c8e1b8c15a5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:48.410794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87afacd-7f13-4989-b017-6c8e1b8c15a5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:48.412245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87afacd-7f13-4989-b017-6c8e1b8c15a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:48.447787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a87afacd-7f13-4989-b017-6c8e1b8c15a5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:48.447787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a87afacd-7f13-4989-b017-6c8e1b8c15a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:48.456560 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1829/3689] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:48.461152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87afacd-7f13-4989-b017-6c8e1b8c15a5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:48.480810 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1861/3690] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:48.486162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87afacd-7f13-4989-b017-6c8e1b8c15a5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:48.490424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f5d72-4ebb-453a-a6ed-fc56f2a82028 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:48.490634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f5d72-4ebb-453a-a6ed-fc56f2a82028 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:48.491119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f5d72-4ebb-453a-a6ed-fc56f2a82028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:48.493830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76392c9b-a998-4921-932a-713aba5d7640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:48.494464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76392c9b-a998-4921-932a-713aba5d7640 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:48.494742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76392c9b-a998-4921-932a-713aba5d7640 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:48.495403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76392c9b-a998-4921-932a-713aba5d7640 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:48.495804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76392c9b-a998-4921-932a-713aba5d7640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:05:48.496150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76392c9b-a998-4921-932a-713aba5d7640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:48.497997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76392c9b-a998-4921-932a-713aba5d7640 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:48.512842 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1830/3691] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:05:48.517363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76392c9b-a998-4921-932a-713aba5d7640 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:48.524434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-101f5d72-4ebb-453a-a6ed-fc56f2a82028 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:48.525887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-101f5d72-4ebb-453a-a6ed-fc56f2a82028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:48.528291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76392c9b-a998-4921-932a-713aba5d7640 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:48.533212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5f750-e332-4d29-b285-9e7977f52863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:48.533916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5f750-e332-4d29-b285-9e7977f52863 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:48.534386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5f750-e332-4d29-b285-9e7977f52863 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:48.534720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5f750-e332-4d29-b285-9e7977f52863 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:48.534859 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1862/3692] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:48.535453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5f750-e332-4d29-b285-9e7977f52863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:48.537488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5f750-e332-4d29-b285-9e7977f52863 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:48.539377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f5d72-4ebb-453a-a6ed-fc56f2a82028 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:48.550470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f5d72-4ebb-453a-a6ed-fc56f2a82028 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:48.555882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb1559-dd81-408c-95ba-87ab8d08ea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:48.556479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb1559-dd81-408c-95ba-87ab8d08ea91 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:48.556642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb1559-dd81-408c-95ba-87ab8d08ea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:48.556985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb1559-dd81-408c-95ba-87ab8d08ea91 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:48.557815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb1559-dd81-408c-95ba-87ab8d08ea91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:05:48.558147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb1559-dd81-408c-95ba-87ab8d08ea91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:48.560548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb1559-dd81-408c-95ba-87ab8d08ea91 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:48.584434 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1863/3693] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:05:48.585797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb1559-dd81-408c-95ba-87ab8d08ea91 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:48.596940 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1831/3694] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:48 2022] POST /identity/v3/projects => generated 367 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:48.600080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5f750-e332-4d29-b285-9e7977f52863 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:48.604951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb1559-dd81-408c-95ba-87ab8d08ea91 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:48.649374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:48.706213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff298e8a-acdf-4c3d-aa87-bc86a462dbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:48.706975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff298e8a-acdf-4c3d-aa87-bc86a462dbc9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:48.707106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff298e8a-acdf-4c3d-aa87-bc86a462dbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:48.707441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff298e8a-acdf-4c3d-aa87-bc86a462dbc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:48.708320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff298e8a-acdf-4c3d-aa87-bc86a462dbc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:48.709998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff298e8a-acdf-4c3d-aa87-bc86a462dbc9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:48.716974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5f750-e332-4d29-b285-9e7977f52863 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:48.722362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e71f0-0874-4a62-a096-7d5e02bb247b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:48.722991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e71f0-0874-4a62-a096-7d5e02bb247b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:48.723269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e71f0-0874-4a62-a096-7d5e02bb247b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:48.723537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e71f0-0874-4a62-a096-7d5e02bb247b admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:48.724170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e71f0-0874-4a62-a096-7d5e02bb247b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:48.726704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e71f0-0874-4a62-a096-7d5e02bb247b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:48.762935 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1832/3695] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:48 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:48.766991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e71f0-0874-4a62-a096-7d5e02bb247b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:48.769641 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1864/3696] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:48 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:48.777383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff298e8a-acdf-4c3d-aa87-bc86a462dbc9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:48.793338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e71f0-0874-4a62-a096-7d5e02bb247b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:48.854486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:48.887951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5c51f-ebf8-4130-88ec-4a9c25480aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:48.888666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5c51f-ebf8-4130-88ec-4a9c25480aae admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:48.888945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5c51f-ebf8-4130-88ec-4a9c25480aae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:48.889213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5c51f-ebf8-4130-88ec-4a9c25480aae admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:48.896609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a5c51f-ebf8-4130-88ec-4a9c25480aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:48.898811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a5c51f-ebf8-4130-88ec-4a9c25480aae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:48.938747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff298e8a-acdf-4c3d-aa87-bc86a462dbc9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:48.943125 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1833/3697] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:48.944082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5c51f-ebf8-4130-88ec-4a9c25480aae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:48.974585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8150b4-c34b-4150-97c8-9fcdb38965d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:48.975225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8150b4-c34b-4150-97c8-9fcdb38965d0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:48.975497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8150b4-c34b-4150-97c8-9fcdb38965d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:48.975936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8150b4-c34b-4150-97c8-9fcdb38965d0 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:48.979565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5c51f-ebf8-4130-88ec-4a9c25480aae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:48.983790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8150b4-c34b-4150-97c8-9fcdb38965d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:48.985826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8150b4-c34b-4150-97c8-9fcdb38965d0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:48.999217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc7b60-8204-4539-8b96-23f90e7580e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:49.000059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc7b60-8204-4539-8b96-23f90e7580e2 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.000187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc7b60-8204-4539-8b96-23f90e7580e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.000358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc7b60-8204-4539-8b96-23f90e7580e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.001041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc7b60-8204-4539-8b96-23f90e7580e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:49.002696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc7b60-8204-4539-8b96-23f90e7580e2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:49.021820 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1834/3698] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:49.027373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc7b60-8204-4539-8b96-23f90e7580e2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:49.037844 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1865/3699] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:48 2022] POST /identity/v3/users => generated 290 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:49.043043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8150b4-c34b-4150-97c8-9fcdb38965d0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:49.043271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc7b60-8204-4539-8b96-23f90e7580e2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:49.052720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c38c6e-5dd9-42e2-ad1f-d00a3ec92508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:49.053478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c38c6e-5dd9-42e2-ad1f-d00a3ec92508 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.053974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c38c6e-5dd9-42e2-ad1f-d00a3ec92508 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.054832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c38c6e-5dd9-42e2-ad1f-d00a3ec92508 admin admin] PATH_INFO: `/v3/projects/8597498ff38b4cadacc981db6c834fff/users/188be8acf9194905a4172cd48b792ddb/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.069020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8150b4-c34b-4150-97c8-9fcdb38965d0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:49.074293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c38c6e-5dd9-42e2-ad1f-d00a3ec92508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8597498ff38b4cadacc981db6c834fff, user_id=188be8acf9194905a4172cd48b792ddb, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:49.077037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c38c6e-5dd9-42e2-ad1f-d00a3ec92508 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:49.084390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4141d89-3341-4fa3-86b0-4128e8f5be0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:49.085901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4141d89-3341-4fa3-86b0-4128e8f5be0a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.086399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4141d89-3341-4fa3-86b0-4128e8f5be0a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.087222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4141d89-3341-4fa3-86b0-4128e8f5be0a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.088434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4141d89-3341-4fa3-86b0-4128e8f5be0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:49.090549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4141d89-3341-4fa3-86b0-4128e8f5be0a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:49.113134 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1835/3700] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:49 2022] PUT /identity/v3/projects/8597498ff38b4cadacc981db6c834fff/users/188be8acf9194905a4172cd48b792ddb/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:49.120642 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1866/3701] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:49.124723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c38c6e-5dd9-42e2-ad1f-d00a3ec92508 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:49.125877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c47d57-06b8-4746-bd72-4c5c5777b95a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.126515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c47d57-06b8-4746-bd72-4c5c5777b95a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.126935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c47d57-06b8-4746-bd72-4c5c5777b95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.197377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b7c47d57-06b8-4746-bd72-4c5c5777b95a None None] MFA Rules not processed for user `188be8acf9194905a4172cd48b792ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:49.198825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b7c47d57-06b8-4746-bd72-4c5c5777b95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:49.206437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c38c6e-5dd9-42e2-ad1f-d00a3ec92508 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:49.229580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b30ad2-de8f-4de9-bbf0-181407b6d789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:49.230207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b30ad2-de8f-4de9-bbf0-181407b6d789 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.231710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b30ad2-de8f-4de9-bbf0-181407b6d789 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.231976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b30ad2-de8f-4de9-bbf0-181407b6d789 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.233039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b30ad2-de8f-4de9-bbf0-181407b6d789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:49.236642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b30ad2-de8f-4de9-bbf0-181407b6d789 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:49.265427 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1836/3702] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:49.267354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b30ad2-de8f-4de9-bbf0-181407b6d789 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:49.289935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b30ad2-de8f-4de9-bbf0-181407b6d789 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:49.304315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31092a5b-af8e-4b7b-bd00-5413bc1318d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:49.305198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31092a5b-af8e-4b7b-bd00-5413bc1318d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.305714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31092a5b-af8e-4b7b-bd00-5413bc1318d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.306217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31092a5b-af8e-4b7b-bd00-5413bc1318d6 admin admin] PATH_INFO: `/v3/projects/af98ae4746f343b39d13b58f18c6c23e/users/77a750effc254df38f53b2c1743adeca/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.338765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:49.346357 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1867/3703] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:49.346945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31092a5b-af8e-4b7b-bd00-5413bc1318d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af98ae4746f343b39d13b58f18c6c23e, user_id=77a750effc254df38f53b2c1743adeca, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:49.347468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c47d57-06b8-4746-bd72-4c5c5777b95a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:49.352749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31092a5b-af8e-4b7b-bd00-5413bc1318d6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:49.367620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c47d57-06b8-4746-bd72-4c5c5777b95a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:49.376987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700ea29-1600-4b23-8b56-5168257a6300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:49.377825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700ea29-1600-4b23-8b56-5168257a6300 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.378103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700ea29-1600-4b23-8b56-5168257a6300 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.379146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700ea29-1600-4b23-8b56-5168257a6300 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.380001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700ea29-1600-4b23-8b56-5168257a6300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:49.384604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700ea29-1600-4b23-8b56-5168257a6300 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:49.392665 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1837/3704] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:49 2022] PUT /identity/v3/projects/af98ae4746f343b39d13b58f18c6c23e/users/77a750effc254df38f53b2c1743adeca/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:49.401002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177b6f0-b2ef-482d-af7a-b34c9b6758d7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.401445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177b6f0-b2ef-482d-af7a-b34c9b6758d7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.401721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177b6f0-b2ef-482d-af7a-b34c9b6758d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.468024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4177b6f0-b2ef-482d-af7a-b34c9b6758d7 None None] MFA Rules not processed for user `77a750effc254df38f53b2c1743adeca`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:49.469546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4177b6f0-b2ef-482d-af7a-b34c9b6758d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:49.477876 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1868/3705] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:49 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:49.477876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700ea29-1600-4b23-8b56-5168257a6300 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:49.480586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4700ea29-1600-4b23-8b56-5168257a6300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:49.616982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700ea29-1600-4b23-8b56-5168257a6300 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:49.626015 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1838/3706] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:49.627171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177b6f0-b2ef-482d-af7a-b34c9b6758d7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:49.628335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef42765-bb6f-4e27-990a-5ffd930a3790 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:49.629191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef42765-bb6f-4e27-990a-5ffd930a3790 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.629700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef42765-bb6f-4e27-990a-5ffd930a3790 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.630168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef42765-bb6f-4e27-990a-5ffd930a3790 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.632671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2ef42765-bb6f-4e27-990a-5ffd930a3790 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:49.670664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef42765-bb6f-4e27-990a-5ffd930a3790 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:49.672281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef42765-bb6f-4e27-990a-5ffd930a3790 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:49.686322 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1869/3707] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:49.687167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef42765-bb6f-4e27-990a-5ffd930a3790 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:49.688633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2ef42765-bb6f-4e27-990a-5ffd930a3790 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:49.700695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:49.730097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177b6f0-b2ef-482d-af7a-b34c9b6758d7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:49.758947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef42765-bb6f-4e27-990a-5ffd930a3790 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:49.763444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81880ca6-0f63-4e1a-8b4c-cde8e8d35819 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:49.764071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81880ca6-0f63-4e1a-8b4c-cde8e8d35819 service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.764345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81880ca6-0f63-4e1a-8b4c-cde8e8d35819 service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.764647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81880ca6-0f63-4e1a-8b4c-cde8e8d35819 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.796067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccf927-a972-488b-934c-499ac3944597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:49.796716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf927-a972-488b-934c-499ac3944597 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.797535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf927-a972-488b-934c-499ac3944597 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.797805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf927-a972-488b-934c-499ac3944597 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.798423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ccf927-a972-488b-934c-499ac3944597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:49.800328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ccf927-a972-488b-934c-499ac3944597 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:49.828415 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1839/3708] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:49 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:49.828415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccf927-a972-488b-934c-499ac3944597 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:49.830820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-62ccf927-a972-488b-934c-499ac3944597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:49.836186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81880ca6-0f63-4e1a-8b4c-cde8e8d35819 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:49.837695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81880ca6-0f63-4e1a-8b4c-cde8e8d35819 service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:49.843902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccf927-a972-488b-934c-499ac3944597 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:49.847483 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1870/3709] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:49.848716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81880ca6-0f63-4e1a-8b4c-cde8e8d35819 service glance] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:49.849114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e15b62-ccfa-44e6-87ce-5233c22e1836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:49.849865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e15b62-ccfa-44e6-87ce-5233c22e1836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.850136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e15b62-ccfa-44e6-87ce-5233c22e1836 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.850401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e15b62-ccfa-44e6-87ce-5233c22e1836 admin admin] PATH_INFO: `/v3/users/647583dffce54234bcf9276d4a61f80e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.853211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e15b62-ccfa-44e6-87ce-5233c22e1836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=647583dffce54234bcf9276d4a61f80e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:49.855025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e15b62-ccfa-44e6-87ce-5233c22e1836 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:49.862708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81880ca6-0f63-4e1a-8b4c-cde8e8d35819 service glance] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:49.869832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:49.870022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d282735-e9dd-4dcf-8a4a-5e63783007b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:49.871159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d282735-e9dd-4dcf-8a4a-5e63783007b3 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.871583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d282735-e9dd-4dcf-8a4a-5e63783007b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.872015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d282735-e9dd-4dcf-8a4a-5e63783007b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.872960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d282735-e9dd-4dcf-8a4a-5e63783007b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:49.875936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d282735-e9dd-4dcf-8a4a-5e63783007b3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:49.890225 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1871/3710] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:49.896340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d282735-e9dd-4dcf-8a4a-5e63783007b3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:49.908716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d282735-e9dd-4dcf-8a4a-5e63783007b3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:49.914524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c215b7-e60b-414d-8297-0f9067697769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:49.915230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c215b7-e60b-414d-8297-0f9067697769 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.915515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c215b7-e60b-414d-8297-0f9067697769 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.916765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c215b7-e60b-414d-8297-0f9067697769 admin admin] PATH_INFO: `/v3/projects/a1e4244f5f524bb2987eb611ddd97056/users/4d839cbe386c4a33ab471f52070f0220/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:49.918194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61e15b62-ccfa-44e6-87ce-5233c22e1836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '647583dffce54234bcf9276d4a61f80e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:49.928440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61e15b62-ccfa-44e6-87ce-5233c22e1836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '647583dffce54234bcf9276d4a61f80e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:49.934990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c215b7-e60b-414d-8297-0f9067697769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1e4244f5f524bb2987eb611ddd97056, user_id=4d839cbe386c4a33ab471f52070f0220, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:49.937728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c215b7-e60b-414d-8297-0f9067697769 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:49.942842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61e15b62-ccfa-44e6-87ce-5233c22e1836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '647583dffce54234bcf9276d4a61f80e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:49.943549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61e15b62-ccfa-44e6-87ce-5233c22e1836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '647583dffce54234bcf9276d4a61f80e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:49.944288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61e15b62-ccfa-44e6-87ce-5233c22e1836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '647583dffce54234bcf9276d4a61f80e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:49.953521 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1872/3711] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:49 2022] PUT /identity/v3/projects/a1e4244f5f524bb2987eb611ddd97056/users/4d839cbe386c4a33ab471f52070f0220/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:49.957224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c215b7-e60b-414d-8297-0f9067697769 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:49.959494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d6c215b7-e60b-414d-8297-0f9067697769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:49.960755 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1840/3712] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:49 2022] DELETE /identity/v3/users/647583dffce54234bcf9276d4a61f80e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:49.989893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f3d14-0b21-4c19-a86a-c1585a412049 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:49.990178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f3d14-0b21-4c19-a86a-c1585a412049 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:49.990453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f3d14-0b21-4c19-a86a-c1585a412049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.014980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a91f3d14-0b21-4c19-a86a-c1585a412049 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:50.016344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a91f3d14-0b21-4c19-a86a-c1585a412049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:50.040225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c215b7-e60b-414d-8297-0f9067697769 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:50.077717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:50.082069 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1841/3713] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:50.104894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f3d14-0b21-4c19-a86a-c1585a412049 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:50.106724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a91f3d14-0b21-4c19-a86a-c1585a412049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:50.109891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443f2fa-40d4-4ef8-a7a4-607f209becc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:50.110743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443f2fa-40d4-4ef8-a7a4-607f209becc3 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.111183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443f2fa-40d4-4ef8-a7a4-607f209becc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.111627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443f2fa-40d4-4ef8-a7a4-607f209becc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.112338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443f2fa-40d4-4ef8-a7a4-607f209becc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:50.114261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443f2fa-40d4-4ef8-a7a4-607f209becc3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:50.126580 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1873/3714] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:05:50.132274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443f2fa-40d4-4ef8-a7a4-607f209becc3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:50.140921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443f2fa-40d4-4ef8-a7a4-607f209becc3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:50.146002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd5d72-53a5-4cd1-8268-98bcde2d9801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:50.146671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd5d72-53a5-4cd1-8268-98bcde2d9801 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.146956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd5d72-53a5-4cd1-8268-98bcde2d9801 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.147344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd5d72-53a5-4cd1-8268-98bcde2d9801 admin admin] PATH_INFO: `/v3/projects/a1e4244f5f524bb2987eb611ddd97056/users/4d839cbe386c4a33ab471f52070f0220/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.149348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd5d72-53a5-4cd1-8268-98bcde2d9801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1e4244f5f524bb2987eb611ddd97056, user_id=4d839cbe386c4a33ab471f52070f0220, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:50.153905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd5d72-53a5-4cd1-8268-98bcde2d9801 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:50.164029 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1874/3715] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:50 2022] PUT /identity/v3/projects/a1e4244f5f524bb2987eb611ddd97056/users/4d839cbe386c4a33ab471f52070f0220/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:50.174397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd5d72-53a5-4cd1-8268-98bcde2d9801 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:50.176982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-37cd5d72-53a5-4cd1-8268-98bcde2d9801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:50.190412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f3d14-0b21-4c19-a86a-c1585a412049 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:50.227491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:50.250584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd5d72-53a5-4cd1-8268-98bcde2d9801 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:50.255269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e1feb-364a-400d-9259-39d37b44e9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:50.255899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1feb-364a-400d-9259-39d37b44e9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.256165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1feb-364a-400d-9259-39d37b44e9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.256449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1feb-364a-400d-9259-39d37b44e9d4 admin admin] PATH_INFO: `/v3/projects/ac3507f6a20140959a61eac7fa4cf2c5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.257726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e1feb-364a-400d-9259-39d37b44e9d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac3507f6a20140959a61eac7fa4cf2c5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:50.258421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3eef4-2b0d-4a05-b3c9-5c1ea2d425b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:50.259067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3eef4-2b0d-4a05-b3c9-5c1ea2d425b3 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.259459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3eef4-2b0d-4a05-b3c9-5c1ea2d425b3 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.259644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e1feb-364a-400d-9259-39d37b44e9d4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:50.259788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3eef4-2b0d-4a05-b3c9-5c1ea2d425b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.309159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f80e1feb-364a-400d-9259-39d37b44e9d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac3507f6a20140959a61eac7fa4cf2c5', 'request_id': 'req-f80e1feb-364a-400d-9259-39d37b44e9d4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:50.317663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3eef4-2b0d-4a05-b3c9-5c1ea2d425b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:50.318537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f80e1feb-364a-400d-9259-39d37b44e9d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac3507f6a20140959a61eac7fa4cf2c5', 'request_id': 'req-f80e1feb-364a-400d-9259-39d37b44e9d4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:50.319364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3eef4-2b0d-4a05-b3c9-5c1ea2d425b3 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:50.325918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f80e1feb-364a-400d-9259-39d37b44e9d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac3507f6a20140959a61eac7fa4cf2c5', 'request_id': 'req-f80e1feb-364a-400d-9259-39d37b44e9d4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:50.328778 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1842/3716] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:05:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:05:50.330194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f009785-ee30-47bf-a330-1dcfa6e6b032 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.330467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f009785-ee30-47bf-a330-1dcfa6e6b032 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.330945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f009785-ee30-47bf-a330-1dcfa6e6b032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.342321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f80e1feb-364a-400d-9259-39d37b44e9d4 admin admin] The token cache is being invalidate because project ac3507f6a20140959a61eac7fa4cf2c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:50.342557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f80e1feb-364a-400d-9259-39d37b44e9d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:50.343279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f80e1feb-364a-400d-9259-39d37b44e9d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:50.345165 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1875/3717] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:50 2022] DELETE /identity/v3/projects/ac3507f6a20140959a61eac7fa4cf2c5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:50.349909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b197f9d-7a08-429d-b171-50d2a63ca0a6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.350188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b197f9d-7a08-429d-b171-50d2a63ca0a6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.350458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b197f9d-7a08-429d-b171-50d2a63ca0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.362317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2f009785-ee30-47bf-a330-1dcfa6e6b032 None None] MFA Rules not processed for user `4d839cbe386c4a33ab471f52070f0220`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:50.363672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2f009785-ee30-47bf-a330-1dcfa6e6b032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:50.373320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6b197f9d-7a08-429d-b171-50d2a63ca0a6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:50.374632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6b197f9d-7a08-429d-b171-50d2a63ca0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:50.426725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:50.454581 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1876/3718] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:50.461178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5afce-c020-4b75-860e-d7816a015688 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.461608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5afce-c020-4b75-860e-d7816a015688 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.462344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5afce-c020-4b75-860e-d7816a015688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.474736 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1843/3719] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:50.479245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f009785-ee30-47bf-a330-1dcfa6e6b032 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:50.485896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2f009785-ee30-47bf-a330-1dcfa6e6b032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:50.486710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dda5afce-c020-4b75-860e-d7816a015688 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:50.487977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dda5afce-c020-4b75-860e-d7816a015688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:50.499660 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1877/3720] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:50.504186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5afce-c020-4b75-860e-d7816a015688 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:50.516033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5afce-c020-4b75-860e-d7816a015688 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:50.522062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ed0ad-2cdf-463c-a472-2b2e2248c343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:50.522062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ed0ad-2cdf-463c-a472-2b2e2248c343 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.522062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ed0ad-2cdf-463c-a472-2b2e2248c343 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.522062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ed0ad-2cdf-463c-a472-2b2e2248c343 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.526691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ed0ad-2cdf-463c-a472-2b2e2248c343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:05:50.526691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ed0ad-2cdf-463c-a472-2b2e2248c343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:50.538802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ed0ad-2cdf-463c-a472-2b2e2248c343 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:50.565450 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1878/3721] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:05:50.582173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ed0ad-2cdf-463c-a472-2b2e2248c343 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:50.606335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ed0ad-2cdf-463c-a472-2b2e2248c343 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:50.607668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f009785-ee30-47bf-a330-1dcfa6e6b032 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:50.612652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46f191-586a-4b3e-807f-94bd39f632c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:50.613221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabc671-b896-46be-bb89-da96920cc7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:50.613352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46f191-586a-4b3e-807f-94bd39f632c4 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.613676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46f191-586a-4b3e-807f-94bd39f632c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.613844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabc671-b896-46be-bb89-da96920cc7af admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.613997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46f191-586a-4b3e-807f-94bd39f632c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.614093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabc671-b896-46be-bb89-da96920cc7af admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.614419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabc671-b896-46be-bb89-da96920cc7af admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.615077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabc671-b896-46be-bb89-da96920cc7af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:50.617157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabc671-b896-46be-bb89-da96920cc7af admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:50.625440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46f191-586a-4b3e-807f-94bd39f632c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:50.626971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46f191-586a-4b3e-807f-94bd39f632c4 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:50.643913 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1844/3722] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:50.665098 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1879/3723] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:50 2022] POST /identity/v3/projects => generated 375 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:50.668465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46f191-586a-4b3e-807f-94bd39f632c4 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:50.681044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46f191-586a-4b3e-807f-94bd39f632c4 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:50.685611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:50.740381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc2af2-af08-4f6a-94c4-be2f351b33ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:50.741213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc2af2-af08-4f6a-94c4-be2f351b33ae admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.741548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc2af2-af08-4f6a-94c4-be2f351b33ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.742149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc2af2-af08-4f6a-94c4-be2f351b33ae admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.742787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc2af2-af08-4f6a-94c4-be2f351b33ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:50.745031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc2af2-af08-4f6a-94c4-be2f351b33ae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:50.767356 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1845/3724] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:50 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:50.771368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabc671-b896-46be-bb89-da96920cc7af admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:50.790755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabc671-b896-46be-bb89-da96920cc7af admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:50.799859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370419b7-2443-417f-b89e-7b6755365c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:50.802541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370419b7-2443-417f-b89e-7b6755365c05 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.802541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370419b7-2443-417f-b89e-7b6755365c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.802541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370419b7-2443-417f-b89e-7b6755365c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.802541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370419b7-2443-417f-b89e-7b6755365c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:50.805637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370419b7-2443-417f-b89e-7b6755365c05 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:50.821328 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1880/3725] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:50.826560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc2af2-af08-4f6a-94c4-be2f351b33ae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:50.838541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc2af2-af08-4f6a-94c4-be2f351b33ae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:50.844528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d0928-6aaa-46c7-bf8d-d42497686c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:50.845286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d0928-6aaa-46c7-bf8d-d42497686c26 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.845828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d0928-6aaa-46c7-bf8d-d42497686c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.846102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d0928-6aaa-46c7-bf8d-d42497686c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.846762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d0928-6aaa-46c7-bf8d-d42497686c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:50.848962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d0928-6aaa-46c7-bf8d-d42497686c26 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:50.865786 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1846/3726] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:05:50.867364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370419b7-2443-417f-b89e-7b6755365c05 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:50.879247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370419b7-2443-417f-b89e-7b6755365c05 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:50.884594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a7173-c7c6-4f09-a9c8-6fc186e2d1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:50.885314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a7173-c7c6-4f09-a9c8-6fc186e2d1e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.885601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a7173-c7c6-4f09-a9c8-6fc186e2d1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.885852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a7173-c7c6-4f09-a9c8-6fc186e2d1e9 admin admin] PATH_INFO: `/v3/projects/7a13c82fcbb542a19af3a42b48f81958/users/30c7e3ac41df480ea0c198af4f0c49ea/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.903922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a7173-c7c6-4f09-a9c8-6fc186e2d1e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a13c82fcbb542a19af3a42b48f81958, user_id=30c7e3ac41df480ea0c198af4f0c49ea, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:50.909656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a7173-c7c6-4f09-a9c8-6fc186e2d1e9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:50.941131 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1881/3727] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:50 2022] PUT /identity/v3/projects/7a13c82fcbb542a19af3a42b48f81958/users/30c7e3ac41df480ea0c198af4f0c49ea/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:50.947235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac847d-3838-4721-acef-7cd01a3fc4c7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:50.947991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac847d-3838-4721-acef-7cd01a3fc4c7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:50.948571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac847d-3838-4721-acef-7cd01a3fc4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:50.976357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ebac847d-3838-4721-acef-7cd01a3fc4c7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:50.978175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ebac847d-3838-4721-acef-7cd01a3fc4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:51.068095 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1847/3728] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:51.082226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a7173-c7c6-4f09-a9c8-6fc186e2d1e9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:51.186417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a7173-c7c6-4f09-a9c8-6fc186e2d1e9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:51.198491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91716621-a6bf-45f8-8106-ee48a4ba66fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:51.199120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91716621-a6bf-45f8-8106-ee48a4ba66fa service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:51.199386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91716621-a6bf-45f8-8106-ee48a4ba66fa service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:51.199649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91716621-a6bf-45f8-8106-ee48a4ba66fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:51.212620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91716621-a6bf-45f8-8106-ee48a4ba66fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:51.214832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91716621-a6bf-45f8-8106-ee48a4ba66fa service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:51.241001 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1882/3729] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:53.090175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac847d-3838-4721-acef-7cd01a3fc4c7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:53.094528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ebac847d-3838-4721-acef-7cd01a3fc4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:53.114519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac847d-3838-4721-acef-7cd01a3fc4c7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:53.120586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:53.121564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:53.121961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:53.122393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] PATH_INFO: `/v3/users/188be8acf9194905a4172cd48b792ddb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:53.123858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=188be8acf9194905a4172cd48b792ddb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:53.126835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:53.213419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '188be8acf9194905a4172cd48b792ddb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:53.230657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '188be8acf9194905a4172cd48b792ddb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:53.254056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '188be8acf9194905a4172cd48b792ddb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:53.254991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '188be8acf9194905a4172cd48b792ddb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:53.255664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '188be8acf9194905a4172cd48b792ddb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:53.264673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:53.281667 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1848/3730] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:53 2022] DELETE /identity/v3/users/188be8acf9194905a4172cd48b792ddb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:53.286308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e191691-9e29-494b-8bdc-f604eaca7dc4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:53.286821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e191691-9e29-494b-8bdc-f604eaca7dc4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:53.286821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e191691-9e29-494b-8bdc-f604eaca7dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:53.310868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3e191691-9e29-494b-8bdc-f604eaca7dc4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:53.313226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3e191691-9e29-494b-8bdc-f604eaca7dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:53.391036 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1883/3731] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:53.416255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:53.418784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:53.529137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edadf7-2f9d-425e-9d5c-0601c4cb2db4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:53.534019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bfc6d-c012-4ad5-a85b-60110a24a237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:53.534646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bfc6d-c012-4ad5-a85b-60110a24a237 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:53.534919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bfc6d-c012-4ad5-a85b-60110a24a237 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:53.535184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bfc6d-c012-4ad5-a85b-60110a24a237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:53.549892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5bfc6d-c012-4ad5-a85b-60110a24a237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:53.553781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5bfc6d-c012-4ad5-a85b-60110a24a237 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:53.573449 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1849/3732] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:54.114352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e191691-9e29-494b-8bdc-f604eaca7dc4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:54.118653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3e191691-9e29-494b-8bdc-f604eaca7dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:54.161409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e191691-9e29-494b-8bdc-f604eaca7dc4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:54.168238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:54.169350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:54.169993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:54.170469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] PATH_INFO: `/v3/projects/8597498ff38b4cadacc981db6c834fff` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:54.171950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8597498ff38b4cadacc981db6c834fff)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:54.174235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:54.186859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:54.254441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8597498ff38b4cadacc981db6c834fff', 'request_id': 'req-dfd27e54-fdd5-4341-93da-34039436d27d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:54.271798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8597498ff38b4cadacc981db6c834fff', 'request_id': 'req-dfd27e54-fdd5-4341-93da-34039436d27d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:54.281106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8597498ff38b4cadacc981db6c834fff', 'request_id': 'req-dfd27e54-fdd5-4341-93da-34039436d27d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:54.297302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] The token cache is being invalidate because project 8597498ff38b4cadacc981db6c834fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:54.297883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:54.299665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:54.302498 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1884/3733] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:54 2022] DELETE /identity/v3/projects/8597498ff38b4cadacc981db6c834fff => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:54.308409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bfc6d-c012-4ad5-a85b-60110a24a237 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:54.319762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bfc6d-c012-4ad5-a85b-60110a24a237 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:54.441268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d4877-acdf-4de4-9fd7-baed7508f0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:54.441916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d4877-acdf-4de4-9fd7-baed7508f0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:54.442350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d4877-acdf-4de4-9fd7-baed7508f0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:54.442640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d4877-acdf-4de4-9fd7-baed7508f0f7 admin admin] PATH_INFO: `/v3/users/4d839cbe386c4a33ab471f52070f0220` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:54.443789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d4877-acdf-4de4-9fd7-baed7508f0f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d839cbe386c4a33ab471f52070f0220)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:54.446908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d4877-acdf-4de4-9fd7-baed7508f0f7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:54.543743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5b4d4877-acdf-4de4-9fd7-baed7508f0f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d839cbe386c4a33ab471f52070f0220'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:54.559105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5b4d4877-acdf-4de4-9fd7-baed7508f0f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d839cbe386c4a33ab471f52070f0220'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:54.594696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5b4d4877-acdf-4de4-9fd7-baed7508f0f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d839cbe386c4a33ab471f52070f0220'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:54.595792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5b4d4877-acdf-4de4-9fd7-baed7508f0f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d839cbe386c4a33ab471f52070f0220'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:54.596513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5b4d4877-acdf-4de4-9fd7-baed7508f0f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d839cbe386c4a33ab471f52070f0220'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:54.613020 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1850/3734] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:54 2022] DELETE /identity/v3/users/4d839cbe386c4a33ab471f52070f0220 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:55.194976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:55.196950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:55.217240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd27e54-fdd5-4341-93da-34039436d27d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:55.325153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0391d-b080-4eed-8fee-23cbfc3d7825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:55.325774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0391d-b080-4eed-8fee-23cbfc3d7825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:55.326047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0391d-b080-4eed-8fee-23cbfc3d7825 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:55.326307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0391d-b080-4eed-8fee-23cbfc3d7825 admin admin] PATH_INFO: `/v3/projects/a1e4244f5f524bb2987eb611ddd97056` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:55.327728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0391d-b080-4eed-8fee-23cbfc3d7825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1e4244f5f524bb2987eb611ddd97056)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:55.329874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0391d-b080-4eed-8fee-23cbfc3d7825 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:55.389848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c0f0391d-b080-4eed-8fee-23cbfc3d7825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1e4244f5f524bb2987eb611ddd97056', 'request_id': 'req-c0f0391d-b080-4eed-8fee-23cbfc3d7825'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:55.402359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c0f0391d-b080-4eed-8fee-23cbfc3d7825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1e4244f5f524bb2987eb611ddd97056', 'request_id': 'req-c0f0391d-b080-4eed-8fee-23cbfc3d7825'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:55.409876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c0f0391d-b080-4eed-8fee-23cbfc3d7825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1e4244f5f524bb2987eb611ddd97056', 'request_id': 'req-c0f0391d-b080-4eed-8fee-23cbfc3d7825'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:55.424054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c0f0391d-b080-4eed-8fee-23cbfc3d7825 admin admin] The token cache is being invalidate because project a1e4244f5f524bb2987eb611ddd97056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:55.424668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c0f0391d-b080-4eed-8fee-23cbfc3d7825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:55.425465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c0f0391d-b080-4eed-8fee-23cbfc3d7825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:55.427539 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1885/3735] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:55 2022] DELETE /identity/v3/projects/a1e4244f5f524bb2987eb611ddd97056 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:55.436113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775097f-36f7-44c5-aa55-299533792c47 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:55.438684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775097f-36f7-44c5-aa55-299533792c47 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:55.439130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775097f-36f7-44c5-aa55-299533792c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:55.473164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a775097f-36f7-44c5-aa55-299533792c47 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:55.474579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a775097f-36f7-44c5-aa55-299533792c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:55.491007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:55.566896 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1851/3736] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:55.578373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347455af-8477-44fa-8334-4324f4cc23db None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:55.578643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347455af-8477-44fa-8334-4324f4cc23db None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:55.578908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347455af-8477-44fa-8334-4324f4cc23db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:55.606119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-347455af-8477-44fa-8334-4324f4cc23db None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:55.607351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-347455af-8477-44fa-8334-4324f4cc23db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:55.615443 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1886/3737] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:55.621300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775097f-36f7-44c5-aa55-299533792c47 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:55.630561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775097f-36f7-44c5-aa55-299533792c47 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:55.636682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05b272-8507-4c64-86f2-314ff029b36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:55.637302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05b272-8507-4c64-86f2-314ff029b36c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:55.637574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05b272-8507-4c64-86f2-314ff029b36c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:55.637839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05b272-8507-4c64-86f2-314ff029b36c admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:55.638490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05b272-8507-4c64-86f2-314ff029b36c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:05:55.638785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05b272-8507-4c64-86f2-314ff029b36c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:55.640400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05b272-8507-4c64-86f2-314ff029b36c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:55.653526 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1852/3738] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:05:55.658117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347455af-8477-44fa-8334-4324f4cc23db None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:55.668635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347455af-8477-44fa-8334-4324f4cc23db None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:55.673638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695986f-4246-4952-9001-8f926a538f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:55.674248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695986f-4246-4952-9001-8f926a538f48 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:55.674520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695986f-4246-4952-9001-8f926a538f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:55.674783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695986f-4246-4952-9001-8f926a538f48 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:55.675405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695986f-4246-4952-9001-8f926a538f48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:55.677748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695986f-4246-4952-9001-8f926a538f48 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:55.707155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:55.722634 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1887/3739] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:55 2022] POST /identity/v3/projects => generated 405 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:55.723753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05b272-8507-4c64-86f2-314ff029b36c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:55.736588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05b272-8507-4c64-86f2-314ff029b36c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:55.819471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e903049-6991-4d7e-a107-d4918faaeeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:55.821034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e903049-6991-4d7e-a107-d4918faaeeae admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:55.821323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e903049-6991-4d7e-a107-d4918faaeeae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:55.821594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e903049-6991-4d7e-a107-d4918faaeeae admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:55.833119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e903049-6991-4d7e-a107-d4918faaeeae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:55.837876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e903049-6991-4d7e-a107-d4918faaeeae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:55.901973 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1853/3740] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:55 2022] POST /identity/v3/users => generated 303 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:55.906228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695986f-4246-4952-9001-8f926a538f48 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:55.919103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695986f-4246-4952-9001-8f926a538f48 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:55.937478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784ca57-3463-4aec-be6f-01f54c0e6e95 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:55.938317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784ca57-3463-4aec-be6f-01f54c0e6e95 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:55.939028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784ca57-3463-4aec-be6f-01f54c0e6e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:55.957188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790448c-307c-4fe4-bdea-fc772acd9870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:55.957836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790448c-307c-4fe4-bdea-fc772acd9870 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:55.958074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790448c-307c-4fe4-bdea-fc772acd9870 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:55.958328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790448c-307c-4fe4-bdea-fc772acd9870 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:55.958993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790448c-307c-4fe4-bdea-fc772acd9870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:55.962750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790448c-307c-4fe4-bdea-fc772acd9870 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:55.962984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9784ca57-3463-4aec-be6f-01f54c0e6e95 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:55.964594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9784ca57-3463-4aec-be6f-01f54c0e6e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:55.997814 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1888/3741] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:56.004517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790448c-307c-4fe4-bdea-fc772acd9870 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:56.026925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790448c-307c-4fe4-bdea-fc772acd9870 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:56.055348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb68dfb-70a2-49ed-8606-96fa1288db38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:56.055987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb68dfb-70a2-49ed-8606-96fa1288db38 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:56.056259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb68dfb-70a2-49ed-8606-96fa1288db38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:56.056557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb68dfb-70a2-49ed-8606-96fa1288db38 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:56.057227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb68dfb-70a2-49ed-8606-96fa1288db38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:56.058907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb68dfb-70a2-49ed-8606-96fa1288db38 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:56.087796 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1854/3742] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:05:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:56.089359 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1889/3743] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:56.090933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784ca57-3463-4aec-be6f-01f54c0e6e95 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:56.103376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb68dfb-70a2-49ed-8606-96fa1288db38 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:56.105347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ceb68dfb-70a2-49ed-8606-96fa1288db38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:56.119291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784ca57-3463-4aec-be6f-01f54c0e6e95 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:56.132209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb89c8-3a5b-4063-8078-40272d9ce0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:56.135951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb89c8-3a5b-4063-8078-40272d9ce0dc admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:56.137372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb89c8-3a5b-4063-8078-40272d9ce0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:56.139322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb89c8-3a5b-4063-8078-40272d9ce0dc admin admin] PATH_INFO: `/v3/projects/1090dfab4c044c30bab4769da4b2c9b2/users/93502ea5425b49a2943334e7e995c7ea/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:56.202262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb89c8-3a5b-4063-8078-40272d9ce0dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1090dfab4c044c30bab4769da4b2c9b2, user_id=93502ea5425b49a2943334e7e995c7ea, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:56.205806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb89c8-3a5b-4063-8078-40272d9ce0dc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:56.232323 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1855/3744] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:56 2022] PUT /identity/v3/projects/1090dfab4c044c30bab4769da4b2c9b2/users/93502ea5425b49a2943334e7e995c7ea/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:56.243285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5fba4-3745-4fcc-ae39-3711112010e8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:56.243742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5fba4-3745-4fcc-ae39-3711112010e8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:56.244189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5fba4-3745-4fcc-ae39-3711112010e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:56.294934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e3f5fba4-3745-4fcc-ae39-3711112010e8 None None] MFA Rules not processed for user `93502ea5425b49a2943334e7e995c7ea`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:56.296519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e3f5fba4-3745-4fcc-ae39-3711112010e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:56.299323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb68dfb-70a2-49ed-8606-96fa1288db38 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:56.308407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:56.409421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63628f4c-5e81-43c1-bd89-f3ba11e4f665 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:56.410295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63628f4c-5e81-43c1-bd89-f3ba11e4f665 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:56.410704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63628f4c-5e81-43c1-bd89-f3ba11e4f665 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:56.411079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63628f4c-5e81-43c1-bd89-f3ba11e4f665 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:56.451621 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1856/3745] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:56 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:56.457376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5fba4-3745-4fcc-ae39-3711112010e8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:56.526432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63628f4c-5e81-43c1-bd89-f3ba11e4f665 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:56.528256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63628f4c-5e81-43c1-bd89-f3ba11e4f665 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:56.538353 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1890/3746] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:05:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:05:56.539866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5fba4-3745-4fcc-ae39-3711112010e8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:56.549149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc6e93-1c88-47ac-83fd-6aa8b64b23f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:56.550235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc6e93-1c88-47ac-83fd-6aa8b64b23f3 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:56.550235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc6e93-1c88-47ac-83fd-6aa8b64b23f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:56.550433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc6e93-1c88-47ac-83fd-6aa8b64b23f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:56.550968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc6e93-1c88-47ac-83fd-6aa8b64b23f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:56.553144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc6e93-1c88-47ac-83fd-6aa8b64b23f3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:56.608656 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1857/3747] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:56 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:56.611524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63628f4c-5e81-43c1-bd89-f3ba11e4f665 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:56.624470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63628f4c-5e81-43c1-bd89-f3ba11e4f665 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:56.644798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:56.702840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc6e93-1c88-47ac-83fd-6aa8b64b23f3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:56.703847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da01d9-0235-4d05-a4bf-5fe200ea56f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:56.704576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da01d9-0235-4d05-a4bf-5fe200ea56f9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:56.704848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da01d9-0235-4d05-a4bf-5fe200ea56f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:56.705138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da01d9-0235-4d05-a4bf-5fe200ea56f9 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:56.705682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-99cc6e93-1c88-47ac-83fd-6aa8b64b23f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:56.705887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da01d9-0235-4d05-a4bf-5fe200ea56f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:56.707927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da01d9-0235-4d05-a4bf-5fe200ea56f9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:56.724228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc6e93-1c88-47ac-83fd-6aa8b64b23f3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:56.732024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e54d2-412b-46c7-99fc-c63709470aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:56.733399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e54d2-412b-46c7-99fc-c63709470aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:56.734083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e54d2-412b-46c7-99fc-c63709470aec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:56.734083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e54d2-412b-46c7-99fc-c63709470aec admin admin] PATH_INFO: `/v3/users/77a750effc254df38f53b2c1743adeca` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:56.735468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e54d2-412b-46c7-99fc-c63709470aec admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a750effc254df38f53b2c1743adeca)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:56.738092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e54d2-412b-46c7-99fc-c63709470aec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:56.740345 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1891/3748] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:56 2022] POST /identity/v3/users => generated 309 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:56.744360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da01d9-0235-4d05-a4bf-5fe200ea56f9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:56.761008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da01d9-0235-4d05-a4bf-5fe200ea56f9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:56.766766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20b49a-0c5c-443c-8334-15b98fd0461a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:56.767965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b49a-0c5c-443c-8334-15b98fd0461a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:56.768242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b49a-0c5c-443c-8334-15b98fd0461a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:56.768545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b49a-0c5c-443c-8334-15b98fd0461a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:56.769202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20b49a-0c5c-443c-8334-15b98fd0461a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:56.772813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20b49a-0c5c-443c-8334-15b98fd0461a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:56.785806 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1892/3749] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:56.789927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20b49a-0c5c-443c-8334-15b98fd0461a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:56.802467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20b49a-0c5c-443c-8334-15b98fd0461a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:56.808352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536648b7-2898-40dc-837c-281e55117e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:56.808538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536648b7-2898-40dc-837c-281e55117e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:56.808709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536648b7-2898-40dc-837c-281e55117e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:56.808972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536648b7-2898-40dc-837c-281e55117e2d admin admin] PATH_INFO: `/v3/projects/a45eac165cf54a57801bc13408a3ef77/users/7ae4e33ad89d40e688e50811196cb0cd/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:56.813688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cc1e54d2-412b-46c7-99fc-c63709470aec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a750effc254df38f53b2c1743adeca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:56.835998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cc1e54d2-412b-46c7-99fc-c63709470aec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a750effc254df38f53b2c1743adeca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:56.836296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536648b7-2898-40dc-837c-281e55117e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a45eac165cf54a57801bc13408a3ef77, user_id=7ae4e33ad89d40e688e50811196cb0cd, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:56.839755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536648b7-2898-40dc-837c-281e55117e2d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:56.856248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cc1e54d2-412b-46c7-99fc-c63709470aec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a750effc254df38f53b2c1743adeca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:56.857037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cc1e54d2-412b-46c7-99fc-c63709470aec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a750effc254df38f53b2c1743adeca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:56.857621 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1893/3750] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:56 2022] PUT /identity/v3/projects/a45eac165cf54a57801bc13408a3ef77/users/7ae4e33ad89d40e688e50811196cb0cd/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:56.860834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cc1e54d2-412b-46c7-99fc-c63709470aec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a750effc254df38f53b2c1743adeca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:56.862024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536648b7-2898-40dc-837c-281e55117e2d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:56.863976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-536648b7-2898-40dc-837c-281e55117e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:56.885330 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1858/3751] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:56 2022] DELETE /identity/v3/users/77a750effc254df38f53b2c1743adeca => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:56.887414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51c6ea-6195-4a3c-9edc-e659a77183b8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:56.887691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51c6ea-6195-4a3c-9edc-e659a77183b8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:56.887961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51c6ea-6195-4a3c-9edc-e659a77183b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:56.902631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:56.918460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ee51c6ea-6195-4a3c-9edc-e659a77183b8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:56.919914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ee51c6ea-6195-4a3c-9edc-e659a77183b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:56.951946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536648b7-2898-40dc-837c-281e55117e2d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:57.005719 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1859/3752] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:57.015724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51c6ea-6195-4a3c-9edc-e659a77183b8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:57.017830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ee51c6ea-6195-4a3c-9edc-e659a77183b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:57.035227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1595905-1802-4ddd-ae32-48fb1aa7ccae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:57.036033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595905-1802-4ddd-ae32-48fb1aa7ccae admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:57.036313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595905-1802-4ddd-ae32-48fb1aa7ccae admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:57.036677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595905-1802-4ddd-ae32-48fb1aa7ccae admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:57.037742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1595905-1802-4ddd-ae32-48fb1aa7ccae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:57.039156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1595905-1802-4ddd-ae32-48fb1aa7ccae admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:57.056328 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1894/3753] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:57.057005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1595905-1802-4ddd-ae32-48fb1aa7ccae admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:57.070848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1595905-1802-4ddd-ae32-48fb1aa7ccae admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:57.076355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea48d6f-cc43-4b28-b466-7f4f66b3102f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:57.077185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea48d6f-cc43-4b28-b466-7f4f66b3102f admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:57.077730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea48d6f-cc43-4b28-b466-7f4f66b3102f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:57.078054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea48d6f-cc43-4b28-b466-7f4f66b3102f admin admin] PATH_INFO: `/v3/projects/a45eac165cf54a57801bc13408a3ef77/users/7ae4e33ad89d40e688e50811196cb0cd/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:57.080545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea48d6f-cc43-4b28-b466-7f4f66b3102f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a45eac165cf54a57801bc13408a3ef77, user_id=7ae4e33ad89d40e688e50811196cb0cd, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:57.083911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea48d6f-cc43-4b28-b466-7f4f66b3102f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:57.097823 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1895/3754] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:57 2022] PUT /identity/v3/projects/a45eac165cf54a57801bc13408a3ef77/users/7ae4e33ad89d40e688e50811196cb0cd/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:57.105222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51c6ea-6195-4a3c-9edc-e659a77183b8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:57.108787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72eed6e-a2f8-47fc-a044-f99d3f384544 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:57.109949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72eed6e-a2f8-47fc-a044-f99d3f384544 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:57.110266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72eed6e-a2f8-47fc-a044-f99d3f384544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:57.131298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:57.166949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe083b3-86a8-4827-8427-a45e5cb159d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:57.167572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe083b3-86a8-4827-8427-a45e5cb159d8 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:57.167842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe083b3-86a8-4827-8427-a45e5cb159d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:57.168107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe083b3-86a8-4827-8427-a45e5cb159d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:57.172529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b72eed6e-a2f8-47fc-a044-f99d3f384544 None None] MFA Rules not processed for user `7ae4e33ad89d40e688e50811196cb0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:57.173859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b72eed6e-a2f8-47fc-a044-f99d3f384544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:57.284502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe083b3-86a8-4827-8427-a45e5cb159d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:57.294845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe083b3-86a8-4827-8427-a45e5cb159d8 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:57.306011 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1860/3755] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:57.307234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d736f9-b4df-49da-91c9-e2404eac3bde None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:57.307516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d736f9-b4df-49da-91c9-e2404eac3bde None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:57.307780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d736f9-b4df-49da-91c9-e2404eac3bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:57.364694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:57.367854 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1896/3756] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:57 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:57.379956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e0d736f9-b4df-49da-91c9-e2404eac3bde None None] MFA Rules not processed for user `30c7e3ac41df480ea0c198af4f0c49ea`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:57.381464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e0d736f9-b4df-49da-91c9-e2404eac3bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:57.387832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72eed6e-a2f8-47fc-a044-f99d3f384544 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:57.389373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b72eed6e-a2f8-47fc-a044-f99d3f384544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:57.414776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72eed6e-a2f8-47fc-a044-f99d3f384544 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:57.426369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ee1d0-0593-4182-8122-5c59ba5db6cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:57.427115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ee1d0-0593-4182-8122-5c59ba5db6cb service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:57.427539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ee1d0-0593-4182-8122-5c59ba5db6cb service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:57.427809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ee1d0-0593-4182-8122-5c59ba5db6cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:57.430909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-932ee1d0-0593-4182-8122-5c59ba5db6cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:57.510866 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1861/3757] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:57.528117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e11c6e-7155-4e3a-8e99-84af3f4341c4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:57.528117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e11c6e-7155-4e3a-8e99-84af3f4341c4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:57.528356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e11c6e-7155-4e3a-8e99-84af3f4341c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:57.566432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-97e11c6e-7155-4e3a-8e99-84af3f4341c4 None None] MFA Rules not processed for user `30c7e3ac41df480ea0c198af4f0c49ea`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:57.567710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-97e11c6e-7155-4e3a-8e99-84af3f4341c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:57.575740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ee1d0-0593-4182-8122-5c59ba5db6cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:57.577527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ee1d0-0593-4182-8122-5c59ba5db6cb service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:57.589088 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1862/3758] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:05:57.599068 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1897/3759] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:57 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:57.627186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e11c6e-7155-4e3a-8e99-84af3f4341c4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:57.629521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-97e11c6e-7155-4e3a-8e99-84af3f4341c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:57.752165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e11c6e-7155-4e3a-8e99-84af3f4341c4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:57.766792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375054a-b175-4f65-b0c3-c059d84ed048 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:57.768788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375054a-b175-4f65-b0c3-c059d84ed048 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:57.769757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375054a-b175-4f65-b0c3-c059d84ed048 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:57.769757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375054a-b175-4f65-b0c3-c059d84ed048 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:57.781349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375054a-b175-4f65-b0c3-c059d84ed048 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:57.782574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375054a-b175-4f65-b0c3-c059d84ed048 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:57.800142 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1863/3760] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:58.052293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ee1d0-0593-4182-8122-5c59ba5db6cb service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:58.054052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-932ee1d0-0593-4182-8122-5c59ba5db6cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:58.081327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ee1d0-0593-4182-8122-5c59ba5db6cb service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:58.091554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb384f1-c660-400d-8e51-f93404640a06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:58.092694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb384f1-c660-400d-8e51-f93404640a06 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:58.093119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb384f1-c660-400d-8e51-f93404640a06 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:58.093598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb384f1-c660-400d-8e51-f93404640a06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:58.114677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb384f1-c660-400d-8e51-f93404640a06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:58.116436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb384f1-c660-400d-8e51-f93404640a06 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:58.128669 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1898/3761] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:58 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:58.219742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375054a-b175-4f65-b0c3-c059d84ed048 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:58.233022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375054a-b175-4f65-b0c3-c059d84ed048 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:58.245319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202862e-0e45-4373-a2a4-545568f6675e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:58.246067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202862e-0e45-4373-a2a4-545568f6675e service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:58.246549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202862e-0e45-4373-a2a4-545568f6675e service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:58.246919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202862e-0e45-4373-a2a4-545568f6675e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:58.252608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a202862e-0e45-4373-a2a4-545568f6675e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:58.261349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb384f1-c660-400d-8e51-f93404640a06 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:58.263058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5bb384f1-c660-400d-8e51-f93404640a06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:58.266141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202862e-0e45-4373-a2a4-545568f6675e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:58.267732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202862e-0e45-4373-a2a4-545568f6675e service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:58.279022 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1864/3762] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:58.283630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb384f1-c660-400d-8e51-f93404640a06 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:58.298981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee26ba-30b2-466a-86ba-d35edf65765a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:58.299714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee26ba-30b2-466a-86ba-d35edf65765a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:58.300062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee26ba-30b2-466a-86ba-d35edf65765a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:58.300425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee26ba-30b2-466a-86ba-d35edf65765a admin admin] PATH_INFO: `/v3/projects/af98ae4746f343b39d13b58f18c6c23e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:58.302760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee26ba-30b2-466a-86ba-d35edf65765a admin admin] RBAC: Authorizing `identity:delete_project(project_id=af98ae4746f343b39d13b58f18c6c23e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:58.306802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee26ba-30b2-466a-86ba-d35edf65765a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:58.364289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cdee26ba-30b2-466a-86ba-d35edf65765a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af98ae4746f343b39d13b58f18c6c23e', 'request_id': 'req-cdee26ba-30b2-466a-86ba-d35edf65765a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:58.373017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cdee26ba-30b2-466a-86ba-d35edf65765a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af98ae4746f343b39d13b58f18c6c23e', 'request_id': 'req-cdee26ba-30b2-466a-86ba-d35edf65765a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:58.378922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cdee26ba-30b2-466a-86ba-d35edf65765a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af98ae4746f343b39d13b58f18c6c23e', 'request_id': 'req-cdee26ba-30b2-466a-86ba-d35edf65765a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:58.387862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:58.393801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cdee26ba-30b2-466a-86ba-d35edf65765a admin admin] The token cache is being invalidate because project af98ae4746f343b39d13b58f18c6c23e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:05:58.394162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cdee26ba-30b2-466a-86ba-d35edf65765a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:58.395148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cdee26ba-30b2-466a-86ba-d35edf65765a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:05:58.396916 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1899/3763] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:05:58 2022] DELETE /identity/v3/projects/af98ae4746f343b39d13b58f18c6c23e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:05:58.402547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7bf1a-7f16-43bc-8abb-a38e17194708 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:58.402927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7bf1a-7f16-43bc-8abb-a38e17194708 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:58.403943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7bf1a-7f16-43bc-8abb-a38e17194708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:58.433944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a3d7bf1a-7f16-43bc-8abb-a38e17194708 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:58.435417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a3d7bf1a-7f16-43bc-8abb-a38e17194708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:58.512667 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1865/3764] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:58.521387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4657c4-5fbe-4465-a035-521e1799b84d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:58.521912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4657c4-5fbe-4465-a035-521e1799b84d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:58.521912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4657c4-5fbe-4465-a035-521e1799b84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:58.542514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6c4657c4-5fbe-4465-a035-521e1799b84d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:58.543722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6c4657c4-5fbe-4465-a035-521e1799b84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:58.552584 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1900/3765] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:58.557205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7bf1a-7f16-43bc-8abb-a38e17194708 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:58.567863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7bf1a-7f16-43bc-8abb-a38e17194708 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:58.573756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fb544-37ce-4a16-ba8b-bcf7924234c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:58.574452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fb544-37ce-4a16-ba8b-bcf7924234c4 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:58.574786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fb544-37ce-4a16-ba8b-bcf7924234c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:58.575117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fb544-37ce-4a16-ba8b-bcf7924234c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:58.575802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fb544-37ce-4a16-ba8b-bcf7924234c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:05:58.576097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fb544-37ce-4a16-ba8b-bcf7924234c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:58.577931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fb544-37ce-4a16-ba8b-bcf7924234c4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:58.579764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:05:58.596153 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1866/3766] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:05:58.597436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4657c4-5fbe-4465-a035-521e1799b84d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:58.612340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4657c4-5fbe-4465-a035-521e1799b84d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:58.635891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529000c-3bd2-4be1-a3be-082d3ed3428d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:58.635891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529000c-3bd2-4be1-a3be-082d3ed3428d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:58.635891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529000c-3bd2-4be1-a3be-082d3ed3428d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:58.635891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529000c-3bd2-4be1-a3be-082d3ed3428d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:58.636190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529000c-3bd2-4be1-a3be-082d3ed3428d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:58.636190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529000c-3bd2-4be1-a3be-082d3ed3428d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:58.695980 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1901/3767] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:05:58 2022] POST /identity/v3/projects => generated 391 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:05:58.700525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fb544-37ce-4a16-ba8b-bcf7924234c4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:58.717550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fb544-37ce-4a16-ba8b-bcf7924234c4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:58.795138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84681224-9df1-422c-aa1d-59d9ae5d4232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:58.795768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84681224-9df1-422c-aa1d-59d9ae5d4232 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:58.796045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84681224-9df1-422c-aa1d-59d9ae5d4232 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:58.796313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84681224-9df1-422c-aa1d-59d9ae5d4232 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:58.797034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84681224-9df1-422c-aa1d-59d9ae5d4232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:58.799887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84681224-9df1-422c-aa1d-59d9ae5d4232 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:58.847008 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1867/3768] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:05:58 2022] POST /identity/v3/users => generated 296 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:05:58.851801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529000c-3bd2-4be1-a3be-082d3ed3428d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:58.865117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529000c-3bd2-4be1-a3be-082d3ed3428d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:58.870078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b2671-ad56-473b-bf80-f6c8ad88ffe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:58.870691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b2671-ad56-473b-bf80-f6c8ad88ffe3 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:58.870964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b2671-ad56-473b-bf80-f6c8ad88ffe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:58.871231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b2671-ad56-473b-bf80-f6c8ad88ffe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:58.872570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b2671-ad56-473b-bf80-f6c8ad88ffe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:58.874402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b2671-ad56-473b-bf80-f6c8ad88ffe3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:58.895305 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1902/3769] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:05:58.899930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84681224-9df1-422c-aa1d-59d9ae5d4232 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:58.913848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84681224-9df1-422c-aa1d-59d9ae5d4232 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:58.924269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede4e03-872d-46fe-b680-4fd4af1bd2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:58.925019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede4e03-872d-46fe-b680-4fd4af1bd2bb admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:58.925289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede4e03-872d-46fe-b680-4fd4af1bd2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:58.925548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede4e03-872d-46fe-b680-4fd4af1bd2bb admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:58.926187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede4e03-872d-46fe-b680-4fd4af1bd2bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:58.927782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede4e03-872d-46fe-b680-4fd4af1bd2bb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:58.943936 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1868/3770] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:05:58.945777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b2671-ad56-473b-bf80-f6c8ad88ffe3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:58.960331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b2671-ad56-473b-bf80-f6c8ad88ffe3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:58.972015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3833d9-8297-44e1-8d11-6a5885fa1e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:58.973194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3833d9-8297-44e1-8d11-6a5885fa1e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:58.973194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3833d9-8297-44e1-8d11-6a5885fa1e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:58.973466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3833d9-8297-44e1-8d11-6a5885fa1e15 admin admin] PATH_INFO: `/v3/projects/fd3bb000eca140b88a168fdbb3a46d01/users/5f2f3dc32ced4c29a6e8329805865f0e/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:59.026384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3833d9-8297-44e1-8d11-6a5885fa1e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3bb000eca140b88a168fdbb3a46d01, user_id=5f2f3dc32ced4c29a6e8329805865f0e, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:59.035565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3833d9-8297-44e1-8d11-6a5885fa1e15 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:59.065664 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1903/3771] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:05:58 2022] PUT /identity/v3/projects/fd3bb000eca140b88a168fdbb3a46d01/users/5f2f3dc32ced4c29a6e8329805865f0e/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:05:59.077696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d3156-6f59-49fc-b05f-d4ec9b38eb32 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:59.077989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d3156-6f59-49fc-b05f-d4ec9b38eb32 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:59.078377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d3156-6f59-49fc-b05f-d4ec9b38eb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:59.125815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-893d3156-6f59-49fc-b05f-d4ec9b38eb32 None None] MFA Rules not processed for user `5f2f3dc32ced4c29a6e8329805865f0e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:05:59.125815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-893d3156-6f59-49fc-b05f-d4ec9b38eb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:59.233206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3833d9-8297-44e1-8d11-6a5885fa1e15 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:59.235320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4b3833d9-8297-44e1-8d11-6a5885fa1e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:05:59.248604 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1869/3772] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:05:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:05:59.431359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3833d9-8297-44e1-8d11-6a5885fa1e15 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:05:59.450624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3d6fc-1974-467f-98f0-0997d711be89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:05:59.453464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3d6fc-1974-467f-98f0-0997d711be89 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:05:59.454040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3d6fc-1974-467f-98f0-0997d711be89 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:05:59.454710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3d6fc-1974-467f-98f0-0997d711be89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:05:59.479325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3d6fc-1974-467f-98f0-0997d711be89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:05:59.484519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3d6fc-1974-467f-98f0-0997d711be89 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:05:59.527438 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1904/3773] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:05:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:05:59.856428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d3156-6f59-49fc-b05f-d4ec9b38eb32 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:05:59.998700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d3156-6f59-49fc-b05f-d4ec9b38eb32 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:00.016724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:00.019242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:00.020123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:00.020663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] PATH_INFO: `/v3/users/5f2f3dc32ced4c29a6e8329805865f0e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:00.021982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f2f3dc32ced4c29a6e8329805865f0e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:00.025574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:00.030956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:00.110963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f2f3dc32ced4c29a6e8329805865f0e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:00.123175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f2f3dc32ced4c29a6e8329805865f0e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:00.138301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f2f3dc32ced4c29a6e8329805865f0e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:00.139172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f2f3dc32ced4c29a6e8329805865f0e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:00.140191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f2f3dc32ced4c29a6e8329805865f0e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:00.157954 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1870/3774] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:05:59 2022] DELETE /identity/v3/users/5f2f3dc32ced4c29a6e8329805865f0e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:00.162727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df64f30-80d9-496d-a9e9-99ef91091106 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:00.163407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df64f30-80d9-496d-a9e9-99ef91091106 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:00.164088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df64f30-80d9-496d-a9e9-99ef91091106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:00.186196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2df64f30-80d9-496d-a9e9-99ef91091106 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:00.187425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2df64f30-80d9-496d-a9e9-99ef91091106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:00.305456 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1905/3775] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:06:00.315772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:00.317686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:00.390081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec915fe-3197-4371-bbd5-0ef1e4af6b85 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:00.393895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359295df-d178-4062-bd47-493a9c25ca84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:00.394537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359295df-d178-4062-bd47-493a9c25ca84 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:00.394951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359295df-d178-4062-bd47-493a9c25ca84 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:00.395221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359295df-d178-4062-bd47-493a9c25ca84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:00.406505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359295df-d178-4062-bd47-493a9c25ca84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:00.407937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359295df-d178-4062-bd47-493a9c25ca84 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:00.419204 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1871/3776] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:00.994669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df64f30-80d9-496d-a9e9-99ef91091106 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:00.997268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2df64f30-80d9-496d-a9e9-99ef91091106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:01.019276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df64f30-80d9-496d-a9e9-99ef91091106 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:01.025549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b48a3-db92-49bb-8a26-c76bfe530dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:01.026426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b48a3-db92-49bb-8a26-c76bfe530dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:01.026851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b48a3-db92-49bb-8a26-c76bfe530dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:01.027323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b48a3-db92-49bb-8a26-c76bfe530dd9 admin admin] PATH_INFO: `/v3/projects/fd3bb000eca140b88a168fdbb3a46d01` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:01.029876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b48a3-db92-49bb-8a26-c76bfe530dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd3bb000eca140b88a168fdbb3a46d01)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:01.033698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b48a3-db92-49bb-8a26-c76bfe530dd9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:01.060913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:01.123664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a22b48a3-db92-49bb-8a26-c76bfe530dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd3bb000eca140b88a168fdbb3a46d01', 'request_id': 'req-a22b48a3-db92-49bb-8a26-c76bfe530dd9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:01.140102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a22b48a3-db92-49bb-8a26-c76bfe530dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd3bb000eca140b88a168fdbb3a46d01', 'request_id': 'req-a22b48a3-db92-49bb-8a26-c76bfe530dd9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:01.148584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a22b48a3-db92-49bb-8a26-c76bfe530dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd3bb000eca140b88a168fdbb3a46d01', 'request_id': 'req-a22b48a3-db92-49bb-8a26-c76bfe530dd9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:01.187028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a22b48a3-db92-49bb-8a26-c76bfe530dd9 admin admin] The token cache is being invalidate because project fd3bb000eca140b88a168fdbb3a46d01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:06:01.187744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a22b48a3-db92-49bb-8a26-c76bfe530dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:01.188601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a22b48a3-db92-49bb-8a26-c76bfe530dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:01.196095 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1906/3777] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:06:00 2022] DELETE /identity/v3/projects/fd3bb000eca140b88a168fdbb3a46d01 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:01.197344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459956bc-bb64-4385-be4f-7648a7251f63 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:01.198134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459956bc-bb64-4385-be4f-7648a7251f63 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:01.198569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459956bc-bb64-4385-be4f-7648a7251f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:01.226430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-459956bc-bb64-4385-be4f-7648a7251f63 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:01.228051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-459956bc-bb64-4385-be4f-7648a7251f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:01.306905 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1872/3778] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:06:01.317308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cccd7e-f80b-43cd-9395-3ee3e2c79c88 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:01.317869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cccd7e-f80b-43cd-9395-3ee3e2c79c88 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:01.318345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cccd7e-f80b-43cd-9395-3ee3e2c79c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:01.385720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f8cccd7e-f80b-43cd-9395-3ee3e2c79c88 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:01.390006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f8cccd7e-f80b-43cd-9395-3ee3e2c79c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:01.452683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459956bc-bb64-4385-be4f-7648a7251f63 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:01.461802 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1907/3779] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:06:01.463397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459956bc-bb64-4385-be4f-7648a7251f63 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:01.473185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2688363-670a-4d59-81ff-6402819f50e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:01.474130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2688363-670a-4d59-81ff-6402819f50e4 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:01.474504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2688363-670a-4d59-81ff-6402819f50e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:01.475173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2688363-670a-4d59-81ff-6402819f50e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:01.477183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2688363-670a-4d59-81ff-6402819f50e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:06:01.477716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2688363-670a-4d59-81ff-6402819f50e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:01.479656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2688363-670a-4d59-81ff-6402819f50e4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:01.500485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cccd7e-f80b-43cd-9395-3ee3e2c79c88 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:01.507978 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1873/3780] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:06:01.519493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cccd7e-f80b-43cd-9395-3ee3e2c79c88 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:01.526345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e58ba-40e2-4001-be68-c1e374c1c49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:01.527260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e58ba-40e2-4001-be68-c1e374c1c49a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:01.527778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e58ba-40e2-4001-be68-c1e374c1c49a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:01.528455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e58ba-40e2-4001-be68-c1e374c1c49a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:01.529495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e58ba-40e2-4001-be68-c1e374c1c49a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:01.531948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e58ba-40e2-4001-be68-c1e374c1c49a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:01.576639 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1908/3781] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:06:01 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:06:01.580771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2688363-670a-4d59-81ff-6402819f50e4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:01.593799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2688363-670a-4d59-81ff-6402819f50e4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:01.612201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:01.718751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251724c-2d47-44dd-a8c3-fa212460fcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:01.719370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251724c-2d47-44dd-a8c3-fa212460fcb5 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:01.719793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251724c-2d47-44dd-a8c3-fa212460fcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:01.720062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251724c-2d47-44dd-a8c3-fa212460fcb5 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:01.721355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251724c-2d47-44dd-a8c3-fa212460fcb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:01.723307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251724c-2d47-44dd-a8c3-fa212460fcb5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:01.752937 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1874/3782] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:06:01 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:06:01.753934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e58ba-40e2-4001-be68-c1e374c1c49a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:01.766817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e58ba-40e2-4001-be68-c1e374c1c49a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:01.773200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3958d-fe3a-4049-8584-a7065344a111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:01.774349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3958d-fe3a-4049-8584-a7065344a111 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:01.774349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3958d-fe3a-4049-8584-a7065344a111 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:01.774554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3958d-fe3a-4049-8584-a7065344a111 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:01.775065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3958d-fe3a-4049-8584-a7065344a111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:01.777641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3958d-fe3a-4049-8584-a7065344a111 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:01.790011 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1909/3783] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:06:01.794079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251724c-2d47-44dd-a8c3-fa212460fcb5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:01.811581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251724c-2d47-44dd-a8c3-fa212460fcb5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:01.818497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b047b-6f86-4703-ae73-677aa1f21beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:01.819067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b047b-6f86-4703-ae73-677aa1f21beb admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:01.819644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b047b-6f86-4703-ae73-677aa1f21beb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:01.820047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b047b-6f86-4703-ae73-677aa1f21beb admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:01.821111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b047b-6f86-4703-ae73-677aa1f21beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:01.823174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b047b-6f86-4703-ae73-677aa1f21beb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:01.841364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3958d-fe3a-4049-8584-a7065344a111 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:01.842364 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1875/3784] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:06:01.859128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3958d-fe3a-4049-8584-a7065344a111 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:01.865394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc45ab-ded7-43ed-9582-67c2f6ba9dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:01.866485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc45ab-ded7-43ed-9582-67c2f6ba9dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:01.867104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc45ab-ded7-43ed-9582-67c2f6ba9dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:01.867587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc45ab-ded7-43ed-9582-67c2f6ba9dc4 admin admin] PATH_INFO: `/v3/projects/b84d9d7e24d643f2a4e500fb574fe7c4/users/74fbd22e52d643cf8c51a2f1ef99debc/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:01.885141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc45ab-ded7-43ed-9582-67c2f6ba9dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b84d9d7e24d643f2a4e500fb574fe7c4, user_id=74fbd22e52d643cf8c51a2f1ef99debc, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:01.888903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc45ab-ded7-43ed-9582-67c2f6ba9dc4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:01.914100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf1d50-2e02-465c-ad0c-a468c735b0da None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:01.914662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf1d50-2e02-465c-ad0c-a468c735b0da None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:01.915172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf1d50-2e02-465c-ad0c-a468c735b0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:01.918996 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1910/3785] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:06:01 2022] PUT /identity/v3/projects/b84d9d7e24d643f2a4e500fb574fe7c4/users/74fbd22e52d643cf8c51a2f1ef99debc/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:06:01.943068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f5bf1d50-2e02-465c-ad0c-a468c735b0da None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:01.944536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f5bf1d50-2e02-465c-ad0c-a468c735b0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:02.076192 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1876/3786] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:02.086208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc45ab-ded7-43ed-9582-67c2f6ba9dc4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:02.088529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7ebc45ab-ded7-43ed-9582-67c2f6ba9dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:02.194412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc45ab-ded7-43ed-9582-67c2f6ba9dc4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:02.198890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716f524-e384-40ea-9478-118b107a2187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:02.199810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716f524-e384-40ea-9478-118b107a2187 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:02.200154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716f524-e384-40ea-9478-118b107a2187 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:02.200527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716f524-e384-40ea-9478-118b107a2187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:02.211602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716f524-e384-40ea-9478-118b107a2187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:02.213630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716f524-e384-40ea-9478-118b107a2187 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:02.247885 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1911/3787] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:03.669862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf1d50-2e02-465c-ad0c-a468c735b0da None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:03.671923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f5bf1d50-2e02-465c-ad0c-a468c735b0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:03.849000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf1d50-2e02-465c-ad0c-a468c735b0da None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:03.857872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0284985-a9bc-4495-9755-e17a2dac7eb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:03.858778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0284985-a9bc-4495-9755-e17a2dac7eb9 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:03.859542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0284985-a9bc-4495-9755-e17a2dac7eb9 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:03.860401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0284985-a9bc-4495-9755-e17a2dac7eb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:03.863129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b0284985-a9bc-4495-9755-e17a2dac7eb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:03.882864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0284985-a9bc-4495-9755-e17a2dac7eb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:03.885114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0284985-a9bc-4495-9755-e17a2dac7eb9 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:03.909878 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1877/3788] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:07.357327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716f524-e384-40ea-9478-118b107a2187 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:07.360920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2716f524-e384-40ea-9478-118b107a2187 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:07.461163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:07.479189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0284985-a9bc-4495-9755-e17a2dac7eb9 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:07.481975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b0284985-a9bc-4495-9755-e17a2dac7eb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:07.503530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716f524-e384-40ea-9478-118b107a2187 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:07.504800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0284985-a9bc-4495-9755-e17a2dac7eb9 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:07.510006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1033cc73-c9f5-4228-a41b-d9a238600a65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:07.511200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033cc73-c9f5-4228-a41b-d9a238600a65 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:07.511630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033cc73-c9f5-4228-a41b-d9a238600a65 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:07.512358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033cc73-c9f5-4228-a41b-d9a238600a65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:07.515478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1033cc73-c9f5-4228-a41b-d9a238600a65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:07.523375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:07.524125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:07.524522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:07.524880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] PATH_INFO: `/v3/users/93502ea5425b49a2943334e7e995c7ea` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:07.526317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93502ea5425b49a2943334e7e995c7ea)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:07.531189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:07.539655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1033cc73-c9f5-4228-a41b-d9a238600a65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:07.542053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1033cc73-c9f5-4228-a41b-d9a238600a65 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:07.555359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:07.561436 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1912/3789] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:07.626254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93502ea5425b49a2943334e7e995c7ea'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:07.639084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93502ea5425b49a2943334e7e995c7ea'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:07.657198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93502ea5425b49a2943334e7e995c7ea'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:07.658483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93502ea5425b49a2943334e7e995c7ea'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:07.659189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93502ea5425b49a2943334e7e995c7ea'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:07.674826 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1878/3790] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:06:07 2022] DELETE /identity/v3/users/93502ea5425b49a2943334e7e995c7ea => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:06:07.679552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417040d-5520-48ec-a43f-bf86f8645491 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:07.679978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417040d-5520-48ec-a43f-bf86f8645491 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:07.680339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417040d-5520-48ec-a43f-bf86f8645491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:07.702895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c417040d-5520-48ec-a43f-bf86f8645491 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:07.704351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c417040d-5520-48ec-a43f-bf86f8645491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:07.833395 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1913/3791] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:07.842094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:07.843900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:07.995283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c4135-e9c6-4c8e-8ef7-ee0d01a15de8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:08.006309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4320bf2-2628-46af-bbc0-1653a41b1f5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:08.007419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4320bf2-2628-46af-bbc0-1653a41b1f5d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:08.007827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4320bf2-2628-46af-bbc0-1653a41b1f5d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:08.008088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4320bf2-2628-46af-bbc0-1653a41b1f5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:08.042809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4320bf2-2628-46af-bbc0-1653a41b1f5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:08.048927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4320bf2-2628-46af-bbc0-1653a41b1f5d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:08.082249 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1879/3792] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:08.778524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417040d-5520-48ec-a43f-bf86f8645491 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:08.778524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c417040d-5520-48ec-a43f-bf86f8645491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:08.792885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417040d-5520-48ec-a43f-bf86f8645491 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:08.799479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:08.800738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:08.801615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:08.802406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3 admin admin] PATH_INFO: `/v3/projects/1090dfab4c044c30bab4769da4b2c9b2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:08.804135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1090dfab4c044c30bab4769da4b2c9b2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:08.807305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:08.906494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1090dfab4c044c30bab4769da4b2c9b2', 'request_id': 'req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:08.924276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1090dfab4c044c30bab4769da4b2c9b2', 'request_id': 'req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:08.933823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1090dfab4c044c30bab4769da4b2c9b2', 'request_id': 'req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:08.953622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3 admin admin] The token cache is being invalidate because project 1090dfab4c044c30bab4769da4b2c9b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:06:08.954043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:08.956827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-07f27541-11c2-4fdf-9eb0-7908c98fb0c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:08.960448 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1914/3793] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:06:08 2022] DELETE /identity/v3/projects/1090dfab4c044c30bab4769da4b2c9b2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:08.964131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4320bf2-2628-46af-bbc0-1653a41b1f5d service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:08.976297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4320bf2-2628-46af-bbc0-1653a41b1f5d service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:09.104260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:09.105182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:09.105671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:09.106125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] PATH_INFO: `/v3/users/7ae4e33ad89d40e688e50811196cb0cd` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:09.107723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae4e33ad89d40e688e50811196cb0cd)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:09.109913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:09.110793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39574ac6-67b6-40ba-aaad-cff7753921c3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:09.111140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39574ac6-67b6-40ba-aaad-cff7753921c3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:09.111474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39574ac6-67b6-40ba-aaad-cff7753921c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:09.164335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-39574ac6-67b6-40ba-aaad-cff7753921c3 None None] MFA Rules not processed for user `74fbd22e52d643cf8c51a2f1ef99debc`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:09.165677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-39574ac6-67b6-40ba-aaad-cff7753921c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:09.175971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:09.251852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae4e33ad89d40e688e50811196cb0cd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:09.265221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae4e33ad89d40e688e50811196cb0cd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:09.279419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:09.285936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae4e33ad89d40e688e50811196cb0cd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:09.287820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae4e33ad89d40e688e50811196cb0cd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:09.288678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae4e33ad89d40e688e50811196cb0cd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:09.335395 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1915/3794] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:09.338590 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1880/3795] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:06:08 2022] DELETE /identity/v3/users/7ae4e33ad89d40e688e50811196cb0cd => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:09.349305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c90b31-8baa-4278-90a4-0eeb3d900e78 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:09.349588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c90b31-8baa-4278-90a4-0eeb3d900e78 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:09.349863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c90b31-8baa-4278-90a4-0eeb3d900e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:09.401733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-26c90b31-8baa-4278-90a4-0eeb3d900e78 None None] MFA Rules not processed for user `74fbd22e52d643cf8c51a2f1ef99debc`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:09.402966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-26c90b31-8baa-4278-90a4-0eeb3d900e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:09.480072 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1916/3796] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:09.560789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:09.562482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:09.579326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf65905-b538-43b9-ae54-d0349dc5327b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:09.579578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf65905-b538-43b9-ae54-d0349dc5327b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:09.579855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf65905-b538-43b9-ae54-d0349dc5327b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:09.617436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1bf65905-b538-43b9-ae54-d0349dc5327b None None] MFA Rules not processed for user `bbc87613dd6349babc15334a66bee64f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:09.618751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1bf65905-b538-43b9-ae54-d0349dc5327b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:09.714259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eea119-2230-4129-aec2-701bfc653ff1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:09.727924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebda37-23b6-4f62-87e2-2ea325697a86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:09.728740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebda37-23b6-4f62-87e2-2ea325697a86 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:09.729115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebda37-23b6-4f62-87e2-2ea325697a86 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:09.729485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebda37-23b6-4f62-87e2-2ea325697a86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:09.759242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebda37-23b6-4f62-87e2-2ea325697a86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:09.762316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebda37-23b6-4f62-87e2-2ea325697a86 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:09.794205 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1917/3797] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:09.794730 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1881/3798] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:10.116845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebda37-23b6-4f62-87e2-2ea325697a86 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:10.119261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4cebda37-23b6-4f62-87e2-2ea325697a86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:10.242025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebda37-23b6-4f62-87e2-2ea325697a86 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:10.252180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87cf7e-ce1e-46a5-8140-670051f38996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:10.253648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87cf7e-ce1e-46a5-8140-670051f38996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:10.253929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87cf7e-ce1e-46a5-8140-670051f38996 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:10.254201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87cf7e-ce1e-46a5-8140-670051f38996 admin admin] PATH_INFO: `/v3/projects/a45eac165cf54a57801bc13408a3ef77` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:10.255482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87cf7e-ce1e-46a5-8140-670051f38996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a45eac165cf54a57801bc13408a3ef77)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:10.258868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87cf7e-ce1e-46a5-8140-670051f38996 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:10.309153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:10.322291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db87cf7e-ce1e-46a5-8140-670051f38996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a45eac165cf54a57801bc13408a3ef77', 'request_id': 'req-db87cf7e-ce1e-46a5-8140-670051f38996'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:10.332016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db87cf7e-ce1e-46a5-8140-670051f38996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a45eac165cf54a57801bc13408a3ef77', 'request_id': 'req-db87cf7e-ce1e-46a5-8140-670051f38996'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:10.341974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db87cf7e-ce1e-46a5-8140-670051f38996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a45eac165cf54a57801bc13408a3ef77', 'request_id': 'req-db87cf7e-ce1e-46a5-8140-670051f38996'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:10.353522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db87cf7e-ce1e-46a5-8140-670051f38996 admin admin] The token cache is being invalidate because project a45eac165cf54a57801bc13408a3ef77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:06:10.353762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db87cf7e-ce1e-46a5-8140-670051f38996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:10.354535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db87cf7e-ce1e-46a5-8140-670051f38996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:10.356135 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1882/3799] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:06:10 2022] DELETE /identity/v3/projects/a45eac165cf54a57801bc13408a3ef77 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:10.361110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d100ef1-cd3b-4bba-8ccf-e45e0f617485 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:10.361474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d100ef1-cd3b-4bba-8ccf-e45e0f617485 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:10.361805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d100ef1-cd3b-4bba-8ccf-e45e0f617485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:10.385738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6d100ef1-cd3b-4bba-8ccf-e45e0f617485 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:10.387146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6d100ef1-cd3b-4bba-8ccf-e45e0f617485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:10.460333 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1918/3800] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:10.556155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31bff9-51e9-4fd2-996f-4488a8631433 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:10.556484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31bff9-51e9-4fd2-996f-4488a8631433 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:10.556761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31bff9-51e9-4fd2-996f-4488a8631433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:10.578123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4f31bff9-51e9-4fd2-996f-4488a8631433 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:10.579336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4f31bff9-51e9-4fd2-996f-4488a8631433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:10.588337 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1883/3801] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:10.592765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d100ef1-cd3b-4bba-8ccf-e45e0f617485 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:10.603311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d100ef1-cd3b-4bba-8ccf-e45e0f617485 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:10.608296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ea610-f705-4869-abf6-d73d0ca7fc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:10.608934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ea610-f705-4869-abf6-d73d0ca7fc78 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:10.609174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ea610-f705-4869-abf6-d73d0ca7fc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:10.609462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ea610-f705-4869-abf6-d73d0ca7fc78 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:10.610099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ea610-f705-4869-abf6-d73d0ca7fc78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:06:10.610396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ea610-f705-4869-abf6-d73d0ca7fc78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:10.611998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ea610-f705-4869-abf6-d73d0ca7fc78 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:10.619395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31bff9-51e9-4fd2-996f-4488a8631433 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:10.620898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4f31bff9-51e9-4fd2-996f-4488a8631433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:10.626010 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1919/3802] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:06:10.630064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ea610-f705-4869-abf6-d73d0ca7fc78 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:10.643173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ea610-f705-4869-abf6-d73d0ca7fc78 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:10.648405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a448d-e08d-48ea-8fdc-02ea6997107a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:10.649082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a448d-e08d-48ea-8fdc-02ea6997107a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:10.649432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a448d-e08d-48ea-8fdc-02ea6997107a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:10.649699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a448d-e08d-48ea-8fdc-02ea6997107a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:10.650332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a448d-e08d-48ea-8fdc-02ea6997107a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:10.652426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a448d-e08d-48ea-8fdc-02ea6997107a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:10.681512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:10.684646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31bff9-51e9-4fd2-996f-4488a8631433 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:10.688650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25341d5a-e996-49e0-92ac-590809fb0283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:10.689264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25341d5a-e996-49e0-92ac-590809fb0283 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:10.689562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25341d5a-e996-49e0-92ac-590809fb0283 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:10.689963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25341d5a-e996-49e0-92ac-590809fb0283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:10.691708 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1920/3803] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:06:10 2022] POST /identity/v3/projects => generated 393 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:06:10.691832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-25341d5a-e996-49e0-92ac-590809fb0283 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:10.693008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a448d-e08d-48ea-8fdc-02ea6997107a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:10.704008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25341d5a-e996-49e0-92ac-590809fb0283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:10.706414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25341d5a-e996-49e0-92ac-590809fb0283 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:10.719436 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1884/3804] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:10.770109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a448d-e08d-48ea-8fdc-02ea6997107a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:10.780836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73c64-3743-479e-93b1-5120f31c6350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:10.780836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73c64-3743-479e-93b1-5120f31c6350 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:10.780836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73c64-3743-479e-93b1-5120f31c6350 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:10.780836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73c64-3743-479e-93b1-5120f31c6350 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:10.781235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f73c64-3743-479e-93b1-5120f31c6350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:10.787092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f73c64-3743-479e-93b1-5120f31c6350 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:10.831249 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1921/3805] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:06:10 2022] POST /identity/v3/users => generated 297 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:06:10.832746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25341d5a-e996-49e0-92ac-590809fb0283 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:10.857505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25341d5a-e996-49e0-92ac-590809fb0283 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:10.889161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64d0f8-c8f7-46ba-bede-26c5794582d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:10.889800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64d0f8-c8f7-46ba-bede-26c5794582d8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:10.890076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64d0f8-c8f7-46ba-bede-26c5794582d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:10.890347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64d0f8-c8f7-46ba-bede-26c5794582d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:10.890998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64d0f8-c8f7-46ba-bede-26c5794582d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:10.892680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64d0f8-c8f7-46ba-bede-26c5794582d8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:10.907702 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1885/3806] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:06:10.912072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73c64-3743-479e-93b1-5120f31c6350 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:10.924807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73c64-3743-479e-93b1-5120f31c6350 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:10.934461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91ce5a-0ff9-4901-ad43-026792bd3688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:10.935846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91ce5a-0ff9-4901-ad43-026792bd3688 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:10.936148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91ce5a-0ff9-4901-ad43-026792bd3688 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:10.936580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91ce5a-0ff9-4901-ad43-026792bd3688 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:10.937239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91ce5a-0ff9-4901-ad43-026792bd3688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:10.938863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91ce5a-0ff9-4901-ad43-026792bd3688 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:10.960250 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1922/3807] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:06:10.961121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64d0f8-c8f7-46ba-bede-26c5794582d8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:10.973220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64d0f8-c8f7-46ba-bede-26c5794582d8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:10.978404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0deb5-a7d9-400d-9fbf-ea3e8c51bd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:10.979203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0deb5-a7d9-400d-9fbf-ea3e8c51bd1f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:10.979578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0deb5-a7d9-400d-9fbf-ea3e8c51bd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:10.979967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0deb5-a7d9-400d-9fbf-ea3e8c51bd1f admin admin] PATH_INFO: `/v3/projects/0c4fab72b68e40c1a3172975b2f8931a/users/ce61530e73f14dc7a9ed1396679b7f82/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:11.004026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0deb5-a7d9-400d-9fbf-ea3e8c51bd1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4fab72b68e40c1a3172975b2f8931a, user_id=ce61530e73f14dc7a9ed1396679b7f82, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:11.007732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0deb5-a7d9-400d-9fbf-ea3e8c51bd1f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:11.045034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15448318-9912-4e22-8d37-b8c532221c96 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:11.045034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15448318-9912-4e22-8d37-b8c532221c96 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:11.045484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15448318-9912-4e22-8d37-b8c532221c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:11.049291 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1886/3808] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:06:10 2022] PUT /identity/v3/projects/0c4fab72b68e40c1a3172975b2f8931a/users/ce61530e73f14dc7a9ed1396679b7f82/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:11.091144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-15448318-9912-4e22-8d37-b8c532221c96 None None] MFA Rules not processed for user `ce61530e73f14dc7a9ed1396679b7f82`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:11.092399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-15448318-9912-4e22-8d37-b8c532221c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:11.233238 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1923/3809] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:11.237891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0deb5-a7d9-400d-9fbf-ea3e8c51bd1f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:11.343656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0deb5-a7d9-400d-9fbf-ea3e8c51bd1f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:11.351115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eea3f2-2154-4a3a-a529-031fea91c1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:11.352488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eea3f2-2154-4a3a-a529-031fea91c1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:11.353193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eea3f2-2154-4a3a-a529-031fea91c1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:11.353660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eea3f2-2154-4a3a-a529-031fea91c1d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:11.354657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eea3f2-2154-4a3a-a529-031fea91c1d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:11.373633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eea3f2-2154-4a3a-a529-031fea91c1d3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:11.503538 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1887/3810] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:06:11 2022] POST /identity/v3/projects => generated 391 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:06:11.508087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15448318-9912-4e22-8d37-b8c532221c96 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:11.535378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15448318-9912-4e22-8d37-b8c532221c96 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:11.669004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d34ff2-c35d-4801-9c55-84e9bd8a3c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:11.669670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d34ff2-c35d-4801-9c55-84e9bd8a3c64 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:11.670037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d34ff2-c35d-4801-9c55-84e9bd8a3c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:11.670205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d34ff2-c35d-4801-9c55-84e9bd8a3c64 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:11.670946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d34ff2-c35d-4801-9c55-84e9bd8a3c64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:11.674402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d34ff2-c35d-4801-9c55-84e9bd8a3c64 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:11.696416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:11.708425 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1924/3811] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:06:11 2022] POST /identity/v3/users => generated 302 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:06:11.712192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eea3f2-2154-4a3a-a529-031fea91c1d3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:11.726754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eea3f2-2154-4a3a-a529-031fea91c1d3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:11.735124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c6b57-cee9-4f67-9b15-0b92ddac369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:11.735921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c6b57-cee9-4f67-9b15-0b92ddac369f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:11.736564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c6b57-cee9-4f67-9b15-0b92ddac369f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:11.736809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c6b57-cee9-4f67-9b15-0b92ddac369f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:11.737495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c6b57-cee9-4f67-9b15-0b92ddac369f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:11.739651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c6b57-cee9-4f67-9b15-0b92ddac369f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:11.747569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:11.757427 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1888/3812] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:06:11.762418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d34ff2-c35d-4801-9c55-84e9bd8a3c64 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:11.776142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d34ff2-c35d-4801-9c55-84e9bd8a3c64 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:11.784110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99508976-3fb8-4ab9-a157-52ba5f56b72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:11.785544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99508976-3fb8-4ab9-a157-52ba5f56b72f admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:11.787257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99508976-3fb8-4ab9-a157-52ba5f56b72f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:11.787742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99508976-3fb8-4ab9-a157-52ba5f56b72f admin admin] PATH_INFO: `/v3/projects/7b305b2cb58346629728f96c3db7db0e/users/5e68e2dae2ac4660aacf43ba3adf9e87/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:11.809043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99508976-3fb8-4ab9-a157-52ba5f56b72f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b305b2cb58346629728f96c3db7db0e, user_id=5e68e2dae2ac4660aacf43ba3adf9e87, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:11.812021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99508976-3fb8-4ab9-a157-52ba5f56b72f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:11.828052 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1925/3813] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:06:11 2022] PUT /identity/v3/projects/7b305b2cb58346629728f96c3db7db0e/users/5e68e2dae2ac4660aacf43ba3adf9e87/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:11.832173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c6b57-cee9-4f67-9b15-0b92ddac369f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:11.843115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c6b57-cee9-4f67-9b15-0b92ddac369f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:11.920091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227855b-b670-4720-b305-4572477d3908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:11.920853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227855b-b670-4720-b305-4572477d3908 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:11.921063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227855b-b670-4720-b305-4572477d3908 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:11.921410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227855b-b670-4720-b305-4572477d3908 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:11.922077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9227855b-b670-4720-b305-4572477d3908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:11.923736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9227855b-b670-4720-b305-4572477d3908 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:11.938394 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1889/3814] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:06:11.942465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99508976-3fb8-4ab9-a157-52ba5f56b72f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:11.955312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99508976-3fb8-4ab9-a157-52ba5f56b72f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:11.961104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defafabf-4519-43d0-9cac-91c4d489c02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:11.961776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defafabf-4519-43d0-9cac-91c4d489c02a admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:11.962053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defafabf-4519-43d0-9cac-91c4d489c02a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:11.962310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defafabf-4519-43d0-9cac-91c4d489c02a admin admin] PATH_INFO: `/v3/projects/7b305b2cb58346629728f96c3db7db0e/users/5e68e2dae2ac4660aacf43ba3adf9e87/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:11.964276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defafabf-4519-43d0-9cac-91c4d489c02a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b305b2cb58346629728f96c3db7db0e, user_id=5e68e2dae2ac4660aacf43ba3adf9e87, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:11.967037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defafabf-4519-43d0-9cac-91c4d489c02a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:11.979769 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1926/3815] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:06:11 2022] PUT /identity/v3/projects/7b305b2cb58346629728f96c3db7db0e/users/5e68e2dae2ac4660aacf43ba3adf9e87/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:06:11.990287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8824c08-65fb-4cf1-bf8f-dd8b2b7b20e0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:11.990565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8824c08-65fb-4cf1-bf8f-dd8b2b7b20e0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:11.990898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8824c08-65fb-4cf1-bf8f-dd8b2b7b20e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:12.031241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a8824c08-65fb-4cf1-bf8f-dd8b2b7b20e0 None None] MFA Rules not processed for user `5e68e2dae2ac4660aacf43ba3adf9e87`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:12.032613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a8824c08-65fb-4cf1-bf8f-dd8b2b7b20e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:12.139930 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1890/3816] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:11 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:12.147668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defafabf-4519-43d0-9cac-91c4d489c02a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:12.150570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-defafabf-4519-43d0-9cac-91c4d489c02a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:12.252149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defafabf-4519-43d0-9cac-91c4d489c02a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:12.257290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935e090-5874-4288-81e9-8d47fad08ce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:12.258511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935e090-5874-4288-81e9-8d47fad08ce5 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:12.258786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935e090-5874-4288-81e9-8d47fad08ce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:12.259051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935e090-5874-4288-81e9-8d47fad08ce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:12.273940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935e090-5874-4288-81e9-8d47fad08ce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:12.275397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935e090-5874-4288-81e9-8d47fad08ce5 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:12.302470 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1927/3817] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:12 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:16.360338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8824c08-65fb-4cf1-bf8f-dd8b2b7b20e0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:16.427682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8824c08-65fb-4cf1-bf8f-dd8b2b7b20e0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:16.433316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:16.434048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:16.434358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:16.434652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] PATH_INFO: `/v3/users/ce61530e73f14dc7a9ed1396679b7f82` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:16.435943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce61530e73f14dc7a9ed1396679b7f82)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:16.437902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:16.466308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:16.498370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce61530e73f14dc7a9ed1396679b7f82'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:16.507208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce61530e73f14dc7a9ed1396679b7f82'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:16.525355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce61530e73f14dc7a9ed1396679b7f82'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:16.526098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce61530e73f14dc7a9ed1396679b7f82'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:16.526704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce61530e73f14dc7a9ed1396679b7f82'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:16.548050 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1891/3818] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:06:16 2022] DELETE /identity/v3/users/ce61530e73f14dc7a9ed1396679b7f82 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:16.554839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd9829-0040-46e8-93e0-360d116f69fc None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:16.555399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd9829-0040-46e8-93e0-360d116f69fc None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:16.555515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd9829-0040-46e8-93e0-360d116f69fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:16.584925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-78fd9829-0040-46e8-93e0-360d116f69fc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:16.586603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-78fd9829-0040-46e8-93e0-360d116f69fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:16.675811 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1928/3819] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:06:16.683600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:16.685711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:16.758421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84954d81-9373-437e-be9d-6c780a154714 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:16.763296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946a7d5-86fa-4582-b521-8f28f0ada95a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:16.764435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946a7d5-86fa-4582-b521-8f28f0ada95a service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:16.764734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946a7d5-86fa-4582-b521-8f28f0ada95a service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:16.764998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946a7d5-86fa-4582-b521-8f28f0ada95a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:16.775797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946a7d5-86fa-4582-b521-8f28f0ada95a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:16.777275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946a7d5-86fa-4582-b521-8f28f0ada95a service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:16.791209 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1892/3820] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:17.206144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd9829-0040-46e8-93e0-360d116f69fc None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:17.207733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-78fd9829-0040-46e8-93e0-360d116f69fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:17.244129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd9829-0040-46e8-93e0-360d116f69fc None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:17.249161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:17.249844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:17.250116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:17.250410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] PATH_INFO: `/v3/projects/0c4fab72b68e40c1a3172975b2f8931a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:17.252009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c4fab72b68e40c1a3172975b2f8931a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:17.254247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:17.313394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c4fab72b68e40c1a3172975b2f8931a', 'request_id': 'req-55e81d69-edf8-4f03-be53-edf8880d8363'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:17.322109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c4fab72b68e40c1a3172975b2f8931a', 'request_id': 'req-55e81d69-edf8-4f03-be53-edf8880d8363'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:17.325053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:17.331187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c4fab72b68e40c1a3172975b2f8931a', 'request_id': 'req-55e81d69-edf8-4f03-be53-edf8880d8363'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:17.352453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] The token cache is being invalidate because project 0c4fab72b68e40c1a3172975b2f8931a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:06:17.352868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:17.353728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:17.359411 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1929/3821] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:06:17 2022] DELETE /identity/v3/projects/0c4fab72b68e40c1a3172975b2f8931a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:17.360429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946a7d5-86fa-4582-b521-8f28f0ada95a service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:17.375357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946a7d5-86fa-4582-b521-8f28f0ada95a service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:17.462023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:17.462673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:17.462949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:17.463229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] PATH_INFO: `/v3/users/5e68e2dae2ac4660aacf43ba3adf9e87` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:17.464432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e68e2dae2ac4660aacf43ba3adf9e87)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:17.467942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:17.519159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:17.521132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:17.550444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:17.553665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e68e2dae2ac4660aacf43ba3adf9e87'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:17.566871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e68e2dae2ac4660aacf43ba3adf9e87'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:17.586955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e68e2dae2ac4660aacf43ba3adf9e87'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:17.588501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e68e2dae2ac4660aacf43ba3adf9e87'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:17.588928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e68e2dae2ac4660aacf43ba3adf9e87'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:17.604992 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1893/3822] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:06:17 2022] DELETE /identity/v3/users/5e68e2dae2ac4660aacf43ba3adf9e87 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:17.656308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e81d69-edf8-4f03-be53-edf8880d8363 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:17.741124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2f5ac-a4ee-42b7-839f-79986cc3469a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:17.741968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2f5ac-a4ee-42b7-839f-79986cc3469a service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:17.742320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2f5ac-a4ee-42b7-839f-79986cc3469a service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:17.742656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2f5ac-a4ee-42b7-839f-79986cc3469a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:17.744957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7ca2f5ac-a4ee-42b7-839f-79986cc3469a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:17.761469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2f5ac-a4ee-42b7-839f-79986cc3469a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:17.763347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2f5ac-a4ee-42b7-839f-79986cc3469a service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:17.782332 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1930/3823] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:17.872856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:17.874388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:17.973071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b91fa8-f0f9-47ca-9f74-c076ea8eac44 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:17.978876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6075c-05cd-4245-9531-a90e31be2a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:17.979566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6075c-05cd-4245-9531-a90e31be2a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:17.979827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6075c-05cd-4245-9531-a90e31be2a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:17.980091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6075c-05cd-4245-9531-a90e31be2a8e admin admin] PATH_INFO: `/v3/projects/7b305b2cb58346629728f96c3db7db0e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:17.981320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6075c-05cd-4245-9531-a90e31be2a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b305b2cb58346629728f96c3db7db0e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:17.983267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6075c-05cd-4245-9531-a90e31be2a8e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:18.048922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d2f6075c-05cd-4245-9531-a90e31be2a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b305b2cb58346629728f96c3db7db0e', 'request_id': 'req-d2f6075c-05cd-4245-9531-a90e31be2a8e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:18.057319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d2f6075c-05cd-4245-9531-a90e31be2a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b305b2cb58346629728f96c3db7db0e', 'request_id': 'req-d2f6075c-05cd-4245-9531-a90e31be2a8e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:18.067279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d2f6075c-05cd-4245-9531-a90e31be2a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b305b2cb58346629728f96c3db7db0e', 'request_id': 'req-d2f6075c-05cd-4245-9531-a90e31be2a8e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:18.081156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d2f6075c-05cd-4245-9531-a90e31be2a8e admin admin] The token cache is being invalidate because project 7b305b2cb58346629728f96c3db7db0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:06:18.081439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d2f6075c-05cd-4245-9531-a90e31be2a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:18.082403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d2f6075c-05cd-4245-9531-a90e31be2a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:18.083693 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1894/3824] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:06:17 2022] DELETE /identity/v3/projects/7b305b2cb58346629728f96c3db7db0e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:18.089045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4b055-ae35-47bc-9df0-7398ed8a968f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:18.089323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4b055-ae35-47bc-9df0-7398ed8a968f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:18.089552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4b055-ae35-47bc-9df0-7398ed8a968f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:18.109684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-22a4b055-ae35-47bc-9df0-7398ed8a968f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:18.111061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-22a4b055-ae35-47bc-9df0-7398ed8a968f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:18.175052 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1931/3825] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:18.183689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f642be-6c79-4f37-b390-e093d202b13e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:18.183966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f642be-6c79-4f37-b390-e093d202b13e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:18.184232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f642be-6c79-4f37-b390-e093d202b13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:18.205650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d2f642be-6c79-4f37-b390-e093d202b13e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:18.206878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d2f642be-6c79-4f37-b390-e093d202b13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:18.217288 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1895/3826] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:18.221909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4b055-ae35-47bc-9df0-7398ed8a968f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:18.232880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4b055-ae35-47bc-9df0-7398ed8a968f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:18.238353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f57413-62a1-4206-a92c-f6f33d785b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:18.239044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f57413-62a1-4206-a92c-f6f33d785b62 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:18.239378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f57413-62a1-4206-a92c-f6f33d785b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:18.239714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f57413-62a1-4206-a92c-f6f33d785b62 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:18.240554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f57413-62a1-4206-a92c-f6f33d785b62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:06:18.241407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f57413-62a1-4206-a92c-f6f33d785b62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:18.243269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f57413-62a1-4206-a92c-f6f33d785b62 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:18.257756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:18.266680 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1932/3827] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:06:18.271195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f642be-6c79-4f37-b390-e093d202b13e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:18.280797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f642be-6c79-4f37-b390-e093d202b13e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:18.285430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fa8da-4a1b-4d38-a14d-ba2e2b6c5d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:18.286075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fa8da-4a1b-4d38-a14d-ba2e2b6c5d71 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:18.286344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fa8da-4a1b-4d38-a14d-ba2e2b6c5d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:18.286602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fa8da-4a1b-4d38-a14d-ba2e2b6c5d71 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:18.287215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101fa8da-4a1b-4d38-a14d-ba2e2b6c5d71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:18.289332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101fa8da-4a1b-4d38-a14d-ba2e2b6c5d71 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:18.320848 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1896/3828] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:06:18 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:06:18.322757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f57413-62a1-4206-a92c-f6f33d785b62 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:18.334187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f57413-62a1-4206-a92c-f6f33d785b62 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:18.404488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb5606-b0d9-4995-9d85-70006ccc6981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:18.405102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb5606-b0d9-4995-9d85-70006ccc6981 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:18.405373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb5606-b0d9-4995-9d85-70006ccc6981 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:18.405645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb5606-b0d9-4995-9d85-70006ccc6981 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:18.406472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb5606-b0d9-4995-9d85-70006ccc6981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:18.408621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb5606-b0d9-4995-9d85-70006ccc6981 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:18.435604 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1933/3829] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:06:18 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:06:18.437258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fa8da-4a1b-4d38-a14d-ba2e2b6c5d71 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:18.449105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fa8da-4a1b-4d38-a14d-ba2e2b6c5d71 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:18.454229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f08dae-d8df-4ae1-8d8a-f9095f7c61c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:18.454836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f08dae-d8df-4ae1-8d8a-f9095f7c61c5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:18.455106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f08dae-d8df-4ae1-8d8a-f9095f7c61c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:18.455364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f08dae-d8df-4ae1-8d8a-f9095f7c61c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:18.455996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f08dae-d8df-4ae1-8d8a-f9095f7c61c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:18.457640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f08dae-d8df-4ae1-8d8a-f9095f7c61c5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:18.468316 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1897/3830] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:06:18.472250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb5606-b0d9-4995-9d85-70006ccc6981 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:18.482951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb5606-b0d9-4995-9d85-70006ccc6981 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:18.488019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10bf9e-1402-4540-9581-74da650e0a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:18.488755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10bf9e-1402-4540-9581-74da650e0a36 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:18.489090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10bf9e-1402-4540-9581-74da650e0a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:18.489429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10bf9e-1402-4540-9581-74da650e0a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:18.490209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10bf9e-1402-4540-9581-74da650e0a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:18.492014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10bf9e-1402-4540-9581-74da650e0a36 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:18.502937 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1934/3831] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:06:18.507213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f08dae-d8df-4ae1-8d8a-f9095f7c61c5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:18.518593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f08dae-d8df-4ae1-8d8a-f9095f7c61c5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:18.523829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe4abf-0e77-4551-91e2-8ac514b4acbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:18.524595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe4abf-0e77-4551-91e2-8ac514b4acbf admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:18.524908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe4abf-0e77-4551-91e2-8ac514b4acbf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:18.525176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe4abf-0e77-4551-91e2-8ac514b4acbf admin admin] PATH_INFO: `/v3/projects/2e6b01267924426b90f763bb19d25ad3/users/e47bd109c2744fe8b859ec3ff0849051/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:18.537743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:18.543076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe4abf-0e77-4551-91e2-8ac514b4acbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6b01267924426b90f763bb19d25ad3, user_id=e47bd109c2744fe8b859ec3ff0849051, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:18.546713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe4abf-0e77-4551-91e2-8ac514b4acbf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:18.561206 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1898/3832] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:06:18 2022] PUT /identity/v3/projects/2e6b01267924426b90f763bb19d25ad3/users/e47bd109c2744fe8b859ec3ff0849051/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:18.572461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9db20-01cf-469b-8e65-baf204b7afb4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:18.572679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9db20-01cf-469b-8e65-baf204b7afb4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:18.572947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9db20-01cf-469b-8e65-baf204b7afb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:18.614057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f4b9db20-01cf-469b-8e65-baf204b7afb4 None None] MFA Rules not processed for user `e47bd109c2744fe8b859ec3ff0849051`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:18.615298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f4b9db20-01cf-469b-8e65-baf204b7afb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:18.682587 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1935/3833] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:18.696387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe4abf-0e77-4551-91e2-8ac514b4acbf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:18.698269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b0fe4abf-0e77-4551-91e2-8ac514b4acbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:18.767258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe4abf-0e77-4551-91e2-8ac514b4acbf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:18.770984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d6ad9-13b1-40dc-b430-7df27cad0798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:18.771589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d6ad9-13b1-40dc-b430-7df27cad0798 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:18.771891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d6ad9-13b1-40dc-b430-7df27cad0798 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:18.772157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d6ad9-13b1-40dc-b430-7df27cad0798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:18.781493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d6ad9-13b1-40dc-b430-7df27cad0798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:18.782937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d6ad9-13b1-40dc-b430-7df27cad0798 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:18.794431 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1899/3834] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:20.756653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f62c9e-6244-4390-9f11-661fe1a9f398 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:20.757236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f62c9e-6244-4390-9f11-661fe1a9f398 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:20.757563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f62c9e-6244-4390-9f11-661fe1a9f398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:20.778431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-95f62c9e-6244-4390-9f11-661fe1a9f398 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:20.780600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-95f62c9e-6244-4390-9f11-661fe1a9f398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:20.840732 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1936/3835] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:20.859157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d6ad9-13b1-40dc-b430-7df27cad0798 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:20.861053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-da3d6ad9-13b1-40dc-b430-7df27cad0798 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:20.974987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d6ad9-13b1-40dc-b430-7df27cad0798 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:20.981222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e9189-f7ff-4610-99db-90765a303f46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:20.982118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e9189-f7ff-4610-99db-90765a303f46 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:20.982545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e9189-f7ff-4610-99db-90765a303f46 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:20.983003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e9189-f7ff-4610-99db-90765a303f46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:20.995265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e9189-f7ff-4610-99db-90765a303f46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:20.999719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e9189-f7ff-4610-99db-90765a303f46 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:21.015063 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1900/3836] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:06:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:06:24.196265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f62c9e-6244-4390-9f11-661fe1a9f398 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:24.197742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-95f62c9e-6244-4390-9f11-661fe1a9f398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:24.229492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:24.405184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f62c9e-6244-4390-9f11-661fe1a9f398 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:24.415738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bb3a7-4eeb-4ea1-b5e9-1bad6fe1c0b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:24.416506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bb3a7-4eeb-4ea1-b5e9-1bad6fe1c0b8 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:24.416784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bb3a7-4eeb-4ea1-b5e9-1bad6fe1c0b8 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:24.417049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bb3a7-4eeb-4ea1-b5e9-1bad6fe1c0b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:24.418843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-532bb3a7-4eeb-4ea1-b5e9-1bad6fe1c0b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:24.513406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bb3a7-4eeb-4ea1-b5e9-1bad6fe1c0b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:24.515287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bb3a7-4eeb-4ea1-b5e9-1bad6fe1c0b8 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:24.545236 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1937/3837] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:06:31.007555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=62,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:34.524040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=51 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:35.940159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e9189-f7ff-4610-99db-90765a303f46 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:35.941988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-435e9189-f7ff-4610-99db-90765a303f46 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:36.014523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bb3a7-4eeb-4ea1-b5e9-1bad6fe1c0b8 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:36.016252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-532bb3a7-4eeb-4ea1-b5e9-1bad6fe1c0b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:36.083463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e9189-f7ff-4610-99db-90765a303f46 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:36.091203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcc3f4-9877-4d71-95f1-b66a54080582 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:36.092856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcc3f4-9877-4d71-95f1-b66a54080582 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:36.093239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcc3f4-9877-4d71-95f1-b66a54080582 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:36.093604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcc3f4-9877-4d71-95f1-b66a54080582 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:36.096072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-90bcc3f4-9877-4d71-95f1-b66a54080582 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:36.118947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcc3f4-9877-4d71-95f1-b66a54080582 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:36.121389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcc3f4-9877-4d71-95f1-b66a54080582 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:36.147032 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1901/3838] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:36.158986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bb3a7-4eeb-4ea1-b5e9-1bad6fe1c0b8 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:36.165685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66643265-f4fe-491a-96d8-50cfd8cf5add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:36.166488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66643265-f4fe-491a-96d8-50cfd8cf5add admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:36.166863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66643265-f4fe-491a-96d8-50cfd8cf5add admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:36.167226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66643265-f4fe-491a-96d8-50cfd8cf5add admin admin] PATH_INFO: `/v3/users/74fbd22e52d643cf8c51a2f1ef99debc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:36.168599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66643265-f4fe-491a-96d8-50cfd8cf5add admin admin] RBAC: Authorizing `identity:delete_user(user_id=74fbd22e52d643cf8c51a2f1ef99debc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:36.170781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66643265-f4fe-491a-96d8-50cfd8cf5add admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:36.257334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-66643265-f4fe-491a-96d8-50cfd8cf5add admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74fbd22e52d643cf8c51a2f1ef99debc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:36.283505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-66643265-f4fe-491a-96d8-50cfd8cf5add admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74fbd22e52d643cf8c51a2f1ef99debc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:36.297850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-66643265-f4fe-491a-96d8-50cfd8cf5add admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74fbd22e52d643cf8c51a2f1ef99debc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:36.323528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-66643265-f4fe-491a-96d8-50cfd8cf5add admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74fbd22e52d643cf8c51a2f1ef99debc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:36.328831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-66643265-f4fe-491a-96d8-50cfd8cf5add admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74fbd22e52d643cf8c51a2f1ef99debc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:36.341048 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1938/3839] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:06:36 2022] DELETE /identity/v3/users/74fbd22e52d643cf8c51a2f1ef99debc => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:06:36.543929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcc3f4-9877-4d71-95f1-b66a54080582 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:36.555552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcc3f4-9877-4d71-95f1-b66a54080582 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:36.643359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e528e-7938-47bd-9a85-e8f7327e5a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:36.645029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e528e-7938-47bd-9a85-e8f7327e5a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:36.646801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e528e-7938-47bd-9a85-e8f7327e5a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:36.647189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e528e-7938-47bd-9a85-e8f7327e5a47 admin admin] PATH_INFO: `/v3/projects/b84d9d7e24d643f2a4e500fb574fe7c4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:36.649239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e528e-7938-47bd-9a85-e8f7327e5a47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b84d9d7e24d643f2a4e500fb574fe7c4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:36.651777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e528e-7938-47bd-9a85-e8f7327e5a47 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:36.734242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b9e528e-7938-47bd-9a85-e8f7327e5a47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b84d9d7e24d643f2a4e500fb574fe7c4', 'request_id': 'req-1b9e528e-7938-47bd-9a85-e8f7327e5a47'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:36.741728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b9e528e-7938-47bd-9a85-e8f7327e5a47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b84d9d7e24d643f2a4e500fb574fe7c4', 'request_id': 'req-1b9e528e-7938-47bd-9a85-e8f7327e5a47'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:36.752424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b9e528e-7938-47bd-9a85-e8f7327e5a47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b84d9d7e24d643f2a4e500fb574fe7c4', 'request_id': 'req-1b9e528e-7938-47bd-9a85-e8f7327e5a47'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:36.790211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b9e528e-7938-47bd-9a85-e8f7327e5a47 admin admin] The token cache is being invalidate because project b84d9d7e24d643f2a4e500fb574fe7c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:06:36.790586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b9e528e-7938-47bd-9a85-e8f7327e5a47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:36.791921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b9e528e-7938-47bd-9a85-e8f7327e5a47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:36.795488 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1902/3840] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:06:36 2022] DELETE /identity/v3/projects/b84d9d7e24d643f2a4e500fb574fe7c4 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:36.800480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206c582-61d9-485f-bc94-8feed1557c13 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:36.801038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206c582-61d9-485f-bc94-8feed1557c13 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:36.801631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206c582-61d9-485f-bc94-8feed1557c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:36.843869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1206c582-61d9-485f-bc94-8feed1557c13 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:36.845315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1206c582-61d9-485f-bc94-8feed1557c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:36.927339 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1939/3841] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:36.938141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a48910-9460-4038-bb37-7793e140fdf5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:36.938623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a48910-9460-4038-bb37-7793e140fdf5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:36.939064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a48910-9460-4038-bb37-7793e140fdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:36.964499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f5a48910-9460-4038-bb37-7793e140fdf5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:36.965731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f5a48910-9460-4038-bb37-7793e140fdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:36.974047 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1903/3842] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:36.979556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206c582-61d9-485f-bc94-8feed1557c13 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:37.000446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206c582-61d9-485f-bc94-8feed1557c13 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:37.018407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006174d3-b4ce-4a24-922b-f048d2396d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:37.020329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006174d3-b4ce-4a24-922b-f048d2396d3c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:37.022099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006174d3-b4ce-4a24-922b-f048d2396d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:37.022561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006174d3-b4ce-4a24-922b-f048d2396d3c admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:37.023524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006174d3-b4ce-4a24-922b-f048d2396d3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:06:37.024164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006174d3-b4ce-4a24-922b-f048d2396d3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:37.026131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006174d3-b4ce-4a24-922b-f048d2396d3c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:37.046191 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1940/3843] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:06:37.047884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a48910-9460-4038-bb37-7793e140fdf5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:37.059836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a48910-9460-4038-bb37-7793e140fdf5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:37.065262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1474a1-35d2-498e-a08a-ba770d347651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:37.065882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1474a1-35d2-498e-a08a-ba770d347651 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:37.066153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1474a1-35d2-498e-a08a-ba770d347651 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:37.066405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1474a1-35d2-498e-a08a-ba770d347651 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:37.067045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1474a1-35d2-498e-a08a-ba770d347651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:37.069395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1474a1-35d2-498e-a08a-ba770d347651 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:37.082461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:37.108094 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1904/3844] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:06:37 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:06:37.112738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006174d3-b4ce-4a24-922b-f048d2396d3c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:37.125219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006174d3-b4ce-4a24-922b-f048d2396d3c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:37.130430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:37.204480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfa927-f893-4977-8d6f-a3b324908a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:37.205212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfa927-f893-4977-8d6f-a3b324908a66 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:37.205384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfa927-f893-4977-8d6f-a3b324908a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:37.205644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfa927-f893-4977-8d6f-a3b324908a66 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:37.206257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfa927-f893-4977-8d6f-a3b324908a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:37.208315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfa927-f893-4977-8d6f-a3b324908a66 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:37.240105 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1941/3845] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:06:37 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:06:37.244854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1474a1-35d2-498e-a08a-ba770d347651 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:37.256162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1474a1-35d2-498e-a08a-ba770d347651 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:37.260852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c536d28-90b9-4386-a829-aa38e3b3bc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:37.261463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c536d28-90b9-4386-a829-aa38e3b3bc70 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:37.261730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c536d28-90b9-4386-a829-aa38e3b3bc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:37.261976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c536d28-90b9-4386-a829-aa38e3b3bc70 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:37.262607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c536d28-90b9-4386-a829-aa38e3b3bc70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:37.264230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c536d28-90b9-4386-a829-aa38e3b3bc70 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:37.278983 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1905/3846] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:06:37.280430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfa927-f893-4977-8d6f-a3b324908a66 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:37.291344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfa927-f893-4977-8d6f-a3b324908a66 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:37.295960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d3b36-c8f2-4105-b765-0f5ee730ee19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:37.305658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d3b36-c8f2-4105-b765-0f5ee730ee19 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:37.305658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d3b36-c8f2-4105-b765-0f5ee730ee19 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:37.305658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d3b36-c8f2-4105-b765-0f5ee730ee19 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:37.305658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d3b36-c8f2-4105-b765-0f5ee730ee19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:37.305658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d3b36-c8f2-4105-b765-0f5ee730ee19 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:37.315064 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1942/3847] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:06:37.317010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c536d28-90b9-4386-a829-aa38e3b3bc70 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:37.329961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c536d28-90b9-4386-a829-aa38e3b3bc70 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:37.336352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921532e-ca9e-4c7a-be39-92f622d83c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:37.337602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921532e-ca9e-4c7a-be39-92f622d83c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:37.337893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921532e-ca9e-4c7a-be39-92f622d83c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:37.338161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921532e-ca9e-4c7a-be39-92f622d83c69 admin admin] PATH_INFO: `/v3/projects/73509e0b7e4c404fb48b66c784d07ce9/users/e9af61808fca4c1bb32f135d92cd9dee/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:37.342922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d3b36-c8f2-4105-b765-0f5ee730ee19 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:37.344061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a65d3b36-c8f2-4105-b765-0f5ee730ee19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:37.358140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921532e-ca9e-4c7a-be39-92f622d83c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73509e0b7e4c404fb48b66c784d07ce9, user_id=e9af61808fca4c1bb32f135d92cd9dee, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:37.358298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d3b36-c8f2-4105-b765-0f5ee730ee19 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:37.362241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921532e-ca9e-4c7a-be39-92f622d83c69 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:37.363028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944600e-7ed3-4d06-9646-cafafcfc2fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:37.363660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944600e-7ed3-4d06-9646-cafafcfc2fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:37.363929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944600e-7ed3-4d06-9646-cafafcfc2fba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:37.364193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944600e-7ed3-4d06-9646-cafafcfc2fba admin admin] PATH_INFO: `/v3/users/e47bd109c2744fe8b859ec3ff0849051` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:37.365670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944600e-7ed3-4d06-9646-cafafcfc2fba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e47bd109c2744fe8b859ec3ff0849051)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:37.367933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944600e-7ed3-4d06-9646-cafafcfc2fba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:37.384958 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1906/3848] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:06:37 2022] PUT /identity/v3/projects/73509e0b7e4c404fb48b66c784d07ce9/users/e9af61808fca4c1bb32f135d92cd9dee/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:37.396111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55117ff-859e-482d-afba-e8abc62da918 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:37.396878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55117ff-859e-482d-afba-e8abc62da918 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:37.396878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55117ff-859e-482d-afba-e8abc62da918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:37.433952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b55117ff-859e-482d-afba-e8abc62da918 None None] MFA Rules not processed for user `e9af61808fca4c1bb32f135d92cd9dee`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:37.435503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b55117ff-859e-482d-afba-e8abc62da918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:37.475211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3944600e-7ed3-4d06-9646-cafafcfc2fba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e47bd109c2744fe8b859ec3ff0849051'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:37.485385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3944600e-7ed3-4d06-9646-cafafcfc2fba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e47bd109c2744fe8b859ec3ff0849051'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:37.495332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3944600e-7ed3-4d06-9646-cafafcfc2fba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e47bd109c2744fe8b859ec3ff0849051'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:37.510783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3944600e-7ed3-4d06-9646-cafafcfc2fba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e47bd109c2744fe8b859ec3ff0849051'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:37.511869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3944600e-7ed3-4d06-9646-cafafcfc2fba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e47bd109c2744fe8b859ec3ff0849051'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:37.518571 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1943/3849] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:06:37 2022] DELETE /identity/v3/users/e47bd109c2744fe8b859ec3ff0849051 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:37.523348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f258e8-f2f8-4213-ac93-ec40f769f2a6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:37.523625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f258e8-f2f8-4213-ac93-ec40f769f2a6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:37.523895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f258e8-f2f8-4213-ac93-ec40f769f2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:37.527237 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1907/3850] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:37.536863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fc529-f577-42ff-bcde-5e034ebbad38 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:37.537208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fc529-f577-42ff-bcde-5e034ebbad38 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:37.537507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fc529-f577-42ff-bcde-5e034ebbad38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:37.543352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-03f258e8-f2f8-4213-ac93-ec40f769f2a6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:37.544766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-03f258e8-f2f8-4213-ac93-ec40f769f2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:37.556412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-502fc529-f577-42ff-bcde-5e034ebbad38 None None] MFA Rules not processed for user `e9af61808fca4c1bb32f135d92cd9dee`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:37.557635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-502fc529-f577-42ff-bcde-5e034ebbad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:37.609111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:37.618626 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1908/3851] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:06:37.625406 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1944/3852] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:37.633700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fc529-f577-42ff-bcde-5e034ebbad38 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:37.635927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f258e8-f2f8-4213-ac93-ec40f769f2a6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:37.637425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-502fc529-f577-42ff-bcde-5e034ebbad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:37.637548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-03f258e8-f2f8-4213-ac93-ec40f769f2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:37.707598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:37.717671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f258e8-f2f8-4213-ac93-ec40f769f2a6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:37.721333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb38dfb-2f85-4fc1-bbd6-1bf61b057c61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:37.721948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb38dfb-2f85-4fc1-bbd6-1bf61b057c61 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:37.722223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb38dfb-2f85-4fc1-bbd6-1bf61b057c61 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:37.722488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb38dfb-2f85-4fc1-bbd6-1bf61b057c61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:37.732242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb38dfb-2f85-4fc1-bbd6-1bf61b057c61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:37.733785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb38dfb-2f85-4fc1-bbd6-1bf61b057c61 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:37.734996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fc529-f577-42ff-bcde-5e034ebbad38 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:37.741605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86fcbf-ffaa-485f-9551-ea139f785dd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:37.742217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86fcbf-ffaa-485f-9551-ea139f785dd3 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:37.742543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86fcbf-ffaa-485f-9551-ea139f785dd3 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:37.742742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86fcbf-ffaa-485f-9551-ea139f785dd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:37.744522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3f86fcbf-ffaa-485f-9551-ea139f785dd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:37.744640 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1945/3853] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:06:37.756763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86fcbf-ffaa-485f-9551-ea139f785dd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:37.758806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86fcbf-ffaa-485f-9551-ea139f785dd3 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:37.771925 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1909/3854] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:37.955782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb38dfb-2f85-4fc1-bbd6-1bf61b057c61 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:37.957604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dfb38dfb-2f85-4fc1-bbd6-1bf61b057c61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:37.984195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb38dfb-2f85-4fc1-bbd6-1bf61b057c61 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:37.989584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f7827-6bfa-45f1-b17d-ba39031da082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:37.990374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f7827-6bfa-45f1-b17d-ba39031da082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:37.991262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f7827-6bfa-45f1-b17d-ba39031da082 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:37.991536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f7827-6bfa-45f1-b17d-ba39031da082 admin admin] PATH_INFO: `/v3/projects/2e6b01267924426b90f763bb19d25ad3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:37.992786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f7827-6bfa-45f1-b17d-ba39031da082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6b01267924426b90f763bb19d25ad3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:37.994666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f7827-6bfa-45f1-b17d-ba39031da082 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:38.044952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a4f7827-6bfa-45f1-b17d-ba39031da082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6b01267924426b90f763bb19d25ad3', 'request_id': 'req-8a4f7827-6bfa-45f1-b17d-ba39031da082'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:38.054846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a4f7827-6bfa-45f1-b17d-ba39031da082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6b01267924426b90f763bb19d25ad3', 'request_id': 'req-8a4f7827-6bfa-45f1-b17d-ba39031da082'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:38.062092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a4f7827-6bfa-45f1-b17d-ba39031da082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6b01267924426b90f763bb19d25ad3', 'request_id': 'req-8a4f7827-6bfa-45f1-b17d-ba39031da082'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:38.074674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a4f7827-6bfa-45f1-b17d-ba39031da082 admin admin] The token cache is being invalidate because project 2e6b01267924426b90f763bb19d25ad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:06:38.075081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a4f7827-6bfa-45f1-b17d-ba39031da082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:38.075787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a4f7827-6bfa-45f1-b17d-ba39031da082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:06:38.077344 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1946/3855] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:06:37 2022] DELETE /identity/v3/projects/2e6b01267924426b90f763bb19d25ad3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:06:38.082261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6d341-49f6-4aec-8f32-5722a71a3bf1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:38.082761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6d341-49f6-4aec-8f32-5722a71a3bf1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:38.082933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6d341-49f6-4aec-8f32-5722a71a3bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:38.102720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2fa6d341-49f6-4aec-8f32-5722a71a3bf1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:38.103935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2fa6d341-49f6-4aec-8f32-5722a71a3bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:38.169260 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1910/3856] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:06:38.178462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32036673-f58d-446b-8625-e79f5fb8d38d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:38.178753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32036673-f58d-446b-8625-e79f5fb8d38d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:38.179016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32036673-f58d-446b-8625-e79f5fb8d38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:38.202269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-32036673-f58d-446b-8625-e79f5fb8d38d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:38.203465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-32036673-f58d-446b-8625-e79f5fb8d38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:38.223316 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1947/3857] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:38.229794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6d341-49f6-4aec-8f32-5722a71a3bf1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:38.242420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6d341-49f6-4aec-8f32-5722a71a3bf1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:38.248132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae708b1-54d8-4540-8059-85cfa091b28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:38.248802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae708b1-54d8-4540-8059-85cfa091b28e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:38.249083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae708b1-54d8-4540-8059-85cfa091b28e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:38.249352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae708b1-54d8-4540-8059-85cfa091b28e admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:38.250005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae708b1-54d8-4540-8059-85cfa091b28e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:06:38.250306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae708b1-54d8-4540-8059-85cfa091b28e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:38.251973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae708b1-54d8-4540-8059-85cfa091b28e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:38.265622 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1911/3858] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:06:38.270594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32036673-f58d-446b-8625-e79f5fb8d38d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:38.287999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32036673-f58d-446b-8625-e79f5fb8d38d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:38.293290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f1edc-8020-48fb-9bea-e9debd405538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:38.293960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f1edc-8020-48fb-9bea-e9debd405538 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:38.294228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f1edc-8020-48fb-9bea-e9debd405538 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:38.294532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f1edc-8020-48fb-9bea-e9debd405538 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:38.295140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f1edc-8020-48fb-9bea-e9debd405538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:38.297118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f1edc-8020-48fb-9bea-e9debd405538 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:38.329791 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1948/3859] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:06:38 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:06:38.334148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae708b1-54d8-4540-8059-85cfa091b28e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:38.344398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae708b1-54d8-4540-8059-85cfa091b28e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:38.428509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b4132-92a8-4e4b-9ab0-527dd59b3ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:38.429152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b4132-92a8-4e4b-9ab0-527dd59b3ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:38.429425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b4132-92a8-4e4b-9ab0-527dd59b3ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:38.429745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b4132-92a8-4e4b-9ab0-527dd59b3ca2 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:38.430386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b4132-92a8-4e4b-9ab0-527dd59b3ca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:38.432345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b4132-92a8-4e4b-9ab0-527dd59b3ca2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:38.461345 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1912/3860] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:06:38 2022] POST /identity/v3/users => generated 284 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:06:38.465426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f1edc-8020-48fb-9bea-e9debd405538 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:38.488434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f1edc-8020-48fb-9bea-e9debd405538 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:38.494770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba30713-a885-430e-808c-4ef2af70e19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:38.495543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba30713-a885-430e-808c-4ef2af70e19e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:38.495822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba30713-a885-430e-808c-4ef2af70e19e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:38.496093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba30713-a885-430e-808c-4ef2af70e19e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:38.496822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba30713-a885-430e-808c-4ef2af70e19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:38.498478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba30713-a885-430e-808c-4ef2af70e19e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:38.510468 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1949/3861] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:06:38.514463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b4132-92a8-4e4b-9ab0-527dd59b3ca2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:38.528628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b4132-92a8-4e4b-9ab0-527dd59b3ca2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:38.533455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057263cf-0c46-47ec-bb8b-73895754c8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:38.534036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057263cf-0c46-47ec-bb8b-73895754c8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:38.534342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057263cf-0c46-47ec-bb8b-73895754c8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:38.534539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057263cf-0c46-47ec-bb8b-73895754c8e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:38.535251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057263cf-0c46-47ec-bb8b-73895754c8e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:38.536808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057263cf-0c46-47ec-bb8b-73895754c8e0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:38.547705 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1913/3862] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:06:38.552238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba30713-a885-430e-808c-4ef2af70e19e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:38.562940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba30713-a885-430e-808c-4ef2af70e19e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:38.567682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e6d49-619b-4d26-a64b-10be2a9f6311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:38.568462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e6d49-619b-4d26-a64b-10be2a9f6311 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:38.568733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e6d49-619b-4d26-a64b-10be2a9f6311 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:38.568995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e6d49-619b-4d26-a64b-10be2a9f6311 admin admin] PATH_INFO: `/v3/projects/4a915791e3be4bc0b116cf4ecf89bb92/users/e2de9161e73441c19b636d9d55446517/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:38.584664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e6d49-619b-4d26-a64b-10be2a9f6311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a915791e3be4bc0b116cf4ecf89bb92, user_id=e2de9161e73441c19b636d9d55446517, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:38.587300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e6d49-619b-4d26-a64b-10be2a9f6311 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:38.604230 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1950/3863] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:06:38 2022] PUT /identity/v3/projects/4a915791e3be4bc0b116cf4ecf89bb92/users/e2de9161e73441c19b636d9d55446517/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:06:38.615678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4e3dc-e61b-42cb-8274-6157b9d33463 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:38.616168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4e3dc-e61b-42cb-8274-6157b9d33463 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:38.616544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4e3dc-e61b-42cb-8274-6157b9d33463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:38.648685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-84f4e3dc-e61b-42cb-8274-6157b9d33463 None None] MFA Rules not processed for user `e2de9161e73441c19b636d9d55446517`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:38.650002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-84f4e3dc-e61b-42cb-8274-6157b9d33463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:38.685509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:38.723450 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1914/3864] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:38.734666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e6d49-619b-4d26-a64b-10be2a9f6311 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:38.736364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7d5e6d49-619b-4d26-a64b-10be2a9f6311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:38.758831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:38.801204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e6d49-619b-4d26-a64b-10be2a9f6311 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:38.805580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef738d2f-f1ea-4600-88f9-33900f047d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:38.806194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef738d2f-f1ea-4600-88f9-33900f047d63 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:38.806463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef738d2f-f1ea-4600-88f9-33900f047d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:38.806723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef738d2f-f1ea-4600-88f9-33900f047d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:38.816011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef738d2f-f1ea-4600-88f9-33900f047d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:38.817753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef738d2f-f1ea-4600-88f9-33900f047d63 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:38.828387 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1951/3865] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:06:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:06:41.012434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bd5f2-6444-44ea-8f43-0701da30c90c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:41.013269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bd5f2-6444-44ea-8f43-0701da30c90c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:41.013710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bd5f2-6444-44ea-8f43-0701da30c90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:41.042576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ea9bd5f2-6444-44ea-8f43-0701da30c90c None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:41.043868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ea9bd5f2-6444-44ea-8f43-0701da30c90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:41.151145 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1915/3866] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:06:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:41.161044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef738d2f-f1ea-4600-88f9-33900f047d63 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:06:41.163382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ef738d2f-f1ea-4600-88f9-33900f047d63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:41.295579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef738d2f-f1ea-4600-88f9-33900f047d63 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:06:41.301395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb96c64-ceae-45ff-afc1-2da4ae20c313 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:06:41.302219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb96c64-ceae-45ff-afc1-2da4ae20c313 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:41.303018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb96c64-ceae-45ff-afc1-2da4ae20c313 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:41.303366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb96c64-ceae-45ff-afc1-2da4ae20c313 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:41.313636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb96c64-ceae-45ff-afc1-2da4ae20c313 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:06:41.315684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb96c64-ceae-45ff-afc1-2da4ae20c313 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:06:41.327245 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1952/3867] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:06:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:06:42.726341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c8121-7729-4a13-99a7-8f4a8c0d6186 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:42.728538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c8121-7729-4a13-99a7-8f4a8c0d6186 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:42.729004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c8121-7729-4a13-99a7-8f4a8c0d6186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:42.755497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-038c8121-7729-4a13-99a7-8f4a8c0d6186 None None] MFA Rules not processed for user `b20aea7088dc4e4fadb737c906cf7ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:42.757497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-038c8121-7729-4a13-99a7-8f4a8c0d6186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:42.847119 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1916/3868] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:06:46.260140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a46a2-deff-4faf-986b-d622b878f376 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:06:46.260788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a46a2-deff-4faf-986b-d622b878f376 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:06:46.261287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a46a2-deff-4faf-986b-d622b878f376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:06:46.295366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cd9a46a2-deff-4faf-986b-d622b878f376 None None] MFA Rules not processed for user `30c7e3ac41df480ea0c198af4f0c49ea`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:06:46.296786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cd9a46a2-deff-4faf-986b-d622b878f376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:06:46.375605 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1953/3869] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:06:52.825701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=63 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:06:56.368087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=74 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:01.651806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872cb8-89bc-496e-984c-ae7e3ed4f369 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:01.652389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872cb8-89bc-496e-984c-ae7e3ed4f369 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:01.652389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872cb8-89bc-496e-984c-ae7e3ed4f369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:01.673262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-08872cb8-89bc-496e-984c-ae7e3ed4f369 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:01.674480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-08872cb8-89bc-496e-984c-ae7e3ed4f369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:01.688521 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1917/3870] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:07:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:01.716621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a46a2-deff-4faf-986b-d622b878f376 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:01.737795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a46a2-deff-4faf-986b-d622b878f376 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:01.751696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc3cc0-b8f1-49ae-8ca5-24213b730508 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:01.753158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc3cc0-b8f1-49ae-8ca5-24213b730508 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:01.753791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc3cc0-b8f1-49ae-8ca5-24213b730508 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:01.755237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc3cc0-b8f1-49ae-8ca5-24213b730508 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:01.778210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc3cc0-b8f1-49ae-8ca5-24213b730508 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:01.779885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc3cc0-b8f1-49ae-8ca5-24213b730508 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:01.794243 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1954/3871] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:07:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:07:07.212932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08872cb8-89bc-496e-984c-ae7e3ed4f369 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:07.215447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-08872cb8-89bc-496e-984c-ae7e3ed4f369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:07.362126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08872cb8-89bc-496e-984c-ae7e3ed4f369 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:07.372503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd2b15-2d24-4e3e-89c5-e5cd90524921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:07.373280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd2b15-2d24-4e3e-89c5-e5cd90524921 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:07.373661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd2b15-2d24-4e3e-89c5-e5cd90524921 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:07.374034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd2b15-2d24-4e3e-89c5-e5cd90524921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:07.377325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-04bd2b15-2d24-4e3e-89c5-e5cd90524921 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:07.552981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd2b15-2d24-4e3e-89c5-e5cd90524921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:07.558755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd2b15-2d24-4e3e-89c5-e5cd90524921 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:07.612247 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1918/3872] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:11.788042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:15.534379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc3cc0-b8f1-49ae-8ca5-24213b730508 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:15.538000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5cbc3cc0-b8f1-49ae-8ca5-24213b730508 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:15.648505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc3cc0-b8f1-49ae-8ca5-24213b730508 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:15.654474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9a4ae-9363-4874-affb-cfc5509ffd4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:15.655255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9a4ae-9363-4874-affb-cfc5509ffd4a service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:15.655566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9a4ae-9363-4874-affb-cfc5509ffd4a service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:15.656014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9a4ae-9363-4874-affb-cfc5509ffd4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:15.658445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-acf9a4ae-9363-4874-affb-cfc5509ffd4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:15.758897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9a4ae-9363-4874-affb-cfc5509ffd4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:15.760563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9a4ae-9363-4874-affb-cfc5509ffd4a service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:15.780648 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1955/3873] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:15 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:16.904943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfee8b-950a-4eaa-beda-04fcd86197f5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:16.905496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfee8b-950a-4eaa-beda-04fcd86197f5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:16.905496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfee8b-950a-4eaa-beda-04fcd86197f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:16.931372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-debfee8b-950a-4eaa-beda-04fcd86197f5 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:16.933366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-debfee8b-950a-4eaa-beda-04fcd86197f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:16.941432 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1919/3874] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:07:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:16.956584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9a4ae-9363-4874-affb-cfc5509ffd4a service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:16.972194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9a4ae-9363-4874-affb-cfc5509ffd4a service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:16.984422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8dc7a-2e6a-40ff-b8eb-6c5471ed56db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:16.985058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8dc7a-2e6a-40ff-b8eb-6c5471ed56db service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:16.985446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8dc7a-2e6a-40ff-b8eb-6c5471ed56db service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:16.985715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8dc7a-2e6a-40ff-b8eb-6c5471ed56db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:16.995439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8dc7a-2e6a-40ff-b8eb-6c5471ed56db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:16.997154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8dc7a-2e6a-40ff-b8eb-6c5471ed56db service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:17.006692 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1956/3875] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:07:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:07:17.123502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfee8b-950a-4eaa-beda-04fcd86197f5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:17.206132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfee8b-950a-4eaa-beda-04fcd86197f5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:17.211046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:17.211693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:17.211966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:17.212232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] PATH_INFO: `/v3/users/e2de9161e73441c19b636d9d55446517` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:17.213379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2de9161e73441c19b636d9d55446517)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:17.218222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:17.241238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:17.277754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2de9161e73441c19b636d9d55446517'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:17.285354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2de9161e73441c19b636d9d55446517'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:17.298399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2de9161e73441c19b636d9d55446517'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:17.299064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2de9161e73441c19b636d9d55446517'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:17.299660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2de9161e73441c19b636d9d55446517'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:17.313218 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1920/3876] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:17 2022] DELETE /identity/v3/users/e2de9161e73441c19b636d9d55446517 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:17.318606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4415e19-5951-49d7-8394-8eef10e614ba None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:17.318895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4415e19-5951-49d7-8394-8eef10e614ba None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:17.319161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4415e19-5951-49d7-8394-8eef10e614ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:17.339685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a4415e19-5951-49d7-8394-8eef10e614ba None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:17.340869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a4415e19-5951-49d7-8394-8eef10e614ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:17.403127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:17.405808 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1957/3877] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:07:17.419087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:17.420594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:17.511580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fb42a-92f8-4cca-b799-bad999ee93c7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:17.515241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62570735-a1f3-4094-9b87-2b802bb0bc6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:17.515852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62570735-a1f3-4094-9b87-2b802bb0bc6b service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:17.516150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62570735-a1f3-4094-9b87-2b802bb0bc6b service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:17.516453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62570735-a1f3-4094-9b87-2b802bb0bc6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:17.526516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62570735-a1f3-4094-9b87-2b802bb0bc6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:17.528058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62570735-a1f3-4094-9b87-2b802bb0bc6b service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:17.538317 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1921/3878] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:17.827113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4415e19-5951-49d7-8394-8eef10e614ba None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:17.828589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a4415e19-5951-49d7-8394-8eef10e614ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:17.841760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4415e19-5951-49d7-8394-8eef10e614ba None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:17.853491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:17.854148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:17.854421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:17.854686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875 admin admin] PATH_INFO: `/v3/projects/4a915791e3be4bc0b116cf4ecf89bb92` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:17.855893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a915791e3be4bc0b116cf4ecf89bb92)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:17.857939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:17.910633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a915791e3be4bc0b116cf4ecf89bb92', 'request_id': 'req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:17.919849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a915791e3be4bc0b116cf4ecf89bb92', 'request_id': 'req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:17.925300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a915791e3be4bc0b116cf4ecf89bb92', 'request_id': 'req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:17.936573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875 admin admin] The token cache is being invalidate because project 4a915791e3be4bc0b116cf4ecf89bb92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:17.936799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:17.937419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-52d11d84-8d9b-45ed-99b1-5c1b5c0e6875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:17.938926 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1958/3879] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:17 2022] DELETE /identity/v3/projects/4a915791e3be4bc0b116cf4ecf89bb92 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:17.944419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c979265-5e57-4193-be74-b096d3b8ee6e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:17.945709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c979265-5e57-4193-be74-b096d3b8ee6e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:17.946031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c979265-5e57-4193-be74-b096d3b8ee6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:17.965914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5c979265-5e57-4193-be74-b096d3b8ee6e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:17.967119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5c979265-5e57-4193-be74-b096d3b8ee6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:18.030468 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1922/3880] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:18.039059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f83f55-711c-4bf2-ac48-3435187e35e2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:18.039588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f83f55-711c-4bf2-ac48-3435187e35e2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:18.039588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f83f55-711c-4bf2-ac48-3435187e35e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:18.062890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-80f83f55-711c-4bf2-ac48-3435187e35e2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:18.064180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-80f83f55-711c-4bf2-ac48-3435187e35e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:18.072617 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1959/3881] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:18.077351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c979265-5e57-4193-be74-b096d3b8ee6e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:18.087965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c979265-5e57-4193-be74-b096d3b8ee6e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:18.093245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb731bf3-38fc-47f6-872a-0ed947b49ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:18.093919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb731bf3-38fc-47f6-872a-0ed947b49ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:18.094191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb731bf3-38fc-47f6-872a-0ed947b49ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:18.094460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb731bf3-38fc-47f6-872a-0ed947b49ae4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:18.095111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb731bf3-38fc-47f6-872a-0ed947b49ae4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:07:18.095399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb731bf3-38fc-47f6-872a-0ed947b49ae4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:18.097060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb731bf3-38fc-47f6-872a-0ed947b49ae4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:18.110193 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1923/3882] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:07:18.114782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f83f55-711c-4bf2-ac48-3435187e35e2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:18.125305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f83f55-711c-4bf2-ac48-3435187e35e2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:18.131201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efac4b-a724-4f50-ac51-e8c86222be71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:18.131892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efac4b-a724-4f50-ac51-e8c86222be71 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:18.132168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efac4b-a724-4f50-ac51-e8c86222be71 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:18.132473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efac4b-a724-4f50-ac51-e8c86222be71 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:18.133127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efac4b-a724-4f50-ac51-e8c86222be71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:18.135035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efac4b-a724-4f50-ac51-e8c86222be71 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:18.168404 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1960/3883] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:18 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:18.173127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb731bf3-38fc-47f6-872a-0ed947b49ae4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:18.184770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb731bf3-38fc-47f6-872a-0ed947b49ae4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:18.204235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:18.245342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7422973-3bdc-423b-b6cd-cfa87c848dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:18.246163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7422973-3bdc-423b-b6cd-cfa87c848dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:18.246643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7422973-3bdc-423b-b6cd-cfa87c848dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:18.246996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7422973-3bdc-423b-b6cd-cfa87c848dd9 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:18.247552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7422973-3bdc-423b-b6cd-cfa87c848dd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:18.249599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7422973-3bdc-423b-b6cd-cfa87c848dd9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:18.274443 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1924/3884] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:18 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:07:18.276294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efac4b-a724-4f50-ac51-e8c86222be71 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:18.287846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efac4b-a724-4f50-ac51-e8c86222be71 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:18.293252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3726b-d1a2-4e2f-adeb-93f00f16f126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:18.293854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3726b-d1a2-4e2f-adeb-93f00f16f126 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:18.294124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3726b-d1a2-4e2f-adeb-93f00f16f126 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:18.294385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3726b-d1a2-4e2f-adeb-93f00f16f126 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:18.295017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3726b-d1a2-4e2f-adeb-93f00f16f126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:18.296819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3726b-d1a2-4e2f-adeb-93f00f16f126 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:18.308294 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1961/3885] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:18.311761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7422973-3bdc-423b-b6cd-cfa87c848dd9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:18.322992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7422973-3bdc-423b-b6cd-cfa87c848dd9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:18.328674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655e744-ca90-4605-a342-e5849655e095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:18.329290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655e744-ca90-4605-a342-e5849655e095 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:18.329548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655e744-ca90-4605-a342-e5849655e095 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:18.330309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655e744-ca90-4605-a342-e5849655e095 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:18.330436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655e744-ca90-4605-a342-e5849655e095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:18.332028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655e744-ca90-4605-a342-e5849655e095 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:18.343170 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1925/3886] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:18.347589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3726b-d1a2-4e2f-adeb-93f00f16f126 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:18.357818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3726b-d1a2-4e2f-adeb-93f00f16f126 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:18.363032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c82a8-f7c5-46dd-abae-ad0871696d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:18.363759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c82a8-f7c5-46dd-abae-ad0871696d21 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:18.364029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c82a8-f7c5-46dd-abae-ad0871696d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:18.364751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c82a8-f7c5-46dd-abae-ad0871696d21 admin admin] PATH_INFO: `/v3/projects/3a13fd62da484de5984962d08f133210/users/a70f1c9f49a34e6abe2dd14070bd78d2/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:18.380962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c82a8-f7c5-46dd-abae-ad0871696d21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a13fd62da484de5984962d08f133210, user_id=a70f1c9f49a34e6abe2dd14070bd78d2, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:18.383917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c82a8-f7c5-46dd-abae-ad0871696d21 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:18.401474 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1962/3887] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:18 2022] PUT /identity/v3/projects/3a13fd62da484de5984962d08f133210/users/a70f1c9f49a34e6abe2dd14070bd78d2/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:18.411654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b349d6-15c7-4026-9c76-2f8298818008 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:18.412073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b349d6-15c7-4026-9c76-2f8298818008 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:18.412424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b349d6-15c7-4026-9c76-2f8298818008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:18.445214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-61b349d6-15c7-4026-9c76-2f8298818008 None None] MFA Rules not processed for user `a70f1c9f49a34e6abe2dd14070bd78d2`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:18.446964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-61b349d6-15c7-4026-9c76-2f8298818008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:18.511610 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1926/3888] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:18.522985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c82a8-f7c5-46dd-abae-ad0871696d21 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:18.526340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1b7c82a8-f7c5-46dd-abae-ad0871696d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:18.603308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c82a8-f7c5-46dd-abae-ad0871696d21 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:18.607882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc59faf-54d9-49e4-bf3c-2ccdc0f23503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:18.608527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc59faf-54d9-49e4-bf3c-2ccdc0f23503 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:18.608801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc59faf-54d9-49e4-bf3c-2ccdc0f23503 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:18.609062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc59faf-54d9-49e4-bf3c-2ccdc0f23503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:18.618756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc59faf-54d9-49e4-bf3c-2ccdc0f23503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:18.620193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc59faf-54d9-49e4-bf3c-2ccdc0f23503 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:18.629918 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1963/3889] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:18.643958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b349d6-15c7-4026-9c76-2f8298818008 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:18.713893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b349d6-15c7-4026-9c76-2f8298818008 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:18.721842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:18.722496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:18.722820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:18.723179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] PATH_INFO: `/v3/users/a70f1c9f49a34e6abe2dd14070bd78d2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:18.724991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a70f1c9f49a34e6abe2dd14070bd78d2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:18.726913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:18.765602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:18.779688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a70f1c9f49a34e6abe2dd14070bd78d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:18.788280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a70f1c9f49a34e6abe2dd14070bd78d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:18.801008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a70f1c9f49a34e6abe2dd14070bd78d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:18.801689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a70f1c9f49a34e6abe2dd14070bd78d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:18.802577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a70f1c9f49a34e6abe2dd14070bd78d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:18.821345 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1927/3890] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:18 2022] DELETE /identity/v3/users/a70f1c9f49a34e6abe2dd14070bd78d2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:18.825938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152e0a8-9eba-4e94-8da9-6a60fafebf9f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:18.826217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152e0a8-9eba-4e94-8da9-6a60fafebf9f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:18.826484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152e0a8-9eba-4e94-8da9-6a60fafebf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:18.851703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a152e0a8-9eba-4e94-8da9-6a60fafebf9f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:18.852499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a152e0a8-9eba-4e94-8da9-6a60fafebf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:18.866990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:18.927916 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1964/3891] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:18.937850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:18.939486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:19.003344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292de6c7-e7fa-49df-90a9-0e94ec8eaba6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:19.006968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747a34e-9047-4a60-8070-f840ee0359a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:19.007561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747a34e-9047-4a60-8070-f840ee0359a8 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:19.007829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747a34e-9047-4a60-8070-f840ee0359a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:19.008088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747a34e-9047-4a60-8070-f840ee0359a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:19.017217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747a34e-9047-4a60-8070-f840ee0359a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:19.018592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747a34e-9047-4a60-8070-f840ee0359a8 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:19.028390 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1928/3892] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:19.352646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152e0a8-9eba-4e94-8da9-6a60fafebf9f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:19.354259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a152e0a8-9eba-4e94-8da9-6a60fafebf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:19.366742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152e0a8-9eba-4e94-8da9-6a60fafebf9f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:19.372784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8003689-7b8e-4678-9f81-ad6f3c238df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:19.373393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8003689-7b8e-4678-9f81-ad6f3c238df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:19.373821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8003689-7b8e-4678-9f81-ad6f3c238df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:19.374348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8003689-7b8e-4678-9f81-ad6f3c238df2 admin admin] PATH_INFO: `/v3/projects/3a13fd62da484de5984962d08f133210` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:19.375654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8003689-7b8e-4678-9f81-ad6f3c238df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a13fd62da484de5984962d08f133210)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:19.377799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8003689-7b8e-4678-9f81-ad6f3c238df2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:19.427820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8003689-7b8e-4678-9f81-ad6f3c238df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a13fd62da484de5984962d08f133210', 'request_id': 'req-c8003689-7b8e-4678-9f81-ad6f3c238df2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:19.437257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8003689-7b8e-4678-9f81-ad6f3c238df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a13fd62da484de5984962d08f133210', 'request_id': 'req-c8003689-7b8e-4678-9f81-ad6f3c238df2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:19.448462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8003689-7b8e-4678-9f81-ad6f3c238df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a13fd62da484de5984962d08f133210', 'request_id': 'req-c8003689-7b8e-4678-9f81-ad6f3c238df2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:19.461183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8003689-7b8e-4678-9f81-ad6f3c238df2 admin admin] The token cache is being invalidate because project 3a13fd62da484de5984962d08f133210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:19.461474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8003689-7b8e-4678-9f81-ad6f3c238df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:19.462294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8003689-7b8e-4678-9f81-ad6f3c238df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:19.463959 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1965/3893] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:19 2022] DELETE /identity/v3/projects/3a13fd62da484de5984962d08f133210 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:19.469146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd51109-a333-4f85-9a96-308c772741de None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:19.469476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd51109-a333-4f85-9a96-308c772741de None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:19.469767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd51109-a333-4f85-9a96-308c772741de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:19.490821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0fd51109-a333-4f85-9a96-308c772741de None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:19.492305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0fd51109-a333-4f85-9a96-308c772741de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:19.553061 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1929/3894] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:19.564104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae4827-5820-4817-ae34-78baa5473a34 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:19.564427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae4827-5820-4817-ae34-78baa5473a34 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:19.564707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae4827-5820-4817-ae34-78baa5473a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:19.585998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9dae4827-5820-4817-ae34-78baa5473a34 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:19.587220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9dae4827-5820-4817-ae34-78baa5473a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:19.595860 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1966/3895] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:19.601115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd51109-a333-4f85-9a96-308c772741de None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:19.611627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd51109-a333-4f85-9a96-308c772741de None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:19.618241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e420e6-7527-47d9-86e4-d0b6ac0eb64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:19.618852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e420e6-7527-47d9-86e4-d0b6ac0eb64c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:19.619125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e420e6-7527-47d9-86e4-d0b6ac0eb64c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:19.619390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e420e6-7527-47d9-86e4-d0b6ac0eb64c admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:19.620033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e420e6-7527-47d9-86e4-d0b6ac0eb64c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:07:19.620316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e420e6-7527-47d9-86e4-d0b6ac0eb64c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:19.622026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e420e6-7527-47d9-86e4-d0b6ac0eb64c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:19.634532 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1930/3896] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:07:19.638952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae4827-5820-4817-ae34-78baa5473a34 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:19.649115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae4827-5820-4817-ae34-78baa5473a34 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:19.653885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8110e-071f-4259-b9f7-833ee906f30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:19.654482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8110e-071f-4259-b9f7-833ee906f30f admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:19.654752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8110e-071f-4259-b9f7-833ee906f30f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:19.655016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8110e-071f-4259-b9f7-833ee906f30f admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:19.655631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8110e-071f-4259-b9f7-833ee906f30f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:19.657563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8110e-071f-4259-b9f7-833ee906f30f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:19.687381 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1967/3897] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:19 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:19.691511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e420e6-7527-47d9-86e4-d0b6ac0eb64c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:19.702333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e420e6-7527-47d9-86e4-d0b6ac0eb64c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:19.742683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:19.764007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c4620-0fd6-461e-99e8-7debe5e3a648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:19.764653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c4620-0fd6-461e-99e8-7debe5e3a648 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:19.764928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c4620-0fd6-461e-99e8-7debe5e3a648 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:19.765199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c4620-0fd6-461e-99e8-7debe5e3a648 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:19.765838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c4620-0fd6-461e-99e8-7debe5e3a648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:19.767770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c4620-0fd6-461e-99e8-7debe5e3a648 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:19.789646 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1931/3898] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:19 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:07:19.793792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8110e-071f-4259-b9f7-833ee906f30f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:19.806036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8110e-071f-4259-b9f7-833ee906f30f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:19.812341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf791b-d015-4651-aefe-bd5f0f700b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:19.813073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf791b-d015-4651-aefe-bd5f0f700b7b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:19.813341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf791b-d015-4651-aefe-bd5f0f700b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:19.813604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf791b-d015-4651-aefe-bd5f0f700b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:19.814245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf791b-d015-4651-aefe-bd5f0f700b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:19.815828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf791b-d015-4651-aefe-bd5f0f700b7b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:19.834539 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1968/3899] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:19.838887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c4620-0fd6-461e-99e8-7debe5e3a648 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:19.850081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c4620-0fd6-461e-99e8-7debe5e3a648 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:19.855534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c08e67-d97f-4390-baef-a067a23dd893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:19.856141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c08e67-d97f-4390-baef-a067a23dd893 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:19.856408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c08e67-d97f-4390-baef-a067a23dd893 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:19.856688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c08e67-d97f-4390-baef-a067a23dd893 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:19.857380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c08e67-d97f-4390-baef-a067a23dd893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:19.859034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c08e67-d97f-4390-baef-a067a23dd893 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:19.877168 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1932/3900] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:19.883241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf791b-d015-4651-aefe-bd5f0f700b7b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:19.894153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf791b-d015-4651-aefe-bd5f0f700b7b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:19.899978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50431b82-f879-4a4f-8a5e-b3dcfa29957b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:19.900650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50431b82-f879-4a4f-8a5e-b3dcfa29957b admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:19.900913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50431b82-f879-4a4f-8a5e-b3dcfa29957b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:19.901173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50431b82-f879-4a4f-8a5e-b3dcfa29957b admin admin] PATH_INFO: `/v3/projects/f5f43b60b06c412ba90f7d296e266a1f/users/01ab827376f9449c9d69a91c693da4ad/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:19.917101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50431b82-f879-4a4f-8a5e-b3dcfa29957b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f43b60b06c412ba90f7d296e266a1f, user_id=01ab827376f9449c9d69a91c693da4ad, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:19.919843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50431b82-f879-4a4f-8a5e-b3dcfa29957b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:19.935870 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1969/3901] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:19 2022] PUT /identity/v3/projects/f5f43b60b06c412ba90f7d296e266a1f/users/01ab827376f9449c9d69a91c693da4ad/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:19.942085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42f175-54e6-4be6-b01e-510a01853b9c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:19.943340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42f175-54e6-4be6-b01e-510a01853b9c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:19.943614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42f175-54e6-4be6-b01e-510a01853b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:19.965488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-dc42f175-54e6-4be6-b01e-510a01853b9c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:19.966716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dc42f175-54e6-4be6-b01e-510a01853b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:20.030292 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1933/3902] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:20.040845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50431b82-f879-4a4f-8a5e-b3dcfa29957b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:20.042607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-50431b82-f879-4a4f-8a5e-b3dcfa29957b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:20.089742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:20.111490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50431b82-f879-4a4f-8a5e-b3dcfa29957b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:20.117223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f126c3b-7881-49ed-a0b2-00c7228dafc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:20.117844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f126c3b-7881-49ed-a0b2-00c7228dafc2 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:20.118113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f126c3b-7881-49ed-a0b2-00c7228dafc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:20.118361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f126c3b-7881-49ed-a0b2-00c7228dafc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:20.133572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f126c3b-7881-49ed-a0b2-00c7228dafc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:20.135323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f126c3b-7881-49ed-a0b2-00c7228dafc2 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:20.147091 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1970/3903] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:20.940594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24a331-479d-4113-bf3e-2975a687bad0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:20.941591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24a331-479d-4113-bf3e-2975a687bad0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:20.941996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24a331-479d-4113-bf3e-2975a687bad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:20.973214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0b24a331-479d-4113-bf3e-2975a687bad0 None None] MFA Rules not processed for user `01ab827376f9449c9d69a91c693da4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:20.974755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0b24a331-479d-4113-bf3e-2975a687bad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:21.045941 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1934/3904] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:20 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:21.050561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f126c3b-7881-49ed-a0b2-00c7228dafc2 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:21.061850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f126c3b-7881-49ed-a0b2-00c7228dafc2 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:21.066467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab90f1-0452-4bc0-8175-32ff66c9fff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:21.067068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab90f1-0452-4bc0-8175-32ff66c9fff0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:21.067438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab90f1-0452-4bc0-8175-32ff66c9fff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:21.067716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab90f1-0452-4bc0-8175-32ff66c9fff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:21.068341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab90f1-0452-4bc0-8175-32ff66c9fff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:21.070279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab90f1-0452-4bc0-8175-32ff66c9fff0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:21.100498 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1971/3905] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:21 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:21.102033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24a331-479d-4113-bf3e-2975a687bad0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:21.113038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24a331-479d-4113-bf3e-2975a687bad0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:21.166353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:21.180265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22de02f-2d67-437b-8f67-1db4efb3805a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:21.181211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22de02f-2d67-437b-8f67-1db4efb3805a admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:21.181557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22de02f-2d67-437b-8f67-1db4efb3805a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:21.181929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22de02f-2d67-437b-8f67-1db4efb3805a admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:21.182920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22de02f-2d67-437b-8f67-1db4efb3805a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:21.186015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22de02f-2d67-437b-8f67-1db4efb3805a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:21.214553 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1935/3906] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:21 2022] POST /identity/v3/users => generated 309 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:21.216026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab90f1-0452-4bc0-8175-32ff66c9fff0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:21.238086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab90f1-0452-4bc0-8175-32ff66c9fff0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:21.250526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0be6ab-4169-4fde-960e-13375bb637ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:21.251392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0be6ab-4169-4fde-960e-13375bb637ad admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:21.251682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0be6ab-4169-4fde-960e-13375bb637ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:21.251934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0be6ab-4169-4fde-960e-13375bb637ad admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:21.252882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0be6ab-4169-4fde-960e-13375bb637ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:21.254728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0be6ab-4169-4fde-960e-13375bb637ad admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:21.266791 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1972/3907] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:21.270988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22de02f-2d67-437b-8f67-1db4efb3805a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:21.281310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22de02f-2d67-437b-8f67-1db4efb3805a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:21.286244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c5559-afa5-40af-b52c-62b7262fe6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:21.286949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c5559-afa5-40af-b52c-62b7262fe6ad admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:21.287220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c5559-afa5-40af-b52c-62b7262fe6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:21.287476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c5559-afa5-40af-b52c-62b7262fe6ad admin admin] PATH_INFO: `/v3/projects/7c67b78f009d42ca911026ebebd4e91c/users/a343db0514324dd1aeb08368a3a711ef/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:21.302352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c5559-afa5-40af-b52c-62b7262fe6ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c67b78f009d42ca911026ebebd4e91c, user_id=a343db0514324dd1aeb08368a3a711ef, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:21.305116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c5559-afa5-40af-b52c-62b7262fe6ad admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:21.322462 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1936/3908] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:21 2022] PUT /identity/v3/projects/7c67b78f009d42ca911026ebebd4e91c/users/a343db0514324dd1aeb08368a3a711ef/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:07:21.326589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0be6ab-4169-4fde-960e-13375bb637ad admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:21.337149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0be6ab-4169-4fde-960e-13375bb637ad admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:21.393315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2147c1-65dd-4568-b880-f909196cd0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:21.393957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2147c1-65dd-4568-b880-f909196cd0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:21.394227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2147c1-65dd-4568-b880-f909196cd0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:21.394583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2147c1-65dd-4568-b880-f909196cd0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:21.395112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2147c1-65dd-4568-b880-f909196cd0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:21.396904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2147c1-65dd-4568-b880-f909196cd0c1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:21.407304 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1973/3909] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:21.412117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c5559-afa5-40af-b52c-62b7262fe6ad admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:21.423349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c5559-afa5-40af-b52c-62b7262fe6ad admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:21.428406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9433f0b-de13-4de4-83b6-674597aa60bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:21.429077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9433f0b-de13-4de4-83b6-674597aa60bc admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:21.429349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9433f0b-de13-4de4-83b6-674597aa60bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:21.429742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9433f0b-de13-4de4-83b6-674597aa60bc admin admin] PATH_INFO: `/v3/projects/7c67b78f009d42ca911026ebebd4e91c/users/a343db0514324dd1aeb08368a3a711ef/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:21.431782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9433f0b-de13-4de4-83b6-674597aa60bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c67b78f009d42ca911026ebebd4e91c, user_id=a343db0514324dd1aeb08368a3a711ef, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:21.434442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9433f0b-de13-4de4-83b6-674597aa60bc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:21.445884 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1937/3910] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:21 2022] PUT /identity/v3/projects/7c67b78f009d42ca911026ebebd4e91c/users/a343db0514324dd1aeb08368a3a711ef/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:22.247945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2f019-68b8-42f1-8f48-39625ed49d68 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:22.248550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2f019-68b8-42f1-8f48-39625ed49d68 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:22.248550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2f019-68b8-42f1-8f48-39625ed49d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:22.279227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-abd2f019-68b8-42f1-8f48-39625ed49d68 None None] MFA Rules not processed for user `a343db0514324dd1aeb08368a3a711ef`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:22.280612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-abd2f019-68b8-42f1-8f48-39625ed49d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:22.355607 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1974/3911] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:22 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:23.605007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9433f0b-de13-4de4-83b6-674597aa60bc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:23.671607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9433f0b-de13-4de4-83b6-674597aa60bc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:23.676678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:23.677367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:23.677635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:23.677893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] PATH_INFO: `/v3/users/01ab827376f9449c9d69a91c693da4ad` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:23.679050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01ab827376f9449c9d69a91c693da4ad)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:23.681005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:23.741538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01ab827376f9449c9d69a91c693da4ad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:23.756525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01ab827376f9449c9d69a91c693da4ad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:23.778897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01ab827376f9449c9d69a91c693da4ad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:23.779728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01ab827376f9449c9d69a91c693da4ad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:23.780649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01ab827376f9449c9d69a91c693da4ad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:23.793973 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1938/3912] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:23 2022] DELETE /identity/v3/users/01ab827376f9449c9d69a91c693da4ad => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:23.990860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2f019-68b8-42f1-8f48-39625ed49d68 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:24.003027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2f019-68b8-42f1-8f48-39625ed49d68 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:24.033152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:24.065309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:24.065980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:24.066254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:24.066518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] PATH_INFO: `/v3/projects/f5f43b60b06c412ba90f7d296e266a1f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:24.067680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f43b60b06c412ba90f7d296e266a1f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:24.070264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:24.124256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f43b60b06c412ba90f7d296e266a1f', 'request_id': 'req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.137444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f43b60b06c412ba90f7d296e266a1f', 'request_id': 'req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.144206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f43b60b06c412ba90f7d296e266a1f', 'request_id': 'req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.156344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] The token cache is being invalidate because project f5f43b60b06c412ba90f7d296e266a1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:24.156841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.157523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.159070 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1975/3913] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:23 2022] DELETE /identity/v3/projects/f5f43b60b06c412ba90f7d296e266a1f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:24.163303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:24.165700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:24.180635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58415f8-39a2-4689-b606-5357930f74b9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:24.207583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:24.258628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911c455-c15b-4767-83cc-8b7d70a4795f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:24.259438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911c455-c15b-4767-83cc-8b7d70a4795f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:24.259809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911c455-c15b-4767-83cc-8b7d70a4795f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:24.260142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911c455-c15b-4767-83cc-8b7d70a4795f admin admin] PATH_INFO: `/v3/users/a343db0514324dd1aeb08368a3a711ef` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:24.261465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911c455-c15b-4767-83cc-8b7d70a4795f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a343db0514324dd1aeb08368a3a711ef)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:24.263580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911c455-c15b-4767-83cc-8b7d70a4795f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:24.315873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5911c455-c15b-4767-83cc-8b7d70a4795f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a343db0514324dd1aeb08368a3a711ef'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.325901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5911c455-c15b-4767-83cc-8b7d70a4795f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a343db0514324dd1aeb08368a3a711ef'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.340655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5911c455-c15b-4767-83cc-8b7d70a4795f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a343db0514324dd1aeb08368a3a711ef'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.341781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5911c455-c15b-4767-83cc-8b7d70a4795f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a343db0514324dd1aeb08368a3a711ef'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.342544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5911c455-c15b-4767-83cc-8b7d70a4795f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a343db0514324dd1aeb08368a3a711ef'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.356091 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1939/3914] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:24 2022] DELETE /identity/v3/users/a343db0514324dd1aeb08368a3a711ef => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:24.541995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:24.543527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:24.558298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3902a-dd0f-4aa2-ae9a-1ba2ea86ed87 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:24.615961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:24.616656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:24.616926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:24.617187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4 admin admin] PATH_INFO: `/v3/projects/7c67b78f009d42ca911026ebebd4e91c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:24.618317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c67b78f009d42ca911026ebebd4e91c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:24.620241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:24.672169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c67b78f009d42ca911026ebebd4e91c', 'request_id': 'req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.680784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c67b78f009d42ca911026ebebd4e91c', 'request_id': 'req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.684680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:24.687243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c67b78f009d42ca911026ebebd4e91c', 'request_id': 'req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.700045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4 admin admin] The token cache is being invalidate because project 7c67b78f009d42ca911026ebebd4e91c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:24.700339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.700962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a85a8b7b-ff5c-4de9-aa15-ec9cf0f567d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:24.702455 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1976/3915] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:24 2022] DELETE /identity/v3/projects/7c67b78f009d42ca911026ebebd4e91c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:24.707513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3804e56-1feb-4dfc-bdf6-5335ed902ba3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:24.707991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3804e56-1feb-4dfc-bdf6-5335ed902ba3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:24.708109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3804e56-1feb-4dfc-bdf6-5335ed902ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:24.728046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a3804e56-1feb-4dfc-bdf6-5335ed902ba3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:24.729268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a3804e56-1feb-4dfc-bdf6-5335ed902ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:24.801232 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1940/3916] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:24.809074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53b2dd-60cd-4337-aec3-4d1cbcbeb3cf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:24.809376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53b2dd-60cd-4337-aec3-4d1cbcbeb3cf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:24.809618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53b2dd-60cd-4337-aec3-4d1cbcbeb3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:24.830903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8c53b2dd-60cd-4337-aec3-4d1cbcbeb3cf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:24.832151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8c53b2dd-60cd-4337-aec3-4d1cbcbeb3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:24.841059 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1977/3917] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:24.845578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3804e56-1feb-4dfc-bdf6-5335ed902ba3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:24.856161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3804e56-1feb-4dfc-bdf6-5335ed902ba3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:24.861773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1d851-1216-421c-9a83-d326a57dcf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:24.862381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1d851-1216-421c-9a83-d326a57dcf31 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:24.862796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1d851-1216-421c-9a83-d326a57dcf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:24.863065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1d851-1216-421c-9a83-d326a57dcf31 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:24.863710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1d851-1216-421c-9a83-d326a57dcf31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:07:24.864006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1d851-1216-421c-9a83-d326a57dcf31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:24.865722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1d851-1216-421c-9a83-d326a57dcf31 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:24.878093 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1941/3918] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:07:24.882589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53b2dd-60cd-4337-aec3-4d1cbcbeb3cf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:24.892466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53b2dd-60cd-4337-aec3-4d1cbcbeb3cf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:24.897342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb885b0c-a31e-4f35-ad9b-c9d634e18e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:24.897984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb885b0c-a31e-4f35-ad9b-c9d634e18e1b admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:24.898254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb885b0c-a31e-4f35-ad9b-c9d634e18e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:24.898519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb885b0c-a31e-4f35-ad9b-c9d634e18e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:24.899136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb885b0c-a31e-4f35-ad9b-c9d634e18e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:24.901134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb885b0c-a31e-4f35-ad9b-c9d634e18e1b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:24.934292 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1978/3919] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:24 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:24.938571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1d851-1216-421c-9a83-d326a57dcf31 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:24.949493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1d851-1216-421c-9a83-d326a57dcf31 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:24.976282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:25.010029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f11b9e-0e44-463b-ba62-84732e2120a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:25.010638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f11b9e-0e44-463b-ba62-84732e2120a3 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:25.010908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f11b9e-0e44-463b-ba62-84732e2120a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:25.011168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f11b9e-0e44-463b-ba62-84732e2120a3 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:25.011777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f11b9e-0e44-463b-ba62-84732e2120a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:25.013699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f11b9e-0e44-463b-ba62-84732e2120a3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:25.036076 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1942/3920] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:24 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:25.040340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb885b0c-a31e-4f35-ad9b-c9d634e18e1b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:25.050960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb885b0c-a31e-4f35-ad9b-c9d634e18e1b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:25.061305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bef93-d8ba-4ad5-bb73-b39f60f29743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:25.061927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bef93-d8ba-4ad5-bb73-b39f60f29743 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:25.062198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bef93-d8ba-4ad5-bb73-b39f60f29743 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:25.062458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bef93-d8ba-4ad5-bb73-b39f60f29743 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:25.063097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9bef93-d8ba-4ad5-bb73-b39f60f29743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:25.064721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9bef93-d8ba-4ad5-bb73-b39f60f29743 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:25.078267 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1979/3921] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:07:25.080897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f11b9e-0e44-463b-ba62-84732e2120a3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:25.091783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f11b9e-0e44-463b-ba62-84732e2120a3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:25.096944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0b658-f5d8-41cc-bc84-26e5693fbabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:25.097549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0b658-f5d8-41cc-bc84-26e5693fbabd admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:25.097823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0b658-f5d8-41cc-bc84-26e5693fbabd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:25.098090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0b658-f5d8-41cc-bc84-26e5693fbabd admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:25.098743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0b658-f5d8-41cc-bc84-26e5693fbabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:25.100576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0b658-f5d8-41cc-bc84-26e5693fbabd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:25.111362 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1943/3922] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:25.115816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bef93-d8ba-4ad5-bb73-b39f60f29743 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:25.126428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bef93-d8ba-4ad5-bb73-b39f60f29743 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:25.131696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b1f3d-1564-4047-a6c1-2d1441a80375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:25.132333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b1f3d-1564-4047-a6c1-2d1441a80375 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:25.132640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b1f3d-1564-4047-a6c1-2d1441a80375 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:25.132924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b1f3d-1564-4047-a6c1-2d1441a80375 admin admin] PATH_INFO: `/v3/projects/c0dc8127254d4b65a28152ce37fe3c77/users/3b65d54cec8d4831b4fe7c6b00de7e84/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:25.148407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b1f3d-1564-4047-a6c1-2d1441a80375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0dc8127254d4b65a28152ce37fe3c77, user_id=3b65d54cec8d4831b4fe7c6b00de7e84, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:25.151065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b1f3d-1564-4047-a6c1-2d1441a80375 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:25.166411 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1980/3923] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:25 2022] PUT /identity/v3/projects/c0dc8127254d4b65a28152ce37fe3c77/users/3b65d54cec8d4831b4fe7c6b00de7e84/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:07:25.176646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50a63b-5116-4c8d-8f0a-3aba64b796bb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:25.177085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50a63b-5116-4c8d-8f0a-3aba64b796bb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:25.177584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50a63b-5116-4c8d-8f0a-3aba64b796bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:25.207957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2e50a63b-5116-4c8d-8f0a-3aba64b796bb None None] MFA Rules not processed for user `3b65d54cec8d4831b4fe7c6b00de7e84`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:25.209327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2e50a63b-5116-4c8d-8f0a-3aba64b796bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:25.276700 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1944/3924] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:25.281776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b1f3d-1564-4047-a6c1-2d1441a80375 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:25.351892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b1f3d-1564-4047-a6c1-2d1441a80375 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:25.356745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0cbb3-5aef-462a-a6eb-c5f12468a162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:25.357338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0cbb3-5aef-462a-a6eb-c5f12468a162 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:25.357605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0cbb3-5aef-462a-a6eb-c5f12468a162 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:25.357867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0cbb3-5aef-462a-a6eb-c5f12468a162 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:25.358516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0cbb3-5aef-462a-a6eb-c5f12468a162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:25.360415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0cbb3-5aef-462a-a6eb-c5f12468a162 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:25.390214 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1981/3925] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:25 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:25.394459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50a63b-5116-4c8d-8f0a-3aba64b796bb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:25.406424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50a63b-5116-4c8d-8f0a-3aba64b796bb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:25.467484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5955d-d3c0-4b99-b28a-9ab4e52673c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:25.468304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5955d-d3c0-4b99-b28a-9ab4e52673c1 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:25.468804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5955d-d3c0-4b99-b28a-9ab4e52673c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:25.469214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5955d-d3c0-4b99-b28a-9ab4e52673c1 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:25.470130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5955d-d3c0-4b99-b28a-9ab4e52673c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:25.473251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5955d-d3c0-4b99-b28a-9ab4e52673c1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:25.495543 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1945/3926] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:25 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:07:25.498977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0cbb3-5aef-462a-a6eb-c5f12468a162 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:25.510398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0cbb3-5aef-462a-a6eb-c5f12468a162 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:25.515423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491331ab-736a-4c8b-b9e2-db7fb51e2af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:25.516019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491331ab-736a-4c8b-b9e2-db7fb51e2af7 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:25.516288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491331ab-736a-4c8b-b9e2-db7fb51e2af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:25.516592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491331ab-736a-4c8b-b9e2-db7fb51e2af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:25.517222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491331ab-736a-4c8b-b9e2-db7fb51e2af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:25.518819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491331ab-736a-4c8b-b9e2-db7fb51e2af7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:25.532741 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1982/3927] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:25.538775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5955d-d3c0-4b99-b28a-9ab4e52673c1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:25.552850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5955d-d3c0-4b99-b28a-9ab4e52673c1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:25.566140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01593c4d-547e-4c13-bc1d-df5b0cbf6c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:25.566807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01593c4d-547e-4c13-bc1d-df5b0cbf6c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:25.567113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01593c4d-547e-4c13-bc1d-df5b0cbf6c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:25.567361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01593c4d-547e-4c13-bc1d-df5b0cbf6c15 admin admin] PATH_INFO: `/v3/projects/327fbe3955f74b1e996f5629944465ef/users/a58f473ce0ff4ed9b4ec45ce5b800c0b/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:25.584197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01593c4d-547e-4c13-bc1d-df5b0cbf6c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=327fbe3955f74b1e996f5629944465ef, user_id=a58f473ce0ff4ed9b4ec45ce5b800c0b, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:25.587028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01593c4d-547e-4c13-bc1d-df5b0cbf6c15 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:25.597935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:25.603598 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1946/3928] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:25 2022] PUT /identity/v3/projects/327fbe3955f74b1e996f5629944465ef/users/a58f473ce0ff4ed9b4ec45ce5b800c0b/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:25.607463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491331ab-736a-4c8b-b9e2-db7fb51e2af7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:25.621725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491331ab-736a-4c8b-b9e2-db7fb51e2af7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:25.679768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2dfd6-ed3b-4373-b4e1-70a8e8ccaccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:25.680400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2dfd6-ed3b-4373-b4e1-70a8e8ccaccb admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:25.680679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2dfd6-ed3b-4373-b4e1-70a8e8ccaccb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:25.680948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2dfd6-ed3b-4373-b4e1-70a8e8ccaccb admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:25.681637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2dfd6-ed3b-4373-b4e1-70a8e8ccaccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:25.683211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2dfd6-ed3b-4373-b4e1-70a8e8ccaccb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:25.684633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:25.695682 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1983/3929] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:25.699982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01593c4d-547e-4c13-bc1d-df5b0cbf6c15 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:25.711047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01593c4d-547e-4c13-bc1d-df5b0cbf6c15 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:25.715861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6039f6d-dabb-43b6-9982-b8734cf7849a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:25.716582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6039f6d-dabb-43b6-9982-b8734cf7849a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:25.716853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6039f6d-dabb-43b6-9982-b8734cf7849a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:25.717134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6039f6d-dabb-43b6-9982-b8734cf7849a admin admin] PATH_INFO: `/v3/projects/327fbe3955f74b1e996f5629944465ef/users/a58f473ce0ff4ed9b4ec45ce5b800c0b/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:25.719037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6039f6d-dabb-43b6-9982-b8734cf7849a admin admin] RBAC: Authorizing `identity:create_grant(project_id=327fbe3955f74b1e996f5629944465ef, user_id=a58f473ce0ff4ed9b4ec45ce5b800c0b, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:25.721722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6039f6d-dabb-43b6-9982-b8734cf7849a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:25.732315 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1947/3930] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:25 2022] PUT /identity/v3/projects/327fbe3955f74b1e996f5629944465ef/users/a58f473ce0ff4ed9b4ec45ce5b800c0b/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:25.744354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b9f45-0109-4058-8d6d-a263f00d1651 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:25.744651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b9f45-0109-4058-8d6d-a263f00d1651 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:25.744917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b9f45-0109-4058-8d6d-a263f00d1651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:25.773247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-407b9f45-0109-4058-8d6d-a263f00d1651 None None] MFA Rules not processed for user `a58f473ce0ff4ed9b4ec45ce5b800c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:25.774444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-407b9f45-0109-4058-8d6d-a263f00d1651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:25.843303 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1984/3931] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:25.883424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6039f6d-dabb-43b6-9982-b8734cf7849a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:25.885452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d6039f6d-dabb-43b6-9982-b8734cf7849a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:25.953010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6039f6d-dabb-43b6-9982-b8734cf7849a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:25.958606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957b956-ce69-4842-92ea-cb49e1799a41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:25.959252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957b956-ce69-4842-92ea-cb49e1799a41 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:25.959523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957b956-ce69-4842-92ea-cb49e1799a41 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:25.959784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957b956-ce69-4842-92ea-cb49e1799a41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:25.969896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957b956-ce69-4842-92ea-cb49e1799a41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:25.971369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957b956-ce69-4842-92ea-cb49e1799a41 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:25.981261 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1948/3932] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:26.128261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b9f45-0109-4058-8d6d-a263f00d1651 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:26.145215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b9f45-0109-4058-8d6d-a263f00d1651 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:26.149087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcdaf3d-9d92-4679-8fcf-c6f9a8858d5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:26.149943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdaf3d-9d92-4679-8fcf-c6f9a8858d5a service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:26.150276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdaf3d-9d92-4679-8fcf-c6f9a8858d5a service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:26.150611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdaf3d-9d92-4679-8fcf-c6f9a8858d5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:26.213051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcdaf3d-9d92-4679-8fcf-c6f9a8858d5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:26.215573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcdaf3d-9d92-4679-8fcf-c6f9a8858d5a service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:26.226241 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1985/3933] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:26.641252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ac10-ba17-465e-98c7-0bcd51b61b30 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:26.641531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ac10-ba17-465e-98c7-0bcd51b61b30 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:26.641812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ac10-ba17-465e-98c7-0bcd51b61b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:26.662184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d6b1ac10-ba17-465e-98c7-0bcd51b61b30 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:26.663565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d6b1ac10-ba17-465e-98c7-0bcd51b61b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:26.723496 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1949/3934] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:07:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:26.736091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcdaf3d-9d92-4679-8fcf-c6f9a8858d5a service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:26.737752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9fcdaf3d-9d92-4679-8fcf-c6f9a8858d5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:26.816426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcdaf3d-9d92-4679-8fcf-c6f9a8858d5a service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:26.822261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237f661-42f6-4c7f-b7b5-61e63d9b686a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:26.822879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237f661-42f6-4c7f-b7b5-61e63d9b686a service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:26.823152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237f661-42f6-4c7f-b7b5-61e63d9b686a service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:26.823420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237f661-42f6-4c7f-b7b5-61e63d9b686a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:26.832001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237f661-42f6-4c7f-b7b5-61e63d9b686a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:26.833946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237f661-42f6-4c7f-b7b5-61e63d9b686a service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:26.842973 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1986/3935] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:07:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:07:27.993229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1ac10-ba17-465e-98c7-0bcd51b61b30 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:28.091423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1ac10-ba17-465e-98c7-0bcd51b61b30 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:28.097066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:28.097731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:28.098008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:28.098273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] PATH_INFO: `/v3/users/3b65d54cec8d4831b4fe7c6b00de7e84` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:28.099840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b65d54cec8d4831b4fe7c6b00de7e84)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:28.101993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:28.161002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:28.182014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b65d54cec8d4831b4fe7c6b00de7e84'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:28.193078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b65d54cec8d4831b4fe7c6b00de7e84'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:28.208850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b65d54cec8d4831b4fe7c6b00de7e84'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:28.209685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b65d54cec8d4831b4fe7c6b00de7e84'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:28.210371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b65d54cec8d4831b4fe7c6b00de7e84'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:28.246955 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1950/3936] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:27 2022] DELETE /identity/v3/users/3b65d54cec8d4831b4fe7c6b00de7e84 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:28.248749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7833b4-9231-4bbe-9722-e58f08b91837 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:28.249151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7833b4-9231-4bbe-9722-e58f08b91837 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:28.249234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7833b4-9231-4bbe-9722-e58f08b91837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:28.283528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bf7833b4-9231-4bbe-9722-e58f08b91837 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:28.284858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bf7833b4-9231-4bbe-9722-e58f08b91837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:28.289282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:28.365644 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1987/3937] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:28.377158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:28.378573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:28.450694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913f853-9773-4937-81c0-051a2264a3be admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:28.454703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bd917-e56d-45c6-aac3-9617a3502a91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:28.455375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bd917-e56d-45c6-aac3-9617a3502a91 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:28.455611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bd917-e56d-45c6-aac3-9617a3502a91 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:28.455866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bd917-e56d-45c6-aac3-9617a3502a91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:28.475035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bd917-e56d-45c6-aac3-9617a3502a91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:28.476519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bd917-e56d-45c6-aac3-9617a3502a91 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:28.495966 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1951/3938] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:28.934613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7833b4-9231-4bbe-9722-e58f08b91837 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:28.938576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bf7833b4-9231-4bbe-9722-e58f08b91837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:28.955660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7833b4-9231-4bbe-9722-e58f08b91837 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:28.964537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:28.965116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:28.965385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:28.965646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] PATH_INFO: `/v3/projects/c0dc8127254d4b65a28152ce37fe3c77` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:28.966775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0dc8127254d4b65a28152ce37fe3c77)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:28.970064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:29.028487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0dc8127254d4b65a28152ce37fe3c77', 'request_id': 'req-a5974b38-a116-4027-af50-dfd7b9090fda'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:29.038705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0dc8127254d4b65a28152ce37fe3c77', 'request_id': 'req-a5974b38-a116-4027-af50-dfd7b9090fda'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:29.045922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0dc8127254d4b65a28152ce37fe3c77', 'request_id': 'req-a5974b38-a116-4027-af50-dfd7b9090fda'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:29.060541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] The token cache is being invalidate because project c0dc8127254d4b65a28152ce37fe3c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:29.061320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:29.062194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:29.063917 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1988/3939] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:28 2022] DELETE /identity/v3/projects/c0dc8127254d4b65a28152ce37fe3c77 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:29.068626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bd917-e56d-45c6-aac3-9617a3502a91 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:29.080566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bd917-e56d-45c6-aac3-9617a3502a91 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:29.143506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04b6a2-1cdf-4df8-a067-86b3ab836010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:29.144054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04b6a2-1cdf-4df8-a067-86b3ab836010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:29.144343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04b6a2-1cdf-4df8-a067-86b3ab836010 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:29.144615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04b6a2-1cdf-4df8-a067-86b3ab836010 admin admin] PATH_INFO: `/v3/users/a58f473ce0ff4ed9b4ec45ce5b800c0b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:29.145923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04b6a2-1cdf-4df8-a067-86b3ab836010 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a58f473ce0ff4ed9b4ec45ce5b800c0b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:29.147858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04b6a2-1cdf-4df8-a067-86b3ab836010 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:29.204017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6f04b6a2-1cdf-4df8-a067-86b3ab836010 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a58f473ce0ff4ed9b4ec45ce5b800c0b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:29.213563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6f04b6a2-1cdf-4df8-a067-86b3ab836010 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a58f473ce0ff4ed9b4ec45ce5b800c0b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:29.219759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:29.235806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6f04b6a2-1cdf-4df8-a067-86b3ab836010 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a58f473ce0ff4ed9b4ec45ce5b800c0b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:29.236977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6f04b6a2-1cdf-4df8-a067-86b3ab836010 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a58f473ce0ff4ed9b4ec45ce5b800c0b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:29.238100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6f04b6a2-1cdf-4df8-a067-86b3ab836010 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a58f473ce0ff4ed9b4ec45ce5b800c0b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:29.267419 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1952/3940] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:29 2022] DELETE /identity/v3/users/a58f473ce0ff4ed9b4ec45ce5b800c0b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:29.953646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:29.955147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:29.975053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5974b38-a116-4027-af50-dfd7b9090fda admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:30.105346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51622d65-054b-4341-8f77-605888a27084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:30.106388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51622d65-054b-4341-8f77-605888a27084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:30.106667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51622d65-054b-4341-8f77-605888a27084 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:30.107080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51622d65-054b-4341-8f77-605888a27084 admin admin] PATH_INFO: `/v3/projects/327fbe3955f74b1e996f5629944465ef` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:30.108851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51622d65-054b-4341-8f77-605888a27084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=327fbe3955f74b1e996f5629944465ef)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:30.111143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51622d65-054b-4341-8f77-605888a27084 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:30.156917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:30.169755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-51622d65-054b-4341-8f77-605888a27084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '327fbe3955f74b1e996f5629944465ef', 'request_id': 'req-51622d65-054b-4341-8f77-605888a27084'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:30.179677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-51622d65-054b-4341-8f77-605888a27084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '327fbe3955f74b1e996f5629944465ef', 'request_id': 'req-51622d65-054b-4341-8f77-605888a27084'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:30.185833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-51622d65-054b-4341-8f77-605888a27084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '327fbe3955f74b1e996f5629944465ef', 'request_id': 'req-51622d65-054b-4341-8f77-605888a27084'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:30.198589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-51622d65-054b-4341-8f77-605888a27084 admin admin] The token cache is being invalidate because project 327fbe3955f74b1e996f5629944465ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:30.198977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-51622d65-054b-4341-8f77-605888a27084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:30.199793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-51622d65-054b-4341-8f77-605888a27084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:30.201767 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1989/3941] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:29 2022] DELETE /identity/v3/projects/327fbe3955f74b1e996f5629944465ef => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:30.207726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae6e6e-27fb-4e5f-b4cd-0db656e8f977 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:30.208357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae6e6e-27fb-4e5f-b4cd-0db656e8f977 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:30.208357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae6e6e-27fb-4e5f-b4cd-0db656e8f977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:30.229607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-53ae6e6e-27fb-4e5f-b4cd-0db656e8f977 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:30.231122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-53ae6e6e-27fb-4e5f-b4cd-0db656e8f977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:30.298427 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1953/3942] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:07:30.308672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9962155-962f-4c74-b48d-fdfda15f2a32 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:30.308988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9962155-962f-4c74-b48d-fdfda15f2a32 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:30.309348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9962155-962f-4c74-b48d-fdfda15f2a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:30.330363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e9962155-962f-4c74-b48d-fdfda15f2a32 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:30.331828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e9962155-962f-4c74-b48d-fdfda15f2a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:30.340941 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1990/3943] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:30.345127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae6e6e-27fb-4e5f-b4cd-0db656e8f977 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:30.355634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae6e6e-27fb-4e5f-b4cd-0db656e8f977 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:30.360391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d487b73-8854-40cb-942e-8a99c7daec5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:30.361008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d487b73-8854-40cb-942e-8a99c7daec5d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:30.361290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d487b73-8854-40cb-942e-8a99c7daec5d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:30.361548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d487b73-8854-40cb-942e-8a99c7daec5d admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:30.362222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d487b73-8854-40cb-942e-8a99c7daec5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:07:30.362521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d487b73-8854-40cb-942e-8a99c7daec5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:30.365345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d487b73-8854-40cb-942e-8a99c7daec5d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:30.376481 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1954/3944] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:07:30.381239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9962155-962f-4c74-b48d-fdfda15f2a32 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:30.391234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9962155-962f-4c74-b48d-fdfda15f2a32 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:30.396616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5435aa3-a46a-4117-b4e8-2c8d38811f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:30.397225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5435aa3-a46a-4117-b4e8-2c8d38811f64 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:30.397495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5435aa3-a46a-4117-b4e8-2c8d38811f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:30.397796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5435aa3-a46a-4117-b4e8-2c8d38811f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:30.398439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5435aa3-a46a-4117-b4e8-2c8d38811f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:30.400434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5435aa3-a46a-4117-b4e8-2c8d38811f64 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:30.434582 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1991/3945] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:30 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:30.435775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d487b73-8854-40cb-942e-8a99c7daec5d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:30.446669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d487b73-8854-40cb-942e-8a99c7daec5d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:30.510420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f391c3-d6a6-4db0-8acf-351d5ba61b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:30.510979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f391c3-d6a6-4db0-8acf-351d5ba61b22 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:30.511252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f391c3-d6a6-4db0-8acf-351d5ba61b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:30.511520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f391c3-d6a6-4db0-8acf-351d5ba61b22 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:30.512156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f391c3-d6a6-4db0-8acf-351d5ba61b22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:30.514227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f391c3-d6a6-4db0-8acf-351d5ba61b22 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:30.539279 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1955/3946] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:30 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:07:30.543718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5435aa3-a46a-4117-b4e8-2c8d38811f64 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:30.557277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5435aa3-a46a-4117-b4e8-2c8d38811f64 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:30.565451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a97869-4340-4ddb-8a36-32ebb259cdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:30.566080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a97869-4340-4ddb-8a36-32ebb259cdc1 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:30.566352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a97869-4340-4ddb-8a36-32ebb259cdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:30.566634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a97869-4340-4ddb-8a36-32ebb259cdc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:30.567315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a97869-4340-4ddb-8a36-32ebb259cdc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:30.570295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a97869-4340-4ddb-8a36-32ebb259cdc1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:30.586585 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1992/3947] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:07:30.590660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f391c3-d6a6-4db0-8acf-351d5ba61b22 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:30.604313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f391c3-d6a6-4db0-8acf-351d5ba61b22 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:30.609220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624603a-6e1e-4d6a-ba4b-646b1a968872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:30.609826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624603a-6e1e-4d6a-ba4b-646b1a968872 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:30.610091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624603a-6e1e-4d6a-ba4b-646b1a968872 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:30.610348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624603a-6e1e-4d6a-ba4b-646b1a968872 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:30.610984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624603a-6e1e-4d6a-ba4b-646b1a968872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:30.612641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624603a-6e1e-4d6a-ba4b-646b1a968872 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:30.625881 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1956/3948] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:30.630788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a97869-4340-4ddb-8a36-32ebb259cdc1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:30.652639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a97869-4340-4ddb-8a36-32ebb259cdc1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:30.659264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975ac79-4b5b-4760-883f-ba4ab23f42df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:30.659969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975ac79-4b5b-4760-883f-ba4ab23f42df admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:30.660358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975ac79-4b5b-4760-883f-ba4ab23f42df admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:30.660665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975ac79-4b5b-4760-883f-ba4ab23f42df admin admin] PATH_INFO: `/v3/projects/d7403064d209452aad48dc9e505f15d0/users/1f99325a125c486492af87a7342cb736/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:30.678591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975ac79-4b5b-4760-883f-ba4ab23f42df admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7403064d209452aad48dc9e505f15d0, user_id=1f99325a125c486492af87a7342cb736, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:30.681269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975ac79-4b5b-4760-883f-ba4ab23f42df admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:30.700365 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1993/3949] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:30 2022] PUT /identity/v3/projects/d7403064d209452aad48dc9e505f15d0/users/1f99325a125c486492af87a7342cb736/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:07:30.711163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5fd2a-c4b4-4d7b-b67d-b77d66625d9a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:30.711425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5fd2a-c4b4-4d7b-b67d-b77d66625d9a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:30.711706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5fd2a-c4b4-4d7b-b67d-b77d66625d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:30.759645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-83c5fd2a-c4b4-4d7b-b67d-b77d66625d9a None None] MFA Rules not processed for user `1f99325a125c486492af87a7342cb736`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:30.761991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-83c5fd2a-c4b4-4d7b-b67d-b77d66625d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:30.807569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:30.878925 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1957/3950] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:30.884077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975ac79-4b5b-4760-883f-ba4ab23f42df admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:30.958498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975ac79-4b5b-4760-883f-ba4ab23f42df admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:30.963617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f2319-8910-432a-964e-66e68d038ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:30.964232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f2319-8910-432a-964e-66e68d038ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:30.964847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f2319-8910-432a-964e-66e68d038ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:30.965122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f2319-8910-432a-964e-66e68d038ff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:30.965742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f2319-8910-432a-964e-66e68d038ff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:30.967846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f2319-8910-432a-964e-66e68d038ff0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:31.001099 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1994/3951] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:30 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:31.002345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5fd2a-c4b4-4d7b-b67d-b77d66625d9a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:31.018094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5fd2a-c4b4-4d7b-b67d-b77d66625d9a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:31.089830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd5eb8-c331-4cfa-9964-9325b4854baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:31.090499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd5eb8-c331-4cfa-9964-9325b4854baf admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:31.090772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd5eb8-c331-4cfa-9964-9325b4854baf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:31.091036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd5eb8-c331-4cfa-9964-9325b4854baf admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:31.091662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd5eb8-c331-4cfa-9964-9325b4854baf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:31.093828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd5eb8-c331-4cfa-9964-9325b4854baf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:31.162253 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1958/3952] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:31 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:07:31.163972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f2319-8910-432a-964e-66e68d038ff0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:31.181670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f2319-8910-432a-964e-66e68d038ff0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:31.192603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63033b-7cc8-4fb6-bd4a-b908f599a095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:31.193782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63033b-7cc8-4fb6-bd4a-b908f599a095 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:31.194276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63033b-7cc8-4fb6-bd4a-b908f599a095 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:31.194810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63033b-7cc8-4fb6-bd4a-b908f599a095 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:31.196000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63033b-7cc8-4fb6-bd4a-b908f599a095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:31.197809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63033b-7cc8-4fb6-bd4a-b908f599a095 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:31.214666 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1995/3953] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:31.215529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd5eb8-c331-4cfa-9964-9325b4854baf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:31.229266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd5eb8-c331-4cfa-9964-9325b4854baf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:31.249745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976eefe-a7e6-4436-b122-337a06d9a652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:31.250745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976eefe-a7e6-4436-b122-337a06d9a652 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:31.251025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976eefe-a7e6-4436-b122-337a06d9a652 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:31.251303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976eefe-a7e6-4436-b122-337a06d9a652 admin admin] PATH_INFO: `/v3/projects/4c3650f9161b46ff9823ad3f77c57399/users/f17e4060ff9f4662a14650496453be5c/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:31.289206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976eefe-a7e6-4436-b122-337a06d9a652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c3650f9161b46ff9823ad3f77c57399, user_id=f17e4060ff9f4662a14650496453be5c, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:31.291960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976eefe-a7e6-4436-b122-337a06d9a652 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:31.311732 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1959/3954] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:31 2022] PUT /identity/v3/projects/4c3650f9161b46ff9823ad3f77c57399/users/f17e4060ff9f4662a14650496453be5c/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:07:31.315784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63033b-7cc8-4fb6-bd4a-b908f599a095 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:31.332714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63033b-7cc8-4fb6-bd4a-b908f599a095 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:31.382590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:31.424970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19defb6b-a718-4829-aa1e-104127b85ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:31.426213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19defb6b-a718-4829-aa1e-104127b85ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:31.426700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19defb6b-a718-4829-aa1e-104127b85ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:31.427215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19defb6b-a718-4829-aa1e-104127b85ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:31.428469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19defb6b-a718-4829-aa1e-104127b85ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:31.431130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19defb6b-a718-4829-aa1e-104127b85ff9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:31.445161 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1996/3955] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:31.449830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976eefe-a7e6-4436-b122-337a06d9a652 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:31.466659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976eefe-a7e6-4436-b122-337a06d9a652 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:31.473209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7836bed-d081-4170-90ab-2c06f32feafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:31.474955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7836bed-d081-4170-90ab-2c06f32feafd admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:31.475328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7836bed-d081-4170-90ab-2c06f32feafd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:31.475667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7836bed-d081-4170-90ab-2c06f32feafd admin admin] PATH_INFO: `/v3/projects/4c3650f9161b46ff9823ad3f77c57399/users/f17e4060ff9f4662a14650496453be5c/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:31.478179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7836bed-d081-4170-90ab-2c06f32feafd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c3650f9161b46ff9823ad3f77c57399, user_id=f17e4060ff9f4662a14650496453be5c, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:31.481222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7836bed-d081-4170-90ab-2c06f32feafd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:31.496410 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1960/3956] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:31 2022] PUT /identity/v3/projects/4c3650f9161b46ff9823ad3f77c57399/users/f17e4060ff9f4662a14650496453be5c/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:07:31.508699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587638f-7dc3-4180-b733-ddc419e24d73 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:31.508964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587638f-7dc3-4180-b733-ddc419e24d73 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:31.509217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587638f-7dc3-4180-b733-ddc419e24d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:31.551130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7587638f-7dc3-4180-b733-ddc419e24d73 None None] MFA Rules not processed for user `f17e4060ff9f4662a14650496453be5c`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:31.552533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7587638f-7dc3-4180-b733-ddc419e24d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:31.660912 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1997/3957] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:31.673842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7836bed-d081-4170-90ab-2c06f32feafd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:31.676013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b7836bed-d081-4170-90ab-2c06f32feafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:31.781564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7836bed-d081-4170-90ab-2c06f32feafd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:31.788852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091ee0f-2ee3-4d11-b538-917b47231dc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:31.789595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091ee0f-2ee3-4d11-b538-917b47231dc2 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:31.789961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091ee0f-2ee3-4d11-b538-917b47231dc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:31.790329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091ee0f-2ee3-4d11-b538-917b47231dc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:31.801128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091ee0f-2ee3-4d11-b538-917b47231dc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:31.804241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091ee0f-2ee3-4d11-b538-917b47231dc2 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:31.830177 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1961/3958] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:31.975656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587638f-7dc3-4180-b733-ddc419e24d73 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:32.059748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587638f-7dc3-4180-b733-ddc419e24d73 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:32.064275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:32.064782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:32.065049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:32.065312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] PATH_INFO: `/v3/users/1f99325a125c486492af87a7342cb736` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:32.066561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f99325a125c486492af87a7342cb736)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:32.068520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:32.163756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f99325a125c486492af87a7342cb736'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:32.178234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f99325a125c486492af87a7342cb736'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:32.181934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:32.202314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f99325a125c486492af87a7342cb736'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:32.220119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f99325a125c486492af87a7342cb736'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:32.220972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f99325a125c486492af87a7342cb736'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:32.229931 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1998/3959] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:31 2022] DELETE /identity/v3/users/1f99325a125c486492af87a7342cb736 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:32.238471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d2504-e15f-439f-9407-778255bb35e1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:32.239052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d2504-e15f-439f-9407-778255bb35e1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:32.242129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d2504-e15f-439f-9407-778255bb35e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:32.266081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fe6d2504-e15f-439f-9407-778255bb35e1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:32.268393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fe6d2504-e15f-439f-9407-778255bb35e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:32.296996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:32.360700 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1962/3960] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:32.370539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:32.373163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:32.450006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230507cf-47d4-400c-9fb8-8007b4b0510d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:32.458932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12643b-24c7-4d1b-9692-e452c839a44a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:32.459730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12643b-24c7-4d1b-9692-e452c839a44a service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:32.460555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12643b-24c7-4d1b-9692-e452c839a44a service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:32.460954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12643b-24c7-4d1b-9692-e452c839a44a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:32.472695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12643b-24c7-4d1b-9692-e452c839a44a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:32.476344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12643b-24c7-4d1b-9692-e452c839a44a service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:32.495853 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 1999/3961] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:07:32.997911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d2504-e15f-439f-9407-778255bb35e1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:33.029673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fe6d2504-e15f-439f-9407-778255bb35e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:33.067015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d2504-e15f-439f-9407-778255bb35e1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:33.073538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:33.078021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:33.078630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:33.079258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] PATH_INFO: `/v3/projects/d7403064d209452aad48dc9e505f15d0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:33.082715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7403064d209452aad48dc9e505f15d0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:33.086917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:33.281599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7403064d209452aad48dc9e505f15d0', 'request_id': 'req-6dcce32b-d355-4bdb-80df-dd36d716a672'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:33.294087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7403064d209452aad48dc9e505f15d0', 'request_id': 'req-6dcce32b-d355-4bdb-80df-dd36d716a672'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:33.309618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7403064d209452aad48dc9e505f15d0', 'request_id': 'req-6dcce32b-d355-4bdb-80df-dd36d716a672'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:33.329891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] The token cache is being invalidate because project d7403064d209452aad48dc9e505f15d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:33.330257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:33.331341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:33.333250 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1963/3962] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:32 2022] DELETE /identity/v3/projects/d7403064d209452aad48dc9e505f15d0 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:33.337734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12643b-24c7-4d1b-9692-e452c839a44a service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:33.350423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12643b-24c7-4d1b-9692-e452c839a44a service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:33.448000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a594a-676b-44ed-8603-2da068177f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:33.448833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a594a-676b-44ed-8603-2da068177f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:33.449163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a594a-676b-44ed-8603-2da068177f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:33.449433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a594a-676b-44ed-8603-2da068177f36 admin admin] PATH_INFO: `/v3/users/f17e4060ff9f4662a14650496453be5c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:33.450620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a594a-676b-44ed-8603-2da068177f36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f17e4060ff9f4662a14650496453be5c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:33.452583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a594a-676b-44ed-8603-2da068177f36 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:33.512269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-041a594a-676b-44ed-8603-2da068177f36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f17e4060ff9f4662a14650496453be5c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:33.522772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-041a594a-676b-44ed-8603-2da068177f36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f17e4060ff9f4662a14650496453be5c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:33.531131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-041a594a-676b-44ed-8603-2da068177f36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f17e4060ff9f4662a14650496453be5c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:33.548029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:33.552125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-041a594a-676b-44ed-8603-2da068177f36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f17e4060ff9f4662a14650496453be5c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:33.554270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-041a594a-676b-44ed-8603-2da068177f36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f17e4060ff9f4662a14650496453be5c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:33.564397 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2000/3963] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:33 2022] DELETE /identity/v3/users/f17e4060ff9f4662a14650496453be5c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:34.013870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:34.017892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:34.058140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144363c5-98d4-4076-a138-5c9a0481f3ce None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:34.058920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144363c5-98d4-4076-a138-5c9a0481f3ce None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:34.059450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144363c5-98d4-4076-a138-5c9a0481f3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:34.062633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcce32b-d355-4bdb-80df-dd36d716a672 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:34.116185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-144363c5-98d4-4076-a138-5c9a0481f3ce None None] MFA Rules not processed for user `b20aea7088dc4e4fadb737c906cf7ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:34.117885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-144363c5-98d4-4076-a138-5c9a0481f3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:34.193735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:34.194968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:34.195238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:34.195962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a admin admin] PATH_INFO: `/v3/projects/4c3650f9161b46ff9823ad3f77c57399` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:34.198301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c3650f9161b46ff9823ad3f77c57399)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:34.202676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:34.211308 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2001/3964] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:34.302968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c3650f9161b46ff9823ad3f77c57399', 'request_id': 'req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:34.304505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:34.314186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c3650f9161b46ff9823ad3f77c57399', 'request_id': 'req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:34.339831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c3650f9161b46ff9823ad3f77c57399', 'request_id': 'req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:34.355664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a admin admin] The token cache is being invalidate because project 4c3650f9161b46ff9823ad3f77c57399 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:34.356069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:34.356994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9c5bd77-3915-4ac2-92f2-ab45297ac02a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:34.358935 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1964/3965] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:34 2022] DELETE /identity/v3/projects/4c3650f9161b46ff9823ad3f77c57399 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:34.363889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929491-59f8-4a51-9f95-a479a3b8fd43 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:34.364479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929491-59f8-4a51-9f95-a479a3b8fd43 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:34.364966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929491-59f8-4a51-9f95-a479a3b8fd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:34.395140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-de929491-59f8-4a51-9f95-a479a3b8fd43 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:34.397695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-de929491-59f8-4a51-9f95-a479a3b8fd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:34.492137 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2002/3966] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:34.493335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fc8dc-29a8-45d6-aaa5-ec5c08c6f39e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:34.493561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fc8dc-29a8-45d6-aaa5-ec5c08c6f39e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:34.493832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fc8dc-29a8-45d6-aaa5-ec5c08c6f39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:34.516573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-934fc8dc-29a8-45d6-aaa5-ec5c08c6f39e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:34.517977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-934fc8dc-29a8-45d6-aaa5-ec5c08c6f39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:34.528418 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1965/3967] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:07:34.532193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de929491-59f8-4a51-9f95-a479a3b8fd43 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:34.545117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de929491-59f8-4a51-9f95-a479a3b8fd43 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:34.553128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836989da-5409-481d-9b61-dbd9bf82f646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:34.554184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836989da-5409-481d-9b61-dbd9bf82f646 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:34.554455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836989da-5409-481d-9b61-dbd9bf82f646 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:34.554719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836989da-5409-481d-9b61-dbd9bf82f646 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:34.555366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836989da-5409-481d-9b61-dbd9bf82f646 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:07:34.555658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836989da-5409-481d-9b61-dbd9bf82f646 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:34.557310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836989da-5409-481d-9b61-dbd9bf82f646 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:34.571949 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2003/3968] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:07:34.577370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fc8dc-29a8-45d6-aaa5-ec5c08c6f39e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:34.593010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fc8dc-29a8-45d6-aaa5-ec5c08c6f39e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:34.598667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c006cc-9cd5-4490-bcec-b1715d72c5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:34.599720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c006cc-9cd5-4490-bcec-b1715d72c5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:34.600430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c006cc-9cd5-4490-bcec-b1715d72c5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:34.601015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c006cc-9cd5-4490-bcec-b1715d72c5f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:34.601751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c006cc-9cd5-4490-bcec-b1715d72c5f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:34.603755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c006cc-9cd5-4490-bcec-b1715d72c5f3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:34.647559 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1966/3969] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:34 2022] POST /identity/v3/projects => generated 363 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:34.649757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836989da-5409-481d-9b61-dbd9bf82f646 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:34.670959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836989da-5409-481d-9b61-dbd9bf82f646 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:34.790362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e971ec-a1b0-4543-b52f-b16bbe62bb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:34.790992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e971ec-a1b0-4543-b52f-b16bbe62bb7f admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:34.791266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e971ec-a1b0-4543-b52f-b16bbe62bb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:34.791533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e971ec-a1b0-4543-b52f-b16bbe62bb7f admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:34.792162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e971ec-a1b0-4543-b52f-b16bbe62bb7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:34.794132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e971ec-a1b0-4543-b52f-b16bbe62bb7f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:34.840288 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2004/3970] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:34 2022] POST /identity/v3/users => generated 282 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:34.844612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c006cc-9cd5-4490-bcec-b1715d72c5f3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:34.856627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c006cc-9cd5-4490-bcec-b1715d72c5f3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:34.862618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e35c9-35f9-48c2-8f96-7873a30ebffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:34.863593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e35c9-35f9-48c2-8f96-7873a30ebffb admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:34.863945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e35c9-35f9-48c2-8f96-7873a30ebffb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:34.864272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e35c9-35f9-48c2-8f96-7873a30ebffb admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:34.865136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e35c9-35f9-48c2-8f96-7873a30ebffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:34.867407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e35c9-35f9-48c2-8f96-7873a30ebffb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:34.886335 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1967/3971] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:34.890392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e971ec-a1b0-4543-b52f-b16bbe62bb7f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:34.907654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e971ec-a1b0-4543-b52f-b16bbe62bb7f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:34.919672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9151a16-d411-4efe-8fe2-9f1799741769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:34.920306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9151a16-d411-4efe-8fe2-9f1799741769 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:34.920628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9151a16-d411-4efe-8fe2-9f1799741769 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:34.920903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9151a16-d411-4efe-8fe2-9f1799741769 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:34.921551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9151a16-d411-4efe-8fe2-9f1799741769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:34.923173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9151a16-d411-4efe-8fe2-9f1799741769 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:34.942630 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2005/3972] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:34.947195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e35c9-35f9-48c2-8f96-7873a30ebffb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:34.958900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e35c9-35f9-48c2-8f96-7873a30ebffb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:34.969339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a00e57-50dd-4c26-aa51-4284bab22a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:34.970672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a00e57-50dd-4c26-aa51-4284bab22a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:34.971156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a00e57-50dd-4c26-aa51-4284bab22a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:34.971752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a00e57-50dd-4c26-aa51-4284bab22a72 admin admin] PATH_INFO: `/v3/projects/1b33df9ae22a450abacc30f1682e24c6/users/877848fa0141431caaca7a5e91a0aa01/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:35.001628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a00e57-50dd-4c26-aa51-4284bab22a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b33df9ae22a450abacc30f1682e24c6, user_id=877848fa0141431caaca7a5e91a0aa01, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:35.005319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a00e57-50dd-4c26-aa51-4284bab22a72 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:35.056496 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1968/3973] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:34 2022] PUT /identity/v3/projects/1b33df9ae22a450abacc30f1682e24c6/users/877848fa0141431caaca7a5e91a0aa01/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:07:35.065532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204230b6-d4f5-4634-9d3b-1349e4457dda None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:35.065898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204230b6-d4f5-4634-9d3b-1349e4457dda None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:35.066856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204230b6-d4f5-4634-9d3b-1349e4457dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:35.081996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:35.139450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-204230b6-d4f5-4634-9d3b-1349e4457dda None None] MFA Rules not processed for user `877848fa0141431caaca7a5e91a0aa01`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:35.143648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-204230b6-d4f5-4634-9d3b-1349e4457dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:35.278838 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2006/3974] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:07:35.283258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a00e57-50dd-4c26-aa51-4284bab22a72 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:35.456117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a00e57-50dd-4c26-aa51-4284bab22a72 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:35.476052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6403977-8cf3-49d6-bdd3-30a007b8e834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:35.476823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6403977-8cf3-49d6-bdd3-30a007b8e834 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:35.477196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6403977-8cf3-49d6-bdd3-30a007b8e834 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:35.477551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6403977-8cf3-49d6-bdd3-30a007b8e834 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:35.480535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6403977-8cf3-49d6-bdd3-30a007b8e834 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:35.482822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6403977-8cf3-49d6-bdd3-30a007b8e834 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:35.530499 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1969/3975] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:35 2022] POST /identity/v3/projects => generated 367 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:35.532534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204230b6-d4f5-4634-9d3b-1349e4457dda None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:35.557549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204230b6-d4f5-4634-9d3b-1349e4457dda None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:35.674204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970db59-d857-4cc0-bed2-0266a2f2c720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:35.675904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970db59-d857-4cc0-bed2-0266a2f2c720 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:35.676657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970db59-d857-4cc0-bed2-0266a2f2c720 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:35.677730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970db59-d857-4cc0-bed2-0266a2f2c720 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:35.680245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970db59-d857-4cc0-bed2-0266a2f2c720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:35.683964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970db59-d857-4cc0-bed2-0266a2f2c720 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:35.716425 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2007/3976] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:35 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:35.720538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6403977-8cf3-49d6-bdd3-30a007b8e834 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:35.734970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6403977-8cf3-49d6-bdd3-30a007b8e834 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:35.749400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d0f69-1c33-4146-bac1-887d39e40a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:35.750119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d0f69-1c33-4146-bac1-887d39e40a6b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:35.750423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d0f69-1c33-4146-bac1-887d39e40a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:35.750881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d0f69-1c33-4146-bac1-887d39e40a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:35.751819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d0f69-1c33-4146-bac1-887d39e40a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:35.753505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d0f69-1c33-4146-bac1-887d39e40a6b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:35.772339 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1970/3977] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:35.773675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970db59-d857-4cc0-bed2-0266a2f2c720 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:35.789268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970db59-d857-4cc0-bed2-0266a2f2c720 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:35.799392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27801d67-d16d-472b-a6a6-97508aa67688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:35.800666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27801d67-d16d-472b-a6a6-97508aa67688 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:35.800666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27801d67-d16d-472b-a6a6-97508aa67688 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:35.801020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27801d67-d16d-472b-a6a6-97508aa67688 admin admin] PATH_INFO: `/v3/projects/920f666ba8be45eb97b2cd8f623e7faf/users/44898a31c9114b61832e7b1f2a125101/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:35.821885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27801d67-d16d-472b-a6a6-97508aa67688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=920f666ba8be45eb97b2cd8f623e7faf, user_id=44898a31c9114b61832e7b1f2a125101, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:35.824801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27801d67-d16d-472b-a6a6-97508aa67688 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:35.853682 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2008/3978] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:35 2022] PUT /identity/v3/projects/920f666ba8be45eb97b2cd8f623e7faf/users/44898a31c9114b61832e7b1f2a125101/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:35.858176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d0f69-1c33-4146-bac1-887d39e40a6b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:35.872542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d0f69-1c33-4146-bac1-887d39e40a6b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:35.962575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:35.982313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b77377-308b-4a54-aecb-deca42df4cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:35.983501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b77377-308b-4a54-aecb-deca42df4cbd admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:35.983841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b77377-308b-4a54-aecb-deca42df4cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:35.984175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b77377-308b-4a54-aecb-deca42df4cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:35.984984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b77377-308b-4a54-aecb-deca42df4cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:35.987478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b77377-308b-4a54-aecb-deca42df4cbd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:36.008764 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1971/3979] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:36.012877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27801d67-d16d-472b-a6a6-97508aa67688 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:36.025659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27801d67-d16d-472b-a6a6-97508aa67688 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:36.032800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f69109-5805-452e-849b-932cf52c0b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:36.033710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f69109-5805-452e-849b-932cf52c0b9a admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:36.034050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f69109-5805-452e-849b-932cf52c0b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:36.034408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f69109-5805-452e-849b-932cf52c0b9a admin admin] PATH_INFO: `/v3/projects/920f666ba8be45eb97b2cd8f623e7faf/users/44898a31c9114b61832e7b1f2a125101/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:36.037285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f69109-5805-452e-849b-932cf52c0b9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=920f666ba8be45eb97b2cd8f623e7faf, user_id=44898a31c9114b61832e7b1f2a125101, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:36.041031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f69109-5805-452e-849b-932cf52c0b9a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:36.066823 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2009/3980] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:36 2022] PUT /identity/v3/projects/920f666ba8be45eb97b2cd8f623e7faf/users/44898a31c9114b61832e7b1f2a125101/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:36.082162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416142d2-1b03-4888-953a-add790f1fea0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:36.082860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416142d2-1b03-4888-953a-add790f1fea0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:36.083664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416142d2-1b03-4888-953a-add790f1fea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:36.125953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-416142d2-1b03-4888-953a-add790f1fea0 None None] MFA Rules not processed for user `44898a31c9114b61832e7b1f2a125101`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:36.127482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-416142d2-1b03-4888-953a-add790f1fea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:36.228541 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1972/3981] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:36.237638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f69109-5805-452e-849b-932cf52c0b9a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:36.245257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b3f69109-5805-452e-849b-932cf52c0b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:36.318358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:36.324063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f69109-5805-452e-849b-932cf52c0b9a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:36.329544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf1f18-e4e6-4f5f-b844-8ea1b043f97c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:36.330436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf1f18-e4e6-4f5f-b844-8ea1b043f97c service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:36.330995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf1f18-e4e6-4f5f-b844-8ea1b043f97c service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:36.331512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf1f18-e4e6-4f5f-b844-8ea1b043f97c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:36.347459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf1f18-e4e6-4f5f-b844-8ea1b043f97c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:36.349927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf1f18-e4e6-4f5f-b844-8ea1b043f97c service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:36.364203 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2010/3982] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:07:36.643434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416142d2-1b03-4888-953a-add790f1fea0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:36.665986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416142d2-1b03-4888-953a-add790f1fea0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:36.669864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383effe-1195-4afd-99ea-b763c2e18e62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:36.670685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383effe-1195-4afd-99ea-b763c2e18e62 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:36.671001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383effe-1195-4afd-99ea-b763c2e18e62 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:36.671267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383effe-1195-4afd-99ea-b763c2e18e62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:36.756129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383effe-1195-4afd-99ea-b763c2e18e62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:36.757478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383effe-1195-4afd-99ea-b763c2e18e62 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:36.767814 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1973/3983] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:39.580642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf1f18-e4e6-4f5f-b844-8ea1b043f97c service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:39.645677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf1f18-e4e6-4f5f-b844-8ea1b043f97c service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:39.652563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56cafc-5af3-4b09-9222-09201cb3215e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:39.653354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56cafc-5af3-4b09-9222-09201cb3215e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:39.653625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56cafc-5af3-4b09-9222-09201cb3215e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:39.653886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56cafc-5af3-4b09-9222-09201cb3215e admin admin] PATH_INFO: `/v3/users/877848fa0141431caaca7a5e91a0aa01` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:39.655277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56cafc-5af3-4b09-9222-09201cb3215e admin admin] RBAC: Authorizing `identity:delete_user(user_id=877848fa0141431caaca7a5e91a0aa01)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:39.657408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56cafc-5af3-4b09-9222-09201cb3215e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:39.736394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcd689-3c63-4412-9b02-4b8a55fe1f66 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:39.737011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcd689-3c63-4412-9b02-4b8a55fe1f66 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:39.737120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcd689-3c63-4412-9b02-4b8a55fe1f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:39.742262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc56cafc-5af3-4b09-9222-09201cb3215e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '877848fa0141431caaca7a5e91a0aa01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:39.757519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc56cafc-5af3-4b09-9222-09201cb3215e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '877848fa0141431caaca7a5e91a0aa01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:39.768565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b1fcd689-3c63-4412-9b02-4b8a55fe1f66 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:39.771563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc56cafc-5af3-4b09-9222-09201cb3215e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '877848fa0141431caaca7a5e91a0aa01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:39.772221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b1fcd689-3c63-4412-9b02-4b8a55fe1f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:39.789476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc56cafc-5af3-4b09-9222-09201cb3215e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '877848fa0141431caaca7a5e91a0aa01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:39.790253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc56cafc-5af3-4b09-9222-09201cb3215e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '877848fa0141431caaca7a5e91a0aa01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:39.808694 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2011/3984] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:39 2022] DELETE /identity/v3/users/877848fa0141431caaca7a5e91a0aa01 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:39.812521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18ece3-e731-4ef2-b0b3-8128577b74a4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:39.812945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18ece3-e731-4ef2-b0b3-8128577b74a4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:39.813045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18ece3-e731-4ef2-b0b3-8128577b74a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:39.855282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-aa18ece3-e731-4ef2-b0b3-8128577b74a4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:39.857165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aa18ece3-e731-4ef2-b0b3-8128577b74a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:39.881238 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1974/3985] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:07:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:39.903696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcd689-3c63-4412-9b02-4b8a55fe1f66 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:39.905253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b1fcd689-3c63-4412-9b02-4b8a55fe1f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:39.932925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:39.933262 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2012/3986] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:39.950658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18ece3-e731-4ef2-b0b3-8128577b74a4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:39.956233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aa18ece3-e731-4ef2-b0b3-8128577b74a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:39.976272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:40.016830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcd689-3c63-4412-9b02-4b8a55fe1f66 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:40.052422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18ece3-e731-4ef2-b0b3-8128577b74a4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:40.058001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb7f05-e9cc-4a07-a6d9-1a1a7e7d80a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:40.058765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb7f05-e9cc-4a07-a6d9-1a1a7e7d80a3 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:40.059133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb7f05-e9cc-4a07-a6d9-1a1a7e7d80a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:40.059479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb7f05-e9cc-4a07-a6d9-1a1a7e7d80a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:40.072717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb7f05-e9cc-4a07-a6d9-1a1a7e7d80a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:40.074198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb7f05-e9cc-4a07-a6d9-1a1a7e7d80a3 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:40.086742 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2013/3987] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:07:40.091591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1de5c3-2a80-4fac-b023-bbdc8455ebd2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:40.092281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1de5c3-2a80-4fac-b023-bbdc8455ebd2 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:40.092645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1de5c3-2a80-4fac-b023-bbdc8455ebd2 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:40.093006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1de5c3-2a80-4fac-b023-bbdc8455ebd2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:40.164029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1de5c3-2a80-4fac-b023-bbdc8455ebd2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:40.165799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1de5c3-2a80-4fac-b023-bbdc8455ebd2 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:40.175542 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1975/3988] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:07:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:07:40.430584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb7f05-e9cc-4a07-a6d9-1a1a7e7d80a3 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:40.432261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c9cb7f05-e9cc-4a07-a6d9-1a1a7e7d80a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:40.449869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb7f05-e9cc-4a07-a6d9-1a1a7e7d80a3 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:40.455479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:40.456389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:40.457089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:40.457352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] PATH_INFO: `/v3/projects/1b33df9ae22a450abacc30f1682e24c6` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:40.459074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b33df9ae22a450abacc30f1682e24c6)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:40.461778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:40.515859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b33df9ae22a450abacc30f1682e24c6', 'request_id': 'req-c36af261-ad7e-4b77-b149-3736675a8ddf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:40.528032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b33df9ae22a450abacc30f1682e24c6', 'request_id': 'req-c36af261-ad7e-4b77-b149-3736675a8ddf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:40.534148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b33df9ae22a450abacc30f1682e24c6', 'request_id': 'req-c36af261-ad7e-4b77-b149-3736675a8ddf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:40.545997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] The token cache is being invalidate because project 1b33df9ae22a450abacc30f1682e24c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:40.546278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:40.546936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:40.548529 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2014/3989] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:40 2022] DELETE /identity/v3/projects/1b33df9ae22a450abacc30f1682e24c6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:40.553344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1de5c3-2a80-4fac-b023-bbdc8455ebd2 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:40.570294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1de5c3-2a80-4fac-b023-bbdc8455ebd2 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:40.629374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c21596-829e-472d-9e9c-57e44b1f7982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:40.630199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c21596-829e-472d-9e9c-57e44b1f7982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:40.630564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c21596-829e-472d-9e9c-57e44b1f7982 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:40.631025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c21596-829e-472d-9e9c-57e44b1f7982 admin admin] PATH_INFO: `/v3/users/44898a31c9114b61832e7b1f2a125101` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:40.632160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c21596-829e-472d-9e9c-57e44b1f7982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44898a31c9114b61832e7b1f2a125101)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:40.634128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c21596-829e-472d-9e9c-57e44b1f7982 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:40.635672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:40.689732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-77c21596-829e-472d-9e9c-57e44b1f7982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44898a31c9114b61832e7b1f2a125101'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:40.704231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-77c21596-829e-472d-9e9c-57e44b1f7982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44898a31c9114b61832e7b1f2a125101'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:40.720505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-77c21596-829e-472d-9e9c-57e44b1f7982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44898a31c9114b61832e7b1f2a125101'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:40.721333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-77c21596-829e-472d-9e9c-57e44b1f7982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44898a31c9114b61832e7b1f2a125101'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:40.721979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-77c21596-829e-472d-9e9c-57e44b1f7982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44898a31c9114b61832e7b1f2a125101'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:40.735684 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1976/3990] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:40 2022] DELETE /identity/v3/users/44898a31c9114b61832e7b1f2a125101 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:41.102585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:41.104324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:41.117881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36af261-ad7e-4b77-b149-3736675a8ddf admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:41.179684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:41.180316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:41.180624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:41.180887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc admin admin] PATH_INFO: `/v3/projects/920f666ba8be45eb97b2cd8f623e7faf` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:41.182069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=920f666ba8be45eb97b2cd8f623e7faf)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:41.184080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:41.233742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:41.242895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '920f666ba8be45eb97b2cd8f623e7faf', 'request_id': 'req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:41.255471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '920f666ba8be45eb97b2cd8f623e7faf', 'request_id': 'req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:41.262241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '920f666ba8be45eb97b2cd8f623e7faf', 'request_id': 'req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:41.277912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc admin admin] The token cache is being invalidate because project 920f666ba8be45eb97b2cd8f623e7faf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:41.278194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:41.278851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fbdc6be3-9d7b-44af-ba59-be889b67b0fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:41.280380 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2015/3991] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:41 2022] DELETE /identity/v3/projects/920f666ba8be45eb97b2cd8f623e7faf => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:41.285599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5df56-dac4-4d65-b621-c6a6c6112020 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:41.286131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5df56-dac4-4d65-b621-c6a6c6112020 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:41.286221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5df56-dac4-4d65-b621-c6a6c6112020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:41.307238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-06d5df56-dac4-4d65-b621-c6a6c6112020 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:41.308500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-06d5df56-dac4-4d65-b621-c6a6c6112020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:41.372172 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1977/3992] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:41.380060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b598f-0cd2-4b8d-ac73-256c8e5d22d9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:41.380331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b598f-0cd2-4b8d-ac73-256c8e5d22d9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:41.380644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b598f-0cd2-4b8d-ac73-256c8e5d22d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:41.401409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5f3b598f-0cd2-4b8d-ac73-256c8e5d22d9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:41.402638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5f3b598f-0cd2-4b8d-ac73-256c8e5d22d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:41.410936 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2016/3993] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:41.415125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5df56-dac4-4d65-b621-c6a6c6112020 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:41.425252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5df56-dac4-4d65-b621-c6a6c6112020 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:41.430356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05204a18-bff5-48a8-9e49-89e186423579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:41.430949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05204a18-bff5-48a8-9e49-89e186423579 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:41.431216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05204a18-bff5-48a8-9e49-89e186423579 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:41.431475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05204a18-bff5-48a8-9e49-89e186423579 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:41.432419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05204a18-bff5-48a8-9e49-89e186423579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:07:41.432741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05204a18-bff5-48a8-9e49-89e186423579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:41.434338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05204a18-bff5-48a8-9e49-89e186423579 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:41.447537 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1978/3994] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:07:41.452207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b598f-0cd2-4b8d-ac73-256c8e5d22d9 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:41.461912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b598f-0cd2-4b8d-ac73-256c8e5d22d9 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:41.467529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753bf39-23ca-4259-9949-a94f89bf79a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:41.468133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753bf39-23ca-4259-9949-a94f89bf79a5 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:41.468420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753bf39-23ca-4259-9949-a94f89bf79a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:41.468679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753bf39-23ca-4259-9949-a94f89bf79a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:41.469307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753bf39-23ca-4259-9949-a94f89bf79a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:41.471242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753bf39-23ca-4259-9949-a94f89bf79a5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:41.502074 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2017/3995] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:41 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:07:41.508047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05204a18-bff5-48a8-9e49-89e186423579 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:41.518259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05204a18-bff5-48a8-9e49-89e186423579 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:41.577706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be55ef-d3ae-4009-aa9a-62b4e3eef891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:41.578219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be55ef-d3ae-4009-aa9a-62b4e3eef891 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:41.578491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be55ef-d3ae-4009-aa9a-62b4e3eef891 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:41.578753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be55ef-d3ae-4009-aa9a-62b4e3eef891 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:41.579370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be55ef-d3ae-4009-aa9a-62b4e3eef891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:41.581331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be55ef-d3ae-4009-aa9a-62b4e3eef891 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:41.592892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:41.607717 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1979/3996] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:41 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:41.612018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753bf39-23ca-4259-9949-a94f89bf79a5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:41.622694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753bf39-23ca-4259-9949-a94f89bf79a5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:41.627424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f53b8-0012-4341-9873-a2214ac22153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:41.628013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f53b8-0012-4341-9873-a2214ac22153 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:41.628285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f53b8-0012-4341-9873-a2214ac22153 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:41.628574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f53b8-0012-4341-9873-a2214ac22153 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:41.629213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f53b8-0012-4341-9873-a2214ac22153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:41.631440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f53b8-0012-4341-9873-a2214ac22153 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:41.641644 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2018/3997] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:41.646158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be55ef-d3ae-4009-aa9a-62b4e3eef891 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:41.656836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be55ef-d3ae-4009-aa9a-62b4e3eef891 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:41.662358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909355e-39ec-44ea-b1ad-51f00a618442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:41.662936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909355e-39ec-44ea-b1ad-51f00a618442 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:41.663207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909355e-39ec-44ea-b1ad-51f00a618442 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:41.663474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909355e-39ec-44ea-b1ad-51f00a618442 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:41.664110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909355e-39ec-44ea-b1ad-51f00a618442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:41.665755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909355e-39ec-44ea-b1ad-51f00a618442 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:41.679579 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1980/3998] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:41.684271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f53b8-0012-4341-9873-a2214ac22153 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:41.698077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f53b8-0012-4341-9873-a2214ac22153 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:41.703821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df070e79-36b6-4735-aa6d-769b8ab387d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:41.704504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df070e79-36b6-4735-aa6d-769b8ab387d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:41.704781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df070e79-36b6-4735-aa6d-769b8ab387d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:41.705039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df070e79-36b6-4735-aa6d-769b8ab387d4 admin admin] PATH_INFO: `/v3/projects/b9e3e933f5eb48ec91a350c8e512ceb0/users/01c831dd9e9046bfb178bb61503132ec/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:41.720405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df070e79-36b6-4735-aa6d-769b8ab387d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e3e933f5eb48ec91a350c8e512ceb0, user_id=01c831dd9e9046bfb178bb61503132ec, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:41.723065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df070e79-36b6-4735-aa6d-769b8ab387d4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:41.740302 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2019/3999] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:41 2022] PUT /identity/v3/projects/b9e3e933f5eb48ec91a350c8e512ceb0/users/01c831dd9e9046bfb178bb61503132ec/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:41.752032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a24ae-a686-4638-9d04-ae87d6d7a244 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:41.752306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a24ae-a686-4638-9d04-ae87d6d7a244 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:41.752600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a24ae-a686-4638-9d04-ae87d6d7a244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:41.781963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ee7a24ae-a686-4638-9d04-ae87d6d7a244 None None] MFA Rules not processed for user `01c831dd9e9046bfb178bb61503132ec`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:41.783226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ee7a24ae-a686-4638-9d04-ae87d6d7a244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:41.853419 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1981/4000] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:41.860661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df070e79-36b6-4735-aa6d-769b8ab387d4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:41.929762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df070e79-36b6-4735-aa6d-769b8ab387d4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:41.934879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933e2d4-bb80-44cc-858e-18462535e12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:41.935485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933e2d4-bb80-44cc-858e-18462535e12e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:41.935755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933e2d4-bb80-44cc-858e-18462535e12e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:41.936021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933e2d4-bb80-44cc-858e-18462535e12e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:41.936672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933e2d4-bb80-44cc-858e-18462535e12e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:41.938867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933e2d4-bb80-44cc-858e-18462535e12e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:41.979415 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2020/4001] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:41 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:41.980644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a24ae-a686-4638-9d04-ae87d6d7a244 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:41.996768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a24ae-a686-4638-9d04-ae87d6d7a244 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:42.079202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd6292-40bd-42b1-8b2b-6ded0435b152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:42.079812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6292-40bd-42b1-8b2b-6ded0435b152 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:42.080084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6292-40bd-42b1-8b2b-6ded0435b152 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:42.080352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6292-40bd-42b1-8b2b-6ded0435b152 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:42.081005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd6292-40bd-42b1-8b2b-6ded0435b152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:42.082985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd6292-40bd-42b1-8b2b-6ded0435b152 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:42.110376 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1982/4002] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:41 2022] POST /identity/v3/users => generated 297 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:07:42.111724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933e2d4-bb80-44cc-858e-18462535e12e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:42.123393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933e2d4-bb80-44cc-858e-18462535e12e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:42.128139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2596f0e-72db-417c-8023-57206c7f47c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:42.128866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2596f0e-72db-417c-8023-57206c7f47c5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:42.129206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2596f0e-72db-417c-8023-57206c7f47c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:42.129536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2596f0e-72db-417c-8023-57206c7f47c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:42.130822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2596f0e-72db-417c-8023-57206c7f47c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:42.132534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2596f0e-72db-417c-8023-57206c7f47c5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:42.143512 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2021/4003] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:42.148359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd6292-40bd-42b1-8b2b-6ded0435b152 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:42.159177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd6292-40bd-42b1-8b2b-6ded0435b152 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:42.164712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff6b96-1766-4044-9644-c034f4776195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:42.165379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff6b96-1766-4044-9644-c034f4776195 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:42.165646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff6b96-1766-4044-9644-c034f4776195 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:42.165909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff6b96-1766-4044-9644-c034f4776195 admin admin] PATH_INFO: `/v3/projects/17121c9e5df6461688652c1aa7769602/users/4e77a5abc29545028fa87d299e53737c/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:42.182518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff6b96-1766-4044-9644-c034f4776195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17121c9e5df6461688652c1aa7769602, user_id=4e77a5abc29545028fa87d299e53737c, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:42.185237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff6b96-1766-4044-9644-c034f4776195 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:42.199694 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1983/4004] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:42 2022] PUT /identity/v3/projects/17121c9e5df6461688652c1aa7769602/users/4e77a5abc29545028fa87d299e53737c/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:42.203751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2596f0e-72db-417c-8023-57206c7f47c5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:42.215172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2596f0e-72db-417c-8023-57206c7f47c5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:42.259071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:42.290821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8ae2c-2df8-407f-81cc-5ed88f922219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:42.291441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8ae2c-2df8-407f-81cc-5ed88f922219 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:42.291713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8ae2c-2df8-407f-81cc-5ed88f922219 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:42.291976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8ae2c-2df8-407f-81cc-5ed88f922219 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:42.292654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8ae2c-2df8-407f-81cc-5ed88f922219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:42.294250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8ae2c-2df8-407f-81cc-5ed88f922219 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:42.304935 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2022/4005] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:42.309355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff6b96-1766-4044-9644-c034f4776195 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:42.320313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff6b96-1766-4044-9644-c034f4776195 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:42.325400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9145c0-46ea-46b7-916d-8b49655cdb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:42.326059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9145c0-46ea-46b7-916d-8b49655cdb3e admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:42.326357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9145c0-46ea-46b7-916d-8b49655cdb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:42.326626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9145c0-46ea-46b7-916d-8b49655cdb3e admin admin] PATH_INFO: `/v3/projects/17121c9e5df6461688652c1aa7769602/users/4e77a5abc29545028fa87d299e53737c/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:42.328615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9145c0-46ea-46b7-916d-8b49655cdb3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17121c9e5df6461688652c1aa7769602, user_id=4e77a5abc29545028fa87d299e53737c, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:42.331253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9145c0-46ea-46b7-916d-8b49655cdb3e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:42.352033 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1984/4006] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:42 2022] PUT /identity/v3/projects/17121c9e5df6461688652c1aa7769602/users/4e77a5abc29545028fa87d299e53737c/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:07:42.355227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e586a8-216f-4a7e-8b44-884ef459912f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:42.355466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e586a8-216f-4a7e-8b44-884ef459912f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:42.355740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e586a8-216f-4a7e-8b44-884ef459912f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:42.414305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-08e586a8-216f-4a7e-8b44-884ef459912f None None] MFA Rules not processed for user `4e77a5abc29545028fa87d299e53737c`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:42.416086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-08e586a8-216f-4a7e-8b44-884ef459912f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:42.539718 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2023/4007] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:42.552989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9145c0-46ea-46b7-916d-8b49655cdb3e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:42.554689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5f9145c0-46ea-46b7-916d-8b49655cdb3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:42.588681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:42.619710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9145c0-46ea-46b7-916d-8b49655cdb3e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:42.623714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635d9eb-72da-4364-8708-a72df424d1eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:42.624123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635d9eb-72da-4364-8708-a72df424d1eb service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:42.624413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635d9eb-72da-4364-8708-a72df424d1eb service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:42.624685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635d9eb-72da-4364-8708-a72df424d1eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:42.681904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635d9eb-72da-4364-8708-a72df424d1eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:42.683516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635d9eb-72da-4364-8708-a72df424d1eb service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:42.694326 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1985/4008] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:07:44.526320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e586a8-216f-4a7e-8b44-884ef459912f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:44.528475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-08e586a8-216f-4a7e-8b44-884ef459912f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:44.616779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e586a8-216f-4a7e-8b44-884ef459912f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:44.621581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:44.622212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:44.622482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:44.622771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] PATH_INFO: `/v3/users/e9af61808fca4c1bb32f135d92cd9dee` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:44.623889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9af61808fca4c1bb32f135d92cd9dee)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:44.625865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:44.677105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9af61808fca4c1bb32f135d92cd9dee'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:44.685592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9af61808fca4c1bb32f135d92cd9dee'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:44.691684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:44.694731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9af61808fca4c1bb32f135d92cd9dee'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:44.709937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9af61808fca4c1bb32f135d92cd9dee'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:44.710853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9af61808fca4c1bb32f135d92cd9dee'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:44.716897 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2024/4009] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:44 2022] DELETE /identity/v3/users/e9af61808fca4c1bb32f135d92cd9dee => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:44.721609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24c823-3d17-4276-884a-9b33644b3935 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:44.722119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24c823-3d17-4276-884a-9b33644b3935 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:44.722299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24c823-3d17-4276-884a-9b33644b3935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:44.741792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ec24c823-3d17-4276-884a-9b33644b3935 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:44.743064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ec24c823-3d17-4276-884a-9b33644b3935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:44.828662 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1986/4010] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:44.837821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:44.839513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:44.905577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa25bdd-2184-406b-847c-5e4513668a3f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:44.909364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28d57f-a58b-49e0-8527-b728e25e0a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:44.910611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28d57f-a58b-49e0-8527-b728e25e0a9b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:44.910889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28d57f-a58b-49e0-8527-b728e25e0a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:44.911262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28d57f-a58b-49e0-8527-b728e25e0a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:44.922585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28d57f-a58b-49e0-8527-b728e25e0a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:44.924046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28d57f-a58b-49e0-8527-b728e25e0a9b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:44.943056 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2025/4011] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:07:45.168544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24c823-3d17-4276-884a-9b33644b3935 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:45.170163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ec24c823-3d17-4276-884a-9b33644b3935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:45.183985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24c823-3d17-4276-884a-9b33644b3935 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:45.188836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da396096-8aba-4012-b903-1d246d222123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:45.189484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da396096-8aba-4012-b903-1d246d222123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:45.189758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da396096-8aba-4012-b903-1d246d222123 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:45.190022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da396096-8aba-4012-b903-1d246d222123 admin admin] PATH_INFO: `/v3/projects/73509e0b7e4c404fb48b66c784d07ce9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:45.191153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da396096-8aba-4012-b903-1d246d222123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73509e0b7e4c404fb48b66c784d07ce9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:45.193150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da396096-8aba-4012-b903-1d246d222123 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:45.248876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-da396096-8aba-4012-b903-1d246d222123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73509e0b7e4c404fb48b66c784d07ce9', 'request_id': 'req-da396096-8aba-4012-b903-1d246d222123'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:45.255234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-da396096-8aba-4012-b903-1d246d222123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73509e0b7e4c404fb48b66c784d07ce9', 'request_id': 'req-da396096-8aba-4012-b903-1d246d222123'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:45.264561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-da396096-8aba-4012-b903-1d246d222123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73509e0b7e4c404fb48b66c784d07ce9', 'request_id': 'req-da396096-8aba-4012-b903-1d246d222123'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:45.276695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-da396096-8aba-4012-b903-1d246d222123 admin admin] The token cache is being invalidate because project 73509e0b7e4c404fb48b66c784d07ce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:45.276981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-da396096-8aba-4012-b903-1d246d222123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:45.277616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-da396096-8aba-4012-b903-1d246d222123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:45.279182 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1987/4012] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:45 2022] DELETE /identity/v3/projects/73509e0b7e4c404fb48b66c784d07ce9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:45.284125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05caffd-600e-4e6c-8423-7f455dea37c1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:45.284430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05caffd-600e-4e6c-8423-7f455dea37c1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:45.284702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05caffd-600e-4e6c-8423-7f455dea37c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:45.305785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a05caffd-600e-4e6c-8423-7f455dea37c1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:45.307036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a05caffd-600e-4e6c-8423-7f455dea37c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:45.370407 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2026/4013] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:45.453230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbb3ca-4e5d-47c9-86f0-da1e3589514e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:45.453498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbb3ca-4e5d-47c9-86f0-da1e3589514e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:45.453767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbb3ca-4e5d-47c9-86f0-da1e3589514e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:45.463097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:45.475814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-97bbb3ca-4e5d-47c9-86f0-da1e3589514e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:45.477456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-97bbb3ca-4e5d-47c9-86f0-da1e3589514e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:45.485581 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1988/4014] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:45.490559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05caffd-600e-4e6c-8423-7f455dea37c1 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:45.506661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05caffd-600e-4e6c-8423-7f455dea37c1 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:45.511687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd284b6f-e05f-4518-b84a-a61e02483087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:45.512296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284b6f-e05f-4518-b84a-a61e02483087 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:45.512592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284b6f-e05f-4518-b84a-a61e02483087 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:45.512852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284b6f-e05f-4518-b84a-a61e02483087 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:45.513492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd284b6f-e05f-4518-b84a-a61e02483087 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:07:45.513780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd284b6f-e05f-4518-b84a-a61e02483087 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:45.515363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd284b6f-e05f-4518-b84a-a61e02483087 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:45.533606 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2027/4015] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:07:45.539022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbb3ca-4e5d-47c9-86f0-da1e3589514e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:45.548860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbb3ca-4e5d-47c9-86f0-da1e3589514e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:45.556477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1de204-6739-4d1c-8401-9c47bbe90104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:45.557719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1de204-6739-4d1c-8401-9c47bbe90104 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:45.557719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1de204-6739-4d1c-8401-9c47bbe90104 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:45.557921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1de204-6739-4d1c-8401-9c47bbe90104 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:45.558639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1de204-6739-4d1c-8401-9c47bbe90104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:45.560647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1de204-6739-4d1c-8401-9c47bbe90104 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:45.603369 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1989/4016] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:45 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:45.606285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd284b6f-e05f-4518-b84a-a61e02483087 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:45.617602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd284b6f-e05f-4518-b84a-a61e02483087 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:45.677190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ec84a-fa6d-4dc3-9e00-1d4b82084dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:45.677828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ec84a-fa6d-4dc3-9e00-1d4b82084dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:45.678055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ec84a-fa6d-4dc3-9e00-1d4b82084dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:45.678317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ec84a-fa6d-4dc3-9e00-1d4b82084dc7 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:45.678936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ec84a-fa6d-4dc3-9e00-1d4b82084dc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:45.681125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ec84a-fa6d-4dc3-9e00-1d4b82084dc7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:45.699995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:45.708038 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2028/4017] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:45 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:45.712396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1de204-6739-4d1c-8401-9c47bbe90104 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:45.723575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1de204-6739-4d1c-8401-9c47bbe90104 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:45.729290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db33a76-7316-4e8b-8f8b-6f6a7ff1a1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:45.729887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db33a76-7316-4e8b-8f8b-6f6a7ff1a1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:45.730208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db33a76-7316-4e8b-8f8b-6f6a7ff1a1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:45.730828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db33a76-7316-4e8b-8f8b-6f6a7ff1a1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:45.731145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db33a76-7316-4e8b-8f8b-6f6a7ff1a1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:45.732875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db33a76-7316-4e8b-8f8b-6f6a7ff1a1c9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:45.747242 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1990/4018] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:45.748100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ec84a-fa6d-4dc3-9e00-1d4b82084dc7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:45.762751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ec84a-fa6d-4dc3-9e00-1d4b82084dc7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:45.773013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74d9bf-dc5d-457c-92fd-ea99af482e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:45.773013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74d9bf-dc5d-457c-92fd-ea99af482e7d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:45.773013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74d9bf-dc5d-457c-92fd-ea99af482e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:45.773013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74d9bf-dc5d-457c-92fd-ea99af482e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:45.773374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74d9bf-dc5d-457c-92fd-ea99af482e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:45.774183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74d9bf-dc5d-457c-92fd-ea99af482e7d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:45.788461 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2029/4019] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:07:45.791501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db33a76-7316-4e8b-8f8b-6f6a7ff1a1c9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:45.802757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db33a76-7316-4e8b-8f8b-6f6a7ff1a1c9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:45.807538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f41b5-16a7-483a-bbfa-419317a59d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:45.808218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f41b5-16a7-483a-bbfa-419317a59d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:45.808530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f41b5-16a7-483a-bbfa-419317a59d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:45.808792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f41b5-16a7-483a-bbfa-419317a59d01 admin admin] PATH_INFO: `/v3/projects/78dc9242bdde4164b557231f3b304094/users/f144c6cdd63d4ab6ac5669aba1a09fb2/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:45.825044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f41b5-16a7-483a-bbfa-419317a59d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78dc9242bdde4164b557231f3b304094, user_id=f144c6cdd63d4ab6ac5669aba1a09fb2, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:45.827702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f41b5-16a7-483a-bbfa-419317a59d01 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:45.841472 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1991/4020] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:45 2022] PUT /identity/v3/projects/78dc9242bdde4164b557231f3b304094/users/f144c6cdd63d4ab6ac5669aba1a09fb2/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:45.852723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3ed2d-ae4c-47f9-b70a-56be842f911b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:45.852944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3ed2d-ae4c-47f9-b70a-56be842f911b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:45.853418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3ed2d-ae4c-47f9-b70a-56be842f911b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:45.882477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-95e3ed2d-ae4c-47f9-b70a-56be842f911b None None] MFA Rules not processed for user `f144c6cdd63d4ab6ac5669aba1a09fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:45.883994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-95e3ed2d-ae4c-47f9-b70a-56be842f911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:45.928491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f41b5-16a7-483a-bbfa-419317a59d01 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:45.930266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-131f41b5-16a7-483a-bbfa-419317a59d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:45.955650 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2030/4021] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:45.971485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3ed2d-ae4c-47f9-b70a-56be842f911b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:45.974831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-95e3ed2d-ae4c-47f9-b70a-56be842f911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:46.014484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f41b5-16a7-483a-bbfa-419317a59d01 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:46.019799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:46.020609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:46.021302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:46.021302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] PATH_INFO: `/v3/users/01c831dd9e9046bfb178bb61503132ec` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:46.022342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c831dd9e9046bfb178bb61503132ec)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:46.024663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:46.073427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3ed2d-ae4c-47f9-b70a-56be842f911b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:46.077894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a2a01-8b64-4e38-aca2-4c13d1d42b4c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:46.078743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a2a01-8b64-4e38-aca2-4c13d1d42b4c service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:46.079018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a2a01-8b64-4e38-aca2-4c13d1d42b4c service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:46.079329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a2a01-8b64-4e38-aca2-4c13d1d42b4c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:46.089639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a2a01-8b64-4e38-aca2-4c13d1d42b4c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:46.091307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a2a01-8b64-4e38-aca2-4c13d1d42b4c service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:46.096957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c831dd9e9046bfb178bb61503132ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.101422 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2031/4022] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:07:46.106984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c831dd9e9046bfb178bb61503132ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.121357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c831dd9e9046bfb178bb61503132ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.122443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c831dd9e9046bfb178bb61503132ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.123223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c831dd9e9046bfb178bb61503132ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.138418 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1992/4023] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:45 2022] DELETE /identity/v3/users/01c831dd9e9046bfb178bb61503132ec => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:07:46.142979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08f72c-b12f-4efc-acd2-65a0f7671091 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:46.144105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08f72c-b12f-4efc-acd2-65a0f7671091 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:46.144409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08f72c-b12f-4efc-acd2-65a0f7671091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:46.171122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-db08f72c-b12f-4efc-acd2-65a0f7671091 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:46.172433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-db08f72c-b12f-4efc-acd2-65a0f7671091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:46.245880 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2032/4024] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:07:46.258776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:46.260403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:46.285480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:46.335155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c9687-d0b1-41b4-8703-7bf142fa4a76 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:46.348676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26aedd-199a-4891-a99a-ffe5740fcd5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:46.348676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26aedd-199a-4891-a99a-ffe5740fcd5b service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:46.348676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26aedd-199a-4891-a99a-ffe5740fcd5b service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:46.348676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26aedd-199a-4891-a99a-ffe5740fcd5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:46.349954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26aedd-199a-4891-a99a-ffe5740fcd5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:46.351379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26aedd-199a-4891-a99a-ffe5740fcd5b service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:46.361136 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1993/4025] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:46.652047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08f72c-b12f-4efc-acd2-65a0f7671091 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:46.654297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-db08f72c-b12f-4efc-acd2-65a0f7671091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:46.681639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08f72c-b12f-4efc-acd2-65a0f7671091 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:46.687548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:46.688193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:46.688532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:46.688879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c admin admin] PATH_INFO: `/v3/projects/b9e3e933f5eb48ec91a350c8e512ceb0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:46.689861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e3e933f5eb48ec91a350c8e512ceb0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:46.691872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:46.710729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:46.745548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e3e933f5eb48ec91a350c8e512ceb0', 'request_id': 'req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.753324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e3e933f5eb48ec91a350c8e512ceb0', 'request_id': 'req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.759377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e3e933f5eb48ec91a350c8e512ceb0', 'request_id': 'req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.771131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c admin admin] The token cache is being invalidate because project b9e3e933f5eb48ec91a350c8e512ceb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:46.771411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.772025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5e6b73a7-4d7c-4516-a177-2fabeacc9c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.773592 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2033/4026] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:46 2022] DELETE /identity/v3/projects/b9e3e933f5eb48ec91a350c8e512ceb0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:46.777660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26aedd-199a-4891-a99a-ffe5740fcd5b service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:46.787597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26aedd-199a-4891-a99a-ffe5740fcd5b service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:46.850801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:46.851469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:46.851741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:46.852011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] PATH_INFO: `/v3/users/4e77a5abc29545028fa87d299e53737c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:46.853263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e77a5abc29545028fa87d299e53737c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:46.855251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:46.904752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e77a5abc29545028fa87d299e53737c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.913876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e77a5abc29545028fa87d299e53737c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.925953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e77a5abc29545028fa87d299e53737c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.926652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e77a5abc29545028fa87d299e53737c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.927258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e77a5abc29545028fa87d299e53737c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:46.939753 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1994/4027] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:46 2022] DELETE /identity/v3/users/4e77a5abc29545028fa87d299e53737c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:47.270946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680bdcf-5e5d-4d01-ac23-863b76d1a5f3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:47.271273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680bdcf-5e5d-4d01-ac23-863b76d1a5f3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:47.271729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680bdcf-5e5d-4d01-ac23-863b76d1a5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:47.294732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1680bdcf-5e5d-4d01-ac23-863b76d1a5f3 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:47.296082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1680bdcf-5e5d-4d01-ac23-863b76d1a5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:47.299438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:47.300902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:47.384901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:47.417948 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2034/4028] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:07:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:07:47.432335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680bdcf-5e5d-4d01-ac23-863b76d1a5f3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:47.434263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1680bdcf-5e5d-4d01-ac23-863b76d1a5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:47.435904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad5794-d801-47e5-8ef5-5f9f13edb91f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:47.452134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90342b2e-8cb5-4475-84ec-bbeed579aee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:47.453317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90342b2e-8cb5-4475-84ec-bbeed579aee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:47.454523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90342b2e-8cb5-4475-84ec-bbeed579aee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:47.455434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90342b2e-8cb5-4475-84ec-bbeed579aee8 admin admin] PATH_INFO: `/v3/projects/17121c9e5df6461688652c1aa7769602` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:47.459776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90342b2e-8cb5-4475-84ec-bbeed579aee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17121c9e5df6461688652c1aa7769602)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:47.463519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90342b2e-8cb5-4475-84ec-bbeed579aee8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:47.556058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680bdcf-5e5d-4d01-ac23-863b76d1a5f3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:47.575411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-90342b2e-8cb5-4475-84ec-bbeed579aee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17121c9e5df6461688652c1aa7769602', 'request_id': 'req-90342b2e-8cb5-4475-84ec-bbeed579aee8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:47.583313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-90342b2e-8cb5-4475-84ec-bbeed579aee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17121c9e5df6461688652c1aa7769602', 'request_id': 'req-90342b2e-8cb5-4475-84ec-bbeed579aee8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:47.602052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-90342b2e-8cb5-4475-84ec-bbeed579aee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17121c9e5df6461688652c1aa7769602', 'request_id': 'req-90342b2e-8cb5-4475-84ec-bbeed579aee8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:47.628749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-90342b2e-8cb5-4475-84ec-bbeed579aee8 admin admin] The token cache is being invalidate because project 17121c9e5df6461688652c1aa7769602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:47.630902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-90342b2e-8cb5-4475-84ec-bbeed579aee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:47.633261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-90342b2e-8cb5-4475-84ec-bbeed579aee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:47.640489 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1995/4029] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:47 2022] DELETE /identity/v3/projects/17121c9e5df6461688652c1aa7769602 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:47.642608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858a68a-4a62-4aa6-b86a-3a4e868da4e0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:47.642994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858a68a-4a62-4aa6-b86a-3a4e868da4e0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:47.643386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858a68a-4a62-4aa6-b86a-3a4e868da4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:47.662814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:47.693234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1858a68a-4a62-4aa6-b86a-3a4e868da4e0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:47.694264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1858a68a-4a62-4aa6-b86a-3a4e868da4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:47.706244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a823d9-d786-4675-a202-ceaf6f147916 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:47.707512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a823d9-d786-4675-a202-ceaf6f147916 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:47.708233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a823d9-d786-4675-a202-ceaf6f147916 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:47.708978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a823d9-d786-4675-a202-ceaf6f147916 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:47.712550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e6a823d9-d786-4675-a202-ceaf6f147916 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:47.786817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a823d9-d786-4675-a202-ceaf6f147916 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:47.788781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a823d9-d786-4675-a202-ceaf6f147916 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:47.823136 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2035/4030] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:07:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:07:47.827858 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1996/4031] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:07:47.836773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caa028-f2ef-4c60-835a-4bda3f8400bd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:47.837436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caa028-f2ef-4c60-835a-4bda3f8400bd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:47.855714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caa028-f2ef-4c60-835a-4bda3f8400bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:47.898859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-04caa028-f2ef-4c60-835a-4bda3f8400bd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:47.900286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-04caa028-f2ef-4c60-835a-4bda3f8400bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:47.914709 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2036/4032] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:47.919119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858a68a-4a62-4aa6-b86a-3a4e868da4e0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:47.930370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858a68a-4a62-4aa6-b86a-3a4e868da4e0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:47.936252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92225518-b88b-4571-b575-395c66214e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:47.936951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92225518-b88b-4571-b575-395c66214e28 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:47.937307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92225518-b88b-4571-b575-395c66214e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:47.937681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92225518-b88b-4571-b575-395c66214e28 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:47.938440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92225518-b88b-4571-b575-395c66214e28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:07:47.938798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92225518-b88b-4571-b575-395c66214e28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:47.941135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92225518-b88b-4571-b575-395c66214e28 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:47.963382 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1997/4033] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:07:47.964603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04caa028-f2ef-4c60-835a-4bda3f8400bd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:47.976548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04caa028-f2ef-4c60-835a-4bda3f8400bd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:48.004543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94caf7-a214-4ab2-ada6-cfce73373e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:48.005352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94caf7-a214-4ab2-ada6-cfce73373e7a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:48.005830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94caf7-a214-4ab2-ada6-cfce73373e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:48.009962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94caf7-a214-4ab2-ada6-cfce73373e7a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:48.010296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94caf7-a214-4ab2-ada6-cfce73373e7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:48.017018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94caf7-a214-4ab2-ada6-cfce73373e7a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:48.087867 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2037/4034] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:47 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:07:48.091906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92225518-b88b-4571-b575-395c66214e28 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:48.105828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92225518-b88b-4571-b575-395c66214e28 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:48.283390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe9971-4703-43be-aaa5-38568ff7a6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:48.284164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe9971-4703-43be-aaa5-38568ff7a6d7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:48.287681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe9971-4703-43be-aaa5-38568ff7a6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:48.288082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe9971-4703-43be-aaa5-38568ff7a6d7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:48.290516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe9971-4703-43be-aaa5-38568ff7a6d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:48.295034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe9971-4703-43be-aaa5-38568ff7a6d7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:48.306384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:48.334102 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1998/4035] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:48 2022] POST /identity/v3/users => generated 285 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:48.335091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94caf7-a214-4ab2-ada6-cfce73373e7a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:48.347279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94caf7-a214-4ab2-ada6-cfce73373e7a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:48.352552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bd6e0-91dd-4b4b-a54f-1bbe465d5190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:48.353179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bd6e0-91dd-4b4b-a54f-1bbe465d5190 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:48.353452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bd6e0-91dd-4b4b-a54f-1bbe465d5190 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:48.353721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bd6e0-91dd-4b4b-a54f-1bbe465d5190 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:48.354339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bd6e0-91dd-4b4b-a54f-1bbe465d5190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:48.355958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bd6e0-91dd-4b4b-a54f-1bbe465d5190 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:48.371543 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2038/4036] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:48.380793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe9971-4703-43be-aaa5-38568ff7a6d7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:48.392134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe9971-4703-43be-aaa5-38568ff7a6d7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:48.397026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07694ca3-a3c8-4d43-b58c-a8a8732fdd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:48.397631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07694ca3-a3c8-4d43-b58c-a8a8732fdd25 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:48.397903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07694ca3-a3c8-4d43-b58c-a8a8732fdd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:48.398169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07694ca3-a3c8-4d43-b58c-a8a8732fdd25 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:48.398830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07694ca3-a3c8-4d43-b58c-a8a8732fdd25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:48.400464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07694ca3-a3c8-4d43-b58c-a8a8732fdd25 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:48.412277 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 1999/4037] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:48.416476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bd6e0-91dd-4b4b-a54f-1bbe465d5190 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:48.427883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bd6e0-91dd-4b4b-a54f-1bbe465d5190 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:48.432685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b6391-e5fe-498e-a6c2-89ca3e34f2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:48.433411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b6391-e5fe-498e-a6c2-89ca3e34f2a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:48.433687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b6391-e5fe-498e-a6c2-89ca3e34f2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:48.433951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b6391-e5fe-498e-a6c2-89ca3e34f2a2 admin admin] PATH_INFO: `/v3/projects/e76576d714a04974abf66c621eda6164/users/d66d4222bd084b89be9b5f3e69f9cb2f/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:48.452593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b6391-e5fe-498e-a6c2-89ca3e34f2a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e76576d714a04974abf66c621eda6164, user_id=d66d4222bd084b89be9b5f3e69f9cb2f, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:48.455714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b6391-e5fe-498e-a6c2-89ca3e34f2a2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:48.472274 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2039/4038] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:48 2022] PUT /identity/v3/projects/e76576d714a04974abf66c621eda6164/users/d66d4222bd084b89be9b5f3e69f9cb2f/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:48.485020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac69870-9c37-4849-9acc-7cc8bf7fc1a3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:48.485611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac69870-9c37-4849-9acc-7cc8bf7fc1a3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:48.486299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac69870-9c37-4849-9acc-7cc8bf7fc1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:48.519035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1ac69870-9c37-4849-9acc-7cc8bf7fc1a3 None None] MFA Rules not processed for user `d66d4222bd084b89be9b5f3e69f9cb2f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:48.520366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1ac69870-9c37-4849-9acc-7cc8bf7fc1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:48.593020 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2000/4039] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:48.605717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b6391-e5fe-498e-a6c2-89ca3e34f2a2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:48.607833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9f5b6391-e5fe-498e-a6c2-89ca3e34f2a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:48.666316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b6391-e5fe-498e-a6c2-89ca3e34f2a2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:48.669988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1247f-13c5-4966-b12a-01b3c51ec40b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:48.670618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1247f-13c5-4966-b12a-01b3c51ec40b service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:48.670888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1247f-13c5-4966-b12a-01b3c51ec40b service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:48.671149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1247f-13c5-4966-b12a-01b3c51ec40b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:48.679653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1247f-13c5-4966-b12a-01b3c51ec40b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:48.681124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1247f-13c5-4966-b12a-01b3c51ec40b service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:48.690116 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2040/4040] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:49.406775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf80f8-34d6-4e66-8523-74982a33cc22 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:49.407441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf80f8-34d6-4e66-8523-74982a33cc22 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:49.407892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf80f8-34d6-4e66-8523-74982a33cc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:49.432046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a6cf80f8-34d6-4e66-8523-74982a33cc22 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:49.433296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a6cf80f8-34d6-4e66-8523-74982a33cc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:49.499713 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2001/4041] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:07:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:49.520012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1247f-13c5-4966-b12a-01b3c51ec40b service glance] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:49.521587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5ca1247f-13c5-4966-b12a-01b3c51ec40b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:49.557049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:49.609961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1247f-13c5-4966-b12a-01b3c51ec40b service glance] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:49.618188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6cfb0-8d62-4e2d-8faf-72b815d7f254 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:49.618847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6cfb0-8d62-4e2d-8faf-72b815d7f254 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:49.619117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6cfb0-8d62-4e2d-8faf-72b815d7f254 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:49.619545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6cfb0-8d62-4e2d-8faf-72b815d7f254 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:49.628644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6cfb0-8d62-4e2d-8faf-72b815d7f254 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:49.630611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6cfb0-8d62-4e2d-8faf-72b815d7f254 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:49.639702 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2041/4042] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:07:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:07:51.038878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e395a9-ce4e-4fc9-a34a-295ece5e1573 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:51.039449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e395a9-ce4e-4fc9-a34a-295ece5e1573 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:51.039449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e395a9-ce4e-4fc9-a34a-295ece5e1573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:51.065137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-71e395a9-ce4e-4fc9-a34a-295ece5e1573 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:51.066666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-71e395a9-ce4e-4fc9-a34a-295ece5e1573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:51.074989 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2002/4043] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:07:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:51.089558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6cfb0-8d62-4e2d-8faf-72b815d7f254 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:51.102136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6cfb0-8d62-4e2d-8faf-72b815d7f254 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:51.108083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9f888-d650-4edf-8d59-2fb009fae329 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:51.108904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9f888-d650-4edf-8d59-2fb009fae329 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:51.109177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9f888-d650-4edf-8d59-2fb009fae329 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:51.109442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9f888-d650-4edf-8d59-2fb009fae329 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:51.126276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9f888-d650-4edf-8d59-2fb009fae329 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:51.129060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9f888-d650-4edf-8d59-2fb009fae329 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:51.138981 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2042/4044] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:07:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:07:51.249528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e395a9-ce4e-4fc9-a34a-295ece5e1573 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:51.256784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-71e395a9-ce4e-4fc9-a34a-295ece5e1573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:51.387661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e395a9-ce4e-4fc9-a34a-295ece5e1573 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:51.393153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83cb77-6f48-4647-a0e7-187dc5c3c370 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:51.394100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83cb77-6f48-4647-a0e7-187dc5c3c370 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:51.394565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83cb77-6f48-4647-a0e7-187dc5c3c370 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:51.394798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83cb77-6f48-4647-a0e7-187dc5c3c370 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:51.397390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9b83cb77-6f48-4647-a0e7-187dc5c3c370 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:51.409346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:51.431915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83cb77-6f48-4647-a0e7-187dc5c3c370 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:51.431915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83cb77-6f48-4647-a0e7-187dc5c3c370 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:51.450347 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2003/4045] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:51.520091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8816323-aeb3-44dd-8747-43671da82a11 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:51.521723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8816323-aeb3-44dd-8747-43671da82a11 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:51.522344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8816323-aeb3-44dd-8747-43671da82a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:51.546638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e8816323-aeb3-44dd-8747-43671da82a11 None None] MFA Rules not processed for user `f144c6cdd63d4ab6ac5669aba1a09fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:51.548598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e8816323-aeb3-44dd-8747-43671da82a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:51.642902 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2043/4046] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:52.120039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83cb77-6f48-4647-a0e7-187dc5c3c370 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:52.224710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83cb77-6f48-4647-a0e7-187dc5c3c370 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:52.234793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60662785-5f00-49c0-8cc3-7974c1996198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:52.235557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60662785-5f00-49c0-8cc3-7974c1996198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:52.235836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60662785-5f00-49c0-8cc3-7974c1996198 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:52.236104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60662785-5f00-49c0-8cc3-7974c1996198 admin admin] PATH_INFO: `/v3/users/d66d4222bd084b89be9b5f3e69f9cb2f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:52.237561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60662785-5f00-49c0-8cc3-7974c1996198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d66d4222bd084b89be9b5f3e69f9cb2f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:52.241552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60662785-5f00-49c0-8cc3-7974c1996198 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:52.334901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ced6-2fc8-4179-9400-ede96390df1e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:52.335963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ced6-2fc8-4179-9400-ede96390df1e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:52.336089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-60662785-5f00-49c0-8cc3-7974c1996198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd66d4222bd084b89be9b5f3e69f9cb2f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:52.336535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ced6-2fc8-4179-9400-ede96390df1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:52.356793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-60662785-5f00-49c0-8cc3-7974c1996198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd66d4222bd084b89be9b5f3e69f9cb2f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:52.364222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-aac7ced6-2fc8-4179-9400-ede96390df1e None None] MFA Rules not processed for user `f144c6cdd63d4ab6ac5669aba1a09fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:52.376999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-60662785-5f00-49c0-8cc3-7974c1996198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd66d4222bd084b89be9b5f3e69f9cb2f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:52.377650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aac7ced6-2fc8-4179-9400-ede96390df1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:52.378135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-60662785-5f00-49c0-8cc3-7974c1996198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd66d4222bd084b89be9b5f3e69f9cb2f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:52.379740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-60662785-5f00-49c0-8cc3-7974c1996198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd66d4222bd084b89be9b5f3e69f9cb2f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:52.402683 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2004/4047] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:52 2022] DELETE /identity/v3/users/d66d4222bd084b89be9b5f3e69f9cb2f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:52.403030 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2044/4048] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:07:52.408047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1441fb-c6d0-44ab-a408-ff6df357dea9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:52.408512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1441fb-c6d0-44ab-a408-ff6df357dea9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:52.408888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1441fb-c6d0-44ab-a408-ff6df357dea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:52.450085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bf1441fb-c6d0-44ab-a408-ff6df357dea9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:52.451562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bf1441fb-c6d0-44ab-a408-ff6df357dea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:52.531530 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2005/4049] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:52.543150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7ced6-2fc8-4179-9400-ede96390df1e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:52.544885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aac7ced6-2fc8-4179-9400-ede96390df1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:52.625303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7ced6-2fc8-4179-9400-ede96390df1e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:52.719052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e927d7b-dc3e-48c5-9e2a-0341f12d546f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:52.719794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e927d7b-dc3e-48c5-9e2a-0341f12d546f service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:52.722318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e927d7b-dc3e-48c5-9e2a-0341f12d546f service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:52.722694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e927d7b-dc3e-48c5-9e2a-0341f12d546f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:52.762413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e927d7b-dc3e-48c5-9e2a-0341f12d546f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:52.771564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e927d7b-dc3e-48c5-9e2a-0341f12d546f service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:52.793098 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2045/4050] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:07:53.501897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1441fb-c6d0-44ab-a408-ff6df357dea9 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:53.503732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bf1441fb-c6d0-44ab-a408-ff6df357dea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:53.519499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1441fb-c6d0-44ab-a408-ff6df357dea9 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:53.527747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938db48-311f-4023-a46a-610b406b5705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:53.531203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938db48-311f-4023-a46a-610b406b5705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:53.543936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938db48-311f-4023-a46a-610b406b5705 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:53.544645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938db48-311f-4023-a46a-610b406b5705 admin admin] PATH_INFO: `/v3/projects/e76576d714a04974abf66c621eda6164` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:53.546677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938db48-311f-4023-a46a-610b406b5705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e76576d714a04974abf66c621eda6164)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:53.549382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938db48-311f-4023-a46a-610b406b5705 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:53.573663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:53.631254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6938db48-311f-4023-a46a-610b406b5705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e76576d714a04974abf66c621eda6164', 'request_id': 'req-6938db48-311f-4023-a46a-610b406b5705'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:53.644006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6938db48-311f-4023-a46a-610b406b5705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e76576d714a04974abf66c621eda6164', 'request_id': 'req-6938db48-311f-4023-a46a-610b406b5705'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:53.656774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6938db48-311f-4023-a46a-610b406b5705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e76576d714a04974abf66c621eda6164', 'request_id': 'req-6938db48-311f-4023-a46a-610b406b5705'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:53.664952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e927d7b-dc3e-48c5-9e2a-0341f12d546f service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:53.666629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6e927d7b-dc3e-48c5-9e2a-0341f12d546f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:53.683149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6938db48-311f-4023-a46a-610b406b5705 admin admin] The token cache is being invalidate because project e76576d714a04974abf66c621eda6164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:53.683685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6938db48-311f-4023-a46a-610b406b5705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:53.684874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6938db48-311f-4023-a46a-610b406b5705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:53.685939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:53.687329 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2006/4051] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:53 2022] DELETE /identity/v3/projects/e76576d714a04974abf66c621eda6164 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:53.691975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db55a6b-ece3-48ff-aaf1-2d62a7c629dc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:53.692571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db55a6b-ece3-48ff-aaf1-2d62a7c629dc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:53.692991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db55a6b-ece3-48ff-aaf1-2d62a7c629dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:53.693725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e927d7b-dc3e-48c5-9e2a-0341f12d546f service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:53.719557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1db55a6b-ece3-48ff-aaf1-2d62a7c629dc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:53.721621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1db55a6b-ece3-48ff-aaf1-2d62a7c629dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:53.784009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c92fcc-45e3-4a92-b0af-68703ca2c6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:53.784757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c92fcc-45e3-4a92-b0af-68703ca2c6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:53.788748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c92fcc-45e3-4a92-b0af-68703ca2c6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:53.789083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c92fcc-45e3-4a92-b0af-68703ca2c6f3 admin admin] PATH_INFO: `/v3/users/30c7e3ac41df480ea0c198af4f0c49ea` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:53.792785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c92fcc-45e3-4a92-b0af-68703ca2c6f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c7e3ac41df480ea0c198af4f0c49ea)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:53.795180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c92fcc-45e3-4a92-b0af-68703ca2c6f3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:53.830555 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2007/4052] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:07:53.839354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c70b96-f032-44ab-a7b7-0dba36730adb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:53.839953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c70b96-f032-44ab-a7b7-0dba36730adb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:53.840637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c70b96-f032-44ab-a7b7-0dba36730adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:53.875403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-56c70b96-f032-44ab-a7b7-0dba36730adb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:53.876852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-56c70b96-f032-44ab-a7b7-0dba36730adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:53.903748 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2008/4053] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:53.906893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c70b96-f032-44ab-a7b7-0dba36730adb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:53.912359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-77c92fcc-45e3-4a92-b0af-68703ca2c6f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c7e3ac41df480ea0c198af4f0c49ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:53.925823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-77c92fcc-45e3-4a92-b0af-68703ca2c6f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c7e3ac41df480ea0c198af4f0c49ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:53.932206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c70b96-f032-44ab-a7b7-0dba36730adb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:53.942327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-77c92fcc-45e3-4a92-b0af-68703ca2c6f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c7e3ac41df480ea0c198af4f0c49ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:53.948145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080583c-dee1-417d-8ace-ce33e9ee9b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:53.949677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080583c-dee1-417d-8ace-ce33e9ee9b4c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:53.950298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080583c-dee1-417d-8ace-ce33e9ee9b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:53.952187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080583c-dee1-417d-8ace-ce33e9ee9b4c admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:53.953055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080583c-dee1-417d-8ace-ce33e9ee9b4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:07:53.953546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080583c-dee1-417d-8ace-ce33e9ee9b4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:53.956062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080583c-dee1-417d-8ace-ce33e9ee9b4c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:53.971804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-77c92fcc-45e3-4a92-b0af-68703ca2c6f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c7e3ac41df480ea0c198af4f0c49ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:53.972841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-77c92fcc-45e3-4a92-b0af-68703ca2c6f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c7e3ac41df480ea0c198af4f0c49ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:53.980133 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2009/4054] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:07:53.987185 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2046/4055] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:07:53 2022] DELETE /identity/v3/users/30c7e3ac41df480ea0c198af4f0c49ea => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:53.987905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080583c-dee1-417d-8ace-ce33e9ee9b4c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:53.990768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5080583c-dee1-417d-8ace-ce33e9ee9b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:54.005422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080583c-dee1-417d-8ace-ce33e9ee9b4c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:54.129076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a773ac0-fd50-496f-a69f-973b5d5e8250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:54.130682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a773ac0-fd50-496f-a69f-973b5d5e8250 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:54.130682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a773ac0-fd50-496f-a69f-973b5d5e8250 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:54.131202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a773ac0-fd50-496f-a69f-973b5d5e8250 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:54.131919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a773ac0-fd50-496f-a69f-973b5d5e8250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:54.134034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a773ac0-fd50-496f-a69f-973b5d5e8250 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:54.181186 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2010/4056] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:53 2022] POST /identity/v3/projects => generated 363 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:54.185295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c92fcc-45e3-4a92-b0af-68703ca2c6f3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:54.199387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c92fcc-45e3-4a92-b0af-68703ca2c6f3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:54.274762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a773ac0-fd50-496f-a69f-973b5d5e8250 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:54.277127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a773ac0-fd50-496f-a69f-973b5d5e8250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:54.286161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:54.296999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad8c55-4086-415e-9b9e-0bc75871f8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:54.297816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad8c55-4086-415e-9b9e-0bc75871f8da admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:54.298104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad8c55-4086-415e-9b9e-0bc75871f8da admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:54.298421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad8c55-4086-415e-9b9e-0bc75871f8da admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:54.302984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad8c55-4086-415e-9b9e-0bc75871f8da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:54.308206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad8c55-4086-415e-9b9e-0bc75871f8da admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:54.377584 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2047/4057] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:54 2022] POST /identity/v3/users => generated 282 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:07:54.379521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad8c55-4086-415e-9b9e-0bc75871f8da admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:54.406300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad8c55-4086-415e-9b9e-0bc75871f8da admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:54.424841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2c432-4492-4e0a-9a20-281fd9bd09ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:54.425565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2c432-4492-4e0a-9a20-281fd9bd09ff admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:54.428236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2c432-4492-4e0a-9a20-281fd9bd09ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:54.428643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2c432-4492-4e0a-9a20-281fd9bd09ff admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:54.429473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2c432-4492-4e0a-9a20-281fd9bd09ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:54.433440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2c432-4492-4e0a-9a20-281fd9bd09ff admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:54.447858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a773ac0-fd50-496f-a69f-973b5d5e8250 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:54.451848 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2048/4058] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:54.453791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2c432-4492-4e0a-9a20-281fd9bd09ff admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:54.461382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:54.462918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:54.467974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2c432-4492-4e0a-9a20-281fd9bd09ff admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:54.475329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13973435-0948-46de-8f0e-b230bcf3f6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:54.476315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13973435-0948-46de-8f0e-b230bcf3f6ad admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:54.476794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13973435-0948-46de-8f0e-b230bcf3f6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:54.478043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:54.478177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13973435-0948-46de-8f0e-b230bcf3f6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:54.478887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13973435-0948-46de-8f0e-b230bcf3f6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:54.480898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13973435-0948-46de-8f0e-b230bcf3f6ad admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:54.481183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:54.482219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b admin admin] PATH_INFO: `/v3/projects/7a13c82fcbb542a19af3a42b48f81958` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:54.487440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a13c82fcbb542a19af3a42b48f81958)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:54.490659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:54.513152 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2049/4059] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:54.515391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13973435-0948-46de-8f0e-b230bcf3f6ad admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:54.533289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13973435-0948-46de-8f0e-b230bcf3f6ad admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:54.539530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964238df-217a-4f6f-a1dc-aac9acddd302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:54.540338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964238df-217a-4f6f-a1dc-aac9acddd302 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:54.540737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964238df-217a-4f6f-a1dc-aac9acddd302 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:54.541214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964238df-217a-4f6f-a1dc-aac9acddd302 admin admin] PATH_INFO: `/v3/projects/fe81ef61979047048c9621b86f7b8b51/users/7dcf48847403418cb0b856126896af94/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:54.561914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964238df-217a-4f6f-a1dc-aac9acddd302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe81ef61979047048c9621b86f7b8b51, user_id=7dcf48847403418cb0b856126896af94, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:54.565103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964238df-217a-4f6f-a1dc-aac9acddd302 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:54.578126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a13c82fcbb542a19af3a42b48f81958', 'request_id': 'req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:54.597192 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2050/4060] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:54 2022] PUT /identity/v3/projects/fe81ef61979047048c9621b86f7b8b51/users/7dcf48847403418cb0b856126896af94/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:07:54.597544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a13c82fcbb542a19af3a42b48f81958', 'request_id': 'req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:54.601479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00e8a4-5c2c-4c49-ab76-0ecb4f9ce1c6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:54.602124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00e8a4-5c2c-4c49-ab76-0ecb4f9ce1c6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:54.602604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00e8a4-5c2c-4c49-ab76-0ecb4f9ce1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:54.620074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a13c82fcbb542a19af3a42b48f81958', 'request_id': 'req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:54.660046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b admin admin] The token cache is being invalidate because project 7a13c82fcbb542a19af3a42b48f81958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:07:54.662244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:54.663079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ef4c724-62f3-4cc1-b7c8-ea90a84f582b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:07:54.669891 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2011/4061] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:07:54 2022] DELETE /identity/v3/projects/7a13c82fcbb542a19af3a42b48f81958 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:54.671499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed660886-c928-422d-b301-438d6c3926e6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:54.672082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed660886-c928-422d-b301-438d6c3926e6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:54.672476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed660886-c928-422d-b301-438d6c3926e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:54.678967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fa00e8a4-5c2c-4c49-ab76-0ecb4f9ce1c6 None None] MFA Rules not processed for user `7dcf48847403418cb0b856126896af94`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:54.687544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fa00e8a4-5c2c-4c49-ab76-0ecb4f9ce1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:54.704592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ed660886-c928-422d-b301-438d6c3926e6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:54.706244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ed660886-c928-422d-b301-438d6c3926e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:54.819277 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2012/4062] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:54.821808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b44959-20e1-42d2-b603-e72bbb8f9458 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:54.822349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b44959-20e1-42d2-b603-e72bbb8f9458 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:54.822825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b44959-20e1-42d2-b603-e72bbb8f9458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:54.823997 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2051/4063] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:54.840920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00e8a4-5c2c-4c49-ab76-0ecb4f9ce1c6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:54.844090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fa00e8a4-5c2c-4c49-ab76-0ecb4f9ce1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:54.850219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-41b44959-20e1-42d2-b603-e72bbb8f9458 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:54.851741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-41b44959-20e1-42d2-b603-e72bbb8f9458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:54.866765 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2013/4064] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:54.867856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b44959-20e1-42d2-b603-e72bbb8f9458 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:54.878719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b44959-20e1-42d2-b603-e72bbb8f9458 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:54.885704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b3572-8748-4b0e-84af-e84e1b46f688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:54.886446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3572-8748-4b0e-84af-e84e1b46f688 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:54.886795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3572-8748-4b0e-84af-e84e1b46f688 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:54.887271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3572-8748-4b0e-84af-e84e1b46f688 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:54.888002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b3572-8748-4b0e-84af-e84e1b46f688 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:07:54.888366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b3572-8748-4b0e-84af-e84e1b46f688 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:54.890197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b3572-8748-4b0e-84af-e84e1b46f688 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:54.906842 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2014/4065] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:07:54.912995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b3572-8748-4b0e-84af-e84e1b46f688 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:54.925083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b3572-8748-4b0e-84af-e84e1b46f688 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:54.930581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83903b67-635d-4ee5-b61a-73737b929101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:54.931386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83903b67-635d-4ee5-b61a-73737b929101 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:54.931877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00e8a4-5c2c-4c49-ab76-0ecb4f9ce1c6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:54.932034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83903b67-635d-4ee5-b61a-73737b929101 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:54.932331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83903b67-635d-4ee5-b61a-73737b929101 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:54.933548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83903b67-635d-4ee5-b61a-73737b929101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:54.935683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83903b67-635d-4ee5-b61a-73737b929101 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:54.937058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843448db-1003-4eca-9d93-064eeeefb9ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:54.937914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843448db-1003-4eca-9d93-064eeeefb9ca service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:54.938680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843448db-1003-4eca-9d93-064eeeefb9ca service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:54.939460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843448db-1003-4eca-9d93-064eeeefb9ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:54.953655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843448db-1003-4eca-9d93-064eeeefb9ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:54.956778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843448db-1003-4eca-9d93-064eeeefb9ca service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:54.957609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:54.987080 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2015/4066] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:07:54 2022] POST /identity/v3/projects => generated 367 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:54.991593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83903b67-635d-4ee5-b61a-73737b929101 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:54.993761 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2052/4067] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:07:55.082494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83903b67-635d-4ee5-b61a-73737b929101 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:55.090510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa60c86-8bb7-4bde-bb4a-236b636de8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:55.091167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa60c86-8bb7-4bde-bb4a-236b636de8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:55.091465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa60c86-8bb7-4bde-bb4a-236b636de8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:55.091946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa60c86-8bb7-4bde-bb4a-236b636de8d2 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:55.092754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa60c86-8bb7-4bde-bb4a-236b636de8d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:55.094693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa60c86-8bb7-4bde-bb4a-236b636de8d2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:55.122004 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2016/4068] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:07:54 2022] POST /identity/v3/users => generated 284 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:07:55.126993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843448db-1003-4eca-9d93-064eeeefb9ca service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:55.143507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843448db-1003-4eca-9d93-064eeeefb9ca service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:55.150918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7b1e1-8c45-4a66-8e93-8a55dfc68bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:55.151564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7b1e1-8c45-4a66-8e93-8a55dfc68bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:55.151932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7b1e1-8c45-4a66-8e93-8a55dfc68bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:55.152195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7b1e1-8c45-4a66-8e93-8a55dfc68bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:55.152750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7b1e1-8c45-4a66-8e93-8a55dfc68bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:55.154346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7b1e1-8c45-4a66-8e93-8a55dfc68bf0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:55.169035 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2053/4069] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:55.170390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa60c86-8bb7-4bde-bb4a-236b636de8d2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:55.182439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa60c86-8bb7-4bde-bb4a-236b636de8d2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:55.188478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572902a-63a9-4739-8eb8-f7ee3215e090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:55.189110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572902a-63a9-4739-8eb8-f7ee3215e090 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:55.189388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572902a-63a9-4739-8eb8-f7ee3215e090 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:55.189747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572902a-63a9-4739-8eb8-f7ee3215e090 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:55.190404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572902a-63a9-4739-8eb8-f7ee3215e090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:55.192074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572902a-63a9-4739-8eb8-f7ee3215e090 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:55.205988 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2017/4070] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:07:55.211166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7b1e1-8c45-4a66-8e93-8a55dfc68bf0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:55.226212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7b1e1-8c45-4a66-8e93-8a55dfc68bf0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:55.231553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab80fd-59dc-45dd-a636-9f5d44856fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:55.232661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab80fd-59dc-45dd-a636-9f5d44856fa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:55.233040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab80fd-59dc-45dd-a636-9f5d44856fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:55.233399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab80fd-59dc-45dd-a636-9f5d44856fa5 admin admin] PATH_INFO: `/v3/projects/77c499d91be4454db2371c4ec99ce38c/users/43dcc039e9d0442bb96ae9c0725a4a76/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:55.252560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab80fd-59dc-45dd-a636-9f5d44856fa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77c499d91be4454db2371c4ec99ce38c, user_id=43dcc039e9d0442bb96ae9c0725a4a76, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:55.255923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab80fd-59dc-45dd-a636-9f5d44856fa5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:55.273978 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2054/4071] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:07:55 2022] PUT /identity/v3/projects/77c499d91be4454db2371c4ec99ce38c/users/43dcc039e9d0442bb96ae9c0725a4a76/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:07:55.285751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991aa5d-dda5-46b7-b83b-2012052d31b4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:55.286547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991aa5d-dda5-46b7-b83b-2012052d31b4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:55.287018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991aa5d-dda5-46b7-b83b-2012052d31b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:55.322689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8991aa5d-dda5-46b7-b83b-2012052d31b4 None None] MFA Rules not processed for user `43dcc039e9d0442bb96ae9c0725a4a76`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:55.324756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8991aa5d-dda5-46b7-b83b-2012052d31b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:55.438570 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2018/4072] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:55.452616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d26784-ed8a-4892-98ca-eda55d5e8b29 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:55.452951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d26784-ed8a-4892-98ca-eda55d5e8b29 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:55.453131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d26784-ed8a-4892-98ca-eda55d5e8b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:55.475892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-36d26784-ed8a-4892-98ca-eda55d5e8b29 None None] MFA Rules not processed for user `43dcc039e9d0442bb96ae9c0725a4a76`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:07:55.477552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-36d26784-ed8a-4892-98ca-eda55d5e8b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:55.495310 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2055/4073] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:07:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:07:55.506602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991aa5d-dda5-46b7-b83b-2012052d31b4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:07:55.508501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8991aa5d-dda5-46b7-b83b-2012052d31b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:07:55.604256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:07:55.626253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991aa5d-dda5-46b7-b83b-2012052d31b4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:07:55.638594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca80b1-e5be-48bb-8f4b-e0ec749438d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:07:55.640769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca80b1-e5be-48bb-8f4b-e0ec749438d2 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:07:55.641119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca80b1-e5be-48bb-8f4b-e0ec749438d2 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:07:55.641398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca80b1-e5be-48bb-8f4b-e0ec749438d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:07:55.653074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca80b1-e5be-48bb-8f4b-e0ec749438d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:07:55.655254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca80b1-e5be-48bb-8f4b-e0ec749438d2 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:07:55.693318 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2019/4074] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:07:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:08:00.412962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d26784-ed8a-4892-98ca-eda55d5e8b29 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:00.415299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-36d26784-ed8a-4892-98ca-eda55d5e8b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:00.448791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:00.514842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d26784-ed8a-4892-98ca-eda55d5e8b29 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:00.519593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:00.520325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:00.520698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:00.521029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] PATH_INFO: `/v3/users/7dcf48847403418cb0b856126896af94` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:00.522264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dcf48847403418cb0b856126896af94)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:00.524323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:00.584661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dcf48847403418cb0b856126896af94'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:00.584661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dcf48847403418cb0b856126896af94'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:00.589469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dcf48847403418cb0b856126896af94'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:00.604427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dcf48847403418cb0b856126896af94'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:00.605088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dcf48847403418cb0b856126896af94'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:00.611140 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2056/4075] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:08:00 2022] DELETE /identity/v3/users/7dcf48847403418cb0b856126896af94 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:00.615873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb5080-a803-404f-bcee-489231e83a49 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:00.616458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb5080-a803-404f-bcee-489231e83a49 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:00.616617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb5080-a803-404f-bcee-489231e83a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:00.636963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d0fb5080-a803-404f-bcee-489231e83a49 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:00.638089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d0fb5080-a803-404f-bcee-489231e83a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:00.703034 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2020/4076] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:00.713206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:00.714647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:00.778138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840e1b6-67ce-4a07-8721-f796e3c5804c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:00.781807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76feb0-5e1d-485c-8c4e-9665105de25a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:00.782449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76feb0-5e1d-485c-8c4e-9665105de25a service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:00.782722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76feb0-5e1d-485c-8c4e-9665105de25a service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:00.782960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76feb0-5e1d-485c-8c4e-9665105de25a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:00.794306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76feb0-5e1d-485c-8c4e-9665105de25a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:00.795761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76feb0-5e1d-485c-8c4e-9665105de25a service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:00.809915 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2057/4077] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:08:01.178988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb5080-a803-404f-bcee-489231e83a49 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:01.181720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d0fb5080-a803-404f-bcee-489231e83a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:01.220970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb5080-a803-404f-bcee-489231e83a49 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:01.223919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c95cac-5387-4025-8934-79713a27388b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:01.224778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c95cac-5387-4025-8934-79713a27388b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:01.225234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c95cac-5387-4025-8934-79713a27388b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:01.225842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c95cac-5387-4025-8934-79713a27388b admin admin] PATH_INFO: `/v3/projects/fe81ef61979047048c9621b86f7b8b51` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:01.227175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c95cac-5387-4025-8934-79713a27388b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe81ef61979047048c9621b86f7b8b51)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:01.229761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c95cac-5387-4025-8934-79713a27388b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:01.316718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4c95cac-5387-4025-8934-79713a27388b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe81ef61979047048c9621b86f7b8b51', 'request_id': 'req-f4c95cac-5387-4025-8934-79713a27388b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:01.323403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4c95cac-5387-4025-8934-79713a27388b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe81ef61979047048c9621b86f7b8b51', 'request_id': 'req-f4c95cac-5387-4025-8934-79713a27388b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:01.337279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4c95cac-5387-4025-8934-79713a27388b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe81ef61979047048c9621b86f7b8b51', 'request_id': 'req-f4c95cac-5387-4025-8934-79713a27388b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:01.355920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4c95cac-5387-4025-8934-79713a27388b admin admin] The token cache is being invalidate because project fe81ef61979047048c9621b86f7b8b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:01.356111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4c95cac-5387-4025-8934-79713a27388b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:01.356918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4c95cac-5387-4025-8934-79713a27388b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:01.363251 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2021/4078] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:08:01 2022] DELETE /identity/v3/projects/fe81ef61979047048c9621b86f7b8b51 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:01.365142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b781ae-63aa-4552-bff1-3f6e3ae98de8 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:01.365255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b781ae-63aa-4552-bff1-3f6e3ae98de8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:01.365873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b781ae-63aa-4552-bff1-3f6e3ae98de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:01.386541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-65b781ae-63aa-4552-bff1-3f6e3ae98de8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:01.387902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-65b781ae-63aa-4552-bff1-3f6e3ae98de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:01.430088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:01.463892 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2058/4079] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:01.472603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b963f82-d498-454c-b202-2e277dd4e123 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:01.472882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b963f82-d498-454c-b202-2e277dd4e123 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:01.473154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b963f82-d498-454c-b202-2e277dd4e123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:01.500060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9b963f82-d498-454c-b202-2e277dd4e123 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:01.501311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9b963f82-d498-454c-b202-2e277dd4e123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:01.511491 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2022/4080] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:01.515810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b781ae-63aa-4552-bff1-3f6e3ae98de8 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:01.538455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b781ae-63aa-4552-bff1-3f6e3ae98de8 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:01.556746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78d958-af85-4248-9807-40e501ed731f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:01.557763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78d958-af85-4248-9807-40e501ed731f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:01.558560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78d958-af85-4248-9807-40e501ed731f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:01.559005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78d958-af85-4248-9807-40e501ed731f admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:01.560097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78d958-af85-4248-9807-40e501ed731f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:08:01.560609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78d958-af85-4248-9807-40e501ed731f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:01.562662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78d958-af85-4248-9807-40e501ed731f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:01.585190 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2059/4081] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:01.589957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b963f82-d498-454c-b202-2e277dd4e123 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:01.606412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b963f82-d498-454c-b202-2e277dd4e123 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:01.611367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1db5e-125c-4dd8-9a06-b7dedbcc4537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:01.611941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1db5e-125c-4dd8-9a06-b7dedbcc4537 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:01.612214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1db5e-125c-4dd8-9a06-b7dedbcc4537 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:01.613358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1db5e-125c-4dd8-9a06-b7dedbcc4537 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:01.614011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1db5e-125c-4dd8-9a06-b7dedbcc4537 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:01.617397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1db5e-125c-4dd8-9a06-b7dedbcc4537 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:01.658069 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2023/4082] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:08:01 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:01.664582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78d958-af85-4248-9807-40e501ed731f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:01.687877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78d958-af85-4248-9807-40e501ed731f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:01.767564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e409c3-b2a5-4ce1-8b58-cfa7c4cd494b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:01.768318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e409c3-b2a5-4ce1-8b58-cfa7c4cd494b admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:01.768634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e409c3-b2a5-4ce1-8b58-cfa7c4cd494b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:01.768900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e409c3-b2a5-4ce1-8b58-cfa7c4cd494b admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:01.769537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e409c3-b2a5-4ce1-8b58-cfa7c4cd494b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:01.771465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e409c3-b2a5-4ce1-8b58-cfa7c4cd494b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:01.798094 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2060/4083] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:08:01 2022] POST /identity/v3/users => generated 289 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:01.802602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1db5e-125c-4dd8-9a06-b7dedbcc4537 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:01.813090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1db5e-125c-4dd8-9a06-b7dedbcc4537 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:01.817725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636ccc6-b34b-4750-9263-fc6a84b4ecd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:01.818321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636ccc6-b34b-4750-9263-fc6a84b4ecd8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:01.818643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636ccc6-b34b-4750-9263-fc6a84b4ecd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:01.818925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636ccc6-b34b-4750-9263-fc6a84b4ecd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:01.819571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636ccc6-b34b-4750-9263-fc6a84b4ecd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:01.821560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636ccc6-b34b-4750-9263-fc6a84b4ecd8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:01.832845 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2024/4084] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:01.837342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e409c3-b2a5-4ce1-8b58-cfa7c4cd494b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:01.848940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e409c3-b2a5-4ce1-8b58-cfa7c4cd494b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:01.854215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bd901-b58c-492d-8def-18a02d31bd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:01.854870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd901-b58c-492d-8def-18a02d31bd09 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:01.855148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd901-b58c-492d-8def-18a02d31bd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:01.855522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd901-b58c-492d-8def-18a02d31bd09 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:01.856176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bd901-b58c-492d-8def-18a02d31bd09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:01.857822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bd901-b58c-492d-8def-18a02d31bd09 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:01.869102 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2061/4085] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:01.873435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636ccc6-b34b-4750-9263-fc6a84b4ecd8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:01.884047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636ccc6-b34b-4750-9263-fc6a84b4ecd8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:01.888996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8d500-5c51-4e6b-b406-346e21338d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:01.889840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8d500-5c51-4e6b-b406-346e21338d85 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:01.890109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8d500-5c51-4e6b-b406-346e21338d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:01.890369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8d500-5c51-4e6b-b406-346e21338d85 admin admin] PATH_INFO: `/v3/projects/a39eb787a51d4029852861a1d0bba7d9/users/d29a50a0da47413eb649d3c63ba3c213/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:01.908277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8d500-5c51-4e6b-b406-346e21338d85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a39eb787a51d4029852861a1d0bba7d9, user_id=d29a50a0da47413eb649d3c63ba3c213, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:01.910960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8d500-5c51-4e6b-b406-346e21338d85 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:01.925062 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2025/4086] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:08:01 2022] PUT /identity/v3/projects/a39eb787a51d4029852861a1d0bba7d9/users/d29a50a0da47413eb649d3c63ba3c213/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:01.930624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdee58b-b27e-45d1-af38-660c52f37849 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:01.930903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdee58b-b27e-45d1-af38-660c52f37849 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:01.931170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdee58b-b27e-45d1-af38-660c52f37849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:01.951451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2fdee58b-b27e-45d1-af38-660c52f37849 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:01.952676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2fdee58b-b27e-45d1-af38-660c52f37849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:02.014964 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2062/4087] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:08:02.026367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8d500-5c51-4e6b-b406-346e21338d85 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:02.028505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b3c8d500-5c51-4e6b-b406-346e21338d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:02.038923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:02.095332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8d500-5c51-4e6b-b406-346e21338d85 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:02.099741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac796f6-7d11-4667-9431-b997bc8e44c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:02.100401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac796f6-7d11-4667-9431-b997bc8e44c1 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:02.100716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac796f6-7d11-4667-9431-b997bc8e44c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:02.100995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac796f6-7d11-4667-9431-b997bc8e44c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:02.112155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac796f6-7d11-4667-9431-b997bc8e44c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:02.114191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac796f6-7d11-4667-9431-b997bc8e44c1 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:02.125894 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2026/4088] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:08:09.861867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3c78f-657f-4935-8e89-134e974b299f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:09.862393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3c78f-657f-4935-8e89-134e974b299f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:09.862393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3c78f-657f-4935-8e89-134e974b299f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:09.934552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-03a3c78f-657f-4935-8e89-134e974b299f None None] MFA Rules not processed for user `d29a50a0da47413eb649d3c63ba3c213`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:09.936448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-03a3c78f-657f-4935-8e89-134e974b299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:10.028968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:10.091545 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2063/4089] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:10.097429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac796f6-7d11-4667-9431-b997bc8e44c1 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:10.109868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac796f6-7d11-4667-9431-b997bc8e44c1 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:10.115616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ae728-f01f-4fb4-9976-813d0480116b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:10.117440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ae728-f01f-4fb4-9976-813d0480116b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:10.117983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ae728-f01f-4fb4-9976-813d0480116b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:10.118595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ae728-f01f-4fb4-9976-813d0480116b admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:10.119448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ae728-f01f-4fb4-9976-813d0480116b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:10.121653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ae728-f01f-4fb4-9976-813d0480116b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:10.161849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3c78f-657f-4935-8e89-134e974b299f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:10.163763 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2027/4090] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:08:10 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:10.176891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3c78f-657f-4935-8e89-134e974b299f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:10.250876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35094536-b95a-401c-98f6-dbb80aa358d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:10.251506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35094536-b95a-401c-98f6-dbb80aa358d0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:10.251903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35094536-b95a-401c-98f6-dbb80aa358d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:10.252043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35094536-b95a-401c-98f6-dbb80aa358d0 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:10.252760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35094536-b95a-401c-98f6-dbb80aa358d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:10.254769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35094536-b95a-401c-98f6-dbb80aa358d0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:10.290711 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2064/4091] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:08:10 2022] POST /identity/v3/users => generated 295 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:08:10.292228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ae728-f01f-4fb4-9976-813d0480116b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:10.310602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ae728-f01f-4fb4-9976-813d0480116b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:10.316468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382172d-aaca-46a2-962b-69975fdc608c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:10.317018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382172d-aaca-46a2-962b-69975fdc608c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:10.317292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382172d-aaca-46a2-962b-69975fdc608c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:10.317559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382172d-aaca-46a2-962b-69975fdc608c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:10.318287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382172d-aaca-46a2-962b-69975fdc608c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:10.320082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382172d-aaca-46a2-962b-69975fdc608c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:10.335984 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2028/4092] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:10.340519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35094536-b95a-401c-98f6-dbb80aa358d0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:10.355225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35094536-b95a-401c-98f6-dbb80aa358d0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:10.359891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d3bb6-f1e6-4667-b7a9-5464c78c65da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:10.361237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d3bb6-f1e6-4667-b7a9-5464c78c65da admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:10.361513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d3bb6-f1e6-4667-b7a9-5464c78c65da admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:10.361780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d3bb6-f1e6-4667-b7a9-5464c78c65da admin admin] PATH_INFO: `/v3/projects/a704089fa75c4f008577d83aee1a6d99/users/c93bdcb41270436ea67cd0aa05984b88/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:10.378198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d3bb6-f1e6-4667-b7a9-5464c78c65da admin admin] RBAC: Authorizing `identity:create_grant(project_id=a704089fa75c4f008577d83aee1a6d99, user_id=c93bdcb41270436ea67cd0aa05984b88, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:10.380976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d3bb6-f1e6-4667-b7a9-5464c78c65da admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:10.395542 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2065/4093] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:08:10 2022] PUT /identity/v3/projects/a704089fa75c4f008577d83aee1a6d99/users/c93bdcb41270436ea67cd0aa05984b88/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:10.399550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382172d-aaca-46a2-962b-69975fdc608c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:10.409583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382172d-aaca-46a2-962b-69975fdc608c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:10.474815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f34568-df48-4a9c-913d-bcf4e744f4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:10.475719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f34568-df48-4a9c-913d-bcf4e744f4bb admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:10.476409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f34568-df48-4a9c-913d-bcf4e744f4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:10.476535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f34568-df48-4a9c-913d-bcf4e744f4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:10.477086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f34568-df48-4a9c-913d-bcf4e744f4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:10.478796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f34568-df48-4a9c-913d-bcf4e744f4bb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:10.490461 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2029/4094] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:10.495764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d3bb6-f1e6-4667-b7a9-5464c78c65da admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:10.507137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d3bb6-f1e6-4667-b7a9-5464c78c65da admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:10.512396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7a329-6972-44d1-8e5e-f1998bc27db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:10.513088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7a329-6972-44d1-8e5e-f1998bc27db6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:10.513358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7a329-6972-44d1-8e5e-f1998bc27db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:10.513622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7a329-6972-44d1-8e5e-f1998bc27db6 admin admin] PATH_INFO: `/v3/projects/a704089fa75c4f008577d83aee1a6d99/users/c93bdcb41270436ea67cd0aa05984b88/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:10.515516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7a329-6972-44d1-8e5e-f1998bc27db6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a704089fa75c4f008577d83aee1a6d99, user_id=c93bdcb41270436ea67cd0aa05984b88, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:10.518177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7a329-6972-44d1-8e5e-f1998bc27db6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:10.530684 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2066/4095] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:08:10 2022] PUT /identity/v3/projects/a704089fa75c4f008577d83aee1a6d99/users/c93bdcb41270436ea67cd0aa05984b88/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:08:12.706051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f34568-df48-4a9c-913d-bcf4e744f4bb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:12.711598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a2f34568-df48-4a9c-913d-bcf4e744f4bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:12.763843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f34568-df48-4a9c-913d-bcf4e744f4bb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:12.908172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0d7f6-cc39-451f-99bd-161ed689197a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:12.909014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0d7f6-cc39-451f-99bd-161ed689197a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:12.921855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0d7f6-cc39-451f-99bd-161ed689197a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:12.925963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0d7f6-cc39-451f-99bd-161ed689197a admin admin] PATH_INFO: `/v3/users/b20aea7088dc4e4fadb737c906cf7ea2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:12.927991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0d7f6-cc39-451f-99bd-161ed689197a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b20aea7088dc4e4fadb737c906cf7ea2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:12.933477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0d7f6-cc39-451f-99bd-161ed689197a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:12.968409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:13.028883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f2c0d7f6-cc39-451f-99bd-161ed689197a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b20aea7088dc4e4fadb737c906cf7ea2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:13.058545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f2c0d7f6-cc39-451f-99bd-161ed689197a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b20aea7088dc4e4fadb737c906cf7ea2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:13.076040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f2c0d7f6-cc39-451f-99bd-161ed689197a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b20aea7088dc4e4fadb737c906cf7ea2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:13.076807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f2c0d7f6-cc39-451f-99bd-161ed689197a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b20aea7088dc4e4fadb737c906cf7ea2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:13.077477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f2c0d7f6-cc39-451f-99bd-161ed689197a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b20aea7088dc4e4fadb737c906cf7ea2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:13.093196 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2030/4096] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:08:12 2022] DELETE /identity/v3/users/b20aea7088dc4e4fadb737c906cf7ea2 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:13.359922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7a329-6972-44d1-8e5e-f1998bc27db6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:13.361274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-13b7a329-6972-44d1-8e5e-f1998bc27db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:13.395982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7a329-6972-44d1-8e5e-f1998bc27db6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:13.504303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:13.508307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:13.508717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:13.508993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff admin admin] PATH_INFO: `/v3/projects/a4f9c633b6d54ac1869108eadb004a87` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:13.510311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f9c633b6d54ac1869108eadb004a87)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:13.512818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:13.610338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:13.631701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f9c633b6d54ac1869108eadb004a87', 'request_id': 'req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:13.655989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f9c633b6d54ac1869108eadb004a87', 'request_id': 'req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:13.667886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f9c633b6d54ac1869108eadb004a87', 'request_id': 'req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:13.700729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff admin admin] The token cache is being invalidate because project a4f9c633b6d54ac1869108eadb004a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:13.701019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:13.701692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-46bb9be7-7ef6-4de3-a156-b6a2ad84e1ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:13.704294 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2067/4097] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:08:13 2022] DELETE /identity/v3/projects/a4f9c633b6d54ac1869108eadb004a87 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:13.708800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab85612-1552-4db5-b279-b6f54d60272c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:13.709382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab85612-1552-4db5-b279-b6f54d60272c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:13.709570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab85612-1552-4db5-b279-b6f54d60272c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:13.733630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fab85612-1552-4db5-b279-b6f54d60272c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:13.735109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fab85612-1552-4db5-b279-b6f54d60272c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:13.853809 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2031/4098] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:13.865778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d4bb-2192-4a57-8ecc-0b5f432cdbda None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:13.866262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d4bb-2192-4a57-8ecc-0b5f432cdbda None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:13.866554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d4bb-2192-4a57-8ecc-0b5f432cdbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:13.893080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4e22d4bb-2192-4a57-8ecc-0b5f432cdbda None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:13.894909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4e22d4bb-2192-4a57-8ecc-0b5f432cdbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:13.905601 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2068/4099] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:13.909828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab85612-1552-4db5-b279-b6f54d60272c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:13.929459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab85612-1552-4db5-b279-b6f54d60272c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:13.935751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328a15e-3f9a-4e2e-93de-4e3f33cb759a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:13.940557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328a15e-3f9a-4e2e-93de-4e3f33cb759a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:13.940952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328a15e-3f9a-4e2e-93de-4e3f33cb759a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:13.941440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328a15e-3f9a-4e2e-93de-4e3f33cb759a admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:13.943010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328a15e-3f9a-4e2e-93de-4e3f33cb759a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:08:13.943424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328a15e-3f9a-4e2e-93de-4e3f33cb759a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:13.945797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328a15e-3f9a-4e2e-93de-4e3f33cb759a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:13.967238 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2032/4100] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:13.968548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22d4bb-2192-4a57-8ecc-0b5f432cdbda None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:13.981387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22d4bb-2192-4a57-8ecc-0b5f432cdbda None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:13.987774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee078cae-5195-463a-8c3d-aa8a4ab45b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:13.988293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee078cae-5195-463a-8c3d-aa8a4ab45b7e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:13.988647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee078cae-5195-463a-8c3d-aa8a4ab45b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:13.990354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee078cae-5195-463a-8c3d-aa8a4ab45b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:13.990354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee078cae-5195-463a-8c3d-aa8a4ab45b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:13.992135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee078cae-5195-463a-8c3d-aa8a4ab45b7e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:14.043126 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2069/4101] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:08:13 2022] POST /identity/v3/projects => generated 389 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:14.047089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328a15e-3f9a-4e2e-93de-4e3f33cb759a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:14.065085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328a15e-3f9a-4e2e-93de-4e3f33cb759a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:14.182156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810da39-b934-48a9-b875-d0f2a7de1fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:14.182918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810da39-b934-48a9-b875-d0f2a7de1fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:14.183255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810da39-b934-48a9-b875-d0f2a7de1fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:14.183744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810da39-b934-48a9-b875-d0f2a7de1fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:14.184489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810da39-b934-48a9-b875-d0f2a7de1fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:14.186498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810da39-b934-48a9-b875-d0f2a7de1fd3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:14.242074 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2033/4102] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:08:14 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:14.246498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee078cae-5195-463a-8c3d-aa8a4ab45b7e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:14.260639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee078cae-5195-463a-8c3d-aa8a4ab45b7e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:14.272435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d9ab8-685c-4145-b447-52635104cfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:14.273178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d9ab8-685c-4145-b447-52635104cfca admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:14.273467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d9ab8-685c-4145-b447-52635104cfca admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:14.273802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d9ab8-685c-4145-b447-52635104cfca admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:14.274382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d9ab8-685c-4145-b447-52635104cfca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:14.276218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d9ab8-685c-4145-b447-52635104cfca admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:14.330336 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2070/4103] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:14.334973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810da39-b934-48a9-b875-d0f2a7de1fd3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:14.352808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810da39-b934-48a9-b875-d0f2a7de1fd3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:14.365552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b872b-9a38-4372-8a14-253fa291fc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:14.366667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b872b-9a38-4372-8a14-253fa291fc08 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:14.367173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b872b-9a38-4372-8a14-253fa291fc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:14.368458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b872b-9a38-4372-8a14-253fa291fc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:14.369512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b872b-9a38-4372-8a14-253fa291fc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:14.372333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b872b-9a38-4372-8a14-253fa291fc08 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:14.375302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:14.396144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d9ab8-685c-4145-b447-52635104cfca admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:14.401144 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2034/4104] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:14.416055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d9ab8-685c-4145-b447-52635104cfca admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:14.430724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83676fdf-940a-45f0-98ff-1f7fd12bd7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:14.431695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83676fdf-940a-45f0-98ff-1f7fd12bd7b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:14.432121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83676fdf-940a-45f0-98ff-1f7fd12bd7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:14.432604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83676fdf-940a-45f0-98ff-1f7fd12bd7b2 admin admin] PATH_INFO: `/v3/projects/83137ec5bbd04ea78c74a4ea576466ae/users/dc3aa9044e0a4ea88c77c3143ea740cb/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:14.464776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83676fdf-940a-45f0-98ff-1f7fd12bd7b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83137ec5bbd04ea78c74a4ea576466ae, user_id=dc3aa9044e0a4ea88c77c3143ea740cb, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:14.504523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83676fdf-940a-45f0-98ff-1f7fd12bd7b2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:14.581387 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2071/4105] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:08:14 2022] PUT /identity/v3/projects/83137ec5bbd04ea78c74a4ea576466ae/users/dc3aa9044e0a4ea88c77c3143ea740cb/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:14.600224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049f35e-f9cd-4065-8c4f-96b7c79ccb07 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:14.601127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049f35e-f9cd-4065-8c4f-96b7c79ccb07 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:14.601127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049f35e-f9cd-4065-8c4f-96b7c79ccb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:14.663612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9049f35e-f9cd-4065-8c4f-96b7c79ccb07 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:14.675301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9049f35e-f9cd-4065-8c4f-96b7c79ccb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:14.789122 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2035/4106] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:14.799815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83676fdf-940a-45f0-98ff-1f7fd12bd7b2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:14.801773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-83676fdf-940a-45f0-98ff-1f7fd12bd7b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:14.878111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83676fdf-940a-45f0-98ff-1f7fd12bd7b2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:14.882888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa69f55-ddc2-4823-9d17-8b0cda0b3475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:14.884005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa69f55-ddc2-4823-9d17-8b0cda0b3475 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:14.884773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa69f55-ddc2-4823-9d17-8b0cda0b3475 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:14.885273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa69f55-ddc2-4823-9d17-8b0cda0b3475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:14.898189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa69f55-ddc2-4823-9d17-8b0cda0b3475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:14.900815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa69f55-ddc2-4823-9d17-8b0cda0b3475 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:14.938585 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2072/4107] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:08:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:08:17.423703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80abbf-05b7-479b-970b-eecc32bfa062 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:17.424244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80abbf-05b7-479b-970b-eecc32bfa062 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:17.424831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80abbf-05b7-479b-970b-eecc32bfa062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:17.467413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8f80abbf-05b7-479b-970b-eecc32bfa062 None None] MFA Rules not processed for user `c93bdcb41270436ea67cd0aa05984b88`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:17.471180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8f80abbf-05b7-479b-970b-eecc32bfa062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:17.628231 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2036/4108] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:17.643398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa69f55-ddc2-4823-9d17-8b0cda0b3475 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:17.647996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dfa69f55-ddc2-4823-9d17-8b0cda0b3475 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:17.771807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa69f55-ddc2-4823-9d17-8b0cda0b3475 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:17.777788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537b4d3-4204-407c-94b0-f02e85518a03 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:17.779092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537b4d3-4204-407c-94b0-f02e85518a03 service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:17.780129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537b4d3-4204-407c-94b0-f02e85518a03 service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:17.780129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537b4d3-4204-407c-94b0-f02e85518a03 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:17.782186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f537b4d3-4204-407c-94b0-f02e85518a03 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:17.798217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:17.933083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537b4d3-4204-407c-94b0-f02e85518a03 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:17.934555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537b4d3-4204-407c-94b0-f02e85518a03 service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:17.970392 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2073/4109] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:08:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:08:19.263916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82189960-c536-4cc4-a1e2-91b58dcf19e3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:19.264479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82189960-c536-4cc4-a1e2-91b58dcf19e3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:19.264479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82189960-c536-4cc4-a1e2-91b58dcf19e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:19.321946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-82189960-c536-4cc4-a1e2-91b58dcf19e3 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:19.334637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-82189960-c536-4cc4-a1e2-91b58dcf19e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:19.420584 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2037/4110] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:08:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:19.453349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537b4d3-4204-407c-94b0-f02e85518a03 service glance] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:19.455203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f537b4d3-4204-407c-94b0-f02e85518a03 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:19.622647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537b4d3-4204-407c-94b0-f02e85518a03 service glance] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:19.644331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e545d-af19-46d8-8703-82e474e6f85a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:19.645448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e545d-af19-46d8-8703-82e474e6f85a service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:19.645938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e545d-af19-46d8-8703-82e474e6f85a service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:19.646459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e545d-af19-46d8-8703-82e474e6f85a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:19.656454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e545d-af19-46d8-8703-82e474e6f85a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:19.658483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e545d-af19-46d8-8703-82e474e6f85a service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:19.668964 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2074/4111] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:08:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:08:22.901407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcb05c-49bc-441b-ac23-52ff24e1841b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:22.902072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcb05c-49bc-441b-ac23-52ff24e1841b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:22.902072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcb05c-49bc-441b-ac23-52ff24e1841b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:22.975172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-87dcb05c-49bc-441b-ac23-52ff24e1841b None None] MFA Rules not processed for user `dc3aa9044e0a4ea88c77c3143ea740cb`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:22.977517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-87dcb05c-49bc-441b-ac23-52ff24e1841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:23.020131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:23.178485 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2038/4112] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:23.190502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc2674-f38a-41b9-9dc3-afb01dbce393 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:23.191410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc2674-f38a-41b9-9dc3-afb01dbce393 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:23.191818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc2674-f38a-41b9-9dc3-afb01dbce393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:23.224789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-44cc2674-f38a-41b9-9dc3-afb01dbce393 None None] MFA Rules not processed for user `dc3aa9044e0a4ea88c77c3143ea740cb`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:23.229094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-44cc2674-f38a-41b9-9dc3-afb01dbce393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:23.252505 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2075/4113] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:23.260420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcb05c-49bc-441b-ac23-52ff24e1841b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:23.262010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-87dcb05c-49bc-441b-ac23-52ff24e1841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:23.403381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcb05c-49bc-441b-ac23-52ff24e1841b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:23.419685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e81d3-9890-47c3-9fb8-3621bdea3023 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:23.421488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e81d3-9890-47c3-9fb8-3621bdea3023 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:23.423247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e81d3-9890-47c3-9fb8-3621bdea3023 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:23.423628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e81d3-9890-47c3-9fb8-3621bdea3023 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:23.445884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e81d3-9890-47c3-9fb8-3621bdea3023 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:23.447610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e81d3-9890-47c3-9fb8-3621bdea3023 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:23.478915 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2039/4114] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:08:23 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:08:27.350060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc2674-f38a-41b9-9dc3-afb01dbce393 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:27.352177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-44cc2674-f38a-41b9-9dc3-afb01dbce393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:27.469695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc2674-f38a-41b9-9dc3-afb01dbce393 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:27.478578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa5229-fbf7-4584-99dc-8d5abb0685a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:27.480704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa5229-fbf7-4584-99dc-8d5abb0685a9 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:27.481256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa5229-fbf7-4584-99dc-8d5abb0685a9 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:27.481773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa5229-fbf7-4584-99dc-8d5abb0685a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:27.483885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b1aa5229-fbf7-4584-99dc-8d5abb0685a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:27.500642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa5229-fbf7-4584-99dc-8d5abb0685a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:27.502276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa5229-fbf7-4584-99dc-8d5abb0685a9 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:27.521569 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2076/4115] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:08:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:08:33.465293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=53 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:37.512479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:42.591043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e81d3-9890-47c3-9fb8-3621bdea3023 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:42.592553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-815e81d3-9890-47c3-9fb8-3621bdea3023 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:42.619958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e81d3-9890-47c3-9fb8-3621bdea3023 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:42.626698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:42.627452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:42.627819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:42.628183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] PATH_INFO: `/v3/users/43dcc039e9d0442bb96ae9c0725a4a76` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:42.629452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43dcc039e9d0442bb96ae9c0725a4a76)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:42.631743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:42.709625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43dcc039e9d0442bb96ae9c0725a4a76'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:42.719338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43dcc039e9d0442bb96ae9c0725a4a76'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:42.732880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43dcc039e9d0442bb96ae9c0725a4a76'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:42.733747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43dcc039e9d0442bb96ae9c0725a4a76'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:42.734430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43dcc039e9d0442bb96ae9c0725a4a76'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:42.748422 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2040/4116] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:08:42 2022] DELETE /identity/v3/users/43dcc039e9d0442bb96ae9c0725a4a76 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:42.753025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003676cc-bb6a-4a41-bc03-8a82e8f1fcfa None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:42.753878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003676cc-bb6a-4a41-bc03-8a82e8f1fcfa None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:42.754351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003676cc-bb6a-4a41-bc03-8a82e8f1fcfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:42.775909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-003676cc-bb6a-4a41-bc03-8a82e8f1fcfa None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:42.777204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-003676cc-bb6a-4a41-bc03-8a82e8f1fcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:42.857362 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2077/4117] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:42.868317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:42.870207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:42.906163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003676cc-bb6a-4a41-bc03-8a82e8f1fcfa None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:42.908071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-003676cc-bb6a-4a41-bc03-8a82e8f1fcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:42.924060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003676cc-bb6a-4a41-bc03-8a82e8f1fcfa None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:42.930610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:42.931237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:42.931505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:42.931765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] PATH_INFO: `/v3/users/d29a50a0da47413eb649d3c63ba3c213` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:42.932915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d29a50a0da47413eb649d3c63ba3c213)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:42.935316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:42.952965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f765e-996d-4389-b125-2a975949e064 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:42.956497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5de67-f895-439c-b8c1-a8150907d2d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:42.957097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5de67-f895-439c-b8c1-a8150907d2d3 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:42.957367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5de67-f895-439c-b8c1-a8150907d2d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:42.957629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5de67-f895-439c-b8c1-a8150907d2d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:42.966979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5de67-f895-439c-b8c1-a8150907d2d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:42.968421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5de67-f895-439c-b8c1-a8150907d2d3 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:42.978871 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2041/4118] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:08:42.993575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd29a50a0da47413eb649d3c63ba3c213'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.002491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd29a50a0da47413eb649d3c63ba3c213'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.010621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd29a50a0da47413eb649d3c63ba3c213'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.024418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd29a50a0da47413eb649d3c63ba3c213'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.025238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd29a50a0da47413eb649d3c63ba3c213'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.031719 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2078/4119] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:08:42 2022] DELETE /identity/v3/users/d29a50a0da47413eb649d3c63ba3c213 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:43.302462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5de67-f895-439c-b8c1-a8150907d2d3 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:43.318953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5de67-f895-439c-b8c1-a8150907d2d3 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:43.332638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:43.336574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:43.405257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:43.406793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:43.407376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:43.408064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] PATH_INFO: `/v3/projects/a39eb787a51d4029852861a1d0bba7d9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:43.409805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a39eb787a51d4029852861a1d0bba7d9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:43.412405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:43.438603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:43.438855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efe452-89b4-453c-9e8d-2c33c9f0f3c2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:43.456135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91285fb3-d86b-4101-a629-7f7fe7de1e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:43.456922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91285fb3-d86b-4101-a629-7f7fe7de1e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:43.457198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91285fb3-d86b-4101-a629-7f7fe7de1e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:43.457466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91285fb3-d86b-4101-a629-7f7fe7de1e1d admin admin] PATH_INFO: `/v3/projects/77c499d91be4454db2371c4ec99ce38c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:43.459278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91285fb3-d86b-4101-a629-7f7fe7de1e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=77c499d91be4454db2371c4ec99ce38c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:43.461761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91285fb3-d86b-4101-a629-7f7fe7de1e1d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:43.483727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:43.504189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a39eb787a51d4029852861a1d0bba7d9', 'request_id': 'req-0827e261-306c-4295-9dd7-a5d359816685'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.510915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a39eb787a51d4029852861a1d0bba7d9', 'request_id': 'req-0827e261-306c-4295-9dd7-a5d359816685'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.518097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91285fb3-d86b-4101-a629-7f7fe7de1e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77c499d91be4454db2371c4ec99ce38c', 'request_id': 'req-91285fb3-d86b-4101-a629-7f7fe7de1e1d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.524028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a39eb787a51d4029852861a1d0bba7d9', 'request_id': 'req-0827e261-306c-4295-9dd7-a5d359816685'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.528257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91285fb3-d86b-4101-a629-7f7fe7de1e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77c499d91be4454db2371c4ec99ce38c', 'request_id': 'req-91285fb3-d86b-4101-a629-7f7fe7de1e1d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.536825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91285fb3-d86b-4101-a629-7f7fe7de1e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77c499d91be4454db2371c4ec99ce38c', 'request_id': 'req-91285fb3-d86b-4101-a629-7f7fe7de1e1d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.538272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] The token cache is being invalidate because project a39eb787a51d4029852861a1d0bba7d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:43.538559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.539274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.542110 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2042/4120] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:08:43 2022] DELETE /identity/v3/projects/a39eb787a51d4029852861a1d0bba7d9 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:43.546159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:43.547816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:43.552125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91285fb3-d86b-4101-a629-7f7fe7de1e1d admin admin] The token cache is being invalidate because project 77c499d91be4454db2371c4ec99ce38c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:43.552567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91285fb3-d86b-4101-a629-7f7fe7de1e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.553251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91285fb3-d86b-4101-a629-7f7fe7de1e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.555171 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2079/4121] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:08:43 2022] DELETE /identity/v3/projects/77c499d91be4454db2371c4ec99ce38c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:43.560220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef390d7e-8da4-44cb-98e1-39e764d58160 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:43.560528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef390d7e-8da4-44cb-98e1-39e764d58160 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:43.560815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef390d7e-8da4-44cb-98e1-39e764d58160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:43.580711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ef390d7e-8da4-44cb-98e1-39e764d58160 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:43.582027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ef390d7e-8da4-44cb-98e1-39e764d58160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:43.627885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827e261-306c-4295-9dd7-a5d359816685 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:43.654516 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2080/4122] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:08:43.662511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cd7aa-94bb-4f3e-bdf2-4fce079d5e0c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:43.662792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cd7aa-94bb-4f3e-bdf2-4fce079d5e0c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:43.663046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cd7aa-94bb-4f3e-bdf2-4fce079d5e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:43.683114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-870cd7aa-94bb-4f3e-bdf2-4fce079d5e0c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:43.684564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-870cd7aa-94bb-4f3e-bdf2-4fce079d5e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:43.693767 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2081/4123] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:08:43.695037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403dfe1-8382-4076-9c91-2c0892dbbd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:43.696026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403dfe1-8382-4076-9c91-2c0892dbbd83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:43.696296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403dfe1-8382-4076-9c91-2c0892dbbd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:43.696602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403dfe1-8382-4076-9c91-2c0892dbbd83 admin admin] PATH_INFO: `/v3/users/c93bdcb41270436ea67cd0aa05984b88` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:43.697818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403dfe1-8382-4076-9c91-2c0892dbbd83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93bdcb41270436ea67cd0aa05984b88)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:43.698442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cd7aa-94bb-4f3e-bdf2-4fce079d5e0c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:43.699764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403dfe1-8382-4076-9c91-2c0892dbbd83 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:43.707517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cd7aa-94bb-4f3e-bdf2-4fce079d5e0c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:43.712262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192733e-a248-4a40-b1e5-58a183bbcfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:43.712900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192733e-a248-4a40-b1e5-58a183bbcfd4 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:43.713166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192733e-a248-4a40-b1e5-58a183bbcfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:43.713426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192733e-a248-4a40-b1e5-58a183bbcfd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:43.714057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192733e-a248-4a40-b1e5-58a183bbcfd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:08:43.714347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192733e-a248-4a40-b1e5-58a183bbcfd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:43.715934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192733e-a248-4a40-b1e5-58a183bbcfd4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:43.728398 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2082/4124] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:08:43.732744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192733e-a248-4a40-b1e5-58a183bbcfd4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:43.743484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192733e-a248-4a40-b1e5-58a183bbcfd4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:43.747984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c403dfe1-8382-4076-9c91-2c0892dbbd83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93bdcb41270436ea67cd0aa05984b88'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.748690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3c764-23e6-487f-aeed-82d4f1a940ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:43.749299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3c764-23e6-487f-aeed-82d4f1a940ea admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:43.749579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3c764-23e6-487f-aeed-82d4f1a940ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:43.749847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3c764-23e6-487f-aeed-82d4f1a940ea admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:43.750486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d3c764-23e6-487f-aeed-82d4f1a940ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:43.752732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d3c764-23e6-487f-aeed-82d4f1a940ea admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:43.754842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c403dfe1-8382-4076-9c91-2c0892dbbd83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93bdcb41270436ea67cd0aa05984b88'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.756153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:43.769689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c403dfe1-8382-4076-9c91-2c0892dbbd83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93bdcb41270436ea67cd0aa05984b88'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.770643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c403dfe1-8382-4076-9c91-2c0892dbbd83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93bdcb41270436ea67cd0aa05984b88'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.771715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c403dfe1-8382-4076-9c91-2c0892dbbd83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93bdcb41270436ea67cd0aa05984b88'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:43.785889 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2083/4125] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:08:43 2022] POST /identity/v3/projects => generated 401 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:43.786484 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2043/4126] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:08:43 2022] DELETE /identity/v3/users/c93bdcb41270436ea67cd0aa05984b88 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:43.788927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3c764-23e6-487f-aeed-82d4f1a940ea admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:43.790653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b4d3c764-23e6-487f-aeed-82d4f1a940ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:43.861599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3c764-23e6-487f-aeed-82d4f1a940ea admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:43.866990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:43.925966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb306e1-1054-4175-8315-569df1733901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:43.926592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb306e1-1054-4175-8315-569df1733901 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:43.926870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb306e1-1054-4175-8315-569df1733901 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:43.927217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb306e1-1054-4175-8315-569df1733901 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:43.927871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb306e1-1054-4175-8315-569df1733901 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:43.931652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb306e1-1054-4175-8315-569df1733901 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:43.972599 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2084/4127] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:08:43 2022] POST /identity/v3/users => generated 301 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:43.975995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403dfe1-8382-4076-9c91-2c0892dbbd83 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:43.994181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403dfe1-8382-4076-9c91-2c0892dbbd83 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:44.000667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eabf97-624c-4ad5-b92e-fa6d1d080dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:44.001342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eabf97-624c-4ad5-b92e-fa6d1d080dea admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:44.001646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eabf97-624c-4ad5-b92e-fa6d1d080dea admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:44.001953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eabf97-624c-4ad5-b92e-fa6d1d080dea admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:44.004423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eabf97-624c-4ad5-b92e-fa6d1d080dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:44.004841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eabf97-624c-4ad5-b92e-fa6d1d080dea admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:44.019340 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2044/4128] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:08:44.020570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eabf97-624c-4ad5-b92e-fa6d1d080dea admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:44.020834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb306e1-1054-4175-8315-569df1733901 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:44.023493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9cb306e1-1054-4175-8315-569df1733901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:44.037603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb306e1-1054-4175-8315-569df1733901 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:44.041329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eabf97-624c-4ad5-b92e-fa6d1d080dea admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:44.045589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83720f78-35e6-4d17-acc8-631c279a8058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:44.046202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83720f78-35e6-4d17-acc8-631c279a8058 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:44.046370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:44.046490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83720f78-35e6-4d17-acc8-631c279a8058 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:44.046770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83720f78-35e6-4d17-acc8-631c279a8058 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:44.046988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:44.047263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:44.047374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83720f78-35e6-4d17-acc8-631c279a8058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:44.047655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] PATH_INFO: `/v3/projects/a704089fa75c4f008577d83aee1a6d99` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:44.048879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a704089fa75c4f008577d83aee1a6d99)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:44.049007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83720f78-35e6-4d17-acc8-631c279a8058 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:44.051022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:44.060982 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2045/4129] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:08:44.065019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83720f78-35e6-4d17-acc8-631c279a8058 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:44.078593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83720f78-35e6-4d17-acc8-631c279a8058 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:44.085082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64451f-ed17-43f8-b54a-5717c1e4714e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:44.085690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64451f-ed17-43f8-b54a-5717c1e4714e admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:44.086084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64451f-ed17-43f8-b54a-5717c1e4714e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:44.086355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64451f-ed17-43f8-b54a-5717c1e4714e admin admin] PATH_INFO: `/v3/projects/97261aff33364af99084756791910cbf/users/b0eb7d1e41fd4d35876a7fbbe1788a5a/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:44.102660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64451f-ed17-43f8-b54a-5717c1e4714e admin admin] RBAC: Authorizing `identity:create_grant(project_id=97261aff33364af99084756791910cbf, user_id=b0eb7d1e41fd4d35876a7fbbe1788a5a, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:44.103239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a704089fa75c4f008577d83aee1a6d99', 'request_id': 'req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:44.105370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64451f-ed17-43f8-b54a-5717c1e4714e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:44.112506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a704089fa75c4f008577d83aee1a6d99', 'request_id': 'req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:44.118397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a704089fa75c4f008577d83aee1a6d99', 'request_id': 'req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:44.121127 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2046/4130] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:08:44 2022] PUT /identity/v3/projects/97261aff33364af99084756791910cbf/users/b0eb7d1e41fd4d35876a7fbbe1788a5a/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:44.131495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] The token cache is being invalidate because project a704089fa75c4f008577d83aee1a6d99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:44.131747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:44.132572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:44.133554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8051783-9227-446f-8dba-a10b3834bdab None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:44.133828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8051783-9227-446f-8dba-a10b3834bdab None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:44.134085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8051783-9227-446f-8dba-a10b3834bdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:44.134281 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2085/4131] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:08:44 2022] DELETE /identity/v3/projects/a704089fa75c4f008577d83aee1a6d99 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:44.170925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c8051783-9227-446f-8dba-a10b3834bdab None None] MFA Rules not processed for user `b0eb7d1e41fd4d35876a7fbbe1788a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:44.172202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c8051783-9227-446f-8dba-a10b3834bdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:44.299766 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2047/4132] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:44 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:44.313453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:44.315468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:44.389897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e349ddf-cfb1-4f4a-82dc-b284cbda0fb6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:44.421799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:44.466416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:44.467165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:44.467491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:44.467820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] PATH_INFO: `/v3/users/b0eb7d1e41fd4d35876a7fbbe1788a5a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:44.469128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0eb7d1e41fd4d35876a7fbbe1788a5a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:44.471479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:44.547856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0eb7d1e41fd4d35876a7fbbe1788a5a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:44.558893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0eb7d1e41fd4d35876a7fbbe1788a5a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:44.566251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0eb7d1e41fd4d35876a7fbbe1788a5a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:44.580278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0eb7d1e41fd4d35876a7fbbe1788a5a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:44.581230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0eb7d1e41fd4d35876a7fbbe1788a5a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:44.589415 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2086/4133] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:08:44 2022] DELETE /identity/v3/users/b0eb7d1e41fd4d35876a7fbbe1788a5a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:44.594080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28039d71-504f-454f-a782-9755c28e3ef4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:44.594379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28039d71-504f-454f-a782-9755c28e3ef4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:44.594649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28039d71-504f-454f-a782-9755c28e3ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:44.615012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-28039d71-504f-454f-a782-9755c28e3ef4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:44.616194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-28039d71-504f-454f-a782-9755c28e3ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:44.683720 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2048/4134] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:08:44.693726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:44.695143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:44.763353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef6405-a0e4-4da6-8207-e9747ca19f0a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:44.766700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b0b95-1137-43e5-895a-c6ddba4d3281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:44.767208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b0b95-1137-43e5-895a-c6ddba4d3281 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:44.767477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b0b95-1137-43e5-895a-c6ddba4d3281 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:44.767740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b0b95-1137-43e5-895a-c6ddba4d3281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:44.779905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b0b95-1137-43e5-895a-c6ddba4d3281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:44.781390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b0b95-1137-43e5-895a-c6ddba4d3281 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:44.792507 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2087/4135] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:08:45.159014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28039d71-504f-454f-a782-9755c28e3ef4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:45.160581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-28039d71-504f-454f-a782-9755c28e3ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:45.173160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28039d71-504f-454f-a782-9755c28e3ef4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:45.177912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d58cb-bf41-4a2c-a599-2b542156b555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:45.178557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d58cb-bf41-4a2c-a599-2b542156b555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:45.178832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d58cb-bf41-4a2c-a599-2b542156b555 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:45.179096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d58cb-bf41-4a2c-a599-2b542156b555 admin admin] PATH_INFO: `/v3/projects/97261aff33364af99084756791910cbf` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:45.180257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d58cb-bf41-4a2c-a599-2b542156b555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97261aff33364af99084756791910cbf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:45.182208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d58cb-bf41-4a2c-a599-2b542156b555 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:45.193842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:45.237564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bd2d58cb-bf41-4a2c-a599-2b542156b555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97261aff33364af99084756791910cbf', 'request_id': 'req-bd2d58cb-bf41-4a2c-a599-2b542156b555'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:45.244445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bd2d58cb-bf41-4a2c-a599-2b542156b555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97261aff33364af99084756791910cbf', 'request_id': 'req-bd2d58cb-bf41-4a2c-a599-2b542156b555'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:45.252691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bd2d58cb-bf41-4a2c-a599-2b542156b555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97261aff33364af99084756791910cbf', 'request_id': 'req-bd2d58cb-bf41-4a2c-a599-2b542156b555'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:45.263068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bd2d58cb-bf41-4a2c-a599-2b542156b555 admin admin] The token cache is being invalidate because project 97261aff33364af99084756791910cbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:45.263345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bd2d58cb-bf41-4a2c-a599-2b542156b555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:45.263954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bd2d58cb-bf41-4a2c-a599-2b542156b555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:45.265509 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2049/4136] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:08:45 2022] DELETE /identity/v3/projects/97261aff33364af99084756791910cbf => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:45.271715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c467ee-19c5-42e2-b83b-5bba57dc45da None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:45.272007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c467ee-19c5-42e2-b83b-5bba57dc45da None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:45.272305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c467ee-19c5-42e2-b83b-5bba57dc45da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:45.292546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b3c467ee-19c5-42e2-b83b-5bba57dc45da None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:45.293742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b3c467ee-19c5-42e2-b83b-5bba57dc45da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:45.341579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:45.358003 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2088/4137] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:45.366088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c62f1a-1417-4660-b7bc-ef9a8410ec64 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:45.366394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c62f1a-1417-4660-b7bc-ef9a8410ec64 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:45.366687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c62f1a-1417-4660-b7bc-ef9a8410ec64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:45.386367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e0c62f1a-1417-4660-b7bc-ef9a8410ec64 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:45.387556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e0c62f1a-1417-4660-b7bc-ef9a8410ec64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:45.395561 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2050/4138] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:45.400540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c467ee-19c5-42e2-b83b-5bba57dc45da None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:45.417808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c467ee-19c5-42e2-b83b-5bba57dc45da None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:45.425296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228bcf2-09dc-4523-bfe8-c74052b4a129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:45.425890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228bcf2-09dc-4523-bfe8-c74052b4a129 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:45.426155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228bcf2-09dc-4523-bfe8-c74052b4a129 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:45.426416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228bcf2-09dc-4523-bfe8-c74052b4a129 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:45.427337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228bcf2-09dc-4523-bfe8-c74052b4a129 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:08:45.427825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228bcf2-09dc-4523-bfe8-c74052b4a129 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:45.429759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228bcf2-09dc-4523-bfe8-c74052b4a129 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:45.441639 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2089/4139] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:45.446478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c62f1a-1417-4660-b7bc-ef9a8410ec64 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:45.455880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c62f1a-1417-4660-b7bc-ef9a8410ec64 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:45.463673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b707d1-9557-45ed-b907-5b25521708cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:45.464410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b707d1-9557-45ed-b907-5b25521708cd admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:45.464699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b707d1-9557-45ed-b907-5b25521708cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:45.464963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b707d1-9557-45ed-b907-5b25521708cd admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:45.465628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b707d1-9557-45ed-b907-5b25521708cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:45.467583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b707d1-9557-45ed-b907-5b25521708cd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:45.499318 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2051/4140] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:08:45 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:45.504995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228bcf2-09dc-4523-bfe8-c74052b4a129 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:45.515267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228bcf2-09dc-4523-bfe8-c74052b4a129 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:45.574585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d8fd4-13a3-486c-90dc-c715bb107d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:45.575200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d8fd4-13a3-486c-90dc-c715bb107d4b admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:45.575473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d8fd4-13a3-486c-90dc-c715bb107d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:45.575737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d8fd4-13a3-486c-90dc-c715bb107d4b admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:45.576358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d8fd4-13a3-486c-90dc-c715bb107d4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:45.578316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d8fd4-13a3-486c-90dc-c715bb107d4b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:45.600575 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2090/4141] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:08:45 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:45.604770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b707d1-9557-45ed-b907-5b25521708cd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:45.617692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b707d1-9557-45ed-b907-5b25521708cd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:45.627095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371b47b-2aa2-43c2-b57f-5b2efb3e8a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:45.627701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371b47b-2aa2-43c2-b57f-5b2efb3e8a66 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:45.627973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371b47b-2aa2-43c2-b57f-5b2efb3e8a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:45.628237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371b47b-2aa2-43c2-b57f-5b2efb3e8a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:45.628903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371b47b-2aa2-43c2-b57f-5b2efb3e8a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:45.630603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371b47b-2aa2-43c2-b57f-5b2efb3e8a66 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:45.644471 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2052/4142] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:45.649704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d8fd4-13a3-486c-90dc-c715bb107d4b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:45.663702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d8fd4-13a3-486c-90dc-c715bb107d4b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:45.668939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3fbc0-900b-4242-92b6-febac335a093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:45.669513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3fbc0-900b-4242-92b6-febac335a093 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:45.669806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3fbc0-900b-4242-92b6-febac335a093 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:45.670063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3fbc0-900b-4242-92b6-febac335a093 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:45.670696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3fbc0-900b-4242-92b6-febac335a093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:45.672290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3fbc0-900b-4242-92b6-febac335a093 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:45.684542 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2091/4143] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:45.688868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371b47b-2aa2-43c2-b57f-5b2efb3e8a66 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:45.700092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371b47b-2aa2-43c2-b57f-5b2efb3e8a66 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:45.705676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28584e2a-5224-4630-9b2e-c2b5ba091b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:45.706433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28584e2a-5224-4630-9b2e-c2b5ba091b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:45.707213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28584e2a-5224-4630-9b2e-c2b5ba091b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:45.708219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28584e2a-5224-4630-9b2e-c2b5ba091b04 admin admin] PATH_INFO: `/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9/users/42f64ead39c04e74b830e1dd647685c5/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:45.726454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28584e2a-5224-4630-9b2e-c2b5ba091b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0624e25029cf4ac6bdabbb28d5bfb5e9, user_id=42f64ead39c04e74b830e1dd647685c5, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:45.729144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28584e2a-5224-4630-9b2e-c2b5ba091b04 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:45.754182 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2053/4144] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:08:45 2022] PUT /identity/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9/users/42f64ead39c04e74b830e1dd647685c5/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:45.759576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf26037-f37b-4860-bda5-512264efd9f2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:45.759868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf26037-f37b-4860-bda5-512264efd9f2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:45.760116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf26037-f37b-4860-bda5-512264efd9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:45.791499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6bf26037-f37b-4860-bda5-512264efd9f2 None None] MFA Rules not processed for user `42f64ead39c04e74b830e1dd647685c5`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:45.792760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6bf26037-f37b-4860-bda5-512264efd9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:45.861370 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2092/4145] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:45.877138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28584e2a-5224-4630-9b2e-c2b5ba091b04 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:45.879073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-28584e2a-5224-4630-9b2e-c2b5ba091b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:45.968716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28584e2a-5224-4630-9b2e-c2b5ba091b04 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:45.978090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d89ff-0112-43ad-ad51-bd7ae0941ec7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:45.978712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d89ff-0112-43ad-ad51-bd7ae0941ec7 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:45.978990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d89ff-0112-43ad-ad51-bd7ae0941ec7 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:45.979254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d89ff-0112-43ad-ad51-bd7ae0941ec7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:45.988442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d89ff-0112-43ad-ad51-bd7ae0941ec7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:45.990535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d89ff-0112-43ad-ad51-bd7ae0941ec7 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:46.001871 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2054/4146] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:08:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:08:46.018446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305811d2-4146-4f76-8711-0049c0191451 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:46.018697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305811d2-4146-4f76-8711-0049c0191451 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:46.019005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305811d2-4146-4f76-8711-0049c0191451 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:46.020342 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2093/4147] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:08:46.025415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d89ff-0112-43ad-ad51-bd7ae0941ec7 service nova] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:08:46.038615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d89ff-0112-43ad-ad51-bd7ae0941ec7 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:46.046396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d89ff-0112-43ad-ad51-bd7ae0941ec7 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:46.050610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55dd6c0-c80a-493c-b190-254311911142 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f64ead39c04e74b830e1dd647685c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0624e25029cf4ac6bdabbb28d5bfb5e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:46.051233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dd6c0-c80a-493c-b190-254311911142 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:46.051509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dd6c0-c80a-493c-b190-254311911142 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:46.051779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dd6c0-c80a-493c-b190-254311911142 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] PATH_INFO: `/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:46.053074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55dd6c0-c80a-493c-b190-254311911142 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: Authorizing `identity:get_project(project_id=0624e25029cf4ac6bdabbb28d5bfb5e9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:46.055073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55dd6c0-c80a-493c-b190-254311911142 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:46.056866 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2055/4148] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:08:46 2022] GET /identity/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9 => generated 363 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:46.088889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed37104-564e-4092-ab75-9b695e9974ee None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:46.089172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed37104-564e-4092-ab75-9b695e9974ee None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:46.089441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed37104-564e-4092-ab75-9b695e9974ee None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:46.091202 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2094/4149] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:08:46.096920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55dd6c0-c80a-493c-b190-254311911142 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:08:46.098830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b55dd6c0-c80a-493c-b190-254311911142 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:46.116719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55dd6c0-c80a-493c-b190-254311911142 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:46.125166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55dd6c0-c80a-493c-b190-254311911142 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:46.129489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9160423-16cf-40ab-ac96-6e9166cefbec tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f64ead39c04e74b830e1dd647685c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0624e25029cf4ac6bdabbb28d5bfb5e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:46.130112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9160423-16cf-40ab-ac96-6e9166cefbec tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:46.130382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9160423-16cf-40ab-ac96-6e9166cefbec tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:46.130678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9160423-16cf-40ab-ac96-6e9166cefbec tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] PATH_INFO: `/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:46.131791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9160423-16cf-40ab-ac96-6e9166cefbec tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: Authorizing `identity:get_project(project_id=0624e25029cf4ac6bdabbb28d5bfb5e9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:46.133639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9160423-16cf-40ab-ac96-6e9166cefbec tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:46.135323 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2056/4150] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:08:46 2022] GET /identity/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9 => generated 363 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:46.172918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f095b-f6ff-491a-b225-99279ab87c43 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:46.173262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f095b-f6ff-491a-b225-99279ab87c43 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:46.173602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f095b-f6ff-491a-b225-99279ab87c43 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:46.175038 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2095/4151] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:08:46.179901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9160423-16cf-40ab-ac96-6e9166cefbec tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:08:46.187196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:46.196740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9160423-16cf-40ab-ac96-6e9166cefbec tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:46.205656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9160423-16cf-40ab-ac96-6e9166cefbec tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:46.210163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cf012-26ff-491d-8ed4-be217ed3f2f2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f64ead39c04e74b830e1dd647685c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0624e25029cf4ac6bdabbb28d5bfb5e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:46.210867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cf012-26ff-491d-8ed4-be217ed3f2f2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:46.211145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cf012-26ff-491d-8ed4-be217ed3f2f2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:46.211418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cf012-26ff-491d-8ed4-be217ed3f2f2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] PATH_INFO: `/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:46.212662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178cf012-26ff-491d-8ed4-be217ed3f2f2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: Authorizing `identity:get_project(project_id=0624e25029cf4ac6bdabbb28d5bfb5e9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:46.214453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178cf012-26ff-491d-8ed4-be217ed3f2f2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:46.216168 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2057/4152] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:08:46 2022] GET /identity/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9 => generated 363 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:46.253572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ca1d0-7580-40f2-8d06-b80dee66b6c6 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:46.254493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ca1d0-7580-40f2-8d06-b80dee66b6c6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:46.254945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ca1d0-7580-40f2-8d06-b80dee66b6c6 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:46.256872 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2096/4153] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:08:46.265686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cf012-26ff-491d-8ed4-be217ed3f2f2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:08:46.289519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cf012-26ff-491d-8ed4-be217ed3f2f2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:46.298102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cf012-26ff-491d-8ed4-be217ed3f2f2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:46.302518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6df1b3-b041-429a-8aaf-5c839beb9ce2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f64ead39c04e74b830e1dd647685c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0624e25029cf4ac6bdabbb28d5bfb5e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:46.303490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6df1b3-b041-429a-8aaf-5c839beb9ce2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:46.303760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6df1b3-b041-429a-8aaf-5c839beb9ce2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:46.304027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6df1b3-b041-429a-8aaf-5c839beb9ce2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] PATH_INFO: `/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:46.305199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6df1b3-b041-429a-8aaf-5c839beb9ce2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: Authorizing `identity:get_project(project_id=0624e25029cf4ac6bdabbb28d5bfb5e9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:46.306985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6df1b3-b041-429a-8aaf-5c839beb9ce2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:46.308988 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2058/4154] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:08:46 2022] GET /identity/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9 => generated 363 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:46.355129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85402aaf-1ba0-4098-bf36-06481833a6f0 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:46.355478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85402aaf-1ba0-4098-bf36-06481833a6f0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:46.355593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85402aaf-1ba0-4098-bf36-06481833a6f0 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:46.357374 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2097/4155] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:08:46.362767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6df1b3-b041-429a-8aaf-5c839beb9ce2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:08:46.390070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6df1b3-b041-429a-8aaf-5c839beb9ce2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:46.402656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6df1b3-b041-429a-8aaf-5c839beb9ce2 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:46.413810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30352680-7a0e-4192-81f9-09cf3a97b679 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f64ead39c04e74b830e1dd647685c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0624e25029cf4ac6bdabbb28d5bfb5e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:46.414585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30352680-7a0e-4192-81f9-09cf3a97b679 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:46.414927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30352680-7a0e-4192-81f9-09cf3a97b679 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:46.415263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30352680-7a0e-4192-81f9-09cf3a97b679 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] PATH_INFO: `/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:46.416514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30352680-7a0e-4192-81f9-09cf3a97b679 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: Authorizing `identity:get_project(project_id=0624e25029cf4ac6bdabbb28d5bfb5e9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:46.418796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30352680-7a0e-4192-81f9-09cf3a97b679 tempest-QuotasTestJSON-372047587 tempest-QuotasTestJSON-372047587-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:46.420479 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2059/4156] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:08:46 2022] GET /identity/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9 => generated 363 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:46.456760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85402aaf-1ba0-4098-bf36-06481833a6f0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:46.510216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:46.527394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85402aaf-1ba0-4098-bf36-06481833a6f0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:46.531812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:46.532501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:46.532772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:46.533036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] PATH_INFO: `/v3/users/42f64ead39c04e74b830e1dd647685c5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:46.534441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=42f64ead39c04e74b830e1dd647685c5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:46.536356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:46.589966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42f64ead39c04e74b830e1dd647685c5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:46.603039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42f64ead39c04e74b830e1dd647685c5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:46.614079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42f64ead39c04e74b830e1dd647685c5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:46.629246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42f64ead39c04e74b830e1dd647685c5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:46.630138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42f64ead39c04e74b830e1dd647685c5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:46.637214 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2098/4157] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:08:46 2022] DELETE /identity/v3/users/42f64ead39c04e74b830e1dd647685c5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:46.641855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e1c2f-8c00-45c2-8218-5f2b02e345df None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:46.653929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e1c2f-8c00-45c2-8218-5f2b02e345df None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:46.653929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e1c2f-8c00-45c2-8218-5f2b02e345df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:46.662009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c12e1c2f-8c00-45c2-8218-5f2b02e345df None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:46.663264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c12e1c2f-8c00-45c2-8218-5f2b02e345df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:46.742220 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2060/4158] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:46.753061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:46.754595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:46.850234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f7fd8-bf2a-4aa2-bc49-018e788d7c1e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:46.855365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf1a8e-8972-4b24-b6ed-1e62705ea5ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:46.856034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf1a8e-8972-4b24-b6ed-1e62705ea5ef service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:46.856437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf1a8e-8972-4b24-b6ed-1e62705ea5ef service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:46.856811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf1a8e-8972-4b24-b6ed-1e62705ea5ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:46.871142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf1a8e-8972-4b24-b6ed-1e62705ea5ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:46.873597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf1a8e-8972-4b24-b6ed-1e62705ea5ef service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:46.905395 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2099/4159] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:08:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:08:47.218791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e1c2f-8c00-45c2-8218-5f2b02e345df None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:47.220300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c12e1c2f-8c00-45c2-8218-5f2b02e345df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:47.238517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e1c2f-8c00-45c2-8218-5f2b02e345df None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:47.243753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff085e53-d597-4e0f-9b7f-eb7ac099db25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:47.244479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085e53-d597-4e0f-9b7f-eb7ac099db25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:47.244736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085e53-d597-4e0f-9b7f-eb7ac099db25 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:47.244995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085e53-d597-4e0f-9b7f-eb7ac099db25 admin admin] PATH_INFO: `/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:47.246264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff085e53-d597-4e0f-9b7f-eb7ac099db25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0624e25029cf4ac6bdabbb28d5bfb5e9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:47.248726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff085e53-d597-4e0f-9b7f-eb7ac099db25 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:47.306279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ff085e53-d597-4e0f-9b7f-eb7ac099db25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0624e25029cf4ac6bdabbb28d5bfb5e9', 'request_id': 'req-ff085e53-d597-4e0f-9b7f-eb7ac099db25'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:47.313496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ff085e53-d597-4e0f-9b7f-eb7ac099db25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0624e25029cf4ac6bdabbb28d5bfb5e9', 'request_id': 'req-ff085e53-d597-4e0f-9b7f-eb7ac099db25'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:47.323821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ff085e53-d597-4e0f-9b7f-eb7ac099db25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0624e25029cf4ac6bdabbb28d5bfb5e9', 'request_id': 'req-ff085e53-d597-4e0f-9b7f-eb7ac099db25'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:47.335051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ff085e53-d597-4e0f-9b7f-eb7ac099db25 admin admin] The token cache is being invalidate because project 0624e25029cf4ac6bdabbb28d5bfb5e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:47.335337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ff085e53-d597-4e0f-9b7f-eb7ac099db25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:47.335980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ff085e53-d597-4e0f-9b7f-eb7ac099db25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:47.337544 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2061/4160] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:08:47 2022] DELETE /identity/v3/projects/0624e25029cf4ac6bdabbb28d5bfb5e9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:47.344693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414c7f4-1063-4871-8b85-0ac780431a9f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:47.344994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414c7f4-1063-4871-8b85-0ac780431a9f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:47.345330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414c7f4-1063-4871-8b85-0ac780431a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:47.365861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e414c7f4-1063-4871-8b85-0ac780431a9f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:47.367093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e414c7f4-1063-4871-8b85-0ac780431a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:47.430503 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2100/4161] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:47.510591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6126a-4f01-46a9-9ad2-f942ae23d0ac None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:47.511005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6126a-4f01-46a9-9ad2-f942ae23d0ac None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:47.511311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6126a-4f01-46a9-9ad2-f942ae23d0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:47.537810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-dea6126a-4f01-46a9-9ad2-f942ae23d0ac None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:47.539417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dea6126a-4f01-46a9-9ad2-f942ae23d0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:47.553395 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2062/4162] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:47.559875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414c7f4-1063-4871-8b85-0ac780431a9f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:47.570891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414c7f4-1063-4871-8b85-0ac780431a9f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:47.576689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78325d91-f67c-4b40-805b-25f960bb497c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:47.577289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78325d91-f67c-4b40-805b-25f960bb497c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:47.577556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78325d91-f67c-4b40-805b-25f960bb497c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:47.577820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78325d91-f67c-4b40-805b-25f960bb497c admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:47.578438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78325d91-f67c-4b40-805b-25f960bb497c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:08:47.578728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78325d91-f67c-4b40-805b-25f960bb497c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:47.580365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78325d91-f67c-4b40-805b-25f960bb497c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:47.582784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:47.593709 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2101/4163] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:47.598017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6126a-4f01-46a9-9ad2-f942ae23d0ac None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:47.607526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6126a-4f01-46a9-9ad2-f942ae23d0ac None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:47.612465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70b15f-ec66-4d26-b158-9d0fef707ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:47.613060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70b15f-ec66-4d26-b158-9d0fef707ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:47.613324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70b15f-ec66-4d26-b158-9d0fef707ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:47.613587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70b15f-ec66-4d26-b158-9d0fef707ac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:47.614201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70b15f-ec66-4d26-b158-9d0fef707ac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:47.616396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70b15f-ec66-4d26-b158-9d0fef707ac2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:47.646224 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2063/4164] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:08:47 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:08:47.650811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78325d91-f67c-4b40-805b-25f960bb497c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:47.662368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78325d91-f67c-4b40-805b-25f960bb497c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:47.724227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db1b24-b136-408f-8e68-29e392f3e9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:47.724873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db1b24-b136-408f-8e68-29e392f3e9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:47.725338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db1b24-b136-408f-8e68-29e392f3e9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:47.725610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db1b24-b136-408f-8e68-29e392f3e9a6 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:47.726228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db1b24-b136-408f-8e68-29e392f3e9a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:47.728205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db1b24-b136-408f-8e68-29e392f3e9a6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:47.756278 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2102/4165] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:08:47 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:47.760682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70b15f-ec66-4d26-b158-9d0fef707ac2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:47.771013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70b15f-ec66-4d26-b158-9d0fef707ac2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:47.775907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34970e-dde9-4759-8cfa-4cca25dc27ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:47.776658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34970e-dde9-4759-8cfa-4cca25dc27ae admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:47.776931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34970e-dde9-4759-8cfa-4cca25dc27ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:47.777192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34970e-dde9-4759-8cfa-4cca25dc27ae admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:47.777817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34970e-dde9-4759-8cfa-4cca25dc27ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:47.779405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34970e-dde9-4759-8cfa-4cca25dc27ae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:47.790783 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2064/4166] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:47.794965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db1b24-b136-408f-8e68-29e392f3e9a6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:47.805718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db1b24-b136-408f-8e68-29e392f3e9a6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:47.810811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193446c-a185-4221-8baf-bf85d37e367e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:47.811575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193446c-a185-4221-8baf-bf85d37e367e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:47.811844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193446c-a185-4221-8baf-bf85d37e367e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:47.812109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193446c-a185-4221-8baf-bf85d37e367e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:47.812788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193446c-a185-4221-8baf-bf85d37e367e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:47.814391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193446c-a185-4221-8baf-bf85d37e367e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:47.827121 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2103/4167] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:47.831448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34970e-dde9-4759-8cfa-4cca25dc27ae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:47.842575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34970e-dde9-4759-8cfa-4cca25dc27ae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:47.847539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4711f-e13b-44cd-820a-49da66de08ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:47.848195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4711f-e13b-44cd-820a-49da66de08ff admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:47.848510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4711f-e13b-44cd-820a-49da66de08ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:47.848782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4711f-e13b-44cd-820a-49da66de08ff admin admin] PATH_INFO: `/v3/projects/225e7973c50e4b8986305fe88e4d000e/users/4d8730857fd041ec83a7bb9c8e621175/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:47.863685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4711f-e13b-44cd-820a-49da66de08ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=225e7973c50e4b8986305fe88e4d000e, user_id=4d8730857fd041ec83a7bb9c8e621175, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:47.866446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4711f-e13b-44cd-820a-49da66de08ff admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:47.870311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:47.882632 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2065/4168] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:08:47 2022] PUT /identity/v3/projects/225e7973c50e4b8986305fe88e4d000e/users/4d8730857fd041ec83a7bb9c8e621175/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:47.893805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17721ff-bb53-4415-aa47-fa95eb13171b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:47.894323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17721ff-bb53-4415-aa47-fa95eb13171b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:47.894466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17721ff-bb53-4415-aa47-fa95eb13171b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:47.926775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e17721ff-bb53-4415-aa47-fa95eb13171b None None] MFA Rules not processed for user `4d8730857fd041ec83a7bb9c8e621175`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:47.928133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e17721ff-bb53-4415-aa47-fa95eb13171b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:48.010206 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2104/4169] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:48.012967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4711f-e13b-44cd-820a-49da66de08ff admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:48.075237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4711f-e13b-44cd-820a-49da66de08ff admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:48.080135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff43d67-71f7-4044-a3ba-0a5536d9119e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:48.080801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff43d67-71f7-4044-a3ba-0a5536d9119e admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:48.081023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff43d67-71f7-4044-a3ba-0a5536d9119e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:48.081290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff43d67-71f7-4044-a3ba-0a5536d9119e admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:48.081894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff43d67-71f7-4044-a3ba-0a5536d9119e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:48.083868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff43d67-71f7-4044-a3ba-0a5536d9119e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:48.113812 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2066/4170] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:08:48 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:48.118299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17721ff-bb53-4415-aa47-fa95eb13171b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:48.131158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17721ff-bb53-4415-aa47-fa95eb13171b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:48.194685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67404a-e99b-42b5-a5de-97861e5ad1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:48.195656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67404a-e99b-42b5-a5de-97861e5ad1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:48.195936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67404a-e99b-42b5-a5de-97861e5ad1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:48.196202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67404a-e99b-42b5-a5de-97861e5ad1e5 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:48.196889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67404a-e99b-42b5-a5de-97861e5ad1e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:48.198946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67404a-e99b-42b5-a5de-97861e5ad1e5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:48.210348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:48.226415 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2105/4171] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:08:48 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:48.231173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff43d67-71f7-4044-a3ba-0a5536d9119e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:48.254388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff43d67-71f7-4044-a3ba-0a5536d9119e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:48.262147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa64d9-6e35-4a62-83f0-757c32024b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:48.262836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa64d9-6e35-4a62-83f0-757c32024b36 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:48.263277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa64d9-6e35-4a62-83f0-757c32024b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:48.263562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa64d9-6e35-4a62-83f0-757c32024b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:48.264308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa64d9-6e35-4a62-83f0-757c32024b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:48.266070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa64d9-6e35-4a62-83f0-757c32024b36 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:48.281099 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2067/4172] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:48.282354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67404a-e99b-42b5-a5de-97861e5ad1e5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:48.293729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67404a-e99b-42b5-a5de-97861e5ad1e5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:48.300125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d1124-6eee-4430-a764-0fe938161994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:48.300832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d1124-6eee-4430-a764-0fe938161994 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:48.301111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d1124-6eee-4430-a764-0fe938161994 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:48.301368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d1124-6eee-4430-a764-0fe938161994 admin admin] PATH_INFO: `/v3/projects/333dacfda99343f79d2c5c80e3089d1f/users/3a9311f440a24d3b8b538d79fae5f201/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:48.317368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d1124-6eee-4430-a764-0fe938161994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=333dacfda99343f79d2c5c80e3089d1f, user_id=3a9311f440a24d3b8b538d79fae5f201, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:48.321421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d1124-6eee-4430-a764-0fe938161994 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:48.337012 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2106/4173] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:08:48 2022] PUT /identity/v3/projects/333dacfda99343f79d2c5c80e3089d1f/users/3a9311f440a24d3b8b538d79fae5f201/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:48.346019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa64d9-6e35-4a62-83f0-757c32024b36 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:48.357192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa64d9-6e35-4a62-83f0-757c32024b36 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:48.414625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b804e8d-c322-44a3-9015-3d883ed9ffc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:48.415235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b804e8d-c322-44a3-9015-3d883ed9ffc2 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:48.415521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b804e8d-c322-44a3-9015-3d883ed9ffc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:48.415786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b804e8d-c322-44a3-9015-3d883ed9ffc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:48.416437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b804e8d-c322-44a3-9015-3d883ed9ffc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:48.418082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b804e8d-c322-44a3-9015-3d883ed9ffc2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:48.429143 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2068/4174] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:48.433645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d1124-6eee-4430-a764-0fe938161994 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:48.444199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d1124-6eee-4430-a764-0fe938161994 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:48.449320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47d4f0-03a3-4459-b412-c3cd6d229841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:48.449969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47d4f0-03a3-4459-b412-c3cd6d229841 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:48.450237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47d4f0-03a3-4459-b412-c3cd6d229841 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:48.450505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47d4f0-03a3-4459-b412-c3cd6d229841 admin admin] PATH_INFO: `/v3/projects/333dacfda99343f79d2c5c80e3089d1f/users/3a9311f440a24d3b8b538d79fae5f201/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:48.452480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47d4f0-03a3-4459-b412-c3cd6d229841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=333dacfda99343f79d2c5c80e3089d1f, user_id=3a9311f440a24d3b8b538d79fae5f201, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:48.455257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47d4f0-03a3-4459-b412-c3cd6d229841 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:48.466656 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2107/4175] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:08:48 2022] PUT /identity/v3/projects/333dacfda99343f79d2c5c80e3089d1f/users/3a9311f440a24d3b8b538d79fae5f201/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:48.480283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b3689-ad0f-48a7-8891-de676f634453 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:48.480718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b3689-ad0f-48a7-8891-de676f634453 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:48.480996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b3689-ad0f-48a7-8891-de676f634453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:48.516094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-673b3689-ad0f-48a7-8891-de676f634453 None None] MFA Rules not processed for user `3a9311f440a24d3b8b538d79fae5f201`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:48.517577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-673b3689-ad0f-48a7-8891-de676f634453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:48.582053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:48.605124 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2069/4176] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:48.609666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47d4f0-03a3-4459-b412-c3cd6d229841 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:48.624359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47d4f0-03a3-4459-b412-c3cd6d229841 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:48.629632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478018f-cb50-4e49-9d81-d8265f4cb7af tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:48.630060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478018f-cb50-4e49-9d81-d8265f4cb7af tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:48.630060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478018f-cb50-4e49-9d81-d8265f4cb7af tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:48.630269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478018f-cb50-4e49-9d81-d8265f4cb7af tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:48.631044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478018f-cb50-4e49-9d81-d8265f4cb7af tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:48.632670 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-f478018f-cb50-4e49-9d81-d8265f4cb7af tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:08:48.632670 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:08:48.632670 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:08:48.633890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478018f-cb50-4e49-9d81-d8265f4cb7af tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:48.664897 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2108/4177] 10.222.0.23 () {60 vars in 1150 bytes} [Sat Aug 20 21:08:48 2022] POST /identity/v3/domains => generated 277 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:48.668780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b3689-ad0f-48a7-8891-de676f634453 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:48.680876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b3689-ad0f-48a7-8891-de676f634453 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:48.770511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05df82f-42bc-4057-b0cf-811845645361 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:48.771208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05df82f-42bc-4057-b0cf-811845645361 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:48.771658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05df82f-42bc-4057-b0cf-811845645361 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:48.772086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05df82f-42bc-4057-b0cf-811845645361 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:48.773096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05df82f-42bc-4057-b0cf-811845645361 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:48.774233 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-a05df82f-42bc-4057-b0cf-811845645361 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:08:48.774233 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:08:48.774233 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:08:48.775518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05df82f-42bc-4057-b0cf-811845645361 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:48.806337 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2070/4178] 10.222.0.23 () {60 vars in 1150 bytes} [Sat Aug 20 21:08:48 2022] POST /identity/v3/domains => generated 276 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:08:48.807842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478018f-cb50-4e49-9d81-d8265f4cb7af tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:48.821355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478018f-cb50-4e49-9d81-d8265f4cb7af tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:48.883384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b3ebe-21d0-4fcb-9c73-569d32f74cc1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:48.883925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b3ebe-21d0-4fcb-9c73-569d32f74cc1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:48.884196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b3ebe-21d0-4fcb-9c73-569d32f74cc1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:48.884498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b3ebe-21d0-4fcb-9c73-569d32f74cc1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:48.885156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b3ebe-21d0-4fcb-9c73-569d32f74cc1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:48.886764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b3ebe-21d0-4fcb-9c73-569d32f74cc1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:48.908750 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2109/4179] 10.222.0.23 () {60 vars in 1150 bytes} [Sat Aug 20 21:08:48 2022] POST /identity/v3/domains => generated 278 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:48.909922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05df82f-42bc-4057-b0cf-811845645361 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:48.920605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05df82f-42bc-4057-b0cf-811845645361 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:48.981218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d961f-0430-4e71-bd2d-cfff95a2347b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:48.981946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d961f-0430-4e71-bd2d-cfff95a2347b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:48.982210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d961f-0430-4e71-bd2d-cfff95a2347b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:48.982476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d961f-0430-4e71-bd2d-cfff95a2347b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:48.983126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d961f-0430-4e71-bd2d-cfff95a2347b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:48.984791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d961f-0430-4e71-bd2d-cfff95a2347b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:49.003654 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2071/4180] 10.222.0.23 () {60 vars in 1150 bytes} [Sat Aug 20 21:08:48 2022] POST /identity/v3/domains => generated 279 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:49.010423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b3ebe-21d0-4fcb-9c73-569d32f74cc1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:49.021621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b3ebe-21d0-4fcb-9c73-569d32f74cc1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:49.106678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:49.107334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:49.107610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:49.107876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/b8659b510ca64fbe82e53b4c70f3b990` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:49.119958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=b8659b510ca64fbe82e53b4c70f3b990)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:49.121092 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:08:49.121092 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:08:49.121092 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:08:49.122089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:49.152314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'b8659b510ca64fbe82e53b4c70f3b990'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.175208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'b8659b510ca64fbe82e53b4c70f3b990'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.177285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:49.232571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8659b510ca64fbe82e53b4c70f3b990', 'request_id': 'req-8298bd2e-7274-44b6-9178-cccc023940ac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.243147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8659b510ca64fbe82e53b4c70f3b990', 'request_id': 'req-8298bd2e-7274-44b6-9178-cccc023940ac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.249106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8659b510ca64fbe82e53b4c70f3b990', 'request_id': 'req-8298bd2e-7274-44b6-9178-cccc023940ac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.260690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] The token cache is being invalidate because project b8659b510ca64fbe82e53b4c70f3b990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:49.260972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.261628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.274747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'b8659b510ca64fbe82e53b4c70f3b990', 'request_id': 'req-8298bd2e-7274-44b6-9178-cccc023940ac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.275494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'b8659b510ca64fbe82e53b4c70f3b990', 'request_id': 'req-8298bd2e-7274-44b6-9178-cccc023940ac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.279627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'b8659b510ca64fbe82e53b4c70f3b990', 'request_id': 'req-8298bd2e-7274-44b6-9178-cccc023940ac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.286239 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2110/4181] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:08:49 2022] DELETE /identity/v3/domains/b8659b510ca64fbe82e53b4c70f3b990 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:49.291256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d961f-0430-4e71-bd2d-cfff95a2347b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:49.293057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1d0d961f-0430-4e71-bd2d-cfff95a2347b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:49.308229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d961f-0430-4e71-bd2d-cfff95a2347b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:49.386789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100ad46-3f91-4448-8709-905aacdeb72f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:49.387402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100ad46-3f91-4448-8709-905aacdeb72f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:49.387676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100ad46-3f91-4448-8709-905aacdeb72f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:49.387938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100ad46-3f91-4448-8709-905aacdeb72f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:49.388616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100ad46-3f91-4448-8709-905aacdeb72f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:49.390206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100ad46-3f91-4448-8709-905aacdeb72f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:49.407202 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2072/4182] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:08:49 2022] POST /identity/v3/domains => generated 249 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:08:49.411496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:49.422274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298bd2e-7274-44b6-9178-cccc023940ac tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:49.480685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd350e59-a35c-49f4-af6e-756b0abc8cb4 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:49.481359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd350e59-a35c-49f4-af6e-756b0abc8cb4 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:49.481634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd350e59-a35c-49f4-af6e-756b0abc8cb4 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:49.481904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd350e59-a35c-49f4-af6e-756b0abc8cb4 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/741f582ba9f34e5db5610c56beea03d1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:49.493672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd350e59-a35c-49f4-af6e-756b0abc8cb4 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=741f582ba9f34e5db5610c56beea03d1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:49.495224 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-dd350e59-a35c-49f4-af6e-756b0abc8cb4 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:08:49.495224 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:08:49.495224 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:08:49.496391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd350e59-a35c-49f4-af6e-756b0abc8cb4 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:49.527837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dd350e59-a35c-49f4-af6e-756b0abc8cb4 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '741f582ba9f34e5db5610c56beea03d1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.527837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dd350e59-a35c-49f4-af6e-756b0abc8cb4 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '741f582ba9f34e5db5610c56beea03d1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.558191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dd350e59-a35c-49f4-af6e-756b0abc8cb4 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '741f582ba9f34e5db5610c56beea03d1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.559209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dd350e59-a35c-49f4-af6e-756b0abc8cb4 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '741f582ba9f34e5db5610c56beea03d1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.562510 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2111/4183] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:08:49 2022] PATCH /identity/v3/domains/741f582ba9f34e5db5610c56beea03d1 => generated 250 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:49.566608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100ad46-3f91-4448-8709-905aacdeb72f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:49.568621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e100ad46-3f91-4448-8709-905aacdeb72f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:49.586071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100ad46-3f91-4448-8709-905aacdeb72f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:49.595903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:49.645905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:49.646882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:49.647162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:49.647430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/741f582ba9f34e5db5610c56beea03d1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:49.658178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=741f582ba9f34e5db5610c56beea03d1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:49.659231 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:08:49.659231 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:08:49.659231 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:08:49.660549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:49.685449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '741f582ba9f34e5db5610c56beea03d1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.691589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '741f582ba9f34e5db5610c56beea03d1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.743333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '741f582ba9f34e5db5610c56beea03d1', 'request_id': 'req-7114cb79-eeec-4b33-8d87-6752221d8f80'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.750642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '741f582ba9f34e5db5610c56beea03d1', 'request_id': 'req-7114cb79-eeec-4b33-8d87-6752221d8f80'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.760804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '741f582ba9f34e5db5610c56beea03d1', 'request_id': 'req-7114cb79-eeec-4b33-8d87-6752221d8f80'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.771849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] The token cache is being invalidate because project 741f582ba9f34e5db5610c56beea03d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:49.772507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.772943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.783078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '741f582ba9f34e5db5610c56beea03d1', 'request_id': 'req-7114cb79-eeec-4b33-8d87-6752221d8f80'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.783848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '741f582ba9f34e5db5610c56beea03d1', 'request_id': 'req-7114cb79-eeec-4b33-8d87-6752221d8f80'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.788678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '741f582ba9f34e5db5610c56beea03d1', 'request_id': 'req-7114cb79-eeec-4b33-8d87-6752221d8f80'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:49.794983 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2073/4184] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:08:49 2022] DELETE /identity/v3/domains/741f582ba9f34e5db5610c56beea03d1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:49.800079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd350e59-a35c-49f4-af6e-756b0abc8cb4 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:49.811910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd350e59-a35c-49f4-af6e-756b0abc8cb4 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:49.845267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:49.878432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320b5f3-152e-49dc-b873-09e199022efb tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:49.879162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320b5f3-152e-49dc-b873-09e199022efb tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:49.879528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320b5f3-152e-49dc-b873-09e199022efb tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:49.879842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320b5f3-152e-49dc-b873-09e199022efb tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:49.880601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320b5f3-152e-49dc-b873-09e199022efb tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:49.882537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320b5f3-152e-49dc-b873-09e199022efb tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:49.903821 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2112/4185] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:08:49 2022] POST /identity/v3/domains => generated 279 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:08:49.905349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:49.908002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:49.920835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114cb79-eeec-4b33-8d87-6752221d8f80 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:49.982997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:49.994112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e8d30-5c4d-4f67-9dd2-3c0a423cc30f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:49.994890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e8d30-5c4d-4f67-9dd2-3c0a423cc30f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:49.995515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e8d30-5c4d-4f67-9dd2-3c0a423cc30f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:49.996105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e8d30-5c4d-4f67-9dd2-3c0a423cc30f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/8f15834ac6504bfc8ecd5e44028d0631` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:50.011125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e8d30-5c4d-4f67-9dd2-3c0a423cc30f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=8f15834ac6504bfc8ecd5e44028d0631)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:50.012302 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-a94e8d30-5c4d-4f67-9dd2-3c0a423cc30f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:08:50.012302 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:08:50.012302 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:08:50.013393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e8d30-5c4d-4f67-9dd2-3c0a423cc30f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:50.035088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a94e8d30-5c4d-4f67-9dd2-3c0a423cc30f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '8f15834ac6504bfc8ecd5e44028d0631'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.035818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a94e8d30-5c4d-4f67-9dd2-3c0a423cc30f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '8f15834ac6504bfc8ecd5e44028d0631'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.065789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a94e8d30-5c4d-4f67-9dd2-3c0a423cc30f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '8f15834ac6504bfc8ecd5e44028d0631'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.066469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a94e8d30-5c4d-4f67-9dd2-3c0a423cc30f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '8f15834ac6504bfc8ecd5e44028d0631'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.069398 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2074/4186] 10.222.0.23 () {60 vars in 1250 bytes} [Sat Aug 20 21:08:49 2022] PATCH /identity/v3/domains/8f15834ac6504bfc8ecd5e44028d0631 => generated 277 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:50.073896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320b5f3-152e-49dc-b873-09e199022efb tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:50.075596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c320b5f3-152e-49dc-b873-09e199022efb tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:50.088436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320b5f3-152e-49dc-b873-09e199022efb tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:50.150958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d8f5b-e0ac-420b-88d5-8b6b1255404d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:50.151624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d8f5b-e0ac-420b-88d5-8b6b1255404d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:50.151901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d8f5b-e0ac-420b-88d5-8b6b1255404d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:50.152166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d8f5b-e0ac-420b-88d5-8b6b1255404d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/8f15834ac6504bfc8ecd5e44028d0631` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:50.165616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d8f5b-e0ac-420b-88d5-8b6b1255404d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=8f15834ac6504bfc8ecd5e44028d0631)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:50.167974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d8f5b-e0ac-420b-88d5-8b6b1255404d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:50.169782 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2113/4187] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:08:50 2022] GET /identity/v3/domains/8f15834ac6504bfc8ecd5e44028d0631 => generated 277 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:50.174014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e8d30-5c4d-4f67-9dd2-3c0a423cc30f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:50.187901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e8d30-5c4d-4f67-9dd2-3c0a423cc30f tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:50.193565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:50.194539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:50.194539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:50.195061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/8f15834ac6504bfc8ecd5e44028d0631` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:50.198804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=8f15834ac6504bfc8ecd5e44028d0631)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:50.200677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:50.240100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '8f15834ac6504bfc8ecd5e44028d0631'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.247776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '8f15834ac6504bfc8ecd5e44028d0631'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.317604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f15834ac6504bfc8ecd5e44028d0631', 'request_id': 'req-cbf31ff3-90b1-483c-bd7c-3b29210aa005'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.324854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f15834ac6504bfc8ecd5e44028d0631', 'request_id': 'req-cbf31ff3-90b1-483c-bd7c-3b29210aa005'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.335089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f15834ac6504bfc8ecd5e44028d0631', 'request_id': 'req-cbf31ff3-90b1-483c-bd7c-3b29210aa005'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.347544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] The token cache is being invalidate because project 8f15834ac6504bfc8ecd5e44028d0631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:50.347822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.348549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.357764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '8f15834ac6504bfc8ecd5e44028d0631', 'request_id': 'req-cbf31ff3-90b1-483c-bd7c-3b29210aa005'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.358514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '8f15834ac6504bfc8ecd5e44028d0631', 'request_id': 'req-cbf31ff3-90b1-483c-bd7c-3b29210aa005'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.362617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '8f15834ac6504bfc8ecd5e44028d0631', 'request_id': 'req-cbf31ff3-90b1-483c-bd7c-3b29210aa005'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.370007 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2075/4188] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:08:50 2022] DELETE /identity/v3/domains/8f15834ac6504bfc8ecd5e44028d0631 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:50.373936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d8f5b-e0ac-420b-88d5-8b6b1255404d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:50.375992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ce0d8f5b-e0ac-420b-88d5-8b6b1255404d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:50.393845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d8f5b-e0ac-420b-88d5-8b6b1255404d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:50.474630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6f854-5996-4d36-a6d4-035ffd3355df tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:50.475625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6f854-5996-4d36-a6d4-035ffd3355df tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:50.475971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6f854-5996-4d36-a6d4-035ffd3355df tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:50.476309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6f854-5996-4d36-a6d4-035ffd3355df tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:50.477091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6f854-5996-4d36-a6d4-035ffd3355df tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:50.479134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6f854-5996-4d36-a6d4-035ffd3355df tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:50.498606 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2114/4189] 10.222.0.23 () {58 vars in 1128 bytes} [Sat Aug 20 21:08:50 2022] GET /identity/v3/domains => generated 1349 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:50.502862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:50.513415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf31ff3-90b1-483c-bd7c-3b29210aa005 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:50.520487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50357d-ccdd-442f-a9e9-651406f07c0d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:50.521221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50357d-ccdd-442f-a9e9-651406f07c0d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:50.521626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50357d-ccdd-442f-a9e9-651406f07c0d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:50.521963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50357d-ccdd-442f-a9e9-651406f07c0d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/8f15834ac6504bfc8ecd5e44028d0631` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:50.530503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50357d-ccdd-442f-a9e9-651406f07c0d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9c50357d-ccdd-442f-a9e9-651406f07c0d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Could not find domain: 8f15834ac6504bfc8ecd5e44028d0631.: keystone.exception.DomainNotFound: Could not find domain: 8f15834ac6504bfc8ecd5e44028d0631. Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 8f15834ac6504bfc8ecd5e44028d0631. Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:08:50.536630 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:08:50.538072 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 8f15834ac6504bfc8ecd5e44028d0631. Aug 20 21:08:50.539219 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:08:50.540347 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2076/4190] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:08:50 2022] PATCH /identity/v3/domains/8f15834ac6504bfc8ecd5e44028d0631 => generated 112 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:08:50.543190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6f854-5996-4d36-a6d4-035ffd3355df tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:50.556622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6f854-5996-4d36-a6d4-035ffd3355df tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:50.565615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c37afb-8f6f-41e4-a36d-157147704006 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:50.566441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c37afb-8f6f-41e4-a36d-157147704006 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:50.566716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c37afb-8f6f-41e4-a36d-157147704006 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:50.567001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c37afb-8f6f-41e4-a36d-157147704006 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:50.567656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c37afb-8f6f-41e4-a36d-157147704006 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:50.569390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c37afb-8f6f-41e4-a36d-157147704006 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:50.596416 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2115/4191] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:08:50 2022] POST /identity/v3/domains => generated 275 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:50.598704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50357d-ccdd-442f-a9e9-651406f07c0d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:50.602395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c3265-2445-423e-a69c-55ea83b1f57c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:50.602674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c3265-2445-423e-a69c-55ea83b1f57c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:50.602945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c3265-2445-423e-a69c-55ea83b1f57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:50.610743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50357d-ccdd-442f-a9e9-651406f07c0d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:50.651625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a73c3265-2445-423e-a69c-55ea83b1f57c None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:50.654638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a73c3265-2445-423e-a69c-55ea83b1f57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:50.657020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:UPDATE=1,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:50.695727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0d1de-0528-44ba-be98-5bcf5d1ee8fe tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:50.696668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0d1de-0528-44ba-be98-5bcf5d1ee8fe tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:50.697734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0d1de-0528-44ba-be98-5bcf5d1ee8fe tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:50.698196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0d1de-0528-44ba-be98-5bcf5d1ee8fe tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:50.699363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0d1de-0528-44ba-be98-5bcf5d1ee8fe tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:50.701622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0d1de-0528-44ba-be98-5bcf5d1ee8fe tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:50.721015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:50.742789 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2077/4192] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:08:50 2022] POST /identity/v3/users => generated 348 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:50.745882 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2116/4193] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:08:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:50.747408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0d1de-0528-44ba-be98-5bcf5d1ee8fe tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:50.758461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0d1de-0528-44ba-be98-5bcf5d1ee8fe tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:50.760715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c3265-2445-423e-a69c-55ea83b1f57c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:50.762295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a73c3265-2445-423e-a69c-55ea83b1f57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:50.764506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409218e-0e22-46e9-ad19-128df1196581 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:50.765368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409218e-0e22-46e9-ad19-128df1196581 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:50.765741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409218e-0e22-46e9-ad19-128df1196581 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:50.766116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409218e-0e22-46e9-ad19-128df1196581 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/groups` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:50.766863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409218e-0e22-46e9-ad19-128df1196581 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:50.768004 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-f409218e-0e22-46e9-ad19-128df1196581 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:08:50.768004 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:08:50.768004 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:08:50.769414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409218e-0e22-46e9-ad19-128df1196581 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:50.790649 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2078/4194] 10.222.0.23 () {60 vars in 1147 bytes} [Sat Aug 20 21:08:50 2022] POST /identity/v3/groups => generated 325 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:50.797532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409218e-0e22-46e9-ad19-128df1196581 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:50.810893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409218e-0e22-46e9-ad19-128df1196581 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:50.816389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40aabd4-9759-43b5-a328-629b481bef2b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:50.817105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aabd4-9759-43b5-a328-629b481bef2b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:50.817403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aabd4-9759-43b5-a328-629b481bef2b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:50.817648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aabd4-9759-43b5-a328-629b481bef2b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/85f891de604e441b86d60634cea55f8d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:50.819815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40aabd4-9759-43b5-a328-629b481bef2b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=85f891de604e441b86d60634cea55f8d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:50.821822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40aabd4-9759-43b5-a328-629b481bef2b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:50.845984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c40aabd4-9759-43b5-a328-629b481bef2b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '85f891de604e441b86d60634cea55f8d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.846967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c40aabd4-9759-43b5-a328-629b481bef2b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '85f891de604e441b86d60634cea55f8d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.847437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c3265-2445-423e-a69c-55ea83b1f57c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:50.888584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c40aabd4-9759-43b5-a328-629b481bef2b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '85f891de604e441b86d60634cea55f8d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.889381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c40aabd4-9759-43b5-a328-629b481bef2b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '85f891de604e441b86d60634cea55f8d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:50.892296 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2079/4195] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:08:50 2022] PATCH /identity/v3/domains/85f891de604e441b86d60634cea55f8d => generated 276 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:50.895971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40aabd4-9759-43b5-a328-629b481bef2b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:50.898311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c40aabd4-9759-43b5-a328-629b481bef2b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:50.935983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845ec4a-1173-47de-a317-2525f5976f53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:50.936647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845ec4a-1173-47de-a317-2525f5976f53 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:50.936922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845ec4a-1173-47de-a317-2525f5976f53 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:50.937189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845ec4a-1173-47de-a317-2525f5976f53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:50.939179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6845ec4a-1173-47de-a317-2525f5976f53 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:50.976753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40aabd4-9759-43b5-a328-629b481bef2b tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:50.987614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:50.988352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:50.988788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:50.989695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/85f891de604e441b86d60634cea55f8d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:51.001088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=85f891de604e441b86d60634cea55f8d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:51.002815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:51.021978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845ec4a-1173-47de-a317-2525f5976f53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:51.023516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845ec4a-1173-47de-a317-2525f5976f53 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:51.028761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '85f891de604e441b86d60634cea55f8d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.033335 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2117/4196] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:08:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:08:51.036428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '85f891de604e441b86d60634cea55f8d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.091183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:UPDATE=1,keystone:DELETE=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:51.191981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dde4d08e4a24c3485129fb15202706e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.202359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dde4d08e4a24c3485129fb15202706e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.219506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dde4d08e4a24c3485129fb15202706e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.220544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dde4d08e4a24c3485129fb15202706e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.221219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dde4d08e4a24c3485129fb15202706e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.295955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85f891de604e441b86d60634cea55f8d', 'request_id': 'req-2886db35-4376-4da9-ab3b-3d572a1e8176'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.304245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85f891de604e441b86d60634cea55f8d', 'request_id': 'req-2886db35-4376-4da9-ab3b-3d572a1e8176'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.316555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85f891de604e441b86d60634cea55f8d', 'request_id': 'req-2886db35-4376-4da9-ab3b-3d572a1e8176'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.353435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] The token cache is being invalidate because project 85f891de604e441b86d60634cea55f8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:51.354165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.355253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.366023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '85f891de604e441b86d60634cea55f8d', 'request_id': 'req-2886db35-4376-4da9-ab3b-3d572a1e8176'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.366750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '85f891de604e441b86d60634cea55f8d', 'request_id': 'req-2886db35-4376-4da9-ab3b-3d572a1e8176'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.375220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '85f891de604e441b86d60634cea55f8d', 'request_id': 'req-2886db35-4376-4da9-ab3b-3d572a1e8176'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:51.386152 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2080/4197] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:08:50 2022] DELETE /identity/v3/domains/85f891de604e441b86d60634cea55f8d => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:08:51.390027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845ec4a-1173-47de-a317-2525f5976f53 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:51.401272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845ec4a-1173-47de-a317-2525f5976f53 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:51.433020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:51.483217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce13d08-516a-4475-9117-5de7b644739a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:51.484118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce13d08-516a-4475-9117-5de7b644739a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:51.484657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce13d08-516a-4475-9117-5de7b644739a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:51.485182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce13d08-516a-4475-9117-5de7b644739a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/85f891de604e441b86d60634cea55f8d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:51.494163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce13d08-516a-4475-9117-5de7b644739a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=85f891de604e441b86d60634cea55f8d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:51.496783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce13d08-516a-4475-9117-5de7b644739a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-3ce13d08-516a-4475-9117-5de7b644739a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Could not find domain: 85f891de604e441b86d60634cea55f8d.: keystone.exception.DomainNotFound: Could not find domain: 85f891de604e441b86d60634cea55f8d. Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 85f891de604e441b86d60634cea55f8d. Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:08:51.518215 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:08:51.519409 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 85f891de604e441b86d60634cea55f8d. Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:08:51.521325 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2118/4198] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:08:51 2022] GET /identity/v3/domains/85f891de604e441b86d60634cea55f8d => generated 112 bytes in 130 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:08:51.525474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:51.527013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:51.542110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886db35-4376-4da9-ab3b-3d572a1e8176 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:51.549053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ad262-0a55-449e-b5fc-16df6a4bbd42 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:51.549932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ad262-0a55-449e-b5fc-16df6a4bbd42 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:51.550371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ad262-0a55-449e-b5fc-16df6a4bbd42 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:51.550814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ad262-0a55-449e-b5fc-16df6a4bbd42 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/users/3dde4d08e4a24c3485129fb15202706e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:51.561027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ad262-0a55-449e-b5fc-16df6a4bbd42 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:get_user(user_id=3dde4d08e4a24c3485129fb15202706e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:51.563106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ad262-0a55-449e-b5fc-16df6a4bbd42 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-3b7ad262-0a55-449e-b5fc-16df6a4bbd42 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Could not find user: 3dde4d08e4a24c3485129fb15202706e.: keystone.exception.UserNotFound: Could not find user: 3dde4d08e4a24c3485129fb15202706e. Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 20 21:08:51.576048 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:08:51.577357 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:08:51.578638 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:08:51.578638 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:08:51.578638 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:08:51.578638 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:08:51.578638 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 21:08:51.578638 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 21:08:51.578638 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 21:08:51.578638 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 21:08:51.578638 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 21:08:51.578638 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 21:08:51.578638 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 3dde4d08e4a24c3485129fb15202706e. Aug 20 21:08:51.578638 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:08:51.578638 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2081/4199] 10.222.0.23 () {58 vars in 1221 bytes} [Sat Aug 20 21:08:51 2022] GET /identity/v3/users/3dde4d08e4a24c3485129fb15202706e => generated 110 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:08:51.582105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce13d08-516a-4475-9117-5de7b644739a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:51.593129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce13d08-516a-4475-9117-5de7b644739a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:51.597954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999f2ec-cc9e-43b7-b74b-a73051ff8633 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:51.598587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999f2ec-cc9e-43b7-b74b-a73051ff8633 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:51.598863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999f2ec-cc9e-43b7-b74b-a73051ff8633 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:51.599134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999f2ec-cc9e-43b7-b74b-a73051ff8633 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/groups/3f7a31a7b2de4c24b20b610b5461cfc4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:51.605188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999f2ec-cc9e-43b7-b74b-a73051ff8633 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:get_group(group_id=3f7a31a7b2de4c24b20b610b5461cfc4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:51.606220 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-1999f2ec-cc9e-43b7-b74b-a73051ff8633 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:08:51.606220 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:08:51.606220 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:08:51.607366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999f2ec-cc9e-43b7-b74b-a73051ff8633 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-1999f2ec-cc9e-43b7-b74b-a73051ff8633 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Could not find group: 3f7a31a7b2de4c24b20b610b5461cfc4.: keystone.exception.GroupNotFound: Could not find group: 3f7a31a7b2de4c24b20b610b5461cfc4. Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 58, in get Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_group(group_id) Aug 20 21:08:51.625586 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 70, in _get_group Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.identity_api.get_group(group_id)) Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:08:51.626692 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:08:51.627711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:08:51.627711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:08:51.627711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:08:51.627711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:08:51.627711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Aug 20 21:08:51.627711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Aug 20 21:08:51.627711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Aug 20 21:08:51.627711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Aug 20 21:08:51.627711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Aug 20 21:08:51.627711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Aug 20 21:08:51.627711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: 3f7a31a7b2de4c24b20b610b5461cfc4. Aug 20 21:08:51.627711 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:08:51.627711 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2119/4200] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:08:51 2022] GET /identity/v3/groups/3f7a31a7b2de4c24b20b610b5461cfc4 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:08:51.631233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ad262-0a55-449e-b5fc-16df6a4bbd42 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:51.643565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ad262-0a55-449e-b5fc-16df6a4bbd42 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:51.648616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd718969-4f2f-4afa-8cdc-3b1a3bf00cfc tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:51.649312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd718969-4f2f-4afa-8cdc-3b1a3bf00cfc tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:51.649648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd718969-4f2f-4afa-8cdc-3b1a3bf00cfc tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:51.650000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd718969-4f2f-4afa-8cdc-3b1a3bf00cfc tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/groups/3f7a31a7b2de4c24b20b610b5461cfc4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:51.652317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=23,keystone:SELECT=75,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:51.655736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd718969-4f2f-4afa-8cdc-3b1a3bf00cfc tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-fd718969-4f2f-4afa-8cdc-3b1a3bf00cfc tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Could not find group: 3f7a31a7b2de4c24b20b610b5461cfc4.: keystone.exception.GroupNotFound: Could not find group: 3f7a31a7b2de4c24b20b610b5461cfc4. Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 133, in delete Aug 20 21:08:51.662448 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_group') Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:08:51.663656 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: 3f7a31a7b2de4c24b20b610b5461cfc4. Aug 20 21:08:51.664743 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:08:51.666946 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2082/4201] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:08:51 2022] DELETE /identity/v3/groups/3f7a31a7b2de4c24b20b610b5461cfc4 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:08:51.668121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999f2ec-cc9e-43b7-b74b-a73051ff8633 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:51.682912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999f2ec-cc9e-43b7-b74b-a73051ff8633 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:51.687951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12d401-89a5-41d8-918d-05020fb26b0e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:51.688642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12d401-89a5-41d8-918d-05020fb26b0e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:51.688914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12d401-89a5-41d8-918d-05020fb26b0e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:51.689177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12d401-89a5-41d8-918d-05020fb26b0e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/users/3dde4d08e4a24c3485129fb15202706e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:51.700326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12d401-89a5-41d8-918d-05020fb26b0e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:delete_user(user_id=3dde4d08e4a24c3485129fb15202706e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:51.702373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12d401-89a5-41d8-918d-05020fb26b0e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-bd12d401-89a5-41d8-918d-05020fb26b0e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Could not find user: 3dde4d08e4a24c3485129fb15202706e.: keystone.exception.UserNotFound: Could not find user: 3dde4d08e4a24c3485129fb15202706e. Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 275, in delete Aug 20 21:08:51.716236 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application PROVIDERS.identity_api.delete_user(user_id) Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1207, in delete_user Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application user_old = self.get_user(user_id) Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:08:51.717330 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 3dde4d08e4a24c3485129fb15202706e. Aug 20 21:08:51.718410 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:08:51.719528 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2120/4202] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:08:51 2022] DELETE /identity/v3/users/3dde4d08e4a24c3485129fb15202706e => generated 110 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:08:51.722082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd718969-4f2f-4afa-8cdc-3b1a3bf00cfc tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:51.734986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd718969-4f2f-4afa-8cdc-3b1a3bf00cfc tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:51.740729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f7b08-d407-4b5e-95dd-200127eef1b2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:51.741590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f7b08-d407-4b5e-95dd-200127eef1b2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:51.741877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f7b08-d407-4b5e-95dd-200127eef1b2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:51.742180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f7b08-d407-4b5e-95dd-200127eef1b2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/85f891de604e441b86d60634cea55f8d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:51.750589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f7b08-d407-4b5e-95dd-200127eef1b2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-263f7b08-d407-4b5e-95dd-200127eef1b2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Could not find domain: 85f891de604e441b86d60634cea55f8d.: keystone.exception.DomainNotFound: Could not find domain: 85f891de604e441b86d60634cea55f8d. Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 85f891de604e441b86d60634cea55f8d. Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:08:51.756818 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:08:51.757995 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 85f891de604e441b86d60634cea55f8d. Aug 20 21:08:51.758957 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:08:51.759962 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2083/4203] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:08:51 2022] PATCH /identity/v3/domains/85f891de604e441b86d60634cea55f8d => generated 112 bytes in 36 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:08:51.764936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12d401-89a5-41d8-918d-05020fb26b0e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:51.782228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12d401-89a5-41d8-918d-05020fb26b0e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:51.789449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1370811-024c-433d-9fac-01fd5faba178 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:51.790106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1370811-024c-433d-9fac-01fd5faba178 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:51.790379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1370811-024c-433d-9fac-01fd5faba178 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:51.790647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1370811-024c-433d-9fac-01fd5faba178 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:51.791267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1370811-024c-433d-9fac-01fd5faba178 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:51.792956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1370811-024c-433d-9fac-01fd5faba178 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:51.810365 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2121/4204] 10.222.0.23 () {58 vars in 1128 bytes} [Sat Aug 20 21:08:51 2022] GET /identity/v3/domains => generated 1349 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:51.815355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f7b08-d407-4b5e-95dd-200127eef1b2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:51.827489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f7b08-d407-4b5e-95dd-200127eef1b2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:51.835512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205184f-8caa-4d2b-9d53-c50f8a8dad6d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:51.836358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205184f-8caa-4d2b-9d53-c50f8a8dad6d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:51.836820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205184f-8caa-4d2b-9d53-c50f8a8dad6d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:51.837106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205184f-8caa-4d2b-9d53-c50f8a8dad6d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:51.838154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205184f-8caa-4d2b-9d53-c50f8a8dad6d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Adding query filter params (enabled=True) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:08:51.838535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205184f-8caa-4d2b-9d53-c50f8a8dad6d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:51.840264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205184f-8caa-4d2b-9d53-c50f8a8dad6d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:51.864425 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2084/4205] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:08:51 2022] GET /identity/v3/domains?enabled=True => generated 1095 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:51.869308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1370811-024c-433d-9fac-01fd5faba178 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:51.881533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1370811-024c-433d-9fac-01fd5faba178 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:51.886803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774b202-0bdc-4b10-91a4-9965e6eb8a9e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:51.887648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774b202-0bdc-4b10-91a4-9965e6eb8a9e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:51.888128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774b202-0bdc-4b10-91a4-9965e6eb8a9e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:51.888640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774b202-0bdc-4b10-91a4-9965e6eb8a9e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:51.889491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774b202-0bdc-4b10-91a4-9965e6eb8a9e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Adding query filter params (name=tempest-test_domain-1589274629) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:08:51.890013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774b202-0bdc-4b10-91a4-9965e6eb8a9e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:51.892079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774b202-0bdc-4b10-91a4-9965e6eb8a9e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:51.905730 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2122/4206] 10.222.0.23 () {58 vars in 1199 bytes} [Sat Aug 20 21:08:51 2022] GET /identity/v3/domains?name=tempest-test_domain-1589274629 => generated 409 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:51.910490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205184f-8caa-4d2b-9d53-c50f8a8dad6d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:51.921984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205184f-8caa-4d2b-9d53-c50f8a8dad6d tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:51.926908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb286449-f5ae-4db3-9cab-50bc70fe2399 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:51.927546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb286449-f5ae-4db3-9cab-50bc70fe2399 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:51.927880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb286449-f5ae-4db3-9cab-50bc70fe2399 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:51.929173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb286449-f5ae-4db3-9cab-50bc70fe2399 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/85f891de604e441b86d60634cea55f8d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:51.938351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb286449-f5ae-4db3-9cab-50bc70fe2399 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-cb286449-f5ae-4db3-9cab-50bc70fe2399 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Could not find domain: 85f891de604e441b86d60634cea55f8d.: keystone.exception.DomainNotFound: Could not find domain: 85f891de604e441b86d60634cea55f8d. Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 85f891de604e441b86d60634cea55f8d. Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:08:51.952752 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:08:51.953907 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 85f891de604e441b86d60634cea55f8d. Aug 20 21:08:51.955035 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:08:51.956126 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2085/4207] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:08:51 2022] PATCH /identity/v3/domains/85f891de604e441b86d60634cea55f8d => generated 112 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:08:51.958289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774b202-0bdc-4b10-91a4-9965e6eb8a9e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:51.973375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774b202-0bdc-4b10-91a4-9965e6eb8a9e tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:51.980127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f34f3-a81a-4e46-b4b5-90ba290c53f1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:51.980939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f34f3-a81a-4e46-b4b5-90ba290c53f1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:51.981215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f34f3-a81a-4e46-b4b5-90ba290c53f1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:51.981481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f34f3-a81a-4e46-b4b5-90ba290c53f1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/0f4ebbd91e164d92b2b77eaaf3f0bc60` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:51.994249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f34f3-a81a-4e46-b4b5-90ba290c53f1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=0f4ebbd91e164d92b2b77eaaf3f0bc60)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:51.995937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f34f3-a81a-4e46-b4b5-90ba290c53f1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:52.056113 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2123/4208] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:08:51 2022] PATCH /identity/v3/domains/0f4ebbd91e164d92b2b77eaaf3f0bc60 => generated 278 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:08:52.057514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb286449-f5ae-4db3-9cab-50bc70fe2399 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:52.071187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb286449-f5ae-4db3-9cab-50bc70fe2399 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:52.075812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:52.076473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:52.076746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:52.077130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/0f4ebbd91e164d92b2b77eaaf3f0bc60` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:52.089048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=0f4ebbd91e164d92b2b77eaaf3f0bc60)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:52.090938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:52.299888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '0f4ebbd91e164d92b2b77eaaf3f0bc60'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.304300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '0f4ebbd91e164d92b2b77eaaf3f0bc60'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.378005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f4ebbd91e164d92b2b77eaaf3f0bc60', 'request_id': 'req-3311457e-56d0-4b97-b4b7-eb411278c154'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.391051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f4ebbd91e164d92b2b77eaaf3f0bc60', 'request_id': 'req-3311457e-56d0-4b97-b4b7-eb411278c154'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.403602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f4ebbd91e164d92b2b77eaaf3f0bc60', 'request_id': 'req-3311457e-56d0-4b97-b4b7-eb411278c154'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.421908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] The token cache is being invalidate because project 0f4ebbd91e164d92b2b77eaaf3f0bc60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:52.422212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.425661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.448971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '0f4ebbd91e164d92b2b77eaaf3f0bc60', 'request_id': 'req-3311457e-56d0-4b97-b4b7-eb411278c154'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.450084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '0f4ebbd91e164d92b2b77eaaf3f0bc60', 'request_id': 'req-3311457e-56d0-4b97-b4b7-eb411278c154'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.454687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '0f4ebbd91e164d92b2b77eaaf3f0bc60', 'request_id': 'req-3311457e-56d0-4b97-b4b7-eb411278c154'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.462053 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2086/4209] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:08:52 2022] DELETE /identity/v3/domains/0f4ebbd91e164d92b2b77eaaf3f0bc60 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:52.466553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f34f3-a81a-4e46-b4b5-90ba290c53f1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:52.469335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d88f34f3-a81a-4e46-b4b5-90ba290c53f1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:52.490781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f34f3-a81a-4e46-b4b5-90ba290c53f1 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:52.611496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb035a0-bfe8-423e-a2b4-b0f67757e7c2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:52.613017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb035a0-bfe8-423e-a2b4-b0f67757e7c2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:52.613239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb035a0-bfe8-423e-a2b4-b0f67757e7c2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:52.614499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb035a0-bfe8-423e-a2b4-b0f67757e7c2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/7db27eb358174704a01101b1c79325d8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:52.628833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb035a0-bfe8-423e-a2b4-b0f67757e7c2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=7db27eb358174704a01101b1c79325d8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:52.631409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb035a0-bfe8-423e-a2b4-b0f67757e7c2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:52.673617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:52.687019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-acb035a0-bfe8-423e-a2b4-b0f67757e7c2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '7db27eb358174704a01101b1c79325d8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.692020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-acb035a0-bfe8-423e-a2b4-b0f67757e7c2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '7db27eb358174704a01101b1c79325d8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.740765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-acb035a0-bfe8-423e-a2b4-b0f67757e7c2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '7db27eb358174704a01101b1c79325d8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.741617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-acb035a0-bfe8-423e-a2b4-b0f67757e7c2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '7db27eb358174704a01101b1c79325d8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.745776 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2124/4210] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:08:52 2022] PATCH /identity/v3/domains/7db27eb358174704a01101b1c79325d8 => generated 277 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:08:52.750593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:52.764849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311457e-56d0-4b97-b4b7-eb411278c154 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:52.826175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:52.835304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:52.836176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:52.836667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:52.837921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/7db27eb358174704a01101b1c79325d8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:52.860330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=7db27eb358174704a01101b1c79325d8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:52.862556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:52.898262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '7db27eb358174704a01101b1c79325d8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:52.902622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '7db27eb358174704a01101b1c79325d8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.012066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7db27eb358174704a01101b1c79325d8', 'request_id': 'req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.024746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7db27eb358174704a01101b1c79325d8', 'request_id': 'req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.056930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7db27eb358174704a01101b1c79325d8', 'request_id': 'req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.076411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] The token cache is being invalidate because project 7db27eb358174704a01101b1c79325d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:53.078528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.080030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.102517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '7db27eb358174704a01101b1c79325d8', 'request_id': 'req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.104740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '7db27eb358174704a01101b1c79325d8', 'request_id': 'req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.114071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '7db27eb358174704a01101b1c79325d8', 'request_id': 'req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.129761 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2087/4211] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:08:52 2022] DELETE /identity/v3/domains/7db27eb358174704a01101b1c79325d8 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:53.131427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb035a0-bfe8-423e-a2b4-b0f67757e7c2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:53.133179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-acb035a0-bfe8-423e-a2b4-b0f67757e7c2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:53.156367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb035a0-bfe8-423e-a2b4-b0f67757e7c2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:53.297471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a436de0-c8d0-496e-82fc-175e8c3fd64a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:53.298307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a436de0-c8d0-496e-82fc-175e8c3fd64a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:53.298686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a436de0-c8d0-496e-82fc-175e8c3fd64a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:53.299113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a436de0-c8d0-496e-82fc-175e8c3fd64a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/55b598013fa54c33b974c40483b8fd85` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:53.314373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a436de0-c8d0-496e-82fc-175e8c3fd64a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=55b598013fa54c33b974c40483b8fd85)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:53.316412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a436de0-c8d0-496e-82fc-175e8c3fd64a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:53.349138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3a436de0-c8d0-496e-82fc-175e8c3fd64a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '55b598013fa54c33b974c40483b8fd85'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.350049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3a436de0-c8d0-496e-82fc-175e8c3fd64a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '55b598013fa54c33b974c40483b8fd85'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.403809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3a436de0-c8d0-496e-82fc-175e8c3fd64a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '55b598013fa54c33b974c40483b8fd85'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.404554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3a436de0-c8d0-496e-82fc-175e8c3fd64a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '55b598013fa54c33b974c40483b8fd85'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.410362 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2125/4212] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:08:53 2022] PATCH /identity/v3/domains/55b598013fa54c33b974c40483b8fd85 => generated 278 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:53.413664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:53.415240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:53.432142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579aa10b-432d-4aaa-8f7f-69093cdcf9f2 tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:53.514972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9311f440a24d3b8b538d79fae5f201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '333dacfda99343f79d2c5c80e3089d1f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:53.515632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:53.515910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:53.516185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] PATH_INFO: `/v3/domains/55b598013fa54c33b974c40483b8fd85` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:53.528704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=55b598013fa54c33b974c40483b8fd85)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:53.530704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:53.565094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '55b598013fa54c33b974c40483b8fd85'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.571447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '55b598013fa54c33b974c40483b8fd85'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.574017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:53.660953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55b598013fa54c33b974c40483b8fd85', 'request_id': 'req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.674545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55b598013fa54c33b974c40483b8fd85', 'request_id': 'req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.687036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55b598013fa54c33b974c40483b8fd85', 'request_id': 'req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.704837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] The token cache is being invalidate because project 55b598013fa54c33b974c40483b8fd85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:53.705140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.705869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.725055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '55b598013fa54c33b974c40483b8fd85', 'request_id': 'req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.727049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '55b598013fa54c33b974c40483b8fd85', 'request_id': 'req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.734711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '55b598013fa54c33b974c40483b8fd85', 'request_id': 'req-22ae8b5e-dce2-40ec-adf4-a1405c1cd94a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:53.745320 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2088/4213] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:08:53 2022] DELETE /identity/v3/domains/55b598013fa54c33b974c40483b8fd85 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:53.746876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a436de0-c8d0-496e-82fc-175e8c3fd64a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:53.749066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3a436de0-c8d0-496e-82fc-175e8c3fd64a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:53.911953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a436de0-c8d0-496e-82fc-175e8c3fd64a tempest-DomainsTestJSON-436849647 tempest-DomainsTestJSON-436849647-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:54.019357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:UPDATE=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:54.049571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:54.050466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:54.050995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:54.051556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] PATH_INFO: `/v3/users/4d8730857fd041ec83a7bb9c8e621175` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:54.053579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d8730857fd041ec83a7bb9c8e621175)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:54.056947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:54.112993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d8730857fd041ec83a7bb9c8e621175'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:54.123420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d8730857fd041ec83a7bb9c8e621175'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:54.130837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d8730857fd041ec83a7bb9c8e621175'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:54.143916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d8730857fd041ec83a7bb9c8e621175'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:54.144667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d8730857fd041ec83a7bb9c8e621175'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:54.150989 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2126/4214] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:08:53 2022] DELETE /identity/v3/users/4d8730857fd041ec83a7bb9c8e621175 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:54.155361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd4c8d-12d8-48cf-a79e-07b2cb627017 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:54.155841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd4c8d-12d8-48cf-a79e-07b2cb627017 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:54.155959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd4c8d-12d8-48cf-a79e-07b2cb627017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:54.175730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-72dd4c8d-12d8-48cf-a79e-07b2cb627017 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:54.176974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-72dd4c8d-12d8-48cf-a79e-07b2cb627017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:54.258702 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2089/4215] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:54.268742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:54.270220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:54.338936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03021e08-47df-4a93-a205-2aa95394b0f8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:54.343577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ae823-eb12-4738-bad3-3f7dd82ff6eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:54.343577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ae823-eb12-4738-bad3-3f7dd82ff6eb service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:54.343840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ae823-eb12-4738-bad3-3f7dd82ff6eb service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:54.344245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ae823-eb12-4738-bad3-3f7dd82ff6eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:54.357441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ae823-eb12-4738-bad3-3f7dd82ff6eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:54.358879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ae823-eb12-4738-bad3-3f7dd82ff6eb service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:54.370587 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2127/4216] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:08:54.721711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd4c8d-12d8-48cf-a79e-07b2cb627017 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:54.723932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-72dd4c8d-12d8-48cf-a79e-07b2cb627017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:54.747824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd4c8d-12d8-48cf-a79e-07b2cb627017 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:54.763320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:54.763992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:54.764265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:54.764557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] PATH_INFO: `/v3/projects/225e7973c50e4b8986305fe88e4d000e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:54.766896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=225e7973c50e4b8986305fe88e4d000e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:54.769703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:54.837929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '225e7973c50e4b8986305fe88e4d000e', 'request_id': 'req-599d50ee-7424-453a-81fc-6812bbd0ab73'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:54.851770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '225e7973c50e4b8986305fe88e4d000e', 'request_id': 'req-599d50ee-7424-453a-81fc-6812bbd0ab73'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:54.869379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '225e7973c50e4b8986305fe88e4d000e', 'request_id': 'req-599d50ee-7424-453a-81fc-6812bbd0ab73'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:54.876915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:54.900756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] The token cache is being invalidate because project 225e7973c50e4b8986305fe88e4d000e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:54.901102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:54.901789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:54.903653 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2090/4217] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:08:54 2022] DELETE /identity/v3/projects/225e7973c50e4b8986305fe88e4d000e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:54.912723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ae823-eb12-4738-bad3-3f7dd82ff6eb service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:54.924748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ae823-eb12-4738-bad3-3f7dd82ff6eb service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:55.123938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f2012-e3e7-4d07-b1fa-e69cf05a8d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:55.124792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f2012-e3e7-4d07-b1fa-e69cf05a8d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:55.125143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f2012-e3e7-4d07-b1fa-e69cf05a8d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:55.125446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f2012-e3e7-4d07-b1fa-e69cf05a8d19 admin admin] PATH_INFO: `/v3/users/3a9311f440a24d3b8b538d79fae5f201` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:55.126918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f2012-e3e7-4d07-b1fa-e69cf05a8d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a9311f440a24d3b8b538d79fae5f201)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:55.129471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f2012-e3e7-4d07-b1fa-e69cf05a8d19 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:55.135545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:55.208434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-568f2012-e3e7-4d07-b1fa-e69cf05a8d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a9311f440a24d3b8b538d79fae5f201'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:55.233928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-568f2012-e3e7-4d07-b1fa-e69cf05a8d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a9311f440a24d3b8b538d79fae5f201'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:55.254724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-568f2012-e3e7-4d07-b1fa-e69cf05a8d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a9311f440a24d3b8b538d79fae5f201'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:55.339419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-568f2012-e3e7-4d07-b1fa-e69cf05a8d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a9311f440a24d3b8b538d79fae5f201'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:55.340738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-568f2012-e3e7-4d07-b1fa-e69cf05a8d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a9311f440a24d3b8b538d79fae5f201'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:55.350595 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2128/4218] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:08:54 2022] DELETE /identity/v3/users/3a9311f440a24d3b8b538d79fae5f201 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:55.936094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:55.938061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:55.953593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d50ee-7424-453a-81fc-6812bbd0ab73 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:56.046316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:56.049211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:56.049505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:56.049784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f admin admin] PATH_INFO: `/v3/projects/333dacfda99343f79d2c5c80e3089d1f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:56.051765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=333dacfda99343f79d2c5c80e3089d1f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:56.053890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:56.127956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '333dacfda99343f79d2c5c80e3089d1f', 'request_id': 'req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:56.137578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '333dacfda99343f79d2c5c80e3089d1f', 'request_id': 'req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:56.148869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '333dacfda99343f79d2c5c80e3089d1f', 'request_id': 'req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:56.166558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f admin admin] The token cache is being invalidate because project 333dacfda99343f79d2c5c80e3089d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:56.166935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:56.167567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e8173f34-af5b-4c33-a94b-bc2e9a4aae9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:56.177277 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2091/4219] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:08:55 2022] DELETE /identity/v3/projects/333dacfda99343f79d2c5c80e3089d1f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:56.180320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8bdc-9ba6-4149-86b0-906ef8dcda37 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:56.180866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8bdc-9ba6-4149-86b0-906ef8dcda37 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:56.180980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8bdc-9ba6-4149-86b0-906ef8dcda37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:56.267225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-713a8bdc-9ba6-4149-86b0-906ef8dcda37 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:56.271064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-713a8bdc-9ba6-4149-86b0-906ef8dcda37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:56.456283 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2129/4220] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:56.458079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa047a8-4c56-403b-a84a-a5305da5eefe None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:56.458459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa047a8-4c56-403b-a84a-a5305da5eefe None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:56.459018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa047a8-4c56-403b-a84a-a5305da5eefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:56.498435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9fa047a8-4c56-403b-a84a-a5305da5eefe None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:56.499653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9fa047a8-4c56-403b-a84a-a5305da5eefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:56.509287 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2092/4221] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:56.514370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a8bdc-9ba6-4149-86b0-906ef8dcda37 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:56.529357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a8bdc-9ba6-4149-86b0-906ef8dcda37 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:56.535697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5d0c6-97b4-42e2-a796-1c351a1b737e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:56.536588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5d0c6-97b4-42e2-a796-1c351a1b737e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:56.536901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5d0c6-97b4-42e2-a796-1c351a1b737e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:56.537196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5d0c6-97b4-42e2-a796-1c351a1b737e admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:56.538049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5d0c6-97b4-42e2-a796-1c351a1b737e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:08:56.538147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5d0c6-97b4-42e2-a796-1c351a1b737e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:56.539785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5d0c6-97b4-42e2-a796-1c351a1b737e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:56.560449 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2130/4222] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:56.561928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa047a8-4c56-403b-a84a-a5305da5eefe None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:56.573349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa047a8-4c56-403b-a84a-a5305da5eefe None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:56.580606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0cee3-8840-4a8e-978d-c0efc0afe610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:56.581720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0cee3-8840-4a8e-978d-c0efc0afe610 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:56.582245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0cee3-8840-4a8e-978d-c0efc0afe610 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:56.582743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0cee3-8840-4a8e-978d-c0efc0afe610 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:56.583666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0cee3-8840-4a8e-978d-c0efc0afe610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:56.586294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0cee3-8840-4a8e-978d-c0efc0afe610 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:56.633330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5d0c6-97b4-42e2-a796-1c351a1b737e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:56.636461 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2093/4223] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:08:56 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:56.649757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5d0c6-97b4-42e2-a796-1c351a1b737e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:56.754452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af210c66-c8c9-4426-bbb2-304543b95e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:56.755090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af210c66-c8c9-4426-bbb2-304543b95e19 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:56.755364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af210c66-c8c9-4426-bbb2-304543b95e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:56.755666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af210c66-c8c9-4426-bbb2-304543b95e19 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:56.756309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af210c66-c8c9-4426-bbb2-304543b95e19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:56.758308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af210c66-c8c9-4426-bbb2-304543b95e19 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:56.781212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:56.792248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0cee3-8840-4a8e-978d-c0efc0afe610 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:56.798445 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2131/4224] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:08:56 2022] POST /identity/v3/users => generated 294 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:08:56.806532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0cee3-8840-4a8e-978d-c0efc0afe610 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:56.814741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e56a6c-06a2-45ee-94a9-57c4e6d87e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:56.815421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e56a6c-06a2-45ee-94a9-57c4e6d87e89 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:56.815821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e56a6c-06a2-45ee-94a9-57c4e6d87e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:56.816103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e56a6c-06a2-45ee-94a9-57c4e6d87e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:56.816995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e56a6c-06a2-45ee-94a9-57c4e6d87e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:56.818950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e56a6c-06a2-45ee-94a9-57c4e6d87e89 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:56.831304 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2094/4225] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:08:56.836726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af210c66-c8c9-4426-bbb2-304543b95e19 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:56.849660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af210c66-c8c9-4426-bbb2-304543b95e19 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:56.854566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d1918-1db4-41ff-9651-6fc373c8e2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:56.855183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d1918-1db4-41ff-9651-6fc373c8e2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:56.856198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d1918-1db4-41ff-9651-6fc373c8e2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:56.856198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d1918-1db4-41ff-9651-6fc373c8e2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:56.856461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d1918-1db4-41ff-9651-6fc373c8e2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:56.858157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d1918-1db4-41ff-9651-6fc373c8e2c4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:56.870175 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2132/4226] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:56.874121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e56a6c-06a2-45ee-94a9-57c4e6d87e89 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:56.887842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e56a6c-06a2-45ee-94a9-57c4e6d87e89 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:56.893242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2256e5-805e-41c0-9015-26c678227693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:56.894149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2256e5-805e-41c0-9015-26c678227693 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:56.894537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2256e5-805e-41c0-9015-26c678227693 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:56.894897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2256e5-805e-41c0-9015-26c678227693 admin admin] PATH_INFO: `/v3/projects/c130bd5aeff842a58574893dda25836e/users/4f6b45fe36bf4aa6bcb941a6c59eb69f/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:56.918946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2256e5-805e-41c0-9015-26c678227693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c130bd5aeff842a58574893dda25836e, user_id=4f6b45fe36bf4aa6bcb941a6c59eb69f, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:56.922845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2256e5-805e-41c0-9015-26c678227693 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:56.940996 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2095/4227] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:08:56 2022] PUT /identity/v3/projects/c130bd5aeff842a58574893dda25836e/users/4f6b45fe36bf4aa6bcb941a6c59eb69f/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:56.952571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f4024-a929-4455-92ef-9a44ccf3c88a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:56.952851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f4024-a929-4455-92ef-9a44ccf3c88a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:56.953113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f4024-a929-4455-92ef-9a44ccf3c88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:56.999320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3a3f4024-a929-4455-92ef-9a44ccf3c88a None None] MFA Rules not processed for user `4f6b45fe36bf4aa6bcb941a6c59eb69f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:57.001039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3a3f4024-a929-4455-92ef-9a44ccf3c88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:57.078403 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2133/4228] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:57.083062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2256e5-805e-41c0-9015-26c678227693 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:57.112823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:57.168123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2256e5-805e-41c0-9015-26c678227693 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:57.176432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6abb57-2827-4e67-a3c5-931c098640a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:57.177338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6abb57-2827-4e67-a3c5-931c098640a9 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:57.177849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6abb57-2827-4e67-a3c5-931c098640a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:57.178291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6abb57-2827-4e67-a3c5-931c098640a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:57.179153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6abb57-2827-4e67-a3c5-931c098640a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:57.181516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6abb57-2827-4e67-a3c5-931c098640a9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:57.218163 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2096/4229] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:08:57 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:08:57.222013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f4024-a929-4455-92ef-9a44ccf3c88a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:57.240941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f4024-a929-4455-92ef-9a44ccf3c88a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:57.355400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76939a66-3bfb-4183-bcdf-515fbd85a0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:57.356301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76939a66-3bfb-4183-bcdf-515fbd85a0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:57.356837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76939a66-3bfb-4183-bcdf-515fbd85a0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:57.357252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76939a66-3bfb-4183-bcdf-515fbd85a0e2 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:57.358872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76939a66-3bfb-4183-bcdf-515fbd85a0e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:57.361123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76939a66-3bfb-4183-bcdf-515fbd85a0e2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:57.413388 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2134/4230] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:08:57 2022] POST /identity/v3/users => generated 300 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:57.414757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6abb57-2827-4e67-a3c5-931c098640a9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:57.442440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6abb57-2827-4e67-a3c5-931c098640a9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:57.456940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbbc27-38e4-4d3c-b09b-fbb3a9da39e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:57.460922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbbc27-38e4-4d3c-b09b-fbb3a9da39e1 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:57.461205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbbc27-38e4-4d3c-b09b-fbb3a9da39e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:57.461530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbbc27-38e4-4d3c-b09b-fbb3a9da39e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:57.462211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbbc27-38e4-4d3c-b09b-fbb3a9da39e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:57.465305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbbc27-38e4-4d3c-b09b-fbb3a9da39e1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:57.499093 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2097/4231] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:57.503089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76939a66-3bfb-4183-bcdf-515fbd85a0e2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:57.528505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76939a66-3bfb-4183-bcdf-515fbd85a0e2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:57.542017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2840c47-429e-4011-9401-493044eb2070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:57.542756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2840c47-429e-4011-9401-493044eb2070 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:57.543032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2840c47-429e-4011-9401-493044eb2070 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:57.543303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2840c47-429e-4011-9401-493044eb2070 admin admin] PATH_INFO: `/v3/projects/f956ce50c15140d7a4e0df85840628ea/users/61761fc1918c4592a8fb0ad885fac8d4/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:57.586729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2840c47-429e-4011-9401-493044eb2070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f956ce50c15140d7a4e0df85840628ea, user_id=61761fc1918c4592a8fb0ad885fac8d4, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:57.589739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2840c47-429e-4011-9401-493044eb2070 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:57.620653 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2135/4232] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:08:57 2022] PUT /identity/v3/projects/f956ce50c15140d7a4e0df85840628ea/users/61761fc1918c4592a8fb0ad885fac8d4/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:57.625151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbbc27-38e4-4d3c-b09b-fbb3a9da39e1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:57.638834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbbc27-38e4-4d3c-b09b-fbb3a9da39e1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:57.759684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9f2b9-7a43-4a44-a098-e9cbbc49615d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:57.760642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9f2b9-7a43-4a44-a098-e9cbbc49615d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:57.761067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9f2b9-7a43-4a44-a098-e9cbbc49615d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:57.761480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9f2b9-7a43-4a44-a098-e9cbbc49615d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:57.762538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9f2b9-7a43-4a44-a098-e9cbbc49615d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:57.764534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9f2b9-7a43-4a44-a098-e9cbbc49615d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:57.778480 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2098/4233] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:08:57.783544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2840c47-429e-4011-9401-493044eb2070 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:57.798481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2840c47-429e-4011-9401-493044eb2070 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:57.811902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f35402-156b-4526-aa3c-24a5d64d2a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:57.812923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f35402-156b-4526-aa3c-24a5d64d2a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:57.813376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f35402-156b-4526-aa3c-24a5d64d2a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:57.813921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f35402-156b-4526-aa3c-24a5d64d2a06 admin admin] PATH_INFO: `/v3/projects/f956ce50c15140d7a4e0df85840628ea/users/61761fc1918c4592a8fb0ad885fac8d4/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:57.816260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f35402-156b-4526-aa3c-24a5d64d2a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f956ce50c15140d7a4e0df85840628ea, user_id=61761fc1918c4592a8fb0ad885fac8d4, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:57.819702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f35402-156b-4526-aa3c-24a5d64d2a06 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:57.839903 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2136/4234] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:08:57 2022] PUT /identity/v3/projects/f956ce50c15140d7a4e0df85840628ea/users/61761fc1918c4592a8fb0ad885fac8d4/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:57.853109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37e26-e798-4ddf-b188-64b9f187ab68 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:57.853868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37e26-e798-4ddf-b188-64b9f187ab68 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:57.854600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37e26-e798-4ddf-b188-64b9f187ab68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:57.893868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b0a37e26-e798-4ddf-b188-64b9f187ab68 None None] MFA Rules not processed for user `61761fc1918c4592a8fb0ad885fac8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:57.895315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b0a37e26-e798-4ddf-b188-64b9f187ab68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:58.011532 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2099/4235] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:58.016814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f35402-156b-4526-aa3c-24a5d64d2a06 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:58.037126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f35402-156b-4526-aa3c-24a5d64d2a06 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:58.044330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ed070-d00f-460d-81c1-686550d72c30 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61761fc1918c4592a8fb0ad885fac8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f956ce50c15140d7a4e0df85840628ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:58.045099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ed070-d00f-460d-81c1-686550d72c30 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:58.045266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ed070-d00f-460d-81c1-686550d72c30 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:58.045525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ed070-d00f-460d-81c1-686550d72c30 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:58.046334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ed070-d00f-460d-81c1-686550d72c30 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:58.048227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ed070-d00f-460d-81c1-686550d72c30 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:58.108142 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2137/4236] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:08:58 2022] POST /identity/v3/projects => generated 359 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:08:58.112697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a37e26-e798-4ddf-b188-64b9f187ab68 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:58.125839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a37e26-e798-4ddf-b188-64b9f187ab68 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:58.153023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:58.277737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4cdfd-8615-4d6c-89ba-861cd16e1bf8 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61761fc1918c4592a8fb0ad885fac8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f956ce50c15140d7a4e0df85840628ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:58.278682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4cdfd-8615-4d6c-89ba-861cd16e1bf8 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:58.278968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4cdfd-8615-4d6c-89ba-861cd16e1bf8 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:58.279285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4cdfd-8615-4d6c-89ba-861cd16e1bf8 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] PATH_INFO: `/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags/tempest-tag-1217320513` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:58.280705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4cdfd-8615-4d6c-89ba-861cd16e1bf8 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorizing `identity:create_project_tag(project_id=9d5b48ea96e94f088dfd8a77d62f4e20, value=tempest-tag-1217320513)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:58.283094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4cdfd-8615-4d6c-89ba-861cd16e1bf8 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:58.375595 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-b7b4cdfd-8615-4d6c-89ba-861cd16e1bf8 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-1217320513. To ensure interoperability, identifiers should be a valid uuid. Aug 20 21:08:58.375595 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Aug 20 21:08:58.375595 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:08:58.385932 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2100/4237] 10.222.0.23 () {60 vars in 1333 bytes} [Sat Aug 20 21:08:58 2022] PUT /identity/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags/tempest-tag-1217320513 => generated 0 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 291 bytes (2 switches on core 0) Aug 20 21:08:58.390219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ed070-d00f-460d-81c1-686550d72c30 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:58.404195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ed070-d00f-460d-81c1-686550d72c30 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:58.432468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b59e9-2db5-4845-9e1a-a64f4d654b60 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61761fc1918c4592a8fb0ad885fac8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f956ce50c15140d7a4e0df85840628ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:58.433423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b59e9-2db5-4845-9e1a-a64f4d654b60 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:58.437461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b59e9-2db5-4845-9e1a-a64f4d654b60 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:58.438224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b59e9-2db5-4845-9e1a-a64f4d654b60 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] PATH_INFO: `/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags/tempest-tag-1217320513` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:58.447071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:58.480749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b59e9-2db5-4845-9e1a-a64f4d654b60 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=9d5b48ea96e94f088dfd8a77d62f4e20, value=tempest-tag-1217320513)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:58.487627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b59e9-2db5-4845-9e1a-a64f4d654b60 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:58.522308 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2138/4238] 10.222.0.23 () {58 vars in 1314 bytes} [Sat Aug 20 21:08:58 2022] GET /identity/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags/tempest-tag-1217320513 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:58.532325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4cdfd-8615-4d6c-89ba-861cd16e1bf8 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:58.558167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4cdfd-8615-4d6c-89ba-861cd16e1bf8 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:58.588645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aa07a-3dc6-496e-877d-f386f5ea85bf tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61761fc1918c4592a8fb0ad885fac8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f956ce50c15140d7a4e0df85840628ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:58.589384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aa07a-3dc6-496e-877d-f386f5ea85bf tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:58.589748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aa07a-3dc6-496e-877d-f386f5ea85bf tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:58.590019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aa07a-3dc6-496e-877d-f386f5ea85bf tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] PATH_INFO: `/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:58.592988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aa07a-3dc6-496e-877d-f386f5ea85bf tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorizing `identity:update_project_tags(project_id=9d5b48ea96e94f088dfd8a77d62f4e20)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:58.604129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aa07a-3dc6-496e-877d-f386f5ea85bf tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:58.707089 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2101/4239] 10.222.0.23 () {60 vars in 1265 bytes} [Sat Aug 20 21:08:58 2022] PUT /identity/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags => generated 141 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:58.715222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b59e9-2db5-4845-9e1a-a64f4d654b60 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:58.732144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b59e9-2db5-4845-9e1a-a64f4d654b60 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:58.741667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7769c2c-e37f-4acc-9c95-76b8db48eba6 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61761fc1918c4592a8fb0ad885fac8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f956ce50c15140d7a4e0df85840628ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:58.741667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7769c2c-e37f-4acc-9c95-76b8db48eba6 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:58.742040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7769c2c-e37f-4acc-9c95-76b8db48eba6 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:58.742343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7769c2c-e37f-4acc-9c95-76b8db48eba6 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] PATH_INFO: `/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:58.768626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7769c2c-e37f-4acc-9c95-76b8db48eba6 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=9d5b48ea96e94f088dfd8a77d62f4e20)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:58.771568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7769c2c-e37f-4acc-9c95-76b8db48eba6 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:58.786610 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2139/4240] 10.222.0.23 () {58 vars in 1245 bytes} [Sat Aug 20 21:08:58 2022] GET /identity/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags => generated 141 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:08:58.791946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aa07a-3dc6-496e-877d-f386f5ea85bf tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:58.808212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aa07a-3dc6-496e-877d-f386f5ea85bf tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:58.814157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706caa5c-3de1-491b-9dbb-1cd4553e7cc5 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61761fc1918c4592a8fb0ad885fac8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f956ce50c15140d7a4e0df85840628ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:58.815266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706caa5c-3de1-491b-9dbb-1cd4553e7cc5 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:58.815784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706caa5c-3de1-491b-9dbb-1cd4553e7cc5 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:58.816265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706caa5c-3de1-491b-9dbb-1cd4553e7cc5 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] PATH_INFO: `/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags/tempest-tag-1714542732` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:58.817828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706caa5c-3de1-491b-9dbb-1cd4553e7cc5 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorizing `identity:delete_project_tag(project_id=9d5b48ea96e94f088dfd8a77d62f4e20, value=tempest-tag-1714542732)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:58.821238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706caa5c-3de1-491b-9dbb-1cd4553e7cc5 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:58.899580 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-706caa5c-3de1-491b-9dbb-1cd4553e7cc5 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-1714542732. To ensure interoperability, identifiers should be a valid uuid. Aug 20 21:08:58.899580 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Aug 20 21:08:58.899580 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:08:58.920527 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2102/4241] 10.222.0.23 () {58 vars in 1317 bytes} [Sat Aug 20 21:08:58 2022] DELETE /identity/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags/tempest-tag-1714542732 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:58.922282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7769c2c-e37f-4acc-9c95-76b8db48eba6 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:58.933139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7769c2c-e37f-4acc-9c95-76b8db48eba6 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:58.939864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1fe05-fc15-4038-a312-c06c9a6fe887 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61761fc1918c4592a8fb0ad885fac8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f956ce50c15140d7a4e0df85840628ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:58.940584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1fe05-fc15-4038-a312-c06c9a6fe887 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:58.940862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1fe05-fc15-4038-a312-c06c9a6fe887 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:58.941128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1fe05-fc15-4038-a312-c06c9a6fe887 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] PATH_INFO: `/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags/tempest-tag-1714542732` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:58.966004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1fe05-fc15-4038-a312-c06c9a6fe887 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=9d5b48ea96e94f088dfd8a77d62f4e20, value=tempest-tag-1714542732)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:58.968491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1fe05-fc15-4038-a312-c06c9a6fe887 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-91f1fe05-fc15-4038-a312-c06c9a6fe887 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Could not find project tag: tempest-tag-1714542732.: keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-1714542732. Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 284, in get Aug 20 21:08:58.996417 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_project_tag(project_id, value) Aug 20 21:08:58.997468 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:08:58.997468 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:08:58.997468 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 960, in get_project_tag Aug 20 21:08:58.997468 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectTagNotFound(project_tag=tag_name) Aug 20 21:08:58.997468 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-1714542732. Aug 20 21:08:58.997468 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:08:58.997846 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2140/4242] 10.222.0.23 () {58 vars in 1314 bytes} [Sat Aug 20 21:08:58 2022] GET /identity/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags/tempest-tag-1714542732 => generated 107 bytes in 76 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:08:59.003271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706caa5c-3de1-491b-9dbb-1cd4553e7cc5 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:59.014661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706caa5c-3de1-491b-9dbb-1cd4553e7cc5 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:59.033118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfa301-fd82-4d85-a680-bcae0f36c752 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61761fc1918c4592a8fb0ad885fac8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f956ce50c15140d7a4e0df85840628ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:59.033913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfa301-fd82-4d85-a680-bcae0f36c752 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:59.034265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfa301-fd82-4d85-a680-bcae0f36c752 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:59.034588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfa301-fd82-4d85-a680-bcae0f36c752 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] PATH_INFO: `/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:59.038263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfa301-fd82-4d85-a680-bcae0f36c752 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorizing `identity:delete_project_tags(project_id=9d5b48ea96e94f088dfd8a77d62f4e20)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:59.040564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfa301-fd82-4d85-a680-bcae0f36c752 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:59.112475 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2103/4243] 10.222.0.23 () {58 vars in 1248 bytes} [Sat Aug 20 21:08:59 2022] DELETE /identity/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:59.117004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1fe05-fc15-4038-a312-c06c9a6fe887 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:59.132248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1fe05-fc15-4038-a312-c06c9a6fe887 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:59.139960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567bdeb-f861-4dd6-be36-198dc289eba4 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61761fc1918c4592a8fb0ad885fac8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f956ce50c15140d7a4e0df85840628ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:59.142372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567bdeb-f861-4dd6-be36-198dc289eba4 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:59.142726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567bdeb-f861-4dd6-be36-198dc289eba4 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:59.143118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567bdeb-f861-4dd6-be36-198dc289eba4 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] PATH_INFO: `/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:59.158802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567bdeb-f861-4dd6-be36-198dc289eba4 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=9d5b48ea96e94f088dfd8a77d62f4e20)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:59.163081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567bdeb-f861-4dd6-be36-198dc289eba4 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:59.178661 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2141/4244] 10.222.0.23 () {58 vars in 1245 bytes} [Sat Aug 20 21:08:59 2022] GET /identity/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20/tags => generated 66 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:08:59.179968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfa301-fd82-4d85-a680-bcae0f36c752 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:59.191937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfa301-fd82-4d85-a680-bcae0f36c752 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:59.199407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61761fc1918c4592a8fb0ad885fac8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f956ce50c15140d7a4e0df85840628ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:59.200088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:59.200393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:59.200660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] PATH_INFO: `/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:59.202794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9d5b48ea96e94f088dfd8a77d62f4e20)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:59.207984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:59.239102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:59.285926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d5b48ea96e94f088dfd8a77d62f4e20', 'request_id': 'req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:59.312009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d5b48ea96e94f088dfd8a77d62f4e20', 'request_id': 'req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:59.325699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d5b48ea96e94f088dfd8a77d62f4e20', 'request_id': 'req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:59.343309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] The token cache is being invalidate because project 9d5b48ea96e94f088dfd8a77d62f4e20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:08:59.343676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:59.344348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d9f06eff-fbd7-47dc-a70d-1c94dda2279c tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:59.346415 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2104/4245] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:08:59 2022] DELETE /identity/v3/projects/9d5b48ea96e94f088dfd8a77d62f4e20 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:59.353685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567bdeb-f861-4dd6-be36-198dc289eba4 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:59.356221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4567bdeb-f861-4dd6-be36-198dc289eba4 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:59.462114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567bdeb-f861-4dd6-be36-198dc289eba4 tempest-IdentityV3ProjectTagsTest-1173723328 tempest-IdentityV3ProjectTagsTest-1173723328-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:08:59.540590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:08:59.541576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:59.541576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:59.541811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] PATH_INFO: `/v3/users/4f6b45fe36bf4aa6bcb941a6c59eb69f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:59.543127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f6b45fe36bf4aa6bcb941a6c59eb69f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:08:59.545304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:08:59.623501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:08:59.656236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f6b45fe36bf4aa6bcb941a6c59eb69f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:59.669328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f6b45fe36bf4aa6bcb941a6c59eb69f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:59.683928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f6b45fe36bf4aa6bcb941a6c59eb69f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:59.716188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f6b45fe36bf4aa6bcb941a6c59eb69f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:59.717067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f6b45fe36bf4aa6bcb941a6c59eb69f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:08:59.725096 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2142/4246] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:08:59 2022] DELETE /identity/v3/users/4f6b45fe36bf4aa6bcb941a6c59eb69f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:08:59.731164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731c5b1-2bf6-4927-aa27-f81ee17038fa None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:08:59.731442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731c5b1-2bf6-4927-aa27-f81ee17038fa None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:08:59.731885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731c5b1-2bf6-4927-aa27-f81ee17038fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:08:59.758071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c731c5b1-2bf6-4927-aa27-f81ee17038fa None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:08:59.759502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c731c5b1-2bf6-4927-aa27-f81ee17038fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:08:59.866527 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2105/4247] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:08:59.883886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:08:59.886001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:00.034919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f75f0d-108d-45bf-b5fc-2cbf64e8c406 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:00.043402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aa839-6d64-45f7-a21a-c04facf04e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:00.044128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aa839-6d64-45f7-a21a-c04facf04e3d service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:00.044618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aa839-6d64-45f7-a21a-c04facf04e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:00.045632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aa839-6d64-45f7-a21a-c04facf04e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:00.058551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795aa839-6d64-45f7-a21a-c04facf04e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:00.060685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795aa839-6d64-45f7-a21a-c04facf04e3d service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:00.075210 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2143/4248] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:00.433733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731c5b1-2bf6-4927-aa27-f81ee17038fa None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:00.435244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c731c5b1-2bf6-4927-aa27-f81ee17038fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:00.455014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731c5b1-2bf6-4927-aa27-f81ee17038fa None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:00.466193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:00.470393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:00.470840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:00.471142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] PATH_INFO: `/v3/users/dc3aa9044e0a4ea88c77c3143ea740cb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:00.472836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc3aa9044e0a4ea88c77c3143ea740cb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:00.474855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:00.595859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc3aa9044e0a4ea88c77c3143ea740cb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:00.620276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc3aa9044e0a4ea88c77c3143ea740cb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:00.653772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aa839-6d64-45f7-a21a-c04facf04e3d service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:00.654881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-795aa839-6d64-45f7-a21a-c04facf04e3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:00.688337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aa839-6d64-45f7-a21a-c04facf04e3d service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:00.694763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:00.696195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:00.696722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:00.697371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] PATH_INFO: `/v3/projects/c130bd5aeff842a58574893dda25836e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:00.698770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c130bd5aeff842a58574893dda25836e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:00.701371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:00.711986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc3aa9044e0a4ea88c77c3143ea740cb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:00.713470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc3aa9044e0a4ea88c77c3143ea740cb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:00.715135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc3aa9044e0a4ea88c77c3143ea740cb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:00.742187 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2106/4249] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:00 2022] DELETE /identity/v3/users/dc3aa9044e0a4ea88c77c3143ea740cb => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:00.820069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c130bd5aeff842a58574893dda25836e', 'request_id': 'req-86a4f943-7749-4693-9a9a-d367aa102da2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:00.841065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c130bd5aeff842a58574893dda25836e', 'request_id': 'req-86a4f943-7749-4693-9a9a-d367aa102da2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:00.848206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c130bd5aeff842a58574893dda25836e', 'request_id': 'req-86a4f943-7749-4693-9a9a-d367aa102da2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:00.870997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] The token cache is being invalidate because project c130bd5aeff842a58574893dda25836e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:00.871881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:00.876755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:00.879374 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2144/4250] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:00 2022] DELETE /identity/v3/projects/c130bd5aeff842a58574893dda25836e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:00.885420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:00.886991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:00.909025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:01.032295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92beee7-603e-4c9c-b318-cb4c61785122 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:01.071595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:01.073493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:01.152242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:01.154405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca344b-1a45-4667-8694-cc4da57bd522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:01.155158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca344b-1a45-4667-8694-cc4da57bd522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:01.155578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca344b-1a45-4667-8694-cc4da57bd522 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:01.155853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca344b-1a45-4667-8694-cc4da57bd522 admin admin] PATH_INFO: `/v3/users/61761fc1918c4592a8fb0ad885fac8d4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:01.157227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca344b-1a45-4667-8694-cc4da57bd522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61761fc1918c4592a8fb0ad885fac8d4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:01.159489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca344b-1a45-4667-8694-cc4da57bd522 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:01.200983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4f943-7749-4693-9a9a-d367aa102da2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:01.206780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be33a3-86b2-441b-92d9-06f6e686de29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:01.207775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be33a3-86b2-441b-92d9-06f6e686de29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:01.208150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be33a3-86b2-441b-92d9-06f6e686de29 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:01.208589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be33a3-86b2-441b-92d9-06f6e686de29 admin admin] PATH_INFO: `/v3/projects/83137ec5bbd04ea78c74a4ea576466ae` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:01.209904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be33a3-86b2-441b-92d9-06f6e686de29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83137ec5bbd04ea78c74a4ea576466ae)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:01.212826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be33a3-86b2-441b-92d9-06f6e686de29 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:01.268675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3aca344b-1a45-4667-8694-cc4da57bd522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61761fc1918c4592a8fb0ad885fac8d4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:01.291836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3aca344b-1a45-4667-8694-cc4da57bd522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61761fc1918c4592a8fb0ad885fac8d4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:01.307255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-95be33a3-86b2-441b-92d9-06f6e686de29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83137ec5bbd04ea78c74a4ea576466ae', 'request_id': 'req-95be33a3-86b2-441b-92d9-06f6e686de29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:01.312929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3aca344b-1a45-4667-8694-cc4da57bd522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61761fc1918c4592a8fb0ad885fac8d4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:01.313862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3aca344b-1a45-4667-8694-cc4da57bd522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61761fc1918c4592a8fb0ad885fac8d4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:01.314564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3aca344b-1a45-4667-8694-cc4da57bd522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61761fc1918c4592a8fb0ad885fac8d4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:01.320247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-95be33a3-86b2-441b-92d9-06f6e686de29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83137ec5bbd04ea78c74a4ea576466ae', 'request_id': 'req-95be33a3-86b2-441b-92d9-06f6e686de29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:01.331953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-95be33a3-86b2-441b-92d9-06f6e686de29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83137ec5bbd04ea78c74a4ea576466ae', 'request_id': 'req-95be33a3-86b2-441b-92d9-06f6e686de29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:01.333527 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2107/4251] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:00 2022] DELETE /identity/v3/users/61761fc1918c4592a8fb0ad885fac8d4 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:01.347126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-95be33a3-86b2-441b-92d9-06f6e686de29 admin admin] The token cache is being invalidate because project 83137ec5bbd04ea78c74a4ea576466ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:01.347802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-95be33a3-86b2-441b-92d9-06f6e686de29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:01.348535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-95be33a3-86b2-441b-92d9-06f6e686de29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:01.350581 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2145/4252] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:01 2022] DELETE /identity/v3/projects/83137ec5bbd04ea78c74a4ea576466ae => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:01.357188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d5a49-979a-4d53-88ce-4ffb8b9e91e5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:01.357563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d5a49-979a-4d53-88ce-4ffb8b9e91e5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:01.357874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d5a49-979a-4d53-88ce-4ffb8b9e91e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:01.379511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-391d5a49-979a-4d53-88ce-4ffb8b9e91e5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:01.380773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-391d5a49-979a-4d53-88ce-4ffb8b9e91e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:01.427971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:01.465484 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2108/4253] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:01.473339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def36455-31ee-46dc-b215-645ea895ace2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:01.473743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def36455-31ee-46dc-b215-645ea895ace2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:01.473870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def36455-31ee-46dc-b215-645ea895ace2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:01.498505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-def36455-31ee-46dc-b215-645ea895ace2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:01.499845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-def36455-31ee-46dc-b215-645ea895ace2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:01.515537 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2146/4254] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:01.517986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d5a49-979a-4d53-88ce-4ffb8b9e91e5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:01.538104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d5a49-979a-4d53-88ce-4ffb8b9e91e5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:01.544193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d23d79-2863-4ef8-acde-449dd72b1e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:01.545480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d23d79-2863-4ef8-acde-449dd72b1e15 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:01.546287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d23d79-2863-4ef8-acde-449dd72b1e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:01.546891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d23d79-2863-4ef8-acde-449dd72b1e15 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:01.548169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d23d79-2863-4ef8-acde-449dd72b1e15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:01.548504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d23d79-2863-4ef8-acde-449dd72b1e15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:01.550396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d23d79-2863-4ef8-acde-449dd72b1e15 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:01.569856 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2109/4255] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:01.571229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def36455-31ee-46dc-b215-645ea895ace2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:01.582864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def36455-31ee-46dc-b215-645ea895ace2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:01.589502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c900bc-6ee1-479a-8106-e25a9bddb70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:01.590123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c900bc-6ee1-479a-8106-e25a9bddb70a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:01.590388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c900bc-6ee1-479a-8106-e25a9bddb70a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:01.590646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c900bc-6ee1-479a-8106-e25a9bddb70a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:01.591264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c900bc-6ee1-479a-8106-e25a9bddb70a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:01.594041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c900bc-6ee1-479a-8106-e25a9bddb70a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:01.644537 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2147/4256] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:01 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:01.645683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d23d79-2863-4ef8-acde-449dd72b1e15 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:01.656839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d23d79-2863-4ef8-acde-449dd72b1e15 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:01.726617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c900bc-6ee1-479a-8106-e25a9bddb70a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:01.732158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b9c900bc-6ee1-479a-8106-e25a9bddb70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:01.773584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8a520-c671-4ca8-9b0b-4ce28c24d822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:01.774390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8a520-c671-4ca8-9b0b-4ce28c24d822 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:01.774680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8a520-c671-4ca8-9b0b-4ce28c24d822 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:01.774966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8a520-c671-4ca8-9b0b-4ce28c24d822 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:01.775710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8a520-c671-4ca8-9b0b-4ce28c24d822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:01.777674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8a520-c671-4ca8-9b0b-4ce28c24d822 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:01.810941 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2110/4257] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:01 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:01.814687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8a520-c671-4ca8-9b0b-4ce28c24d822 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:01.826952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8a520-c671-4ca8-9b0b-4ce28c24d822 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:01.849852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabf44c-5ecc-4309-86cf-66a2c744415f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:01.850641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabf44c-5ecc-4309-86cf-66a2c744415f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:01.850924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabf44c-5ecc-4309-86cf-66a2c744415f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:01.851193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabf44c-5ecc-4309-86cf-66a2c744415f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:01.851884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabf44c-5ecc-4309-86cf-66a2c744415f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:01.854568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabf44c-5ecc-4309-86cf-66a2c744415f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:01.872147 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2111/4258] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:01.872686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c900bc-6ee1-479a-8106-e25a9bddb70a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:01.877392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabf44c-5ecc-4309-86cf-66a2c744415f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:01.880741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7380916-c67d-432d-ac0d-ce5381a714aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:01.881633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7380916-c67d-432d-ac0d-ce5381a714aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:01.882129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7380916-c67d-432d-ac0d-ce5381a714aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:01.882519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7380916-c67d-432d-ac0d-ce5381a714aa admin admin] PATH_INFO: `/v3/projects/f956ce50c15140d7a4e0df85840628ea` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:01.884111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7380916-c67d-432d-ac0d-ce5381a714aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f956ce50c15140d7a4e0df85840628ea)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:01.886341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7380916-c67d-432d-ac0d-ce5381a714aa admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:01.901354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabf44c-5ecc-4309-86cf-66a2c744415f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:01.914129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8e9e5-9b88-4453-8ab6-ebc0356f5452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:01.914377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8e9e5-9b88-4453-8ab6-ebc0356f5452 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:01.916429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8e9e5-9b88-4453-8ab6-ebc0356f5452 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:01.916565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8e9e5-9b88-4453-8ab6-ebc0356f5452 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:01.917669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8e9e5-9b88-4453-8ab6-ebc0356f5452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:01.919283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8e9e5-9b88-4453-8ab6-ebc0356f5452 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:01.941035 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2112/4259] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:01.945934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8e9e5-9b88-4453-8ab6-ebc0356f5452 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:01.957798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8e9e5-9b88-4453-8ab6-ebc0356f5452 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:01.966378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:01.996826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d7380916-c67d-432d-ac0d-ce5381a714aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f956ce50c15140d7a4e0df85840628ea', 'request_id': 'req-d7380916-c67d-432d-ac0d-ce5381a714aa'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:02.010377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d7380916-c67d-432d-ac0d-ce5381a714aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f956ce50c15140d7a4e0df85840628ea', 'request_id': 'req-d7380916-c67d-432d-ac0d-ce5381a714aa'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:02.022857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d7380916-c67d-432d-ac0d-ce5381a714aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f956ce50c15140d7a4e0df85840628ea', 'request_id': 'req-d7380916-c67d-432d-ac0d-ce5381a714aa'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:02.037348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d7380916-c67d-432d-ac0d-ce5381a714aa admin admin] The token cache is being invalidate because project f956ce50c15140d7a4e0df85840628ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:02.037719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d7380916-c67d-432d-ac0d-ce5381a714aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:02.038504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d7380916-c67d-432d-ac0d-ce5381a714aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:02.040435 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2148/4260] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:01 2022] DELETE /identity/v3/projects/f956ce50c15140d7a4e0df85840628ea => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:02.045131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd0297-7f7e-42f7-85a9-0fce2179abb5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:02.045395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd0297-7f7e-42f7-85a9-0fce2179abb5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:02.045719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd0297-7f7e-42f7-85a9-0fce2179abb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:02.049109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6a2c3-c862-4bd4-902d-a01d002390bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:02.049944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6a2c3-c862-4bd4-902d-a01d002390bf admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:02.050354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6a2c3-c862-4bd4-902d-a01d002390bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:02.050632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6a2c3-c862-4bd4-902d-a01d002390bf admin admin] PATH_INFO: `/v3/projects/bd4e1f3de3af4dba8cc95846592e21cb/users/6c8268c69e134211b81ac3fd4c030dff/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:02.070955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6a2c3-c862-4bd4-902d-a01d002390bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd4e1f3de3af4dba8cc95846592e21cb, user_id=6c8268c69e134211b81ac3fd4c030dff, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:02.074268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6a2c3-c862-4bd4-902d-a01d002390bf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:02.075412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-eecd0297-7f7e-42f7-85a9-0fce2179abb5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:02.076804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-eecd0297-7f7e-42f7-85a9-0fce2179abb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:02.089320 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2149/4261] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:02.092424 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2113/4262] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:01 2022] PUT /identity/v3/projects/bd4e1f3de3af4dba8cc95846592e21cb/users/6c8268c69e134211b81ac3fd4c030dff/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:02.096674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808c946-707d-48cb-95fa-cafb201e332b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:02.096950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808c946-707d-48cb-95fa-cafb201e332b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:02.097220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808c946-707d-48cb-95fa-cafb201e332b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:02.098392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e0db-5615-41d1-8e2d-8e6519b10cb5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:02.098839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e0db-5615-41d1-8e2d-8e6519b10cb5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:02.099359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e0db-5615-41d1-8e2d-8e6519b10cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:02.123997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0b88e0db-5615-41d1-8e2d-8e6519b10cb5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:02.125949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0b88e0db-5615-41d1-8e2d-8e6519b10cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:02.131837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1808c946-707d-48cb-95fa-cafb201e332b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:02.133315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1808c946-707d-48cb-95fa-cafb201e332b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:02.139098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:02.211845 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2150/4263] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:02.214738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808c946-707d-48cb-95fa-cafb201e332b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:02.222655 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2114/4264] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:02.239724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808c946-707d-48cb-95fa-cafb201e332b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:02.245564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39225100-a000-4fb1-bfd0-221e3a9ef712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:02.245795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88e0db-5615-41d1-8e2d-8e6519b10cb5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:02.245914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39225100-a000-4fb1-bfd0-221e3a9ef712 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:02.246170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39225100-a000-4fb1-bfd0-221e3a9ef712 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:02.246474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39225100-a000-4fb1-bfd0-221e3a9ef712 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:02.247328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39225100-a000-4fb1-bfd0-221e3a9ef712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:02.247568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39225100-a000-4fb1-bfd0-221e3a9ef712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:02.248348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0b88e0db-5615-41d1-8e2d-8e6519b10cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:02.249419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39225100-a000-4fb1-bfd0-221e3a9ef712 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:02.267210 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2151/4265] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:02.268878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39225100-a000-4fb1-bfd0-221e3a9ef712 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:02.283057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39225100-a000-4fb1-bfd0-221e3a9ef712 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:02.289124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be41a4e-922f-4d04-a1a7-d677303c3122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:02.290227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be41a4e-922f-4d04-a1a7-d677303c3122 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:02.290514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be41a4e-922f-4d04-a1a7-d677303c3122 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:02.290776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be41a4e-922f-4d04-a1a7-d677303c3122 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:02.293097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be41a4e-922f-4d04-a1a7-d677303c3122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:02.297226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be41a4e-922f-4d04-a1a7-d677303c3122 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:02.338018 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2152/4266] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:02 2022] POST /identity/v3/projects => generated 395 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:02.343110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be41a4e-922f-4d04-a1a7-d677303c3122 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:02.393260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88e0db-5615-41d1-8e2d-8e6519b10cb5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:02.430727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be41a4e-922f-4d04-a1a7-d677303c3122 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:02.438028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab6a7b-7026-4aa5-bd73-84fb92628f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:02.438644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab6a7b-7026-4aa5-bd73-84fb92628f8e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:02.438973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab6a7b-7026-4aa5-bd73-84fb92628f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:02.439242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab6a7b-7026-4aa5-bd73-84fb92628f8e admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:02.439922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab6a7b-7026-4aa5-bd73-84fb92628f8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:02.444347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab6a7b-7026-4aa5-bd73-84fb92628f8e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:02.482700 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2153/4267] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:02 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:02.484442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab6a7b-7026-4aa5-bd73-84fb92628f8e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:02.500734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab6a7b-7026-4aa5-bd73-84fb92628f8e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:02.508477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81dbe82-b28a-4e07-8c32-5fe8942c032b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:02.509204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81dbe82-b28a-4e07-8c32-5fe8942c032b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:02.509579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81dbe82-b28a-4e07-8c32-5fe8942c032b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:02.509941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81dbe82-b28a-4e07-8c32-5fe8942c032b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:02.510690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81dbe82-b28a-4e07-8c32-5fe8942c032b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:02.512399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81dbe82-b28a-4e07-8c32-5fe8942c032b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:02.513081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f8f22-2d71-4317-901e-8d1de8920b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:02.513696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f8f22-2d71-4317-901e-8d1de8920b9d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:02.513973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f8f22-2d71-4317-901e-8d1de8920b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:02.514237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f8f22-2d71-4317-901e-8d1de8920b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:02.525084 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2154/4268] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:02.529129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f8f22-2d71-4317-901e-8d1de8920b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:02.529288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81dbe82-b28a-4e07-8c32-5fe8942c032b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:02.530884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f8f22-2d71-4317-901e-8d1de8920b9d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:02.534717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:02.540324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81dbe82-b28a-4e07-8c32-5fe8942c032b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:02.545726 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2115/4269] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:02.546740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3e01a-f450-420d-ba63-0de26f87f64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:02.547412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3e01a-f450-420d-ba63-0de26f87f64f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:02.547778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3e01a-f450-420d-ba63-0de26f87f64f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:02.548220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3e01a-f450-420d-ba63-0de26f87f64f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:02.548987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3e01a-f450-420d-ba63-0de26f87f64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:02.552484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3e01a-f450-420d-ba63-0de26f87f64f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:02.565142 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2155/4270] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:02.571838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f8f22-2d71-4317-901e-8d1de8920b9d service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:02.595815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f8f22-2d71-4317-901e-8d1de8920b9d service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:02.609016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711e322-fb90-4a62-9eb9-2af540f38062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:02.609796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711e322-fb90-4a62-9eb9-2af540f38062 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:02.610033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711e322-fb90-4a62-9eb9-2af540f38062 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:02.610555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711e322-fb90-4a62-9eb9-2af540f38062 admin admin] PATH_INFO: `/v3/projects/8c5d1d97223b418fbf7fbfcebc03ab65/users/122fda1eca5f4583b19b2dffb305b5b3/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:02.633826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711e322-fb90-4a62-9eb9-2af540f38062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c5d1d97223b418fbf7fbfcebc03ab65, user_id=122fda1eca5f4583b19b2dffb305b5b3, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:02.637622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711e322-fb90-4a62-9eb9-2af540f38062 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:02.659645 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2116/4271] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:02 2022] PUT /identity/v3/projects/8c5d1d97223b418fbf7fbfcebc03ab65/users/122fda1eca5f4583b19b2dffb305b5b3/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:02.671199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5aa46d-0bb2-45a0-ae15-7e83c1685988 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:02.671533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5aa46d-0bb2-45a0-ae15-7e83c1685988 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:02.671944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5aa46d-0bb2-45a0-ae15-7e83c1685988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:02.712295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ee5aa46d-0bb2-45a0-ae15-7e83c1685988 None None] MFA Rules not processed for user `122fda1eca5f4583b19b2dffb305b5b3`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:02.713563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ee5aa46d-0bb2-45a0-ae15-7e83c1685988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:02.816964 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2156/4272] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:02 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:02.821951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711e322-fb90-4a62-9eb9-2af540f38062 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:02.941859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:02.968964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711e322-fb90-4a62-9eb9-2af540f38062 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:02.974618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ac467-bf62-40ee-910d-b2103020feee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:02.975394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ac467-bf62-40ee-910d-b2103020feee admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:02.975670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ac467-bf62-40ee-910d-b2103020feee admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:02.975939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ac467-bf62-40ee-910d-b2103020feee admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:02.976690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ac467-bf62-40ee-910d-b2103020feee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:02.979022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ac467-bf62-40ee-910d-b2103020feee admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:03.041293 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2117/4273] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:02 2022] POST /identity/v3/projects => generated 395 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:03.042809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5aa46d-0bb2-45a0-ae15-7e83c1685988 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:03.058505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5aa46d-0bb2-45a0-ae15-7e83c1685988 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:03.152887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed619e-bcb0-4e3f-83b4-8c383093cec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:03.153757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed619e-bcb0-4e3f-83b4-8c383093cec5 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:03.154077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed619e-bcb0-4e3f-83b4-8c383093cec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:03.154349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed619e-bcb0-4e3f-83b4-8c383093cec5 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:03.155045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed619e-bcb0-4e3f-83b4-8c383093cec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:03.157106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed619e-bcb0-4e3f-83b4-8c383093cec5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:03.194946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ac467-bf62-40ee-910d-b2103020feee admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:03.202582 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2157/4274] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:03 2022] POST /identity/v3/users => generated 304 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:03.207157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ac467-bf62-40ee-910d-b2103020feee admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:03.212639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc4420-a20a-4956-9c94-219ad31183aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:03.213529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc4420-a20a-4956-9c94-219ad31183aa admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:03.213972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc4420-a20a-4956-9c94-219ad31183aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:03.214243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc4420-a20a-4956-9c94-219ad31183aa admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:03.214945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc4420-a20a-4956-9c94-219ad31183aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:03.216829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc4420-a20a-4956-9c94-219ad31183aa admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:03.242321 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2118/4275] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:03.246736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed619e-bcb0-4e3f-83b4-8c383093cec5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:03.258627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed619e-bcb0-4e3f-83b4-8c383093cec5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:03.263717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0584230-a0d6-4388-8242-2ac3a3940156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:03.264439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0584230-a0d6-4388-8242-2ac3a3940156 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:03.264783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0584230-a0d6-4388-8242-2ac3a3940156 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:03.265121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0584230-a0d6-4388-8242-2ac3a3940156 admin admin] PATH_INFO: `/v3/projects/7ec3450d8b8d4d9ba7a6a7ab2bbaedd4/users/801afad3240144f5bf9af26bd23cccff/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:03.284686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0584230-a0d6-4388-8242-2ac3a3940156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec3450d8b8d4d9ba7a6a7ab2bbaedd4, user_id=801afad3240144f5bf9af26bd23cccff, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:03.288490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0584230-a0d6-4388-8242-2ac3a3940156 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:03.309009 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2158/4276] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:03 2022] PUT /identity/v3/projects/7ec3450d8b8d4d9ba7a6a7ab2bbaedd4/users/801afad3240144f5bf9af26bd23cccff/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:03.312948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc4420-a20a-4956-9c94-219ad31183aa admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:03.326072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc4420-a20a-4956-9c94-219ad31183aa admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:03.426284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d9ffd-e19a-403e-8b82-b1dd37f925d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:03.427014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d9ffd-e19a-403e-8b82-b1dd37f925d3 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:03.427449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d9ffd-e19a-403e-8b82-b1dd37f925d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:03.427786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d9ffd-e19a-403e-8b82-b1dd37f925d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:03.428541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d9ffd-e19a-403e-8b82-b1dd37f925d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:03.430578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d9ffd-e19a-403e-8b82-b1dd37f925d3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:03.444266 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2119/4277] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:03.449090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0584230-a0d6-4388-8242-2ac3a3940156 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:03.470163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0584230-a0d6-4388-8242-2ac3a3940156 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:03.483665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabd694-7be0-4f07-807d-b245c38a7ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:03.484687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabd694-7be0-4f07-807d-b245c38a7ceb admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:03.485135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabd694-7be0-4f07-807d-b245c38a7ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:03.485559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabd694-7be0-4f07-807d-b245c38a7ceb admin admin] PATH_INFO: `/v3/projects/7ec3450d8b8d4d9ba7a6a7ab2bbaedd4/users/801afad3240144f5bf9af26bd23cccff/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:03.490320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabd694-7be0-4f07-807d-b245c38a7ceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec3450d8b8d4d9ba7a6a7ab2bbaedd4, user_id=801afad3240144f5bf9af26bd23cccff, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:03.501137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabd694-7be0-4f07-807d-b245c38a7ceb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:03.529093 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2159/4278] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:03 2022] PUT /identity/v3/projects/7ec3450d8b8d4d9ba7a6a7ab2bbaedd4/users/801afad3240144f5bf9af26bd23cccff/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:03.541945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fdfd2-0d4c-483a-add0-cccf4515054b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:03.542478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fdfd2-0d4c-483a-add0-cccf4515054b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:03.542693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fdfd2-0d4c-483a-add0-cccf4515054b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:03.617348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-818fdfd2-0d4c-483a-add0-cccf4515054b None None] MFA Rules not processed for user `801afad3240144f5bf9af26bd23cccff`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:03.620058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-818fdfd2-0d4c-483a-add0-cccf4515054b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:03.760439 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2120/4279] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:03.766735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabd694-7be0-4f07-807d-b245c38a7ceb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:03.853947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:03.914013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabd694-7be0-4f07-807d-b245c38a7ceb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:03.922216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6bdda-e3ec-4647-a6cb-d052306974f3 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '122fda1eca5f4583b19b2dffb305b5b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c5d1d97223b418fbf7fbfcebc03ab65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:03.922878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6bdda-e3ec-4647-a6cb-d052306974f3 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:03.923154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6bdda-e3ec-4647-a6cb-d052306974f3 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:03.923424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6bdda-e3ec-4647-a6cb-d052306974f3 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:03.924076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6bdda-e3ec-4647-a6cb-d052306974f3 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-a4b6bdda-e3ec-4647-a6cb-d052306974f3 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] You are not authorized to perform the requested action: identity:create_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 162, in post Aug 20 21:09:03.931398 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Aug 20 21:09:03.932572 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Aug 20 21:09:03.932572 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application enforcer_obj._enforce( Aug 20 21:09:03.932572 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Aug 20 21:09:03.932572 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Aug 20 21:09:03.932572 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Aug 20 21:09:03.932572 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Aug 20 21:09:03.932572 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Aug 20 21:09:03.932572 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:03.933117 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2160/4280] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:09:03 2022] POST /identity/v3/projects => generated 136 bytes in 168 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:09:03.938633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fdfd2-0d4c-483a-add0-cccf4515054b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:03.949586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fdfd2-0d4c-483a-add0-cccf4515054b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:03.963038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7e363-73b8-4d48-927b-28bb971557ea tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '801afad3240144f5bf9af26bd23cccff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec3450d8b8d4d9ba7a6a7ab2bbaedd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:03.963866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7e363-73b8-4d48-927b-28bb971557ea tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:03.964145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7e363-73b8-4d48-927b-28bb971557ea tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:03.964603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7e363-73b8-4d48-927b-28bb971557ea tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:03.966411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7e363-73b8-4d48-927b-28bb971557ea tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:03.967088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7e363-73b8-4d48-927b-28bb971557ea tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-b9c7e363-73b8-4d48-927b-28bb971557ea tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Invalid input for field 'name': '' is too short Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: Failed validating 'minLength' in schema['properties']['name']: Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: On instance['name']: Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: Failed validating 'minLength' in schema['properties']['name']: Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: On instance['name']: Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: '' Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise error Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application On instance['name']: Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application '' Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:03.990409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Aug 20 21:09:03.992300 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Aug 20 21:09:03.994187 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Aug 20 21:09:03.994187 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Aug 20 21:09:03.994187 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:03.994187 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Aug 20 21:09:03.994187 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:03.994187 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:03.994187 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application On instance['name']: Aug 20 21:09:03.994187 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application '' Aug 20 21:09:03.994187 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:03.997959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6bdda-e3ec-4647-a6cb-d052306974f3 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:03.998492 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2121/4281] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:09:03 2022] POST /identity/v3/projects => generated 283 bytes in 59 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 20 21:09:04.016007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6bdda-e3ec-4647-a6cb-d052306974f3 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:04.025874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d8eb0-f710-4a66-bede-f1a7abc1f55d tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '801afad3240144f5bf9af26bd23cccff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec3450d8b8d4d9ba7a6a7ab2bbaedd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:04.026508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d8eb0-f710-4a66-bede-f1a7abc1f55d tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:04.026940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d8eb0-f710-4a66-bede-f1a7abc1f55d tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:04.027231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d8eb0-f710-4a66-bede-f1a7abc1f55d tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:04.027837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d8eb0-f710-4a66-bede-f1a7abc1f55d tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:04.029722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d8eb0-f710-4a66-bede-f1a7abc1f55d tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-593d8eb0-f710-4a66-bede-f1a7abc1f55d tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: Failed validating 'maxLength' in schema['properties']['name']: Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: On instance['name']: Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: Failed validating 'maxLength' in schema['properties']['name']: Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: On instance['name']: Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise error Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Aug 20 21:09:04.077775 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application On instance['name']: Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:04.079772 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Aug 20 21:09:04.088848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Aug 20 21:09:04.088848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Aug 20 21:09:04.088848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Aug 20 21:09:04.088848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Aug 20 21:09:04.088848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Aug 20 21:09:04.088848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Aug 20 21:09:04.088848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.088848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Aug 20 21:09:04.088848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:04.088848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.088848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application On instance['name']: Aug 20 21:09:04.088848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Aug 20 21:09:04.088848 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.089789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7e363-73b8-4d48-927b-28bb971557ea tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:04.089887 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2161/4282] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:09:03 2022] POST /identity/v3/projects => generated 412 bytes in 47 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 20 21:09:04.108945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7e363-73b8-4d48-927b-28bb971557ea tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:04.123111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe02a7-3134-414d-b349-698eac45398c tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '801afad3240144f5bf9af26bd23cccff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec3450d8b8d4d9ba7a6a7ab2bbaedd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:04.124131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe02a7-3134-414d-b349-698eac45398c tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:04.124757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe02a7-3134-414d-b349-698eac45398c tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:04.124989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe02a7-3134-414d-b349-698eac45398c tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] PATH_INFO: `/v3/projects/04abaa8bfa8d4e3c816944c132ac4ed3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:04.156954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe02a7-3134-414d-b349-698eac45398c tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: Authorizing `identity:delete_project(project_id=04abaa8bfa8d4e3c816944c132ac4ed3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:04.163676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe02a7-3134-414d-b349-698eac45398c tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-5cbe02a7-3134-414d-b349-698eac45398c tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Could not find project: 04abaa8bfa8d4e3c816944c132ac4ed3.: keystone.exception.ProjectNotFound: Could not find project: 04abaa8bfa8d4e3c816944c132ac4ed3. Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Aug 20 21:09:04.188842 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Aug 20 21:09:04.190003 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:04.190003 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:04.190003 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Aug 20 21:09:04.190003 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Aug 20 21:09:04.190003 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:09:04.190003 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:09:04.190003 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:09:04.190003 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:09:04.190003 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 04abaa8bfa8d4e3c816944c132ac4ed3. Aug 20 21:09:04.190003 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:04.190836 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2122/4283] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:04 2022] DELETE /identity/v3/projects/04abaa8bfa8d4e3c816944c132ac4ed3 => generated 113 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:09:04.197229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d8eb0-f710-4a66-bede-f1a7abc1f55d tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:04.212133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d8eb0-f710-4a66-bede-f1a7abc1f55d tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:04.246140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35381c42-14f8-4d5d-888b-40730cd5b188 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '122fda1eca5f4583b19b2dffb305b5b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c5d1d97223b418fbf7fbfcebc03ab65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:04.246785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35381c42-14f8-4d5d-888b-40730cd5b188 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:04.247141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35381c42-14f8-4d5d-888b-40730cd5b188 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:04.247320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35381c42-14f8-4d5d-888b-40730cd5b188 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:04.247965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35381c42-14f8-4d5d-888b-40730cd5b188 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:04.249153 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-35381c42-14f8-4d5d-888b-40730cd5b188 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:04.249153 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:09:04.249153 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-35381c42-14f8-4d5d-888b-40730cd5b188 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] You are not authorized to perform the requested action: identity:list_projects.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 153, in get Aug 20 21:09:04.254986 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._list_projects() Aug 20 21:09:04.256387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 120, in _list_projects Aug 20 21:09:04.256387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:list_projects', Aug 20 21:09:04.256387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Aug 20 21:09:04.256387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application enforcer_obj._enforce( Aug 20 21:09:04.256387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Aug 20 21:09:04.256387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Aug 20 21:09:04.256387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Aug 20 21:09:04.256387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Aug 20 21:09:04.256387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Aug 20 21:09:04.256387 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.257296 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2162/4284] 10.222.0.23 () {58 vars in 1131 bytes} [Sat Aug 20 21:09:04 2022] GET /identity/v3/projects => generated 135 bytes in 60 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:09:04.264333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe02a7-3134-414d-b349-698eac45398c tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:04.283731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe02a7-3134-414d-b349-698eac45398c tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:04.289656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d6c2d-dc84-4701-aa63-0f7d08630681 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '801afad3240144f5bf9af26bd23cccff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec3450d8b8d4d9ba7a6a7ab2bbaedd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:04.290395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6c2d-dc84-4701-aa63-0f7d08630681 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:04.290770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6c2d-dc84-4701-aa63-0f7d08630681 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:04.291155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6c2d-dc84-4701-aa63-0f7d08630681 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:04.291878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d6c2d-dc84-4701-aa63-0f7d08630681 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:04.294021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d6c2d-dc84-4701-aa63-0f7d08630681 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:04.378486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35381c42-14f8-4d5d-888b-40730cd5b188 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:04.389297 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2123/4285] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:04 2022] POST /identity/v3/projects => generated 358 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:04.395635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35381c42-14f8-4d5d-888b-40730cd5b188 tempest-ProjectsNegativeStaticTestJSON-217799270 tempest-ProjectsNegativeStaticTestJSON-217799270-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:04.513794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc866a-8dbb-4354-ac0d-409ca3bd0a01 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '801afad3240144f5bf9af26bd23cccff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec3450d8b8d4d9ba7a6a7ab2bbaedd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:04.515102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc866a-8dbb-4354-ac0d-409ca3bd0a01 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:04.515542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc866a-8dbb-4354-ac0d-409ca3bd0a01 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:04.516776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc866a-8dbb-4354-ac0d-409ca3bd0a01 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:04.517582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc866a-8dbb-4354-ac0d-409ca3bd0a01 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:04.519702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc866a-8dbb-4354-ac0d-409ca3bd0a01 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:04.556298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.sql.core [None req-97fc866a-8dbb-4354-ac0d-409ca3bd0a01 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-1555545766' for key 'project.ixu_project_name_domain_id'") Aug 20 21:09:04.556298 ncdv-master-1 devstack@keystone.service[104118]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 20 21:09:04.556298 ncdv-master-1 devstack@keystone.service[104118]: [parameters: {'id': 'f57a17e0eb7f4eaf96a1d834c5a77625', 'name': 'tempest-project-dup-1555545766', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 20 21:09:04.556298 ncdv-master-1 devstack@keystone.service[104118]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=104118) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 20 21:09:04.571366 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-97fc866a-8dbb-4354-ac0d-409ca3bd0a01 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-1555545766, project id f57a17e0eb7f4eaf96a1d834c5a77625.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-1555545766, project id f57a17e0eb7f4eaf96a1d834c5a77625. Aug 20 21:09:04.571366 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:04.571366 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 20 21:09:04.571366 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 20 21:09:04.571366 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 20 21:09:04.571366 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 20 21:09:04.571366 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 20 21:09:04.571366 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result = self._query(query) Aug 20 21:09:04.571366 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 20 21:09:04.571366 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application conn.query(q) Aug 20 21:09:04.571366 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 20 21:09:04.571366 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result.read() Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-tempest-project-dup-1555545766' for key 'project.ixu_project_name_domain_id'") Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 20 21:09:04.572487 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application next(self.gen) Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application yield resource Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application next(self.gen) Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.session.rollback() Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.force_reraise() Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise self.value Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 20 21:09:04.573828 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application session.commit() Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._prepare_impl() Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.session.flush() Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._flush(objects) Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application compat.raise_( Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application flush_context.execute() Aug 20 21:09:04.575847 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rec.execute(self) Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 20 21:09:04.577313 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result = self._query(query) Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application conn.query(q) Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result.read() Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 20 21:09:04.578664 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-1555545766' for key 'project.ixu_project_name_domain_id'") Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [parameters: {'id': 'f57a17e0eb7f4eaf96a1d834c5a77625', 'name': 'tempest-project-dup-1555545766', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:04.580880 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-project-dup-1555545766 at domain ID default. Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:04.582014 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:04.586481 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:04.586481 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 20 21:09:04.586481 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 20 21:09:04.586481 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:04.586481 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:04.586481 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 20 21:09:04.586481 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 20 21:09:04.586481 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-1555545766, project id f57a17e0eb7f4eaf96a1d834c5a77625. Aug 20 21:09:04.586481 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:04.589596 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2163/4286] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:09:04 2022] POST /identity/v3/projects => generated 285 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 20 21:09:04.595212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d6c2d-dc84-4701-aa63-0f7d08630681 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:04.621814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d6c2d-dc84-4701-aa63-0f7d08630681 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:04.636901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '801afad3240144f5bf9af26bd23cccff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec3450d8b8d4d9ba7a6a7ab2bbaedd4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:04.638242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:04.638523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:04.638796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] PATH_INFO: `/v3/projects/6390579809204f56b9bb7f2661389973` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:04.640005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6390579809204f56b9bb7f2661389973)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:04.642689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:04.645645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:04.751743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6390579809204f56b9bb7f2661389973', 'request_id': 'req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:04.762022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6390579809204f56b9bb7f2661389973', 'request_id': 'req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:04.779214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6390579809204f56b9bb7f2661389973', 'request_id': 'req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:04.794344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] The token cache is being invalidate because project 6390579809204f56b9bb7f2661389973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:04.794705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:04.795636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b3072fca-bc9b-44c4-8d8b-a32bf273fdba tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:04.797497 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2124/4287] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:04 2022] DELETE /identity/v3/projects/6390579809204f56b9bb7f2661389973 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:04.803610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc866a-8dbb-4354-ac0d-409ca3bd0a01 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:04.805701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-97fc866a-8dbb-4354-ac0d-409ca3bd0a01 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:05.142694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc866a-8dbb-4354-ac0d-409ca3bd0a01 tempest-ProjectsNegativeStaticTestJSON-668660941 tempest-ProjectsNegativeStaticTestJSON-668660941-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:05.220877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:05.221685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:05.221968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:05.222234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] PATH_INFO: `/v3/users/122fda1eca5f4583b19b2dffb305b5b3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:05.223404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=122fda1eca5f4583b19b2dffb305b5b3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:05.242992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:05.258074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:05.338330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '122fda1eca5f4583b19b2dffb305b5b3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:05.353103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '122fda1eca5f4583b19b2dffb305b5b3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:05.362924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '122fda1eca5f4583b19b2dffb305b5b3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:05.376521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '122fda1eca5f4583b19b2dffb305b5b3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:05.377479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '122fda1eca5f4583b19b2dffb305b5b3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:05.389382 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2164/4288] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:04 2022] DELETE /identity/v3/users/122fda1eca5f4583b19b2dffb305b5b3 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:05.394364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa94e98-8098-41c4-8f20-2362815cd72e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:05.394713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa94e98-8098-41c4-8f20-2362815cd72e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:05.395049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa94e98-8098-41c4-8f20-2362815cd72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:05.419379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5fa94e98-8098-41c4-8f20-2362815cd72e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:05.421544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5fa94e98-8098-41c4-8f20-2362815cd72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:05.500599 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2125/4289] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:05.532320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:05.533931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:05.607391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c024f4f-736b-46f1-9c4e-2901d764a7ae admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:05.611358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b510446-bc97-486d-85ef-3de054ae0621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:05.611962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b510446-bc97-486d-85ef-3de054ae0621 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:05.612285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b510446-bc97-486d-85ef-3de054ae0621 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:05.612575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b510446-bc97-486d-85ef-3de054ae0621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:05.623229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b510446-bc97-486d-85ef-3de054ae0621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:05.624710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b510446-bc97-486d-85ef-3de054ae0621 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:05.635654 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2165/4290] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:09:06.084990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa94e98-8098-41c4-8f20-2362815cd72e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:06.086333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5fa94e98-8098-41c4-8f20-2362815cd72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:06.103123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa94e98-8098-41c4-8f20-2362815cd72e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:06.113311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03828041-8674-453a-8049-24565e27160f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:06.114688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03828041-8674-453a-8049-24565e27160f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:06.115087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03828041-8674-453a-8049-24565e27160f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:06.116866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03828041-8674-453a-8049-24565e27160f admin admin] PATH_INFO: `/v3/projects/8c5d1d97223b418fbf7fbfcebc03ab65` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:06.118759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03828041-8674-453a-8049-24565e27160f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c5d1d97223b418fbf7fbfcebc03ab65)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:06.121188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03828041-8674-453a-8049-24565e27160f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:06.184111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03828041-8674-453a-8049-24565e27160f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c5d1d97223b418fbf7fbfcebc03ab65', 'request_id': 'req-03828041-8674-453a-8049-24565e27160f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:06.193244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03828041-8674-453a-8049-24565e27160f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c5d1d97223b418fbf7fbfcebc03ab65', 'request_id': 'req-03828041-8674-453a-8049-24565e27160f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:06.206737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03828041-8674-453a-8049-24565e27160f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c5d1d97223b418fbf7fbfcebc03ab65', 'request_id': 'req-03828041-8674-453a-8049-24565e27160f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:06.251081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03828041-8674-453a-8049-24565e27160f admin admin] The token cache is being invalidate because project 8c5d1d97223b418fbf7fbfcebc03ab65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:06.252146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03828041-8674-453a-8049-24565e27160f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:06.252950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03828041-8674-453a-8049-24565e27160f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:06.255447 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2126/4291] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:06 2022] DELETE /identity/v3/projects/8c5d1d97223b418fbf7fbfcebc03ab65 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:06.260016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b510446-bc97-486d-85ef-3de054ae0621 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:06.273230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b510446-bc97-486d-85ef-3de054ae0621 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:06.371876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e84ab-a93f-4444-a550-df2ebe620327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:06.372566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e84ab-a93f-4444-a550-df2ebe620327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:06.372841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e84ab-a93f-4444-a550-df2ebe620327 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:06.373111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e84ab-a93f-4444-a550-df2ebe620327 admin admin] PATH_INFO: `/v3/users/801afad3240144f5bf9af26bd23cccff` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:06.375624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e84ab-a93f-4444-a550-df2ebe620327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=801afad3240144f5bf9af26bd23cccff)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:06.377609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e84ab-a93f-4444-a550-df2ebe620327 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:06.425103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:06.448823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d49e84ab-a93f-4444-a550-df2ebe620327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '801afad3240144f5bf9af26bd23cccff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:06.462066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d49e84ab-a93f-4444-a550-df2ebe620327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '801afad3240144f5bf9af26bd23cccff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:06.473795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d49e84ab-a93f-4444-a550-df2ebe620327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '801afad3240144f5bf9af26bd23cccff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:06.492412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d49e84ab-a93f-4444-a550-df2ebe620327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '801afad3240144f5bf9af26bd23cccff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:06.493255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d49e84ab-a93f-4444-a550-df2ebe620327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '801afad3240144f5bf9af26bd23cccff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:06.503790 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2166/4292] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:06 2022] DELETE /identity/v3/users/801afad3240144f5bf9af26bd23cccff => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:06.869366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03828041-8674-453a-8049-24565e27160f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:06.871611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-03828041-8674-453a-8049-24565e27160f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:06.897081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03828041-8674-453a-8049-24565e27160f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:06.912118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:06.990460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6e35c-ece1-4227-af13-54b2a16952a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:06.991133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6e35c-ece1-4227-af13-54b2a16952a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:06.991392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6e35c-ece1-4227-af13-54b2a16952a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:06.991656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6e35c-ece1-4227-af13-54b2a16952a0 admin admin] PATH_INFO: `/v3/projects/7ec3450d8b8d4d9ba7a6a7ab2bbaedd4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:06.992962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6e35c-ece1-4227-af13-54b2a16952a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec3450d8b8d4d9ba7a6a7ab2bbaedd4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:06.995016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6e35c-ece1-4227-af13-54b2a16952a0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:07.056770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5e6e35c-ece1-4227-af13-54b2a16952a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec3450d8b8d4d9ba7a6a7ab2bbaedd4', 'request_id': 'req-c5e6e35c-ece1-4227-af13-54b2a16952a0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:07.064231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5e6e35c-ece1-4227-af13-54b2a16952a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec3450d8b8d4d9ba7a6a7ab2bbaedd4', 'request_id': 'req-c5e6e35c-ece1-4227-af13-54b2a16952a0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:07.078297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5e6e35c-ece1-4227-af13-54b2a16952a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec3450d8b8d4d9ba7a6a7ab2bbaedd4', 'request_id': 'req-c5e6e35c-ece1-4227-af13-54b2a16952a0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:07.094763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5e6e35c-ece1-4227-af13-54b2a16952a0 admin admin] The token cache is being invalidate because project 7ec3450d8b8d4d9ba7a6a7ab2bbaedd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:07.096198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5e6e35c-ece1-4227-af13-54b2a16952a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:07.097100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5e6e35c-ece1-4227-af13-54b2a16952a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:07.099479 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2127/4293] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:06 2022] DELETE /identity/v3/projects/7ec3450d8b8d4d9ba7a6a7ab2bbaedd4 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:07.104663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ce58a-d882-42a0-87b9-ae55a4ba956f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:07.104943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ce58a-d882-42a0-87b9-ae55a4ba956f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:07.106736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ce58a-d882-42a0-87b9-ae55a4ba956f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:07.169181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c38ce58a-d882-42a0-87b9-ae55a4ba956f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:07.170423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c38ce58a-d882-42a0-87b9-ae55a4ba956f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:07.315785 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2167/4294] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:07.322909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fb9bf-4fd2-47f2-8e4c-0e6b51594e96 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:07.323374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fb9bf-4fd2-47f2-8e4c-0e6b51594e96 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:07.323651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fb9bf-4fd2-47f2-8e4c-0e6b51594e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:07.370364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-655fb9bf-4fd2-47f2-8e4c-0e6b51594e96 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:07.371791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-655fb9bf-4fd2-47f2-8e4c-0e6b51594e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:07.394930 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2128/4295] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:07.399301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ce58a-d882-42a0-87b9-ae55a4ba956f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:07.416643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ce58a-d882-42a0-87b9-ae55a4ba956f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:07.446634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67d50b-8fab-4947-9d1c-0b257e2cd882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:07.446976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67d50b-8fab-4947-9d1c-0b257e2cd882 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:07.447449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67d50b-8fab-4947-9d1c-0b257e2cd882 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:07.447746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67d50b-8fab-4947-9d1c-0b257e2cd882 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:07.448362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67d50b-8fab-4947-9d1c-0b257e2cd882 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:07.453746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67d50b-8fab-4947-9d1c-0b257e2cd882 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:07.456067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67d50b-8fab-4947-9d1c-0b257e2cd882 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:07.489131 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2168/4296] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:07.490433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fb9bf-4fd2-47f2-8e4c-0e6b51594e96 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:07.500786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fb9bf-4fd2-47f2-8e4c-0e6b51594e96 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:07.513868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0586886-bcb4-4b91-b5e1-8957a7f417a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:07.514578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0586886-bcb4-4b91-b5e1-8957a7f417a0 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:07.514914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0586886-bcb4-4b91-b5e1-8957a7f417a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:07.515427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0586886-bcb4-4b91-b5e1-8957a7f417a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:07.516303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0586886-bcb4-4b91-b5e1-8957a7f417a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:07.518357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0586886-bcb4-4b91-b5e1-8957a7f417a0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:07.558091 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2129/4297] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:07 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:07.562273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67d50b-8fab-4947-9d1c-0b257e2cd882 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:07.575146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67d50b-8fab-4947-9d1c-0b257e2cd882 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:07.651872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f18b8-0b87-4f4d-80a1-e061086c8faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:07.652529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f18b8-0b87-4f4d-80a1-e061086c8faa admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:07.652802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f18b8-0b87-4f4d-80a1-e061086c8faa admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:07.653064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f18b8-0b87-4f4d-80a1-e061086c8faa admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:07.653678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f18b8-0b87-4f4d-80a1-e061086c8faa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:07.655597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f18b8-0b87-4f4d-80a1-e061086c8faa admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:07.684421 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2169/4298] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:07 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:07.687242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0586886-bcb4-4b91-b5e1-8957a7f417a0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:07.700483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0586886-bcb4-4b91-b5e1-8957a7f417a0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:07.705415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4ab74-3d31-4621-acce-8ec7d8ccd241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:07.706064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4ab74-3d31-4621-acce-8ec7d8ccd241 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:07.706336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4ab74-3d31-4621-acce-8ec7d8ccd241 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:07.706631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4ab74-3d31-4621-acce-8ec7d8ccd241 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:07.707320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4ab74-3d31-4621-acce-8ec7d8ccd241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:07.709098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4ab74-3d31-4621-acce-8ec7d8ccd241 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:07.721203 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2130/4299] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:07.725235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f18b8-0b87-4f4d-80a1-e061086c8faa admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:07.736163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f18b8-0b87-4f4d-80a1-e061086c8faa admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:07.741526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714a287-5249-4b45-aa46-d3957f614a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:07.742682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714a287-5249-4b45-aa46-d3957f614a2f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:07.742682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714a287-5249-4b45-aa46-d3957f614a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:07.742682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714a287-5249-4b45-aa46-d3957f614a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:07.743305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714a287-5249-4b45-aa46-d3957f614a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:07.744958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714a287-5249-4b45-aa46-d3957f614a2f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:07.756745 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2170/4300] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:07.761298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4ab74-3d31-4621-acce-8ec7d8ccd241 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:07.774015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4ab74-3d31-4621-acce-8ec7d8ccd241 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:07.779073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a6935-9619-44cf-a909-dc7d39078b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:07.779781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6935-9619-44cf-a909-dc7d39078b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:07.780051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6935-9619-44cf-a909-dc7d39078b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:07.780468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6935-9619-44cf-a909-dc7d39078b7d admin admin] PATH_INFO: `/v3/projects/e108f56c7b2d409591d2c7260ab07446/users/0c5b969e7405485096edfea59c7983ed/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:07.801657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a6935-9619-44cf-a909-dc7d39078b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e108f56c7b2d409591d2c7260ab07446, user_id=0c5b969e7405485096edfea59c7983ed, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:07.804527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a6935-9619-44cf-a909-dc7d39078b7d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:07.824755 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2131/4301] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:07 2022] PUT /identity/v3/projects/e108f56c7b2d409591d2c7260ab07446/users/0c5b969e7405485096edfea59c7983ed/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:07.836557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fafb1a-7580-4017-9a8c-2d239147094c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:07.836811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fafb1a-7580-4017-9a8c-2d239147094c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:07.837103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fafb1a-7580-4017-9a8c-2d239147094c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:07.868018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d0fafb1a-7580-4017-9a8c-2d239147094c None None] MFA Rules not processed for user `0c5b969e7405485096edfea59c7983ed`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:07.869271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d0fafb1a-7580-4017-9a8c-2d239147094c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:07.874231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:07.956912 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2171/4302] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:07.961708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a6935-9619-44cf-a909-dc7d39078b7d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:08.084867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:08.111209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a6935-9619-44cf-a909-dc7d39078b7d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:08.135773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3558c3-b582-4863-a2d8-a4eed741287a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:08.136587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3558c3-b582-4863-a2d8-a4eed741287a admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:08.137082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3558c3-b582-4863-a2d8-a4eed741287a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:08.137596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3558c3-b582-4863-a2d8-a4eed741287a admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:08.139048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3558c3-b582-4863-a2d8-a4eed741287a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:08.141294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3558c3-b582-4863-a2d8-a4eed741287a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:08.178422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974eb5c-8a74-41a5-a26c-ac81798eac14 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:08.178619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974eb5c-8a74-41a5-a26c-ac81798eac14 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:08.178868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974eb5c-8a74-41a5-a26c-ac81798eac14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:08.181046 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2132/4303] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:07 2022] POST /identity/v3/projects => generated 385 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:08.185350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3558c3-b582-4863-a2d8-a4eed741287a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:08.259390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a974eb5c-8a74-41a5-a26c-ac81798eac14 None None] MFA Rules not processed for user `6c8268c69e134211b81ac3fd4c030dff`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:08.260724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a974eb5c-8a74-41a5-a26c-ac81798eac14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:08.296478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3558c3-b582-4863-a2d8-a4eed741287a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:08.301650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b80a72-715f-45ad-91dc-1e5c1905eceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:08.302323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b80a72-715f-45ad-91dc-1e5c1905eceb admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:08.302617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b80a72-715f-45ad-91dc-1e5c1905eceb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:08.302860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b80a72-715f-45ad-91dc-1e5c1905eceb admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:08.303463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b80a72-715f-45ad-91dc-1e5c1905eceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:08.305465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b80a72-715f-45ad-91dc-1e5c1905eceb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:08.341836 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2133/4304] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:08 2022] POST /identity/v3/users => generated 299 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:08.345534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b80a72-715f-45ad-91dc-1e5c1905eceb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:08.370686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b80a72-715f-45ad-91dc-1e5c1905eceb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:08.379010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22517025-c591-4297-ac33-b21a58345d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:08.379389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22517025-c591-4297-ac33-b21a58345d7e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:08.379763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22517025-c591-4297-ac33-b21a58345d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:08.380118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22517025-c591-4297-ac33-b21a58345d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:08.382813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22517025-c591-4297-ac33-b21a58345d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:08.385813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22517025-c591-4297-ac33-b21a58345d7e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:08.399432 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2134/4305] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:08.404157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22517025-c591-4297-ac33-b21a58345d7e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:08.424942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22517025-c591-4297-ac33-b21a58345d7e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:08.429261 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2172/4306] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:08.437441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603afb45-b5db-4be5-aed3-62e03436a594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:08.438258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603afb45-b5db-4be5-aed3-62e03436a594 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:08.438639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603afb45-b5db-4be5-aed3-62e03436a594 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:08.439028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603afb45-b5db-4be5-aed3-62e03436a594 admin admin] PATH_INFO: `/v3/projects/b5160740bbeb4b7bba933ceafed38239/users/58caab0aa986444eb3708d027251b06a/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:08.449132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626c879-0d87-4cb7-82fe-4409faf32149 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:08.449408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626c879-0d87-4cb7-82fe-4409faf32149 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:08.449681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626c879-0d87-4cb7-82fe-4409faf32149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:08.473610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1626c879-0d87-4cb7-82fe-4409faf32149 None None] MFA Rules not processed for user `6c8268c69e134211b81ac3fd4c030dff`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:08.475081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1626c879-0d87-4cb7-82fe-4409faf32149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:08.476418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603afb45-b5db-4be5-aed3-62e03436a594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5160740bbeb4b7bba933ceafed38239, user_id=58caab0aa986444eb3708d027251b06a, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:08.479350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603afb45-b5db-4be5-aed3-62e03436a594 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:08.484634 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2173/4307] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:08.499407 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2135/4308] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:08 2022] PUT /identity/v3/projects/b5160740bbeb4b7bba933ceafed38239/users/58caab0aa986444eb3708d027251b06a/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:08.503351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626c879-0d87-4cb7-82fe-4409faf32149 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:08.510661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603afb45-b5db-4be5-aed3-62e03436a594 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:08.512431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-603afb45-b5db-4be5-aed3-62e03436a594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:08.520515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626c879-0d87-4cb7-82fe-4409faf32149 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:08.607456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23987e23-7df4-46b7-9244-e0a91e75c83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:08.608096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23987e23-7df4-46b7-9244-e0a91e75c83b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:08.608476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23987e23-7df4-46b7-9244-e0a91e75c83b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:08.608887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23987e23-7df4-46b7-9244-e0a91e75c83b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:08.609618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23987e23-7df4-46b7-9244-e0a91e75c83b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:08.611244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23987e23-7df4-46b7-9244-e0a91e75c83b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:08.622577 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2174/4309] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:08.626914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23987e23-7df4-46b7-9244-e0a91e75c83b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:08.632134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603afb45-b5db-4be5-aed3-62e03436a594 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:08.638117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f322b-74dc-41df-b737-78599513985f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:08.638346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23987e23-7df4-46b7-9244-e0a91e75c83b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:08.638795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f322b-74dc-41df-b737-78599513985f service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:08.639137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f322b-74dc-41df-b737-78599513985f service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:08.639351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f322b-74dc-41df-b737-78599513985f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:08.643370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b6402-8376-4c38-8666-435424c08345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:08.644122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b6402-8376-4c38-8666-435424c08345 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:08.644503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b6402-8376-4c38-8666-435424c08345 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:08.644821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b6402-8376-4c38-8666-435424c08345 admin admin] PATH_INFO: `/v3/projects/b5160740bbeb4b7bba933ceafed38239/users/58caab0aa986444eb3708d027251b06a/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:08.647523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b6402-8376-4c38-8666-435424c08345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5160740bbeb4b7bba933ceafed38239, user_id=58caab0aa986444eb3708d027251b06a, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:08.650428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b6402-8376-4c38-8666-435424c08345 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:08.654699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:08.664294 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2175/4310] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:08 2022] PUT /identity/v3/projects/b5160740bbeb4b7bba933ceafed38239/users/58caab0aa986444eb3708d027251b06a/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:08.674985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451de060-76b2-4a8b-bbb8-ed0dd9fedb0f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:08.675387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451de060-76b2-4a8b-bbb8-ed0dd9fedb0f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:08.675658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451de060-76b2-4a8b-bbb8-ed0dd9fedb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:08.699395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:08.708304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-451de060-76b2-4a8b-bbb8-ed0dd9fedb0f None None] MFA Rules not processed for user `58caab0aa986444eb3708d027251b06a`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:08.708959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f322b-74dc-41df-b737-78599513985f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:08.710566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-451de060-76b2-4a8b-bbb8-ed0dd9fedb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:08.710810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f322b-74dc-41df-b737-78599513985f service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:08.721692 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2136/4311] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:08.800568 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2176/4312] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:08.805145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f322b-74dc-41df-b737-78599513985f service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:08.911726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f322b-74dc-41df-b737-78599513985f service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:08.920193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d3a5f-eced-457f-84fa-e2fdb7c006c5 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58caab0aa986444eb3708d027251b06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5160740bbeb4b7bba933ceafed38239', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:08.920880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d3a5f-eced-457f-84fa-e2fdb7c006c5 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:08.921296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d3a5f-eced-457f-84fa-e2fdb7c006c5 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:08.922084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d3a5f-eced-457f-84fa-e2fdb7c006c5 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:08.922764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d3a5f-eced-457f-84fa-e2fdb7c006c5 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:08.925283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d3a5f-eced-457f-84fa-e2fdb7c006c5 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:08.997266 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2137/4313] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:08 2022] POST /identity/v3/projects => generated 360 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:09.002746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451de060-76b2-4a8b-bbb8-ed0dd9fedb0f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:09.087731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451de060-76b2-4a8b-bbb8-ed0dd9fedb0f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:09.161056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b793c6f-3844-44d5-9b37-18e2600f339b tempest-ProjectsNegativeTestJSON-1065984173 tempest-ProjectsNegativeTestJSON-1065984173-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c5b969e7405485096edfea59c7983ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e108f56c7b2d409591d2c7260ab07446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:09.162311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b793c6f-3844-44d5-9b37-18e2600f339b tempest-ProjectsNegativeTestJSON-1065984173 tempest-ProjectsNegativeTestJSON-1065984173-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:09.162802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b793c6f-3844-44d5-9b37-18e2600f339b tempest-ProjectsNegativeTestJSON-1065984173 tempest-ProjectsNegativeTestJSON-1065984173-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:09.163355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b793c6f-3844-44d5-9b37-18e2600f339b tempest-ProjectsNegativeTestJSON-1065984173 tempest-ProjectsNegativeTestJSON-1065984173-project] PATH_INFO: `/v3/projects/256e9d5d1da04ed196f8764d1fc33b30` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:09.165239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b793c6f-3844-44d5-9b37-18e2600f339b tempest-ProjectsNegativeTestJSON-1065984173 tempest-ProjectsNegativeTestJSON-1065984173-project] RBAC: Authorizing `identity:delete_project(project_id=256e9d5d1da04ed196f8764d1fc33b30)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-9b793c6f-3844-44d5-9b37-18e2600f339b tempest-ProjectsNegativeTestJSON-1065984173 tempest-ProjectsNegativeTestJSON-1065984173-project] You are not authorized to perform the requested action: identity:delete_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 207, in delete Aug 20 21:09:09.176246 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Aug 20 21:09:09.177537 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Aug 20 21:09:09.177537 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application enforcer_obj._enforce( Aug 20 21:09:09.177537 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Aug 20 21:09:09.177537 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Aug 20 21:09:09.177537 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Aug 20 21:09:09.177537 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Aug 20 21:09:09.177537 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Aug 20 21:09:09.177537 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:09.179144 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2177/4314] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:09 2022] DELETE /identity/v3/projects/256e9d5d1da04ed196f8764d1fc33b30 => generated 136 bytes in 178 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:09:09.183969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d3a5f-eced-457f-84fa-e2fdb7c006c5 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:09.266641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d3a5f-eced-457f-84fa-e2fdb7c006c5 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:09.272258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0350b9e-d201-444e-92ed-c43e51c57c64 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58caab0aa986444eb3708d027251b06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5160740bbeb4b7bba933ceafed38239', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:09.273262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0350b9e-d201-444e-92ed-c43e51c57c64 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:09.273597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0350b9e-d201-444e-92ed-c43e51c57c64 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:09.273970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0350b9e-d201-444e-92ed-c43e51c57c64 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] PATH_INFO: `/v3/projects/256e9d5d1da04ed196f8764d1fc33b30` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:09.275389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0350b9e-d201-444e-92ed-c43e51c57c64 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] RBAC: Authorizing `identity:delete_project(project_id=256e9d5d1da04ed196f8764d1fc33b30)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:09.277637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0350b9e-d201-444e-92ed-c43e51c57c64 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:09.330820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:09.335464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e0350b9e-d201-444e-92ed-c43e51c57c64 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '256e9d5d1da04ed196f8764d1fc33b30', 'request_id': 'req-e0350b9e-d201-444e-92ed-c43e51c57c64'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:09.343820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e0350b9e-d201-444e-92ed-c43e51c57c64 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '256e9d5d1da04ed196f8764d1fc33b30', 'request_id': 'req-e0350b9e-d201-444e-92ed-c43e51c57c64'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:09.355621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e0350b9e-d201-444e-92ed-c43e51c57c64 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '256e9d5d1da04ed196f8764d1fc33b30', 'request_id': 'req-e0350b9e-d201-444e-92ed-c43e51c57c64'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:09.368366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e0350b9e-d201-444e-92ed-c43e51c57c64 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] The token cache is being invalidate because project 256e9d5d1da04ed196f8764d1fc33b30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:09.368814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e0350b9e-d201-444e-92ed-c43e51c57c64 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:09.369656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e0350b9e-d201-444e-92ed-c43e51c57c64 tempest-ProjectsNegativeTestJSON-1650365999 tempest-ProjectsNegativeTestJSON-1650365999-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:09.371557 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2138/4315] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:09 2022] DELETE /identity/v3/projects/256e9d5d1da04ed196f8764d1fc33b30 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:09.376208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b793c6f-3844-44d5-9b37-18e2600f339b tempest-ProjectsNegativeTestJSON-1065984173 tempest-ProjectsNegativeTestJSON-1065984173-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:09.378131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9b793c6f-3844-44d5-9b37-18e2600f339b tempest-ProjectsNegativeTestJSON-1065984173 tempest-ProjectsNegativeTestJSON-1065984173-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:09.471646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:09.479021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b793c6f-3844-44d5-9b37-18e2600f339b tempest-ProjectsNegativeTestJSON-1065984173 tempest-ProjectsNegativeTestJSON-1065984173-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:09.562678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:09.563590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:09.563864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:09.564127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] PATH_INFO: `/v3/users/0c5b969e7405485096edfea59c7983ed` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:09.565327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c5b969e7405485096edfea59c7983ed)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:09.567418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:09.702399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c5b969e7405485096edfea59c7983ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:09.713239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c5b969e7405485096edfea59c7983ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:09.733553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c5b969e7405485096edfea59c7983ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:09.750394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c5b969e7405485096edfea59c7983ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:09.751305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c5b969e7405485096edfea59c7983ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:09.759437 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2178/4316] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:09 2022] DELETE /identity/v3/users/0c5b969e7405485096edfea59c7983ed => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:09.764055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556808e-3d28-45f2-83e5-b79c63d94217 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:09.764333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556808e-3d28-45f2-83e5-b79c63d94217 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:09.764629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556808e-3d28-45f2-83e5-b79c63d94217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:09.790081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5556808e-3d28-45f2-83e5-b79c63d94217 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:09.791641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5556808e-3d28-45f2-83e5-b79c63d94217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:09.887847 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2139/4317] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:09.899783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:09.902252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:09.981197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177b06d-1d60-4bd2-82f0-6ab3b9d04d67 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:09.985969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bafca8-539d-48a0-9507-ba34659aaf02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:09.987032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bafca8-539d-48a0-9507-ba34659aaf02 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:09.987492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bafca8-539d-48a0-9507-ba34659aaf02 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:09.987785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bafca8-539d-48a0-9507-ba34659aaf02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:10.004193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bafca8-539d-48a0-9507-ba34659aaf02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:10.006542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bafca8-539d-48a0-9507-ba34659aaf02 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:10.018607 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2179/4318] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:10.367166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556808e-3d28-45f2-83e5-b79c63d94217 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:10.368699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5556808e-3d28-45f2-83e5-b79c63d94217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:10.387237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556808e-3d28-45f2-83e5-b79c63d94217 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:10.396689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:10.397353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:10.397633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:10.397999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] PATH_INFO: `/v3/projects/e108f56c7b2d409591d2c7260ab07446` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:10.399163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e108f56c7b2d409591d2c7260ab07446)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:10.401424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:10.464540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e108f56c7b2d409591d2c7260ab07446', 'request_id': 'req-b7cc7f01-3aeb-4457-973d-436b33598e88'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:10.472121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e108f56c7b2d409591d2c7260ab07446', 'request_id': 'req-b7cc7f01-3aeb-4457-973d-436b33598e88'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:10.481120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e108f56c7b2d409591d2c7260ab07446', 'request_id': 'req-b7cc7f01-3aeb-4457-973d-436b33598e88'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:10.492195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] The token cache is being invalidate because project e108f56c7b2d409591d2c7260ab07446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:10.492509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:10.493162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:10.494678 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2140/4319] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:10 2022] DELETE /identity/v3/projects/e108f56c7b2d409591d2c7260ab07446 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:10.498792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bafca8-539d-48a0-9507-ba34659aaf02 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:10.509853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bafca8-539d-48a0-9507-ba34659aaf02 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:10.536207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:10.582184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42248bd2-72ec-43a2-9497-d7a6db5a7cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:10.583184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42248bd2-72ec-43a2-9497-d7a6db5a7cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:10.583437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42248bd2-72ec-43a2-9497-d7a6db5a7cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:10.583701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42248bd2-72ec-43a2-9497-d7a6db5a7cb0 admin admin] PATH_INFO: `/v3/users/58caab0aa986444eb3708d027251b06a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:10.585150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42248bd2-72ec-43a2-9497-d7a6db5a7cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58caab0aa986444eb3708d027251b06a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:10.587217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42248bd2-72ec-43a2-9497-d7a6db5a7cb0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:10.645557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-42248bd2-72ec-43a2-9497-d7a6db5a7cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58caab0aa986444eb3708d027251b06a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:10.654541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-42248bd2-72ec-43a2-9497-d7a6db5a7cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58caab0aa986444eb3708d027251b06a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:10.665652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-42248bd2-72ec-43a2-9497-d7a6db5a7cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58caab0aa986444eb3708d027251b06a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:10.680986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-42248bd2-72ec-43a2-9497-d7a6db5a7cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58caab0aa986444eb3708d027251b06a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:10.680986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-42248bd2-72ec-43a2-9497-d7a6db5a7cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58caab0aa986444eb3708d027251b06a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:10.688790 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2180/4320] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:10 2022] DELETE /identity/v3/users/58caab0aa986444eb3708d027251b06a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:11.379508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:11.382360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:11.405115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc7f01-3aeb-4457-973d-436b33598e88 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:11.481587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:11.492898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cf6e4-7d45-428b-8984-67d748d93f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:11.493583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cf6e4-7d45-428b-8984-67d748d93f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:11.494212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cf6e4-7d45-428b-8984-67d748d93f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:11.494992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cf6e4-7d45-428b-8984-67d748d93f38 admin admin] PATH_INFO: `/v3/projects/b5160740bbeb4b7bba933ceafed38239` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:11.496531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cf6e4-7d45-428b-8984-67d748d93f38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5160740bbeb4b7bba933ceafed38239)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:11.498697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cf6e4-7d45-428b-8984-67d748d93f38 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:11.567713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-895cf6e4-7d45-428b-8984-67d748d93f38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5160740bbeb4b7bba933ceafed38239', 'request_id': 'req-895cf6e4-7d45-428b-8984-67d748d93f38'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:11.577818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-895cf6e4-7d45-428b-8984-67d748d93f38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5160740bbeb4b7bba933ceafed38239', 'request_id': 'req-895cf6e4-7d45-428b-8984-67d748d93f38'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:11.588930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-895cf6e4-7d45-428b-8984-67d748d93f38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5160740bbeb4b7bba933ceafed38239', 'request_id': 'req-895cf6e4-7d45-428b-8984-67d748d93f38'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:11.605806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-895cf6e4-7d45-428b-8984-67d748d93f38 admin admin] The token cache is being invalidate because project b5160740bbeb4b7bba933ceafed38239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:11.606234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-895cf6e4-7d45-428b-8984-67d748d93f38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:11.607485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-895cf6e4-7d45-428b-8984-67d748d93f38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:11.614128 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2141/4321] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:11 2022] DELETE /identity/v3/projects/b5160740bbeb4b7bba933ceafed38239 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:11.620243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b3f6c-f1fb-458e-bc34-eee66d1a5fe2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:11.621169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b3f6c-f1fb-458e-bc34-eee66d1a5fe2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:11.621584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b3f6c-f1fb-458e-bc34-eee66d1a5fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:11.644495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-536b3f6c-f1fb-458e-bc34-eee66d1a5fe2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:11.646190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-536b3f6c-f1fb-458e-bc34-eee66d1a5fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:11.719435 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2181/4322] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:11.727708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65a66f-c87e-4c5a-a8c7-47a8075c7bab None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:11.728071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65a66f-c87e-4c5a-a8c7-47a8075c7bab None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:11.728343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65a66f-c87e-4c5a-a8c7-47a8075c7bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:11.749147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ab65a66f-c87e-4c5a-a8c7-47a8075c7bab None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:11.750431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ab65a66f-c87e-4c5a-a8c7-47a8075c7bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:11.759859 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2142/4323] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:11.765736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b3f6c-f1fb-458e-bc34-eee66d1a5fe2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:11.782951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b3f6c-f1fb-458e-bc34-eee66d1a5fe2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:11.788162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2da6d3-b3e5-4c3d-af50-883894c244cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:11.788846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2da6d3-b3e5-4c3d-af50-883894c244cd admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:11.789535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2da6d3-b3e5-4c3d-af50-883894c244cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:11.792288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2da6d3-b3e5-4c3d-af50-883894c244cd admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:11.792801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2da6d3-b3e5-4c3d-af50-883894c244cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:11.793043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2da6d3-b3e5-4c3d-af50-883894c244cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:11.794884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2da6d3-b3e5-4c3d-af50-883894c244cd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:11.817324 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2182/4324] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:11.822403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65a66f-c87e-4c5a-a8c7-47a8075c7bab None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:11.832676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65a66f-c87e-4c5a-a8c7-47a8075c7bab None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:11.838342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f9fe9-867b-417f-9b94-957391797f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:11.846285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f9fe9-867b-417f-9b94-957391797f52 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:11.846560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f9fe9-867b-417f-9b94-957391797f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:11.846827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f9fe9-867b-417f-9b94-957391797f52 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:11.847471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f9fe9-867b-417f-9b94-957391797f52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:11.849971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f9fe9-867b-417f-9b94-957391797f52 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:11.912900 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2143/4325] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:11 2022] POST /identity/v3/projects => generated 369 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:11.914492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2da6d3-b3e5-4c3d-af50-883894c244cd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:11.936871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2da6d3-b3e5-4c3d-af50-883894c244cd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:12.025860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:12.107216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef512ae7-d382-4238-b5e1-c7c5efd4a418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:12.107606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef512ae7-d382-4238-b5e1-c7c5efd4a418 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:12.107874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef512ae7-d382-4238-b5e1-c7c5efd4a418 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:12.108148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef512ae7-d382-4238-b5e1-c7c5efd4a418 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:12.108824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef512ae7-d382-4238-b5e1-c7c5efd4a418 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:12.110767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef512ae7-d382-4238-b5e1-c7c5efd4a418 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:12.170753 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2183/4326] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:11 2022] POST /identity/v3/users => generated 285 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:12.175565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f9fe9-867b-417f-9b94-957391797f52 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:12.192355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f9fe9-867b-417f-9b94-957391797f52 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:12.208465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393363b3-7fba-4ab4-aeac-e43f73d6b38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:12.209254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393363b3-7fba-4ab4-aeac-e43f73d6b38a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:12.209526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393363b3-7fba-4ab4-aeac-e43f73d6b38a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:12.209792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393363b3-7fba-4ab4-aeac-e43f73d6b38a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:12.212453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393363b3-7fba-4ab4-aeac-e43f73d6b38a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:12.218522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393363b3-7fba-4ab4-aeac-e43f73d6b38a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:12.242260 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2144/4327] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:12.247533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef512ae7-d382-4238-b5e1-c7c5efd4a418 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:12.266387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef512ae7-d382-4238-b5e1-c7c5efd4a418 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:12.282432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93aa91-6e50-4522-a657-7e2821a23e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:12.283555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93aa91-6e50-4522-a657-7e2821a23e4a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:12.284013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93aa91-6e50-4522-a657-7e2821a23e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:12.284486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93aa91-6e50-4522-a657-7e2821a23e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:12.285296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93aa91-6e50-4522-a657-7e2821a23e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:12.287802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93aa91-6e50-4522-a657-7e2821a23e4a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:12.307006 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2184/4328] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:12.311153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393363b3-7fba-4ab4-aeac-e43f73d6b38a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:12.329425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393363b3-7fba-4ab4-aeac-e43f73d6b38a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:12.341542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c52e9-49e7-4cc1-b823-fcbe0391fccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:12.342892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c52e9-49e7-4cc1-b823-fcbe0391fccd admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:12.343570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c52e9-49e7-4cc1-b823-fcbe0391fccd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:12.343849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c52e9-49e7-4cc1-b823-fcbe0391fccd admin admin] PATH_INFO: `/v3/projects/d64206fece414e70b231268263d8678a/users/789f31a888294c8392e0ce14d6e92e68/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:12.382863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c52e9-49e7-4cc1-b823-fcbe0391fccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64206fece414e70b231268263d8678a, user_id=789f31a888294c8392e0ce14d6e92e68, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:12.386068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c52e9-49e7-4cc1-b823-fcbe0391fccd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:12.416787 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2145/4329] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:12 2022] PUT /identity/v3/projects/d64206fece414e70b231268263d8678a/users/789f31a888294c8392e0ce14d6e92e68/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:12.434413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752514d8-e515-4065-bdf0-52429bfb4ef2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:12.434711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752514d8-e515-4065-bdf0-52429bfb4ef2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:12.434943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752514d8-e515-4065-bdf0-52429bfb4ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:12.491733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-752514d8-e515-4065-bdf0-52429bfb4ef2 None None] MFA Rules not processed for user `789f31a888294c8392e0ce14d6e92e68`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:12.492768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-752514d8-e515-4065-bdf0-52429bfb4ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:12.637835 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2185/4330] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:12.643506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c52e9-49e7-4cc1-b823-fcbe0391fccd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:12.862195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c52e9-49e7-4cc1-b823-fcbe0391fccd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:12.875733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ec306-2dcc-4de5-9b93-a8dee2369a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:12.878900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ec306-2dcc-4de5-9b93-a8dee2369a78 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:12.879387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ec306-2dcc-4de5-9b93-a8dee2369a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:12.879852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ec306-2dcc-4de5-9b93-a8dee2369a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:12.882548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ec306-2dcc-4de5-9b93-a8dee2369a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:12.887297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ec306-2dcc-4de5-9b93-a8dee2369a78 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:12.957389 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2146/4331] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:12 2022] POST /identity/v3/projects => generated 369 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:12.961192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752514d8-e515-4065-bdf0-52429bfb4ef2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:12.979027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752514d8-e515-4065-bdf0-52429bfb4ef2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:13.105945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e199f94-9e0c-4d4b-9e2f-38b09c7e2033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:13.107162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e199f94-9e0c-4d4b-9e2f-38b09c7e2033 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:13.107735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e199f94-9e0c-4d4b-9e2f-38b09c7e2033 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:13.108010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e199f94-9e0c-4d4b-9e2f-38b09c7e2033 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:13.108676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e199f94-9e0c-4d4b-9e2f-38b09c7e2033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:13.110946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e199f94-9e0c-4d4b-9e2f-38b09c7e2033 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:13.152436 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2186/4332] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:12 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:13.155180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ec306-2dcc-4de5-9b93-a8dee2369a78 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:13.167923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ec306-2dcc-4de5-9b93-a8dee2369a78 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:13.173629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1a922-8eab-4e67-b069-fb3de55ae682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:13.174367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1a922-8eab-4e67-b069-fb3de55ae682 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:13.175311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1a922-8eab-4e67-b069-fb3de55ae682 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:13.175649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1a922-8eab-4e67-b069-fb3de55ae682 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:13.176404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1a922-8eab-4e67-b069-fb3de55ae682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:13.178264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1a922-8eab-4e67-b069-fb3de55ae682 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:13.179925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:13.204043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e199f94-9e0c-4d4b-9e2f-38b09c7e2033 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:13.212703 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2147/4333] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:13.217426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e199f94-9e0c-4d4b-9e2f-38b09c7e2033 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:13.222817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfba99-3b63-4708-addd-a82104357de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:13.223873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfba99-3b63-4708-addd-a82104357de1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:13.225441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfba99-3b63-4708-addd-a82104357de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:13.225994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfba99-3b63-4708-addd-a82104357de1 admin admin] PATH_INFO: `/v3/projects/1c1a313be3e14dfc903ee36e51514f0f/users/39f931c959f34c3a967966ccd8d1533e/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:13.253207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfba99-3b63-4708-addd-a82104357de1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c1a313be3e14dfc903ee36e51514f0f, user_id=39f931c959f34c3a967966ccd8d1533e, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:13.256413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfba99-3b63-4708-addd-a82104357de1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:13.294270 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2187/4334] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:13 2022] PUT /identity/v3/projects/1c1a313be3e14dfc903ee36e51514f0f/users/39f931c959f34c3a967966ccd8d1533e/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:13.299352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1a922-8eab-4e67-b069-fb3de55ae682 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:13.314329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1a922-8eab-4e67-b069-fb3de55ae682 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:13.421576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de82c4f-3076-4c17-95b2-83b2f368bb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:13.422312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de82c4f-3076-4c17-95b2-83b2f368bb8c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:13.422537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de82c4f-3076-4c17-95b2-83b2f368bb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:13.422805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de82c4f-3076-4c17-95b2-83b2f368bb8c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:13.423510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de82c4f-3076-4c17-95b2-83b2f368bb8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:13.425290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de82c4f-3076-4c17-95b2-83b2f368bb8c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:13.468536 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2148/4335] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:13.470679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfba99-3b63-4708-addd-a82104357de1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:13.483454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:13.493669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfba99-3b63-4708-addd-a82104357de1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:13.512708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3a9b6-134b-48b1-b607-892a61e3bfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:13.513397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3a9b6-134b-48b1-b607-892a61e3bfd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:13.513666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3a9b6-134b-48b1-b607-892a61e3bfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:13.513931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3a9b6-134b-48b1-b607-892a61e3bfd6 admin admin] PATH_INFO: `/v3/projects/1c1a313be3e14dfc903ee36e51514f0f/users/39f931c959f34c3a967966ccd8d1533e/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:13.517413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3a9b6-134b-48b1-b607-892a61e3bfd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c1a313be3e14dfc903ee36e51514f0f, user_id=39f931c959f34c3a967966ccd8d1533e, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:13.520304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3a9b6-134b-48b1-b607-892a61e3bfd6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:13.535398 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2188/4336] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:13 2022] PUT /identity/v3/projects/1c1a313be3e14dfc903ee36e51514f0f/users/39f931c959f34c3a967966ccd8d1533e/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:13.547139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87cf46-b7b4-4891-b329-e6b619c68075 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:13.548117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87cf46-b7b4-4891-b329-e6b619c68075 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:13.548527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87cf46-b7b4-4891-b329-e6b619c68075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:13.583485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ce87cf46-b7b4-4891-b329-e6b619c68075 None None] MFA Rules not processed for user `39f931c959f34c3a967966ccd8d1533e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:13.584851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ce87cf46-b7b4-4891-b329-e6b619c68075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:13.693991 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2149/4337] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:13.698904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3a9b6-134b-48b1-b607-892a61e3bfd6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:13.783401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3a9b6-134b-48b1-b607-892a61e3bfd6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:13.791473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acca268-24ee-4d8a-b3ed-763503e27fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:13.792113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acca268-24ee-4d8a-b3ed-763503e27fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:13.792399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acca268-24ee-4d8a-b3ed-763503e27fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:13.792673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acca268-24ee-4d8a-b3ed-763503e27fb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:13.793297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acca268-24ee-4d8a-b3ed-763503e27fb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:13.795668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acca268-24ee-4d8a-b3ed-763503e27fb5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:13.875266 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2189/4338] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:13 2022] POST /identity/v3/projects => generated 369 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:13.879531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87cf46-b7b4-4891-b329-e6b619c68075 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:13.894188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87cf46-b7b4-4891-b329-e6b619c68075 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:14.001998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33727699-ccc2-4885-b3f6-c40fa4dd18f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:14.002645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33727699-ccc2-4885-b3f6-c40fa4dd18f6 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:14.002985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33727699-ccc2-4885-b3f6-c40fa4dd18f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:14.003252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33727699-ccc2-4885-b3f6-c40fa4dd18f6 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:14.003983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33727699-ccc2-4885-b3f6-c40fa4dd18f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:14.006112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33727699-ccc2-4885-b3f6-c40fa4dd18f6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:14.064809 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2150/4339] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:13 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:14.066034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acca268-24ee-4d8a-b3ed-763503e27fb5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:14.083119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acca268-24ee-4d8a-b3ed-763503e27fb5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:14.103322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a7a30-78a2-435f-a137-b676d578401e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:14.104239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a7a30-78a2-435f-a137-b676d578401e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:14.105070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a7a30-78a2-435f-a137-b676d578401e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:14.105070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a7a30-78a2-435f-a137-b676d578401e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:14.105639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a7a30-78a2-435f-a137-b676d578401e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:14.107240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a7a30-78a2-435f-a137-b676d578401e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:14.129106 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2190/4340] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:14.135050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33727699-ccc2-4885-b3f6-c40fa4dd18f6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:14.154922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33727699-ccc2-4885-b3f6-c40fa4dd18f6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:14.161062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff94b9-158f-44b7-a21e-462c103a880c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:14.161812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff94b9-158f-44b7-a21e-462c103a880c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:14.162089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff94b9-158f-44b7-a21e-462c103a880c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:14.162359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff94b9-158f-44b7-a21e-462c103a880c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:14.167434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff94b9-158f-44b7-a21e-462c103a880c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:14.172674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff94b9-158f-44b7-a21e-462c103a880c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:14.174354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:14.201839 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2151/4341] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:14.203046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a7a30-78a2-435f-a137-b676d578401e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:14.222143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a7a30-78a2-435f-a137-b676d578401e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:14.229869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120f5bb-8c39-49a8-bdb7-76c7af9eb7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:14.230774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120f5bb-8c39-49a8-bdb7-76c7af9eb7d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:14.231314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120f5bb-8c39-49a8-bdb7-76c7af9eb7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:14.231731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120f5bb-8c39-49a8-bdb7-76c7af9eb7d7 admin admin] PATH_INFO: `/v3/projects/4e6b7b46a6004ee483a4824756e61204/users/e182dc8f2ade4e66abc883423eba9040/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:14.254037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120f5bb-8c39-49a8-bdb7-76c7af9eb7d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6b7b46a6004ee483a4824756e61204, user_id=e182dc8f2ade4e66abc883423eba9040, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:14.257177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120f5bb-8c39-49a8-bdb7-76c7af9eb7d7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:14.288450 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2191/4342] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:14 2022] PUT /identity/v3/projects/4e6b7b46a6004ee483a4824756e61204/users/e182dc8f2ade4e66abc883423eba9040/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:14.299887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf22cf9-798e-4d07-a4d6-11b34e89850b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:14.299887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf22cf9-798e-4d07-a4d6-11b34e89850b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:14.300315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf22cf9-798e-4d07-a4d6-11b34e89850b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:14.352097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-acf22cf9-798e-4d07-a4d6-11b34e89850b None None] MFA Rules not processed for user `e182dc8f2ade4e66abc883423eba9040`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:14.353440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-acf22cf9-798e-4d07-a4d6-11b34e89850b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:14.496147 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2152/4343] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:14.501167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120f5bb-8c39-49a8-bdb7-76c7af9eb7d7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:14.591284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120f5bb-8c39-49a8-bdb7-76c7af9eb7d7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:14.596297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9f83d-952d-44fe-88b4-97d2ef96816c tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:14.596969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9f83d-952d-44fe-88b4-97d2ef96816c tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:14.597238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9f83d-952d-44fe-88b4-97d2ef96816c tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:14.597509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9f83d-952d-44fe-88b4-97d2ef96816c tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:14.598216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9f83d-952d-44fe-88b4-97d2ef96816c tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:14.599874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9f83d-952d-44fe-88b4-97d2ef96816c tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:14.633826 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2192/4344] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:09:14 2022] POST /identity/v3/domains => generated 276 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:14.637815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf22cf9-798e-4d07-a4d6-11b34e89850b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:14.651499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf22cf9-798e-4d07-a4d6-11b34e89850b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:14.721379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2fb3f-61b6-4860-846e-8d33b94d88ec tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:14.722310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2fb3f-61b6-4860-846e-8d33b94d88ec tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:14.722762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2fb3f-61b6-4860-846e-8d33b94d88ec tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:14.723505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2fb3f-61b6-4860-846e-8d33b94d88ec tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:14.725224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2fb3f-61b6-4860-846e-8d33b94d88ec tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:14.726987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2fb3f-61b6-4860-846e-8d33b94d88ec tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:14.750565 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2153/4345] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:09:14 2022] POST /identity/v3/roles => generated 239 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:14.753479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9f83d-952d-44fe-88b4-97d2ef96816c tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:14.768452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9f83d-952d-44fe-88b4-97d2ef96816c tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:14.774123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b8403-0675-436e-a3de-1a2a99b0545e tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:14.775273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b8403-0675-436e-a3de-1a2a99b0545e tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:14.775737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b8403-0675-436e-a3de-1a2a99b0545e tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:14.776197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b8403-0675-436e-a3de-1a2a99b0545e tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/839183bb6e2c4e46bb498860d9e45a14/users/e182dc8f2ade4e66abc883423eba9040/roles/f0cbe7f8f9bf4cd0a2e8731f8f661dce` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:14.795919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b8403-0675-436e-a3de-1a2a99b0545e tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=839183bb6e2c4e46bb498860d9e45a14, user_id=e182dc8f2ade4e66abc883423eba9040, role_id=f0cbe7f8f9bf4cd0a2e8731f8f661dce)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:14.799814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b8403-0675-436e-a3de-1a2a99b0545e tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:14.818726 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2193/4346] 10.222.0.23 () {60 vars in 1480 bytes} [Sat Aug 20 21:09:14 2022] PUT /identity/v3/domains/839183bb6e2c4e46bb498860d9e45a14/users/e182dc8f2ade4e66abc883423eba9040/roles/f0cbe7f8f9bf4cd0a2e8731f8f661dce => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:14.823637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2fb3f-61b6-4860-846e-8d33b94d88ec tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:14.836216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2fb3f-61b6-4860-846e-8d33b94d88ec tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:14.919800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541e2fd-06d8-4689-8b6e-379504bf5f80 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:14.920962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541e2fd-06d8-4689-8b6e-379504bf5f80 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:14.921442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541e2fd-06d8-4689-8b6e-379504bf5f80 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:14.921974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541e2fd-06d8-4689-8b6e-379504bf5f80 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:14.922647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541e2fd-06d8-4689-8b6e-379504bf5f80 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:14.924683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541e2fd-06d8-4689-8b6e-379504bf5f80 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:14.952066 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2154/4347] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:09:14 2022] POST /identity/v3/domains => generated 276 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:14.953356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b8403-0675-436e-a3de-1a2a99b0545e tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:14.967756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b8403-0675-436e-a3de-1a2a99b0545e tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:14.979287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:15.033044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d6468-f236-4c94-8648-be47e7e4cd15 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:15.033807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d6468-f236-4c94-8648-be47e7e4cd15 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:15.034090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d6468-f236-4c94-8648-be47e7e4cd15 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:15.034358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d6468-f236-4c94-8648-be47e7e4cd15 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:15.035016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d6468-f236-4c94-8648-be47e7e4cd15 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:15.036878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d6468-f236-4c94-8648-be47e7e4cd15 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:15.055010 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2194/4348] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:09:14 2022] POST /identity/v3/roles => generated 240 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:15.059459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541e2fd-06d8-4689-8b6e-379504bf5f80 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:15.071415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541e2fd-06d8-4689-8b6e-379504bf5f80 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:15.077674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f100b5-1179-4002-a9d2-3eb1b0cf5bb0 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:15.078602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f100b5-1179-4002-a9d2-3eb1b0cf5bb0 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:15.079072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f100b5-1179-4002-a9d2-3eb1b0cf5bb0 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:15.079681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f100b5-1179-4002-a9d2-3eb1b0cf5bb0 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/groups` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:15.080661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f100b5-1179-4002-a9d2-3eb1b0cf5bb0 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:15.083145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f100b5-1179-4002-a9d2-3eb1b0cf5bb0 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:15.095719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:15.113794 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2155/4349] 10.222.0.23 () {60 vars in 1147 bytes} [Sat Aug 20 21:09:15 2022] POST /identity/v3/groups => generated 325 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:15.118193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d6468-f236-4c94-8648-be47e7e4cd15 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:15.128598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d6468-f236-4c94-8648-be47e7e4cd15 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:15.133884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38392a9-72a3-4c17-8e5b-b0229493bfea tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:15.134531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38392a9-72a3-4c17-8e5b-b0229493bfea tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:15.134802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38392a9-72a3-4c17-8e5b-b0229493bfea tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:15.135463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38392a9-72a3-4c17-8e5b-b0229493bfea tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/groups/81995cde30094e8b8a29bd908c62be6d/users/e182dc8f2ade4e66abc883423eba9040` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:15.141782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38392a9-72a3-4c17-8e5b-b0229493bfea tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=81995cde30094e8b8a29bd908c62be6d, user_id=e182dc8f2ade4e66abc883423eba9040)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:15.142921 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-f38392a9-72a3-4c17-8e5b-b0229493bfea tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:15.142921 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:09:15.142921 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:09:15.144412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38392a9-72a3-4c17-8e5b-b0229493bfea tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:15.214726 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2195/4350] 10.222.0.23 () {60 vars in 1360 bytes} [Sat Aug 20 21:09:15 2022] PUT /identity/v3/groups/81995cde30094e8b8a29bd908c62be6d/users/e182dc8f2ade4e66abc883423eba9040 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:15.220165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f100b5-1179-4002-a9d2-3eb1b0cf5bb0 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:15.239137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f100b5-1179-4002-a9d2-3eb1b0cf5bb0 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:15.304385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b8907-e407-475c-a46b-61b814a540b5 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:15.305085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b8907-e407-475c-a46b-61b814a540b5 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:15.305366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b8907-e407-475c-a46b-61b814a540b5 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:15.305665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b8907-e407-475c-a46b-61b814a540b5 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/d652d6aa6952461dbec876c41cd7597e/groups/81995cde30094e8b8a29bd908c62be6d/roles/bc3dd8abd88644b98f8f91e0e095d8cc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:15.307894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b8907-e407-475c-a46b-61b814a540b5 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=d652d6aa6952461dbec876c41cd7597e, group_id=81995cde30094e8b8a29bd908c62be6d, role_id=bc3dd8abd88644b98f8f91e0e095d8cc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:15.310605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b8907-e407-475c-a46b-61b814a540b5 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:15.331587 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2156/4351] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:15 2022] PUT /identity/v3/domains/d652d6aa6952461dbec876c41cd7597e/groups/81995cde30094e8b8a29bd908c62be6d/roles/bc3dd8abd88644b98f8f91e0e095d8cc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:15.333163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38392a9-72a3-4c17-8e5b-b0229493bfea tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:15.349515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38392a9-72a3-4c17-8e5b-b0229493bfea tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:15.435248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732cb99-887f-485a-ad51-828013885382 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:15.435958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732cb99-887f-485a-ad51-828013885382 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:15.436300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732cb99-887f-485a-ad51-828013885382 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:15.436633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732cb99-887f-485a-ad51-828013885382 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:15.437324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732cb99-887f-485a-ad51-828013885382 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:15.438911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732cb99-887f-485a-ad51-828013885382 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:15.456009 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2196/4352] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:09:15 2022] POST /identity/v3/domains => generated 276 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:15.460402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b8907-e407-475c-a46b-61b814a540b5 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:15.471189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b8907-e407-475c-a46b-61b814a540b5 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:15.546763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e3286-1963-4cb4-a155-3f1f7d63e43a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:15.547478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e3286-1963-4cb4-a155-3f1f7d63e43a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:15.547842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e3286-1963-4cb4-a155-3f1f7d63e43a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:15.548168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e3286-1963-4cb4-a155-3f1f7d63e43a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:15.549205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e3286-1963-4cb4-a155-3f1f7d63e43a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:15.551140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e3286-1963-4cb4-a155-3f1f7d63e43a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:15.572696 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2157/4353] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:09:15 2022] POST /identity/v3/roles => generated 239 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:15.574574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732cb99-887f-485a-ad51-828013885382 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:15.587495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732cb99-887f-485a-ad51-828013885382 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:15.592581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbd142-f592-4732-a223-5c57895ea923 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:15.593276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbd142-f592-4732-a223-5c57895ea923 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:15.593545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbd142-f592-4732-a223-5c57895ea923 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:15.593810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbd142-f592-4732-a223-5c57895ea923 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/da10f4e86869489b8e0c624d9107f743/users/e182dc8f2ade4e66abc883423eba9040/roles/350f43da7bf940bc8c5462da7ba14fea` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:15.606935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbd142-f592-4732-a223-5c57895ea923 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=da10f4e86869489b8e0c624d9107f743, user_id=e182dc8f2ade4e66abc883423eba9040, role_id=350f43da7bf940bc8c5462da7ba14fea)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:15.609880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbd142-f592-4732-a223-5c57895ea923 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:15.628030 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2197/4354] 10.222.0.23 () {60 vars in 1480 bytes} [Sat Aug 20 21:09:15 2022] PUT /identity/v3/domains/da10f4e86869489b8e0c624d9107f743/users/e182dc8f2ade4e66abc883423eba9040/roles/350f43da7bf940bc8c5462da7ba14fea => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:15.632454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e3286-1963-4cb4-a155-3f1f7d63e43a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:15.643224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e3286-1963-4cb4-a155-3f1f7d63e43a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:15.711592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfc866-78a5-440b-8b8e-91ca17136d88 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:15.712359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfc866-78a5-440b-8b8e-91ca17136d88 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:15.712669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfc866-78a5-440b-8b8e-91ca17136d88 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:15.712943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfc866-78a5-440b-8b8e-91ca17136d88 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:15.713605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfc866-78a5-440b-8b8e-91ca17136d88 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:15.715236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfc866-78a5-440b-8b8e-91ca17136d88 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:15.732660 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2158/4355] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:09:15 2022] POST /identity/v3/domains => generated 275 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:15.737253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbd142-f592-4732-a223-5c57895ea923 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:15.749714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbd142-f592-4732-a223-5c57895ea923 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:15.800868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:15.813003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53fa5f7-388f-4a81-9035-3cad10cbd567 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:15.813618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53fa5f7-388f-4a81-9035-3cad10cbd567 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:15.813893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53fa5f7-388f-4a81-9035-3cad10cbd567 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:15.814260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53fa5f7-388f-4a81-9035-3cad10cbd567 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:15.814936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53fa5f7-388f-4a81-9035-3cad10cbd567 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:15.816567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53fa5f7-388f-4a81-9035-3cad10cbd567 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:15.835059 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2198/4356] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:09:15 2022] POST /identity/v3/roles => generated 239 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:15.839316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfc866-78a5-440b-8b8e-91ca17136d88 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:15.850771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfc866-78a5-440b-8b8e-91ca17136d88 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:15.858068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d3a3a-1179-4afd-b931-bbdd3de0a3cf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:15.858937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3a3a-1179-4afd-b931-bbdd3de0a3cf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:15.859522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3a3a-1179-4afd-b931-bbdd3de0a3cf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:15.859970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3a3a-1179-4afd-b931-bbdd3de0a3cf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/groups` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:15.860910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d3a3a-1179-4afd-b931-bbdd3de0a3cf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:15.863634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d3a3a-1179-4afd-b931-bbdd3de0a3cf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:15.882825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399051fc-5b8f-44d7-a0d0-9ffa6da44901 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:15.882825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399051fc-5b8f-44d7-a0d0-9ffa6da44901 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:15.882825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399051fc-5b8f-44d7-a0d0-9ffa6da44901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:15.887843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:15.910663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-399051fc-5b8f-44d7-a0d0-9ffa6da44901 None None] MFA Rules not processed for user `bbc87613dd6349babc15334a66bee64f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:15.912768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-399051fc-5b8f-44d7-a0d0-9ffa6da44901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:15.924023 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2159/4357] 10.222.0.23 () {60 vars in 1147 bytes} [Sat Aug 20 21:09:15 2022] POST /identity/v3/groups => generated 325 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:15.931460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d3a3a-1179-4afd-b931-bbdd3de0a3cf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:15.949725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d3a3a-1179-4afd-b931-bbdd3de0a3cf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:15.956710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15d8f0-a0c2-44cf-89d6-bc50c1944e52 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:15.957483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15d8f0-a0c2-44cf-89d6-bc50c1944e52 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:15.958120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15d8f0-a0c2-44cf-89d6-bc50c1944e52 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:15.958120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15d8f0-a0c2-44cf-89d6-bc50c1944e52 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/groups/4e0823e2da764393a3db170e320ae27f/users/e182dc8f2ade4e66abc883423eba9040` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:15.989241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15d8f0-a0c2-44cf-89d6-bc50c1944e52 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=4e0823e2da764393a3db170e320ae27f, user_id=e182dc8f2ade4e66abc883423eba9040)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:15.990071 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-3c15d8f0-a0c2-44cf-89d6-bc50c1944e52 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:15.990071 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:09:15.990071 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:09:15.991687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15d8f0-a0c2-44cf-89d6-bc50c1944e52 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:16.054146 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2160/4358] 10.222.0.23 () {60 vars in 1360 bytes} [Sat Aug 20 21:09:15 2022] PUT /identity/v3/groups/4e0823e2da764393a3db170e320ae27f/users/e182dc8f2ade4e66abc883423eba9040 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:16.054353 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2199/4359] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:16.060494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15d8f0-a0c2-44cf-89d6-bc50c1944e52 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:16.164141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15d8f0-a0c2-44cf-89d6-bc50c1944e52 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:16.169953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a5442-7ac6-4ec8-b514-b9eff045a222 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:16.170622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a5442-7ac6-4ec8-b514-b9eff045a222 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:16.170894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a5442-7ac6-4ec8-b514-b9eff045a222 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:16.171208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a5442-7ac6-4ec8-b514-b9eff045a222 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/011648f608d94448ae88cbac66227167/groups/4e0823e2da764393a3db170e320ae27f/roles/52bd5fdf1f2d4c2a816fec69a65bc523` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:16.173854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a5442-7ac6-4ec8-b514-b9eff045a222 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=011648f608d94448ae88cbac66227167, group_id=4e0823e2da764393a3db170e320ae27f, role_id=52bd5fdf1f2d4c2a816fec69a65bc523)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:16.176170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a5442-7ac6-4ec8-b514-b9eff045a222 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:16.197707 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2161/4360] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:16 2022] PUT /identity/v3/domains/011648f608d94448ae88cbac66227167/groups/4e0823e2da764393a3db170e320ae27f/roles/52bd5fdf1f2d4c2a816fec69a65bc523 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:16.202356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399051fc-5b8f-44d7-a0d0-9ffa6da44901 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:16.337580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399051fc-5b8f-44d7-a0d0-9ffa6da44901 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:16.440480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:16.458306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b9e31-8544-4fb4-a158-238227390e18 tempest-TokensV3TestJSON-1890703781 tempest-TokensV3TestJSON-1890703781-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e182dc8f2ade4e66abc883423eba9040', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e6b7b46a6004ee483a4824756e61204', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:16.458922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b9e31-8544-4fb4-a158-238227390e18 tempest-TokensV3TestJSON-1890703781 tempest-TokensV3TestJSON-1890703781-project] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:16.460066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b9e31-8544-4fb4-a158-238227390e18 tempest-TokensV3TestJSON-1890703781 tempest-TokensV3TestJSON-1890703781-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:16.461104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b9e31-8544-4fb4-a158-238227390e18 tempest-TokensV3TestJSON-1890703781 tempest-TokensV3TestJSON-1890703781-project] PATH_INFO: `/v3/auth/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:16.461700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b9e31-8544-4fb4-a158-238227390e18 tempest-TokensV3TestJSON-1890703781 tempest-TokensV3TestJSON-1890703781-project] RBAC: Authorizing `identity:get_auth_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:16.463308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b9e31-8544-4fb4-a158-238227390e18 tempest-TokensV3TestJSON-1890703781 tempest-TokensV3TestJSON-1890703781-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:16.563033 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2200/4361] 10.222.0.23 () {58 vars in 1143 bytes} [Sat Aug 20 21:09:16 2022] GET /identity/v3/auth/domains => generated 1171 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:16.567174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a5442-7ac6-4ec8-b514-b9eff045a222 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:16.665728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a5442-7ac6-4ec8-b514-b9eff045a222 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:16.670580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b633a21-64a6-4728-b28d-c569a01560d2 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:16.671096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b633a21-64a6-4728-b28d-c569a01560d2 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:16.671370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b633a21-64a6-4728-b28d-c569a01560d2 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:16.671636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b633a21-64a6-4728-b28d-c569a01560d2 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/011648f608d94448ae88cbac66227167/groups/4e0823e2da764393a3db170e320ae27f/roles/52bd5fdf1f2d4c2a816fec69a65bc523` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:16.673966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b633a21-64a6-4728-b28d-c569a01560d2 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=011648f608d94448ae88cbac66227167, group_id=4e0823e2da764393a3db170e320ae27f, role_id=52bd5fdf1f2d4c2a816fec69a65bc523)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:16.675005 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-4b633a21-64a6-4728-b28d-c569a01560d2 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:16.675005 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:09:16.675005 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:09:16.677164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b633a21-64a6-4728-b28d-c569a01560d2 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:16.685334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4b633a21-64a6-4728-b28d-c569a01560d2 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invalidating the token cache because role 52bd5fdf1f2d4c2a816fec69a65bc523 was removed from group 4e0823e2da764393a3db170e320ae27f on domain 011648f608d94448ae88cbac66227167. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:16.685624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4b633a21-64a6-4728-b28d-c569a01560d2 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:16.686372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4b633a21-64a6-4728-b28d-c569a01560d2 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:16.702195 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2162/4362] 10.222.0.23 () {58 vars in 1467 bytes} [Sat Aug 20 21:09:16 2022] DELETE /identity/v3/domains/011648f608d94448ae88cbac66227167/groups/4e0823e2da764393a3db170e320ae27f/roles/52bd5fdf1f2d4c2a816fec69a65bc523 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:16.706426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b9e31-8544-4fb4-a158-238227390e18 tempest-TokensV3TestJSON-1890703781 tempest-TokensV3TestJSON-1890703781-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:16.708491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7f6b9e31-8544-4fb4-a158-238227390e18 tempest-TokensV3TestJSON-1890703781 tempest-TokensV3TestJSON-1890703781-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:16.722412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b9e31-8544-4fb4-a158-238227390e18 tempest-TokensV3TestJSON-1890703781 tempest-TokensV3TestJSON-1890703781-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:16.783545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94f9bc-3263-4453-a220-972e61eea85d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:16.784386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94f9bc-3263-4453-a220-972e61eea85d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:16.784688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94f9bc-3263-4453-a220-972e61eea85d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:16.785352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94f9bc-3263-4453-a220-972e61eea85d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/groups/4e0823e2da764393a3db170e320ae27f/users/e182dc8f2ade4e66abc883423eba9040` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:16.787280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94f9bc-3263-4453-a220-972e61eea85d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=4e0823e2da764393a3db170e320ae27f, user_id=e182dc8f2ade4e66abc883423eba9040)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:16.788331 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-8d94f9bc-3263-4453-a220-972e61eea85d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:16.788331 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:09:16.788331 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:09:16.789567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94f9bc-3263-4453-a220-972e61eea85d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:16.797985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8d94f9bc-3263-4453-a220-972e61eea85d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'e182dc8f2ade4e66abc883423eba9040'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:16.814152 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2201/4363] 10.222.0.23 () {58 vars in 1344 bytes} [Sat Aug 20 21:09:16 2022] DELETE /identity/v3/groups/4e0823e2da764393a3db170e320ae27f/users/e182dc8f2ade4e66abc883423eba9040 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:16.818550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b633a21-64a6-4728-b28d-c569a01560d2 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:16.830821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b633a21-64a6-4728-b28d-c569a01560d2 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:16.888702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:16.920721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e3343-06b2-4782-8388-c877988ba20d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:16.921985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e3343-06b2-4782-8388-c877988ba20d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:16.922571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e3343-06b2-4782-8388-c877988ba20d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:16.922920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e3343-06b2-4782-8388-c877988ba20d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/groups/4e0823e2da764393a3db170e320ae27f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:16.938378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e3343-06b2-4782-8388-c877988ba20d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_group(group_id=4e0823e2da764393a3db170e320ae27f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:16.938378 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-041e3343-06b2-4782-8388-c877988ba20d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:16.938378 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:09:16.938378 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:09:16.938378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e3343-06b2-4782-8388-c877988ba20d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:17.000924 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2163/4364] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:09:16 2022] DELETE /identity/v3/groups/4e0823e2da764393a3db170e320ae27f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:17.009624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94f9bc-3263-4453-a220-972e61eea85d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:17.025692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94f9bc-3263-4453-a220-972e61eea85d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:17.094363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67729061-4255-49af-89a0-c196b2367ec8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:17.094995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67729061-4255-49af-89a0-c196b2367ec8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:17.095278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67729061-4255-49af-89a0-c196b2367ec8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:17.095543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67729061-4255-49af-89a0-c196b2367ec8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/roles/52bd5fdf1f2d4c2a816fec69a65bc523` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:17.097071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67729061-4255-49af-89a0-c196b2367ec8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_role(role_id=52bd5fdf1f2d4c2a816fec69a65bc523)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:17.098663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67729061-4255-49af-89a0-c196b2367ec8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:17.117358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:17.141542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-67729061-4255-49af-89a0-c196b2367ec8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invalidating the token cache because role 52bd5fdf1f2d4c2a816fec69a65bc523 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:17.141743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-67729061-4255-49af-89a0-c196b2367ec8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.142378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-67729061-4255-49af-89a0-c196b2367ec8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.144178 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2202/4365] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:17 2022] DELETE /identity/v3/roles/52bd5fdf1f2d4c2a816fec69a65bc523 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:17.148849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e3343-06b2-4782-8388-c877988ba20d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:17.150660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-041e3343-06b2-4782-8388-c877988ba20d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:17.177739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e3343-06b2-4782-8388-c877988ba20d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:17.268958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae9277-182a-446f-96e9-eea939af3ff1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:17.269610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae9277-182a-446f-96e9-eea939af3ff1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:17.269882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae9277-182a-446f-96e9-eea939af3ff1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:17.270154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae9277-182a-446f-96e9-eea939af3ff1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/011648f608d94448ae88cbac66227167` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:17.271591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae9277-182a-446f-96e9-eea939af3ff1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=011648f608d94448ae88cbac66227167)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:17.273247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae9277-182a-446f-96e9-eea939af3ff1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:17.295756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10ae9277-182a-446f-96e9-eea939af3ff1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '011648f608d94448ae88cbac66227167'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.296676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10ae9277-182a-446f-96e9-eea939af3ff1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '011648f608d94448ae88cbac66227167'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.327291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10ae9277-182a-446f-96e9-eea939af3ff1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '011648f608d94448ae88cbac66227167'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.328692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10ae9277-182a-446f-96e9-eea939af3ff1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '011648f608d94448ae88cbac66227167'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.336931 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2164/4366] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:09:17 2022] PATCH /identity/v3/domains/011648f608d94448ae88cbac66227167 => generated 276 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:17.339430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67729061-4255-49af-89a0-c196b2367ec8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:17.375471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67729061-4255-49af-89a0-c196b2367ec8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:17.431056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:17.431618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:17.431878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:17.432145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/011648f608d94448ae88cbac66227167` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:17.443786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=011648f608d94448ae88cbac66227167)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:17.445854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:17.469322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '011648f608d94448ae88cbac66227167'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.480080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '011648f608d94448ae88cbac66227167'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.523826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011648f608d94448ae88cbac66227167', 'request_id': 'req-3467132f-0a85-43d0-b12d-38323d39ef42'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.533940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011648f608d94448ae88cbac66227167', 'request_id': 'req-3467132f-0a85-43d0-b12d-38323d39ef42'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.540676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011648f608d94448ae88cbac66227167', 'request_id': 'req-3467132f-0a85-43d0-b12d-38323d39ef42'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.562510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] The token cache is being invalidate because project 011648f608d94448ae88cbac66227167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:17.563367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.564589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.574507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '011648f608d94448ae88cbac66227167', 'request_id': 'req-3467132f-0a85-43d0-b12d-38323d39ef42'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.576015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '011648f608d94448ae88cbac66227167', 'request_id': 'req-3467132f-0a85-43d0-b12d-38323d39ef42'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.577528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:17.580296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '011648f608d94448ae88cbac66227167', 'request_id': 'req-3467132f-0a85-43d0-b12d-38323d39ef42'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.589968 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2203/4367] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:17 2022] DELETE /identity/v3/domains/011648f608d94448ae88cbac66227167 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:17.591232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae9277-182a-446f-96e9-eea939af3ff1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:17.592997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-10ae9277-182a-446f-96e9-eea939af3ff1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:17.609435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae9277-182a-446f-96e9-eea939af3ff1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:17.653411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:17.685214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29d2b7-3921-479e-8bb3-218f5bf64b84 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:17.685619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29d2b7-3921-479e-8bb3-218f5bf64b84 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:17.685956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29d2b7-3921-479e-8bb3-218f5bf64b84 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:17.686406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29d2b7-3921-479e-8bb3-218f5bf64b84 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/da10f4e86869489b8e0c624d9107f743/users/e182dc8f2ade4e66abc883423eba9040/roles/350f43da7bf940bc8c5462da7ba14fea` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:17.689036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29d2b7-3921-479e-8bb3-218f5bf64b84 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=da10f4e86869489b8e0c624d9107f743, user_id=e182dc8f2ade4e66abc883423eba9040, role_id=350f43da7bf940bc8c5462da7ba14fea)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:17.693450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29d2b7-3921-479e-8bb3-218f5bf64b84 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:17.701170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b29d2b7-3921-479e-8bb3-218f5bf64b84 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invalidating the token cache because role 350f43da7bf940bc8c5462da7ba14fea was removed from user e182dc8f2ade4e66abc883423eba9040 on domain da10f4e86869489b8e0c624d9107f743. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:17.701458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b29d2b7-3921-479e-8bb3-218f5bf64b84 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.702152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b29d2b7-3921-479e-8bb3-218f5bf64b84 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.721148 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2165/4368] 10.222.0.23 () {58 vars in 1464 bytes} [Sat Aug 20 21:09:17 2022] DELETE /identity/v3/domains/da10f4e86869489b8e0c624d9107f743/users/e182dc8f2ade4e66abc883423eba9040/roles/350f43da7bf940bc8c5462da7ba14fea => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:17.725343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:17.727106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:17.744841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467132f-0a85-43d0-b12d-38323d39ef42 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:17.803926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c33c9f-7c2f-4f5d-be30-2a57dbe8eb1f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:17.804608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c33c9f-7c2f-4f5d-be30-2a57dbe8eb1f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:17.804882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c33c9f-7c2f-4f5d-be30-2a57dbe8eb1f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:17.805149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c33c9f-7c2f-4f5d-be30-2a57dbe8eb1f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/roles/350f43da7bf940bc8c5462da7ba14fea` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:17.806550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c33c9f-7c2f-4f5d-be30-2a57dbe8eb1f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_role(role_id=350f43da7bf940bc8c5462da7ba14fea)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:17.808261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c33c9f-7c2f-4f5d-be30-2a57dbe8eb1f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:17.844176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-17c33c9f-7c2f-4f5d-be30-2a57dbe8eb1f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invalidating the token cache because role 350f43da7bf940bc8c5462da7ba14fea has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:17.844731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-17c33c9f-7c2f-4f5d-be30-2a57dbe8eb1f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.845263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-17c33c9f-7c2f-4f5d-be30-2a57dbe8eb1f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.847075 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2204/4369] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:17 2022] DELETE /identity/v3/roles/350f43da7bf940bc8c5462da7ba14fea => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:17.851537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29d2b7-3921-479e-8bb3-218f5bf64b84 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:17.852964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1b29d2b7-3921-479e-8bb3-218f5bf64b84 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:17.866389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29d2b7-3921-479e-8bb3-218f5bf64b84 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:17.926839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fcc81-df81-4dab-bd7a-10a9f363ac62 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:17.927920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fcc81-df81-4dab-bd7a-10a9f363ac62 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:17.928194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fcc81-df81-4dab-bd7a-10a9f363ac62 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:17.928499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fcc81-df81-4dab-bd7a-10a9f363ac62 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/da10f4e86869489b8e0c624d9107f743` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:17.929777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fcc81-df81-4dab-bd7a-10a9f363ac62 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=da10f4e86869489b8e0c624d9107f743)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:17.931375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fcc81-df81-4dab-bd7a-10a9f363ac62 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:17.968301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4d1fcc81-df81-4dab-bd7a-10a9f363ac62 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'da10f4e86869489b8e0c624d9107f743'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:17.970134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4d1fcc81-df81-4dab-bd7a-10a9f363ac62 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'da10f4e86869489b8e0c624d9107f743'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.040315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4d1fcc81-df81-4dab-bd7a-10a9f363ac62 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'da10f4e86869489b8e0c624d9107f743'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.043151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4d1fcc81-df81-4dab-bd7a-10a9f363ac62 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'da10f4e86869489b8e0c624d9107f743'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.050482 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2166/4370] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:09:17 2022] PATCH /identity/v3/domains/da10f4e86869489b8e0c624d9107f743 => generated 277 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:18.054633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c33c9f-7c2f-4f5d-be30-2a57dbe8eb1f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:18.056148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-17c33c9f-7c2f-4f5d-be30-2a57dbe8eb1f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:18.099809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c33c9f-7c2f-4f5d-be30-2a57dbe8eb1f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:18.228303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:18.228990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:18.229273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:18.229544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/da10f4e86869489b8e0c624d9107f743` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:18.253157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=da10f4e86869489b8e0c624d9107f743)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:18.254838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:18.301369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'da10f4e86869489b8e0c624d9107f743'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.314468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'da10f4e86869489b8e0c624d9107f743'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.349791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=6,keystone:SELECT=94 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:18.385303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da10f4e86869489b8e0c624d9107f743', 'request_id': 'req-f34df6c2-d177-4a01-92cb-68fee7d09344'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.399011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da10f4e86869489b8e0c624d9107f743', 'request_id': 'req-f34df6c2-d177-4a01-92cb-68fee7d09344'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.410895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da10f4e86869489b8e0c624d9107f743', 'request_id': 'req-f34df6c2-d177-4a01-92cb-68fee7d09344'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.426440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] The token cache is being invalidate because project da10f4e86869489b8e0c624d9107f743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:18.426680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.427613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.450602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'da10f4e86869489b8e0c624d9107f743', 'request_id': 'req-f34df6c2-d177-4a01-92cb-68fee7d09344'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.453140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'da10f4e86869489b8e0c624d9107f743', 'request_id': 'req-f34df6c2-d177-4a01-92cb-68fee7d09344'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.459799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f34df6c2-d177-4a01-92cb-68fee7d09344 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'da10f4e86869489b8e0c624d9107f743', 'request_id': 'req-f34df6c2-d177-4a01-92cb-68fee7d09344'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.468519 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2205/4371] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:18 2022] DELETE /identity/v3/domains/da10f4e86869489b8e0c624d9107f743 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:18.473259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fcc81-df81-4dab-bd7a-10a9f363ac62 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:18.474970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4d1fcc81-df81-4dab-bd7a-10a9f363ac62 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:18.490909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fcc81-df81-4dab-bd7a-10a9f363ac62 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:18.496585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f232824-d2ca-4f88-bc3c-188aef386bba None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:18.496972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f232824-d2ca-4f88-bc3c-188aef386bba None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:18.497367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f232824-d2ca-4f88-bc3c-188aef386bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:18.517679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2f232824-d2ca-4f88-bc3c-188aef386bba None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:18.518873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2f232824-d2ca-4f88-bc3c-188aef386bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:18.565025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70eef7-ac95-4c37-84f9-9793cc2493fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:18.566116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70eef7-ac95-4c37-84f9-9793cc2493fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:18.566423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70eef7-ac95-4c37-84f9-9793cc2493fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:18.566983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70eef7-ac95-4c37-84f9-9793cc2493fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/d652d6aa6952461dbec876c41cd7597e/groups/81995cde30094e8b8a29bd908c62be6d/roles/bc3dd8abd88644b98f8f91e0e095d8cc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:18.569845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70eef7-ac95-4c37-84f9-9793cc2493fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=d652d6aa6952461dbec876c41cd7597e, group_id=81995cde30094e8b8a29bd908c62be6d, role_id=bc3dd8abd88644b98f8f91e0e095d8cc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:18.572954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70eef7-ac95-4c37-84f9-9793cc2493fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:18.579581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4a70eef7-ac95-4c37-84f9-9793cc2493fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invalidating the token cache because role bc3dd8abd88644b98f8f91e0e095d8cc was removed from group 81995cde30094e8b8a29bd908c62be6d on domain d652d6aa6952461dbec876c41cd7597e. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:18.579868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4a70eef7-ac95-4c37-84f9-9793cc2493fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.580581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4a70eef7-ac95-4c37-84f9-9793cc2493fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.582815 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2206/4372] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:09:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:18.596313 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2167/4373] 10.222.0.23 () {58 vars in 1467 bytes} [Sat Aug 20 21:09:18 2022] DELETE /identity/v3/domains/d652d6aa6952461dbec876c41cd7597e/groups/81995cde30094e8b8a29bd908c62be6d/roles/bc3dd8abd88644b98f8f91e0e095d8cc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:18.601130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f232824-d2ca-4f88-bc3c-188aef386bba None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:18.602914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2f232824-d2ca-4f88-bc3c-188aef386bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:18.609388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70eef7-ac95-4c37-84f9-9793cc2493fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:18.611112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4a70eef7-ac95-4c37-84f9-9793cc2493fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:18.619544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f232824-d2ca-4f88-bc3c-188aef386bba None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:18.644681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:UPDATE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:18.685446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d0c31-f2dc-4359-af92-de3cf6abaeab tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:18.686240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d0c31-f2dc-4359-af92-de3cf6abaeab tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:18.686582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d0c31-f2dc-4359-af92-de3cf6abaeab tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:18.686924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d0c31-f2dc-4359-af92-de3cf6abaeab tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/groups/81995cde30094e8b8a29bd908c62be6d/users/e182dc8f2ade4e66abc883423eba9040` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:18.688678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d0c31-f2dc-4359-af92-de3cf6abaeab tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=81995cde30094e8b8a29bd908c62be6d, user_id=e182dc8f2ade4e66abc883423eba9040)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:18.690665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d0c31-f2dc-4359-af92-de3cf6abaeab tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:18.700653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b17d0c31-f2dc-4359-af92-de3cf6abaeab tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'e182dc8f2ade4e66abc883423eba9040'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:18.701707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70eef7-ac95-4c37-84f9-9793cc2493fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:18.707412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462db050-a647-42e6-8444-40a62b374be0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:18.708033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462db050-a647-42e6-8444-40a62b374be0 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:18.710736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462db050-a647-42e6-8444-40a62b374be0 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:18.710956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462db050-a647-42e6-8444-40a62b374be0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:18.713137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-462db050-a647-42e6-8444-40a62b374be0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:18.720582 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2207/4374] 10.222.0.23 () {58 vars in 1344 bytes} [Sat Aug 20 21:09:18 2022] DELETE /identity/v3/groups/81995cde30094e8b8a29bd908c62be6d/users/e182dc8f2ade4e66abc883423eba9040 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:18.723726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d0c31-f2dc-4359-af92-de3cf6abaeab tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:18.726343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b17d0c31-f2dc-4359-af92-de3cf6abaeab tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:18.769935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:18.797772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462db050-a647-42e6-8444-40a62b374be0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:18.799743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462db050-a647-42e6-8444-40a62b374be0 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:18.810235 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2168/4375] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:09:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:09:18.828612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d0c31-f2dc-4359-af92-de3cf6abaeab tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:18.838097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e6c9d-7b93-46d7-b4f5-3aa9faea6878 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:18.847890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e6c9d-7b93-46d7-b4f5-3aa9faea6878 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:18.848422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e6c9d-7b93-46d7-b4f5-3aa9faea6878 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:18.848564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e6c9d-7b93-46d7-b4f5-3aa9faea6878 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/groups/81995cde30094e8b8a29bd908c62be6d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:18.852644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e6c9d-7b93-46d7-b4f5-3aa9faea6878 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_group(group_id=81995cde30094e8b8a29bd908c62be6d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:18.853699 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-fd5e6c9d-7b93-46d7-b4f5-3aa9faea6878 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:18.853699 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:09:18.853699 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:09:18.858549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e6c9d-7b93-46d7-b4f5-3aa9faea6878 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:18.948475 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2208/4376] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:09:18 2022] DELETE /identity/v3/groups/81995cde30094e8b8a29bd908c62be6d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:18.953345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462db050-a647-42e6-8444-40a62b374be0 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:18.976435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462db050-a647-42e6-8444-40a62b374be0 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:19.117817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b4c90-709a-496b-b5d6-d931825b86f1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:19.118979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b4c90-709a-496b-b5d6-d931825b86f1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:19.119333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b4c90-709a-496b-b5d6-d931825b86f1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:19.120096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b4c90-709a-496b-b5d6-d931825b86f1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/roles/bc3dd8abd88644b98f8f91e0e095d8cc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:19.122489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b4c90-709a-496b-b5d6-d931825b86f1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_role(role_id=bc3dd8abd88644b98f8f91e0e095d8cc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:19.125146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b4c90-709a-496b-b5d6-d931825b86f1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:19.233994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-762b4c90-709a-496b-b5d6-d931825b86f1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invalidating the token cache because role bc3dd8abd88644b98f8f91e0e095d8cc has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:19.233994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-762b4c90-709a-496b-b5d6-d931825b86f1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.234384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-762b4c90-709a-496b-b5d6-d931825b86f1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.237159 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2169/4377] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:18 2022] DELETE /identity/v3/roles/bc3dd8abd88644b98f8f91e0e095d8cc => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:19.241509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e6c9d-7b93-46d7-b4f5-3aa9faea6878 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:19.243316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fd5e6c9d-7b93-46d7-b4f5-3aa9faea6878 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:19.282253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e6c9d-7b93-46d7-b4f5-3aa9faea6878 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:19.435370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcdfa5-b1fe-4c83-ad07-857115aa137b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:19.436131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcdfa5-b1fe-4c83-ad07-857115aa137b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:19.436941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcdfa5-b1fe-4c83-ad07-857115aa137b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:19.436941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcdfa5-b1fe-4c83-ad07-857115aa137b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/d652d6aa6952461dbec876c41cd7597e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:19.438732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcdfa5-b1fe-4c83-ad07-857115aa137b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=d652d6aa6952461dbec876c41cd7597e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:19.440542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcdfa5-b1fe-4c83-ad07-857115aa137b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:19.500694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aedcdfa5-b1fe-4c83-ad07-857115aa137b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'd652d6aa6952461dbec876c41cd7597e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.504254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aedcdfa5-b1fe-4c83-ad07-857115aa137b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'd652d6aa6952461dbec876c41cd7597e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.579964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aedcdfa5-b1fe-4c83-ad07-857115aa137b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'd652d6aa6952461dbec876c41cd7597e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.580755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aedcdfa5-b1fe-4c83-ad07-857115aa137b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'd652d6aa6952461dbec876c41cd7597e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.587521 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2209/4378] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:09:19 2022] PATCH /identity/v3/domains/d652d6aa6952461dbec876c41cd7597e => generated 277 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:09:19.588041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b4c90-709a-496b-b5d6-d931825b86f1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:19.606706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b4c90-709a-496b-b5d6-d931825b86f1 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:19.663024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:19.707701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:19.709319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:19.709720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:19.710571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/d652d6aa6952461dbec876c41cd7597e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:19.728778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=d652d6aa6952461dbec876c41cd7597e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:19.732861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:19.793793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'd652d6aa6952461dbec876c41cd7597e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.799852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'd652d6aa6952461dbec876c41cd7597e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.869247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd652d6aa6952461dbec876c41cd7597e', 'request_id': 'req-42e980a3-410b-441f-a138-a4d807dca3e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.876623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd652d6aa6952461dbec876c41cd7597e', 'request_id': 'req-42e980a3-410b-441f-a138-a4d807dca3e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.887450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd652d6aa6952461dbec876c41cd7597e', 'request_id': 'req-42e980a3-410b-441f-a138-a4d807dca3e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.902255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] The token cache is being invalidate because project d652d6aa6952461dbec876c41cd7597e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:19.902542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.903329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.912651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'd652d6aa6952461dbec876c41cd7597e', 'request_id': 'req-42e980a3-410b-441f-a138-a4d807dca3e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.913560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'd652d6aa6952461dbec876c41cd7597e', 'request_id': 'req-42e980a3-410b-441f-a138-a4d807dca3e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.917879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'd652d6aa6952461dbec876c41cd7597e', 'request_id': 'req-42e980a3-410b-441f-a138-a4d807dca3e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:19.931479 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2170/4379] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:19 2022] DELETE /identity/v3/domains/d652d6aa6952461dbec876c41cd7597e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:19.935863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcdfa5-b1fe-4c83-ad07-857115aa137b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:19.937686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aedcdfa5-b1fe-4c83-ad07-857115aa137b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:19.967233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcdfa5-b1fe-4c83-ad07-857115aa137b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:20.037316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:20.054892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238b1c5-ff16-4ba5-8ba4-d50109479068 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:20.055610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238b1c5-ff16-4ba5-8ba4-d50109479068 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:20.055947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238b1c5-ff16-4ba5-8ba4-d50109479068 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:20.056198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238b1c5-ff16-4ba5-8ba4-d50109479068 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/839183bb6e2c4e46bb498860d9e45a14/users/e182dc8f2ade4e66abc883423eba9040/roles/f0cbe7f8f9bf4cd0a2e8731f8f661dce` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:20.059365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238b1c5-ff16-4ba5-8ba4-d50109479068 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=839183bb6e2c4e46bb498860d9e45a14, user_id=e182dc8f2ade4e66abc883423eba9040, role_id=f0cbe7f8f9bf4cd0a2e8731f8f661dce)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:20.060804 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-1238b1c5-ff16-4ba5-8ba4-d50109479068 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:20.060804 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:09:20.060804 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:09:20.062968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238b1c5-ff16-4ba5-8ba4-d50109479068 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:20.071972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1238b1c5-ff16-4ba5-8ba4-d50109479068 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invalidating the token cache because role f0cbe7f8f9bf4cd0a2e8731f8f661dce was removed from user e182dc8f2ade4e66abc883423eba9040 on domain 839183bb6e2c4e46bb498860d9e45a14. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:20.072256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1238b1c5-ff16-4ba5-8ba4-d50109479068 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:20.073307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1238b1c5-ff16-4ba5-8ba4-d50109479068 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:20.101794 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2210/4380] 10.222.0.23 () {58 vars in 1464 bytes} [Sat Aug 20 21:09:19 2022] DELETE /identity/v3/domains/839183bb6e2c4e46bb498860d9e45a14/users/e182dc8f2ade4e66abc883423eba9040/roles/f0cbe7f8f9bf4cd0a2e8731f8f661dce => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:20.103326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:20.106640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:20.124616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e980a3-410b-441f-a138-a4d807dca3e6 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:20.222070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e029e-a81a-4e7d-b579-f14075ab1f39 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:20.229458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e029e-a81a-4e7d-b579-f14075ab1f39 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:20.234568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e029e-a81a-4e7d-b579-f14075ab1f39 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:20.235242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e029e-a81a-4e7d-b579-f14075ab1f39 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/roles/f0cbe7f8f9bf4cd0a2e8731f8f661dce` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:20.238556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e029e-a81a-4e7d-b579-f14075ab1f39 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f0cbe7f8f9bf4cd0a2e8731f8f661dce)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:20.241384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e029e-a81a-4e7d-b579-f14075ab1f39 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:20.265944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:20.333920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db9e029e-a81a-4e7d-b579-f14075ab1f39 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invalidating the token cache because role f0cbe7f8f9bf4cd0a2e8731f8f661dce has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:20.336404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db9e029e-a81a-4e7d-b579-f14075ab1f39 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:20.346811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db9e029e-a81a-4e7d-b579-f14075ab1f39 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:20.359203 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2171/4381] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:20 2022] DELETE /identity/v3/roles/f0cbe7f8f9bf4cd0a2e8731f8f661dce => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:20.360358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238b1c5-ff16-4ba5-8ba4-d50109479068 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:20.362833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1238b1c5-ff16-4ba5-8ba4-d50109479068 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:20.386822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238b1c5-ff16-4ba5-8ba4-d50109479068 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:20.510687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b0e79-a6eb-44b8-a2ae-63cd8c4a3b8b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:20.512093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b0e79-a6eb-44b8-a2ae-63cd8c4a3b8b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:20.512498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b0e79-a6eb-44b8-a2ae-63cd8c4a3b8b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:20.512864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b0e79-a6eb-44b8-a2ae-63cd8c4a3b8b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/839183bb6e2c4e46bb498860d9e45a14` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:20.514885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b0e79-a6eb-44b8-a2ae-63cd8c4a3b8b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=839183bb6e2c4e46bb498860d9e45a14)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:20.517650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b0e79-a6eb-44b8-a2ae-63cd8c4a3b8b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:20.550173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-856b0e79-a6eb-44b8-a2ae-63cd8c4a3b8b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '839183bb6e2c4e46bb498860d9e45a14'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:20.551190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-856b0e79-a6eb-44b8-a2ae-63cd8c4a3b8b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '839183bb6e2c4e46bb498860d9e45a14'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:20.590409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-856b0e79-a6eb-44b8-a2ae-63cd8c4a3b8b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '839183bb6e2c4e46bb498860d9e45a14'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:20.590692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-856b0e79-a6eb-44b8-a2ae-63cd8c4a3b8b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '839183bb6e2c4e46bb498860d9e45a14'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:20.594010 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2211/4382] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:09:20 2022] PATCH /identity/v3/domains/839183bb6e2c4e46bb498860d9e45a14 => generated 277 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:20.597998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e029e-a81a-4e7d-b579-f14075ab1f39 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:20.600393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-db9e029e-a81a-4e7d-b579-f14075ab1f39 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:20.631603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e029e-a81a-4e7d-b579-f14075ab1f39 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:20.768247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:20.769213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:20.769653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:20.770108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/839183bb6e2c4e46bb498860d9e45a14` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:20.800553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=839183bb6e2c4e46bb498860d9e45a14)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:20.803283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:20.853639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '839183bb6e2c4e46bb498860d9e45a14'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:20.858637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '839183bb6e2c4e46bb498860d9e45a14'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:20.927723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '839183bb6e2c4e46bb498860d9e45a14', 'request_id': 'req-f6cbd368-c58b-4969-a66a-bacc241d1dd3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:20.956328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '839183bb6e2c4e46bb498860d9e45a14', 'request_id': 'req-f6cbd368-c58b-4969-a66a-bacc241d1dd3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:20.973954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '839183bb6e2c4e46bb498860d9e45a14', 'request_id': 'req-f6cbd368-c58b-4969-a66a-bacc241d1dd3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:21.031538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] The token cache is being invalidate because project 839183bb6e2c4e46bb498860d9e45a14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:21.031827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:21.032679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:21.057912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '839183bb6e2c4e46bb498860d9e45a14', 'request_id': 'req-f6cbd368-c58b-4969-a66a-bacc241d1dd3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:21.059213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '839183bb6e2c4e46bb498860d9e45a14', 'request_id': 'req-f6cbd368-c58b-4969-a66a-bacc241d1dd3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:21.065239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '839183bb6e2c4e46bb498860d9e45a14', 'request_id': 'req-f6cbd368-c58b-4969-a66a-bacc241d1dd3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:21.073385 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2172/4383] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:20 2022] DELETE /identity/v3/domains/839183bb6e2c4e46bb498860d9e45a14 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:21.080780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b0e79-a6eb-44b8-a2ae-63cd8c4a3b8b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:21.084014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-856b0e79-a6eb-44b8-a2ae-63cd8c4a3b8b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:21.102937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b0e79-a6eb-44b8-a2ae-63cd8c4a3b8b tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:21.184714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6025e13-3e15-44dc-b621-e4465a28ff26 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:21.185425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6025e13-3e15-44dc-b621-e4465a28ff26 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:21.185700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6025e13-3e15-44dc-b621-e4465a28ff26 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:21.185968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6025e13-3e15-44dc-b621-e4465a28ff26 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:21.187249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6025e13-3e15-44dc-b621-e4465a28ff26 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:21.189023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6025e13-3e15-44dc-b621-e4465a28ff26 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:21.204728 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2212/4384] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:21.210008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:21.211580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:21.226191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cbd368-c58b-4969-a66a-bacc241d1dd3 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:21.232348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae1c8c-9f49-4e60-b28d-6f15680d95fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:21.233126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae1c8c-9f49-4e60-b28d-6f15680d95fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:21.233471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae1c8c-9f49-4e60-b28d-6f15680d95fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:21.233810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae1c8c-9f49-4e60-b28d-6f15680d95fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/projects/d64206fece414e70b231268263d8678a/users/39f931c959f34c3a967966ccd8d1533e/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:21.235890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae1c8c-9f49-4e60-b28d-6f15680d95fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_grant(project_id=d64206fece414e70b231268263d8678a, user_id=39f931c959f34c3a967966ccd8d1533e, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:21.238889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae1c8c-9f49-4e60-b28d-6f15680d95fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:21.259203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:21.270314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6025e13-3e15-44dc-b621-e4465a28ff26 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:21.271424 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2173/4385] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:21 2022] PUT /identity/v3/projects/d64206fece414e70b231268263d8678a/users/39f931c959f34c3a967966ccd8d1533e/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:21.285330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6025e13-3e15-44dc-b621-e4465a28ff26 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:21.389199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:UPDATE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:21.443948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf168e15-6baa-474e-a050-75d83791e8e8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:21.445161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf168e15-6baa-474e-a050-75d83791e8e8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:21.445850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf168e15-6baa-474e-a050-75d83791e8e8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:21.447080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf168e15-6baa-474e-a050-75d83791e8e8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/auth/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:21.447923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf168e15-6baa-474e-a050-75d83791e8e8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:get_auth_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:21.449246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf168e15-6baa-474e-a050-75d83791e8e8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:21.528689 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2213/4386] 10.222.0.23 () {58 vars in 1146 bytes} [Sat Aug 20 21:09:21 2022] GET /identity/v3/auth/projects => generated 828 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:21.534607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae1c8c-9f49-4e60-b28d-6f15680d95fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:21.550855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae1c8c-9f49-4e60-b28d-6f15680d95fc tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:21.556851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b281d1-1828-4749-ae08-7bba20f53b36 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:21.557835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b281d1-1828-4749-ae08-7bba20f53b36 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:21.557835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b281d1-1828-4749-ae08-7bba20f53b36 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:21.558029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b281d1-1828-4749-ae08-7bba20f53b36 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/projects/d64206fece414e70b231268263d8678a/users/39f931c959f34c3a967966ccd8d1533e/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:21.559968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b281d1-1828-4749-ae08-7bba20f53b36 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=d64206fece414e70b231268263d8678a, user_id=39f931c959f34c3a967966ccd8d1533e, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:21.564010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b281d1-1828-4749-ae08-7bba20f53b36 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:21.572107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e5b281d1-1828-4749-ae08-7bba20f53b36 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invalidating the token cache because role 1e7f1276e60442b0bb784f6e1d0a4647 was removed from user 39f931c959f34c3a967966ccd8d1533e on project d64206fece414e70b231268263d8678a. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:21.572296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e5b281d1-1828-4749-ae08-7bba20f53b36 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:21.573420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e5b281d1-1828-4749-ae08-7bba20f53b36 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:21.605539 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2174/4387] 10.222.0.23 () {58 vars in 1467 bytes} [Sat Aug 20 21:09:21 2022] DELETE /identity/v3/projects/d64206fece414e70b231268263d8678a/users/39f931c959f34c3a967966ccd8d1533e/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:21.606076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf168e15-6baa-474e-a050-75d83791e8e8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:21.607999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cf168e15-6baa-474e-a050-75d83791e8e8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:21.624597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf168e15-6baa-474e-a050-75d83791e8e8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:21.719762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2e199-57f7-47c4-ac4e-6d97b7909407 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:21.720409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2e199-57f7-47c4-ac4e-6d97b7909407 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:21.720695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2e199-57f7-47c4-ac4e-6d97b7909407 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:21.720965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2e199-57f7-47c4-ac4e-6d97b7909407 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:21.721601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2e199-57f7-47c4-ac4e-6d97b7909407 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:21.723722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2e199-57f7-47c4-ac4e-6d97b7909407 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:21.756070 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2214/4388] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:21 2022] POST /identity/v3/users => generated 321 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:21.760943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b281d1-1828-4749-ae08-7bba20f53b36 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:21.775674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b281d1-1828-4749-ae08-7bba20f53b36 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:21.782346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d57793-8c95-4507-a3b7-58433dabca97 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:21.783479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d57793-8c95-4507-a3b7-58433dabca97 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:21.783856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d57793-8c95-4507-a3b7-58433dabca97 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:21.784536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d57793-8c95-4507-a3b7-58433dabca97 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:21.785434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d57793-8c95-4507-a3b7-58433dabca97 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:21.787673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d57793-8c95-4507-a3b7-58433dabca97 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:21.829628 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2175/4389] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:21 2022] POST /identity/v3/projects => generated 363 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:21.833750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2e199-57f7-47c4-ac4e-6d97b7909407 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:21.851900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2e199-57f7-47c4-ac4e-6d97b7909407 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:21.957172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc2dd2-830d-439b-a565-8d39f26366c8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:21.957859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc2dd2-830d-439b-a565-8d39f26366c8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:21.958170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc2dd2-830d-439b-a565-8d39f26366c8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:21.958510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc2dd2-830d-439b-a565-8d39f26366c8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:21.959119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc2dd2-830d-439b-a565-8d39f26366c8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:21.961141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc2dd2-830d-439b-a565-8d39f26366c8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:22.015540 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2215/4390] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:21 2022] POST /identity/v3/projects => generated 362 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:22.019984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d57793-8c95-4507-a3b7-58433dabca97 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:22.035091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d57793-8c95-4507-a3b7-58433dabca97 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:22.111919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83613e81-817f-4aa3-8511-e3994118b54a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:22.112586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83613e81-817f-4aa3-8511-e3994118b54a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:22.112866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83613e81-817f-4aa3-8511-e3994118b54a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:22.113138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83613e81-817f-4aa3-8511-e3994118b54a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:22.113815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83613e81-817f-4aa3-8511-e3994118b54a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:22.123196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83613e81-817f-4aa3-8511-e3994118b54a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:22.154202 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2176/4391] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:09:22 2022] POST /identity/v3/roles => generated 240 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:22.155550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc2dd2-830d-439b-a565-8d39f26366c8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:22.177407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc2dd2-830d-439b-a565-8d39f26366c8 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:22.192869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86937251-665d-4389-acea-5922148f1ff4 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:22.193729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86937251-665d-4389-acea-5922148f1ff4 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:22.194020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86937251-665d-4389-acea-5922148f1ff4 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:22.194275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86937251-665d-4389-acea-5922148f1ff4 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/projects/a4f95a48a63e4a7fb9a7c9decb6da382/users/31458a56f52641ca8adbc984519f5962/roles/f34c6d2375ee4ffd9ff1b0c8a184533e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:22.229354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86937251-665d-4389-acea-5922148f1ff4 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_grant(project_id=a4f95a48a63e4a7fb9a7c9decb6da382, user_id=31458a56f52641ca8adbc984519f5962, role_id=f34c6d2375ee4ffd9ff1b0c8a184533e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:22.232264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86937251-665d-4389-acea-5922148f1ff4 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:22.235093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:22.263212 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2216/4392] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:22 2022] PUT /identity/v3/projects/a4f95a48a63e4a7fb9a7c9decb6da382/users/31458a56f52641ca8adbc984519f5962/roles/f34c6d2375ee4ffd9ff1b0c8a184533e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:22.294683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83613e81-817f-4aa3-8511-e3994118b54a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:22.307203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83613e81-817f-4aa3-8511-e3994118b54a tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:22.420437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca19d3-41b8-4ba9-b879-0157176c4011 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:22.420687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca19d3-41b8-4ba9-b879-0157176c4011 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:22.420779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca19d3-41b8-4ba9-b879-0157176c4011 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:22.421034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca19d3-41b8-4ba9-b879-0157176c4011 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/projects/0e50413838ff4fa3b3a4706ef8b8a457/users/31458a56f52641ca8adbc984519f5962/roles/f34c6d2375ee4ffd9ff1b0c8a184533e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:22.427782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca19d3-41b8-4ba9-b879-0157176c4011 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:create_grant(project_id=0e50413838ff4fa3b3a4706ef8b8a457, user_id=31458a56f52641ca8adbc984519f5962, role_id=f34c6d2375ee4ffd9ff1b0c8a184533e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:22.432566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca19d3-41b8-4ba9-b879-0157176c4011 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:22.461826 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2177/4393] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:22 2022] PUT /identity/v3/projects/0e50413838ff4fa3b3a4706ef8b8a457/users/31458a56f52641ca8adbc984519f5962/roles/f34c6d2375ee4ffd9ff1b0c8a184533e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:22.466311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105d6d6-e3da-4ead-b058-c6f15402a861 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:22.466661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105d6d6-e3da-4ead-b058-c6f15402a861 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:22.467002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105d6d6-e3da-4ead-b058-c6f15402a861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:22.542284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a105d6d6-e3da-4ead-b058-c6f15402a861 None None] MFA Rules not processed for user `31458a56f52641ca8adbc984519f5962`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:22.544119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a105d6d6-e3da-4ead-b058-c6f15402a861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:22.548000 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2217/4394] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:22 2022] POST /identity/v3/auth/tokens => generated 334 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:09:22.552350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831c509-1846-4b50-a99f-185f8fa53ffc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:22.552852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831c509-1846-4b50-a99f-185f8fa53ffc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:22.553276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831c509-1846-4b50-a99f-185f8fa53ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:22.573124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f831c509-1846-4b50-a99f-185f8fa53ffc None None] MFA Rules not processed for user `31458a56f52641ca8adbc984519f5962`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:22.574550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f831c509-1846-4b50-a99f-185f8fa53ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:22.652045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:22.675187 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2178/4395] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:22 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 406 bytes (2 switches on core 0) Aug 20 21:09:22.679321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105d6d6-e3da-4ead-b058-c6f15402a861 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:22.755176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105d6d6-e3da-4ead-b058-c6f15402a861 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:22.763611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b79eac-805c-4588-867f-b03724e35e51 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:22.763611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b79eac-805c-4588-867f-b03724e35e51 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:22.763611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b79eac-805c-4588-867f-b03724e35e51 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:22.764495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b79eac-805c-4588-867f-b03724e35e51 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:22.778392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b79eac-805c-4588-867f-b03724e35e51 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:revoke_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:22.781444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b79eac-805c-4588-867f-b03724e35e51 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:22.809561 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2218/4396] 10.222.0.23 () {56 vars in 1308 bytes} [Sat Aug 20 21:09:22 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:22.812459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075c489-c0a9-46fa-82e3-69636aae59e0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:22.814916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075c489-c0a9-46fa-82e3-69636aae59e0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:22.814916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075c489-c0a9-46fa-82e3-69636aae59e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:22.828366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e075c489-c0a9-46fa-82e3-69636aae59e0 None None] MFA Rules not processed for user `31458a56f52641ca8adbc984519f5962`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:22.829573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e075c489-c0a9-46fa-82e3-69636aae59e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:22.924932 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2179/4397] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:22 2022] POST /identity/v3/auth/tokens => generated 3223 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Aug 20 21:09:22.926504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b79eac-805c-4588-867f-b03724e35e51 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:22.937097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b79eac-805c-4588-867f-b03724e35e51 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:22.942087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63c7f9-92e7-4686-923a-450dd0fa148d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:22.942866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63c7f9-92e7-4686-923a-450dd0fa148d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:22.943226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63c7f9-92e7-4686-923a-450dd0fa148d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:22.943554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63c7f9-92e7-4686-923a-450dd0fa148d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/roles/f34c6d2375ee4ffd9ff1b0c8a184533e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:22.945101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63c7f9-92e7-4686-923a-450dd0fa148d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f34c6d2375ee4ffd9ff1b0c8a184533e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:22.947145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63c7f9-92e7-4686-923a-450dd0fa148d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:23.004951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5e63c7f9-92e7-4686-923a-450dd0fa148d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invalidating the token cache because role f34c6d2375ee4ffd9ff1b0c8a184533e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:23.005382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5e63c7f9-92e7-4686-923a-450dd0fa148d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.009115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5e63c7f9-92e7-4686-923a-450dd0fa148d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.014229 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2219/4398] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:22 2022] DELETE /identity/v3/roles/f34c6d2375ee4ffd9ff1b0c8a184533e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:23.017324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075c489-c0a9-46fa-82e3-69636aae59e0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:23.030761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075c489-c0a9-46fa-82e3-69636aae59e0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:23.117410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:23.117988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:23.118259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:23.118759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/projects/0e50413838ff4fa3b3a4706ef8b8a457` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:23.131385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0e50413838ff4fa3b3a4706ef8b8a457)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:23.131385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:23.261419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e50413838ff4fa3b3a4706ef8b8a457', 'request_id': 'req-8f47b575-afd7-41b6-acb4-a7463a0ca884'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.273438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e50413838ff4fa3b3a4706ef8b8a457', 'request_id': 'req-8f47b575-afd7-41b6-acb4-a7463a0ca884'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.326635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e50413838ff4fa3b3a4706ef8b8a457', 'request_id': 'req-8f47b575-afd7-41b6-acb4-a7463a0ca884'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.354078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] The token cache is being invalidate because project 0e50413838ff4fa3b3a4706ef8b8a457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:23.354315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.363605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.366878 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2180/4399] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:23 2022] DELETE /identity/v3/projects/0e50413838ff4fa3b3a4706ef8b8a457 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:23.370920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63c7f9-92e7-4686-923a-450dd0fa148d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:23.372394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5e63c7f9-92e7-4686-923a-450dd0fa148d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:23.387921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63c7f9-92e7-4686-923a-450dd0fa148d tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:23.461132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1c547-3c47-4948-ac71-28a93685d112 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:23.462115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1c547-3c47-4948-ac71-28a93685d112 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:23.462394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1c547-3c47-4948-ac71-28a93685d112 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:23.463547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1c547-3c47-4948-ac71-28a93685d112 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/projects/0e50413838ff4fa3b3a4706ef8b8a457` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:23.471013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1c547-3c47-4948-ac71-28a93685d112 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0e50413838ff4fa3b3a4706ef8b8a457)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:23.473026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1c547-3c47-4948-ac71-28a93685d112 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-fdb1c547-3c47-4948-ac71-28a93685d112 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Could not find project: 0e50413838ff4fa3b3a4706ef8b8a457.: keystone.exception.ProjectNotFound: Could not find project: 0e50413838ff4fa3b3a4706ef8b8a457. Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Aug 20 21:09:23.487310 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Aug 20 21:09:23.488407 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:23.488407 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:23.488407 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Aug 20 21:09:23.488407 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Aug 20 21:09:23.488407 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:09:23.488407 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:09:23.488407 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:09:23.488407 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:09:23.488407 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 0e50413838ff4fa3b3a4706ef8b8a457. Aug 20 21:09:23.488407 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:23.492707 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2220/4400] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:23 2022] DELETE /identity/v3/projects/0e50413838ff4fa3b3a4706ef8b8a457 => generated 113 bytes in 122 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:09:23.493235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:23.494683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:23.520274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47b575-afd7-41b6-acb4-a7463a0ca884 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:23.526371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:23.527025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:23.527298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:23.527565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/projects/a4f95a48a63e4a7fb9a7c9decb6da382` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:23.529115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a4f95a48a63e4a7fb9a7c9decb6da382)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:23.531146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:23.606230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:23.717184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f95a48a63e4a7fb9a7c9decb6da382', 'request_id': 'req-7381199c-573a-407f-b50d-2200a53bad5f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.731161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f95a48a63e4a7fb9a7c9decb6da382', 'request_id': 'req-7381199c-573a-407f-b50d-2200a53bad5f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.753094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f95a48a63e4a7fb9a7c9decb6da382', 'request_id': 'req-7381199c-573a-407f-b50d-2200a53bad5f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.767261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] The token cache is being invalidate because project a4f95a48a63e4a7fb9a7c9decb6da382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:23.767689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.768734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.770463 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2181/4401] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:23 2022] DELETE /identity/v3/projects/a4f95a48a63e4a7fb9a7c9decb6da382 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:23.774756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1c547-3c47-4948-ac71-28a93685d112 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:23.776713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fdb1c547-3c47-4948-ac71-28a93685d112 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:23.792365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1c547-3c47-4948-ac71-28a93685d112 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:23.813738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=2,keystone:SELECT=94,keystone:DELETE=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:23.871803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:23.872896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:23.873594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:23.874045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/users/31458a56f52641ca8adbc984519f5962` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:23.875415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorizing `identity:delete_user(user_id=31458a56f52641ca8adbc984519f5962)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:23.877749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:23.957678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31458a56f52641ca8adbc984519f5962'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.973207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31458a56f52641ca8adbc984519f5962'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:23.988337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31458a56f52641ca8adbc984519f5962'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:24.007988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31458a56f52641ca8adbc984519f5962'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:24.008760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31458a56f52641ca8adbc984519f5962'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:24.021345 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2221/4402] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:23 2022] DELETE /identity/v3/users/31458a56f52641ca8adbc984519f5962 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:24.022368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:24.040132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381199c-573a-407f-b50d-2200a53bad5f tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:24.114663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff6037-371b-40f2-9f8b-1dbd966b7376 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:24.115509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff6037-371b-40f2-9f8b-1dbd966b7376 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:24.115890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff6037-371b-40f2-9f8b-1dbd966b7376 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:24.116263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff6037-371b-40f2-9f8b-1dbd966b7376 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/011648f608d94448ae88cbac66227167` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:24.126442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff6037-371b-40f2-9f8b-1dbd966b7376 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-ffff6037-371b-40f2-9f8b-1dbd966b7376 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Could not find domain: 011648f608d94448ae88cbac66227167.: keystone.exception.DomainNotFound: Could not find domain: 011648f608d94448ae88cbac66227167. Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 011648f608d94448ae88cbac66227167. Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:24.132700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:24.134097 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 011648f608d94448ae88cbac66227167. Aug 20 21:09:24.135318 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:24.137994 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2182/4403] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:09:24 2022] PATCH /identity/v3/domains/011648f608d94448ae88cbac66227167 => generated 112 bytes in 116 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:09:24.138275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:24.139786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:24.157170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd6f65-ade1-4ee2-bb50-5721b08795be tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:24.163179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5822b5-5bcb-4bcb-9ae2-27d0fc920383 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:24.163838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5822b5-5bcb-4bcb-9ae2-27d0fc920383 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:24.164113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5822b5-5bcb-4bcb-9ae2-27d0fc920383 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:24.164467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5822b5-5bcb-4bcb-9ae2-27d0fc920383 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/da10f4e86869489b8e0c624d9107f743` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:24.172685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5822b5-5bcb-4bcb-9ae2-27d0fc920383 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-6e5822b5-5bcb-4bcb-9ae2-27d0fc920383 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Could not find domain: da10f4e86869489b8e0c624d9107f743.: keystone.exception.DomainNotFound: Could not find domain: da10f4e86869489b8e0c624d9107f743. Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: da10f4e86869489b8e0c624d9107f743. Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:24.178175 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:24.179422 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: da10f4e86869489b8e0c624d9107f743. Aug 20 21:09:24.184089 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:24.185810 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2222/4404] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:09:24 2022] PATCH /identity/v3/domains/da10f4e86869489b8e0c624d9107f743 => generated 112 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Aug 20 21:09:24.185912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff6037-371b-40f2-9f8b-1dbd966b7376 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:24.197150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff6037-371b-40f2-9f8b-1dbd966b7376 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:24.202456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364af7e-8c2c-415c-abe3-4883ec2dae05 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:24.203107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364af7e-8c2c-415c-abe3-4883ec2dae05 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:24.203367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364af7e-8c2c-415c-abe3-4883ec2dae05 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:24.203628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364af7e-8c2c-415c-abe3-4883ec2dae05 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/d652d6aa6952461dbec876c41cd7597e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:24.212928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364af7e-8c2c-415c-abe3-4883ec2dae05 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-0364af7e-8c2c-415c-abe3-4883ec2dae05 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Could not find domain: d652d6aa6952461dbec876c41cd7597e.: keystone.exception.DomainNotFound: Could not find domain: d652d6aa6952461dbec876c41cd7597e. Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d652d6aa6952461dbec876c41cd7597e. Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:24.230929 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:24.232296 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: d652d6aa6952461dbec876c41cd7597e. Aug 20 21:09:24.233468 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:24.244914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5822b5-5bcb-4bcb-9ae2-27d0fc920383 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:24.255270 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2183/4405] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:09:24 2022] PATCH /identity/v3/domains/d652d6aa6952461dbec876c41cd7597e => generated 112 bytes in 71 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Aug 20 21:09:24.258054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5822b5-5bcb-4bcb-9ae2-27d0fc920383 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:24.269929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2301d8-270b-494a-8ac3-9e67adc0edcf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39f931c959f34c3a967966ccd8d1533e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c1a313be3e14dfc903ee36e51514f0f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:24.270822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2301d8-270b-494a-8ac3-9e67adc0edcf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:24.271337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2301d8-270b-494a-8ac3-9e67adc0edcf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:24.271337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2301d8-270b-494a-8ac3-9e67adc0edcf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] PATH_INFO: `/v3/domains/839183bb6e2c4e46bb498860d9e45a14` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:24.282789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2301d8-270b-494a-8ac3-9e67adc0edcf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-ac2301d8-270b-494a-8ac3-9e67adc0edcf tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Could not find domain: 839183bb6e2c4e46bb498860d9e45a14.: keystone.exception.DomainNotFound: Could not find domain: 839183bb6e2c4e46bb498860d9e45a14. Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 839183bb6e2c4e46bb498860d9e45a14. Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:24.290557 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:24.292022 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 839183bb6e2c4e46bb498860d9e45a14. Aug 20 21:09:24.293219 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:24.296225 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2223/4406] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:09:24 2022] PATCH /identity/v3/domains/839183bb6e2c4e46bb498860d9e45a14 => generated 112 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Aug 20 21:09:24.300318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364af7e-8c2c-415c-abe3-4883ec2dae05 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:24.302224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0364af7e-8c2c-415c-abe3-4883ec2dae05 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:24.419509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364af7e-8c2c-415c-abe3-4883ec2dae05 tempest-TokensV3TestJSON-1068802847 tempest-TokensV3TestJSON-1068802847-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:24.434092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:24.434747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:24.435035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:24.435303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] PATH_INFO: `/v3/users/789f31a888294c8392e0ce14d6e92e68` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:24.437844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=789f31a888294c8392e0ce14d6e92e68)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:24.439846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:24.451581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:24.557615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '789f31a888294c8392e0ce14d6e92e68'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:24.573561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '789f31a888294c8392e0ce14d6e92e68'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:24.594719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '789f31a888294c8392e0ce14d6e92e68'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:24.596519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '789f31a888294c8392e0ce14d6e92e68'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:24.600905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '789f31a888294c8392e0ce14d6e92e68'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:24.622126 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2184/4407] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:24 2022] DELETE /identity/v3/users/789f31a888294c8392e0ce14d6e92e68 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:24.626645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68773967-c8b2-434b-b4fb-29632629b15d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:24.627165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68773967-c8b2-434b-b4fb-29632629b15d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:24.627294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68773967-c8b2-434b-b4fb-29632629b15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:24.689370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-68773967-c8b2-434b-b4fb-29632629b15d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:24.690723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-68773967-c8b2-434b-b4fb-29632629b15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:24.841775 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2224/4408] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:24.853949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:24.855479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:25.029956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948fb64-0a5c-4e60-a620-f094404f9070 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:25.034135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd90e0e-2f91-465c-80ec-e9d5de36fe3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:25.034892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd90e0e-2f91-465c-80ec-e9d5de36fe3a service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:25.035619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd90e0e-2f91-465c-80ec-e9d5de36fe3a service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:25.035984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd90e0e-2f91-465c-80ec-e9d5de36fe3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:25.051256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd90e0e-2f91-465c-80ec-e9d5de36fe3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:25.053211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd90e0e-2f91-465c-80ec-e9d5de36fe3a service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:25.065483 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2185/4409] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:25.568566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68773967-c8b2-434b-b4fb-29632629b15d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:25.570021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-68773967-c8b2-434b-b4fb-29632629b15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:25.584889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68773967-c8b2-434b-b4fb-29632629b15d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:25.591135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:25.591798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:25.592066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:25.592334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] PATH_INFO: `/v3/projects/d64206fece414e70b231268263d8678a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:25.593509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d64206fece414e70b231268263d8678a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:25.596210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:25.611362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:25.660592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd64206fece414e70b231268263d8678a', 'request_id': 'req-68ca0560-9921-4a53-a350-7d3c5abc21d8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:25.671189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd64206fece414e70b231268263d8678a', 'request_id': 'req-68ca0560-9921-4a53-a350-7d3c5abc21d8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:25.679461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd64206fece414e70b231268263d8678a', 'request_id': 'req-68ca0560-9921-4a53-a350-7d3c5abc21d8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:25.697239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] The token cache is being invalidate because project d64206fece414e70b231268263d8678a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:25.697621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:25.699232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:25.703313 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2225/4410] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:25 2022] DELETE /identity/v3/projects/d64206fece414e70b231268263d8678a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:25.704703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd90e0e-2f91-465c-80ec-e9d5de36fe3a service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:25.717667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd90e0e-2f91-465c-80ec-e9d5de36fe3a service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:25.803251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:25.822259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:25.823003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:25.823450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:25.823831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] PATH_INFO: `/v3/users/39f931c959f34c3a967966ccd8d1533e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:25.825133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=39f931c959f34c3a967966ccd8d1533e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:25.827503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:25.896331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39f931c959f34c3a967966ccd8d1533e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:25.905485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39f931c959f34c3a967966ccd8d1533e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:25.938785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39f931c959f34c3a967966ccd8d1533e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:25.938785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39f931c959f34c3a967966ccd8d1533e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:25.938785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39f931c959f34c3a967966ccd8d1533e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:25.954343 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2186/4411] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:25 2022] DELETE /identity/v3/users/39f931c959f34c3a967966ccd8d1533e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:26.588856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:26.590725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:26.619289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca0560-9921-4a53-a350-7d3c5abc21d8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:26.712066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:26.713094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:26.713242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:26.713492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] PATH_INFO: `/v3/projects/1c1a313be3e14dfc903ee36e51514f0f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:26.715315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c1a313be3e14dfc903ee36e51514f0f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:26.717326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:26.778683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c1a313be3e14dfc903ee36e51514f0f', 'request_id': 'req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:26.789687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c1a313be3e14dfc903ee36e51514f0f', 'request_id': 'req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:26.798465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c1a313be3e14dfc903ee36e51514f0f', 'request_id': 'req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:26.818104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] The token cache is being invalidate because project 1c1a313be3e14dfc903ee36e51514f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:26.818104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:26.818815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:26.822908 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2226/4412] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:26 2022] DELETE /identity/v3/projects/1c1a313be3e14dfc903ee36e51514f0f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:26.826910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:26.828523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:26.843916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162891b-8549-4108-b24a-6a6234a958eb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:26.949801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8f0e5-d5ef-4103-b81e-13555b80f25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:26.950557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8f0e5-d5ef-4103-b81e-13555b80f25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:26.950828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8f0e5-d5ef-4103-b81e-13555b80f25b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:26.951370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8f0e5-d5ef-4103-b81e-13555b80f25b admin admin] PATH_INFO: `/v3/users/e182dc8f2ade4e66abc883423eba9040` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:26.953099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8f0e5-d5ef-4103-b81e-13555b80f25b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e182dc8f2ade4e66abc883423eba9040)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:26.955204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8f0e5-d5ef-4103-b81e-13555b80f25b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:27.062186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-91d8f0e5-d5ef-4103-b81e-13555b80f25b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e182dc8f2ade4e66abc883423eba9040'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:27.078244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-91d8f0e5-d5ef-4103-b81e-13555b80f25b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e182dc8f2ade4e66abc883423eba9040'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:27.121747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-91d8f0e5-d5ef-4103-b81e-13555b80f25b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e182dc8f2ade4e66abc883423eba9040'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:27.123272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-91d8f0e5-d5ef-4103-b81e-13555b80f25b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e182dc8f2ade4e66abc883423eba9040'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:27.125824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-91d8f0e5-d5ef-4103-b81e-13555b80f25b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e182dc8f2ade4e66abc883423eba9040'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:27.131391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:27.157676 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2187/4413] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:26 2022] DELETE /identity/v3/users/e182dc8f2ade4e66abc883423eba9040 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:27.919121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:27.920675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:27.939037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f983b7-65d1-40b7-a236-8ef5c0c7f30b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:28.021918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:28.071627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec17bd6-916b-432c-a88b-318a6b18abfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:28.072440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec17bd6-916b-432c-a88b-318a6b18abfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:28.072936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec17bd6-916b-432c-a88b-318a6b18abfb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:28.073438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec17bd6-916b-432c-a88b-318a6b18abfb admin admin] PATH_INFO: `/v3/projects/4e6b7b46a6004ee483a4824756e61204` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:28.074993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec17bd6-916b-432c-a88b-318a6b18abfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e6b7b46a6004ee483a4824756e61204)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:28.077567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec17bd6-916b-432c-a88b-318a6b18abfb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:28.184356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ec17bd6-916b-432c-a88b-318a6b18abfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e6b7b46a6004ee483a4824756e61204', 'request_id': 'req-5ec17bd6-916b-432c-a88b-318a6b18abfb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:28.206832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ec17bd6-916b-432c-a88b-318a6b18abfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e6b7b46a6004ee483a4824756e61204', 'request_id': 'req-5ec17bd6-916b-432c-a88b-318a6b18abfb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:28.224892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ec17bd6-916b-432c-a88b-318a6b18abfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e6b7b46a6004ee483a4824756e61204', 'request_id': 'req-5ec17bd6-916b-432c-a88b-318a6b18abfb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:28.254338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ec17bd6-916b-432c-a88b-318a6b18abfb admin admin] The token cache is being invalidate because project 4e6b7b46a6004ee483a4824756e61204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:28.254695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ec17bd6-916b-432c-a88b-318a6b18abfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:28.255704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ec17bd6-916b-432c-a88b-318a6b18abfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:28.263182 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2227/4414] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:27 2022] DELETE /identity/v3/projects/4e6b7b46a6004ee483a4824756e61204 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:28.264249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdb7b0-e998-4268-83d2-39dbfc440342 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:28.264900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdb7b0-e998-4268-83d2-39dbfc440342 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:28.265055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdb7b0-e998-4268-83d2-39dbfc440342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:28.319800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-03cdb7b0-e998-4268-83d2-39dbfc440342 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:28.324049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-03cdb7b0-e998-4268-83d2-39dbfc440342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:28.453900 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2188/4415] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:28.461504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27327991-d8d8-4fe3-a0ec-6aa7c2263e5c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:28.461978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27327991-d8d8-4fe3-a0ec-6aa7c2263e5c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:28.462258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27327991-d8d8-4fe3-a0ec-6aa7c2263e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:28.499169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-27327991-d8d8-4fe3-a0ec-6aa7c2263e5c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:28.501256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-27327991-d8d8-4fe3-a0ec-6aa7c2263e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:28.513647 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2228/4416] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:28.517034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdb7b0-e998-4268-83d2-39dbfc440342 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:28.532734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdb7b0-e998-4268-83d2-39dbfc440342 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:28.553334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7761e-127f-42c8-ba7f-cfbc2648268d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:28.554258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7761e-127f-42c8-ba7f-cfbc2648268d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:28.554724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7761e-127f-42c8-ba7f-cfbc2648268d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:28.555024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7761e-127f-42c8-ba7f-cfbc2648268d admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:28.555696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7761e-127f-42c8-ba7f-cfbc2648268d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:28.556069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7761e-127f-42c8-ba7f-cfbc2648268d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:28.557556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7761e-127f-42c8-ba7f-cfbc2648268d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:28.583183 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2189/4417] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:28.588056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27327991-d8d8-4fe3-a0ec-6aa7c2263e5c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:28.631246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27327991-d8d8-4fe3-a0ec-6aa7c2263e5c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:28.667046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9560fd-1d43-4e04-bdd1-f052cf842a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:28.671368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9560fd-1d43-4e04-bdd1-f052cf842a04 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:28.672820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9560fd-1d43-4e04-bdd1-f052cf842a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:28.680390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9560fd-1d43-4e04-bdd1-f052cf842a04 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:28.681096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9560fd-1d43-4e04-bdd1-f052cf842a04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:28.690250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9560fd-1d43-4e04-bdd1-f052cf842a04 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:28.770556 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2229/4418] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:28 2022] POST /identity/v3/projects => generated 367 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:28.774947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7761e-127f-42c8-ba7f-cfbc2648268d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:28.790802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7761e-127f-42c8-ba7f-cfbc2648268d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:28.896520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537d409-0017-475d-9b95-3b8b33826836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:28.897277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537d409-0017-475d-9b95-3b8b33826836 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:28.897643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537d409-0017-475d-9b95-3b8b33826836 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:28.898000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537d409-0017-475d-9b95-3b8b33826836 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:28.898996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537d409-0017-475d-9b95-3b8b33826836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:28.901494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537d409-0017-475d-9b95-3b8b33826836 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:28.947913 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2190/4419] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:28 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:28.968232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9560fd-1d43-4e04-bdd1-f052cf842a04 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:28.990590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9560fd-1d43-4e04-bdd1-f052cf842a04 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:29.004620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b2122-5db8-469a-b09d-d82c13a87430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:29.005466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b2122-5db8-469a-b09d-d82c13a87430 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:29.005799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b2122-5db8-469a-b09d-d82c13a87430 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:29.006630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b2122-5db8-469a-b09d-d82c13a87430 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:29.007363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b2122-5db8-469a-b09d-d82c13a87430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:29.011271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b2122-5db8-469a-b09d-d82c13a87430 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:29.036922 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2230/4420] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:29.039774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537d409-0017-475d-9b95-3b8b33826836 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:29.058062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537d409-0017-475d-9b95-3b8b33826836 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:29.065181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a8f12-b495-4868-bea8-eb6c082589fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:29.066407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a8f12-b495-4868-bea8-eb6c082589fe admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:29.066684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a8f12-b495-4868-bea8-eb6c082589fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:29.066948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a8f12-b495-4868-bea8-eb6c082589fe admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:29.067665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a8f12-b495-4868-bea8-eb6c082589fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:29.071575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a8f12-b495-4868-bea8-eb6c082589fe admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:29.088424 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2191/4421] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:29.093210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b2122-5db8-469a-b09d-d82c13a87430 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:29.105886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b2122-5db8-469a-b09d-d82c13a87430 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:29.113207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9755e82-b294-437b-9155-f61da967aa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:29.114261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9755e82-b294-437b-9155-f61da967aa0d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:29.114450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9755e82-b294-437b-9155-f61da967aa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:29.114826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9755e82-b294-437b-9155-f61da967aa0d admin admin] PATH_INFO: `/v3/projects/78f31458e1d841108f753773b78e1a14/users/4b7bcade95e34f8d93ee81c3198c98ba/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:29.138761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9755e82-b294-437b-9155-f61da967aa0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f31458e1d841108f753773b78e1a14, user_id=4b7bcade95e34f8d93ee81c3198c98ba, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:29.142237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9755e82-b294-437b-9155-f61da967aa0d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:29.166694 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2231/4422] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:29 2022] PUT /identity/v3/projects/78f31458e1d841108f753773b78e1a14/users/4b7bcade95e34f8d93ee81c3198c98ba/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:29.177055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdd72f-d236-4691-ab58-af826e13c19d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:29.177484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdd72f-d236-4691-ab58-af826e13c19d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:29.177750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdd72f-d236-4691-ab58-af826e13c19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:29.231565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-01cdd72f-d236-4691-ab58-af826e13c19d None None] MFA Rules not processed for user `4b7bcade95e34f8d93ee81c3198c98ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:29.233194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-01cdd72f-d236-4691-ab58-af826e13c19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:29.340983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:29.429440 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2192/4423] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:29.435516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9755e82-b294-437b-9155-f61da967aa0d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:29.574165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:29.584626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9755e82-b294-437b-9155-f61da967aa0d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:29.602913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ab593-8be8-46ca-83ad-58649c87c703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:29.603325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ab593-8be8-46ca-83ad-58649c87c703 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:29.603675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ab593-8be8-46ca-83ad-58649c87c703 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:29.604052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ab593-8be8-46ca-83ad-58649c87c703 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:29.604836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ab593-8be8-46ca-83ad-58649c87c703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:29.607122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ab593-8be8-46ca-83ad-58649c87c703 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:29.666300 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2232/4424] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:29 2022] POST /identity/v3/projects => generated 367 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:29.667844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdd72f-d236-4691-ab58-af826e13c19d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:29.687385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdd72f-d236-4691-ab58-af826e13c19d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:29.777051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5ecea-c1fc-4422-a3ab-73c77a3f8a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:29.777762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5ecea-c1fc-4422-a3ab-73c77a3f8a32 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:29.778130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5ecea-c1fc-4422-a3ab-73c77a3f8a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:29.778517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5ecea-c1fc-4422-a3ab-73c77a3f8a32 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:29.780017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5ecea-c1fc-4422-a3ab-73c77a3f8a32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:29.782145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5ecea-c1fc-4422-a3ab-73c77a3f8a32 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:29.824992 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2193/4425] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:29 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:29.825502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ab593-8be8-46ca-83ad-58649c87c703 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:29.838540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ab593-8be8-46ca-83ad-58649c87c703 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:29.848503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1a46c-5802-4a19-a661-240887e3c4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:29.849615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1a46c-5802-4a19-a661-240887e3c4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:29.850079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1a46c-5802-4a19-a661-240887e3c4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:29.850575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1a46c-5802-4a19-a661-240887e3c4a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:29.852950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1a46c-5802-4a19-a661-240887e3c4a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:29.853363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1a46c-5802-4a19-a661-240887e3c4a3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:29.887168 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2233/4426] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:29.891090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5ecea-c1fc-4422-a3ab-73c77a3f8a32 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:29.907729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5ecea-c1fc-4422-a3ab-73c77a3f8a32 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:29.919365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d32e0-4fdb-4f0c-99c3-b39f4fdbabe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:29.920327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d32e0-4fdb-4f0c-99c3-b39f4fdbabe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:29.920499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d32e0-4fdb-4f0c-99c3-b39f4fdbabe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:29.920675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d32e0-4fdb-4f0c-99c3-b39f4fdbabe4 admin admin] PATH_INFO: `/v3/projects/64a9a01c8a234babb6bb1e93e4f70ad9/users/9512f567658e4cbdb0e8830178105a37/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:29.944272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d32e0-4fdb-4f0c-99c3-b39f4fdbabe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a9a01c8a234babb6bb1e93e4f70ad9, user_id=9512f567658e4cbdb0e8830178105a37, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:29.947824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d32e0-4fdb-4f0c-99c3-b39f4fdbabe4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:29.993345 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2194/4427] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:29 2022] PUT /identity/v3/projects/64a9a01c8a234babb6bb1e93e4f70ad9/users/9512f567658e4cbdb0e8830178105a37/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:29.995123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1a46c-5802-4a19-a661-240887e3c4a3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:30.024277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1a46c-5802-4a19-a661-240887e3c4a3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:30.200678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed30ba2-531b-4850-b219-131fc87032c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:30.201300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed30ba2-531b-4850-b219-131fc87032c3 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:30.201576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed30ba2-531b-4850-b219-131fc87032c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:30.201843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed30ba2-531b-4850-b219-131fc87032c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:30.202499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed30ba2-531b-4850-b219-131fc87032c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:30.204111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed30ba2-531b-4850-b219-131fc87032c3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:30.230626 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2234/4428] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:30.235359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d32e0-4fdb-4f0c-99c3-b39f4fdbabe4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:30.257687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d32e0-4fdb-4f0c-99c3-b39f4fdbabe4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:30.271497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7062ae-d004-451f-8d97-b0d12f6e066f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:30.272187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7062ae-d004-451f-8d97-b0d12f6e066f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:30.272509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7062ae-d004-451f-8d97-b0d12f6e066f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:30.272779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7062ae-d004-451f-8d97-b0d12f6e066f admin admin] PATH_INFO: `/v3/projects/64a9a01c8a234babb6bb1e93e4f70ad9/users/9512f567658e4cbdb0e8830178105a37/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:30.281353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7062ae-d004-451f-8d97-b0d12f6e066f admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a9a01c8a234babb6bb1e93e4f70ad9, user_id=9512f567658e4cbdb0e8830178105a37, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:30.284051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7062ae-d004-451f-8d97-b0d12f6e066f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:30.332325 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2195/4429] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:30 2022] PUT /identity/v3/projects/64a9a01c8a234babb6bb1e93e4f70ad9/users/9512f567658e4cbdb0e8830178105a37/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:30.346325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bebf3-0e69-40d3-9f5c-af3cf1ebfd21 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:30.346700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bebf3-0e69-40d3-9f5c-af3cf1ebfd21 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:30.346953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bebf3-0e69-40d3-9f5c-af3cf1ebfd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:30.404304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1c8bebf3-0e69-40d3-9f5c-af3cf1ebfd21 None None] MFA Rules not processed for user `9512f567658e4cbdb0e8830178105a37`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:30.405587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1c8bebf3-0e69-40d3-9f5c-af3cf1ebfd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:30.565962 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2235/4430] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:30.571131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7062ae-d004-451f-8d97-b0d12f6e066f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:30.601937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7062ae-d004-451f-8d97-b0d12f6e066f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:30.614470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40aca81-50ef-46c4-bb06-0ee103e25aeb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:30.615570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aca81-50ef-46c4-bb06-0ee103e25aeb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:30.616069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aca81-50ef-46c4-bb06-0ee103e25aeb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:30.616643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aca81-50ef-46c4-bb06-0ee103e25aeb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:30.618996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40aca81-50ef-46c4-bb06-0ee103e25aeb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:30.626423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40aca81-50ef-46c4-bb06-0ee103e25aeb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:30.786681 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2196/4431] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:30 2022] POST /identity/v3/projects => generated 359 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:30.791571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bebf3-0e69-40d3-9f5c-af3cf1ebfd21 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:30.828292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bebf3-0e69-40d3-9f5c-af3cf1ebfd21 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:30.968720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:30.991162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9161e1a-78a6-4000-911d-6539ad89f377 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:30.991873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161e1a-78a6-4000-911d-6539ad89f377 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:30.992151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161e1a-78a6-4000-911d-6539ad89f377 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:30.992638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161e1a-78a6-4000-911d-6539ad89f377 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:30.993819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9161e1a-78a6-4000-911d-6539ad89f377 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:30.997588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9161e1a-78a6-4000-911d-6539ad89f377 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:31.046104 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2236/4432] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:30 2022] POST /identity/v3/users => generated 371 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:31.047658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40aca81-50ef-46c4-bb06-0ee103e25aeb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:31.062397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40aca81-50ef-46c4-bb06-0ee103e25aeb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:31.068213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59ee0d-4076-47a4-b4c3-7244cd709d77 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:31.068857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59ee0d-4076-47a4-b4c3-7244cd709d77 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:31.069135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59ee0d-4076-47a4-b4c3-7244cd709d77 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:31.069399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59ee0d-4076-47a4-b4c3-7244cd709d77 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/7db2de0770ac4091b3c9a7e88b5ff62a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:31.086838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59ee0d-4076-47a4-b4c3-7244cd709d77 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:get_user(user_id=7db2de0770ac4091b3c9a7e88b5ff62a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:31.089139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59ee0d-4076-47a4-b4c3-7244cd709d77 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:31.094093 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2197/4433] 10.222.0.23 () {58 vars in 1221 bytes} [Sat Aug 20 21:09:31 2022] GET /identity/v3/users/7db2de0770ac4091b3c9a7e88b5ff62a => generated 371 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:31.097088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9161e1a-78a6-4000-911d-6539ad89f377 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:31.109983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9161e1a-78a6-4000-911d-6539ad89f377 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:31.116045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:31.117580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:31.117994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:31.118265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/7db2de0770ac4091b3c9a7e88b5ff62a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:31.119648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:delete_user(user_id=7db2de0770ac4091b3c9a7e88b5ff62a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:31.121807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:31.174012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59ee0d-4076-47a4-b4c3-7244cd709d77 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:31.176503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-be59ee0d-4076-47a4-b4c3-7244cd709d77 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:31.211709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db2de0770ac4091b3c9a7e88b5ff62a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.231136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db2de0770ac4091b3c9a7e88b5ff62a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.248850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db2de0770ac4091b3c9a7e88b5ff62a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.280165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db2de0770ac4091b3c9a7e88b5ff62a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.281698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db2de0770ac4091b3c9a7e88b5ff62a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.293346 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2237/4434] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:31 2022] DELETE /identity/v3/users/7db2de0770ac4091b3c9a7e88b5ff62a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:31.297570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:31.300170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:31.304174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:31.327076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59ee0d-4076-47a4-b4c3-7244cd709d77 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:31.401836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:31.402480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:31.402818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:31.403081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] PATH_INFO: `/v3/users/6c8268c69e134211b81ac3fd4c030dff` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:31.404138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c8268c69e134211b81ac3fd4c030dff)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:31.406376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:31.444645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c35759-f0cf-4824-b37b-4ae4ca6f41be tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:31.450699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:31.451344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:31.453171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:31.453444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects/20b13c31b26b4935be3f50607f44a826` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:31.454778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:delete_project(project_id=20b13c31b26b4935be3f50607f44a826)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:31.457035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:31.480502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c8268c69e134211b81ac3fd4c030dff'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.495992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c8268c69e134211b81ac3fd4c030dff'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.514756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c8268c69e134211b81ac3fd4c030dff'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.516337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c8268c69e134211b81ac3fd4c030dff'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.519291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c8268c69e134211b81ac3fd4c030dff'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.523721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20b13c31b26b4935be3f50607f44a826', 'request_id': 'req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.528046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=80,keystone:INSERT=5,keystone:DELETE=15 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:31.537561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20b13c31b26b4935be3f50607f44a826', 'request_id': 'req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.556302 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2198/4435] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:31 2022] DELETE /identity/v3/users/6c8268c69e134211b81ac3fd4c030dff => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:31.565684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20b13c31b26b4935be3f50607f44a826', 'request_id': 'req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.603682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] The token cache is being invalidate because project 20b13c31b26b4935be3f50607f44a826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:31.603682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.606891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:31.608969 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2238/4436] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:31 2022] DELETE /identity/v3/projects/20b13c31b26b4935be3f50607f44a826 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:31.617756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:31.619451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:31.711876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144b2d2-6061-436a-ad17-182e69dd2837 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:31.717417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb1439-fa1e-4453-a843-caad1a348625 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:31.718038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb1439-fa1e-4453-a843-caad1a348625 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:31.718309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb1439-fa1e-4453-a843-caad1a348625 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:31.718572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb1439-fa1e-4453-a843-caad1a348625 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:31.719190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb1439-fa1e-4453-a843-caad1a348625 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:31.721153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb1439-fa1e-4453-a843-caad1a348625 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:31.764214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:31.764989 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2199/4437] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:31 2022] POST /identity/v3/projects => generated 359 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:31.766201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:31.768809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb1439-fa1e-4453-a843-caad1a348625 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:31.771199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-22fb1439-fa1e-4453-a843-caad1a348625 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:31.779071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:31.878398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb1439-fa1e-4453-a843-caad1a348625 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:31.883092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee4fb1-97ef-47a2-a4e4-08ce0b622e2d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:31.883266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59beaa-64a9-4b54-af46-a3614adf5d2c tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:31.883973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59beaa-64a9-4b54-af46-a3614adf5d2c tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:31.884358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59beaa-64a9-4b54-af46-a3614adf5d2c tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:31.884691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59beaa-64a9-4b54-af46-a3614adf5d2c tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:31.885506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59beaa-64a9-4b54-af46-a3614adf5d2c tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:31.888351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59beaa-64a9-4b54-af46-a3614adf5d2c tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:31.956708 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2200/4438] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:31 2022] POST /identity/v3/users => generated 417 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:31.958768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59beaa-64a9-4b54-af46-a3614adf5d2c tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:31.969318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:31.970333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:31.970791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:31.971143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3 admin admin] PATH_INFO: `/v3/projects/bd4e1f3de3af4dba8cc95846592e21cb` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:31.972749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd4e1f3de3af4dba8cc95846592e21cb)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:31.975335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:31.979915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59beaa-64a9-4b54-af46-a3614adf5d2c tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:31.996772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c76af-bacd-41bf-b244-4b642f78349b tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:32.012624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c76af-bacd-41bf-b244-4b642f78349b tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:32.012624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c76af-bacd-41bf-b244-4b642f78349b tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:32.012624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c76af-bacd-41bf-b244-4b642f78349b tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:32.012624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c76af-bacd-41bf-b244-4b642f78349b tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:32.014941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c76af-bacd-41bf-b244-4b642f78349b tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:32.060915 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2201/4439] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:09:31 2022] POST /identity/v3/roles => generated 240 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:32.062178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c76af-bacd-41bf-b244-4b642f78349b tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:32.083109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c76af-bacd-41bf-b244-4b642f78349b tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:32.112938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd4ad3-a759-4187-966b-ad68074e7449 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:32.113993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd4ad3-a759-4187-966b-ad68074e7449 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:32.114677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd4ad3-a759-4187-966b-ad68074e7449 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:32.115206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd4ad3-a759-4187-966b-ad68074e7449 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/5027ccbfd8f54ee6b639e51aa1bbc76d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:32.149714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd4ad3-a759-4187-966b-ad68074e7449 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:get_user(user_id=5027ccbfd8f54ee6b639e51aa1bbc76d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:32.152519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd4ad3-a759-4187-966b-ad68074e7449 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:32.153078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd4e1f3de3af4dba8cc95846592e21cb', 'request_id': 'req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:32.154950 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2202/4440] 10.222.0.23 () {58 vars in 1221 bytes} [Sat Aug 20 21:09:32 2022] GET /identity/v3/users/5027ccbfd8f54ee6b639e51aa1bbc76d => generated 417 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:09:32.177141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd4ad3-a759-4187-966b-ad68074e7449 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:32.177675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd4e1f3de3af4dba8cc95846592e21cb', 'request_id': 'req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:32.177675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd4e1f3de3af4dba8cc95846592e21cb', 'request_id': 'req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:32.196150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3 admin admin] The token cache is being invalidate because project bd4e1f3de3af4dba8cc95846592e21cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:32.196744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:32.206649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a747bb5c-ed08-44e2-b5fc-52ed3271afd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:32.206649 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2239/4441] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:31 2022] DELETE /identity/v3/projects/bd4e1f3de3af4dba8cc95846592e21cb => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:32.211119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3b463-5f0b-44cf-9721-7af8e794a630 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:32.211389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3b463-5f0b-44cf-9721-7af8e794a630 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:32.211663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3b463-5f0b-44cf-9721-7af8e794a630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:32.216419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd4ad3-a759-4187-966b-ad68074e7449 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:32.294060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-41a3b463-5f0b-44cf-9721-7af8e794a630 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:32.300000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-41a3b463-5f0b-44cf-9721-7af8e794a630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:32.333283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:32.390206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c48ba-3c38-4ae1-954f-4c3efbf57fd3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:32.391470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c48ba-3c38-4ae1-954f-4c3efbf57fd3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:32.391990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c48ba-3c38-4ae1-954f-4c3efbf57fd3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:32.393087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c48ba-3c38-4ae1-954f-4c3efbf57fd3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/roles/1c38be71ee764c4389654c47d28209ee` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:32.395646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c48ba-3c38-4ae1-954f-4c3efbf57fd3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:get_role(role_id=1c38be71ee764c4389654c47d28209ee)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:32.397034 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-709c48ba-3c38-4ae1-954f-4c3efbf57fd3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:32.397034 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:09:32.397034 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:09:32.398851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c48ba-3c38-4ae1-954f-4c3efbf57fd3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:32.406183 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2203/4442] 10.222.0.23 () {58 vars in 1221 bytes} [Sat Aug 20 21:09:32 2022] GET /identity/v3/roles/1c38be71ee764c4389654c47d28209ee => generated 240 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:32.407515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c48ba-3c38-4ae1-954f-4c3efbf57fd3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:32.409984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-709c48ba-3c38-4ae1-954f-4c3efbf57fd3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:32.427602 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2240/4443] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:32.428177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c48ba-3c38-4ae1-954f-4c3efbf57fd3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:32.434151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ff9b0-f448-400b-935b-a6bf4d602536 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:32.434769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ff9b0-f448-400b-935b-a6bf4d602536 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:32.435045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ff9b0-f448-400b-935b-a6bf4d602536 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:32.436292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1837d-9ad0-4f68-b256-c442503f8867 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:32.436719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1837d-9ad0-4f68-b256-c442503f8867 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:32.437054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1837d-9ad0-4f68-b256-c442503f8867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:32.446736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ff9b0-f448-400b-935b-a6bf4d602536 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:32.450195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ff9b0-f448-400b-935b-a6bf4d602536 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:32.460293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7ad1837d-9ad0-4f68-b256-c442503f8867 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:32.460909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ff9b0-f448-400b-935b-a6bf4d602536 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:32.462055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7ad1837d-9ad0-4f68-b256-c442503f8867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:32.474835 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2241/4444] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:32.490640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1837d-9ad0-4f68-b256-c442503f8867 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:32.528406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1837d-9ad0-4f68-b256-c442503f8867 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:32.531821 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2204/4445] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:32 2022] POST /identity/v3/projects => generated 358 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:32.536529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ff9b0-f448-400b-935b-a6bf4d602536 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:32.648538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49397536-ba48-48cc-bb56-b0a7711b5b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:32.649367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49397536-ba48-48cc-bb56-b0a7711b5b71 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:32.649643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49397536-ba48-48cc-bb56-b0a7711b5b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:32.649908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49397536-ba48-48cc-bb56-b0a7711b5b71 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:32.650688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49397536-ba48-48cc-bb56-b0a7711b5b71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:32.650991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49397536-ba48-48cc-bb56-b0a7711b5b71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:32.657064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49397536-ba48-48cc-bb56-b0a7711b5b71 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:32.665856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:32.687965 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2242/4446] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:32.694012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49397536-ba48-48cc-bb56-b0a7711b5b71 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:32.704219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ff9b0-f448-400b-935b-a6bf4d602536 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:32.708333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49397536-ba48-48cc-bb56-b0a7711b5b71 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:32.712740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3cb40-8c74-409c-b793-0754c42fd6f2 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:32.713407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3cb40-8c74-409c-b793-0754c42fd6f2 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:32.713681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3cb40-8c74-409c-b793-0754c42fd6f2 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:32.713948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3cb40-8c74-409c-b793-0754c42fd6f2 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects/3faddd3d88c94a3eb3c26e1ca4fee53a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:32.714791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e87be-0e45-44b5-9b03-08a9c7e57b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:32.715100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3cb40-8c74-409c-b793-0754c42fd6f2 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:get_project(project_id=3faddd3d88c94a3eb3c26e1ca4fee53a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:32.715399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e87be-0e45-44b5-9b03-08a9c7e57b93 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:32.715668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e87be-0e45-44b5-9b03-08a9c7e57b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:32.715932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e87be-0e45-44b5-9b03-08a9c7e57b93 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:32.717181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e87be-0e45-44b5-9b03-08a9c7e57b93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:32.717669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3cb40-8c74-409c-b793-0754c42fd6f2 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:32.719223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e87be-0e45-44b5-9b03-08a9c7e57b93 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:32.722539 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2205/4447] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:32 2022] GET /identity/v3/projects/3faddd3d88c94a3eb3c26e1ca4fee53a => generated 358 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:32.725612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3cb40-8c74-409c-b793-0754c42fd6f2 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:32.745573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3cb40-8c74-409c-b793-0754c42fd6f2 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:32.757976 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2243/4448] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:32 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:32.759368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8e95c-30f9-43cb-8941-5104861ace49 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:32.760709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e87be-0e45-44b5-9b03-08a9c7e57b93 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:32.761765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8e95c-30f9-43cb-8941-5104861ace49 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:32.763002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8e95c-30f9-43cb-8941-5104861ace49 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:32.763522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8e95c-30f9-43cb-8941-5104861ace49 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects/3faddd3d88c94a3eb3c26e1ca4fee53a/users/5027ccbfd8f54ee6b639e51aa1bbc76d/roles/1c38be71ee764c4389654c47d28209ee` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:32.768522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8e95c-30f9-43cb-8941-5104861ace49 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:create_grant(project_id=3faddd3d88c94a3eb3c26e1ca4fee53a, user_id=5027ccbfd8f54ee6b639e51aa1bbc76d, role_id=1c38be71ee764c4389654c47d28209ee)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:32.772789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8e95c-30f9-43cb-8941-5104861ace49 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:32.797863 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2206/4449] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:32 2022] PUT /identity/v3/projects/3faddd3d88c94a3eb3c26e1ca4fee53a/users/5027ccbfd8f54ee6b639e51aa1bbc76d/roles/1c38be71ee764c4389654c47d28209ee => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:32.804896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8e95c-30f9-43cb-8941-5104861ace49 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:32.898180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e87be-0e45-44b5-9b03-08a9c7e57b93 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:32.904281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8e95c-30f9-43cb-8941-5104861ace49 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:32.911261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849a03a-4c0c-4bf4-a9a7-a84f688b7880 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:32.911529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849a03a-4c0c-4bf4-a9a7-a84f688b7880 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:32.911760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849a03a-4c0c-4bf4-a9a7-a84f688b7880 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:32.913276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849a03a-4c0c-4bf4-a9a7-a84f688b7880 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:32.913910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849a03a-4c0c-4bf4-a9a7-a84f688b7880 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:32.916184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849a03a-4c0c-4bf4-a9a7-a84f688b7880 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:32.946054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:32.989038 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2207/4450] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:32 2022] POST /identity/v3/projects => generated 360 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:32.991481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849a03a-4c0c-4bf4-a9a7-a84f688b7880 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:33.018965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384324fa-617d-4c34-8f39-fbce89114a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:33.019740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384324fa-617d-4c34-8f39-fbce89114a23 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:33.020108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384324fa-617d-4c34-8f39-fbce89114a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:33.020576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384324fa-617d-4c34-8f39-fbce89114a23 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:33.021302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384324fa-617d-4c34-8f39-fbce89114a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:33.023331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384324fa-617d-4c34-8f39-fbce89114a23 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:33.059295 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2244/4451] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:32 2022] POST /identity/v3/users => generated 289 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:33.063519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384324fa-617d-4c34-8f39-fbce89114a23 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:33.084261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384324fa-617d-4c34-8f39-fbce89114a23 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:33.147922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849a03a-4c0c-4bf4-a9a7-a84f688b7880 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:33.153451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e4fea-d4b0-4c13-ac2f-ae19e0a06de5 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:33.154132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e4fea-d4b0-4c13-ac2f-ae19e0a06de5 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:33.154411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e4fea-d4b0-4c13-ac2f-ae19e0a06de5 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:33.154676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e4fea-d4b0-4c13-ac2f-ae19e0a06de5 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects/4ddb483a66204ca9bc91dd8afd2c515a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:33.160931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e4fea-d4b0-4c13-ac2f-ae19e0a06de5 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:get_project(project_id=4ddb483a66204ca9bc91dd8afd2c515a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:33.163835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e4fea-d4b0-4c13-ac2f-ae19e0a06de5 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:33.170984 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2208/4452] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:32 2022] GET /identity/v3/projects/4ddb483a66204ca9bc91dd8afd2c515a => generated 360 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:33.174795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e4fea-d4b0-4c13-ac2f-ae19e0a06de5 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:33.203831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e4fea-d4b0-4c13-ac2f-ae19e0a06de5 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:33.209198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1420d-0450-4b0e-86dd-c80a6c009110 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:33.209961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1420d-0450-4b0e-86dd-c80a6c009110 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:33.210308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1420d-0450-4b0e-86dd-c80a6c009110 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:33.210689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1420d-0450-4b0e-86dd-c80a6c009110 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects/4ddb483a66204ca9bc91dd8afd2c515a/users/5027ccbfd8f54ee6b639e51aa1bbc76d/roles/1c38be71ee764c4389654c47d28209ee` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:33.213174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014ddd1-5048-4c25-8c3e-8ed690b8a014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:33.213736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1420d-0450-4b0e-86dd-c80a6c009110 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:create_grant(project_id=4ddb483a66204ca9bc91dd8afd2c515a, user_id=5027ccbfd8f54ee6b639e51aa1bbc76d, role_id=1c38be71ee764c4389654c47d28209ee)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:33.213855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014ddd1-5048-4c25-8c3e-8ed690b8a014 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:33.214052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014ddd1-5048-4c25-8c3e-8ed690b8a014 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:33.214313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014ddd1-5048-4c25-8c3e-8ed690b8a014 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:33.214943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014ddd1-5048-4c25-8c3e-8ed690b8a014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:33.216550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014ddd1-5048-4c25-8c3e-8ed690b8a014 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:33.217081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1420d-0450-4b0e-86dd-c80a6c009110 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:33.252461 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2245/4453] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:33 2022] GET /identity/v3/roles => generated 1651 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:33.254675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014ddd1-5048-4c25-8c3e-8ed690b8a014 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:33.284088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014ddd1-5048-4c25-8c3e-8ed690b8a014 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:33.287672 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2209/4454] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:33 2022] PUT /identity/v3/projects/4ddb483a66204ca9bc91dd8afd2c515a/users/5027ccbfd8f54ee6b639e51aa1bbc76d/roles/1c38be71ee764c4389654c47d28209ee => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:33.299199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1420d-0450-4b0e-86dd-c80a6c009110 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:33.406557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:33.421870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1420d-0450-4b0e-86dd-c80a6c009110 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:33.430242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca4dbe-13cc-4d6e-b8a4-246daa00b4ef tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:33.431241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca4dbe-13cc-4d6e-b8a4-246daa00b4ef tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:33.431241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca4dbe-13cc-4d6e-b8a4-246daa00b4ef tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:33.431557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca4dbe-13cc-4d6e-b8a4-246daa00b4ef tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/5027ccbfd8f54ee6b639e51aa1bbc76d/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:33.433706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca4dbe-13cc-4d6e-b8a4-246daa00b4ef tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:list_user_projects(user_id=5027ccbfd8f54ee6b639e51aa1bbc76d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:33.435658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca4dbe-13cc-4d6e-b8a4-246daa00b4ef tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:33.444551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71ba4b-2a20-4c1d-b4d7-a0b6a95ad30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:33.445182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71ba4b-2a20-4c1d-b4d7-a0b6a95ad30e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:33.445815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71ba4b-2a20-4c1d-b4d7-a0b6a95ad30e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:33.446075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71ba4b-2a20-4c1d-b4d7-a0b6a95ad30e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:33.446908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71ba4b-2a20-4c1d-b4d7-a0b6a95ad30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:33.448485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71ba4b-2a20-4c1d-b4d7-a0b6a95ad30e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:33.479391 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2246/4455] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:33 2022] GET /identity/v3/roles => generated 1651 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:33.483389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71ba4b-2a20-4c1d-b4d7-a0b6a95ad30e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:33.503728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71ba4b-2a20-4c1d-b4d7-a0b6a95ad30e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:33.512837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5054b-afc4-4e6b-be9a-ffbedf38411a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:33.513201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5054b-afc4-4e6b-be9a-ffbedf38411a admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:33.513470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5054b-afc4-4e6b-be9a-ffbedf38411a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:33.513734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5054b-afc4-4e6b-be9a-ffbedf38411a admin admin] PATH_INFO: `/v3/projects/02a62e8f59af4d4a8c1abc150e4d0dcc/users/d442b62a763b4ec58e6776602890e200/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:33.546138 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2210/4456] 10.222.0.23 () {58 vars in 1248 bytes} [Sat Aug 20 21:09:33 2022] GET /identity/v3/users/5027ccbfd8f54ee6b639e51aa1bbc76d/projects => generated 842 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:33.549631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca4dbe-13cc-4d6e-b8a4-246daa00b4ef tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:33.554539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5054b-afc4-4e6b-be9a-ffbedf38411a admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a62e8f59af4d4a8c1abc150e4d0dcc, user_id=d442b62a763b4ec58e6776602890e200, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:33.557389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5054b-afc4-4e6b-be9a-ffbedf38411a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:33.572016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca4dbe-13cc-4d6e-b8a4-246daa00b4ef tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:33.605321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:33.606047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:33.606311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:33.606579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects/4ddb483a66204ca9bc91dd8afd2c515a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:33.608074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4ddb483a66204ca9bc91dd8afd2c515a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:33.611201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:33.625023 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2247/4457] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:33 2022] PUT /identity/v3/projects/02a62e8f59af4d4a8c1abc150e4d0dcc/users/d442b62a763b4ec58e6776602890e200/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:33.643039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24422c-73cc-4921-a204-4862c8fa1e57 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:33.643552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24422c-73cc-4921-a204-4862c8fa1e57 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:33.643825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24422c-73cc-4921-a204-4862c8fa1e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:33.712055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4b24422c-73cc-4921-a204-4862c8fa1e57 None None] MFA Rules not processed for user `d442b62a763b4ec58e6776602890e200`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:33.713687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ddb483a66204ca9bc91dd8afd2c515a', 'request_id': 'req-cc274cf9-8ba0-4719-ada7-598cf291b161'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:33.714215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4b24422c-73cc-4921-a204-4862c8fa1e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:33.720741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ddb483a66204ca9bc91dd8afd2c515a', 'request_id': 'req-cc274cf9-8ba0-4719-ada7-598cf291b161'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:33.733111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ddb483a66204ca9bc91dd8afd2c515a', 'request_id': 'req-cc274cf9-8ba0-4719-ada7-598cf291b161'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:33.738893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:33.746189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] The token cache is being invalidate because project 4ddb483a66204ca9bc91dd8afd2c515a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:33.746478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:33.747309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:33.751921 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2211/4458] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:33 2022] DELETE /identity/v3/projects/4ddb483a66204ca9bc91dd8afd2c515a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:33.753093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:33.756636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:33.807031 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2248/4459] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:33.808791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24422c-73cc-4921-a204-4862c8fa1e57 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:33.811656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4b24422c-73cc-4921-a204-4862c8fa1e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:33.873698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc274cf9-8ba0-4719-ada7-598cf291b161 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:33.880270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:33.881865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:33.882379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:33.883278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects/3faddd3d88c94a3eb3c26e1ca4fee53a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:33.893489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3faddd3d88c94a3eb3c26e1ca4fee53a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:33.896361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:33.965875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24422c-73cc-4921-a204-4862c8fa1e57 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:33.973373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7412f-7139-4155-a2eb-fa0cc3633eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:33.974777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7412f-7139-4155-a2eb-fa0cc3633eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:33.975155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7412f-7139-4155-a2eb-fa0cc3633eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:33.975506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7412f-7139-4155-a2eb-fa0cc3633eb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:33.976243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7412f-7139-4155-a2eb-fa0cc3633eb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:33.978397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7412f-7139-4155-a2eb-fa0cc3633eb9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:34.006743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:34.010428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3faddd3d88c94a3eb3c26e1ca4fee53a', 'request_id': 'req-13b87064-dcce-4799-a641-19087c2cfbb3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.023668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3faddd3d88c94a3eb3c26e1ca4fee53a', 'request_id': 'req-13b87064-dcce-4799-a641-19087c2cfbb3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.040663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3faddd3d88c94a3eb3c26e1ca4fee53a', 'request_id': 'req-13b87064-dcce-4799-a641-19087c2cfbb3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.057253 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2249/4460] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:33 2022] POST /identity/v3/projects => generated 375 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:34.060791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7412f-7139-4155-a2eb-fa0cc3633eb9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:34.070220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] The token cache is being invalidate because project 3faddd3d88c94a3eb3c26e1ca4fee53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:34.070889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.073021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.076482 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2212/4461] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:33 2022] DELETE /identity/v3/projects/3faddd3d88c94a3eb3c26e1ca4fee53a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:34.079152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:34.083850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:34.145038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7412f-7139-4155-a2eb-fa0cc3633eb9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:34.150746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dd632-a5ce-4fe3-b6ee-1f08193d8735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:34.151367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dd632-a5ce-4fe3-b6ee-1f08193d8735 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:34.151636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dd632-a5ce-4fe3-b6ee-1f08193d8735 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:34.151920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dd632-a5ce-4fe3-b6ee-1f08193d8735 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:34.152558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2dd632-a5ce-4fe3-b6ee-1f08193d8735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:34.154698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2dd632-a5ce-4fe3-b6ee-1f08193d8735 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:34.177017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b87064-dcce-4799-a641-19087c2cfbb3 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:34.197386 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2250/4462] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:34 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:34.201078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dd632-a5ce-4fe3-b6ee-1f08193d8735 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:34.212766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9d2dd632-a5ce-4fe3-b6ee-1f08193d8735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:34.235482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dd632-a5ce-4fe3-b6ee-1f08193d8735 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:34.244953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767775a-4fd4-483a-b49d-65dbeb172194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:34.245579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767775a-4fd4-483a-b49d-65dbeb172194 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:34.245848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767775a-4fd4-483a-b49d-65dbeb172194 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:34.246109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767775a-4fd4-483a-b49d-65dbeb172194 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:34.246759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767775a-4fd4-483a-b49d-65dbeb172194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:34.248566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767775a-4fd4-483a-b49d-65dbeb172194 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:34.262873 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2251/4463] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:34 2022] GET /identity/v3/roles => generated 1651 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:34.266492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005e71b-9729-4bcc-b671-67505e9508bb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:34.266773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767775a-4fd4-483a-b49d-65dbeb172194 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:34.267147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005e71b-9729-4bcc-b671-67505e9508bb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:34.267426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005e71b-9729-4bcc-b671-67505e9508bb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:34.267692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005e71b-9729-4bcc-b671-67505e9508bb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/roles/1c38be71ee764c4389654c47d28209ee` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:34.269280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005e71b-9729-4bcc-b671-67505e9508bb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:delete_role(role_id=1c38be71ee764c4389654c47d28209ee)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:34.273458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005e71b-9729-4bcc-b671-67505e9508bb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:34.281871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767775a-4fd4-483a-b49d-65dbeb172194 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:34.288970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c722079-4dd8-4174-9247-737a4149d598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:34.291262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c722079-4dd8-4174-9247-737a4149d598 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:34.291993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c722079-4dd8-4174-9247-737a4149d598 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:34.293280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c722079-4dd8-4174-9247-737a4149d598 admin admin] PATH_INFO: `/v3/projects/7e5d6bd9f7d943d189a7211ef7c0b4d1/users/75c6a0bbf6cf43a2baa3371925d5a7e8/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:34.315951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c722079-4dd8-4174-9247-737a4149d598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5d6bd9f7d943d189a7211ef7c0b4d1, user_id=75c6a0bbf6cf43a2baa3371925d5a7e8, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:34.319447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c722079-4dd8-4174-9247-737a4149d598 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:34.323072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9005e71b-9729-4bcc-b671-67505e9508bb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invalidating the token cache because role 1c38be71ee764c4389654c47d28209ee has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:34.323366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9005e71b-9729-4bcc-b671-67505e9508bb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.324090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9005e71b-9729-4bcc-b671-67505e9508bb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.326766 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2213/4464] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:34 2022] DELETE /identity/v3/roles/1c38be71ee764c4389654c47d28209ee => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:34.330544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005e71b-9729-4bcc-b671-67505e9508bb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:34.332024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9005e71b-9729-4bcc-b671-67505e9508bb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:34.333475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:34.339669 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2252/4465] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:34 2022] PUT /identity/v3/projects/7e5d6bd9f7d943d189a7211ef7c0b4d1/users/75c6a0bbf6cf43a2baa3371925d5a7e8/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:34.344473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c722079-4dd8-4174-9247-737a4149d598 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:34.346753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5c722079-4dd8-4174-9247-737a4149d598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:34.392654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:34.410745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005e71b-9729-4bcc-b671-67505e9508bb tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:34.417965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c722079-4dd8-4174-9247-737a4149d598 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:34.422604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebaf0d3-42c6-438e-ba64-2ff0847fa93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:34.423206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaf0d3-42c6-438e-ba64-2ff0847fa93d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:34.423475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaf0d3-42c6-438e-ba64-2ff0847fa93d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:34.423741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaf0d3-42c6-438e-ba64-2ff0847fa93d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:34.424394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebaf0d3-42c6-438e-ba64-2ff0847fa93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:34.425994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebaf0d3-42c6-438e-ba64-2ff0847fa93d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:34.436284 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2253/4466] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:34.437924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e6bcf-c31a-419a-ad56-6e521653f3fd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:34.438202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e6bcf-c31a-419a-ad56-6e521653f3fd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:34.438471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e6bcf-c31a-419a-ad56-6e521653f3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:34.459220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ec1e6bcf-c31a-419a-ad56-6e521653f3fd None None] MFA Rules not processed for user `bbc87613dd6349babc15334a66bee64f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:34.460625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ec1e6bcf-c31a-419a-ad56-6e521653f3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:34.472481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:34.475174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:34.475461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:34.475731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/5027ccbfd8f54ee6b639e51aa1bbc76d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:34.477192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:delete_user(user_id=5027ccbfd8f54ee6b639e51aa1bbc76d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:34.478951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:34.531924 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2254/4467] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:34.532651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5027ccbfd8f54ee6b639e51aa1bbc76d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.533147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e6bcf-c31a-419a-ad56-6e521653f3fd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:34.543589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5027ccbfd8f54ee6b639e51aa1bbc76d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.545074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e6bcf-c31a-419a-ad56-6e521653f3fd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:34.549847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50adba-787c-4cea-8c3e-db65bccb2d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:34.550509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50adba-787c-4cea-8c3e-db65bccb2d8c admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:34.550779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50adba-787c-4cea-8c3e-db65bccb2d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:34.551052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50adba-787c-4cea-8c3e-db65bccb2d8c admin admin] PATH_INFO: `/v3/projects/7e5d6bd9f7d943d189a7211ef7c0b4d1/users/75c6a0bbf6cf43a2baa3371925d5a7e8/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:34.553775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50adba-787c-4cea-8c3e-db65bccb2d8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5d6bd9f7d943d189a7211ef7c0b4d1, user_id=75c6a0bbf6cf43a2baa3371925d5a7e8, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:34.556482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50adba-787c-4cea-8c3e-db65bccb2d8c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:34.559366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5027ccbfd8f54ee6b639e51aa1bbc76d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.560249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5027ccbfd8f54ee6b639e51aa1bbc76d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.561045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5027ccbfd8f54ee6b639e51aa1bbc76d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.572440 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2255/4468] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:34 2022] PUT /identity/v3/projects/7e5d6bd9f7d943d189a7211ef7c0b4d1/users/75c6a0bbf6cf43a2baa3371925d5a7e8/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:34.582958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454b915-e741-4720-b38b-b731df8465fa None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:34.583773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454b915-e741-4720-b38b-b731df8465fa None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:34.584106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454b915-e741-4720-b38b-b731df8465fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:34.584473 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2214/4469] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:34 2022] DELETE /identity/v3/users/5027ccbfd8f54ee6b639e51aa1bbc76d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:34.585594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:34.587232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:34.616966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0454b915-e741-4720-b38b-b731df8465fa None None] MFA Rules not processed for user `75c6a0bbf6cf43a2baa3371925d5a7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:34.618495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0454b915-e741-4720-b38b-b731df8465fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:34.656747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feff2ee-0376-4913-bd04-b88882a24284 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:34.663866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:34.664822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:34.664822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:34.665057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects/52cdf8f37a2641768acbd1ef3c5d8531` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:34.666225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:delete_project(project_id=52cdf8f37a2641768acbd1ef3c5d8531)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:34.668142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:34.685965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:34.699727 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2256/4470] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:34.704593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454b915-e741-4720-b38b-b731df8465fa None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:34.723323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454b915-e741-4720-b38b-b731df8465fa None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:34.738671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52cdf8f37a2641768acbd1ef3c5d8531', 'request_id': 'req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.744447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:34.746847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52cdf8f37a2641768acbd1ef3c5d8531', 'request_id': 'req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.769270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52cdf8f37a2641768acbd1ef3c5d8531', 'request_id': 'req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.782598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] The token cache is being invalidate because project 52cdf8f37a2641768acbd1ef3c5d8531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:34.782846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.783519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:34.785077 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2215/4471] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:34 2022] DELETE /identity/v3/projects/52cdf8f37a2641768acbd1ef3c5d8531 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:34.790341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:34.792464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:34.809679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6bf5d-dc0d-44e2-9098-720ffb08bc97 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75c6a0bbf6cf43a2baa3371925d5a7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e5d6bd9f7d943d189a7211ef7c0b4d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:34.810430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6bf5d-dc0d-44e2-9098-720ffb08bc97 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:34.810798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6bf5d-dc0d-44e2-9098-720ffb08bc97 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:34.811160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6bf5d-dc0d-44e2-9098-720ffb08bc97 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:34.811902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6bf5d-dc0d-44e2-9098-720ffb08bc97 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:34.813791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6bf5d-dc0d-44e2-9098-720ffb08bc97 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:34.845249 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2257/4472] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:09:34 2022] POST /identity/v3/domains => generated 246 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:34.849268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6bf5d-dc0d-44e2-9098-720ffb08bc97 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:34.851740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-32a6bf5d-dc0d-44e2-9098-720ffb08bc97 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:34.889433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4404e-a11d-4e4e-bc4e-8685eb3a01c7 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:34.934116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6bf5d-dc0d-44e2-9098-720ffb08bc97 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:34.940178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f6587-35bf-4a54-8501-9df997b13263 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75c6a0bbf6cf43a2baa3371925d5a7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e5d6bd9f7d943d189a7211ef7c0b4d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:34.940839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f6587-35bf-4a54-8501-9df997b13263 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:34.941115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f6587-35bf-4a54-8501-9df997b13263 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:34.941385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f6587-35bf-4a54-8501-9df997b13263 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:34.942027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f6587-35bf-4a54-8501-9df997b13263 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:34.944393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f6587-35bf-4a54-8501-9df997b13263 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:34.964694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b2f72-486a-441f-a83a-0304e7a1d9ca tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:34.965548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b2f72-486a-441f-a83a-0304e7a1d9ca tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:34.965923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b2f72-486a-441f-a83a-0304e7a1d9ca tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:34.966332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b2f72-486a-441f-a83a-0304e7a1d9ca tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/4b7bcade95e34f8d93ee81c3198c98ba` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:34.968270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b2f72-486a-441f-a83a-0304e7a1d9ca tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:update_user(user_id=4b7bcade95e34f8d93ee81c3198c98ba)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:34.970280 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-544b2f72-486a-441f-a83a-0304e7a1d9ca tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:34.970280 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:09:34.970280 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:09:34.972601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b2f72-486a-441f-a83a-0304e7a1d9ca tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:34.990608 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2258/4473] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:34 2022] POST /identity/v3/projects => generated 410 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:34.992781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f6587-35bf-4a54-8501-9df997b13263 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:35.020975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-544b2f72-486a-441f-a83a-0304e7a1d9ca tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '4b7bcade95e34f8d93ee81c3198c98ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.029873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-544b2f72-486a-441f-a83a-0304e7a1d9ca tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invalidating the token cache because user 4b7bcade95e34f8d93ee81c3198c98ba was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:35.030536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-544b2f72-486a-441f-a83a-0304e7a1d9ca tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.031303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-544b2f72-486a-441f-a83a-0304e7a1d9ca tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.033561 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2216/4474] 10.222.0.23 () {60 vars in 1243 bytes} [Sat Aug 20 21:09:34 2022] PATCH /identity/v3/users/4b7bcade95e34f8d93ee81c3198c98ba => generated 297 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:09:35.038990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91417abc-fc6b-4765-b0e9-30939af3271d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.039475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91417abc-fc6b-4765-b0e9-30939af3271d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.039927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91417abc-fc6b-4765-b0e9-30939af3271d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.066174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f6587-35bf-4a54-8501-9df997b13263 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:35.071811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f9cc4-425a-4edd-be8e-b9f478d4a036 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75c6a0bbf6cf43a2baa3371925d5a7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e5d6bd9f7d943d189a7211ef7c0b4d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:35.072574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f9cc4-425a-4edd-be8e-b9f478d4a036 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.073047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f9cc4-425a-4edd-be8e-b9f478d4a036 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.073409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f9cc4-425a-4edd-be8e-b9f478d4a036 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.074119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f9cc4-425a-4edd-be8e-b9f478d4a036 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:35.074308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-91417abc-fc6b-4765-b0e9-30939af3271d None None] MFA Rules not processed for user `4b7bcade95e34f8d93ee81c3198c98ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:35.075643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-91417abc-fc6b-4765-b0e9-30939af3271d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:35.076432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f9cc4-425a-4edd-be8e-b9f478d4a036 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:35.079087 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2217/4475] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:35 2022] POST /identity/v3/auth/tokens => generated 349 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:09:35.083502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91417abc-fc6b-4765-b0e9-30939af3271d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:35.084923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-91417abc-fc6b-4765-b0e9-30939af3271d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:35.096866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:35.105418 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2259/4476] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:34 2022] POST /identity/v3/users => generated 348 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:35.110468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f9cc4-425a-4edd-be8e-b9f478d4a036 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:35.113422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b07f9cc4-425a-4edd-be8e-b9f478d4a036 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:35.131699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f9cc4-425a-4edd-be8e-b9f478d4a036 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:35.137885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b7bac-a51b-46df-a5de-c77ee4ff8394 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75c6a0bbf6cf43a2baa3371925d5a7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e5d6bd9f7d943d189a7211ef7c0b4d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:35.138808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b7bac-a51b-46df-a5de-c77ee4ff8394 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.139416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b7bac-a51b-46df-a5de-c77ee4ff8394 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.139851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b7bac-a51b-46df-a5de-c77ee4ff8394 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.140842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b7bac-a51b-46df-a5de-c77ee4ff8394 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:35.142669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b7bac-a51b-46df-a5de-c77ee4ff8394 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:35.156023 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2260/4477] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:35.160691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b7bac-a51b-46df-a5de-c77ee4ff8394 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:35.167268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:35.173451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b7bac-a51b-46df-a5de-c77ee4ff8394 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:35.181252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91417abc-fc6b-4765-b0e9-30939af3271d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:35.181878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b40ff-5b73-4b00-96fc-c748406f0378 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75c6a0bbf6cf43a2baa3371925d5a7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e5d6bd9f7d943d189a7211ef7c0b4d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:35.182676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b40ff-5b73-4b00-96fc-c748406f0378 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.183005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b40ff-5b73-4b00-96fc-c748406f0378 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.183399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b40ff-5b73-4b00-96fc-c748406f0378 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] PATH_INFO: `/v3/projects/8188b83a23c2484eb748cdc2430698a8/users/76dfcaed590e41ac92b9c9b585d72064/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.188166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0ef7a-3968-4165-bfcf-f1081d05a5dd tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:35.188166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0ef7a-3968-4165-bfcf-f1081d05a5dd tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.188488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0ef7a-3968-4165-bfcf-f1081d05a5dd tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.188611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0ef7a-3968-4165-bfcf-f1081d05a5dd tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/4b7bcade95e34f8d93ee81c3198c98ba` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.189948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0ef7a-3968-4165-bfcf-f1081d05a5dd tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:update_user(user_id=4b7bcade95e34f8d93ee81c3198c98ba)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:35.191884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0ef7a-3968-4165-bfcf-f1081d05a5dd tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:35.202447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b40ff-5b73-4b00-96fc-c748406f0378 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorizing `identity:create_grant(project_id=8188b83a23c2484eb748cdc2430698a8, user_id=76dfcaed590e41ac92b9c9b585d72064, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:35.205862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b40ff-5b73-4b00-96fc-c748406f0378 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:35.224408 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2261/4478] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:35 2022] PUT /identity/v3/projects/8188b83a23c2484eb748cdc2430698a8/users/76dfcaed590e41ac92b9c9b585d72064/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:35.229276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0c4d1-a4d0-4f9e-983a-e699228bc923 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.229760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0c4d1-a4d0-4f9e-983a-e699228bc923 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.230305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0c4d1-a4d0-4f9e-983a-e699228bc923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.236004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-29d0ef7a-3968-4165-bfcf-f1081d05a5dd tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '4b7bcade95e34f8d93ee81c3198c98ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.247989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-29d0ef7a-3968-4165-bfcf-f1081d05a5dd tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invalidating the token cache because user 4b7bcade95e34f8d93ee81c3198c98ba was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:35.248324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-29d0ef7a-3968-4165-bfcf-f1081d05a5dd tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.249489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-29d0ef7a-3968-4165-bfcf-f1081d05a5dd tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.251921 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2218/4479] 10.222.0.23 () {60 vars in 1243 bytes} [Sat Aug 20 21:09:35 2022] PATCH /identity/v3/users/4b7bcade95e34f8d93ee81c3198c98ba => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:09:35.256994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0ef7a-3968-4165-bfcf-f1081d05a5dd tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:35.258664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-29d0ef7a-3968-4165-bfcf-f1081d05a5dd tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:35.274021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0ef7a-3968-4165-bfcf-f1081d05a5dd tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:35.356632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc9bc0-97fd-4eed-84bc-f6d012b41f86 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:35.357994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc9bc0-97fd-4eed-84bc-f6d012b41f86 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.358274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc9bc0-97fd-4eed-84bc-f6d012b41f86 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.358546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc9bc0-97fd-4eed-84bc-f6d012b41f86 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/4b7bcade95e34f8d93ee81c3198c98ba` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.377469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc9bc0-97fd-4eed-84bc-f6d012b41f86 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:update_user(user_id=4b7bcade95e34f8d93ee81c3198c98ba)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:35.380450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc9bc0-97fd-4eed-84bc-f6d012b41f86 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:35.383099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1ed0c4d1-a4d0-4f9e-983a-e699228bc923 None None] MFA Rules not processed for user `76dfcaed590e41ac92b9c9b585d72064`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:35.384546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1ed0c4d1-a4d0-4f9e-983a-e699228bc923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:35.403932 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2262/4480] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:35 2022] POST /identity/v3/auth/tokens => generated 3363 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:35.415412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97aab0-cbe3-4f56-b051-7e077cea97a6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.415942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97aab0-cbe3-4f56-b051-7e077cea97a6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.416274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97aab0-cbe3-4f56-b051-7e077cea97a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.424647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59bc9bc0-97fd-4eed-84bc-f6d012b41f86 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '4b7bcade95e34f8d93ee81c3198c98ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.436321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59bc9bc0-97fd-4eed-84bc-f6d012b41f86 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invalidating the token cache because user 4b7bcade95e34f8d93ee81c3198c98ba was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:35.436908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59bc9bc0-97fd-4eed-84bc-f6d012b41f86 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.438748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59bc9bc0-97fd-4eed-84bc-f6d012b41f86 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.440276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1a97aab0-cbe3-4f56-b051-7e077cea97a6 None None] MFA Rules not processed for user `76dfcaed590e41ac92b9c9b585d72064`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:35.441485 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2219/4481] 10.222.0.23 () {60 vars in 1243 bytes} [Sat Aug 20 21:09:35 2022] PATCH /identity/v3/users/4b7bcade95e34f8d93ee81c3198c98ba => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:09:35.441636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1a97aab0-cbe3-4f56-b051-7e077cea97a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:35.451296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24408c4-951d-4470-9529-f196261ccf48 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.451617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24408c4-951d-4470-9529-f196261ccf48 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.452051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24408c4-951d-4470-9529-f196261ccf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.453564 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2263/4482] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:35 2022] POST /identity/v3/auth/tokens => generated 3363 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:35.458170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97aab0-cbe3-4f56-b051-7e077cea97a6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:35.469273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97aab0-cbe3-4f56-b051-7e077cea97a6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:35.475561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71ba05-f143-462e-8bac-38f14a5771a9 tempest-test_project-1085957676 tempest-user-1375487218] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dfcaed590e41ac92b9c9b585d72064', 'user_domain_id': 'dc0bce09c24743d59a15f6768dc49bd5', 'system_scope': None, 'project_id': '8188b83a23c2484eb748cdc2430698a8', 'project_domain_id': 'dc0bce09c24743d59a15f6768dc49bd5', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:35.476191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71ba05-f143-462e-8bac-38f14a5771a9 tempest-test_project-1085957676 tempest-user-1375487218] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.476502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71ba05-f143-462e-8bac-38f14a5771a9 tempest-test_project-1085957676 tempest-user-1375487218] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.476773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71ba05-f143-462e-8bac-38f14a5771a9 tempest-test_project-1085957676 tempest-user-1375487218] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.488294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71ba05-f143-462e-8bac-38f14a5771a9 tempest-test_project-1085957676 tempest-user-1375487218] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:35.490599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71ba05-f143-462e-8bac-38f14a5771a9 tempest-test_project-1085957676 tempest-user-1375487218] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:35.500419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e24408c4-951d-4470-9529-f196261ccf48 None None] MFA Rules not processed for user `4b7bcade95e34f8d93ee81c3198c98ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:35.501813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e24408c4-951d-4470-9529-f196261ccf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:35.504552 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2264/4483] 10.222.0.23 () {56 vars in 1284 bytes} [Sat Aug 20 21:09:35 2022] GET /identity/v3/auth/tokens => generated 3363 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:09:35.507613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71ba05-f143-462e-8bac-38f14a5771a9 tempest-test_project-1085957676 tempest-user-1375487218] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:35.510140 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2220/4484] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:35 2022] POST /identity/v3/auth/tokens => generated 349 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:09:35.510564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1e71ba05-f143-462e-8bac-38f14a5771a9 tempest-test_project-1085957676 tempest-user-1375487218] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:35.513768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24408c4-951d-4470-9529-f196261ccf48 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:35.516012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e24408c4-951d-4470-9529-f196261ccf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:35.555530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24408c4-951d-4470-9529-f196261ccf48 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:35.565550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062803e5-4fc1-4fa9-8baf-7ec4272b1dea tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:35.566208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062803e5-4fc1-4fa9-8baf-7ec4272b1dea tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.566487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062803e5-4fc1-4fa9-8baf-7ec4272b1dea tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.566746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062803e5-4fc1-4fa9-8baf-7ec4272b1dea tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/4b7bcade95e34f8d93ee81c3198c98ba` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.569186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062803e5-4fc1-4fa9-8baf-7ec4272b1dea tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:update_user(user_id=4b7bcade95e34f8d93ee81c3198c98ba)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:35.571706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062803e5-4fc1-4fa9-8baf-7ec4272b1dea tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:35.610310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71ba05-f143-462e-8bac-38f14a5771a9 tempest-test_project-1085957676 tempest-user-1375487218] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:35.615256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75c6a0bbf6cf43a2baa3371925d5a7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e5d6bd9f7d943d189a7211ef7c0b4d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:35.615910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.616177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.616481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] PATH_INFO: `/v3/users/76dfcaed590e41ac92b9c9b585d72064` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.617805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorizing `identity:delete_user(user_id=76dfcaed590e41ac92b9c9b585d72064)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:35.619739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:35.672320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-062803e5-4fc1-4fa9-8baf-7ec4272b1dea tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '4b7bcade95e34f8d93ee81c3198c98ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.678541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:UPDATE=3,keystone:INSERT=6,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:35.687340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-062803e5-4fc1-4fa9-8baf-7ec4272b1dea tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invalidating the token cache because user 4b7bcade95e34f8d93ee81c3198c98ba was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:35.687637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-062803e5-4fc1-4fa9-8baf-7ec4272b1dea tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.688628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-062803e5-4fc1-4fa9-8baf-7ec4272b1dea tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.692678 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2221/4485] 10.222.0.23 () {60 vars in 1243 bytes} [Sat Aug 20 21:09:35 2022] PATCH /identity/v3/users/4b7bcade95e34f8d93ee81c3198c98ba => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:35.693165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:35.698717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062803e5-4fc1-4fa9-8baf-7ec4272b1dea tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:35.705282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-062803e5-4fc1-4fa9-8baf-7ec4272b1dea tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:35.734049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76dfcaed590e41ac92b9c9b585d72064'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.742093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062803e5-4fc1-4fa9-8baf-7ec4272b1dea tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:35.747398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522ca22-eff7-4e2b-acac-cf122faa077d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:35.748144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522ca22-eff7-4e2b-acac-cf122faa077d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.748556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522ca22-eff7-4e2b-acac-cf122faa077d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.748830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522ca22-eff7-4e2b-acac-cf122faa077d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.749449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522ca22-eff7-4e2b-acac-cf122faa077d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:35.750990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76dfcaed590e41ac92b9c9b585d72064'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.751550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522ca22-eff7-4e2b-acac-cf122faa077d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:35.763498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76dfcaed590e41ac92b9c9b585d72064'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.781124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76dfcaed590e41ac92b9c9b585d72064'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.782166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76dfcaed590e41ac92b9c9b585d72064'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:35.793620 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2265/4486] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:35 2022] DELETE /identity/v3/users/76dfcaed590e41ac92b9c9b585d72064 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:35.795295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:35.796964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:35.805929 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2222/4487] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:09:35 2022] POST /identity/v3/users => generated 265 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:35.810113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522ca22-eff7-4e2b-acac-cf122faa077d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:35.813411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2522ca22-eff7-4e2b-acac-cf122faa077d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:35.853353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522ca22-eff7-4e2b-acac-cf122faa077d tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:35.934981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c06d9-a3ab-4e1c-b695-cc4afa8f25dc tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:35.941084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75c6a0bbf6cf43a2baa3371925d5a7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e5d6bd9f7d943d189a7211ef7c0b4d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:35.941733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.942005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.942337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] PATH_INFO: `/v3/projects/8188b83a23c2484eb748cdc2430698a8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:35.943809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8188b83a23c2484eb748cdc2430698a8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:35.946152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:35.952841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c06732-66e5-4624-8750-e6cfbd20e7c8 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:35.964408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c06732-66e5-4624-8750-e6cfbd20e7c8 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:35.964408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c06732-66e5-4624-8750-e6cfbd20e7c8 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:35.964408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c06732-66e5-4624-8750-e6cfbd20e7c8 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/e5563d8e50a74153b76c606a776f07b1/password` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:36.031256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8188b83a23c2484eb748cdc2430698a8', 'request_id': 'req-c2639af9-13fc-4ad7-9b92-b793db129807'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.057463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8188b83a23c2484eb748cdc2430698a8', 'request_id': 'req-c2639af9-13fc-4ad7-9b92-b793db129807'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.061555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e9c06732-66e5-4624-8750-e6cfbd20e7c8 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'e5563d8e50a74153b76c606a776f07b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.072061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8188b83a23c2484eb748cdc2430698a8', 'request_id': 'req-c2639af9-13fc-4ad7-9b92-b793db129807'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.082924 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2223/4488] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:09:35 2022] POST /identity/v3/users/e5563d8e50a74153b76c606a776f07b1/password => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:36.085001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] The token cache is being invalidate because project 8188b83a23c2484eb748cdc2430698a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:36.085309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.085934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.087829 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2266/4489] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:35 2022] DELETE /identity/v3/projects/8188b83a23c2484eb748cdc2430698a8 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:36.092038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c06732-66e5-4624-8750-e6cfbd20e7c8 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:36.093825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e9c06732-66e5-4624-8750-e6cfbd20e7c8 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:36.109345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c06732-66e5-4624-8750-e6cfbd20e7c8 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:36.186779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eef6f7-00e2-447d-9bdf-7cd0f8c5a6d5 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75c6a0bbf6cf43a2baa3371925d5a7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e5d6bd9f7d943d189a7211ef7c0b4d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:36.187922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eef6f7-00e2-447d-9bdf-7cd0f8c5a6d5 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:36.188427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eef6f7-00e2-447d-9bdf-7cd0f8c5a6d5 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:36.189180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eef6f7-00e2-447d-9bdf-7cd0f8c5a6d5 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] PATH_INFO: `/v3/domains/dc0bce09c24743d59a15f6768dc49bd5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:36.191255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eef6f7-00e2-447d-9bdf-7cd0f8c5a6d5 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=dc0bce09c24743d59a15f6768dc49bd5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:36.194351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eef6f7-00e2-447d-9bdf-7cd0f8c5a6d5 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:36.221630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f0eef6f7-00e2-447d-9bdf-7cd0f8c5a6d5 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'dc0bce09c24743d59a15f6768dc49bd5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.222766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f0eef6f7-00e2-447d-9bdf-7cd0f8c5a6d5 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'dc0bce09c24743d59a15f6768dc49bd5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.260151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f0eef6f7-00e2-447d-9bdf-7cd0f8c5a6d5 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'dc0bce09c24743d59a15f6768dc49bd5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.260893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f0eef6f7-00e2-447d-9bdf-7cd0f8c5a6d5 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'dc0bce09c24743d59a15f6768dc49bd5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.263403 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2224/4490] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:09:36 2022] PATCH /identity/v3/domains/dc0bce09c24743d59a15f6768dc49bd5 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:36.267921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:36.286677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2639af9-13fc-4ad7-9b92-b793db129807 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:36.355432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:36.412508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75c6a0bbf6cf43a2baa3371925d5a7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e5d6bd9f7d943d189a7211ef7c0b4d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:36.413598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:36.414088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:36.414794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] PATH_INFO: `/v3/domains/dc0bce09c24743d59a15f6768dc49bd5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:36.428014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=dc0bce09c24743d59a15f6768dc49bd5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:36.429740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:36.469103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'dc0bce09c24743d59a15f6768dc49bd5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.485735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'dc0bce09c24743d59a15f6768dc49bd5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.536024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc0bce09c24743d59a15f6768dc49bd5', 'request_id': 'req-e36636f7-6da1-46c8-8867-a54605dba6e4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.548585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc0bce09c24743d59a15f6768dc49bd5', 'request_id': 'req-e36636f7-6da1-46c8-8867-a54605dba6e4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.558823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc0bce09c24743d59a15f6768dc49bd5', 'request_id': 'req-e36636f7-6da1-46c8-8867-a54605dba6e4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.572615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] The token cache is being invalidate because project dc0bce09c24743d59a15f6768dc49bd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:36.572901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.573566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.581539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'dc0bce09c24743d59a15f6768dc49bd5', 'request_id': 'req-e36636f7-6da1-46c8-8867-a54605dba6e4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.582203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'dc0bce09c24743d59a15f6768dc49bd5', 'request_id': 'req-e36636f7-6da1-46c8-8867-a54605dba6e4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.585614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e36636f7-6da1-46c8-8867-a54605dba6e4 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'dc0bce09c24743d59a15f6768dc49bd5', 'request_id': 'req-e36636f7-6da1-46c8-8867-a54605dba6e4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.591920 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2267/4491] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:36 2022] DELETE /identity/v3/domains/dc0bce09c24743d59a15f6768dc49bd5 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:36.598498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eef6f7-00e2-447d-9bdf-7cd0f8c5a6d5 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:36.599973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f0eef6f7-00e2-447d-9bdf-7cd0f8c5a6d5 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:36.605352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=2,keystone:SELECT=91,keystone:INSERT=5,keystone:UPDATE=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:36.692027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eef6f7-00e2-447d-9bdf-7cd0f8c5a6d5 tempest-TestDefaultProjectId-480408711 tempest-TestDefaultProjectId-480408711-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:36.760772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:36.761459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:36.761708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:36.762015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] PATH_INFO: `/v3/users/d442b62a763b4ec58e6776602890e200` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:36.763353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=d442b62a763b4ec58e6776602890e200)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:36.765300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:36.832778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd442b62a763b4ec58e6776602890e200'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.866193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd442b62a763b4ec58e6776602890e200'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.889914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd442b62a763b4ec58e6776602890e200'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.890977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd442b62a763b4ec58e6776602890e200'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.892070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd442b62a763b4ec58e6776602890e200'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:36.926734 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2225/4492] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:36 2022] DELETE /identity/v3/users/d442b62a763b4ec58e6776602890e200 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:36.932871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd325eb6-ab98-4a08-81a1-653df58e8177 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:36.933196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd325eb6-ab98-4a08-81a1-653df58e8177 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:36.933404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd325eb6-ab98-4a08-81a1-653df58e8177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:36.955340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dd325eb6-ab98-4a08-81a1-653df58e8177 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:36.956666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dd325eb6-ab98-4a08-81a1-653df58e8177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:37.049575 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2268/4493] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:37.061577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:37.062896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:37.091482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cc615-6dd0-4628-883b-14843fd8d90d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:37.091767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cc615-6dd0-4628-883b-14843fd8d90d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:37.092130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cc615-6dd0-4628-883b-14843fd8d90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:37.127426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b36cc615-6dd0-4628-883b-14843fd8d90d None None] MFA Rules not processed for user `e5563d8e50a74153b76c606a776f07b1`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:37.129789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b36cc615-6dd0-4628-883b-14843fd8d90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:37.144463 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2269/4494] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:37 2022] POST /identity/v3/auth/tokens => generated 330 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:09:37.145644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:37.148847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cc615-6dd0-4628-883b-14843fd8d90d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:37.152029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b36cc615-6dd0-4628-883b-14843fd8d90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:37.171591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:37.226666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c821da-a0ba-4c09-800e-5536927a82ee admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:37.238122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61369764-535b-4482-b632-ca4961444f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:37.238407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61369764-535b-4482-b632-ca4961444f56 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:37.238677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61369764-535b-4482-b632-ca4961444f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:37.238950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61369764-535b-4482-b632-ca4961444f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:37.248911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cc615-6dd0-4628-883b-14843fd8d90d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:37.260607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab185368-c82f-4f59-b19d-a3faf8745c00 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:37.260607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab185368-c82f-4f59-b19d-a3faf8745c00 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:37.260607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab185368-c82f-4f59-b19d-a3faf8745c00 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:37.260987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab185368-c82f-4f59-b19d-a3faf8745c00 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:37.268594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61369764-535b-4482-b632-ca4961444f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:37.270065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61369764-535b-4482-b632-ca4961444f56 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:37.276683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab185368-c82f-4f59-b19d-a3faf8745c00 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:37.282596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab185368-c82f-4f59-b19d-a3faf8745c00 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:37.296472 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2226/4495] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:37.301782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61369764-535b-4482-b632-ca4961444f56 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:37.301967 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2270/4496] 10.222.0.23 () {56 vars in 1263 bytes} [Sat Aug 20 21:09:37 2022] GET /identity/v3/auth/tokens => generated 330 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 20 21:09:37.321139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61369764-535b-4482-b632-ca4961444f56 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:37.328754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ae211-6506-41e8-8f20-06f4d6a5d371 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:37.329311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ae211-6506-41e8-8f20-06f4d6a5d371 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:37.329979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ae211-6506-41e8-8f20-06f4d6a5d371 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:37.330273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ae211-6506-41e8-8f20-06f4d6a5d371 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/e5563d8e50a74153b76c606a776f07b1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:37.331530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ae211-6506-41e8-8f20-06f4d6a5d371 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:delete_user(user_id=e5563d8e50a74153b76c606a776f07b1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:37.333672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ae211-6506-41e8-8f20-06f4d6a5d371 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:37.452016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0b6ae211-6506-41e8-8f20-06f4d6a5d371 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5563d8e50a74153b76c606a776f07b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:37.489865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0b6ae211-6506-41e8-8f20-06f4d6a5d371 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5563d8e50a74153b76c606a776f07b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:37.502773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0b6ae211-6506-41e8-8f20-06f4d6a5d371 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5563d8e50a74153b76c606a776f07b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:37.503696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0b6ae211-6506-41e8-8f20-06f4d6a5d371 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5563d8e50a74153b76c606a776f07b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:37.505364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0b6ae211-6506-41e8-8f20-06f4d6a5d371 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5563d8e50a74153b76c606a776f07b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:37.535014 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2227/4497] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:37 2022] DELETE /identity/v3/users/e5563d8e50a74153b76c606a776f07b1 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:37.540606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab185368-c82f-4f59-b19d-a3faf8745c00 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:37.553971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab185368-c82f-4f59-b19d-a3faf8745c00 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:37.564655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f93f8-7955-4c8a-a251-fa47d2932145 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:37.565140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f93f8-7955-4c8a-a251-fa47d2932145 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:37.565626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f93f8-7955-4c8a-a251-fa47d2932145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:37.590447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9f6f93f8-7955-4c8a-a251-fa47d2932145 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:37.592040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9f6f93f8-7955-4c8a-a251-fa47d2932145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:37.632866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b1015-9003-445a-9d7e-3feb72dd5f52 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:37.633504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b1015-9003-445a-9d7e-3feb72dd5f52 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:37.633869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b1015-9003-445a-9d7e-3feb72dd5f52 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:37.634360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b1015-9003-445a-9d7e-3feb72dd5f52 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:37.634804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b1015-9003-445a-9d7e-3feb72dd5f52 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:37.636964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b1015-9003-445a-9d7e-3feb72dd5f52 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:37.692434 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2271/4498] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:37 2022] POST /identity/v3/users => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:37.692669 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2228/4499] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:09:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:37.697732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b1015-9003-445a-9d7e-3feb72dd5f52 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:37.703096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-510b1015-9003-445a-9d7e-3feb72dd5f52 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:37.709245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f93f8-7955-4c8a-a251-fa47d2932145 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:37.710807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9f6f93f8-7955-4c8a-a251-fa47d2932145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:37.732481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b1015-9003-445a-9d7e-3feb72dd5f52 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:37.745437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b14b44-8366-4a25-a416-2973a32d634f tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:37.746263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b14b44-8366-4a25-a416-2973a32d634f tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:37.746727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b14b44-8366-4a25-a416-2973a32d634f tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:37.747286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b14b44-8366-4a25-a416-2973a32d634f tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:37.748060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b14b44-8366-4a25-a416-2973a32d634f tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:37.750328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b14b44-8366-4a25-a416-2973a32d634f tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:37.779771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:37.793693 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2272/4500] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:37 2022] POST /identity/v3/projects => generated 359 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:37.798546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b14b44-8366-4a25-a416-2973a32d634f tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:37.805205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f93f8-7955-4c8a-a251-fa47d2932145 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:37.882011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b14b44-8366-4a25-a416-2973a32d634f tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:37.885301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e5bd4-229a-4d99-ab46-19f10ed54958 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:37.885919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e5bd4-229a-4d99-ab46-19f10ed54958 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:37.886191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e5bd4-229a-4d99-ab46-19f10ed54958 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:37.886453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e5bd4-229a-4d99-ab46-19f10ed54958 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:37.888277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e62dbe-bf27-4036-8255-0eb482e53d25 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:37.889372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e62dbe-bf27-4036-8255-0eb482e53d25 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:37.889561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e62dbe-bf27-4036-8255-0eb482e53d25 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:37.889990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e62dbe-bf27-4036-8255-0eb482e53d25 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/8a8a9e9df22c4699b2e732f9df850c36` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:37.914416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e62dbe-bf27-4036-8255-0eb482e53d25 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:update_user(user_id=8a8a9e9df22c4699b2e732f9df850c36)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:37.917063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e62dbe-bf27-4036-8255-0eb482e53d25 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:37.924734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:37.988954 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2273/4501] 10.222.0.23 () {60 vars in 1244 bytes} [Sat Aug 20 21:09:37 2022] PATCH /identity/v3/users/8a8a9e9df22c4699b2e732f9df850c36 => generated 522 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:09:37.990579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e62dbe-bf27-4036-8255-0eb482e53d25 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:37.997136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-51e62dbe-bf27-4036-8255-0eb482e53d25 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:38.002323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e5bd4-229a-4d99-ab46-19f10ed54958 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:38.004438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e5bd4-229a-4d99-ab46-19f10ed54958 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:38.028479 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2229/4502] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:09:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:09:38.029716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e5bd4-229a-4d99-ab46-19f10ed54958 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:38.043983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e5bd4-229a-4d99-ab46-19f10ed54958 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:38.048915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3e888-33dd-45da-9388-c70ba9971a24 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:38.049665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3e888-33dd-45da-9388-c70ba9971a24 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:38.050133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3e888-33dd-45da-9388-c70ba9971a24 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:38.050461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3e888-33dd-45da-9388-c70ba9971a24 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/8a8a9e9df22c4699b2e732f9df850c36` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:38.072504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3e888-33dd-45da-9388-c70ba9971a24 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:get_user(user_id=8a8a9e9df22c4699b2e732f9df850c36)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:38.075954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3e888-33dd-45da-9388-c70ba9971a24 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:38.081560 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2230/4503] 10.222.0.23 () {58 vars in 1221 bytes} [Sat Aug 20 21:09:38 2022] GET /identity/v3/users/8a8a9e9df22c4699b2e732f9df850c36 => generated 389 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:38.086036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3e888-33dd-45da-9388-c70ba9971a24 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:38.113420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3e888-33dd-45da-9388-c70ba9971a24 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:38.131324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:38.132686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:38.133062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:38.133790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/projects/7db89487c17c443eae88d04e343c07b8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:38.135554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7db89487c17c443eae88d04e343c07b8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:38.138576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:38.179979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e62dbe-bf27-4036-8255-0eb482e53d25 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:38.193291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:38.193774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:38.193905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:38.194239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] PATH_INFO: `/v3/projects/02a62e8f59af4d4a8c1abc150e4d0dcc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:38.201548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02a62e8f59af4d4a8c1abc150e4d0dcc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:38.203705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:38.277666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7db89487c17c443eae88d04e343c07b8', 'request_id': 'req-fc7c087b-369e-4445-96a4-1e2ddf7fae82'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.314599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7db89487c17c443eae88d04e343c07b8', 'request_id': 'req-fc7c087b-369e-4445-96a4-1e2ddf7fae82'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.329050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7db89487c17c443eae88d04e343c07b8', 'request_id': 'req-fc7c087b-369e-4445-96a4-1e2ddf7fae82'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.340965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:38.348168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] The token cache is being invalidate because project 7db89487c17c443eae88d04e343c07b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:38.350471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.352993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.355213 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2231/4504] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:38 2022] DELETE /identity/v3/projects/7db89487c17c443eae88d04e343c07b8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:38.360350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02a62e8f59af4d4a8c1abc150e4d0dcc', 'request_id': 'req-8b1ff019-7af5-4349-9e9b-596215943912'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.361329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:38.365804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:38.380891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02a62e8f59af4d4a8c1abc150e4d0dcc', 'request_id': 'req-8b1ff019-7af5-4349-9e9b-596215943912'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.395071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02a62e8f59af4d4a8c1abc150e4d0dcc', 'request_id': 'req-8b1ff019-7af5-4349-9e9b-596215943912'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.420992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] The token cache is being invalidate because project 02a62e8f59af4d4a8c1abc150e4d0dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:38.421222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.422733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.427616 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2274/4505] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:37 2022] DELETE /identity/v3/projects/02a62e8f59af4d4a8c1abc150e4d0dcc => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:38.432754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:38.436332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:38.483587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c087b-369e-4445-96a4-1e2ddf7fae82 tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:38.562145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9512f567658e4cbdb0e8830178105a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a9a01c8a234babb6bb1e93e4f70ad9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:38.562801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:38.563223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:38.563676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] PATH_INFO: `/v3/users/8a8a9e9df22c4699b2e732f9df850c36` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:38.566019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorizing `identity:delete_user(user_id=8a8a9e9df22c4699b2e732f9df850c36)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:38.569762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:38.578187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ff019-7af5-4349-9e9b-596215943912 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:38.593164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d17a4-be57-4fdc-8ba8-6c2447555189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:38.594346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d17a4-be57-4fdc-8ba8-6c2447555189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:38.594623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d17a4-be57-4fdc-8ba8-6c2447555189 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:38.594885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d17a4-be57-4fdc-8ba8-6c2447555189 admin admin] PATH_INFO: `/v3/users/75c6a0bbf6cf43a2baa3371925d5a7e8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:38.596181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d17a4-be57-4fdc-8ba8-6c2447555189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75c6a0bbf6cf43a2baa3371925d5a7e8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:38.598243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d17a4-be57-4fdc-8ba8-6c2447555189 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:38.629454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:UPDATE=2,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:38.692874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9f3d17a4-be57-4fdc-8ba8-6c2447555189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75c6a0bbf6cf43a2baa3371925d5a7e8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.697645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8a9e9df22c4699b2e732f9df850c36'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.704822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9f3d17a4-be57-4fdc-8ba8-6c2447555189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75c6a0bbf6cf43a2baa3371925d5a7e8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.709984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8a9e9df22c4699b2e732f9df850c36'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.716531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9f3d17a4-be57-4fdc-8ba8-6c2447555189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75c6a0bbf6cf43a2baa3371925d5a7e8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.730999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8a9e9df22c4699b2e732f9df850c36'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.732088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8a9e9df22c4699b2e732f9df850c36'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.732909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8a9e9df22c4699b2e732f9df850c36'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.735380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9f3d17a4-be57-4fdc-8ba8-6c2447555189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75c6a0bbf6cf43a2baa3371925d5a7e8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.736186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9f3d17a4-be57-4fdc-8ba8-6c2447555189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75c6a0bbf6cf43a2baa3371925d5a7e8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:38.743069 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2275/4506] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:38 2022] DELETE /identity/v3/users/75c6a0bbf6cf43a2baa3371925d5a7e8 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:38.746096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ccc72-c7e1-4861-9c76-085e17360bbe None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:38.746096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ccc72-c7e1-4861-9c76-085e17360bbe None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:38.746096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ccc72-c7e1-4861-9c76-085e17360bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:38.752814 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2232/4507] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:38 2022] DELETE /identity/v3/users/8a8a9e9df22c4699b2e732f9df850c36 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:38.768561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:38.771570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-261ccc72-c7e1-4861-9c76-085e17360bbe None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:38.771812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-261ccc72-c7e1-4861-9c76-085e17360bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:38.774026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:38.849903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:38.883041 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2276/4508] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:09:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:38.895997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ccc72-c7e1-4861-9c76-085e17360bbe None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:38.897621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-261ccc72-c7e1-4861-9c76-085e17360bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:38.903023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cef57b-361b-4280-aca5-79bfddf0c6fc tempest-UsersV3TestJSON-1147217935 tempest-UsersV3TestJSON-1147217935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:38.910085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:38.911062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:38.911445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:38.911688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] PATH_INFO: `/v3/users/4b7bcade95e34f8d93ee81c3198c98ba` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:38.934171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b7bcade95e34f8d93ee81c3198c98ba)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:38.936499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:39.009463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ccc72-c7e1-4861-9c76-085e17360bbe None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:39.010260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b7bcade95e34f8d93ee81c3198c98ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:39.015032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5fb3d6-897e-4cd1-82cb-d559391c9f81 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:39.015745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fb3d6-897e-4cd1-82cb-d559391c9f81 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:39.016094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fb3d6-897e-4cd1-82cb-d559391c9f81 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:39.016394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fb3d6-897e-4cd1-82cb-d559391c9f81 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:39.023105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b7bcade95e34f8d93ee81c3198c98ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:39.028514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5fb3d6-897e-4cd1-82cb-d559391c9f81 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:39.030403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5fb3d6-897e-4cd1-82cb-d559391c9f81 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:39.045107 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2277/4509] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:09:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:09:39.049157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b7bcade95e34f8d93ee81c3198c98ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:39.050245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b7bcade95e34f8d93ee81c3198c98ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:39.050887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b7bcade95e34f8d93ee81c3198c98ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:39.073573 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2233/4510] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:38 2022] DELETE /identity/v3/users/4b7bcade95e34f8d93ee81c3198c98ba => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:39.078231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b18077-38e3-4ec5-b124-f9b3d9697d95 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:39.078589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b18077-38e3-4ec5-b124-f9b3d9697d95 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:39.078912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b18077-38e3-4ec5-b124-f9b3d9697d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:39.114895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e0b18077-38e3-4ec5-b124-f9b3d9697d95 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:39.116712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e0b18077-38e3-4ec5-b124-f9b3d9697d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:39.146659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:39.201552 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2278/4511] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:39.213516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:39.215962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:39.291151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10dac47-a2f9-434e-a055-0535c0fcb949 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:39.295238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99dfb2-360d-4db4-ac0f-a6f08f6deb4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:39.295996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99dfb2-360d-4db4-ac0f-a6f08f6deb4c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:39.296439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99dfb2-360d-4db4-ac0f-a6f08f6deb4c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:39.296708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99dfb2-360d-4db4-ac0f-a6f08f6deb4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:39.310038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99dfb2-360d-4db4-ac0f-a6f08f6deb4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:39.311787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99dfb2-360d-4db4-ac0f-a6f08f6deb4c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:39.327240 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2234/4512] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:39.353207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b18077-38e3-4ec5-b124-f9b3d9697d95 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:39.357730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e0b18077-38e3-4ec5-b124-f9b3d9697d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:39.379476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b18077-38e3-4ec5-b124-f9b3d9697d95 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:39.391832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:39.395535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:39.396174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:39.396731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e admin admin] PATH_INFO: `/v3/projects/7e5d6bd9f7d943d189a7211ef7c0b4d1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:39.404917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e5d6bd9f7d943d189a7211ef7c0b4d1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:39.412609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:39.530057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e5d6bd9f7d943d189a7211ef7c0b4d1', 'request_id': 'req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:39.546052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e5d6bd9f7d943d189a7211ef7c0b4d1', 'request_id': 'req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:39.565940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e5d6bd9f7d943d189a7211ef7c0b4d1', 'request_id': 'req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:39.607116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e admin admin] The token cache is being invalidate because project 7e5d6bd9f7d943d189a7211ef7c0b4d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:39.608189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:39.609208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8916b84d-24ed-4bbf-a1a7-ada104ccac2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:39.614974 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2279/4513] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:39 2022] DELETE /identity/v3/projects/7e5d6bd9f7d943d189a7211ef7c0b4d1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:39.621527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6874a-0bcc-4e54-b395-71036208a095 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:39.622121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6874a-0bcc-4e54-b395-71036208a095 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:39.624139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6874a-0bcc-4e54-b395-71036208a095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:39.649163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7ae6874a-0bcc-4e54-b395-71036208a095 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:39.650453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7ae6874a-0bcc-4e54-b395-71036208a095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:39.692041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:39.738210 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2235/4514] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:39.750280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a088d4e-4a7b-4814-b94e-9eb1fc74fecf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:39.750722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a088d4e-4a7b-4814-b94e-9eb1fc74fecf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:39.751359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a088d4e-4a7b-4814-b94e-9eb1fc74fecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:39.786781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5a088d4e-4a7b-4814-b94e-9eb1fc74fecf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:39.788304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5a088d4e-4a7b-4814-b94e-9eb1fc74fecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:39.793554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148cc7c-946c-4a89-af5b-6cdbc0ff9d8f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:39.794014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148cc7c-946c-4a89-af5b-6cdbc0ff9d8f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:39.796733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148cc7c-946c-4a89-af5b-6cdbc0ff9d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:39.801885 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2280/4515] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:39.814440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a088d4e-4a7b-4814-b94e-9eb1fc74fecf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:39.819689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a148cc7c-946c-4a89-af5b-6cdbc0ff9d8f None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:39.821529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a148cc7c-946c-4a89-af5b-6cdbc0ff9d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:39.830656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a088d4e-4a7b-4814-b94e-9eb1fc74fecf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:39.844582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce8d4a-7172-45f9-bad6-f0dee99a4d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:39.853033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce8d4a-7172-45f9-bad6-f0dee99a4d1e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:39.853487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce8d4a-7172-45f9-bad6-f0dee99a4d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:39.854488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce8d4a-7172-45f9-bad6-f0dee99a4d1e admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:39.855187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce8d4a-7172-45f9-bad6-f0dee99a4d1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:39.855546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce8d4a-7172-45f9-bad6-f0dee99a4d1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:39.864041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce8d4a-7172-45f9-bad6-f0dee99a4d1e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:39.902171 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2236/4516] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:09:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:39.903162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148cc7c-946c-4a89-af5b-6cdbc0ff9d8f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:39.904714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a148cc7c-946c-4a89-af5b-6cdbc0ff9d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:39.912197 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2281/4517] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:39.916896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce8d4a-7172-45f9-bad6-f0dee99a4d1e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:39.932308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148cc7c-946c-4a89-af5b-6cdbc0ff9d8f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:39.937888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:39.938673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:39.939079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce8d4a-7172-45f9-bad6-f0dee99a4d1e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:39.939787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:39.940189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] PATH_INFO: `/v3/projects/78f31458e1d841108f753773b78e1a14` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:39.941550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] RBAC: Authorizing `identity:delete_project(project_id=78f31458e1d841108f753773b78e1a14)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:39.945061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:39.948094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac4a88-f49d-4c3d-b903-27d2d61834ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:39.949157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac4a88-f49d-4c3d-b903-27d2d61834ed admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:39.949610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac4a88-f49d-4c3d-b903-27d2d61834ed admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:39.950040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac4a88-f49d-4c3d-b903-27d2d61834ed admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:39.951235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac4a88-f49d-4c3d-b903-27d2d61834ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:39.955041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac4a88-f49d-4c3d-b903-27d2d61834ed admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:40.012527 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2282/4518] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:39 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:40.013436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac4a88-f49d-4c3d-b903-27d2d61834ed admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:40.015142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-48ac4a88-f49d-4c3d-b903-27d2d61834ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:40.016630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78f31458e1d841108f753773b78e1a14', 'request_id': 'req-b6354b36-74cd-49c9-9204-1eb9ecefd12b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:40.023990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78f31458e1d841108f753773b78e1a14', 'request_id': 'req-b6354b36-74cd-49c9-9204-1eb9ecefd12b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:40.033307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78f31458e1d841108f753773b78e1a14', 'request_id': 'req-b6354b36-74cd-49c9-9204-1eb9ecefd12b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:40.045191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] The token cache is being invalidate because project 78f31458e1d841108f753773b78e1a14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:40.045545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:40.046238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:40.047850 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2237/4519] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:39 2022] DELETE /identity/v3/projects/78f31458e1d841108f753773b78e1a14 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:40.050950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:40.050950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:40.088994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac4a88-f49d-4c3d-b903-27d2d61834ed admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:40.094165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bc5c8-4b38-4ee6-b805-008fbdd2f31b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:40.094868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bc5c8-4b38-4ee6-b805-008fbdd2f31b service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:40.095211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bc5c8-4b38-4ee6-b805-008fbdd2f31b service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:40.095541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bc5c8-4b38-4ee6-b805-008fbdd2f31b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:40.097516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-670bc5c8-4b38-4ee6-b805-008fbdd2f31b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:40.109663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:40.119428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6354b36-74cd-49c9-9204-1eb9ecefd12b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:40.129463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:40.166986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bc5c8-4b38-4ee6-b805-008fbdd2f31b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:40.168613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bc5c8-4b38-4ee6-b805-008fbdd2f31b service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:40.178389 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2283/4520] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:09:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:09:40.179240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bc5c8-4b38-4ee6-b805-008fbdd2f31b service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:40.180648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-670bc5c8-4b38-4ee6-b805-008fbdd2f31b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:40.198308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df7e9-312b-4006-9acf-b28d8e2332d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:40.199081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df7e9-312b-4006-9acf-b28d8e2332d9 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:40.199485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df7e9-312b-4006-9acf-b28d8e2332d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:40.199836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df7e9-312b-4006-9acf-b28d8e2332d9 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:40.200565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45df7e9-312b-4006-9acf-b28d8e2332d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:40.202647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45df7e9-312b-4006-9acf-b28d8e2332d9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:40.236412 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2238/4521] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:40 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:40.240578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df7e9-312b-4006-9acf-b28d8e2332d9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:40.252780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df7e9-312b-4006-9acf-b28d8e2332d9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:40.258220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c9085-046b-40d7-bdca-11be6d3a9c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:40.258835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c9085-046b-40d7-bdca-11be6d3a9c15 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:40.259156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c9085-046b-40d7-bdca-11be6d3a9c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:40.259475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c9085-046b-40d7-bdca-11be6d3a9c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:40.260127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c9085-046b-40d7-bdca-11be6d3a9c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:40.261761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c9085-046b-40d7-bdca-11be6d3a9c15 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:40.266817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bc5c8-4b38-4ee6-b805-008fbdd2f31b service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:40.272242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:40.272628 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2239/4522] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:40.272949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:40.273198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:40.273460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] PATH_INFO: `/v3/users/9512f567658e4cbdb0e8830178105a37` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:40.274640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] RBAC: Authorizing `identity:delete_user(user_id=9512f567658e4cbdb0e8830178105a37)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:40.276216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c9085-046b-40d7-bdca-11be6d3a9c15 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:40.276671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:40.287726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c9085-046b-40d7-bdca-11be6d3a9c15 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:40.292748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ce8a6-ece4-418c-8ec5-83281eb0d3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:40.293367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ce8a6-ece4-418c-8ec5-83281eb0d3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:40.293622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ce8a6-ece4-418c-8ec5-83281eb0d3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:40.293892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ce8a6-ece4-418c-8ec5-83281eb0d3d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:40.294773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ce8a6-ece4-418c-8ec5-83281eb0d3d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:40.296557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ce8a6-ece4-418c-8ec5-83281eb0d3d3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:40.309999 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2240/4523] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:40.317187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ce8a6-ece4-418c-8ec5-83281eb0d3d3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:40.328783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ce8a6-ece4-418c-8ec5-83281eb0d3d3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:40.334437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba138c4-8fc1-405f-b62b-cb0d0368e8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:40.335052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba138c4-8fc1-405f-b62b-cb0d0368e8b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:40.335323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba138c4-8fc1-405f-b62b-cb0d0368e8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:40.335823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba138c4-8fc1-405f-b62b-cb0d0368e8b5 admin admin] PATH_INFO: `/v3/projects/86f7990dfbdb407e88e2ddf7623fe307/users/36768cfb8ede43a5bb9a4c2ff1e8d311/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:40.336703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9512f567658e4cbdb0e8830178105a37'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:40.347950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9512f567658e4cbdb0e8830178105a37'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:40.356741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba138c4-8fc1-405f-b62b-cb0d0368e8b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f7990dfbdb407e88e2ddf7623fe307, user_id=36768cfb8ede43a5bb9a4c2ff1e8d311, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:40.357153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9512f567658e4cbdb0e8830178105a37'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:40.359943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba138c4-8fc1-405f-b62b-cb0d0368e8b5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:40.370699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9512f567658e4cbdb0e8830178105a37'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:40.371501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9512f567658e4cbdb0e8830178105a37'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:40.392403 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2241/4524] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:40 2022] PUT /identity/v3/projects/86f7990dfbdb407e88e2ddf7623fe307/users/36768cfb8ede43a5bb9a4c2ff1e8d311/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:40.393401 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2284/4525] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:40 2022] DELETE /identity/v3/users/9512f567658e4cbdb0e8830178105a37 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:40.464045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df15d5-da4c-4239-8bb5-6ec0a08d6cbe None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:40.464356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df15d5-da4c-4239-8bb5-6ec0a08d6cbe None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:40.464356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df15d5-da4c-4239-8bb5-6ec0a08d6cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:40.499630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-86df15d5-da4c-4239-8bb5-6ec0a08d6cbe None None] MFA Rules not processed for user `36768cfb8ede43a5bb9a4c2ff1e8d311`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:40.500874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-86df15d5-da4c-4239-8bb5-6ec0a08d6cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:40.578714 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2242/4526] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:40.583369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:40.585054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:40.622142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:40.662708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b97b0e-5a22-4984-a814-54e03b46feff admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:40.673022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25aa8d4-7b3c-4bcb-84b6-017bb384f076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:40.673731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25aa8d4-7b3c-4bcb-84b6-017bb384f076 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:40.674180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25aa8d4-7b3c-4bcb-84b6-017bb384f076 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:40.674551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25aa8d4-7b3c-4bcb-84b6-017bb384f076 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:40.675241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25aa8d4-7b3c-4bcb-84b6-017bb384f076 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:40.678633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25aa8d4-7b3c-4bcb-84b6-017bb384f076 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:40.724503 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2285/4527] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:40 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:40.725940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df15d5-da4c-4239-8bb5-6ec0a08d6cbe None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:40.739324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25aa8d4-7b3c-4bcb-84b6-017bb384f076 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:40.741214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df15d5-da4c-4239-8bb5-6ec0a08d6cbe None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:40.743478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a25aa8d4-7b3c-4bcb-84b6-017bb384f076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:40.838384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:40.854013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d0c86-3b05-43ba-8123-b7945ac9a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:40.854818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d0c86-3b05-43ba-8123-b7945ac9a6ee admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:40.855195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d0c86-3b05-43ba-8123-b7945ac9a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:40.855562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d0c86-3b05-43ba-8123-b7945ac9a6ee admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:40.856489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d0c86-3b05-43ba-8123-b7945ac9a6ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:40.858725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d0c86-3b05-43ba-8123-b7945ac9a6ee admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:40.873711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25aa8d4-7b3c-4bcb-84b6-017bb384f076 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:40.882073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b80b9e-222e-4d30-978e-77d1704d4d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:40.883052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b80b9e-222e-4d30-978e-77d1704d4d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:40.883354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b80b9e-222e-4d30-978e-77d1704d4d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:40.883619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b80b9e-222e-4d30-978e-77d1704d4d5a admin admin] PATH_INFO: `/v3/projects/64a9a01c8a234babb6bb1e93e4f70ad9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:40.884914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b80b9e-222e-4d30-978e-77d1704d4d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a9a01c8a234babb6bb1e93e4f70ad9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:40.888327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b80b9e-222e-4d30-978e-77d1704d4d5a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:40.896436 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2243/4528] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:40 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:40.899665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d0c86-3b05-43ba-8123-b7945ac9a6ee admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:40.917265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d0c86-3b05-43ba-8123-b7945ac9a6ee admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:40.925462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287ffe2-9399-4bb9-9c05-1feba0fe7a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:40.926356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287ffe2-9399-4bb9-9c05-1feba0fe7a9f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:40.927020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287ffe2-9399-4bb9-9c05-1feba0fe7a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:40.927020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287ffe2-9399-4bb9-9c05-1feba0fe7a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:40.928006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287ffe2-9399-4bb9-9c05-1feba0fe7a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:40.929727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287ffe2-9399-4bb9-9c05-1feba0fe7a9f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:40.943901 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2244/4529] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:40.948816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287ffe2-9399-4bb9-9c05-1feba0fe7a9f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:40.957142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5b80b9e-222e-4d30-978e-77d1704d4d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a9a01c8a234babb6bb1e93e4f70ad9', 'request_id': 'req-c5b80b9e-222e-4d30-978e-77d1704d4d5a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:40.973148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287ffe2-9399-4bb9-9c05-1feba0fe7a9f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:40.974830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5b80b9e-222e-4d30-978e-77d1704d4d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a9a01c8a234babb6bb1e93e4f70ad9', 'request_id': 'req-c5b80b9e-222e-4d30-978e-77d1704d4d5a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:41.008164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5b80b9e-222e-4d30-978e-77d1704d4d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a9a01c8a234babb6bb1e93e4f70ad9', 'request_id': 'req-c5b80b9e-222e-4d30-978e-77d1704d4d5a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:41.031211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5b80b9e-222e-4d30-978e-77d1704d4d5a admin admin] The token cache is being invalidate because project 64a9a01c8a234babb6bb1e93e4f70ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:41.031211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5b80b9e-222e-4d30-978e-77d1704d4d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:41.031928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5b80b9e-222e-4d30-978e-77d1704d4d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:41.049548 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2286/4530] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:40 2022] DELETE /identity/v3/projects/64a9a01c8a234babb6bb1e93e4f70ad9 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:41.056070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ee31f-9b16-47d2-bc6a-fd3906ec9373 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.056448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ee31f-9b16-47d2-bc6a-fd3906ec9373 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.057344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ee31f-9b16-47d2-bc6a-fd3906ec9373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.089252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e0c3f-56b4-4f4a-8bf7-3302060b9bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.090183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e0c3f-56b4-4f4a-8bf7-3302060b9bc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.090631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e0c3f-56b4-4f4a-8bf7-3302060b9bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.091069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e0c3f-56b4-4f4a-8bf7-3302060b9bc3 admin admin] PATH_INFO: `/v3/projects/bafac58f22974dbf81a10ec5527b6ac2/users/d806161de6ce47f394a21ca7a044f8d2/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.100776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9c9ee31f-9b16-47d2-bc6a-fd3906ec9373 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:41.102049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9c9ee31f-9b16-47d2-bc6a-fd3906ec9373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:41.113420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e0c3f-56b4-4f4a-8bf7-3302060b9bc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bafac58f22974dbf81a10ec5527b6ac2, user_id=d806161de6ce47f394a21ca7a044f8d2, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.117217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e0c3f-56b4-4f4a-8bf7-3302060b9bc3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.118085 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2287/4531] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:41.119558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adba89a-0df9-4813-936d-07e4342b3a43 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.119836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adba89a-0df9-4813-936d-07e4342b3a43 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.120088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adba89a-0df9-4813-936d-07e4342b3a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.135629 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2245/4532] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:40 2022] PUT /identity/v3/projects/bafac58f22974dbf81a10ec5527b6ac2/users/d806161de6ce47f394a21ca7a044f8d2/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:41.139433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e0c3f-56b4-4f4a-8bf7-3302060b9bc3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.140918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5adba89a-0df9-4813-936d-07e4342b3a43 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:41.141085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-600e0c3f-56b4-4f4a-8bf7-3302060b9bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:41.142076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5adba89a-0df9-4813-936d-07e4342b3a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:41.184796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:41.229947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e0c3f-56b4-4f4a-8bf7-3302060b9bc3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.236027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c97f5-f5b4-4508-beb5-53f260b32349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.236697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c97f5-f5b4-4508-beb5-53f260b32349 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.236970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c97f5-f5b4-4508-beb5-53f260b32349 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.237335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c97f5-f5b4-4508-beb5-53f260b32349 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.237881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c97f5-f5b4-4508-beb5-53f260b32349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.239698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c97f5-f5b4-4508-beb5-53f260b32349 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.255134 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2288/4533] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:41.259557 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2246/4534] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:41.259930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adba89a-0df9-4813-936d-07e4342b3a43 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.260501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c97f5-f5b4-4508-beb5-53f260b32349 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.270575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adba89a-0df9-4813-936d-07e4342b3a43 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.271507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c97f5-f5b4-4508-beb5-53f260b32349 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.275492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086f311-a567-43bc-891a-99a5fe217b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.276223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086f311-a567-43bc-891a-99a5fe217b69 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.276539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086f311-a567-43bc-891a-99a5fe217b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.277186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d565c-5806-4957-a408-c65a67b1d8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.277357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086f311-a567-43bc-891a-99a5fe217b69 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.277461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d565c-5806-4957-a408-c65a67b1d8d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.277549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086f311-a567-43bc-891a-99a5fe217b69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:41.277809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086f311-a567-43bc-891a-99a5fe217b69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.277923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d565c-5806-4957-a408-c65a67b1d8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.278098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d565c-5806-4957-a408-c65a67b1d8d9 admin admin] PATH_INFO: `/v3/projects/bafac58f22974dbf81a10ec5527b6ac2/users/d806161de6ce47f394a21ca7a044f8d2/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.279303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086f311-a567-43bc-891a-99a5fe217b69 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.282374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d565c-5806-4957-a408-c65a67b1d8d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bafac58f22974dbf81a10ec5527b6ac2, user_id=d806161de6ce47f394a21ca7a044f8d2, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.285217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d565c-5806-4957-a408-c65a67b1d8d9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.300466 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2289/4535] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:41.300673 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2247/4536] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:41 2022] PUT /identity/v3/projects/bafac58f22974dbf81a10ec5527b6ac2/users/d806161de6ce47f394a21ca7a044f8d2/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:41.305542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d565c-5806-4957-a408-c65a67b1d8d9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.322668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd95df1-cb8f-494a-827d-43140db9de80 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.323195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd95df1-cb8f-494a-827d-43140db9de80 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.323462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd95df1-cb8f-494a-827d-43140db9de80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.343130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:41.354078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1cd95df1-cb8f-494a-827d-43140db9de80 None None] MFA Rules not processed for user `d806161de6ce47f394a21ca7a044f8d2`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:41.355404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1cd95df1-cb8f-494a-827d-43140db9de80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:41.386497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d565c-5806-4957-a408-c65a67b1d8d9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.391979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbec5d-39f7-4261-abc9-da1af7e95d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.392628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbec5d-39f7-4261-abc9-da1af7e95d38 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.392903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbec5d-39f7-4261-abc9-da1af7e95d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.393842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbec5d-39f7-4261-abc9-da1af7e95d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.394481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbec5d-39f7-4261-abc9-da1af7e95d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.396462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbec5d-39f7-4261-abc9-da1af7e95d38 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.429412 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2248/4537] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:41 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:41.430309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbec5d-39f7-4261-abc9-da1af7e95d38 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.442483 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2290/4538] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:41.447838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd95df1-cb8f-494a-827d-43140db9de80 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.458529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd95df1-cb8f-494a-827d-43140db9de80 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.500602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbec5d-39f7-4261-abc9-da1af7e95d38 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.508110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b55ef8-b68c-481c-9e2e-25de1434e38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.508862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b55ef8-b68c-481c-9e2e-25de1434e38c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.509245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b55ef8-b68c-481c-9e2e-25de1434e38c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.509577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b55ef8-b68c-481c-9e2e-25de1434e38c admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.510286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b55ef8-b68c-481c-9e2e-25de1434e38c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.512704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b55ef8-b68c-481c-9e2e-25de1434e38c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.525727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d835b-ae1c-4cd9-b570-e563f8fbccf7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd806161de6ce47f394a21ca7a044f8d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bafac58f22974dbf81a10ec5527b6ac2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.526436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d835b-ae1c-4cd9-b570-e563f8fbccf7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.526684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d835b-ae1c-4cd9-b570-e563f8fbccf7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.527106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d835b-ae1c-4cd9-b570-e563f8fbccf7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.527657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d835b-ae1c-4cd9-b570-e563f8fbccf7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.529355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d835b-ae1c-4cd9-b570-e563f8fbccf7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-2e1d835b-ae1c-4cd9-b570-e563f8fbccf7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invalid input for field 'name': '' is too short Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: Failed validating 'minLength' in schema['properties']['name']: Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: On instance['name']: Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: Failed validating 'minLength' in schema['properties']['name']: Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: On instance['name']: Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: '' Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise error Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application On instance['name']: Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application '' Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.536454 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Aug 20 21:09:41.538559 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Aug 20 21:09:41.539737 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Aug 20 21:09:41.539737 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Aug 20 21:09:41.539737 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.539737 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Aug 20 21:09:41.539737 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:41.539737 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.539737 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application On instance['name']: Aug 20 21:09:41.539737 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application '' Aug 20 21:09:41.539737 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.539737 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2291/4539] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:09:41 2022] POST /identity/v3/domains => generated 283 bytes in 90 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Aug 20 21:09:41.541155 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2249/4540] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:41 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:41.542628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d835b-ae1c-4cd9-b570-e563f8fbccf7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.542761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b55ef8-b68c-481c-9e2e-25de1434e38c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.554148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b55ef8-b68c-481c-9e2e-25de1434e38c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.554944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d835b-ae1c-4cd9-b570-e563f8fbccf7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.559748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe83dc-5cd2-4374-a2ea-114f6960289e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.560270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827b6d4-a34a-48e4-a0ba-9c8d0b9e6953 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd806161de6ce47f394a21ca7a044f8d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bafac58f22974dbf81a10ec5527b6ac2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.560459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe83dc-5cd2-4374-a2ea-114f6960289e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.560660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe83dc-5cd2-4374-a2ea-114f6960289e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.560926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe83dc-5cd2-4374-a2ea-114f6960289e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.561271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827b6d4-a34a-48e4-a0ba-9c8d0b9e6953 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.561606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe83dc-5cd2-4374-a2ea-114f6960289e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.561725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827b6d4-a34a-48e4-a0ba-9c8d0b9e6953 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.562198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827b6d4-a34a-48e4-a0ba-9c8d0b9e6953 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.562946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827b6d4-a34a-48e4-a0ba-9c8d0b9e6953 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.563271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe83dc-5cd2-4374-a2ea-114f6960289e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.564755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827b6d4-a34a-48e4-a0ba-9c8d0b9e6953 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-e827b6d4-a34a-48e4-a0ba-9c8d0b9e6953 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: Failed validating 'maxLength' in schema['properties']['name']: Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: On instance['name']: Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: Failed validating 'maxLength' in schema['properties']['name']: Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: On instance['name']: Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise error Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:41.572149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application On instance['name']: Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Aug 20 21:09:41.573964 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Aug 20 21:09:41.575450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Aug 20 21:09:41.575450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Aug 20 21:09:41.575450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Aug 20 21:09:41.575450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Aug 20 21:09:41.575450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Aug 20 21:09:41.575450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.575450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Aug 20 21:09:41.575450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Aug 20 21:09:41.575450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.575450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application On instance['name']: Aug 20 21:09:41.575450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Aug 20 21:09:41.575450 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:41.576243 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2250/4541] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:41.576385 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2292/4542] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:09:41 2022] POST /identity/v3/domains => generated 412 bytes in 33 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 20 21:09:41.580554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827b6d4-a34a-48e4-a0ba-9c8d0b9e6953 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.580836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe83dc-5cd2-4374-a2ea-114f6960289e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.592056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe83dc-5cd2-4374-a2ea-114f6960289e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.593618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827b6d4-a34a-48e4-a0ba-9c8d0b9e6953 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.596924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3982a-09c7-4eb6-9fc2-d1bea6a6b2f7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd806161de6ce47f394a21ca7a044f8d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bafac58f22974dbf81a10ec5527b6ac2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.597635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3982a-09c7-4eb6-9fc2-d1bea6a6b2f7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.597975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3982a-09c7-4eb6-9fc2-d1bea6a6b2f7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.598343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3982a-09c7-4eb6-9fc2-d1bea6a6b2f7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.598525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b425574-da46-4a73-bbcb-39a19072c9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.599117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3982a-09c7-4eb6-9fc2-d1bea6a6b2f7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.599241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b425574-da46-4a73-bbcb-39a19072c9fc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.599494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b425574-da46-4a73-bbcb-39a19072c9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.599753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b425574-da46-4a73-bbcb-39a19072c9fc admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.600390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b425574-da46-4a73-bbcb-39a19072c9fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.601174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3982a-09c7-4eb6-9fc2-d1bea6a6b2f7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.601979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b425574-da46-4a73-bbcb-39a19072c9fc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.613504 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2293/4543] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:41.617577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b425574-da46-4a73-bbcb-39a19072c9fc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.618842 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2251/4544] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:09:41 2022] POST /identity/v3/domains => generated 276 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:41.622973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3982a-09c7-4eb6-9fc2-d1bea6a6b2f7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.627696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b425574-da46-4a73-bbcb-39a19072c9fc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.660343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:41.684000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b64775-2a96-45ea-984c-04b3f2ced56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.684752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b64775-2a96-45ea-984c-04b3f2ced56d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.685025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b64775-2a96-45ea-984c-04b3f2ced56d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.685275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b64775-2a96-45ea-984c-04b3f2ced56d admin admin] PATH_INFO: `/v3/projects/30647907e07c41fea7a2c713d5829814/users/e023b25bcb984a1288f523b9c331dfcc/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.688808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3982a-09c7-4eb6-9fc2-d1bea6a6b2f7 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.693630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de36479-5700-4de9-9000-2a6786a425a6 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd806161de6ce47f394a21ca7a044f8d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bafac58f22974dbf81a10ec5527b6ac2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.694263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de36479-5700-4de9-9000-2a6786a425a6 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.694534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de36479-5700-4de9-9000-2a6786a425a6 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.694828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de36479-5700-4de9-9000-2a6786a425a6 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] PATH_INFO: `/v3/domains/469afa945ab548d0a5c9dc530d2dcaf3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.700300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b64775-2a96-45ea-984c-04b3f2ced56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=30647907e07c41fea7a2c713d5829814, user_id=e023b25bcb984a1288f523b9c331dfcc, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.703008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b64775-2a96-45ea-984c-04b3f2ced56d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.706727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de36479-5700-4de9-9000-2a6786a425a6 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=469afa945ab548d0a5c9dc530d2dcaf3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.708520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de36479-5700-4de9-9000-2a6786a425a6 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.716682 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2294/4545] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:41 2022] PUT /identity/v3/projects/30647907e07c41fea7a2c713d5829814/users/e023b25bcb984a1288f523b9c331dfcc/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-5de36479-5700-4de9-9000-2a6786a425a6 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Cannot delete a domain that is enabled, please disable it first.: keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 151, in delete Aug 20 21:09:41.722937 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_domain( Aug 20 21:09:41.724268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:41.724268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:41.724268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 802, in delete_domain Aug 20 21:09:41.724268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._delete_domain(domain, initiator) Aug 20 21:09:41.724268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 815, in _delete_domain Aug 20 21:09:41.724268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Aug 20 21:09:41.724268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Aug 20 21:09:41.724268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:41.725426 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2252/4546] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:41 2022] DELETE /identity/v3/domains/469afa945ab548d0a5c9dc530d2dcaf3 => generated 120 bytes in 103 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Aug 20 21:09:41.727313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce93a6d-fb52-4fcc-a12b-03e30c3d833d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.727640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce93a6d-fb52-4fcc-a12b-03e30c3d833d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.727944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce93a6d-fb52-4fcc-a12b-03e30c3d833d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.729146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de36479-5700-4de9-9000-2a6786a425a6 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.739148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de36479-5700-4de9-9000-2a6786a425a6 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.742953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:41.756920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0ce93a6d-fb52-4fcc-a12b-03e30c3d833d None None] MFA Rules not processed for user `e023b25bcb984a1288f523b9c331dfcc`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:41.758049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0ce93a6d-fb52-4fcc-a12b-03e30c3d833d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:41.798644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeeb5b7-ad7a-42ae-9f57-12254e015867 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd806161de6ce47f394a21ca7a044f8d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bafac58f22974dbf81a10ec5527b6ac2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.799273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb5b7-ad7a-42ae-9f57-12254e015867 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.799557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb5b7-ad7a-42ae-9f57-12254e015867 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.800116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb5b7-ad7a-42ae-9f57-12254e015867 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] PATH_INFO: `/v3/domains/469afa945ab548d0a5c9dc530d2dcaf3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.801754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeeb5b7-ad7a-42ae-9f57-12254e015867 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=469afa945ab548d0a5c9dc530d2dcaf3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.803396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeeb5b7-ad7a-42ae-9f57-12254e015867 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.823112 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2295/4547] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:41.825519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-daeeb5b7-ad7a-42ae-9f57-12254e015867 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '469afa945ab548d0a5c9dc530d2dcaf3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:41.826242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-daeeb5b7-ad7a-42ae-9f57-12254e015867 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '469afa945ab548d0a5c9dc530d2dcaf3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:41.828191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce93a6d-fb52-4fcc-a12b-03e30c3d833d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.836875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce93a6d-fb52-4fcc-a12b-03e30c3d833d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.856780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-daeeb5b7-ad7a-42ae-9f57-12254e015867 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '469afa945ab548d0a5c9dc530d2dcaf3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:41.857488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-daeeb5b7-ad7a-42ae-9f57-12254e015867 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '469afa945ab548d0a5c9dc530d2dcaf3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:41.860250 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2253/4548] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:09:41 2022] PATCH /identity/v3/domains/469afa945ab548d0a5c9dc530d2dcaf3 => generated 277 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:41.864506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeeb5b7-ad7a-42ae-9f57-12254e015867 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.865866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-daeeb5b7-ad7a-42ae-9f57-12254e015867 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:41.898559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e412b3-4550-45b2-bd9b-9edb419ed81c tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e023b25bcb984a1288f523b9c331dfcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30647907e07c41fea7a2c713d5829814', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.899189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e412b3-4550-45b2-bd9b-9edb419ed81c tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.899460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e412b3-4550-45b2-bd9b-9edb419ed81c tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.899737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e412b3-4550-45b2-bd9b-9edb419ed81c tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.900327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e412b3-4550-45b2-bd9b-9edb419ed81c tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=e023b25bcb984a1288f523b9c331dfcc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.901838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e412b3-4550-45b2-bd9b-9edb419ed81c tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.928423 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2296/4549] 10.222.0.23 () {60 vars in 1299 bytes} [Sat Aug 20 21:09:41 2022] POST /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2 => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:41.931234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e412b3-4550-45b2-bd9b-9edb419ed81c tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.933303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-27e412b3-4550-45b2-bd9b-9edb419ed81c tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:41.938684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeeb5b7-ad7a-42ae-9f57-12254e015867 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.943642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd806161de6ce47f394a21ca7a044f8d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bafac58f22974dbf81a10ec5527b6ac2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.944276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.944598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.944865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] PATH_INFO: `/v3/domains/469afa945ab548d0a5c9dc530d2dcaf3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.944987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e412b3-4550-45b2-bd9b-9edb419ed81c tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.949619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac3612-f514-4280-92f2-53cc6265fb35 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e023b25bcb984a1288f523b9c331dfcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30647907e07c41fea7a2c713d5829814', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.950253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac3612-f514-4280-92f2-53cc6265fb35 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.950528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac3612-f514-4280-92f2-53cc6265fb35 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.950798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac3612-f514-4280-92f2-53cc6265fb35 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/c96b9eaebf1d472c8dc45ca305142c6f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.952792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac3612-f514-4280-92f2-53cc6265fb35 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=e023b25bcb984a1288f523b9c331dfcc, credential_id=c96b9eaebf1d472c8dc45ca305142c6f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.954239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac3612-f514-4280-92f2-53cc6265fb35 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.955915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=469afa945ab548d0a5c9dc530d2dcaf3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.957539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:41.972805 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2297/4550] 10.222.0.23 () {58 vars in 1380 bytes} [Sat Aug 20 21:09:41 2022] DELETE /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/c96b9eaebf1d472c8dc45ca305142c6f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:41.977997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac3612-f514-4280-92f2-53cc6265fb35 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:41.982240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '469afa945ab548d0a5c9dc530d2dcaf3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:41.985795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '469afa945ab548d0a5c9dc530d2dcaf3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:41.987939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac3612-f514-4280-92f2-53cc6265fb35 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:41.992874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb147e3-4b17-48ef-95f7-7b2b1a6773d8 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e023b25bcb984a1288f523b9c331dfcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30647907e07c41fea7a2c713d5829814', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:41.993496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb147e3-4b17-48ef-95f7-7b2b1a6773d8 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:41.993775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb147e3-4b17-48ef-95f7-7b2b1a6773d8 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:41.994045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb147e3-4b17-48ef-95f7-7b2b1a6773d8 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:41.994666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb147e3-4b17-48ef-95f7-7b2b1a6773d8 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=e023b25bcb984a1288f523b9c331dfcc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:41.996132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb147e3-4b17-48ef-95f7-7b2b1a6773d8 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.016407 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2298/4551] 10.222.0.23 () {60 vars in 1299 bytes} [Sat Aug 20 21:09:41 2022] POST /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2 => generated 367 bytes in 36 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:42.017616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb147e3-4b17-48ef-95f7-7b2b1a6773d8 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.027844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb147e3-4b17-48ef-95f7-7b2b1a6773d8 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.033550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffb84e-b11d-47c8-9c1c-4582d03ce497 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e023b25bcb984a1288f523b9c331dfcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30647907e07c41fea7a2c713d5829814', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.034217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffb84e-b11d-47c8-9c1c-4582d03ce497 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.034490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffb84e-b11d-47c8-9c1c-4582d03ce497 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.034759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffb84e-b11d-47c8-9c1c-4582d03ce497 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/d5275d1a1ada4445842897dcda4f8780` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.036892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffb84e-b11d-47c8-9c1c-4582d03ce497 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=e023b25bcb984a1288f523b9c331dfcc, credential_id=d5275d1a1ada4445842897dcda4f8780)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:42.038493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffb84e-b11d-47c8-9c1c-4582d03ce497 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.044466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:42.055063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '469afa945ab548d0a5c9dc530d2dcaf3', 'request_id': 'req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.069046 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2299/4552] 10.222.0.23 () {58 vars in 1380 bytes} [Sat Aug 20 21:09:42 2022] DELETE /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/d5275d1a1ada4445842897dcda4f8780 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:42.069980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffb84e-b11d-47c8-9c1c-4582d03ce497 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.072278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '469afa945ab548d0a5c9dc530d2dcaf3', 'request_id': 'req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.077808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:42.082287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffb84e-b11d-47c8-9c1c-4582d03ce497 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.088160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '469afa945ab548d0a5c9dc530d2dcaf3', 'request_id': 'req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.109956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] The token cache is being invalidate because project 469afa945ab548d0a5c9dc530d2dcaf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:42.110248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.111015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.126519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '469afa945ab548d0a5c9dc530d2dcaf3', 'request_id': 'req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.128176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '469afa945ab548d0a5c9dc530d2dcaf3', 'request_id': 'req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.134286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '469afa945ab548d0a5c9dc530d2dcaf3', 'request_id': 'req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.142333 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2254/4553] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:41 2022] DELETE /identity/v3/domains/469afa945ab548d0a5c9dc530d2dcaf3 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:42.152481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.154744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:42.160456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18db7f7-534c-415f-bbde-0cfb65f6a51a tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e023b25bcb984a1288f523b9c331dfcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30647907e07c41fea7a2c713d5829814', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.161561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18db7f7-534c-415f-bbde-0cfb65f6a51a tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.161977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18db7f7-534c-415f-bbde-0cfb65f6a51a tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.162320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18db7f7-534c-415f-bbde-0cfb65f6a51a tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/d5275d1a1ada4445842897dcda4f8780` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-d18db7f7-534c-415f-bbde-0cfb65f6a51a tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Could not find credential: f13cd093e59bb4b9183f3b4e002cd888ee5380e0bd9f95799ef4604f3fb8e911.: keystone.exception.CredentialNotFound: Could not find credential: f13cd093e59bb4b9183f3b4e002cd888ee5380e0bd9f95799ef4604f3fb8e911. Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 426, in get Aug 20 21:09:42.181682 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 119, in _build_enforcer_target_data_owner_and_user_id_match Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref['credential'] = PROVIDERS.credential_api.get_credential( Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 119, in get_credential Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application credential = self._get_credential(credential_id) Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:09:42.183065 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 124, in _get_credential Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.driver.get_credential(credential_id) Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 88, in get_credential Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_credential(session, credential_id).to_dict() Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 83, in _get_credential Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.CredentialNotFound(credential_id=credential_id) Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.CredentialNotFound: Could not find credential: f13cd093e59bb4b9183f3b4e002cd888ee5380e0bd9f95799ef4604f3fb8e911. Aug 20 21:09:42.185527 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:42.200849 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2300/4554] 10.222.0.23 () {58 vars in 1377 bytes} [Sat Aug 20 21:09:42 2022] GET /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/d5275d1a1ada4445842897dcda4f8780 => generated 148 bytes in 120 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Aug 20 21:09:42.200849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18db7f7-534c-415f-bbde-0cfb65f6a51a tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.200849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d18db7f7-534c-415f-bbde-0cfb65f6a51a tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:42.238828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18db7f7-534c-415f-bbde-0cfb65f6a51a tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.246390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec69ac-0c78-42a9-9cdc-7bdb853113e1 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e023b25bcb984a1288f523b9c331dfcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30647907e07c41fea7a2c713d5829814', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.247186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec69ac-0c78-42a9-9cdc-7bdb853113e1 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.247556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec69ac-0c78-42a9-9cdc-7bdb853113e1 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.247912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec69ac-0c78-42a9-9cdc-7bdb853113e1 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.248667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec69ac-0c78-42a9-9cdc-7bdb853113e1 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=e023b25bcb984a1288f523b9c331dfcc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:42.250270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec69ac-0c78-42a9-9cdc-7bdb853113e1 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.257768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e9132-cd3c-4c84-b4e4-9207b0ddc17d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.262914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3068d-195b-4c6b-a0e9-505758d42b72 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd806161de6ce47f394a21ca7a044f8d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bafac58f22974dbf81a10ec5527b6ac2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.263953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3068d-195b-4c6b-a0e9-505758d42b72 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.264133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3068d-195b-4c6b-a0e9-505758d42b72 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.264679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3068d-195b-4c6b-a0e9-505758d42b72 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] PATH_INFO: `/v3/domains/fcfd2dda36854bf6b1e6c0e815a34765` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.274170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3068d-195b-4c6b-a0e9-505758d42b72 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-a4a3068d-195b-4c6b-a0e9-505758d42b72 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Could not find domain: fcfd2dda36854bf6b1e6c0e815a34765.: keystone.exception.DomainNotFound: Could not find domain: fcfd2dda36854bf6b1e6c0e815a34765. Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: fcfd2dda36854bf6b1e6c0e815a34765. Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:42.290874 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 150, in delete Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_domain') Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:42.292470 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:09:42.293789 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: fcfd2dda36854bf6b1e6c0e815a34765. Aug 20 21:09:42.295144 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:42.295242 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2301/4555] 10.222.0.23 () {60 vars in 1299 bytes} [Sat Aug 20 21:09:42 2022] POST /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2 => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:42.296360 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2255/4556] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:42 2022] DELETE /identity/v3/domains/fcfd2dda36854bf6b1e6c0e815a34765 => generated 112 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:09:42.297390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec69ac-0c78-42a9-9cdc-7bdb853113e1 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.301795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3068d-195b-4c6b-a0e9-505758d42b72 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.320118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec69ac-0c78-42a9-9cdc-7bdb853113e1 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.323702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3068d-195b-4c6b-a0e9-505758d42b72 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.331261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150f4f0-31d8-47d9-a142-2e1d0c20f1c6 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e023b25bcb984a1288f523b9c331dfcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30647907e07c41fea7a2c713d5829814', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.331904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150f4f0-31d8-47d9-a142-2e1d0c20f1c6 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.332178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150f4f0-31d8-47d9-a142-2e1d0c20f1c6 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.333352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150f4f0-31d8-47d9-a142-2e1d0c20f1c6 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.334172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150f4f0-31d8-47d9-a142-2e1d0c20f1c6 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=e023b25bcb984a1288f523b9c331dfcc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:42.335697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150f4f0-31d8-47d9-a142-2e1d0c20f1c6 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.337404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd076e94-15c2-485d-8b0f-c94e6f6d0ebe tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd806161de6ce47f394a21ca7a044f8d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bafac58f22974dbf81a10ec5527b6ac2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.338336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd076e94-15c2-485d-8b0f-c94e6f6d0ebe tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.338670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd076e94-15c2-485d-8b0f-c94e6f6d0ebe tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.338932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd076e94-15c2-485d-8b0f-c94e6f6d0ebe tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.339737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd076e94-15c2-485d-8b0f-c94e6f6d0ebe tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:42.341367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd076e94-15c2-485d-8b0f-c94e6f6d0ebe tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.370704 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2302/4557] 10.222.0.23 () {60 vars in 1299 bytes} [Sat Aug 20 21:09:42 2022] POST /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2 => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:42.374842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150f4f0-31d8-47d9-a142-2e1d0c20f1c6 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.384349 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2256/4558] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:09:42 2022] POST /identity/v3/domains => generated 253 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:42.385566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd076e94-15c2-485d-8b0f-c94e6f6d0ebe tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.386587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150f4f0-31d8-47d9-a142-2e1d0c20f1c6 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.482885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432688c3-5a87-4ca2-932a-a52e1bebb3d0 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e023b25bcb984a1288f523b9c331dfcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30647907e07c41fea7a2c713d5829814', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.483954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432688c3-5a87-4ca2-932a-a52e1bebb3d0 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.484441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432688c3-5a87-4ca2-932a-a52e1bebb3d0 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.485127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432688c3-5a87-4ca2-932a-a52e1bebb3d0 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.485895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432688c3-5a87-4ca2-932a-a52e1bebb3d0 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorizing `identity:ec2_list_credentials(user_id=e023b25bcb984a1288f523b9c331dfcc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:42.489290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432688c3-5a87-4ca2-932a-a52e1bebb3d0 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.494344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd076e94-15c2-485d-8b0f-c94e6f6d0ebe tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.499043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ec465-8b67-45fc-a417-bab8bc9731a8 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd806161de6ce47f394a21ca7a044f8d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bafac58f22974dbf81a10ec5527b6ac2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.499694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ec465-8b67-45fc-a417-bab8bc9731a8 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.499969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ec465-8b67-45fc-a417-bab8bc9731a8 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.500237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ec465-8b67-45fc-a417-bab8bc9731a8 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.500924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ec465-8b67-45fc-a417-bab8bc9731a8 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:42.502702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ec465-8b67-45fc-a417-bab8bc9731a8 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.511466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.sql.core [None req-144ec465-8b67-45fc-a417-bab8bc9731a8 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-1412097280' for key 'project.ixu_project_name_domain_id'") Aug 20 21:09:42.511466 ncdv-master-1 devstack@keystone.service[104119]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 20 21:09:42.511466 ncdv-master-1 devstack@keystone.service[104119]: [parameters: {'id': '0a86bc438a5c47fdaa868e81bd3a39ca', 'name': 'tempest-domain-dup-1412097280', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Aug 20 21:09:42.511466 ncdv-master-1 devstack@keystone.service[104119]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=104119) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 20 21:09:42.515638 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2303/4559] 10.222.0.23 () {58 vars in 1278 bytes} [Sat Aug 20 21:09:42 2022] GET /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2 => generated 865 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:42.517706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432688c3-5a87-4ca2-932a-a52e1bebb3d0 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.519501 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-144ec465-8b67-45fc-a417-bab8bc9731a8 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1412097280.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1412097280. Aug 20 21:09:42.519501 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:42.519501 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 20 21:09:42.519501 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 20 21:09:42.519501 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 20 21:09:42.519501 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 20 21:09:42.519501 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 20 21:09:42.519501 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application result = self._query(query) Aug 20 21:09:42.519501 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 20 21:09:42.519501 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application conn.query(q) Aug 20 21:09:42.519501 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 20 21:09:42.519501 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 20 21:09:42.519501 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application result.read() Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-tempest-domain-dup-1412097280' for key 'project.ixu_project_name_domain_id'") Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 20 21:09:42.521276 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application next(self.gen) Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application yield resource Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application next(self.gen) Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.session.rollback() Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.force_reraise() Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise self.value Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application session.commit() Aug 20 21:09:42.522571 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._prepare_impl() Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.session.flush() Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._flush(objects) Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application compat.raise_( Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application flush_context.execute() Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 20 21:09:42.523705 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rec.execute(self) Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 20 21:09:42.524939 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application result = self._query(query) Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application conn.query(q) Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application result.read() Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 20 21:09:42.526220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-1412097280' for key 'project.ixu_project_name_domain_id'") Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [parameters: {'id': '0a86bc438a5c47fdaa868e81bd3a39ca', 'name': 'tempest-domain-dup-1412097280', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:42.527615 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-domain-dup-1412097280. Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:42.528733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1412097280. Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:42.532111 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2257/4560] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:09:42 2022] POST /identity/v3/domains => generated 212 bytes in 136 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 20 21:09:42.533336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ec465-8b67-45fc-a417-bab8bc9731a8 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.540975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432688c3-5a87-4ca2-932a-a52e1bebb3d0 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.543912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ec465-8b67-45fc-a417-bab8bc9731a8 tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.545685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebe5e2-089e-4e2d-8dd2-d177838ad4ac tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e023b25bcb984a1288f523b9c331dfcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30647907e07c41fea7a2c713d5829814', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.546416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebe5e2-089e-4e2d-8dd2-d177838ad4ac tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.546809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebe5e2-089e-4e2d-8dd2-d177838ad4ac tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.547139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebe5e2-089e-4e2d-8dd2-d177838ad4ac tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/9701e310e8fd456095de2489b2968add` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.549306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebe5e2-089e-4e2d-8dd2-d177838ad4ac tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=e023b25bcb984a1288f523b9c331dfcc, credential_id=9701e310e8fd456095de2489b2968add)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:42.550685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171fa35-0bc0-4bd1-8adf-5522d4bc3f7f tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd806161de6ce47f394a21ca7a044f8d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bafac58f22974dbf81a10ec5527b6ac2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.550866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171fa35-0bc0-4bd1-8adf-5522d4bc3f7f tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.551165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171fa35-0bc0-4bd1-8adf-5522d4bc3f7f tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.552203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171fa35-0bc0-4bd1-8adf-5522d4bc3f7f tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] PATH_INFO: `/v3/domains/eaf1455c32fb4046b61fb72f1df6cdac` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.556971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebe5e2-089e-4e2d-8dd2-d177838ad4ac tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.565751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171fa35-0bc0-4bd1-8adf-5522d4bc3f7f tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=eaf1455c32fb4046b61fb72f1df6cdac)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:42.567911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171fa35-0bc0-4bd1-8adf-5522d4bc3f7f tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.580849 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2304/4561] 10.222.0.23 () {58 vars in 1380 bytes} [Sat Aug 20 21:09:42 2022] DELETE /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/9701e310e8fd456095de2489b2968add => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:42.586237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebe5e2-089e-4e2d-8dd2-d177838ad4ac tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.593854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b171fa35-0bc0-4bd1-8adf-5522d4bc3f7f tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'eaf1455c32fb4046b61fb72f1df6cdac'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.594749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b171fa35-0bc0-4bd1-8adf-5522d4bc3f7f tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'eaf1455c32fb4046b61fb72f1df6cdac'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.596921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:42.602859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebe5e2-089e-4e2d-8dd2-d177838ad4ac tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.642776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b171fa35-0bc0-4bd1-8adf-5522d4bc3f7f tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'eaf1455c32fb4046b61fb72f1df6cdac'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.643771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b171fa35-0bc0-4bd1-8adf-5522d4bc3f7f tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'eaf1455c32fb4046b61fb72f1df6cdac'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.650222 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2258/4562] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:09:42 2022] PATCH /identity/v3/domains/eaf1455c32fb4046b61fb72f1df6cdac => generated 254 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:42.651275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171fa35-0bc0-4bd1-8adf-5522d4bc3f7f tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.652891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b171fa35-0bc0-4bd1-8adf-5522d4bc3f7f tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:42.674002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356b7e9-d3b3-4977-8354-4e751f57ddfb tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e023b25bcb984a1288f523b9c331dfcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30647907e07c41fea7a2c713d5829814', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.674653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356b7e9-d3b3-4977-8354-4e751f57ddfb tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.674928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356b7e9-d3b3-4977-8354-4e751f57ddfb tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.675381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356b7e9-d3b3-4977-8354-4e751f57ddfb tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/84ba87ca21244727b49d3f27c1bd53dd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.678113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356b7e9-d3b3-4977-8354-4e751f57ddfb tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=e023b25bcb984a1288f523b9c331dfcc, credential_id=84ba87ca21244727b49d3f27c1bd53dd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:42.678824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356b7e9-d3b3-4977-8354-4e751f57ddfb tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.699317 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2305/4563] 10.222.0.23 () {58 vars in 1380 bytes} [Sat Aug 20 21:09:42 2022] DELETE /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/84ba87ca21244727b49d3f27c1bd53dd => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:42.704415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356b7e9-d3b3-4977-8354-4e751f57ddfb tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.706238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a356b7e9-d3b3-4977-8354-4e751f57ddfb tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:42.715234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:42.719458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356b7e9-d3b3-4977-8354-4e751f57ddfb tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.725938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f6187-6b95-4f58-bba0-6b15552f2db7 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e023b25bcb984a1288f523b9c331dfcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30647907e07c41fea7a2c713d5829814', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.726586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f6187-6b95-4f58-bba0-6b15552f2db7 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.726872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f6187-6b95-4f58-bba0-6b15552f2db7 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.727118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f6187-6b95-4f58-bba0-6b15552f2db7 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.727730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f6187-6b95-4f58-bba0-6b15552f2db7 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=e023b25bcb984a1288f523b9c331dfcc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:42.729368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f6187-6b95-4f58-bba0-6b15552f2db7 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.738544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171fa35-0bc0-4bd1-8adf-5522d4bc3f7f tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.744003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd806161de6ce47f394a21ca7a044f8d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bafac58f22974dbf81a10ec5527b6ac2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.744962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.745132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.745463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] PATH_INFO: `/v3/domains/eaf1455c32fb4046b61fb72f1df6cdac` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.747577 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2306/4564] 10.222.0.23 () {60 vars in 1299 bytes} [Sat Aug 20 21:09:42 2022] POST /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2 => generated 367 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:42.751540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f6187-6b95-4f58-bba0-6b15552f2db7 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.758192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=eaf1455c32fb4046b61fb72f1df6cdac)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:42.759901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.763603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f6187-6b95-4f58-bba0-6b15552f2db7 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.767916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8ca9c-c601-4e45-8bc8-eead6023485d tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e023b25bcb984a1288f523b9c331dfcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30647907e07c41fea7a2c713d5829814', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.768612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8ca9c-c601-4e45-8bc8-eead6023485d tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.768893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8ca9c-c601-4e45-8bc8-eead6023485d tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.769167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8ca9c-c601-4e45-8bc8-eead6023485d tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/a5731f1eb5384677bc058ce43757a6f4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.771907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8ca9c-c601-4e45-8bc8-eead6023485d tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorizing `identity:ec2_get_credential(user_id=e023b25bcb984a1288f523b9c331dfcc, credential_id=a5731f1eb5384677bc058ce43757a6f4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:42.773572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8ca9c-c601-4e45-8bc8-eead6023485d tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.779501 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2307/4565] 10.222.0.23 () {58 vars in 1377 bytes} [Sat Aug 20 21:09:42 2022] GET /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/a5731f1eb5384677bc058ce43757a6f4 => generated 367 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:42.780541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8ca9c-c601-4e45-8bc8-eead6023485d tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.791126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8ca9c-c601-4e45-8bc8-eead6023485d tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.791525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'eaf1455c32fb4046b61fb72f1df6cdac'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.795963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'eaf1455c32fb4046b61fb72f1df6cdac'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.797971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68e159-408c-4ed4-a36c-7dba5778a256 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e023b25bcb984a1288f523b9c331dfcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '30647907e07c41fea7a2c713d5829814', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.798816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68e159-408c-4ed4-a36c-7dba5778a256 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.799144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68e159-408c-4ed4-a36c-7dba5778a256 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.799423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68e159-408c-4ed4-a36c-7dba5778a256 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/a5731f1eb5384677bc058ce43757a6f4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.802004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68e159-408c-4ed4-a36c-7dba5778a256 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=e023b25bcb984a1288f523b9c331dfcc, credential_id=a5731f1eb5384677bc058ce43757a6f4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:42.804017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68e159-408c-4ed4-a36c-7dba5778a256 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:42.830775 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2308/4566] 10.222.0.23 () {58 vars in 1380 bytes} [Sat Aug 20 21:09:42 2022] DELETE /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc/credentials/OS-EC2/a5731f1eb5384677bc058ce43757a6f4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:42.832100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68e159-408c-4ed4-a36c-7dba5778a256 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.835766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ad68e159-408c-4ed4-a36c-7dba5778a256 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:42.858327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf1455c32fb4046b61fb72f1df6cdac', 'request_id': 'req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.867442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf1455c32fb4046b61fb72f1df6cdac', 'request_id': 'req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.879786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf1455c32fb4046b61fb72f1df6cdac', 'request_id': 'req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.901383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] The token cache is being invalidate because project eaf1455c32fb4046b61fb72f1df6cdac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:42.901951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.903051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.917584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'eaf1455c32fb4046b61fb72f1df6cdac', 'request_id': 'req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.919474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'eaf1455c32fb4046b61fb72f1df6cdac', 'request_id': 'req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.924074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'eaf1455c32fb4046b61fb72f1df6cdac', 'request_id': 'req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:42.924336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68e159-408c-4ed4-a36c-7dba5778a256 tempest-EC2CredentialsTest-968123088 tempest-EC2CredentialsTest-968123088-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:42.933721 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2259/4567] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:42 2022] DELETE /identity/v3/domains/eaf1455c32fb4046b61fb72f1df6cdac => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:42.939614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:42.941133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:42.967632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:UPDATE=1,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:42.996408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0819c8-198a-42b8-be3e-c1ad7ee73187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:42.997176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0819c8-198a-42b8-be3e-c1ad7ee73187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:42.997450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0819c8-198a-42b8-be3e-c1ad7ee73187 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:42.997719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0819c8-198a-42b8-be3e-c1ad7ee73187 admin admin] PATH_INFO: `/v3/users/e023b25bcb984a1288f523b9c331dfcc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:42.998854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0819c8-198a-42b8-be3e-c1ad7ee73187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e023b25bcb984a1288f523b9c331dfcc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:43.000833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0819c8-198a-42b8-be3e-c1ad7ee73187 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:43.023943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d3b3b-f29c-48c4-a528-8f6d847f031d tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:43.030125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b57221-127e-44da-bc35-44b1bf53ae6a tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd806161de6ce47f394a21ca7a044f8d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bafac58f22974dbf81a10ec5527b6ac2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:43.030806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b57221-127e-44da-bc35-44b1bf53ae6a tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:43.031179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b57221-127e-44da-bc35-44b1bf53ae6a tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:43.031407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b57221-127e-44da-bc35-44b1bf53ae6a tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] PATH_INFO: `/v3/domains/469afa945ab548d0a5c9dc530d2dcaf3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:43.039483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b57221-127e-44da-bc35-44b1bf53ae6a tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-65b57221-127e-44da-bc35-44b1bf53ae6a tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Could not find domain: 469afa945ab548d0a5c9dc530d2dcaf3.: keystone.exception.DomainNotFound: Could not find domain: 469afa945ab548d0a5c9dc530d2dcaf3. Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 469afa945ab548d0a5c9dc530d2dcaf3. Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:43.044543 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:43.046220 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:09:43.047650 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 469afa945ab548d0a5c9dc530d2dcaf3. Aug 20 21:09:43.048880 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:43.049210 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2260/4568] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:09:42 2022] PATCH /identity/v3/domains/469afa945ab548d0a5c9dc530d2dcaf3 => generated 112 bytes in 111 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Aug 20 21:09:43.050112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b57221-127e-44da-bc35-44b1bf53ae6a tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:43.052012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-65b57221-127e-44da-bc35-44b1bf53ae6a tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:43.066506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b57221-127e-44da-bc35-44b1bf53ae6a tempest-DomainsNegativeTestJSON-902642552 tempest-DomainsNegativeTestJSON-902642552-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:43.069511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3c0819c8-198a-42b8-be3e-c1ad7ee73187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e023b25bcb984a1288f523b9c331dfcc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:43.075120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2dcce-8ebf-41d9-aa7e-ea107847a06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:43.075409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2dcce-8ebf-41d9-aa7e-ea107847a06e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:43.075409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2dcce-8ebf-41d9-aa7e-ea107847a06e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:43.075689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2dcce-8ebf-41d9-aa7e-ea107847a06e admin admin] PATH_INFO: `/v3/users/36768cfb8ede43a5bb9a4c2ff1e8d311` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:43.077073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2dcce-8ebf-41d9-aa7e-ea107847a06e admin admin] RBAC: Authorizing `identity:delete_user(user_id=36768cfb8ede43a5bb9a4c2ff1e8d311)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:43.078954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2dcce-8ebf-41d9-aa7e-ea107847a06e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:43.084634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3c0819c8-198a-42b8-be3e-c1ad7ee73187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e023b25bcb984a1288f523b9c331dfcc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:43.102439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3c0819c8-198a-42b8-be3e-c1ad7ee73187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e023b25bcb984a1288f523b9c331dfcc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:43.119978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3c0819c8-198a-42b8-be3e-c1ad7ee73187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e023b25bcb984a1288f523b9c331dfcc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:43.121516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3c0819c8-198a-42b8-be3e-c1ad7ee73187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e023b25bcb984a1288f523b9c331dfcc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:43.123764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:43.140402 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2309/4569] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:42 2022] DELETE /identity/v3/users/e023b25bcb984a1288f523b9c331dfcc => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:43.147990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afc51c-3303-45fe-bac8-3c0dce08ac8c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:43.148573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afc51c-3303-45fe-bac8-3c0dce08ac8c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:43.149852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afc51c-3303-45fe-bac8-3c0dce08ac8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:43.157359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cfc2dcce-8ebf-41d9-aa7e-ea107847a06e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36768cfb8ede43a5bb9a4c2ff1e8d311'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:43.168148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cfc2dcce-8ebf-41d9-aa7e-ea107847a06e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36768cfb8ede43a5bb9a4c2ff1e8d311'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:43.179153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-13afc51c-3303-45fe-bac8-3c0dce08ac8c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:43.180364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-13afc51c-3303-45fe-bac8-3c0dce08ac8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:43.190206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cfc2dcce-8ebf-41d9-aa7e-ea107847a06e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36768cfb8ede43a5bb9a4c2ff1e8d311'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:43.192102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cfc2dcce-8ebf-41d9-aa7e-ea107847a06e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36768cfb8ede43a5bb9a4c2ff1e8d311'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:43.194055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cfc2dcce-8ebf-41d9-aa7e-ea107847a06e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36768cfb8ede43a5bb9a4c2ff1e8d311'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:43.218891 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2261/4570] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:43 2022] DELETE /identity/v3/users/36768cfb8ede43a5bb9a4c2ff1e8d311 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:43.221442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a105723-04ed-438e-8319-08d1e94bd78a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:43.221724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a105723-04ed-438e-8319-08d1e94bd78a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:43.222186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a105723-04ed-438e-8319-08d1e94bd78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:43.251093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4a105723-04ed-438e-8319-08d1e94bd78a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:43.252332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4a105723-04ed-438e-8319-08d1e94bd78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:43.294582 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2310/4571] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:43.305522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13afc51c-3303-45fe-bac8-3c0dce08ac8c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:43.306881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-13afc51c-3303-45fe-bac8-3c0dce08ac8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:43.322739 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2262/4572] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:43.332428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a105723-04ed-438e-8319-08d1e94bd78a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:43.334122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4a105723-04ed-438e-8319-08d1e94bd78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:43.371403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13afc51c-3303-45fe-bac8-3c0dce08ac8c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:43.374337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:43.410034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a105723-04ed-438e-8319-08d1e94bd78a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:43.416936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d844a1-66f6-4404-b4ab-15ca798d0744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:43.417880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d844a1-66f6-4404-b4ab-15ca798d0744 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:43.418151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d844a1-66f6-4404-b4ab-15ca798d0744 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:43.418412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d844a1-66f6-4404-b4ab-15ca798d0744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:43.428766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d844a1-66f6-4404-b4ab-15ca798d0744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:43.430268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d844a1-66f6-4404-b4ab-15ca798d0744 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:43.441177 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2263/4573] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:09:43.442164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d4617-afff-4f74-b226-6e1ba791b3af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:43.442906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d4617-afff-4f74-b226-6e1ba791b3af service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:43.443245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d4617-afff-4f74-b226-6e1ba791b3af service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:43.443714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d4617-afff-4f74-b226-6e1ba791b3af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:43.459727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d4617-afff-4f74-b226-6e1ba791b3af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:43.461249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d4617-afff-4f74-b226-6e1ba791b3af service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:43.483619 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2311/4574] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:09:43.947180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d844a1-66f6-4404-b4ab-15ca798d0744 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:43.948771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-73d844a1-66f6-4404-b4ab-15ca798d0744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:43.964191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d844a1-66f6-4404-b4ab-15ca798d0744 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:43.969772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:43.970501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:43.971225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:43.971225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] PATH_INFO: `/v3/projects/86f7990dfbdb407e88e2ddf7623fe307` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:43.972575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=86f7990dfbdb407e88e2ddf7623fe307)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:43.974701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:44.029181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d4617-afff-4f74-b226-6e1ba791b3af service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:44.030588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ee1d4617-afff-4f74-b226-6e1ba791b3af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:44.046309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86f7990dfbdb407e88e2ddf7623fe307', 'request_id': 'req-8d82d153-74d0-4960-a581-351e9c9183ea'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.046821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d4617-afff-4f74-b226-6e1ba791b3af service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:44.053958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86f7990dfbdb407e88e2ddf7623fe307', 'request_id': 'req-8d82d153-74d0-4960-a581-351e9c9183ea'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.067664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86f7990dfbdb407e88e2ddf7623fe307', 'request_id': 'req-8d82d153-74d0-4960-a581-351e9c9183ea'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.084442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] The token cache is being invalidate because project 86f7990dfbdb407e88e2ddf7623fe307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:44.084738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.085407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.087528 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2264/4575] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:43 2022] DELETE /identity/v3/projects/86f7990dfbdb407e88e2ddf7623fe307 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:44.091191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:44.093161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:44.114883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:44.122936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:44.123579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:44.124090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:44.124090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80 admin admin] PATH_INFO: `/v3/projects/30647907e07c41fea7a2c713d5829814` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:44.125362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30647907e07c41fea7a2c713d5829814)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:44.128136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:44.172695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82d153-74d0-4960-a581-351e9c9183ea admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:44.185707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30647907e07c41fea7a2c713d5829814', 'request_id': 'req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.194791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30647907e07c41fea7a2c713d5829814', 'request_id': 'req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.201878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30647907e07c41fea7a2c713d5829814', 'request_id': 'req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.216870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80 admin admin] The token cache is being invalidate because project 30647907e07c41fea7a2c713d5829814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:44.217329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.218322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fca54dbd-9a00-41a9-aaf5-54783c1d3b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.220812 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2312/4576] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:44 2022] DELETE /identity/v3/projects/30647907e07c41fea7a2c713d5829814 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:44.227579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e4f19-4e58-4d23-9848-9f68dbc653b9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:44.227882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e4f19-4e58-4d23-9848-9f68dbc653b9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:44.228157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e4f19-4e58-4d23-9848-9f68dbc653b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:44.258080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-241e4f19-4e58-4d23-9848-9f68dbc653b9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:44.259365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-241e4f19-4e58-4d23-9848-9f68dbc653b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:44.263126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:44.263883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:44.264221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:44.264620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] PATH_INFO: `/v3/users/d806161de6ce47f394a21ca7a044f8d2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:44.265856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d806161de6ce47f394a21ca7a044f8d2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:44.268446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:44.269717 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2313/4577] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:44.275573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:44.278762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beeafd2-53d8-406e-b4cf-ffd1a1f61aff None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:44.278976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beeafd2-53d8-406e-b4cf-ffd1a1f61aff None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:44.279176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beeafd2-53d8-406e-b4cf-ffd1a1f61aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:44.305205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8beeafd2-53d8-406e-b4cf-ffd1a1f61aff None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:44.306338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8beeafd2-53d8-406e-b4cf-ffd1a1f61aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:44.319611 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2314/4578] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:44.320170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beeafd2-53d8-406e-b4cf-ffd1a1f61aff None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:44.330488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beeafd2-53d8-406e-b4cf-ffd1a1f61aff None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:44.336879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cceea-0b21-4cb1-ab07-19a143c58929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:44.337261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cceea-0b21-4cb1-ab07-19a143c58929 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:44.337538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cceea-0b21-4cb1-ab07-19a143c58929 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:44.337788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cceea-0b21-4cb1-ab07-19a143c58929 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:44.338472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3cceea-0b21-4cb1-ab07-19a143c58929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:44.338765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3cceea-0b21-4cb1-ab07-19a143c58929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:44.340430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3cceea-0b21-4cb1-ab07-19a143c58929 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:44.353686 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2315/4579] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:44.358268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cceea-0b21-4cb1-ab07-19a143c58929 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:44.365883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd806161de6ce47f394a21ca7a044f8d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.369205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cceea-0b21-4cb1-ab07-19a143c58929 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:44.374378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57daa4-dbc8-485f-8129-8c4d4c11bd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:44.374973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57daa4-dbc8-485f-8129-8c4d4c11bd71 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:44.375245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57daa4-dbc8-485f-8129-8c4d4c11bd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:44.375510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57daa4-dbc8-485f-8129-8c4d4c11bd71 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:44.376134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57daa4-dbc8-485f-8129-8c4d4c11bd71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:44.377325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd806161de6ce47f394a21ca7a044f8d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.378751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57daa4-dbc8-485f-8129-8c4d4c11bd71 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:44.392935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd806161de6ce47f394a21ca7a044f8d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.393689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd806161de6ce47f394a21ca7a044f8d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.394303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd806161de6ce47f394a21ca7a044f8d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.408215 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2265/4580] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:44 2022] DELETE /identity/v3/users/d806161de6ce47f394a21ca7a044f8d2 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:44.409396 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2316/4581] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:44 2022] POST /identity/v3/projects => generated 397 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:44.413989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:44.415386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:44.503921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bf741-a51b-46e1-a0d8-995204e01be3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:44.564685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a1518-3749-419a-bc69-6f7f19577ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:44.565219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a1518-3749-419a-bc69-6f7f19577ceb admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:44.565491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a1518-3749-419a-bc69-6f7f19577ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:44.565754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a1518-3749-419a-bc69-6f7f19577ceb admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:44.566386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a1518-3749-419a-bc69-6f7f19577ceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:44.569006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a1518-3749-419a-bc69-6f7f19577ceb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:44.599237 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2266/4582] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:44 2022] POST /identity/v3/users => generated 305 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:44.603891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57daa4-dbc8-485f-8129-8c4d4c11bd71 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:44.615987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57daa4-dbc8-485f-8129-8c4d4c11bd71 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:44.621621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da1f36-56d2-4d01-9fce-deab957ab20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:44.622286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da1f36-56d2-4d01-9fce-deab957ab20a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:44.622611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da1f36-56d2-4d01-9fce-deab957ab20a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:44.623560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da1f36-56d2-4d01-9fce-deab957ab20a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:44.624290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da1f36-56d2-4d01-9fce-deab957ab20a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:44.626035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da1f36-56d2-4d01-9fce-deab957ab20a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:44.640136 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2317/4583] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:44.644659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a1518-3749-419a-bc69-6f7f19577ceb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:44.666804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a1518-3749-419a-bc69-6f7f19577ceb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:44.671996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0646ce5-13f6-4902-bf42-e302883ed604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:44.672786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0646ce5-13f6-4902-bf42-e302883ed604 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:44.673448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0646ce5-13f6-4902-bf42-e302883ed604 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:44.673782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0646ce5-13f6-4902-bf42-e302883ed604 admin admin] PATH_INFO: `/v3/projects/897ea60a208d4878afdf74c901a018af/users/f7db36b258cc40a5b208f48cd2ae12e7/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:44.712403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0646ce5-13f6-4902-bf42-e302883ed604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897ea60a208d4878afdf74c901a018af, user_id=f7db36b258cc40a5b208f48cd2ae12e7, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:44.712403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0646ce5-13f6-4902-bf42-e302883ed604 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:44.712403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:44.730935 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2267/4584] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:44 2022] PUT /identity/v3/projects/897ea60a208d4878afdf74c901a018af/users/f7db36b258cc40a5b208f48cd2ae12e7/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:44.732287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da1f36-56d2-4d01-9fce-deab957ab20a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:44.745452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da1f36-56d2-4d01-9fce-deab957ab20a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:44.767232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0646ce5-13f6-4902-bf42-e302883ed604 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:44.769934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b0646ce5-13f6-4902-bf42-e302883ed604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:44.863582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdf824-136f-4a10-a73c-aaab4de7b765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:44.864483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdf824-136f-4a10-a73c-aaab4de7b765 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:44.864759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdf824-136f-4a10-a73c-aaab4de7b765 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:44.865212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdf824-136f-4a10-a73c-aaab4de7b765 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:44.866031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdf824-136f-4a10-a73c-aaab4de7b765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:44.867694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdf824-136f-4a10-a73c-aaab4de7b765 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:44.880444 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2318/4585] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:44.884682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdf824-136f-4a10-a73c-aaab4de7b765 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:44.886998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0646ce5-13f6-4902-bf42-e302883ed604 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:44.894807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc4411-312e-4e50-961f-f5b21b28b5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:44.897026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc4411-312e-4e50-961f-f5b21b28b5e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:44.897410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc4411-312e-4e50-961f-f5b21b28b5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:44.898284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc4411-312e-4e50-961f-f5b21b28b5e3 admin admin] PATH_INFO: `/v3/projects/bafac58f22974dbf81a10ec5527b6ac2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:44.900473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdf824-136f-4a10-a73c-aaab4de7b765 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:44.900813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc4411-312e-4e50-961f-f5b21b28b5e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bafac58f22974dbf81a10ec5527b6ac2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:44.903613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc4411-312e-4e50-961f-f5b21b28b5e3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:44.905796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e02efa-1c6a-4b95-90da-c702650b0ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:44.906903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e02efa-1c6a-4b95-90da-c702650b0ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:44.907214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e02efa-1c6a-4b95-90da-c702650b0ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:44.907429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e02efa-1c6a-4b95-90da-c702650b0ce8 admin admin] PATH_INFO: `/v3/projects/897ea60a208d4878afdf74c901a018af/users/f7db36b258cc40a5b208f48cd2ae12e7/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:44.909615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e02efa-1c6a-4b95-90da-c702650b0ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897ea60a208d4878afdf74c901a018af, user_id=f7db36b258cc40a5b208f48cd2ae12e7, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:44.914240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e02efa-1c6a-4b95-90da-c702650b0ce8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:44.925119 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2319/4586] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:44 2022] PUT /identity/v3/projects/897ea60a208d4878afdf74c901a018af/users/f7db36b258cc40a5b208f48cd2ae12e7/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:44.936124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2f1f7-3daf-4463-9e90-dc184280cc99 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:44.936491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2f1f7-3daf-4463-9e90-dc184280cc99 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:44.936724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2f1f7-3daf-4463-9e90-dc184280cc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:44.960778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:44.967191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cddc4411-312e-4e50-961f-f5b21b28b5e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bafac58f22974dbf81a10ec5527b6ac2', 'request_id': 'req-cddc4411-312e-4e50-961f-f5b21b28b5e3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.969086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-92a2f1f7-3daf-4463-9e90-dc184280cc99 None None] MFA Rules not processed for user `f7db36b258cc40a5b208f48cd2ae12e7`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:44.970928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-92a2f1f7-3daf-4463-9e90-dc184280cc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:44.977510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cddc4411-312e-4e50-961f-f5b21b28b5e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bafac58f22974dbf81a10ec5527b6ac2', 'request_id': 'req-cddc4411-312e-4e50-961f-f5b21b28b5e3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:44.990045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cddc4411-312e-4e50-961f-f5b21b28b5e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bafac58f22974dbf81a10ec5527b6ac2', 'request_id': 'req-cddc4411-312e-4e50-961f-f5b21b28b5e3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:45.011673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cddc4411-312e-4e50-961f-f5b21b28b5e3 admin admin] The token cache is being invalidate because project bafac58f22974dbf81a10ec5527b6ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:45.011673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cddc4411-312e-4e50-961f-f5b21b28b5e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:45.012265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cddc4411-312e-4e50-961f-f5b21b28b5e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:45.013879 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2268/4587] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:44 2022] DELETE /identity/v3/projects/bafac58f22974dbf81a10ec5527b6ac2 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:45.019819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c22679-8c90-42a8-b9d1-76a6945519a7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.021779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c22679-8c90-42a8-b9d1-76a6945519a7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.022155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c22679-8c90-42a8-b9d1-76a6945519a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.044153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-80c22679-8c90-42a8-b9d1-76a6945519a7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:45.045545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-80c22679-8c90-42a8-b9d1-76a6945519a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:45.060141 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2320/4588] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:44 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:45.064732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2f1f7-3daf-4463-9e90-dc184280cc99 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.066244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-92a2f1f7-3daf-4463-9e90-dc184280cc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:45.136939 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2269/4589] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:45.146026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de888e-830d-4304-9baa-a0ec51677809 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.146291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de888e-830d-4304-9baa-a0ec51677809 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.146550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de888e-830d-4304-9baa-a0ec51677809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.146664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2f1f7-3daf-4463-9e90-dc184280cc99 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.151520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7268d2-da37-42e9-88dd-d47bdeee54f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.152120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7268d2-da37-42e9-88dd-d47bdeee54f2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.152407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7268d2-da37-42e9-88dd-d47bdeee54f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.152823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7268d2-da37-42e9-88dd-d47bdeee54f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.153447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7268d2-da37-42e9-88dd-d47bdeee54f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.155564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7268d2-da37-42e9-88dd-d47bdeee54f2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:45.168846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a9de888e-830d-4304-9baa-a0ec51677809 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:45.170052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a9de888e-830d-4304-9baa-a0ec51677809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:45.178770 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2270/4590] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:45.184017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de888e-830d-4304-9baa-a0ec51677809 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.186795 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2321/4591] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:45 2022] POST /identity/v3/projects => generated 397 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:45.190558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7268d2-da37-42e9-88dd-d47bdeee54f2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.193842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de888e-830d-4304-9baa-a0ec51677809 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.211151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:45.256784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674cf26-6714-417e-aa7b-54dd866fc82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.257398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674cf26-6714-417e-aa7b-54dd866fc82c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.257671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674cf26-6714-417e-aa7b-54dd866fc82c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.257937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674cf26-6714-417e-aa7b-54dd866fc82c admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.258577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674cf26-6714-417e-aa7b-54dd866fc82c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:45.258840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674cf26-6714-417e-aa7b-54dd866fc82c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.260499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674cf26-6714-417e-aa7b-54dd866fc82c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:45.264595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7268d2-da37-42e9-88dd-d47bdeee54f2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.270333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4b747-cd07-484b-a99e-1740b148edca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.271279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4b747-cd07-484b-a99e-1740b148edca admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.271741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4b747-cd07-484b-a99e-1740b148edca admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.272079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4b747-cd07-484b-a99e-1740b148edca admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.272757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4b747-cd07-484b-a99e-1740b148edca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.273703 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2271/4592] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:45.274769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4b747-cd07-484b-a99e-1740b148edca admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:45.278168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674cf26-6714-417e-aa7b-54dd866fc82c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.288739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674cf26-6714-417e-aa7b-54dd866fc82c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.294018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b3181-2d3b-4981-986f-7101007b12e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.294632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b3181-2d3b-4981-986f-7101007b12e4 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.294941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b3181-2d3b-4981-986f-7101007b12e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.295211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b3181-2d3b-4981-986f-7101007b12e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.295891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b3181-2d3b-4981-986f-7101007b12e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.297860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b3181-2d3b-4981-986f-7101007b12e4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:45.299431 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2322/4593] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:45 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:45.303186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4b747-cd07-484b-a99e-1740b148edca admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.312997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4b747-cd07-484b-a99e-1740b148edca admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.318450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8615e27-fbaf-4066-8a16-55bbe825b3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.319077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8615e27-fbaf-4066-8a16-55bbe825b3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.319404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8615e27-fbaf-4066-8a16-55bbe825b3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.319609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8615e27-fbaf-4066-8a16-55bbe825b3c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.320675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8615e27-fbaf-4066-8a16-55bbe825b3c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.322225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8615e27-fbaf-4066-8a16-55bbe825b3c6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:45.327589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:45.327914 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2272/4594] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:45 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:45.331956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b3181-2d3b-4981-986f-7101007b12e4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.339851 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2323/4595] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:45.340836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8615e27-fbaf-4066-8a16-55bbe825b3c6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.351496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8615e27-fbaf-4066-8a16-55bbe825b3c6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.408048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b3181-2d3b-4981-986f-7101007b12e4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.413422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654224b-f392-4a7d-9a49-931d36185923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.414186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654224b-f392-4a7d-9a49-931d36185923 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.414459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654224b-f392-4a7d-9a49-931d36185923 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.414721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654224b-f392-4a7d-9a49-931d36185923 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.415353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654224b-f392-4a7d-9a49-931d36185923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.417093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654224b-f392-4a7d-9a49-931d36185923 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:45.423758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffb4fb-c922-4ea8-beea-62ab693f9063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.424403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffb4fb-c922-4ea8-beea-62ab693f9063 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.424672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffb4fb-c922-4ea8-beea-62ab693f9063 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.424937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffb4fb-c922-4ea8-beea-62ab693f9063 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.425560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffb4fb-c922-4ea8-beea-62ab693f9063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.427561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffb4fb-c922-4ea8-beea-62ab693f9063 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:45.433484 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2324/4596] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:45.437471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654224b-f392-4a7d-9a49-931d36185923 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.448872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654224b-f392-4a7d-9a49-931d36185923 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.452614 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2273/4597] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:45 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:45.453363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c521e-b6bc-4985-9a6f-48b5915b904d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.454290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c521e-b6bc-4985-9a6f-48b5915b904d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.454561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c521e-b6bc-4985-9a6f-48b5915b904d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.454820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c521e-b6bc-4985-9a6f-48b5915b904d admin admin] PATH_INFO: `/v3/projects/236663ad486f473e85f641f53efb4069/users/f1c1ed780e404dc583f7effaf1ccbfe6/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.456410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffb4fb-c922-4ea8-beea-62ab693f9063 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.467284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffb4fb-c922-4ea8-beea-62ab693f9063 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.469706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c521e-b6bc-4985-9a6f-48b5915b904d admin admin] RBAC: Authorizing `identity:create_grant(project_id=236663ad486f473e85f641f53efb4069, user_id=f1c1ed780e404dc583f7effaf1ccbfe6, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.471963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e243cca-7c23-41f1-8576-301f4d7aec22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.472364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c521e-b6bc-4985-9a6f-48b5915b904d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:45.472588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e243cca-7c23-41f1-8576-301f4d7aec22 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.472861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e243cca-7c23-41f1-8576-301f4d7aec22 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.473133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e243cca-7c23-41f1-8576-301f4d7aec22 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.473828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e243cca-7c23-41f1-8576-301f4d7aec22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.475547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e243cca-7c23-41f1-8576-301f4d7aec22 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:45.489249 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2274/4598] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:45.489497 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2325/4599] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:45 2022] PUT /identity/v3/projects/236663ad486f473e85f641f53efb4069/users/f1c1ed780e404dc583f7effaf1ccbfe6/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:45.490714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e243cca-7c23-41f1-8576-301f4d7aec22 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.497193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a26935-7a82-4c5c-bee2-997424d0bffd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.497575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a26935-7a82-4c5c-bee2-997424d0bffd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.497932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a26935-7a82-4c5c-bee2-997424d0bffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.501750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e243cca-7c23-41f1-8576-301f4d7aec22 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.532747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-16a26935-7a82-4c5c-bee2-997424d0bffd None None] MFA Rules not processed for user `f1c1ed780e404dc583f7effaf1ccbfe6`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:45.533967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-16a26935-7a82-4c5c-bee2-997424d0bffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:45.563580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f3862-943b-4a8f-9171-f015995e0013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.564192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f3862-943b-4a8f-9171-f015995e0013 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.564502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f3862-943b-4a8f-9171-f015995e0013 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.564774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f3862-943b-4a8f-9171-f015995e0013 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.565411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f3862-943b-4a8f-9171-f015995e0013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.567108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f3862-943b-4a8f-9171-f015995e0013 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:45.587189 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2275/4600] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:45.591350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f3862-943b-4a8f-9171-f015995e0013 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.601580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f3862-943b-4a8f-9171-f015995e0013 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.606281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74665f2c-bfcf-4e17-8719-0fa975e2058d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.606940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74665f2c-bfcf-4e17-8719-0fa975e2058d admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.607246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74665f2c-bfcf-4e17-8719-0fa975e2058d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.607523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74665f2c-bfcf-4e17-8719-0fa975e2058d admin admin] PATH_INFO: `/v3/projects/be3b845ecb4446b78fb729cfd55bf850/users/91eb3c6c60724027baf734cf79e28b55/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.609909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:45.611652 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2326/4601] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:45 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:45.624286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a26935-7a82-4c5c-bee2-997424d0bffd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.625730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-16a26935-7a82-4c5c-bee2-997424d0bffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:45.626586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74665f2c-bfcf-4e17-8719-0fa975e2058d admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3b845ecb4446b78fb729cfd55bf850, user_id=91eb3c6c60724027baf734cf79e28b55, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.629341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74665f2c-bfcf-4e17-8719-0fa975e2058d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:45.644325 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2276/4602] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:45 2022] PUT /identity/v3/projects/be3b845ecb4446b78fb729cfd55bf850/users/91eb3c6c60724027baf734cf79e28b55/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:45.655556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34664f61-9880-4bbc-84c9-b4c90e456553 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.655857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34664f61-9880-4bbc-84c9-b4c90e456553 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.656093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34664f61-9880-4bbc-84c9-b4c90e456553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.687962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-34664f61-9880-4bbc-84c9-b4c90e456553 None None] MFA Rules not processed for user `91eb3c6c60724027baf734cf79e28b55`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:45.690771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-34664f61-9880-4bbc-84c9-b4c90e456553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:45.695021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a26935-7a82-4c5c-bee2-997424d0bffd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.709854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:45.774914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286fd19-1290-42a7-b9a3-12b7dddfa78f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.775562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286fd19-1290-42a7-b9a3-12b7dddfa78f service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.775927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286fd19-1290-42a7-b9a3-12b7dddfa78f service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.776118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286fd19-1290-42a7-b9a3-12b7dddfa78f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.789143 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2277/4603] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:45.793779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34664f61-9880-4bbc-84c9-b4c90e456553 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.853912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286fd19-1290-42a7-b9a3-12b7dddfa78f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.855294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286fd19-1290-42a7-b9a3-12b7dddfa78f service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:45.865075 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2327/4604] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:45 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:09:45.871695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34664f61-9880-4bbc-84c9-b4c90e456553 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.876340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887191e-aff3-4fcf-9fb1-6a2c7d6e3a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.877037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887191e-aff3-4fcf-9fb1-6a2c7d6e3a5e admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.877376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887191e-aff3-4fcf-9fb1-6a2c7d6e3a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.877734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887191e-aff3-4fcf-9fb1-6a2c7d6e3a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.878415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887191e-aff3-4fcf-9fb1-6a2c7d6e3a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.880949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887191e-aff3-4fcf-9fb1-6a2c7d6e3a5e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:45.909406 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2278/4605] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:45 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:45.914517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286fd19-1290-42a7-b9a3-12b7dddfa78f service glance] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:45.925392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286fd19-1290-42a7-b9a3-12b7dddfa78f service glance] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:45.991338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86270737-5519-4503-be09-71bace3ed783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:45.992175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86270737-5519-4503-be09-71bace3ed783 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:45.992306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86270737-5519-4503-be09-71bace3ed783 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:45.992544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86270737-5519-4503-be09-71bace3ed783 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:45.993192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86270737-5519-4503-be09-71bace3ed783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:45.995331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86270737-5519-4503-be09-71bace3ed783 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.020003 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2328/4606] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:45 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:46.024459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887191e-aff3-4fcf-9fb1-6a2c7d6e3a5e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.035058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887191e-aff3-4fcf-9fb1-6a2c7d6e3a5e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.040190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f07e2-d971-4cf8-8af9-8af499c50f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.040828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f07e2-d971-4cf8-8af9-8af499c50f42 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.041160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f07e2-d971-4cf8-8af9-8af499c50f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.041381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f07e2-d971-4cf8-8af9-8af499c50f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.042047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f07e2-d971-4cf8-8af9-8af499c50f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.043679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f07e2-d971-4cf8-8af9-8af499c50f42 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.054491 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2279/4607] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:46.058885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86270737-5519-4503-be09-71bace3ed783 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.071018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86270737-5519-4503-be09-71bace3ed783 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.076924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c696d4-c05c-43ab-b46b-ef8b801f2ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.077580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c696d4-c05c-43ab-b46b-ef8b801f2ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.077847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c696d4-c05c-43ab-b46b-ef8b801f2ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.078110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c696d4-c05c-43ab-b46b-ef8b801f2ac3 admin admin] PATH_INFO: `/v3/projects/a095482cd44f48388f9c2befac2e919f/users/37ee4b1f2cee495ebd91c5ea720d03f1/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.096595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c696d4-c05c-43ab-b46b-ef8b801f2ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a095482cd44f48388f9c2befac2e919f, user_id=37ee4b1f2cee495ebd91c5ea720d03f1, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.100816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c696d4-c05c-43ab-b46b-ef8b801f2ac3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.122187 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2329/4608] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:46 2022] PUT /identity/v3/projects/a095482cd44f48388f9c2befac2e919f/users/37ee4b1f2cee495ebd91c5ea720d03f1/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:46.126327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f07e2-d971-4cf8-8af9-8af499c50f42 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.138153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f07e2-d971-4cf8-8af9-8af499c50f42 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.199028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:46.224413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff21e8-d04d-4ff0-be38-5d4076bcf5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.225051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff21e8-d04d-4ff0-be38-5d4076bcf5e5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.225334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff21e8-d04d-4ff0-be38-5d4076bcf5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.225604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff21e8-d04d-4ff0-be38-5d4076bcf5e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.229196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff21e8-d04d-4ff0-be38-5d4076bcf5e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.231071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff21e8-d04d-4ff0-be38-5d4076bcf5e5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.247801 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2280/4609] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:46.252014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c696d4-c05c-43ab-b46b-ef8b801f2ac3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.263743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c696d4-c05c-43ab-b46b-ef8b801f2ac3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.269052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2c55d-7bbc-4c06-8e52-81b2b0509c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.269722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2c55d-7bbc-4c06-8e52-81b2b0509c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.270010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2c55d-7bbc-4c06-8e52-81b2b0509c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.270432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2c55d-7bbc-4c06-8e52-81b2b0509c42 admin admin] PATH_INFO: `/v3/projects/a095482cd44f48388f9c2befac2e919f/users/37ee4b1f2cee495ebd91c5ea720d03f1/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.272575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2c55d-7bbc-4c06-8e52-81b2b0509c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a095482cd44f48388f9c2befac2e919f, user_id=37ee4b1f2cee495ebd91c5ea720d03f1, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.275273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2c55d-7bbc-4c06-8e52-81b2b0509c42 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.287595 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2330/4610] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:46 2022] PUT /identity/v3/projects/a095482cd44f48388f9c2befac2e919f/users/37ee4b1f2cee495ebd91c5ea720d03f1/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:46.299097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3a936-f542-4640-875b-e5993e943b88 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.299357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3a936-f542-4640-875b-e5993e943b88 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.299626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3a936-f542-4640-875b-e5993e943b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.311183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2c55d-7bbc-4c06-8e52-81b2b0509c42 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.336911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-99f3a936-f542-4640-875b-e5993e943b88 None None] MFA Rules not processed for user `37ee4b1f2cee495ebd91c5ea720d03f1`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:46.337153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:46.338207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-99f3a936-f542-4640-875b-e5993e943b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:46.408730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2c55d-7bbc-4c06-8e52-81b2b0509c42 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.413969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf619465-d352-4fec-b0d9-9525a50a58e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.414724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf619465-d352-4fec-b0d9-9525a50a58e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.415098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf619465-d352-4fec-b0d9-9525a50a58e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.415443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf619465-d352-4fec-b0d9-9525a50a58e3 admin admin] PATH_INFO: `/v3/users/f7db36b258cc40a5b208f48cd2ae12e7` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.416733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf619465-d352-4fec-b0d9-9525a50a58e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7db36b258cc40a5b208f48cd2ae12e7)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.418685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf619465-d352-4fec-b0d9-9525a50a58e3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.439565 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2281/4611] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:46.445878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3a936-f542-4640-875b-e5993e943b88 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.456527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3a936-f542-4640-875b-e5993e943b88 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.463642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0f92b-47b8-4d4e-a34b-cef3b0c39dd5 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.464578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0f92b-47b8-4d4e-a34b-cef3b0c39dd5 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.464916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0f92b-47b8-4d4e-a34b-cef3b0c39dd5 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.465282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0f92b-47b8-4d4e-a34b-cef3b0c39dd5 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.466005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0f92b-47b8-4d4e-a34b-cef3b0c39dd5 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.469322 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-ada0f92b-47b8-4d4e-a34b-cef3b0c39dd5 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:46.469322 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:09:46.469322 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:09:46.470979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0f92b-47b8-4d4e-a34b-cef3b0c39dd5 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.502443 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2282/4612] 10.222.0.23 () {60 vars in 1185 bytes} [Sat Aug 20 21:09:46 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:46.506200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bf619465-d352-4fec-b0d9-9525a50a58e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7db36b258cc40a5b208f48cd2ae12e7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:46.506668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0f92b-47b8-4d4e-a34b-cef3b0c39dd5 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.517072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0f92b-47b8-4d4e-a34b-cef3b0c39dd5 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.519930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bf619465-d352-4fec-b0d9-9525a50a58e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7db36b258cc40a5b208f48cd2ae12e7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:46.526649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dd7eb-b919-4c89-a129-e3ba4b9c5998 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.527541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dd7eb-b919-4c89-a129-e3ba4b9c5998 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.527818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dd7eb-b919-4c89-a129-e3ba4b9c5998 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.528105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dd7eb-b919-4c89-a129-e3ba4b9c5998 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/f087e879406048e4b870b548bbf2d1c8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.528735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dd7eb-b919-4c89-a129-e3ba4b9c5998 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=f087e879406048e4b870b548bbf2d1c8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.529753 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-000dd7eb-b919-4c89-a129-e3ba4b9c5998 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:46.529753 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:09:46.529753 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:09:46.530889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dd7eb-b919-4c89-a129-e3ba4b9c5998 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.532606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bf619465-d352-4fec-b0d9-9525a50a58e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7db36b258cc40a5b208f48cd2ae12e7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:46.540261 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2283/4613] 10.222.0.23 () {58 vars in 1263 bytes} [Sat Aug 20 21:09:46 2022] GET /identity/v3/OS-OAUTH1/consumers/f087e879406048e4b870b548bbf2d1c8 => generated 221 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:09:46.542764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dd7eb-b919-4c89-a129-e3ba4b9c5998 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.552773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bf619465-d352-4fec-b0d9-9525a50a58e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7db36b258cc40a5b208f48cd2ae12e7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:46.553499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bf619465-d352-4fec-b0d9-9525a50a58e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7db36b258cc40a5b208f48cd2ae12e7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:46.555457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dd7eb-b919-4c89-a129-e3ba4b9c5998 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.560284 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2331/4614] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:46 2022] DELETE /identity/v3/users/f7db36b258cc40a5b208f48cd2ae12e7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:46.561982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.562633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.563076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.563295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/f087e879406048e4b870b548bbf2d1c8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.563864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=f087e879406048e4b870b548bbf2d1c8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.564413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a81af-d558-4250-8e9f-f48afc348aae None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.564679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a81af-d558-4250-8e9f-f48afc348aae None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.564946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a81af-d558-4250-8e9f-f48afc348aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.565064 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:46.565064 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:09:46.565064 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:09:46.566077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.566349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invalidating token cache because consumer f087e879406048e4b870b548bbf2d1c8 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:46.566651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:46.567477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:46.587032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-870a81af-d558-4250-8e9f-f48afc348aae None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:46.588285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-870a81af-d558-4250-8e9f-f48afc348aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:46.591761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'f087e879406048e4b870b548bbf2d1c8', 'request_id': 'req-016895d8-8b8f-411c-af22-7f216c7ead17'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:46.612043 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2284/4615] 10.222.0.23 () {58 vars in 1266 bytes} [Sat Aug 20 21:09:46 2022] DELETE /identity/v3/OS-OAUTH1/consumers/f087e879406048e4b870b548bbf2d1c8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:46.617166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.618790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:46.633506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016895d8-8b8f-411c-af22-7f216c7ead17 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.672476 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2332/4616] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:46.688677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a81af-d558-4250-8e9f-f48afc348aae None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.692098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-870a81af-d558-4250-8e9f-f48afc348aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:46.707376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2b58e-2ba9-472d-a884-0f76c45284bd tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.708407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2b58e-2ba9-472d-a884-0f76c45284bd tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.708638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2b58e-2ba9-472d-a884-0f76c45284bd tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.708906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2b58e-2ba9-472d-a884-0f76c45284bd tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.710502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab2b58e-2ba9-472d-a884-0f76c45284bd tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.715710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab2b58e-2ba9-472d-a884-0f76c45284bd tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.744644 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2285/4617] 10.222.0.23 () {60 vars in 1185 bytes} [Sat Aug 20 21:09:46 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:46.748279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2b58e-2ba9-472d-a884-0f76c45284bd tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.759601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:46.760643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2b58e-2ba9-472d-a884-0f76c45284bd tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.765852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64f801-f75e-4aac-9892-633bf0b6074b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.766601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64f801-f75e-4aac-9892-633bf0b6074b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.766888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64f801-f75e-4aac-9892-633bf0b6074b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.767159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64f801-f75e-4aac-9892-633bf0b6074b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/796a2b45b242439a8f89ace018900904` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.767811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64f801-f75e-4aac-9892-633bf0b6074b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=796a2b45b242439a8f89ace018900904)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.769441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64f801-f75e-4aac-9892-633bf0b6074b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.775240 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2286/4618] 10.222.0.23 () {58 vars in 1263 bytes} [Sat Aug 20 21:09:46 2022] GET /identity/v3/OS-OAUTH1/consumers/796a2b45b242439a8f89ace018900904 => generated 220 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:46.778869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64f801-f75e-4aac-9892-633bf0b6074b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.785400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:46.788962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a81af-d558-4250-8e9f-f48afc348aae None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.790401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64f801-f75e-4aac-9892-633bf0b6074b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.795062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f6f7e-92dc-4f18-9fc6-24a8dee32548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.795828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f6f7e-92dc-4f18-9fc6-24a8dee32548 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.796129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f6f7e-92dc-4f18-9fc6-24a8dee32548 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.796864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f6f7e-92dc-4f18-9fc6-24a8dee32548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.797376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57a5a8-da38-4b48-b710-5bff6dc4a927 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.798028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57a5a8-da38-4b48-b710-5bff6dc4a927 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.798804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57a5a8-da38-4b48-b710-5bff6dc4a927 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.799357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57a5a8-da38-4b48-b710-5bff6dc4a927 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/796a2b45b242439a8f89ace018900904` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.800413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57a5a8-da38-4b48-b710-5bff6dc4a927 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=796a2b45b242439a8f89ace018900904)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.802583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57a5a8-da38-4b48-b710-5bff6dc4a927 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.803045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c57a5a8-da38-4b48-b710-5bff6dc4a927 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invalidating token cache because consumer 796a2b45b242439a8f89ace018900904 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:46.803400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c57a5a8-da38-4b48-b710-5bff6dc4a927 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:46.804168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c57a5a8-da38-4b48-b710-5bff6dc4a927 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:46.807199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f6f7e-92dc-4f18-9fc6-24a8dee32548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.808649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f6f7e-92dc-4f18-9fc6-24a8dee32548 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.816312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c57a5a8-da38-4b48-b710-5bff6dc4a927 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '796a2b45b242439a8f89ace018900904', 'request_id': 'req-9c57a5a8-da38-4b48-b710-5bff6dc4a927'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:46.819154 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2333/4619] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:09:46.831868 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2287/4620] 10.222.0.23 () {58 vars in 1266 bytes} [Sat Aug 20 21:09:46 2022] DELETE /identity/v3/OS-OAUTH1/consumers/796a2b45b242439a8f89ace018900904 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:46.836468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f6f7e-92dc-4f18-9fc6-24a8dee32548 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.847046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f6f7e-92dc-4f18-9fc6-24a8dee32548 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.855717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9313e-e669-40db-b917-096ee24d43a1 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.855717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9313e-e669-40db-b917-096ee24d43a1 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.855717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9313e-e669-40db-b917-096ee24d43a1 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.856359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9313e-e669-40db-b917-096ee24d43a1 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/796a2b45b242439a8f89ace018900904` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.857301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9313e-e669-40db-b917-096ee24d43a1 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=796a2b45b242439a8f89ace018900904)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.857974 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-d4a9313e-e669-40db-b917-096ee24d43a1 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:46.857974 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:09:46.857974 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:09:46.858849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9313e-e669-40db-b917-096ee24d43a1 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-d4a9313e-e669-40db-b917-096ee24d43a1 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 100, in get Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_consumer(consumer_id) Aug 20 21:09:46.880632 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 95, in _get_consumer Aug 20 21:09:46.881890 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.oauth_api.get_consumer(consumer_id)) Aug 20 21:09:46.881890 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 101, in get_consumer Aug 20 21:09:46.881890 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.get_consumer_with_secret(consumer_id)) Aug 20 21:09:46.881890 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 96, in get_consumer_with_secret Aug 20 21:09:46.881890 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Aug 20 21:09:46.881890 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Aug 20 21:09:46.881890 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Aug 20 21:09:46.881890 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Aug 20 21:09:46.881890 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:09:46.885803 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2334/4621] 10.222.0.23 () {58 vars in 1263 bytes} [Sat Aug 20 21:09:46 2022] GET /identity/v3/OS-OAUTH1/consumers/796a2b45b242439a8f89ace018900904 => generated 74 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 20 21:09:46.886222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57a5a8-da38-4b48-b710-5bff6dc4a927 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.887892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9c57a5a8-da38-4b48-b710-5bff6dc4a927 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:46.902380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57a5a8-da38-4b48-b710-5bff6dc4a927 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.906956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4d350-5491-477e-b968-afe60cfed556 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.907594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4d350-5491-477e-b968-afe60cfed556 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.907869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4d350-5491-477e-b968-afe60cfed556 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.908135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4d350-5491-477e-b968-afe60cfed556 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/796a2b45b242439a8f89ace018900904` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.908782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4d350-5491-477e-b968-afe60cfed556 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=796a2b45b242439a8f89ace018900904)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.910374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4d350-5491-477e-b968-afe60cfed556 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.910626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e4e4d350-5491-477e-b968-afe60cfed556 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invalidating token cache because consumer 796a2b45b242439a8f89ace018900904 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:46.910949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e4e4d350-5491-477e-b968-afe60cfed556 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:46.911705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e4e4d350-5491-477e-b968-afe60cfed556 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-e4e4d350-5491-477e-b968-afe60cfed556 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 123, in delete Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application PROVIDERS.oauth_api.delete_consumer( Aug 20 21:09:46.928514 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:46.929730 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:46.929730 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/core.py", line 139, in delete_consumer Aug 20 21:09:46.929730 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ret = self.driver.delete_consumer(consumer_id) Aug 20 21:09:46.929730 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 133, in delete_consumer Aug 20 21:09:46.929730 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._delete_consumer(session, consumer_id) Aug 20 21:09:46.929730 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 110, in _delete_consumer Aug 20 21:09:46.929730 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Aug 20 21:09:46.929730 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Aug 20 21:09:46.929730 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Aug 20 21:09:46.929730 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Aug 20 21:09:46.929730 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:46.930864 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2288/4622] 10.222.0.23 () {58 vars in 1266 bytes} [Sat Aug 20 21:09:46 2022] DELETE /identity/v3/OS-OAUTH1/consumers/796a2b45b242439a8f89ace018900904 => generated 74 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 21:09:46.935811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9313e-e669-40db-b917-096ee24d43a1 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:46.937591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d4a9313e-e669-40db-b917-096ee24d43a1 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:46.952345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9313e-e669-40db-b917-096ee24d43a1 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:46.959881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4006a-98a4-4f7a-8efd-d8d7d01807ca tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:46.960653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4006a-98a4-4f7a-8efd-d8d7d01807ca tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:46.961298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4006a-98a4-4f7a-8efd-d8d7d01807ca tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:46.961298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4006a-98a4-4f7a-8efd-d8d7d01807ca tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:46.961845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4006a-98a4-4f7a-8efd-d8d7d01807ca tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:46.963277 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-51e4006a-98a4-4f7a-8efd-d8d7d01807ca tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:46.963277 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:09:46.963277 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:09:46.964181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4006a-98a4-4f7a-8efd-d8d7d01807ca tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:46.987443 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2335/4623] 10.222.0.23 () {60 vars in 1185 bytes} [Sat Aug 20 21:09:46 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:46.988450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4d350-5491-477e-b968-afe60cfed556 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:47.001543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4d350-5491-477e-b968-afe60cfed556 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:47.007786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62773ec3-a80d-481f-90b3-d22175fdbaed tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:47.008552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62773ec3-a80d-481f-90b3-d22175fdbaed tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:47.008923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62773ec3-a80d-481f-90b3-d22175fdbaed tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:47.009289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62773ec3-a80d-481f-90b3-d22175fdbaed tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:47.010017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62773ec3-a80d-481f-90b3-d22175fdbaed tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:47.011721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62773ec3-a80d-481f-90b3-d22175fdbaed tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:47.028418 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2289/4624] 10.222.0.23 () {60 vars in 1185 bytes} [Sat Aug 20 21:09:46 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:47.032437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4006a-98a4-4f7a-8efd-d8d7d01807ca tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:47.043073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4006a-98a4-4f7a-8efd-d8d7d01807ca tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:47.047760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3442a1-b988-4ca7-9e06-28736f1c77e0 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:47.048397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3442a1-b988-4ca7-9e06-28736f1c77e0 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:47.048682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3442a1-b988-4ca7-9e06-28736f1c77e0 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:47.048952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3442a1-b988-4ca7-9e06-28736f1c77e0 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:47.049514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3442a1-b988-4ca7-9e06-28736f1c77e0 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:list_consumers()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:47.050593 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-cb3442a1-b988-4ca7-9e06-28736f1c77e0 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_consumers": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:47.050593 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:09:47.050593 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:09:47.051533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3442a1-b988-4ca7-9e06-28736f1c77e0 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:47.057615 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2336/4625] 10.222.0.23 () {58 vars in 1164 bytes} [Sat Aug 20 21:09:47 2022] GET /identity/v3/OS-OAUTH1/consumers => generated 536 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:09:47.061576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62773ec3-a80d-481f-90b3-d22175fdbaed tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:47.072217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62773ec3-a80d-481f-90b3-d22175fdbaed tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:47.076929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be4c9b-9e80-4285-ad9d-ae19b0cbf413 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:47.077564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be4c9b-9e80-4285-ad9d-ae19b0cbf413 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:47.077839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be4c9b-9e80-4285-ad9d-ae19b0cbf413 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:47.078103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be4c9b-9e80-4285-ad9d-ae19b0cbf413 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/0db45afa8f6642d796743596e036c71e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:47.078742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be4c9b-9e80-4285-ad9d-ae19b0cbf413 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=0db45afa8f6642d796743596e036c71e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:47.080347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be4c9b-9e80-4285-ad9d-ae19b0cbf413 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:47.080674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17be4c9b-9e80-4285-ad9d-ae19b0cbf413 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invalidating token cache because consumer 0db45afa8f6642d796743596e036c71e has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:47.080947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17be4c9b-9e80-4285-ad9d-ae19b0cbf413 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.081614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17be4c9b-9e80-4285-ad9d-ae19b0cbf413 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.095519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17be4c9b-9e80-4285-ad9d-ae19b0cbf413 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '0db45afa8f6642d796743596e036c71e', 'request_id': 'req-17be4c9b-9e80-4285-ad9d-ae19b0cbf413'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.109975 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2290/4626] 10.222.0.23 () {58 vars in 1266 bytes} [Sat Aug 20 21:09:47 2022] DELETE /identity/v3/OS-OAUTH1/consumers/0db45afa8f6642d796743596e036c71e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:47.114268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3442a1-b988-4ca7-9e06-28736f1c77e0 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:47.117035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cb3442a1-b988-4ca7-9e06-28736f1c77e0 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:47.131067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3442a1-b988-4ca7-9e06-28736f1c77e0 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:47.136027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:47.137789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:47.138066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:47.138347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/e636cfb90171499a9eedbd55d3d75d56` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:47.138924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=e636cfb90171499a9eedbd55d3d75d56)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:47.140010 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:47.140010 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:09:47.140010 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:09:47.141133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:47.141444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invalidating token cache because consumer e636cfb90171499a9eedbd55d3d75d56 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:47.141697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.142792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.162222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'e636cfb90171499a9eedbd55d3d75d56', 'request_id': 'req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.178015 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2337/4627] 10.222.0.23 () {58 vars in 1266 bytes} [Sat Aug 20 21:09:47 2022] DELETE /identity/v3/OS-OAUTH1/consumers/e636cfb90171499a9eedbd55d3d75d56 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:47.183218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be4c9b-9e80-4285-ad9d-ae19b0cbf413 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:47.197518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be4c9b-9e80-4285-ad9d-ae19b0cbf413 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:47.202648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a9a44-e9eb-4a28-ac7a-43f3c4b1184d tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:47.203604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a9a44-e9eb-4a28-ac7a-43f3c4b1184d tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:47.203824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a9a44-e9eb-4a28-ac7a-43f3c4b1184d tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:47.204092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a9a44-e9eb-4a28-ac7a-43f3c4b1184d tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:47.204829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a9a44-e9eb-4a28-ac7a-43f3c4b1184d tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:47.207067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:47.207418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a9a44-e9eb-4a28-ac7a-43f3c4b1184d tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:47.208850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:47.224361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ce11c-8bb9-42c8-bbe0-cba082820bbe tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:47.231478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:47.232241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:47.232559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:47.232824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] PATH_INFO: `/v3/projects/897ea60a208d4878afdf74c901a018af` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:47.234035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=897ea60a208d4878afdf74c901a018af)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:47.235948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:47.249204 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2291/4628] 10.222.0.23 () {60 vars in 1185 bytes} [Sat Aug 20 21:09:47 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:47.250734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a9a44-e9eb-4a28-ac7a-43f3c4b1184d tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:47.253743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-421a9a44-e9eb-4a28-ac7a-43f3c4b1184d tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:47.271067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a9a44-e9eb-4a28-ac7a-43f3c4b1184d tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:47.276986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33277b-e314-4768-8fc4-c23f826cf71e tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:47.277704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33277b-e314-4768-8fc4-c23f826cf71e tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:47.278008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33277b-e314-4768-8fc4-c23f826cf71e tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:47.278297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33277b-e314-4768-8fc4-c23f826cf71e tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/8ca0b56d53704fe49e58dfe68a06a6f7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:47.278985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33277b-e314-4768-8fc4-c23f826cf71e tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:update_consumer(consumer_id=8ca0b56d53704fe49e58dfe68a06a6f7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:47.280132 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-ca33277b-e314-4768-8fc4-c23f826cf71e tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:47.280132 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:09:47.280132 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:09:47.281365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33277b-e314-4768-8fc4-c23f826cf71e tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:47.303961 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2292/4629] 10.222.0.23 () {60 vars in 1285 bytes} [Sat Aug 20 21:09:47 2022] PATCH /identity/v3/OS-OAUTH1/consumers/8ca0b56d53704fe49e58dfe68a06a6f7 => generated 220 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:47.306022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33277b-e314-4768-8fc4-c23f826cf71e tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:47.309617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '897ea60a208d4878afdf74c901a018af', 'request_id': 'req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.316047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33277b-e314-4768-8fc4-c23f826cf71e tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:47.319657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '897ea60a208d4878afdf74c901a018af', 'request_id': 'req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.326925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '897ea60a208d4878afdf74c901a018af', 'request_id': 'req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.340018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] The token cache is being invalidate because project 897ea60a208d4878afdf74c901a018af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:47.340297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.340986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.343092 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2338/4630] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:47 2022] DELETE /identity/v3/projects/897ea60a208d4878afdf74c901a018af => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:47.346682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:47.348043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:47.384239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd126ce6-412d-46c3-a81b-cf9452bf6727 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:47.385371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd126ce6-412d-46c3-a81b-cf9452bf6727 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:47.385747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd126ce6-412d-46c3-a81b-cf9452bf6727 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:47.386114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd126ce6-412d-46c3-a81b-cf9452bf6727 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/8ca0b56d53704fe49e58dfe68a06a6f7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:47.386921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd126ce6-412d-46c3-a81b-cf9452bf6727 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=8ca0b56d53704fe49e58dfe68a06a6f7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:47.388879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd126ce6-412d-46c3-a81b-cf9452bf6727 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:47.395482 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2293/4631] 10.222.0.23 () {58 vars in 1263 bytes} [Sat Aug 20 21:09:47 2022] GET /identity/v3/OS-OAUTH1/consumers/8ca0b56d53704fe49e58dfe68a06a6f7 => generated 220 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:09:47.401871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd126ce6-412d-46c3-a81b-cf9452bf6727 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:47.403926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fd126ce6-412d-46c3-a81b-cf9452bf6727 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:47.421151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd126ce6-412d-46c3-a81b-cf9452bf6727 tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:47.426407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c02d3-62c4-445f-b5a6-785f9177697b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ee4b1f2cee495ebd91c5ea720d03f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a095482cd44f48388f9c2befac2e919f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:47.427420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c02d3-62c4-445f-b5a6-785f9177697b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:47.427792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c02d3-62c4-445f-b5a6-785f9177697b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:47.428274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c02d3-62c4-445f-b5a6-785f9177697b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/8ca0b56d53704fe49e58dfe68a06a6f7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:47.428911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f9767-5ab1-49cb-aee4-a4c5374942b6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:47.429052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c02d3-62c4-445f-b5a6-785f9177697b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=8ca0b56d53704fe49e58dfe68a06a6f7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:47.430817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c02d3-62c4-445f-b5a6-785f9177697b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:47.431179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f79c02d3-62c4-445f-b5a6-785f9177697b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invalidating token cache because consumer 8ca0b56d53704fe49e58dfe68a06a6f7 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:47.431557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f79c02d3-62c4-445f-b5a6-785f9177697b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.432321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f79c02d3-62c4-445f-b5a6-785f9177697b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.434226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88946649-4608-4722-a70a-fe3884e959bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:47.435002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88946649-4608-4722-a70a-fe3884e959bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:47.435275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88946649-4608-4722-a70a-fe3884e959bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:47.435644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88946649-4608-4722-a70a-fe3884e959bd admin admin] PATH_INFO: `/v3/users/f1c1ed780e404dc583f7effaf1ccbfe6` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:47.437003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88946649-4608-4722-a70a-fe3884e959bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1c1ed780e404dc583f7effaf1ccbfe6)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:47.439133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88946649-4608-4722-a70a-fe3884e959bd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:47.451404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f79c02d3-62c4-445f-b5a6-785f9177697b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '8ca0b56d53704fe49e58dfe68a06a6f7', 'request_id': 'req-f79c02d3-62c4-445f-b5a6-785f9177697b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.454449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:47.476339 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2294/4632] 10.222.0.23 () {58 vars in 1266 bytes} [Sat Aug 20 21:09:47 2022] DELETE /identity/v3/OS-OAUTH1/consumers/8ca0b56d53704fe49e58dfe68a06a6f7 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:47.481463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c02d3-62c4-445f-b5a6-785f9177697b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:47.483314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f79c02d3-62c4-445f-b5a6-785f9177697b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:47.513887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c02d3-62c4-445f-b5a6-785f9177697b tempest-OAUTHConsumersV3Test-1744336105 tempest-OAUTHConsumersV3Test-1744336105-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:47.519380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-88946649-4608-4722-a70a-fe3884e959bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1c1ed780e404dc583f7effaf1ccbfe6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.529176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:47.529838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:47.530141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:47.530414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] PATH_INFO: `/v3/users/91eb3c6c60724027baf734cf79e28b55` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:47.533554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=91eb3c6c60724027baf734cf79e28b55)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:47.534771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-88946649-4608-4722-a70a-fe3884e959bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1c1ed780e404dc583f7effaf1ccbfe6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.536076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:47.546506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-88946649-4608-4722-a70a-fe3884e959bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1c1ed780e404dc583f7effaf1ccbfe6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.559111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-88946649-4608-4722-a70a-fe3884e959bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1c1ed780e404dc583f7effaf1ccbfe6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.560815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-88946649-4608-4722-a70a-fe3884e959bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1c1ed780e404dc583f7effaf1ccbfe6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.567584 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2339/4633] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:47 2022] DELETE /identity/v3/users/f1c1ed780e404dc583f7effaf1ccbfe6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:47.589169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=5,keystone:UPDATE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:47.637280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91eb3c6c60724027baf734cf79e28b55'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.649063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91eb3c6c60724027baf734cf79e28b55'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.665709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91eb3c6c60724027baf734cf79e28b55'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.668487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91eb3c6c60724027baf734cf79e28b55'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.668864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91eb3c6c60724027baf734cf79e28b55'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:47.686648 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2295/4634] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:47 2022] DELETE /identity/v3/users/91eb3c6c60724027baf734cf79e28b55 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:47.691372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2f2e3-9452-4850-91bf-51aeeb329187 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:47.691646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2f2e3-9452-4850-91bf-51aeeb329187 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:47.691941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2f2e3-9452-4850-91bf-51aeeb329187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:47.712996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2ae2f2e3-9452-4850-91bf-51aeeb329187 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:47.714226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2ae2f2e3-9452-4850-91bf-51aeeb329187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:47.778410 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2340/4635] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:47.789808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:47.791267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:47.869038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4e0e4-2976-4dce-b50b-7fba0479a0ab admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:47.873534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b04b1-7023-4aa5-9015-5f24d368e708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:47.874244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b04b1-7023-4aa5-9015-5f24d368e708 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:47.874586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b04b1-7023-4aa5-9015-5f24d368e708 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:47.875136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b04b1-7023-4aa5-9015-5f24d368e708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:47.887622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b04b1-7023-4aa5-9015-5f24d368e708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:47.889223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b04b1-7023-4aa5-9015-5f24d368e708 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:47.902463 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2296/4636] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:48.307086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2f2e3-9452-4850-91bf-51aeeb329187 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:48.308690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2ae2f2e3-9452-4850-91bf-51aeeb329187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:48.342589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2f2e3-9452-4850-91bf-51aeeb329187 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:48.352995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ac209-c1fe-404a-a056-a725867e4004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:48.354520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ac209-c1fe-404a-a056-a725867e4004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:48.355097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ac209-c1fe-404a-a056-a725867e4004 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:48.355724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ac209-c1fe-404a-a056-a725867e4004 admin admin] PATH_INFO: `/v3/projects/236663ad486f473e85f641f53efb4069` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:48.361438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ac209-c1fe-404a-a056-a725867e4004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=236663ad486f473e85f641f53efb4069)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:48.363698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ac209-c1fe-404a-a056-a725867e4004 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:48.427296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b04b1-7023-4aa5-9015-5f24d368e708 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:48.429298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-be6b04b1-7023-4aa5-9015-5f24d368e708 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:48.453007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-082ac209-c1fe-404a-a056-a725867e4004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '236663ad486f473e85f641f53efb4069', 'request_id': 'req-082ac209-c1fe-404a-a056-a725867e4004'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.484246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-082ac209-c1fe-404a-a056-a725867e4004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '236663ad486f473e85f641f53efb4069', 'request_id': 'req-082ac209-c1fe-404a-a056-a725867e4004'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.493877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-082ac209-c1fe-404a-a056-a725867e4004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '236663ad486f473e85f641f53efb4069', 'request_id': 'req-082ac209-c1fe-404a-a056-a725867e4004'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.499971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b04b1-7023-4aa5-9015-5f24d368e708 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:48.509912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-082ac209-c1fe-404a-a056-a725867e4004 admin admin] The token cache is being invalidate because project 236663ad486f473e85f641f53efb4069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:48.510287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-082ac209-c1fe-404a-a056-a725867e4004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.511567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-082ac209-c1fe-404a-a056-a725867e4004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.515013 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2341/4637] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:48 2022] DELETE /identity/v3/projects/236663ad486f473e85f641f53efb4069 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:48.519866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd2143-a9f1-4078-b119-b6d16b37d860 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:48.520408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd2143-a9f1-4078-b119-b6d16b37d860 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:48.520768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd2143-a9f1-4078-b119-b6d16b37d860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:48.546627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-16cd2143-a9f1-4078-b119-b6d16b37d860 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:48.547850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-16cd2143-a9f1-4078-b119-b6d16b37d860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:48.560913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:48.659702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:48.660359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:48.660786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:48.661011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] PATH_INFO: `/v3/projects/be3b845ecb4446b78fb729cfd55bf850` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:48.662204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3b845ecb4446b78fb729cfd55bf850)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:48.664133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:48.672429 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2342/4638] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:48.678980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeae53-3a64-453b-95d0-24886935f9e5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:48.680690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeae53-3a64-453b-95d0-24886935f9e5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:48.681132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeae53-3a64-453b-95d0-24886935f9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:48.733121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5abeae53-3a64-453b-95d0-24886935f9e5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:48.734303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5abeae53-3a64-453b-95d0-24886935f9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:48.746360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:48.750813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3b845ecb4446b78fb729cfd55bf850', 'request_id': 'req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.762718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3b845ecb4446b78fb729cfd55bf850', 'request_id': 'req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.772280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3b845ecb4446b78fb729cfd55bf850', 'request_id': 'req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.789791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] The token cache is being invalidate because project be3b845ecb4446b78fb729cfd55bf850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:48.790081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.790896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.792919 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2297/4639] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:48 2022] DELETE /identity/v3/projects/be3b845ecb4446b78fb729cfd55bf850 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:48.797079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:48.798570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:48.821159 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2343/4640] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:48.825666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeae53-3a64-453b-95d0-24886935f9e5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:48.837622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeae53-3a64-453b-95d0-24886935f9e5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:48.844980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479c3c0-b6f9-4a88-99b9-495687a72726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:48.845600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479c3c0-b6f9-4a88-99b9-495687a72726 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:48.845871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479c3c0-b6f9-4a88-99b9-495687a72726 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:48.846142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479c3c0-b6f9-4a88-99b9-495687a72726 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:48.846794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479c3c0-b6f9-4a88-99b9-495687a72726 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:48.847092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479c3c0-b6f9-4a88-99b9-495687a72726 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:48.848864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479c3c0-b6f9-4a88-99b9-495687a72726 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:48.868417 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2344/4641] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:48.871504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479c3c0-b6f9-4a88-99b9-495687a72726 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:48.882334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479c3c0-b6f9-4a88-99b9-495687a72726 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:48.886940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeffc83-5dd4-401f-aa7f-1193e4c15812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:48.887535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeffc83-5dd4-401f-aa7f-1193e4c15812 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:48.887813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeffc83-5dd4-401f-aa7f-1193e4c15812 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:48.888080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeffc83-5dd4-401f-aa7f-1193e4c15812 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:48.888742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeffc83-5dd4-401f-aa7f-1193e4c15812 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:48.890646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeffc83-5dd4-401f-aa7f-1193e4c15812 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:48.893366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4a81d-3a13-4e22-8e3c-79c2be5ce094 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:48.899332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:48.900311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:48.900623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:48.900895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] PATH_INFO: `/v3/users/37ee4b1f2cee495ebd91c5ea720d03f1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:48.902082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37ee4b1f2cee495ebd91c5ea720d03f1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:48.904073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:48.923084 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2345/4642] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:48 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:48.924677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeffc83-5dd4-401f-aa7f-1193e4c15812 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:48.968948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37ee4b1f2cee495ebd91c5ea720d03f1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.981252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37ee4b1f2cee495ebd91c5ea720d03f1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.998029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37ee4b1f2cee495ebd91c5ea720d03f1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.998881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37ee4b1f2cee495ebd91c5ea720d03f1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:48.999499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37ee4b1f2cee495ebd91c5ea720d03f1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:49.002012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeffc83-5dd4-401f-aa7f-1193e4c15812 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:49.007630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b2b75-8ee0-44c6-b50f-a902ed792c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:49.008542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b2b75-8ee0-44c6-b50f-a902ed792c6d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.008842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b2b75-8ee0-44c6-b50f-a902ed792c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.009107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b2b75-8ee0-44c6-b50f-a902ed792c6d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.009721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b2b75-8ee0-44c6-b50f-a902ed792c6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:49.011634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b2b75-8ee0-44c6-b50f-a902ed792c6d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:49.013105 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2298/4643] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:48 2022] DELETE /identity/v3/users/37ee4b1f2cee495ebd91c5ea720d03f1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:49.037603 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2346/4644] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:48 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:49.041964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:49.043388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:49.107852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:49.125203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc31310-964f-443d-bb97-664fe20d8755 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:49.131962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21c1d0-bbf0-4a21-a775-4d9f22f483eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:49.132708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21c1d0-bbf0-4a21-a775-4d9f22f483eb admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.132986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21c1d0-bbf0-4a21-a775-4d9f22f483eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.133311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21c1d0-bbf0-4a21-a775-4d9f22f483eb admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.133904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21c1d0-bbf0-4a21-a775-4d9f22f483eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:49.135645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21c1d0-bbf0-4a21-a775-4d9f22f483eb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:49.151044 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2299/4645] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:49.152329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b2b75-8ee0-44c6-b50f-a902ed792c6d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:49.164098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b2b75-8ee0-44c6-b50f-a902ed792c6d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:49.170178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc5919-f135-4c5f-b393-f73ea86cf930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:49.171050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc5919-f135-4c5f-b393-f73ea86cf930 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.171268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc5919-f135-4c5f-b393-f73ea86cf930 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.171534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc5919-f135-4c5f-b393-f73ea86cf930 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.172175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc5919-f135-4c5f-b393-f73ea86cf930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:49.173842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc5919-f135-4c5f-b393-f73ea86cf930 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:49.175395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:49.206893 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2347/4646] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:49.217093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21c1d0-bbf0-4a21-a775-4d9f22f483eb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:49.249162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21c1d0-bbf0-4a21-a775-4d9f22f483eb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:49.255711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163390b-4c93-41dd-9c4e-71463c392115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:49.256822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163390b-4c93-41dd-9c4e-71463c392115 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.257205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163390b-4c93-41dd-9c4e-71463c392115 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.257786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163390b-4c93-41dd-9c4e-71463c392115 admin admin] PATH_INFO: `/v3/projects/281f765786b8478da7aed89922e8305f/users/70f5c962d0ec4c5b8f0be043d31e1bbb/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.280335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163390b-4c93-41dd-9c4e-71463c392115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281f765786b8478da7aed89922e8305f, user_id=70f5c962d0ec4c5b8f0be043d31e1bbb, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:49.284733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163390b-4c93-41dd-9c4e-71463c392115 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:49.304438 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2300/4647] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:49 2022] PUT /identity/v3/projects/281f765786b8478da7aed89922e8305f/users/70f5c962d0ec4c5b8f0be043d31e1bbb/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:49.314893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100958b-05fd-472c-9744-370e3f8c32ce None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.315446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100958b-05fd-472c-9744-370e3f8c32ce None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.315446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100958b-05fd-472c-9744-370e3f8c32ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.346452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7100958b-05fd-472c-9744-370e3f8c32ce None None] MFA Rules not processed for user `70f5c962d0ec4c5b8f0be043d31e1bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:49.347819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7100958b-05fd-472c-9744-370e3f8c32ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:49.348815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163390b-4c93-41dd-9c4e-71463c392115 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:49.350689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3163390b-4c93-41dd-9c4e-71463c392115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:49.419035 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2348/4648] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:49.423344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100958b-05fd-472c-9744-370e3f8c32ce None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:49.436255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163390b-4c93-41dd-9c4e-71463c392115 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:49.441335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03437de1-4f45-4294-a92b-6180bc1339c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:49.442261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03437de1-4f45-4294-a92b-6180bc1339c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.442534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03437de1-4f45-4294-a92b-6180bc1339c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.442839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03437de1-4f45-4294-a92b-6180bc1339c9 admin admin] PATH_INFO: `/v3/projects/a095482cd44f48388f9c2befac2e919f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.444060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03437de1-4f45-4294-a92b-6180bc1339c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a095482cd44f48388f9c2befac2e919f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:49.446425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03437de1-4f45-4294-a92b-6180bc1339c9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:49.490412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100958b-05fd-472c-9744-370e3f8c32ce None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:49.496951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03437de1-4f45-4294-a92b-6180bc1339c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a095482cd44f48388f9c2befac2e919f', 'request_id': 'req-03437de1-4f45-4294-a92b-6180bc1339c9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:49.504012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03437de1-4f45-4294-a92b-6180bc1339c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a095482cd44f48388f9c2befac2e919f', 'request_id': 'req-03437de1-4f45-4294-a92b-6180bc1339c9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:49.512808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03437de1-4f45-4294-a92b-6180bc1339c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a095482cd44f48388f9c2befac2e919f', 'request_id': 'req-03437de1-4f45-4294-a92b-6180bc1339c9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:49.524290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03437de1-4f45-4294-a92b-6180bc1339c9 admin admin] The token cache is being invalidate because project a095482cd44f48388f9c2befac2e919f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:49.524606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03437de1-4f45-4294-a92b-6180bc1339c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:49.525261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-03437de1-4f45-4294-a92b-6180bc1339c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:49.527029 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2301/4649] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:49 2022] DELETE /identity/v3/projects/a095482cd44f48388f9c2befac2e919f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:49.532493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a1bfd-745b-47b8-8c78-96b94fa123cc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.533072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a1bfd-745b-47b8-8c78-96b94fa123cc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.533482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a1bfd-745b-47b8-8c78-96b94fa123cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.553599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96d424-154a-4838-ab6b-cee3528464ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:49.554203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96d424-154a-4838-ab6b-cee3528464ce admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.554473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96d424-154a-4838-ab6b-cee3528464ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.554731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96d424-154a-4838-ab6b-cee3528464ce admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.555337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96d424-154a-4838-ab6b-cee3528464ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:49.556348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-930a1bfd-745b-47b8-8c78-96b94fa123cc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:49.557410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96d424-154a-4838-ab6b-cee3528464ce admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:49.557591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-930a1bfd-745b-47b8-8c78-96b94fa123cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:49.565723 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2302/4650] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:49.575297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20453da-4e80-4c07-83c0-dc1532eac804 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.575562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20453da-4e80-4c07-83c0-dc1532eac804 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.575822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20453da-4e80-4c07-83c0-dc1532eac804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.591100 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2349/4651] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:49 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:49.593324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96d424-154a-4838-ab6b-cee3528464ce admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:49.594908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bb96d424-154a-4838-ab6b-cee3528464ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:49.603322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b20453da-4e80-4c07-83c0-dc1532eac804 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:49.605157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b20453da-4e80-4c07-83c0-dc1532eac804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:49.618435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:49.659724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:49.687088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96d424-154a-4838-ab6b-cee3528464ce admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:49.696066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0d1ff-0799-46ba-a24f-b059a53fc45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:49.696715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0d1ff-0799-46ba-a24f-b059a53fc45c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.696986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0d1ff-0799-46ba-a24f-b059a53fc45c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.697249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0d1ff-0799-46ba-a24f-b059a53fc45c admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.697866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0d1ff-0799-46ba-a24f-b059a53fc45c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:49.698436 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2303/4652] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:49.699935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0d1ff-0799-46ba-a24f-b059a53fc45c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:49.702736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20453da-4e80-4c07-83c0-dc1532eac804 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:49.712516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20453da-4e80-4c07-83c0-dc1532eac804 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:49.719342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c69f36-fe73-4cdb-bd99-cfb5971f699d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:49.719342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c69f36-fe73-4cdb-bd99-cfb5971f699d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.719342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c69f36-fe73-4cdb-bd99-cfb5971f699d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.719342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c69f36-fe73-4cdb-bd99-cfb5971f699d admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.719827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c69f36-fe73-4cdb-bd99-cfb5971f699d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:49.719827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c69f36-fe73-4cdb-bd99-cfb5971f699d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:49.721533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c69f36-fe73-4cdb-bd99-cfb5971f699d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:49.731756 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2350/4653] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:49 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:49.732886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0d1ff-0799-46ba-a24f-b059a53fc45c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:49.736814 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2304/4654] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:09:49.741168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c69f36-fe73-4cdb-bd99-cfb5971f699d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:49.743396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0d1ff-0799-46ba-a24f-b059a53fc45c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:49.748083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8830d37-d510-490c-a090-903acfca28de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:49.748753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8830d37-d510-490c-a090-903acfca28de admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.748995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8830d37-d510-490c-a090-903acfca28de admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.749261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8830d37-d510-490c-a090-903acfca28de admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.749900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8830d37-d510-490c-a090-903acfca28de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:49.753122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8830d37-d510-490c-a090-903acfca28de admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:49.756026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c69f36-fe73-4cdb-bd99-cfb5971f699d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:49.762895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4221ff8-10d1-4a81-905c-deca18f5d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:49.763580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4221ff8-10d1-4a81-905c-deca18f5d4ee admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.763921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4221ff8-10d1-4a81-905c-deca18f5d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.764254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4221ff8-10d1-4a81-905c-deca18f5d4ee admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.765019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4221ff8-10d1-4a81-905c-deca18f5d4ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:49.768448 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2351/4655] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:49.768598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4221ff8-10d1-4a81-905c-deca18f5d4ee admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:49.769715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8830d37-d510-490c-a090-903acfca28de admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:49.781321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8830d37-d510-490c-a090-903acfca28de admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:49.791100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16732c8b-1a66-431f-97de-f39678b0602e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:49.791764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16732c8b-1a66-431f-97de-f39678b0602e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.792109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16732c8b-1a66-431f-97de-f39678b0602e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.792466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16732c8b-1a66-431f-97de-f39678b0602e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.793170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16732c8b-1a66-431f-97de-f39678b0602e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:49.794889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16732c8b-1a66-431f-97de-f39678b0602e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:49.808408 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2305/4656] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:49 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:49.810772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4221ff8-10d1-4a81-905c-deca18f5d4ee admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:49.818867 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2352/4657] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:09:49.825554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16732c8b-1a66-431f-97de-f39678b0602e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:49.840482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16732c8b-1a66-431f-97de-f39678b0602e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:49.891123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4221ff8-10d1-4a81-905c-deca18f5d4ee admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:49.899194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a3a24-d7f9-4bc8-b9ae-014cc786d001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:49.899383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a3a24-d7f9-4bc8-b9ae-014cc786d001 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.899635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a3a24-d7f9-4bc8-b9ae-014cc786d001 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.899969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a3a24-d7f9-4bc8-b9ae-014cc786d001 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.900824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a3a24-d7f9-4bc8-b9ae-014cc786d001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:49.903420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a3a24-d7f9-4bc8-b9ae-014cc786d001 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:49.923002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c7c35-d039-46d0-9a2b-d085592ced1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:49.923756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c7c35-d039-46d0-9a2b-d085592ced1b admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.924091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c7c35-d039-46d0-9a2b-d085592ced1b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.924446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c7c35-d039-46d0-9a2b-d085592ced1b admin admin] PATH_INFO: `/v3/projects/e159fa978ca44f24b1e7f2bc7736c317/users/348b88a9e8e34ec7bb89c3883dce53cd/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.933029 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2306/4658] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:49 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:49.933029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39d9b4-962a-442a-8095-45da7d332248 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:49.933029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39d9b4-962a-442a-8095-45da7d332248 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:49.933029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39d9b4-962a-442a-8095-45da7d332248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:49.946223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c7c35-d039-46d0-9a2b-d085592ced1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e159fa978ca44f24b1e7f2bc7736c317, user_id=348b88a9e8e34ec7bb89c3883dce53cd, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:49.950135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c7c35-d039-46d0-9a2b-d085592ced1b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:49.953087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ad39d9b4-962a-442a-8095-45da7d332248 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:49.954275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ad39d9b4-962a-442a-8095-45da7d332248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:49.965220 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2353/4659] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:49 2022] PUT /identity/v3/projects/e159fa978ca44f24b1e7f2bc7736c317/users/348b88a9e8e34ec7bb89c3883dce53cd/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:49.966222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c7c35-d039-46d0-9a2b-d085592ced1b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:50.015418 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2307/4660] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:50.016762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07ed66-0039-42fb-a7af-ed469f17d585 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.017008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07ed66-0039-42fb-a7af-ed469f17d585 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.017271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07ed66-0039-42fb-a7af-ed469f17d585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.039713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c7c35-d039-46d0-9a2b-d085592ced1b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:50.044267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd00da8-e4ad-43b8-b92b-85a354f454ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:50.044894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd00da8-e4ad-43b8-b92b-85a354f454ee admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.045162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd00da8-e4ad-43b8-b92b-85a354f454ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.045424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd00da8-e4ad-43b8-b92b-85a354f454ee admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.046051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd00da8-e4ad-43b8-b92b-85a354f454ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:50.046472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ea07ed66-0039-42fb-a7af-ed469f17d585 None None] MFA Rules not processed for user `348b88a9e8e34ec7bb89c3883dce53cd`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:50.047625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd00da8-e4ad-43b8-b92b-85a354f454ee admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:50.047822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ea07ed66-0039-42fb-a7af-ed469f17d585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:50.057751 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2354/4661] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:50.058666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd00da8-e4ad-43b8-b92b-85a354f454ee admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:50.060242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5bd00da8-e4ad-43b8-b92b-85a354f454ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:50.066621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:50.091787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:50.119544 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2308/4662] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:50.120573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07ed66-0039-42fb-a7af-ed469f17d585 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:50.131794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07ed66-0039-42fb-a7af-ed469f17d585 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:50.136595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3c3a6-a502-42f0-99b1-f113ca6da839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:50.137026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd00da8-e4ad-43b8-b92b-85a354f454ee admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:50.137198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3c3a6-a502-42f0-99b1-f113ca6da839 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.137464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3c3a6-a502-42f0-99b1-f113ca6da839 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.137725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3c3a6-a502-42f0-99b1-f113ca6da839 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.138354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3c3a6-a502-42f0-99b1-f113ca6da839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:50.139946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3c3a6-a502-42f0-99b1-f113ca6da839 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:50.142215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fea384-c4bc-48b8-b530-8787f58f5878 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:50.142887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fea384-c4bc-48b8-b530-8787f58f5878 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.143160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fea384-c4bc-48b8-b530-8787f58f5878 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.143424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fea384-c4bc-48b8-b530-8787f58f5878 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.153878 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2309/4663] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:50.153878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3c3a6-a502-42f0-99b1-f113ca6da839 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:50.153878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e6f3c3a6-a502-42f0-99b1-f113ca6da839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:50.204724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fea384-c4bc-48b8-b530-8787f58f5878 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:50.206253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fea384-c4bc-48b8-b530-8787f58f5878 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:50.214688 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2355/4664] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:09:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:09:50.215678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fea384-c4bc-48b8-b530-8787f58f5878 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:50.215972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3c3a6-a502-42f0-99b1-f113ca6da839 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:50.220849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8a754-c3b3-4fef-a821-39f37d53d9f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:50.221481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8a754-c3b3-4fef-a821-39f37d53d9f1 service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.221754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8a754-c3b3-4fef-a821-39f37d53d9f1 service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.222608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8a754-c3b3-4fef-a821-39f37d53d9f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.224472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-daf8a754-c3b3-4fef-a821-39f37d53d9f1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:50.230722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fea384-c4bc-48b8-b530-8787f58f5878 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:50.236409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee06267-8348-4614-af92-2ba0bfd0af9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:50.237347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee06267-8348-4614-af92-2ba0bfd0af9c admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.237916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee06267-8348-4614-af92-2ba0bfd0af9c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.238282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee06267-8348-4614-af92-2ba0bfd0af9c admin admin] PATH_INFO: `/v3/projects/5cda1ca23e0440998fec70a040112e98/users/d48af3beb4c54fa9a9834b27bc976fba/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.256092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee06267-8348-4614-af92-2ba0bfd0af9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cda1ca23e0440998fec70a040112e98, user_id=d48af3beb4c54fa9a9834b27bc976fba, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:50.259000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee06267-8348-4614-af92-2ba0bfd0af9c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:50.276238 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2356/4665] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:50 2022] PUT /identity/v3/projects/5cda1ca23e0440998fec70a040112e98/users/d48af3beb4c54fa9a9834b27bc976fba/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:50.287041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7afa81-9689-4b72-8b6d-18c6151970ed None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.287314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7afa81-9689-4b72-8b6d-18c6151970ed None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.287704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7afa81-9689-4b72-8b6d-18c6151970ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.299144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8a754-c3b3-4fef-a821-39f37d53d9f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:50.300869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8a754-c3b3-4fef-a821-39f37d53d9f1 service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:50.312228 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2310/4666] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:50.318782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8b7afa81-9689-4b72-8b6d-18c6151970ed None None] MFA Rules not processed for user `d48af3beb4c54fa9a9834b27bc976fba`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:50.320013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8b7afa81-9689-4b72-8b6d-18c6151970ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:50.385821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c7cb2-b269-477a-a7ca-4d2c0af33e8a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.386174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c7cb2-b269-477a-a7ca-4d2c0af33e8a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.386483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c7cb2-b269-477a-a7ca-4d2c0af33e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.395721 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2357/4667] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:50.401527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7afa81-9689-4b72-8b6d-18c6151970ed None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:50.413084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c13c7cb2-b269-477a-a7ca-4d2c0af33e8a None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:50.417466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c13c7cb2-b269-477a-a7ca-4d2c0af33e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:50.428867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:50.488954 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2311/4668] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:09:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:50.489962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c7cb2-b269-477a-a7ca-4d2c0af33e8a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:50.495015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7afa81-9689-4b72-8b6d-18c6151970ed None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:50.500742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50932ad-fa53-444a-aea3-fe2b6fdd7c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:50.501610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50932ad-fa53-444a-aea3-fe2b6fdd7c2c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.501944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50932ad-fa53-444a-aea3-fe2b6fdd7c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.502210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50932ad-fa53-444a-aea3-fe2b6fdd7c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.502831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50932ad-fa53-444a-aea3-fe2b6fdd7c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:50.504810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50932ad-fa53-444a-aea3-fe2b6fdd7c2c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:50.520702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:50.534554 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2358/4669] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:50 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:50.537740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50932ad-fa53-444a-aea3-fe2b6fdd7c2c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:50.551867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c7cb2-b269-477a-a7ca-4d2c0af33e8a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:50.601963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0688dbc-5bac-4804-b4c5-29691717aa12 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:50.602572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0688dbc-5bac-4804-b4c5-29691717aa12 service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.602845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0688dbc-5bac-4804-b4c5-29691717aa12 service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.603111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0688dbc-5bac-4804-b4c5-29691717aa12 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.606594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50932ad-fa53-444a-aea3-fe2b6fdd7c2c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:50.612340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35331e81-2331-4cd3-954a-b56e4ee4880a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:50.612978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35331e81-2331-4cd3-954a-b56e4ee4880a service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.613252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35331e81-2331-4cd3-954a-b56e4ee4880a service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.613525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35331e81-2331-4cd3-954a-b56e4ee4880a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.661099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0688dbc-5bac-4804-b4c5-29691717aa12 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:50.662602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0688dbc-5bac-4804-b4c5-29691717aa12 service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:50.668739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35331e81-2331-4cd3-954a-b56e4ee4880a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:50.670324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35331e81-2331-4cd3-954a-b56e4ee4880a service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:50.672138 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2312/4670] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:50.673008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0688dbc-5bac-4804-b4c5-29691717aa12 service glance] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:50.679115 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2359/4671] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:09:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:09:50.772355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0688dbc-5bac-4804-b4c5-29691717aa12 service glance] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:50.777999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3c551-0399-4b34-a9cd-93a550321dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:50.778736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3c551-0399-4b34-a9cd-93a550321dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.779206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3c551-0399-4b34-a9cd-93a550321dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.779607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3c551-0399-4b34-a9cd-93a550321dc8 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.780691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3c551-0399-4b34-a9cd-93a550321dc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:50.784017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3c551-0399-4b34-a9cd-93a550321dc8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:50.828364 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2313/4672] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:50 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:50.833987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35331e81-2331-4cd3-954a-b56e4ee4880a service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:50.849382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35331e81-2331-4cd3-954a-b56e4ee4880a service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:50.856141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1acd84-2c02-441f-977f-5e9288b6956b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:50.856948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1acd84-2c02-441f-977f-5e9288b6956b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.857225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1acd84-2c02-441f-977f-5e9288b6956b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.857498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1acd84-2c02-441f-977f-5e9288b6956b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.858131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1acd84-2c02-441f-977f-5e9288b6956b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:50.863435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1acd84-2c02-441f-977f-5e9288b6956b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:50.874077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ae420-2eb8-4a55-aa99-3fddbfe091ff None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.874350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ae420-2eb8-4a55-aa99-3fddbfe091ff None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.874616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ae420-2eb8-4a55-aa99-3fddbfe091ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.878939 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2360/4673] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:50.883673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1acd84-2c02-441f-977f-5e9288b6956b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:50.894503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1acd84-2c02-441f-977f-5e9288b6956b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:50.896939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3d1ae420-2eb8-4a55-aa99-3fddbfe091ff None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:50.898368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3d1ae420-2eb8-4a55-aa99-3fddbfe091ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:50.899966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f48a68-c56b-4535-9aca-78981f0a6152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:50.900950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f48a68-c56b-4535-9aca-78981f0a6152 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:50.901349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f48a68-c56b-4535-9aca-78981f0a6152 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:50.902003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f48a68-c56b-4535-9aca-78981f0a6152 admin admin] PATH_INFO: `/v3/projects/2c0161f599f645e584efed2906c1eb22/users/475d1c123b7c427e8d7987dea6c7eec3/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:50.917245 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2314/4674] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:09:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:50.923862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f48a68-c56b-4535-9aca-78981f0a6152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0161f599f645e584efed2906c1eb22, user_id=475d1c123b7c427e8d7987dea6c7eec3, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:50.927065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ae420-2eb8-4a55-aa99-3fddbfe091ff None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:50.928118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f48a68-c56b-4535-9aca-78981f0a6152 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:50.942776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ae420-2eb8-4a55-aa99-3fddbfe091ff None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:50.951626 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2361/4675] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:50 2022] PUT /identity/v3/projects/2c0161f599f645e584efed2906c1eb22/users/475d1c123b7c427e8d7987dea6c7eec3/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:50.953349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f48a68-c56b-4535-9aca-78981f0a6152 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:50.960240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:50.988744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:51.049529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f48a68-c56b-4535-9aca-78981f0a6152 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:51.057697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8ba64-8ac2-4937-b519-64698524a690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:51.058556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8ba64-8ac2-4937-b519-64698524a690 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.059287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8ba64-8ac2-4937-b519-64698524a690 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.059731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8ba64-8ac2-4937-b519-64698524a690 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.060686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f8ba64-8ac2-4937-b519-64698524a690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:51.062852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f8ba64-8ac2-4937-b519-64698524a690 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:51.076969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03a3b7-13ac-4d85-8172-47dd9ec0cd07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:51.077825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03a3b7-13ac-4d85-8172-47dd9ec0cd07 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.078171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03a3b7-13ac-4d85-8172-47dd9ec0cd07 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.078492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03a3b7-13ac-4d85-8172-47dd9ec0cd07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.081609 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2362/4676] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:51.086137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8ba64-8ac2-4937-b519-64698524a690 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:51.096895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8ba64-8ac2-4937-b519-64698524a690 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:51.103655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c44011-901e-4f15-9cac-2fe8f8d4b844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:51.104481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c44011-901e-4f15-9cac-2fe8f8d4b844 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.104814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c44011-901e-4f15-9cac-2fe8f8d4b844 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.105191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c44011-901e-4f15-9cac-2fe8f8d4b844 admin admin] PATH_INFO: `/v3/projects/2c0161f599f645e584efed2906c1eb22/users/475d1c123b7c427e8d7987dea6c7eec3/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.108476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c44011-901e-4f15-9cac-2fe8f8d4b844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0161f599f645e584efed2906c1eb22, user_id=475d1c123b7c427e8d7987dea6c7eec3, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:51.112584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c44011-901e-4f15-9cac-2fe8f8d4b844 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:51.131738 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2363/4677] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:51 2022] PUT /identity/v3/projects/2c0161f599f645e584efed2906c1eb22/users/475d1c123b7c427e8d7987dea6c7eec3/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:51.132659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c44011-901e-4f15-9cac-2fe8f8d4b844 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:51.173602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03a3b7-13ac-4d85-8172-47dd9ec0cd07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:51.175649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03a3b7-13ac-4d85-8172-47dd9ec0cd07 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:51.192706 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2315/4678] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:09:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:09:51.194056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9fec4-1316-46a0-9650-daa81ce606cd None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.194471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9fec4-1316-46a0-9650-daa81ce606cd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.194956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9fec4-1316-46a0-9650-daa81ce606cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.220097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c44011-901e-4f15-9cac-2fe8f8d4b844 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:51.236957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-13e9fec4-1316-46a0-9650-daa81ce606cd None None] MFA Rules not processed for user `475d1c123b7c427e8d7987dea6c7eec3`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:51.238406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-13e9fec4-1316-46a0-9650-daa81ce606cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:51.241091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:51.244916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.245431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.246515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] PATH_INFO: `/v3/users/70f5c962d0ec4c5b8f0be043d31e1bbb` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.248567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70f5c962d0ec4c5b8f0be043d31e1bbb)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:51.253067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:51.315043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70f5c962d0ec4c5b8f0be043d31e1bbb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:51.326321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70f5c962d0ec4c5b8f0be043d31e1bbb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:51.336307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70f5c962d0ec4c5b8f0be043d31e1bbb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:51.340543 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2316/4679] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:51.341504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9fec4-1316-46a0-9650-daa81ce606cd None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:51.342815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-13e9fec4-1316-46a0-9650-daa81ce606cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:51.351062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70f5c962d0ec4c5b8f0be043d31e1bbb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:51.351957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70f5c962d0ec4c5b8f0be043d31e1bbb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:51.357257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9fec4-1316-46a0-9650-daa81ce606cd None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:51.357942 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2364/4680] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:51 2022] DELETE /identity/v3/users/70f5c962d0ec4c5b8f0be043d31e1bbb => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:51.358903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:51.361098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:51.368906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:51.417071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:51.431597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:51.432245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.432554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.432839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] PATH_INFO: `/v3/users/f144c6cdd63d4ab6ac5669aba1a09fb2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.433988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f144c6cdd63d4ab6ac5669aba1a09fb2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:51.435982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:51.450856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b804b8-5213-4888-995e-8d4ed5814fc5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:51.455477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30684b77-6e15-436e-90fa-6351fb7008f5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:51.456110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30684b77-6e15-436e-90fa-6351fb7008f5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.456399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30684b77-6e15-436e-90fa-6351fb7008f5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.456673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30684b77-6e15-436e-90fa-6351fb7008f5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.457301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30684b77-6e15-436e-90fa-6351fb7008f5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:51.458505 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-30684b77-6e15-436e-90fa-6351fb7008f5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:51.458505 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:09:51.458505 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:09:51.459374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30684b77-6e15-436e-90fa-6351fb7008f5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:51.484189 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2365/4681] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:09:51 2022] POST /identity/v3/regions => generated 224 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:51.485810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c54c4-df09-4e0d-a9bd-f1ada267565b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.486194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c54c4-df09-4e0d-a9bd-f1ada267565b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.486570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c54c4-df09-4e0d-a9bd-f1ada267565b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.492173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f144c6cdd63d4ab6ac5669aba1a09fb2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:51.501490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f144c6cdd63d4ab6ac5669aba1a09fb2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:51.509111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0a6c54c4-df09-4e0d-a9bd-f1ada267565b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:51.510309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0a6c54c4-df09-4e0d-a9bd-f1ada267565b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:51.517049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f144c6cdd63d4ab6ac5669aba1a09fb2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:51.517839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f144c6cdd63d4ab6ac5669aba1a09fb2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:51.518946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f144c6cdd63d4ab6ac5669aba1a09fb2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:51.528787 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2366/4682] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:51.529994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001dd49-c8b6-4d27-a653-66070bc01303 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.530266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001dd49-c8b6-4d27-a653-66070bc01303 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.530529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001dd49-c8b6-4d27-a653-66070bc01303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.534008 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2317/4683] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:51 2022] DELETE /identity/v3/users/f144c6cdd63d4ab6ac5669aba1a09fb2 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:51.537451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:51.539094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:51.550622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0001dd49-c8b6-4d27-a653-66070bc01303 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:51.552013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0001dd49-c8b6-4d27-a653-66070bc01303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:51.614264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3c268-e3df-4cf1-9fad-d87a15d02689 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:51.618075 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2367/4684] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:09:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:51.618929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001dd49-c8b6-4d27-a653-66070bc01303 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:51.620247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0001dd49-c8b6-4d27-a653-66070bc01303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:51.635191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58db131-9727-4eb2-9a79-f82a4bb36481 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:51.635843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58db131-9727-4eb2-9a79-f82a4bb36481 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.636115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58db131-9727-4eb2-9a79-f82a4bb36481 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.636412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58db131-9727-4eb2-9a79-f82a4bb36481 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.637049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58db131-9727-4eb2-9a79-f82a4bb36481 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:51.638059 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-a58db131-9727-4eb2-9a79-f82a4bb36481 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:51.638059 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:09:51.638059 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:09:51.638976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58db131-9727-4eb2-9a79-f82a4bb36481 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:51.665026 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2318/4685] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:09:51 2022] POST /identity/v3/regions => generated 224 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:51.666563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1df2f-e0d6-4a26-8192-ad1a0748b76c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.667367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1df2f-e0d6-4a26-8192-ad1a0748b76c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.667703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1df2f-e0d6-4a26-8192-ad1a0748b76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.688913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bba1df2f-e0d6-4a26-8192-ad1a0748b76c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:51.690420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bba1df2f-e0d6-4a26-8192-ad1a0748b76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:51.700505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001dd49-c8b6-4d27-a653-66070bc01303 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:51.714821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31daaf2-8527-43b1-8ea3-abe8ef994979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:51.715720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31daaf2-8527-43b1-8ea3-abe8ef994979 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.716149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31daaf2-8527-43b1-8ea3-abe8ef994979 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.716444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31daaf2-8527-43b1-8ea3-abe8ef994979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.718643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d31daaf2-8527-43b1-8ea3-abe8ef994979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:51.729408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:51.735051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:51.767958 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2319/4686] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:51.768967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1df2f-e0d6-4a26-8192-ad1a0748b76c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:51.782077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1df2f-e0d6-4a26-8192-ad1a0748b76c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:51.786129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920834a6-731c-4523-844d-af3b8a9660b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:51.786731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920834a6-731c-4523-844d-af3b8a9660b0 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.786999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920834a6-731c-4523-844d-af3b8a9660b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.787259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920834a6-731c-4523-844d-af3b8a9660b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.789180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-920834a6-731c-4523-844d-af3b8a9660b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:51.801729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920834a6-731c-4523-844d-af3b8a9660b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:51.803212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920834a6-731c-4523-844d-af3b8a9660b0 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:51.806839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31daaf2-8527-43b1-8ea3-abe8ef994979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:51.809385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31daaf2-8527-43b1-8ea3-abe8ef994979 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:51.813607 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2320/4687] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:51.814428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920834a6-731c-4523-844d-af3b8a9660b0 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:51.815815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-920834a6-731c-4523-844d-af3b8a9660b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:51.842555 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2368/4688] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:51.843419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31daaf2-8527-43b1-8ea3-abe8ef994979 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:51.868627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31daaf2-8527-43b1-8ea3-abe8ef994979 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:51.873663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db3584-b515-45ca-af6a-6413041cec1b tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:51.874439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db3584-b515-45ca-af6a-6413041cec1b tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.874724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db3584-b515-45ca-af6a-6413041cec1b tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.874994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db3584-b515-45ca-af6a-6413041cec1b tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions/da8b1e00-fb09-4771-9054-a16c370cb74d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.875762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db3584-b515-45ca-af6a-6413041cec1b tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:create_region(region_id=da8b1e00-fb09-4771-9054-a16c370cb74d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:51.878015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db3584-b515-45ca-af6a-6413041cec1b tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:51.906822 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2369/4689] 10.222.0.23 () {60 vars in 1259 bytes} [Sat Aug 20 21:09:51 2022] PUT /identity/v3/regions/da8b1e00-fb09-4771-9054-a16c370cb74d => generated 232 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:51.907709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db3584-b515-45ca-af6a-6413041cec1b tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:51.925293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920834a6-731c-4523-844d-af3b8a9660b0 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:51.940213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db3584-b515-45ca-af6a-6413041cec1b tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:51.946234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f95b5-6fa6-4633-887a-bde71463b725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:51.946870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f95b5-6fa6-4633-887a-bde71463b725 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.947132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f95b5-6fa6-4633-887a-bde71463b725 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.947395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f95b5-6fa6-4633-887a-bde71463b725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.961176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f95b5-6fa6-4633-887a-bde71463b725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:51.962829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f95b5-6fa6-4633-887a-bde71463b725 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:51.966022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf0570-ac59-4ec5-862d-5e3438a3a8f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:51.966894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf0570-ac59-4ec5-862d-5e3438a3a8f9 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:51.967974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf0570-ac59-4ec5-862d-5e3438a3a8f9 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:51.968433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf0570-ac59-4ec5-862d-5e3438a3a8f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:51.988042 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2370/4690] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:51.990032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f95b5-6fa6-4633-887a-bde71463b725 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:51.995012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf0570-ac59-4ec5-862d-5e3438a3a8f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:51.996902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf0570-ac59-4ec5-862d-5e3438a3a8f9 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:52.006237 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2321/4691] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:09:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:09:52.022267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f95b5-6fa6-4633-887a-bde71463b725 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:52.027987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89ea1a-b3fe-4718-b460-3be7bd352b8c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:52.028817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89ea1a-b3fe-4718-b460-3be7bd352b8c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:52.029141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89ea1a-b3fe-4718-b460-3be7bd352b8c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:52.029416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89ea1a-b3fe-4718-b460-3be7bd352b8c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions/da8b1e00-fb09-4771-9054-a16c370cb74d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:52.029910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89ea1a-b3fe-4718-b460-3be7bd352b8c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:delete_region(region_id=da8b1e00-fb09-4771-9054-a16c370cb74d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:52.031601 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-9d89ea1a-b3fe-4718-b460-3be7bd352b8c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:52.031601 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:09:52.031601 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:09:52.032542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89ea1a-b3fe-4718-b460-3be7bd352b8c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:52.114238 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2371/4692] 10.222.0.23 () {58 vars in 1242 bytes} [Sat Aug 20 21:09:51 2022] DELETE /identity/v3/regions/da8b1e00-fb09-4771-9054-a16c370cb74d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:52.119485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf0570-ac59-4ec5-862d-5e3438a3a8f9 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:52.139348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf0570-ac59-4ec5-862d-5e3438a3a8f9 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:52.207967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d840c0-ff27-40ec-a7a8-0bfcaab0952d tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:52.208654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d840c0-ff27-40ec-a7a8-0bfcaab0952d tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:52.208939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d840c0-ff27-40ec-a7a8-0bfcaab0952d tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:52.209468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d840c0-ff27-40ec-a7a8-0bfcaab0952d tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:52.213804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d840c0-ff27-40ec-a7a8-0bfcaab0952d tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:52.215828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d840c0-ff27-40ec-a7a8-0bfcaab0952d tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:52.270364 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2322/4693] 10.222.0.23 () {60 vars in 1150 bytes} [Sat Aug 20 21:09:52 2022] POST /identity/v3/regions => generated 254 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:52.275208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89ea1a-b3fe-4718-b460-3be7bd352b8c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:52.294667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89ea1a-b3fe-4718-b460-3be7bd352b8c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:52.329360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3745b2-dd55-4871-931e-3b18e8b731fa tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:52.330321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3745b2-dd55-4871-931e-3b18e8b731fa tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:52.330688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3745b2-dd55-4871-931e-3b18e8b731fa tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:52.331259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3745b2-dd55-4871-931e-3b18e8b731fa tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions/5f527ad17cba41ba9395fb2c99b64482` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:52.332051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3745b2-dd55-4871-931e-3b18e8b731fa tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:update_region(region_id=5f527ad17cba41ba9395fb2c99b64482)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:52.333530 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-de3745b2-dd55-4871-931e-3b18e8b731fa tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:52.333530 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:09:52.333530 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:09:52.335194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3745b2-dd55-4871-931e-3b18e8b731fa tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:52.336363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d840c0-ff27-40ec-a7a8-0bfcaab0952d tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:52.366893 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2372/4694] 10.222.0.23 () {60 vars in 1250 bytes} [Sat Aug 20 21:09:52 2022] PATCH /identity/v3/regions/5f527ad17cba41ba9395fb2c99b64482 => generated 253 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:52.371349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3745b2-dd55-4871-931e-3b18e8b731fa tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:52.393775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d840c0-ff27-40ec-a7a8-0bfcaab0952d tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:52.398045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3745b2-dd55-4871-931e-3b18e8b731fa tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:52.407032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c0013-fa5f-4a26-a2df-5194b8c6dbab tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:52.408028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c0013-fa5f-4a26-a2df-5194b8c6dbab tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:52.408403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c0013-fa5f-4a26-a2df-5194b8c6dbab tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:52.408901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c0013-fa5f-4a26-a2df-5194b8c6dbab tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions/5f527ad17cba41ba9395fb2c99b64482` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:52.409585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c0013-fa5f-4a26-a2df-5194b8c6dbab tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:get_region(region_id=5f527ad17cba41ba9395fb2c99b64482)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:52.411189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c0013-fa5f-4a26-a2df-5194b8c6dbab tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:52.438519 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2373/4695] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:09:52 2022] GET /identity/v3/regions/5f527ad17cba41ba9395fb2c99b64482 => generated 253 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:52.439800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c0013-fa5f-4a26-a2df-5194b8c6dbab tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:52.442436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fe4c0013-fa5f-4a26-a2df-5194b8c6dbab tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:52.446304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328ca8-a210-4f89-aba6-864afe509aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:52.447638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328ca8-a210-4f89-aba6-864afe509aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:52.448606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328ca8-a210-4f89-aba6-864afe509aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:52.454715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328ca8-a210-4f89-aba6-864afe509aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:52.458257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8e328ca8-a210-4f89-aba6-864afe509aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:52.472133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c0013-fa5f-4a26-a2df-5194b8c6dbab tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:52.478679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e328ca8-a210-4f89-aba6-864afe509aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:52.480162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e328ca8-a210-4f89-aba6-864afe509aa0 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:52.480893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:52.481641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:52.482128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:52.482504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] PATH_INFO: `/v3/projects/281f765786b8478da7aed89922e8305f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:52.483944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=281f765786b8478da7aed89922e8305f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:52.486176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:52.488614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=1,keystone:UPDATE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:52.542139 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2323/4696] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:52.543312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328ca8-a210-4f89-aba6-864afe509aa0 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:52.544874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8e328ca8-a210-4f89-aba6-864afe509aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:52.575162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '281f765786b8478da7aed89922e8305f', 'request_id': 'req-0cec175a-7be0-458a-b79b-2c3f15238e0b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:52.580764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328ca8-a210-4f89-aba6-864afe509aa0 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:52.593763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '281f765786b8478da7aed89922e8305f', 'request_id': 'req-0cec175a-7be0-458a-b79b-2c3f15238e0b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:52.608418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '281f765786b8478da7aed89922e8305f', 'request_id': 'req-0cec175a-7be0-458a-b79b-2c3f15238e0b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:52.624343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] The token cache is being invalidate because project 281f765786b8478da7aed89922e8305f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:52.627264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:52.628748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:52.637681 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2374/4697] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:52 2022] DELETE /identity/v3/projects/281f765786b8478da7aed89922e8305f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:52.639352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:52.641612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:52.650762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:52.722803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:52.723958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:52.724481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:52.724932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] PATH_INFO: `/v3/projects/78dc9242bdde4164b557231f3b304094` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:52.726428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78dc9242bdde4164b557231f3b304094)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:52.728846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:52.734342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec175a-7be0-458a-b79b-2c3f15238e0b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:52.742304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0dc95-b187-4fc1-b5a5-25383edc9f2c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:52.743101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0dc95-b187-4fc1-b5a5-25383edc9f2c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:52.743478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0dc95-b187-4fc1-b5a5-25383edc9f2c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:52.743849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0dc95-b187-4fc1-b5a5-25383edc9f2c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions/5f527ad17cba41ba9395fb2c99b64482` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:52.745017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0dc95-b187-4fc1-b5a5-25383edc9f2c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:delete_region(region_id=5f527ad17cba41ba9395fb2c99b64482)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:52.746904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0dc95-b187-4fc1-b5a5-25383edc9f2c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:52.783408 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2375/4698] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:52 2022] DELETE /identity/v3/regions/5f527ad17cba41ba9395fb2c99b64482 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:52.785002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0dc95-b187-4fc1-b5a5-25383edc9f2c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:52.789446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-acc0dc95-b187-4fc1-b5a5-25383edc9f2c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:52.816900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78dc9242bdde4164b557231f3b304094', 'request_id': 'req-b03beb99-c99f-42dc-a8cf-64f433b9c112'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:52.836503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78dc9242bdde4164b557231f3b304094', 'request_id': 'req-b03beb99-c99f-42dc-a8cf-64f433b9c112'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:52.845805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0dc95-b187-4fc1-b5a5-25383edc9f2c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:52.850708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78dc9242bdde4164b557231f3b304094', 'request_id': 'req-b03beb99-c99f-42dc-a8cf-64f433b9c112'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:52.874371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] The token cache is being invalidate because project 78dc9242bdde4164b557231f3b304094 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:52.874702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:52.875381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:52.876979 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2324/4699] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:52 2022] DELETE /identity/v3/projects/78dc9242bdde4164b557231f3b304094 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:52.881884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:52.883385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:52.946972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806a67c-a008-4d39-a777-cbe3d72151f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:52.947951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806a67c-a008-4d39-a777-cbe3d72151f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:52.948547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806a67c-a008-4d39-a777-cbe3d72151f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:52.948981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806a67c-a008-4d39-a777-cbe3d72151f1 admin admin] PATH_INFO: `/v3/users/348b88a9e8e34ec7bb89c3883dce53cd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:52.950347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8806a67c-a008-4d39-a777-cbe3d72151f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=348b88a9e8e34ec7bb89c3883dce53cd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:52.952611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8806a67c-a008-4d39-a777-cbe3d72151f1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:52.960507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:52.969382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03beb99-c99f-42dc-a8cf-64f433b9c112 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:52.998826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3564a7-cc5d-4432-a950-50ea0bedca21 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.000003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3564a7-cc5d-4432-a950-50ea0bedca21 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.000532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3564a7-cc5d-4432-a950-50ea0bedca21 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.001135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3564a7-cc5d-4432-a950-50ea0bedca21 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.002751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3564a7-cc5d-4432-a950-50ea0bedca21 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.004442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3564a7-cc5d-4432-a950-50ea0bedca21 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.017249 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2325/4700] 10.222.0.23 () {58 vars in 1128 bytes} [Sat Aug 20 21:09:52 2022] GET /identity/v3/regions => generated 670 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:53.019539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c1ee4-bb4d-465b-9761-595abc80c15f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.020677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c1ee4-bb4d-465b-9761-595abc80c15f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.021704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c1ee4-bb4d-465b-9761-595abc80c15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.063805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8806a67c-a008-4d39-a777-cbe3d72151f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348b88a9e8e34ec7bb89c3883dce53cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:53.067116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-040c1ee4-bb4d-465b-9761-595abc80c15f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:53.069514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-040c1ee4-bb4d-465b-9761-595abc80c15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:53.079074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8806a67c-a008-4d39-a777-cbe3d72151f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348b88a9e8e34ec7bb89c3883dce53cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:53.081800 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2326/4701] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:53.082753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c1ee4-bb4d-465b-9761-595abc80c15f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.089195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8806a67c-a008-4d39-a777-cbe3d72151f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348b88a9e8e34ec7bb89c3883dce53cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:53.100097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c1ee4-bb4d-465b-9761-595abc80c15f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:53.106703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8806a67c-a008-4d39-a777-cbe3d72151f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348b88a9e8e34ec7bb89c3883dce53cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:53.107241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19354b-4afb-47f7-add7-d92872917199 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.107902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8806a67c-a008-4d39-a777-cbe3d72151f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348b88a9e8e34ec7bb89c3883dce53cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:53.108103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19354b-4afb-47f7-add7-d92872917199 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.108243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19354b-4afb-47f7-add7-d92872917199 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.108544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19354b-4afb-47f7-add7-d92872917199 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions/5f527ad17cba41ba9395fb2c99b64482` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.109255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19354b-4afb-47f7-add7-d92872917199 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:delete_region(region_id=5f527ad17cba41ba9395fb2c99b64482)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.110444 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-8a19354b-4afb-47f7-add7-d92872917199 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:09:53.110444 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:09:53.110444 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:09:53.111908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19354b-4afb-47f7-add7-d92872917199 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.117165 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2376/4702] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:52 2022] DELETE /identity/v3/users/348b88a9e8e34ec7bb89c3883dce53cd => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:53.118652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a306c28-dd85-49b1-ae11-3e06ceb7aca2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.118996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a306c28-dd85-49b1-ae11-3e06ceb7aca2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.119387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a306c28-dd85-49b1-ae11-3e06ceb7aca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-8a19354b-4afb-47f7-add7-d92872917199 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Could not find region: 5f527ad17cba41ba9395fb2c99b64482.: keystone.exception.RegionNotFound: Could not find region: 5f527ad17cba41ba9395fb2c99b64482. Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 129, in delete_region Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ret = self.driver.delete_region(region_id) Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 152, in delete_region Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ref = self._get_region(session, region_id) Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 103, in _get_region Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 5f527ad17cba41ba9395fb2c99b64482. Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:09:53.133088 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/regions.py", line 92, in delete Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return PROVIDERS.catalog_api.delete_region( Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 135, in delete_region Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 5f527ad17cba41ba9395fb2c99b64482. Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:09:53.134745 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2327/4703] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:53 2022] DELETE /identity/v3/regions/5f527ad17cba41ba9395fb2c99b64482 => generated 112 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:09:53.139472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19354b-4afb-47f7-add7-d92872917199 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.141235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a19354b-4afb-47f7-add7-d92872917199 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:53.146430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9a306c28-dd85-49b1-ae11-3e06ceb7aca2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:53.148038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9a306c28-dd85-49b1-ae11-3e06ceb7aca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:53.156968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19354b-4afb-47f7-add7-d92872917199 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:53.197604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:53.230439 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2377/4704] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:53.234403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a306c28-dd85-49b1-ae11-3e06ceb7aca2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.250655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a306c28-dd85-49b1-ae11-3e06ceb7aca2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:53.258681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5f1f2-6879-4bf3-bc47-c4dc531ef58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.259427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5f1f2-6879-4bf3-bc47-c4dc531ef58f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.259813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5f1f2-6879-4bf3-bc47-c4dc531ef58f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.260212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5f1f2-6879-4bf3-bc47-c4dc531ef58f admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.261085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da5f1f2-6879-4bf3-bc47-c4dc531ef58f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:53.261844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da5f1f2-6879-4bf3-bc47-c4dc531ef58f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.262442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954df925-1b3c-4e7a-a60b-415785130e3c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.263113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954df925-1b3c-4e7a-a60b-415785130e3c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.263387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954df925-1b3c-4e7a-a60b-415785130e3c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.263724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954df925-1b3c-4e7a-a60b-415785130e3c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.264713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954df925-1b3c-4e7a-a60b-415785130e3c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.265600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da5f1f2-6879-4bf3-bc47-c4dc531ef58f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.266004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954df925-1b3c-4e7a-a60b-415785130e3c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.280006 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2328/4705] 10.222.0.23 () {58 vars in 1128 bytes} [Sat Aug 20 21:09:53 2022] GET /identity/v3/regions => generated 670 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:53.281267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954df925-1b3c-4e7a-a60b-415785130e3c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.286562 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2378/4706] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:53.295915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954df925-1b3c-4e7a-a60b-415785130e3c tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:53.302459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70254681-4535-48f4-92d3-b8585e20624a tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.303083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70254681-4535-48f4-92d3-b8585e20624a tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.303364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70254681-4535-48f4-92d3-b8585e20624a tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.303687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70254681-4535-48f4-92d3-b8585e20624a tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.304967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70254681-4535-48f4-92d3-b8585e20624a tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.306736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70254681-4535-48f4-92d3-b8585e20624a tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.309337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5f1f2-6879-4bf3-bc47-c4dc531ef58f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.333641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5f1f2-6879-4bf3-bc47-c4dc531ef58f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:53.340778 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2329/4707] 10.222.0.23 () {60 vars in 1150 bytes} [Sat Aug 20 21:09:53 2022] POST /identity/v3/regions => generated 254 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:53.345608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70254681-4535-48f4-92d3-b8585e20624a tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.346312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422db5a-276a-4b4c-b4c0-4585632f59b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.348042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422db5a-276a-4b4c-b4c0-4585632f59b9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.348476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422db5a-276a-4b4c-b4c0-4585632f59b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.348844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422db5a-276a-4b4c-b4c0-4585632f59b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.349548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422db5a-276a-4b4c-b4c0-4585632f59b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.355905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422db5a-276a-4b4c-b4c0-4585632f59b9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.398167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70254681-4535-48f4-92d3-b8585e20624a tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:53.399930 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2379/4708] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:53 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:53.404813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422db5a-276a-4b4c-b4c0-4585632f59b9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.494176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77f50b-60b4-498a-ac5c-0fc7335b02e5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.495126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f50b-60b4-498a-ac5c-0fc7335b02e5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.495588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f50b-60b4-498a-ac5c-0fc7335b02e5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.496016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f50b-60b4-498a-ac5c-0fc7335b02e5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.496831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77f50b-60b4-498a-ac5c-0fc7335b02e5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Adding query filter params (parent_region_id=00adc734681746ca84221a3a83c9c835) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:53.497295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77f50b-60b4-498a-ac5c-0fc7335b02e5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.498578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77f50b-60b4-498a-ac5c-0fc7335b02e5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.509380 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2330/4709] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:09:53 2022] GET /identity/v3/regions?parent_region_id=00adc734681746ca84221a3a83c9c835 => generated 400 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:53.513346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77f50b-60b4-498a-ac5c-0fc7335b02e5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.517642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:53.535849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77f50b-60b4-498a-ac5c-0fc7335b02e5 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:53.541022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422db5a-276a-4b4c-b4c0-4585632f59b9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:53.552761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de69006-826e-4a80-adf2-d9f1b2fb6d69 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.553603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de69006-826e-4a80-adf2-d9f1b2fb6d69 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.553912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de69006-826e-4a80-adf2-d9f1b2fb6d69 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.554189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de69006-826e-4a80-adf2-d9f1b2fb6d69 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions/b1e78269c408483daac22cc62ae44589` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.554854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de69006-826e-4a80-adf2-d9f1b2fb6d69 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:delete_region(region_id=b1e78269c408483daac22cc62ae44589)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.556667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de69006-826e-4a80-adf2-d9f1b2fb6d69 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.566623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ec310-bb43-488d-b6da-af8747483348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.568278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ec310-bb43-488d-b6da-af8747483348 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.568794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ec310-bb43-488d-b6da-af8747483348 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.569264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ec310-bb43-488d-b6da-af8747483348 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.571747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ec310-bb43-488d-b6da-af8747483348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.576148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ec310-bb43-488d-b6da-af8747483348 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.593338 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2331/4710] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:53 2022] DELETE /identity/v3/regions/b1e78269c408483daac22cc62ae44589 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:53.594679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de69006-826e-4a80-adf2-d9f1b2fb6d69 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.598906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5de69006-826e-4a80-adf2-d9f1b2fb6d69 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:53.638431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de69006-826e-4a80-adf2-d9f1b2fb6d69 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:53.644363 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2380/4711] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:53 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:53.647490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.648784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.649151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.649424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] PATH_INFO: `/v3/projects/e159fa978ca44f24b1e7f2bc7736c317` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.651262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e159fa978ca44f24b1e7f2bc7736c317)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.652113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ec310-bb43-488d-b6da-af8747483348 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.655338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.670449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ec310-bb43-488d-b6da-af8747483348 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:53.690105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09118b88-87c5-4b33-ab99-05317295ec3e tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.690733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09118b88-87c5-4b33-ab99-05317295ec3e tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.691007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09118b88-87c5-4b33-ab99-05317295ec3e tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.691288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09118b88-87c5-4b33-ab99-05317295ec3e tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions/ad5bbdf2cb804dc29601eb032edbb963` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.691902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09118b88-87c5-4b33-ab99-05317295ec3e tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:delete_region(region_id=ad5bbdf2cb804dc29601eb032edbb963)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.694046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09118b88-87c5-4b33-ab99-05317295ec3e tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.727081 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2381/4712] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:53 2022] DELETE /identity/v3/regions/ad5bbdf2cb804dc29601eb032edbb963 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:53.729220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09118b88-87c5-4b33-ab99-05317295ec3e tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.731801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e159fa978ca44f24b1e7f2bc7736c317', 'request_id': 'req-355ce832-786c-4454-84a4-fa32cc773306'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:53.738550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e159fa978ca44f24b1e7f2bc7736c317', 'request_id': 'req-355ce832-786c-4454-84a4-fa32cc773306'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:53.750293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e159fa978ca44f24b1e7f2bc7736c317', 'request_id': 'req-355ce832-786c-4454-84a4-fa32cc773306'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:53.755893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:53.769664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] The token cache is being invalidate because project e159fa978ca44f24b1e7f2bc7736c317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:53.770666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:53.771679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:53.773786 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2332/4713] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:53 2022] DELETE /identity/v3/projects/e159fa978ca44f24b1e7f2bc7736c317 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:53.774900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.777105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09118b88-87c5-4b33-ab99-05317295ec3e tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:53.780081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:53.849433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d807f23-ab75-4adc-858a-c809fec13c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.850051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d807f23-ab75-4adc-858a-c809fec13c10 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.850456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d807f23-ab75-4adc-858a-c809fec13c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.850834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d807f23-ab75-4adc-858a-c809fec13c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.851578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d807f23-ab75-4adc-858a-c809fec13c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.853896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d807f23-ab75-4adc-858a-c809fec13c10 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.868399 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2382/4714] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:53.868399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a05393-291d-4c02-8147-6988622cf025 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.868399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a05393-291d-4c02-8147-6988622cf025 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.868399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a05393-291d-4c02-8147-6988622cf025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.876924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ce832-786c-4454-84a4-fa32cc773306 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:53.896177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-29a05393-291d-4c02-8147-6988622cf025 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:53.897603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6854e-da62-4c27-87b3-d904f50657a3 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '475d1c123b7c427e8d7987dea6c7eec3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c0161f599f645e584efed2906c1eb22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.897764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-29a05393-291d-4c02-8147-6988622cf025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:53.898258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6854e-da62-4c27-87b3-d904f50657a3 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.898531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6854e-da62-4c27-87b3-d904f50657a3 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.898890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6854e-da62-4c27-87b3-d904f50657a3 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] PATH_INFO: `/v3/regions/00adc734681746ca84221a3a83c9c835` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.899483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6854e-da62-4c27-87b3-d904f50657a3 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorizing `identity:delete_region(region_id=00adc734681746ca84221a3a83c9c835)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.901488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6854e-da62-4c27-87b3-d904f50657a3 tempest-RegionsTestJSON-1228172385 tempest-RegionsTestJSON-1228172385-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.912997 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2383/4715] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:53.914283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a05393-291d-4c02-8147-6988622cf025 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.922876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-29a05393-291d-4c02-8147-6988622cf025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:53.944185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a05393-291d-4c02-8147-6988622cf025 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:53.950239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e48ee-a5b0-4d1a-8319-8bc01d51a25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:53.950851 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2333/4716] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:09:53 2022] DELETE /identity/v3/regions/00adc734681746ca84221a3a83c9c835 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:53.951005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e48ee-a5b0-4d1a-8319-8bc01d51a25f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.951101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e48ee-a5b0-4d1a-8319-8bc01d51a25f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.951370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e48ee-a5b0-4d1a-8319-8bc01d51a25f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.952099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e48ee-a5b0-4d1a-8319-8bc01d51a25f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:53.952709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ff0f6-d917-4148-87cd-1ad418b57155 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:53.952990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ff0f6-d917-4148-87cd-1ad418b57155 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:53.953261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ff0f6-d917-4148-87cd-1ad418b57155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:53.954173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e48ee-a5b0-4d1a-8319-8bc01d51a25f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:53.984782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-052ff0f6-d917-4148-87cd-1ad418b57155 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:53.986000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-052ff0f6-d917-4148-87cd-1ad418b57155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:53.987914 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2384/4717] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:53.989069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e48ee-a5b0-4d1a-8319-8bc01d51a25f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:53.991567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-254e48ee-a5b0-4d1a-8319-8bc01d51a25f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:54.005676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e48ee-a5b0-4d1a-8319-8bc01d51a25f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:54.023574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bdc3f-a3b5-45be-993c-935838a8310e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:54.024427 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2334/4718] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:54.024567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bdc3f-a3b5-45be-993c-935838a8310e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:54.024910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bdc3f-a3b5-45be-993c-935838a8310e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:54.025186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ff0f6-d917-4148-87cd-1ad418b57155 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:54.025404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bdc3f-a3b5-45be-993c-935838a8310e admin admin] PATH_INFO: `/v3/users/d48af3beb4c54fa9a9834b27bc976fba` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:54.028594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16bdc3f-a3b5-45be-993c-935838a8310e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d48af3beb4c54fa9a9834b27bc976fba)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:54.030920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16bdc3f-a3b5-45be-993c-935838a8310e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:54.035591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ff0f6-d917-4148-87cd-1ad418b57155 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:54.048324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502d558-8c83-4b10-80fa-e73de0dfbd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:54.049208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502d558-8c83-4b10-80fa-e73de0dfbd2f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:54.049496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502d558-8c83-4b10-80fa-e73de0dfbd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:54.049750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502d558-8c83-4b10-80fa-e73de0dfbd2f admin admin] PATH_INFO: `/v3/projects/19b41386891848609ba30a7c36c57dd6/users/ca5972c2e2b246958e91c5ba52abb9cd/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:54.073204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502d558-8c83-4b10-80fa-e73de0dfbd2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b41386891848609ba30a7c36c57dd6, user_id=ca5972c2e2b246958e91c5ba52abb9cd, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:54.076875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502d558-8c83-4b10-80fa-e73de0dfbd2f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:54.120185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a16bdc3f-a3b5-45be-993c-935838a8310e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd48af3beb4c54fa9a9834b27bc976fba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:54.134955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a16bdc3f-a3b5-45be-993c-935838a8310e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd48af3beb4c54fa9a9834b27bc976fba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:54.137773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:54.144490 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2335/4719] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:54 2022] PUT /identity/v3/projects/19b41386891848609ba30a7c36c57dd6/users/ca5972c2e2b246958e91c5ba52abb9cd/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:54.147011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502d558-8c83-4b10-80fa-e73de0dfbd2f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:54.147327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a16bdc3f-a3b5-45be-993c-935838a8310e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd48af3beb4c54fa9a9834b27bc976fba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:54.175665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a16bdc3f-a3b5-45be-993c-935838a8310e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd48af3beb4c54fa9a9834b27bc976fba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:54.178965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a16bdc3f-a3b5-45be-993c-935838a8310e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd48af3beb4c54fa9a9834b27bc976fba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:54.190833 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2385/4720] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:53 2022] DELETE /identity/v3/users/d48af3beb4c54fa9a9834b27bc976fba => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:54.196422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bad27-9b81-4b9c-ad2f-c013230ddceb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:54.196422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bad27-9b81-4b9c-ad2f-c013230ddceb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:54.196422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bad27-9b81-4b9c-ad2f-c013230ddceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:54.277659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ce4bad27-9b81-4b9c-ad2f-c013230ddceb None None] MFA Rules not processed for user `ca5972c2e2b246958e91c5ba52abb9cd`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:54.278741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ce4bad27-9b81-4b9c-ad2f-c013230ddceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:54.301180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502d558-8c83-4b10-80fa-e73de0dfbd2f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:54.387601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:54.423380 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2386/4721] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:54.426980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcef25-4c94-45e2-b190-94bec1b06486 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:54.428961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcef25-4c94-45e2-b190-94bec1b06486 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:54.429382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcef25-4c94-45e2-b190-94bec1b06486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:54.450863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423cf96-d0ef-449c-bb4f-1b18a91daa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:54.453137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423cf96-d0ef-449c-bb4f-1b18a91daa74 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:54.454532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423cf96-d0ef-449c-bb4f-1b18a91daa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:54.455399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423cf96-d0ef-449c-bb4f-1b18a91daa74 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:54.459413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423cf96-d0ef-449c-bb4f-1b18a91daa74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:54.459413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423cf96-d0ef-449c-bb4f-1b18a91daa74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:54.461552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423cf96-d0ef-449c-bb4f-1b18a91daa74 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:54.471223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9dbcef25-4c94-45e2-b190-94bec1b06486 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:54.473182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9dbcef25-4c94-45e2-b190-94bec1b06486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:54.481130 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2336/4722] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:54.482291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423cf96-d0ef-449c-bb4f-1b18a91daa74 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:54.486059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f423cf96-d0ef-449c-bb4f-1b18a91daa74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:54.491504 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2387/4723] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:54.496957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcef25-4c94-45e2-b190-94bec1b06486 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:54.498872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9dbcef25-4c94-45e2-b190-94bec1b06486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:54.519738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcef25-4c94-45e2-b190-94bec1b06486 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:54.537947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd70ff7-f357-4290-9fab-6e0f0ce63519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:54.538737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd70ff7-f357-4290-9fab-6e0f0ce63519 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:54.539120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd70ff7-f357-4290-9fab-6e0f0ce63519 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:54.539483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd70ff7-f357-4290-9fab-6e0f0ce63519 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:54.540295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd70ff7-f357-4290-9fab-6e0f0ce63519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:54.542308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd70ff7-f357-4290-9fab-6e0f0ce63519 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:54.588038 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2388/4724] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:54 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:54.588038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd70ff7-f357-4290-9fab-6e0f0ce63519 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:54.590224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2fd70ff7-f357-4290-9fab-6e0f0ce63519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:54.629817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423cf96-d0ef-449c-bb4f-1b18a91daa74 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:54.679150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd70ff7-f357-4290-9fab-6e0f0ce63519 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:54.683999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359edd0-8962-43dd-960d-b174fe3f4e82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:54.684648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359edd0-8962-43dd-960d-b174fe3f4e82 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:54.684980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359edd0-8962-43dd-960d-b174fe3f4e82 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:54.685244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359edd0-8962-43dd-960d-b174fe3f4e82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:54.726570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1799a0e-185c-4bda-98ac-47a943fa3e44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:54.727529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1799a0e-185c-4bda-98ac-47a943fa3e44 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:54.727986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1799a0e-185c-4bda-98ac-47a943fa3e44 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:54.728450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1799a0e-185c-4bda-98ac-47a943fa3e44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:54.732668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:54.765538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359edd0-8962-43dd-960d-b174fe3f4e82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:54.768320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359edd0-8962-43dd-960d-b174fe3f4e82 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:54.788540 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2389/4725] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:54.789806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359edd0-8962-43dd-960d-b174fe3f4e82 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:54.805801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359edd0-8962-43dd-960d-b174fe3f4e82 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:54.807621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1799a0e-185c-4bda-98ac-47a943fa3e44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:54.809338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1799a0e-185c-4bda-98ac-47a943fa3e44 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:54.811738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e78011-ec36-4988-9e41-eeb1044c83f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:54.812578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e78011-ec36-4988-9e41-eeb1044c83f8 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:54.813028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e78011-ec36-4988-9e41-eeb1044c83f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:54.814819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:54.815373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e78011-ec36-4988-9e41-eeb1044c83f8 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:54.819307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e78011-ec36-4988-9e41-eeb1044c83f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:54.823448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e78011-ec36-4988-9e41-eeb1044c83f8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:54.827681 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2337/4726] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:54.878669 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2390/4727] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:54 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:54.883103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1799a0e-185c-4bda-98ac-47a943fa3e44 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:54.900321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1799a0e-185c-4bda-98ac-47a943fa3e44 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:54.914608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94586a6-c193-4456-8aab-0bc2f98e173c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:54.915244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94586a6-c193-4456-8aab-0bc2f98e173c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:54.915515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94586a6-c193-4456-8aab-0bc2f98e173c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:54.915777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94586a6-c193-4456-8aab-0bc2f98e173c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:54.916651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94586a6-c193-4456-8aab-0bc2f98e173c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:54.919814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94586a6-c193-4456-8aab-0bc2f98e173c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:54.943381 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2338/4728] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:54.947747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e78011-ec36-4988-9e41-eeb1044c83f8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:54.977473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e78011-ec36-4988-9e41-eeb1044c83f8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:55.002880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72bb76-f6c0-49cd-b270-263e87d1c7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:55.003553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72bb76-f6c0-49cd-b270-263e87d1c7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:55.003830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72bb76-f6c0-49cd-b270-263e87d1c7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:55.006032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72bb76-f6c0-49cd-b270-263e87d1c7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:55.006726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72bb76-f6c0-49cd-b270-263e87d1c7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:55.008387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72bb76-f6c0-49cd-b270-263e87d1c7a9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:55.040651 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2391/4729] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:55.041947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94586a6-c193-4456-8aab-0bc2f98e173c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:55.061232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94586a6-c193-4456-8aab-0bc2f98e173c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:55.081181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea645880-59c6-46e4-86a6-27f95333aafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:55.081935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea645880-59c6-46e4-86a6-27f95333aafd admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:55.082272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea645880-59c6-46e4-86a6-27f95333aafd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:55.083549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea645880-59c6-46e4-86a6-27f95333aafd admin admin] PATH_INFO: `/v3/projects/2ab01f6e58774255af430c4bb1e24d04/users/3545abe50b7542c4a34ead955b313bf2/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:55.155724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea645880-59c6-46e4-86a6-27f95333aafd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab01f6e58774255af430c4bb1e24d04, user_id=3545abe50b7542c4a34ead955b313bf2, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:55.161177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea645880-59c6-46e4-86a6-27f95333aafd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:55.200410 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2339/4730] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:55 2022] PUT /identity/v3/projects/2ab01f6e58774255af430c4bb1e24d04/users/3545abe50b7542c4a34ead955b313bf2/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:55.210078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd418048-0786-4683-8cbf-72f696d226d6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:55.210078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd418048-0786-4683-8cbf-72f696d226d6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:55.210078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd418048-0786-4683-8cbf-72f696d226d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:55.269236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cd418048-0786-4683-8cbf-72f696d226d6 None None] MFA Rules not processed for user `3545abe50b7542c4a34ead955b313bf2`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:55.270701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cd418048-0786-4683-8cbf-72f696d226d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:55.436147 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2392/4731] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:55.440559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea645880-59c6-46e4-86a6-27f95333aafd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:55.442275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ea645880-59c6-46e4-86a6-27f95333aafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:55.512500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd418048-0786-4683-8cbf-72f696d226d6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:55.514687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cd418048-0786-4683-8cbf-72f696d226d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:55.609383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea645880-59c6-46e4-86a6-27f95333aafd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:55.613722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac821b-a52a-4dcd-9d01-8bfe224ae830 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:55.615384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac821b-a52a-4dcd-9d01-8bfe224ae830 service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:55.615862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac821b-a52a-4dcd-9d01-8bfe224ae830 service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:55.616788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac821b-a52a-4dcd-9d01-8bfe224ae830 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:55.635940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac821b-a52a-4dcd-9d01-8bfe224ae830 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:55.648909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac821b-a52a-4dcd-9d01-8bfe224ae830 service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:55.663158 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2340/4732] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:09:55.664474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac821b-a52a-4dcd-9d01-8bfe224ae830 service glance] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:55.666271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9aac821b-a52a-4dcd-9d01-8bfe224ae830 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:55.666432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd418048-0786-4683-8cbf-72f696d226d6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:55.673010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:55.673654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:55.673921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:55.674183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] PATH_INFO: `/v3/projects/5cda1ca23e0440998fec70a040112e98` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:55.675334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cda1ca23e0440998fec70a040112e98)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:55.681909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:55.689115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac821b-a52a-4dcd-9d01-8bfe224ae830 service glance] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:55.750146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ae167-b779-4db3-8045-61f086b56c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:55.751217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ae167-b779-4db3-8045-61f086b56c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:55.751666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ae167-b779-4db3-8045-61f086b56c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:55.753571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ae167-b779-4db3-8045-61f086b56c27 admin admin] PATH_INFO: `/v3/users/ca5972c2e2b246958e91c5ba52abb9cd` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:55.756613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ae167-b779-4db3-8045-61f086b56c27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca5972c2e2b246958e91c5ba52abb9cd)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:55.760024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ae167-b779-4db3-8045-61f086b56c27 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:55.883422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c10ae167-b779-4db3-8045-61f086b56c27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca5972c2e2b246958e91c5ba52abb9cd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:55.885633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:55.901424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cda1ca23e0440998fec70a040112e98', 'request_id': 'req-4b33bbb9-6e50-45c6-941f-caa6126e92b8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:55.914268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c10ae167-b779-4db3-8045-61f086b56c27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca5972c2e2b246958e91c5ba52abb9cd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:55.926360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cda1ca23e0440998fec70a040112e98', 'request_id': 'req-4b33bbb9-6e50-45c6-941f-caa6126e92b8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:55.948502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cda1ca23e0440998fec70a040112e98', 'request_id': 'req-4b33bbb9-6e50-45c6-941f-caa6126e92b8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:55.955940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c10ae167-b779-4db3-8045-61f086b56c27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca5972c2e2b246958e91c5ba52abb9cd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:55.963510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c10ae167-b779-4db3-8045-61f086b56c27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca5972c2e2b246958e91c5ba52abb9cd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:55.964934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c10ae167-b779-4db3-8045-61f086b56c27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca5972c2e2b246958e91c5ba52abb9cd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:55.983399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] The token cache is being invalidate because project 5cda1ca23e0440998fec70a040112e98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:55.983717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:55.984549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:55.991407 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2393/4733] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:55 2022] DELETE /identity/v3/projects/5cda1ca23e0440998fec70a040112e98 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:55.996426 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2341/4734] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:55 2022] DELETE /identity/v3/users/ca5972c2e2b246958e91c5ba52abb9cd => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:56.001130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:56.002747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419f724-5ee7-4522-813a-922a1b79a73c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:56.004854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419f724-5ee7-4522-813a-922a1b79a73c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:56.005360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419f724-5ee7-4522-813a-922a1b79a73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:56.007677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:56.041930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a419f724-5ee7-4522-813a-922a1b79a73c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:56.043706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a419f724-5ee7-4522-813a-922a1b79a73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:56.087663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:56.143592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33bbb9-6e50-45c6-941f-caa6126e92b8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:56.175904 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2342/4735] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:56.187835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419f724-5ee7-4522-813a-922a1b79a73c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:56.189748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a419f724-5ee7-4522-813a-922a1b79a73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:56.274576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:56.275786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:56.276162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:56.276570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] PATH_INFO: `/v3/users/475d1c123b7c427e8d7987dea6c7eec3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:56.280484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=475d1c123b7c427e8d7987dea6c7eec3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:56.283833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:56.345723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419f724-5ee7-4522-813a-922a1b79a73c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:56.359143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3c6f1-b10d-46d8-8b77-d10096fdca33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:56.359750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3c6f1-b10d-46d8-8b77-d10096fdca33 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:56.359937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3c6f1-b10d-46d8-8b77-d10096fdca33 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:56.361004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3c6f1-b10d-46d8-8b77-d10096fdca33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:56.396622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3c6f1-b10d-46d8-8b77-d10096fdca33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:56.398100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3c6f1-b10d-46d8-8b77-d10096fdca33 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:56.400257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '475d1c123b7c427e8d7987dea6c7eec3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:56.415022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '475d1c123b7c427e8d7987dea6c7eec3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:56.422746 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2343/4736] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:09:56.461661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '475d1c123b7c427e8d7987dea6c7eec3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:56.501251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:56.509326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '475d1c123b7c427e8d7987dea6c7eec3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:56.514374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '475d1c123b7c427e8d7987dea6c7eec3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:56.534494 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2394/4737] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:56 2022] DELETE /identity/v3/users/475d1c123b7c427e8d7987dea6c7eec3 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:56.954941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3c6f1-b10d-46d8-8b77-d10096fdca33 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:56.969522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3c6f1-b10d-46d8-8b77-d10096fdca33 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:57.096783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170293b7-2b05-46dc-8652-4cefe6ccdb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:57.097455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170293b7-2b05-46dc-8652-4cefe6ccdb65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:57.097776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170293b7-2b05-46dc-8652-4cefe6ccdb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:57.098520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170293b7-2b05-46dc-8652-4cefe6ccdb65 admin admin] PATH_INFO: `/v3/projects/2c0161f599f645e584efed2906c1eb22` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:57.100325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170293b7-2b05-46dc-8652-4cefe6ccdb65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c0161f599f645e584efed2906c1eb22)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:57.102893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170293b7-2b05-46dc-8652-4cefe6ccdb65 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:57.111124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:57.114811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:57.142232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a07012-4672-41fb-a281-e2aa52b585d7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:57.149213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54386c-397b-4be4-a564-fc71133b65e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:57.150267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54386c-397b-4be4-a564-fc71133b65e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:57.150722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54386c-397b-4be4-a564-fc71133b65e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:57.151172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54386c-397b-4be4-a564-fc71133b65e9 admin admin] PATH_INFO: `/v3/users/3545abe50b7542c4a34ead955b313bf2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:57.152658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54386c-397b-4be4-a564-fc71133b65e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3545abe50b7542c4a34ead955b313bf2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:57.154852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54386c-397b-4be4-a564-fc71133b65e9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:57.186818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-170293b7-2b05-46dc-8652-4cefe6ccdb65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c0161f599f645e584efed2906c1eb22', 'request_id': 'req-170293b7-2b05-46dc-8652-4cefe6ccdb65'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:57.196475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-170293b7-2b05-46dc-8652-4cefe6ccdb65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c0161f599f645e584efed2906c1eb22', 'request_id': 'req-170293b7-2b05-46dc-8652-4cefe6ccdb65'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:57.211577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-170293b7-2b05-46dc-8652-4cefe6ccdb65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c0161f599f645e584efed2906c1eb22', 'request_id': 'req-170293b7-2b05-46dc-8652-4cefe6ccdb65'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:57.227521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b54386c-397b-4be4-a564-fc71133b65e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3545abe50b7542c4a34ead955b313bf2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:57.230001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-170293b7-2b05-46dc-8652-4cefe6ccdb65 admin admin] The token cache is being invalidate because project 2c0161f599f645e584efed2906c1eb22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:57.231593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-170293b7-2b05-46dc-8652-4cefe6ccdb65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:57.233046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-170293b7-2b05-46dc-8652-4cefe6ccdb65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:57.237000 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2344/4738] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:56 2022] DELETE /identity/v3/projects/2c0161f599f645e584efed2906c1eb22 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:57.242736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eaca4c-44a4-4c66-ba0e-79895141ce70 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:57.243300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eaca4c-44a4-4c66-ba0e-79895141ce70 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:57.243743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eaca4c-44a4-4c66-ba0e-79895141ce70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:57.253943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b54386c-397b-4be4-a564-fc71133b65e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3545abe50b7542c4a34ead955b313bf2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:57.262683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:57.268601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b54386c-397b-4be4-a564-fc71133b65e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3545abe50b7542c4a34ead955b313bf2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:57.288128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a9eaca4c-44a4-4c66-ba0e-79895141ce70 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:57.289628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a9eaca4c-44a4-4c66-ba0e-79895141ce70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:57.305404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b54386c-397b-4be4-a564-fc71133b65e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3545abe50b7542c4a34ead955b313bf2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:57.306202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b54386c-397b-4be4-a564-fc71133b65e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3545abe50b7542c4a34ead955b313bf2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:57.319706 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2395/4739] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:09:57 2022] DELETE /identity/v3/users/3545abe50b7542c4a34ead955b313bf2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:57.324424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd554f-43ce-4dd4-9f77-643a569d28ce None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:57.324711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd554f-43ce-4dd4-9f77-643a569d28ce None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:57.324979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd554f-43ce-4dd4-9f77-643a569d28ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:57.363803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-84cd554f-43ce-4dd4-9f77-643a569d28ce None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:57.365358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-84cd554f-43ce-4dd4-9f77-643a569d28ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:57.365973 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2345/4740] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:57.377292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7768b01-53d6-45ac-b491-d339e0b61f3f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:57.377561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7768b01-53d6-45ac-b491-d339e0b61f3f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:57.377827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7768b01-53d6-45ac-b491-d339e0b61f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:57.414936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d7768b01-53d6-45ac-b491-d339e0b61f3f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:57.416360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d7768b01-53d6-45ac-b491-d339e0b61f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:57.451247 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2396/4741] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:57.465065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd554f-43ce-4dd4-9f77-643a569d28ce None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:57.466531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-84cd554f-43ce-4dd4-9f77-643a569d28ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:57.540535 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2346/4742] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:57.541884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7768b01-53d6-45ac-b491-d339e0b61f3f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:57.551891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd554f-43ce-4dd4-9f77-643a569d28ce None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:57.555917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7baeb-c742-47d4-9554-1a83a2b6bba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:57.556732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7baeb-c742-47d4-9554-1a83a2b6bba9 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:57.557002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7baeb-c742-47d4-9554-1a83a2b6bba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:57.557262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7baeb-c742-47d4-9554-1a83a2b6bba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:57.560401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7768b01-53d6-45ac-b491-d339e0b61f3f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:57.572273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7baeb-c742-47d4-9554-1a83a2b6bba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:57.573459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7baeb-c742-47d4-9554-1a83a2b6bba9 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:57.574756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cedbd-dfbe-4bdb-a071-c7796f2680fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:57.575757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cedbd-dfbe-4bdb-a071-c7796f2680fe admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:57.576047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cedbd-dfbe-4bdb-a071-c7796f2680fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:57.576320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cedbd-dfbe-4bdb-a071-c7796f2680fe admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:57.577001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cedbd-dfbe-4bdb-a071-c7796f2680fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:57.577449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cedbd-dfbe-4bdb-a071-c7796f2680fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:57.585015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cedbd-dfbe-4bdb-a071-c7796f2680fe admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:57.607918 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2347/4744] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:57.609138 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2397/4744] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:09:57.612454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cedbd-dfbe-4bdb-a071-c7796f2680fe admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:57.628990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cedbd-dfbe-4bdb-a071-c7796f2680fe admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:57.636936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e415a0-386b-4bf4-bc14-893848286222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:57.637713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e415a0-386b-4bf4-bc14-893848286222 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:57.637980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e415a0-386b-4bf4-bc14-893848286222 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:57.638249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e415a0-386b-4bf4-bc14-893848286222 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:57.638945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e415a0-386b-4bf4-bc14-893848286222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:57.640937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e415a0-386b-4bf4-bc14-893848286222 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:57.685581 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2348/4745] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:57 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:57.687030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7baeb-c742-47d4-9554-1a83a2b6bba9 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:57.702842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7baeb-c742-47d4-9554-1a83a2b6bba9 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:57.710594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:57.817960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2d96d-b01c-4be6-8714-3ea19c79f0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:57.818676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2d96d-b01c-4be6-8714-3ea19c79f0bb admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:57.818676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2d96d-b01c-4be6-8714-3ea19c79f0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:57.818676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2d96d-b01c-4be6-8714-3ea19c79f0bb admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:57.821111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2d96d-b01c-4be6-8714-3ea19c79f0bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:57.824301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2d96d-b01c-4be6-8714-3ea19c79f0bb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:57.869310 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2398/4746] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:57 2022] POST /identity/v3/users => generated 284 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:57.870807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e415a0-386b-4bf4-bc14-893848286222 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:57.888863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e415a0-386b-4bf4-bc14-893848286222 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:57.899608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f7d6b-b4b7-4516-99d5-ea92ca71069d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:57.905077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f7d6b-b4b7-4516-99d5-ea92ca71069d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:57.905744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f7d6b-b4b7-4516-99d5-ea92ca71069d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:57.906080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f7d6b-b4b7-4516-99d5-ea92ca71069d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:57.907269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f7d6b-b4b7-4516-99d5-ea92ca71069d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:57.909942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f7d6b-b4b7-4516-99d5-ea92ca71069d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:57.929022 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2349/4747] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:57.934280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2d96d-b01c-4be6-8714-3ea19c79f0bb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:57.959466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2d96d-b01c-4be6-8714-3ea19c79f0bb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:57.970146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b1386-3767-48c0-909b-e9d887251bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:57.975039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1386-3767-48c0-909b-e9d887251bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:57.975691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1386-3767-48c0-909b-e9d887251bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:57.975937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1386-3767-48c0-909b-e9d887251bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:57.977225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b1386-3767-48c0-909b-e9d887251bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:57.980605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b1386-3767-48c0-909b-e9d887251bc8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:57.982231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f7d6b-b4b7-4516-99d5-ea92ca71069d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:57.984354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e47f7d6b-b4b7-4516-99d5-ea92ca71069d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:57.994470 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2399/4748] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:57.999361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b1386-3767-48c0-909b-e9d887251bc8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:58.006439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f7d6b-b4b7-4516-99d5-ea92ca71069d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:58.021416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c480b-a28a-4827-97d4-ec8ea747d9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:58.022704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c480b-a28a-4827-97d4-ec8ea747d9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:58.023237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c480b-a28a-4827-97d4-ec8ea747d9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:58.025244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c480b-a28a-4827-97d4-ec8ea747d9aa admin admin] PATH_INFO: `/v3/projects/19b41386891848609ba30a7c36c57dd6` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:58.026762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c480b-a28a-4827-97d4-ec8ea747d9aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b41386891848609ba30a7c36c57dd6)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:58.031891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b1386-3767-48c0-909b-e9d887251bc8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:58.035469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c480b-a28a-4827-97d4-ec8ea747d9aa admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:58.039167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1973b4e-2290-4e0b-8d0e-2b5f553d511a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:58.040776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1973b4e-2290-4e0b-8d0e-2b5f553d511a admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:58.041908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1973b4e-2290-4e0b-8d0e-2b5f553d511a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:58.042507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1973b4e-2290-4e0b-8d0e-2b5f553d511a admin admin] PATH_INFO: `/v3/projects/f7bc4969ba8f4974a03c71244f63bb18/users/e27a739b6324457e88fbaeace3aa0574/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:58.064883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1973b4e-2290-4e0b-8d0e-2b5f553d511a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7bc4969ba8f4974a03c71244f63bb18, user_id=e27a739b6324457e88fbaeace3aa0574, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:58.069911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1973b4e-2290-4e0b-8d0e-2b5f553d511a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:58.105356 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2400/4749] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:57 2022] PUT /identity/v3/projects/f7bc4969ba8f4974a03c71244f63bb18/users/e27a739b6324457e88fbaeace3aa0574/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:58.118004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a39c480b-a28a-4827-97d4-ec8ea747d9aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b41386891848609ba30a7c36c57dd6', 'request_id': 'req-a39c480b-a28a-4827-97d4-ec8ea747d9aa'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:58.118637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad9b3e-5d65-4239-ac0b-12103b3fc510 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:58.119151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad9b3e-5d65-4239-ac0b-12103b3fc510 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:58.119467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad9b3e-5d65-4239-ac0b-12103b3fc510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:58.122268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:58.137283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a39c480b-a28a-4827-97d4-ec8ea747d9aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b41386891848609ba30a7c36c57dd6', 'request_id': 'req-a39c480b-a28a-4827-97d4-ec8ea747d9aa'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:58.149916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a39c480b-a28a-4827-97d4-ec8ea747d9aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b41386891848609ba30a7c36c57dd6', 'request_id': 'req-a39c480b-a28a-4827-97d4-ec8ea747d9aa'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:58.168077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a39c480b-a28a-4827-97d4-ec8ea747d9aa admin admin] The token cache is being invalidate because project 19b41386891848609ba30a7c36c57dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:58.169584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a39c480b-a28a-4827-97d4-ec8ea747d9aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:58.171274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b7ad9b3e-5d65-4239-ac0b-12103b3fc510 None None] MFA Rules not processed for user `e27a739b6324457e88fbaeace3aa0574`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:58.172780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b7ad9b3e-5d65-4239-ac0b-12103b3fc510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:58.173048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a39c480b-a28a-4827-97d4-ec8ea747d9aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:58.176557 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2350/4750] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:57 2022] DELETE /identity/v3/projects/19b41386891848609ba30a7c36c57dd6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:09:58.181567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6539c9-0122-468a-a554-15ae06a7b372 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:58.182099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6539c9-0122-468a-a554-15ae06a7b372 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:58.183766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6539c9-0122-468a-a554-15ae06a7b372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:58.243781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1f6539c9-0122-468a-a554-15ae06a7b372 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:58.247450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1f6539c9-0122-468a-a554-15ae06a7b372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:58.332611 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2401/4751] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:09:58.338854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad9b3e-5d65-4239-ac0b-12103b3fc510 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:58.341876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b7ad9b3e-5d65-4239-ac0b-12103b3fc510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:58.431880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:58.432462 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2351/4752] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:58.440738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6539c9-0122-468a-a554-15ae06a7b372 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:58.442456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1f6539c9-0122-468a-a554-15ae06a7b372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:58.471615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6539c9-0122-468a-a554-15ae06a7b372 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:58.472385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad9b3e-5d65-4239-ac0b-12103b3fc510 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:58.477634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f5d19-8e9f-4587-8524-90fde62736c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:58.478190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f5d19-8e9f-4587-8524-90fde62736c3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:58.478463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f5d19-8e9f-4587-8524-90fde62736c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:58.478726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f5d19-8e9f-4587-8524-90fde62736c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:58.479237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:58.479405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f5d19-8e9f-4587-8524-90fde62736c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:58.480199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:58.480685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:58.481113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] PATH_INFO: `/v3/projects/2ab01f6e58774255af430c4bb1e24d04` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:58.481329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f5d19-8e9f-4587-8524-90fde62736c3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:58.482896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab01f6e58774255af430c4bb1e24d04)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:58.485245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:58.543152 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2402/4753] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:58 2022] POST /identity/v3/projects => generated 367 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:09:58.543152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33d2f3-349c-48d1-8641-a376f7ca75d7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:58.543152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33d2f3-349c-48d1-8641-a376f7ca75d7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:58.543152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33d2f3-349c-48d1-8641-a376f7ca75d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:58.562181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab01f6e58774255af430c4bb1e24d04', 'request_id': 'req-fbb5cf24-7463-4247-a726-728bb77da4d6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:58.569862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9b33d2f3-349c-48d1-8641-a376f7ca75d7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:58.571008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab01f6e58774255af430c4bb1e24d04', 'request_id': 'req-fbb5cf24-7463-4247-a726-728bb77da4d6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:58.571359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9b33d2f3-349c-48d1-8641-a376f7ca75d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:58.580667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab01f6e58774255af430c4bb1e24d04', 'request_id': 'req-fbb5cf24-7463-4247-a726-728bb77da4d6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:58.595673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] The token cache is being invalidate because project 2ab01f6e58774255af430c4bb1e24d04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:09:58.595997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:58.596797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:09:58.598675 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2352/4754] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:09:58 2022] DELETE /identity/v3/projects/2ab01f6e58774255af430c4bb1e24d04 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:58.599518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:58.601194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:58.683952 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2403/4755] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:58.683952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8a15f-e10a-47c8-99d5-644eee1dd5ba None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:58.683952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8a15f-e10a-47c8-99d5-644eee1dd5ba None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:58.683952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8a15f-e10a-47c8-99d5-644eee1dd5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:58.706794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-68f8a15f-e10a-47c8-99d5-644eee1dd5ba None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:58.709382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-68f8a15f-e10a-47c8-99d5-644eee1dd5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:58.709990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5cf24-7463-4247-a726-728bb77da4d6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:58.715907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89660466-74e2-4c13-8835-70197aeebf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:58.717050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89660466-74e2-4c13-8835-70197aeebf0d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:58.717724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89660466-74e2-4c13-8835-70197aeebf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:58.718193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89660466-74e2-4c13-8835-70197aeebf0d admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:58.719287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89660466-74e2-4c13-8835-70197aeebf0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:58.720145 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2404/4756] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:58.721300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8a15f-e10a-47c8-99d5-644eee1dd5ba None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:58.722521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89660466-74e2-4c13-8835-70197aeebf0d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:58.735220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8a15f-e10a-47c8-99d5-644eee1dd5ba None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:58.746372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:58.751235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202a8c8-7aa9-4ee4-8c58-cd2ff73cf0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:58.751887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202a8c8-7aa9-4ee4-8c58-cd2ff73cf0da admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:58.752158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202a8c8-7aa9-4ee4-8c58-cd2ff73cf0da admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:58.752445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202a8c8-7aa9-4ee4-8c58-cd2ff73cf0da admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:58.753592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202a8c8-7aa9-4ee4-8c58-cd2ff73cf0da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:58.754172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202a8c8-7aa9-4ee4-8c58-cd2ff73cf0da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:58.757475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202a8c8-7aa9-4ee4-8c58-cd2ff73cf0da admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:58.790950 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2405/4757] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:58.792176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202a8c8-7aa9-4ee4-8c58-cd2ff73cf0da admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:58.797604 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2353/4758] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:58 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:58.801868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89660466-74e2-4c13-8835-70197aeebf0d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:58.827446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89660466-74e2-4c13-8835-70197aeebf0d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:58.830442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202a8c8-7aa9-4ee4-8c58-cd2ff73cf0da admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:58.835415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfd112-a52f-4c05-9cdd-4f2ae0d284d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:58.836559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfd112-a52f-4c05-9cdd-4f2ae0d284d5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:58.837121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfd112-a52f-4c05-9cdd-4f2ae0d284d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:58.837546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfd112-a52f-4c05-9cdd-4f2ae0d284d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:58.838402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfd112-a52f-4c05-9cdd-4f2ae0d284d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:58.841769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfd112-a52f-4c05-9cdd-4f2ae0d284d5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:58.848935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6473636-d654-4008-b7b5-8a3dfa7c5626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:58.851704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6473636-d654-4008-b7b5-8a3dfa7c5626 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:58.851704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6473636-d654-4008-b7b5-8a3dfa7c5626 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:58.851704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6473636-d654-4008-b7b5-8a3dfa7c5626 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:58.851704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6473636-d654-4008-b7b5-8a3dfa7c5626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:58.853968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6473636-d654-4008-b7b5-8a3dfa7c5626 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:58.892746 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2354/4759] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:58.894196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1971fb3-0de6-45de-bbfd-b896f1f4ae46 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:58.894514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1971fb3-0de6-45de-bbfd-b896f1f4ae46 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:58.894872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1971fb3-0de6-45de-bbfd-b896f1f4ae46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:58.914405 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2406/4760] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:58 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:58.916707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6473636-d654-4008-b7b5-8a3dfa7c5626 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:58.935702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c1971fb3-0de6-45de-bbfd-b896f1f4ae46 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:58.936970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c1971fb3-0de6-45de-bbfd-b896f1f4ae46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:59.007314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6473636-d654-4008-b7b5-8a3dfa7c5626 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.010225 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2355/4761] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:59.011105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1971fb3-0de6-45de-bbfd-b896f1f4ae46 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.019570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3692097-646a-4d90-a623-37d3ec3e5ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.020808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3692097-646a-4d90-a623-37d3ec3e5ab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.021320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3692097-646a-4d90-a623-37d3ec3e5ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.021955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3692097-646a-4d90-a623-37d3ec3e5ab0 admin admin] PATH_INFO: `/v3/projects/76334a932f3346aaae842d51c8f9a139/users/5adde001844b4eaa9f0908a116077501/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.025077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1971fb3-0de6-45de-bbfd-b896f1f4ae46 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.029725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e617bba-d9ef-4abb-b470-823c5d5d05a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.030326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e617bba-d9ef-4abb-b470-823c5d5d05a2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.030594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e617bba-d9ef-4abb-b470-823c5d5d05a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.030857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e617bba-d9ef-4abb-b470-823c5d5d05a2 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.033049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e617bba-d9ef-4abb-b470-823c5d5d05a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.038326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e617bba-d9ef-4abb-b470-823c5d5d05a2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.041414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3692097-646a-4d90-a623-37d3ec3e5ab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76334a932f3346aaae842d51c8f9a139, user_id=5adde001844b4eaa9f0908a116077501, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.044084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3692097-646a-4d90-a623-37d3ec3e5ab0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.066141 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2407/4762] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:58 2022] PUT /identity/v3/projects/76334a932f3346aaae842d51c8f9a139/users/5adde001844b4eaa9f0908a116077501/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:59.067319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3692097-646a-4d90-a623-37d3ec3e5ab0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.085167 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2356/4763] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:59 2022] POST /identity/v3/users => generated 288 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:59.086552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e617bba-d9ef-4abb-b470-823c5d5d05a2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.097889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e617bba-d9ef-4abb-b470-823c5d5d05a2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.101592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:59.157961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3692097-646a-4d90-a623-37d3ec3e5ab0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.165067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbaffb2-ae20-449b-a6f3-071d3764a90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.165922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbaffb2-ae20-449b-a6f3-071d3764a90b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.166384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbaffb2-ae20-449b-a6f3-071d3764a90b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.166649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbaffb2-ae20-449b-a6f3-071d3764a90b admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.167372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbaffb2-ae20-449b-a6f3-071d3764a90b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:09:59.167676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbaffb2-ae20-449b-a6f3-071d3764a90b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.169336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbaffb2-ae20-449b-a6f3-071d3764a90b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.177713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b17696-7946-42d3-bd75-533fd85de094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.178809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b17696-7946-42d3-bd75-533fd85de094 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.179085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b17696-7946-42d3-bd75-533fd85de094 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.179408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b17696-7946-42d3-bd75-533fd85de094 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.180036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b17696-7946-42d3-bd75-533fd85de094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.182092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b17696-7946-42d3-bd75-533fd85de094 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.185958 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2408/4764] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:09:59.187102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbaffb2-ae20-449b-a6f3-071d3764a90b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.201434 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2357/4765] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:59.201434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b17696-7946-42d3-bd75-533fd85de094 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.204268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbaffb2-ae20-449b-a6f3-071d3764a90b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.211034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527f59c-acf4-4a55-b44f-b237e9b8c9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.211642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527f59c-acf4-4a55-b44f-b237e9b8c9cb admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.211770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b17696-7946-42d3-bd75-533fd85de094 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.211979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527f59c-acf4-4a55-b44f-b237e9b8c9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.212164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527f59c-acf4-4a55-b44f-b237e9b8c9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.213233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527f59c-acf4-4a55-b44f-b237e9b8c9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.215576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527f59c-acf4-4a55-b44f-b237e9b8c9cb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.216908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9ea1f-dae9-45d0-9137-6c453ebdbaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.217513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9ea1f-dae9-45d0-9137-6c453ebdbaf9 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.217818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9ea1f-dae9-45d0-9137-6c453ebdbaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.218090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9ea1f-dae9-45d0-9137-6c453ebdbaf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.218715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9ea1f-dae9-45d0-9137-6c453ebdbaf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.220660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9ea1f-dae9-45d0-9137-6c453ebdbaf9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.231053 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2409/4766] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:59.235384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527f59c-acf4-4a55-b44f-b237e9b8c9cb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.249853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527f59c-acf4-4a55-b44f-b237e9b8c9cb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.258500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db7123-f119-4519-90a4-a76670160744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.259182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db7123-f119-4519-90a4-a76670160744 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.259627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db7123-f119-4519-90a4-a76670160744 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.259889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db7123-f119-4519-90a4-a76670160744 admin admin] PATH_INFO: `/v3/projects/76334a932f3346aaae842d51c8f9a139/users/5adde001844b4eaa9f0908a116077501/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.262005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db7123-f119-4519-90a4-a76670160744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76334a932f3346aaae842d51c8f9a139, user_id=5adde001844b4eaa9f0908a116077501, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.264799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db7123-f119-4519-90a4-a76670160744 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.271356 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2358/4767] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:09:59 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:59.271356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9ea1f-dae9-45d0-9137-6c453ebdbaf9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.282704 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2410/4768] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:59 2022] PUT /identity/v3/projects/76334a932f3346aaae842d51c8f9a139/users/5adde001844b4eaa9f0908a116077501/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:59.283740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db7123-f119-4519-90a4-a76670160744 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.351507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db7123-f119-4519-90a4-a76670160744 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.353049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:59.364561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac0c27-780f-40f7-82ba-e298fea859a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.364561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac0c27-780f-40f7-82ba-e298fea859a0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.364561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac0c27-780f-40f7-82ba-e298fea859a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.364561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac0c27-780f-40f7-82ba-e298fea859a0 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.365284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac0c27-780f-40f7-82ba-e298fea859a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.365614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac0c27-780f-40f7-82ba-e298fea859a0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.381089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9ea1f-dae9-45d0-9137-6c453ebdbaf9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.390027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fad592-9a1f-4b92-a644-22ac404507db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.390027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fad592-9a1f-4b92-a644-22ac404507db admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.390027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fad592-9a1f-4b92-a644-22ac404507db admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.390027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fad592-9a1f-4b92-a644-22ac404507db admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.390947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fad592-9a1f-4b92-a644-22ac404507db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.391643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fad592-9a1f-4b92-a644-22ac404507db admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.394164 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2411/4769] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:09:59 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:59.395624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3aaa0-ca48-4302-91f9-0f148a3708bd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.395978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3aaa0-ca48-4302-91f9-0f148a3708bd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.396261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3aaa0-ca48-4302-91f9-0f148a3708bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.408519 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2359/4770] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:59.408519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fad592-9a1f-4b92-a644-22ac404507db admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.420532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fad592-9a1f-4b92-a644-22ac404507db admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.425809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b3d3aaa0-ca48-4302-91f9-0f148a3708bd None None] MFA Rules not processed for user `5adde001844b4eaa9f0908a116077501`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:59.427354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b3d3aaa0-ca48-4302-91f9-0f148a3708bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:59.429898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7224608-724b-4f65-b1b6-edaaf8bb6c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.430527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7224608-724b-4f65-b1b6-edaaf8bb6c0b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.430806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7224608-724b-4f65-b1b6-edaaf8bb6c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.431073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7224608-724b-4f65-b1b6-edaaf8bb6c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.431963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7224608-724b-4f65-b1b6-edaaf8bb6c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.434037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7224608-724b-4f65-b1b6-edaaf8bb6c0b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.446734 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2360/4771] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:59.447554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7224608-724b-4f65-b1b6-edaaf8bb6c0b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.463735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7224608-724b-4f65-b1b6-edaaf8bb6c0b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.470080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adcd61-4d43-4d57-9426-1ba95f859aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.470927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adcd61-4d43-4d57-9426-1ba95f859aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.471260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adcd61-4d43-4d57-9426-1ba95f859aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.471592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adcd61-4d43-4d57-9426-1ba95f859aa0 admin admin] PATH_INFO: `/v3/projects/18895d700e874d019b1f7636f2ed5e40/users/97a1575a72e74fa2a188ba9eef8ace8a/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.494359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adcd61-4d43-4d57-9426-1ba95f859aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18895d700e874d019b1f7636f2ed5e40, user_id=97a1575a72e74fa2a188ba9eef8ace8a, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.497203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adcd61-4d43-4d57-9426-1ba95f859aa0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.517699 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2361/4772] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:59 2022] PUT /identity/v3/projects/18895d700e874d019b1f7636f2ed5e40/users/97a1575a72e74fa2a188ba9eef8ace8a/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:59.517699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adcd61-4d43-4d57-9426-1ba95f859aa0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.518035 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2412/4773] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:59.522014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3aaa0-ca48-4302-91f9-0f148a3708bd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.538839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:09:59.547753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3aaa0-ca48-4302-91f9-0f148a3708bd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.625356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adcd61-4d43-4d57-9426-1ba95f859aa0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.636148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2b0d3-0d4d-44d0-bf67-f8f5b2834fbb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.636804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2b0d3-0d4d-44d0-bf67-f8f5b2834fbb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.637078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2b0d3-0d4d-44d0-bf67-f8f5b2834fbb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.637409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2b0d3-0d4d-44d0-bf67-f8f5b2834fbb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.638415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2b0d3-0d4d-44d0-bf67-f8f5b2834fbb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.640416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2b0d3-0d4d-44d0-bf67-f8f5b2834fbb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.647098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c004491-8ebe-4064-ad37-abd7c239fd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.647747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c004491-8ebe-4064-ad37-abd7c239fd15 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.648019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c004491-8ebe-4064-ad37-abd7c239fd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.648280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c004491-8ebe-4064-ad37-abd7c239fd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.649018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c004491-8ebe-4064-ad37-abd7c239fd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.650829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c004491-8ebe-4064-ad37-abd7c239fd15 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.664194 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2413/4774] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:09:59 2022] POST /identity/v3/roles => generated 235 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:59.664194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f135bf-c4fe-4cf0-9f38-6a54d5a51de3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.664194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f135bf-c4fe-4cf0-9f38-6a54d5a51de3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.664194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f135bf-c4fe-4cf0-9f38-6a54d5a51de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.664734 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2362/4775] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:09:59 2022] GET /identity/v3/roles => generated 1646 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:09:59.667578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c004491-8ebe-4064-ad37-abd7c239fd15 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.690847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c004491-8ebe-4064-ad37-abd7c239fd15 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.707573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-43f135bf-c4fe-4cf0-9f38-6a54d5a51de3 None None] MFA Rules not processed for user `97a1575a72e74fa2a188ba9eef8ace8a`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:09:59.709511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecc06f-3e12-4ab8-b2b0-7c37344216eb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.709688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-43f135bf-c4fe-4cf0-9f38-6a54d5a51de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:59.710333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc06f-3e12-4ab8-b2b0-7c37344216eb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.710609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc06f-3e12-4ab8-b2b0-7c37344216eb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.710876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc06f-3e12-4ab8-b2b0-7c37344216eb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.711660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecc06f-3e12-4ab8-b2b0-7c37344216eb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.713265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecc06f-3e12-4ab8-b2b0-7c37344216eb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.740533 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2363/4776] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:09:59 2022] POST /identity/v3/roles => generated 235 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:59.740533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecc06f-3e12-4ab8-b2b0-7c37344216eb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.768521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecc06f-3e12-4ab8-b2b0-7c37344216eb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.775349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e5c6f-1ac0-452a-a027-58e14ea9d5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.776182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e5c6f-1ac0-452a-a027-58e14ea9d5b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.776790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e5c6f-1ac0-452a-a027-58e14ea9d5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.777331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e5c6f-1ac0-452a-a027-58e14ea9d5b9 admin admin] PATH_INFO: `/v3/projects/46d6021c84034c16aed842b25a8b54c9/users/53e5c96846c3473c91d0893de46f5d59/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.800896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e5c6f-1ac0-452a-a027-58e14ea9d5b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46d6021c84034c16aed842b25a8b54c9, user_id=53e5c96846c3473c91d0893de46f5d59, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.801383 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2414/4777] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:09:59.808791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f135bf-c4fe-4cf0-9f38-6a54d5a51de3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.808900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e5c6f-1ac0-452a-a027-58e14ea9d5b9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.831035 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2364/4778] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:09:59 2022] PUT /identity/v3/projects/46d6021c84034c16aed842b25a8b54c9/users/53e5c96846c3473c91d0893de46f5d59/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:09:59.832693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e5c6f-1ac0-452a-a027-58e14ea9d5b9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:09:59.835958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f135bf-c4fe-4cf0-9f38-6a54d5a51de3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:09:59.846795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-217e5c6f-1ac0-452a-a027-58e14ea9d5b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:09:59.929424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0ba45-c87f-4758-8e61-c03ece4271ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:09:59.930258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0ba45-c87f-4758-8e61-c03ece4271ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.930602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0ba45-c87f-4758-8e61-c03ece4271ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.930944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0ba45-c87f-4758-8e61-c03ece4271ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.932088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0ba45-c87f-4758-8e61-c03ece4271ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:09:59.933906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0ba45-c87f-4758-8e61-c03ece4271ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:09:59.959694 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2415/4779] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:09:59 2022] POST /identity/v3/roles => generated 235 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:09:59.961852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dc550-be35-43a6-8378-f3e05a47aa52 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:09:59.961852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dc550-be35-43a6-8378-f3e05a47aa52 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:09:59.962139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dc550-be35-43a6-8378-f3e05a47aa52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:09:59.974450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:00.012115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-837dc550-be35-43a6-8378-f3e05a47aa52 None None] MFA Rules not processed for user `53e5c96846c3473c91d0893de46f5d59`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:00.012986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e5c6f-1ac0-452a-a027-58e14ea9d5b9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.013631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-837dc550-be35-43a6-8378-f3e05a47aa52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:00.022115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcddf9-2c2b-4719-b4ea-404b03d77310 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.022988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcddf9-2c2b-4719-b4ea-404b03d77310 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.023266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcddf9-2c2b-4719-b4ea-404b03d77310 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.023531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcddf9-2c2b-4719-b4ea-404b03d77310 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.025454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:00.105264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcddf9-2c2b-4719-b4ea-404b03d77310 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.107383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcddf9-2c2b-4719-b4ea-404b03d77310 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.109506 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2416/4780] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:00.110422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dc550-be35-43a6-8378-f3e05a47aa52 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.127471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dc550-be35-43a6-8378-f3e05a47aa52 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.133725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ec1d7-9474-41bc-b361-d7a601b631d5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.134558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ec1d7-9474-41bc-b361-d7a601b631d5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.134895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ec1d7-9474-41bc-b361-d7a601b631d5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.135484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ec1d7-9474-41bc-b361-d7a601b631d5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.135621 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2365/4781] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:09:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:00.136178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcddf9-2c2b-4719-b4ea-404b03d77310 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.136501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ec1d7-9474-41bc-b361-d7a601b631d5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.138459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ec1d7-9474-41bc-b361-d7a601b631d5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.168839 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2417/4782] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:10:00 2022] POST /identity/v3/domains => generated 275 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:00.174429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ec1d7-9474-41bc-b361-d7a601b631d5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.218453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcddf9-2c2b-4719-b4ea-404b03d77310 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.296649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ec1d7-9474-41bc-b361-d7a601b631d5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.305079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1447f3-5298-48e5-8057-ace58f8d8114 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.305589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b292fd-31ce-41d9-9d1e-c004294d59a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.305755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1447f3-5298-48e5-8057-ace58f8d8114 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.305973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1447f3-5298-48e5-8057-ace58f8d8114 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.306246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1447f3-5298-48e5-8057-ace58f8d8114 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.306839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b292fd-31ce-41d9-9d1e-c004294d59a9 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.306969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1447f3-5298-48e5-8057-ace58f8d8114 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.307302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b292fd-31ce-41d9-9d1e-c004294d59a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.307664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b292fd-31ce-41d9-9d1e-c004294d59a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.308401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b292fd-31ce-41d9-9d1e-c004294d59a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.308996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1447f3-5298-48e5-8057-ace58f8d8114 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.310579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b292fd-31ce-41d9-9d1e-c004294d59a9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.357924 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2366/4783] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:00 2022] POST /identity/v3/projects => generated 373 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:00.362138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b292fd-31ce-41d9-9d1e-c004294d59a9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.398222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:00.418063 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2418/4784] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:00 2022] POST /identity/v3/projects => generated 400 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:00.418978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1447f3-5298-48e5-8057-ace58f8d8114 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.436916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b292fd-31ce-41d9-9d1e-c004294d59a9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.531366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcb535-b7f5-492f-8a9f-bae1cf8bd190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.531985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcb535-b7f5-492f-8a9f-bae1cf8bd190 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.532258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcb535-b7f5-492f-8a9f-bae1cf8bd190 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.532600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcb535-b7f5-492f-8a9f-bae1cf8bd190 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.533041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1447f3-5298-48e5-8057-ace58f8d8114 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.533513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcb535-b7f5-492f-8a9f-bae1cf8bd190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.535198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcb535-b7f5-492f-8a9f-bae1cf8bd190 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.539103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f60b3-3ff2-4fb4-a88e-969467a2e36c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.539845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f60b3-3ff2-4fb4-a88e-969467a2e36c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.540216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f60b3-3ff2-4fb4-a88e-969467a2e36c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.540621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f60b3-3ff2-4fb4-a88e-969467a2e36c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/groups` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.542330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f60b3-3ff2-4fb4-a88e-969467a2e36c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.543459 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-979f60b3-3ff2-4fb4-a88e-969467a2e36c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:00.543459 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:00.543459 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:00.545025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f60b3-3ff2-4fb4-a88e-969467a2e36c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.569070 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2367/4785] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:00 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:00.574617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcb535-b7f5-492f-8a9f-bae1cf8bd190 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.575053 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2419/4786] 10.222.0.23 () {60 vars in 1147 bytes} [Sat Aug 20 21:10:00 2022] POST /identity/v3/groups => generated 301 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:00.581757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f60b3-3ff2-4fb4-a88e-969467a2e36c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.593648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f60b3-3ff2-4fb4-a88e-969467a2e36c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.598859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb84ee1-a56d-486d-9f92-b93fd80d5527 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.599976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb84ee1-a56d-486d-9f92-b93fd80d5527 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.600363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb84ee1-a56d-486d-9f92-b93fd80d5527 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.600796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb84ee1-a56d-486d-9f92-b93fd80d5527 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.601548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb84ee1-a56d-486d-9f92-b93fd80d5527 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.603250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb84ee1-a56d-486d-9f92-b93fd80d5527 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.608281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:00.618558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcb535-b7f5-492f-8a9f-bae1cf8bd190 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.625423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2e461-97a6-4f4e-a077-fcfadd1b0356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.626502 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2420/4787] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:10:00 2022] POST /identity/v3/roles => generated 235 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:00.630084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2e461-97a6-4f4e-a077-fcfadd1b0356 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.630576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2e461-97a6-4f4e-a077-fcfadd1b0356 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.630922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2e461-97a6-4f4e-a077-fcfadd1b0356 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.631617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb84ee1-a56d-486d-9f92-b93fd80d5527 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.631837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2e461-97a6-4f4e-a077-fcfadd1b0356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.633683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2e461-97a6-4f4e-a077-fcfadd1b0356 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.653242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb84ee1-a56d-486d-9f92-b93fd80d5527 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.656361 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2368/4788] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:00 2022] GET /identity/v3/roles => generated 2324 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:00.657730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2e461-97a6-4f4e-a077-fcfadd1b0356 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.665720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560d683-5aa7-420e-8b23-ddc8bd37fa1e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.669203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560d683-5aa7-420e-8b23-ddc8bd37fa1e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.669610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560d683-5aa7-420e-8b23-ddc8bd37fa1e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.669749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560d683-5aa7-420e-8b23-ddc8bd37fa1e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.670474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560d683-5aa7-420e-8b23-ddc8bd37fa1e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.672440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560d683-5aa7-420e-8b23-ddc8bd37fa1e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.682708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2e461-97a6-4f4e-a077-fcfadd1b0356 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.697599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593df11f-2c74-4a5e-80a0-67b05933d7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.698889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593df11f-2c74-4a5e-80a0-67b05933d7f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.699442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593df11f-2c74-4a5e-80a0-67b05933d7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.700100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593df11f-2c74-4a5e-80a0-67b05933d7f0 admin admin] PATH_INFO: `/v3/projects/f6c564242cec4c29a0f6f980d0a83a3c/users/70711adcb5584d4d9cd46b9c0acb9b7d/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.717064 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2421/4789] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:00 2022] POST /identity/v3/users => generated 439 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:00.718157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560d683-5aa7-420e-8b23-ddc8bd37fa1e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.729588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593df11f-2c74-4a5e-80a0-67b05933d7f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c564242cec4c29a0f6f980d0a83a3c, user_id=70711adcb5584d4d9cd46b9c0acb9b7d, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.735373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593df11f-2c74-4a5e-80a0-67b05933d7f0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.737793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560d683-5aa7-420e-8b23-ddc8bd37fa1e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.758872 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2369/4790] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:00 2022] PUT /identity/v3/projects/f6c564242cec4c29a0f6f980d0a83a3c/users/70711adcb5584d4d9cd46b9c0acb9b7d/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:00.761323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c12b4-bce7-4f2a-aebb-6a62e06b9978 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.762025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c12b4-bce7-4f2a-aebb-6a62e06b9978 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.762280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c12b4-bce7-4f2a-aebb-6a62e06b9978 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.762560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c12b4-bce7-4f2a-aebb-6a62e06b9978 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.763375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c12b4-bce7-4f2a-aebb-6a62e06b9978 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.763517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593df11f-2c74-4a5e-80a0-67b05933d7f0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.764688 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-6c3c12b4-bce7-4f2a-aebb-6a62e06b9978 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:00.764688 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:00.764688 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:00.768547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c12b4-bce7-4f2a-aebb-6a62e06b9978 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.801389 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2422/4791] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:10:00 2022] POST /identity/v3/roles => generated 270 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:00.805532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c12b4-bce7-4f2a-aebb-6a62e06b9978 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.840530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593df11f-2c74-4a5e-80a0-67b05933d7f0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.843017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c12b4-bce7-4f2a-aebb-6a62e06b9978 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.845629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b319c4-6049-4b25-8e30-2ceb5fd37065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.846282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b319c4-6049-4b25-8e30-2ceb5fd37065 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.846555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b319c4-6049-4b25-8e30-2ceb5fd37065 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.846816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b319c4-6049-4b25-8e30-2ceb5fd37065 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.847623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b319c4-6049-4b25-8e30-2ceb5fd37065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.849667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b319c4-6049-4b25-8e30-2ceb5fd37065 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.861989 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2370/4792] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:00 2022] GET /identity/v3/roles => generated 2324 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:00.870836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b319c4-6049-4b25-8e30-2ceb5fd37065 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.883840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b319c4-6049-4b25-8e30-2ceb5fd37065 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:00.890681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55affcd-1d62-49a8-b1f9-50bc90c0598c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.891586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55affcd-1d62-49a8-b1f9-50bc90c0598c admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.891956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55affcd-1d62-49a8-b1f9-50bc90c0598c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.892316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55affcd-1d62-49a8-b1f9-50bc90c0598c admin admin] PATH_INFO: `/v3/projects/f6c564242cec4c29a0f6f980d0a83a3c/users/70711adcb5584d4d9cd46b9c0acb9b7d/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.894629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55affcd-1d62-49a8-b1f9-50bc90c0598c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c564242cec4c29a0f6f980d0a83a3c, user_id=70711adcb5584d4d9cd46b9c0acb9b7d, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.898130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55affcd-1d62-49a8-b1f9-50bc90c0598c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.915523 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2371/4793] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:00 2022] PUT /identity/v3/projects/f6c564242cec4c29a0f6f980d0a83a3c/users/70711adcb5584d4d9cd46b9c0acb9b7d/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:00.919926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41543b-44af-4ca7-a20d-9ae69127b682 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:00.921250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41543b-44af-4ca7-a20d-9ae69127b682 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.921732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41543b-44af-4ca7-a20d-9ae69127b682 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.922174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41543b-44af-4ca7-a20d-9ae69127b682 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/1343eccf1de84fd480c2ac5edd85892f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.926648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee2f3c-c03d-49ab-a57f-55819e97dc1b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:00.926936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee2f3c-c03d-49ab-a57f-55819e97dc1b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:00.927234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee2f3c-c03d-49ab-a57f-55819e97dc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:00.946439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41543b-44af-4ca7-a20d-9ae69127b682 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_grant(project_id=3e37435363bb4abc9199670f3780c04c, user_id=8401b16d587845afa1487456fee0bfaf, role_id=1343eccf1de84fd480c2ac5edd85892f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:00.952408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41543b-44af-4ca7-a20d-9ae69127b682 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:00.980167 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2423/4794] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:00 2022] PUT /identity/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/1343eccf1de84fd480c2ac5edd85892f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:00.981783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:00.984391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41543b-44af-4ca7-a20d-9ae69127b682 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:00.994271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cfee2f3c-c03d-49ab-a57f-55819e97dc1b None None] MFA Rules not processed for user `70711adcb5584d4d9cd46b9c0acb9b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:00.997204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cfee2f3c-c03d-49ab-a57f-55819e97dc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:01.124062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41543b-44af-4ca7-a20d-9ae69127b682 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:01.136273 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2372/4795] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:01.142475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee2f3c-c03d-49ab-a57f-55819e97dc1b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:01.155136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e78662-29ed-425c-94a6-e56ee262dd57 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:01.155992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e78662-29ed-425c-94a6-e56ee262dd57 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:01.157347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e78662-29ed-425c-94a6-e56ee262dd57 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:01.158442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e78662-29ed-425c-94a6-e56ee262dd57 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:01.159312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e78662-29ed-425c-94a6-e56ee262dd57 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Adding query filter params (scope.project.id=3e37435363bb4abc9199670f3780c04c, user.id=8401b16d587845afa1487456fee0bfaf) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:01.162507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee2f3c-c03d-49ab-a57f-55819e97dc1b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:01.163710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e78662-29ed-425c-94a6-e56ee262dd57 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:01.165679 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-75e78662-29ed-425c-94a6-e56ee262dd57 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:01.165679 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:01.165679 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:01.173615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda3cbe-2554-4383-b236-c52a2c8f551b tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70711adcb5584d4d9cd46b9c0acb9b7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6c564242cec4c29a0f6f980d0a83a3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:01.174257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda3cbe-2554-4383-b236-c52a2c8f551b tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:01.174536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda3cbe-2554-4383-b236-c52a2c8f551b tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:01.174807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda3cbe-2554-4383-b236-c52a2c8f551b tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:01.175465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda3cbe-2554-4383-b236-c52a2c8f551b tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:01.175770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda3cbe-2554-4383-b236-c52a2c8f551b tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:01.176110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e78662-29ed-425c-94a6-e56ee262dd57 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:01.177490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda3cbe-2554-4383-b236-c52a2c8f551b tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:01.220244 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2373/4796] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:10:01.221571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda3cbe-2554-4383-b236-c52a2c8f551b tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:01.290622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda3cbe-2554-4383-b236-c52a2c8f551b tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:01.311916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805695bb-d4b4-40ec-8224-7de7af3e3c91 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70711adcb5584d4d9cd46b9c0acb9b7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6c564242cec4c29a0f6f980d0a83a3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:01.312945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805695bb-d4b4-40ec-8224-7de7af3e3c91 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:01.313401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805695bb-d4b4-40ec-8224-7de7af3e3c91 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:01.314732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805695bb-d4b4-40ec-8224-7de7af3e3c91 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:01.316140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805695bb-d4b4-40ec-8224-7de7af3e3c91 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:01.326668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805695bb-d4b4-40ec-8224-7de7af3e3c91 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:01.340499 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2424/4797] 10.222.0.23 () {58 vars in 1356 bytes} [Sat Aug 20 21:10:00 2022] GET /identity/v3/role_assignments?scope.project.id=3e37435363bb4abc9199670f3780c04c&user.id=8401b16d587845afa1487456fee0bfaf&effective => generated 228 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:10:01.345854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e78662-29ed-425c-94a6-e56ee262dd57 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:01.412641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e78662-29ed-425c-94a6-e56ee262dd57 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:01.435449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33cbbb7-1ee1-44a9-a55d-5f6f11d305db tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:01.436692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33cbbb7-1ee1-44a9-a55d-5f6f11d305db tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:01.438267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33cbbb7-1ee1-44a9-a55d-5f6f11d305db tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:01.439385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33cbbb7-1ee1-44a9-a55d-5f6f11d305db tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/1343eccf1de84fd480c2ac5edd85892f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:01.462507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33cbbb7-1ee1-44a9-a55d-5f6f11d305db tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=3e37435363bb4abc9199670f3780c04c, user_id=8401b16d587845afa1487456fee0bfaf, role_id=1343eccf1de84fd480c2ac5edd85892f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:01.478341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33cbbb7-1ee1-44a9-a55d-5f6f11d305db tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:01.509010 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2374/4798] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:01 2022] POST /identity/v3/projects => generated 350 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:01.516514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805695bb-d4b4-40ec-8224-7de7af3e3c91 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:01.516772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:01.539775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c33cbbb7-1ee1-44a9-a55d-5f6f11d305db tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role 1343eccf1de84fd480c2ac5edd85892f was removed from user 8401b16d587845afa1487456fee0bfaf on project 3e37435363bb4abc9199670f3780c04c. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:01.542499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c33cbbb7-1ee1-44a9-a55d-5f6f11d305db tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:01.543734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c33cbbb7-1ee1-44a9-a55d-5f6f11d305db tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:01.566008 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2425/4799] 10.222.0.23 () {58 vars in 1467 bytes} [Sat Aug 20 21:10:01 2022] DELETE /identity/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/1343eccf1de84fd480c2ac5edd85892f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:01.571072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33cbbb7-1ee1-44a9-a55d-5f6f11d305db tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:01.572877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c33cbbb7-1ee1-44a9-a55d-5f6f11d305db tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:01.668787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33cbbb7-1ee1-44a9-a55d-5f6f11d305db tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:01.674626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688155c6-d6b8-45f5-a8a0-fe763a47409c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:01.675029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688155c6-d6b8-45f5-a8a0-fe763a47409c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:01.675315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688155c6-d6b8-45f5-a8a0-fe763a47409c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:01.675589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688155c6-d6b8-45f5-a8a0-fe763a47409c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/1343eccf1de84fd480c2ac5edd85892f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:01.676946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688155c6-d6b8-45f5-a8a0-fe763a47409c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=1343eccf1de84fd480c2ac5edd85892f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:01.678847 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-688155c6-d6b8-45f5-a8a0-fe763a47409c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:01.678847 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:01.678847 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:01.680180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688155c6-d6b8-45f5-a8a0-fe763a47409c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:01.747771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-688155c6-d6b8-45f5-a8a0-fe763a47409c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role 1343eccf1de84fd480c2ac5edd85892f has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:01.748337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-688155c6-d6b8-45f5-a8a0-fe763a47409c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:01.749323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-688155c6-d6b8-45f5-a8a0-fe763a47409c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:01.752515 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2426/4800] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:01 2022] DELETE /identity/v3/roles/1343eccf1de84fd480c2ac5edd85892f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:01.758607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688155c6-d6b8-45f5-a8a0-fe763a47409c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:01.760710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805695bb-d4b4-40ec-8224-7de7af3e3c91 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:01.761634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-688155c6-d6b8-45f5-a8a0-fe763a47409c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:01.877276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36491f5e-a5d9-4b53-a880-0efd229d5985 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70711adcb5584d4d9cd46b9c0acb9b7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6c564242cec4c29a0f6f980d0a83a3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:01.877902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36491f5e-a5d9-4b53-a880-0efd229d5985 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:01.878183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36491f5e-a5d9-4b53-a880-0efd229d5985 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:01.878704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36491f5e-a5d9-4b53-a880-0efd229d5985 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:01.879974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36491f5e-a5d9-4b53-a880-0efd229d5985 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:01.880317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36491f5e-a5d9-4b53-a880-0efd229d5985 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:01.883088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36491f5e-a5d9-4b53-a880-0efd229d5985 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:01.886193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688155c6-d6b8-45f5-a8a0-fe763a47409c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:01.891758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145a954-dc73-46f2-a59f-eca100c0e1a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:01.893658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145a954-dc73-46f2-a59f-eca100c0e1a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:01.894174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145a954-dc73-46f2-a59f-eca100c0e1a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:01.894631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145a954-dc73-46f2-a59f-eca100c0e1a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/be50ad6b14b342a9b9347d2bf643b250` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:01.897651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145a954-dc73-46f2-a59f-eca100c0e1a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_grant(project_id=3e37435363bb4abc9199670f3780c04c, user_id=8401b16d587845afa1487456fee0bfaf, role_id=be50ad6b14b342a9b9347d2bf643b250)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:01.898238 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2375/4801] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:01.901847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145a954-dc73-46f2-a59f-eca100c0e1a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:01.908943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36491f5e-a5d9-4b53-a880-0efd229d5985 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:01.911487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-36491f5e-a5d9-4b53-a880-0efd229d5985 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:01.920749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:01.939761 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2427/4802] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:01 2022] PUT /identity/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/be50ad6b14b342a9b9347d2bf643b250 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:01.947508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145a954-dc73-46f2-a59f-eca100c0e1a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:01.992795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36491f5e-a5d9-4b53-a880-0efd229d5985 tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:02.054383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4957bb-e745-40de-bf17-f28c22e97979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:02.055108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4957bb-e745-40de-bf17-f28c22e97979 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:02.055613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4957bb-e745-40de-bf17-f28c22e97979 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:02.056015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4957bb-e745-40de-bf17-f28c22e97979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:02.058069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ad4957bb-e745-40de-bf17-f28c22e97979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:02.058649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:02.085315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145a954-dc73-46f2-a59f-eca100c0e1a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:02.090705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7960231-1a6b-42a3-ba8a-55e6844b8843 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:02.091673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7960231-1a6b-42a3-ba8a-55e6844b8843 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:02.092134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7960231-1a6b-42a3-ba8a-55e6844b8843 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:02.092453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7960231-1a6b-42a3-ba8a-55e6844b8843 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:02.094406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7960231-1a6b-42a3-ba8a-55e6844b8843 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:02.099191 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-d7960231-1a6b-42a3-ba8a-55e6844b8843 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:02.099191 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:02.099191 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:02.100245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7960231-1a6b-42a3-ba8a-55e6844b8843 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:02.137504 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2428/4803] 10.222.0.23 () {60 vars in 1363 bytes} [Sat Aug 20 21:10:01 2022] PUT /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5 => generated 521 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:02.138750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4957bb-e745-40de-bf17-f28c22e97979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:02.139216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7960231-1a6b-42a3-ba8a-55e6844b8843 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:02.144682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4957bb-e745-40de-bf17-f28c22e97979 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:02.159143 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2376/4804] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:02.229720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7960231-1a6b-42a3-ba8a-55e6844b8843 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:02.246572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0cf9a-9ff7-44b7-9f98-bfdc3ce2ba21 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:02.247520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0cf9a-9ff7-44b7-9f98-bfdc3ce2ba21 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:02.247862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0cf9a-9ff7-44b7-9f98-bfdc3ce2ba21 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:02.248195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0cf9a-9ff7-44b7-9f98-bfdc3ce2ba21 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:02.249622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0cf9a-9ff7-44b7-9f98-bfdc3ce2ba21 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Adding query filter params (scope.project.id=3e37435363bb4abc9199670f3780c04c, user.id=8401b16d587845afa1487456fee0bfaf) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:02.250112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0cf9a-9ff7-44b7-9f98-bfdc3ce2ba21 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:02.252632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0cf9a-9ff7-44b7-9f98-bfdc3ce2ba21 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:02.321929 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2429/4805] 10.222.0.23 () {58 vars in 1356 bytes} [Sat Aug 20 21:10:02 2022] GET /identity/v3/role_assignments?scope.project.id=3e37435363bb4abc9199670f3780c04c&user.id=8401b16d587845afa1487456fee0bfaf&effective => generated 1061 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:02.326562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4957bb-e745-40de-bf17-f28c22e97979 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:02.342992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4957bb-e745-40de-bf17-f28c22e97979 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:02.353819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e0c75-7369-434b-b5e7-79a68fcd23bb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:02.363750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e0c75-7369-434b-b5e7-79a68fcd23bb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:02.364352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e0c75-7369-434b-b5e7-79a68fcd23bb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:02.364525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e0c75-7369-434b-b5e7-79a68fcd23bb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:02.371326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e0c75-7369-434b-b5e7-79a68fcd23bb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:02.372424 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-b34e0c75-7369-434b-b5e7-79a68fcd23bb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:02.372424 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:02.372424 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:02.373661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e0c75-7369-434b-b5e7-79a68fcd23bb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:02.407307 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2377/4806] 10.222.0.23 () {58 vars in 1347 bytes} [Sat Aug 20 21:10:02 2022] DELETE /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:02.411783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0cf9a-9ff7-44b7-9f98-bfdc3ce2ba21 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:02.431842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0cf9a-9ff7-44b7-9f98-bfdc3ce2ba21 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:02.474404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e0c75-7369-434b-b5e7-79a68fcd23bb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:02.476247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b34e0c75-7369-434b-b5e7-79a68fcd23bb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:02.538129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f6ecd-45da-45a6-b99d-a885890b2f41 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:02.538961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f6ecd-45da-45a6-b99d-a885890b2f41 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:02.539780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f6ecd-45da-45a6-b99d-a885890b2f41 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:02.540151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f6ecd-45da-45a6-b99d-a885890b2f41 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:02.541098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f6ecd-45da-45a6-b99d-a885890b2f41 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Adding query filter params (scope.project.id=3e37435363bb4abc9199670f3780c04c, user.id=8401b16d587845afa1487456fee0bfaf) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:02.541502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f6ecd-45da-45a6-b99d-a885890b2f41 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:02.543793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f6ecd-45da-45a6-b99d-a885890b2f41 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:02.588148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e0c75-7369-434b-b5e7-79a68fcd23bb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:02.595848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0a43c-d77b-45b3-a30d-e53fbdcbcabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:02.596845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0a43c-d77b-45b3-a30d-e53fbdcbcabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:02.597183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0a43c-d77b-45b3-a30d-e53fbdcbcabf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:02.597579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0a43c-d77b-45b3-a30d-e53fbdcbcabf admin admin] PATH_INFO: `/v3/users/97a1575a72e74fa2a188ba9eef8ace8a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:02.600070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0a43c-d77b-45b3-a30d-e53fbdcbcabf admin admin] RBAC: Authorizing `identity:delete_user(user_id=97a1575a72e74fa2a188ba9eef8ace8a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:02.602442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:02.602876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0a43c-d77b-45b3-a30d-e53fbdcbcabf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:02.619225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:02.664174 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2430/4807] 10.222.0.23 () {58 vars in 1356 bytes} [Sat Aug 20 21:10:02 2022] GET /identity/v3/role_assignments?scope.project.id=3e37435363bb4abc9199670f3780c04c&user.id=8401b16d587845afa1487456fee0bfaf&effective => generated 577 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:02.665348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f6ecd-45da-45a6-b99d-a885890b2f41 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:02.687225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f6ecd-45da-45a6-b99d-a885890b2f41 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:02.701101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2121d-4a6a-44ae-88d1-43163db9114f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:02.702804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2121d-4a6a-44ae-88d1-43163db9114f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:02.703380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2121d-4a6a-44ae-88d1-43163db9114f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:02.705605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2121d-4a6a-44ae-88d1-43163db9114f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:02.712073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2121d-4a6a-44ae-88d1-43163db9114f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:02.713429 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-4ac2121d-4a6a-44ae-88d1-43163db9114f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:02.713429 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:02.713429 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:02.717355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2121d-4a6a-44ae-88d1-43163db9114f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-4ac2121d-4a6a-44ae-88d1-43163db9114f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] be50ad6b14b342a9b9347d2bf643b250 does not imply 7d0cc3f88e39496d87c9fed6401b9be5.: keystone.exception.ImpliedRoleNotFound: be50ad6b14b342a9b9347d2bf643b250 does not imply 7d0cc3f88e39496d87c9fed6401b9be5. Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = query.one() Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._iter().one() Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._only_one_row( Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exc.NoResultFound( Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:10:02.738519 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Aug 20 21:10:02.740101 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Aug 20 21:10:02.741539 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Aug 20 21:10:02.741539 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Aug 20 21:10:02.741539 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Aug 20 21:10:02.741539 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: be50ad6b14b342a9b9347d2bf643b250 does not imply 7d0cc3f88e39496d87c9fed6401b9be5. Aug 20 21:10:02.741539 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:10:02.743865 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2431/4808] 10.222.0.23 () {58 vars in 1347 bytes} [Sat Aug 20 21:10:02 2022] DELETE /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5 => generated 137 bytes in 78 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:10:02.749540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2121d-4a6a-44ae-88d1-43163db9114f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:02.767255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2121d-4a6a-44ae-88d1-43163db9114f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:02.774760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff1db3-0ba7-48ec-94e6-f1d7990ff848 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:02.775603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff1db3-0ba7-48ec-94e6-f1d7990ff848 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:02.775966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff1db3-0ba7-48ec-94e6-f1d7990ff848 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:02.776342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff1db3-0ba7-48ec-94e6-f1d7990ff848 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/be50ad6b14b342a9b9347d2bf643b250` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:02.779442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff1db3-0ba7-48ec-94e6-f1d7990ff848 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=3e37435363bb4abc9199670f3780c04c, user_id=8401b16d587845afa1487456fee0bfaf, role_id=be50ad6b14b342a9b9347d2bf643b250)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:02.782457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff1db3-0ba7-48ec-94e6-f1d7990ff848 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:02.786337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-79e0a43c-d77b-45b3-a30d-e53fbdcbcabf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97a1575a72e74fa2a188ba9eef8ace8a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:02.806226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1ff1db3-0ba7-48ec-94e6-f1d7990ff848 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role be50ad6b14b342a9b9347d2bf643b250 was removed from user 8401b16d587845afa1487456fee0bfaf on project 3e37435363bb4abc9199670f3780c04c. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:02.808846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1ff1db3-0ba7-48ec-94e6-f1d7990ff848 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:02.811345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1ff1db3-0ba7-48ec-94e6-f1d7990ff848 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:02.811507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-79e0a43c-d77b-45b3-a30d-e53fbdcbcabf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97a1575a72e74fa2a188ba9eef8ace8a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:02.842768 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2432/4809] 10.222.0.23 () {58 vars in 1467 bytes} [Sat Aug 20 21:10:02 2022] DELETE /identity/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/be50ad6b14b342a9b9347d2bf643b250 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:02.851020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff1db3-0ba7-48ec-94e6-f1d7990ff848 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:02.868777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-79e0a43c-d77b-45b3-a30d-e53fbdcbcabf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97a1575a72e74fa2a188ba9eef8ace8a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:02.868997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c1ff1db3-0ba7-48ec-94e6-f1d7990ff848 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:02.869728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-79e0a43c-d77b-45b3-a30d-e53fbdcbcabf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97a1575a72e74fa2a188ba9eef8ace8a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:02.870422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-79e0a43c-d77b-45b3-a30d-e53fbdcbcabf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97a1575a72e74fa2a188ba9eef8ace8a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:02.888860 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2378/4810] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:02 2022] DELETE /identity/v3/users/97a1575a72e74fa2a188ba9eef8ace8a => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:02.894435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576913f-ccf0-496f-8bd6-7c17725fc77f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:02.894999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576913f-ccf0-496f-8bd6-7c17725fc77f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:02.895717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576913f-ccf0-496f-8bd6-7c17725fc77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:02.940558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8576913f-ccf0-496f-8bd6-7c17725fc77f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:02.943352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8576913f-ccf0-496f-8bd6-7c17725fc77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:03.093492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff1db3-0ba7-48ec-94e6-f1d7990ff848 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:03.108643 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2379/4811] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:03.126502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576913f-ccf0-496f-8bd6-7c17725fc77f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:03.128191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8576913f-ccf0-496f-8bd6-7c17725fc77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:03.206598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b5732-60d1-4ceb-b65b-e34da6d20529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:03.207321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b5732-60d1-4ceb-b65b-e34da6d20529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:03.207620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b5732-60d1-4ceb-b65b-e34da6d20529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:03.207984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b5732-60d1-4ceb-b65b-e34da6d20529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:03.208677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b5732-60d1-4ceb-b65b-e34da6d20529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:03.210286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b5732-60d1-4ceb-b65b-e34da6d20529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:03.230439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576913f-ccf0-496f-8bd6-7c17725fc77f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:03.239321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf97cc-52b5-4132-9bfc-3e328c06ed6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:03.239979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf97cc-52b5-4132-9bfc-3e328c06ed6f service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:03.240444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf97cc-52b5-4132-9bfc-3e328c06ed6f service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:03.240786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf97cc-52b5-4132-9bfc-3e328c06ed6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:03.256024 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2433/4812] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:02 2022] POST /identity/v3/roles => generated 272 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:03.261455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf97cc-52b5-4132-9bfc-3e328c06ed6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:03.262620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b5732-60d1-4ceb-b65b-e34da6d20529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:03.264961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4f5b5732-60d1-4ceb-b65b-e34da6d20529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:03.266877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf97cc-52b5-4132-9bfc-3e328c06ed6f service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:03.283200 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2380/4813] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:10:03.299976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b5732-60d1-4ceb-b65b-e34da6d20529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:03.305493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364c39f-c606-454c-a1f4-dd27ca0f01ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:03.306426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364c39f-c606-454c-a1f4-dd27ca0f01ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:03.306913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364c39f-c606-454c-a1f4-dd27ca0f01ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:03.307461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364c39f-c606-454c-a1f4-dd27ca0f01ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:03.308269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364c39f-c606-454c-a1f4-dd27ca0f01ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Adding query filter params (domain_id=7742da72f13d416aa9ec51962a89d55f) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:03.308705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364c39f-c606-454c-a1f4-dd27ca0f01ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:03.309938 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-2364c39f-c606-454c-a1f4-dd27ca0f01ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domain_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:03.309938 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:03.309938 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:03.311110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364c39f-c606-454c-a1f4-dd27ca0f01ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:03.324703 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2434/4814] 10.222.0.23 () {58 vars in 1207 bytes} [Sat Aug 20 21:10:03 2022] GET /identity/v3/roles?domain_id=7742da72f13d416aa9ec51962a89d55f => generated 409 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:03.329094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf97cc-52b5-4132-9bfc-3e328c06ed6f service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:03.341966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf97cc-52b5-4132-9bfc-3e328c06ed6f service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:03.349016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0299d0c-9be9-4a90-9319-8b9a8d96014f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:03.349816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0299d0c-9be9-4a90-9319-8b9a8d96014f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:03.350194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0299d0c-9be9-4a90-9319-8b9a8d96014f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:03.350575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0299d0c-9be9-4a90-9319-8b9a8d96014f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/0029aa8cc5244bf6a82629d49ed67511` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:03.352047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0299d0c-9be9-4a90-9319-8b9a8d96014f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=0029aa8cc5244bf6a82629d49ed67511)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:03.353438 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-d0299d0c-9be9-4a90-9319-8b9a8d96014f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:03.353438 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:03.353438 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:03.354807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0299d0c-9be9-4a90-9319-8b9a8d96014f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:03.376585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:03.432821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d0299d0c-9be9-4a90-9319-8b9a8d96014f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role 0029aa8cc5244bf6a82629d49ed67511 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:03.433265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d0299d0c-9be9-4a90-9319-8b9a8d96014f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:03.434197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d0299d0c-9be9-4a90-9319-8b9a8d96014f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:03.439563 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2381/4815] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:03 2022] DELETE /identity/v3/roles/0029aa8cc5244bf6a82629d49ed67511 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:03.441254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364c39f-c606-454c-a1f4-dd27ca0f01ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:03.443381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2364c39f-c606-454c-a1f4-dd27ca0f01ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:03.470954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364c39f-c606-454c-a1f4-dd27ca0f01ea tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:03.560805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:03.583638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d750b2-5b0d-4987-aa26-afe7e0b7fa5d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:03.584292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d750b2-5b0d-4987-aa26-afe7e0b7fa5d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:03.584617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d750b2-5b0d-4987-aa26-afe7e0b7fa5d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:03.584881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d750b2-5b0d-4987-aa26-afe7e0b7fa5d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:03.585538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d750b2-5b0d-4987-aa26-afe7e0b7fa5d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Adding query filter params (domain_id=7742da72f13d416aa9ec51962a89d55f) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:03.585830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d750b2-5b0d-4987-aa26-afe7e0b7fa5d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:03.587437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d750b2-5b0d-4987-aa26-afe7e0b7fa5d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:03.594931 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2435/4816] 10.222.0.23 () {58 vars in 1207 bytes} [Sat Aug 20 21:10:03 2022] GET /identity/v3/roles?domain_id=7742da72f13d416aa9ec51962a89d55f => generated 148 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:03.599297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0299d0c-9be9-4a90-9319-8b9a8d96014f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:03.611322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0299d0c-9be9-4a90-9319-8b9a8d96014f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:03.619746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16720013-915c-4f56-b214-166da9efc266 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:03.620789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16720013-915c-4f56-b214-166da9efc266 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:03.620971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16720013-915c-4f56-b214-166da9efc266 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:03.621341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16720013-915c-4f56-b214-166da9efc266 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/0029aa8cc5244bf6a82629d49ed67511` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:03.635724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16720013-915c-4f56-b214-166da9efc266 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-16720013-915c-4f56-b214-166da9efc266 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Could not find role: 0029aa8cc5244bf6a82629d49ed67511.: keystone.exception.RoleNotFound: Could not find role: 0029aa8cc5244bf6a82629d49ed67511. Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 165, in delete Aug 20 21:10:03.641697 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_role') Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:10:03.642990 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:10:03.644316 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:10:03.644316 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:10:03.644316 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:10:03.644316 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:10:03.644316 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 20 21:10:03.644316 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 20 21:10:03.644316 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 20 21:10:03.644316 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 20 21:10:03.644316 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 20 21:10:03.644316 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 20 21:10:03.644316 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 0029aa8cc5244bf6a82629d49ed67511. Aug 20 21:10:03.644316 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:10:03.649143 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2382/4817] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:03 2022] DELETE /identity/v3/roles/0029aa8cc5244bf6a82629d49ed67511 => generated 110 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:10:03.657180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d750b2-5b0d-4987-aa26-afe7e0b7fa5d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:03.689501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d750b2-5b0d-4987-aa26-afe7e0b7fa5d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:03.704082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14461719-8dbe-48cb-a7b5-42a94743e4f4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:03.704837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14461719-8dbe-48cb-a7b5-42a94743e4f4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:03.705116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14461719-8dbe-48cb-a7b5-42a94743e4f4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:03.705384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14461719-8dbe-48cb-a7b5-42a94743e4f4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains/7742da72f13d416aa9ec51962a89d55f/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:03.721038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14461719-8dbe-48cb-a7b5-42a94743e4f4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=7742da72f13d416aa9ec51962a89d55f, group_id=913a950456c24a9fa04fb93a0de83b04, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:03.725163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14461719-8dbe-48cb-a7b5-42a94743e4f4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:03.762143 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2436/4818] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:03 2022] PUT /identity/v3/domains/7742da72f13d416aa9ec51962a89d55f/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:03.766772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16720013-915c-4f56-b214-166da9efc266 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:03.780833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16720013-915c-4f56-b214-166da9efc266 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:03.869138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14461719-8dbe-48cb-a7b5-42a94743e4f4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:03.870935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-14461719-8dbe-48cb-a7b5-42a94743e4f4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:03.987922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce641dd-fd29-4b3e-9a13-9e2550cd58b5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:03.989241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce641dd-fd29-4b3e-9a13-9e2550cd58b5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:03.989703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce641dd-fd29-4b3e-9a13-9e2550cd58b5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:03.990119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce641dd-fd29-4b3e-9a13-9e2550cd58b5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains/7742da72f13d416aa9ec51962a89d55f/groups/913a950456c24a9fa04fb93a0de83b04/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:04.006489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce641dd-fd29-4b3e-9a13-9e2550cd58b5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=7742da72f13d416aa9ec51962a89d55f, group_id=913a950456c24a9fa04fb93a0de83b04)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:04.006489 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-7ce641dd-fd29-4b3e-9a13-9e2550cd58b5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:04.006489 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:04.006489 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:04.006489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce641dd-fd29-4b3e-9a13-9e2550cd58b5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:04.029564 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2383/4819] 10.222.0.23 () {58 vars in 1365 bytes} [Sat Aug 20 21:10:03 2022] GET /identity/v3/domains/7742da72f13d416aa9ec51962a89d55f/groups/913a950456c24a9fa04fb93a0de83b04/roles => generated 410 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:04.033486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce641dd-fd29-4b3e-9a13-9e2550cd58b5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:04.046265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce641dd-fd29-4b3e-9a13-9e2550cd58b5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:04.055518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1423ba3-4db3-496c-b459-8c5c5c3d9138 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:04.056336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1423ba3-4db3-496c-b459-8c5c5c3d9138 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:04.056717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1423ba3-4db3-496c-b459-8c5c5c3d9138 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:04.057079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1423ba3-4db3-496c-b459-8c5c5c3d9138 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains/7742da72f13d416aa9ec51962a89d55f/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:04.062929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1423ba3-4db3-496c-b459-8c5c5c3d9138 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=7742da72f13d416aa9ec51962a89d55f, group_id=913a950456c24a9fa04fb93a0de83b04, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:04.064064 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-a1423ba3-4db3-496c-b459-8c5c5c3d9138 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:04.064064 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:04.064064 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:04.064906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14461719-8dbe-48cb-a7b5-42a94743e4f4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:04.067508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1423ba3-4db3-496c-b459-8c5c5c3d9138 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:04.071743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0ac04-602f-426f-ad81-5904ed4b9a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:04.073444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0ac04-602f-426f-ad81-5904ed4b9a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:04.074164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0ac04-602f-426f-ad81-5904ed4b9a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:04.074785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0ac04-602f-426f-ad81-5904ed4b9a77 admin admin] PATH_INFO: `/v3/projects/18895d700e874d019b1f7636f2ed5e40` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:04.078704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0ac04-602f-426f-ad81-5904ed4b9a77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18895d700e874d019b1f7636f2ed5e40)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:04.083569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0ac04-602f-426f-ad81-5904ed4b9a77 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:04.084844 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2384/4820] 10.222.0.23 () {58 vars in 1465 bytes} [Sat Aug 20 21:10:04 2022] HEAD /identity/v3/domains/7742da72f13d416aa9ec51962a89d55f/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:04.089715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1423ba3-4db3-496c-b459-8c5c5c3d9138 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:04.111002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1423ba3-4db3-496c-b459-8c5c5c3d9138 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:04.132783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7609b-53fb-48e7-adb4-e262fa5a3cb0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:04.133595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7609b-53fb-48e7-adb4-e262fa5a3cb0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:04.134026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7609b-53fb-48e7-adb4-e262fa5a3cb0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:04.134416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7609b-53fb-48e7-adb4-e262fa5a3cb0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains/7742da72f13d416aa9ec51962a89d55f/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:04.138035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7609b-53fb-48e7-adb4-e262fa5a3cb0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=7742da72f13d416aa9ec51962a89d55f, group_id=913a950456c24a9fa04fb93a0de83b04, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:04.140739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7609b-53fb-48e7-adb4-e262fa5a3cb0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:04.150250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-09a7609b-53fb-48e7-adb4-e262fa5a3cb0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role bd889ce606ae4308bb72b30879315f83 was removed from group 913a950456c24a9fa04fb93a0de83b04 on domain 7742da72f13d416aa9ec51962a89d55f. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:04.150624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-09a7609b-53fb-48e7-adb4-e262fa5a3cb0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:04.151747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-09a7609b-53fb-48e7-adb4-e262fa5a3cb0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:04.177486 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2385/4821] 10.222.0.23 () {58 vars in 1467 bytes} [Sat Aug 20 21:10:04 2022] DELETE /identity/v3/domains/7742da72f13d416aa9ec51962a89d55f/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:04.183823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ba0ac04-602f-426f-ad81-5904ed4b9a77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18895d700e874d019b1f7636f2ed5e40', 'request_id': 'req-5ba0ac04-602f-426f-ad81-5904ed4b9a77'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:04.184353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7609b-53fb-48e7-adb4-e262fa5a3cb0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:04.187411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-09a7609b-53fb-48e7-adb4-e262fa5a3cb0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:04.207310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ba0ac04-602f-426f-ad81-5904ed4b9a77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18895d700e874d019b1f7636f2ed5e40', 'request_id': 'req-5ba0ac04-602f-426f-ad81-5904ed4b9a77'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:04.245302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ba0ac04-602f-426f-ad81-5904ed4b9a77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18895d700e874d019b1f7636f2ed5e40', 'request_id': 'req-5ba0ac04-602f-426f-ad81-5904ed4b9a77'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:04.271791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ba0ac04-602f-426f-ad81-5904ed4b9a77 admin admin] The token cache is being invalidate because project 18895d700e874d019b1f7636f2ed5e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:04.272225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ba0ac04-602f-426f-ad81-5904ed4b9a77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:04.273628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ba0ac04-602f-426f-ad81-5904ed4b9a77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:04.273930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7609b-53fb-48e7-adb4-e262fa5a3cb0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:04.275493 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2437/4822] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:03 2022] DELETE /identity/v3/projects/18895d700e874d019b1f7636f2ed5e40 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:04.280931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f7a02-a645-45d6-93f3-c233a09a2173 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:04.281365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f7a02-a645-45d6-93f3-c233a09a2173 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:04.281597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f7a02-a645-45d6-93f3-c233a09a2173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:04.285705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b67264-e4c9-4157-93bc-079292c277d3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:04.289070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b67264-e4c9-4157-93bc-079292c277d3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:04.292569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b67264-e4c9-4157-93bc-079292c277d3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:04.293629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b67264-e4c9-4157-93bc-079292c277d3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects/3e37435363bb4abc9199670f3780c04c/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:04.297317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b67264-e4c9-4157-93bc-079292c277d3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_grant(project_id=3e37435363bb4abc9199670f3780c04c, group_id=913a950456c24a9fa04fb93a0de83b04, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:04.300144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b67264-e4c9-4157-93bc-079292c277d3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:04.313500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a88f7a02-a645-45d6-93f3-c233a09a2173 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:04.314766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a88f7a02-a645-45d6-93f3-c233a09a2173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:04.326680 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2386/4823] 10.222.0.23 () {60 vars in 1486 bytes} [Sat Aug 20 21:10:04 2022] PUT /identity/v3/projects/3e37435363bb4abc9199670f3780c04c/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:04.330595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b67264-e4c9-4157-93bc-079292c277d3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:04.333665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-57b67264-e4c9-4157-93bc-079292c277d3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:04.380287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:04.389297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:04.416794 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2438/4824] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:04.419659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98a34c-ff9b-4cfc-99de-385f7e359c76 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:04.419917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98a34c-ff9b-4cfc-99de-385f7e359c76 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:04.420251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98a34c-ff9b-4cfc-99de-385f7e359c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:04.457899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dd98a34c-ff9b-4cfc-99de-385f7e359c76 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:04.460444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dd98a34c-ff9b-4cfc-99de-385f7e359c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:04.471841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b67264-e4c9-4157-93bc-079292c277d3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:04.476764 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2439/4825] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:04.479199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98a34c-ff9b-4cfc-99de-385f7e359c76 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:04.493449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98a34c-ff9b-4cfc-99de-385f7e359c76 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:04.498864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f174bbb-82bc-464f-afc8-73d6832521f9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:04.500108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f174bbb-82bc-464f-afc8-73d6832521f9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:04.500590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f174bbb-82bc-464f-afc8-73d6832521f9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:04.501046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca1dce-ab0c-48ee-b0c1-c548d9e252ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:04.501212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f174bbb-82bc-464f-afc8-73d6832521f9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects/3e37435363bb4abc9199670f3780c04c/groups/913a950456c24a9fa04fb93a0de83b04/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:04.501841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca1dce-ab0c-48ee-b0c1-c548d9e252ce admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:04.502208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca1dce-ab0c-48ee-b0c1-c548d9e252ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:04.502598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca1dce-ab0c-48ee-b0c1-c548d9e252ce admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:04.503334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca1dce-ab0c-48ee-b0c1-c548d9e252ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:04.503724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca1dce-ab0c-48ee-b0c1-c548d9e252ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:04.505076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f174bbb-82bc-464f-afc8-73d6832521f9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_grants(project_id=3e37435363bb4abc9199670f3780c04c, group_id=913a950456c24a9fa04fb93a0de83b04)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:04.506231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca1dce-ab0c-48ee-b0c1-c548d9e252ce admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:04.508316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f174bbb-82bc-464f-afc8-73d6832521f9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:04.525671 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2440/4826] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:10:04.530048 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2387/4827] 10.222.0.23 () {58 vars in 1368 bytes} [Sat Aug 20 21:10:04 2022] GET /identity/v3/projects/3e37435363bb4abc9199670f3780c04c/groups/913a950456c24a9fa04fb93a0de83b04/roles => generated 411 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:04.533576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca1dce-ab0c-48ee-b0c1-c548d9e252ce admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:04.534775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f174bbb-82bc-464f-afc8-73d6832521f9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:04.557244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f174bbb-82bc-464f-afc8-73d6832521f9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:04.564035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca1dce-ab0c-48ee-b0c1-c548d9e252ce admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:04.568326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4908d9d-47a5-4674-bf91-921a9af492c1 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:04.569887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4908d9d-47a5-4674-bf91-921a9af492c1 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:04.571045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4908d9d-47a5-4674-bf91-921a9af492c1 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:04.571480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4908d9d-47a5-4674-bf91-921a9af492c1 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/groups/913a950456c24a9fa04fb93a0de83b04/users/8401b16d587845afa1487456fee0bfaf` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:04.573942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4908d9d-47a5-4674-bf91-921a9af492c1 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=913a950456c24a9fa04fb93a0de83b04, user_id=8401b16d587845afa1487456fee0bfaf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:04.576327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4908d9d-47a5-4674-bf91-921a9af492c1 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:04.581161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576efd0b-78f6-4e49-af00-12a8cded92ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:04.582274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576efd0b-78f6-4e49-af00-12a8cded92ee admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:04.582661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576efd0b-78f6-4e49-af00-12a8cded92ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:04.583352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576efd0b-78f6-4e49-af00-12a8cded92ee admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:04.586171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576efd0b-78f6-4e49-af00-12a8cded92ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:04.596629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576efd0b-78f6-4e49-af00-12a8cded92ee admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:04.621805 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2388/4828] 10.222.0.23 () {60 vars in 1360 bytes} [Sat Aug 20 21:10:04 2022] PUT /identity/v3/groups/913a950456c24a9fa04fb93a0de83b04/users/8401b16d587845afa1487456fee0bfaf => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:04.621805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4908d9d-47a5-4674-bf91-921a9af492c1 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:04.624043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b4908d9d-47a5-4674-bf91-921a9af492c1 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:04.646795 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2441/4829] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:04 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:04.650164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47aaf9-cdef-45f7-a7fe-35c95861fc7b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:04.650603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47aaf9-cdef-45f7-a7fe-35c95861fc7b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:04.650944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47aaf9-cdef-45f7-a7fe-35c95861fc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:04.715951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4908d9d-47a5-4674-bf91-921a9af492c1 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:04.725384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5c47aaf9-cdef-45f7-a7fe-35c95861fc7b None None] MFA Rules not processed for user `8401b16d587845afa1487456fee0bfaf`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:04.726641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5c47aaf9-cdef-45f7-a7fe-35c95861fc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:04.796243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70711adcb5584d4d9cd46b9c0acb9b7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6c564242cec4c29a0f6f980d0a83a3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:04.797231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:04.797508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:04.797782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] PATH_INFO: `/v3/projects/e1576072b2c844ff9840ed393451e4d9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:04.799283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e1576072b2c844ff9840ed393451e4d9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:04.803087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:04.866874 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2442/4830] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:04 2022] POST /identity/v3/auth/tokens => generated 3265 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:04.866874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47aaf9-cdef-45f7-a7fe-35c95861fc7b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:04.881950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:04.891567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1576072b2c844ff9840ed393451e4d9', 'request_id': 'req-2f93b21f-01c9-4eed-9046-29794561b89d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:04.904511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1576072b2c844ff9840ed393451e4d9', 'request_id': 'req-2f93b21f-01c9-4eed-9046-29794561b89d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:04.921060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1576072b2c844ff9840ed393451e4d9', 'request_id': 'req-2f93b21f-01c9-4eed-9046-29794561b89d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:04.946915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] The token cache is being invalidate because project e1576072b2c844ff9840ed393451e4d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:04.947278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:04.948900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:04.950871 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2389/4831] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:04 2022] DELETE /identity/v3/projects/e1576072b2c844ff9840ed393451e4d9 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:04.951748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:04.954772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:04.981862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47aaf9-cdef-45f7-a7fe-35c95861fc7b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:05.011322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:05.069044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad162d-0a10-43d8-a2a6-aecd2307453c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:05.070034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad162d-0a10-43d8-a2a6-aecd2307453c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.070839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad162d-0a10-43d8-a2a6-aecd2307453c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.071275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad162d-0a10-43d8-a2a6-aecd2307453c admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.072049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad162d-0a10-43d8-a2a6-aecd2307453c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:05.072821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93b21f-01c9-4eed-9046-29794561b89d tempest-QuotasNegativeTest-1817535116 tempest-QuotasNegativeTest-1817535116-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:05.074552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad162d-0a10-43d8-a2a6-aecd2307453c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:05.084514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4180d06-a96a-4091-890d-3904049eb2e8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:05.084514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4180d06-a96a-4091-890d-3904049eb2e8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.084514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4180d06-a96a-4091-890d-3904049eb2e8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.084959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4180d06-a96a-4091-890d-3904049eb2e8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects/3e37435363bb4abc9199670f3780c04c/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.090187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4180d06-a96a-4091-890d-3904049eb2e8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:check_grant(project_id=3e37435363bb4abc9199670f3780c04c, group_id=913a950456c24a9fa04fb93a0de83b04, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:05.090187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4180d06-a96a-4091-890d-3904049eb2e8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:05.096193 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2390/4832] 10.222.0.23 () {58 vars in 1468 bytes} [Sat Aug 20 21:10:04 2022] HEAD /identity/v3/projects/3e37435363bb4abc9199670f3780c04c/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:05.097121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4180d06-a96a-4091-890d-3904049eb2e8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:05.098968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a4180d06-a96a-4091-890d-3904049eb2e8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:05.115917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4180d06-a96a-4091-890d-3904049eb2e8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:05.123834 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2443/4833] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:04 2022] POST /identity/v3/users => generated 291 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:05.123834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad162d-0a10-43d8-a2a6-aecd2307453c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:05.131631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:05.132277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.132580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.132867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] PATH_INFO: `/v3/users/53e5c96846c3473c91d0893de46f5d59` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.134045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e5c96846c3473c91d0893de46f5d59)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:05.134513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad162d-0a10-43d8-a2a6-aecd2307453c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:05.136318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:05.142133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248afe4-4fa8-46e5-8c8e-b92a52f390b3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:05.142880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248afe4-4fa8-46e5-8c8e-b92a52f390b3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.143993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248afe4-4fa8-46e5-8c8e-b92a52f390b3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.143993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248afe4-4fa8-46e5-8c8e-b92a52f390b3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects/3e37435363bb4abc9199670f3780c04c/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.148603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248afe4-4fa8-46e5-8c8e-b92a52f390b3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=3e37435363bb4abc9199670f3780c04c, group_id=913a950456c24a9fa04fb93a0de83b04, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:05.152541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248afe4-4fa8-46e5-8c8e-b92a52f390b3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:05.159202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c248afe4-4fa8-46e5-8c8e-b92a52f390b3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role bd889ce606ae4308bb72b30879315f83 was removed from group 913a950456c24a9fa04fb93a0de83b04 on project 3e37435363bb4abc9199670f3780c04c. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:05.159646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c248afe4-4fa8-46e5-8c8e-b92a52f390b3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:05.160773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c248afe4-4fa8-46e5-8c8e-b92a52f390b3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:05.186610 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2444/4834] 10.222.0.23 () {58 vars in 1470 bytes} [Sat Aug 20 21:10:05 2022] DELETE /identity/v3/projects/3e37435363bb4abc9199670f3780c04c/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:05.188769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248afe4-4fa8-46e5-8c8e-b92a52f390b3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:05.190328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c248afe4-4fa8-46e5-8c8e-b92a52f390b3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:05.219974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e5c96846c3473c91d0893de46f5d59'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:05.232274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e5c96846c3473c91d0893de46f5d59'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:05.256743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e5c96846c3473c91d0893de46f5d59'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:05.258022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e5c96846c3473c91d0893de46f5d59'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:05.259240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e5c96846c3473c91d0893de46f5d59'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:05.279038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248afe4-4fa8-46e5-8c8e-b92a52f390b3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:05.280305 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2391/4835] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:05 2022] DELETE /identity/v3/users/53e5c96846c3473c91d0893de46f5d59 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:05.281177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:05.282726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:05.343421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:05.367650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc07758-e884-4bd0-8c9f-b5d484a609a0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:05.373205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148223c5-0e08-4300-8b65-b359ebe9b3ee tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:05.374064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148223c5-0e08-4300-8b65-b359ebe9b3ee tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.374553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148223c5-0e08-4300-8b65-b359ebe9b3ee tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.374897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148223c5-0e08-4300-8b65-b359ebe9b3ee tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/groups/913a950456c24a9fa04fb93a0de83b04/users/8401b16d587845afa1487456fee0bfaf` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.376773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148223c5-0e08-4300-8b65-b359ebe9b3ee tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=913a950456c24a9fa04fb93a0de83b04, user_id=8401b16d587845afa1487456fee0bfaf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:05.378024 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-148223c5-0e08-4300-8b65-b359ebe9b3ee tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:05.378024 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:05.378024 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:05.380045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148223c5-0e08-4300-8b65-b359ebe9b3ee tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:05.386483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be120583-77dd-4a9d-ab88-a59cde195e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:05.387202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be120583-77dd-4a9d-ab88-a59cde195e83 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.387568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be120583-77dd-4a9d-ab88-a59cde195e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.387936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be120583-77dd-4a9d-ab88-a59cde195e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.388680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be120583-77dd-4a9d-ab88-a59cde195e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:05.390643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be120583-77dd-4a9d-ab88-a59cde195e83 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:05.393791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-148223c5-0e08-4300-8b65-b359ebe9b3ee tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '8401b16d587845afa1487456fee0bfaf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:05.407787 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2445/4836] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:05 2022] GET /identity/v3/roles => generated 2324 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:05.409255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea9ac6-f926-445e-9d10-0c4354bb81e4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.409530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea9ac6-f926-445e-9d10-0c4354bb81e4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.409793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea9ac6-f926-445e-9d10-0c4354bb81e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.418360 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2392/4837] 10.222.0.23 () {58 vars in 1344 bytes} [Sat Aug 20 21:10:05 2022] DELETE /identity/v3/groups/913a950456c24a9fa04fb93a0de83b04/users/8401b16d587845afa1487456fee0bfaf => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:05.421259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148223c5-0e08-4300-8b65-b359ebe9b3ee tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:05.423472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-148223c5-0e08-4300-8b65-b359ebe9b3ee tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:05.443720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b9ea9ac6-f926-445e-9d10-0c4354bb81e4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:05.445600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b9ea9ac6-f926-445e-9d10-0c4354bb81e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:05.488597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:05.529386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148223c5-0e08-4300-8b65-b359ebe9b3ee tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:05.531256 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2446/4838] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:05.532193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea9ac6-f926-445e-9d10-0c4354bb81e4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:05.541401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3f966-612b-48dd-8cd0-e209b613217a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:05.542293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3f966-612b-48dd-8cd0-e209b613217a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.542667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3f966-612b-48dd-8cd0-e209b613217a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.543151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3f966-612b-48dd-8cd0-e209b613217a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.543903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3f966-612b-48dd-8cd0-e209b613217a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:05.546052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3f966-612b-48dd-8cd0-e209b613217a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:05.566946 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2393/4839] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:05 2022] GET /identity/v3/roles => generated 2324 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:05.566946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3f966-612b-48dd-8cd0-e209b613217a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:05.566946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b9f3f966-612b-48dd-8cd0-e209b613217a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:05.621059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea9ac6-f926-445e-9d10-0c4354bb81e4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:05.632271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaeb71b-8933-49b8-bac0-d3abae4735ec tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:05.633353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaeb71b-8933-49b8-bac0-d3abae4735ec tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.633901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaeb71b-8933-49b8-bac0-d3abae4735ec tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.634265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaeb71b-8933-49b8-bac0-d3abae4735ec tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/system/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.639695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaeb71b-8933-49b8-bac0-d3abae4735ec tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_system_grant_for_group(group_id=913a950456c24a9fa04fb93a0de83b04, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:05.640724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3f966-612b-48dd-8cd0-e209b613217a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:05.640979 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-ccaeb71b-8933-49b8-bac0-d3abae4735ec tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:05.640979 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:05.640979 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:05.643134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaeb71b-8933-49b8-bac0-d3abae4735ec tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:05.644403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0237b05-c0b1-4b72-832c-44fded66a730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:05.645127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0237b05-c0b1-4b72-832c-44fded66a730 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.645398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0237b05-c0b1-4b72-832c-44fded66a730 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.645695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0237b05-c0b1-4b72-832c-44fded66a730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.658401 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2447/4840] 10.222.0.23 () {60 vars in 1381 bytes} [Sat Aug 20 21:10:05 2022] PUT /identity/v3/system/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:05.658587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaeb71b-8933-49b8-bac0-d3abae4735ec tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:05.668235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0237b05-c0b1-4b72-832c-44fded66a730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:05.669946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0237b05-c0b1-4b72-832c-44fded66a730 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:05.673807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaeb71b-8933-49b8-bac0-d3abae4735ec tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:05.680904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311ecd5-95ba-4fb2-a747-e1fc1e994fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:05.681758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311ecd5-95ba-4fb2-a747-e1fc1e994fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.682095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311ecd5-95ba-4fb2-a747-e1fc1e994fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.682440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311ecd5-95ba-4fb2-a747-e1fc1e994fa4 admin admin] PATH_INFO: `/v3/projects/28e9c7707d7645d5b191af40f4c6da31/users/1cf36cef64b14630850a3fc3d2974e79/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.685553 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2394/4841] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:05.686534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0237b05-c0b1-4b72-832c-44fded66a730 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:05.700498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0237b05-c0b1-4b72-832c-44fded66a730 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:05.704189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311ecd5-95ba-4fb2-a747-e1fc1e994fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e9c7707d7645d5b191af40f4c6da31, user_id=1cf36cef64b14630850a3fc3d2974e79, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:05.708053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311ecd5-95ba-4fb2-a747-e1fc1e994fa4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:05.717351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4ab8c-d297-4fb5-bc98-3617a859a8fb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:05.718131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4ab8c-d297-4fb5-bc98-3617a859a8fb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.718386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4ab8c-d297-4fb5-bc98-3617a859a8fb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.718695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4ab8c-d297-4fb5-bc98-3617a859a8fb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/system/groups/913a950456c24a9fa04fb93a0de83b04/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.719944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4ab8c-d297-4fb5-bc98-3617a859a8fb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_system_grants_for_group(group_id=913a950456c24a9fa04fb93a0de83b04)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:05.721020 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-0da4ab8c-d297-4fb5-bc98-3617a859a8fb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:05.721020 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:05.721020 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:05.722013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4ab8c-d297-4fb5-bc98-3617a859a8fb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:05.733474 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2448/4842] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:05 2022] PUT /identity/v3/projects/28e9c7707d7645d5b191af40f4c6da31/users/1cf36cef64b14630850a3fc3d2974e79/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:05.738895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ec02a-84bb-478f-9037-551bae6cfd56 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.739406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ec02a-84bb-478f-9037-551bae6cfd56 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.739843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ec02a-84bb-478f-9037-551bae6cfd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.743259 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2395/4843] 10.222.0.23 () {58 vars in 1263 bytes} [Sat Aug 20 21:10:05 2022] GET /identity/v3/system/groups/913a950456c24a9fa04fb93a0de83b04/roles => generated 376 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:05.744399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4ab8c-d297-4fb5-bc98-3617a859a8fb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:05.769830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4ab8c-d297-4fb5-bc98-3617a859a8fb tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:05.773065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-594ec02a-84bb-478f-9037-551bae6cfd56 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:05.774285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-594ec02a-84bb-478f-9037-551bae6cfd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:05.865204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:05.881767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32196494-23c0-4664-aabe-0ef702ddf949 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:05.882875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32196494-23c0-4664-aabe-0ef702ddf949 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.883243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32196494-23c0-4664-aabe-0ef702ddf949 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.883643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32196494-23c0-4664-aabe-0ef702ddf949 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/system/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.885721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32196494-23c0-4664-aabe-0ef702ddf949 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:check_system_grant_for_group(group_id=913a950456c24a9fa04fb93a0de83b04, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:05.885917 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2449/4844] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:05.886854 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-32196494-23c0-4664-aabe-0ef702ddf949 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:05.886854 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:05.886854 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:05.888052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32196494-23c0-4664-aabe-0ef702ddf949 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:05.897439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ec02a-84bb-478f-9037-551bae6cfd56 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:05.900262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-594ec02a-84bb-478f-9037-551bae6cfd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:05.933340 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2396/4845] 10.222.0.23 () {58 vars in 1363 bytes} [Sat Aug 20 21:10:05 2022] HEAD /identity/v3/system/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:05.941381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32196494-23c0-4664-aabe-0ef702ddf949 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:05.966109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32196494-23c0-4664-aabe-0ef702ddf949 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:05.979833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604d663-c356-4674-b062-83cce552964f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:05.980839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604d663-c356-4674-b062-83cce552964f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:05.981289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604d663-c356-4674-b062-83cce552964f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:05.981743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604d663-c356-4674-b062-83cce552964f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/system/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:05.983706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604d663-c356-4674-b062-83cce552964f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_group(group_id=913a950456c24a9fa04fb93a0de83b04, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:05.986829 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-2604d663-c356-4674-b062-83cce552964f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:05.986829 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:05.986829 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:05.989284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604d663-c356-4674-b062-83cce552964f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:06.009947 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2397/4846] 10.222.0.23 () {58 vars in 1365 bytes} [Sat Aug 20 21:10:05 2022] DELETE /identity/v3/system/groups/913a950456c24a9fa04fb93a0de83b04/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:06.015170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604d663-c356-4674-b062-83cce552964f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:06.034816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604d663-c356-4674-b062-83cce552964f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:06.036001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ec02a-84bb-478f-9037-551bae6cfd56 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:06.049408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b2e25-f2ec-4e68-9003-e0bff22bed19 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:06.049408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b2e25-f2ec-4e68-9003-e0bff22bed19 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:06.049408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b2e25-f2ec-4e68-9003-e0bff22bed19 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:06.049769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b2e25-f2ec-4e68-9003-e0bff22bed19 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains/7742da72f13d416aa9ec51962a89d55f/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:06.049769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b2e25-f2ec-4e68-9003-e0bff22bed19 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=7742da72f13d416aa9ec51962a89d55f, user_id=8401b16d587845afa1487456fee0bfaf, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:06.049769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b2e25-f2ec-4e68-9003-e0bff22bed19 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:06.053595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bd9be-78da-4685-bcbe-3b4d031f417d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:06.053595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bd9be-78da-4685-bcbe-3b4d031f417d service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:06.053595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bd9be-78da-4685-bcbe-3b4d031f417d service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:06.053595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bd9be-78da-4685-bcbe-3b4d031f417d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:06.061108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:06.066145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6bd9be-78da-4685-bcbe-3b4d031f417d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:06.068702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6bd9be-78da-4685-bcbe-3b4d031f417d service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:06.078902 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2398/4847] 10.222.0.23 () {60 vars in 1480 bytes} [Sat Aug 20 21:10:06 2022] PUT /identity/v3/domains/7742da72f13d416aa9ec51962a89d55f/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:06.080416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b2e25-f2ec-4e68-9003-e0bff22bed19 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:06.086746 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2450/4848] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:10:06.213464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b2e25-f2ec-4e68-9003-e0bff22bed19 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:06.218927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dddea4-ff8a-4fe4-bbd5-19b529260bb7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:06.219595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dddea4-ff8a-4fe4-bbd5-19b529260bb7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:06.219932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dddea4-ff8a-4fe4-bbd5-19b529260bb7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:06.220264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dddea4-ff8a-4fe4-bbd5-19b529260bb7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains/7742da72f13d416aa9ec51962a89d55f/users/8401b16d587845afa1487456fee0bfaf/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:06.222155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dddea4-ff8a-4fe4-bbd5-19b529260bb7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=7742da72f13d416aa9ec51962a89d55f, user_id=8401b16d587845afa1487456fee0bfaf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:06.230877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dddea4-ff8a-4fe4-bbd5-19b529260bb7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:06.231444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bd9be-78da-4685-bcbe-3b4d031f417d service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:06.231444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4a6bd9be-78da-4685-bcbe-3b4d031f417d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:06.254683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bd9be-78da-4685-bcbe-3b4d031f417d service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:06.254980 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2399/4849] 10.222.0.23 () {58 vars in 1362 bytes} [Sat Aug 20 21:10:06 2022] GET /identity/v3/domains/7742da72f13d416aa9ec51962a89d55f/users/8401b16d587845afa1487456fee0bfaf/roles => generated 409 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:06.256294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dddea4-ff8a-4fe4-bbd5-19b529260bb7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:06.278435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dddea4-ff8a-4fe4-bbd5-19b529260bb7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:06.284703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e1196-33aa-4e87-a51d-12cfbcd66bed tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:06.285991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e1196-33aa-4e87-a51d-12cfbcd66bed tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:06.298969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e1196-33aa-4e87-a51d-12cfbcd66bed tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:06.299253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e1196-33aa-4e87-a51d-12cfbcd66bed tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains/7742da72f13d416aa9ec51962a89d55f/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:06.303605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e1196-33aa-4e87-a51d-12cfbcd66bed tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=7742da72f13d416aa9ec51962a89d55f, user_id=8401b16d587845afa1487456fee0bfaf, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:06.307579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e1196-33aa-4e87-a51d-12cfbcd66bed tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:06.320107 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2400/4850] 10.222.0.23 () {58 vars in 1462 bytes} [Sat Aug 20 21:10:06 2022] HEAD /identity/v3/domains/7742da72f13d416aa9ec51962a89d55f/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:06.325374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e1196-33aa-4e87-a51d-12cfbcd66bed tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:06.337723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e1196-33aa-4e87-a51d-12cfbcd66bed tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:06.343596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d40aba-dc26-4495-a3cc-c7ba76f7755d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:06.344676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d40aba-dc26-4495-a3cc-c7ba76f7755d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:06.345019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d40aba-dc26-4495-a3cc-c7ba76f7755d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:06.345357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d40aba-dc26-4495-a3cc-c7ba76f7755d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains/7742da72f13d416aa9ec51962a89d55f/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:06.368463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d40aba-dc26-4495-a3cc-c7ba76f7755d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=7742da72f13d416aa9ec51962a89d55f, user_id=8401b16d587845afa1487456fee0bfaf, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:06.371882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d40aba-dc26-4495-a3cc-c7ba76f7755d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:06.379656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-54d40aba-dc26-4495-a3cc-c7ba76f7755d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role bd889ce606ae4308bb72b30879315f83 was removed from user 8401b16d587845afa1487456fee0bfaf on domain 7742da72f13d416aa9ec51962a89d55f. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:06.380030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-54d40aba-dc26-4495-a3cc-c7ba76f7755d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:06.381034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-54d40aba-dc26-4495-a3cc-c7ba76f7755d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:06.432483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:06.432483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:06.432483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:06.432483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] PATH_INFO: `/v3/projects/46d6021c84034c16aed842b25a8b54c9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:06.433109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] RBAC: Authorizing `identity:delete_project(project_id=46d6021c84034c16aed842b25a8b54c9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:06.433109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:06.433288 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2401/4851] 10.222.0.23 () {58 vars in 1464 bytes} [Sat Aug 20 21:10:06 2022] DELETE /identity/v3/domains/7742da72f13d416aa9ec51962a89d55f/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:06.439934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d40aba-dc26-4495-a3cc-c7ba76f7755d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:06.442690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-54d40aba-dc26-4495-a3cc-c7ba76f7755d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:06.539211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46d6021c84034c16aed842b25a8b54c9', 'request_id': 'req-b6f8e505-8279-4f8a-9b3e-bfb414babeba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:06.554696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d40aba-dc26-4495-a3cc-c7ba76f7755d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:06.556056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46d6021c84034c16aed842b25a8b54c9', 'request_id': 'req-b6f8e505-8279-4f8a-9b3e-bfb414babeba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:06.563125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46d6021c84034c16aed842b25a8b54c9', 'request_id': 'req-b6f8e505-8279-4f8a-9b3e-bfb414babeba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:06.602910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] The token cache is being invalidate because project 46d6021c84034c16aed842b25a8b54c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:06.602910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:06.605177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:06.612785 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2451/4852] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:06 2022] DELETE /identity/v3/projects/46d6021c84034c16aed842b25a8b54c9 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:06.615309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:06.616651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:06.620958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:06.640564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d24751-f30c-4013-8d89-120fa31006a7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:06.641239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d24751-f30c-4013-8d89-120fa31006a7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:06.641533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d24751-f30c-4013-8d89-120fa31006a7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:06.641772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d24751-f30c-4013-8d89-120fa31006a7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:06.644197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d24751-f30c-4013-8d89-120fa31006a7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_grant(project_id=3e37435363bb4abc9199670f3780c04c, user_id=8401b16d587845afa1487456fee0bfaf, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:06.648055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d24751-f30c-4013-8d89-120fa31006a7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:06.672427 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2402/4853] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:06 2022] PUT /identity/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:06.677540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d24751-f30c-4013-8d89-120fa31006a7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:06.682380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-71d24751-f30c-4013-8d89-120fa31006a7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:06.748788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8e505-8279-4f8a-9b3e-bfb414babeba admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:06.794362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d24751-f30c-4013-8d89-120fa31006a7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:06.809592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c3a1e-4297-4a8c-b91e-37d3bb8f6fbf tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:06.811733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c3a1e-4297-4a8c-b91e-37d3bb8f6fbf tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:06.812099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c3a1e-4297-4a8c-b91e-37d3bb8f6fbf tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:06.813563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c3a1e-4297-4a8c-b91e-37d3bb8f6fbf tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:06.816747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:06.817151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c3a1e-4297-4a8c-b91e-37d3bb8f6fbf tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_grants(project_id=3e37435363bb4abc9199670f3780c04c, user_id=8401b16d587845afa1487456fee0bfaf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:06.821790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c3a1e-4297-4a8c-b91e-37d3bb8f6fbf tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:06.853695 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2403/4854] 10.222.0.23 () {58 vars in 1365 bytes} [Sat Aug 20 21:10:06 2022] GET /identity/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles => generated 410 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:06.861938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c3a1e-4297-4a8c-b91e-37d3bb8f6fbf tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:06.884107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c3a1e-4297-4a8c-b91e-37d3bb8f6fbf tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:06.900681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:06.901648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06ad8a-33f4-44c4-a00e-3af1eec4b804 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:06.901841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:06.902258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:06.902689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] PATH_INFO: `/v3/users/70711adcb5584d4d9cd46b9c0acb9b7d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:06.903784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06ad8a-33f4-44c4-a00e-3af1eec4b804 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:06.904711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06ad8a-33f4-44c4-a00e-3af1eec4b804 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:06.905102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06ad8a-33f4-44c4-a00e-3af1eec4b804 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:06.905554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70711adcb5584d4d9cd46b9c0acb9b7d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:06.908739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06ad8a-33f4-44c4-a00e-3af1eec4b804 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:check_grant(project_id=3e37435363bb4abc9199670f3780c04c, user_id=8401b16d587845afa1487456fee0bfaf, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:06.909358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:06.915487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06ad8a-33f4-44c4-a00e-3af1eec4b804 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:06.929979 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2404/4855] 10.222.0.23 () {58 vars in 1465 bytes} [Sat Aug 20 21:10:06 2022] HEAD /identity/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:06.937900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06ad8a-33f4-44c4-a00e-3af1eec4b804 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:06.956056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06ad8a-33f4-44c4-a00e-3af1eec4b804 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:06.965691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ee02a-d29c-4100-8731-965310301fe0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:06.966583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ee02a-d29c-4100-8731-965310301fe0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:06.967149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ee02a-d29c-4100-8731-965310301fe0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:06.967515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ee02a-d29c-4100-8731-965310301fe0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:06.971006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ee02a-d29c-4100-8731-965310301fe0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=3e37435363bb4abc9199670f3780c04c, user_id=8401b16d587845afa1487456fee0bfaf, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:06.975432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ee02a-d29c-4100-8731-965310301fe0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:06.983571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e91ee02a-d29c-4100-8731-965310301fe0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role bd889ce606ae4308bb72b30879315f83 was removed from user 8401b16d587845afa1487456fee0bfaf on project 3e37435363bb4abc9199670f3780c04c. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:06.984146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e91ee02a-d29c-4100-8731-965310301fe0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:06.985587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e91ee02a-d29c-4100-8731-965310301fe0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:07.028821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70711adcb5584d4d9cd46b9c0acb9b7d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:07.031602 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2405/4856] 10.222.0.23 () {58 vars in 1467 bytes} [Sat Aug 20 21:10:06 2022] DELETE /identity/v3/projects/3e37435363bb4abc9199670f3780c04c/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:07.049931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ee02a-d29c-4100-8731-965310301fe0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:07.049931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e91ee02a-d29c-4100-8731-965310301fe0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:07.071428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70711adcb5584d4d9cd46b9c0acb9b7d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:07.093378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70711adcb5584d4d9cd46b9c0acb9b7d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:07.126614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70711adcb5584d4d9cd46b9c0acb9b7d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:07.126614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70711adcb5584d4d9cd46b9c0acb9b7d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:07.134365 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2452/4857] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:06 2022] DELETE /identity/v3/users/70711adcb5584d4d9cd46b9c0acb9b7d => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:07.158532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ee02a-d29c-4100-8731-965310301fe0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:07.267998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:07.309048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4176f-f790-4306-84d5-c66bd2589539 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:07.314558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4176f-f790-4306-84d5-c66bd2589539 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:07.314857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4176f-f790-4306-84d5-c66bd2589539 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:07.315124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4176f-f790-4306-84d5-c66bd2589539 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/system/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:07.322611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4176f-f790-4306-84d5-c66bd2589539 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8401b16d587845afa1487456fee0bfaf, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:07.323656 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-e3b4176f-f790-4306-84d5-c66bd2589539 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:07.323656 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:07.323656 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:07.324679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4176f-f790-4306-84d5-c66bd2589539 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:07.343574 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2406/4858] 10.222.0.23 () {60 vars in 1378 bytes} [Sat Aug 20 21:10:07 2022] PUT /identity/v3/system/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:07.348730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:07.350351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:07.395585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c4ee7-2455-4554-acba-1e115ed61195 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:07.413734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9983e-eae6-43b3-a787-83eb574eb3e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:07.416832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9983e-eae6-43b3-a787-83eb574eb3e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:07.417342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9983e-eae6-43b3-a787-83eb574eb3e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:07.417683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9983e-eae6-43b3-a787-83eb574eb3e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/system/users/8401b16d587845afa1487456fee0bfaf/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:07.420894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9983e-eae6-43b3-a787-83eb574eb3e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_system_grants_for_user(user_id=8401b16d587845afa1487456fee0bfaf)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:07.423644 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-d0b9983e-eae6-43b3-a787-83eb574eb3e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:07.423644 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:07.423644 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:07.426611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9983e-eae6-43b3-a787-83eb574eb3e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:07.463643 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2453/4859] 10.222.0.23 () {58 vars in 1260 bytes} [Sat Aug 20 21:10:07 2022] GET /identity/v3/system/users/8401b16d587845afa1487456fee0bfaf/roles => generated 375 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:07.468684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4176f-f790-4306-84d5-c66bd2589539 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:07.484278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4176f-f790-4306-84d5-c66bd2589539 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:07.493631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6de8b-5434-4f02-b8a8-487e4c8df567 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:07.494805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6de8b-5434-4f02-b8a8-487e4c8df567 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:07.495704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6de8b-5434-4f02-b8a8-487e4c8df567 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:07.496047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6de8b-5434-4f02-b8a8-487e4c8df567 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/system/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:07.498314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6de8b-5434-4f02-b8a8-487e4c8df567 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:check_system_grant_for_user(user_id=8401b16d587845afa1487456fee0bfaf, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:07.499884 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-92d6de8b-5434-4f02-b8a8-487e4c8df567 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:07.499884 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:07.499884 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:07.501560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6de8b-5434-4f02-b8a8-487e4c8df567 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:07.508092 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2407/4860] 10.222.0.23 () {58 vars in 1360 bytes} [Sat Aug 20 21:10:07 2022] HEAD /identity/v3/system/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:07.512289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9983e-eae6-43b3-a787-83eb574eb3e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:07.526618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9983e-eae6-43b3-a787-83eb574eb3e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:07.538599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b0e17-dc33-4eaa-af39-af9129cbe2a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:07.541403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b0e17-dc33-4eaa-af39-af9129cbe2a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:07.541962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b0e17-dc33-4eaa-af39-af9129cbe2a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:07.542306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b0e17-dc33-4eaa-af39-af9129cbe2a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/system/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:07.544510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b0e17-dc33-4eaa-af39-af9129cbe2a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_user(user_id=8401b16d587845afa1487456fee0bfaf, role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:07.547832 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-c53b0e17-dc33-4eaa-af39-af9129cbe2a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:07.547832 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:07.547832 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:07.551345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b0e17-dc33-4eaa-af39-af9129cbe2a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:07.564412 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2454/4861] 10.222.0.23 () {58 vars in 1362 bytes} [Sat Aug 20 21:10:07 2022] DELETE /identity/v3/system/users/8401b16d587845afa1487456fee0bfaf/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:07.569783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6de8b-5434-4f02-b8a8-487e4c8df567 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:07.571918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-92d6de8b-5434-4f02-b8a8-487e4c8df567 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:07.577437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b0e17-dc33-4eaa-af39-af9129cbe2a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:07.593258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b0e17-dc33-4eaa-af39-af9129cbe2a5 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:07.598546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e9abe-ef83-4189-b7c1-8daa1bc059a4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:07.599272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e9abe-ef83-4189-b7c1-8daa1bc059a4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:07.600179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e9abe-ef83-4189-b7c1-8daa1bc059a4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:07.600492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e9abe-ef83-4189-b7c1-8daa1bc059a4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:07.601167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e9abe-ef83-4189-b7c1-8daa1bc059a4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:07.602815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e9abe-ef83-4189-b7c1-8daa1bc059a4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:07.635563 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2455/4862] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:10:07 2022] POST /identity/v3/roles => generated 270 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:07.636745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e9abe-ef83-4189-b7c1-8daa1bc059a4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:07.647927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e9abe-ef83-4189-b7c1-8daa1bc059a4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:07.654373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1ca2e-b643-43a3-a8ae-d513df4257cc tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:07.655226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1ca2e-b643-43a3-a8ae-d513df4257cc tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:07.655719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1ca2e-b643-43a3-a8ae-d513df4257cc tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:07.656127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1ca2e-b643-43a3-a8ae-d513df4257cc tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:07.656917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1ca2e-b643-43a3-a8ae-d513df4257cc tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:07.658771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1ca2e-b643-43a3-a8ae-d513df4257cc tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:07.688363 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2456/4863] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:10:07 2022] POST /identity/v3/roles => generated 269 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:07.690311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1ca2e-b643-43a3-a8ae-d513df4257cc tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:07.693846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6de8b-5434-4f02-b8a8-487e4c8df567 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:07.703838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1ca2e-b643-43a3-a8ae-d513df4257cc tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:07.705018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:07.705889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:07.706814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:07.707315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320 admin admin] PATH_INFO: `/v3/projects/f6c564242cec4c29a0f6f980d0a83a3c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:07.709209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6c564242cec4c29a0f6f980d0a83a3c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:07.709840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20f9e0-6537-4dea-8148-0199eca2a1e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:07.710605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20f9e0-6537-4dea-8148-0199eca2a1e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:07.710972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20f9e0-6537-4dea-8148-0199eca2a1e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:07.711329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20f9e0-6537-4dea-8148-0199eca2a1e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/e43654b0bb75444195ea39f232845752/implies/88160463b95f4fdead952d6a37a083fc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:07.712729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:07.715024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf20f9e0-6537-4dea-8148-0199eca2a1e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=e43654b0bb75444195ea39f232845752, implied_role_id=88160463b95f4fdead952d6a37a083fc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:07.717465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf20f9e0-6537-4dea-8148-0199eca2a1e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:07.757303 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2457/4864] 10.222.0.23 () {60 vars in 1363 bytes} [Sat Aug 20 21:10:07 2022] PUT /identity/v3/roles/e43654b0bb75444195ea39f232845752/implies/88160463b95f4fdead952d6a37a083fc => generated 530 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:07.760808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20f9e0-6537-4dea-8148-0199eca2a1e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:07.781603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6c564242cec4c29a0f6f980d0a83a3c', 'request_id': 'req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:07.791153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6c564242cec4c29a0f6f980d0a83a3c', 'request_id': 'req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:07.802578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6c564242cec4c29a0f6f980d0a83a3c', 'request_id': 'req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:07.808061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:07.821258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320 admin admin] The token cache is being invalidate because project f6c564242cec4c29a0f6f980d0a83a3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:07.821258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:07.822167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ce35a2c8-e60f-4fb1-aeeb-532ea2fe7320 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:07.824150 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2408/4865] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:07 2022] DELETE /identity/v3/projects/f6c564242cec4c29a0f6f980d0a83a3c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:07.829297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2bb09-23dd-42ca-9e24-62761d49eb17 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:07.829667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2bb09-23dd-42ca-9e24-62761d49eb17 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:07.830091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2bb09-23dd-42ca-9e24-62761d49eb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:07.845240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20f9e0-6537-4dea-8148-0199eca2a1e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:07.855319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2ba2bb09-23dd-42ca-9e24-62761d49eb17 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:07.857500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2ba2bb09-23dd-42ca-9e24-62761d49eb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:07.929741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd57fe-d724-4839-8626-aa32ce80d119 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:07.930360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd57fe-d724-4839-8626-aa32ce80d119 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:07.930630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd57fe-d724-4839-8626-aa32ce80d119 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:07.930891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd57fe-d724-4839-8626-aa32ce80d119 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:07.931552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd57fe-d724-4839-8626-aa32ce80d119 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:07.934925 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2409/4866] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:07.935492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd57fe-d724-4839-8626-aa32ce80d119 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:07.944667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1346c-530d-4068-9117-393068e5a48e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:07.945627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1346c-530d-4068-9117-393068e5a48e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:07.945834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1346c-530d-4068-9117-393068e5a48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:07.972936 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2458/4867] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:10:07 2022] POST /identity/v3/domains => generated 277 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:07.976584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd57fe-d724-4839-8626-aa32ce80d119 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:07.978456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dfdd57fe-d724-4839-8626-aa32ce80d119 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:07.984798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-12d1346c-530d-4068-9117-393068e5a48e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:07.987983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-12d1346c-530d-4068-9117-393068e5a48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:08.003273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:08.074612 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2410/4868] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:08.075609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1346c-530d-4068-9117-393068e5a48e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:08.080255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd57fe-d724-4839-8626-aa32ce80d119 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:08.086063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1346c-530d-4068-9117-393068e5a48e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:08.091055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0290d-c3ef-4a37-b462-1fa2410c6195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:08.091687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0290d-c3ef-4a37-b462-1fa2410c6195 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.091939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0290d-c3ef-4a37-b462-1fa2410c6195 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.092201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0290d-c3ef-4a37-b462-1fa2410c6195 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.093051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0290d-c3ef-4a37-b462-1fa2410c6195 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:08.093225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0290d-c3ef-4a37-b462-1fa2410c6195 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:08.094826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0290d-c3ef-4a37-b462-1fa2410c6195 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:08.102029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e544d42-cdbe-4f7c-a1e9-b9da3a33e246 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:08.102815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e544d42-cdbe-4f7c-a1e9-b9da3a33e246 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.103637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e544d42-cdbe-4f7c-a1e9-b9da3a33e246 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.103637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e544d42-cdbe-4f7c-a1e9-b9da3a33e246 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.104595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e544d42-cdbe-4f7c-a1e9-b9da3a33e246 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:08.111968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e544d42-cdbe-4f7c-a1e9-b9da3a33e246 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:08.125020 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2411/4869] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:08.127962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0290d-c3ef-4a37-b462-1fa2410c6195 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:08.139600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0290d-c3ef-4a37-b462-1fa2410c6195 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:08.144726 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2459/4870] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:10:07 2022] POST /identity/v3/roles => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:08.149591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab450d1d-3712-4021-a698-2d9d4e4a8a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:08.150247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab450d1d-3712-4021-a698-2d9d4e4a8a39 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.150519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab450d1d-3712-4021-a698-2d9d4e4a8a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.150783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab450d1d-3712-4021-a698-2d9d4e4a8a39 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.151110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e544d42-cdbe-4f7c-a1e9-b9da3a33e246 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:08.151620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab450d1d-3712-4021-a698-2d9d4e4a8a39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:08.153964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab450d1d-3712-4021-a698-2d9d4e4a8a39 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:08.173282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e544d42-cdbe-4f7c-a1e9-b9da3a33e246 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:08.184293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8b143-7271-45cb-bb16-ecd1a8a80a23 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:08.185092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8b143-7271-45cb-bb16-ecd1a8a80a23 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.185614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8b143-7271-45cb-bb16-ecd1a8a80a23 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.185871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8b143-7271-45cb-bb16-ecd1a8a80a23 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/e43654b0bb75444195ea39f232845752/implies/9b61e1f1ec384989a084bdf37151aaad` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.187688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8b143-7271-45cb-bb16-ecd1a8a80a23 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=e43654b0bb75444195ea39f232845752, implied_role_id=9b61e1f1ec384989a084bdf37151aaad)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:08.190626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8b143-7271-45cb-bb16-ecd1a8a80a23 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:08.208016 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2412/4871] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:08 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:08.208016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab450d1d-3712-4021-a698-2d9d4e4a8a39 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:08.227803 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2460/4872] 10.222.0.23 () {60 vars in 1363 bytes} [Sat Aug 20 21:10:08 2022] PUT /identity/v3/roles/e43654b0bb75444195ea39f232845752/implies/9b61e1f1ec384989a084bdf37151aaad => generated 530 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:08.232251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8b143-7271-45cb-bb16-ecd1a8a80a23 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:08.337583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:08.345495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab450d1d-3712-4021-a698-2d9d4e4a8a39 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:08.377548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8b143-7271-45cb-bb16-ecd1a8a80a23 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:08.389919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cd58f-0b4d-49a3-a896-e973759a3973 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:08.390610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cd58f-0b4d-49a3-a896-e973759a3973 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.390884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cd58f-0b4d-49a3-a896-e973759a3973 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.391800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cd58f-0b4d-49a3-a896-e973759a3973 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/e43654b0bb75444195ea39f232845752/implies/bd889ce606ae4308bb72b30879315f83` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.394354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cd58f-0b4d-49a3-a896-e973759a3973 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=e43654b0bb75444195ea39f232845752, implied_role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:08.396412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cd58f-0b4d-49a3-a896-e973759a3973 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:08.436406 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2461/4873] 10.222.0.23 () {60 vars in 1363 bytes} [Sat Aug 20 21:10:08 2022] PUT /identity/v3/roles/e43654b0bb75444195ea39f232845752/implies/bd889ce606ae4308bb72b30879315f83 => generated 526 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:08.438542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0217699-723e-4202-8c9c-bbeb0db90754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:08.439101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0217699-723e-4202-8c9c-bbeb0db90754 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.439375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0217699-723e-4202-8c9c-bbeb0db90754 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.439640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0217699-723e-4202-8c9c-bbeb0db90754 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.440255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0217699-723e-4202-8c9c-bbeb0db90754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:08.440918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cd58f-0b4d-49a3-a896-e973759a3973 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:08.442537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0217699-723e-4202-8c9c-bbeb0db90754 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:08.471728 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2413/4874] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:08 2022] POST /identity/v3/users => generated 287 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:08.476411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0217699-723e-4202-8c9c-bbeb0db90754 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:08.491733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0217699-723e-4202-8c9c-bbeb0db90754 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:08.511812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:08.562178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cd58f-0b4d-49a3-a896-e973759a3973 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:08.597265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ac592-0947-4a71-8a8e-1ea2515e3201 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:08.599053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ac592-0947-4a71-8a8e-1ea2515e3201 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.599258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ac592-0947-4a71-8a8e-1ea2515e3201 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.599810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ac592-0947-4a71-8a8e-1ea2515e3201 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/bd889ce606ae4308bb72b30879315f83/implies/e43654b0bb75444195ea39f232845752` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.605430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ac592-0947-4a71-8a8e-1ea2515e3201 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=bd889ce606ae4308bb72b30879315f83, implied_role_id=e43654b0bb75444195ea39f232845752)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:08.613057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ac592-0947-4a71-8a8e-1ea2515e3201 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-496ac592-0947-4a71-8a8e-1ea2515e3201 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] e43654b0bb75444195ea39f232845752 cannot be an implied roles.: keystone.exception.InvalidImpliedRole: e43654b0bb75444195ea39f232845752 cannot be an implied roles. Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 259, in put Aug 20 21:10:08.641268 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application PROVIDERS.role_api.create_implied_role(prior_role_id, implied_role_id) Aug 20 21:10:08.642620 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:10:08.642620 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:10:08.642620 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1361, in create_implied_role Aug 20 21:10:08.642620 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.InvalidImpliedRole(msg, Aug 20 21:10:08.642620 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.InvalidImpliedRole: e43654b0bb75444195ea39f232845752 cannot be an implied roles. Aug 20 21:10:08.642620 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:10:08.647707 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2462/4875] 10.222.0.23 () {60 vars in 1363 bytes} [Sat Aug 20 21:10:08 2022] PUT /identity/v3/roles/bd889ce606ae4308bb72b30879315f83/implies/e43654b0bb75444195ea39f232845752 => generated 116 bytes in 207 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:10:08.649525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ac592-0947-4a71-8a8e-1ea2515e3201 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:08.649969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe1273-160f-4831-9349-e27b5f164b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:08.650702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe1273-160f-4831-9349-e27b5f164b88 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.651625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe1273-160f-4831-9349-e27b5f164b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.651901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe1273-160f-4831-9349-e27b5f164b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.652636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe1273-160f-4831-9349-e27b5f164b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:08.656208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe1273-160f-4831-9349-e27b5f164b88 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:08.662677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ac592-0947-4a71-8a8e-1ea2515e3201 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:08.672042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b596cd-b9cc-4e84-9d95-a1257b34ab93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:08.672857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b596cd-b9cc-4e84-9d95-a1257b34ab93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.673245 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2414/4876] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:08 2022] GET /identity/v3/roles => generated 2324 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:08.676584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b596cd-b9cc-4e84-9d95-a1257b34ab93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.676944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b596cd-b9cc-4e84-9d95-a1257b34ab93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/e43654b0bb75444195ea39f232845752/implies/bd889ce606ae4308bb72b30879315f83` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.677174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe1273-160f-4831-9349-e27b5f164b88 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:08.678995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b596cd-b9cc-4e84-9d95-a1257b34ab93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=e43654b0bb75444195ea39f232845752, implied_role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:08.680737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b596cd-b9cc-4e84-9d95-a1257b34ab93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:08.690797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe1273-160f-4831-9349-e27b5f164b88 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:08.694375 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2463/4877] 10.222.0.23 () {58 vars in 1347 bytes} [Sat Aug 20 21:10:08 2022] DELETE /identity/v3/roles/e43654b0bb75444195ea39f232845752/implies/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:08.698051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb9074-7fe0-4340-a77e-8ba3dc587b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:08.698690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb9074-7fe0-4340-a77e-8ba3dc587b0f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.698986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b596cd-b9cc-4e84-9d95-a1257b34ab93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:08.699151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb9074-7fe0-4340-a77e-8ba3dc587b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.699269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb9074-7fe0-4340-a77e-8ba3dc587b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.700094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb9074-7fe0-4340-a77e-8ba3dc587b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:08.702274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb9074-7fe0-4340-a77e-8ba3dc587b0f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:08.717973 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2415/4878] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:08 2022] GET /identity/v3/roles => generated 2324 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:08.722121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb9074-7fe0-4340-a77e-8ba3dc587b0f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:08.734939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb9074-7fe0-4340-a77e-8ba3dc587b0f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:08.789192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b596cd-b9cc-4e84-9d95-a1257b34ab93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:08.794760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59162562-28be-407c-ad13-eb6556d2ca9a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:08.794965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59162562-28be-407c-ad13-eb6556d2ca9a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.795359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59162562-28be-407c-ad13-eb6556d2ca9a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.795491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59162562-28be-407c-ad13-eb6556d2ca9a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/e43654b0bb75444195ea39f232845752/implies/9b61e1f1ec384989a084bdf37151aaad` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.797126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59162562-28be-407c-ad13-eb6556d2ca9a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=e43654b0bb75444195ea39f232845752, implied_role_id=9b61e1f1ec384989a084bdf37151aaad)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:08.798949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59162562-28be-407c-ad13-eb6556d2ca9a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:08.807625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc042b-3855-431b-900c-0178caf7d808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:08.808579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc042b-3855-431b-900c-0178caf7d808 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.809196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc042b-3855-431b-900c-0178caf7d808 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.809668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc042b-3855-431b-900c-0178caf7d808 admin admin] PATH_INFO: `/v3/projects/5592ede086c7438eaa83dfb8cd73c1fb/users/055f740ebe494119973307b8d209fe6f/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.813399 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2464/4879] 10.222.0.23 () {58 vars in 1347 bytes} [Sat Aug 20 21:10:08 2022] DELETE /identity/v3/roles/e43654b0bb75444195ea39f232845752/implies/9b61e1f1ec384989a084bdf37151aaad => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:08.814549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59162562-28be-407c-ad13-eb6556d2ca9a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:08.831358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc042b-3855-431b-900c-0178caf7d808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5592ede086c7438eaa83dfb8cd73c1fb, user_id=055f740ebe494119973307b8d209fe6f, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:08.834613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc042b-3855-431b-900c-0178caf7d808 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:08.855812 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2416/4880] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:08 2022] PUT /identity/v3/projects/5592ede086c7438eaa83dfb8cd73c1fb/users/055f740ebe494119973307b8d209fe6f/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:08.865955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:08.866330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005008c-29dc-4a3f-8fc2-69069063e08b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.866617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005008c-29dc-4a3f-8fc2-69069063e08b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.867144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005008c-29dc-4a3f-8fc2-69069063e08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.897520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59162562-28be-407c-ad13-eb6556d2ca9a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:08.898844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9005008c-29dc-4a3f-8fc2-69069063e08b None None] MFA Rules not processed for user `055f740ebe494119973307b8d209fe6f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:08.900205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9005008c-29dc-4a3f-8fc2-69069063e08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:08.984998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440f41a-be50-41b3-a5e3-a96c2a316850 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:08.986055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440f41a-be50-41b3-a5e3-a96c2a316850 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:08.986328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440f41a-be50-41b3-a5e3-a96c2a316850 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:08.986595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440f41a-be50-41b3-a5e3-a96c2a316850 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/9b61e1f1ec384989a084bdf37151aaad` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:08.987735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440f41a-be50-41b3-a5e3-a96c2a316850 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=9b61e1f1ec384989a084bdf37151aaad)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:08.989770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440f41a-be50-41b3-a5e3-a96c2a316850 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:08.994292 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2417/4881] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:09.009323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005008c-29dc-4a3f-8fc2-69069063e08b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:09.012087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9005008c-29dc-4a3f-8fc2-69069063e08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:09.025553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:09.038629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c440f41a-be50-41b3-a5e3-a96c2a316850 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role 9b61e1f1ec384989a084bdf37151aaad has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:09.038980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c440f41a-be50-41b3-a5e3-a96c2a316850 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.039891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c440f41a-be50-41b3-a5e3-a96c2a316850 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.042673 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2465/4882] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:08 2022] DELETE /identity/v3/roles/9b61e1f1ec384989a084bdf37151aaad => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:09.046773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440f41a-be50-41b3-a5e3-a96c2a316850 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:09.048153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c440f41a-be50-41b3-a5e3-a96c2a316850 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:09.089850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005008c-29dc-4a3f-8fc2-69069063e08b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:09.130018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440f41a-be50-41b3-a5e3-a96c2a316850 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:09.135694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb5206-1c4f-452e-a70a-4f91ef5fdfb4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:09.136517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb5206-1c4f-452e-a70a-4f91ef5fdfb4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:09.136798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb5206-1c4f-452e-a70a-4f91ef5fdfb4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:09.137120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb5206-1c4f-452e-a70a-4f91ef5fdfb4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains/f42d71442931441d9bbd0fd41e362063` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:09.147582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb834e-d8aa-4762-871f-3d2a8fffbac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:09.149254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb834e-d8aa-4762-871f-3d2a8fffbac7 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:09.149634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb834e-d8aa-4762-871f-3d2a8fffbac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:09.151109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb834e-d8aa-4762-871f-3d2a8fffbac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:09.152602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb5206-1c4f-452e-a70a-4f91ef5fdfb4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f42d71442931441d9bbd0fd41e362063)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:09.153412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4bbb834e-d8aa-4762-871f-3d2a8fffbac7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:09.154471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb5206-1c4f-452e-a70a-4f91ef5fdfb4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:09.182123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24cb5206-1c4f-452e-a70a-4f91ef5fdfb4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f42d71442931441d9bbd0fd41e362063'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.183021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24cb5206-1c4f-452e-a70a-4f91ef5fdfb4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f42d71442931441d9bbd0fd41e362063'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.223312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24cb5206-1c4f-452e-a70a-4f91ef5fdfb4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f42d71442931441d9bbd0fd41e362063'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.228983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24cb5206-1c4f-452e-a70a-4f91ef5fdfb4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f42d71442931441d9bbd0fd41e362063'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.233781 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2466/4883] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:10:09 2022] PATCH /identity/v3/domains/f42d71442931441d9bbd0fd41e362063 => generated 278 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:10:09.242494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb5206-1c4f-452e-a70a-4f91ef5fdfb4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:09.247011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-24cb5206-1c4f-452e-a70a-4f91ef5fdfb4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:09.259147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb834e-d8aa-4762-871f-3d2a8fffbac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:09.260898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb834e-d8aa-4762-871f-3d2a8fffbac7 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:09.269185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:UPDATE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:09.274657 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2418/4884] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:10:09.344965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb5206-1c4f-452e-a70a-4f91ef5fdfb4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:09.351894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:09.352969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:09.353359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:09.354112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains/f42d71442931441d9bbd0fd41e362063` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:09.369604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f42d71442931441d9bbd0fd41e362063)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:09.372223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:09.407508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f42d71442931441d9bbd0fd41e362063'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.420646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f42d71442931441d9bbd0fd41e362063'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.476815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f42d71442931441d9bbd0fd41e362063', 'request_id': 'req-4c0c6cd3-f216-4377-adfd-88a3222d1c05'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.490410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f42d71442931441d9bbd0fd41e362063', 'request_id': 'req-4c0c6cd3-f216-4377-adfd-88a3222d1c05'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.510771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f42d71442931441d9bbd0fd41e362063', 'request_id': 'req-4c0c6cd3-f216-4377-adfd-88a3222d1c05'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.528520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] The token cache is being invalidate because project f42d71442931441d9bbd0fd41e362063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:09.529087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.529788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.538912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f42d71442931441d9bbd0fd41e362063', 'request_id': 'req-4c0c6cd3-f216-4377-adfd-88a3222d1c05'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.539911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f42d71442931441d9bbd0fd41e362063', 'request_id': 'req-4c0c6cd3-f216-4377-adfd-88a3222d1c05'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.543907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f42d71442931441d9bbd0fd41e362063', 'request_id': 'req-4c0c6cd3-f216-4377-adfd-88a3222d1c05'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.550626 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2467/4885] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:10:09 2022] DELETE /identity/v3/domains/f42d71442931441d9bbd0fd41e362063 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:09.555300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb834e-d8aa-4762-871f-3d2a8fffbac7 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:09.592229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb834e-d8aa-4762-871f-3d2a8fffbac7 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:09.717995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d7ea4-17d5-4b88-8f67-abdd7547cbc0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:09.718988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d7ea4-17d5-4b88-8f67-abdd7547cbc0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:09.719464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d7ea4-17d5-4b88-8f67-abdd7547cbc0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:09.719978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d7ea4-17d5-4b88-8f67-abdd7547cbc0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/e43654b0bb75444195ea39f232845752/implies/88160463b95f4fdead952d6a37a083fc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:09.722538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d7ea4-17d5-4b88-8f67-abdd7547cbc0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=e43654b0bb75444195ea39f232845752, implied_role_id=88160463b95f4fdead952d6a37a083fc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:09.724942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d7ea4-17d5-4b88-8f67-abdd7547cbc0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:09.741658 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2419/4886] 10.222.0.23 () {58 vars in 1347 bytes} [Sat Aug 20 21:10:09 2022] DELETE /identity/v3/roles/e43654b0bb75444195ea39f232845752/implies/88160463b95f4fdead952d6a37a083fc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:09.746195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:09.748044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:09.763129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c6cd3-f216-4377-adfd-88a3222d1c05 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:09.818356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:09.847878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09706171-624e-48ac-8cda-ac998138915a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:09.848659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09706171-624e-48ac-8cda-ac998138915a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:09.849006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09706171-624e-48ac-8cda-ac998138915a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:09.849384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09706171-624e-48ac-8cda-ac998138915a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/88160463b95f4fdead952d6a37a083fc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:09.850635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09706171-624e-48ac-8cda-ac998138915a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=88160463b95f4fdead952d6a37a083fc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:09.852532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09706171-624e-48ac-8cda-ac998138915a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:09.927596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-09706171-624e-48ac-8cda-ac998138915a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role 88160463b95f4fdead952d6a37a083fc has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:09.927896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-09706171-624e-48ac-8cda-ac998138915a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.928610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-09706171-624e-48ac-8cda-ac998138915a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:09.933719 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2468/4887] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:09 2022] DELETE /identity/v3/roles/88160463b95f4fdead952d6a37a083fc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:09.935482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d7ea4-17d5-4b88-8f67-abdd7547cbc0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:09.937243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-462d7ea4-17d5-4b88-8f67-abdd7547cbc0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:09.953563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d7ea4-17d5-4b88-8f67-abdd7547cbc0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:09.973450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:10.093836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfddc38-7746-44a5-9413-b4778c74ca3e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:10.094714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfddc38-7746-44a5-9413-b4778c74ca3e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:10.095074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfddc38-7746-44a5-9413-b4778c74ca3e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:10.095436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfddc38-7746-44a5-9413-b4778c74ca3e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/e43654b0bb75444195ea39f232845752` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:10.096913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfddc38-7746-44a5-9413-b4778c74ca3e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=e43654b0bb75444195ea39f232845752)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:10.098935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfddc38-7746-44a5-9413-b4778c74ca3e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:10.161601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-dcfddc38-7746-44a5-9413-b4778c74ca3e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role e43654b0bb75444195ea39f232845752 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:10.161924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-dcfddc38-7746-44a5-9413-b4778c74ca3e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:10.162575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-dcfddc38-7746-44a5-9413-b4778c74ca3e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:10.164667 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2420/4888] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:09 2022] DELETE /identity/v3/roles/e43654b0bb75444195ea39f232845752 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:10.170340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09706171-624e-48ac-8cda-ac998138915a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:10.196164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09706171-624e-48ac-8cda-ac998138915a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:10.273286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccc746-5838-456c-85a6-f0d9bb82107c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:10.274244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccc746-5838-456c-85a6-f0d9bb82107c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:10.274375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccc746-5838-456c-85a6-f0d9bb82107c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:10.274650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccc746-5838-456c-85a6-f0d9bb82107c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:10.276833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccc746-5838-456c-85a6-f0d9bb82107c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:10.278478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccc746-5838-456c-85a6-f0d9bb82107c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:10.312438 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2469/4889] 10.222.0.23 () {60 vars in 1363 bytes} [Sat Aug 20 21:10:10 2022] PUT /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5 => generated 521 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:10.316787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfddc38-7746-44a5-9413-b4778c74ca3e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:10.318519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dcfddc38-7746-44a5-9413-b4778c74ca3e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:10.339658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfddc38-7746-44a5-9413-b4778c74ca3e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:10.438654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c1db3-d94c-4aa4-b9c8-a0a6121971e4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:10.439833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c1db3-d94c-4aa4-b9c8-a0a6121971e4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:10.440420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c1db3-d94c-4aa4-b9c8-a0a6121971e4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:10.440699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c1db3-d94c-4aa4-b9c8-a0a6121971e4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:10.457479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c1db3-d94c-4aa4-b9c8-a0a6121971e4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:check_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:10.458569 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-006c1db3-d94c-4aa4-b9c8-a0a6121971e4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:10.458569 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:10.458569 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:10.459492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c1db3-d94c-4aa4-b9c8-a0a6121971e4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:10.461041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c1db3-d94c-4aa4-b9c8-a0a6121971e4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:10.461942 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-006c1db3-d94c-4aa4-b9c8-a0a6121971e4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:10.461942 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:10.461942 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:10.463026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c1db3-d94c-4aa4-b9c8-a0a6121971e4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:10.471681 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2421/4890] 10.222.0.23 () {58 vars in 1345 bytes} [Sat Aug 20 21:10:10 2022] HEAD /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:10.475744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccc746-5838-456c-85a6-f0d9bb82107c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:10.490345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccc746-5838-456c-85a6-f0d9bb82107c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:10.509338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc774f8-2c8d-4810-9e7c-bcf1324c690e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:10.510946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc774f8-2c8d-4810-9e7c-bcf1324c690e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:10.511700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc774f8-2c8d-4810-9e7c-bcf1324c690e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:10.512335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc774f8-2c8d-4810-9e7c-bcf1324c690e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:10.514923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc774f8-2c8d-4810-9e7c-bcf1324c690e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:10.517220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c1db3-d94c-4aa4-b9c8-a0a6121971e4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:10.520974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-006c1db3-d94c-4aa4-b9c8-a0a6121971e4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:10.530277 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-dfc774f8-2c8d-4810-9e7c-bcf1324c690e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:10.530277 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:10.530277 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:10.536624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc774f8-2c8d-4810-9e7c-bcf1324c690e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:10.557526 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2470/4891] 10.222.0.23 () {58 vars in 1344 bytes} [Sat Aug 20 21:10:10 2022] GET /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5 => generated 521 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:10.563013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc774f8-2c8d-4810-9e7c-bcf1324c690e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:10.586068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc774f8-2c8d-4810-9e7c-bcf1324c690e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:10.595101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2c084-5670-4d1a-831a-a844bc422ab8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:10.596293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2c084-5670-4d1a-831a-a844bc422ab8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:10.596830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2c084-5670-4d1a-831a-a844bc422ab8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:10.597120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2c084-5670-4d1a-831a-a844bc422ab8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:10.602971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2c084-5670-4d1a-831a-a844bc422ab8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:10.605392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2c084-5670-4d1a-831a-a844bc422ab8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:10.619673 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2471/4892] 10.222.0.23 () {58 vars in 1347 bytes} [Sat Aug 20 21:10:10 2022] DELETE /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:10.629695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2c084-5670-4d1a-831a-a844bc422ab8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:10.689146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c1db3-d94c-4aa4-b9c8-a0a6121971e4 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:10.740656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2c084-5670-4d1a-831a-a844bc422ab8 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:10.741831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:10.757901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35defca-2846-4d49-95e6-c025e954639c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:10.759273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35defca-2846-4d49-95e6-c025e954639c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:10.759273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35defca-2846-4d49-95e6-c025e954639c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:10.759273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35defca-2846-4d49-95e6-c025e954639c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:10.760829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35defca-2846-4d49-95e6-c025e954639c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:10.763349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35defca-2846-4d49-95e6-c025e954639c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-f35defca-2846-4d49-95e6-c025e954639c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] be50ad6b14b342a9b9347d2bf643b250 does not imply 7d0cc3f88e39496d87c9fed6401b9be5.: keystone.exception.ImpliedRoleNotFound: be50ad6b14b342a9b9347d2bf643b250 does not imply 7d0cc3f88e39496d87c9fed6401b9be5. Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = query.one() Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._iter().one() Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._only_one_row( Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exc.NoResultFound( Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:10:10.787327 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 235, in get Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_implied_role(prior_role_id, implied_role_id) Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 239, in _get_implied_role Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application PROVIDERS.role_api.get_implied_role( Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 149, in get_implied_role Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Aug 20 21:10:10.788667 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Aug 20 21:10:10.789763 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Aug 20 21:10:10.789763 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: be50ad6b14b342a9b9347d2bf643b250 does not imply 7d0cc3f88e39496d87c9fed6401b9be5. Aug 20 21:10:10.789763 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:10:10.802007 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2472/4893] 10.222.0.23 () {58 vars in 1344 bytes} [Sat Aug 20 21:10:10 2022] GET /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5 => generated 137 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:10:10.808422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35defca-2846-4d49-95e6-c025e954639c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:10.809436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a2997-0da2-4b5c-b018-8baa47122f8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:10.810040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a2997-0da2-4b5c-b018-8baa47122f8e service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:10.810040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a2997-0da2-4b5c-b018-8baa47122f8e service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:10.810315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a2997-0da2-4b5c-b018-8baa47122f8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:10.812457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-524a2997-0da2-4b5c-b018-8baa47122f8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:10.839076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35defca-2846-4d49-95e6-c025e954639c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:10.846196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd7deb-0aec-4ac6-9c46-bf65df939db6 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:10.848334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd7deb-0aec-4ac6-9c46-bf65df939db6 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:10.849218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd7deb-0aec-4ac6-9c46-bf65df939db6 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:10.849730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd7deb-0aec-4ac6-9c46-bf65df939db6 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:10.852808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd7deb-0aec-4ac6-9c46-bf65df939db6 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:10.856645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd7deb-0aec-4ac6-9c46-bf65df939db6 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:10.862553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-d6cd7deb-0aec-4ac6-9c46-bf65df939db6 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] be50ad6b14b342a9b9347d2bf643b250 does not imply 7d0cc3f88e39496d87c9fed6401b9be5.: keystone.exception.ImpliedRoleNotFound: be50ad6b14b342a9b9347d2bf643b250 does not imply 7d0cc3f88e39496d87c9fed6401b9be5. Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = query.one() Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._iter().one() Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._only_one_row( Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exc.NoResultFound( Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:10:10.874900 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Aug 20 21:10:10.876195 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Aug 20 21:10:10.878190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Aug 20 21:10:10.878190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Aug 20 21:10:10.878190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Aug 20 21:10:10.878190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: be50ad6b14b342a9b9347d2bf643b250 does not imply 7d0cc3f88e39496d87c9fed6401b9be5. Aug 20 21:10:10.878190 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:10:10.881345 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2473/4894] 10.222.0.23 () {58 vars in 1347 bytes} [Sat Aug 20 21:10:10 2022] DELETE /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5 => generated 137 bytes in 74 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:10:10.885867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd7deb-0aec-4ac6-9c46-bf65df939db6 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:10.904138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd7deb-0aec-4ac6-9c46-bf65df939db6 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:10.913543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070d72d-be7e-4130-9495-f962e95ee18c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:10.915597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070d72d-be7e-4130-9495-f962e95ee18c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:10.917316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070d72d-be7e-4130-9495-f962e95ee18c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:10.917822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070d72d-be7e-4130-9495-f962e95ee18c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:10.922018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070d72d-be7e-4130-9495-f962e95ee18c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:10.927548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070d72d-be7e-4130-9495-f962e95ee18c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:10.954228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a2997-0da2-4b5c-b018-8baa47122f8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:10.956716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a2997-0da2-4b5c-b018-8baa47122f8e service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:10.973857 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2474/4895] 10.222.0.23 () {60 vars in 1363 bytes} [Sat Aug 20 21:10:10 2022] PUT /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5 => generated 521 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:10.978649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070d72d-be7e-4130-9495-f962e95ee18c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:10.980711 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2422/4896] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:11.067197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070d72d-be7e-4130-9495-f962e95ee18c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:11.074931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f4791-79e5-4fc3-867c-f64740e0ca93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:11.075710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f4791-79e5-4fc3-867c-f64740e0ca93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:11.075982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f4791-79e5-4fc3-867c-f64740e0ca93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:11.076250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f4791-79e5-4fc3-867c-f64740e0ca93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/e14738b703a5459d817ddc87891bc9d8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:11.078362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f4791-79e5-4fc3-867c-f64740e0ca93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=e14738b703a5459d817ddc87891bc9d8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:11.081461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f4791-79e5-4fc3-867c-f64740e0ca93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:11.125874 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2475/4897] 10.222.0.23 () {60 vars in 1363 bytes} [Sat Aug 20 21:10:10 2022] PUT /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/e14738b703a5459d817ddc87891bc9d8 => generated 521 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:11.126779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a2997-0da2-4b5c-b018-8baa47122f8e service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:11.142701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a2997-0da2-4b5c-b018-8baa47122f8e service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:11.214285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ec6d1-49bf-4a74-b0ef-9d6853ca7960 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:11.214948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ec6d1-49bf-4a74-b0ef-9d6853ca7960 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:11.215240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ec6d1-49bf-4a74-b0ef-9d6853ca7960 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:11.215485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ec6d1-49bf-4a74-b0ef-9d6853ca7960 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/e14738b703a5459d817ddc87891bc9d8/implies/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:11.217206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ec6d1-49bf-4a74-b0ef-9d6853ca7960 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=e14738b703a5459d817ddc87891bc9d8, implied_role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:11.218237 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-b46ec6d1-49bf-4a74-b0ef-9d6853ca7960 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:11.218237 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:11.218237 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:11.219143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ec6d1-49bf-4a74-b0ef-9d6853ca7960 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:11.254504 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2423/4898] 10.222.0.23 () {60 vars in 1363 bytes} [Sat Aug 20 21:10:11 2022] PUT /identity/v3/roles/e14738b703a5459d817ddc87891bc9d8/implies/bd889ce606ae4308bb72b30879315f83 => generated 521 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:11.258458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f4791-79e5-4fc3-867c-f64740e0ca93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:11.269891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f4791-79e5-4fc3-867c-f64740e0ca93 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:11.343687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5184d-adf3-4615-9cc8-5af010fe3228 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:11.344320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5184d-adf3-4615-9cc8-5af010fe3228 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:11.344649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5184d-adf3-4615-9cc8-5af010fe3228 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:11.344917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5184d-adf3-4615-9cc8-5af010fe3228 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/role_inferences` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:11.345487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5184d-adf3-4615-9cc8-5af010fe3228 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_role_inference_rules()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:11.346556 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-56c5184d-adf3-4615-9cc8-5af010fe3228 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_inference_rules": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:11.346556 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:11.346556 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:11.347776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5184d-adf3-4615-9cc8-5af010fe3228 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:11.392672 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2476/4899] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Aug 20 21:10:11 2022] GET /identity/v3/role_inferences => generated 1608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:11.393525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ec6d1-49bf-4a74-b0ef-9d6853ca7960 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:11.406624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ec6d1-49bf-4a74-b0ef-9d6853ca7960 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:11.411771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd538c0-308c-44f1-b2ed-e231f943e199 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:11.412948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd538c0-308c-44f1-b2ed-e231f943e199 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:11.413381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd538c0-308c-44f1-b2ed-e231f943e199 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:11.413757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd538c0-308c-44f1-b2ed-e231f943e199 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/e14738b703a5459d817ddc87891bc9d8/implies/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:11.415979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd538c0-308c-44f1-b2ed-e231f943e199 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=e14738b703a5459d817ddc87891bc9d8, implied_role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:11.417728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd538c0-308c-44f1-b2ed-e231f943e199 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:11.437658 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2424/4900] 10.222.0.23 () {58 vars in 1347 bytes} [Sat Aug 20 21:10:11 2022] DELETE /identity/v3/roles/e14738b703a5459d817ddc87891bc9d8/implies/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:11.439019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5184d-adf3-4615-9cc8-5af010fe3228 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:11.452357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5184d-adf3-4615-9cc8-5af010fe3228 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:11.566548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:11.594244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e48f0f-a2d3-4887-b794-d02b24ccc975 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:11.594836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e48f0f-a2d3-4887-b794-d02b24ccc975 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:11.595204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e48f0f-a2d3-4887-b794-d02b24ccc975 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:11.595569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e48f0f-a2d3-4887-b794-d02b24ccc975 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/e14738b703a5459d817ddc87891bc9d8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:11.598758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e48f0f-a2d3-4887-b794-d02b24ccc975 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=e14738b703a5459d817ddc87891bc9d8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:11.600621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e48f0f-a2d3-4887-b794-d02b24ccc975 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:11.616988 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2477/4901] 10.222.0.23 () {58 vars in 1347 bytes} [Sat Aug 20 21:10:11 2022] DELETE /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/e14738b703a5459d817ddc87891bc9d8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:11.621837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd538c0-308c-44f1-b2ed-e231f943e199 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:11.648984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd538c0-308c-44f1-b2ed-e231f943e199 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:11.679330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:11.743745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0479b-12ea-4c45-a99e-b4b5b496a529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:11.745666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0479b-12ea-4c45-a99e-b4b5b496a529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:11.745964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0479b-12ea-4c45-a99e-b4b5b496a529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:11.746241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0479b-12ea-4c45-a99e-b4b5b496a529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:11.748323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0479b-12ea-4c45-a99e-b4b5b496a529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:11.750101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0479b-12ea-4c45-a99e-b4b5b496a529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:11.773404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e48f0f-a2d3-4887-b794-d02b24ccc975 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:11.776460 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2425/4902] 10.222.0.23 () {58 vars in 1347 bytes} [Sat Aug 20 21:10:11 2022] DELETE /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:11.788193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e48f0f-a2d3-4887-b794-d02b24ccc975 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:11.920171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd3ad9-ad98-4ee1-812f-92d98b479a68 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:11.921185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd3ad9-ad98-4ee1-812f-92d98b479a68 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:11.921619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd3ad9-ad98-4ee1-812f-92d98b479a68 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:11.922015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd3ad9-ad98-4ee1-812f-92d98b479a68 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:11.922857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd3ad9-ad98-4ee1-812f-92d98b479a68 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:11.924878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd3ad9-ad98-4ee1-812f-92d98b479a68 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:11.961215 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2478/4903] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:11 2022] GET /identity/v3/roles => generated 2324 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:11.966723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0479b-12ea-4c45-a99e-b4b5b496a529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:11.978449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0479b-12ea-4c45-a99e-b4b5b496a529 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:11.984433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb978732-4061-4eb4-a39e-a96f66267512 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:11.985174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb978732-4061-4eb4-a39e-a96f66267512 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:11.985544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb978732-4061-4eb4-a39e-a96f66267512 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:11.985925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb978732-4061-4eb4-a39e-a96f66267512 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:11.986686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb978732-4061-4eb4-a39e-a96f66267512 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:11.988468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb978732-4061-4eb4-a39e-a96f66267512 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:12.009808 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2426/4904] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:10:11 2022] POST /identity/v3/roles => generated 234 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:12.014344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd3ad9-ad98-4ee1-812f-92d98b479a68 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:12.034683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd3ad9-ad98-4ee1-812f-92d98b479a68 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:12.044191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e520a8-e337-4112-bd46-c891e91bd4dd tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:12.045493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e520a8-e337-4112-bd46-c891e91bd4dd tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:12.045705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e520a8-e337-4112-bd46-c891e91bd4dd tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:12.046168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e520a8-e337-4112-bd46-c891e91bd4dd tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/5e030d227a714c1c80f207f039725782` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:12.047910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e520a8-e337-4112-bd46-c891e91bd4dd tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:update_role(role_id=5e030d227a714c1c80f207f039725782)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:12.049249 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-d4e520a8-e337-4112-bd46-c891e91bd4dd tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:12.049249 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:12.049249 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:12.050391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e520a8-e337-4112-bd46-c891e91bd4dd tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:12.154669 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2479/4905] 10.222.0.23 () {60 vars in 1243 bytes} [Sat Aug 20 21:10:12 2022] PATCH /identity/v3/roles/5e030d227a714c1c80f207f039725782 => generated 237 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:12.156482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb978732-4061-4eb4-a39e-a96f66267512 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:12.184210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb978732-4061-4eb4-a39e-a96f66267512 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:12.204752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc41ea-ca41-41ff-a436-c87d837459e7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:12.205731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc41ea-ca41-41ff-a436-c87d837459e7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:12.206160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc41ea-ca41-41ff-a436-c87d837459e7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:12.206594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc41ea-ca41-41ff-a436-c87d837459e7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/5e030d227a714c1c80f207f039725782` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:12.225102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc41ea-ca41-41ff-a436-c87d837459e7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:get_role(role_id=5e030d227a714c1c80f207f039725782)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:12.226751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc41ea-ca41-41ff-a436-c87d837459e7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:12.229712 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2427/4906] 10.222.0.23 () {58 vars in 1221 bytes} [Sat Aug 20 21:10:12 2022] GET /identity/v3/roles/5e030d227a714c1c80f207f039725782 => generated 237 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:10:12.234036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e520a8-e337-4112-bd46-c891e91bd4dd tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:12.248990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e520a8-e337-4112-bd46-c891e91bd4dd tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:12.261153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8cc10-801e-47f8-8dbb-3f8be19e4f66 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:12.261980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8cc10-801e-47f8-8dbb-3f8be19e4f66 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:12.262264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8cc10-801e-47f8-8dbb-3f8be19e4f66 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:12.262608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8cc10-801e-47f8-8dbb-3f8be19e4f66 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:12.263411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8cc10-801e-47f8-8dbb-3f8be19e4f66 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:12.265562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8cc10-801e-47f8-8dbb-3f8be19e4f66 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:12.286353 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2480/4907] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:12 2022] GET /identity/v3/roles => generated 2552 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:12.295539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc41ea-ca41-41ff-a436-c87d837459e7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:12.313710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc41ea-ca41-41ff-a436-c87d837459e7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:12.327000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a295-e38f-46e6-863a-caae4c0e2eab tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:12.327733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a295-e38f-46e6-863a-caae4c0e2eab tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:12.327955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a295-e38f-46e6-863a-caae4c0e2eab tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:12.328205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a295-e38f-46e6-863a-caae4c0e2eab tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/5e030d227a714c1c80f207f039725782` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:12.341798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a295-e38f-46e6-863a-caae4c0e2eab tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_role(role_id=5e030d227a714c1c80f207f039725782)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:12.349100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a295-e38f-46e6-863a-caae4c0e2eab tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:12.422584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-83e6a295-e38f-46e6-863a-caae4c0e2eab tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role 5e030d227a714c1c80f207f039725782 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:12.423687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-83e6a295-e38f-46e6-863a-caae4c0e2eab tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:12.424585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-83e6a295-e38f-46e6-863a-caae4c0e2eab tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:12.428035 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2428/4908] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:12 2022] DELETE /identity/v3/roles/5e030d227a714c1c80f207f039725782 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:12.433969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8cc10-801e-47f8-8dbb-3f8be19e4f66 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:12.435741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-52a8cc10-801e-47f8-8dbb-3f8be19e4f66 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:12.467770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8cc10-801e-47f8-8dbb-3f8be19e4f66 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:12.624157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51599ea5-ea18-4228-b9ee-e6597688567c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:12.624554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51599ea5-ea18-4228-b9ee-e6597688567c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:12.624832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51599ea5-ea18-4228-b9ee-e6597688567c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:12.625101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51599ea5-ea18-4228-b9ee-e6597688567c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:12.627372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51599ea5-ea18-4228-b9ee-e6597688567c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:12.630235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51599ea5-ea18-4228-b9ee-e6597688567c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:12.698869 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2481/4909] 10.222.0.23 () {60 vars in 1363 bytes} [Sat Aug 20 21:10:12 2022] PUT /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5 => generated 521 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:12.703625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a295-e38f-46e6-863a-caae4c0e2eab tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:12.722853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a295-e38f-46e6-863a-caae4c0e2eab tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:12.849021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c6990-3702-445a-a8dc-e7bf8f4dcce7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:12.849764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c6990-3702-445a-a8dc-e7bf8f4dcce7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:12.849957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c6990-3702-445a-a8dc-e7bf8f4dcce7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:12.850221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c6990-3702-445a-a8dc-e7bf8f4dcce7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/e14738b703a5459d817ddc87891bc9d8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:12.860700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c6990-3702-445a-a8dc-e7bf8f4dcce7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=e14738b703a5459d817ddc87891bc9d8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:12.862533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c6990-3702-445a-a8dc-e7bf8f4dcce7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:12.911055 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2429/4910] 10.222.0.23 () {60 vars in 1363 bytes} [Sat Aug 20 21:10:12 2022] PUT /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/e14738b703a5459d817ddc87891bc9d8 => generated 521 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:12.911845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51599ea5-ea18-4228-b9ee-e6597688567c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:12.927094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51599ea5-ea18-4228-b9ee-e6597688567c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:12.981609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:13.041042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d982a-f3e8-4972-ba6e-bf14d9168090 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:13.042133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d982a-f3e8-4972-ba6e-bf14d9168090 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:13.043032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d982a-f3e8-4972-ba6e-bf14d9168090 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:13.043499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d982a-f3e8-4972-ba6e-bf14d9168090 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/e14738b703a5459d817ddc87891bc9d8/implies/bd889ce606ae4308bb72b30879315f83` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:13.045617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d982a-f3e8-4972-ba6e-bf14d9168090 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=e14738b703a5459d817ddc87891bc9d8, implied_role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:13.047632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d982a-f3e8-4972-ba6e-bf14d9168090 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:13.093804 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2482/4911] 10.222.0.23 () {60 vars in 1363 bytes} [Sat Aug 20 21:10:12 2022] PUT /identity/v3/roles/e14738b703a5459d817ddc87891bc9d8/implies/bd889ce606ae4308bb72b30879315f83 => generated 521 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:13.097575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c6990-3702-445a-a8dc-e7bf8f4dcce7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:13.119084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c6990-3702-445a-a8dc-e7bf8f4dcce7 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:13.172930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:13.225171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32601f-6f1c-454e-a149-25993c84167d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:13.225762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32601f-6f1c-454e-a149-25993c84167d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:13.226136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32601f-6f1c-454e-a149-25993c84167d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:13.226579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32601f-6f1c-454e-a149-25993c84167d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/e14738b703a5459d817ddc87891bc9d8/implies` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:13.227928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32601f-6f1c-454e-a149-25993c84167d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=e14738b703a5459d817ddc87891bc9d8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:13.229181 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-7f32601f-6f1c-454e-a149-25993c84167d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:13.229181 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:10:13.229181 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:10:13.230740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32601f-6f1c-454e-a149-25993c84167d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:13.242401 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2430/4912] 10.222.0.23 () {58 vars in 1245 bytes} [Sat Aug 20 21:10:13 2022] GET /identity/v3/roles/e14738b703a5459d817ddc87891bc9d8/implies => generated 490 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:10:13.244737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d982a-f3e8-4972-ba6e-bf14d9168090 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:13.259573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d982a-f3e8-4972-ba6e-bf14d9168090 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:13.265545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b39f8-7999-4fd7-a34e-5cf08ebcb6e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:13.266348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b39f8-7999-4fd7-a34e-5cf08ebcb6e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:13.266684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b39f8-7999-4fd7-a34e-5cf08ebcb6e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:13.267224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b39f8-7999-4fd7-a34e-5cf08ebcb6e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:13.269059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b39f8-7999-4fd7-a34e-5cf08ebcb6e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=be50ad6b14b342a9b9347d2bf643b250)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:13.270047 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-825b39f8-7999-4fd7-a34e-5cf08ebcb6e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:10:13.270047 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:10:13.270047 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:10:13.272118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b39f8-7999-4fd7-a34e-5cf08ebcb6e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:13.282286 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2483/4913] 10.222.0.23 () {58 vars in 1245 bytes} [Sat Aug 20 21:10:13 2022] GET /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies => generated 661 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:13.285972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32601f-6f1c-454e-a149-25993c84167d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:13.309011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32601f-6f1c-454e-a149-25993c84167d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:13.314379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5afd2-49f3-4086-af65-07bf24e28be3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:13.314379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5afd2-49f3-4086-af65-07bf24e28be3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:13.314770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5afd2-49f3-4086-af65-07bf24e28be3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:13.315277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5afd2-49f3-4086-af65-07bf24e28be3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/e14738b703a5459d817ddc87891bc9d8/implies/bd889ce606ae4308bb72b30879315f83` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:13.319102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5afd2-49f3-4086-af65-07bf24e28be3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=e14738b703a5459d817ddc87891bc9d8, implied_role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:13.321748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5afd2-49f3-4086-af65-07bf24e28be3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:13.370039 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2431/4914] 10.222.0.23 () {58 vars in 1347 bytes} [Sat Aug 20 21:10:13 2022] DELETE /identity/v3/roles/e14738b703a5459d817ddc87891bc9d8/implies/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:13.374053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b39f8-7999-4fd7-a34e-5cf08ebcb6e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:13.390581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b39f8-7999-4fd7-a34e-5cf08ebcb6e9 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:13.462927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cb76e-a8e7-4dd0-831e-a5b8cfd1e753 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:13.463835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cb76e-a8e7-4dd0-831e-a5b8cfd1e753 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:13.464162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cb76e-a8e7-4dd0-831e-a5b8cfd1e753 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:13.464582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cb76e-a8e7-4dd0-831e-a5b8cfd1e753 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/e14738b703a5459d817ddc87891bc9d8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:13.466855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cb76e-a8e7-4dd0-831e-a5b8cfd1e753 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=e14738b703a5459d817ddc87891bc9d8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:13.468793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cb76e-a8e7-4dd0-831e-a5b8cfd1e753 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:13.483456 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2484/4915] 10.222.0.23 () {58 vars in 1347 bytes} [Sat Aug 20 21:10:13 2022] DELETE /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/e14738b703a5459d817ddc87891bc9d8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:13.487902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5afd2-49f3-4086-af65-07bf24e28be3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:13.506797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5afd2-49f3-4086-af65-07bf24e28be3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:13.616887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ac544-07da-443d-9133-211ce315a8ac tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:13.617839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ac544-07da-443d-9133-211ce315a8ac tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:13.618117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ac544-07da-443d-9133-211ce315a8ac tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:13.618385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ac544-07da-443d-9133-211ce315a8ac tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:13.623608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ac544-07da-443d-9133-211ce315a8ac tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=be50ad6b14b342a9b9347d2bf643b250, implied_role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:13.625293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ac544-07da-443d-9133-211ce315a8ac tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:13.653972 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2432/4916] 10.222.0.23 () {58 vars in 1347 bytes} [Sat Aug 20 21:10:13 2022] DELETE /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250/implies/7d0cc3f88e39496d87c9fed6401b9be5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:13.659666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cb76e-a8e7-4dd0-831e-a5b8cfd1e753 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:13.677337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cb76e-a8e7-4dd0-831e-a5b8cfd1e753 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:13.856278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ca438-17a8-462f-abc0-a8db74e8b56b tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:13.857390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ca438-17a8-462f-abc0-a8db74e8b56b tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:13.857663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ca438-17a8-462f-abc0-a8db74e8b56b tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:13.857976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ca438-17a8-462f-abc0-a8db74e8b56b tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains/f42d71442931441d9bbd0fd41e362063` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:13.877099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ca438-17a8-462f-abc0-a8db74e8b56b tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-e85ca438-17a8-462f-abc0-a8db74e8b56b tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Could not find domain: f42d71442931441d9bbd0fd41e362063.: keystone.exception.DomainNotFound: Could not find domain: f42d71442931441d9bbd0fd41e362063. Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f42d71442931441d9bbd0fd41e362063. Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:10:13.886351 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:10:13.887864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: f42d71442931441d9bbd0fd41e362063. Aug 20 21:10:13.889350 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:10:13.895621 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2485/4917] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:10:13 2022] PATCH /identity/v3/domains/f42d71442931441d9bbd0fd41e362063 => generated 112 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:10:13.896101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ac544-07da-443d-9133-211ce315a8ac tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:13.911575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ac544-07da-443d-9133-211ce315a8ac tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:13.918269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7a92d-fb3c-4ac9-9030-e842d4c8fd2d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:13.918269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7a92d-fb3c-4ac9-9030-e842d4c8fd2d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:13.918615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7a92d-fb3c-4ac9-9030-e842d4c8fd2d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:13.918810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7a92d-fb3c-4ac9-9030-e842d4c8fd2d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/users/8401b16d587845afa1487456fee0bfaf` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:13.920127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7a92d-fb3c-4ac9-9030-e842d4c8fd2d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_user(user_id=8401b16d587845afa1487456fee0bfaf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:13.922631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7a92d-fb3c-4ac9-9030-e842d4c8fd2d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:14.001232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bde7a92d-fb3c-4ac9-9030-e842d4c8fd2d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8401b16d587845afa1487456fee0bfaf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:14.019405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bde7a92d-fb3c-4ac9-9030-e842d4c8fd2d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8401b16d587845afa1487456fee0bfaf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:14.062445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bde7a92d-fb3c-4ac9-9030-e842d4c8fd2d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8401b16d587845afa1487456fee0bfaf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:14.063282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bde7a92d-fb3c-4ac9-9030-e842d4c8fd2d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8401b16d587845afa1487456fee0bfaf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:14.063890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bde7a92d-fb3c-4ac9-9030-e842d4c8fd2d tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8401b16d587845afa1487456fee0bfaf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:14.091621 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2433/4918] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:13 2022] DELETE /identity/v3/users/8401b16d587845afa1487456fee0bfaf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:14.097224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ca438-17a8-462f-abc0-a8db74e8b56b tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:14.101275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e85ca438-17a8-462f-abc0-a8db74e8b56b tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:14.141054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ca438-17a8-462f-abc0-a8db74e8b56b tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:14.186606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:14.197007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22dcd8-05ba-47c2-aa68-112a2ccd033a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:14.197373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22dcd8-05ba-47c2-aa68-112a2ccd033a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:14.197694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22dcd8-05ba-47c2-aa68-112a2ccd033a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:14.245237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2c22dcd8-05ba-47c2-aa68-112a2ccd033a None None] MFA Rules not processed for user `1cf36cef64b14630850a3fc3d2974e79`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:14.246717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2c22dcd8-05ba-47c2-aa68-112a2ccd033a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:14.287728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:14.303226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3541f3f-bf5f-4945-b632-c83304812a7c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:14.304003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3541f3f-bf5f-4945-b632-c83304812a7c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:14.304327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3541f3f-bf5f-4945-b632-c83304812a7c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:14.304800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3541f3f-bf5f-4945-b632-c83304812a7c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/bd889ce606ae4308bb72b30879315f83` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:14.307179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3541f3f-bf5f-4945-b632-c83304812a7c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_role(role_id=bd889ce606ae4308bb72b30879315f83)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:14.313044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3541f3f-bf5f-4945-b632-c83304812a7c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:14.388882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c3541f3f-bf5f-4945-b632-c83304812a7c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role bd889ce606ae4308bb72b30879315f83 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:14.389179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c3541f3f-bf5f-4945-b632-c83304812a7c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:14.389890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c3541f3f-bf5f-4945-b632-c83304812a7c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:14.397447 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2486/4919] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:14 2022] DELETE /identity/v3/roles/bd889ce606ae4308bb72b30879315f83 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:14.398452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3541f3f-bf5f-4945-b632-c83304812a7c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:14.401658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c3541f3f-bf5f-4945-b632-c83304812a7c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:14.403195 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2434/4920] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:14.411469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b1475-7144-495e-8c99-5902d3b9b248 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:14.422542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b1475-7144-495e-8c99-5902d3b9b248 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:14.422959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b1475-7144-495e-8c99-5902d3b9b248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:14.454784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-702b1475-7144-495e-8c99-5902d3b9b248 None None] MFA Rules not processed for user `1cf36cef64b14630850a3fc3d2974e79`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:14.456096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-702b1475-7144-495e-8c99-5902d3b9b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:14.527626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3541f3f-bf5f-4945-b632-c83304812a7c tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:14.542617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236f2c9-05bc-4515-8efb-0c14e07bef3a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:14.543577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236f2c9-05bc-4515-8efb-0c14e07bef3a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:14.544204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236f2c9-05bc-4515-8efb-0c14e07bef3a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:14.545590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236f2c9-05bc-4515-8efb-0c14e07bef3a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/groups/913a950456c24a9fa04fb93a0de83b04` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:14.547948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236f2c9-05bc-4515-8efb-0c14e07bef3a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_group(group_id=913a950456c24a9fa04fb93a0de83b04)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:14.550486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236f2c9-05bc-4515-8efb-0c14e07bef3a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:14.553828 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2435/4921] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:14.569016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b1475-7144-495e-8c99-5902d3b9b248 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:14.572319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-702b1475-7144-495e-8c99-5902d3b9b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:14.637644 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2487/4922] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:10:14 2022] DELETE /identity/v3/groups/913a950456c24a9fa04fb93a0de83b04 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:14.646384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236f2c9-05bc-4515-8efb-0c14e07bef3a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:14.706079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b1475-7144-495e-8c99-5902d3b9b248 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:14.734230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:14.744270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236f2c9-05bc-4515-8efb-0c14e07bef3a tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:14.786713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:14.791466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:14.791778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:14.792025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/projects/3e37435363bb4abc9199670f3780c04c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:14.793791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3e37435363bb4abc9199670f3780c04c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:14.796341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:14.819532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:14.838641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b263d3-fb46-439d-9175-89920c04e2c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:14.839549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b263d3-fb46-439d-9175-89920c04e2c2 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:14.839919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b263d3-fb46-439d-9175-89920c04e2c2 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:14.840278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b263d3-fb46-439d-9175-89920c04e2c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:14.846992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-48b263d3-fb46-439d-9175-89920c04e2c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:14.913739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e37435363bb4abc9199670f3780c04c', 'request_id': 'req-d56b7134-fe65-4d46-863c-0e2a341352e0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:14.928625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e37435363bb4abc9199670f3780c04c', 'request_id': 'req-d56b7134-fe65-4d46-863c-0e2a341352e0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:14.937471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e37435363bb4abc9199670f3780c04c', 'request_id': 'req-d56b7134-fe65-4d46-863c-0e2a341352e0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:14.954504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] The token cache is being invalidate because project 3e37435363bb4abc9199670f3780c04c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:14.954860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:14.955628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:14.960414 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2488/4923] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:14 2022] DELETE /identity/v3/projects/3e37435363bb4abc9199670f3780c04c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:14.961941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:14.965770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:14.970344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b263d3-fb46-439d-9175-89920c04e2c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:14.975553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b263d3-fb46-439d-9175-89920c04e2c2 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:14.999681 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2436/4924] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:15.142006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b7134-fe65-4d46-863c-0e2a341352e0 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:15.153357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91dfd4-9788-45ae-905e-909449697fe3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:15.154920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91dfd4-9788-45ae-905e-909449697fe3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:15.155515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91dfd4-9788-45ae-905e-909449697fe3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:15.155945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91dfd4-9788-45ae-905e-909449697fe3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains/7742da72f13d416aa9ec51962a89d55f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:15.158189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91dfd4-9788-45ae-905e-909449697fe3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=7742da72f13d416aa9ec51962a89d55f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:15.160043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91dfd4-9788-45ae-905e-909449697fe3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:15.202316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e91dfd4-9788-45ae-905e-909449697fe3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '7742da72f13d416aa9ec51962a89d55f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.203241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e91dfd4-9788-45ae-905e-909449697fe3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '7742da72f13d416aa9ec51962a89d55f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.253265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e91dfd4-9788-45ae-905e-909449697fe3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '7742da72f13d416aa9ec51962a89d55f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.254812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e91dfd4-9788-45ae-905e-909449697fe3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '7742da72f13d416aa9ec51962a89d55f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.260393 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2489/4925] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:10:14 2022] PATCH /identity/v3/domains/7742da72f13d416aa9ec51962a89d55f => generated 276 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:15.261281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b263d3-fb46-439d-9175-89920c04e2c2 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:15.276772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b263d3-fb46-439d-9175-89920c04e2c2 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:15.343446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:15.344100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:15.344667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:15.344941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/domains/7742da72f13d416aa9ec51962a89d55f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:15.357867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=7742da72f13d416aa9ec51962a89d55f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:15.359512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:15.404275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '7742da72f13d416aa9ec51962a89d55f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.408728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '7742da72f13d416aa9ec51962a89d55f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.524482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7742da72f13d416aa9ec51962a89d55f', 'request_id': 'req-eb5a9461-06a2-48b3-bed0-b7454c0e227e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.538789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7742da72f13d416aa9ec51962a89d55f', 'request_id': 'req-eb5a9461-06a2-48b3-bed0-b7454c0e227e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.554788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7742da72f13d416aa9ec51962a89d55f', 'request_id': 'req-eb5a9461-06a2-48b3-bed0-b7454c0e227e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.565622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:15.598511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] The token cache is being invalidate because project 7742da72f13d416aa9ec51962a89d55f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:15.599613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.600594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.618727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '7742da72f13d416aa9ec51962a89d55f', 'request_id': 'req-eb5a9461-06a2-48b3-bed0-b7454c0e227e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.619478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '7742da72f13d416aa9ec51962a89d55f', 'request_id': 'req-eb5a9461-06a2-48b3-bed0-b7454c0e227e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.623942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '7742da72f13d416aa9ec51962a89d55f', 'request_id': 'req-eb5a9461-06a2-48b3-bed0-b7454c0e227e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.634626 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2437/4926] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:10:15 2022] DELETE /identity/v3/domains/7742da72f13d416aa9ec51962a89d55f => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:15.639919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91dfd4-9788-45ae-905e-909449697fe3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:15.641369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8e91dfd4-9788-45ae-905e-909449697fe3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:15.656363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91dfd4-9788-45ae-905e-909449697fe3 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:15.719855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:15.740144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737ffcd-bb0a-45ba-8ee0-357883fa49de tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:15.741079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737ffcd-bb0a-45ba-8ee0-357883fa49de tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:15.741578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737ffcd-bb0a-45ba-8ee0-357883fa49de tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:15.742622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737ffcd-bb0a-45ba-8ee0-357883fa49de tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/e14738b703a5459d817ddc87891bc9d8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:15.746049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737ffcd-bb0a-45ba-8ee0-357883fa49de tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_role(role_id=e14738b703a5459d817ddc87891bc9d8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:15.749563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737ffcd-bb0a-45ba-8ee0-357883fa49de tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:15.807711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b737ffcd-bb0a-45ba-8ee0-357883fa49de tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role e14738b703a5459d817ddc87891bc9d8 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:15.808090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b737ffcd-bb0a-45ba-8ee0-357883fa49de tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.809228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b737ffcd-bb0a-45ba-8ee0-357883fa49de tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:15.812717 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2490/4927] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:15 2022] DELETE /identity/v3/roles/e14738b703a5459d817ddc87891bc9d8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:15.817289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:15.819172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:15.843766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a9461-06a2-48b3-bed0-b7454c0e227e tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:15.969869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb974ad-9430-4e81-8af6-aa06a45071f2 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:15.970667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb974ad-9430-4e81-8af6-aa06a45071f2 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:15.971318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb974ad-9430-4e81-8af6-aa06a45071f2 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:15.971318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb974ad-9430-4e81-8af6-aa06a45071f2 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/7d0cc3f88e39496d87c9fed6401b9be5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:15.974122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb974ad-9430-4e81-8af6-aa06a45071f2 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_role(role_id=7d0cc3f88e39496d87c9fed6401b9be5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:15.975861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb974ad-9430-4e81-8af6-aa06a45071f2 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:16.058706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecb974ad-9430-4e81-8af6-aa06a45071f2 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role 7d0cc3f88e39496d87c9fed6401b9be5 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:16.059362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecb974ad-9430-4e81-8af6-aa06a45071f2 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:16.060708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecb974ad-9430-4e81-8af6-aa06a45071f2 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:16.071439 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2438/4928] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:15 2022] DELETE /identity/v3/roles/7d0cc3f88e39496d87c9fed6401b9be5 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:16.073382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737ffcd-bb0a-45ba-8ee0-357883fa49de tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:16.076207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b737ffcd-bb0a-45ba-8ee0-357883fa49de tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:16.109554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737ffcd-bb0a-45ba-8ee0-357883fa49de tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:16.271170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a2b01-4e9b-4cfd-96d0-67868cba163f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5adde001844b4eaa9f0908a116077501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76334a932f3346aaae842d51c8f9a139', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:16.272226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a2b01-4e9b-4cfd-96d0-67868cba163f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:16.272774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a2b01-4e9b-4cfd-96d0-67868cba163f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:16.273496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a2b01-4e9b-4cfd-96d0-67868cba163f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] PATH_INFO: `/v3/roles/be50ad6b14b342a9b9347d2bf643b250` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:16.276006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a2b01-4e9b-4cfd-96d0-67868cba163f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorizing `identity:delete_role(role_id=be50ad6b14b342a9b9347d2bf643b250)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:16.278570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a2b01-4e9b-4cfd-96d0-67868cba163f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:16.358175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-697a2b01-4e9b-4cfd-96d0-67868cba163f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invalidating the token cache because role be50ad6b14b342a9b9347d2bf643b250 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:16.358770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-697a2b01-4e9b-4cfd-96d0-67868cba163f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:16.360681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-697a2b01-4e9b-4cfd-96d0-67868cba163f tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:16.368935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb974ad-9430-4e81-8af6-aa06a45071f2 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:16.370750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ecb974ad-9430-4e81-8af6-aa06a45071f2 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:16.380168 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2491/4929] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:16 2022] DELETE /identity/v3/roles/be50ad6b14b342a9b9347d2bf643b250 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:16.546209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb974ad-9430-4e81-8af6-aa06a45071f2 tempest-RolesV3TestJSON-1310279574 tempest-RolesV3TestJSON-1310279574-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:16.618491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:16.644455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:16.645295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:16.645714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:16.645992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] PATH_INFO: `/v3/users/e27a739b6324457e88fbaeace3aa0574` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:16.647775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e27a739b6324457e88fbaeace3aa0574)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:16.652940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:16.737294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e27a739b6324457e88fbaeace3aa0574'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:16.748955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e27a739b6324457e88fbaeace3aa0574'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:16.774477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e27a739b6324457e88fbaeace3aa0574'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:16.775426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e27a739b6324457e88fbaeace3aa0574'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:16.776770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e27a739b6324457e88fbaeace3aa0574'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:16.803212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79f107-0c12-4c43-9c2b-26f9a27c2502 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:16.803482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79f107-0c12-4c43-9c2b-26f9a27c2502 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:16.803977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79f107-0c12-4c43-9c2b-26f9a27c2502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:16.813288 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2439/4930] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:16 2022] DELETE /identity/v3/users/e27a739b6324457e88fbaeace3aa0574 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:16.836272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4c79f107-0c12-4c43-9c2b-26f9a27c2502 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:16.838439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4c79f107-0c12-4c43-9c2b-26f9a27c2502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:16.932416 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2492/4931] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:16.938377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:16.941921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:17.028253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db483a4c-5a28-4a5a-87b6-ed0009a2b2b2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:17.032532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90a02e-a9f5-48aa-9588-b4afa9119eca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:17.033306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90a02e-a9f5-48aa-9588-b4afa9119eca service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:17.033581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90a02e-a9f5-48aa-9588-b4afa9119eca service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:17.033845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90a02e-a9f5-48aa-9588-b4afa9119eca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:17.045125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90a02e-a9f5-48aa-9588-b4afa9119eca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:17.046687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90a02e-a9f5-48aa-9588-b4afa9119eca service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:17.068482 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2440/4932] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:17.566670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79f107-0c12-4c43-9c2b-26f9a27c2502 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:17.568155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4c79f107-0c12-4c43-9c2b-26f9a27c2502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:17.600300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79f107-0c12-4c43-9c2b-26f9a27c2502 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:17.608118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:17.608926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:17.609060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:17.609328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] PATH_INFO: `/v3/projects/f7bc4969ba8f4974a03c71244f63bb18` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:17.610461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7bc4969ba8f4974a03c71244f63bb18)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:17.613025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:17.637776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:17.685127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7bc4969ba8f4974a03c71244f63bb18', 'request_id': 'req-81f76192-524b-4626-a53e-2a4fd5d7da75'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:17.696235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7bc4969ba8f4974a03c71244f63bb18', 'request_id': 'req-81f76192-524b-4626-a53e-2a4fd5d7da75'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:17.703322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7bc4969ba8f4974a03c71244f63bb18', 'request_id': 'req-81f76192-524b-4626-a53e-2a4fd5d7da75'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:17.718384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] The token cache is being invalidate because project f7bc4969ba8f4974a03c71244f63bb18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:17.718384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:17.719128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:17.721334 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2493/4933] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:17 2022] DELETE /identity/v3/projects/f7bc4969ba8f4974a03c71244f63bb18 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:17.725175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90a02e-a9f5-48aa-9588-b4afa9119eca service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:17.736899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90a02e-a9f5-48aa-9588-b4afa9119eca service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:17.804819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:17.806108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:17.806387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:17.806655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] PATH_INFO: `/v3/users/5adde001844b4eaa9f0908a116077501` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:17.808205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5adde001844b4eaa9f0908a116077501)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:17.817974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:17.874993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:17.967940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5adde001844b4eaa9f0908a116077501'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:17.982851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5adde001844b4eaa9f0908a116077501'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:18.011636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5adde001844b4eaa9f0908a116077501'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:18.014048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5adde001844b4eaa9f0908a116077501'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:18.021686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5adde001844b4eaa9f0908a116077501'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:18.077110 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2441/4934] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:17 2022] DELETE /identity/v3/users/5adde001844b4eaa9f0908a116077501 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:19.541238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:19.543428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:19.576694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:19.578852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:19.706445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f76192-524b-4626-a53e-2a4fd5d7da75 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:19.773031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369976e4-6549-40f7-aa65-0b014f0e60bd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:19.807491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ce8bb-f298-4fa3-863d-9a87ce947927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:19.809589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ce8bb-f298-4fa3-863d-9a87ce947927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:19.811420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ce8bb-f298-4fa3-863d-9a87ce947927 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:19.811901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ce8bb-f298-4fa3-863d-9a87ce947927 admin admin] PATH_INFO: `/v3/projects/76334a932f3346aaae842d51c8f9a139` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:19.817211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ce8bb-f298-4fa3-863d-9a87ce947927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76334a932f3346aaae842d51c8f9a139)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:19.820818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ce8bb-f298-4fa3-863d-9a87ce947927 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:19.824998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506c35b-d092-45ad-acbd-e8afa5f36e7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:19.827480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506c35b-d092-45ad-acbd-e8afa5f36e7e service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:19.827842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506c35b-d092-45ad-acbd-e8afa5f36e7e service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:19.828004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506c35b-d092-45ad-acbd-e8afa5f36e7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:19.837180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7506c35b-d092-45ad-acbd-e8afa5f36e7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:19.869736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506c35b-d092-45ad-acbd-e8afa5f36e7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:19.878192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506c35b-d092-45ad-acbd-e8afa5f36e7e service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:19.928782 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2494/4935] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:19.990754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3d4ce8bb-f298-4fa3-863d-9a87ce947927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76334a932f3346aaae842d51c8f9a139', 'request_id': 'req-3d4ce8bb-f298-4fa3-863d-9a87ce947927'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:20.002119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3d4ce8bb-f298-4fa3-863d-9a87ce947927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76334a932f3346aaae842d51c8f9a139', 'request_id': 'req-3d4ce8bb-f298-4fa3-863d-9a87ce947927'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:20.023974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3d4ce8bb-f298-4fa3-863d-9a87ce947927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76334a932f3346aaae842d51c8f9a139', 'request_id': 'req-3d4ce8bb-f298-4fa3-863d-9a87ce947927'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:20.042153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3d4ce8bb-f298-4fa3-863d-9a87ce947927 admin admin] The token cache is being invalidate because project 76334a932f3346aaae842d51c8f9a139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:20.042457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3d4ce8bb-f298-4fa3-863d-9a87ce947927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:20.049712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3d4ce8bb-f298-4fa3-863d-9a87ce947927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:20.052808 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2442/4936] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:19 2022] DELETE /identity/v3/projects/76334a932f3346aaae842d51c8f9a139 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:20.063629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f470f35-7600-40d8-807a-1262a41f4aad None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:20.063629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f470f35-7600-40d8-807a-1262a41f4aad None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:20.063629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f470f35-7600-40d8-807a-1262a41f4aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:20.101049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4f470f35-7600-40d8-807a-1262a41f4aad None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:20.104698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4f470f35-7600-40d8-807a-1262a41f4aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:20.283966 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2495/4937] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:20.290107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07ab93-99b2-4a0c-9713-7ce0bba76c6d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:20.290384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07ab93-99b2-4a0c-9713-7ce0bba76c6d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:20.290650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07ab93-99b2-4a0c-9713-7ce0bba76c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:20.332964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7e07ab93-99b2-4a0c-9713-7ce0bba76c6d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:20.334198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7e07ab93-99b2-4a0c-9713-7ce0bba76c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:20.351895 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2443/4938] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:20.356718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f470f35-7600-40d8-807a-1262a41f4aad None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:20.392055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:20.404610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f470f35-7600-40d8-807a-1262a41f4aad None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:20.415107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea6233-73c2-42e0-8624-cc3c34870a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:20.415973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea6233-73c2-42e0-8624-cc3c34870a54 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:20.416716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea6233-73c2-42e0-8624-cc3c34870a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:20.420365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea6233-73c2-42e0-8624-cc3c34870a54 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:20.428719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea6233-73c2-42e0-8624-cc3c34870a54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:20.429081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea6233-73c2-42e0-8624-cc3c34870a54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:20.431381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea6233-73c2-42e0-8624-cc3c34870a54 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:20.457944 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2496/4939] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:20.464242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07ab93-99b2-4a0c-9713-7ce0bba76c6d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:20.478741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07ab93-99b2-4a0c-9713-7ce0bba76c6d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:20.489207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd07e8-36e0-4149-b2ee-98a3d96bd31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:20.489966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd07e8-36e0-4149-b2ee-98a3d96bd31a admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:20.492721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd07e8-36e0-4149-b2ee-98a3d96bd31a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:20.493098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd07e8-36e0-4149-b2ee-98a3d96bd31a admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:20.494075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd07e8-36e0-4149-b2ee-98a3d96bd31a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:20.496235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd07e8-36e0-4149-b2ee-98a3d96bd31a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:20.522766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:20.576822 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2444/4940] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:20 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:20.581183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea6233-73c2-42e0-8624-cc3c34870a54 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:20.607194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea6233-73c2-42e0-8624-cc3c34870a54 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:20.687205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49dbab-d2e7-4112-8cf7-70f41a571d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:20.688414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49dbab-d2e7-4112-8cf7-70f41a571d57 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:20.688697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49dbab-d2e7-4112-8cf7-70f41a571d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:20.689156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49dbab-d2e7-4112-8cf7-70f41a571d57 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:20.690453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49dbab-d2e7-4112-8cf7-70f41a571d57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:20.692980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49dbab-d2e7-4112-8cf7-70f41a571d57 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:20.720229 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2497/4941] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:20 2022] POST /identity/v3/users => generated 290 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:20.724593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd07e8-36e0-4149-b2ee-98a3d96bd31a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:20.737132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd07e8-36e0-4149-b2ee-98a3d96bd31a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:20.742482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62908453-bcfc-48c7-80dc-99c43da3f230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:20.743097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62908453-bcfc-48c7-80dc-99c43da3f230 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:20.743371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62908453-bcfc-48c7-80dc-99c43da3f230 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:20.743638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62908453-bcfc-48c7-80dc-99c43da3f230 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:20.744281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62908453-bcfc-48c7-80dc-99c43da3f230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:20.746549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62908453-bcfc-48c7-80dc-99c43da3f230 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:20.757698 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2445/4942] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:20.761817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49dbab-d2e7-4112-8cf7-70f41a571d57 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:20.772636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49dbab-d2e7-4112-8cf7-70f41a571d57 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:20.779118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a6cd0-a144-461e-930a-60c84bb7d2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:20.779823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a6cd0-a144-461e-930a-60c84bb7d2ce admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:20.780161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a6cd0-a144-461e-930a-60c84bb7d2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:20.780560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a6cd0-a144-461e-930a-60c84bb7d2ce admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:20.781286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a6cd0-a144-461e-930a-60c84bb7d2ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:20.783036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a6cd0-a144-461e-930a-60c84bb7d2ce admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:20.797886 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2498/4943] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:20.802748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62908453-bcfc-48c7-80dc-99c43da3f230 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:20.814287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62908453-bcfc-48c7-80dc-99c43da3f230 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:20.819667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5f777-d820-4fd0-8ead-cc6f3c286030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:20.820725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5f777-d820-4fd0-8ead-cc6f3c286030 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:20.821001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5f777-d820-4fd0-8ead-cc6f3c286030 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:20.821270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5f777-d820-4fd0-8ead-cc6f3c286030 admin admin] PATH_INFO: `/v3/projects/af7498ca10ac43cd9ffccfbb0cde3f7a/users/762ce176aa7b4e448f388d4d746f6691/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:20.855486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5f777-d820-4fd0-8ead-cc6f3c286030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af7498ca10ac43cd9ffccfbb0cde3f7a, user_id=762ce176aa7b4e448f388d4d746f6691, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:20.862055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5f777-d820-4fd0-8ead-cc6f3c286030 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:20.887598 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2446/4944] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:20 2022] PUT /identity/v3/projects/af7498ca10ac43cd9ffccfbb0cde3f7a/users/762ce176aa7b4e448f388d4d746f6691/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:20.904740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb9d51-b046-4b9d-8389-04bdbd89df2a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:20.905066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb9d51-b046-4b9d-8389-04bdbd89df2a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:20.905341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb9d51-b046-4b9d-8389-04bdbd89df2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:21.054827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-62cb9d51-b046-4b9d-8389-04bdbd89df2a None None] MFA Rules not processed for user `762ce176aa7b4e448f388d4d746f6691`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:21.056678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-62cb9d51-b046-4b9d-8389-04bdbd89df2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:21.211948 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2499/4945] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:21.217240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5f777-d820-4fd0-8ead-cc6f3c286030 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:21.315154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5f777-d820-4fd0-8ead-cc6f3c286030 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:21.327964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91967b-8d53-4cd3-88f9-4a87fe848188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:21.330014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91967b-8d53-4cd3-88f9-4a87fe848188 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:21.330295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91967b-8d53-4cd3-88f9-4a87fe848188 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:21.330551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91967b-8d53-4cd3-88f9-4a87fe848188 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:21.331206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91967b-8d53-4cd3-88f9-4a87fe848188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:21.333234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91967b-8d53-4cd3-88f9-4a87fe848188 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:21.405063 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2447/4946] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:21 2022] POST /identity/v3/projects => generated 379 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:21.409361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb9d51-b046-4b9d-8389-04bdbd89df2a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:21.428228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb9d51-b046-4b9d-8389-04bdbd89df2a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:21.568825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6de901-fb9a-4cd3-9472-e8cbeae1a7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:21.569928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6de901-fb9a-4cd3-9472-e8cbeae1a7b2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:21.570818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6de901-fb9a-4cd3-9472-e8cbeae1a7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:21.571775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6de901-fb9a-4cd3-9472-e8cbeae1a7b2 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:21.575359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6de901-fb9a-4cd3-9472-e8cbeae1a7b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:21.580427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6de901-fb9a-4cd3-9472-e8cbeae1a7b2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:21.606232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:21.633015 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2500/4947] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:21 2022] POST /identity/v3/users => generated 290 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:21.638162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91967b-8d53-4cd3-88f9-4a87fe848188 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:21.672255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91967b-8d53-4cd3-88f9-4a87fe848188 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:21.677587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25faa6b-af88-4de7-8b42-e49f0191be92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:21.678220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25faa6b-af88-4de7-8b42-e49f0191be92 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:21.678495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25faa6b-af88-4de7-8b42-e49f0191be92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:21.678765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25faa6b-af88-4de7-8b42-e49f0191be92 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:21.679553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25faa6b-af88-4de7-8b42-e49f0191be92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:21.681234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25faa6b-af88-4de7-8b42-e49f0191be92 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:21.705868 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2448/4948] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:21.707309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6de901-fb9a-4cd3-9472-e8cbeae1a7b2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:21.722154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6de901-fb9a-4cd3-9472-e8cbeae1a7b2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:21.730087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1664f-82ba-48dc-8c3a-0bc987ffb8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:21.730822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1664f-82ba-48dc-8c3a-0bc987ffb8dc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:21.731277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1664f-82ba-48dc-8c3a-0bc987ffb8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:21.731637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1664f-82ba-48dc-8c3a-0bc987ffb8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:21.732383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd1664f-82ba-48dc-8c3a-0bc987ffb8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:21.734274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd1664f-82ba-48dc-8c3a-0bc987ffb8dc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:21.754853 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2501/4949] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:21.759244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25faa6b-af88-4de7-8b42-e49f0191be92 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:21.770854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1664f-82ba-48dc-8c3a-0bc987ffb8dc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:21.771660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25faa6b-af88-4de7-8b42-e49f0191be92 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:21.773368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bbd1664f-82ba-48dc-8c3a-0bc987ffb8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:21.776761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff492b-8d13-4e1e-951a-210772b6c5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:21.777437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff492b-8d13-4e1e-951a-210772b6c5de admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:21.777721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff492b-8d13-4e1e-951a-210772b6c5de admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:21.778666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff492b-8d13-4e1e-951a-210772b6c5de admin admin] PATH_INFO: `/v3/projects/9fb1f6e1bd5d4d86b44901b94d8f0759/users/025bde8f4c764aa8b69e7cc757d2b159/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:21.799977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff492b-8d13-4e1e-951a-210772b6c5de admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb1f6e1bd5d4d86b44901b94d8f0759, user_id=025bde8f4c764aa8b69e7cc757d2b159, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:21.803335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff492b-8d13-4e1e-951a-210772b6c5de admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:21.837454 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2449/4950] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:21 2022] PUT /identity/v3/projects/9fb1f6e1bd5d4d86b44901b94d8f0759/users/025bde8f4c764aa8b69e7cc757d2b159/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:21.851949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2addbb-abf2-45a3-add9-383f234140ca None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:21.852806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2addbb-abf2-45a3-add9-383f234140ca None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:21.853512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2addbb-abf2-45a3-add9-383f234140ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:21.886029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1664f-82ba-48dc-8c3a-0bc987ffb8dc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:21.903278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1e2addbb-abf2-45a3-add9-383f234140ca None None] MFA Rules not processed for user `025bde8f4c764aa8b69e7cc757d2b159`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:21.905960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1e2addbb-abf2-45a3-add9-383f234140ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:22.043187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ac2ca-39f7-449a-8fab-7ba3e2cff4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:22.043662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ac2ca-39f7-449a-8fab-7ba3e2cff4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:22.044096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ac2ca-39f7-449a-8fab-7ba3e2cff4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:22.044424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ac2ca-39f7-449a-8fab-7ba3e2cff4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:22.049171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-134ac2ca-39f7-449a-8fab-7ba3e2cff4e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:22.073645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ac2ca-39f7-449a-8fab-7ba3e2cff4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:22.075148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ac2ca-39f7-449a-8fab-7ba3e2cff4e5 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:22.119074 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2502/4951] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:22.152263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:22.198735 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2450/4952] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:22.198898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ac2ca-39f7-449a-8fab-7ba3e2cff4e5 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:22.361665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ac2ca-39f7-449a-8fab-7ba3e2cff4e5 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:22.376159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e91969-477e-4928-b11a-3aa3f9d10920 tempest-IdentityV3ProjectsTest-620862354 tempest-IdentityV3ProjectsTest-620862354-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '762ce176aa7b4e448f388d4d746f6691', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af7498ca10ac43cd9ffccfbb0cde3f7a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:22.377302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e91969-477e-4928-b11a-3aa3f9d10920 tempest-IdentityV3ProjectsTest-620862354 tempest-IdentityV3ProjectsTest-620862354-project] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:22.377814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e91969-477e-4928-b11a-3aa3f9d10920 tempest-IdentityV3ProjectsTest-620862354 tempest-IdentityV3ProjectsTest-620862354-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:22.378917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e91969-477e-4928-b11a-3aa3f9d10920 tempest-IdentityV3ProjectsTest-620862354 tempest-IdentityV3ProjectsTest-620862354-project] PATH_INFO: `/v3/users/762ce176aa7b4e448f388d4d746f6691/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:22.381912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e91969-477e-4928-b11a-3aa3f9d10920 tempest-IdentityV3ProjectsTest-620862354 tempest-IdentityV3ProjectsTest-620862354-project] RBAC: Authorizing `identity:list_user_projects(user_id=762ce176aa7b4e448f388d4d746f6691)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:22.386547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e91969-477e-4928-b11a-3aa3f9d10920 tempest-IdentityV3ProjectsTest-620862354 tempest-IdentityV3ProjectsTest-620862354-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:22.485437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55aeb5-9e92-4155-884d-21beaa98188c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:22.485782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55aeb5-9e92-4155-884d-21beaa98188c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:22.485986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55aeb5-9e92-4155-884d-21beaa98188c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:22.492056 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2503/4953] 10.222.0.23 () {58 vars in 1248 bytes} [Sat Aug 20 21:10:22 2022] GET /identity/v3/users/762ce176aa7b4e448f388d4d746f6691/projects => generated 515 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:22.552313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2f55aeb5-9e92-4155-884d-21beaa98188c None None] MFA Rules not processed for user `762ce176aa7b4e448f388d4d746f6691`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:22.556319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2f55aeb5-9e92-4155-884d-21beaa98188c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:22.585022 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2451/4954] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:22.591676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5a087-2a4a-4a62-9e8a-1ba6640d15c9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:22.593598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5a087-2a4a-4a62-9e8a-1ba6640d15c9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:22.593978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5a087-2a4a-4a62-9e8a-1ba6640d15c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:22.623581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:22.643113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0bc5a087-2a4a-4a62-9e8a-1ba6640d15c9 None None] MFA Rules not processed for user `762ce176aa7b4e448f388d4d746f6691`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:22.648469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0bc5a087-2a4a-4a62-9e8a-1ba6640d15c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:22.753057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.models.token_model [None req-0bc5a087-2a4a-4a62-9e8a-1ba6640d15c9 None None] User 762ce176aa7b4e448f388d4d746f6691 has no access to project 9fb1f6e1bd5d4d86b44901b94d8f0759 {{(pid=104118) _validate_project_scope /opt/stack/keystone/keystone/models/token_model.py:539}} Aug 20 21:10:22.763703 ncdv-master-1 devstack@keystone.service[104118]: WARNING keystone.server.flask.application [None req-0bc5a087-2a4a-4a62-9e8a-1ba6640d15c9 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.23: keystone.exception.Unauthorized: The request you have made requires authentication. Aug 20 21:10:22.765927 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2504/4955] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:22 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 175 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Aug 20 21:10:22.773750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55aeb5-9e92-4155-884d-21beaa98188c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:22.888931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55aeb5-9e92-4155-884d-21beaa98188c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:22.899324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:22.900952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:22.901909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:22.902418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] PATH_INFO: `/v3/users/762ce176aa7b4e448f388d4d746f6691` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:22.903964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=762ce176aa7b4e448f388d4d746f6691)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:22.906302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:22.986254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '762ce176aa7b4e448f388d4d746f6691'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:22.997106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '762ce176aa7b4e448f388d4d746f6691'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.015887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '762ce176aa7b4e448f388d4d746f6691'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.016771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '762ce176aa7b4e448f388d4d746f6691'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.017370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '762ce176aa7b4e448f388d4d746f6691'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.039325 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2452/4956] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:22 2022] DELETE /identity/v3/users/762ce176aa7b4e448f388d4d746f6691 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:23.044225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0fc2b-7570-4191-a06c-bdd73eb06be2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:23.044595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0fc2b-7570-4191-a06c-bdd73eb06be2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:23.044909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0fc2b-7570-4191-a06c-bdd73eb06be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:23.075441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-aff0fc2b-7570-4191-a06c-bdd73eb06be2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:23.076763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aff0fc2b-7570-4191-a06c-bdd73eb06be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:23.155066 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2505/4957] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:23.166082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:23.167523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:23.242522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74919076-2ce9-40d6-b145-7aaf1e5731aa admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:23.246427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa7253-85f3-4f99-aae0-fa552340665c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:23.247046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa7253-85f3-4f99-aae0-fa552340665c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:23.247320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa7253-85f3-4f99-aae0-fa552340665c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:23.247587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa7253-85f3-4f99-aae0-fa552340665c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:23.258918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa7253-85f3-4f99-aae0-fa552340665c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:23.260356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa7253-85f3-4f99-aae0-fa552340665c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:23.271008 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2453/4958] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:23.665379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0fc2b-7570-4191-a06c-bdd73eb06be2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:23.668201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aff0fc2b-7570-4191-a06c-bdd73eb06be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:23.684853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0fc2b-7570-4191-a06c-bdd73eb06be2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:23.690835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:23.691614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:23.691950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:23.692536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] PATH_INFO: `/v3/projects/af7498ca10ac43cd9ffccfbb0cde3f7a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:23.694091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af7498ca10ac43cd9ffccfbb0cde3f7a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:23.697223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:23.747439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af7498ca10ac43cd9ffccfbb0cde3f7a', 'request_id': 'req-e4d46eac-43e7-4988-9545-aa0f8ded96a5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.758192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af7498ca10ac43cd9ffccfbb0cde3f7a', 'request_id': 'req-e4d46eac-43e7-4988-9545-aa0f8ded96a5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.765602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af7498ca10ac43cd9ffccfbb0cde3f7a', 'request_id': 'req-e4d46eac-43e7-4988-9545-aa0f8ded96a5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.779059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] The token cache is being invalidate because project af7498ca10ac43cd9ffccfbb0cde3f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:23.779315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.779971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.781519 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2506/4959] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:23 2022] DELETE /identity/v3/projects/af7498ca10ac43cd9ffccfbb0cde3f7a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:23.785944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa7253-85f3-4f99-aae0-fa552340665c service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:23.792300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:23.796299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa7253-85f3-4f99-aae0-fa552340665c service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:23.856200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769889c-af23-47a8-b8c5-f092ff91a07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:23.856857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769889c-af23-47a8-b8c5-f092ff91a07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:23.857129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769889c-af23-47a8-b8c5-f092ff91a07b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:23.857391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769889c-af23-47a8-b8c5-f092ff91a07b admin admin] PATH_INFO: `/v3/users/025bde8f4c764aa8b69e7cc757d2b159` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:23.858603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769889c-af23-47a8-b8c5-f092ff91a07b admin admin] RBAC: Authorizing `identity:delete_user(user_id=025bde8f4c764aa8b69e7cc757d2b159)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:23.860571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769889c-af23-47a8-b8c5-f092ff91a07b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:23.908250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d769889c-af23-47a8-b8c5-f092ff91a07b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '025bde8f4c764aa8b69e7cc757d2b159'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.917796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d769889c-af23-47a8-b8c5-f092ff91a07b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '025bde8f4c764aa8b69e7cc757d2b159'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.930771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d769889c-af23-47a8-b8c5-f092ff91a07b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '025bde8f4c764aa8b69e7cc757d2b159'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.931457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d769889c-af23-47a8-b8c5-f092ff91a07b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '025bde8f4c764aa8b69e7cc757d2b159'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.932035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d769889c-af23-47a8-b8c5-f092ff91a07b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '025bde8f4c764aa8b69e7cc757d2b159'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:23.950055 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2454/4960] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:23 2022] DELETE /identity/v3/users/025bde8f4c764aa8b69e7cc757d2b159 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:24.123982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:24.125452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:24.143944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d46eac-43e7-4988-9545-aa0f8ded96a5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:24.204925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:24.251554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:24.252272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:24.252585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:24.252858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] PATH_INFO: `/v3/users/055f740ebe494119973307b8d209fe6f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:24.254228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=055f740ebe494119973307b8d209fe6f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:24.256913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:24.321856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '055f740ebe494119973307b8d209fe6f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:24.331542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '055f740ebe494119973307b8d209fe6f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:24.340323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '055f740ebe494119973307b8d209fe6f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:24.354402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '055f740ebe494119973307b8d209fe6f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:24.355170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '055f740ebe494119973307b8d209fe6f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:24.361514 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2507/4961] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:24 2022] DELETE /identity/v3/users/055f740ebe494119973307b8d209fe6f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:24.365908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f5bc4-68bb-460a-9511-c4eafab5e0e6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:24.366186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f5bc4-68bb-460a-9511-c4eafab5e0e6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:24.366447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f5bc4-68bb-460a-9511-c4eafab5e0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:24.402467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-082f5bc4-68bb-460a-9511-c4eafab5e0e6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:24.403809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-082f5bc4-68bb-460a-9511-c4eafab5e0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:24.447942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:24.450103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:24.525784 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2455/4962] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:24.535643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f5bc4-68bb-460a-9511-c4eafab5e0e6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:24.537246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-082f5bc4-68bb-460a-9511-c4eafab5e0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:24.552970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c87e5-6a00-442e-918d-dee82d44e198 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:24.558220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:24.558952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:24.559111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:24.559384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960 admin admin] PATH_INFO: `/v3/projects/9fb1f6e1bd5d4d86b44901b94d8f0759` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:24.560863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb1f6e1bd5d4d86b44901b94d8f0759)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:24.562963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:24.571365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:24.621942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f5bc4-68bb-460a-9511-c4eafab5e0e6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:24.626608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb1f6e1bd5d4d86b44901b94d8f0759', 'request_id': 'req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:24.652600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb1f6e1bd5d4d86b44901b94d8f0759', 'request_id': 'req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:24.662631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb1f6e1bd5d4d86b44901b94d8f0759', 'request_id': 'req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:24.664427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:24.678667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960 admin admin] The token cache is being invalidate because project 9fb1f6e1bd5d4d86b44901b94d8f0759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:24.678667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:24.680045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8df2e6a-46ea-4ce4-ab62-e8cd82191960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:24.687681 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2508/4963] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:24 2022] DELETE /identity/v3/projects/9fb1f6e1bd5d4d86b44901b94d8f0759 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:24.689794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeaab39-c98a-4ce8-b5ff-de4e2f07617e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:24.690090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeaab39-c98a-4ce8-b5ff-de4e2f07617e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:24.690345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeaab39-c98a-4ce8-b5ff-de4e2f07617e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:24.701081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52009b-d3cd-4920-982d-c0bc2577952c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:24.702075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52009b-d3cd-4920-982d-c0bc2577952c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:24.702472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52009b-d3cd-4920-982d-c0bc2577952c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:24.703020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52009b-d3cd-4920-982d-c0bc2577952c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:24.717184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8c52009b-d3cd-4920-982d-c0bc2577952c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:24.717636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-faeaab39-c98a-4ce8-b5ff-de4e2f07617e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:24.719623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-faeaab39-c98a-4ce8-b5ff-de4e2f07617e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:24.796270 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2509/4964] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:24.796669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52009b-d3cd-4920-982d-c0bc2577952c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:24.796669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52009b-d3cd-4920-982d-c0bc2577952c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:24.798389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77b00a-257d-4efb-8c99-75e8fd143ea9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:24.798685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77b00a-257d-4efb-8c99-75e8fd143ea9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:24.798960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77b00a-257d-4efb-8c99-75e8fd143ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:24.809224 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2456/4965] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:10:24.822921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4a77b00a-257d-4efb-8c99-75e8fd143ea9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:24.824472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4a77b00a-257d-4efb-8c99-75e8fd143ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:24.837557 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2510/4966] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:24.839584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52009b-d3cd-4920-982d-c0bc2577952c service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:24.851091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52009b-d3cd-4920-982d-c0bc2577952c service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:24.856362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2856a-ff85-449c-b44e-02d70aea1e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:24.857103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2856a-ff85-449c-b44e-02d70aea1e75 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:24.857385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2856a-ff85-449c-b44e-02d70aea1e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:24.857645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2856a-ff85-449c-b44e-02d70aea1e75 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:24.858334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2856a-ff85-449c-b44e-02d70aea1e75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:24.858633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2856a-ff85-449c-b44e-02d70aea1e75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:24.860536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2856a-ff85-449c-b44e-02d70aea1e75 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:24.874683 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2457/4967] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:24.878479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77b00a-257d-4efb-8c99-75e8fd143ea9 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:24.889225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77b00a-257d-4efb-8c99-75e8fd143ea9 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:24.896274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eab018-bc8c-4517-85bd-ea2b13bdd544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:24.896918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eab018-bc8c-4517-85bd-ea2b13bdd544 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:24.897196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eab018-bc8c-4517-85bd-ea2b13bdd544 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:24.897459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eab018-bc8c-4517-85bd-ea2b13bdd544 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:24.898087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eab018-bc8c-4517-85bd-ea2b13bdd544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:24.901249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eab018-bc8c-4517-85bd-ea2b13bdd544 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:24.938176 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2511/4968] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:24 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:24.939551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2856a-ff85-449c-b44e-02d70aea1e75 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:24.952604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2856a-ff85-449c-b44e-02d70aea1e75 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.030246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eab018-bc8c-4517-85bd-ea2b13bdd544 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.032135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c2eab018-bc8c-4517-85bd-ea2b13bdd544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:25.049145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1021a-0f66-4d4f-b15e-fcf8a5d35b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.050096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1021a-0f66-4d4f-b15e-fcf8a5d35b1c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.050096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1021a-0f66-4d4f-b15e-fcf8a5d35b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.050656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1021a-0f66-4d4f-b15e-fcf8a5d35b1c admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.051292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1021a-0f66-4d4f-b15e-fcf8a5d35b1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.054065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1021a-0f66-4d4f-b15e-fcf8a5d35b1c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.088970 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2458/4969] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:24 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:25.090095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1021a-0f66-4d4f-b15e-fcf8a5d35b1c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.107756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1021a-0f66-4d4f-b15e-fcf8a5d35b1c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.114463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccade999-9bdb-414a-bf0d-9f9b501c6469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.115203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccade999-9bdb-414a-bf0d-9f9b501c6469 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.115596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccade999-9bdb-414a-bf0d-9f9b501c6469 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.115803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccade999-9bdb-414a-bf0d-9f9b501c6469 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.116698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccade999-9bdb-414a-bf0d-9f9b501c6469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.118613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccade999-9bdb-414a-bf0d-9f9b501c6469 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.120082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:25.132902 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2459/4970] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:25.135741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccade999-9bdb-414a-bf0d-9f9b501c6469 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.146099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccade999-9bdb-414a-bf0d-9f9b501c6469 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.146267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eab018-bc8c-4517-85bd-ea2b13bdd544 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.150639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cf884-4ee1-4966-8cfa-7c5aad8ac91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.150805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddb381-f0be-4fdb-be96-e167647365f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.151258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cf884-4ee1-4966-8cfa-7c5aad8ac91a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.151385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddb381-f0be-4fdb-be96-e167647365f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.151529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cf884-4ee1-4966-8cfa-7c5aad8ac91a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.151654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddb381-f0be-4fdb-be96-e167647365f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.151795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cf884-4ee1-4966-8cfa-7c5aad8ac91a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.151910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddb381-f0be-4fdb-be96-e167647365f2 admin admin] PATH_INFO: `/v3/projects/5592ede086c7438eaa83dfb8cd73c1fb` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.152447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cf884-4ee1-4966-8cfa-7c5aad8ac91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.153007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ddb381-f0be-4fdb-be96-e167647365f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5592ede086c7438eaa83dfb8cd73c1fb)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.154031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cf884-4ee1-4966-8cfa-7c5aad8ac91a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.154933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ddb381-f0be-4fdb-be96-e167647365f2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.164547 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2460/4971] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:25.168844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cf884-4ee1-4966-8cfa-7c5aad8ac91a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.178783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cf884-4ee1-4966-8cfa-7c5aad8ac91a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.183713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d49c9-9956-4242-951a-d6a82cb1265b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.184364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d49c9-9956-4242-951a-d6a82cb1265b admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.184661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d49c9-9956-4242-951a-d6a82cb1265b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.184928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d49c9-9956-4242-951a-d6a82cb1265b admin admin] PATH_INFO: `/v3/projects/82e04a0b669e487e9e09bf61b70b74e7/users/aa1cf46b75f1467ab8ebddb27ff9b8b9/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.204764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d49c9-9956-4242-951a-d6a82cb1265b admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e04a0b669e487e9e09bf61b70b74e7, user_id=aa1cf46b75f1467ab8ebddb27ff9b8b9, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.208476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c3ddb381-f0be-4fdb-be96-e167647365f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5592ede086c7438eaa83dfb8cd73c1fb', 'request_id': 'req-c3ddb381-f0be-4fdb-be96-e167647365f2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:25.208860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d49c9-9956-4242-951a-d6a82cb1265b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.224641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c3ddb381-f0be-4fdb-be96-e167647365f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5592ede086c7438eaa83dfb8cd73c1fb', 'request_id': 'req-c3ddb381-f0be-4fdb-be96-e167647365f2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:25.239130 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2461/4972] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:25 2022] PUT /identity/v3/projects/82e04a0b669e487e9e09bf61b70b74e7/users/aa1cf46b75f1467ab8ebddb27ff9b8b9/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:25.239393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c3ddb381-f0be-4fdb-be96-e167647365f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5592ede086c7438eaa83dfb8cd73c1fb', 'request_id': 'req-c3ddb381-f0be-4fdb-be96-e167647365f2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:25.239393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:25.244183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60830efa-1f41-40d8-8d4b-f3c9ea5c3f7d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.244760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60830efa-1f41-40d8-8d4b-f3c9ea5c3f7d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.245297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60830efa-1f41-40d8-8d4b-f3c9ea5c3f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.248167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c3ddb381-f0be-4fdb-be96-e167647365f2 admin admin] The token cache is being invalidate because project 5592ede086c7438eaa83dfb8cd73c1fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:25.248675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c3ddb381-f0be-4fdb-be96-e167647365f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:25.249833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c3ddb381-f0be-4fdb-be96-e167647365f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:25.252284 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2512/4973] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:25 2022] DELETE /identity/v3/projects/5592ede086c7438eaa83dfb8cd73c1fb => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:25.260241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65daa66-054f-44a7-983a-cc2c6c005612 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.260241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65daa66-054f-44a7-983a-cc2c6c005612 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.260241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65daa66-054f-44a7-983a-cc2c6c005612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.281341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-60830efa-1f41-40d8-8d4b-f3c9ea5c3f7d None None] MFA Rules not processed for user `aa1cf46b75f1467ab8ebddb27ff9b8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:25.282595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-60830efa-1f41-40d8-8d4b-f3c9ea5c3f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:25.289760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a65daa66-054f-44a7-983a-cc2c6c005612 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:25.291469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a65daa66-054f-44a7-983a-cc2c6c005612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:25.360394 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2513/4974] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:25.362067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a7af9-d08e-4407-b629-3409573a8d6c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.362344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a7af9-d08e-4407-b629-3409573a8d6c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.362612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a7af9-d08e-4407-b629-3409573a8d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.370524 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2462/4975] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:25.374938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60830efa-1f41-40d8-8d4b-f3c9ea5c3f7d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.376351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-60830efa-1f41-40d8-8d4b-f3c9ea5c3f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:25.389488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-118a7af9-d08e-4407-b629-3409573a8d6c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:25.390580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-118a7af9-d08e-4407-b629-3409573a8d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:25.392363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60830efa-1f41-40d8-8d4b-f3c9ea5c3f7d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.399047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8bd39-6060-4c0f-92db-9a4eb84e43e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.399451 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2514/4976] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:25.399960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8bd39-6060-4c0f-92db-9a4eb84e43e6 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.400318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8bd39-6060-4c0f-92db-9a4eb84e43e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.400778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8bd39-6060-4c0f-92db-9a4eb84e43e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.401474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8bd39-6060-4c0f-92db-9a4eb84e43e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.403246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a7af9-d08e-4407-b629-3409573a8d6c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.404452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8bd39-6060-4c0f-92db-9a4eb84e43e6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.414810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a7af9-d08e-4407-b629-3409573a8d6c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.420667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143e6cd-0800-4b03-ba26-6f3f6ce4c9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.421533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143e6cd-0800-4b03-ba26-6f3f6ce4c9f5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.421878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143e6cd-0800-4b03-ba26-6f3f6ce4c9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.422212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143e6cd-0800-4b03-ba26-6f3f6ce4c9f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.422931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143e6cd-0800-4b03-ba26-6f3f6ce4c9f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:25.423296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143e6cd-0800-4b03-ba26-6f3f6ce4c9f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.424994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143e6cd-0800-4b03-ba26-6f3f6ce4c9f5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.446186 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2515/4977] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:25.450164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143e6cd-0800-4b03-ba26-6f3f6ce4c9f5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.463087 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2463/4978] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:25 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:25.464021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8bd39-6060-4c0f-92db-9a4eb84e43e6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.484822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143e6cd-0800-4b03-ba26-6f3f6ce4c9f5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.551859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8bd39-6060-4c0f-92db-9a4eb84e43e6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.558956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411ee75-ef1f-4b1e-a937-834be7cef084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.560176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411ee75-ef1f-4b1e-a937-834be7cef084 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.561187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411ee75-ef1f-4b1e-a937-834be7cef084 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.561946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411ee75-ef1f-4b1e-a937-834be7cef084 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.562700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411ee75-ef1f-4b1e-a937-834be7cef084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.564758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411ee75-ef1f-4b1e-a937-834be7cef084 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.567031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d73a26-bf66-4071-8deb-b8e078d6d35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.567637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d73a26-bf66-4071-8deb-b8e078d6d35a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.567906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d73a26-bf66-4071-8deb-b8e078d6d35a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.568167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d73a26-bf66-4071-8deb-b8e078d6d35a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.568815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d73a26-bf66-4071-8deb-b8e078d6d35a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.571096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d73a26-bf66-4071-8deb-b8e078d6d35a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.589178 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2464/4979] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:25 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:25.593449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411ee75-ef1f-4b1e-a937-834be7cef084 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.602232 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2516/4980] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:25 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:25.603788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411ee75-ef1f-4b1e-a937-834be7cef084 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.606312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d73a26-bf66-4071-8deb-b8e078d6d35a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.617193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:25.677249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276aaab-7a0d-48c3-a748-8689ce1bc794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.677504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d73a26-bf66-4071-8deb-b8e078d6d35a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.677765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276aaab-7a0d-48c3-a748-8689ce1bc794 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.678540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276aaab-7a0d-48c3-a748-8689ce1bc794 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.678745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276aaab-7a0d-48c3-a748-8689ce1bc794 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.679500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276aaab-7a0d-48c3-a748-8689ce1bc794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.681171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276aaab-7a0d-48c3-a748-8689ce1bc794 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.682332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722da9c-13bc-4486-8170-5663f57f72e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.682945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722da9c-13bc-4486-8170-5663f57f72e3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.683218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722da9c-13bc-4486-8170-5663f57f72e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.683489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722da9c-13bc-4486-8170-5663f57f72e3 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.684210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722da9c-13bc-4486-8170-5663f57f72e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.686261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722da9c-13bc-4486-8170-5663f57f72e3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.692769 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2465/4981] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:25.697068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276aaab-7a0d-48c3-a748-8689ce1bc794 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.699360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:25.708341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276aaab-7a0d-48c3-a748-8689ce1bc794 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.714039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8aa904-0992-4b96-bc23-cd066cee629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.714683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8aa904-0992-4b96-bc23-cd066cee629f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.714914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8aa904-0992-4b96-bc23-cd066cee629f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.715176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8aa904-0992-4b96-bc23-cd066cee629f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.715819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8aa904-0992-4b96-bc23-cd066cee629f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.719860 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2517/4982] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:25 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:25.720685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8aa904-0992-4b96-bc23-cd066cee629f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.720987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722da9c-13bc-4486-8170-5663f57f72e3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.728747 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2466/4983] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:25.733625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8aa904-0992-4b96-bc23-cd066cee629f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.733799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722da9c-13bc-4486-8170-5663f57f72e3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.740182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23031bc-d173-43bc-93f9-1f4a4acfacaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.741575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23031bc-d173-43bc-93f9-1f4a4acfacaf admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.741851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23031bc-d173-43bc-93f9-1f4a4acfacaf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.742120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23031bc-d173-43bc-93f9-1f4a4acfacaf admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.742957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23031bc-d173-43bc-93f9-1f4a4acfacaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.745057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23031bc-d173-43bc-93f9-1f4a4acfacaf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.745543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8aa904-0992-4b96-bc23-cd066cee629f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.750848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49900223-5b38-4d37-9671-572fe3504781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.751800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49900223-5b38-4d37-9671-572fe3504781 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.752316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49900223-5b38-4d37-9671-572fe3504781 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.752850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49900223-5b38-4d37-9671-572fe3504781 admin admin] PATH_INFO: `/v3/projects/5e5cf321ca004485a1003925da822420/users/87185bd5eda9492eaf4d79ce3cbf49c5/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.758567 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2518/4984] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:25.762193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23031bc-d173-43bc-93f9-1f4a4acfacaf admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.770500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49900223-5b38-4d37-9671-572fe3504781 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e5cf321ca004485a1003925da822420, user_id=87185bd5eda9492eaf4d79ce3cbf49c5, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.773240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49900223-5b38-4d37-9671-572fe3504781 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.775223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23031bc-d173-43bc-93f9-1f4a4acfacaf admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.780780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7d9de-817a-4211-8ad0-4db9eaecf3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.781419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7d9de-817a-4211-8ad0-4db9eaecf3ba admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.781804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7d9de-817a-4211-8ad0-4db9eaecf3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.782074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7d9de-817a-4211-8ad0-4db9eaecf3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.783311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7d9de-817a-4211-8ad0-4db9eaecf3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.784734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7d9de-817a-4211-8ad0-4db9eaecf3ba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.790759 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2467/4985] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:25 2022] PUT /identity/v3/projects/5e5cf321ca004485a1003925da822420/users/87185bd5eda9492eaf4d79ce3cbf49c5/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:25.800151 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2519/4986] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:25.800960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7d9de-817a-4211-8ad0-4db9eaecf3ba admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.801092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcbf2b-5663-4c99-8ebd-6b9a9c605d51 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.801562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcbf2b-5663-4c99-8ebd-6b9a9c605d51 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.801562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcbf2b-5663-4c99-8ebd-6b9a9c605d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.812540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7d9de-817a-4211-8ad0-4db9eaecf3ba admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.837489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0dfcbf2b-5663-4c99-8ebd-6b9a9c605d51 None None] MFA Rules not processed for user `87185bd5eda9492eaf4d79ce3cbf49c5`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:25.839014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0dfcbf2b-5663-4c99-8ebd-6b9a9c605d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:25.882040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e64f73-c8be-492c-900c-66aee16a661d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.882986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e64f73-c8be-492c-900c-66aee16a661d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.883377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e64f73-c8be-492c-900c-66aee16a661d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.883744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e64f73-c8be-492c-900c-66aee16a661d admin admin] PATH_INFO: `/v3/projects/570eb829c83e45b380a9a6df986b9b35/users/a75178e93f9a463fbab9a21e1a34793e/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.903977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e64f73-c8be-492c-900c-66aee16a661d admin admin] RBAC: Authorizing `identity:create_grant(project_id=570eb829c83e45b380a9a6df986b9b35, user_id=a75178e93f9a463fbab9a21e1a34793e, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.906812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e64f73-c8be-492c-900c-66aee16a661d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:25.907789 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2468/4987] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:25.912153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfcbf2b-5663-4c99-8ebd-6b9a9c605d51 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:25.924532 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2520/4988] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:25 2022] PUT /identity/v3/projects/570eb829c83e45b380a9a6df986b9b35/users/a75178e93f9a463fbab9a21e1a34793e/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:25.930867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfcbf2b-5663-4c99-8ebd-6b9a9c605d51 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:25.937007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4ce09-8985-46ab-a817-480c1ddb8a68 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.937484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4ce09-8985-46ab-a817-480c1ddb8a68 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.937752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4ce09-8985-46ab-a817-480c1ddb8a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.970197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-07f4ce09-8985-46ab-a817-480c1ddb8a68 None None] MFA Rules not processed for user `a75178e93f9a463fbab9a21e1a34793e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:25.971540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-07f4ce09-8985-46ab-a817-480c1ddb8a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:25.995328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb8e8c-9c77-4db9-9909-53c1eb910913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:25.995949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb8e8c-9c77-4db9-9909-53c1eb910913 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:25.996220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb8e8c-9c77-4db9-9909-53c1eb910913 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:25.996519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb8e8c-9c77-4db9-9909-53c1eb910913 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:25.997150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb8e8c-9c77-4db9-9909-53c1eb910913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:25.999071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb8e8c-9c77-4db9-9909-53c1eb910913 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:26.014787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:26.033023 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2469/4989] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:25 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:26.033887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb8e8c-9c77-4db9-9909-53c1eb910913 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:26.048995 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2521/4990] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:26.056034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4ce09-8985-46ab-a817-480c1ddb8a68 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:26.057951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-07f4ce09-8985-46ab-a817-480c1ddb8a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:26.108044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb8e8c-9c77-4db9-9909-53c1eb910913 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:26.112561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:26.115363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c234f-528d-4575-9d94-7c742a4bf698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:26.115954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c234f-528d-4575-9d94-7c742a4bf698 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:26.116227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c234f-528d-4575-9d94-7c742a4bf698 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:26.116615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c234f-528d-4575-9d94-7c742a4bf698 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:26.117296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c234f-528d-4575-9d94-7c742a4bf698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:26.119265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c234f-528d-4575-9d94-7c742a4bf698 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:26.147440 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2470/4991] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:26 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:26.150932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4ce09-8985-46ab-a817-480c1ddb8a68 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:26.151344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c234f-528d-4575-9d94-7c742a4bf698 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:26.163494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c234f-528d-4575-9d94-7c742a4bf698 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:26.170065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed0ea3-3676-432d-8033-1bfadf8dd25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:26.170956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed0ea3-3676-432d-8033-1bfadf8dd25b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:26.171332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed0ea3-3676-432d-8033-1bfadf8dd25b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:26.171718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed0ea3-3676-432d-8033-1bfadf8dd25b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:26.172607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed0ea3-3676-432d-8033-1bfadf8dd25b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:26.175088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed0ea3-3676-432d-8033-1bfadf8dd25b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:26.204013 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2471/4992] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:26.208437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed0ea3-3676-432d-8033-1bfadf8dd25b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:26.209972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdbf11-3eea-4686-a2d6-47f1b9314bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:26.210643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdbf11-3eea-4686-a2d6-47f1b9314bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:26.210964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdbf11-3eea-4686-a2d6-47f1b9314bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:26.211419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdbf11-3eea-4686-a2d6-47f1b9314bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:26.228525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed0ea3-3676-432d-8033-1bfadf8dd25b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:26.238559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acaa53-7f92-40ea-8ac8-45b2e58edd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:26.239245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acaa53-7f92-40ea-8ac8-45b2e58edd3f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:26.239515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acaa53-7f92-40ea-8ac8-45b2e58edd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:26.239918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acaa53-7f92-40ea-8ac8-45b2e58edd3f admin admin] PATH_INFO: `/v3/projects/5c748e76dafd409d87a8637645bf36f9/users/3e849f3556c04fb69a01b0e1f7a95bef/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:26.265943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acaa53-7f92-40ea-8ac8-45b2e58edd3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c748e76dafd409d87a8637645bf36f9, user_id=3e849f3556c04fb69a01b0e1f7a95bef, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:26.269524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acaa53-7f92-40ea-8ac8-45b2e58edd3f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:26.283869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bdbf11-3eea-4686-a2d6-47f1b9314bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:26.286579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bdbf11-3eea-4686-a2d6-47f1b9314bb8 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:26.296549 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2472/4993] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:26 2022] PUT /identity/v3/projects/5c748e76dafd409d87a8637645bf36f9/users/3e849f3556c04fb69a01b0e1f7a95bef/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:26.297738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acaa53-7f92-40ea-8ac8-45b2e58edd3f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:26.312597 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2522/4994] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:10:26.436024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acaa53-7f92-40ea-8ac8-45b2e58edd3f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:26.443929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfe694-2411-4a79-93a5-a76698330010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:26.449108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfe694-2411-4a79-93a5-a76698330010 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:26.449258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfe694-2411-4a79-93a5-a76698330010 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:26.449530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfe694-2411-4a79-93a5-a76698330010 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:26.450206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfe694-2411-4a79-93a5-a76698330010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:26.452051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfe694-2411-4a79-93a5-a76698330010 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:26.477041 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2473/4995] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:26.481304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdbf11-3eea-4686-a2d6-47f1b9314bb8 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:26.497099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdbf11-3eea-4686-a2d6-47f1b9314bb8 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:26.502984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befedf3d-9888-4e04-ab4c-9b83d6d5475b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:26.504599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befedf3d-9888-4e04-ab4c-9b83d6d5475b admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:26.505044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befedf3d-9888-4e04-ab4c-9b83d6d5475b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:26.505482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befedf3d-9888-4e04-ab4c-9b83d6d5475b admin admin] PATH_INFO: `/v3/projects/5c748e76dafd409d87a8637645bf36f9/users/3e849f3556c04fb69a01b0e1f7a95bef/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:26.507526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befedf3d-9888-4e04-ab4c-9b83d6d5475b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c748e76dafd409d87a8637645bf36f9, user_id=3e849f3556c04fb69a01b0e1f7a95bef, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:26.510296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befedf3d-9888-4e04-ab4c-9b83d6d5475b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:26.523402 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2523/4996] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:26 2022] PUT /identity/v3/projects/5c748e76dafd409d87a8637645bf36f9/users/3e849f3556c04fb69a01b0e1f7a95bef/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:26.535382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e0b1f-ca62-4020-9111-184449ebcf11 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:26.535666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e0b1f-ca62-4020-9111-184449ebcf11 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:26.536088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e0b1f-ca62-4020-9111-184449ebcf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:26.598087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ed9e0b1f-ca62-4020-9111-184449ebcf11 None None] MFA Rules not processed for user `3e849f3556c04fb69a01b0e1f7a95bef`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:26.599345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ed9e0b1f-ca62-4020-9111-184449ebcf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:26.658032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:26.709919 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2474/4997] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:26 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:26.726894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befedf3d-9888-4e04-ab4c-9b83d6d5475b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:26.805720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befedf3d-9888-4e04-ab4c-9b83d6d5475b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:26.809720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5756a-fa0f-4db6-b2da-7825b5813853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:26.810467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5756a-fa0f-4db6-b2da-7825b5813853 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:26.810596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5756a-fa0f-4db6-b2da-7825b5813853 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:26.810851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5756a-fa0f-4db6-b2da-7825b5813853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:26.887110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:26.891003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5756a-fa0f-4db6-b2da-7825b5813853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:26.893255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5756a-fa0f-4db6-b2da-7825b5813853 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:26.908879 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2524/4998] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:10:33.869645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57428cbc-0434-436e-b484-a6117ae64453 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:33.870295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57428cbc-0434-436e-b484-a6117ae64453 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:33.870295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57428cbc-0434-436e-b484-a6117ae64453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:33.889922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-57428cbc-0434-436e-b484-a6117ae64453 None None] MFA Rules not processed for user `1cf36cef64b14630850a3fc3d2974e79`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:33.891139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-57428cbc-0434-436e-b484-a6117ae64453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:33.951310 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2475/4999] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:34.761523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5756a-fa0f-4db6-b2da-7825b5813853 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:34.858148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5756a-fa0f-4db6-b2da-7825b5813853 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:34.863418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:34.864094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:34.864389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:34.864663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] PATH_INFO: `/v3/users/a75178e93f9a463fbab9a21e1a34793e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:34.865827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a75178e93f9a463fbab9a21e1a34793e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:34.869105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:34.942811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a75178e93f9a463fbab9a21e1a34793e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:34.965921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a75178e93f9a463fbab9a21e1a34793e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:34.975932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a75178e93f9a463fbab9a21e1a34793e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:35.006386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a75178e93f9a463fbab9a21e1a34793e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:35.007105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a75178e93f9a463fbab9a21e1a34793e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:35.015290 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2525/5000] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:34 2022] DELETE /identity/v3/users/a75178e93f9a463fbab9a21e1a34793e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:35.019677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e6836-542b-4807-bc63-a0c45e8983f7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:35.020696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e6836-542b-4807-bc63-a0c45e8983f7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:35.021144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e6836-542b-4807-bc63-a0c45e8983f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:35.047002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ce6e6836-542b-4807-bc63-a0c45e8983f7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:35.048919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ce6e6836-542b-4807-bc63-a0c45e8983f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:35.155832 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2476/5001] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:35.191191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:35.193587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:35.333460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e33969-9e66-4042-b753-153cc1f3a8c2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:35.339287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a6729-da95-4d0a-8e65-80b3e86de78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:35.340462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a6729-da95-4d0a-8e65-80b3e86de78b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:35.340929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a6729-da95-4d0a-8e65-80b3e86de78b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:35.341407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a6729-da95-4d0a-8e65-80b3e86de78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:35.368937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a6729-da95-4d0a-8e65-80b3e86de78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:35.377617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a6729-da95-4d0a-8e65-80b3e86de78b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:35.402961 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2526/5002] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:35.994807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e6836-542b-4807-bc63-a0c45e8983f7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:35.997969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ce6e6836-542b-4807-bc63-a0c45e8983f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:36.030575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e6836-542b-4807-bc63-a0c45e8983f7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:36.064199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477115f5-3f05-4d38-909f-7862a0d156c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:36.065336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477115f5-3f05-4d38-909f-7862a0d156c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:36.065336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477115f5-3f05-4d38-909f-7862a0d156c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:36.065567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477115f5-3f05-4d38-909f-7862a0d156c4 admin admin] PATH_INFO: `/v3/projects/570eb829c83e45b380a9a6df986b9b35` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:36.068063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477115f5-3f05-4d38-909f-7862a0d156c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=570eb829c83e45b380a9a6df986b9b35)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:36.070518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477115f5-3f05-4d38-909f-7862a0d156c4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:36.270491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-477115f5-3f05-4d38-909f-7862a0d156c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '570eb829c83e45b380a9a6df986b9b35', 'request_id': 'req-477115f5-3f05-4d38-909f-7862a0d156c4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:36.290425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-477115f5-3f05-4d38-909f-7862a0d156c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '570eb829c83e45b380a9a6df986b9b35', 'request_id': 'req-477115f5-3f05-4d38-909f-7862a0d156c4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:36.304877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-477115f5-3f05-4d38-909f-7862a0d156c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '570eb829c83e45b380a9a6df986b9b35', 'request_id': 'req-477115f5-3f05-4d38-909f-7862a0d156c4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:36.318240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:36.343399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-477115f5-3f05-4d38-909f-7862a0d156c4 admin admin] The token cache is being invalidate because project 570eb829c83e45b380a9a6df986b9b35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:36.344144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-477115f5-3f05-4d38-909f-7862a0d156c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:36.345464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-477115f5-3f05-4d38-909f-7862a0d156c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:36.356830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb621ec4-1ac6-4052-a867-62a888316ad3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:36.357480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb621ec4-1ac6-4052-a867-62a888316ad3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:36.357849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb621ec4-1ac6-4052-a867-62a888316ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:36.361127 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2477/5003] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:35 2022] DELETE /identity/v3/projects/570eb829c83e45b380a9a6df986b9b35 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:36.449018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fb621ec4-1ac6-4052-a867-62a888316ad3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:36.464404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fb621ec4-1ac6-4052-a867-62a888316ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:36.483580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:36.616930 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2527/5004] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:36.624948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5c80f-99f6-4e14-8a91-984888c1405c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:36.625233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5c80f-99f6-4e14-8a91-984888c1405c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:36.625726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5c80f-99f6-4e14-8a91-984888c1405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:36.663772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7be5c80f-99f6-4e14-8a91-984888c1405c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:36.667502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7be5c80f-99f6-4e14-8a91-984888c1405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:36.681383 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2478/5005] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:36.682221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb621ec4-1ac6-4052-a867-62a888316ad3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:36.704752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb621ec4-1ac6-4052-a867-62a888316ad3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:36.719291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637af31-43d9-4226-bb1f-db2410c8535b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:36.719985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637af31-43d9-4226-bb1f-db2410c8535b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:36.720274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637af31-43d9-4226-bb1f-db2410c8535b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:36.721220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637af31-43d9-4226-bb1f-db2410c8535b admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:36.722400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637af31-43d9-4226-bb1f-db2410c8535b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:36.722710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637af31-43d9-4226-bb1f-db2410c8535b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:36.728952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637af31-43d9-4226-bb1f-db2410c8535b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:36.786477 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2528/5006] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:36.789076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5c80f-99f6-4e14-8a91-984888c1405c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:36.808710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5c80f-99f6-4e14-8a91-984888c1405c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:36.818605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dff604-bb12-4dc0-ab3f-41c579c617c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:36.824153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dff604-bb12-4dc0-ab3f-41c579c617c8 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:36.826563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dff604-bb12-4dc0-ab3f-41c579c617c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:36.826916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dff604-bb12-4dc0-ab3f-41c579c617c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:36.827650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dff604-bb12-4dc0-ab3f-41c579c617c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:36.832961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dff604-bb12-4dc0-ab3f-41c579c617c8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:36.908425 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2479/5007] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:36 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:36.913858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637af31-43d9-4226-bb1f-db2410c8535b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:36.929818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637af31-43d9-4226-bb1f-db2410c8535b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:37.057087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98406f72-2d81-4861-857b-1b154f712a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:37.057863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98406f72-2d81-4861-857b-1b154f712a96 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:37.058140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98406f72-2d81-4861-857b-1b154f712a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:37.058401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98406f72-2d81-4861-857b-1b154f712a96 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:37.059034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98406f72-2d81-4861-857b-1b154f712a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:37.061062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98406f72-2d81-4861-857b-1b154f712a96 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:37.089071 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2529/5008] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:36 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:37.093187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dff604-bb12-4dc0-ab3f-41c579c617c8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:37.107236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dff604-bb12-4dc0-ab3f-41c579c617c8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:37.114493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22f4d1-72f6-48a5-b0c1-5401733991b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:37.116144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22f4d1-72f6-48a5-b0c1-5401733991b9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:37.116735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22f4d1-72f6-48a5-b0c1-5401733991b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:37.117278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22f4d1-72f6-48a5-b0c1-5401733991b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:37.118314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22f4d1-72f6-48a5-b0c1-5401733991b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:37.120565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22f4d1-72f6-48a5-b0c1-5401733991b9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:37.146611 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2480/5009] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:37.150046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98406f72-2d81-4861-857b-1b154f712a96 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:37.170885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98406f72-2d81-4861-857b-1b154f712a96 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:37.180559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246d5d5-011d-46c6-9294-633da502be44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:37.181552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246d5d5-011d-46c6-9294-633da502be44 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:37.182139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246d5d5-011d-46c6-9294-633da502be44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:37.182277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246d5d5-011d-46c6-9294-633da502be44 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:37.183247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246d5d5-011d-46c6-9294-633da502be44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:37.187981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246d5d5-011d-46c6-9294-633da502be44 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:37.209435 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2530/5010] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:37.211054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22f4d1-72f6-48a5-b0c1-5401733991b9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:37.225650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22f4d1-72f6-48a5-b0c1-5401733991b9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:37.256716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23434559-05ef-4b97-bd3f-661696818691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:37.260528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23434559-05ef-4b97-bd3f-661696818691 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:37.264888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23434559-05ef-4b97-bd3f-661696818691 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:37.266837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23434559-05ef-4b97-bd3f-661696818691 admin admin] PATH_INFO: `/v3/projects/21945e06c391470584cfbf61d808859b/users/18aacc4bbf774e2b8645be682089cf78/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:37.325283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23434559-05ef-4b97-bd3f-661696818691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21945e06c391470584cfbf61d808859b, user_id=18aacc4bbf774e2b8645be682089cf78, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:37.330820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23434559-05ef-4b97-bd3f-661696818691 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:37.388428 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2481/5011] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:37 2022] PUT /identity/v3/projects/21945e06c391470584cfbf61d808859b/users/18aacc4bbf774e2b8645be682089cf78/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:37.394996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b8f8b-5ab4-429c-9d7c-0e68a62c0cc7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:37.395538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b8f8b-5ab4-429c-9d7c-0e68a62c0cc7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:37.395538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b8f8b-5ab4-429c-9d7c-0e68a62c0cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:37.456917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-875b8f8b-5ab4-429c-9d7c-0e68a62c0cc7 None None] MFA Rules not processed for user `18aacc4bbf774e2b8645be682089cf78`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:37.458229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-875b8f8b-5ab4-429c-9d7c-0e68a62c0cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:37.624946 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2531/5012] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:37.625927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23434559-05ef-4b97-bd3f-661696818691 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:37.738004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23434559-05ef-4b97-bd3f-661696818691 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:37.748244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07930a-0400-48e2-afea-2f27caad7161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:37.748796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07930a-0400-48e2-afea-2f27caad7161 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:37.749183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07930a-0400-48e2-afea-2f27caad7161 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:37.749436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07930a-0400-48e2-afea-2f27caad7161 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:37.750167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07930a-0400-48e2-afea-2f27caad7161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:37.752973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07930a-0400-48e2-afea-2f27caad7161 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:37.796922 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2482/5013] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:37 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:37.797682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b8f8b-5ab4-429c-9d7c-0e68a62c0cc7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:37.813701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b8f8b-5ab4-429c-9d7c-0e68a62c0cc7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:37.835028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:37.919642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff16a6-4e36-43bf-b448-c262e33318ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:37.920281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff16a6-4e36-43bf-b448-c262e33318ec admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:37.920613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff16a6-4e36-43bf-b448-c262e33318ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:37.920875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff16a6-4e36-43bf-b448-c262e33318ec admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:37.921513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff16a6-4e36-43bf-b448-c262e33318ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:37.928700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff16a6-4e36-43bf-b448-c262e33318ec admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:37.982160 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2532/5014] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:37 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:37.987011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07930a-0400-48e2-afea-2f27caad7161 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:38.000055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07930a-0400-48e2-afea-2f27caad7161 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:38.005060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e40b5-d059-4db3-a4eb-b46c9ac2309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:38.006202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e40b5-d059-4db3-a4eb-b46c9ac2309b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:38.006636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e40b5-d059-4db3-a4eb-b46c9ac2309b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:38.006886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e40b5-d059-4db3-a4eb-b46c9ac2309b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:38.007674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e40b5-d059-4db3-a4eb-b46c9ac2309b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:38.010061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e40b5-d059-4db3-a4eb-b46c9ac2309b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:38.032402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff16a6-4e36-43bf-b448-c262e33318ec admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:38.036991 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2483/5015] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:38.046945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff16a6-4e36-43bf-b448-c262e33318ec admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:38.059945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b80f8a-ac9f-40aa-9695-e7ca824ba75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:38.060791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b80f8a-ac9f-40aa-9695-e7ca824ba75f admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:38.061205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b80f8a-ac9f-40aa-9695-e7ca824ba75f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:38.061476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b80f8a-ac9f-40aa-9695-e7ca824ba75f admin admin] PATH_INFO: `/v3/projects/9a64ef5bb0c84698a5a48d5f787791b5/users/1159d8ea06cf4778b005e6957a5bb830/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:38.071121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210dd65-94a9-4043-938e-b9b4919f361a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:38.071671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210dd65-94a9-4043-938e-b9b4919f361a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:38.072245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210dd65-94a9-4043-938e-b9b4919f361a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:38.081586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b80f8a-ac9f-40aa-9695-e7ca824ba75f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a64ef5bb0c84698a5a48d5f787791b5, user_id=1159d8ea06cf4778b005e6957a5bb830, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:38.085060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b80f8a-ac9f-40aa-9695-e7ca824ba75f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:38.108217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b210dd65-94a9-4043-938e-b9b4919f361a None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:38.110995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b210dd65-94a9-4043-938e-b9b4919f361a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:38.115273 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2533/5016] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:38 2022] PUT /identity/v3/projects/9a64ef5bb0c84698a5a48d5f787791b5/users/1159d8ea06cf4778b005e6957a5bb830/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:38.119302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b80f8a-ac9f-40aa-9695-e7ca824ba75f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:38.249081 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2484/5017] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:10:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:38.283364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210dd65-94a9-4043-938e-b9b4919f361a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:38.285403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b210dd65-94a9-4043-938e-b9b4919f361a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:38.295258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:38.317370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b80f8a-ac9f-40aa-9695-e7ca824ba75f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:38.325553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7784a-28a0-483a-8eed-de266755d8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:38.326746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7784a-28a0-483a-8eed-de266755d8fb admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:38.327045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7784a-28a0-483a-8eed-de266755d8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:38.327312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7784a-28a0-483a-8eed-de266755d8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:38.329018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7784a-28a0-483a-8eed-de266755d8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:38.331930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7784a-28a0-483a-8eed-de266755d8fb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:38.352685 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2534/5018] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:38.356330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7784a-28a0-483a-8eed-de266755d8fb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:38.369980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7784a-28a0-483a-8eed-de266755d8fb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:38.376801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f8150-0b79-469e-b7e0-400dc71019ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:38.377393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f8150-0b79-469e-b7e0-400dc71019ba admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:38.377735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f8150-0b79-469e-b7e0-400dc71019ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:38.378096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f8150-0b79-469e-b7e0-400dc71019ba admin admin] PATH_INFO: `/v3/projects/9a64ef5bb0c84698a5a48d5f787791b5/users/1159d8ea06cf4778b005e6957a5bb830/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:38.380303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f8150-0b79-469e-b7e0-400dc71019ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a64ef5bb0c84698a5a48d5f787791b5, user_id=1159d8ea06cf4778b005e6957a5bb830, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:38.383523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f8150-0b79-469e-b7e0-400dc71019ba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:38.407888 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2535/5019] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:38 2022] PUT /identity/v3/projects/9a64ef5bb0c84698a5a48d5f787791b5/users/1159d8ea06cf4778b005e6957a5bb830/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:38.408938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa6b9b-a7ea-47dd-8887-ae363c1790fa None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:38.409300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa6b9b-a7ea-47dd-8887-ae363c1790fa None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:38.409777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa6b9b-a7ea-47dd-8887-ae363c1790fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:38.470731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9faa6b9b-a7ea-47dd-8887-ae363c1790fa None None] MFA Rules not processed for user `1159d8ea06cf4778b005e6957a5bb830`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:38.472194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9faa6b9b-a7ea-47dd-8887-ae363c1790fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:38.485280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210dd65-94a9-4043-938e-b9b4919f361a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:38.568977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463d7d4-6f19-4a2d-846b-26595a3387d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:38.570894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463d7d4-6f19-4a2d-846b-26595a3387d5 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:38.571186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463d7d4-6f19-4a2d-846b-26595a3387d5 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:38.571461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463d7d4-6f19-4a2d-846b-26595a3387d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:38.587768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:38.648639 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2536/5020] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:38.665742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa6b9b-a7ea-47dd-8887-ae363c1790fa None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:38.667710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9faa6b9b-a7ea-47dd-8887-ae363c1790fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:38.673671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463d7d4-6f19-4a2d-846b-26595a3387d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:38.675521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463d7d4-6f19-4a2d-846b-26595a3387d5 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:38.690153 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2485/5021] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:10:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:10:38.765225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa6b9b-a7ea-47dd-8887-ae363c1790fa None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:38.769736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c763c06-8994-4f43-b963-431ca8f1ff96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:38.770531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c763c06-8994-4f43-b963-431ca8f1ff96 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:38.770897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c763c06-8994-4f43-b963-431ca8f1ff96 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:38.771279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c763c06-8994-4f43-b963-431ca8f1ff96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:38.791737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c763c06-8994-4f43-b963-431ca8f1ff96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:38.793446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c763c06-8994-4f43-b963-431ca8f1ff96 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:38.817818 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2537/5022] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:10:39.069578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463d7d4-6f19-4a2d-846b-26595a3387d5 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:39.084294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463d7d4-6f19-4a2d-846b-26595a3387d5 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:39.094839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a95a6-e7d0-408b-9d5d-a07738f86ea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:39.095479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a95a6-e7d0-408b-9d5d-a07738f86ea7 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:39.095829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a95a6-e7d0-408b-9d5d-a07738f86ea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:39.096006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a95a6-e7d0-408b-9d5d-a07738f86ea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:39.186620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:39.231950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a95a6-e7d0-408b-9d5d-a07738f86ea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:39.237801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a95a6-e7d0-408b-9d5d-a07738f86ea7 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:39.266888 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2486/5023] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:39.568795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c763c06-8994-4f43-b963-431ca8f1ff96 service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:39.570641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c763c06-8994-4f43-b963-431ca8f1ff96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:39.646967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c763c06-8994-4f43-b963-431ca8f1ff96 service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:39.651350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae6a23-b81f-43c2-982b-36d56ea677f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:39.651961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae6a23-b81f-43c2-982b-36d56ea677f1 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:39.652668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae6a23-b81f-43c2-982b-36d56ea677f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:39.653114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae6a23-b81f-43c2-982b-36d56ea677f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:39.655072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c9ae6a23-b81f-43c2-982b-36d56ea677f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:39.748051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae6a23-b81f-43c2-982b-36d56ea677f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:39.749722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae6a23-b81f-43c2-982b-36d56ea677f1 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:39.760185 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2538/5024] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:40.655749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a95a6-e7d0-408b-9d5d-a07738f86ea7 service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:40.657259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a45a95a6-e7d0-408b-9d5d-a07738f86ea7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:40.803760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a95a6-e7d0-408b-9d5d-a07738f86ea7 service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:40.809792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f6f89-9731-410c-8188-f4f662c89d1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:40.810428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f6f89-9731-410c-8188-f4f662c89d1c service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:40.810702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f6f89-9731-410c-8188-f4f662c89d1c service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:40.810967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f6f89-9731-410c-8188-f4f662c89d1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:40.812805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c97f6f89-9731-410c-8188-f4f662c89d1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:40.829140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f6f89-9731-410c-8188-f4f662c89d1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:40.830765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f6f89-9731-410c-8188-f4f662c89d1c service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:40.844650 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2487/5025] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:44.475325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae6a23-b81f-43c2-982b-36d56ea677f1 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:44.527180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:44.553955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae6a23-b81f-43c2-982b-36d56ea677f1 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:44.559075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:44.559905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:44.560180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:44.560483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] PATH_INFO: `/v3/users/18aacc4bbf774e2b8645be682089cf78` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:44.561626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18aacc4bbf774e2b8645be682089cf78)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:44.563592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:44.628395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18aacc4bbf774e2b8645be682089cf78'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:44.644843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18aacc4bbf774e2b8645be682089cf78'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:44.655974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18aacc4bbf774e2b8645be682089cf78'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:44.681797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18aacc4bbf774e2b8645be682089cf78'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:44.682796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18aacc4bbf774e2b8645be682089cf78'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:44.694705 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2539/5026] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:44 2022] DELETE /identity/v3/users/18aacc4bbf774e2b8645be682089cf78 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:44.699733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8e88e-7be4-4093-bad7-f4bf16a9fad4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:44.700199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8e88e-7be4-4093-bad7-f4bf16a9fad4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:44.700199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8e88e-7be4-4093-bad7-f4bf16a9fad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:44.734707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-84e8e88e-7be4-4093-bad7-f4bf16a9fad4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:44.736151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-84e8e88e-7be4-4093-bad7-f4bf16a9fad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:44.852889 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2488/5027] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:44.862742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:44.865049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:44.965353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545c755-d687-4761-8ce1-176632b53c37 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:44.969024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5687b7-dce5-4366-9bf0-f6ed0e806c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:44.969674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5687b7-dce5-4366-9bf0-f6ed0e806c2b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:44.969945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5687b7-dce5-4366-9bf0-f6ed0e806c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:44.970207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5687b7-dce5-4366-9bf0-f6ed0e806c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:44.980118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5687b7-dce5-4366-9bf0-f6ed0e806c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:44.981599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5687b7-dce5-4366-9bf0-f6ed0e806c2b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:44.991559 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2540/5028] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:45.343583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8e88e-7be4-4093-bad7-f4bf16a9fad4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:45.345262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-84e8e88e-7be4-4093-bad7-f4bf16a9fad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:45.367190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8e88e-7be4-4093-bad7-f4bf16a9fad4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:45.372478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:45.373935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:45.373935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:45.373935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] PATH_INFO: `/v3/projects/21945e06c391470584cfbf61d808859b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:45.375261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21945e06c391470584cfbf61d808859b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:45.378608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:45.443375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21945e06c391470584cfbf61d808859b', 'request_id': 'req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:45.451827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21945e06c391470584cfbf61d808859b', 'request_id': 'req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:45.453750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:45.463716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21945e06c391470584cfbf61d808859b', 'request_id': 'req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:45.478718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] The token cache is being invalidate because project 21945e06c391470584cfbf61d808859b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:45.479103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:45.480055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:45.481968 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2489/5029] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:45 2022] DELETE /identity/v3/projects/21945e06c391470584cfbf61d808859b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:45.485866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5687b7-dce5-4366-9bf0-f6ed0e806c2b service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:45.496455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5687b7-dce5-4366-9bf0-f6ed0e806c2b service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:45.571315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a88256-6900-4902-838a-a29949802df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:45.572765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a88256-6900-4902-838a-a29949802df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:45.574017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a88256-6900-4902-838a-a29949802df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:45.574474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a88256-6900-4902-838a-a29949802df0 admin admin] PATH_INFO: `/v3/users/1159d8ea06cf4778b005e6957a5bb830` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:45.576787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a88256-6900-4902-838a-a29949802df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1159d8ea06cf4778b005e6957a5bb830)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:45.579045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a88256-6900-4902-838a-a29949802df0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:45.581053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:45.635334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c2a88256-6900-4902-838a-a29949802df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1159d8ea06cf4778b005e6957a5bb830'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:45.645697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c2a88256-6900-4902-838a-a29949802df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1159d8ea06cf4778b005e6957a5bb830'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:45.656843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c2a88256-6900-4902-838a-a29949802df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1159d8ea06cf4778b005e6957a5bb830'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:45.670828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c2a88256-6900-4902-838a-a29949802df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1159d8ea06cf4778b005e6957a5bb830'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:45.671679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c2a88256-6900-4902-838a-a29949802df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1159d8ea06cf4778b005e6957a5bb830'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:45.680467 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2541/5030] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:45 2022] DELETE /identity/v3/users/1159d8ea06cf4778b005e6957a5bb830 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:45.981572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:45.983753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:46.003860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ae5f5-d290-4d13-a18a-3e4f83831cf1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:46.094032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:46.094774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:46.095115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:46.095450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] PATH_INFO: `/v3/projects/9a64ef5bb0c84698a5a48d5f787791b5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:46.096683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a64ef5bb0c84698a5a48d5f787791b5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:46.099087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:46.159162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a64ef5bb0c84698a5a48d5f787791b5', 'request_id': 'req-12c54a61-ccbf-4586-b6ff-72f184ce74a3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:46.166326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a64ef5bb0c84698a5a48d5f787791b5', 'request_id': 'req-12c54a61-ccbf-4586-b6ff-72f184ce74a3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:46.176013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a64ef5bb0c84698a5a48d5f787791b5', 'request_id': 'req-12c54a61-ccbf-4586-b6ff-72f184ce74a3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:46.190350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] The token cache is being invalidate because project 9a64ef5bb0c84698a5a48d5f787791b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:46.191059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:46.191918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:46.193776 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2490/5031] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:45 2022] DELETE /identity/v3/projects/9a64ef5bb0c84698a5a48d5f787791b5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:46.201019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04816f8c-e884-4c63-adb7-c228000c19bf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:46.201297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04816f8c-e884-4c63-adb7-c228000c19bf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:46.201593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04816f8c-e884-4c63-adb7-c228000c19bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:46.234368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-04816f8c-e884-4c63-adb7-c228000c19bf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:46.236298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-04816f8c-e884-4c63-adb7-c228000c19bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:46.248724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:46.250811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:46.322023 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2542/5032] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:46.329927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f703576-be65-4415-b839-7c6da277bc5d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:46.331699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f703576-be65-4415-b839-7c6da277bc5d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:46.332064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f703576-be65-4415-b839-7c6da277bc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:46.334621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c54a61-ccbf-4586-b6ff-72f184ce74a3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:46.341433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857c7ef-1985-4f43-a251-2c78e27bf5f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:46.342420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857c7ef-1985-4f43-a251-2c78e27bf5f0 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:46.343480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857c7ef-1985-4f43-a251-2c78e27bf5f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:46.344134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857c7ef-1985-4f43-a251-2c78e27bf5f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:46.346650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7857c7ef-1985-4f43-a251-2c78e27bf5f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:46.366265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5f703576-be65-4415-b839-7c6da277bc5d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:46.367952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5f703576-be65-4415-b839-7c6da277bc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:46.377831 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2543/5033] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:46.381858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f703576-be65-4415-b839-7c6da277bc5d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:46.400057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:46.404942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f703576-be65-4415-b839-7c6da277bc5d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:46.413726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08393085-83a1-44b6-a91d-190244f2c4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:46.416147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08393085-83a1-44b6-a91d-190244f2c4ad admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:46.416696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08393085-83a1-44b6-a91d-190244f2c4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:46.417532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08393085-83a1-44b6-a91d-190244f2c4ad admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:46.418517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08393085-83a1-44b6-a91d-190244f2c4ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:46.419068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08393085-83a1-44b6-a91d-190244f2c4ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:46.421180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08393085-83a1-44b6-a91d-190244f2c4ad admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:46.448342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857c7ef-1985-4f43-a251-2c78e27bf5f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:46.449795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857c7ef-1985-4f43-a251-2c78e27bf5f0 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:46.461582 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2544/5034] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:10:46.463814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08393085-83a1-44b6-a91d-190244f2c4ad admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:46.464717 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2491/5035] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:46.495247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08393085-83a1-44b6-a91d-190244f2c4ad admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:46.502301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b0390-0613-4b9d-b1f5-8d2e8102c23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:46.503539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b0390-0613-4b9d-b1f5-8d2e8102c23d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:46.504234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b0390-0613-4b9d-b1f5-8d2e8102c23d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:46.504983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b0390-0613-4b9d-b1f5-8d2e8102c23d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:46.506143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b0390-0613-4b9d-b1f5-8d2e8102c23d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:46.508896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b0390-0613-4b9d-b1f5-8d2e8102c23d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:46.548163 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2545/5036] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:46 2022] POST /identity/v3/projects => generated 363 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:46.558205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857c7ef-1985-4f43-a251-2c78e27bf5f0 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:46.574094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857c7ef-1985-4f43-a251-2c78e27bf5f0 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:46.650315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919962e-c4f4-45c3-915d-9d690af36954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:46.650927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919962e-c4f4-45c3-915d-9d690af36954 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:46.651204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919962e-c4f4-45c3-915d-9d690af36954 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:46.651451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919962e-c4f4-45c3-915d-9d690af36954 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:46.652087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919962e-c4f4-45c3-915d-9d690af36954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:46.654064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919962e-c4f4-45c3-915d-9d690af36954 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:46.680009 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2492/5037] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:46 2022] POST /identity/v3/users => generated 282 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:46.681270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b0390-0613-4b9d-b1f5-8d2e8102c23d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:46.693466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b0390-0613-4b9d-b1f5-8d2e8102c23d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:46.698745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd3ea0-0ae0-4dbb-9f76-c3577a538c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:46.699391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd3ea0-0ae0-4dbb-9f76-c3577a538c2f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:46.699681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd3ea0-0ae0-4dbb-9f76-c3577a538c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:46.699946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd3ea0-0ae0-4dbb-9f76-c3577a538c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:46.700680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd3ea0-0ae0-4dbb-9f76-c3577a538c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:46.702303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd3ea0-0ae0-4dbb-9f76-c3577a538c2f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:46.713450 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2546/5038] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:46.717847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919962e-c4f4-45c3-915d-9d690af36954 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:46.729326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919962e-c4f4-45c3-915d-9d690af36954 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:46.734611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4152194-711c-408d-b9c4-afe411b8fe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:46.735347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4152194-711c-408d-b9c4-afe411b8fe0b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:46.736513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4152194-711c-408d-b9c4-afe411b8fe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:46.736513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4152194-711c-408d-b9c4-afe411b8fe0b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:46.736917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4152194-711c-408d-b9c4-afe411b8fe0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:46.738655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4152194-711c-408d-b9c4-afe411b8fe0b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:46.749830 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2493/5039] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:46.753798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd3ea0-0ae0-4dbb-9f76-c3577a538c2f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:46.763900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd3ea0-0ae0-4dbb-9f76-c3577a538c2f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:46.768743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ede47-8b7c-47f6-a8d2-87b8c4dcb9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:46.769634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ede47-8b7c-47f6-a8d2-87b8c4dcb9df admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:46.769998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ede47-8b7c-47f6-a8d2-87b8c4dcb9df admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:46.770273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ede47-8b7c-47f6-a8d2-87b8c4dcb9df admin admin] PATH_INFO: `/v3/projects/baa8fe0a18544deab360decffd074333/users/4f9ed9209564470c96c01a82f5ead19e/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:46.785449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ede47-8b7c-47f6-a8d2-87b8c4dcb9df admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa8fe0a18544deab360decffd074333, user_id=4f9ed9209564470c96c01a82f5ead19e, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:46.788254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ede47-8b7c-47f6-a8d2-87b8c4dcb9df admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:46.810391 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2547/5040] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:46 2022] PUT /identity/v3/projects/baa8fe0a18544deab360decffd074333/users/4f9ed9209564470c96c01a82f5ead19e/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:46.821289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d383082-4eae-4d31-abe8-79a4000bcd62 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:46.821571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d383082-4eae-4d31-abe8-79a4000bcd62 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:46.821839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d383082-4eae-4d31-abe8-79a4000bcd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:46.853533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5d383082-4eae-4d31-abe8-79a4000bcd62 None None] MFA Rules not processed for user `4f9ed9209564470c96c01a82f5ead19e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:46.855025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5d383082-4eae-4d31-abe8-79a4000bcd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:46.937284 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2494/5041] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:46.938725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ede47-8b7c-47f6-a8d2-87b8c4dcb9df admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:46.944573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:47.017300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ede47-8b7c-47f6-a8d2-87b8c4dcb9df admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:47.023429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade52670-a79f-4b20-98b5-f553d43ed8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:47.024141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade52670-a79f-4b20-98b5-f553d43ed8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:47.024508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade52670-a79f-4b20-98b5-f553d43ed8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:47.024837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade52670-a79f-4b20-98b5-f553d43ed8f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:47.025534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade52670-a79f-4b20-98b5-f553d43ed8f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:47.027499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade52670-a79f-4b20-98b5-f553d43ed8f9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:47.058299 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2548/5042] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:46 2022] POST /identity/v3/projects => generated 363 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:47.059544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d383082-4eae-4d31-abe8-79a4000bcd62 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:47.071582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d383082-4eae-4d31-abe8-79a4000bcd62 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:47.122193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:47.151629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b7eaa-4700-4c89-9b04-2da5e2dfa4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:47.152441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b7eaa-4700-4c89-9b04-2da5e2dfa4ae admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:47.152717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b7eaa-4700-4c89-9b04-2da5e2dfa4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:47.152982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b7eaa-4700-4c89-9b04-2da5e2dfa4ae admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:47.153623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b7eaa-4700-4c89-9b04-2da5e2dfa4ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:47.155547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b7eaa-4700-4c89-9b04-2da5e2dfa4ae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:47.181272 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2495/5043] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:47 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:47.185084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade52670-a79f-4b20-98b5-f553d43ed8f9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:47.196890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade52670-a79f-4b20-98b5-f553d43ed8f9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:47.201773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb34e4-4484-4d8d-a131-8231858f8535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:47.202448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb34e4-4484-4d8d-a131-8231858f8535 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:47.202790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb34e4-4484-4d8d-a131-8231858f8535 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:47.203125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb34e4-4484-4d8d-a131-8231858f8535 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:47.203835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb34e4-4484-4d8d-a131-8231858f8535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:47.205626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb34e4-4484-4d8d-a131-8231858f8535 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:47.216782 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2549/5044] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:47.221248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b7eaa-4700-4c89-9b04-2da5e2dfa4ae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:47.234117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b7eaa-4700-4c89-9b04-2da5e2dfa4ae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:47.239722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a277df-1c0e-4aaa-b24d-442c27c43dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:47.241254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a277df-1c0e-4aaa-b24d-442c27c43dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:47.243092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a277df-1c0e-4aaa-b24d-442c27c43dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:47.243422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a277df-1c0e-4aaa-b24d-442c27c43dc8 admin admin] PATH_INFO: `/v3/projects/8a3d6197a60a4e4d8f35bc0359aa90be/users/8f7fb88129534d9ea403b4038bcdd51f/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:47.259356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a277df-1c0e-4aaa-b24d-442c27c43dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a3d6197a60a4e4d8f35bc0359aa90be, user_id=8f7fb88129534d9ea403b4038bcdd51f, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:47.262333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a277df-1c0e-4aaa-b24d-442c27c43dc8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:47.275592 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2496/5045] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:47 2022] PUT /identity/v3/projects/8a3d6197a60a4e4d8f35bc0359aa90be/users/8f7fb88129534d9ea403b4038bcdd51f/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:47.279511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb34e4-4484-4d8d-a131-8231858f8535 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:47.291155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb34e4-4484-4d8d-a131-8231858f8535 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:47.358611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae66b4d-cd13-4d02-86f7-d76c077c2bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:47.359226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae66b4d-cd13-4d02-86f7-d76c077c2bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:47.359519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae66b4d-cd13-4d02-86f7-d76c077c2bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:47.359790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae66b4d-cd13-4d02-86f7-d76c077c2bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:47.360487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae66b4d-cd13-4d02-86f7-d76c077c2bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:47.362088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae66b4d-cd13-4d02-86f7-d76c077c2bf5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:47.376534 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2550/5046] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:47.377965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a277df-1c0e-4aaa-b24d-442c27c43dc8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:47.390404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a277df-1c0e-4aaa-b24d-442c27c43dc8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:47.395261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647540c-e21c-48ef-97d2-7e87dab8147c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:47.395919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647540c-e21c-48ef-97d2-7e87dab8147c admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:47.396188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647540c-e21c-48ef-97d2-7e87dab8147c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:47.396481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647540c-e21c-48ef-97d2-7e87dab8147c admin admin] PATH_INFO: `/v3/projects/8a3d6197a60a4e4d8f35bc0359aa90be/users/8f7fb88129534d9ea403b4038bcdd51f/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:47.398683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647540c-e21c-48ef-97d2-7e87dab8147c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a3d6197a60a4e4d8f35bc0359aa90be, user_id=8f7fb88129534d9ea403b4038bcdd51f, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:47.401396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647540c-e21c-48ef-97d2-7e87dab8147c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:47.412638 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2497/5047] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:47 2022] PUT /identity/v3/projects/8a3d6197a60a4e4d8f35bc0359aa90be/users/8f7fb88129534d9ea403b4038bcdd51f/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:47.424577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b74cd-eecd-4de2-b6d3-8037890166f0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:47.424977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b74cd-eecd-4de2-b6d3-8037890166f0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:47.425951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b74cd-eecd-4de2-b6d3-8037890166f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:47.463455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d01b74cd-eecd-4de2-b6d3-8037890166f0 None None] MFA Rules not processed for user `8f7fb88129534d9ea403b4038bcdd51f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:47.464844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d01b74cd-eecd-4de2-b6d3-8037890166f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:47.543402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:47.563230 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2551/5048] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:47.591959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647540c-e21c-48ef-97d2-7e87dab8147c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:47.598509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6647540c-e21c-48ef-97d2-7e87dab8147c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:47.699518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647540c-e21c-48ef-97d2-7e87dab8147c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:47.703628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6917ac-73aa-4c52-ab60-6d43694e38ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:47.704518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6917ac-73aa-4c52-ab60-6d43694e38ad service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:47.705216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6917ac-73aa-4c52-ab60-6d43694e38ad service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:47.705641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6917ac-73aa-4c52-ab60-6d43694e38ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:47.717207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6917ac-73aa-4c52-ab60-6d43694e38ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:47.718930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6917ac-73aa-4c52-ab60-6d43694e38ad service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:47.730872 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2498/5049] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:47 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:47.826110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b74cd-eecd-4de2-b6d3-8037890166f0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:47.863072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b74cd-eecd-4de2-b6d3-8037890166f0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:47.868279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359500c-24f4-4b59-b623-301330bc08f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:47.869090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359500c-24f4-4b59-b623-301330bc08f4 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:47.869876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359500c-24f4-4b59-b623-301330bc08f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:47.869876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359500c-24f4-4b59-b623-301330bc08f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:47.933164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359500c-24f4-4b59-b623-301330bc08f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:47.935684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359500c-24f4-4b59-b623-301330bc08f4 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:47.951983 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2552/5050] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:49.686485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6917ac-73aa-4c52-ab60-6d43694e38ad service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:49.702580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359500c-24f4-4b59-b623-301330bc08f4 service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:49.704569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a359500c-24f4-4b59-b623-301330bc08f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:49.858172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6917ac-73aa-4c52-ab60-6d43694e38ad service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:49.868191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359500c-24f4-4b59-b623-301330bc08f4 service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:49.868877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:49.869539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:49.869814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:49.870142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] PATH_INFO: `/v3/users/4f9ed9209564470c96c01a82f5ead19e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:49.871728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f9ed9209564470c96c01a82f5ead19e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:49.873720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:49.884475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b2787-698c-4467-bd48-de32b74e0cb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:49.886687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2787-698c-4467-bd48-de32b74e0cb3 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:49.887170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2787-698c-4467-bd48-de32b74e0cb3 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:49.888655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2787-698c-4467-bd48-de32b74e0cb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:49.893327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cc8b2787-698c-4467-bd48-de32b74e0cb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:49.922663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:49.931533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b2787-698c-4467-bd48-de32b74e0cb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:49.933744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b2787-698c-4467-bd48-de32b74e0cb3 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:49.955385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f9ed9209564470c96c01a82f5ead19e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:49.963573 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2553/5051] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:49.972775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f9ed9209564470c96c01a82f5ead19e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:50.008420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f9ed9209564470c96c01a82f5ead19e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:50.008420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f9ed9209564470c96c01a82f5ead19e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:50.008420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f9ed9209564470c96c01a82f5ead19e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:50.023137 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2499/5052] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:49 2022] DELETE /identity/v3/users/4f9ed9209564470c96c01a82f5ead19e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:50.027995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567478d-b41e-4274-a425-10df7e59f76a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:50.028398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567478d-b41e-4274-a425-10df7e59f76a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:50.028739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567478d-b41e-4274-a425-10df7e59f76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:50.065989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1567478d-b41e-4274-a425-10df7e59f76a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:50.068016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1567478d-b41e-4274-a425-10df7e59f76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:50.155279 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2554/5053] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:50.168159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:50.170020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:50.258303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29193118-d77a-44a0-a254-8424916ff142 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:50.262064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2e1f5-b0bc-4ad6-8cc5-2a9cf4213180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:50.263669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2e1f5-b0bc-4ad6-8cc5-2a9cf4213180 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:50.264044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2e1f5-b0bc-4ad6-8cc5-2a9cf4213180 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:50.264399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2e1f5-b0bc-4ad6-8cc5-2a9cf4213180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:50.274825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2e1f5-b0bc-4ad6-8cc5-2a9cf4213180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:50.276315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2e1f5-b0bc-4ad6-8cc5-2a9cf4213180 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:50.288399 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2500/5054] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:50.769579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567478d-b41e-4274-a425-10df7e59f76a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:50.771493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1567478d-b41e-4274-a425-10df7e59f76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:50.798615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567478d-b41e-4274-a425-10df7e59f76a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:50.814687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:50.821157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:50.822805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:50.823219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] PATH_INFO: `/v3/projects/baa8fe0a18544deab360decffd074333` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:50.825556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baa8fe0a18544deab360decffd074333)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:50.829949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:50.869910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:50.906057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baa8fe0a18544deab360decffd074333', 'request_id': 'req-31faa4c2-a4fc-4f51-804f-448096750f64'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:50.915759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baa8fe0a18544deab360decffd074333', 'request_id': 'req-31faa4c2-a4fc-4f51-804f-448096750f64'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:50.923923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baa8fe0a18544deab360decffd074333', 'request_id': 'req-31faa4c2-a4fc-4f51-804f-448096750f64'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:50.936746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] The token cache is being invalidate because project baa8fe0a18544deab360decffd074333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:50.937035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:50.937695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:50.939240 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2555/5055] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:50 2022] DELETE /identity/v3/projects/baa8fe0a18544deab360decffd074333 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:50.944197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2e1f5-b0bc-4ad6-8cc5-2a9cf4213180 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:50.955801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2e1f5-b0bc-4ad6-8cc5-2a9cf4213180 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:51.047072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc2ad2-533f-487f-853a-42fca5d7e95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:51.047886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2ad2-533f-487f-853a-42fca5d7e95e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:51.048229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2ad2-533f-487f-853a-42fca5d7e95e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:51.048641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2ad2-533f-487f-853a-42fca5d7e95e admin admin] PATH_INFO: `/v3/users/8f7fb88129534d9ea403b4038bcdd51f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:51.050263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc2ad2-533f-487f-853a-42fca5d7e95e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f7fb88129534d9ea403b4038bcdd51f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:51.052738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc2ad2-533f-487f-853a-42fca5d7e95e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:51.131012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e1bc2ad2-533f-487f-853a-42fca5d7e95e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f7fb88129534d9ea403b4038bcdd51f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:51.139980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e1bc2ad2-533f-487f-853a-42fca5d7e95e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f7fb88129534d9ea403b4038bcdd51f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:51.143069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:51.168701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e1bc2ad2-533f-487f-853a-42fca5d7e95e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f7fb88129534d9ea403b4038bcdd51f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:51.169565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e1bc2ad2-533f-487f-853a-42fca5d7e95e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f7fb88129534d9ea403b4038bcdd51f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:51.170239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e1bc2ad2-533f-487f-853a-42fca5d7e95e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f7fb88129534d9ea403b4038bcdd51f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:51.193317 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2501/5056] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:50 2022] DELETE /identity/v3/users/8f7fb88129534d9ea403b4038bcdd51f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:51.560457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:51.562305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:51.588435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31faa4c2-a4fc-4f51-804f-448096750f64 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:51.661026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac681d7-97d9-4fe3-bbef-add703f965e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:51.661781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac681d7-97d9-4fe3-bbef-add703f965e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:51.662185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac681d7-97d9-4fe3-bbef-add703f965e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:51.662460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac681d7-97d9-4fe3-bbef-add703f965e1 admin admin] PATH_INFO: `/v3/projects/8a3d6197a60a4e4d8f35bc0359aa90be` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:51.663609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac681d7-97d9-4fe3-bbef-add703f965e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a3d6197a60a4e4d8f35bc0359aa90be)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:51.665601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac681d7-97d9-4fe3-bbef-add703f965e1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:51.715486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cac681d7-97d9-4fe3-bbef-add703f965e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a3d6197a60a4e4d8f35bc0359aa90be', 'request_id': 'req-cac681d7-97d9-4fe3-bbef-add703f965e1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:51.731951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cac681d7-97d9-4fe3-bbef-add703f965e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a3d6197a60a4e4d8f35bc0359aa90be', 'request_id': 'req-cac681d7-97d9-4fe3-bbef-add703f965e1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:51.739592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cac681d7-97d9-4fe3-bbef-add703f965e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a3d6197a60a4e4d8f35bc0359aa90be', 'request_id': 'req-cac681d7-97d9-4fe3-bbef-add703f965e1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:51.751013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cac681d7-97d9-4fe3-bbef-add703f965e1 admin admin] The token cache is being invalidate because project 8a3d6197a60a4e4d8f35bc0359aa90be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:51.751300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cac681d7-97d9-4fe3-bbef-add703f965e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:51.751967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cac681d7-97d9-4fe3-bbef-add703f965e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:51.753681 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2556/5057] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:51 2022] DELETE /identity/v3/projects/8a3d6197a60a4e4d8f35bc0359aa90be => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:51.758854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61e48f-7021-4e1c-918c-cf3b57ead5b2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:51.759214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61e48f-7021-4e1c-918c-cf3b57ead5b2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:51.759584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61e48f-7021-4e1c-918c-cf3b57ead5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:51.780966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-af61e48f-7021-4e1c-918c-cf3b57ead5b2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:51.782440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-af61e48f-7021-4e1c-918c-cf3b57ead5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:51.857276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f6de-7470-4231-a9df-c1e670275ce1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:51.857832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f6de-7470-4231-a9df-c1e670275ce1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:51.857832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f6de-7470-4231-a9df-c1e670275ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:51.860503 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2502/5058] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:51.885482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b801f6de-7470-4231-a9df-c1e670275ce1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:51.886776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b801f6de-7470-4231-a9df-c1e670275ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:51.901983 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2557/5059] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:51.910421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61e48f-7021-4e1c-918c-cf3b57ead5b2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:51.920390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61e48f-7021-4e1c-918c-cf3b57ead5b2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:51.933527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98149476-59ff-4ca7-887b-fcae778bae53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:51.934827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98149476-59ff-4ca7-887b-fcae778bae53 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:51.935451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98149476-59ff-4ca7-887b-fcae778bae53 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:51.935980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98149476-59ff-4ca7-887b-fcae778bae53 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:51.936895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98149476-59ff-4ca7-887b-fcae778bae53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:51.937290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98149476-59ff-4ca7-887b-fcae778bae53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:51.939787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98149476-59ff-4ca7-887b-fcae778bae53 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:51.957845 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2503/5060] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:51.960934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801f6de-7470-4231-a9df-c1e670275ce1 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:51.971030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801f6de-7470-4231-a9df-c1e670275ce1 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:51.996683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a37ac-1493-4151-bc45-3d1cbfa42c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:51.997517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a37ac-1493-4151-bc45-3d1cbfa42c9d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:51.997947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a37ac-1493-4151-bc45-3d1cbfa42c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:51.998392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a37ac-1493-4151-bc45-3d1cbfa42c9d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:52.004720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a37ac-1493-4151-bc45-3d1cbfa42c9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:52.006768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a37ac-1493-4151-bc45-3d1cbfa42c9d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:52.066704 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2558/5061] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:51 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:52.071375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98149476-59ff-4ca7-887b-fcae778bae53 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:52.090243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98149476-59ff-4ca7-887b-fcae778bae53 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:52.249073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a50407-ce9a-450d-b6ba-ca2996d73cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:52.250048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a50407-ce9a-450d-b6ba-ca2996d73cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:52.250384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a50407-ce9a-450d-b6ba-ca2996d73cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:52.250769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a50407-ce9a-450d-b6ba-ca2996d73cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:52.251424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a50407-ce9a-450d-b6ba-ca2996d73cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:52.253623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a50407-ce9a-450d-b6ba-ca2996d73cb2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:52.282760 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2504/5062] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:52 2022] POST /identity/v3/users => generated 282 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:52.286924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a37ac-1493-4151-bc45-3d1cbfa42c9d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:52.311683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a37ac-1493-4151-bc45-3d1cbfa42c9d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:52.322270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8901d27-02a0-4d55-8f04-1029760aa048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:52.323128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8901d27-02a0-4d55-8f04-1029760aa048 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:52.323495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8901d27-02a0-4d55-8f04-1029760aa048 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:52.323856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8901d27-02a0-4d55-8f04-1029760aa048 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:52.324635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8901d27-02a0-4d55-8f04-1029760aa048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:52.326508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8901d27-02a0-4d55-8f04-1029760aa048 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:52.328323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:52.343597 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2559/5063] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:52.347829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a50407-ce9a-450d-b6ba-ca2996d73cb2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:52.358977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a50407-ce9a-450d-b6ba-ca2996d73cb2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:52.364179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d8304-4d6d-4f49-8803-3cb7436e3a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:52.365162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d8304-4d6d-4f49-8803-3cb7436e3a90 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:52.365523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d8304-4d6d-4f49-8803-3cb7436e3a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:52.365839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d8304-4d6d-4f49-8803-3cb7436e3a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:52.366561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d8304-4d6d-4f49-8803-3cb7436e3a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:52.368238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d8304-4d6d-4f49-8803-3cb7436e3a90 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:52.380550 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2505/5064] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:52.384872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8901d27-02a0-4d55-8f04-1029760aa048 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:52.395770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8901d27-02a0-4d55-8f04-1029760aa048 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:52.400486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa767dc0-3ff6-4491-9000-97d12b46b1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:52.401263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa767dc0-3ff6-4491-9000-97d12b46b1fc admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:52.401614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa767dc0-3ff6-4491-9000-97d12b46b1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:52.401945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa767dc0-3ff6-4491-9000-97d12b46b1fc admin admin] PATH_INFO: `/v3/projects/03de8dce8aaa4749bf75ac3553035b7c/users/2ef84f959ef949c087b772be3d8948ef/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:52.418959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa767dc0-3ff6-4491-9000-97d12b46b1fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=03de8dce8aaa4749bf75ac3553035b7c, user_id=2ef84f959ef949c087b772be3d8948ef, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:52.422041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa767dc0-3ff6-4491-9000-97d12b46b1fc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:52.437567 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2560/5065] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:52 2022] PUT /identity/v3/projects/03de8dce8aaa4749bf75ac3553035b7c/users/2ef84f959ef949c087b772be3d8948ef/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:52.449979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5e151-20e4-4d0e-b89c-a6647d57ccb6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:52.450296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5e151-20e4-4d0e-b89c-a6647d57ccb6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:52.450634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5e151-20e4-4d0e-b89c-a6647d57ccb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:52.479949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-aeb5e151-20e4-4d0e-b89c-a6647d57ccb6 None None] MFA Rules not processed for user `2ef84f959ef949c087b772be3d8948ef`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:52.481253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-aeb5e151-20e4-4d0e-b89c-a6647d57ccb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:52.508199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:52.568514 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2506/5066] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:52.571869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa767dc0-3ff6-4491-9000-97d12b46b1fc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:52.663589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa767dc0-3ff6-4491-9000-97d12b46b1fc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:52.670368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22848139-8736-43c5-90c9-bcdb1d02a7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:52.671001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22848139-8736-43c5-90c9-bcdb1d02a7e8 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:52.671277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22848139-8736-43c5-90c9-bcdb1d02a7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:52.671717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22848139-8736-43c5-90c9-bcdb1d02a7e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:52.673393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22848139-8736-43c5-90c9-bcdb1d02a7e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:52.677477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22848139-8736-43c5-90c9-bcdb1d02a7e8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:52.739349 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2561/5067] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:52 2022] POST /identity/v3/projects => generated 363 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:52.743504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5e151-20e4-4d0e-b89c-a6647d57ccb6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:52.759068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5e151-20e4-4d0e-b89c-a6647d57ccb6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:52.829093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60aa36-6034-407e-814c-f2f09060efef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:52.829810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60aa36-6034-407e-814c-f2f09060efef admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:52.830182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60aa36-6034-407e-814c-f2f09060efef admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:52.830516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60aa36-6034-407e-814c-f2f09060efef admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:52.831418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60aa36-6034-407e-814c-f2f09060efef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:52.834648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60aa36-6034-407e-814c-f2f09060efef admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:52.864394 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2507/5068] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:52 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:52.865535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22848139-8736-43c5-90c9-bcdb1d02a7e8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:52.876761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22848139-8736-43c5-90c9-bcdb1d02a7e8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:52.882330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c5aa6-e737-4ee8-9ae9-69f5ebefc997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:52.882736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c5aa6-e737-4ee8-9ae9-69f5ebefc997 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:52.883342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c5aa6-e737-4ee8-9ae9-69f5ebefc997 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:52.883645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c5aa6-e737-4ee8-9ae9-69f5ebefc997 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:52.884239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c5aa6-e737-4ee8-9ae9-69f5ebefc997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:52.886208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c5aa6-e737-4ee8-9ae9-69f5ebefc997 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:52.903064 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2562/5069] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:52.904001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60aa36-6034-407e-814c-f2f09060efef admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:52.914735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60aa36-6034-407e-814c-f2f09060efef admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:52.929882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7651f57-1659-41c1-94a8-bfedfc48f156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:52.930714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7651f57-1659-41c1-94a8-bfedfc48f156 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:52.930995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7651f57-1659-41c1-94a8-bfedfc48f156 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:52.931264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7651f57-1659-41c1-94a8-bfedfc48f156 admin admin] PATH_INFO: `/v3/projects/c933a20ba9644d0e909136ab67e35381/users/11e6e4a57ed84e5791bfaaa059894354/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:52.965127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7651f57-1659-41c1-94a8-bfedfc48f156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c933a20ba9644d0e909136ab67e35381, user_id=11e6e4a57ed84e5791bfaaa059894354, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:52.969802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7651f57-1659-41c1-94a8-bfedfc48f156 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:52.988648 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2508/5070] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:52 2022] PUT /identity/v3/projects/c933a20ba9644d0e909136ab67e35381/users/11e6e4a57ed84e5791bfaaa059894354/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:52.992640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c5aa6-e737-4ee8-9ae9-69f5ebefc997 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:53.004351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c5aa6-e737-4ee8-9ae9-69f5ebefc997 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:53.084179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5d045-2d77-464d-8097-427a46b4a345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:53.085128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5d045-2d77-464d-8097-427a46b4a345 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:53.085406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5d045-2d77-464d-8097-427a46b4a345 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:53.085675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5d045-2d77-464d-8097-427a46b4a345 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:53.086320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5d045-2d77-464d-8097-427a46b4a345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:53.087969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5d045-2d77-464d-8097-427a46b4a345 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:53.101525 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2563/5071] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:53.104683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7651f57-1659-41c1-94a8-bfedfc48f156 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:53.117275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7651f57-1659-41c1-94a8-bfedfc48f156 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:53.121918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c651c-1b37-491c-a2ca-30d0708e9ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:53.122580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c651c-1b37-491c-a2ca-30d0708e9ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:53.122852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c651c-1b37-491c-a2ca-30d0708e9ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:53.123115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c651c-1b37-491c-a2ca-30d0708e9ff0 admin admin] PATH_INFO: `/v3/projects/c933a20ba9644d0e909136ab67e35381/users/11e6e4a57ed84e5791bfaaa059894354/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:53.125055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c651c-1b37-491c-a2ca-30d0708e9ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c933a20ba9644d0e909136ab67e35381, user_id=11e6e4a57ed84e5791bfaaa059894354, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:53.127713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c651c-1b37-491c-a2ca-30d0708e9ff0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:53.140251 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2509/5072] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:53 2022] PUT /identity/v3/projects/c933a20ba9644d0e909136ab67e35381/users/11e6e4a57ed84e5791bfaaa059894354/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:53.150743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f31bbc-7ede-46e8-a5f1-b915944ac123 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:53.151170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f31bbc-7ede-46e8-a5f1-b915944ac123 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:53.151576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f31bbc-7ede-46e8-a5f1-b915944ac123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:53.183175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-86f31bbc-7ede-46e8-a5f1-b915944ac123 None None] MFA Rules not processed for user `11e6e4a57ed84e5791bfaaa059894354`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:53.184392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-86f31bbc-7ede-46e8-a5f1-b915944ac123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:53.215243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:53.258152 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2564/5073] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:53.271128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c651c-1b37-491c-a2ca-30d0708e9ff0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:53.273033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-994c651c-1b37-491c-a2ca-30d0708e9ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:53.334534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c651c-1b37-491c-a2ca-30d0708e9ff0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:53.339294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90dec19-ee5a-4b8e-991c-23af0ff247f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:53.339922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dec19-ee5a-4b8e-991c-23af0ff247f7 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:53.340193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dec19-ee5a-4b8e-991c-23af0ff247f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:53.340451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dec19-ee5a-4b8e-991c-23af0ff247f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:53.350071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90dec19-ee5a-4b8e-991c-23af0ff247f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:53.351504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90dec19-ee5a-4b8e-991c-23af0ff247f7 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:53.361272 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2510/5074] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:53.462666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f31bbc-7ede-46e8-a5f1-b915944ac123 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:53.474691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f31bbc-7ede-46e8-a5f1-b915944ac123 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:53.478733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288e6df-9b97-4ff7-8b6d-b05b7dbd6182 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:53.479430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e6df-9b97-4ff7-8b6d-b05b7dbd6182 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:53.479771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e6df-9b97-4ff7-8b6d-b05b7dbd6182 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:53.480126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e6df-9b97-4ff7-8b6d-b05b7dbd6182 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:53.537915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288e6df-9b97-4ff7-8b6d-b05b7dbd6182 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:53.539363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288e6df-9b97-4ff7-8b6d-b05b7dbd6182 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:53.548535 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2565/5075] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:55.159237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90dec19-ee5a-4b8e-991c-23af0ff247f7 service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:55.191576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:55.262212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90dec19-ee5a-4b8e-991c-23af0ff247f7 service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:55.267478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:55.268613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:55.269329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:55.269725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] PATH_INFO: `/v3/users/2ef84f959ef949c087b772be3d8948ef` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:55.271037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef84f959ef949c087b772be3d8948ef)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:55.273164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:55.370698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef84f959ef949c087b772be3d8948ef'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:55.385666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef84f959ef949c087b772be3d8948ef'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:55.419418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef84f959ef949c087b772be3d8948ef'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:55.420510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef84f959ef949c087b772be3d8948ef'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:55.421659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef84f959ef949c087b772be3d8948ef'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:55.452271 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2511/5076] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:55 2022] DELETE /identity/v3/users/2ef84f959ef949c087b772be3d8948ef => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:55.456817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b601b-1788-4c5b-9d08-751b83461790 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:55.457581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b601b-1788-4c5b-9d08-751b83461790 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:55.457581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b601b-1788-4c5b-9d08-751b83461790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:55.504743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-aa8b601b-1788-4c5b-9d08-751b83461790 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:55.505910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aa8b601b-1788-4c5b-9d08-751b83461790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:55.603685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:55.605380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:55.634083 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2566/5077] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:10:55.644007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b601b-1788-4c5b-9d08-751b83461790 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:55.645449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aa8b601b-1788-4c5b-9d08-751b83461790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:55.674833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a1b67-fd05-4b4b-bf74-c2de5386e445 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:55.679634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:55.680443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:55.680803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:55.681098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] PATH_INFO: `/v3/users/aa1cf46b75f1467ab8ebddb27ff9b8b9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:55.682243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa1cf46b75f1467ab8ebddb27ff9b8b9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:55.684345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:55.709487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b601b-1788-4c5b-9d08-751b83461790 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:55.714535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59bd4ec-2246-4ea8-8d04-2d4555ec40e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:55.715147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bd4ec-2246-4ea8-8d04-2d4555ec40e9 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:55.715419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bd4ec-2246-4ea8-8d04-2d4555ec40e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:55.715677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bd4ec-2246-4ea8-8d04-2d4555ec40e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:55.729669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59bd4ec-2246-4ea8-8d04-2d4555ec40e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:55.731604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59bd4ec-2246-4ea8-8d04-2d4555ec40e9 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:55.740188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa1cf46b75f1467ab8ebddb27ff9b8b9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:55.742520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:55.748604 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2567/5078] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:10:55.753194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa1cf46b75f1467ab8ebddb27ff9b8b9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:55.781284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa1cf46b75f1467ab8ebddb27ff9b8b9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:55.782378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa1cf46b75f1467ab8ebddb27ff9b8b9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:55.783146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa1cf46b75f1467ab8ebddb27ff9b8b9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:55.804068 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2512/5079] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:55 2022] DELETE /identity/v3/users/aa1cf46b75f1467ab8ebddb27ff9b8b9 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:55.808869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a71925-cd17-4040-9559-b7e9c62dd307 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:55.809152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a71925-cd17-4040-9559-b7e9c62dd307 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:55.809420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a71925-cd17-4040-9559-b7e9c62dd307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:55.836393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-27a71925-cd17-4040-9559-b7e9c62dd307 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:55.838043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-27a71925-cd17-4040-9559-b7e9c62dd307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:55.843090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:55.937730 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2568/5080] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:55.949474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:55.950937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:56.033012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b70a50-07dd-4c80-b236-c98709468fd3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:56.039459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3167f-4703-4057-aeb6-2c874a13beb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:56.040299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3167f-4703-4057-aeb6-2c874a13beb0 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:56.040779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3167f-4703-4057-aeb6-2c874a13beb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:56.040779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3167f-4703-4057-aeb6-2c874a13beb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:56.058390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3167f-4703-4057-aeb6-2c874a13beb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:56.060243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3167f-4703-4057-aeb6-2c874a13beb0 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:56.073482 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2513/5081] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:10:56.265503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a71925-cd17-4040-9559-b7e9c62dd307 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:56.267060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-27a71925-cd17-4040-9559-b7e9c62dd307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:56.280928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a71925-cd17-4040-9559-b7e9c62dd307 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:56.286743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:56.287406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:56.287681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:56.287957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] PATH_INFO: `/v3/projects/82e04a0b669e487e9e09bf61b70b74e7` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:56.289675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82e04a0b669e487e9e09bf61b70b74e7)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:56.291631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:56.344484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82e04a0b669e487e9e09bf61b70b74e7', 'request_id': 'req-e066dd2d-9494-43f2-b029-c95814600c77'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.353360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82e04a0b669e487e9e09bf61b70b74e7', 'request_id': 'req-e066dd2d-9494-43f2-b029-c95814600c77'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.359635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82e04a0b669e487e9e09bf61b70b74e7', 'request_id': 'req-e066dd2d-9494-43f2-b029-c95814600c77'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.373318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] The token cache is being invalidate because project 82e04a0b669e487e9e09bf61b70b74e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:56.373716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.374530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.376564 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2569/5082] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:56 2022] DELETE /identity/v3/projects/82e04a0b669e487e9e09bf61b70b74e7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:56.384767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3167f-4703-4057-aeb6-2c874a13beb0 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:56.398970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3167f-4703-4057-aeb6-2c874a13beb0 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:56.484809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:56.485583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:56.485831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:56.486122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] PATH_INFO: `/v3/users/87185bd5eda9492eaf4d79ce3cbf49c5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:56.487406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87185bd5eda9492eaf4d79ce3cbf49c5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:56.489925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:56.510179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:56.516121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:56.541888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066dd2d-9494-43f2-b029-c95814600c77 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:56.547706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:56.548955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:56.549262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:56.549672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] PATH_INFO: `/v3/projects/03de8dce8aaa4749bf75ac3553035b7c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:56.551302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03de8dce8aaa4749bf75ac3553035b7c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:56.553613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:56.574137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87185bd5eda9492eaf4d79ce3cbf49c5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.586021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87185bd5eda9492eaf4d79ce3cbf49c5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.601899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:56.614817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87185bd5eda9492eaf4d79ce3cbf49c5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.616742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87185bd5eda9492eaf4d79ce3cbf49c5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.617415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87185bd5eda9492eaf4d79ce3cbf49c5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.634090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03de8dce8aaa4749bf75ac3553035b7c', 'request_id': 'req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.636998 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2514/5083] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:56 2022] DELETE /identity/v3/users/87185bd5eda9492eaf4d79ce3cbf49c5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:56.653068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03de8dce8aaa4749bf75ac3553035b7c', 'request_id': 'req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.663228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03de8dce8aaa4749bf75ac3553035b7c', 'request_id': 'req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.700554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] The token cache is being invalidate because project 03de8dce8aaa4749bf75ac3553035b7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:56.701240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.702227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.704261 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2570/5084] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:56 2022] DELETE /identity/v3/projects/03de8dce8aaa4749bf75ac3553035b7c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:56.710256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:56.711700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:56.822011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61018452-4332-41b7-90e0-50a4e559e173 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:56.828687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:56.829613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:56.830440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:56.830807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] PATH_INFO: `/v3/users/11e6e4a57ed84e5791bfaaa059894354` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:56.832512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e6e4a57ed84e5791bfaaa059894354)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:56.835021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:56.921233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e6e4a57ed84e5791bfaaa059894354'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:56.947137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e6e4a57ed84e5791bfaaa059894354'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:57.029627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e6e4a57ed84e5791bfaaa059894354'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:57.031737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e6e4a57ed84e5791bfaaa059894354'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:57.034777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e6e4a57ed84e5791bfaaa059894354'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:57.080607 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2515/5085] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:56 2022] DELETE /identity/v3/users/11e6e4a57ed84e5791bfaaa059894354 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:57.555975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:57.557153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:57.577944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:57.718407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f6544-12ac-49c0-b27c-3bb1fa8e0d54 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:57.822222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:57.824567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:57.824883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:57.825149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] PATH_INFO: `/v3/projects/5e5cf321ca004485a1003925da822420` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:57.826448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e5cf321ca004485a1003925da822420)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:57.830406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:57.849227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:57.850725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:57.874082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca78226-a99e-4754-b73c-fceca97b673f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:57.885698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096657e-971e-44e7-ab32-21bf2f5029ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:57.887159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096657e-971e-44e7-ab32-21bf2f5029ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:57.887449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096657e-971e-44e7-ab32-21bf2f5029ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:57.887738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096657e-971e-44e7-ab32-21bf2f5029ca admin admin] PATH_INFO: `/v3/projects/c933a20ba9644d0e909136ab67e35381` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:57.889094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096657e-971e-44e7-ab32-21bf2f5029ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=c933a20ba9644d0e909136ab67e35381)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:57.892524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096657e-971e-44e7-ab32-21bf2f5029ca admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:57.943900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e5cf321ca004485a1003925da822420', 'request_id': 'req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:57.959839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e5cf321ca004485a1003925da822420', 'request_id': 'req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:57.971108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e5cf321ca004485a1003925da822420', 'request_id': 'req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:57.975372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0096657e-971e-44e7-ab32-21bf2f5029ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c933a20ba9644d0e909136ab67e35381', 'request_id': 'req-0096657e-971e-44e7-ab32-21bf2f5029ca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:57.987973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] The token cache is being invalidate because project 5e5cf321ca004485a1003925da822420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:57.988619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:57.989684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:57.991719 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2571/5086] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:57 2022] DELETE /identity/v3/projects/5e5cf321ca004485a1003925da822420 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:57.997001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:58.000134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:58.008245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0096657e-971e-44e7-ab32-21bf2f5029ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c933a20ba9644d0e909136ab67e35381', 'request_id': 'req-0096657e-971e-44e7-ab32-21bf2f5029ca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:58.011079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:58.023591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0096657e-971e-44e7-ab32-21bf2f5029ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c933a20ba9644d0e909136ab67e35381', 'request_id': 'req-0096657e-971e-44e7-ab32-21bf2f5029ca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:58.042391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0096657e-971e-44e7-ab32-21bf2f5029ca admin admin] The token cache is being invalidate because project c933a20ba9644d0e909136ab67e35381 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:58.042990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0096657e-971e-44e7-ab32-21bf2f5029ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:58.044252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0096657e-971e-44e7-ab32-21bf2f5029ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:58.046259 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2516/5087] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:57 2022] DELETE /identity/v3/projects/c933a20ba9644d0e909136ab67e35381 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:58.052610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3bb7d-3358-4b8d-acb1-600417a1d7bb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:58.053701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3bb7d-3358-4b8d-acb1-600417a1d7bb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:58.054085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3bb7d-3358-4b8d-acb1-600417a1d7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:58.102434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9af3bb7d-3358-4b8d-acb1-600417a1d7bb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:58.104347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9af3bb7d-3358-4b8d-acb1-600417a1d7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:58.148527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:58.191163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91627be-f1bf-41aa-9ff5-e46bcb5b1273 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:58.258419 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2517/5088] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:58.266474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34db7b-afa0-414f-9a1a-aafa6a9b041c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:58.266756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34db7b-afa0-414f-9a1a-aafa6a9b041c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:58.267023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34db7b-afa0-414f-9a1a-aafa6a9b041c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:58.293561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2f34db7b-afa0-414f-9a1a-aafa6a9b041c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:58.295215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2f34db7b-afa0-414f-9a1a-aafa6a9b041c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:58.301696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca97bfa-352a-42ee-9128-2f82debfa340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:58.302348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca97bfa-352a-42ee-9128-2f82debfa340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:58.303268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca97bfa-352a-42ee-9128-2f82debfa340 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:58.303555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca97bfa-352a-42ee-9128-2f82debfa340 admin admin] PATH_INFO: `/v3/users/3e849f3556c04fb69a01b0e1f7a95bef` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:58.305605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca97bfa-352a-42ee-9128-2f82debfa340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e849f3556c04fb69a01b0e1f7a95bef)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:58.308176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca97bfa-352a-42ee-9128-2f82debfa340 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:58.317514 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2518/5089] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:58.321998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34db7b-afa0-414f-9a1a-aafa6a9b041c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:58.351426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34db7b-afa0-414f-9a1a-aafa6a9b041c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:58.356809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7da78-3de5-4a74-b709-7610506a88c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:58.357527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7da78-3de5-4a74-b709-7610506a88c4 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:58.357825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7da78-3de5-4a74-b709-7610506a88c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:58.358091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7da78-3de5-4a74-b709-7610506a88c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:58.358731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7da78-3de5-4a74-b709-7610506a88c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:58.359034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7da78-3de5-4a74-b709-7610506a88c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:58.360671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7da78-3de5-4a74-b709-7610506a88c4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:58.377941 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2519/5090] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:10:58.378949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7da78-3de5-4a74-b709-7610506a88c4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:58.381229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2ca97bfa-352a-42ee-9128-2f82debfa340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e849f3556c04fb69a01b0e1f7a95bef'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:58.392104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7da78-3de5-4a74-b709-7610506a88c4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:58.397879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f719f-ce4c-483a-98d3-9c2a85b9f669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:58.398499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f719f-ce4c-483a-98d3-9c2a85b9f669 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:58.398657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2ca97bfa-352a-42ee-9128-2f82debfa340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e849f3556c04fb69a01b0e1f7a95bef'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:58.398879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f719f-ce4c-483a-98d3-9c2a85b9f669 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:58.399107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f719f-ce4c-483a-98d3-9c2a85b9f669 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:58.400106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f719f-ce4c-483a-98d3-9c2a85b9f669 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:58.401714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f719f-ce4c-483a-98d3-9c2a85b9f669 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:58.411721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2ca97bfa-352a-42ee-9128-2f82debfa340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e849f3556c04fb69a01b0e1f7a95bef'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:58.431529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2ca97bfa-352a-42ee-9128-2f82debfa340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e849f3556c04fb69a01b0e1f7a95bef'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:58.432810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2ca97bfa-352a-42ee-9128-2f82debfa340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e849f3556c04fb69a01b0e1f7a95bef'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:58.443849 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2520/5092] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:58 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:58.443849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f719f-ce4c-483a-98d3-9c2a85b9f669 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:58.445670 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2572/5092] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:10:57 2022] DELETE /identity/v3/users/3e849f3556c04fb69a01b0e1f7a95bef => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:58.446280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e14f719f-ce4c-483a-98d3-9c2a85b9f669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:58.555720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f719f-ce4c-483a-98d3-9c2a85b9f669 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:58.616697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:58.632774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181460a8-ba2c-4692-95c5-3d89c14688d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:58.633411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181460a8-ba2c-4692-95c5-3d89c14688d7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:58.633680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181460a8-ba2c-4692-95c5-3d89c14688d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:58.633944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181460a8-ba2c-4692-95c5-3d89c14688d7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:58.634573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181460a8-ba2c-4692-95c5-3d89c14688d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:58.636732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181460a8-ba2c-4692-95c5-3d89c14688d7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:58.669189 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2521/5093] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:58 2022] POST /identity/v3/users => generated 291 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:58.672184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca97bfa-352a-42ee-9128-2f82debfa340 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:58.684694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca97bfa-352a-42ee-9128-2f82debfa340 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:58.690720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f249da3-23f6-4666-a24a-9a1b8a75285d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:58.691827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f249da3-23f6-4666-a24a-9a1b8a75285d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:58.692099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f249da3-23f6-4666-a24a-9a1b8a75285d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:58.692389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f249da3-23f6-4666-a24a-9a1b8a75285d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:58.693157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f249da3-23f6-4666-a24a-9a1b8a75285d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:58.695515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f249da3-23f6-4666-a24a-9a1b8a75285d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:58.711462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181460a8-ba2c-4692-95c5-3d89c14688d7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:58.712527 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2573/5094] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:58.723131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181460a8-ba2c-4692-95c5-3d89c14688d7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:58.728312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adc142-3f61-408f-9b18-ed5618a5a9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:58.728989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adc142-3f61-408f-9b18-ed5618a5a9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:58.729264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adc142-3f61-408f-9b18-ed5618a5a9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:58.729534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adc142-3f61-408f-9b18-ed5618a5a9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:58.730200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57adc142-3f61-408f-9b18-ed5618a5a9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:58.732031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57adc142-3f61-408f-9b18-ed5618a5a9b9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:58.748911 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2522/5095] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:58.750609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f249da3-23f6-4666-a24a-9a1b8a75285d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:58.763279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f249da3-23f6-4666-a24a-9a1b8a75285d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:58.769090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa03e24-34bb-4ec5-9a1d-aab4825d3f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:58.770138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa03e24-34bb-4ec5-9a1d-aab4825d3f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:58.770613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa03e24-34bb-4ec5-9a1d-aab4825d3f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:58.771080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa03e24-34bb-4ec5-9a1d-aab4825d3f39 admin admin] PATH_INFO: `/v3/projects/51bfae5fb554467fbd0b6b8794e58d9c/users/f6bd3b52be874cd2b10fea2e3b98bcf9/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:58.802915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa03e24-34bb-4ec5-9a1d-aab4825d3f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51bfae5fb554467fbd0b6b8794e58d9c, user_id=f6bd3b52be874cd2b10fea2e3b98bcf9, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:58.805953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa03e24-34bb-4ec5-9a1d-aab4825d3f39 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:58.826915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adc142-3f61-408f-9b18-ed5618a5a9b9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:58.829446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-57adc142-3f61-408f-9b18-ed5618a5a9b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:58.834947 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2574/5096] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:58 2022] PUT /identity/v3/projects/51bfae5fb554467fbd0b6b8794e58d9c/users/f6bd3b52be874cd2b10fea2e3b98bcf9/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:10:58.836558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0f43a-3c32-4603-aa54-653651bd731d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:58.836829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0f43a-3c32-4603-aa54-653651bd731d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:58.837104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0f43a-3c32-4603-aa54-653651bd731d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:58.851592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adc142-3f61-408f-9b18-ed5618a5a9b9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:58.878051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-62a0f43a-3c32-4603-aa54-653651bd731d None None] MFA Rules not processed for user `f6bd3b52be874cd2b10fea2e3b98bcf9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:58.879544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-62a0f43a-3c32-4603-aa54-653651bd731d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:58.921776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:58.969182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3430bfc-b4be-4206-9601-e744d8f22862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:58.970389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3430bfc-b4be-4206-9601-e744d8f22862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:58.970890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3430bfc-b4be-4206-9601-e744d8f22862 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:58.971345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3430bfc-b4be-4206-9601-e744d8f22862 admin admin] PATH_INFO: `/v3/projects/5c748e76dafd409d87a8637645bf36f9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:58.972754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3430bfc-b4be-4206-9601-e744d8f22862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c748e76dafd409d87a8637645bf36f9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:58.973169 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2575/5097] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:58.975836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3430bfc-b4be-4206-9601-e744d8f22862 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:58.979222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0f43a-3c32-4603-aa54-653651bd731d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:58.996556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0f43a-3c32-4603-aa54-653651bd731d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:59.008154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ff3fb-1fa7-4bb1-ace6-8940386e5d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:59.009321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ff3fb-1fa7-4bb1-ace6-8940386e5d63 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.009609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ff3fb-1fa7-4bb1-ace6-8940386e5d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.009963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ff3fb-1fa7-4bb1-ace6-8940386e5d63 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.010646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ff3fb-1fa7-4bb1-ace6-8940386e5d63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:59.013163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ff3fb-1fa7-4bb1-ace6-8940386e5d63 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:59.034612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3430bfc-b4be-4206-9601-e744d8f22862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c748e76dafd409d87a8637645bf36f9', 'request_id': 'req-a3430bfc-b4be-4206-9601-e744d8f22862'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:59.045369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3430bfc-b4be-4206-9601-e744d8f22862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c748e76dafd409d87a8637645bf36f9', 'request_id': 'req-a3430bfc-b4be-4206-9601-e744d8f22862'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:59.059779 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2576/5098] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:58 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:59.061417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ff3fb-1fa7-4bb1-ace6-8940386e5d63 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:59.063254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3430bfc-b4be-4206-9601-e744d8f22862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c748e76dafd409d87a8637645bf36f9', 'request_id': 'req-a3430bfc-b4be-4206-9601-e744d8f22862'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:59.092679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3430bfc-b4be-4206-9601-e744d8f22862 admin admin] The token cache is being invalidate because project 5c748e76dafd409d87a8637645bf36f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:10:59.093026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3430bfc-b4be-4206-9601-e744d8f22862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:59.095297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3430bfc-b4be-4206-9601-e744d8f22862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:10:59.097885 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2523/5099] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:10:58 2022] DELETE /identity/v3/projects/5c748e76dafd409d87a8637645bf36f9 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:59.103729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141706b3-1d0c-4c9d-8c8b-5a843bc21cc6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.104045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141706b3-1d0c-4c9d-8c8b-5a843bc21cc6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.104295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141706b3-1d0c-4c9d-8c8b-5a843bc21cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.158484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-141706b3-1d0c-4c9d-8c8b-5a843bc21cc6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:59.160928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-141706b3-1d0c-4c9d-8c8b-5a843bc21cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:59.196679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ff3fb-1fa7-4bb1-ace6-8940386e5d63 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:59.202934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ec95e-544c-4495-9493-013bbd2bb199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:59.203621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ec95e-544c-4495-9493-013bbd2bb199 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.203949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ec95e-544c-4495-9493-013bbd2bb199 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.204860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ec95e-544c-4495-9493-013bbd2bb199 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.206318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ec95e-544c-4495-9493-013bbd2bb199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:59.208465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ec95e-544c-4495-9493-013bbd2bb199 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:59.254239 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2577/5100] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:59 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:10:59.255318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ec95e-544c-4495-9493-013bbd2bb199 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:59.258165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-869ec95e-544c-4495-9493-013bbd2bb199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:59.267019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:59.278252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ec95e-544c-4495-9493-013bbd2bb199 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:59.283002 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2524/5101] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:59.285974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfc251-be41-4cf0-8b11-1fe1e7ca7f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:59.286680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfc251-be41-4cf0-8b11-1fe1e7ca7f13 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.287040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfc251-be41-4cf0-8b11-1fe1e7ca7f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.287526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfc251-be41-4cf0-8b11-1fe1e7ca7f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.288246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfc251-be41-4cf0-8b11-1fe1e7ca7f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:59.290483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfc251-be41-4cf0-8b11-1fe1e7ca7f13 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:59.292483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3f829-4781-476b-b7d9-a0faa3d0d59f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.293272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3f829-4781-476b-b7d9-a0faa3d0d59f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.293772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3f829-4781-476b-b7d9-a0faa3d0d59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.303853 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2578/5102] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:10:59.318991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfc251-be41-4cf0-8b11-1fe1e7ca7f13 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:59.324366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a9a3f829-4781-476b-b7d9-a0faa3d0d59f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:59.326069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a9a3f829-4781-476b-b7d9-a0faa3d0d59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:59.331326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfc251-be41-4cf0-8b11-1fe1e7ca7f13 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:59.337616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5c06f-1177-4160-9dbf-e9a184fe4ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:59.337921 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2525/5103] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:59.338594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5c06f-1177-4160-9dbf-e9a184fe4ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.338939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5c06f-1177-4160-9dbf-e9a184fe4ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.339277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5c06f-1177-4160-9dbf-e9a184fe4ce1 admin admin] PATH_INFO: `/v3/projects/f450b10ec5d4468484d41c3a2a31679d/users/d20c5b6017a9441d887c391be116cd51/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.341861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3f829-4781-476b-b7d9-a0faa3d0d59f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:59.354489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3f829-4781-476b-b7d9-a0faa3d0d59f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:59.360705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb1d27-9e5b-48eb-956e-211fabd1487e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:59.361371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb1d27-9e5b-48eb-956e-211fabd1487e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.361650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb1d27-9e5b-48eb-956e-211fabd1487e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.361970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb1d27-9e5b-48eb-956e-211fabd1487e admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.363471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb1d27-9e5b-48eb-956e-211fabd1487e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:10:59.363654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb1d27-9e5b-48eb-956e-211fabd1487e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:59.365622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb1d27-9e5b-48eb-956e-211fabd1487e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:59.376477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5c06f-1177-4160-9dbf-e9a184fe4ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f450b10ec5d4468484d41c3a2a31679d, user_id=d20c5b6017a9441d887c391be116cd51, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:59.383250 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2526/5104] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:10:59.384764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb1d27-9e5b-48eb-956e-211fabd1487e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:59.386420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5c06f-1177-4160-9dbf-e9a184fe4ce1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:59.396513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb1d27-9e5b-48eb-956e-211fabd1487e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:59.402409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3484cd8-6782-46c1-829b-d18df95fb102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:59.403215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3484cd8-6782-46c1-829b-d18df95fb102 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.403874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3484cd8-6782-46c1-829b-d18df95fb102 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.403874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3484cd8-6782-46c1-829b-d18df95fb102 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.404438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3484cd8-6782-46c1-829b-d18df95fb102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:59.407296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3484cd8-6782-46c1-829b-d18df95fb102 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:59.408472 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2579/5105] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:59 2022] PUT /identity/v3/projects/f450b10ec5d4468484d41c3a2a31679d/users/d20c5b6017a9441d887c391be116cd51/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:59.411991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5c06f-1177-4160-9dbf-e9a184fe4ce1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:59.445973 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2527/5106] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:10:59 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:59.450212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3484cd8-6782-46c1-829b-d18df95fb102 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:59.544393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5c06f-1177-4160-9dbf-e9a184fe4ce1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:59.565096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:59.581070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3484cd8-6782-46c1-829b-d18df95fb102 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:59.589800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb18428-81f1-4d9b-b8fc-c5b75bdd306c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:59.590771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb18428-81f1-4d9b-b8fc-c5b75bdd306c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.591260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb18428-81f1-4d9b-b8fc-c5b75bdd306c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.591924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb18428-81f1-4d9b-b8fc-c5b75bdd306c admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.593371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb18428-81f1-4d9b-b8fc-c5b75bdd306c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:59.595554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb18428-81f1-4d9b-b8fc-c5b75bdd306c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:59.623787 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2528/5107] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:10:59 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:10:59.627845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb18428-81f1-4d9b-b8fc-c5b75bdd306c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:59.642467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb18428-81f1-4d9b-b8fc-c5b75bdd306c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:59.649597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72add8-cfc7-435e-ac72-802819261277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:59.650331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72add8-cfc7-435e-ac72-802819261277 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.650677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72add8-cfc7-435e-ac72-802819261277 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.650838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db242598-9c54-4958-8f37-0fa4133cffaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:59.650955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72add8-cfc7-435e-ac72-802819261277 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.651922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72add8-cfc7-435e-ac72-802819261277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:59.652582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db242598-9c54-4958-8f37-0fa4133cffaf admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.652886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db242598-9c54-4958-8f37-0fa4133cffaf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.653155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db242598-9c54-4958-8f37-0fa4133cffaf admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.653794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db242598-9c54-4958-8f37-0fa4133cffaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:59.654062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72add8-cfc7-435e-ac72-802819261277 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:59.658705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db242598-9c54-4958-8f37-0fa4133cffaf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:59.670908 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2529/5108] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:59.674344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72add8-cfc7-435e-ac72-802819261277 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:59.674653 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2580/5109] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:59.679260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db242598-9c54-4958-8f37-0fa4133cffaf admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:59.695959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72add8-cfc7-435e-ac72-802819261277 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:59.704755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078c0a6-87e2-4996-ad5e-19f3950fb742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:59.705389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078c0a6-87e2-4996-ad5e-19f3950fb742 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.705667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078c0a6-87e2-4996-ad5e-19f3950fb742 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.706020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078c0a6-87e2-4996-ad5e-19f3950fb742 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.706852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078c0a6-87e2-4996-ad5e-19f3950fb742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:59.708513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078c0a6-87e2-4996-ad5e-19f3950fb742 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:59.712672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db242598-9c54-4958-8f37-0fa4133cffaf admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:59.719515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa93839-b912-490b-837e-d343d84eda3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:59.721380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa93839-b912-490b-837e-d343d84eda3f admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.721886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa93839-b912-490b-837e-d343d84eda3f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.723129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa93839-b912-490b-837e-d343d84eda3f admin admin] PATH_INFO: `/v3/projects/f450b10ec5d4468484d41c3a2a31679d/users/d20c5b6017a9441d887c391be116cd51/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.726370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa93839-b912-490b-837e-d343d84eda3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f450b10ec5d4468484d41c3a2a31679d, user_id=d20c5b6017a9441d887c391be116cd51, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:59.733447 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2530/5110] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:10:59.734882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078c0a6-87e2-4996-ad5e-19f3950fb742 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:59.737340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa93839-b912-490b-837e-d343d84eda3f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:59.749246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078c0a6-87e2-4996-ad5e-19f3950fb742 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:10:59.770707 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2581/5111] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:59 2022] PUT /identity/v3/projects/f450b10ec5d4468484d41c3a2a31679d/users/d20c5b6017a9441d887c391be116cd51/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:59.774657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f3dc8-3942-4f10-bf6e-052b64ffe9bd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.774930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f3dc8-3942-4f10-bf6e-052b64ffe9bd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.775197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f3dc8-3942-4f10-bf6e-052b64ffe9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.815779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e16f3dc8-3942-4f10-bf6e-052b64ffe9bd None None] MFA Rules not processed for user `d20c5b6017a9441d887c391be116cd51`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:59.821433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e16f3dc8-3942-4f10-bf6e-052b64ffe9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:59.848553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d80bf-18a3-47a5-b484-bd9482ee891d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:10:59.850961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d80bf-18a3-47a5-b484-bd9482ee891d admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.851244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d80bf-18a3-47a5-b484-bd9482ee891d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.851515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d80bf-18a3-47a5-b484-bd9482ee891d admin admin] PATH_INFO: `/v3/projects/c21663600eb34db7b534db8a33e15344/users/80d645a43ba249208f07968cd5930832/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.893603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d80bf-18a3-47a5-b484-bd9482ee891d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21663600eb34db7b534db8a33e15344, user_id=80d645a43ba249208f07968cd5930832, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:10:59.897878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d80bf-18a3-47a5-b484-bd9482ee891d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:10:59.927109 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2531/5112] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:10:59 2022] PUT /identity/v3/projects/c21663600eb34db7b534db8a33e15344/users/80d645a43ba249208f07968cd5930832/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:10:59.941080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523079b-0756-45f4-8a7d-c285c6eb3eb7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:10:59.941443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523079b-0756-45f4-8a7d-c285c6eb3eb7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:10:59.941635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523079b-0756-45f4-8a7d-c285c6eb3eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:10:59.967743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:10:59.980139 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2582/5113] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:10:59.990053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e523079b-0756-45f4-8a7d-c285c6eb3eb7 None None] MFA Rules not processed for user `80d645a43ba249208f07968cd5930832`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:10:59.990412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f3dc8-3942-4f10-bf6e-052b64ffe9bd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:10:59.992348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e523079b-0756-45f4-8a7d-c285c6eb3eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:10:59.993034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e16f3dc8-3942-4f10-bf6e-052b64ffe9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:00.087561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f3dc8-3942-4f10-bf6e-052b64ffe9bd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:00.101056 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2532/5114] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:10:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:00.103332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523079b-0756-45f4-8a7d-c285c6eb3eb7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:00.105144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e523079b-0756-45f4-8a7d-c285c6eb3eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:00.147335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:00.171174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39307ed6-cf0a-48b7-90e2-a398ae376472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:00.171876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39307ed6-cf0a-48b7-90e2-a398ae376472 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:00.172116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39307ed6-cf0a-48b7-90e2-a398ae376472 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:00.172340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39307ed6-cf0a-48b7-90e2-a398ae376472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:00.176564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523079b-0756-45f4-8a7d-c285c6eb3eb7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:00.180846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec672385-bc66-401d-b791-fe5dd50522c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:00.181464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec672385-bc66-401d-b791-fe5dd50522c0 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:00.181735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec672385-bc66-401d-b791-fe5dd50522c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:00.181998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec672385-bc66-401d-b791-fe5dd50522c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:00.184943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ec672385-bc66-401d-b791-fe5dd50522c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:00.257160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39307ed6-cf0a-48b7-90e2-a398ae376472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:00.258559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec672385-bc66-401d-b791-fe5dd50522c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:00.258763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39307ed6-cf0a-48b7-90e2-a398ae376472 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:00.260132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec672385-bc66-401d-b791-fe5dd50522c0 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:00.269379 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2583/5115] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:10:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:00.270186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39307ed6-cf0a-48b7-90e2-a398ae376472 service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:00.273722 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2533/5116] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:00.281656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39307ed6-cf0a-48b7-90e2-a398ae376472 service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:00.288492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a716c-0df8-40b0-aa9a-7fdfb4e08246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:00.289123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a716c-0df8-40b0-aa9a-7fdfb4e08246 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:00.289399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a716c-0df8-40b0-aa9a-7fdfb4e08246 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:00.289656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a716c-0df8-40b0-aa9a-7fdfb4e08246 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:00.290392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a716c-0df8-40b0-aa9a-7fdfb4e08246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:00.292253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a716c-0df8-40b0-aa9a-7fdfb4e08246 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:00.341888 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2584/5117] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:00 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:00.346638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec672385-bc66-401d-b791-fe5dd50522c0 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:00.363163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec672385-bc66-401d-b791-fe5dd50522c0 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:00.502941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:00.520819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5cf93-e05e-4081-8563-d067b999ccd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:00.522356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5cf93-e05e-4081-8563-d067b999ccd7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:00.524158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5cf93-e05e-4081-8563-d067b999ccd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:00.524862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5cf93-e05e-4081-8563-d067b999ccd7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:00.525842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5cf93-e05e-4081-8563-d067b999ccd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:00.528120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5cf93-e05e-4081-8563-d067b999ccd7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:00.571487 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2534/5118] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:00 2022] POST /identity/v3/users => generated 295 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:00.574877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a716c-0df8-40b0-aa9a-7fdfb4e08246 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:00.594720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a716c-0df8-40b0-aa9a-7fdfb4e08246 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:00.601232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f2b74-d439-4724-808a-145054cd89fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:00.602696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f2b74-d439-4724-808a-145054cd89fe admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:00.603263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f2b74-d439-4724-808a-145054cd89fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:00.604096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f2b74-d439-4724-808a-145054cd89fe admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:00.604899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f2b74-d439-4724-808a-145054cd89fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:00.607288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f2b74-d439-4724-808a-145054cd89fe admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:00.628907 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2585/5119] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:00.630906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5cf93-e05e-4081-8563-d067b999ccd7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:00.643931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5cf93-e05e-4081-8563-d067b999ccd7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:00.650117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999698a-2a1f-46c8-8d95-3312260abb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:00.651198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999698a-2a1f-46c8-8d95-3312260abb6c admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:00.651479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999698a-2a1f-46c8-8d95-3312260abb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:00.651746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999698a-2a1f-46c8-8d95-3312260abb6c admin admin] PATH_INFO: `/v3/projects/654136bb95e84a238901b4f282ef64de/users/c16e351e92be4a18bcd02a937d051457/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:00.674811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999698a-2a1f-46c8-8d95-3312260abb6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=654136bb95e84a238901b4f282ef64de, user_id=c16e351e92be4a18bcd02a937d051457, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:00.679453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999698a-2a1f-46c8-8d95-3312260abb6c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:00.710541 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2535/5120] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:00 2022] PUT /identity/v3/projects/654136bb95e84a238901b4f282ef64de/users/c16e351e92be4a18bcd02a937d051457/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:00.712166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f2b74-d439-4724-808a-145054cd89fe admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:00.729634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f2b74-d439-4724-808a-145054cd89fe admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:00.844130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023210e-b21f-4f5d-98ff-119083b0a5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:00.845078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023210e-b21f-4f5d-98ff-119083b0a5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:00.845629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023210e-b21f-4f5d-98ff-119083b0a5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:00.846492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023210e-b21f-4f5d-98ff-119083b0a5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:00.847396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023210e-b21f-4f5d-98ff-119083b0a5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:00.850595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023210e-b21f-4f5d-98ff-119083b0a5c3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:00.867352 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2586/5121] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:00.871802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999698a-2a1f-46c8-8d95-3312260abb6c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:00.892597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999698a-2a1f-46c8-8d95-3312260abb6c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:00.899804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d32a8-210a-4cc9-8b7a-15c7b5d4ee53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:00.900794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d32a8-210a-4cc9-8b7a-15c7b5d4ee53 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:00.901130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d32a8-210a-4cc9-8b7a-15c7b5d4ee53 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:00.901463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d32a8-210a-4cc9-8b7a-15c7b5d4ee53 admin admin] PATH_INFO: `/v3/projects/654136bb95e84a238901b4f282ef64de/users/c16e351e92be4a18bcd02a937d051457/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:00.904545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d32a8-210a-4cc9-8b7a-15c7b5d4ee53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=654136bb95e84a238901b4f282ef64de, user_id=c16e351e92be4a18bcd02a937d051457, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:00.907608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d32a8-210a-4cc9-8b7a-15c7b5d4ee53 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:00.929142 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2536/5122] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:00 2022] PUT /identity/v3/projects/654136bb95e84a238901b4f282ef64de/users/c16e351e92be4a18bcd02a937d051457/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:00.942447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccaed1-d9c5-4097-8664-2263de835977 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:00.943084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccaed1-d9c5-4097-8664-2263de835977 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:00.943821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccaed1-d9c5-4097-8664-2263de835977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:00.972235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d32a8-210a-4cc9-8b7a-15c7b5d4ee53 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:01.035169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7eccaed1-d9c5-4097-8664-2263de835977 None None] MFA Rules not processed for user `c16e351e92be4a18bcd02a937d051457`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:01.038787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7eccaed1-d9c5-4097-8664-2263de835977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:01.120277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d32a8-210a-4cc9-8b7a-15c7b5d4ee53 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:01.121082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:01.126358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e4d22-0036-4ebc-8c99-9ee00556a5bc tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd20c5b6017a9441d887c391be116cd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f450b10ec5d4468484d41c3a2a31679d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:01.127880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e4d22-0036-4ebc-8c99-9ee00556a5bc tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:01.128162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e4d22-0036-4ebc-8c99-9ee00556a5bc tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:01.128460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e4d22-0036-4ebc-8c99-9ee00556a5bc tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:01.129554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e4d22-0036-4ebc-8c99-9ee00556a5bc tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:01.129868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e4d22-0036-4ebc-8c99-9ee00556a5bc tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:01.132050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e4d22-0036-4ebc-8c99-9ee00556a5bc tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:01.146880 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2537/5123] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:11:01.151147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e4d22-0036-4ebc-8c99-9ee00556a5bc tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:01.162023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e4d22-0036-4ebc-8c99-9ee00556a5bc tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:01.162914 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2587/5124] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:01.171236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88057e5e-2cb9-47aa-bc7c-c66ead531618 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd20c5b6017a9441d887c391be116cd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f450b10ec5d4468484d41c3a2a31679d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:01.172179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88057e5e-2cb9-47aa-bc7c-c66ead531618 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:01.172489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88057e5e-2cb9-47aa-bc7c-c66ead531618 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:01.172763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88057e5e-2cb9-47aa-bc7c-c66ead531618 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:01.173501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88057e5e-2cb9-47aa-bc7c-c66ead531618 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:01.175041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eccaed1-d9c5-4097-8664-2263de835977 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:01.176940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88057e5e-2cb9-47aa-bc7c-c66ead531618 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:01.219979 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2538/5125] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:01 2022] POST /identity/v3/projects => generated 354 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:01.222346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88057e5e-2cb9-47aa-bc7c-c66ead531618 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:01.248621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eccaed1-d9c5-4097-8664-2263de835977 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:01.307942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8196a-0b20-4992-a32b-d87ea2231134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:01.308922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8196a-0b20-4992-a32b-d87ea2231134 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:01.309347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8196a-0b20-4992-a32b-d87ea2231134 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:01.309800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8196a-0b20-4992-a32b-d87ea2231134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:01.328892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88057e5e-2cb9-47aa-bc7c-c66ead531618 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:01.339119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e6a48-ca4e-471c-b155-824f633ff574 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd20c5b6017a9441d887c391be116cd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f450b10ec5d4468484d41c3a2a31679d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:01.339754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e6a48-ca4e-471c-b155-824f633ff574 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:01.340180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e6a48-ca4e-471c-b155-824f633ff574 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:01.340487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e6a48-ca4e-471c-b155-824f633ff574 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:01.341681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e6a48-ca4e-471c-b155-824f633ff574 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:01.342394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e6a48-ca4e-471c-b155-824f633ff574 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:01.346099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e6a48-ca4e-471c-b155-824f633ff574 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:01.367996 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2539/5126] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:01.402992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8196a-0b20-4992-a32b-d87ea2231134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:01.404503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8196a-0b20-4992-a32b-d87ea2231134 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:01.420288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e6a48-ca4e-471c-b155-824f633ff574 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:01.433232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e6a48-ca4e-471c-b155-824f633ff574 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:01.433661 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2588/5127] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:01.441606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86065d3-4b78-4298-8a62-00d132fdb222 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd20c5b6017a9441d887c391be116cd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f450b10ec5d4468484d41c3a2a31679d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:01.442517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86065d3-4b78-4298-8a62-00d132fdb222 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:01.443068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86065d3-4b78-4298-8a62-00d132fdb222 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:01.444899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86065d3-4b78-4298-8a62-00d132fdb222 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] PATH_INFO: `/v3/projects/16127fe1e32f4dbdb44311d4060f3eb2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:01.457097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86065d3-4b78-4298-8a62-00d132fdb222 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: Authorizing `identity:delete_project(project_id=16127fe1e32f4dbdb44311d4060f3eb2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:01.459333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86065d3-4b78-4298-8a62-00d132fdb222 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:01.461636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:01.576029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a86065d3-4b78-4298-8a62-00d132fdb222 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16127fe1e32f4dbdb44311d4060f3eb2', 'request_id': 'req-a86065d3-4b78-4298-8a62-00d132fdb222'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:01.591484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a86065d3-4b78-4298-8a62-00d132fdb222 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16127fe1e32f4dbdb44311d4060f3eb2', 'request_id': 'req-a86065d3-4b78-4298-8a62-00d132fdb222'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:01.615841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a86065d3-4b78-4298-8a62-00d132fdb222 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16127fe1e32f4dbdb44311d4060f3eb2', 'request_id': 'req-a86065d3-4b78-4298-8a62-00d132fdb222'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:01.641223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a86065d3-4b78-4298-8a62-00d132fdb222 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] The token cache is being invalidate because project 16127fe1e32f4dbdb44311d4060f3eb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:01.642649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a86065d3-4b78-4298-8a62-00d132fdb222 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:01.643604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a86065d3-4b78-4298-8a62-00d132fdb222 tempest-VolumeQuotaClassesTest-807914385 tempest-VolumeQuotaClassesTest-807914385-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:01.647635 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2540/5128] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:01 2022] DELETE /identity/v3/projects/16127fe1e32f4dbdb44311d4060f3eb2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:02.120672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8196a-0b20-4992-a32b-d87ea2231134 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:02.190315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:02.259810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8196a-0b20-4992-a32b-d87ea2231134 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:02.387030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:02.388260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:02.388260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:02.388591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] PATH_INFO: `/v3/users/f6bd3b52be874cd2b10fea2e3b98bcf9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:02.390876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6bd3b52be874cd2b10fea2e3b98bcf9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:02.393505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:02.503121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6bd3b52be874cd2b10fea2e3b98bcf9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:02.513021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6bd3b52be874cd2b10fea2e3b98bcf9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:02.539422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6bd3b52be874cd2b10fea2e3b98bcf9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:02.566802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6bd3b52be874cd2b10fea2e3b98bcf9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:02.567746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6bd3b52be874cd2b10fea2e3b98bcf9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:02.579281 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2589/5129] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:02 2022] DELETE /identity/v3/users/f6bd3b52be874cd2b10fea2e3b98bcf9 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:02.580780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492611c-9600-4c6b-b8ca-d3d4d9e42291 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:02.581342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492611c-9600-4c6b-b8ca-d3d4d9e42291 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:02.581771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492611c-9600-4c6b-b8ca-d3d4d9e42291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:02.623432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d492611c-9600-4c6b-b8ca-d3d4d9e42291 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:02.625075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d492611c-9600-4c6b-b8ca-d3d4d9e42291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:02.730052 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2541/5130] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:02.766622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:02.768464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:02.890049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9254f8-8695-48fa-b66f-007193d54894 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:02.896428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694994db-e4c9-4a12-8251-d91af0f91ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:02.897086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694994db-e4c9-4a12-8251-d91af0f91ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:02.897362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694994db-e4c9-4a12-8251-d91af0f91ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:02.897629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694994db-e4c9-4a12-8251-d91af0f91ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:02.908844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694994db-e4c9-4a12-8251-d91af0f91ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:02.910457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694994db-e4c9-4a12-8251-d91af0f91ef4 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:02.922350 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2590/5131] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:03.768047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492611c-9600-4c6b-b8ca-d3d4d9e42291 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:03.770618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d492611c-9600-4c6b-b8ca-d3d4d9e42291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:03.806137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694994db-e4c9-4a12-8251-d91af0f91ef4 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:03.808317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-694994db-e4c9-4a12-8251-d91af0f91ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:03.812757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492611c-9600-4c6b-b8ca-d3d4d9e42291 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:03.835387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694994db-e4c9-4a12-8251-d91af0f91ef4 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:03.840963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:03.841685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:03.841971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:03.842240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] PATH_INFO: `/v3/users/1cf36cef64b14630850a3fc3d2974e79` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:03.843374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cf36cef64b14630850a3fc3d2974e79)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:03.845453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:03.847088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:03.847735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:03.848019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:03.848288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] PATH_INFO: `/v3/projects/51bfae5fb554467fbd0b6b8794e58d9c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:03.855155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51bfae5fb554467fbd0b6b8794e58d9c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:03.857149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:03.873160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:03.902429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cf36cef64b14630850a3fc3d2974e79'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:03.916436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cf36cef64b14630850a3fc3d2974e79'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:03.932552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cf36cef64b14630850a3fc3d2974e79'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:03.933578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cf36cef64b14630850a3fc3d2974e79'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:03.934614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cf36cef64b14630850a3fc3d2974e79'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:03.952623 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2542/5132] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:03 2022] DELETE /identity/v3/users/1cf36cef64b14630850a3fc3d2974e79 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:03.958492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51bfae5fb554467fbd0b6b8794e58d9c', 'request_id': 'req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:03.973113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51bfae5fb554467fbd0b6b8794e58d9c', 'request_id': 'req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:03.985610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51bfae5fb554467fbd0b6b8794e58d9c', 'request_id': 'req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:04.001841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] The token cache is being invalidate because project 51bfae5fb554467fbd0b6b8794e58d9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:04.002585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:04.003589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:04.012637 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2591/5133] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:03 2022] DELETE /identity/v3/projects/51bfae5fb554467fbd0b6b8794e58d9c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:04.017753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:04.019561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:04.025176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:04.108529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df77b6-5dfd-4b9e-89c6-34f73b619f0f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:04.115183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:04.117195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:04.117195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:04.117195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] PATH_INFO: `/v3/users/d20c5b6017a9441d887c391be116cd51` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:04.118562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20c5b6017a9441d887c391be116cd51)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:04.121512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:04.181551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20c5b6017a9441d887c391be116cd51'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:04.191732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20c5b6017a9441d887c391be116cd51'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:04.207873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20c5b6017a9441d887c391be116cd51'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:04.208654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20c5b6017a9441d887c391be116cd51'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:04.209337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20c5b6017a9441d887c391be116cd51'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:04.226606 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2543/5134] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:04 2022] DELETE /identity/v3/users/d20c5b6017a9441d887c391be116cd51 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:04.521668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:04.525856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:04.698564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b99d83-0d2c-47ac-a064-14b8a7b1bff0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:04.810046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331c6f8-ec72-4c20-b284-9921560c9acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:04.810706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331c6f8-ec72-4c20-b284-9921560c9acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:04.810981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331c6f8-ec72-4c20-b284-9921560c9acf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:04.812125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331c6f8-ec72-4c20-b284-9921560c9acf admin admin] PATH_INFO: `/v3/projects/28e9c7707d7645d5b191af40f4c6da31` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:04.813038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331c6f8-ec72-4c20-b284-9921560c9acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=28e9c7707d7645d5b191af40f4c6da31)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:04.816439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331c6f8-ec72-4c20-b284-9921560c9acf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:04.938013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:04.949667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:04.952986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:04.960527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2331c6f8-ec72-4c20-b284-9921560c9acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28e9c7707d7645d5b191af40f4c6da31', 'request_id': 'req-2331c6f8-ec72-4c20-b284-9921560c9acf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:04.978414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee246c-fdc6-4161-b5d1-ae7b1dbadcea admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:04.984333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2331c6f8-ec72-4c20-b284-9921560c9acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28e9c7707d7645d5b191af40f4c6da31', 'request_id': 'req-2331c6f8-ec72-4c20-b284-9921560c9acf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:04.998831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2331c6f8-ec72-4c20-b284-9921560c9acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28e9c7707d7645d5b191af40f4c6da31', 'request_id': 'req-2331c6f8-ec72-4c20-b284-9921560c9acf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:05.021617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2331c6f8-ec72-4c20-b284-9921560c9acf admin admin] The token cache is being invalidate because project 28e9c7707d7645d5b191af40f4c6da31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:05.021999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2331c6f8-ec72-4c20-b284-9921560c9acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:05.023453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2331c6f8-ec72-4c20-b284-9921560c9acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:05.033134 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2592/5135] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:04 2022] DELETE /identity/v3/projects/28e9c7707d7645d5b191af40f4c6da31 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:05.035956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6691c-61e3-4b1b-a805-a0b84a90a9bb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:05.036239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6691c-61e3-4b1b-a805-a0b84a90a9bb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:05.036541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6691c-61e3-4b1b-a805-a0b84a90a9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:05.106126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-14a6691c-61e3-4b1b-a805-a0b84a90a9bb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:05.110155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-14a6691c-61e3-4b1b-a805-a0b84a90a9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:05.131142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03098a6-2af7-424b-956a-180e80cc9b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:05.132583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03098a6-2af7-424b-956a-180e80cc9b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:05.133594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03098a6-2af7-424b-956a-180e80cc9b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:05.134282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03098a6-2af7-424b-956a-180e80cc9b4f admin admin] PATH_INFO: `/v3/projects/f450b10ec5d4468484d41c3a2a31679d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:05.136901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03098a6-2af7-424b-956a-180e80cc9b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f450b10ec5d4468484d41c3a2a31679d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:05.138916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03098a6-2af7-424b-956a-180e80cc9b4f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:05.180589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:05.217136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d03098a6-2af7-424b-956a-180e80cc9b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f450b10ec5d4468484d41c3a2a31679d', 'request_id': 'req-d03098a6-2af7-424b-956a-180e80cc9b4f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:05.241982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d03098a6-2af7-424b-956a-180e80cc9b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f450b10ec5d4468484d41c3a2a31679d', 'request_id': 'req-d03098a6-2af7-424b-956a-180e80cc9b4f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:05.244885 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2593/5136] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:05.252859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfad4c2-c6ec-4709-a585-04d8868e244d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:05.253324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfad4c2-c6ec-4709-a585-04d8868e244d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:05.253606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfad4c2-c6ec-4709-a585-04d8868e244d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:05.279405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d03098a6-2af7-424b-956a-180e80cc9b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f450b10ec5d4468484d41c3a2a31679d', 'request_id': 'req-d03098a6-2af7-424b-956a-180e80cc9b4f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:05.296805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8dfad4c2-c6ec-4709-a585-04d8868e244d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:05.299603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8dfad4c2-c6ec-4709-a585-04d8868e244d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:05.300055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d03098a6-2af7-424b-956a-180e80cc9b4f admin admin] The token cache is being invalidate because project f450b10ec5d4468484d41c3a2a31679d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:05.300341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d03098a6-2af7-424b-956a-180e80cc9b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:05.301101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d03098a6-2af7-424b-956a-180e80cc9b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:05.303015 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2544/5137] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:04 2022] DELETE /identity/v3/projects/f450b10ec5d4468484d41c3a2a31679d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:05.325024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904e9b-9ca2-4ef9-acc5-e5a8205c9792 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:05.325304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904e9b-9ca2-4ef9-acc5-e5a8205c9792 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:05.325575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904e9b-9ca2-4ef9-acc5-e5a8205c9792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:05.352833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-81904e9b-9ca2-4ef9-acc5-e5a8205c9792 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:05.354234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-81904e9b-9ca2-4ef9-acc5-e5a8205c9792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:05.396896 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2594/5138] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:05.401021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfad4c2-c6ec-4709-a585-04d8868e244d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:05.421358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfad4c2-c6ec-4709-a585-04d8868e244d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:05.434442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2721e40-6ba9-4a3f-82bc-6bc1a631a287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:05.435833 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2545/5139] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:05.436349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2721e40-6ba9-4a3f-82bc-6bc1a631a287 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:05.437085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2721e40-6ba9-4a3f-82bc-6bc1a631a287 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:05.437636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2721e40-6ba9-4a3f-82bc-6bc1a631a287 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:05.439106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2721e40-6ba9-4a3f-82bc-6bc1a631a287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:05.439970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2721e40-6ba9-4a3f-82bc-6bc1a631a287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:05.443253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2721e40-6ba9-4a3f-82bc-6bc1a631a287 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:05.450907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49e3c-1c41-40cd-a63f-d1134fd7ec79 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:05.451293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49e3c-1c41-40cd-a63f-d1134fd7ec79 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:05.451795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49e3c-1c41-40cd-a63f-d1134fd7ec79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:05.474003 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2595/5140] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:05.481485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2721e40-6ba9-4a3f-82bc-6bc1a631a287 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:05.492850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cce49e3c-1c41-40cd-a63f-d1134fd7ec79 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:05.494184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cce49e3c-1c41-40cd-a63f-d1134fd7ec79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:05.498212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2721e40-6ba9-4a3f-82bc-6bc1a631a287 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:05.509704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781015f1-9b88-4997-84aa-a08a413c92b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:05.510254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781015f1-9b88-4997-84aa-a08a413c92b8 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:05.510522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781015f1-9b88-4997-84aa-a08a413c92b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:05.510774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781015f1-9b88-4997-84aa-a08a413c92b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:05.511401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781015f1-9b88-4997-84aa-a08a413c92b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:05.512723 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2546/5141] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:05.513528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781015f1-9b88-4997-84aa-a08a413c92b8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:05.517245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce49e3c-1c41-40cd-a63f-d1134fd7ec79 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:05.541248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce49e3c-1c41-40cd-a63f-d1134fd7ec79 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:05.569078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a61cc5-dac9-4099-b31d-443a4eee2018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:05.569078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a61cc5-dac9-4099-b31d-443a4eee2018 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:05.569078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a61cc5-dac9-4099-b31d-443a4eee2018 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:05.569078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a61cc5-dac9-4099-b31d-443a4eee2018 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:05.570070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a61cc5-dac9-4099-b31d-443a4eee2018 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:05.570070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a61cc5-dac9-4099-b31d-443a4eee2018 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:05.570637 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2596/5142] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:05 2022] POST /identity/v3/projects => generated 395 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:05.571173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781015f1-9b88-4997-84aa-a08a413c92b8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:05.575066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a61cc5-dac9-4099-b31d-443a4eee2018 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:05.606828 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2547/5143] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:05.607801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a61cc5-dac9-4099-b31d-443a4eee2018 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:05.635442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a61cc5-dac9-4099-b31d-443a4eee2018 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:05.648881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:05.696850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781015f1-9b88-4997-84aa-a08a413c92b8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:05.709302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca46603-56e2-4fff-b9d9-dcd0a3f2db14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:05.709838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca46603-56e2-4fff-b9d9-dcd0a3f2db14 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:05.710111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca46603-56e2-4fff-b9d9-dcd0a3f2db14 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:05.710374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca46603-56e2-4fff-b9d9-dcd0a3f2db14 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:05.711005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca46603-56e2-4fff-b9d9-dcd0a3f2db14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:05.713027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca46603-56e2-4fff-b9d9-dcd0a3f2db14 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:05.738179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b1cc1-2f23-4a63-89f1-d1e026970822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:05.739462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b1cc1-2f23-4a63-89f1-d1e026970822 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:05.739462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b1cc1-2f23-4a63-89f1-d1e026970822 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:05.739462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b1cc1-2f23-4a63-89f1-d1e026970822 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:05.739908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b1cc1-2f23-4a63-89f1-d1e026970822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:05.742397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b1cc1-2f23-4a63-89f1-d1e026970822 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:05.747850 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2597/5144] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:05 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:05.753290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca46603-56e2-4fff-b9d9-dcd0a3f2db14 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:05.768519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca46603-56e2-4fff-b9d9-dcd0a3f2db14 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:05.778363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d8d1a-482f-4026-a17a-fa9603369d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:05.779409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d8d1a-482f-4026-a17a-fa9603369d54 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:05.780004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d8d1a-482f-4026-a17a-fa9603369d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:05.780346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d8d1a-482f-4026-a17a-fa9603369d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:05.781229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d8d1a-482f-4026-a17a-fa9603369d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:05.783344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d8d1a-482f-4026-a17a-fa9603369d54 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:05.785947 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2548/5145] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:05 2022] POST /identity/v3/projects => generated 391 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:05.791427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b1cc1-2f23-4a63-89f1-d1e026970822 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:05.801157 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2598/5146] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:11:05.813427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d8d1a-482f-4026-a17a-fa9603369d54 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:05.826268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:05.827971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d8d1a-482f-4026-a17a-fa9603369d54 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:05.895027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b1cc1-2f23-4a63-89f1-d1e026970822 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:05.903215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537aaa1b-75bd-4468-aef2-14f7a49ef3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:05.903633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537aaa1b-75bd-4468-aef2-14f7a49ef3ae admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:05.903976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537aaa1b-75bd-4468-aef2-14f7a49ef3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:05.904328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537aaa1b-75bd-4468-aef2-14f7a49ef3ae admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:05.905131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537aaa1b-75bd-4468-aef2-14f7a49ef3ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:05.907850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537aaa1b-75bd-4468-aef2-14f7a49ef3ae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:05.942128 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2549/5147] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:05 2022] POST /identity/v3/users => generated 296 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:05.949927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537aaa1b-75bd-4468-aef2-14f7a49ef3ae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:05.985779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537aaa1b-75bd-4468-aef2-14f7a49ef3ae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:05.990357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee48cb-d357-409b-81bf-9b459716fe86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:05.990553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee48cb-d357-409b-81bf-9b459716fe86 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:05.991224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee48cb-d357-409b-81bf-9b459716fe86 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:05.994231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee48cb-d357-409b-81bf-9b459716fe86 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:05.994924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee48cb-d357-409b-81bf-9b459716fe86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:05.996803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee48cb-d357-409b-81bf-9b459716fe86 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:05.999370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac8246-0ca6-4d0e-8d5b-3fa615a6dc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:06.000488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac8246-0ca6-4d0e-8d5b-3fa615a6dc8f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:06.000488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac8246-0ca6-4d0e-8d5b-3fa615a6dc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:06.000697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac8246-0ca6-4d0e-8d5b-3fa615a6dc8f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:06.001521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac8246-0ca6-4d0e-8d5b-3fa615a6dc8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:06.006947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac8246-0ca6-4d0e-8d5b-3fa615a6dc8f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:06.018404 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2599/5148] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:06.022836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee48cb-d357-409b-81bf-9b459716fe86 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:06.024685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-abee48cb-d357-409b-81bf-9b459716fe86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:06.025196 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2550/5149] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:06.028996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac8246-0ca6-4d0e-8d5b-3fa615a6dc8f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:06.045085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee48cb-d357-409b-81bf-9b459716fe86 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:06.047416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac8246-0ca6-4d0e-8d5b-3fa615a6dc8f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:06.053282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2aa8b-4968-4cf5-80cb-f634676e2c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:06.054140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2aa8b-4968-4cf5-80cb-f634676e2c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:06.054440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2aa8b-4968-4cf5-80cb-f634676e2c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:06.054706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2aa8b-4968-4cf5-80cb-f634676e2c69 admin admin] PATH_INFO: `/v3/users/bbc87613dd6349babc15334a66bee64f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:06.056098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a8b47-f4c4-4bf9-adfe-239e93051349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:06.057526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a8b47-f4c4-4bf9-adfe-239e93051349 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:06.059085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a8b47-f4c4-4bf9-adfe-239e93051349 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:06.059850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a8b47-f4c4-4bf9-adfe-239e93051349 admin admin] PATH_INFO: `/v3/projects/7af26a7f1fc742d39f13dbed6031f8b6/users/bb87b3c690584dbdbb0916d69e87065b/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:06.072518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2aa8b-4968-4cf5-80cb-f634676e2c69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbc87613dd6349babc15334a66bee64f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:06.077331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2aa8b-4968-4cf5-80cb-f634676e2c69 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:06.079322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a8b47-f4c4-4bf9-adfe-239e93051349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af26a7f1fc742d39f13dbed6031f8b6, user_id=bb87b3c690584dbdbb0916d69e87065b, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:06.082937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a8b47-f4c4-4bf9-adfe-239e93051349 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:06.108743 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2551/5150] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:06 2022] PUT /identity/v3/projects/7af26a7f1fc742d39f13dbed6031f8b6/users/bb87b3c690584dbdbb0916d69e87065b/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:06.109787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a8b47-f4c4-4bf9-adfe-239e93051349 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:06.187098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a2aa8b-4968-4cf5-80cb-f634676e2c69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbc87613dd6349babc15334a66bee64f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:06.197088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a2aa8b-4968-4cf5-80cb-f634676e2c69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbc87613dd6349babc15334a66bee64f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:06.206431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a2aa8b-4968-4cf5-80cb-f634676e2c69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbc87613dd6349babc15334a66bee64f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:06.219178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a2aa8b-4968-4cf5-80cb-f634676e2c69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbc87613dd6349babc15334a66bee64f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:06.221039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a2aa8b-4968-4cf5-80cb-f634676e2c69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbc87613dd6349babc15334a66bee64f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:06.227987 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2600/5151] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:06 2022] DELETE /identity/v3/users/bbc87613dd6349babc15334a66bee64f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:06.247190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e5a3f-c52b-4e51-857a-db806a0e607a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:06.247309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e5a3f-c52b-4e51-857a-db806a0e607a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:06.247561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e5a3f-c52b-4e51-857a-db806a0e607a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:06.266555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a8b47-f4c4-4bf9-adfe-239e93051349 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:06.293880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-963e5a3f-c52b-4e51-857a-db806a0e607a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:06.295623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-963e5a3f-c52b-4e51-857a-db806a0e607a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:06.319690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:06.353335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acfccd-d9b9-4d69-8631-a68467f770ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:06.353954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acfccd-d9b9-4d69-8631-a68467f770ea admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:06.354224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acfccd-d9b9-4d69-8631-a68467f770ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:06.354484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acfccd-d9b9-4d69-8631-a68467f770ea admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:06.355552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acfccd-d9b9-4d69-8631-a68467f770ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:06.357363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acfccd-d9b9-4d69-8631-a68467f770ea admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:06.382218 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2552/5152] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:06.383718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acfccd-d9b9-4d69-8631-a68467f770ea admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:06.387303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d3acfccd-d9b9-4d69-8631-a68467f770ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:06.404274 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2601/5153] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:06.404609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acfccd-d9b9-4d69-8631-a68467f770ea admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:06.411072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb2bf0-899a-48cd-a21e-989619c6aeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:06.413188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb2bf0-899a-48cd-a21e-989619c6aeb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:06.413478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb2bf0-899a-48cd-a21e-989619c6aeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:06.413750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb2bf0-899a-48cd-a21e-989619c6aeb9 admin admin] PATH_INFO: `/v3/projects/64e425c3426d451b96189d0ae915c0ef/users/16513d91960b44dca3ff68c39969a673/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:06.421279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e5a3f-c52b-4e51-857a-db806a0e607a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:06.422976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-963e5a3f-c52b-4e51-857a-db806a0e607a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:06.433107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:06.438851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb2bf0-899a-48cd-a21e-989619c6aeb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e425c3426d451b96189d0ae915c0ef, user_id=16513d91960b44dca3ff68c39969a673, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:06.442654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb2bf0-899a-48cd-a21e-989619c6aeb9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:06.469863 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2553/5154] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:06 2022] PUT /identity/v3/projects/64e425c3426d451b96189d0ae915c0ef/users/16513d91960b44dca3ff68c39969a673/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:06.481853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1f03c-c313-4b76-a555-f6cd2aa70a55 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:06.482252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1f03c-c313-4b76-a555-f6cd2aa70a55 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:06.482693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1f03c-c313-4b76-a555-f6cd2aa70a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:06.521521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e5a3f-c52b-4e51-857a-db806a0e607a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:06.528109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-22d1f03c-c313-4b76-a555-f6cd2aa70a55 None None] MFA Rules not processed for user `16513d91960b44dca3ff68c39969a673`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:06.529432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-22d1f03c-c313-4b76-a555-f6cd2aa70a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:06.601394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3ff9c-c61d-489e-bb67-15489e5c24d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:06.602084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3ff9c-c61d-489e-bb67-15489e5c24d9 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:06.602359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3ff9c-c61d-489e-bb67-15489e5c24d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:06.602809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3ff9c-c61d-489e-bb67-15489e5c24d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:06.612099 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2554/5155] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:06.612099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1f03c-c313-4b76-a555-f6cd2aa70a55 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:06.612099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-22d1f03c-c313-4b76-a555-f6cd2aa70a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:06.676520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3ff9c-c61d-489e-bb67-15489e5c24d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:06.678134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3ff9c-c61d-489e-bb67-15489e5c24d9 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:06.689058 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2602/5156] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:06.689904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3ff9c-c61d-489e-bb67-15489e5c24d9 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:06.702240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3ff9c-c61d-489e-bb67-15489e5c24d9 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:06.703889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1f03c-c313-4b76-a555-f6cd2aa70a55 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:06.708900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91716ca-98ec-4eb1-8eca-de005c6a1729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:06.710057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91716ca-98ec-4eb1-8eca-de005c6a1729 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:06.710252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91716ca-98ec-4eb1-8eca-de005c6a1729 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:06.710522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91716ca-98ec-4eb1-8eca-de005c6a1729 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:06.710800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca6729-fb50-46b2-8203-7d2f041ab599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:06.711340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91716ca-98ec-4eb1-8eca-de005c6a1729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:06.711609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca6729-fb50-46b2-8203-7d2f041ab599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:06.711898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca6729-fb50-46b2-8203-7d2f041ab599 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:06.712190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca6729-fb50-46b2-8203-7d2f041ab599 admin admin] PATH_INFO: `/v3/projects/b1584c72d057406ea00af9d0277fa47f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:06.713383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91716ca-98ec-4eb1-8eca-de005c6a1729 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:06.737831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca6729-fb50-46b2-8203-7d2f041ab599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1584c72d057406ea00af9d0277fa47f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:06.739868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca6729-fb50-46b2-8203-7d2f041ab599 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:06.744272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:06.759195 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2603/5157] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:06 2022] POST /identity/v3/projects => generated 393 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:06.763258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91716ca-98ec-4eb1-8eca-de005c6a1729 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:06.815603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7dca6729-fb50-46b2-8203-7d2f041ab599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1584c72d057406ea00af9d0277fa47f', 'request_id': 'req-7dca6729-fb50-46b2-8203-7d2f041ab599'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:06.832661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7dca6729-fb50-46b2-8203-7d2f041ab599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1584c72d057406ea00af9d0277fa47f', 'request_id': 'req-7dca6729-fb50-46b2-8203-7d2f041ab599'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:06.845530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91716ca-98ec-4eb1-8eca-de005c6a1729 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:06.846582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7dca6729-fb50-46b2-8203-7d2f041ab599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1584c72d057406ea00af9d0277fa47f', 'request_id': 'req-7dca6729-fb50-46b2-8203-7d2f041ab599'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:06.861039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7dca6729-fb50-46b2-8203-7d2f041ab599 admin admin] The token cache is being invalidate because project b1584c72d057406ea00af9d0277fa47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:06.861327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7dca6729-fb50-46b2-8203-7d2f041ab599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:06.863194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7dca6729-fb50-46b2-8203-7d2f041ab599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:06.865731 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2555/5158] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:06 2022] DELETE /identity/v3/projects/b1584c72d057406ea00af9d0277fa47f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:06.871357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d217b51-7784-4f5e-80de-a6ba0c5070de None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:06.871701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d217b51-7784-4f5e-80de-a6ba0c5070de None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:06.872649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d217b51-7784-4f5e-80de-a6ba0c5070de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:06.882762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:06.899053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5d217b51-7784-4f5e-80de-a6ba0c5070de None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:06.901150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5d217b51-7784-4f5e-80de-a6ba0c5070de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:06.952695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38840c6-4446-4f07-be25-0923d18ddca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:06.953351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38840c6-4446-4f07-be25-0923d18ddca2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:06.954676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38840c6-4446-4f07-be25-0923d18ddca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:06.955027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38840c6-4446-4f07-be25-0923d18ddca2 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:06.955831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38840c6-4446-4f07-be25-0923d18ddca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:06.958330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38840c6-4446-4f07-be25-0923d18ddca2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:06.999848 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2604/5159] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:06 2022] POST /identity/v3/users => generated 303 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:07.004686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38840c6-4446-4f07-be25-0923d18ddca2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:07.007835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b38840c6-4446-4f07-be25-0923d18ddca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:07.026075 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2556/5160] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:07.028270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38840c6-4446-4f07-be25-0923d18ddca2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:07.037105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d13616-2ed2-42e1-a148-8ef512b1e489 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:07.038281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d13616-2ed2-42e1-a148-8ef512b1e489 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:07.038618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d13616-2ed2-42e1-a148-8ef512b1e489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:07.039820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0384c4d-2c8e-43ac-9446-5f536695ac51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:07.040551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0384c4d-2c8e-43ac-9446-5f536695ac51 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:07.040828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0384c4d-2c8e-43ac-9446-5f536695ac51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:07.041523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0384c4d-2c8e-43ac-9446-5f536695ac51 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:07.042184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0384c4d-2c8e-43ac-9446-5f536695ac51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:07.044602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0384c4d-2c8e-43ac-9446-5f536695ac51 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:07.089493 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2605/5161] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:07.093383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0384c4d-2c8e-43ac-9446-5f536695ac51 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:07.100293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-76d13616-2ed2-42e1-a148-8ef512b1e489 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:07.111559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-76d13616-2ed2-42e1-a148-8ef512b1e489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:07.132724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0384c4d-2c8e-43ac-9446-5f536695ac51 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:07.146039 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2557/5162] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:07.148316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82313302-72c3-4050-8388-ccd2112b2d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:07.149029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82313302-72c3-4050-8388-ccd2112b2d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:07.151408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82313302-72c3-4050-8388-ccd2112b2d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:07.151716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82313302-72c3-4050-8388-ccd2112b2d0e admin admin] PATH_INFO: `/v3/projects/73d6766729114253bd3b864cc7681d51/users/d235a69333794df38191f429797d1d48/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:07.151864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d13616-2ed2-42e1-a148-8ef512b1e489 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:07.164637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d13616-2ed2-42e1-a148-8ef512b1e489 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:07.178722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82313302-72c3-4050-8388-ccd2112b2d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=73d6766729114253bd3b864cc7681d51, user_id=d235a69333794df38191f429797d1d48, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:07.182012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82313302-72c3-4050-8388-ccd2112b2d0e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:07.186342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294387c9-b033-48cc-b436-a3f294ae524d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:07.186971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294387c9-b033-48cc-b436-a3f294ae524d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:07.187252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294387c9-b033-48cc-b436-a3f294ae524d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:07.187506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294387c9-b033-48cc-b436-a3f294ae524d admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:07.188730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294387c9-b033-48cc-b436-a3f294ae524d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:07.189039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294387c9-b033-48cc-b436-a3f294ae524d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:07.191160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294387c9-b033-48cc-b436-a3f294ae524d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:07.209906 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2606/5163] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:07 2022] PUT /identity/v3/projects/73d6766729114253bd3b864cc7681d51/users/d235a69333794df38191f429797d1d48/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:07.210806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82313302-72c3-4050-8388-ccd2112b2d0e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:07.228003 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2558/5164] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:07.229710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294387c9-b033-48cc-b436-a3f294ae524d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:07.244906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294387c9-b033-48cc-b436-a3f294ae524d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:07.347246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9bf8b-d5e2-40c3-889a-1ebbcb1cbb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:07.348205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9bf8b-d5e2-40c3-889a-1ebbcb1cbb7b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:07.348539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9bf8b-d5e2-40c3-889a-1ebbcb1cbb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:07.348960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9bf8b-d5e2-40c3-889a-1ebbcb1cbb7b admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:07.349892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9bf8b-d5e2-40c3-889a-1ebbcb1cbb7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:07.351943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9bf8b-d5e2-40c3-889a-1ebbcb1cbb7b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:07.400493 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2559/5165] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:07 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:07.402646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9bf8b-d5e2-40c3-889a-1ebbcb1cbb7b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:07.404348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82313302-72c3-4050-8388-ccd2112b2d0e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:07.409516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:07.480842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07051533-4ead-4068-ba21-94a3f9d3cd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:07.481674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07051533-4ead-4068-ba21-94a3f9d3cd9f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:07.481984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07051533-4ead-4068-ba21-94a3f9d3cd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:07.482253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07051533-4ead-4068-ba21-94a3f9d3cd9f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:07.483193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07051533-4ead-4068-ba21-94a3f9d3cd9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:07.484931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07051533-4ead-4068-ba21-94a3f9d3cd9f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:07.498121 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2607/5166] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:07.503306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07051533-4ead-4068-ba21-94a3f9d3cd9f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:07.516748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07051533-4ead-4068-ba21-94a3f9d3cd9f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:07.522058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a467baa-9540-493f-9532-2807ff8c7a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:07.522924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a467baa-9540-493f-9532-2807ff8c7a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:07.523193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a467baa-9540-493f-9532-2807ff8c7a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:07.523496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a467baa-9540-493f-9532-2807ff8c7a87 admin admin] PATH_INFO: `/v3/projects/73d6766729114253bd3b864cc7681d51/users/d235a69333794df38191f429797d1d48/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:07.525689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a467baa-9540-493f-9532-2807ff8c7a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73d6766729114253bd3b864cc7681d51, user_id=d235a69333794df38191f429797d1d48, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:07.528639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a467baa-9540-493f-9532-2807ff8c7a87 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:07.551147 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2608/5167] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:07 2022] PUT /identity/v3/projects/73d6766729114253bd3b864cc7681d51/users/d235a69333794df38191f429797d1d48/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:07.553137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cb2f4-fe82-4a61-b076-94a916beb6c2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:07.553402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cb2f4-fe82-4a61-b076-94a916beb6c2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:07.553670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cb2f4-fe82-4a61-b076-94a916beb6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:07.573096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9bf8b-d5e2-40c3-889a-1ebbcb1cbb7b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:07.589679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d62cb2f4-fe82-4a61-b076-94a916beb6c2 None None] MFA Rules not processed for user `d235a69333794df38191f429797d1d48`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:07.591403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d62cb2f4-fe82-4a61-b076-94a916beb6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:07.603407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:07.702187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece0f65-915d-4fcd-853f-a9eb5fe00f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:07.702898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece0f65-915d-4fcd-853f-a9eb5fe00f88 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:07.703505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece0f65-915d-4fcd-853f-a9eb5fe00f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:07.703844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece0f65-915d-4fcd-853f-a9eb5fe00f88 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:07.705740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece0f65-915d-4fcd-853f-a9eb5fe00f88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:07.710729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece0f65-915d-4fcd-853f-a9eb5fe00f88 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:07.745265 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2609/5168] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:07 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:07.748911 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2560/5169] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:07 2022] POST /identity/v3/users => generated 294 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:07.752953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cb2f4-fe82-4a61-b076-94a916beb6c2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:07.759151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece0f65-915d-4fcd-853f-a9eb5fe00f88 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:07.764207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7ece0f65-915d-4fcd-853f-a9eb5fe00f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:07.790349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cb2f4-fe82-4a61-b076-94a916beb6c2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:07.805069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1aff1-d69d-4edf-94f6-6f671d756520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:07.805712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1aff1-d69d-4edf-94f6-6f671d756520 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:07.805985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1aff1-d69d-4edf-94f6-6f671d756520 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:07.806250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1aff1-d69d-4edf-94f6-6f671d756520 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:07.806896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1aff1-d69d-4edf-94f6-6f671d756520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:07.814782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1aff1-d69d-4edf-94f6-6f671d756520 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:07.841585 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2610/5170] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:07.843887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1aff1-d69d-4edf-94f6-6f671d756520 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:07.862701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1aff1-d69d-4edf-94f6-6f671d756520 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:07.876883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38834ed6-49b4-4337-a90a-80082800c1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:07.880125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38834ed6-49b4-4337-a90a-80082800c1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:07.880936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38834ed6-49b4-4337-a90a-80082800c1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:07.881432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38834ed6-49b4-4337-a90a-80082800c1a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:07.883565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38834ed6-49b4-4337-a90a-80082800c1a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:07.888147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38834ed6-49b4-4337-a90a-80082800c1a6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:07.896221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece0f65-915d-4fcd-853f-a9eb5fe00f88 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:07.901265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbe1bc-520d-4e8c-a7c1-013a00473873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:07.901971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbe1bc-520d-4e8c-a7c1-013a00473873 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:07.902464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbe1bc-520d-4e8c-a7c1-013a00473873 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:07.902742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbe1bc-520d-4e8c-a7c1-013a00473873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:07.914610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbe1bc-520d-4e8c-a7c1-013a00473873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:07.919193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbe1bc-520d-4e8c-a7c1-013a00473873 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:07.919961 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2611/5171] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:07.921303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38834ed6-49b4-4337-a90a-80082800c1a6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:07.927377 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2561/5172] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:07 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:07.942407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38834ed6-49b4-4337-a90a-80082800c1a6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:07.952648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475e7f5-a7aa-4399-8689-323aaeb2f2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:07.953957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e7f5-a7aa-4399-8689-323aaeb2f2bb admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:07.954424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e7f5-a7aa-4399-8689-323aaeb2f2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:07.954625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e7f5-a7aa-4399-8689-323aaeb2f2bb admin admin] PATH_INFO: `/v3/projects/6c9379c9bf394e25ab8a843967a8cd45/users/61b87253caae43d6a2c5836d553ebf24/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:07.998479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475e7f5-a7aa-4399-8689-323aaeb2f2bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c9379c9bf394e25ab8a843967a8cd45, user_id=61b87253caae43d6a2c5836d553ebf24, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:07.998842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbe1bc-520d-4e8c-a7c1-013a00473873 service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:08.001996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475e7f5-a7aa-4399-8689-323aaeb2f2bb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:08.012809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbe1bc-520d-4e8c-a7c1-013a00473873 service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:08.017933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92632b-5723-44f8-b9ef-1e80ea005ad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:08.019064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92632b-5723-44f8-b9ef-1e80ea005ad2 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:08.019358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92632b-5723-44f8-b9ef-1e80ea005ad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:08.019720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92632b-5723-44f8-b9ef-1e80ea005ad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:08.022112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a92632b-5723-44f8-b9ef-1e80ea005ad2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:08.028301 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2612/5173] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:07 2022] PUT /identity/v3/projects/6c9379c9bf394e25ab8a843967a8cd45/users/61b87253caae43d6a2c5836d553ebf24/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:08.033876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350c6c0-a15a-4ba7-becf-6a52f67f9386 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:08.034620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350c6c0-a15a-4ba7-becf-6a52f67f9386 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:08.035207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350c6c0-a15a-4ba7-becf-6a52f67f9386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:08.082533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3350c6c0-a15a-4ba7-becf-6a52f67f9386 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:08.083944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3350c6c0-a15a-4ba7-becf-6a52f67f9386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:08.113589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:08.133238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92632b-5723-44f8-b9ef-1e80ea005ad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:08.134717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92632b-5723-44f8-b9ef-1e80ea005ad2 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:08.147416 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2562/5174] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:08.193177 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2613/5175] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:08.206908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92632b-5723-44f8-b9ef-1e80ea005ad2 service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:08.208393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a92632b-5723-44f8-b9ef-1e80ea005ad2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:08.317921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:08.331184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92632b-5723-44f8-b9ef-1e80ea005ad2 service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:08.457349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f60682-ee94-4d22-a2f3-d916d9ab0458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:08.458015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f60682-ee94-4d22-a2f3-d916d9ab0458 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:08.458380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f60682-ee94-4d22-a2f3-d916d9ab0458 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:08.458697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f60682-ee94-4d22-a2f3-d916d9ab0458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:08.469882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f60682-ee94-4d22-a2f3-d916d9ab0458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:08.471838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f60682-ee94-4d22-a2f3-d916d9ab0458 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:08.483453 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2563/5176] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:09.567988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350c6c0-a15a-4ba7-becf-6a52f67f9386 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:09.569803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3350c6c0-a15a-4ba7-becf-6a52f67f9386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:09.654987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350c6c0-a15a-4ba7-becf-6a52f67f9386 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:09.662139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b038bc5-f8c8-412e-9722-ee3e4e340892 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:09.662921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b038bc5-f8c8-412e-9722-ee3e4e340892 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:09.663293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b038bc5-f8c8-412e-9722-ee3e4e340892 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:09.663658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b038bc5-f8c8-412e-9722-ee3e4e340892 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:09.665740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0b038bc5-f8c8-412e-9722-ee3e4e340892 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:09.684292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b038bc5-f8c8-412e-9722-ee3e4e340892 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:09.687497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b038bc5-f8c8-412e-9722-ee3e4e340892 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:09.703190 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2614/5177] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:09.711604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f60682-ee94-4d22-a2f3-d916d9ab0458 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:09.714871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-01f60682-ee94-4d22-a2f3-d916d9ab0458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:09.896722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f60682-ee94-4d22-a2f3-d916d9ab0458 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:09.906194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c4ff7-a9f3-4c90-9400-9652b4bc3930 tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16e351e92be4a18bcd02a937d051457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '654136bb95e84a238901b4f282ef64de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:09.906954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c4ff7-a9f3-4c90-9400-9652b4bc3930 tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:09.908504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c4ff7-a9f3-4c90-9400-9652b4bc3930 tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:09.909466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c4ff7-a9f3-4c90-9400-9652b4bc3930 tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:09.910676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c4ff7-a9f3-4c90-9400-9652b4bc3930 tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:09.912611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c4ff7-a9f3-4c90-9400-9652b4bc3930 tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:09.915088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c4ff7-a9f3-4c90-9400-9652b4bc3930 tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:09.945842 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2564/5178] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:09.947213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b038bc5-f8c8-412e-9722-ee3e4e340892 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:09.961818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b038bc5-f8c8-412e-9722-ee3e4e340892 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:09.983598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bae0f7-f864-4205-8959-fbb838995a3f tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16e351e92be4a18bcd02a937d051457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '654136bb95e84a238901b4f282ef64de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:09.984528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bae0f7-f864-4205-8959-fbb838995a3f tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:09.987268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bae0f7-f864-4205-8959-fbb838995a3f tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:09.987558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bae0f7-f864-4205-8959-fbb838995a3f tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:09.988198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bae0f7-f864-4205-8959-fbb838995a3f tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:10.003517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bae0f7-f864-4205-8959-fbb838995a3f tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:10.077049 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2615/5179] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:09 2022] POST /identity/v3/projects => generated 348 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:10.079808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c4ff7-a9f3-4c90-9400-9652b4bc3930 tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:10.096659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c4ff7-a9f3-4c90-9400-9652b4bc3930 tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:10.212186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88520df9-1528-4673-bf16-893731dec9bf tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16e351e92be4a18bcd02a937d051457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '654136bb95e84a238901b4f282ef64de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:10.212931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88520df9-1528-4673-bf16-893731dec9bf tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:10.213209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88520df9-1528-4673-bf16-893731dec9bf tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:10.215332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88520df9-1528-4673-bf16-893731dec9bf tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:10.216110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88520df9-1528-4673-bf16-893731dec9bf tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:10.216574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88520df9-1528-4673-bf16-893731dec9bf tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:10.218467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88520df9-1528-4673-bf16-893731dec9bf tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:10.238595 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2565/5180] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:10.529726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bae0f7-f864-4205-8959-fbb838995a3f tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:10.701556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bae0f7-f864-4205-8959-fbb838995a3f tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:10.711318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:10.714808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:10.715557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:10.716514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] PATH_INFO: `/v3/users/16513d91960b44dca3ff68c39969a673` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:10.718664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16513d91960b44dca3ff68c39969a673)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:10.725896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:10.805894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:10.824746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16513d91960b44dca3ff68c39969a673'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:10.846638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16513d91960b44dca3ff68c39969a673'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:10.858045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16513d91960b44dca3ff68c39969a673'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:10.888228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16513d91960b44dca3ff68c39969a673'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:10.889111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16513d91960b44dca3ff68c39969a673'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:10.911880 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2616/5181] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:10 2022] DELETE /identity/v3/users/16513d91960b44dca3ff68c39969a673 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:10.916325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fba777-c9d6-43f8-a7d9-76ca974f3cf2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:10.916863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fba777-c9d6-43f8-a7d9-76ca974f3cf2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:10.916863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fba777-c9d6-43f8-a7d9-76ca974f3cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:10.969452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-27fba777-c9d6-43f8-a7d9-76ca974f3cf2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:10.970889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-27fba777-c9d6-43f8-a7d9-76ca974f3cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:11.071086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:11.172480 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2566/5182] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:11.183223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:11.184689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:11.291539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72beeee6-eed7-48d6-8c54-eddac0377422 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:11.306249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045dcf7-934f-464c-9f4a-760173c131dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:11.306939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045dcf7-934f-464c-9f4a-760173c131dc service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:11.307949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045dcf7-934f-464c-9f4a-760173c131dc service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:11.307949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045dcf7-934f-464c-9f4a-760173c131dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:11.317907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fba777-c9d6-43f8-a7d9-76ca974f3cf2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:11.321720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-27fba777-c9d6-43f8-a7d9-76ca974f3cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:11.341768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045dcf7-934f-464c-9f4a-760173c131dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:11.346296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045dcf7-934f-464c-9f4a-760173c131dc service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:11.370841 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2617/5183] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:11.494300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fba777-c9d6-43f8-a7d9-76ca974f3cf2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:11.499758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16e351e92be4a18bcd02a937d051457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '654136bb95e84a238901b4f282ef64de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:11.500725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:11.501225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:11.501695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] PATH_INFO: `/v3/projects/24a3d1d7af4e48a6ba59995315179101` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:11.503454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: Authorizing `identity:delete_project(project_id=24a3d1d7af4e48a6ba59995315179101)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:11.505647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:11.582223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a3d1d7af4e48a6ba59995315179101', 'request_id': 'req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:11.592852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a3d1d7af4e48a6ba59995315179101', 'request_id': 'req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:11.624616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a3d1d7af4e48a6ba59995315179101', 'request_id': 'req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:11.639167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] The token cache is being invalidate because project 24a3d1d7af4e48a6ba59995315179101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:11.639447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:11.640709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:11.643227 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2567/5184] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:11 2022] DELETE /identity/v3/projects/24a3d1d7af4e48a6ba59995315179101 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:11.661810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045dcf7-934f-464c-9f4a-760173c131dc service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:11.663190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a045dcf7-934f-464c-9f4a-760173c131dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:11.686737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045dcf7-934f-464c-9f4a-760173c131dc service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:11.792121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f82ac1-c2ff-424b-8708-3dc3a2c29302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:11.792725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f82ac1-c2ff-424b-8708-3dc3a2c29302 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:11.792951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f82ac1-c2ff-424b-8708-3dc3a2c29302 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:11.793220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f82ac1-c2ff-424b-8708-3dc3a2c29302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:11.795837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-08f82ac1-c2ff-424b-8708-3dc3a2c29302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:11.917819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f82ac1-c2ff-424b-8708-3dc3a2c29302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:11.919271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f82ac1-c2ff-424b-8708-3dc3a2c29302 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:11.938703 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2618/5185] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:12.526059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:12.528080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:12.684466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe7847-ad9f-4f34-9366-78c1e20ce9fc tempest-RoutersIpV6AdminTest-1967360814 tempest-RoutersIpV6AdminTest-1967360814-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:12.694205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:12.696765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:12.698306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:12.703910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb admin admin] PATH_INFO: `/v3/projects/64e425c3426d451b96189d0ae915c0ef` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:12.705953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e425c3426d451b96189d0ae915c0ef)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:12.710307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:12.722964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:12.875972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e425c3426d451b96189d0ae915c0ef', 'request_id': 'req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:12.886058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e425c3426d451b96189d0ae915c0ef', 'request_id': 'req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:12.897521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e425c3426d451b96189d0ae915c0ef', 'request_id': 'req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:12.914740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb admin admin] The token cache is being invalidate because project 64e425c3426d451b96189d0ae915c0ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:12.916623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:12.917721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2c08c97-8aee-4fad-9d31-3f202eecc5bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:12.923736 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2568/5186] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:12 2022] DELETE /identity/v3/projects/64e425c3426d451b96189d0ae915c0ef => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:12.924872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f82ac1-c2ff-424b-8708-3dc3a2c29302 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:12.940395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f82ac1-c2ff-424b-8708-3dc3a2c29302 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:12.955294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:13.158141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:13.160317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:13.160735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:13.161110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] PATH_INFO: `/v3/users/d235a69333794df38191f429797d1d48` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:13.164338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d235a69333794df38191f429797d1d48)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:13.167038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:13.289583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd235a69333794df38191f429797d1d48'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:13.313961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd235a69333794df38191f429797d1d48'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:13.335937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd235a69333794df38191f429797d1d48'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:13.370317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd235a69333794df38191f429797d1d48'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:13.371424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd235a69333794df38191f429797d1d48'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:13.381175 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2619/5187] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:12 2022] DELETE /identity/v3/users/d235a69333794df38191f429797d1d48 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:14.334889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317605f-f94a-417b-af1a-3d8b88d3ffd2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:14.335773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317605f-f94a-417b-af1a-3d8b88d3ffd2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:14.335773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317605f-f94a-417b-af1a-3d8b88d3ffd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:14.400736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:14.403512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:14.455877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c317605f-f94a-417b-af1a-3d8b88d3ffd2 None None] MFA Rules not processed for user `bb87b3c690584dbdbb0916d69e87065b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:14.457241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c317605f-f94a-417b-af1a-3d8b88d3ffd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:14.593829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec5fa6-fc57-446a-a13f-ed0534dfe25b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:14.609232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:14.610198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:14.610662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:14.611097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b admin admin] PATH_INFO: `/v3/projects/73d6766729114253bd3b864cc7681d51` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:14.613194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b admin admin] RBAC: Authorizing `identity:delete_project(project_id=73d6766729114253bd3b864cc7681d51)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:14.615358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:14.640072 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2569/5188] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:14 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:14.652143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c1299-95af-4123-a203-b38153f77827 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:14.652484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c1299-95af-4123-a203-b38153f77827 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:14.652910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c1299-95af-4123-a203-b38153f77827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:14.695848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0b1c1299-95af-4123-a203-b38153f77827 None None] MFA Rules not processed for user `bb87b3c690584dbdbb0916d69e87065b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:14.697165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0b1c1299-95af-4123-a203-b38153f77827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:14.703933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:14.729954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73d6766729114253bd3b864cc7681d51', 'request_id': 'req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:14.746505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73d6766729114253bd3b864cc7681d51', 'request_id': 'req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:14.757875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73d6766729114253bd3b864cc7681d51', 'request_id': 'req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:14.784886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b admin admin] The token cache is being invalidate because project 73d6766729114253bd3b864cc7681d51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:14.786319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:14.788213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d56b1c5e-6ade-44a4-95cc-dfd8d8ae346b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:14.794758 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2620/5189] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:14 2022] DELETE /identity/v3/projects/73d6766729114253bd3b864cc7681d51 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:14.809213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a1efe-e131-4181-adc8-77caa8048240 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:14.809566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a1efe-e131-4181-adc8-77caa8048240 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:14.809777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a1efe-e131-4181-adc8-77caa8048240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:14.833949 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2570/5190] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:14 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:14.854772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c1299-95af-4123-a203-b38153f77827 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:14.856560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0b1c1299-95af-4123-a203-b38153f77827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:14.870330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-934a1efe-e131-4181-adc8-77caa8048240 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:14.872528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-934a1efe-e131-4181-adc8-77caa8048240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:14.937097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:15.004541 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2621/5191] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:15.008868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf7475-eea8-40f6-b4ac-e02c27375ed5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:15.009432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf7475-eea8-40f6-b4ac-e02c27375ed5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:15.009794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf7475-eea8-40f6-b4ac-e02c27375ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:15.032637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c1299-95af-4123-a203-b38153f77827 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:15.037339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dcdf7475-eea8-40f6-b4ac-e02c27375ed5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:15.038605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dcdf7475-eea8-40f6-b4ac-e02c27375ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:15.053137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da6485-f42c-4902-807a-5eec4bc0f3fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:15.053777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da6485-f42c-4902-807a-5eec4bc0f3fa service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:15.054054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da6485-f42c-4902-807a-5eec4bc0f3fa service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:15.054342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da6485-f42c-4902-807a-5eec4bc0f3fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:15.056140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e3da6485-f42c-4902-807a-5eec4bc0f3fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:15.056281 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2622/5192] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:15.060998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf7475-eea8-40f6-b4ac-e02c27375ed5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:15.072586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf7475-eea8-40f6-b4ac-e02c27375ed5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:15.077879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cebcaa-efbd-4f89-932b-5672c99ded10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:15.078627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cebcaa-efbd-4f89-932b-5672c99ded10 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:15.079000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cebcaa-efbd-4f89-932b-5672c99ded10 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:15.079364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cebcaa-efbd-4f89-932b-5672c99ded10 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:15.079758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da6485-f42c-4902-807a-5eec4bc0f3fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:15.080210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cebcaa-efbd-4f89-932b-5672c99ded10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:15.081652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cebcaa-efbd-4f89-932b-5672c99ded10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:15.083507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cebcaa-efbd-4f89-932b-5672c99ded10 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:15.084295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da6485-f42c-4902-807a-5eec4bc0f3fa service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:15.096835 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2623/5193] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:11:15.101747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cebcaa-efbd-4f89-932b-5672c99ded10 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:15.107848 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2571/5194] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:14 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:11:15.130780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cebcaa-efbd-4f89-932b-5672c99ded10 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:15.142866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d521c9-d696-4e3b-82a1-d78582a27067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:15.144899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d521c9-d696-4e3b-82a1-d78582a27067 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:15.145500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d521c9-d696-4e3b-82a1-d78582a27067 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:15.146017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d521c9-d696-4e3b-82a1-d78582a27067 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:15.147043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d521c9-d696-4e3b-82a1-d78582a27067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:15.150603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d521c9-d696-4e3b-82a1-d78582a27067 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:15.207580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da6485-f42c-4902-807a-5eec4bc0f3fa service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:15.207867 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2624/5195] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:15 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:15.220041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da6485-f42c-4902-807a-5eec4bc0f3fa service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:15.368535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca81bc-319b-44bb-9c33-cc2b58524cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:15.369946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca81bc-319b-44bb-9c33-cc2b58524cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:15.370377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca81bc-319b-44bb-9c33-cc2b58524cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:15.370810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca81bc-319b-44bb-9c33-cc2b58524cf4 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:15.371641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca81bc-319b-44bb-9c33-cc2b58524cf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:15.374037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca81bc-319b-44bb-9c33-cc2b58524cf4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:15.430577 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2572/5196] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:15 2022] POST /identity/v3/users => generated 291 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:15.436811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d521c9-d696-4e3b-82a1-d78582a27067 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:15.463006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d521c9-d696-4e3b-82a1-d78582a27067 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:15.479322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b0f2d-3f7e-486d-95bc-c0d6c8738ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:15.481761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b0f2d-3f7e-486d-95bc-c0d6c8738ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:15.482152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b0f2d-3f7e-486d-95bc-c0d6c8738ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:15.482517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b0f2d-3f7e-486d-95bc-c0d6c8738ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:15.484035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b0f2d-3f7e-486d-95bc-c0d6c8738ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:15.490372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b0f2d-3f7e-486d-95bc-c0d6c8738ce5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:15.520280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca81bc-319b-44bb-9c33-cc2b58524cf4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:15.521777 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2625/5197] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:15.540418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca81bc-319b-44bb-9c33-cc2b58524cf4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:15.546213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926bbc8-98d4-4724-8a2e-b0da722a688d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:15.546705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926bbc8-98d4-4724-8a2e-b0da722a688d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:15.546984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926bbc8-98d4-4724-8a2e-b0da722a688d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:15.547253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926bbc8-98d4-4724-8a2e-b0da722a688d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:15.547915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926bbc8-98d4-4724-8a2e-b0da722a688d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:15.549751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926bbc8-98d4-4724-8a2e-b0da722a688d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:15.572731 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2573/5198] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:15.575138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b0f2d-3f7e-486d-95bc-c0d6c8738ce5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:15.596096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b0f2d-3f7e-486d-95bc-c0d6c8738ce5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:15.623771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c84cd9-7ee0-4185-8fc2-a1296e173808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:15.625421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c84cd9-7ee0-4185-8fc2-a1296e173808 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:15.625940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c84cd9-7ee0-4185-8fc2-a1296e173808 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:15.626577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c84cd9-7ee0-4185-8fc2-a1296e173808 admin admin] PATH_INFO: `/v3/projects/40d1d17de7374bfea67096a7075b9933/users/9cf674aceaef4bafb15472466a55bea1/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:15.650157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c84cd9-7ee0-4185-8fc2-a1296e173808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d1d17de7374bfea67096a7075b9933, user_id=9cf674aceaef4bafb15472466a55bea1, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:15.653301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c84cd9-7ee0-4185-8fc2-a1296e173808 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:15.670099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926bbc8-98d4-4724-8a2e-b0da722a688d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:15.672174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4926bbc8-98d4-4724-8a2e-b0da722a688d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:15.709220 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2626/5199] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:15 2022] PUT /identity/v3/projects/40d1d17de7374bfea67096a7075b9933/users/9cf674aceaef4bafb15472466a55bea1/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:15.713033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97eb7f5-fbad-4dbd-acca-163bfe4f7c1c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:15.713637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97eb7f5-fbad-4dbd-acca-163bfe4f7c1c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:15.714127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97eb7f5-fbad-4dbd-acca-163bfe4f7c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:15.760237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d97eb7f5-fbad-4dbd-acca-163bfe4f7c1c None None] MFA Rules not processed for user `9cf674aceaef4bafb15472466a55bea1`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:15.762077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d97eb7f5-fbad-4dbd-acca-163bfe4f7c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:15.818985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926bbc8-98d4-4724-8a2e-b0da722a688d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:15.838700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:15.897061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:15.905509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b6a50-d400-4f42-beba-1789fe3ae04b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:15.905871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b6a50-d400-4f42-beba-1789fe3ae04b service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:15.906058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b6a50-d400-4f42-beba-1789fe3ae04b service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:15.906401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b6a50-d400-4f42-beba-1789fe3ae04b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:15.910095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f78b6a50-d400-4f42-beba-1789fe3ae04b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:15.941477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b6a50-d400-4f42-beba-1789fe3ae04b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:15.943239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b6a50-d400-4f42-beba-1789fe3ae04b service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:15.943728 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2627/5200] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:15.945906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97eb7f5-fbad-4dbd-acca-163bfe4f7c1c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:15.957678 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2574/5201] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:11:16.079413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97eb7f5-fbad-4dbd-acca-163bfe4f7c1c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:16.086637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215339b-3002-4ba6-a4a4-4938fb82ffc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:16.087417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215339b-3002-4ba6-a4a4-4938fb82ffc0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:16.087893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215339b-3002-4ba6-a4a4-4938fb82ffc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:16.088262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215339b-3002-4ba6-a4a4-4938fb82ffc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:16.089051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215339b-3002-4ba6-a4a4-4938fb82ffc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:16.091367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215339b-3002-4ba6-a4a4-4938fb82ffc0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:16.143863 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2628/5202] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:15 2022] POST /identity/v3/projects => generated 379 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:16.148361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b6a50-d400-4f42-beba-1789fe3ae04b service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:16.165729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b6a50-d400-4f42-beba-1789fe3ae04b service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:16.180402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b4e0e-4a4f-48b7-b301-89a98724e1af None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:16.180870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b4e0e-4a4f-48b7-b301-89a98724e1af None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:16.180991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b4e0e-4a4f-48b7-b301-89a98724e1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:16.269950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e94b4e0e-4a4f-48b7-b301-89a98724e1af None None] MFA Rules not processed for user `61b87253caae43d6a2c5836d553ebf24`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:16.269950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e94b4e0e-4a4f-48b7-b301-89a98724e1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:16.321069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a363c-77f6-4624-b44e-542fe82e7035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:16.322134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a363c-77f6-4624-b44e-542fe82e7035 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:16.322642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a363c-77f6-4624-b44e-542fe82e7035 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:16.323119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a363c-77f6-4624-b44e-542fe82e7035 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:16.323963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a363c-77f6-4624-b44e-542fe82e7035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:16.333079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a363c-77f6-4624-b44e-542fe82e7035 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:16.368837 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2575/5203] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:16 2022] POST /identity/v3/users => generated 296 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:16.370787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a363c-77f6-4624-b44e-542fe82e7035 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:16.398671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a363c-77f6-4624-b44e-542fe82e7035 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:16.410128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa09d7-3627-4f31-b474-3666a2a7c12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:16.413613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa09d7-3627-4f31-b474-3666a2a7c12d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:16.414192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa09d7-3627-4f31-b474-3666a2a7c12d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:16.414703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa09d7-3627-4f31-b474-3666a2a7c12d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:16.415579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa09d7-3627-4f31-b474-3666a2a7c12d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:16.419187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa09d7-3627-4f31-b474-3666a2a7c12d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:16.431894 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2629/5204] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:16.439722 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2576/5205] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:16.444463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b4e0e-4a4f-48b7-b301-89a98724e1af None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:16.459463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b4e0e-4a4f-48b7-b301-89a98724e1af None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:16.475381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33168c6-c934-4cf3-aa34-97185d9dec40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:16.484316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33168c6-c934-4cf3-aa34-97185d9dec40 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:16.484996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33168c6-c934-4cf3-aa34-97185d9dec40 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:16.485479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33168c6-c934-4cf3-aa34-97185d9dec40 admin admin] PATH_INFO: `/v3/projects/6aeeb05ef129436c9ead34abec7425af/users/ad51907e4d4c492b99ba2ce8c8e4c170/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:16.524697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33168c6-c934-4cf3-aa34-97185d9dec40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aeeb05ef129436c9ead34abec7425af, user_id=ad51907e4d4c492b99ba2ce8c8e4c170, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:16.529056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33168c6-c934-4cf3-aa34-97185d9dec40 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:16.573767 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2630/5206] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:16 2022] PUT /identity/v3/projects/6aeeb05ef129436c9ead34abec7425af/users/ad51907e4d4c492b99ba2ce8c8e4c170/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:16.577944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa09d7-3627-4f31-b474-3666a2a7c12d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:16.592779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa09d7-3627-4f31-b474-3666a2a7c12d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:16.761383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a046af-95b2-4667-b48e-b61dd24bd1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:16.762329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a046af-95b2-4667-b48e-b61dd24bd1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:16.762774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a046af-95b2-4667-b48e-b61dd24bd1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:16.763250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a046af-95b2-4667-b48e-b61dd24bd1c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:16.763867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a046af-95b2-4667-b48e-b61dd24bd1c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:16.770601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a046af-95b2-4667-b48e-b61dd24bd1c7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:16.796516 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2577/5207] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:16.798905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33168c6-c934-4cf3-aa34-97185d9dec40 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:16.819234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33168c6-c934-4cf3-aa34-97185d9dec40 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:16.833601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5aba4-5e58-4102-a638-dd0ed2c4af3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:16.834633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5aba4-5e58-4102-a638-dd0ed2c4af3f admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:16.835077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5aba4-5e58-4102-a638-dd0ed2c4af3f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:16.835517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5aba4-5e58-4102-a638-dd0ed2c4af3f admin admin] PATH_INFO: `/v3/projects/6aeeb05ef129436c9ead34abec7425af/users/ad51907e4d4c492b99ba2ce8c8e4c170/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:16.839913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5aba4-5e58-4102-a638-dd0ed2c4af3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aeeb05ef129436c9ead34abec7425af, user_id=ad51907e4d4c492b99ba2ce8c8e4c170, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:16.843083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5aba4-5e58-4102-a638-dd0ed2c4af3f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:16.878350 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2631/5208] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:16 2022] PUT /identity/v3/projects/6aeeb05ef129436c9ead34abec7425af/users/ad51907e4d4c492b99ba2ce8c8e4c170/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:16.890461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba824-00c5-4f2c-a59f-47b5b7edbb05 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:16.890719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba824-00c5-4f2c-a59f-47b5b7edbb05 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:16.890841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba824-00c5-4f2c-a59f-47b5b7edbb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:16.938490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5aba4-5e58-4102-a638-dd0ed2c4af3f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:16.940707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c8b5aba4-5e58-4102-a638-dd0ed2c4af3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:16.984971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:17.004159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8a3ba824-00c5-4f2c-a59f-47b5b7edbb05 None None] MFA Rules not processed for user `ad51907e4d4c492b99ba2ce8c8e4c170`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:17.005806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a3ba824-00c5-4f2c-a59f-47b5b7edbb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:17.130170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:17.148138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5aba4-5e58-4102-a638-dd0ed2c4af3f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:17.154539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18123f9-b8ab-4575-997c-2c1e9b15652d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:17.155589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18123f9-b8ab-4575-997c-2c1e9b15652d service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:17.156084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18123f9-b8ab-4575-997c-2c1e9b15652d service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:17.157088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18123f9-b8ab-4575-997c-2c1e9b15652d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:17.159807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e18123f9-b8ab-4575-997c-2c1e9b15652d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:17.179283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18123f9-b8ab-4575-997c-2c1e9b15652d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:17.181879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18123f9-b8ab-4575-997c-2c1e9b15652d service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:17.184890 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2578/5209] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:17.196115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba824-00c5-4f2c-a59f-47b5b7edbb05 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:17.200124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a3ba824-00c5-4f2c-a59f-47b5b7edbb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:17.217295 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2632/5210] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:17.330993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba824-00c5-4f2c-a59f-47b5b7edbb05 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:17.340711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b255fb-127f-4d74-b512-5adc5ea5ef72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:17.342264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b255fb-127f-4d74-b512-5adc5ea5ef72 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:17.343326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b255fb-127f-4d74-b512-5adc5ea5ef72 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:17.343326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b255fb-127f-4d74-b512-5adc5ea5ef72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:17.374340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b255fb-127f-4d74-b512-5adc5ea5ef72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:17.376570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b255fb-127f-4d74-b512-5adc5ea5ef72 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:17.388860 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2579/5211] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:11:18.593821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18123f9-b8ab-4575-997c-2c1e9b15652d service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:18.754688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18123f9-b8ab-4575-997c-2c1e9b15652d service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:18.763367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:18.764598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:18.764598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:18.764856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] PATH_INFO: `/v3/users/9cf674aceaef4bafb15472466a55bea1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:18.773321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cf674aceaef4bafb15472466a55bea1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:18.776202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:18.889150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cf674aceaef4bafb15472466a55bea1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:18.910303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cf674aceaef4bafb15472466a55bea1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:18.924946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cf674aceaef4bafb15472466a55bea1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:18.955011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cf674aceaef4bafb15472466a55bea1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:18.957237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cf674aceaef4bafb15472466a55bea1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:18.978924 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2633/5212] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:18 2022] DELETE /identity/v3/users/9cf674aceaef4bafb15472466a55bea1 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:18.988984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ed744-6704-41f7-802d-8b4b459d8228 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:18.989566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ed744-6704-41f7-802d-8b4b459d8228 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:18.990166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ed744-6704-41f7-802d-8b4b459d8228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:19.032528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e73ed744-6704-41f7-802d-8b4b459d8228 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:19.037995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e73ed744-6704-41f7-802d-8b4b459d8228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:19.233678 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2580/5213] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:19.270854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:19.272307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:19.322907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:19.396136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721985ae-7763-4091-ab7c-d35ad4bf9c39 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:19.405825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2ab34-be1a-40fb-8b52-721820962f8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:19.406531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2ab34-be1a-40fb-8b52-721820962f8c service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:19.407153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2ab34-be1a-40fb-8b52-721820962f8c service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:19.407492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2ab34-be1a-40fb-8b52-721820962f8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:19.435135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2ab34-be1a-40fb-8b52-721820962f8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:19.437113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2ab34-be1a-40fb-8b52-721820962f8c service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:19.462350 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2634/5214] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:20.068234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ed744-6704-41f7-802d-8b4b459d8228 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:20.071618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e73ed744-6704-41f7-802d-8b4b459d8228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:20.093854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ed744-6704-41f7-802d-8b4b459d8228 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:20.103513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:20.105328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:20.106313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:20.106670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] PATH_INFO: `/v3/projects/40d1d17de7374bfea67096a7075b9933` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:20.108431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d1d17de7374bfea67096a7075b9933)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:20.111680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:20.255321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d1d17de7374bfea67096a7075b9933', 'request_id': 'req-13d020b7-5847-4af7-bf17-645bf8380beb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:20.265466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d1d17de7374bfea67096a7075b9933', 'request_id': 'req-13d020b7-5847-4af7-bf17-645bf8380beb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:20.281849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d1d17de7374bfea67096a7075b9933', 'request_id': 'req-13d020b7-5847-4af7-bf17-645bf8380beb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:20.297288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] The token cache is being invalidate because project 40d1d17de7374bfea67096a7075b9933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:20.297852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:20.298716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:20.310277 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2581/5215] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:20 2022] DELETE /identity/v3/projects/40d1d17de7374bfea67096a7075b9933 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:20.312749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2ab34-be1a-40fb-8b52-721820962f8c service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:20.334583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2ab34-be1a-40fb-8b52-721820962f8c service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:20.436057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:20.436884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:20.437318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:20.437669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] PATH_INFO: `/v3/users/ad51907e4d4c492b99ba2ce8c8e4c170` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:20.439461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad51907e4d4c492b99ba2ce8c8e4c170)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:20.444601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:20.558656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad51907e4d4c492b99ba2ce8c8e4c170'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:20.577477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad51907e4d4c492b99ba2ce8c8e4c170'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:20.596153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad51907e4d4c492b99ba2ce8c8e4c170'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:20.621896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad51907e4d4c492b99ba2ce8c8e4c170'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:20.622690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad51907e4d4c492b99ba2ce8c8e4c170'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:20.630711 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2635/5216] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:20 2022] DELETE /identity/v3/users/ad51907e4d4c492b99ba2ce8c8e4c170 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:21.966064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:21.968541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:22.001531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d020b7-5847-4af7-bf17-645bf8380beb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:22.031878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:22.153561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f18f6-0718-4742-a157-e7712977c71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:22.154341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f18f6-0718-4742-a157-e7712977c71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.154618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f18f6-0718-4742-a157-e7712977c71e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.154917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f18f6-0718-4742-a157-e7712977c71e admin admin] PATH_INFO: `/v3/projects/6aeeb05ef129436c9ead34abec7425af` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.156254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f18f6-0718-4742-a157-e7712977c71e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aeeb05ef129436c9ead34abec7425af)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:22.158571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f18f6-0718-4742-a157-e7712977c71e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:22.171292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:22.172763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:22.211856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441e476-cc7d-432e-9b6c-1be5f4c39b9e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:22.229655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882a0d2-7ed1-41d3-a4dd-46da8722ee8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:22.229655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882a0d2-7ed1-41d3-a4dd-46da8722ee8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.229655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882a0d2-7ed1-41d3-a4dd-46da8722ee8f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.229655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882a0d2-7ed1-41d3-a4dd-46da8722ee8f admin admin] PATH_INFO: `/v3/users/61b87253caae43d6a2c5836d553ebf24` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.240607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882a0d2-7ed1-41d3-a4dd-46da8722ee8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=61b87253caae43d6a2c5836d553ebf24)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:22.247631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882a0d2-7ed1-41d3-a4dd-46da8722ee8f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:22.252865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd7f18f6-0718-4742-a157-e7712977c71e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aeeb05ef129436c9ead34abec7425af', 'request_id': 'req-cd7f18f6-0718-4742-a157-e7712977c71e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.267687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd7f18f6-0718-4742-a157-e7712977c71e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aeeb05ef129436c9ead34abec7425af', 'request_id': 'req-cd7f18f6-0718-4742-a157-e7712977c71e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.284693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd7f18f6-0718-4742-a157-e7712977c71e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aeeb05ef129436c9ead34abec7425af', 'request_id': 'req-cd7f18f6-0718-4742-a157-e7712977c71e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.304347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd7f18f6-0718-4742-a157-e7712977c71e admin admin] The token cache is being invalidate because project 6aeeb05ef129436c9ead34abec7425af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:22.304553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd7f18f6-0718-4742-a157-e7712977c71e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.306218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd7f18f6-0718-4742-a157-e7712977c71e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.312842 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2582/5217] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:21 2022] DELETE /identity/v3/projects/6aeeb05ef129436c9ead34abec7425af => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:22.314918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fa0ad-9c89-4622-9ac7-4846e2d058f2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.316181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fa0ad-9c89-4622-9ac7-4846e2d058f2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.316648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fa0ad-9c89-4622-9ac7-4846e2d058f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.339573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:22.344149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a85fa0ad-9c89-4622-9ac7-4846e2d058f2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:22.346011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a85fa0ad-9c89-4622-9ac7-4846e2d058f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:22.349980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e882a0d2-7ed1-41d3-a4dd-46da8722ee8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61b87253caae43d6a2c5836d553ebf24'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.365375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e882a0d2-7ed1-41d3-a4dd-46da8722ee8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61b87253caae43d6a2c5836d553ebf24'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.379042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e882a0d2-7ed1-41d3-a4dd-46da8722ee8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61b87253caae43d6a2c5836d553ebf24'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.405272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e882a0d2-7ed1-41d3-a4dd-46da8722ee8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61b87253caae43d6a2c5836d553ebf24'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.406054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e882a0d2-7ed1-41d3-a4dd-46da8722ee8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61b87253caae43d6a2c5836d553ebf24'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.415784 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2636/5218] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:22 2022] DELETE /identity/v3/users/61b87253caae43d6a2c5836d553ebf24 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:22.430111 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2583/5219] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:22.438044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5a154-f404-4d96-bd75-254bc8a2feab None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.438324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5a154-f404-4d96-bd75-254bc8a2feab None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.438732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5a154-f404-4d96-bd75-254bc8a2feab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.474007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-27c5a154-f404-4d96-bd75-254bc8a2feab None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:22.475394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-27c5a154-f404-4d96-bd75-254bc8a2feab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:22.550148 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2637/5220] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:22.554043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fa0ad-9c89-4622-9ac7-4846e2d058f2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:22.570009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fa0ad-9c89-4622-9ac7-4846e2d058f2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:22.582895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2c27d-f653-4593-89f3-e8b0b5ea09ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:22.583740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2c27d-f653-4593-89f3-e8b0b5ea09ec admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.584075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2c27d-f653-4593-89f3-e8b0b5ea09ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.584818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2c27d-f653-4593-89f3-e8b0b5ea09ec admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.585851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2c27d-f653-4593-89f3-e8b0b5ea09ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:22.585851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2c27d-f653-4593-89f3-e8b0b5ea09ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:22.587794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2c27d-f653-4593-89f3-e8b0b5ea09ec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:22.601665 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2584/5221] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:22.605776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5a154-f404-4d96-bd75-254bc8a2feab None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:22.619184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5a154-f404-4d96-bd75-254bc8a2feab None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:22.625623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a255a-eeca-410f-9aa0-18c1728fb3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:22.626464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a255a-eeca-410f-9aa0-18c1728fb3b2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.626732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a255a-eeca-410f-9aa0-18c1728fb3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.626991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a255a-eeca-410f-9aa0-18c1728fb3b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.627736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a255a-eeca-410f-9aa0-18c1728fb3b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:22.630307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a255a-eeca-410f-9aa0-18c1728fb3b2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:22.646420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2c27d-f653-4593-89f3-e8b0b5ea09ec admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:22.657590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d3b2c27d-f653-4593-89f3-e8b0b5ea09ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:22.678433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2c27d-f653-4593-89f3-e8b0b5ea09ec admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:22.684758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb338701-883c-43b4-8f40-dd95ea138e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:22.685731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb338701-883c-43b4-8f40-dd95ea138e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.686008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb338701-883c-43b4-8f40-dd95ea138e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.686281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb338701-883c-43b4-8f40-dd95ea138e53 admin admin] PATH_INFO: `/v3/projects/6c9379c9bf394e25ab8a843967a8cd45` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.687474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb338701-883c-43b4-8f40-dd95ea138e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c9379c9bf394e25ab8a843967a8cd45)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:22.689732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb338701-883c-43b4-8f40-dd95ea138e53 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:22.700498 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2638/5222] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:22 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:22.704566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a255a-eeca-410f-9aa0-18c1728fb3b2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:22.728529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:22.749210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb338701-883c-43b4-8f40-dd95ea138e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c9379c9bf394e25ab8a843967a8cd45', 'request_id': 'req-eb338701-883c-43b4-8f40-dd95ea138e53'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.758993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb338701-883c-43b4-8f40-dd95ea138e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c9379c9bf394e25ab8a843967a8cd45', 'request_id': 'req-eb338701-883c-43b4-8f40-dd95ea138e53'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.768857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb338701-883c-43b4-8f40-dd95ea138e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c9379c9bf394e25ab8a843967a8cd45', 'request_id': 'req-eb338701-883c-43b4-8f40-dd95ea138e53'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.781745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb338701-883c-43b4-8f40-dd95ea138e53 admin admin] The token cache is being invalidate because project 6c9379c9bf394e25ab8a843967a8cd45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:22.781974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a255a-eeca-410f-9aa0-18c1728fb3b2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:22.782495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb338701-883c-43b4-8f40-dd95ea138e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.783020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb338701-883c-43b4-8f40-dd95ea138e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:22.785028 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2585/5223] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:22 2022] DELETE /identity/v3/projects/6c9379c9bf394e25ab8a843967a8cd45 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:22.790695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386c2bd-44c3-4014-8640-879b972478df None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.790992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386c2bd-44c3-4014-8640-879b972478df None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.791349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386c2bd-44c3-4014-8640-879b972478df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.813013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e386c2bd-44c3-4014-8640-879b972478df None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:22.814352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e386c2bd-44c3-4014-8640-879b972478df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:22.837122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:22.859598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4482e15-8e8c-47ec-b4f4-a191cbb5fc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:22.860221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4482e15-8e8c-47ec-b4f4-a191cbb5fc3b admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.860543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4482e15-8e8c-47ec-b4f4-a191cbb5fc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.860808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4482e15-8e8c-47ec-b4f4-a191cbb5fc3b admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.861598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4482e15-8e8c-47ec-b4f4-a191cbb5fc3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:22.863734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4482e15-8e8c-47ec-b4f4-a191cbb5fc3b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:22.884129 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2586/5224] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:22.891723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ba051-31a8-4ab9-9ebf-240fa1e89113 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.892025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ba051-31a8-4ab9-9ebf-240fa1e89113 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.892261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ba051-31a8-4ab9-9ebf-240fa1e89113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.893483 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2639/5225] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:22 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:22.894411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4482e15-8e8c-47ec-b4f4-a191cbb5fc3b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:22.896072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d4482e15-8e8c-47ec-b4f4-a191cbb5fc3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:22.911070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4482e15-8e8c-47ec-b4f4-a191cbb5fc3b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:22.914780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-282ba051-31a8-4ab9-9ebf-240fa1e89113 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:22.916532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-282ba051-31a8-4ab9-9ebf-240fa1e89113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:22.916710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb47f9-995b-4809-b97e-84506d80a8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:22.917279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb47f9-995b-4809-b97e-84506d80a8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.917279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb47f9-995b-4809-b97e-84506d80a8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.917535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb47f9-995b-4809-b97e-84506d80a8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.918137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb47f9-995b-4809-b97e-84506d80a8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:22.919795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb47f9-995b-4809-b97e-84506d80a8f0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:22.926435 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2587/5226] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:22.930480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ba051-31a8-4ab9-9ebf-240fa1e89113 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:22.932791 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2640/5227] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:22.937569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb47f9-995b-4809-b97e-84506d80a8f0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:22.946466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ba051-31a8-4ab9-9ebf-240fa1e89113 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:22.948345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb47f9-995b-4809-b97e-84506d80a8f0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:22.951830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d0394-cc1f-46ac-84e0-e0e088111346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:22.952517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d0394-cc1f-46ac-84e0-e0e088111346 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.952788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d0394-cc1f-46ac-84e0-e0e088111346 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.953050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d0394-cc1f-46ac-84e0-e0e088111346 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.953399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d57df-8e2a-457c-9035-5e5956206b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:22.953693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d0394-cc1f-46ac-84e0-e0e088111346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:22.953989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d0394-cc1f-46ac-84e0-e0e088111346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:22.954333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d57df-8e2a-457c-9035-5e5956206b29 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.954662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d57df-8e2a-457c-9035-5e5956206b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.954985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d57df-8e2a-457c-9035-5e5956206b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.955738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d57df-8e2a-457c-9035-5e5956206b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:22.955867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d0394-cc1f-46ac-84e0-e0e088111346 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:22.957737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d57df-8e2a-457c-9035-5e5956206b29 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:22.971732 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2641/5228] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:22.972509 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2588/5229] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:22.973417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d0394-cc1f-46ac-84e0-e0e088111346 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:22.975721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d57df-8e2a-457c-9035-5e5956206b29 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:22.984641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d0394-cc1f-46ac-84e0-e0e088111346 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:22.987324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d57df-8e2a-457c-9035-5e5956206b29 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:22.990418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c58fc-2eaa-4926-a6c3-7963cbc0f1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:22.991026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c58fc-2eaa-4926-a6c3-7963cbc0f1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.991742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c58fc-2eaa-4926-a6c3-7963cbc0f1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.991742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c58fc-2eaa-4926-a6c3-7963cbc0f1d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:22.992156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c58fc-2eaa-4926-a6c3-7963cbc0f1d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:22.992987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fbd96-ec18-4b38-ba3a-a379297b61aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:22.993715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fbd96-ec18-4b38-ba3a-a379297b61aa admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:22.994047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fbd96-ec18-4b38-ba3a-a379297b61aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:22.994179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c58fc-2eaa-4926-a6c3-7963cbc0f1d9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:22.994327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fbd96-ec18-4b38-ba3a-a379297b61aa admin admin] PATH_INFO: `/v3/projects/caaf396164354112aba40343b8c2e9eb/users/1485643e4fb94562b8b516456b4ff2d7/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.011537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003fbd96-ec18-4b38-ba3a-a379297b61aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=caaf396164354112aba40343b8c2e9eb, user_id=1485643e4fb94562b8b516456b4ff2d7, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:23.014427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003fbd96-ec18-4b38-ba3a-a379297b61aa admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:23.026974 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2589/5230] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:22 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:23.031482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c58fc-2eaa-4926-a6c3-7963cbc0f1d9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:23.032513 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2642/5231] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:22 2022] PUT /identity/v3/projects/caaf396164354112aba40343b8c2e9eb/users/1485643e4fb94562b8b516456b4ff2d7/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:23.042849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cce82-90b4-43be-82c6-30ae98817c66 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:23.043127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cce82-90b4-43be-82c6-30ae98817c66 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:23.043533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cce82-90b4-43be-82c6-30ae98817c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.076193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-186cce82-90b4-43be-82c6-30ae98817c66 None None] MFA Rules not processed for user `1485643e4fb94562b8b516456b4ff2d7`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:23.077704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-186cce82-90b4-43be-82c6-30ae98817c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:23.127316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c58fc-2eaa-4926-a6c3-7963cbc0f1d9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:23.137568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23669d4d-7eb3-4e60-95d0-dffeb79c8988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:23.139168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23669d4d-7eb3-4e60-95d0-dffeb79c8988 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:23.139856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23669d4d-7eb3-4e60-95d0-dffeb79c8988 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:23.140203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23669d4d-7eb3-4e60-95d0-dffeb79c8988 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.140985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23669d4d-7eb3-4e60-95d0-dffeb79c8988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:23.143828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23669d4d-7eb3-4e60-95d0-dffeb79c8988 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:23.175911 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2590/5232] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:23 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:23.177169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23669d4d-7eb3-4e60-95d0-dffeb79c8988 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:23.187208 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2643/5233] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:23.191607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186cce82-90b4-43be-82c6-30ae98817c66 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:23.192436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23669d4d-7eb3-4e60-95d0-dffeb79c8988 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:23.198237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c136bf-1a1d-4a5f-a780-7c3cfcee3761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:23.198987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c136bf-1a1d-4a5f-a780-7c3cfcee3761 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:23.199351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c136bf-1a1d-4a5f-a780-7c3cfcee3761 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:23.199711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c136bf-1a1d-4a5f-a780-7c3cfcee3761 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.200609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c136bf-1a1d-4a5f-a780-7c3cfcee3761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:23.202363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c136bf-1a1d-4a5f-a780-7c3cfcee3761 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:23.202666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186cce82-90b4-43be-82c6-30ae98817c66 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:23.204199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:23.208985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c745b5-cd69-44dc-aae5-778e0448b4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:23.209869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c745b5-cd69-44dc-aae5-778e0448b4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:23.210191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c745b5-cd69-44dc-aae5-778e0448b4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:23.210461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c745b5-cd69-44dc-aae5-778e0448b4b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.212137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c745b5-cd69-44dc-aae5-778e0448b4b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:23.213729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c745b5-cd69-44dc-aae5-778e0448b4b3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:23.220441 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2591/5234] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:23.224656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c136bf-1a1d-4a5f-a780-7c3cfcee3761 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:23.244487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c136bf-1a1d-4a5f-a780-7c3cfcee3761 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:23.249743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e41da-8ee4-47f5-8e71-837c2d1825d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:23.250272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e41da-8ee4-47f5-8e71-837c2d1825d0 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:23.250541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e41da-8ee4-47f5-8e71-837c2d1825d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:23.250947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e41da-8ee4-47f5-8e71-837c2d1825d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.251587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e41da-8ee4-47f5-8e71-837c2d1825d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:23.253637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e41da-8ee4-47f5-8e71-837c2d1825d0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:23.255020 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2644/5235] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:23 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:23.256079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c745b5-cd69-44dc-aae5-778e0448b4b3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:23.276567 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2592/5236] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:23.276567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e41da-8ee4-47f5-8e71-837c2d1825d0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:23.281353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-610e41da-8ee4-47f5-8e71-837c2d1825d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:23.338518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:23.360917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c745b5-cd69-44dc-aae5-778e0448b4b3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:23.369537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080811c-0793-4b8e-b029-60871a1fe131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:23.370745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080811c-0793-4b8e-b029-60871a1fe131 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:23.371517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080811c-0793-4b8e-b029-60871a1fe131 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:23.372038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080811c-0793-4b8e-b029-60871a1fe131 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.373126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080811c-0793-4b8e-b029-60871a1fe131 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:23.375824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080811c-0793-4b8e-b029-60871a1fe131 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:23.388232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e41da-8ee4-47f5-8e71-837c2d1825d0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:23.413821 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2645/5237] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:23 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:23.415021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080811c-0793-4b8e-b029-60871a1fe131 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:23.433466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080811c-0793-4b8e-b029-60871a1fe131 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:23.447438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a35ce-021b-409a-b5bc-09940a7892ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:23.448128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a35ce-021b-409a-b5bc-09940a7892ba admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:23.448435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a35ce-021b-409a-b5bc-09940a7892ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:23.448711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a35ce-021b-409a-b5bc-09940a7892ba admin admin] PATH_INFO: `/v3/projects/b2c04bc7811c44958c5792df10e83296/users/178fecfee69843ff84a0f8ed67fce86f/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.467169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a35ce-021b-409a-b5bc-09940a7892ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c04bc7811c44958c5792df10e83296, user_id=178fecfee69843ff84a0f8ed67fce86f, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:23.469943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a35ce-021b-409a-b5bc-09940a7892ba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:23.491384 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2646/5238] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:23 2022] PUT /identity/v3/projects/b2c04bc7811c44958c5792df10e83296/users/178fecfee69843ff84a0f8ed67fce86f/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:23.492793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a35ce-021b-409a-b5bc-09940a7892ba admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:23.533967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a590334-d310-4243-8a26-b64d76142a4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:23.534983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a590334-d310-4243-8a26-b64d76142a4f service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:23.535472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a590334-d310-4243-8a26-b64d76142a4f service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:23.536618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a590334-d310-4243-8a26-b64d76142a4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.540308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a590334-d310-4243-8a26-b64d76142a4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:23.592749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a590334-d310-4243-8a26-b64d76142a4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:23.596094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a590334-d310-4243-8a26-b64d76142a4f service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:23.608583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a35ce-021b-409a-b5bc-09940a7892ba admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:23.613772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3c2fa-2e64-4cc3-a609-69006b9a8a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:23.614462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3c2fa-2e64-4cc3-a609-69006b9a8a5c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:23.614805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3c2fa-2e64-4cc3-a609-69006b9a8a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:23.615138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3c2fa-2e64-4cc3-a609-69006b9a8a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.616022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3c2fa-2e64-4cc3-a609-69006b9a8a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:23.617864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3c2fa-2e64-4cc3-a609-69006b9a8a5c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:23.629339 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2647/5239] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:23.630800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bb647-d452-4315-a9b8-8a3297463268 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:23.631182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bb647-d452-4315-a9b8-8a3297463268 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:23.631678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bb647-d452-4315-a9b8-8a3297463268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.633893 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2593/5240] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:23.639834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a590334-d310-4243-8a26-b64d76142a4f service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:23.672690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a590334-d310-4243-8a26-b64d76142a4f service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:23.686385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717754e1-e7c4-4c4c-b04f-ff3e95596586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:23.687685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717754e1-e7c4-4c4c-b04f-ff3e95596586 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:23.688175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717754e1-e7c4-4c4c-b04f-ff3e95596586 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:23.688743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717754e1-e7c4-4c4c-b04f-ff3e95596586 admin admin] PATH_INFO: `/v3/projects/2bd307a1c78249fbba9d998492def546/users/e1957235951042c191e56467ef7c92f3/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.706769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5e8bb647-d452-4315-a9b8-8a3297463268 None None] MFA Rules not processed for user `178fecfee69843ff84a0f8ed67fce86f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:23.710546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5e8bb647-d452-4315-a9b8-8a3297463268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:23.724033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717754e1-e7c4-4c4c-b04f-ff3e95596586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bd307a1c78249fbba9d998492def546, user_id=e1957235951042c191e56467ef7c92f3, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:23.728162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717754e1-e7c4-4c4c-b04f-ff3e95596586 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:23.756621 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2594/5241] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:23 2022] PUT /identity/v3/projects/2bd307a1c78249fbba9d998492def546/users/e1957235951042c191e56467ef7c92f3/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:23.761803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717754e1-e7c4-4c4c-b04f-ff3e95596586 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:23.824112 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2648/5242] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:23.838187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bb647-d452-4315-a9b8-8a3297463268 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:23.853627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:23.876358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717754e1-e7c4-4c4c-b04f-ff3e95596586 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:23.885644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507b35d-1897-4891-b04d-d229df661c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:23.886560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507b35d-1897-4891-b04d-d229df661c4c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:23.887037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507b35d-1897-4891-b04d-d229df661c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:23.887462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507b35d-1897-4891-b04d-d229df661c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.888534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507b35d-1897-4891-b04d-d229df661c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:23.890719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507b35d-1897-4891-b04d-d229df661c4c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:23.901526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:23.907081 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2595/5243] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:11:23.913878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507b35d-1897-4891-b04d-d229df661c4c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:23.934781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507b35d-1897-4891-b04d-d229df661c4c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:23.941735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bb647-d452-4315-a9b8-8a3297463268 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:23.944730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682620d-d8c0-4e3c-a11a-8ec7576cf250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:23.947357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682620d-d8c0-4e3c-a11a-8ec7576cf250 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:23.948552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682620d-d8c0-4e3c-a11a-8ec7576cf250 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:23.949018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682620d-d8c0-4e3c-a11a-8ec7576cf250 admin admin] PATH_INFO: `/v3/projects/2bd307a1c78249fbba9d998492def546/users/e1957235951042c191e56467ef7c92f3/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:23.951371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682620d-d8c0-4e3c-a11a-8ec7576cf250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bd307a1c78249fbba9d998492def546, user_id=e1957235951042c191e56467ef7c92f3, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:23.954791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682620d-d8c0-4e3c-a11a-8ec7576cf250 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:23.970548 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2596/5244] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:23 2022] PUT /identity/v3/projects/2bd307a1c78249fbba9d998492def546/users/e1957235951042c191e56467ef7c92f3/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:24.044884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817f29a-86d8-458d-9c50-283fbca606fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:24.045828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817f29a-86d8-458d-9c50-283fbca606fc service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:24.046261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817f29a-86d8-458d-9c50-283fbca606fc service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:24.046826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817f29a-86d8-458d-9c50-283fbca606fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:24.089030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af79b50-aeda-4480-a98a-5807d3d3c348 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:24.089537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af79b50-aeda-4480-a98a-5807d3d3c348 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:24.089537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af79b50-aeda-4480-a98a-5807d3d3c348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:24.122712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1af79b50-aeda-4480-a98a-5807d3d3c348 None None] MFA Rules not processed for user `e1957235951042c191e56467ef7c92f3`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:24.125632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1af79b50-aeda-4480-a98a-5807d3d3c348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:24.148982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817f29a-86d8-458d-9c50-283fbca606fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:24.151170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817f29a-86d8-458d-9c50-283fbca606fc service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:24.164616 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2649/5245] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:24.252253 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2597/5246] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:24.550415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817f29a-86d8-458d-9c50-283fbca606fc service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:24.552152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2817f29a-86d8-458d-9c50-283fbca606fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:24.642147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817f29a-86d8-458d-9c50-283fbca606fc service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:24.707180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:24.728524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626251c3-1083-49bb-9b26-5deb72e6dafa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:24.729343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626251c3-1083-49bb-9b26-5deb72e6dafa service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:24.729804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626251c3-1083-49bb-9b26-5deb72e6dafa service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:24.730074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626251c3-1083-49bb-9b26-5deb72e6dafa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:24.741735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626251c3-1083-49bb-9b26-5deb72e6dafa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:24.743695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626251c3-1083-49bb-9b26-5deb72e6dafa service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:24.758925 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2650/5247] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:25.041831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af79b50-aeda-4480-a98a-5807d3d3c348 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:25.050643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af79b50-aeda-4480-a98a-5807d3d3c348 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:25.055119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d073219-d6d3-4d22-b2be-92e2cecfbb7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:25.055727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d073219-d6d3-4d22-b2be-92e2cecfbb7a service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:25.055997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d073219-d6d3-4d22-b2be-92e2cecfbb7a service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:25.056261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d073219-d6d3-4d22-b2be-92e2cecfbb7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:25.130936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d073219-d6d3-4d22-b2be-92e2cecfbb7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:25.133008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d073219-d6d3-4d22-b2be-92e2cecfbb7a service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:25.145137 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2598/5248] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:25.523248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626251c3-1083-49bb-9b26-5deb72e6dafa service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:25.602083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626251c3-1083-49bb-9b26-5deb72e6dafa service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:25.608866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:25.609659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:25.610027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:25.610511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] PATH_INFO: `/v3/users/1485643e4fb94562b8b516456b4ff2d7` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:25.612412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1485643e4fb94562b8b516456b4ff2d7)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:25.614393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:25.672340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1485643e4fb94562b8b516456b4ff2d7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:25.683028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1485643e4fb94562b8b516456b4ff2d7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:25.693557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1485643e4fb94562b8b516456b4ff2d7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:25.710293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1485643e4fb94562b8b516456b4ff2d7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:25.711469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1485643e4fb94562b8b516456b4ff2d7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:25.722909 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2651/5249] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:25 2022] DELETE /identity/v3/users/1485643e4fb94562b8b516456b4ff2d7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:25.727348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883af94d-3b09-48dd-9f1b-05f30cbbd5f0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:25.727628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883af94d-3b09-48dd-9f1b-05f30cbbd5f0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:25.728340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883af94d-3b09-48dd-9f1b-05f30cbbd5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:25.752101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-883af94d-3b09-48dd-9f1b-05f30cbbd5f0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:25.753536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-883af94d-3b09-48dd-9f1b-05f30cbbd5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:25.816283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:25.834764 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2599/5250] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:25.846111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:25.847780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:25.927611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aefe0-d01f-4570-84bc-0a6986e660bd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:25.931675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867c615-0c51-4659-88f0-c9af3da0802f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:25.933068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867c615-0c51-4659-88f0-c9af3da0802f service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:25.933068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867c615-0c51-4659-88f0-c9af3da0802f service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:25.933068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867c615-0c51-4659-88f0-c9af3da0802f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:25.944006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867c615-0c51-4659-88f0-c9af3da0802f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:25.945952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867c615-0c51-4659-88f0-c9af3da0802f service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:25.951670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:25.962255 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2652/5251] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:26.367583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883af94d-3b09-48dd-9f1b-05f30cbbd5f0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:26.369234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-883af94d-3b09-48dd-9f1b-05f30cbbd5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:26.392237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883af94d-3b09-48dd-9f1b-05f30cbbd5f0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:26.401714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:26.402660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:26.403103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:26.403689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] PATH_INFO: `/v3/projects/caaf396164354112aba40343b8c2e9eb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:26.405686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] RBAC: Authorizing `identity:delete_project(project_id=caaf396164354112aba40343b8c2e9eb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:26.408438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:26.505779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caaf396164354112aba40343b8c2e9eb', 'request_id': 'req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.507184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867c615-0c51-4659-88f0-c9af3da0802f service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:26.508653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4867c615-0c51-4659-88f0-c9af3da0802f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:26.513492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caaf396164354112aba40343b8c2e9eb', 'request_id': 'req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.535459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867c615-0c51-4659-88f0-c9af3da0802f service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:26.539236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caaf396164354112aba40343b8c2e9eb', 'request_id': 'req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.556329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] The token cache is being invalidate because project caaf396164354112aba40343b8c2e9eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:26.557015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.557804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.559977 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2600/5252] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:26 2022] DELETE /identity/v3/projects/caaf396164354112aba40343b8c2e9eb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:26.565425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:26.567040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:26.644746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86c6e4-6be4-4064-b2fa-d4f5c7acb66e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:26.645294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b22aab-05dd-49db-876e-541427aa5518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:26.645943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b22aab-05dd-49db-876e-541427aa5518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:26.646438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b22aab-05dd-49db-876e-541427aa5518 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:26.646705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b22aab-05dd-49db-876e-541427aa5518 admin admin] PATH_INFO: `/v3/users/178fecfee69843ff84a0f8ed67fce86f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:26.648049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b22aab-05dd-49db-876e-541427aa5518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=178fecfee69843ff84a0f8ed67fce86f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:26.649772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:26.650025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b22aab-05dd-49db-876e-541427aa5518 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:26.650532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:26.650866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:26.651213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] PATH_INFO: `/v3/users/e1957235951042c191e56467ef7c92f3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:26.652454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1957235951042c191e56467ef7c92f3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:26.654430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:26.723704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7b22aab-05dd-49db-876e-541427aa5518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '178fecfee69843ff84a0f8ed67fce86f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.732687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1957235951042c191e56467ef7c92f3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.737627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7b22aab-05dd-49db-876e-541427aa5518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '178fecfee69843ff84a0f8ed67fce86f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.745707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1957235951042c191e56467ef7c92f3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.747575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7b22aab-05dd-49db-876e-541427aa5518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '178fecfee69843ff84a0f8ed67fce86f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.768751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1957235951042c191e56467ef7c92f3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.769760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1957235951042c191e56467ef7c92f3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.770519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1957235951042c191e56467ef7c92f3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.774024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7b22aab-05dd-49db-876e-541427aa5518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '178fecfee69843ff84a0f8ed67fce86f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.774828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7b22aab-05dd-49db-876e-541427aa5518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '178fecfee69843ff84a0f8ed67fce86f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:26.777943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:26.784671 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2653/5253] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:26 2022] DELETE /identity/v3/users/178fecfee69843ff84a0f8ed67fce86f => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:26.790057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376fd0b-5bf6-4961-950d-8fa3232d5327 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:26.790462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376fd0b-5bf6-4961-950d-8fa3232d5327 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:26.792425 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2601/5254] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:26 2022] DELETE /identity/v3/users/e1957235951042c191e56467ef7c92f3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:26.792591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376fd0b-5bf6-4961-950d-8fa3232d5327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:26.834347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0376fd0b-5bf6-4961-950d-8fa3232d5327 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:26.835594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0376fd0b-5bf6-4961-950d-8fa3232d5327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:26.933564 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2654/5255] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:26.968147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:26.969654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:27.115532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f573a-4fb8-4c79-b0ea-b574b057c7fb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:27.119537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c6175-597b-43c0-8ca7-490beb0558c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:27.120164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c6175-597b-43c0-8ca7-490beb0558c1 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:27.120643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c6175-597b-43c0-8ca7-490beb0558c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:27.120890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c6175-597b-43c0-8ca7-490beb0558c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:27.131723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c6175-597b-43c0-8ca7-490beb0558c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:27.133237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c6175-597b-43c0-8ca7-490beb0558c1 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:27.145013 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2602/5256] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:27.552851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376fd0b-5bf6-4961-950d-8fa3232d5327 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:27.555230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0376fd0b-5bf6-4961-950d-8fa3232d5327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:27.570000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376fd0b-5bf6-4961-950d-8fa3232d5327 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:27.578436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:27.579396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:27.579882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:27.580204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] PATH_INFO: `/v3/projects/2bd307a1c78249fbba9d998492def546` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:27.581773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bd307a1c78249fbba9d998492def546)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:27.585266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:27.606253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:27.654632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bd307a1c78249fbba9d998492def546', 'request_id': 'req-b593265d-cf1f-4bef-982f-26359d8083cc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:27.678085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bd307a1c78249fbba9d998492def546', 'request_id': 'req-b593265d-cf1f-4bef-982f-26359d8083cc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:27.684882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bd307a1c78249fbba9d998492def546', 'request_id': 'req-b593265d-cf1f-4bef-982f-26359d8083cc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:27.697941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] The token cache is being invalidate because project 2bd307a1c78249fbba9d998492def546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:27.698510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:27.699373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:27.702163 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2655/5257] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:27 2022] DELETE /identity/v3/projects/2bd307a1c78249fbba9d998492def546 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:27.707568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaeaad-217e-40dc-b6ca-2e30fdf9bb40 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:27.708205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaeaad-217e-40dc-b6ca-2e30fdf9bb40 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:27.708334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaeaad-217e-40dc-b6ca-2e30fdf9bb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:27.734607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4ebaeaad-217e-40dc-b6ca-2e30fdf9bb40 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:27.738442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4ebaeaad-217e-40dc-b6ca-2e30fdf9bb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:27.802316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:27.803835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:27.832426 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2603/5258] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:27.839886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cd54f-1745-42cc-b98d-f381609b2ffa None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:27.840598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cd54f-1745-42cc-b98d-f381609b2ffa None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:27.841669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cd54f-1745-42cc-b98d-f381609b2ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:27.875458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fe5cd54f-1745-42cc-b98d-f381609b2ffa None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:27.876906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fe5cd54f-1745-42cc-b98d-f381609b2ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:27.887970 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2604/5259] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:27.892949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cd54f-1745-42cc-b98d-f381609b2ffa None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:27.906071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cd54f-1745-42cc-b98d-f381609b2ffa None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:27.914570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54120a-0a9d-4bc3-b9ba-c3bd78839141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:27.915345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54120a-0a9d-4bc3-b9ba-c3bd78839141 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:27.915617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54120a-0a9d-4bc3-b9ba-c3bd78839141 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:27.916180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54120a-0a9d-4bc3-b9ba-c3bd78839141 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:27.917049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54120a-0a9d-4bc3-b9ba-c3bd78839141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:27.917515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54120a-0a9d-4bc3-b9ba-c3bd78839141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:27.919397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54120a-0a9d-4bc3-b9ba-c3bd78839141 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:27.932777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593265d-cf1f-4bef-982f-26359d8083cc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:27.936779 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2605/5260] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:27.938655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54120a-0a9d-4bc3-b9ba-c3bd78839141 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:27.941378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:27.942124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:27.942604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:27.942882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5 admin admin] PATH_INFO: `/v3/projects/b2c04bc7811c44958c5792df10e83296` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:27.944278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2c04bc7811c44958c5792df10e83296)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:27.948319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:27.953670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54120a-0a9d-4bc3-b9ba-c3bd78839141 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:27.965054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8909a24-a866-4a8a-8260-e635b3f77f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:27.965885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8909a24-a866-4a8a-8260-e635b3f77f2f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:27.966262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8909a24-a866-4a8a-8260-e635b3f77f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:27.966722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8909a24-a866-4a8a-8260-e635b3f77f2f admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:27.969701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8909a24-a866-4a8a-8260-e635b3f77f2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:27.974967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8909a24-a866-4a8a-8260-e635b3f77f2f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:28.028195 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2606/5261] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:27 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:28.032543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2c04bc7811c44958c5792df10e83296', 'request_id': 'req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:28.048592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2c04bc7811c44958c5792df10e83296', 'request_id': 'req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:28.051930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8909a24-a866-4a8a-8260-e635b3f77f2f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:28.057960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2c04bc7811c44958c5792df10e83296', 'request_id': 'req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:28.082452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5 admin admin] The token cache is being invalidate because project b2c04bc7811c44958c5792df10e83296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:28.085031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:28.087583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2af33d74-95ac-4503-b6cb-dad48e3e9fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:28.090417 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2656/5262] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:27 2022] DELETE /identity/v3/projects/b2c04bc7811c44958c5792df10e83296 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:28.095298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f8b38-7845-4f78-ae8b-9db86f599a58 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.095725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f8b38-7845-4f78-ae8b-9db86f599a58 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.096524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f8b38-7845-4f78-ae8b-9db86f599a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.133968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9d6f8b38-7845-4f78-ae8b-9db86f599a58 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:28.135652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9d6f8b38-7845-4f78-ae8b-9db86f599a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:28.186806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:28.189521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:28.191989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8909a24-a866-4a8a-8260-e635b3f77f2f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:28.207743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f05d4-e098-4e03-912d-93e6d4c7e254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:28.208835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f05d4-e098-4e03-912d-93e6d4c7e254 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.209215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f05d4-e098-4e03-912d-93e6d4c7e254 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.209623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f05d4-e098-4e03-912d-93e6d4c7e254 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.210424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f05d4-e098-4e03-912d-93e6d4c7e254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:28.212997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f05d4-e098-4e03-912d-93e6d4c7e254 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:28.247911 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2607/5263] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:28 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:28.256601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f05d4-e098-4e03-912d-93e6d4c7e254 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:28.259643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ac7f05d4-e098-4e03-912d-93e6d4c7e254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:28.274611 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2657/5264] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:28.284513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747567d-50b3-4186-89ba-e7af341d44f5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.284863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747567d-50b3-4186-89ba-e7af341d44f5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.285059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747567d-50b3-4186-89ba-e7af341d44f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.285475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f05d4-e098-4e03-912d-93e6d4c7e254 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:28.290452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255cab3-74c8-4885-a85e-635dfd5f7567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:28.291223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255cab3-74c8-4885-a85e-635dfd5f7567 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.291504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255cab3-74c8-4885-a85e-635dfd5f7567 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.292086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255cab3-74c8-4885-a85e-635dfd5f7567 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.292788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255cab3-74c8-4885-a85e-635dfd5f7567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:28.294714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255cab3-74c8-4885-a85e-635dfd5f7567 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:28.309456 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2608/5265] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:28.316594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8747567d-50b3-4186-89ba-e7af341d44f5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:28.317273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255cab3-74c8-4885-a85e-635dfd5f7567 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:28.320436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8747567d-50b3-4186-89ba-e7af341d44f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:28.339186 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2658/5266] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:28.342756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747567d-50b3-4186-89ba-e7af341d44f5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:28.345578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255cab3-74c8-4885-a85e-635dfd5f7567 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:28.357619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747567d-50b3-4186-89ba-e7af341d44f5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:28.359839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699fcf0-c4ed-4fde-b671-9c51447110d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:28.361214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699fcf0-c4ed-4fde-b671-9c51447110d6 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.361214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699fcf0-c4ed-4fde-b671-9c51447110d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.361953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699fcf0-c4ed-4fde-b671-9c51447110d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.364463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699fcf0-c4ed-4fde-b671-9c51447110d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:28.366994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699fcf0-c4ed-4fde-b671-9c51447110d6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:28.369281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3d986-cbcc-4eb7-8109-24a3ddc954a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:28.370165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3d986-cbcc-4eb7-8109-24a3ddc954a2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.370894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3d986-cbcc-4eb7-8109-24a3ddc954a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.371776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3d986-cbcc-4eb7-8109-24a3ddc954a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.373627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3d986-cbcc-4eb7-8109-24a3ddc954a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:28.374224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3d986-cbcc-4eb7-8109-24a3ddc954a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:28.377439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3d986-cbcc-4eb7-8109-24a3ddc954a2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:28.388497 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2609/5267] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:28.392225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699fcf0-c4ed-4fde-b671-9c51447110d6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:28.396352 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2659/5268] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:28.402388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3d986-cbcc-4eb7-8109-24a3ddc954a2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:28.412970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699fcf0-c4ed-4fde-b671-9c51447110d6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:28.422675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53a714-1ae5-46d5-9d8e-f6f58b6a7462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:28.423747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53a714-1ae5-46d5-9d8e-f6f58b6a7462 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.427608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3d986-cbcc-4eb7-8109-24a3ddc954a2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:28.430682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53a714-1ae5-46d5-9d8e-f6f58b6a7462 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.433038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53a714-1ae5-46d5-9d8e-f6f58b6a7462 admin admin] PATH_INFO: `/v3/projects/a13ab8c6aec0422ca95024c3d6dd4e8c/users/d9f72c907381424689273cdcb70b6a28/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.436216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0905d3-fb32-44a0-9a2d-8a29ebfdb69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:28.436990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0905d3-fb32-44a0-9a2d-8a29ebfdb69b admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.437345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0905d3-fb32-44a0-9a2d-8a29ebfdb69b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.437703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0905d3-fb32-44a0-9a2d-8a29ebfdb69b admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.438430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0905d3-fb32-44a0-9a2d-8a29ebfdb69b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:28.440982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0905d3-fb32-44a0-9a2d-8a29ebfdb69b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:28.462932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53a714-1ae5-46d5-9d8e-f6f58b6a7462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a13ab8c6aec0422ca95024c3d6dd4e8c, user_id=d9f72c907381424689273cdcb70b6a28, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:28.470232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53a714-1ae5-46d5-9d8e-f6f58b6a7462 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:28.475555 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2660/5269] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:28 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:28.479716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0905d3-fb32-44a0-9a2d-8a29ebfdb69b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:28.494946 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2610/5270] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:28 2022] PUT /identity/v3/projects/a13ab8c6aec0422ca95024c3d6dd4e8c/users/d9f72c907381424689273cdcb70b6a28/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:28.505069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8257e63-0832-4460-9d6c-b95169cac13e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.506896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8257e63-0832-4460-9d6c-b95169cac13e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.507210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8257e63-0832-4460-9d6c-b95169cac13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.590118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c8257e63-0832-4460-9d6c-b95169cac13e None None] MFA Rules not processed for user `d9f72c907381424689273cdcb70b6a28`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:28.592072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c8257e63-0832-4460-9d6c-b95169cac13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:28.642973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0905d3-fb32-44a0-9a2d-8a29ebfdb69b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:28.719572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80572f7e-339d-4f2c-8685-a7b0a5b3f464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:28.720724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80572f7e-339d-4f2c-8685-a7b0a5b3f464 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.721227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80572f7e-339d-4f2c-8685-a7b0a5b3f464 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.721775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80572f7e-339d-4f2c-8685-a7b0a5b3f464 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.722617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80572f7e-339d-4f2c-8685-a7b0a5b3f464 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:28.726075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80572f7e-339d-4f2c-8685-a7b0a5b3f464 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:28.728854 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2611/5271] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:28.742254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8257e63-0832-4460-9d6c-b95169cac13e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:28.746199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c8257e63-0832-4460-9d6c-b95169cac13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:28.757785 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2661/5272] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:28 2022] POST /identity/v3/users => generated 287 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:28.759400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80572f7e-339d-4f2c-8685-a7b0a5b3f464 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:28.771809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80572f7e-339d-4f2c-8685-a7b0a5b3f464 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:28.777276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13398858-3555-47cc-96d4-e06c4950693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:28.778085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13398858-3555-47cc-96d4-e06c4950693f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.778360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13398858-3555-47cc-96d4-e06c4950693f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.778757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13398858-3555-47cc-96d4-e06c4950693f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.779543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13398858-3555-47cc-96d4-e06c4950693f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:28.781231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13398858-3555-47cc-96d4-e06c4950693f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:28.793166 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2662/5273] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:28.797071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13398858-3555-47cc-96d4-e06c4950693f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:28.810164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13398858-3555-47cc-96d4-e06c4950693f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:28.816321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87055d6-8ae6-4703-bc83-779982732bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:28.817333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87055d6-8ae6-4703-bc83-779982732bed admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.817333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87055d6-8ae6-4703-bc83-779982732bed admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.817543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87055d6-8ae6-4703-bc83-779982732bed admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.818138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87055d6-8ae6-4703-bc83-779982732bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:28.820928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87055d6-8ae6-4703-bc83-779982732bed admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:28.838082 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2663/5274] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:28.839005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87055d6-8ae6-4703-bc83-779982732bed admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:28.850699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87055d6-8ae6-4703-bc83-779982732bed admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:28.854535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8257e63-0832-4460-9d6c-b95169cac13e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:28.858931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e607450-9778-4eba-8e9a-d9fdc68692d3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:28.859820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e607450-9778-4eba-8e9a-d9fdc68692d3 service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.860192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e607450-9778-4eba-8e9a-d9fdc68692d3 service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.860591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e607450-9778-4eba-8e9a-d9fdc68692d3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.868871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec7e34-725c-42b6-8c40-00c4bf4ae272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:28.870802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec7e34-725c-42b6-8c40-00c4bf4ae272 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.870802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec7e34-725c-42b6-8c40-00c4bf4ae272 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.871963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e607450-9778-4eba-8e9a-d9fdc68692d3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:28.872150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec7e34-725c-42b6-8c40-00c4bf4ae272 admin admin] PATH_INFO: `/v3/projects/07cee74c4f6a4f27b18462fae3f1ea07/users/adc0f328125e4273b3f0515113344a01/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:28.874010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e607450-9778-4eba-8e9a-d9fdc68692d3 service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:28.875657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:28.890540 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2612/5275] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:28.898547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec7e34-725c-42b6-8c40-00c4bf4ae272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07cee74c4f6a4f27b18462fae3f1ea07, user_id=adc0f328125e4273b3f0515113344a01, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:28.901925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec7e34-725c-42b6-8c40-00c4bf4ae272 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:28.920898 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2664/5276] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:28 2022] PUT /identity/v3/projects/07cee74c4f6a4f27b18462fae3f1ea07/users/adc0f328125e4273b3f0515113344a01/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:28.932626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde27f7-222b-466c-9ba1-aee2359d5ace None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:28.932902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde27f7-222b-466c-9ba1-aee2359d5ace None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:28.933552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde27f7-222b-466c-9ba1-aee2359d5ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:29.006779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7dde27f7-222b-466c-9ba1-aee2359d5ace None None] MFA Rules not processed for user `adc0f328125e4273b3f0515113344a01`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:29.006779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7dde27f7-222b-466c-9ba1-aee2359d5ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:29.019452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:29.102498 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2613/5277] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:29.109180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec7e34-725c-42b6-8c40-00c4bf4ae272 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:29.111105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-56ec7e34-725c-42b6-8c40-00c4bf4ae272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:29.225838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec7e34-725c-42b6-8c40-00c4bf4ae272 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:29.237103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b335d-4241-4041-a37b-60efcde8ed01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:29.237896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b335d-4241-4041-a37b-60efcde8ed01 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:29.238245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b335d-4241-4041-a37b-60efcde8ed01 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:29.238657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b335d-4241-4041-a37b-60efcde8ed01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:29.255795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b335d-4241-4041-a37b-60efcde8ed01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:29.258073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b335d-4241-4041-a37b-60efcde8ed01 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:29.277924 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2665/5278] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:31.139640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde27f7-222b-466c-9ba1-aee2359d5ace None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:31.215561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde27f7-222b-466c-9ba1-aee2359d5ace None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:31.220551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15510b4a-7bbd-4166-a8db-57cb831a3d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:31.221464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15510b4a-7bbd-4166-a8db-57cb831a3d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:31.221846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15510b4a-7bbd-4166-a8db-57cb831a3d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:31.222116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15510b4a-7bbd-4166-a8db-57cb831a3d44 admin admin] PATH_INFO: `/v3/users/d9f72c907381424689273cdcb70b6a28` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:31.223253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15510b4a-7bbd-4166-a8db-57cb831a3d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9f72c907381424689273cdcb70b6a28)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:31.225223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15510b4a-7bbd-4166-a8db-57cb831a3d44 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:31.285746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-15510b4a-7bbd-4166-a8db-57cb831a3d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9f72c907381424689273cdcb70b6a28'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:31.300882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-15510b4a-7bbd-4166-a8db-57cb831a3d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9f72c907381424689273cdcb70b6a28'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:31.316209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-15510b4a-7bbd-4166-a8db-57cb831a3d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9f72c907381424689273cdcb70b6a28'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:31.317143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-15510b4a-7bbd-4166-a8db-57cb831a3d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9f72c907381424689273cdcb70b6a28'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:31.318152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-15510b4a-7bbd-4166-a8db-57cb831a3d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9f72c907381424689273cdcb70b6a28'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:31.338425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b335d-4241-4041-a37b-60efcde8ed01 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:31.350752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b335d-4241-4041-a37b-60efcde8ed01 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:31.352966 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2614/5279] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:31 2022] DELETE /identity/v3/users/d9f72c907381424689273cdcb70b6a28 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:31.357231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a588f4-56de-4f2d-9243-44d79d68ff51 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:31.357675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a588f4-56de-4f2d-9243-44d79d68ff51 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:31.357950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a588f4-56de-4f2d-9243-44d79d68ff51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:31.379127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a1a588f4-56de-4f2d-9243-44d79d68ff51 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:31.380348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a1a588f4-56de-4f2d-9243-44d79d68ff51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:31.422700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616a1b4-5d48-4206-a39b-e1f31abc0648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:31.423567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616a1b4-5d48-4206-a39b-e1f31abc0648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:31.423971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616a1b4-5d48-4206-a39b-e1f31abc0648 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:31.424406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616a1b4-5d48-4206-a39b-e1f31abc0648 admin admin] PATH_INFO: `/v3/users/adc0f328125e4273b3f0515113344a01` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:31.425790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616a1b4-5d48-4206-a39b-e1f31abc0648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adc0f328125e4273b3f0515113344a01)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:31.428299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:31.428867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616a1b4-5d48-4206-a39b-e1f31abc0648 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:31.460912 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2615/5280] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:31.473380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a588f4-56de-4f2d-9243-44d79d68ff51 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:31.475771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a1a588f4-56de-4f2d-9243-44d79d68ff51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:31.486108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8616a1b4-5d48-4206-a39b-e1f31abc0648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adc0f328125e4273b3f0515113344a01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:31.496312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8616a1b4-5d48-4206-a39b-e1f31abc0648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adc0f328125e4273b3f0515113344a01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:31.505291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8616a1b4-5d48-4206-a39b-e1f31abc0648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adc0f328125e4273b3f0515113344a01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:31.513442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:31.528155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8616a1b4-5d48-4206-a39b-e1f31abc0648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adc0f328125e4273b3f0515113344a01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:31.529600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8616a1b4-5d48-4206-a39b-e1f31abc0648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adc0f328125e4273b3f0515113344a01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:31.543304 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2666/5281] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:31 2022] DELETE /identity/v3/users/adc0f328125e4273b3f0515113344a01 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:31.545607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a973f-e2d1-4853-bb68-d438b007aef4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:31.545923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a973f-e2d1-4853-bb68-d438b007aef4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:31.546330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a973f-e2d1-4853-bb68-d438b007aef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:31.551891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a588f4-56de-4f2d-9243-44d79d68ff51 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:31.570827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-176a973f-e2d1-4853-bb68-d438b007aef4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:31.572266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-176a973f-e2d1-4853-bb68-d438b007aef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:31.623744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043a0b5-d374-4294-b666-b619071ea9ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:31.624774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043a0b5-d374-4294-b666-b619071ea9ca service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:31.625499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043a0b5-d374-4294-b666-b619071ea9ca service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:31.626207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043a0b5-d374-4294-b666-b619071ea9ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:31.630323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1043a0b5-d374-4294-b666-b619071ea9ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:31.672502 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2667/5282] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:31.683069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a973f-e2d1-4853-bb68-d438b007aef4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:31.685292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-176a973f-e2d1-4853-bb68-d438b007aef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:31.705955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a973f-e2d1-4853-bb68-d438b007aef4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:31.710755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae239e-4b56-402a-8cf3-b13f9d726077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:31.711611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae239e-4b56-402a-8cf3-b13f9d726077 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:31.712020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae239e-4b56-402a-8cf3-b13f9d726077 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:31.712495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae239e-4b56-402a-8cf3-b13f9d726077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:31.731587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae239e-4b56-402a-8cf3-b13f9d726077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:31.733071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae239e-4b56-402a-8cf3-b13f9d726077 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:31.735071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043a0b5-d374-4294-b666-b619071ea9ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:31.739971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043a0b5-d374-4294-b666-b619071ea9ca service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:31.753479 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2668/5283] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:11:31.755740 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2616/5284] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:32.415914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae239e-4b56-402a-8cf3-b13f9d726077 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:32.417478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-87ae239e-4b56-402a-8cf3-b13f9d726077 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:32.431014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae239e-4b56-402a-8cf3-b13f9d726077 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:32.434741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043a0b5-d374-4294-b666-b619071ea9ca service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:32.437552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1043a0b5-d374-4294-b666-b619071ea9ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:32.447494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631c38c-4309-4adc-a087-08a71d513cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:32.448190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631c38c-4309-4adc-a087-08a71d513cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:32.448702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631c38c-4309-4adc-a087-08a71d513cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:32.448975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631c38c-4309-4adc-a087-08a71d513cf2 admin admin] PATH_INFO: `/v3/projects/a13ab8c6aec0422ca95024c3d6dd4e8c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:32.450121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631c38c-4309-4adc-a087-08a71d513cf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a13ab8c6aec0422ca95024c3d6dd4e8c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:32.452035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631c38c-4309-4adc-a087-08a71d513cf2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:32.464226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043a0b5-d374-4294-b666-b619071ea9ca service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:32.479092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:32.480529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:32.481004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:32.481496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175 admin admin] PATH_INFO: `/v3/projects/07cee74c4f6a4f27b18462fae3f1ea07` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:32.482638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07cee74c4f6a4f27b18462fae3f1ea07)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:32.485411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:32.507446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6631c38c-4309-4adc-a087-08a71d513cf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a13ab8c6aec0422ca95024c3d6dd4e8c', 'request_id': 'req-6631c38c-4309-4adc-a087-08a71d513cf2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:32.518925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6631c38c-4309-4adc-a087-08a71d513cf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a13ab8c6aec0422ca95024c3d6dd4e8c', 'request_id': 'req-6631c38c-4309-4adc-a087-08a71d513cf2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:32.527755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6631c38c-4309-4adc-a087-08a71d513cf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a13ab8c6aec0422ca95024c3d6dd4e8c', 'request_id': 'req-6631c38c-4309-4adc-a087-08a71d513cf2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:32.542356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:32.544004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6631c38c-4309-4adc-a087-08a71d513cf2 admin admin] The token cache is being invalidate because project a13ab8c6aec0422ca95024c3d6dd4e8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:32.544295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6631c38c-4309-4adc-a087-08a71d513cf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:32.545059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6631c38c-4309-4adc-a087-08a71d513cf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:32.546833 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2669/5285] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:32 2022] DELETE /identity/v3/projects/a13ab8c6aec0422ca95024c3d6dd4e8c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:32.551845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07cee74c4f6a4f27b18462fae3f1ea07', 'request_id': 'req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:32.559418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07cee74c4f6a4f27b18462fae3f1ea07', 'request_id': 'req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:32.560420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca540825-0821-45a6-aea7-655fe4a1173c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:32.560420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca540825-0821-45a6-aea7-655fe4a1173c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:32.560420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca540825-0821-45a6-aea7-655fe4a1173c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:32.567935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07cee74c4f6a4f27b18462fae3f1ea07', 'request_id': 'req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:32.582707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175 admin admin] The token cache is being invalidate because project 07cee74c4f6a4f27b18462fae3f1ea07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:32.582948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ca540825-0821-45a6-aea7-655fe4a1173c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:32.583073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:32.583625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-36c976ba-2df8-4ce2-bcc0-98e1bae2a175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:32.584416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ca540825-0821-45a6-aea7-655fe4a1173c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:32.585403 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2617/5286] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:32 2022] DELETE /identity/v3/projects/07cee74c4f6a4f27b18462fae3f1ea07 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:32.591976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f283cd-8ea3-436b-9093-b3696884ce8b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:32.592254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f283cd-8ea3-436b-9093-b3696884ce8b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:32.592614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f283cd-8ea3-436b-9093-b3696884ce8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:32.624401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-49f283cd-8ea3-436b-9093-b3696884ce8b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:32.626988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-49f283cd-8ea3-436b-9093-b3696884ce8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:32.646357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:32.668919 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2670/5287] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:32.679448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92fa80-eecb-4281-929f-4f4a3f1017e4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:32.679735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92fa80-eecb-4281-929f-4f4a3f1017e4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:32.680134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92fa80-eecb-4281-929f-4f4a3f1017e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:32.719250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7f92fa80-eecb-4281-929f-4f4a3f1017e4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:32.720877 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2618/5288] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:32.724706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7f92fa80-eecb-4281-929f-4f4a3f1017e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:32.728352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c931b19-074c-41ce-9d61-89abe925b463 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:32.728884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c931b19-074c-41ce-9d61-89abe925b463 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:32.729076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c931b19-074c-41ce-9d61-89abe925b463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:32.735184 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2671/5289] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:32.739700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92fa80-eecb-4281-929f-4f4a3f1017e4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:32.749164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92fa80-eecb-4281-929f-4f4a3f1017e4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:32.751938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4c931b19-074c-41ce-9d61-89abe925b463 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:32.753481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4c931b19-074c-41ce-9d61-89abe925b463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:32.754518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95bc68-5195-4928-91cd-5ac8fcbb4c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:32.755127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95bc68-5195-4928-91cd-5ac8fcbb4c45 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:32.755396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95bc68-5195-4928-91cd-5ac8fcbb4c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:32.755872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95bc68-5195-4928-91cd-5ac8fcbb4c45 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:32.756503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95bc68-5195-4928-91cd-5ac8fcbb4c45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:32.756798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95bc68-5195-4928-91cd-5ac8fcbb4c45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:32.758386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95bc68-5195-4928-91cd-5ac8fcbb4c45 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:32.763040 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2619/5290] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:32.767227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c931b19-074c-41ce-9d61-89abe925b463 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:32.771912 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2672/5291] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:32.776088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95bc68-5195-4928-91cd-5ac8fcbb4c45 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:32.777679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c931b19-074c-41ce-9d61-89abe925b463 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:32.783119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d6761-61cb-4b52-8ea0-15d26670487d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:32.784425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d6761-61cb-4b52-8ea0-15d26670487d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:32.784715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d6761-61cb-4b52-8ea0-15d26670487d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:32.785064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d6761-61cb-4b52-8ea0-15d26670487d admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:32.785811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d6761-61cb-4b52-8ea0-15d26670487d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:32.786100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d6761-61cb-4b52-8ea0-15d26670487d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:32.787711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d6761-61cb-4b52-8ea0-15d26670487d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:32.788023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95bc68-5195-4928-91cd-5ac8fcbb4c45 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:32.793969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15136bfc-4b87-4f7c-9c7d-61d4fb4e1299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:32.794601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15136bfc-4b87-4f7c-9c7d-61d4fb4e1299 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:32.794872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15136bfc-4b87-4f7c-9c7d-61d4fb4e1299 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:32.795328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15136bfc-4b87-4f7c-9c7d-61d4fb4e1299 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:32.795752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15136bfc-4b87-4f7c-9c7d-61d4fb4e1299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:32.797699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15136bfc-4b87-4f7c-9c7d-61d4fb4e1299 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:32.801882 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2620/5292] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:32.806688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d6761-61cb-4b52-8ea0-15d26670487d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:32.819413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d6761-61cb-4b52-8ea0-15d26670487d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:32.825706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0c7b1-59c4-4aac-8f53-2638bb3b6897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:32.826593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0c7b1-59c4-4aac-8f53-2638bb3b6897 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:32.826930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0c7b1-59c4-4aac-8f53-2638bb3b6897 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:32.827730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0c7b1-59c4-4aac-8f53-2638bb3b6897 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:32.828366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0c7b1-59c4-4aac-8f53-2638bb3b6897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:32.833442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0c7b1-59c4-4aac-8f53-2638bb3b6897 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:32.833589 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2673/5293] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:32 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:32.834597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15136bfc-4b87-4f7c-9c7d-61d4fb4e1299 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:32.873339 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2621/5294] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:32 2022] POST /identity/v3/projects => generated 401 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:32.875221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0c7b1-59c4-4aac-8f53-2638bb3b6897 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:32.913940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15136bfc-4b87-4f7c-9c7d-61d4fb4e1299 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:32.990982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0c7b1-59c4-4aac-8f53-2638bb3b6897 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:32.998015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c786e-4cb9-41ad-b9b0-a17b8afd6a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:32.998778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c786e-4cb9-41ad-b9b0-a17b8afd6a00 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:32.999078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c786e-4cb9-41ad-b9b0-a17b8afd6a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:32.999320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c786e-4cb9-41ad-b9b0-a17b8afd6a00 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.000048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c786e-4cb9-41ad-b9b0-a17b8afd6a00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.002087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101314ae-d013-45f0-b426-79a7bea4733a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:33.002303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c786e-4cb9-41ad-b9b0-a17b8afd6a00 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.003474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101314ae-d013-45f0-b426-79a7bea4733a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.003942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101314ae-d013-45f0-b426-79a7bea4733a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.004452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101314ae-d013-45f0-b426-79a7bea4733a admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.005418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101314ae-d013-45f0-b426-79a7bea4733a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.009554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101314ae-d013-45f0-b426-79a7bea4733a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.032684 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2622/5295] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:32 2022] POST /identity/v3/users => generated 301 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:33.043004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c786e-4cb9-41ad-b9b0-a17b8afd6a00 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:33.052767 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2674/5296] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:32 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:33.057715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101314ae-d013-45f0-b426-79a7bea4733a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:33.062978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c786e-4cb9-41ad-b9b0-a17b8afd6a00 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:33.070690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32fdf5-5dac-424a-8ea9-5ad70ecd981a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:33.073436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32fdf5-5dac-424a-8ea9-5ad70ecd981a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.073820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32fdf5-5dac-424a-8ea9-5ad70ecd981a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.074167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32fdf5-5dac-424a-8ea9-5ad70ecd981a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.074900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32fdf5-5dac-424a-8ea9-5ad70ecd981a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.076846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32fdf5-5dac-424a-8ea9-5ad70ecd981a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.077264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101314ae-d013-45f0-b426-79a7bea4733a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:33.083715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06329e2c-8af7-44d0-a8d2-578ab6568e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:33.084243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06329e2c-8af7-44d0-a8d2-578ab6568e43 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.085080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06329e2c-8af7-44d0-a8d2-578ab6568e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.085362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06329e2c-8af7-44d0-a8d2-578ab6568e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.086022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06329e2c-8af7-44d0-a8d2-578ab6568e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.088925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06329e2c-8af7-44d0-a8d2-578ab6568e43 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.091340 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2623/5297] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:11:33.096683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32fdf5-5dac-424a-8ea9-5ad70ecd981a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:33.113909 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2675/5298] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:11:33.115534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06329e2c-8af7-44d0-a8d2-578ab6568e43 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:33.118178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32fdf5-5dac-424a-8ea9-5ad70ecd981a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:33.125255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21073a8-31fa-4db3-8a7e-aa1c1fd065b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:33.127454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21073a8-31fa-4db3-8a7e-aa1c1fd065b7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.129061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21073a8-31fa-4db3-8a7e-aa1c1fd065b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.129569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21073a8-31fa-4db3-8a7e-aa1c1fd065b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.130434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21073a8-31fa-4db3-8a7e-aa1c1fd065b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.134852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21073a8-31fa-4db3-8a7e-aa1c1fd065b7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.142623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:33.144640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06329e2c-8af7-44d0-a8d2-578ab6568e43 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:33.150209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25da9af-8434-448b-a85d-6ccb6c2eea74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:33.150819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25da9af-8434-448b-a85d-6ccb6c2eea74 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.151639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25da9af-8434-448b-a85d-6ccb6c2eea74 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.152242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25da9af-8434-448b-a85d-6ccb6c2eea74 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.152927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25da9af-8434-448b-a85d-6ccb6c2eea74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.154934 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2624/5299] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:11:33.155054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25da9af-8434-448b-a85d-6ccb6c2eea74 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.163396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21073a8-31fa-4db3-8a7e-aa1c1fd065b7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:33.167907 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2676/5300] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:33.171779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25da9af-8434-448b-a85d-6ccb6c2eea74 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:33.178532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21073a8-31fa-4db3-8a7e-aa1c1fd065b7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:33.184522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91e1fc-3115-46a6-856d-d2131693a3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:33.186658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91e1fc-3115-46a6-856d-d2131693a3f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.186766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25da9af-8434-448b-a85d-6ccb6c2eea74 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:33.187246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91e1fc-3115-46a6-856d-d2131693a3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.187721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91e1fc-3115-46a6-856d-d2131693a3f6 admin admin] PATH_INFO: `/v3/projects/f78b7f0e52c94512bbd9d77e2044dcdb/users/c557dd180991414bbbc6336b54f3206e/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.192015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e2066-80f5-4321-929c-a36e581496fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:33.192924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e2066-80f5-4321-929c-a36e581496fd admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.193343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e2066-80f5-4321-929c-a36e581496fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.193789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e2066-80f5-4321-929c-a36e581496fd admin admin] PATH_INFO: `/v3/projects/f7246aade32b4cbba93ef4e3a2bb8556/users/7a4fc8c4eebd458d994b847245b2b886/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.206724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91e1fc-3115-46a6-856d-d2131693a3f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f78b7f0e52c94512bbd9d77e2044dcdb, user_id=c557dd180991414bbbc6336b54f3206e, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.209548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91e1fc-3115-46a6-856d-d2131693a3f6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.213662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e2066-80f5-4321-929c-a36e581496fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7246aade32b4cbba93ef4e3a2bb8556, user_id=7a4fc8c4eebd458d994b847245b2b886, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.219093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e2066-80f5-4321-929c-a36e581496fd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.234890 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2625/5301] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:33 2022] PUT /identity/v3/projects/f78b7f0e52c94512bbd9d77e2044dcdb/users/c557dd180991414bbbc6336b54f3206e/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:33.249127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08486b39-3e8f-4cc9-a812-6da77cfcb484 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.249415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08486b39-3e8f-4cc9-a812-6da77cfcb484 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.249688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08486b39-3e8f-4cc9-a812-6da77cfcb484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.250711 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2677/5302] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:33 2022] PUT /identity/v3/projects/f7246aade32b4cbba93ef4e3a2bb8556/users/7a4fc8c4eebd458d994b847245b2b886/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:33.260943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9227352-2b07-4f27-9a7c-15308986d95a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.261273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9227352-2b07-4f27-9a7c-15308986d95a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.261489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9227352-2b07-4f27-9a7c-15308986d95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.276626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:33.290520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-08486b39-3e8f-4cc9-a812-6da77cfcb484 None None] MFA Rules not processed for user `c557dd180991414bbbc6336b54f3206e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:33.291812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-08486b39-3e8f-4cc9-a812-6da77cfcb484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:33.297935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f9227352-2b07-4f27-9a7c-15308986d95a None None] MFA Rules not processed for user `7a4fc8c4eebd458d994b847245b2b886`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:33.299456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f9227352-2b07-4f27-9a7c-15308986d95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:33.416419 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2626/5303] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:33 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:33.421041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08486b39-3e8f-4cc9-a812-6da77cfcb484 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:33.425094 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2678/5304] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:33.440019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9227352-2b07-4f27-9a7c-15308986d95a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:33.443097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f9227352-2b07-4f27-9a7c-15308986d95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:33.512657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08486b39-3e8f-4cc9-a812-6da77cfcb484 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:33.513201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9227352-2b07-4f27-9a7c-15308986d95a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:33.517260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20db81c-a07f-4d1e-994c-4e8eb8ee56be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:33.518283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20db81c-a07f-4d1e-994c-4e8eb8ee56be service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.518562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea989d0-dea3-485b-9aa5-7c7f33afb761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:33.518729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20db81c-a07f-4d1e-994c-4e8eb8ee56be service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.519090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20db81c-a07f-4d1e-994c-4e8eb8ee56be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.519603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea989d0-dea3-485b-9aa5-7c7f33afb761 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.520060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea989d0-dea3-485b-9aa5-7c7f33afb761 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.520562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea989d0-dea3-485b-9aa5-7c7f33afb761 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.521322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea989d0-dea3-485b-9aa5-7c7f33afb761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.523568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea989d0-dea3-485b-9aa5-7c7f33afb761 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.530931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20db81c-a07f-4d1e-994c-4e8eb8ee56be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.532548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20db81c-a07f-4d1e-994c-4e8eb8ee56be service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.543427 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2679/5305] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:33.573932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20db81c-a07f-4d1e-994c-4e8eb8ee56be service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:33.575633 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2627/5306] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:33 2022] POST /identity/v3/projects => generated 403 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:33.587193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20db81c-a07f-4d1e-994c-4e8eb8ee56be service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:33.684022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2040c3-793f-48d4-9165-6a40c06b58e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:33.684752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2040c3-793f-48d4-9165-6a40c06b58e3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.685088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2040c3-793f-48d4-9165-6a40c06b58e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.685797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2040c3-793f-48d4-9165-6a40c06b58e3 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.686510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2040c3-793f-48d4-9165-6a40c06b58e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.688791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2040c3-793f-48d4-9165-6a40c06b58e3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.720156 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2680/5307] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:33 2022] POST /identity/v3/users => generated 308 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:33.725514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea989d0-dea3-485b-9aa5-7c7f33afb761 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:33.739452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea989d0-dea3-485b-9aa5-7c7f33afb761 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:33.746905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490843e-d365-4278-862a-b98a242941ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:33.748131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490843e-d365-4278-862a-b98a242941ec admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.748542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490843e-d365-4278-862a-b98a242941ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.748950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490843e-d365-4278-862a-b98a242941ec admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.749689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490843e-d365-4278-862a-b98a242941ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.753138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490843e-d365-4278-862a-b98a242941ec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.767818 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2628/5308] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:11:33.772759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2040c3-793f-48d4-9165-6a40c06b58e3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:33.787329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2040c3-793f-48d4-9165-6a40c06b58e3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:33.798266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa108bca-c52b-4568-b291-de699cb21202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:33.799382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa108bca-c52b-4568-b291-de699cb21202 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.799785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa108bca-c52b-4568-b291-de699cb21202 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.800144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa108bca-c52b-4568-b291-de699cb21202 admin admin] PATH_INFO: `/v3/projects/2942c5e1812a497e847a55f75468baf9/users/eef95b60b2294e96b0e54e95e985ba21/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.820601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa108bca-c52b-4568-b291-de699cb21202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2942c5e1812a497e847a55f75468baf9, user_id=eef95b60b2294e96b0e54e95e985ba21, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.825026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa108bca-c52b-4568-b291-de699cb21202 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.847740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490843e-d365-4278-862a-b98a242941ec admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:33.849594 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2681/5309] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:33 2022] PUT /identity/v3/projects/2942c5e1812a497e847a55f75468baf9/users/eef95b60b2294e96b0e54e95e985ba21/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:33.866541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490843e-d365-4278-862a-b98a242941ec admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:33.914684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:33.954817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f439ee-e518-4ee7-ba6d-aba0af9679f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:33.963968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f439ee-e518-4ee7-ba6d-aba0af9679f8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:33.964291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f439ee-e518-4ee7-ba6d-aba0af9679f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:33.964594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f439ee-e518-4ee7-ba6d-aba0af9679f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:33.966499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f439ee-e518-4ee7-ba6d-aba0af9679f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:33.968212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f439ee-e518-4ee7-ba6d-aba0af9679f8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:33.989127 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2629/5310] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:33.994719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa108bca-c52b-4568-b291-de699cb21202 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:34.006073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:34.013843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa108bca-c52b-4568-b291-de699cb21202 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:34.024733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456f734-7b28-4bac-89c1-d4e00cf62190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:34.025736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456f734-7b28-4bac-89c1-d4e00cf62190 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:34.026189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456f734-7b28-4bac-89c1-d4e00cf62190 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:34.026560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456f734-7b28-4bac-89c1-d4e00cf62190 admin admin] PATH_INFO: `/v3/projects/2942c5e1812a497e847a55f75468baf9/users/eef95b60b2294e96b0e54e95e985ba21/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:34.029132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456f734-7b28-4bac-89c1-d4e00cf62190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2942c5e1812a497e847a55f75468baf9, user_id=eef95b60b2294e96b0e54e95e985ba21, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:34.032884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456f734-7b28-4bac-89c1-d4e00cf62190 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:34.052982 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2682/5311] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:33 2022] PUT /identity/v3/projects/2942c5e1812a497e847a55f75468baf9/users/eef95b60b2294e96b0e54e95e985ba21/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:34.065908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e9aa1-aea8-4d86-b5ed-fd4e2e423d91 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:34.066191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e9aa1-aea8-4d86-b5ed-fd4e2e423d91 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:34.066711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e9aa1-aea8-4d86-b5ed-fd4e2e423d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:34.116091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-943e9aa1-aea8-4d86-b5ed-fd4e2e423d91 None None] MFA Rules not processed for user `eef95b60b2294e96b0e54e95e985ba21`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:34.117572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-943e9aa1-aea8-4d86-b5ed-fd4e2e423d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:34.211239 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2630/5312] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:34 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:34.216191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456f734-7b28-4bac-89c1-d4e00cf62190 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:34.231499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456f734-7b28-4bac-89c1-d4e00cf62190 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:34.236602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69cf40-d29c-4cc7-93e3-6008b63c4728 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef95b60b2294e96b0e54e95e985ba21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2942c5e1812a497e847a55f75468baf9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:34.237379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69cf40-d29c-4cc7-93e3-6008b63c4728 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:34.237701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69cf40-d29c-4cc7-93e3-6008b63c4728 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:34.238035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69cf40-d29c-4cc7-93e3-6008b63c4728 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:34.238776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69cf40-d29c-4cc7-93e3-6008b63c4728 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:34.240528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69cf40-d29c-4cc7-93e3-6008b63c4728 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:34.258779 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2683/5313] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:11:34 2022] POST /identity/v3/roles => generated 240 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:34.262997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e9aa1-aea8-4d86-b5ed-fd4e2e423d91 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:34.273170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e9aa1-aea8-4d86-b5ed-fd4e2e423d91 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:34.277969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c30a7-a788-4dae-9d61-b99dfcfb99e8 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef95b60b2294e96b0e54e95e985ba21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2942c5e1812a497e847a55f75468baf9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:34.278650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c30a7-a788-4dae-9d61-b99dfcfb99e8 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:34.278922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c30a7-a788-4dae-9d61-b99dfcfb99e8 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:34.279193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c30a7-a788-4dae-9d61-b99dfcfb99e8 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] PATH_INFO: `/v3/projects/f78b7f0e52c94512bbd9d77e2044dcdb/users/c557dd180991414bbbc6336b54f3206e/roles/b9654c851295474abf58a98ae0e1b006` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:34.281471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c30a7-a788-4dae-9d61-b99dfcfb99e8 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] RBAC: Authorizing `identity:create_grant(project_id=f78b7f0e52c94512bbd9d77e2044dcdb, user_id=c557dd180991414bbbc6336b54f3206e, role_id=b9654c851295474abf58a98ae0e1b006)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:34.284639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c30a7-a788-4dae-9d61-b99dfcfb99e8 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:34.301931 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2631/5314] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:34 2022] PUT /identity/v3/projects/f78b7f0e52c94512bbd9d77e2044dcdb/users/c557dd180991414bbbc6336b54f3206e/roles/b9654c851295474abf58a98ae0e1b006 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:34.306623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69cf40-d29c-4cc7-93e3-6008b63c4728 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:34.374083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69cf40-d29c-4cc7-93e3-6008b63c4728 tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:34.435367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fda16-9f9c-47fc-95f4-3d4d997eeeb9 tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c557dd180991414bbbc6336b54f3206e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f78b7f0e52c94512bbd9d77e2044dcdb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'tempest-test_role-1971435284'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:34.436100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fda16-9f9c-47fc-95f4-3d4d997eeeb9 tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:34.436279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fda16-9f9c-47fc-95f4-3d4d997eeeb9 tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:34.436588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fda16-9f9c-47fc-95f4-3d4d997eeeb9 tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] PATH_INFO: `/v3/users/c557dd180991414bbbc6336b54f3206e/application_credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:34.437248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fda16-9f9c-47fc-95f4-3d4d997eeeb9 tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] RBAC: Authorizing `identity:create_application_credential(user_id=c557dd180991414bbbc6336b54f3206e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:34.438462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fda16-9f9c-47fc-95f4-3d4d997eeeb9 tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:34.521145 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2684/5315] 10.222.0.23 () {60 vars in 1315 bytes} [Sat Aug 20 21:11:34 2022] POST /identity/v3/users/c557dd180991414bbbc6336b54f3206e/application_credentials => generated 659 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:34.527312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750d8fb-d9bf-412e-ae12-3777a1d5fb3f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:34.527925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750d8fb-d9bf-412e-ae12-3777a1d5fb3f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:34.527925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750d8fb-d9bf-412e-ae12-3777a1d5fb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:34.558500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b750d8fb-d9bf-412e-ae12-3777a1d5fb3f None None] MFA Rules not processed for user `c557dd180991414bbbc6336b54f3206e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:34.559741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b750d8fb-d9bf-412e-ae12-3777a1d5fb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:34.656683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:34.683396 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2632/5316] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:34 2022] POST /identity/v3/auth/tokens => generated 3394 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Aug 20 21:11:34.687824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fda16-9f9c-47fc-95f4-3d4d997eeeb9 tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:34.699641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fda16-9f9c-47fc-95f4-3d4d997eeeb9 tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:34.704434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972d116-10d8-440a-a310-dda1458b554f tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c557dd180991414bbbc6336b54f3206e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f78b7f0e52c94512bbd9d77e2044dcdb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'tempest-test_role-1971435284'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:34.705231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972d116-10d8-440a-a310-dda1458b554f tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:34.706166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972d116-10d8-440a-a310-dda1458b554f tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:34.706441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972d116-10d8-440a-a310-dda1458b554f tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] PATH_INFO: `/v3/users/c557dd180991414bbbc6336b54f3206e/application_credentials/fb0fa3ac6e624d55bc5a817356a9ef8b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:34.707648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972d116-10d8-440a-a310-dda1458b554f tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] RBAC: Authorizing `identity:delete_application_credential(user_id=c557dd180991414bbbc6336b54f3206e, application_credential_id=fb0fa3ac6e624d55bc5a817356a9ef8b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:34.709197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972d116-10d8-440a-a310-dda1458b554f tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:34.748560 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2685/5317] 10.222.0.23 () {58 vars in 1395 bytes} [Sat Aug 20 21:11:34 2022] DELETE /identity/v3/users/c557dd180991414bbbc6336b54f3206e/application_credentials/fb0fa3ac6e624d55bc5a817356a9ef8b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:34.750355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750d8fb-d9bf-412e-ae12-3777a1d5fb3f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:34.824365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750d8fb-d9bf-412e-ae12-3777a1d5fb3f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:34.829106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f344f-b7d3-4783-85a7-013e34dee8ca tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef95b60b2294e96b0e54e95e985ba21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2942c5e1812a497e847a55f75468baf9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:34.829748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f344f-b7d3-4783-85a7-013e34dee8ca tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:34.830263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f344f-b7d3-4783-85a7-013e34dee8ca tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:34.831078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f344f-b7d3-4783-85a7-013e34dee8ca tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] PATH_INFO: `/v3/roles/b9654c851295474abf58a98ae0e1b006` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:34.832579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f344f-b7d3-4783-85a7-013e34dee8ca tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] RBAC: Authorizing `identity:delete_role(role_id=b9654c851295474abf58a98ae0e1b006)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:34.834141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f344f-b7d3-4783-85a7-013e34dee8ca tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:34.878587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6e2f344f-b7d3-4783-85a7-013e34dee8ca tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] Invalidating the token cache because role b9654c851295474abf58a98ae0e1b006 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:34.878885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6e2f344f-b7d3-4783-85a7-013e34dee8ca tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:34.879535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6e2f344f-b7d3-4783-85a7-013e34dee8ca tempest-ApplicationCredentialsV3AdminTest-1340587729 tempest-ApplicationCredentialsV3AdminTest-1340587729-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:34.881412 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2633/5318] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:34 2022] DELETE /identity/v3/roles/b9654c851295474abf58a98ae0e1b006 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:34.886660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972d116-10d8-440a-a310-dda1458b554f tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:34.888392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4972d116-10d8-440a-a310-dda1458b554f tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:34.894097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:34.958812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972d116-10d8-440a-a310-dda1458b554f tempest-ApplicationCredentialsV3AdminTest-317022112 tempest-ApplicationCredentialsV3AdminTest-317022112-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:35.019067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:35.019724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:35.020122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:35.020429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] PATH_INFO: `/v3/users/c557dd180991414bbbc6336b54f3206e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:35.021622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c557dd180991414bbbc6336b54f3206e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:35.023717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:35.083863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c557dd180991414bbbc6336b54f3206e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:35.094674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c557dd180991414bbbc6336b54f3206e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:35.103881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c557dd180991414bbbc6336b54f3206e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:35.135443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c557dd180991414bbbc6336b54f3206e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:35.136801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c557dd180991414bbbc6336b54f3206e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:35.146529 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2686/5319] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:34 2022] DELETE /identity/v3/users/c557dd180991414bbbc6336b54f3206e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:35.151265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b66e5-2137-4548-8670-95af1aabc359 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:35.151542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b66e5-2137-4548-8670-95af1aabc359 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:35.151810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b66e5-2137-4548-8670-95af1aabc359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:35.171510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e44b66e5-2137-4548-8670-95af1aabc359 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:35.172781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e44b66e5-2137-4548-8670-95af1aabc359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:35.257270 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2634/5320] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:35.270315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:35.271898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:35.319166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:35.356257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e9a40-3f97-488f-be85-d94aacc90898 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:35.360017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f650b-59b1-43ba-9220-39fa71d98e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:35.360734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f650b-59b1-43ba-9220-39fa71d98e05 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:35.361008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f650b-59b1-43ba-9220-39fa71d98e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:35.361430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f650b-59b1-43ba-9220-39fa71d98e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:35.370880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f650b-59b1-43ba-9220-39fa71d98e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:35.372344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f650b-59b1-43ba-9220-39fa71d98e05 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:35.383931 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2687/5321] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:11:35.392478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b66e5-2137-4548-8670-95af1aabc359 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:35.394257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e44b66e5-2137-4548-8670-95af1aabc359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:35.407250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b66e5-2137-4548-8670-95af1aabc359 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:35.412470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:35.413119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:35.413648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:35.413838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] PATH_INFO: `/v3/users/7a4fc8c4eebd458d994b847245b2b886` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:35.414990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a4fc8c4eebd458d994b847245b2b886)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:35.416958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:35.463529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:35.470265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a4fc8c4eebd458d994b847245b2b886'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:35.478162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a4fc8c4eebd458d994b847245b2b886'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:35.492205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a4fc8c4eebd458d994b847245b2b886'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:35.493284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a4fc8c4eebd458d994b847245b2b886'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:35.494012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a4fc8c4eebd458d994b847245b2b886'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:35.511243 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2635/5322] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:35 2022] DELETE /identity/v3/users/7a4fc8c4eebd458d994b847245b2b886 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:35.516336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bccf9-a2f5-42da-8737-3dcf99e5adf5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:35.516856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bccf9-a2f5-42da-8737-3dcf99e5adf5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:35.516856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bccf9-a2f5-42da-8737-3dcf99e5adf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:35.539632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-099bccf9-a2f5-42da-8737-3dcf99e5adf5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:35.540902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-099bccf9-a2f5-42da-8737-3dcf99e5adf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:35.607633 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2688/5323] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:35.617758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:35.619175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:35.686275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85291a68-541e-4ebe-8a96-12fc933f7fb0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:35.690895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b3220-8c21-4a9b-95e6-aa6074859309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:35.691511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3220-8c21-4a9b-95e6-aa6074859309 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:35.691783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3220-8c21-4a9b-95e6-aa6074859309 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:35.692099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3220-8c21-4a9b-95e6-aa6074859309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:35.702454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b3220-8c21-4a9b-95e6-aa6074859309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:35.703975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b3220-8c21-4a9b-95e6-aa6074859309 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:35.715490 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2636/5324] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:36.004556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099bccf9-a2f5-42da-8737-3dcf99e5adf5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:36.006829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-099bccf9-a2f5-42da-8737-3dcf99e5adf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:36.019499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099bccf9-a2f5-42da-8737-3dcf99e5adf5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:36.024859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:36.025476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:36.025735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:36.026000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] PATH_INFO: `/v3/projects/f78b7f0e52c94512bbd9d77e2044dcdb` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:36.027158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f78b7f0e52c94512bbd9d77e2044dcdb)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:36.029213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:36.086809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f78b7f0e52c94512bbd9d77e2044dcdb', 'request_id': 'req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.096688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f78b7f0e52c94512bbd9d77e2044dcdb', 'request_id': 'req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.103223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f78b7f0e52c94512bbd9d77e2044dcdb', 'request_id': 'req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.118988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] The token cache is being invalidate because project f78b7f0e52c94512bbd9d77e2044dcdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:36.119377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.120535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.126355 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2689/5325] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:36 2022] DELETE /identity/v3/projects/f78b7f0e52c94512bbd9d77e2044dcdb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:36.128070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b3220-8c21-4a9b-95e6-aa6074859309 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:36.140930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b3220-8c21-4a9b-95e6-aa6074859309 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:36.233581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73272a68-28ed-42c5-baef-68b0395287bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:36.234486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73272a68-28ed-42c5-baef-68b0395287bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:36.234849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73272a68-28ed-42c5-baef-68b0395287bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:36.235406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73272a68-28ed-42c5-baef-68b0395287bb admin admin] PATH_INFO: `/v3/users/eef95b60b2294e96b0e54e95e985ba21` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:36.241601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73272a68-28ed-42c5-baef-68b0395287bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef95b60b2294e96b0e54e95e985ba21)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:36.245012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73272a68-28ed-42c5-baef-68b0395287bb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:36.293661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:36.297819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:36.321892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139affc8-fbd8-4c5a-9f44-d170fc5eeba1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:36.332320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:36.334145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:36.334692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:36.335183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c admin admin] PATH_INFO: `/v3/projects/f7246aade32b4cbba93ef4e3a2bb8556` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:36.336985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7246aade32b4cbba93ef4e3a2bb8556)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:36.340017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:36.341678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73272a68-28ed-42c5-baef-68b0395287bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef95b60b2294e96b0e54e95e985ba21'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.352300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73272a68-28ed-42c5-baef-68b0395287bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef95b60b2294e96b0e54e95e985ba21'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.373956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:36.390359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73272a68-28ed-42c5-baef-68b0395287bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef95b60b2294e96b0e54e95e985ba21'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.391163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73272a68-28ed-42c5-baef-68b0395287bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef95b60b2294e96b0e54e95e985ba21'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.391840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73272a68-28ed-42c5-baef-68b0395287bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef95b60b2294e96b0e54e95e985ba21'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.413339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7246aade32b4cbba93ef4e3a2bb8556', 'request_id': 'req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.417041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:36.419243 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2637/5326] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:36 2022] DELETE /identity/v3/users/eef95b60b2294e96b0e54e95e985ba21 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:36.437836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7246aade32b4cbba93ef4e3a2bb8556', 'request_id': 'req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.449103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7246aade32b4cbba93ef4e3a2bb8556', 'request_id': 'req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.464591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c admin admin] The token cache is being invalidate because project f7246aade32b4cbba93ef4e3a2bb8556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:36.464591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.467695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc1fb222-9e99-4085-bf3e-4ca6373ef12c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.467695 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2690/5327] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:36 2022] DELETE /identity/v3/projects/f7246aade32b4cbba93ef4e3a2bb8556 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:36.474461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d71aef-bbee-4409-b334-eae92007179f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:36.474738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d71aef-bbee-4409-b334-eae92007179f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:36.475164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d71aef-bbee-4409-b334-eae92007179f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:36.502000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c4d71aef-bbee-4409-b334-eae92007179f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:36.504033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c4d71aef-bbee-4409-b334-eae92007179f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:36.591264 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2638/5328] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:36.594546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f682ed-45c8-4e98-8801-eb413a5f3129 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:36.595018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f682ed-45c8-4e98-8801-eb413a5f3129 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:36.595102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f682ed-45c8-4e98-8801-eb413a5f3129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:36.621074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-98f682ed-45c8-4e98-8801-eb413a5f3129 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:36.622290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-98f682ed-45c8-4e98-8801-eb413a5f3129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:36.632313 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2691/5329] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:36.637564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d71aef-bbee-4409-b334-eae92007179f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:36.654901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d71aef-bbee-4409-b334-eae92007179f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:36.660492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c42d9-c847-4f4c-9a63-e0b43524e535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:36.661453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c42d9-c847-4f4c-9a63-e0b43524e535 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:36.662198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c42d9-c847-4f4c-9a63-e0b43524e535 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:36.662681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c42d9-c847-4f4c-9a63-e0b43524e535 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:36.663409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c42d9-c847-4f4c-9a63-e0b43524e535 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:36.663785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c42d9-c847-4f4c-9a63-e0b43524e535 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:36.665610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c42d9-c847-4f4c-9a63-e0b43524e535 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:36.678866 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2639/5330] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:36.683033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f682ed-45c8-4e98-8801-eb413a5f3129 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:36.699113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f682ed-45c8-4e98-8801-eb413a5f3129 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:36.704009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8432a3-b1d4-49d1-a046-588fad5a566a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:36.704768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8432a3-b1d4-49d1-a046-588fad5a566a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:36.705915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8432a3-b1d4-49d1-a046-588fad5a566a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:36.706190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8432a3-b1d4-49d1-a046-588fad5a566a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:36.706853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8432a3-b1d4-49d1-a046-588fad5a566a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:36.708885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8432a3-b1d4-49d1-a046-588fad5a566a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:36.743021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c42d9-c847-4f4c-9a63-e0b43524e535 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:36.745239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4b8c42d9-c847-4f4c-9a63-e0b43524e535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:36.752278 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2692/5331] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:36 2022] POST /identity/v3/projects => generated 389 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:36.759579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8432a3-b1d4-49d1-a046-588fad5a566a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:36.780180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c42d9-c847-4f4c-9a63-e0b43524e535 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:36.839860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8432a3-b1d4-49d1-a046-588fad5a566a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:36.845196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac596f-2bef-464b-8f9e-f7cd3fb82ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:36.846179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac596f-2bef-464b-8f9e-f7cd3fb82ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:36.846533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac596f-2bef-464b-8f9e-f7cd3fb82ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:36.846930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac596f-2bef-464b-8f9e-f7cd3fb82ac7 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:36.847657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac596f-2bef-464b-8f9e-f7cd3fb82ac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:36.850315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac596f-2bef-464b-8f9e-f7cd3fb82ac7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:36.872025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131ed6a-daef-459e-a7b2-da10d5c1c54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:36.872684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131ed6a-daef-459e-a7b2-da10d5c1c54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:36.872957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131ed6a-daef-459e-a7b2-da10d5c1c54f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:36.873228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131ed6a-daef-459e-a7b2-da10d5c1c54f admin admin] PATH_INFO: `/v3/projects/2942c5e1812a497e847a55f75468baf9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:36.874451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131ed6a-daef-459e-a7b2-da10d5c1c54f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2942c5e1812a497e847a55f75468baf9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:36.876425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131ed6a-daef-459e-a7b2-da10d5c1c54f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:36.882523 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2693/5332] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:36 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:36.884945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac596f-2bef-464b-8f9e-f7cd3fb82ac7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:36.897185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac596f-2bef-464b-8f9e-f7cd3fb82ac7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:36.902641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b771b5-fbee-4d18-8264-b54bb4cb9bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:36.903271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b771b5-fbee-4d18-8264-b54bb4cb9bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:36.903540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b771b5-fbee-4d18-8264-b54bb4cb9bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:36.903806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b771b5-fbee-4d18-8264-b54bb4cb9bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:36.904476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b771b5-fbee-4d18-8264-b54bb4cb9bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:36.906109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b771b5-fbee-4d18-8264-b54bb4cb9bb4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:36.918605 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2694/5333] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:11:36.922429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b771b5-fbee-4d18-8264-b54bb4cb9bb4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:36.940360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b771b5-fbee-4d18-8264-b54bb4cb9bb4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:36.949018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9131ed6a-daef-459e-a7b2-da10d5c1c54f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2942c5e1812a497e847a55f75468baf9', 'request_id': 'req-9131ed6a-daef-459e-a7b2-da10d5c1c54f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.956338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9131ed6a-daef-459e-a7b2-da10d5c1c54f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2942c5e1812a497e847a55f75468baf9', 'request_id': 'req-9131ed6a-daef-459e-a7b2-da10d5c1c54f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.967469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9131ed6a-daef-459e-a7b2-da10d5c1c54f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2942c5e1812a497e847a55f75468baf9', 'request_id': 'req-9131ed6a-daef-459e-a7b2-da10d5c1c54f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.975448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:36.983135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9131ed6a-daef-459e-a7b2-da10d5c1c54f admin admin] The token cache is being invalidate because project 2942c5e1812a497e847a55f75468baf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:36.983622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9131ed6a-daef-459e-a7b2-da10d5c1c54f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.984814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9131ed6a-daef-459e-a7b2-da10d5c1c54f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:36.986505 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2640/5334] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:36 2022] DELETE /identity/v3/projects/2942c5e1812a497e847a55f75468baf9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:36.991689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26d94d-c86f-43d1-bbe9-3e26b8b7981c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:36.992212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26d94d-c86f-43d1-bbe9-3e26b8b7981c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:36.992642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26d94d-c86f-43d1-bbe9-3e26b8b7981c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.011542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc608-ec18-4024-b6af-e49ba1cb90f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:37.012198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc608-ec18-4024-b6af-e49ba1cb90f4 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.012507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc608-ec18-4024-b6af-e49ba1cb90f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.012878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc608-ec18-4024-b6af-e49ba1cb90f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.013543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebc608-ec18-4024-b6af-e49ba1cb90f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:37.015598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebc608-ec18-4024-b6af-e49ba1cb90f4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:37.015695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6e26d94d-c86f-43d1-bbe9-3e26b8b7981c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:37.016831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6e26d94d-c86f-43d1-bbe9-3e26b8b7981c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:37.025582 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2641/5335] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:37.031795 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2695/5336] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:37.034434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07eaa7a-0883-4984-b7e3-de4ff441b62a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.034720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07eaa7a-0883-4984-b7e3-de4ff441b62a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.035018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07eaa7a-0883-4984-b7e3-de4ff441b62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.038223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc608-ec18-4024-b6af-e49ba1cb90f4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:37.039987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5cebc608-ec18-4024-b6af-e49ba1cb90f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:37.056894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d07eaa7a-0883-4984-b7e3-de4ff441b62a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:37.058397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d07eaa7a-0883-4984-b7e3-de4ff441b62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:37.061307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc608-ec18-4024-b6af-e49ba1cb90f4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:37.067553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505fdf5-9774-49e4-8adf-167e14f6c009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:37.069302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505fdf5-9774-49e4-8adf-167e14f6c009 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.069581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505fdf5-9774-49e4-8adf-167e14f6c009 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.069853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505fdf5-9774-49e4-8adf-167e14f6c009 admin admin] PATH_INFO: `/v3/projects/c6850a387bdb460abf25f6e63f7e1f0b/users/07a9def94e1f4480a60bc91d904f57d9/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.070710 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2642/5337] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:37.075260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07eaa7a-0883-4984-b7e3-de4ff441b62a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:37.086657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07eaa7a-0883-4984-b7e3-de4ff441b62a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:37.089606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505fdf5-9774-49e4-8adf-167e14f6c009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6850a387bdb460abf25f6e63f7e1f0b, user_id=07a9def94e1f4480a60bc91d904f57d9, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:37.092179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe40f4-a582-4c30-8150-68f7e22c9cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:37.092842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe40f4-a582-4c30-8150-68f7e22c9cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.093114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe40f4-a582-4c30-8150-68f7e22c9cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.093381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe40f4-a582-4c30-8150-68f7e22c9cb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.094030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe40f4-a582-4c30-8150-68f7e22c9cb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:37.094330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe40f4-a582-4c30-8150-68f7e22c9cb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:37.096327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe40f4-a582-4c30-8150-68f7e22c9cb1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:37.102114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505fdf5-9774-49e4-8adf-167e14f6c009 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:37.110159 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2643/5338] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:37.115292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe40f4-a582-4c30-8150-68f7e22c9cb1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:37.117231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=6,keystone:DELETE=3,keystone:SELECT=91 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:37.124290 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2696/5339] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:37 2022] PUT /identity/v3/projects/c6850a387bdb460abf25f6e63f7e1f0b/users/07a9def94e1f4480a60bc91d904f57d9/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:37.139312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25327e05-d945-4ba8-9fa9-f56bc8677964 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.139629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25327e05-d945-4ba8-9fa9-f56bc8677964 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.139899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25327e05-d945-4ba8-9fa9-f56bc8677964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.143253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe40f4-a582-4c30-8150-68f7e22c9cb1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:37.212751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-25327e05-d945-4ba8-9fa9-f56bc8677964 None None] MFA Rules not processed for user `07a9def94e1f4480a60bc91d904f57d9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:37.213978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-25327e05-d945-4ba8-9fa9-f56bc8677964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:37.222473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604efbc6-13e7-42e2-a885-cabe1766314c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:37.223189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604efbc6-13e7-42e2-a885-cabe1766314c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.235781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604efbc6-13e7-42e2-a885-cabe1766314c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.236246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604efbc6-13e7-42e2-a885-cabe1766314c admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.236884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604efbc6-13e7-42e2-a885-cabe1766314c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:37.238827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604efbc6-13e7-42e2-a885-cabe1766314c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:37.283330 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2644/5340] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:37 2022] POST /identity/v3/projects => generated 371 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:37.287759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604efbc6-13e7-42e2-a885-cabe1766314c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:37.292829 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2697/5341] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:37.304020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25327e05-d945-4ba8-9fa9-f56bc8677964 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:37.305563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-25327e05-d945-4ba8-9fa9-f56bc8677964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:37.362417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604efbc6-13e7-42e2-a885-cabe1766314c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:37.369400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0f31b-9f0c-4dff-a7dc-a755ec3bc7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:37.370017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0f31b-9f0c-4dff-a7dc-a755ec3bc7ef admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.370338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0f31b-9f0c-4dff-a7dc-a755ec3bc7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.371975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0f31b-9f0c-4dff-a7dc-a755ec3bc7ef admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.371975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0f31b-9f0c-4dff-a7dc-a755ec3bc7ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:37.373186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0f31b-9f0c-4dff-a7dc-a755ec3bc7ef admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:37.374752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25327e05-d945-4ba8-9fa9-f56bc8677964 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:37.400507 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2645/5342] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:37 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:37.401623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0f31b-9f0c-4dff-a7dc-a755ec3bc7ef admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:37.412922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0f31b-9f0c-4dff-a7dc-a755ec3bc7ef admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:37.419341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5961fce-ba27-4f5f-aea8-a396e7ada4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:37.420142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5961fce-ba27-4f5f-aea8-a396e7ada4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.420780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5961fce-ba27-4f5f-aea8-a396e7ada4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.420780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5961fce-ba27-4f5f-aea8-a396e7ada4b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.421626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5961fce-ba27-4f5f-aea8-a396e7ada4b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:37.423946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5961fce-ba27-4f5f-aea8-a396e7ada4b6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:37.435566 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2646/5343] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:37.439997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5961fce-ba27-4f5f-aea8-a396e7ada4b6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:37.441585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ab542-2559-4761-9e0f-7ac948d653ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:37.442543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ab542-2559-4761-9e0f-7ac948d653ae service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.442948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ab542-2559-4761-9e0f-7ac948d653ae service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.443392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ab542-2559-4761-9e0f-7ac948d653ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.450598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5961fce-ba27-4f5f-aea8-a396e7ada4b6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:37.457570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3c22c-44cf-4643-bda3-72550b89131e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:37.458171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3c22c-44cf-4643-bda3-72550b89131e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.458478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3c22c-44cf-4643-bda3-72550b89131e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.458702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3c22c-44cf-4643-bda3-72550b89131e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.459437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3c22c-44cf-4643-bda3-72550b89131e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:37.460978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3c22c-44cf-4643-bda3-72550b89131e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:37.477701 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2647/5344] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:11:37.481820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3c22c-44cf-4643-bda3-72550b89131e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:37.492670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3c22c-44cf-4643-bda3-72550b89131e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:37.497545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef666e-ac88-4388-b422-182cf72bb1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:37.498327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef666e-ac88-4388-b422-182cf72bb1e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.498734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef666e-ac88-4388-b422-182cf72bb1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.499014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef666e-ac88-4388-b422-182cf72bb1e4 admin admin] PATH_INFO: `/v3/projects/424b08483b7c49f5b9b60ebb3a4abe96/users/73d0aad0c8714b2eb599f4bbf81856c4/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.509207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ab542-2559-4761-9e0f-7ac948d653ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:37.510653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ab542-2559-4761-9e0f-7ac948d653ae service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:37.515432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:37.517600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef666e-ac88-4388-b422-182cf72bb1e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=424b08483b7c49f5b9b60ebb3a4abe96, user_id=73d0aad0c8714b2eb599f4bbf81856c4, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:37.521064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef666e-ac88-4388-b422-182cf72bb1e4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:37.521191 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2698/5345] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:11:37.531183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:37.538252 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2648/5346] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:37 2022] PUT /identity/v3/projects/424b08483b7c49f5b9b60ebb3a4abe96/users/73d0aad0c8714b2eb599f4bbf81856c4/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:37.548681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e12b7b-d46a-41bc-a412-7c5befd55c51 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.548996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e12b7b-d46a-41bc-a412-7c5befd55c51 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.549369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e12b7b-d46a-41bc-a412-7c5befd55c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.582448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-87e12b7b-d46a-41bc-a412-7c5befd55c51 None None] MFA Rules not processed for user `73d0aad0c8714b2eb599f4bbf81856c4`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:37.584341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-87e12b7b-d46a-41bc-a412-7c5befd55c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:37.670599 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2699/5347] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:37.677380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef666e-ac88-4388-b422-182cf72bb1e4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:37.762601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef666e-ac88-4388-b422-182cf72bb1e4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:37.769055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e278d-5ab3-42d4-a5ea-6abeaa3e414b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:37.769721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e278d-5ab3-42d4-a5ea-6abeaa3e414b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.770038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e278d-5ab3-42d4-a5ea-6abeaa3e414b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.770260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e278d-5ab3-42d4-a5ea-6abeaa3e414b admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.770901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e278d-5ab3-42d4-a5ea-6abeaa3e414b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:37.773061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e278d-5ab3-42d4-a5ea-6abeaa3e414b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:37.810107 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2649/5348] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:37 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:37.813573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e12b7b-d46a-41bc-a412-7c5befd55c51 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:37.847458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e12b7b-d46a-41bc-a412-7c5befd55c51 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:37.968471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be996807-f4a7-45cf-a108-4795fccccad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:37.969547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be996807-f4a7-45cf-a108-4795fccccad5 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:37.969983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be996807-f4a7-45cf-a108-4795fccccad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:37.970464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be996807-f4a7-45cf-a108-4795fccccad5 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:37.971655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be996807-f4a7-45cf-a108-4795fccccad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:37.975306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be996807-f4a7-45cf-a108-4795fccccad5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.014628 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2700/5349] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:37 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:38.023655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e278d-5ab3-42d4-a5ea-6abeaa3e414b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.040483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e278d-5ab3-42d4-a5ea-6abeaa3e414b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.046958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97516c-4c75-4c36-b741-0cf1a660b820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.048221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97516c-4c75-4c36-b741-0cf1a660b820 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.049787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97516c-4c75-4c36-b741-0cf1a660b820 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.050234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97516c-4c75-4c36-b741-0cf1a660b820 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.051454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97516c-4c75-4c36-b741-0cf1a660b820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.054291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97516c-4c75-4c36-b741-0cf1a660b820 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.069477 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2650/5350] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:38.076127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be996807-f4a7-45cf-a108-4795fccccad5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.091040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be996807-f4a7-45cf-a108-4795fccccad5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.096207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0620113-403f-46fa-847d-db44e80fb644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.096889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0620113-403f-46fa-847d-db44e80fb644 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.097190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0620113-403f-46fa-847d-db44e80fb644 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.097565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0620113-403f-46fa-847d-db44e80fb644 admin admin] PATH_INFO: `/v3/projects/9963bd6ccc5a4e098875e8e4d59bb266/users/6854aff251ea4cf19c219b6149340aac/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.113722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0620113-403f-46fa-847d-db44e80fb644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9963bd6ccc5a4e098875e8e4d59bb266, user_id=6854aff251ea4cf19c219b6149340aac, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.116565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0620113-403f-46fa-847d-db44e80fb644 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.132321 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2701/5351] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:38 2022] PUT /identity/v3/projects/9963bd6ccc5a4e098875e8e4d59bb266/users/6854aff251ea4cf19c219b6149340aac/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:38.136492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97516c-4c75-4c36-b741-0cf1a660b820 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.151039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97516c-4c75-4c36-b741-0cf1a660b820 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.207773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689afc8c-24eb-4cba-b282-3cbfcda883e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.208416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689afc8c-24eb-4cba-b282-3cbfcda883e9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.208695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689afc8c-24eb-4cba-b282-3cbfcda883e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.208958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689afc8c-24eb-4cba-b282-3cbfcda883e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.209733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689afc8c-24eb-4cba-b282-3cbfcda883e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.211321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689afc8c-24eb-4cba-b282-3cbfcda883e9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.221660 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2651/5352] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:11:38.226091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0620113-403f-46fa-847d-db44e80fb644 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.240322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0620113-403f-46fa-847d-db44e80fb644 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.245055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950494e2-3fc1-43eb-9305-a77f7f75d226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.245810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950494e2-3fc1-43eb-9305-a77f7f75d226 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.246149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950494e2-3fc1-43eb-9305-a77f7f75d226 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.246481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950494e2-3fc1-43eb-9305-a77f7f75d226 admin admin] PATH_INFO: `/v3/projects/9963bd6ccc5a4e098875e8e4d59bb266/users/6854aff251ea4cf19c219b6149340aac/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.249726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950494e2-3fc1-43eb-9305-a77f7f75d226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9963bd6ccc5a4e098875e8e4d59bb266, user_id=6854aff251ea4cf19c219b6149340aac, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.252569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950494e2-3fc1-43eb-9305-a77f7f75d226 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.279731 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2702/5353] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:38 2022] PUT /identity/v3/projects/9963bd6ccc5a4e098875e8e4d59bb266/users/6854aff251ea4cf19c219b6149340aac/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:38.300483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e85530-d990-4af1-bc63-88a80b6d5c5d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.300483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e85530-d990-4af1-bc63-88a80b6d5c5d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.300483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e85530-d990-4af1-bc63-88a80b6d5c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.331400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c0e85530-d990-4af1-bc63-88a80b6d5c5d None None] MFA Rules not processed for user `6854aff251ea4cf19c219b6149340aac`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:38.332789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c0e85530-d990-4af1-bc63-88a80b6d5c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:38.398678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:38.411492 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2652/5354] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:38.416227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950494e2-3fc1-43eb-9305-a77f7f75d226 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.428279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950494e2-3fc1-43eb-9305-a77f7f75d226 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.433431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04107ac8-0df3-4089-96cf-01574c1bc091 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.434412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04107ac8-0df3-4089-96cf-01574c1bc091 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.434714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04107ac8-0df3-4089-96cf-01574c1bc091 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.434977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04107ac8-0df3-4089-96cf-01574c1bc091 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.435630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04107ac8-0df3-4089-96cf-01574c1bc091 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.436910 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-04107ac8-0df3-4089-96cf-01574c1bc091 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:38.436910 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:11:38.436910 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:38.437811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04107ac8-0df3-4089-96cf-01574c1bc091 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.459932 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2703/5355] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:38 2022] POST /identity/v3/services => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:38.463967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e85530-d990-4af1-bc63-88a80b6d5c5d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.473958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e85530-d990-4af1-bc63-88a80b6d5c5d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.489982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554ff1f-64a4-4cd0-9d90-8b11d71d358e tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.490611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554ff1f-64a4-4cd0-9d90-8b11d71d358e tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.490885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554ff1f-64a4-4cd0-9d90-8b11d71d358e tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.491154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554ff1f-64a4-4cd0-9d90-8b11d71d358e tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.491825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554ff1f-64a4-4cd0-9d90-8b11d71d358e tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.492884 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-8554ff1f-64a4-4cd0-9d90-8b11d71d358e tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:38.492884 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:11:38.492884 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:38.493790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554ff1f-64a4-4cd0-9d90-8b11d71d358e tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.506624 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2653/5356] 10.222.0.23 () {60 vars in 1213 bytes} [Sat Aug 20 21:11:38 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:38.511362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04107ac8-0df3-4089-96cf-01574c1bc091 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.523240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04107ac8-0df3-4089-96cf-01574c1bc091 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.528741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e86bc-cf79-4c8e-a4e3-4a0ac927ea22 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.529366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e86bc-cf79-4c8e-a4e3-4a0ac927ea22 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.529654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e86bc-cf79-4c8e-a4e3-4a0ac927ea22 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.529891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e86bc-cf79-4c8e-a4e3-4a0ac927ea22 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.530614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e86bc-cf79-4c8e-a4e3-4a0ac927ea22 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.532242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e86bc-cf79-4c8e-a4e3-4a0ac927ea22 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.551350 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2704/5357] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:38 2022] POST /identity/v3/services => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:38.555853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554ff1f-64a4-4cd0-9d90-8b11d71d358e tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.566997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554ff1f-64a4-4cd0-9d90-8b11d71d358e tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.584731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ec995-1232-44ed-b6ef-63bdb74c8184 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.585345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ec995-1232-44ed-b6ef-63bdb74c8184 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.585618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ec995-1232-44ed-b6ef-63bdb74c8184 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.585883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ec995-1232-44ed-b6ef-63bdb74c8184 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.586534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ec995-1232-44ed-b6ef-63bdb74c8184 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.588139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ec995-1232-44ed-b6ef-63bdb74c8184 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.598637 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2654/5358] 10.222.0.23 () {60 vars in 1213 bytes} [Sat Aug 20 21:11:38 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 332 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:38.603002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e86bc-cf79-4c8e-a4e3-4a0ac927ea22 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.614821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e86bc-cf79-4c8e-a4e3-4a0ac927ea22 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.620161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ea556-0af2-4732-893d-f3a0ca33b177 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.620861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ea556-0af2-4732-893d-f3a0ca33b177 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.621138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ea556-0af2-4732-893d-f3a0ca33b177 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.621409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ea556-0af2-4732-893d-f3a0ca33b177 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.622031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ea556-0af2-4732-893d-f3a0ca33b177 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.623063 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-c80ea556-0af2-4732-893d-f3a0ca33b177 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoint_groups": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:38.623063 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:11:38.623063 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:38.623964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ea556-0af2-4732-893d-f3a0ca33b177 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.635074 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2705/5359] 10.222.0.23 () {58 vars in 1191 bytes} [Sat Aug 20 21:11:38 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 763 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:38.636920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ec995-1232-44ed-b6ef-63bdb74c8184 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.647785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ec995-1232-44ed-b6ef-63bdb74c8184 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.652620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f59c31-802a-43f3-9d33-5a7b6fa135f5 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.653251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f59c31-802a-43f3-9d33-5a7b6fa135f5 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.653553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f59c31-802a-43f3-9d33-5a7b6fa135f5 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.653792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f59c31-802a-43f3-9d33-5a7b6fa135f5 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/2d52e92ab75442229eadb760b3459a68` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.654530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f59c31-802a-43f3-9d33-5a7b6fa135f5 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=2d52e92ab75442229eadb760b3459a68)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.655525 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-e1f59c31-802a-43f3-9d33-5a7b6fa135f5 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:38.655525 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:11:38.655525 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:38.656427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f59c31-802a-43f3-9d33-5a7b6fa135f5 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.668100 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2655/5360] 10.222.0.23 () {58 vars in 1290 bytes} [Sat Aug 20 21:11:38 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups/2d52e92ab75442229eadb760b3459a68 => generated 332 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:38.669556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ea556-0af2-4732-893d-f3a0ca33b177 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.681496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ea556-0af2-4732-893d-f3a0ca33b177 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.686262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f72250-3e21-479d-9221-753a1d708fea tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.686888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f72250-3e21-479d-9221-753a1d708fea tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.687164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f72250-3e21-479d-9221-753a1d708fea tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.687429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f72250-3e21-479d-9221-753a1d708fea tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/2d52e92ab75442229eadb760b3459a68` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.688040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f72250-3e21-479d-9221-753a1d708fea tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=2d52e92ab75442229eadb760b3459a68)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.689091 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-90f72250-3e21-479d-9221-753a1d708fea tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:38.689091 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:11:38.689091 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:38.689971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f72250-3e21-479d-9221-753a1d708fea tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.691401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:38.698374 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2706/5361] 10.222.0.23 () {58 vars in 1291 bytes} [Sat Aug 20 21:11:38 2022] HEAD /identity/v3/OS-EP-FILTER/endpoint_groups/2d52e92ab75442229eadb760b3459a68 => generated 0 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Aug 20 21:11:38.702429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f59c31-802a-43f3-9d33-5a7b6fa135f5 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.712829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f59c31-802a-43f3-9d33-5a7b6fa135f5 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.717673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a66cb-210c-481e-8c59-a0da02879422 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.718369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a66cb-210c-481e-8c59-a0da02879422 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.718650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a66cb-210c-481e-8c59-a0da02879422 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.718922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a66cb-210c-481e-8c59-a0da02879422 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/2d52e92ab75442229eadb760b3459a68` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.719516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a66cb-210c-481e-8c59-a0da02879422 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=2d52e92ab75442229eadb760b3459a68)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.720783 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-b39a66cb-210c-481e-8c59-a0da02879422 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:38.720783 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:11:38.720783 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:38.721752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a66cb-210c-481e-8c59-a0da02879422 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.732752 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2656/5362] 10.222.0.23 () {58 vars in 1293 bytes} [Sat Aug 20 21:11:38 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/2d52e92ab75442229eadb760b3459a68 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:38.736800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f72250-3e21-479d-9221-753a1d708fea tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.747084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f72250-3e21-479d-9221-753a1d708fea tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.751829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20582b-e79f-4f42-a7c3-f519580a466b tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.752439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20582b-e79f-4f42-a7c3-f519580a466b tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.752718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20582b-e79f-4f42-a7c3-f519580a466b tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.753006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20582b-e79f-4f42-a7c3-f519580a466b tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.753610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20582b-e79f-4f42-a7c3-f519580a466b tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.755210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20582b-e79f-4f42-a7c3-f519580a466b tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.760035 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2707/5363] 10.222.0.23 () {58 vars in 1191 bytes} [Sat Aug 20 21:11:38 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 450 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:38.764144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a66cb-210c-481e-8c59-a0da02879422 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.774589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a66cb-210c-481e-8c59-a0da02879422 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.779378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfca69-595c-4356-a77e-cf92082d4881 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.780008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfca69-595c-4356-a77e-cf92082d4881 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.780284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfca69-595c-4356-a77e-cf92082d4881 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.780586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfca69-595c-4356-a77e-cf92082d4881 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/2d52e92ab75442229eadb760b3459a68` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.781199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfca69-595c-4356-a77e-cf92082d4881 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=2d52e92ab75442229eadb760b3459a68)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.782790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfca69-595c-4356-a77e-cf92082d4881 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-1bcfca69-595c-4356-a77e-cf92082d4881 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Could not find Endpoint Group: 2d52e92ab75442229eadb760b3459a68.: keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: 2d52e92ab75442229eadb760b3459a68. Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_ep_filter.py", line 107, in delete Aug 20 21:11:38.802157 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return (PROVIDERS.catalog_api.delete_endpoint_group(endpoint_group_id), Aug 20 21:11:38.803033 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 519, in delete_endpoint_group Aug 20 21:11:38.803033 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application endpoint_group_ref = self._get_endpoint_group(session, Aug 20 21:11:38.803033 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 495, in _get_endpoint_group Aug 20 21:11:38.803033 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.EndpointGroupNotFound( Aug 20 21:11:38.803033 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: 2d52e92ab75442229eadb760b3459a68. Aug 20 21:11:38.803033 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:38.803329 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2657/5364] 10.222.0.23 () {58 vars in 1293 bytes} [Sat Aug 20 21:11:38 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/2d52e92ab75442229eadb760b3459a68 => generated 120 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:11:38.808988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20582b-e79f-4f42-a7c3-f519580a466b tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.823321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20582b-e79f-4f42-a7c3-f519580a466b tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.828597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cc4fc-5a44-4503-9a1c-91fd7758eba8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.829248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cc4fc-5a44-4503-9a1c-91fd7758eba8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.829526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cc4fc-5a44-4503-9a1c-91fd7758eba8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.829813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cc4fc-5a44-4503-9a1c-91fd7758eba8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/services/c7e6f33500784d6f92f37fd2f90f5fe2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.830763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cc4fc-5a44-4503-9a1c-91fd7758eba8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:delete_service(service_id=c7e6f33500784d6f92f37fd2f90f5fe2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.831907 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-909cc4fc-5a44-4503-9a1c-91fd7758eba8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:38.831907 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:11:38.831907 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:38.832820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cc4fc-5a44-4503-9a1c-91fd7758eba8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.868998 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2708/5365] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:38 2022] DELETE /identity/v3/services/c7e6f33500784d6f92f37fd2f90f5fe2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:38.874255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfca69-595c-4356-a77e-cf92082d4881 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.886517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfca69-595c-4356-a77e-cf92082d4881 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.904352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c72b0b-f427-427f-bc22-c135e8a4c17d tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.904701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c72b0b-f427-427f-bc22-c135e8a4c17d tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.904976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c72b0b-f427-427f-bc22-c135e8a4c17d tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.905239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c72b0b-f427-427f-bc22-c135e8a4c17d tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.905881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c72b0b-f427-427f-bc22-c135e8a4c17d tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.907076 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-97c72b0b-f427-427f-bc22-c135e8a4c17d tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:38.907076 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:11:38.907076 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:38.909061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c72b0b-f427-427f-bc22-c135e8a4c17d tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.925112 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2658/5366] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:38 2022] POST /identity/v3/services => generated 289 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:38.929225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cc4fc-5a44-4503-9a1c-91fd7758eba8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:38.943746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cc4fc-5a44-4503-9a1c-91fd7758eba8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:38.969797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f2f0d-b076-451b-a02b-addd24c9ef4c tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:38.970938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f2f0d-b076-451b-a02b-addd24c9ef4c tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:38.971247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f2f0d-b076-451b-a02b-addd24c9ef4c tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:38.971510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f2f0d-b076-451b-a02b-addd24c9ef4c tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:38.972174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f2f0d-b076-451b-a02b-addd24c9ef4c tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:38.973571 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-f76f2f0d-b076-451b-a02b-addd24c9ef4c tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:38.973571 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:11:38.973571 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:38.974534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f2f0d-b076-451b-a02b-addd24c9ef4c tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:38.996653 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2709/5367] 10.222.0.23 () {60 vars in 1213 bytes} [Sat Aug 20 21:11:38 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 334 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:39.003207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c72b0b-f427-427f-bc22-c135e8a4c17d tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:39.016504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c72b0b-f427-427f-bc22-c135e8a4c17d tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:39.022253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d5349-10d3-48b7-9eee-3d3d635e01b8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:39.022872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d5349-10d3-48b7-9eee-3d3d635e01b8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:39.023148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d5349-10d3-48b7-9eee-3d3d635e01b8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:39.023409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d5349-10d3-48b7-9eee-3d3d635e01b8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:39.024076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d5349-10d3-48b7-9eee-3d3d635e01b8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:39.025718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d5349-10d3-48b7-9eee-3d3d635e01b8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:39.058762 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2659/5368] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:39 2022] POST /identity/v3/services => generated 288 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:39.060128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f2f0d-b076-451b-a02b-addd24c9ef4c tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:39.073831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f2f0d-b076-451b-a02b-addd24c9ef4c tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:39.095367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2b71e-2312-4577-8b8c-9eb7902c0042 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:39.096330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b71e-2312-4577-8b8c-9eb7902c0042 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:39.096648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b71e-2312-4577-8b8c-9eb7902c0042 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:39.096920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b71e-2312-4577-8b8c-9eb7902c0042 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/f394e4efea11458884bff8c7d15dee46` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:39.097589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2b71e-2312-4577-8b8c-9eb7902c0042 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:update_endpoint_group(endpoint_group_id=f394e4efea11458884bff8c7d15dee46)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:39.098609 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-12b2b71e-2312-4577-8b8c-9eb7902c0042 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:39.098609 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:11:39.098609 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:39.099536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2b71e-2312-4577-8b8c-9eb7902c0042 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:39.113301 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2710/5369] 10.222.0.23 () {60 vars in 1313 bytes} [Sat Aug 20 21:11:39 2022] PATCH /identity/v3/OS-EP-FILTER/endpoint_groups/f394e4efea11458884bff8c7d15dee46 => generated 334 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:11:39.118384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d5349-10d3-48b7-9eee-3d3d635e01b8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:39.134930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d5349-10d3-48b7-9eee-3d3d635e01b8 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:39.142341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b8957-a975-475b-b6f9-e2d8c8c051a7 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:39.143229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b8957-a975-475b-b6f9-e2d8c8c051a7 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:39.143507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b8957-a975-475b-b6f9-e2d8c8c051a7 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:39.143772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b8957-a975-475b-b6f9-e2d8c8c051a7 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/services/343179c564294f3692f629e8892f66c3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:39.144363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b8957-a975-475b-b6f9-e2d8c8c051a7 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:delete_service(service_id=343179c564294f3692f629e8892f66c3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:39.146048 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-dd1b8957-a975-475b-b6f9-e2d8c8c051a7 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:39.146048 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:11:39.146048 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:39.146947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b8957-a975-475b-b6f9-e2d8c8c051a7 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:39.179405 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2660/5370] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:39 2022] DELETE /identity/v3/services/343179c564294f3692f629e8892f66c3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:39.184408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2b71e-2312-4577-8b8c-9eb7902c0042 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:39.197740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2b71e-2312-4577-8b8c-9eb7902c0042 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:39.214699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46577dfc-25f0-48a3-822e-10fb641048f0 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:39.215420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46577dfc-25f0-48a3-822e-10fb641048f0 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:39.215775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46577dfc-25f0-48a3-822e-10fb641048f0 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:39.216060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46577dfc-25f0-48a3-822e-10fb641048f0 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/f394e4efea11458884bff8c7d15dee46` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:39.217044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46577dfc-25f0-48a3-822e-10fb641048f0 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=f394e4efea11458884bff8c7d15dee46)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:39.218245 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-46577dfc-25f0-48a3-822e-10fb641048f0 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:39.218245 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:11:39.218245 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:39.219295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46577dfc-25f0-48a3-822e-10fb641048f0 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:39.234724 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2711/5371] 10.222.0.23 () {58 vars in 1293 bytes} [Sat Aug 20 21:11:39 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/f394e4efea11458884bff8c7d15dee46 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:39.239469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b8957-a975-475b-b6f9-e2d8c8c051a7 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:39.250641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b8957-a975-475b-b6f9-e2d8c8c051a7 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:39.257235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b7046-9359-4336-9f3e-e2fb4651e632 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:39.258207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b7046-9359-4336-9f3e-e2fb4651e632 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:39.258207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b7046-9359-4336-9f3e-e2fb4651e632 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:39.258454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b7046-9359-4336-9f3e-e2fb4651e632 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/services/642551b149954a24a020c4fa7856a0c0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:39.259055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b7046-9359-4336-9f3e-e2fb4651e632 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:delete_service(service_id=642551b149954a24a020c4fa7856a0c0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:39.260692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b7046-9359-4336-9f3e-e2fb4651e632 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:39.303455 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2661/5372] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:39 2022] DELETE /identity/v3/services/642551b149954a24a020c4fa7856a0c0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:39.308289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46577dfc-25f0-48a3-822e-10fb641048f0 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:39.321679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46577dfc-25f0-48a3-822e-10fb641048f0 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:39.331404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b7046-9359-4336-9f3e-e2fb4651e632 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:39.346297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920df94b-0950-49a1-babf-e41cf5cb438a tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:39.347148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920df94b-0950-49a1-babf-e41cf5cb438a tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:39.347489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920df94b-0950-49a1-babf-e41cf5cb438a tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:39.348048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920df94b-0950-49a1-babf-e41cf5cb438a tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/9968e1fea38240d4bc3aa3930a33271f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:39.348990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920df94b-0950-49a1-babf-e41cf5cb438a tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=9968e1fea38240d4bc3aa3930a33271f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:39.351160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920df94b-0950-49a1-babf-e41cf5cb438a tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:39.367240 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2712/5373] 10.222.0.23 () {58 vars in 1293 bytes} [Sat Aug 20 21:11:39 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/9968e1fea38240d4bc3aa3930a33271f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:39.371708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920df94b-0950-49a1-babf-e41cf5cb438a tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:39.388635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920df94b-0950-49a1-babf-e41cf5cb438a tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:39.396336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76217589-a9b7-43cb-b0b1-8dce7586c440 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6854aff251ea4cf19c219b6149340aac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9963bd6ccc5a4e098875e8e4d59bb266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:39.397104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76217589-a9b7-43cb-b0b1-8dce7586c440 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:39.398393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76217589-a9b7-43cb-b0b1-8dce7586c440 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:39.398682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76217589-a9b7-43cb-b0b1-8dce7586c440 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] PATH_INFO: `/v3/services/1b79d669e9304323b3c9062f19d0f281` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:39.399284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76217589-a9b7-43cb-b0b1-8dce7586c440 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorizing `identity:delete_service(service_id=1b79d669e9304323b3c9062f19d0f281)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:39.401863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76217589-a9b7-43cb-b0b1-8dce7586c440 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:39.427057 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2713/5374] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:39 2022] DELETE /identity/v3/services/1b79d669e9304323b3c9062f19d0f281 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:39.431897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76217589-a9b7-43cb-b0b1-8dce7586c440 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:39.439452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:39.443050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b7046-9359-4336-9f3e-e2fb4651e632 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:39.465730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d83f65-96b9-4b95-914a-ecec0de8fe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:39.465945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76217589-a9b7-43cb-b0b1-8dce7586c440 tempest-EndPointGroupsTest-1226888938 tempest-EndPointGroupsTest-1226888938-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:39.466037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d83f65-96b9-4b95-914a-ecec0de8fe13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:39.466037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d83f65-96b9-4b95-914a-ecec0de8fe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:39.466202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d83f65-96b9-4b95-914a-ecec0de8fe13 admin admin] PATH_INFO: `/v3/users/07a9def94e1f4480a60bc91d904f57d9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:39.467406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d83f65-96b9-4b95-914a-ecec0de8fe13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07a9def94e1f4480a60bc91d904f57d9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:39.469387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d83f65-96b9-4b95-914a-ecec0de8fe13 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:39.480975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faac2bd-6067-4278-af67-409faa2b940b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:39.481632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faac2bd-6067-4278-af67-409faa2b940b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:39.481939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faac2bd-6067-4278-af67-409faa2b940b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:39.482211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faac2bd-6067-4278-af67-409faa2b940b admin admin] PATH_INFO: `/v3/users/73d0aad0c8714b2eb599f4bbf81856c4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:39.483502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faac2bd-6067-4278-af67-409faa2b940b admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d0aad0c8714b2eb599f4bbf81856c4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:39.486382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faac2bd-6067-4278-af67-409faa2b940b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:39.539490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17d83f65-96b9-4b95-914a-ecec0de8fe13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07a9def94e1f4480a60bc91d904f57d9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:39.558111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17d83f65-96b9-4b95-914a-ecec0de8fe13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07a9def94e1f4480a60bc91d904f57d9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:39.562106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7faac2bd-6067-4278-af67-409faa2b940b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d0aad0c8714b2eb599f4bbf81856c4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:39.574183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7faac2bd-6067-4278-af67-409faa2b940b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d0aad0c8714b2eb599f4bbf81856c4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:39.581387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17d83f65-96b9-4b95-914a-ecec0de8fe13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07a9def94e1f4480a60bc91d904f57d9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:39.582193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17d83f65-96b9-4b95-914a-ecec0de8fe13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07a9def94e1f4480a60bc91d904f57d9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:39.584304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-17d83f65-96b9-4b95-914a-ecec0de8fe13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07a9def94e1f4480a60bc91d904f57d9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:39.586018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7faac2bd-6067-4278-af67-409faa2b940b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d0aad0c8714b2eb599f4bbf81856c4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:39.589080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:39.606807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7faac2bd-6067-4278-af67-409faa2b940b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d0aad0c8714b2eb599f4bbf81856c4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:39.607534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7faac2bd-6067-4278-af67-409faa2b940b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d0aad0c8714b2eb599f4bbf81856c4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:39.612168 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2662/5375] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:39 2022] DELETE /identity/v3/users/07a9def94e1f4480a60bc91d904f57d9 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:39.617342 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2714/5376] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:39 2022] DELETE /identity/v3/users/73d0aad0c8714b2eb599f4bbf81856c4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:39.617510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4eeb2-7daa-49c4-8946-c9558bfed4d5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:39.617510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4eeb2-7daa-49c4-8946-c9558bfed4d5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:39.617510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4eeb2-7daa-49c4-8946-c9558bfed4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:39.619699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e22f6f-c939-4e07-807c-4e58163fa3c2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:39.619979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e22f6f-c939-4e07-807c-4e58163fa3c2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:39.620493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e22f6f-c939-4e07-807c-4e58163fa3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:39.644407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-12f4eeb2-7daa-49c4-8946-c9558bfed4d5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:39.645868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-12f4eeb2-7daa-49c4-8946-c9558bfed4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:39.655476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-20e22f6f-c939-4e07-807c-4e58163fa3c2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:39.657479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-20e22f6f-c939-4e07-807c-4e58163fa3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:39.725020 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2663/5377] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:39.732101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4eeb2-7daa-49c4-8946-c9558bfed4d5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:39.733592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-12f4eeb2-7daa-49c4-8946-c9558bfed4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:39.760363 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2715/5378] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:39.771637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e22f6f-c939-4e07-807c-4e58163fa3c2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:39.774129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-20e22f6f-c939-4e07-807c-4e58163fa3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:39.821499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4eeb2-7daa-49c4-8946-c9558bfed4d5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:39.825595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7b43a-f854-4047-9b7e-286c8af9337f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:39.826203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7b43a-f854-4047-9b7e-286c8af9337f service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:39.826476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7b43a-f854-4047-9b7e-286c8af9337f service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:39.826743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7b43a-f854-4047-9b7e-286c8af9337f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:39.836729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7b43a-f854-4047-9b7e-286c8af9337f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:39.838179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7b43a-f854-4047-9b7e-286c8af9337f service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:39.851650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e22f6f-c939-4e07-807c-4e58163fa3c2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:39.853337 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2664/5379] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:11:39.856838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bc966-2074-46e0-8dde-2e18a22daa36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:39.857696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bc966-2074-46e0-8dde-2e18a22daa36 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:39.857944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bc966-2074-46e0-8dde-2e18a22daa36 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:39.858222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bc966-2074-46e0-8dde-2e18a22daa36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:39.869159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bc966-2074-46e0-8dde-2e18a22daa36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:39.870662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bc966-2074-46e0-8dde-2e18a22daa36 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:39.882787 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2716/5380] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:11:40.311378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7b43a-f854-4047-9b7e-286c8af9337f service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:40.314014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8ef7b43a-f854-4047-9b7e-286c8af9337f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:40.328614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:40.337740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7b43a-f854-4047-9b7e-286c8af9337f service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:40.345624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b614b7-428b-4501-8fed-38893a09dce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:40.346230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b614b7-428b-4501-8fed-38893a09dce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:40.346703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b614b7-428b-4501-8fed-38893a09dce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:40.347301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b614b7-428b-4501-8fed-38893a09dce7 admin admin] PATH_INFO: `/v3/projects/c6850a387bdb460abf25f6e63f7e1f0b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:40.348783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b614b7-428b-4501-8fed-38893a09dce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6850a387bdb460abf25f6e63f7e1f0b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:40.351015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b614b7-428b-4501-8fed-38893a09dce7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:40.427961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-40b614b7-428b-4501-8fed-38893a09dce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6850a387bdb460abf25f6e63f7e1f0b', 'request_id': 'req-40b614b7-428b-4501-8fed-38893a09dce7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.444992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-40b614b7-428b-4501-8fed-38893a09dce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6850a387bdb460abf25f6e63f7e1f0b', 'request_id': 'req-40b614b7-428b-4501-8fed-38893a09dce7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.446098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bc966-2074-46e0-8dde-2e18a22daa36 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:40.448637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b70bc966-2074-46e0-8dde-2e18a22daa36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:40.462255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-40b614b7-428b-4501-8fed-38893a09dce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6850a387bdb460abf25f6e63f7e1f0b', 'request_id': 'req-40b614b7-428b-4501-8fed-38893a09dce7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.464607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bc966-2074-46e0-8dde-2e18a22daa36 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:40.475436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-40b614b7-428b-4501-8fed-38893a09dce7 admin admin] The token cache is being invalidate because project c6850a387bdb460abf25f6e63f7e1f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:40.475723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-40b614b7-428b-4501-8fed-38893a09dce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.476403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-40b614b7-428b-4501-8fed-38893a09dce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.478512 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2665/5381] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:40 2022] DELETE /identity/v3/projects/c6850a387bdb460abf25f6e63f7e1f0b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:40.490174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d2067-db65-480e-b30e-219e144d0b3a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:40.491922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d2067-db65-480e-b30e-219e144d0b3a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:40.492199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d2067-db65-480e-b30e-219e144d0b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:40.512847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8a9d2067-db65-480e-b30e-219e144d0b3a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:40.514059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a9d2067-db65-480e-b30e-219e144d0b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:40.546773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:40.547478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:40.547861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:40.548125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] PATH_INFO: `/v3/projects/424b08483b7c49f5b9b60ebb3a4abe96` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:40.549475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=424b08483b7c49f5b9b60ebb3a4abe96)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:40.551412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:40.586651 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2666/5382] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:40.593147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:40.594684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22e19d-37db-4380-96ce-b4f1324b04a8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:40.594993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22e19d-37db-4380-96ce-b4f1324b04a8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:40.595225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22e19d-37db-4380-96ce-b4f1324b04a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:40.611686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '424b08483b7c49f5b9b60ebb3a4abe96', 'request_id': 'req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.615518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7d22e19d-37db-4380-96ce-b4f1324b04a8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:40.616845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7d22e19d-37db-4380-96ce-b4f1324b04a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:40.628936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '424b08483b7c49f5b9b60ebb3a4abe96', 'request_id': 'req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.635549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '424b08483b7c49f5b9b60ebb3a4abe96', 'request_id': 'req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.647505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] The token cache is being invalidate because project 424b08483b7c49f5b9b60ebb3a4abe96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:40.647774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.648446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.650253 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2717/5383] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:40 2022] DELETE /identity/v3/projects/424b08483b7c49f5b9b60ebb3a4abe96 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:40.657220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:40.658865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:40.684440 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2667/5384] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:40.689274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22e19d-37db-4380-96ce-b4f1324b04a8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:40.700540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22e19d-37db-4380-96ce-b4f1324b04a8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:40.705059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fdca6-bd7b-4f7d-a7ff-10f81c13a7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:40.705659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fdca6-bd7b-4f7d-a7ff-10f81c13a7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:40.705930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fdca6-bd7b-4f7d-a7ff-10f81c13a7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:40.706194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fdca6-bd7b-4f7d-a7ff-10f81c13a7f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:40.706835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fdca6-bd7b-4f7d-a7ff-10f81c13a7f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:40.707136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fdca6-bd7b-4f7d-a7ff-10f81c13a7f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:40.708800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fdca6-bd7b-4f7d-a7ff-10f81c13a7f7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:40.721406 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2668/5385] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:11:40.726083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fdca6-bd7b-4f7d-a7ff-10f81c13a7f7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:40.732392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a583487-aa0f-4985-bef8-41a3bd4b9ad7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:40.739213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:40.740128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:40.740598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:40.741081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fdca6-bd7b-4f7d-a7ff-10f81c13a7f7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:40.741286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] PATH_INFO: `/v3/users/6854aff251ea4cf19c219b6149340aac` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:40.743322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6854aff251ea4cf19c219b6149340aac)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:40.746256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:40.747869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f5703-3320-49e8-812d-1ca020734589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:40.748520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f5703-3320-49e8-812d-1ca020734589 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:40.748796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f5703-3320-49e8-812d-1ca020734589 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:40.749140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f5703-3320-49e8-812d-1ca020734589 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:40.750029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f5703-3320-49e8-812d-1ca020734589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:40.752524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f5703-3320-49e8-812d-1ca020734589 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:40.765703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:40.789103 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2669/5386] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:40 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:40.792968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f5703-3320-49e8-812d-1ca020734589 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:40.812175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6854aff251ea4cf19c219b6149340aac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.821924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6854aff251ea4cf19c219b6149340aac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.829398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6854aff251ea4cf19c219b6149340aac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.842101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6854aff251ea4cf19c219b6149340aac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.842824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6854aff251ea4cf19c219b6149340aac'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:40.849351 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2718/5387] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:40 2022] DELETE /identity/v3/users/6854aff251ea4cf19c219b6149340aac => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:40.868084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f5703-3320-49e8-812d-1ca020734589 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:40.941932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e086e-b130-450a-bad2-5e88fc869dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:40.942556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e086e-b130-450a-bad2-5e88fc869dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:40.942832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e086e-b130-450a-bad2-5e88fc869dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:40.943104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e086e-b130-450a-bad2-5e88fc869dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:40.944144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e086e-b130-450a-bad2-5e88fc869dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:40.946593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e086e-b130-450a-bad2-5e88fc869dc5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:40.975705 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2670/5388] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:40 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:40.979833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:40.981912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:41.013223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b8f9a-bbbc-4a74-bfcd-c662864e7d77 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:41.019071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db80b50-2e51-4351-9dc3-32233d1ddbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:41.020483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db80b50-2e51-4351-9dc3-32233d1ddbe8 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.020898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db80b50-2e51-4351-9dc3-32233d1ddbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.021300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db80b50-2e51-4351-9dc3-32233d1ddbe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.022503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db80b50-2e51-4351-9dc3-32233d1ddbe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:41.025557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db80b50-2e51-4351-9dc3-32233d1ddbe8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:41.041221 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2719/5389] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:41.042064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e086e-b130-450a-bad2-5e88fc869dc5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:41.057738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e086e-b130-450a-bad2-5e88fc869dc5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:41.063855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d16d10-2167-45da-bb1a-e0255c631546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:41.064799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d16d10-2167-45da-bb1a-e0255c631546 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.065407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d16d10-2167-45da-bb1a-e0255c631546 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.065870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d16d10-2167-45da-bb1a-e0255c631546 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.066712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d16d10-2167-45da-bb1a-e0255c631546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:41.068594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d16d10-2167-45da-bb1a-e0255c631546 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:41.081062 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2671/5390] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:11:41.085017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db80b50-2e51-4351-9dc3-32233d1ddbe8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:41.098298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db80b50-2e51-4351-9dc3-32233d1ddbe8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:41.103669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b57d1e-c56f-4864-b8e1-abdfdce59171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:41.104273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b57d1e-c56f-4864-b8e1-abdfdce59171 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.104680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b57d1e-c56f-4864-b8e1-abdfdce59171 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.105232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b57d1e-c56f-4864-b8e1-abdfdce59171 admin admin] PATH_INFO: `/v3/projects/a265133763e7444e80adc0cd8541340c/users/f127011ceab246a8834d50f01f426487/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.122825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b57d1e-c56f-4864-b8e1-abdfdce59171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a265133763e7444e80adc0cd8541340c, user_id=f127011ceab246a8834d50f01f426487, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:41.125740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b57d1e-c56f-4864-b8e1-abdfdce59171 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:41.139213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:41.152720 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2720/5391] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:41 2022] PUT /identity/v3/projects/a265133763e7444e80adc0cd8541340c/users/f127011ceab246a8834d50f01f426487/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:41.154487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3869ad-5986-49c8-b749-b6f836bc951d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.154765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3869ad-5986-49c8-b749-b6f836bc951d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.155176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3869ad-5986-49c8-b749-b6f836bc951d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.178786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5e3869ad-5986-49c8-b749-b6f836bc951d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:41.180022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5e3869ad-5986-49c8-b749-b6f836bc951d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:41.232216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b57d1e-c56f-4864-b8e1-abdfdce59171 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:41.235848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-90b57d1e-c56f-4864-b8e1-abdfdce59171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:41.283921 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2672/5392] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:41.292908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3869ad-5986-49c8-b749-b6f836bc951d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:41.294741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5e3869ad-5986-49c8-b749-b6f836bc951d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:41.316233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b57d1e-c56f-4864-b8e1-abdfdce59171 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:41.319481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:41.323433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804e878-f0b7-4f9f-b8c2-061e49f53d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:41.324081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804e878-f0b7-4f9f-b8c2-061e49f53d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.324358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804e878-f0b7-4f9f-b8c2-061e49f53d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.324653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804e878-f0b7-4f9f-b8c2-061e49f53d89 admin admin] PATH_INFO: `/v3/projects/9963bd6ccc5a4e098875e8e4d59bb266` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.326426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804e878-f0b7-4f9f-b8c2-061e49f53d89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9963bd6ccc5a4e098875e8e4d59bb266)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:41.328385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804e878-f0b7-4f9f-b8c2-061e49f53d89 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:41.371748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3869ad-5986-49c8-b749-b6f836bc951d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:41.384018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5804e878-f0b7-4f9f-b8c2-061e49f53d89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9963bd6ccc5a4e098875e8e4d59bb266', 'request_id': 'req-5804e878-f0b7-4f9f-b8c2-061e49f53d89'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:41.393888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5804e878-f0b7-4f9f-b8c2-061e49f53d89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9963bd6ccc5a4e098875e8e4d59bb266', 'request_id': 'req-5804e878-f0b7-4f9f-b8c2-061e49f53d89'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:41.405222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5804e878-f0b7-4f9f-b8c2-061e49f53d89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9963bd6ccc5a4e098875e8e4d59bb266', 'request_id': 'req-5804e878-f0b7-4f9f-b8c2-061e49f53d89'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:41.418135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5804e878-f0b7-4f9f-b8c2-061e49f53d89 admin admin] The token cache is being invalidate because project 9963bd6ccc5a4e098875e8e4d59bb266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:41.418401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5804e878-f0b7-4f9f-b8c2-061e49f53d89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:41.419404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5804e878-f0b7-4f9f-b8c2-061e49f53d89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:41.424641 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2721/5393] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:41 2022] DELETE /identity/v3/projects/9963bd6ccc5a4e098875e8e4d59bb266 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:41.425969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f0806-73cf-4c2e-907d-61874d41bd36 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.426251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f0806-73cf-4c2e-907d-61874d41bd36 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.426522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f0806-73cf-4c2e-907d-61874d41bd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.437850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23f613-6675-40fb-a221-29be44db52ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:41.438565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23f613-6675-40fb-a221-29be44db52ca service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.438905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23f613-6675-40fb-a221-29be44db52ca service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.439356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23f613-6675-40fb-a221-29be44db52ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.441493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2a23f613-6675-40fb-a221-29be44db52ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:41.448107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-990f0806-73cf-4c2e-907d-61874d41bd36 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:41.451778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-990f0806-73cf-4c2e-907d-61874d41bd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:41.525777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23f613-6675-40fb-a221-29be44db52ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:41.527379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23f613-6675-40fb-a221-29be44db52ca service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:41.534937 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2722/5394] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:41.540309 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2673/5395] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:41.543795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274803fe-54b9-4e66-87ce-92f54f169d67 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.544132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274803fe-54b9-4e66-87ce-92f54f169d67 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.544498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274803fe-54b9-4e66-87ce-92f54f169d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.565864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-274803fe-54b9-4e66-87ce-92f54f169d67 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:41.567793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-274803fe-54b9-4e66-87ce-92f54f169d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:41.581621 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2723/5396] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:41.582685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23f613-6675-40fb-a221-29be44db52ca service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:41.593301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23f613-6675-40fb-a221-29be44db52ca service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:41.598715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49592329-b3e7-4de5-ae56-33d17db0b1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:41.599345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49592329-b3e7-4de5-ae56-33d17db0b1ad admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.599621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49592329-b3e7-4de5-ae56-33d17db0b1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.600417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49592329-b3e7-4de5-ae56-33d17db0b1ad admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.600587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49592329-b3e7-4de5-ae56-33d17db0b1ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:41.600883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49592329-b3e7-4de5-ae56-33d17db0b1ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:41.602535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49592329-b3e7-4de5-ae56-33d17db0b1ad admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:41.621579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274803fe-54b9-4e66-87ce-92f54f169d67 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:41.627014 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2674/5397] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:11:41.636823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274803fe-54b9-4e66-87ce-92f54f169d67 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:41.642002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7412dc-a99c-41d5-bf6e-e7577000ca9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:41.642907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7412dc-a99c-41d5-bf6e-e7577000ca9b admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.642907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7412dc-a99c-41d5-bf6e-e7577000ca9b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.643455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7412dc-a99c-41d5-bf6e-e7577000ca9b admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.644053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7412dc-a99c-41d5-bf6e-e7577000ca9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:41.646316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7412dc-a99c-41d5-bf6e-e7577000ca9b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:41.682649 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2724/5398] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:41 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:41.687193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49592329-b3e7-4de5-ae56-33d17db0b1ad admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:41.700019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49592329-b3e7-4de5-ae56-33d17db0b1ad admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:41.772975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba40b8d-cee9-4f56-a586-e27be30b5b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:41.773664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba40b8d-cee9-4f56-a586-e27be30b5b7f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.774004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba40b8d-cee9-4f56-a586-e27be30b5b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.774336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba40b8d-cee9-4f56-a586-e27be30b5b7f admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.775046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba40b8d-cee9-4f56-a586-e27be30b5b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:41.777160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba40b8d-cee9-4f56-a586-e27be30b5b7f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:41.795970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:41.801816 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2675/5399] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:41 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:41.805956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7412dc-a99c-41d5-bf6e-e7577000ca9b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:41.811739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:41.818099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7412dc-a99c-41d5-bf6e-e7577000ca9b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:41.824894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206653b8-bdc6-4478-9001-e42ca52e826c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:41.825558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206653b8-bdc6-4478-9001-e42ca52e826c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.825889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206653b8-bdc6-4478-9001-e42ca52e826c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.826212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206653b8-bdc6-4478-9001-e42ca52e826c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.826847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206653b8-bdc6-4478-9001-e42ca52e826c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:41.828540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206653b8-bdc6-4478-9001-e42ca52e826c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:41.842587 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2725/5400] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:11:41.843790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba40b8d-cee9-4f56-a586-e27be30b5b7f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:41.854051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba40b8d-cee9-4f56-a586-e27be30b5b7f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:41.858795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcba26-b931-48a5-ba57-d18646377ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:41.859388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcba26-b931-48a5-ba57-d18646377ddd admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.859625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcba26-b931-48a5-ba57-d18646377ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.859913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcba26-b931-48a5-ba57-d18646377ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.860590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcba26-b931-48a5-ba57-d18646377ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:41.862382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcba26-b931-48a5-ba57-d18646377ddd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:41.873117 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2676/5401] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:41.878218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206653b8-bdc6-4478-9001-e42ca52e826c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:41.888841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206653b8-bdc6-4478-9001-e42ca52e826c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:41.894096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4d101-3562-4503-9c15-ed552537d6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:41.894750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4d101-3562-4503-9c15-ed552537d6be admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.895019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4d101-3562-4503-9c15-ed552537d6be admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.895283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4d101-3562-4503-9c15-ed552537d6be admin admin] PATH_INFO: `/v3/projects/a841c0735d87429f863cc32827c79f3a/users/2ff25e3225ce4b169fee2a8ff15e96d4/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.910283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4d101-3562-4503-9c15-ed552537d6be admin admin] RBAC: Authorizing `identity:create_grant(project_id=a841c0735d87429f863cc32827c79f3a, user_id=2ff25e3225ce4b169fee2a8ff15e96d4, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:41.912973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4d101-3562-4503-9c15-ed552537d6be admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:41.927829 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2726/5402] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:41 2022] PUT /identity/v3/projects/a841c0735d87429f863cc32827c79f3a/users/2ff25e3225ce4b169fee2a8ff15e96d4/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:41.938275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e013005-8b2d-4dcc-975e-5e6d1bdc9264 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:41.938535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e013005-8b2d-4dcc-975e-5e6d1bdc9264 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:41.938803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e013005-8b2d-4dcc-975e-5e6d1bdc9264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:41.967115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9e013005-8b2d-4dcc-975e-5e6d1bdc9264 None None] MFA Rules not processed for user `2ff25e3225ce4b169fee2a8ff15e96d4`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:41.968326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9e013005-8b2d-4dcc-975e-5e6d1bdc9264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:42.034447 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2677/5403] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:42.038537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4d101-3562-4503-9c15-ed552537d6be admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:42.122018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4d101-3562-4503-9c15-ed552537d6be admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:42.131082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e3ff7-2b0e-4a75-a484-ff49371d795a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:42.132099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e3ff7-2b0e-4a75-a484-ff49371d795a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:42.132675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e3ff7-2b0e-4a75-a484-ff49371d795a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:42.132952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e3ff7-2b0e-4a75-a484-ff49371d795a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:42.133702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e3ff7-2b0e-4a75-a484-ff49371d795a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:42.136676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e3ff7-2b0e-4a75-a484-ff49371d795a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:42.187517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e013005-8b2d-4dcc-975e-5e6d1bdc9264 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:42.200122 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2727/5404] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:42 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:42.207726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e013005-8b2d-4dcc-975e-5e6d1bdc9264 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:42.312599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f602c-f915-4fde-b903-a1d0e2b26aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:42.313384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f602c-f915-4fde-b903-a1d0e2b26aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:42.313634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f602c-f915-4fde-b903-a1d0e2b26aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:42.313990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f602c-f915-4fde-b903-a1d0e2b26aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:42.315232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f602c-f915-4fde-b903-a1d0e2b26aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:42.317512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f602c-f915-4fde-b903-a1d0e2b26aa0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:42.350096 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2678/5405] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:42 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:42.354396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e3ff7-2b0e-4a75-a484-ff49371d795a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:42.366185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e3ff7-2b0e-4a75-a484-ff49371d795a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:42.371626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6eecc8-0b78-44b9-aeda-108dcbe7a18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:42.372251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6eecc8-0b78-44b9-aeda-108dcbe7a18d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:42.372572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6eecc8-0b78-44b9-aeda-108dcbe7a18d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:42.372832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6eecc8-0b78-44b9-aeda-108dcbe7a18d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:42.373473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6eecc8-0b78-44b9-aeda-108dcbe7a18d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:42.375079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6eecc8-0b78-44b9-aeda-108dcbe7a18d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:42.390580 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2728/5406] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:42.391818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f602c-f915-4fde-b903-a1d0e2b26aa0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:42.404284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f602c-f915-4fde-b903-a1d0e2b26aa0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:42.420124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b88a25-4e6f-440f-85d2-1d95e1588a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:42.420874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b88a25-4e6f-440f-85d2-1d95e1588a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:42.421213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b88a25-4e6f-440f-85d2-1d95e1588a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:42.421590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b88a25-4e6f-440f-85d2-1d95e1588a40 admin admin] PATH_INFO: `/v3/projects/f749e0b3182a4b2f977b76d4580003fa/users/ec27efbdcfa64eed9aa08aa518f6cc41/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:42.437886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b88a25-4e6f-440f-85d2-1d95e1588a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f749e0b3182a4b2f977b76d4580003fa, user_id=ec27efbdcfa64eed9aa08aa518f6cc41, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:42.441028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b88a25-4e6f-440f-85d2-1d95e1588a40 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:42.461964 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2679/5407] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:42 2022] PUT /identity/v3/projects/f749e0b3182a4b2f977b76d4580003fa/users/ec27efbdcfa64eed9aa08aa518f6cc41/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:42.466136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6eecc8-0b78-44b9-aeda-108dcbe7a18d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:42.476665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6eecc8-0b78-44b9-aeda-108dcbe7a18d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:42.597209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58374afc-2403-4c65-ad88-787a0232aa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:42.598247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58374afc-2403-4c65-ad88-787a0232aa83 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:42.599102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58374afc-2403-4c65-ad88-787a0232aa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:42.599936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58374afc-2403-4c65-ad88-787a0232aa83 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:42.600832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58374afc-2403-4c65-ad88-787a0232aa83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:42.614035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58374afc-2403-4c65-ad88-787a0232aa83 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:42.624413 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2729/5408] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:42.628407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b88a25-4e6f-440f-85d2-1d95e1588a40 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:42.642369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b88a25-4e6f-440f-85d2-1d95e1588a40 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:42.648990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733c203-c75a-4848-b2c5-4a1665696068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:42.649941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733c203-c75a-4848-b2c5-4a1665696068 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:42.650525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733c203-c75a-4848-b2c5-4a1665696068 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:42.652333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733c203-c75a-4848-b2c5-4a1665696068 admin admin] PATH_INFO: `/v3/projects/f749e0b3182a4b2f977b76d4580003fa/users/ec27efbdcfa64eed9aa08aa518f6cc41/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:42.655198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733c203-c75a-4848-b2c5-4a1665696068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f749e0b3182a4b2f977b76d4580003fa, user_id=ec27efbdcfa64eed9aa08aa518f6cc41, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:42.659638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733c203-c75a-4848-b2c5-4a1665696068 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:42.685648 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2680/5409] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:42 2022] PUT /identity/v3/projects/f749e0b3182a4b2f977b76d4580003fa/users/ec27efbdcfa64eed9aa08aa518f6cc41/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:42.686908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210f01f-5c00-457c-b804-6c3a68573d48 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:42.687200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210f01f-5c00-457c-b804-6c3a68573d48 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:42.687468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210f01f-5c00-457c-b804-6c3a68573d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:42.722138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d210f01f-5c00-457c-b804-6c3a68573d48 None None] MFA Rules not processed for user `ec27efbdcfa64eed9aa08aa518f6cc41`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:42.723385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d210f01f-5c00-457c-b804-6c3a68573d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:42.753107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:42.807591 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2730/5410] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:42.809107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733c203-c75a-4848-b2c5-4a1665696068 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:42.823550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733c203-c75a-4848-b2c5-4a1665696068 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:42.831027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9013b5-f886-454c-920d-a4ef605f889f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:42.832071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9013b5-f886-454c-920d-a4ef605f889f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:42.833033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9013b5-f886-454c-920d-a4ef605f889f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:42.833501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9013b5-f886-454c-920d-a4ef605f889f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:42.834790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9013b5-f886-454c-920d-a4ef605f889f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:42.837287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9013b5-f886-454c-920d-a4ef605f889f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:42.865262 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2681/5411] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:42 2022] POST /identity/v3/services => generated 289 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:42.865453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210f01f-5c00-457c-b804-6c3a68573d48 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:42.886995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210f01f-5c00-457c-b804-6c3a68573d48 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:42.918298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7353a-b613-4eeb-83cc-6613404f2940 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:42.919247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7353a-b613-4eeb-83cc-6613404f2940 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:42.919527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7353a-b613-4eeb-83cc-6613404f2940 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:42.920077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7353a-b613-4eeb-83cc-6613404f2940 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:42.920564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7353a-b613-4eeb-83cc-6613404f2940 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:42.921574 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-64e7353a-b613-4eeb-83cc-6613404f2940 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:42.921574 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:11:42.921574 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:42.922486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7353a-b613-4eeb-83cc-6613404f2940 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:42.946716 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-64e7353a-b613-4eeb-83cc-6613404f2940 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-683764309. To ensure interoperability, identifiers should be a valid uuid. Aug 20 21:11:42.946716 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Aug 20 21:11:42.946716 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:42.998774 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2731/5412] 10.222.0.23 () {60 vars in 1156 bytes} [Sat Aug 20 21:11:42 2022] POST /identity/v3/endpoints => generated 357 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:43.000274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9013b5-f886-454c-920d-a4ef605f889f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:43.016858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9013b5-f886-454c-920d-a4ef605f889f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:43.040306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825ae7d-3660-4619-bfdc-0fb7631f8ec8 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:43.041218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825ae7d-3660-4619-bfdc-0fb7631f8ec8 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:43.041497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825ae7d-3660-4619-bfdc-0fb7631f8ec8 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:43.042055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825ae7d-3660-4619-bfdc-0fb7631f8ec8 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/regions/tempest-region-683764309` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:43.043181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825ae7d-3660-4619-bfdc-0fb7631f8ec8 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-683764309)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:43.044669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825ae7d-3660-4619-bfdc-0fb7631f8ec8 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:43.050187 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2682/5413] 10.222.0.23 () {58 vars in 1203 bytes} [Sat Aug 20 21:11:42 2022] GET /identity/v3/regions/tempest-region-683764309 => generated 178 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:43.051459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7353a-b613-4eeb-83cc-6613404f2940 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:43.062549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7353a-b613-4eeb-83cc-6613404f2940 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:43.067695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303fd63-6cef-43aa-b6aa-d98f50f8f294 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:43.068323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303fd63-6cef-43aa-b6aa-d98f50f8f294 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:43.068689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303fd63-6cef-43aa-b6aa-d98f50f8f294 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:43.068959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303fd63-6cef-43aa-b6aa-d98f50f8f294 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:43.069611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303fd63-6cef-43aa-b6aa-d98f50f8f294 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:43.071616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303fd63-6cef-43aa-b6aa-d98f50f8f294 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:43.092985 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2732/5414] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:43 2022] POST /identity/v3/services => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:43.099022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825ae7d-3660-4619-bfdc-0fb7631f8ec8 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:43.106758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:43.111197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825ae7d-3660-4619-bfdc-0fb7631f8ec8 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:43.131178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2087d-aa43-4da3-8fbe-b2afcd81ab22 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:43.131786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2087d-aa43-4da3-8fbe-b2afcd81ab22 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:43.132120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2087d-aa43-4da3-8fbe-b2afcd81ab22 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:43.132308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2087d-aa43-4da3-8fbe-b2afcd81ab22 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:43.133146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2087d-aa43-4da3-8fbe-b2afcd81ab22 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:43.134301 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-4fa2087d-aa43-4da3-8fbe-b2afcd81ab22 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:43.134301 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:11:43.134301 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:43.135249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2087d-aa43-4da3-8fbe-b2afcd81ab22 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:43.159234 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-4fa2087d-aa43-4da3-8fbe-b2afcd81ab22 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-155505591. To ensure interoperability, identifiers should be a valid uuid. Aug 20 21:11:43.159234 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Aug 20 21:11:43.159234 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:43.220433 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2683/5415] 10.222.0.23 () {60 vars in 1156 bytes} [Sat Aug 20 21:11:43 2022] POST /identity/v3/endpoints => generated 361 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:43.222059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303fd63-6cef-43aa-b6aa-d98f50f8f294 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:43.251117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303fd63-6cef-43aa-b6aa-d98f50f8f294 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:43.272809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b62843-b805-4637-9b45-1cf1fc8ed640 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:43.274613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b62843-b805-4637-9b45-1cf1fc8ed640 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:43.274893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b62843-b805-4637-9b45-1cf1fc8ed640 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:43.275165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b62843-b805-4637-9b45-1cf1fc8ed640 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/regions/tempest-region-155505591` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:43.275769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b62843-b805-4637-9b45-1cf1fc8ed640 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-155505591)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:43.288145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b62843-b805-4637-9b45-1cf1fc8ed640 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:43.297862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2087d-aa43-4da3-8fbe-b2afcd81ab22 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:43.306074 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2733/5416] 10.222.0.23 () {58 vars in 1203 bytes} [Sat Aug 20 21:11:43 2022] GET /identity/v3/regions/tempest-region-155505591 => generated 178 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:43.322194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2087d-aa43-4da3-8fbe-b2afcd81ab22 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:43.338587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981c347-f387-4797-b1ad-22d7905b9b06 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:43.338587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981c347-f387-4797-b1ad-22d7905b9b06 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:43.338587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981c347-f387-4797-b1ad-22d7905b9b06 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:43.339121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981c347-f387-4797-b1ad-22d7905b9b06 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:43.339739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981c347-f387-4797-b1ad-22d7905b9b06 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:43.343968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981c347-f387-4797-b1ad-22d7905b9b06 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:43.409195 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-2981c347-f387-4797-b1ad-22d7905b9b06 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-42591065. To ensure interoperability, identifiers should be a valid uuid. Aug 20 21:11:43.409195 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Aug 20 21:11:43.409195 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:43.470124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b62843-b805-4637-9b45-1cf1fc8ed640 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:43.470656 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2684/5417] 10.222.0.23 () {60 vars in 1156 bytes} [Sat Aug 20 21:11:43 2022] POST /identity/v3/endpoints => generated 355 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:43.483517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b62843-b805-4637-9b45-1cf1fc8ed640 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:43.518574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38bfc1-3e9f-4146-8ce9-b5a43e0e935f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:43.519655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38bfc1-3e9f-4146-8ce9-b5a43e0e935f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:43.520113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38bfc1-3e9f-4146-8ce9-b5a43e0e935f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:43.521532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38bfc1-3e9f-4146-8ce9-b5a43e0e935f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/regions/tempest-region-42591065` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:43.522404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38bfc1-3e9f-4146-8ce9-b5a43e0e935f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-42591065)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:43.529710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38bfc1-3e9f-4146-8ce9-b5a43e0e935f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:43.533316 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2734/5418] 10.222.0.23 () {58 vars in 1200 bytes} [Sat Aug 20 21:11:43 2022] GET /identity/v3/regions/tempest-region-42591065 => generated 176 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:11:43.540951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981c347-f387-4797-b1ad-22d7905b9b06 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:43.557336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981c347-f387-4797-b1ad-22d7905b9b06 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:43.569712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1973adb-1e77-4bf7-a433-2b850604e584 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:43.570783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1973adb-1e77-4bf7-a433-2b850604e584 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:43.571058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1973adb-1e77-4bf7-a433-2b850604e584 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:43.571321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1973adb-1e77-4bf7-a433-2b850604e584 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:43.571957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1973adb-1e77-4bf7-a433-2b850604e584 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:43.573010 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-f1973adb-1e77-4bf7-a433-2b850604e584 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:43.573010 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:11:43.573010 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:43.573908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1973adb-1e77-4bf7-a433-2b850604e584 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:43.590885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38bfc1-3e9f-4146-8ce9-b5a43e0e935f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:43.593736 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2685/5419] 10.222.0.23 () {58 vars in 1134 bytes} [Sat Aug 20 21:11:43 2022] GET /identity/v3/endpoints => generated 4387 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:43.604621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38bfc1-3e9f-4146-8ce9-b5a43e0e935f tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:43.612821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33c030-48da-4189-b436-2e119c0870f7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:43.613582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33c030-48da-4189-b436-2e119c0870f7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:43.614250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33c030-48da-4189-b436-2e119c0870f7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:43.614624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33c030-48da-4189-b436-2e119c0870f7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints/9a9aab0467df4c508032c55762b7596d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:43.624758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33c030-48da-4189-b436-2e119c0870f7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:get_endpoint(endpoint_id=9a9aab0467df4c508032c55762b7596d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:43.626002 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-da33c030-48da-4189-b436-2e119c0870f7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:43.626002 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:11:43.626002 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:43.627448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33c030-48da-4189-b436-2e119c0870f7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:43.630177 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2735/5420] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:43 2022] GET /identity/v3/endpoints/9a9aab0467df4c508032c55762b7596d => generated 355 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:43.634240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1973adb-1e77-4bf7-a433-2b850604e584 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:43.679962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1973adb-1e77-4bf7-a433-2b850604e584 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:43.694057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a633e-030c-4f3a-972d-384901a03bed tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:43.694649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a633e-030c-4f3a-972d-384901a03bed tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:43.694927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a633e-030c-4f3a-972d-384901a03bed tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:43.695560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a633e-030c-4f3a-972d-384901a03bed tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints/9a9aab0467df4c508032c55762b7596d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:43.699012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a633e-030c-4f3a-972d-384901a03bed tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=9a9aab0467df4c508032c55762b7596d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:43.699283 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-cb2a633e-030c-4f3a-972d-384901a03bed tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:43.699283 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:11:43.699283 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:43.700167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a633e-030c-4f3a-972d-384901a03bed tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:43.723709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cb2a633e-030c-4f3a-972d-384901a03bed tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '9a9aab0467df4c508032c55762b7596d', 'request_id': 'req-cb2a633e-030c-4f3a-972d-384901a03bed'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:43.751981 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2686/5421] 10.222.0.23 () {58 vars in 1236 bytes} [Sat Aug 20 21:11:43 2022] DELETE /identity/v3/endpoints/9a9aab0467df4c508032c55762b7596d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:43.755796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33c030-48da-4189-b436-2e119c0870f7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:43.768832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33c030-48da-4189-b436-2e119c0870f7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:43.800363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d1e67-4c7e-4e97-b754-5f44ad7bde3c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:43.801382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d1e67-4c7e-4e97-b754-5f44ad7bde3c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:43.801670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d1e67-4c7e-4e97-b754-5f44ad7bde3c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:43.801942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d1e67-4c7e-4e97-b754-5f44ad7bde3c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:43.802578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d1e67-4c7e-4e97-b754-5f44ad7bde3c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:43.809270 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-430d1e67-4c7e-4e97-b754-5f44ad7bde3c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:43.809270 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:11:43.809270 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:43.810224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d1e67-4c7e-4e97-b754-5f44ad7bde3c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:43.827755 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2736/5422] 10.222.0.23 () {58 vars in 1134 bytes} [Sat Aug 20 21:11:43 2022] GET /identity/v3/endpoints => generated 4045 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:43.832481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a633e-030c-4f3a-972d-384901a03bed tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:43.849363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a633e-030c-4f3a-972d-384901a03bed tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:43.860711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc1e34-c8fd-46a0-bde0-b5805ee696c4 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:43.861730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc1e34-c8fd-46a0-bde0-b5805ee696c4 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:43.861971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc1e34-c8fd-46a0-bde0-b5805ee696c4 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:43.862693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc1e34-c8fd-46a0-bde0-b5805ee696c4 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints/9a9aab0467df4c508032c55762b7596d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:43.868961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc1e34-c8fd-46a0-bde0-b5805ee696c4 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-2ffc1e34-c8fd-46a0-bde0-b5805ee696c4 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Could not find endpoint: 9a9aab0467df4c508032c55762b7596d.: keystone.exception.EndpointNotFound: Could not find endpoint: 9a9aab0467df4c508032c55762b7596d. Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 236, in _get_endpoint Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return session.query(Endpoint).filter_by(id=endpoint_id).one() Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._iter().one() Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._only_one_row( Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exc.NoResultFound( Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:43.884700 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 228, in get_endpoint Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_endpoint(endpoint_id) Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 242, in get_endpoint Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_endpoint(session, endpoint_id).to_dict() Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 238, in _get_endpoint Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 9a9aab0467df4c508032c55762b7596d. Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:11:43.885762 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 113, in delete Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_endpoint') Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:11:43.886617 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:11:43.887409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:11:43.887409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:11:43.887409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:11:43.887409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:11:43.887409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:11:43.887409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:11:43.887409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:11:43.887409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:11:43.887409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:11:43.887409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 230, in get_endpoint Aug 20 21:11:43.887409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Aug 20 21:11:43.887409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 9a9aab0467df4c508032c55762b7596d. Aug 20 21:11:43.887409 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:43.889575 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2687/5423] 10.222.0.23 () {58 vars in 1236 bytes} [Sat Aug 20 21:11:43 2022] DELETE /identity/v3/endpoints/9a9aab0467df4c508032c55762b7596d => generated 114 bytes in 58 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:11:43.893729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d1e67-4c7e-4e97-b754-5f44ad7bde3c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:43.906413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d1e67-4c7e-4e97-b754-5f44ad7bde3c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:43.914702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9529f-a2a2-4463-9148-f299514fad64 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:43.920563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9529f-a2a2-4463-9148-f299514fad64 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:43.920848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9529f-a2a2-4463-9148-f299514fad64 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:43.921126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9529f-a2a2-4463-9148-f299514fad64 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/regions/tempest-region-42591065` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:43.921750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9529f-a2a2-4463-9148-f299514fad64 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-42591065)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:43.924073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9529f-a2a2-4463-9148-f299514fad64 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:43.943529 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-c6c9529f-a2a2-4463-9148-f299514fad64 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-42591065. To ensure interoperability, identifiers should be a valid uuid. Aug 20 21:11:43.943529 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Aug 20 21:11:43.943529 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:43.952464 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2737/5424] 10.222.0.23 () {58 vars in 1203 bytes} [Sat Aug 20 21:11:43 2022] DELETE /identity/v3/regions/tempest-region-42591065 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:43.961971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc1e34-c8fd-46a0-bde0-b5805ee696c4 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:43.974255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc1e34-c8fd-46a0-bde0-b5805ee696c4 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:44.001394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6672076-67cc-4100-b6dc-a48bab6906c2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:44.002522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6672076-67cc-4100-b6dc-a48bab6906c2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:44.002937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6672076-67cc-4100-b6dc-a48bab6906c2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:44.003423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6672076-67cc-4100-b6dc-a48bab6906c2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:44.004219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6672076-67cc-4100-b6dc-a48bab6906c2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:44.005973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6672076-67cc-4100-b6dc-a48bab6906c2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:44.016962 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2688/5425] 10.222.0.23 () {58 vars in 1134 bytes} [Sat Aug 20 21:11:43 2022] GET /identity/v3/endpoints => generated 4045 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:44.022260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9529f-a2a2-4463-9148-f299514fad64 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:44.033948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9529f-a2a2-4463-9148-f299514fad64 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:44.043362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daea94b-1231-48b9-a8b2-746205fbb619 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:44.044125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daea94b-1231-48b9-a8b2-746205fbb619 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:44.044558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daea94b-1231-48b9-a8b2-746205fbb619 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:44.045022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daea94b-1231-48b9-a8b2-746205fbb619 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:44.045811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daea94b-1231-48b9-a8b2-746205fbb619 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Adding query filter params (service_id=c26dc4dc75ef4fb3804a1cbdc72038ec) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:44.046216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daea94b-1231-48b9-a8b2-746205fbb619 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:44.048159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daea94b-1231-48b9-a8b2-746205fbb619 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:44.061976 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2738/5426] 10.222.0.23 () {58 vars in 1221 bytes} [Sat Aug 20 21:11:44 2022] GET /identity/v3/endpoints?service_id=c26dc4dc75ef4fb3804a1cbdc72038ec => generated 499 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:44.062775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6672076-67cc-4100-b6dc-a48bab6906c2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:44.082787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6672076-67cc-4100-b6dc-a48bab6906c2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:44.089160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e04da-ba84-42f9-9ad8-11572e0ebd0b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:44.090838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e04da-ba84-42f9-9ad8-11572e0ebd0b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:44.091186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e04da-ba84-42f9-9ad8-11572e0ebd0b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:44.091679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e04da-ba84-42f9-9ad8-11572e0ebd0b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:44.092658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e04da-ba84-42f9-9ad8-11572e0ebd0b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Adding query filter params (service_id=d4e5228794be4d47a944b1fd75519409) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:44.093031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e04da-ba84-42f9-9ad8-11572e0ebd0b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:44.094822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e04da-ba84-42f9-9ad8-11572e0ebd0b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:44.107635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daea94b-1231-48b9-a8b2-746205fbb619 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:44.119672 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2689/5427] 10.222.0.23 () {58 vars in 1221 bytes} [Sat Aug 20 21:11:44 2022] GET /identity/v3/endpoints?service_id=d4e5228794be4d47a944b1fd75519409 => generated 503 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:11:44.125885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daea94b-1231-48b9-a8b2-746205fbb619 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:44.145444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a2e3d-6381-4118-88e4-571abd439f5c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:44.146423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a2e3d-6381-4118-88e4-571abd439f5c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:44.146902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a2e3d-6381-4118-88e4-571abd439f5c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:44.147369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a2e3d-6381-4118-88e4-571abd439f5c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:44.148226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a2e3d-6381-4118-88e4-571abd439f5c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Adding query filter params (interface=public) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:44.148772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a2e3d-6381-4118-88e4-571abd439f5c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:44.150584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a2e3d-6381-4118-88e4-571abd439f5c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:44.173091 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2739/5428] 10.222.0.23 () {58 vars in 1167 bytes} [Sat Aug 20 21:11:44 2022] GET /identity/v3/endpoints?interface=public => generated 3402 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:44.175157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e04da-ba84-42f9-9ad8-11572e0ebd0b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:44.199834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e04da-ba84-42f9-9ad8-11572e0ebd0b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:44.208768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbef1e-efe4-4c80-9a24-e839b997b9fc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:44.209912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbef1e-efe4-4c80-9a24-e839b997b9fc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:44.210195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbef1e-efe4-4c80-9a24-e839b997b9fc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:44.210467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbef1e-efe4-4c80-9a24-e839b997b9fc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:44.211132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbef1e-efe4-4c80-9a24-e839b997b9fc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Adding query filter params (interface=internal) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:44.213463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbef1e-efe4-4c80-9a24-e839b997b9fc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:44.215421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbef1e-efe4-4c80-9a24-e839b997b9fc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:44.223251 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2690/5429] 10.222.0.23 () {58 vars in 1171 bytes} [Sat Aug 20 21:11:44 2022] GET /identity/v3/endpoints?interface=internal => generated 478 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:44.236986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a2e3d-6381-4118-88e4-571abd439f5c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:44.249769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a2e3d-6381-4118-88e4-571abd439f5c tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:44.255286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511fe80-2290-41fc-b932-39eacb214690 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:44.256033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511fe80-2290-41fc-b932-39eacb214690 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:44.256491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511fe80-2290-41fc-b932-39eacb214690 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:44.257006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511fe80-2290-41fc-b932-39eacb214690 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:44.257840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511fe80-2290-41fc-b932-39eacb214690 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:44.259486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511fe80-2290-41fc-b932-39eacb214690 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:44.285131 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2740/5430] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:44 2022] POST /identity/v3/services => generated 289 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:44.289017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbef1e-efe4-4c80-9a24-e839b997b9fc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:44.304140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbef1e-efe4-4c80-9a24-e839b997b9fc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:44.332780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd28b5-bac9-4388-8881-9ca6261fb6cc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:44.333347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd28b5-bac9-4388-8881-9ca6261fb6cc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:44.333706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd28b5-bac9-4388-8881-9ca6261fb6cc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:44.334030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd28b5-bac9-4388-8881-9ca6261fb6cc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:44.334772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd28b5-bac9-4388-8881-9ca6261fb6cc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:44.337238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd28b5-bac9-4388-8881-9ca6261fb6cc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:44.367482 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-fefd28b5-bac9-4388-8881-9ca6261fb6cc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1656201885. To ensure interoperability, identifiers should be a valid uuid. Aug 20 21:11:44.367482 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Aug 20 21:11:44.367482 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:44.417841 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2691/5431] 10.222.0.23 () {60 vars in 1156 bytes} [Sat Aug 20 21:11:44 2022] POST /identity/v3/endpoints => generated 361 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:44.418582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511fe80-2290-41fc-b932-39eacb214690 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:44.446116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511fe80-2290-41fc-b932-39eacb214690 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:44.499340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b1cad-8881-4557-8bfe-13bce90985ba tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:44.500023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b1cad-8881-4557-8bfe-13bce90985ba tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:44.500311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b1cad-8881-4557-8bfe-13bce90985ba tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:44.504848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b1cad-8881-4557-8bfe-13bce90985ba tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/regions/tempest-region-1656201885` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:44.505714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b1cad-8881-4557-8bfe-13bce90985ba tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1656201885)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:44.509339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b1cad-8881-4557-8bfe-13bce90985ba tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:44.516909 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2741/5432] 10.222.0.23 () {58 vars in 1206 bytes} [Sat Aug 20 21:11:44 2022] GET /identity/v3/regions/tempest-region-1656201885 => generated 180 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:44.522447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd28b5-bac9-4388-8881-9ca6261fb6cc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:44.539350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd28b5-bac9-4388-8881-9ca6261fb6cc tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:44.555028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f06ca0-5d67-4073-a492-511c6f68fef1 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:44.555892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f06ca0-5d67-4073-a492-511c6f68fef1 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:44.556123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f06ca0-5d67-4073-a492-511c6f68fef1 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:44.556670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f06ca0-5d67-4073-a492-511c6f68fef1 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints/90a5e08933da41c2ac67349b6b2e08ca` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:44.573816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f06ca0-5d67-4073-a492-511c6f68fef1 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=90a5e08933da41c2ac67349b6b2e08ca)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:44.574925 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-55f06ca0-5d67-4073-a492-511c6f68fef1 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:44.574925 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:11:44.574925 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:44.576004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f06ca0-5d67-4073-a492-511c6f68fef1 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:44.617911 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-55f06ca0-5d67-4073-a492-511c6f68fef1 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1522406253. To ensure interoperability, identifiers should be a valid uuid. Aug 20 21:11:44.617911 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Aug 20 21:11:44.617911 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:44.620542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=7,keystone:DELETE=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:44.695873 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2692/5433] 10.222.0.23 () {60 vars in 1256 bytes} [Sat Aug 20 21:11:44 2022] PATCH /identity/v3/endpoints/90a5e08933da41c2ac67349b6b2e08ca => generated 364 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:44.701409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b1cad-8881-4557-8bfe-13bce90985ba tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:44.708971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:44.728730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b1cad-8881-4557-8bfe-13bce90985ba tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:44.778149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a259cb-2f46-4726-a2a2-603fecc837a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:44.778798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a259cb-2f46-4726-a2a2-603fecc837a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:44.779083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a259cb-2f46-4726-a2a2-603fecc837a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:44.779497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a259cb-2f46-4726-a2a2-603fecc837a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/regions/tempest-region-1522406253` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:44.780145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a259cb-2f46-4726-a2a2-603fecc837a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1522406253)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:44.781751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a259cb-2f46-4726-a2a2-603fecc837a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:44.787715 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2742/5434] 10.222.0.23 () {58 vars in 1206 bytes} [Sat Aug 20 21:11:44 2022] GET /identity/v3/regions/tempest-region-1522406253 => generated 180 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:44.795391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f06ca0-5d67-4073-a492-511c6f68fef1 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:44.812328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f06ca0-5d67-4073-a492-511c6f68fef1 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:44.826343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f7766-974d-4acc-bbbc-72254000e51b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:44.827012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f7766-974d-4acc-bbbc-72254000e51b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:44.827445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f7766-974d-4acc-bbbc-72254000e51b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:44.827560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f7766-974d-4acc-bbbc-72254000e51b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints/90a5e08933da41c2ac67349b6b2e08ca` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:44.839261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f7766-974d-4acc-bbbc-72254000e51b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=90a5e08933da41c2ac67349b6b2e08ca)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:44.842503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f7766-974d-4acc-bbbc-72254000e51b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:44.862828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f14f7766-974d-4acc-bbbc-72254000e51b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '90a5e08933da41c2ac67349b6b2e08ca', 'request_id': 'req-f14f7766-974d-4acc-bbbc-72254000e51b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:44.889816 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2693/5435] 10.222.0.23 () {58 vars in 1236 bytes} [Sat Aug 20 21:11:44 2022] DELETE /identity/v3/endpoints/90a5e08933da41c2ac67349b6b2e08ca => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:44.894323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a259cb-2f46-4726-a2a2-603fecc837a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:44.918860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a259cb-2f46-4726-a2a2-603fecc837a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:44.944128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d319e99-45bb-4387-8755-2c0bc40230b7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:44.944846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d319e99-45bb-4387-8755-2c0bc40230b7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:44.945612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d319e99-45bb-4387-8755-2c0bc40230b7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:44.945940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d319e99-45bb-4387-8755-2c0bc40230b7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/regions/tempest-region-1522406253` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:44.946538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d319e99-45bb-4387-8755-2c0bc40230b7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1522406253)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:44.948161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d319e99-45bb-4387-8755-2c0bc40230b7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:44.966165 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-7d319e99-45bb-4387-8755-2c0bc40230b7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1522406253. To ensure interoperability, identifiers should be a valid uuid. Aug 20 21:11:44.966165 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Aug 20 21:11:44.966165 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:44.977979 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2743/5436] 10.222.0.23 () {58 vars in 1209 bytes} [Sat Aug 20 21:11:44 2022] DELETE /identity/v3/regions/tempest-region-1522406253 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:44.982024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f7766-974d-4acc-bbbc-72254000e51b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:44.993372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f7766-974d-4acc-bbbc-72254000e51b tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:45.016695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa001203-9d08-49ce-9e4f-870c30fc30f2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:45.017424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa001203-9d08-49ce-9e4f-870c30fc30f2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:45.017698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa001203-9d08-49ce-9e4f-870c30fc30f2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:45.017952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa001203-9d08-49ce-9e4f-870c30fc30f2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/regions/tempest-region-1656201885` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:45.018544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa001203-9d08-49ce-9e4f-870c30fc30f2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1656201885)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:45.020750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa001203-9d08-49ce-9e4f-870c30fc30f2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:45.053756 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2694/5437] 10.222.0.23 () {58 vars in 1209 bytes} [Sat Aug 20 21:11:44 2022] DELETE /identity/v3/regions/tempest-region-1656201885 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:45.057746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d319e99-45bb-4387-8755-2c0bc40230b7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:45.074864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d319e99-45bb-4387-8755-2c0bc40230b7 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:45.096002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b44d2d-ed9c-493d-9d10-b1c5d64a4578 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:45.097055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b44d2d-ed9c-493d-9d10-b1c5d64a4578 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:45.097514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b44d2d-ed9c-493d-9d10-b1c5d64a4578 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:45.098057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b44d2d-ed9c-493d-9d10-b1c5d64a4578 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/services/46ea808323234963aca2ded11a41bb4a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:45.098817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b44d2d-ed9c-493d-9d10-b1c5d64a4578 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:delete_service(service_id=46ea808323234963aca2ded11a41bb4a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:45.101850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b44d2d-ed9c-493d-9d10-b1c5d64a4578 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:45.151677 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2744/5438] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:45 2022] DELETE /identity/v3/services/46ea808323234963aca2ded11a41bb4a => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:45.157181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa001203-9d08-49ce-9e4f-870c30fc30f2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:45.171553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa001203-9d08-49ce-9e4f-870c30fc30f2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:45.191305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede420c-9759-45da-a9c5-a11601b6db68 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:45.192110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede420c-9759-45da-a9c5-a11601b6db68 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:45.192404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede420c-9759-45da-a9c5-a11601b6db68 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:45.192674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede420c-9759-45da-a9c5-a11601b6db68 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints/ae2d7574096044b8805d10c65b4c3a06` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:45.199568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede420c-9759-45da-a9c5-a11601b6db68 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=ae2d7574096044b8805d10c65b4c3a06)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:45.201246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede420c-9759-45da-a9c5-a11601b6db68 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:45.211144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1ede420c-9759-45da-a9c5-a11601b6db68 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'ae2d7574096044b8805d10c65b4c3a06', 'request_id': 'req-1ede420c-9759-45da-a9c5-a11601b6db68'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:45.246253 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2695/5439] 10.222.0.23 () {58 vars in 1236 bytes} [Sat Aug 20 21:11:45 2022] DELETE /identity/v3/endpoints/ae2d7574096044b8805d10c65b4c3a06 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:45.250967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b44d2d-ed9c-493d-9d10-b1c5d64a4578 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:45.262726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b44d2d-ed9c-493d-9d10-b1c5d64a4578 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:45.289021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58354f-7416-4516-bc31-0217754dfe54 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:45.290293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58354f-7416-4516-bc31-0217754dfe54 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:45.290742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58354f-7416-4516-bc31-0217754dfe54 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:45.291649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58354f-7416-4516-bc31-0217754dfe54 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/regions/tempest-region-155505591` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:45.292693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58354f-7416-4516-bc31-0217754dfe54 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-155505591)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:45.296181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58354f-7416-4516-bc31-0217754dfe54 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:45.324598 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-1d58354f-7416-4516-bc31-0217754dfe54 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-155505591. To ensure interoperability, identifiers should be a valid uuid. Aug 20 21:11:45.324598 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Aug 20 21:11:45.324598 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:45.340428 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2745/5440] 10.222.0.23 () {58 vars in 1206 bytes} [Sat Aug 20 21:11:45 2022] DELETE /identity/v3/regions/tempest-region-155505591 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:45.341746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede420c-9759-45da-a9c5-a11601b6db68 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:45.353543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede420c-9759-45da-a9c5-a11601b6db68 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:45.379230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596386f-bd1f-4644-8f83-1c58466e5629 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:45.380478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596386f-bd1f-4644-8f83-1c58466e5629 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:45.380768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596386f-bd1f-4644-8f83-1c58466e5629 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:45.381176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596386f-bd1f-4644-8f83-1c58466e5629 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/services/d4e5228794be4d47a944b1fd75519409` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:45.381835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596386f-bd1f-4644-8f83-1c58466e5629 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:delete_service(service_id=d4e5228794be4d47a944b1fd75519409)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:45.383968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596386f-bd1f-4644-8f83-1c58466e5629 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:45.422205 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2696/5441] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:45 2022] DELETE /identity/v3/services/d4e5228794be4d47a944b1fd75519409 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:45.426083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58354f-7416-4516-bc31-0217754dfe54 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:45.441388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58354f-7416-4516-bc31-0217754dfe54 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:45.461904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb01eb-cf04-4c2c-8935-66d0aed0143d tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:45.462642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb01eb-cf04-4c2c-8935-66d0aed0143d tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:45.462984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb01eb-cf04-4c2c-8935-66d0aed0143d tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:45.463350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb01eb-cf04-4c2c-8935-66d0aed0143d tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/endpoints/32ee1034f0884582ac1cdb4b209d366c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:45.469634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb01eb-cf04-4c2c-8935-66d0aed0143d tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=32ee1034f0884582ac1cdb4b209d366c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:45.471803 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-22cb01eb-cf04-4c2c-8935-66d0aed0143d tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:45.471803 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:11:45.471803 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:45.472977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb01eb-cf04-4c2c-8935-66d0aed0143d tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:45.485227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-22cb01eb-cf04-4c2c-8935-66d0aed0143d tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '32ee1034f0884582ac1cdb4b209d366c', 'request_id': 'req-22cb01eb-cf04-4c2c-8935-66d0aed0143d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:45.499867 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2746/5442] 10.222.0.23 () {58 vars in 1236 bytes} [Sat Aug 20 21:11:45 2022] DELETE /identity/v3/endpoints/32ee1034f0884582ac1cdb4b209d366c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:45.504470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596386f-bd1f-4644-8f83-1c58466e5629 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:45.522432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596386f-bd1f-4644-8f83-1c58466e5629 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:45.548340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a426b69-ed84-4192-91eb-82139f9120a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:45.549535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a426b69-ed84-4192-91eb-82139f9120a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:45.549535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a426b69-ed84-4192-91eb-82139f9120a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:45.549925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a426b69-ed84-4192-91eb-82139f9120a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/regions/tempest-region-683764309` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:45.551236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a426b69-ed84-4192-91eb-82139f9120a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-683764309)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:45.553327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a426b69-ed84-4192-91eb-82139f9120a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:45.573008 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-0a426b69-ed84-4192-91eb-82139f9120a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-683764309. To ensure interoperability, identifiers should be a valid uuid. Aug 20 21:11:45.573008 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Aug 20 21:11:45.573008 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:45.582754 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2697/5443] 10.222.0.23 () {58 vars in 1206 bytes} [Sat Aug 20 21:11:45 2022] DELETE /identity/v3/regions/tempest-region-683764309 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:45.586740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb01eb-cf04-4c2c-8935-66d0aed0143d tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:45.607434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb01eb-cf04-4c2c-8935-66d0aed0143d tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:45.624856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50855d-06a1-4d64-b27b-8f42c403c124 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec27efbdcfa64eed9aa08aa518f6cc41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f749e0b3182a4b2f977b76d4580003fa', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:45.625595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50855d-06a1-4d64-b27b-8f42c403c124 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:45.625933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50855d-06a1-4d64-b27b-8f42c403c124 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:45.626269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50855d-06a1-4d64-b27b-8f42c403c124 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] PATH_INFO: `/v3/services/c26dc4dc75ef4fb3804a1cbdc72038ec` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:45.627302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50855d-06a1-4d64-b27b-8f42c403c124 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorizing `identity:delete_service(service_id=c26dc4dc75ef4fb3804a1cbdc72038ec)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:45.629010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50855d-06a1-4d64-b27b-8f42c403c124 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:45.664216 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2747/5444] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:45 2022] DELETE /identity/v3/services/c26dc4dc75ef4fb3804a1cbdc72038ec => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:45.668239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a426b69-ed84-4192-91eb-82139f9120a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:45.742426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:UPDATE=1,keystone:DELETE=8 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:45.776498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a426b69-ed84-4192-91eb-82139f9120a2 tempest-EndPointsTestJSON-1163866935 tempest-EndPointsTestJSON-1163866935-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:45.795585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:45.795585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:45.808814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:45.809260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] PATH_INFO: `/v3/users/2ff25e3225ce4b169fee2a8ff15e96d4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:45.811665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ff25e3225ce4b169fee2a8ff15e96d4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:45.813862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:45.891560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ff25e3225ce4b169fee2a8ff15e96d4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:45.901265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ff25e3225ce4b169fee2a8ff15e96d4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:45.919720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ff25e3225ce4b169fee2a8ff15e96d4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:45.921150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ff25e3225ce4b169fee2a8ff15e96d4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:45.922781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ff25e3225ce4b169fee2a8ff15e96d4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:45.945005 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2698/5445] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:45 2022] DELETE /identity/v3/users/2ff25e3225ce4b169fee2a8ff15e96d4 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:45.949861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82147067-fa27-4c30-b11f-4d7807ae9672 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:45.950739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82147067-fa27-4c30-b11f-4d7807ae9672 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:45.951333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82147067-fa27-4c30-b11f-4d7807ae9672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:45.987115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-82147067-fa27-4c30-b11f-4d7807ae9672 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:45.988826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-82147067-fa27-4c30-b11f-4d7807ae9672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:46.050588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:46.077908 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2748/5446] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:46.085101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:46.087124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:46.215677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b855da6-4bdb-463c-90c1-3a3195ced5dd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:46.220977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e3cde-8c3c-4803-a22d-2a9cce51c173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:46.221636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e3cde-8c3c-4803-a22d-2a9cce51c173 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:46.222137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e3cde-8c3c-4803-a22d-2a9cce51c173 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:46.222467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e3cde-8c3c-4803-a22d-2a9cce51c173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:46.235551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e3cde-8c3c-4803-a22d-2a9cce51c173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:46.237995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e3cde-8c3c-4803-a22d-2a9cce51c173 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:46.254104 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2699/5447] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:46.691273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82147067-fa27-4c30-b11f-4d7807ae9672 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:46.692818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-82147067-fa27-4c30-b11f-4d7807ae9672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:46.709986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82147067-fa27-4c30-b11f-4d7807ae9672 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:46.714808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:46.715531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:46.716003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:46.716335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] PATH_INFO: `/v3/projects/a841c0735d87429f863cc32827c79f3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:46.717685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a841c0735d87429f863cc32827c79f3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:46.719876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:46.805610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a841c0735d87429f863cc32827c79f3a', 'request_id': 'req-68ddda3f-f7ac-4655-9b02-2426a0a59a28'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:46.819484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a841c0735d87429f863cc32827c79f3a', 'request_id': 'req-68ddda3f-f7ac-4655-9b02-2426a0a59a28'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:46.828350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a841c0735d87429f863cc32827c79f3a', 'request_id': 'req-68ddda3f-f7ac-4655-9b02-2426a0a59a28'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:46.840553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] The token cache is being invalidate because project a841c0735d87429f863cc32827c79f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:46.841589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:46.841589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:46.843361 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2749/5448] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:46 2022] DELETE /identity/v3/projects/a841c0735d87429f863cc32827c79f3a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:46.847316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e3cde-8c3c-4803-a22d-2a9cce51c173 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:46.857784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e3cde-8c3c-4803-a22d-2a9cce51c173 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:46.912881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:46.936248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284582b3-2216-4f0e-943f-e819e316d580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:46.936928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284582b3-2216-4f0e-943f-e819e316d580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:46.937199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284582b3-2216-4f0e-943f-e819e316d580 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:46.937601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284582b3-2216-4f0e-943f-e819e316d580 admin admin] PATH_INFO: `/v3/users/ec27efbdcfa64eed9aa08aa518f6cc41` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:46.938720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284582b3-2216-4f0e-943f-e819e316d580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec27efbdcfa64eed9aa08aa518f6cc41)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:46.940937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284582b3-2216-4f0e-943f-e819e316d580 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:47.005658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-284582b3-2216-4f0e-943f-e819e316d580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec27efbdcfa64eed9aa08aa518f6cc41'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:47.022397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-284582b3-2216-4f0e-943f-e819e316d580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec27efbdcfa64eed9aa08aa518f6cc41'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:47.047108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-284582b3-2216-4f0e-943f-e819e316d580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec27efbdcfa64eed9aa08aa518f6cc41'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:47.048914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-284582b3-2216-4f0e-943f-e819e316d580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec27efbdcfa64eed9aa08aa518f6cc41'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:47.050319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-284582b3-2216-4f0e-943f-e819e316d580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec27efbdcfa64eed9aa08aa518f6cc41'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:47.080474 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2700/5449] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:46 2022] DELETE /identity/v3/users/ec27efbdcfa64eed9aa08aa518f6cc41 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:48.065867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:48.067503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:48.082072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddda3f-f7ac-4655-9b02-2426a0a59a28 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:48.170801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcef8ba-5837-4ad2-8bae-1f861125aff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:48.171593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcef8ba-5837-4ad2-8bae-1f861125aff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:48.171829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcef8ba-5837-4ad2-8bae-1f861125aff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:48.172097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcef8ba-5837-4ad2-8bae-1f861125aff4 admin admin] PATH_INFO: `/v3/projects/f749e0b3182a4b2f977b76d4580003fa` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:48.173249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcef8ba-5837-4ad2-8bae-1f861125aff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f749e0b3182a4b2f977b76d4580003fa)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:48.175297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcef8ba-5837-4ad2-8bae-1f861125aff4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:48.233620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bbcef8ba-5837-4ad2-8bae-1f861125aff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f749e0b3182a4b2f977b76d4580003fa', 'request_id': 'req-bbcef8ba-5837-4ad2-8bae-1f861125aff4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:48.245140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bbcef8ba-5837-4ad2-8bae-1f861125aff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f749e0b3182a4b2f977b76d4580003fa', 'request_id': 'req-bbcef8ba-5837-4ad2-8bae-1f861125aff4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:48.253978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bbcef8ba-5837-4ad2-8bae-1f861125aff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f749e0b3182a4b2f977b76d4580003fa', 'request_id': 'req-bbcef8ba-5837-4ad2-8bae-1f861125aff4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:48.275298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bbcef8ba-5837-4ad2-8bae-1f861125aff4 admin admin] The token cache is being invalidate because project f749e0b3182a4b2f977b76d4580003fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:48.275863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bbcef8ba-5837-4ad2-8bae-1f861125aff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:48.276643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bbcef8ba-5837-4ad2-8bae-1f861125aff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:48.278805 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2750/5450] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:48 2022] DELETE /identity/v3/projects/f749e0b3182a4b2f977b76d4580003fa => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:48.284663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d01f7-e903-4735-99b0-356efea1d1b3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:48.285231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d01f7-e903-4735-99b0-356efea1d1b3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:48.285231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d01f7-e903-4735-99b0-356efea1d1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:48.307621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9b2d01f7-e903-4735-99b0-356efea1d1b3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:48.309073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9b2d01f7-e903-4735-99b0-356efea1d1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:48.404825 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2701/5451] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:48.405698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d1f69-f12d-49fe-ba20-2e525e0f6c1e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:48.405979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d1f69-f12d-49fe-ba20-2e525e0f6c1e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:48.406273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d1f69-f12d-49fe-ba20-2e525e0f6c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:48.419695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:48.447230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-260d1f69-f12d-49fe-ba20-2e525e0f6c1e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:48.448727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-260d1f69-f12d-49fe-ba20-2e525e0f6c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:48.458089 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2751/5452] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:48.463730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d01f7-e903-4735-99b0-356efea1d1b3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:48.481683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d01f7-e903-4735-99b0-356efea1d1b3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:48.495191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6018e1c-0f55-4052-b0a1-f317adaedab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:48.495988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6018e1c-0f55-4052-b0a1-f317adaedab1 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:48.496359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6018e1c-0f55-4052-b0a1-f317adaedab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:48.496752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6018e1c-0f55-4052-b0a1-f317adaedab1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:48.497509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6018e1c-0f55-4052-b0a1-f317adaedab1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:48.497902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6018e1c-0f55-4052-b0a1-f317adaedab1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:48.499608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6018e1c-0f55-4052-b0a1-f317adaedab1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:48.518372 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2702/5453] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:48.519527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d1f69-f12d-49fe-ba20-2e525e0f6c1e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:48.536728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d1f69-f12d-49fe-ba20-2e525e0f6c1e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:48.543561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cb339-effb-4ae0-b5a0-dcb7a4890080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:48.544538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cb339-effb-4ae0-b5a0-dcb7a4890080 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:48.544947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cb339-effb-4ae0-b5a0-dcb7a4890080 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:48.545346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cb339-effb-4ae0-b5a0-dcb7a4890080 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:48.546125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cb339-effb-4ae0-b5a0-dcb7a4890080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:48.548191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cb339-effb-4ae0-b5a0-dcb7a4890080 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:48.620260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6018e1c-0f55-4052-b0a1-f317adaedab1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:48.636659 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2752/5454] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:48 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:48.640673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6018e1c-0f55-4052-b0a1-f317adaedab1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:48.719455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:48.755470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede27ed-e68d-4e3f-a189-788015fdd865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:48.756326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede27ed-e68d-4e3f-a189-788015fdd865 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:48.761966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede27ed-e68d-4e3f-a189-788015fdd865 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:48.761966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede27ed-e68d-4e3f-a189-788015fdd865 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:48.761966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede27ed-e68d-4e3f-a189-788015fdd865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:48.763715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede27ed-e68d-4e3f-a189-788015fdd865 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:48.799639 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2703/5455] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:48 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:48.801019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cb339-effb-4ae0-b5a0-dcb7a4890080 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:48.822864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cb339-effb-4ae0-b5a0-dcb7a4890080 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:48.836319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91020851-c210-4266-8694-4df2ca36f75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:48.837476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91020851-c210-4266-8694-4df2ca36f75d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:48.837758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91020851-c210-4266-8694-4df2ca36f75d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:48.838018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91020851-c210-4266-8694-4df2ca36f75d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:48.838712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91020851-c210-4266-8694-4df2ca36f75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:48.840686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91020851-c210-4266-8694-4df2ca36f75d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:48.885322 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2753/5456] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:48.889070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede27ed-e68d-4e3f-a189-788015fdd865 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:48.909059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede27ed-e68d-4e3f-a189-788015fdd865 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:48.922296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac16834-f7d8-477b-936f-46799b2ec883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:48.922925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac16834-f7d8-477b-936f-46799b2ec883 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:48.923204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac16834-f7d8-477b-936f-46799b2ec883 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:48.923512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac16834-f7d8-477b-936f-46799b2ec883 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:48.924506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac16834-f7d8-477b-936f-46799b2ec883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:48.929245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac16834-f7d8-477b-936f-46799b2ec883 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:48.956474 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2704/5457] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:48.957952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91020851-c210-4266-8694-4df2ca36f75d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:48.976149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91020851-c210-4266-8694-4df2ca36f75d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:48.998093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de65b0-c263-4c48-b19f-50c9de751239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:48.998996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de65b0-c263-4c48-b19f-50c9de751239 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:48.999445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de65b0-c263-4c48-b19f-50c9de751239 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:48.999887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de65b0-c263-4c48-b19f-50c9de751239 admin admin] PATH_INFO: `/v3/projects/77f13bae08d343d9a34677c5e2c83fcc/users/340832bba11b4110b2ba3e04158accf9/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.019052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de65b0-c263-4c48-b19f-50c9de751239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f13bae08d343d9a34677c5e2c83fcc, user_id=340832bba11b4110b2ba3e04158accf9, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:49.021787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de65b0-c263-4c48-b19f-50c9de751239 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:49.049021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f3e36-885d-4be9-9cc0-ce41a264a409 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.049424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f3e36-885d-4be9-9cc0-ce41a264a409 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.049763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f3e36-885d-4be9-9cc0-ce41a264a409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.055285 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2754/5458] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:48 2022] PUT /identity/v3/projects/77f13bae08d343d9a34677c5e2c83fcc/users/340832bba11b4110b2ba3e04158accf9/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:49.067361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228bb05-bee9-4404-95ff-61b72db159b4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.068036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228bb05-bee9-4404-95ff-61b72db159b4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.068036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228bb05-bee9-4404-95ff-61b72db159b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.098162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1228bb05-bee9-4404-95ff-61b72db159b4 None None] MFA Rules not processed for user `340832bba11b4110b2ba3e04158accf9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:49.099421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c69f3e36-885d-4be9-9cc0-ce41a264a409 None None] MFA Rules not processed for user `f127011ceab246a8834d50f01f426487`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:49.099574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1228bb05-bee9-4404-95ff-61b72db159b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:49.100801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c69f3e36-885d-4be9-9cc0-ce41a264a409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:49.198512 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2755/5459] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:49.199616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228bb05-bee9-4404-95ff-61b72db159b4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:49.261589 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2705/5460] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:49.262913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f3e36-885d-4be9-9cc0-ce41a264a409 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:49.264508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c69f3e36-885d-4be9-9cc0-ce41a264a409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:49.308758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228bb05-bee9-4404-95ff-61b72db159b4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:49.317168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baca59f-5a06-4d98-a5ea-a16f2c480f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:49.317945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baca59f-5a06-4d98-a5ea-a16f2c480f49 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.318403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baca59f-5a06-4d98-a5ea-a16f2c480f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.318767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baca59f-5a06-4d98-a5ea-a16f2c480f49 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.319502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baca59f-5a06-4d98-a5ea-a16f2c480f49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:49.322558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baca59f-5a06-4d98-a5ea-a16f2c480f49 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:49.364499 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2756/5461] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:49 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:49.368269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baca59f-5a06-4d98-a5ea-a16f2c480f49 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:49.386614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f3e36-885d-4be9-9cc0-ce41a264a409 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:49.388004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:49.451308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baca59f-5a06-4d98-a5ea-a16f2c480f49 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:49.454310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98533e7d-0c97-475c-8c07-4570f14ffc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:49.455197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98533e7d-0c97-475c-8c07-4570f14ffc48 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.455688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98533e7d-0c97-475c-8c07-4570f14ffc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.456132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98533e7d-0c97-475c-8c07-4570f14ffc48 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.456911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd9917-6270-4798-99f8-db622886d702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:49.457158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98533e7d-0c97-475c-8c07-4570f14ffc48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:49.457532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd9917-6270-4798-99f8-db622886d702 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.457805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd9917-6270-4798-99f8-db622886d702 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.458070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd9917-6270-4798-99f8-db622886d702 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.458684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd9917-6270-4798-99f8-db622886d702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:49.460639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd9917-6270-4798-99f8-db622886d702 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:49.460866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98533e7d-0c97-475c-8c07-4570f14ffc48 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:49.471659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:49.488934 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2757/5462] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:49 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:49.492702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd9917-6270-4798-99f8-db622886d702 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:49.501612 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2706/5463] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:49 2022] POST /identity/v3/projects => generated 375 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:49.504256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd9917-6270-4798-99f8-db622886d702 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:49.505488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98533e7d-0c97-475c-8c07-4570f14ffc48 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:49.577535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dddbc2-af7f-40ca-a265-b791e5cebc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:49.578303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dddbc2-af7f-40ca-a265-b791e5cebc88 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.578565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dddbc2-af7f-40ca-a265-b791e5cebc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.578834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dddbc2-af7f-40ca-a265-b791e5cebc88 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.579649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dddbc2-af7f-40ca-a265-b791e5cebc88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:49.581645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dddbc2-af7f-40ca-a265-b791e5cebc88 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:49.593954 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2758/5464] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:49.594169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98533e7d-0c97-475c-8c07-4570f14ffc48 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:49.598488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dddbc2-af7f-40ca-a265-b791e5cebc88 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:49.599564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91171826-e4d8-473e-8299-1107b338fbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:49.600009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91171826-e4d8-473e-8299-1107b338fbb9 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.600205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91171826-e4d8-473e-8299-1107b338fbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.600502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91171826-e4d8-473e-8299-1107b338fbb9 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.601111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91171826-e4d8-473e-8299-1107b338fbb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:49.603125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91171826-e4d8-473e-8299-1107b338fbb9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:49.620595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dddbc2-af7f-40ca-a265-b791e5cebc88 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:49.628614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05e365-dd1d-4838-b107-1a953109cfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:49.629792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05e365-dd1d-4838-b107-1a953109cfe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.630312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05e365-dd1d-4838-b107-1a953109cfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.630722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05e365-dd1d-4838-b107-1a953109cfe9 admin admin] PATH_INFO: `/v3/projects/5a1c210dabe94aa6961957f622f212ee/users/9aade25b2471481685d0cb1d96e0d136/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.635510 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2707/5465] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:49 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:49.636909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91171826-e4d8-473e-8299-1107b338fbb9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:49.650065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91171826-e4d8-473e-8299-1107b338fbb9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:49.652212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05e365-dd1d-4838-b107-1a953109cfe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1c210dabe94aa6961957f622f212ee, user_id=9aade25b2471481685d0cb1d96e0d136, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:49.655580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3975dd-469b-44a1-98aa-d3b710112683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:49.655731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3975dd-469b-44a1-98aa-d3b710112683 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.655989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3975dd-469b-44a1-98aa-d3b710112683 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.656261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3975dd-469b-44a1-98aa-d3b710112683 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.656945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3975dd-469b-44a1-98aa-d3b710112683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:49.658266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05e365-dd1d-4838-b107-1a953109cfe9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:49.659037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3975dd-469b-44a1-98aa-d3b710112683 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:49.672689 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2708/5466] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:49.676621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3975dd-469b-44a1-98aa-d3b710112683 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:49.677410 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2759/5467] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:49 2022] PUT /identity/v3/projects/5a1c210dabe94aa6961957f622f212ee/users/9aade25b2471481685d0cb1d96e0d136/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:49.681976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05e365-dd1d-4838-b107-1a953109cfe9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:49.690497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3975dd-469b-44a1-98aa-d3b710112683 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:49.765064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f055d-d009-44f9-bd88-1916678bf193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:49.765294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05e365-dd1d-4838-b107-1a953109cfe9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:49.765827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f055d-d009-44f9-bd88-1916678bf193 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.766111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f055d-d009-44f9-bd88-1916678bf193 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.766630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f055d-d009-44f9-bd88-1916678bf193 admin admin] PATH_INFO: `/v3/projects/582064e0834345e2a13aec61f9f5bee2/users/7cec2ab9334d4317bd17951c53ef2560/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.772849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7a6be-6473-41e2-b0ff-9e5ddb4f741c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:49.773739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7a6be-6473-41e2-b0ff-9e5ddb4f741c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.774091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7a6be-6473-41e2-b0ff-9e5ddb4f741c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.774431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7a6be-6473-41e2-b0ff-9e5ddb4f741c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.775244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7a6be-6473-41e2-b0ff-9e5ddb4f741c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:49.777238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7a6be-6473-41e2-b0ff-9e5ddb4f741c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:49.786118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f055d-d009-44f9-bd88-1916678bf193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=582064e0834345e2a13aec61f9f5bee2, user_id=7cec2ab9334d4317bd17951c53ef2560, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:49.792390 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2760/5468] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:49.792579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f055d-d009-44f9-bd88-1916678bf193 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:49.793550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7a6be-6473-41e2-b0ff-9e5ddb4f741c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:49.806491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7a6be-6473-41e2-b0ff-9e5ddb4f741c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:49.816215 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2709/5469] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:49 2022] PUT /identity/v3/projects/582064e0834345e2a13aec61f9f5bee2/users/7cec2ab9334d4317bd17951c53ef2560/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:49.817487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:49.818389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f055d-d009-44f9-bd88-1916678bf193 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:49.916058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:49.929607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8dfc3-34a6-43fa-9605-e4e732a4d87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:49.930590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8dfc3-34a6-43fa-9605-e4e732a4d87a admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.930866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8dfc3-34a6-43fa-9605-e4e732a4d87a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.931130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8dfc3-34a6-43fa-9605-e4e732a4d87a admin admin] PATH_INFO: `/v3/projects/5a1c210dabe94aa6961957f622f212ee/users/9aade25b2471481685d0cb1d96e0d136/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.931374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f055d-d009-44f9-bd88-1916678bf193 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:49.933195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8dfc3-34a6-43fa-9605-e4e732a4d87a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1c210dabe94aa6961957f622f212ee, user_id=9aade25b2471481685d0cb1d96e0d136, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:49.936138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8dfc3-34a6-43fa-9605-e4e732a4d87a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:49.937625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d925d-9ecf-4a02-a86e-6b0a693d5595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:49.938358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d925d-9ecf-4a02-a86e-6b0a693d5595 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.938785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d925d-9ecf-4a02-a86e-6b0a693d5595 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.939112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d925d-9ecf-4a02-a86e-6b0a693d5595 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.939858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d925d-9ecf-4a02-a86e-6b0a693d5595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:49.941594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d925d-9ecf-4a02-a86e-6b0a693d5595 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:49.951292 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2761/5470] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:49 2022] PUT /identity/v3/projects/5a1c210dabe94aa6961957f622f212ee/users/9aade25b2471481685d0cb1d96e0d136/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:49.966407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a5846-3785-4284-90a2-cdb4acfb0c78 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:49.966702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a5846-3785-4284-90a2-cdb4acfb0c78 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:49.977750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a5846-3785-4284-90a2-cdb4acfb0c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:49.998755 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2710/5471] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:49.999685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d925d-9ecf-4a02-a86e-6b0a693d5595 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:50.024005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d925d-9ecf-4a02-a86e-6b0a693d5595 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:50.084320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-415a5846-3785-4284-90a2-cdb4acfb0c78 None None] MFA Rules not processed for user `9aade25b2471481685d0cb1d96e0d136`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:50.088622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-415a5846-3785-4284-90a2-cdb4acfb0c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:50.142786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245701d4-a00c-4f30-9193-b3ff1cca8508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:50.143923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245701d4-a00c-4f30-9193-b3ff1cca8508 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:50.144691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245701d4-a00c-4f30-9193-b3ff1cca8508 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:50.145123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245701d4-a00c-4f30-9193-b3ff1cca8508 admin admin] PATH_INFO: `/v3/projects/582064e0834345e2a13aec61f9f5bee2/users/7cec2ab9334d4317bd17951c53ef2560/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:50.149038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245701d4-a00c-4f30-9193-b3ff1cca8508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=582064e0834345e2a13aec61f9f5bee2, user_id=7cec2ab9334d4317bd17951c53ef2560, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:50.153126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245701d4-a00c-4f30-9193-b3ff1cca8508 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:50.167517 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2711/5472] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:49 2022] PUT /identity/v3/projects/582064e0834345e2a13aec61f9f5bee2/users/7cec2ab9334d4317bd17951c53ef2560/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:50.256157 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2762/5473] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:50.257502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245701d4-a00c-4f30-9193-b3ff1cca8508 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:50.338918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245701d4-a00c-4f30-9193-b3ff1cca8508 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:50.343904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f01ae-36cd-49fc-971f-ab600d187f20 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:50.344551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f01ae-36cd-49fc-971f-ab600d187f20 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:50.344831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f01ae-36cd-49fc-971f-ab600d187f20 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:50.345103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f01ae-36cd-49fc-971f-ab600d187f20 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:50.345874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f01ae-36cd-49fc-971f-ab600d187f20 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:50.347744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f01ae-36cd-49fc-971f-ab600d187f20 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:50.375794 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2712/5474] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:50 2022] POST /identity/v3/services => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:50.380755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a5846-3785-4284-90a2-cdb4acfb0c78 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:50.393022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a5846-3785-4284-90a2-cdb4acfb0c78 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:50.414199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138208f4-6e61-4466-b3dc-8d22d8599863 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:50.415203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138208f4-6e61-4466-b3dc-8d22d8599863 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:50.415694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138208f4-6e61-4466-b3dc-8d22d8599863 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:50.416025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138208f4-6e61-4466-b3dc-8d22d8599863 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:50.416936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138208f4-6e61-4466-b3dc-8d22d8599863 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:50.419073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138208f4-6e61-4466-b3dc-8d22d8599863 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-138208f4-6e61-4466-b3dc-8d22d8599863 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: On instance['enabled']: Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: On instance['enabled']: Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: 'False' Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise error Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application On instance['enabled']: Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application 'False' Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.426390 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Aug 20 21:11:50.428864 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Aug 20 21:11:50.430191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Aug 20 21:11:50.430191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Aug 20 21:11:50.430191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.430191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:50.430191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:50.430191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.430191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application On instance['enabled']: Aug 20 21:11:50.430191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application 'False' Aug 20 21:11:50.430191 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.430191 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2763/5475] 10.222.0.23 () {60 vars in 1156 bytes} [Sat Aug 20 21:11:50 2022] POST /identity/v3/endpoints => generated 283 bytes in 48 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Aug 20 21:11:50.434814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f01ae-36cd-49fc-971f-ab600d187f20 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:50.450290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f01ae-36cd-49fc-971f-ab600d187f20 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:50.464957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f56d78-6767-4c20-931d-47dd0d1a07af tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:50.465629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f56d78-6767-4c20-931d-47dd0d1a07af tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:50.466095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f56d78-6767-4c20-931d-47dd0d1a07af tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:50.466397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f56d78-6767-4c20-931d-47dd0d1a07af tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:50.467205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f56d78-6767-4c20-931d-47dd0d1a07af tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:50.468952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f56d78-6767-4c20-931d-47dd0d1a07af tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-04f56d78-6767-4c20-931d-47dd0d1a07af tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: On instance['enabled']: Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: On instance['enabled']: Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: 'True' Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise error Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application On instance['enabled']: Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application 'True' Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:50.475733 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Aug 20 21:11:50.477043 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Aug 20 21:11:50.478278 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Aug 20 21:11:50.478278 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Aug 20 21:11:50.478278 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:50.478278 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:50.478278 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:50.478278 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:50.478278 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application On instance['enabled']: Aug 20 21:11:50.478278 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application 'True' Aug 20 21:11:50.478278 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:50.478278 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2713/5476] 10.222.0.23 () {60 vars in 1156 bytes} [Sat Aug 20 21:11:50 2022] POST /identity/v3/endpoints => generated 281 bytes in 43 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 20 21:11:50.485105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138208f4-6e61-4466-b3dc-8d22d8599863 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:50.500764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138208f4-6e61-4466-b3dc-8d22d8599863 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:50.507503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57aa54-c221-446f-9a85-64fa4b39455f tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:50.508711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57aa54-c221-446f-9a85-64fa4b39455f tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:50.508923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57aa54-c221-446f-9a85-64fa4b39455f tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:50.509157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57aa54-c221-446f-9a85-64fa4b39455f tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:50.509814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57aa54-c221-446f-9a85-64fa4b39455f tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:50.512009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57aa54-c221-446f-9a85-64fa4b39455f tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:50.549382 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-0f57aa54-c221-446f-9a85-64fa4b39455f tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1969321502. To ensure interoperability, identifiers should be a valid uuid. Aug 20 21:11:50.549382 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Aug 20 21:11:50.549382 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:50.611669 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2764/5477] 10.222.0.23 () {60 vars in 1156 bytes} [Sat Aug 20 21:11:50 2022] POST /identity/v3/endpoints => generated 361 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:50.616062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f56d78-6767-4c20-931d-47dd0d1a07af tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:50.638936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f56d78-6767-4c20-931d-47dd0d1a07af tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:50.668141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c007a-b33f-4e18-827a-6f90442776ab tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:50.668963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c007a-b33f-4e18-827a-6f90442776ab tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:50.669333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c007a-b33f-4e18-827a-6f90442776ab tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:50.669696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c007a-b33f-4e18-827a-6f90442776ab tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/regions/tempest-region-1969321502` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:50.670376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c007a-b33f-4e18-827a-6f90442776ab tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1969321502)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:50.673582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c007a-b33f-4e18-827a-6f90442776ab tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:50.677751 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2714/5478] 10.222.0.23 () {58 vars in 1206 bytes} [Sat Aug 20 21:11:50 2022] GET /identity/v3/regions/tempest-region-1969321502 => generated 180 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:50.681864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57aa54-c221-446f-9a85-64fa4b39455f tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:50.694282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57aa54-c221-446f-9a85-64fa4b39455f tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:50.699620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37632bcb-5870-455e-af2d-3fd5fb1083f7 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:50.700416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37632bcb-5870-455e-af2d-3fd5fb1083f7 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:50.700740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37632bcb-5870-455e-af2d-3fd5fb1083f7 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:50.701149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37632bcb-5870-455e-af2d-3fd5fb1083f7 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/endpoints/0e278ebded4a4de79a8b37dfbadd0c88` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:50.709098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37632bcb-5870-455e-af2d-3fd5fb1083f7 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=0e278ebded4a4de79a8b37dfbadd0c88)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:50.710419 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-37632bcb-5870-455e-af2d-3fd5fb1083f7 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:11:50.710419 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:11:50.710419 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:11:50.711611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37632bcb-5870-455e-af2d-3fd5fb1083f7 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-37632bcb-5870-455e-af2d-3fd5fb1083f7 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: On instance['enabled']: Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: On instance['enabled']: Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: 'False' Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise error Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application On instance['enabled']: Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application 'False' Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.720612 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Aug 20 21:11:50.722196 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Aug 20 21:11:50.723247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Aug 20 21:11:50.723247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Aug 20 21:11:50.723247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.723247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:50.723247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:50.723247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.723247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application On instance['enabled']: Aug 20 21:11:50.723247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application 'False' Aug 20 21:11:50.723247 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:50.729895 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2765/5479] 10.222.0.23 () {60 vars in 1255 bytes} [Sat Aug 20 21:11:50 2022] PATCH /identity/v3/endpoints/0e278ebded4a4de79a8b37dfbadd0c88 => generated 283 bytes in 49 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Aug 20 21:11:50.730654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c007a-b33f-4e18-827a-6f90442776ab tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:50.742658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c007a-b33f-4e18-827a-6f90442776ab tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:50.749262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b12465-68cd-459b-838e-9c735f10ee77 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:50.749906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b12465-68cd-459b-838e-9c735f10ee77 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:50.750232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b12465-68cd-459b-838e-9c735f10ee77 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:50.750730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b12465-68cd-459b-838e-9c735f10ee77 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/endpoints/0e278ebded4a4de79a8b37dfbadd0c88` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:50.751833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b12465-68cd-459b-838e-9c735f10ee77 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=0e278ebded4a4de79a8b37dfbadd0c88)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:50.754493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b12465-68cd-459b-838e-9c735f10ee77 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:50.767624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a4b12465-68cd-459b-838e-9c735f10ee77 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '0e278ebded4a4de79a8b37dfbadd0c88', 'request_id': 'req-a4b12465-68cd-459b-838e-9c735f10ee77'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:50.786138 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2715/5480] 10.222.0.23 () {58 vars in 1236 bytes} [Sat Aug 20 21:11:50 2022] DELETE /identity/v3/endpoints/0e278ebded4a4de79a8b37dfbadd0c88 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:50.788448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37632bcb-5870-455e-af2d-3fd5fb1083f7 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:50.803357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37632bcb-5870-455e-af2d-3fd5fb1083f7 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:50.819911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f3ecd-d494-4987-843b-3d4e3f757374 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:50.821031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f3ecd-d494-4987-843b-3d4e3f757374 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:50.821307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f3ecd-d494-4987-843b-3d4e3f757374 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:50.821572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f3ecd-d494-4987-843b-3d4e3f757374 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/regions/tempest-region-1969321502` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:50.822202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f3ecd-d494-4987-843b-3d4e3f757374 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1969321502)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:50.823821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f3ecd-d494-4987-843b-3d4e3f757374 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:50.845289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:50.865475 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2766/5481] 10.222.0.23 () {58 vars in 1209 bytes} [Sat Aug 20 21:11:50 2022] DELETE /identity/v3/regions/tempest-region-1969321502 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:50.867095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b12465-68cd-459b-838e-9c735f10ee77 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:50.884926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b12465-68cd-459b-838e-9c735f10ee77 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:50.922953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030a26b-ff6d-4a92-b85e-a7b946b46549 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:50.924023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030a26b-ff6d-4a92-b85e-a7b946b46549 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:50.926222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030a26b-ff6d-4a92-b85e-a7b946b46549 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:50.926852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030a26b-ff6d-4a92-b85e-a7b946b46549 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:50.927733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030a26b-ff6d-4a92-b85e-a7b946b46549 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:50.933817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030a26b-ff6d-4a92-b85e-a7b946b46549 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:51.001971 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-a030a26b-ff6d-4a92-b85e-a7b946b46549 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1288565278. To ensure interoperability, identifiers should be a valid uuid. Aug 20 21:11:51.001971 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Aug 20 21:11:51.001971 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:11:51.031686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:51.072939 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2716/5482] 10.222.0.23 () {60 vars in 1156 bytes} [Sat Aug 20 21:11:50 2022] POST /identity/v3/endpoints => generated 360 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:51.074226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f3ecd-d494-4987-843b-3d4e3f757374 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:51.096138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f3ecd-d494-4987-843b-3d4e3f757374 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:51.141512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca74c1-5da8-4832-a9c7-34a282124742 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:51.143441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca74c1-5da8-4832-a9c7-34a282124742 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:51.144417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca74c1-5da8-4832-a9c7-34a282124742 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:51.144841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca74c1-5da8-4832-a9c7-34a282124742 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/regions/tempest-region-1288565278` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:51.145761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca74c1-5da8-4832-a9c7-34a282124742 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1288565278)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:51.148612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca74c1-5da8-4832-a9c7-34a282124742 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:51.154713 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2767/5483] 10.222.0.23 () {58 vars in 1206 bytes} [Sat Aug 20 21:11:51 2022] GET /identity/v3/regions/tempest-region-1288565278 => generated 180 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:51.159412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030a26b-ff6d-4a92-b85e-a7b946b46549 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:51.184235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030a26b-ff6d-4a92-b85e-a7b946b46549 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:51.189356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88be705-c0e4-4b2e-8081-d1144aa44a2d tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:51.190301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88be705-c0e4-4b2e-8081-d1144aa44a2d tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:51.190417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88be705-c0e4-4b2e-8081-d1144aa44a2d tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:51.190898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88be705-c0e4-4b2e-8081-d1144aa44a2d tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/endpoints/d83e6f44c13245efa8c1d0c17fe02283` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:51.197540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88be705-c0e4-4b2e-8081-d1144aa44a2d tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=d83e6f44c13245efa8c1d0c17fe02283)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:51.199296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88be705-c0e4-4b2e-8081-d1144aa44a2d tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-c88be705-c0e4-4b2e-8081-d1144aa44a2d tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: On instance['enabled']: Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: On instance['enabled']: Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: 'True' Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise error Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application On instance['enabled']: Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application 'True' Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:51.208717 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Aug 20 21:11:51.210206 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Aug 20 21:11:51.211324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Aug 20 21:11:51.211324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Aug 20 21:11:51.211324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:51.211324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Aug 20 21:11:51.211324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Aug 20 21:11:51.211324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:51.211324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application On instance['enabled']: Aug 20 21:11:51.211324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application 'True' Aug 20 21:11:51.211324 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:51.213173 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2717/5484] 10.222.0.23 () {60 vars in 1255 bytes} [Sat Aug 20 21:11:51 2022] PATCH /identity/v3/endpoints/d83e6f44c13245efa8c1d0c17fe02283 => generated 281 bytes in 54 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Aug 20 21:11:51.217957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca74c1-5da8-4832-a9c7-34a282124742 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:51.252020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca74c1-5da8-4832-a9c7-34a282124742 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:51.264080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35bf7b-e69b-42e2-ab65-fead18ce4531 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:51.264769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35bf7b-e69b-42e2-ab65-fead18ce4531 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:51.265050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35bf7b-e69b-42e2-ab65-fead18ce4531 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:51.265317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35bf7b-e69b-42e2-ab65-fead18ce4531 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/endpoints/d83e6f44c13245efa8c1d0c17fe02283` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:51.266790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35bf7b-e69b-42e2-ab65-fead18ce4531 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=d83e6f44c13245efa8c1d0c17fe02283)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:51.269467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35bf7b-e69b-42e2-ab65-fead18ce4531 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:51.285151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cd35bf7b-e69b-42e2-ab65-fead18ce4531 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'd83e6f44c13245efa8c1d0c17fe02283', 'request_id': 'req-cd35bf7b-e69b-42e2-ab65-fead18ce4531'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:51.316095 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2768/5485] 10.222.0.23 () {58 vars in 1236 bytes} [Sat Aug 20 21:11:51 2022] DELETE /identity/v3/endpoints/d83e6f44c13245efa8c1d0c17fe02283 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:51.318037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88be705-c0e4-4b2e-8081-d1144aa44a2d tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:51.342410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88be705-c0e4-4b2e-8081-d1144aa44a2d tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:51.361096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821ee7-8814-43ae-ac0f-a0f92a4c5fda tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:51.361886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821ee7-8814-43ae-ac0f-a0f92a4c5fda tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:51.362254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821ee7-8814-43ae-ac0f-a0f92a4c5fda tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:51.362610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821ee7-8814-43ae-ac0f-a0f92a4c5fda tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/regions/tempest-region-1288565278` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:51.363327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9821ee7-8814-43ae-ac0f-a0f92a4c5fda tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1288565278)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:51.365395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9821ee7-8814-43ae-ac0f-a0f92a4c5fda tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:51.398166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35bf7b-e69b-42e2-ab65-fead18ce4531 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:51.399081 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2718/5486] 10.222.0.23 () {58 vars in 1209 bytes} [Sat Aug 20 21:11:51 2022] DELETE /identity/v3/regions/tempest-region-1288565278 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:51.412752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35bf7b-e69b-42e2-ab65-fead18ce4531 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:51.453447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd3c9f-02d6-41f7-a131-ca553ff05df1 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aade25b2471481685d0cb1d96e0d136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a1c210dabe94aa6961957f622f212ee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:51.454159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd3c9f-02d6-41f7-a131-ca553ff05df1 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:51.454378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd3c9f-02d6-41f7-a131-ca553ff05df1 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:51.454951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd3c9f-02d6-41f7-a131-ca553ff05df1 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] PATH_INFO: `/v3/services/6f7095bea5c94f36ab754596c3e93a50` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:51.455314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd3c9f-02d6-41f7-a131-ca553ff05df1 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorizing `identity:delete_service(service_id=6f7095bea5c94f36ab754596c3e93a50)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:51.457052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd3c9f-02d6-41f7-a131-ca553ff05df1 tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:51.487844 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2769/5487] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:51 2022] DELETE /identity/v3/services/6f7095bea5c94f36ab754596c3e93a50 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:51.491654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821ee7-8814-43ae-ac0f-a0f92a4c5fda tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:51.639850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821ee7-8814-43ae-ac0f-a0f92a4c5fda tempest-EndpointsNegativeTestJSON-46603696 tempest-EndpointsNegativeTestJSON-46603696-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:51.660358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:51.661215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:51.661727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:51.662686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] PATH_INFO: `/v3/users/340832bba11b4110b2ba3e04158accf9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:51.663512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=340832bba11b4110b2ba3e04158accf9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:51.665769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:51.779314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '340832bba11b4110b2ba3e04158accf9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:51.796885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '340832bba11b4110b2ba3e04158accf9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:51.825319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '340832bba11b4110b2ba3e04158accf9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:51.826686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '340832bba11b4110b2ba3e04158accf9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:51.828799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '340832bba11b4110b2ba3e04158accf9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:51.848637 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2719/5488] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:51 2022] DELETE /identity/v3/users/340832bba11b4110b2ba3e04158accf9 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:51.853049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bf319-9a31-44f4-8f07-74d9a8d0c7e6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:51.853316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bf319-9a31-44f4-8f07-74d9a8d0c7e6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:51.853594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bf319-9a31-44f4-8f07-74d9a8d0c7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:51.893040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-755bf319-9a31-44f4-8f07-74d9a8d0c7e6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:51.894350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-755bf319-9a31-44f4-8f07-74d9a8d0c7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:52.038653 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2770/5489] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:52.055660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:52.058138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:52.087440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=11 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:52.238698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffaa95-cc76-4bfb-857c-294bab5099fc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:52.243885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec52607-2d07-4bfe-a391-bb2568dab57c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:52.244531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec52607-2d07-4bfe-a391-bb2568dab57c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:52.244909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec52607-2d07-4bfe-a391-bb2568dab57c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:52.245276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec52607-2d07-4bfe-a391-bb2568dab57c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:52.265253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec52607-2d07-4bfe-a391-bb2568dab57c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:52.270457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec52607-2d07-4bfe-a391-bb2568dab57c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:52.309824 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2720/5490] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:53.076873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bf319-9a31-44f4-8f07-74d9a8d0c7e6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:53.081151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-755bf319-9a31-44f4-8f07-74d9a8d0c7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:53.134448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bf319-9a31-44f4-8f07-74d9a8d0c7e6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:53.157214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:53.158038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:53.158312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:53.158579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] PATH_INFO: `/v3/users/80d645a43ba249208f07968cd5930832` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:53.161107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80d645a43ba249208f07968cd5930832)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:53.163084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:53.251501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80d645a43ba249208f07968cd5930832'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.266801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80d645a43ba249208f07968cd5930832'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.278209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80d645a43ba249208f07968cd5930832'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.295076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80d645a43ba249208f07968cd5930832'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.298165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80d645a43ba249208f07968cd5930832'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.304600 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2771/5491] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:53 2022] DELETE /identity/v3/users/80d645a43ba249208f07968cd5930832 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:53.309809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570905c0-0a46-4e5a-8e23-ffb70ffb10ae None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:53.310266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570905c0-0a46-4e5a-8e23-ffb70ffb10ae None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:53.310684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570905c0-0a46-4e5a-8e23-ffb70ffb10ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:53.340637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-570905c0-0a46-4e5a-8e23-ffb70ffb10ae None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:53.341971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-570905c0-0a46-4e5a-8e23-ffb70ffb10ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:53.435659 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2721/5492] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:53.457008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:53.465842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:53.480438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:53.487830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570905c0-0a46-4e5a-8e23-ffb70ffb10ae None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:53.490026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-570905c0-0a46-4e5a-8e23-ffb70ffb10ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:53.511205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570905c0-0a46-4e5a-8e23-ffb70ffb10ae None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:53.527398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:53.528131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:53.528471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:53.528731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] PATH_INFO: `/v3/projects/77f13bae08d343d9a34677c5e2c83fcc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:53.530041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] RBAC: Authorizing `identity:delete_project(project_id=77f13bae08d343d9a34677c5e2c83fcc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:53.532560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:53.576201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1c214-fc8f-4c04-8663-4726591607a0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:53.582576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8e1c1-cadd-4abb-ba33-a13e02c60205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:53.583585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8e1c1-cadd-4abb-ba33-a13e02c60205 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:53.584288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8e1c1-cadd-4abb-ba33-a13e02c60205 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:53.584602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8e1c1-cadd-4abb-ba33-a13e02c60205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:53.597433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77f13bae08d343d9a34677c5e2c83fcc', 'request_id': 'req-60b91549-ffa9-4209-b467-105bed61a27d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.604533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77f13bae08d343d9a34677c5e2c83fcc', 'request_id': 'req-60b91549-ffa9-4209-b467-105bed61a27d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.604787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8e1c1-cadd-4abb-ba33-a13e02c60205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:53.610918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8e1c1-cadd-4abb-ba33-a13e02c60205 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:53.617479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77f13bae08d343d9a34677c5e2c83fcc', 'request_id': 'req-60b91549-ffa9-4209-b467-105bed61a27d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.628243 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2772/5493] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:53.650376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] The token cache is being invalidate because project 77f13bae08d343d9a34677c5e2c83fcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:53.651162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.651917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.657140 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2722/5494] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:53 2022] DELETE /identity/v3/projects/77f13bae08d343d9a34677c5e2c83fcc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:53.658749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8e1c1-cadd-4abb-ba33-a13e02c60205 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:53.670585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8e1c1-cadd-4abb-ba33-a13e02c60205 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:53.753768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:53.754756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:53.755129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:53.755486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] PATH_INFO: `/v3/users/9aade25b2471481685d0cb1d96e0d136` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:53.756764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aade25b2471481685d0cb1d96e0d136)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:53.758970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:53.832807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aade25b2471481685d0cb1d96e0d136'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.844161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aade25b2471481685d0cb1d96e0d136'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.853776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aade25b2471481685d0cb1d96e0d136'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.868200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aade25b2471481685d0cb1d96e0d136'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.869284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aade25b2471481685d0cb1d96e0d136'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:53.882603 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2773/5495] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:53 2022] DELETE /identity/v3/users/9aade25b2471481685d0cb1d96e0d136 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:53.981861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:53.983320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:54.006270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b91549-ffa9-4209-b467-105bed61a27d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:54.048879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:54.101634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:54.101953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:54.102038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:54.102297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] PATH_INFO: `/v3/projects/c21663600eb34db7b534db8a33e15344` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:54.103522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c21663600eb34db7b534db8a33e15344)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:54.105561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:54.216217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c21663600eb34db7b534db8a33e15344', 'request_id': 'req-08f45f36-237a-4a61-9d87-093bda4a4353'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.246810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c21663600eb34db7b534db8a33e15344', 'request_id': 'req-08f45f36-237a-4a61-9d87-093bda4a4353'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.292393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c21663600eb34db7b534db8a33e15344', 'request_id': 'req-08f45f36-237a-4a61-9d87-093bda4a4353'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.310887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] The token cache is being invalidate because project c21663600eb34db7b534db8a33e15344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:54.312688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.313596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.316354 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2723/5496] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:53 2022] DELETE /identity/v3/projects/c21663600eb34db7b534db8a33e15344 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:54.326272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:54.328313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:54.354914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0df61-6bca-469a-81e9-83e29692cfaa admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:54.440714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:54.511945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b94e14-cd95-4bca-8f5a-9c9d32fc47df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:54.512626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b94e14-cd95-4bca-8f5a-9c9d32fc47df admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:54.512903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b94e14-cd95-4bca-8f5a-9c9d32fc47df admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:54.513164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b94e14-cd95-4bca-8f5a-9c9d32fc47df admin admin] PATH_INFO: `/v3/users/c16e351e92be4a18bcd02a937d051457` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:54.521473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b94e14-cd95-4bca-8f5a-9c9d32fc47df admin admin] RBAC: Authorizing `identity:delete_user(user_id=c16e351e92be4a18bcd02a937d051457)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:54.524544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b94e14-cd95-4bca-8f5a-9c9d32fc47df admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:54.582391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:54.584106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:54.609163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f45f36-237a-4a61-9d87-093bda4a4353 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:54.615481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d48178-7aba-4a56-be2b-b9f5d6088a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:54.616142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d48178-7aba-4a56-be2b-b9f5d6088a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:54.616450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d48178-7aba-4a56-be2b-b9f5d6088a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:54.616725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d48178-7aba-4a56-be2b-b9f5d6088a94 admin admin] PATH_INFO: `/v3/projects/5a1c210dabe94aa6961957f622f212ee` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:54.617945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d48178-7aba-4a56-be2b-b9f5d6088a94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1c210dabe94aa6961957f622f212ee)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:54.619882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d48178-7aba-4a56-be2b-b9f5d6088a94 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:54.629342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8b94e14-cd95-4bca-8f5a-9c9d32fc47df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c16e351e92be4a18bcd02a937d051457'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.642036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8b94e14-cd95-4bca-8f5a-9c9d32fc47df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c16e351e92be4a18bcd02a937d051457'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.654575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8b94e14-cd95-4bca-8f5a-9c9d32fc47df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c16e351e92be4a18bcd02a937d051457'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.675619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8b94e14-cd95-4bca-8f5a-9c9d32fc47df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c16e351e92be4a18bcd02a937d051457'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.676915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c8b94e14-cd95-4bca-8f5a-9c9d32fc47df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c16e351e92be4a18bcd02a937d051457'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.683562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-97d48178-7aba-4a56-be2b-b9f5d6088a94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1c210dabe94aa6961957f622f212ee', 'request_id': 'req-97d48178-7aba-4a56-be2b-b9f5d6088a94'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.685411 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2774/5497] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:54 2022] DELETE /identity/v3/users/c16e351e92be4a18bcd02a937d051457 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:54.694501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-97d48178-7aba-4a56-be2b-b9f5d6088a94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1c210dabe94aa6961957f622f212ee', 'request_id': 'req-97d48178-7aba-4a56-be2b-b9f5d6088a94'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.712829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-97d48178-7aba-4a56-be2b-b9f5d6088a94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1c210dabe94aa6961957f622f212ee', 'request_id': 'req-97d48178-7aba-4a56-be2b-b9f5d6088a94'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.728047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-97d48178-7aba-4a56-be2b-b9f5d6088a94 admin admin] The token cache is being invalidate because project 5a1c210dabe94aa6961957f622f212ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:54.728334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-97d48178-7aba-4a56-be2b-b9f5d6088a94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.729096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-97d48178-7aba-4a56-be2b-b9f5d6088a94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:54.731091 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2724/5498] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:54 2022] DELETE /identity/v3/projects/5a1c210dabe94aa6961957f622f212ee => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:54.735918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f76300-7ede-4216-ad45-90e41969d35a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:54.736275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f76300-7ede-4216-ad45-90e41969d35a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:54.736644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f76300-7ede-4216-ad45-90e41969d35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:54.771303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d1f76300-7ede-4216-ad45-90e41969d35a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:54.774562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d1f76300-7ede-4216-ad45-90e41969d35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:54.866080 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2775/5499] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:54.874557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead6aed-cfc7-407c-9f00-328d36fffb4f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:54.874919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead6aed-cfc7-407c-9f00-328d36fffb4f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:54.875269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead6aed-cfc7-407c-9f00-328d36fffb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:54.910158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bead6aed-cfc7-407c-9f00-328d36fffb4f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:54.915616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bead6aed-cfc7-407c-9f00-328d36fffb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:54.929940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f76300-7ede-4216-ad45-90e41969d35a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:54.937011 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2725/5500] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:54.937987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead6aed-cfc7-407c-9f00-328d36fffb4f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:54.940840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bead6aed-cfc7-407c-9f00-328d36fffb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:54.946505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f76300-7ede-4216-ad45-90e41969d35a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:54.957340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089869c-2437-4c78-8334-334eef843aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:54.957973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089869c-2437-4c78-8334-334eef843aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:54.958248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089869c-2437-4c78-8334-334eef843aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:54.958512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089869c-2437-4c78-8334-334eef843aa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:54.959170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089869c-2437-4c78-8334-334eef843aa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:54.959605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089869c-2437-4c78-8334-334eef843aa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:54.961369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089869c-2437-4c78-8334-334eef843aa8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:54.968858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead6aed-cfc7-407c-9f00-328d36fffb4f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:54.977040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:54.977950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:54.978272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:54.978539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458 admin admin] PATH_INFO: `/v3/projects/654136bb95e84a238901b4f282ef64de` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:54.979841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=654136bb95e84a238901b4f282ef64de)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:54.982059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:54.983170 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2776/5501] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:54.984151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089869c-2437-4c78-8334-334eef843aa8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:55.025788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089869c-2437-4c78-8334-334eef843aa8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:55.031835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe01b0-7fff-4671-a6c6-71d213be9d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:55.032650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe01b0-7fff-4671-a6c6-71d213be9d2c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.032859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe01b0-7fff-4671-a6c6-71d213be9d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.033115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe01b0-7fff-4671-a6c6-71d213be9d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.033750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe01b0-7fff-4671-a6c6-71d213be9d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:55.036151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe01b0-7fff-4671-a6c6-71d213be9d2c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:55.045081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:55.064513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654136bb95e84a238901b4f282ef64de', 'request_id': 'req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:55.072936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654136bb95e84a238901b4f282ef64de', 'request_id': 'req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:55.083686 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2777/5502] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:54 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:55.086522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe01b0-7fff-4671-a6c6-71d213be9d2c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:55.089087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654136bb95e84a238901b4f282ef64de', 'request_id': 'req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:55.103491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:55.110167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458 admin admin] The token cache is being invalidate because project 654136bb95e84a238901b4f282ef64de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:55.111390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:55.113043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4dc1eaf3-3dfc-48fb-9a86-7a6991fea458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:55.115665 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2726/5503] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:54 2022] DELETE /identity/v3/projects/654136bb95e84a238901b4f282ef64de => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:55.121871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4126c4-e2b1-4955-8c4c-9be51d479fce None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.122283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4126c4-e2b1-4955-8c4c-9be51d479fce None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.122751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4126c4-e2b1-4955-8c4c-9be51d479fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.152106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0d4126c4-e2b1-4955-8c4c-9be51d479fce None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:55.153802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0d4126c4-e2b1-4955-8c4c-9be51d479fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:55.196025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe01b0-7fff-4671-a6c6-71d213be9d2c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:55.205902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecde35-48a2-48df-a46a-a0be8cb0f2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:55.207751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecde35-48a2-48df-a46a-a0be8cb0f2ee admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.208038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecde35-48a2-48df-a46a-a0be8cb0f2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.208314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecde35-48a2-48df-a46a-a0be8cb0f2ee admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.209551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecde35-48a2-48df-a46a-a0be8cb0f2ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:55.213760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecde35-48a2-48df-a46a-a0be8cb0f2ee admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:55.240762 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2727/5504] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:55.250385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada2846-8e96-48aa-9051-679d1c4ed2bb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.250822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada2846-8e96-48aa-9051-679d1c4ed2bb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.251270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada2846-8e96-48aa-9051-679d1c4ed2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.259357 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2778/5505] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:55 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:55.262200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecde35-48a2-48df-a46a-a0be8cb0f2ee admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:55.265292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-52ecde35-48a2-48df-a46a-a0be8cb0f2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:55.279146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1ada2846-8e96-48aa-9051-679d1c4ed2bb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:55.280498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1ada2846-8e96-48aa-9051-679d1c4ed2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:55.292709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecde35-48a2-48df-a46a-a0be8cb0f2ee admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:55.296364 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2728/5506] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:55.301097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada2846-8e96-48aa-9051-679d1c4ed2bb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:55.301482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01623693-e5d0-46d1-a669-50753170fbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:55.302174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01623693-e5d0-46d1-a669-50753170fbc5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.302511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01623693-e5d0-46d1-a669-50753170fbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.302840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01623693-e5d0-46d1-a669-50753170fbc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.303614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01623693-e5d0-46d1-a669-50753170fbc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:55.305479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01623693-e5d0-46d1-a669-50753170fbc5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:55.316221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada2846-8e96-48aa-9051-679d1c4ed2bb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:55.320420 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2779/5507] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:55.323960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01623693-e5d0-46d1-a669-50753170fbc5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:55.330862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9bf8b-bcc9-402b-bb91-da03e922e118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:55.331967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9bf8b-bcc9-402b-bb91-da03e922e118 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.332474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9bf8b-bcc9-402b-bb91-da03e922e118 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.332879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9bf8b-bcc9-402b-bb91-da03e922e118 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.333663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9bf8b-bcc9-402b-bb91-da03e922e118 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:55.334097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9bf8b-bcc9-402b-bb91-da03e922e118 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:55.336555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9bf8b-bcc9-402b-bb91-da03e922e118 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:55.351429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01623693-e5d0-46d1-a669-50753170fbc5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:55.355533 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2729/5508] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:11:55.361745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9bf8b-bcc9-402b-bb91-da03e922e118 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:55.369182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2c8fb-cb29-41e9-9960-33740ef78eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:55.369956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c8fb-cb29-41e9-9960-33740ef78eae admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.371894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c8fb-cb29-41e9-9960-33740ef78eae admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.373659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c8fb-cb29-41e9-9960-33740ef78eae admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.374630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2c8fb-cb29-41e9-9960-33740ef78eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:55.386138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2c8fb-cb29-41e9-9960-33740ef78eae admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:55.388252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9bf8b-bcc9-402b-bb91-da03e922e118 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:55.393807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38a39c-b774-4bda-aafa-a7552a2666bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:55.394436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38a39c-b774-4bda-aafa-a7552a2666bc admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.394723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38a39c-b774-4bda-aafa-a7552a2666bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.394982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38a39c-b774-4bda-aafa-a7552a2666bc admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.395611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38a39c-b774-4bda-aafa-a7552a2666bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:55.397627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38a39c-b774-4bda-aafa-a7552a2666bc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:55.419045 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2780/5509] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:11:55.422017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2c8fb-cb29-41e9-9960-33740ef78eae admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:55.436771 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2730/5510] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:55 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:55.438245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2c8fb-cb29-41e9-9960-33740ef78eae admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:55.441564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38a39c-b774-4bda-aafa-a7552a2666bc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:55.551700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2c24e-a8d0-44e8-93bb-957ca59cc99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:55.552244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2c24e-a8d0-44e8-93bb-957ca59cc99a admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.552553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2c24e-a8d0-44e8-93bb-957ca59cc99a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.552829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2c24e-a8d0-44e8-93bb-957ca59cc99a admin admin] PATH_INFO: `/v3/projects/9b8032359adf4128960906a9f1d865cf/users/14c095b880c14cbbbacd989bd8411995/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.563249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38a39c-b774-4bda-aafa-a7552a2666bc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:55.568818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2501a2-611c-4ea4-b7ae-a849bae3269a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:55.569947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2501a2-611c-4ea4-b7ae-a849bae3269a admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.570682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2501a2-611c-4ea4-b7ae-a849bae3269a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.571044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2501a2-611c-4ea4-b7ae-a849bae3269a admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.573694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2501a2-611c-4ea4-b7ae-a849bae3269a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:55.576066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2501a2-611c-4ea4-b7ae-a849bae3269a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:55.587292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2c24e-a8d0-44e8-93bb-957ca59cc99a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b8032359adf4128960906a9f1d865cf, user_id=14c095b880c14cbbbacd989bd8411995, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:55.590352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2c24e-a8d0-44e8-93bb-957ca59cc99a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:55.618706 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2781/5511] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:55 2022] PUT /identity/v3/projects/9b8032359adf4128960906a9f1d865cf/users/14c095b880c14cbbbacd989bd8411995/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:55.622541 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2731/5512] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:55 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:11:55.638572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2501a2-611c-4ea4-b7ae-a849bae3269a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:55.639485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d785ce-50df-41fb-a474-8d6ac10c50bc None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.639922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d785ce-50df-41fb-a474-8d6ac10c50bc None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.640765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d785ce-50df-41fb-a474-8d6ac10c50bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.652761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2501a2-611c-4ea4-b7ae-a849bae3269a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:55.674714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:55.699885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-04d785ce-50df-41fb-a474-8d6ac10c50bc None None] MFA Rules not processed for user `14c095b880c14cbbbacd989bd8411995`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:55.701271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-04d785ce-50df-41fb-a474-8d6ac10c50bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:55.765551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0203045-57db-42e1-9ffc-b7df80f41b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:55.766427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0203045-57db-42e1-9ffc-b7df80f41b64 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.766711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0203045-57db-42e1-9ffc-b7df80f41b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.767043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0203045-57db-42e1-9ffc-b7df80f41b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.768704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:55.770688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0203045-57db-42e1-9ffc-b7df80f41b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:55.772362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0203045-57db-42e1-9ffc-b7df80f41b64 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:55.792494 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2732/5513] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:55.796462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0203045-57db-42e1-9ffc-b7df80f41b64 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:55.798001 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2782/5514] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:55.802798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d785ce-50df-41fb-a474-8d6ac10c50bc None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:55.814565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0203045-57db-42e1-9ffc-b7df80f41b64 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:55.816245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d785ce-50df-41fb-a474-8d6ac10c50bc None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:55.822443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824448d-f4c9-46a1-8d09-6a2715cf84f0 tempest-DefaultDomainTestJSON-1130610530 tempest-DefaultDomainTestJSON-1130610530-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14c095b880c14cbbbacd989bd8411995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b8032359adf4128960906a9f1d865cf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:55.823361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824448d-f4c9-46a1-8d09-6a2715cf84f0 tempest-DefaultDomainTestJSON-1130610530 tempest-DefaultDomainTestJSON-1130610530-project] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.823854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824448d-f4c9-46a1-8d09-6a2715cf84f0 tempest-DefaultDomainTestJSON-1130610530 tempest-DefaultDomainTestJSON-1130610530-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.824309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824448d-f4c9-46a1-8d09-6a2715cf84f0 tempest-DefaultDomainTestJSON-1130610530 tempest-DefaultDomainTestJSON-1130610530-project] PATH_INFO: `/v3/domains/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.826315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824448d-f4c9-46a1-8d09-6a2715cf84f0 tempest-DefaultDomainTestJSON-1130610530 tempest-DefaultDomainTestJSON-1130610530-project] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:55.829033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824448d-f4c9-46a1-8d09-6a2715cf84f0 tempest-DefaultDomainTestJSON-1130610530 tempest-DefaultDomainTestJSON-1130610530-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:55.830475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f873e72-1f86-43eb-8fe3-76f36059f329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:55.831152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f873e72-1f86-43eb-8fe3-76f36059f329 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.831433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f873e72-1f86-43eb-8fe3-76f36059f329 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.831754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f873e72-1f86-43eb-8fe3-76f36059f329 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.832365 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2783/5515] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Aug 20 21:11:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:55.837206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824448d-f4c9-46a1-8d09-6a2715cf84f0 tempest-DefaultDomainTestJSON-1130610530 tempest-DefaultDomainTestJSON-1130610530-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:55.837637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f873e72-1f86-43eb-8fe3-76f36059f329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:55.839353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f873e72-1f86-43eb-8fe3-76f36059f329 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:55.852397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824448d-f4c9-46a1-8d09-6a2715cf84f0 tempest-DefaultDomainTestJSON-1130610530 tempest-DefaultDomainTestJSON-1130610530-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:55.859018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd7bc9-c605-480a-88b3-6ba3c7f66a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:55.859917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd7bc9-c605-480a-88b3-6ba3c7f66a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.860405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd7bc9-c605-480a-88b3-6ba3c7f66a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.861119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd7bc9-c605-480a-88b3-6ba3c7f66a9e admin admin] PATH_INFO: `/v3/users/14c095b880c14cbbbacd989bd8411995` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.862601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd7bc9-c605-480a-88b3-6ba3c7f66a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=14c095b880c14cbbbacd989bd8411995)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:55.864846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd7bc9-c605-480a-88b3-6ba3c7f66a9e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:55.885060 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2733/5516] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:55.891695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f873e72-1f86-43eb-8fe3-76f36059f329 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:55.918331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f873e72-1f86-43eb-8fe3-76f36059f329 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:55.929049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289986a5-f4d9-4543-924e-0e3db6fe71d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:55.929769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289986a5-f4d9-4543-924e-0e3db6fe71d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:55.930964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289986a5-f4d9-4543-924e-0e3db6fe71d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:55.932314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289986a5-f4d9-4543-924e-0e3db6fe71d6 admin admin] PATH_INFO: `/v3/projects/5f16924ecb4e466e94a815b44528130d/users/549ccd150d7b4a8a882e6fbda33129ea/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:55.939268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-40bd7bc9-c605-480a-88b3-6ba3c7f66a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14c095b880c14cbbbacd989bd8411995'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:55.958677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-40bd7bc9-c605-480a-88b3-6ba3c7f66a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14c095b880c14cbbbacd989bd8411995'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:55.958802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289986a5-f4d9-4543-924e-0e3db6fe71d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f16924ecb4e466e94a815b44528130d, user_id=549ccd150d7b4a8a882e6fbda33129ea, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:55.961824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289986a5-f4d9-4543-924e-0e3db6fe71d6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:55.979116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-40bd7bc9-c605-480a-88b3-6ba3c7f66a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14c095b880c14cbbbacd989bd8411995'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:55.993006 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2734/5517] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:55 2022] PUT /identity/v3/projects/5f16924ecb4e466e94a815b44528130d/users/549ccd150d7b4a8a882e6fbda33129ea/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:55.998714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-40bd7bc9-c605-480a-88b3-6ba3c7f66a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14c095b880c14cbbbacd989bd8411995'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:55.999009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-40bd7bc9-c605-480a-88b3-6ba3c7f66a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14c095b880c14cbbbacd989bd8411995'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:56.000490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b993c6f-fae2-4e06-abbc-31ad831cadbb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:56.000490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b993c6f-fae2-4e06-abbc-31ad831cadbb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:56.000637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b993c6f-fae2-4e06-abbc-31ad831cadbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:56.018458 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2784/5518] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:11:55 2022] DELETE /identity/v3/users/14c095b880c14cbbbacd989bd8411995 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:56.024504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33186720-ee41-4d61-ae08-64cba72a2440 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:56.025072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33186720-ee41-4d61-ae08-64cba72a2440 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:56.026731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33186720-ee41-4d61-ae08-64cba72a2440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:56.049200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8b993c6f-fae2-4e06-abbc-31ad831cadbb None None] MFA Rules not processed for user `549ccd150d7b4a8a882e6fbda33129ea`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:56.050802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8b993c6f-fae2-4e06-abbc-31ad831cadbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:56.058989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-33186720-ee41-4d61-ae08-64cba72a2440 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:56.060924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-33186720-ee41-4d61-ae08-64cba72a2440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:56.156418 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2785/5519] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:56.166916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33186720-ee41-4d61-ae08-64cba72a2440 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:56.170171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-33186720-ee41-4d61-ae08-64cba72a2440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:56.220930 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2735/5520] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:56.233502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b993c6f-fae2-4e06-abbc-31ad831cadbb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:56.236077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8b993c6f-fae2-4e06-abbc-31ad831cadbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:56.298390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33186720-ee41-4d61-ae08-64cba72a2440 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:56.302757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a826503-511d-471a-a3a9-f2e562276816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:56.303518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a826503-511d-471a-a3a9-f2e562276816 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:56.304311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a826503-511d-471a-a3a9-f2e562276816 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:56.304756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a826503-511d-471a-a3a9-f2e562276816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:56.317396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a826503-511d-471a-a3a9-f2e562276816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:56.319462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a826503-511d-471a-a3a9-f2e562276816 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:56.328292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b993c6f-fae2-4e06-abbc-31ad831cadbb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:56.333818 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2786/5521] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:56.340859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fafb13-cccc-4ba5-b798-ed896af3f741 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:56.342186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fafb13-cccc-4ba5-b798-ed896af3f741 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:56.342569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fafb13-cccc-4ba5-b798-ed896af3f741 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:56.343102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fafb13-cccc-4ba5-b798-ed896af3f741 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:56.356136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fafb13-cccc-4ba5-b798-ed896af3f741 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:56.358408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fafb13-cccc-4ba5-b798-ed896af3f741 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:56.401824 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2736/5522] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:56.745548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a13fd0-a21c-4988-a5f3-967069234855 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:56.746105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a13fd0-a21c-4988-a5f3-967069234855 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:56.746105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a13fd0-a21c-4988-a5f3-967069234855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:56.755284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:56.812128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-45a13fd0-a21c-4988-a5f3-967069234855 None None] MFA Rules not processed for user `7cec2ab9334d4317bd17951c53ef2560`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:56.813636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-45a13fd0-a21c-4988-a5f3-967069234855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:56.914153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fafb13-cccc-4ba5-b798-ed896af3f741 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:56.918655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-01fafb13-cccc-4ba5-b798-ed896af3f741 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:56.924709 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2787/5523] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:56.934114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a13fd0-a21c-4988-a5f3-967069234855 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:56.935985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-45a13fd0-a21c-4988-a5f3-967069234855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:56.958950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:56.972986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fafb13-cccc-4ba5-b798-ed896af3f741 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:56.981417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b7962-d922-4166-997b-3abca0a14741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:56.982068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b7962-d922-4166-997b-3abca0a14741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:56.982337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b7962-d922-4166-997b-3abca0a14741 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:56.983089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b7962-d922-4166-997b-3abca0a14741 admin admin] PATH_INFO: `/v3/projects/9b8032359adf4128960906a9f1d865cf` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:56.984383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b7962-d922-4166-997b-3abca0a14741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b8032359adf4128960906a9f1d865cf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:56.987384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b7962-d922-4166-997b-3abca0a14741 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:57.086369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0f5b7962-d922-4166-997b-3abca0a14741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b8032359adf4128960906a9f1d865cf', 'request_id': 'req-0f5b7962-d922-4166-997b-3abca0a14741'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:57.122317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0f5b7962-d922-4166-997b-3abca0a14741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b8032359adf4128960906a9f1d865cf', 'request_id': 'req-0f5b7962-d922-4166-997b-3abca0a14741'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:57.146083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0f5b7962-d922-4166-997b-3abca0a14741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b8032359adf4128960906a9f1d865cf', 'request_id': 'req-0f5b7962-d922-4166-997b-3abca0a14741'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:57.166398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a13fd0-a21c-4988-a5f3-967069234855 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:57.182648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0f5b7962-d922-4166-997b-3abca0a14741 admin admin] The token cache is being invalidate because project 9b8032359adf4128960906a9f1d865cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:11:57.183326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0f5b7962-d922-4166-997b-3abca0a14741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:57.186561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0f5b7962-d922-4166-997b-3abca0a14741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:57.189303 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2737/5524] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:11:56 2022] DELETE /identity/v3/projects/9b8032359adf4128960906a9f1d865cf => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:57.195841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe9f15-34fc-42f7-86f4-2c131877ff09 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:57.196128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe9f15-34fc-42f7-86f4-2c131877ff09 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:57.196401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe9f15-34fc-42f7-86f4-2c131877ff09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:57.259485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-91fe9f15-34fc-42f7-86f4-2c131877ff09 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:57.264604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-91fe9f15-34fc-42f7-86f4-2c131877ff09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:57.335471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d43b0b-a995-4d88-9dc0-b20795b882ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:57.336846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d43b0b-a995-4d88-9dc0-b20795b882ed service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:57.337330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d43b0b-a995-4d88-9dc0-b20795b882ed service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:57.337805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d43b0b-a995-4d88-9dc0-b20795b882ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:57.341843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-96d43b0b-a995-4d88-9dc0-b20795b882ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:57.371846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:57.420266 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2738/5525] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:57.427126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f888f9-9c97-49e4-a4ab-fdfd5d6f0c05 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:57.427402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f888f9-9c97-49e4-a4ab-fdfd5d6f0c05 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:57.427669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f888f9-9c97-49e4-a4ab-fdfd5d6f0c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:57.458792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-58f888f9-9c97-49e4-a4ab-fdfd5d6f0c05 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:57.460439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-58f888f9-9c97-49e4-a4ab-fdfd5d6f0c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:57.461108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d43b0b-a995-4d88-9dc0-b20795b882ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:57.462821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d43b0b-a995-4d88-9dc0-b20795b882ed service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:57.473810 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2739/5526] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:11:57.477441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f888f9-9c97-49e4-a4ab-fdfd5d6f0c05 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:57.489038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f888f9-9c97-49e4-a4ab-fdfd5d6f0c05 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:57.489987 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2788/5527] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:57.500843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b4bbc-cc8a-4978-bb40-c23df7a0e11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:57.502460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b4bbc-cc8a-4978-bb40-c23df7a0e11d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:57.502698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b4bbc-cc8a-4978-bb40-c23df7a0e11d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:57.502964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b4bbc-cc8a-4978-bb40-c23df7a0e11d admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:57.503620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b4bbc-cc8a-4978-bb40-c23df7a0e11d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:11:57.503909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b4bbc-cc8a-4978-bb40-c23df7a0e11d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:57.509173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b4bbc-cc8a-4978-bb40-c23df7a0e11d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:57.540557 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2740/5528] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:11:57.543460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d43b0b-a995-4d88-9dc0-b20795b882ed service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:57.558724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d43b0b-a995-4d88-9dc0-b20795b882ed service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:57.575269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee68789-1bfe-474e-8377-03e456d72a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:57.575976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee68789-1bfe-474e-8377-03e456d72a30 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:57.576312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee68789-1bfe-474e-8377-03e456d72a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:57.576711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee68789-1bfe-474e-8377-03e456d72a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:57.577532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee68789-1bfe-474e-8377-03e456d72a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:57.579597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee68789-1bfe-474e-8377-03e456d72a30 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:57.595369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52e83c-84b6-436e-89da-80ebd7cbbed3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:57.595661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52e83c-84b6-436e-89da-80ebd7cbbed3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:57.595939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52e83c-84b6-436e-89da-80ebd7cbbed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:57.625078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-df52e83c-84b6-436e-89da-80ebd7cbbed3 None None] MFA Rules not processed for user `f127011ceab246a8834d50f01f426487`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:57.626516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-df52e83c-84b6-436e-89da-80ebd7cbbed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:57.641010 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2789/5529] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:11:57 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:57.642568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee68789-1bfe-474e-8377-03e456d72a30 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:57.753021 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2741/5530] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:57.771046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee68789-1bfe-474e-8377-03e456d72a30 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:57.776788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626cb40-baac-438a-a72e-24b9a9510bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:57.777675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626cb40-baac-438a-a72e-24b9a9510bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:57.778199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626cb40-baac-438a-a72e-24b9a9510bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:57.778438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626cb40-baac-438a-a72e-24b9a9510bb2 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:57.780073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626cb40-baac-438a-a72e-24b9a9510bb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:57.782280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626cb40-baac-438a-a72e-24b9a9510bb2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:57.819976 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2790/5531] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:11:57 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:11:57.823897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52e83c-84b6-436e-89da-80ebd7cbbed3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:57.831241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:57.837436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52e83c-84b6-436e-89da-80ebd7cbbed3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:57.854953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8b40e-d66d-4875-b877-e390d0ded365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:57.855586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8b40e-d66d-4875-b877-e390d0ded365 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:57.855860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8b40e-d66d-4875-b877-e390d0ded365 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:57.856128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8b40e-d66d-4875-b877-e390d0ded365 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:57.856806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8b40e-d66d-4875-b877-e390d0ded365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:57.858549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8b40e-d66d-4875-b877-e390d0ded365 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:57.884025 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2742/5532] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:57.886100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626cb40-baac-438a-a72e-24b9a9510bb2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:57.896722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626cb40-baac-438a-a72e-24b9a9510bb2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:57.901556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9e338-32c1-4242-a809-d7e82deee779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:57.902302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9e338-32c1-4242-a809-d7e82deee779 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:57.902588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9e338-32c1-4242-a809-d7e82deee779 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:57.902937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9e338-32c1-4242-a809-d7e82deee779 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:57.903572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c9e338-32c1-4242-a809-d7e82deee779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:57.905243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c9e338-32c1-4242-a809-d7e82deee779 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:57.916807 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2791/5533] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:11:57.922417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8b40e-d66d-4875-b877-e390d0ded365 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:57.935384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8b40e-d66d-4875-b877-e390d0ded365 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:57.940405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c235fa-21ee-4408-a20c-0961f596a95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:57.941063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c235fa-21ee-4408-a20c-0961f596a95a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:57.941334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c235fa-21ee-4408-a20c-0961f596a95a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:57.941601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c235fa-21ee-4408-a20c-0961f596a95a admin admin] PATH_INFO: `/v3/projects/a697d7497f2f4f6eb63189a800bed008/users/77743eed1b5f449bb12a00de89246722/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:57.957662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c235fa-21ee-4408-a20c-0961f596a95a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a697d7497f2f4f6eb63189a800bed008, user_id=77743eed1b5f449bb12a00de89246722, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:57.960317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c235fa-21ee-4408-a20c-0961f596a95a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:57.974813 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2743/5534] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:11:57 2022] PUT /identity/v3/projects/a697d7497f2f4f6eb63189a800bed008/users/77743eed1b5f449bb12a00de89246722/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:11:57.985684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bf58e-b9c6-42e9-a621-e112f67713a3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:57.986040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bf58e-b9c6-42e9-a621-e112f67713a3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:57.986262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bf58e-b9c6-42e9-a621-e112f67713a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:58.028098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bf9bf58e-b9c6-42e9-a621-e112f67713a3 None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:58.029449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bf9bf58e-b9c6-42e9-a621-e112f67713a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:58.136258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:11:58.155257 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2792/5535] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:58.160248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c235fa-21ee-4408-a20c-0961f596a95a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:58.184995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c235fa-21ee-4408-a20c-0961f596a95a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:58.197077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068cb18-a259-4f89-be3b-e35dfb6dbfeb tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77743eed1b5f449bb12a00de89246722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a697d7497f2f4f6eb63189a800bed008', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:58.197824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068cb18-a259-4f89-be3b-e35dfb6dbfeb tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:58.198171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068cb18-a259-4f89-be3b-e35dfb6dbfeb tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:58.198514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068cb18-a259-4f89-be3b-e35dfb6dbfeb tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] PATH_INFO: `/v3/users/77743eed1b5f449bb12a00de89246722/password` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:58.280788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bf58e-b9c6-42e9-a621-e112f67713a3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:58.284679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bf9bf58e-b9c6-42e9-a621-e112f67713a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:58.296766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1068cb18-a259-4f89-be3b-e35dfb6dbfeb tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:58.315105 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2744/5536] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:11:58 2022] POST /identity/v3/users/77743eed1b5f449bb12a00de89246722/password => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:11:58.374518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bf58e-b9c6-42e9-a621-e112f67713a3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:58.378634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ff750-a851-4820-8311-9f5760e09bc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:58.379367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ff750-a851-4820-8311-9f5760e09bc3 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:58.379809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ff750-a851-4820-8311-9f5760e09bc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:58.380170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ff750-a851-4820-8311-9f5760e09bc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:58.382394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9a1ff750-a851-4820-8311-9f5760e09bc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:58.398978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ff750-a851-4820-8311-9f5760e09bc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:11:58.400800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ff750-a851-4820-8311-9f5760e09bc3 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:11:58.416441 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2793/5537] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:11:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:11:59.319243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b73fb-01c2-4934-b17f-e97bbc0c0d93 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:59.319777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b73fb-01c2-4934-b17f-e97bbc0c0d93 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:59.319777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b73fb-01c2-4934-b17f-e97bbc0c0d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:59.345166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f96b73fb-01c2-4934-b17f-e97bbc0c0d93 None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:59.346450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f96b73fb-01c2-4934-b17f-e97bbc0c0d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:59.352527 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2745/5538] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:59 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:11:59.365240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac778d-2555-47da-ac7c-a57fcd7a5299 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:59.365847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac778d-2555-47da-ac7c-a57fcd7a5299 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:59.366574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac778d-2555-47da-ac7c-a57fcd7a5299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:59.406365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5fac778d-2555-47da-ac7c-a57fcd7a5299 None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:11:59.408005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5fac778d-2555-47da-ac7c-a57fcd7a5299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:11:59.419851 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2794/5539] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:11:59.424159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b73fb-01c2-4934-b17f-e97bbc0c0d93 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:59.438103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b73fb-01c2-4934-b17f-e97bbc0c0d93 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:59.444341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bc365-cf0c-478a-a540-9734a1791e91 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77743eed1b5f449bb12a00de89246722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a697d7497f2f4f6eb63189a800bed008', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:59.445391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bc365-cf0c-478a-a540-9734a1791e91 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:59.445664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bc365-cf0c-478a-a540-9734a1791e91 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:59.445925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bc365-cf0c-478a-a540-9734a1791e91 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] PATH_INFO: `/v3/users/77743eed1b5f449bb12a00de89246722/password` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:59.526339 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-c70bc365-cf0c-478a-a540-9734a1791e91 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Aug 20 21:11:59.526339 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:59.526339 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:11:59.526339 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:11:59.526339 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:11:59.526339 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:11:59.526339 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:11:59.526339 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:11:59.526339 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:11:59.526339 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:11:59.526339 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:11:59.526339 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:11:59.526339 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Aug 20 21:11:59.527245 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:11:59.529908 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2746/5540] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:11:59 2022] POST /identity/v3/users/77743eed1b5f449bb12a00de89246722/password => generated 188 bytes in 106 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 20 21:11:59.534577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac778d-2555-47da-ac7c-a57fcd7a5299 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:59.544199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac778d-2555-47da-ac7c-a57fcd7a5299 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:59.556645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d6329-e0d6-4977-9789-cd9def79cf6b tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77743eed1b5f449bb12a00de89246722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a697d7497f2f4f6eb63189a800bed008', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:59.557024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d6329-e0d6-4977-9789-cd9def79cf6b tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:59.557395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d6329-e0d6-4977-9789-cd9def79cf6b tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:59.557883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d6329-e0d6-4977-9789-cd9def79cf6b tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] PATH_INFO: `/v3/users/77743eed1b5f449bb12a00de89246722/password` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:59.612013 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-4c0d6329-e0d6-4977-9789-cd9def79cf6b tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Aug 20 21:11:59.612013 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:11:59.612013 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:11:59.612013 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:11:59.612013 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:11:59.612013 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:11:59.612013 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:11:59.612013 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:11:59.612013 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:11:59.612013 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:11:59.612013 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:11:59.612013 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:11:59.612013 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Aug 20 21:11:59.612902 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:11:59.618445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bc365-cf0c-478a-a540-9734a1791e91 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:11:59.618924 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2795/5541] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:11:59 2022] POST /identity/v3/users/77743eed1b5f449bb12a00de89246722/password => generated 188 bytes in 83 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Aug 20 21:11:59.629275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bc365-cf0c-478a-a540-9734a1791e91 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:11:59.633880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177cf26-437a-4590-bbfd-0165ffea8d10 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77743eed1b5f449bb12a00de89246722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a697d7497f2f4f6eb63189a800bed008', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:11:59.634563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177cf26-437a-4590-bbfd-0165ffea8d10 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:11:59.634803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177cf26-437a-4590-bbfd-0165ffea8d10 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:11:59.635615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177cf26-437a-4590-bbfd-0165ffea8d10 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] PATH_INFO: `/v3/users/77743eed1b5f449bb12a00de89246722/password` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:11:59.706247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4177cf26-437a-4590-bbfd-0165ffea8d10 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:11:59.728163 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2747/5542] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:11:59 2022] POST /identity/v3/users/77743eed1b5f449bb12a00de89246722/password => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:00.734149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe35398-e22a-43d4-a8c8-6f56e7031ccf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:00.734657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe35398-e22a-43d4-a8c8-6f56e7031ccf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:00.734809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe35398-e22a-43d4-a8c8-6f56e7031ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:00.768887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dbe35398-e22a-43d4-a8c8-6f56e7031ccf None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:00.772914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dbe35398-e22a-43d4-a8c8-6f56e7031ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:00.779015 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2796/5543] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:00 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:12:00.783473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6a95-c271-444c-8622-9c24ca488575 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:00.784782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6a95-c271-444c-8622-9c24ca488575 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:00.785230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6a95-c271-444c-8622-9c24ca488575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:00.825143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9e7f6a95-c271-444c-8622-9c24ca488575 None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:00.826607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9e7f6a95-c271-444c-8622-9c24ca488575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:00.848163 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2748/5544] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:00.848420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe35398-e22a-43d4-a8c8-6f56e7031ccf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:00.868141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe35398-e22a-43d4-a8c8-6f56e7031ccf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:00.875655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8150b-58fd-47cf-b3e5-7432a3e17b0d tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77743eed1b5f449bb12a00de89246722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a697d7497f2f4f6eb63189a800bed008', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:00.878007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8150b-58fd-47cf-b3e5-7432a3e17b0d tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:00.878717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8150b-58fd-47cf-b3e5-7432a3e17b0d tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:00.879414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8150b-58fd-47cf-b3e5-7432a3e17b0d tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] PATH_INFO: `/v3/users/77743eed1b5f449bb12a00de89246722/password` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:00.962185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-75c8150b-58fd-47cf-b3e5-7432a3e17b0d tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:00.983997 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2797/5545] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:12:00 2022] POST /identity/v3/users/77743eed1b5f449bb12a00de89246722/password => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:01.989047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58a6b1-ed85-4174-a3df-d2ecefabd4a8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:01.989636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58a6b1-ed85-4174-a3df-d2ecefabd4a8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:01.989636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58a6b1-ed85-4174-a3df-d2ecefabd4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:02.071150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5b58a6b1-ed85-4174-a3df-d2ecefabd4a8 None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:02.072405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5b58a6b1-ed85-4174-a3df-d2ecefabd4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:02.084389 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2749/5546] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:01 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:12:02.088540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f03a1-bc90-4081-b8ef-ee9db4ea902d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:02.089617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f03a1-bc90-4081-b8ef-ee9db4ea902d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:02.089728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f03a1-bc90-4081-b8ef-ee9db4ea902d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:02.133118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-724f03a1-bc90-4081-b8ef-ee9db4ea902d None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:02.145618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-724f03a1-bc90-4081-b8ef-ee9db4ea902d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:02.198707 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2798/5547] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:02.200714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58a6b1-ed85-4174-a3df-d2ecefabd4a8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:02.225611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58a6b1-ed85-4174-a3df-d2ecefabd4a8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:02.261853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604a9a6-79e3-4a48-9da2-afd76bf1a612 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77743eed1b5f449bb12a00de89246722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a697d7497f2f4f6eb63189a800bed008', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:02.272582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604a9a6-79e3-4a48-9da2-afd76bf1a612 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:02.279833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604a9a6-79e3-4a48-9da2-afd76bf1a612 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:02.281139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604a9a6-79e3-4a48-9da2-afd76bf1a612 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] PATH_INFO: `/v3/users/77743eed1b5f449bb12a00de89246722/password` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:02.422188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5604a9a6-79e3-4a48-9da2-afd76bf1a612 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:02.442608 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2750/5548] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:12:02 2022] POST /identity/v3/users/77743eed1b5f449bb12a00de89246722/password => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:03.448210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb455d-843c-43c3-bfb4-5478206a270d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:03.448970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb455d-843c-43c3-bfb4-5478206a270d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:03.449433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb455d-843c-43c3-bfb4-5478206a270d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:03.479162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-31eb455d-843c-43c3-bfb4-5478206a270d None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:03.480406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-31eb455d-843c-43c3-bfb4-5478206a270d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:03.491680 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2799/5549] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:03 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:12:03.495212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7c8a6-51b5-4047-a6db-8ac201d2fd87 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:03.495691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7c8a6-51b5-4047-a6db-8ac201d2fd87 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:03.495691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7c8a6-51b5-4047-a6db-8ac201d2fd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:03.521242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2ae7c8a6-51b5-4047-a6db-8ac201d2fd87 None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:03.522471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2ae7c8a6-51b5-4047-a6db-8ac201d2fd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:03.530284 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2751/5550] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:03.536188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb455d-843c-43c3-bfb4-5478206a270d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:03.550251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb455d-843c-43c3-bfb4-5478206a270d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:03.555494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f2284-5c03-41de-95da-4164aee8c60e tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77743eed1b5f449bb12a00de89246722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a697d7497f2f4f6eb63189a800bed008', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:03.556547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f2284-5c03-41de-95da-4164aee8c60e tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:03.557135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f2284-5c03-41de-95da-4164aee8c60e tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:03.557315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f2284-5c03-41de-95da-4164aee8c60e tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] PATH_INFO: `/v3/users/77743eed1b5f449bb12a00de89246722/password` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:03.614195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-425f2284-5c03-41de-95da-4164aee8c60e tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:03.624855 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2800/5551] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:12:03 2022] POST /identity/v3/users/77743eed1b5f449bb12a00de89246722/password => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:04.630697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076214f3-af44-4947-a49d-a5da30db87d1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:04.632145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076214f3-af44-4947-a49d-a5da30db87d1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:04.632672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076214f3-af44-4947-a49d-a5da30db87d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:04.680966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-076214f3-af44-4947-a49d-a5da30db87d1 None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:04.685244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-076214f3-af44-4947-a49d-a5da30db87d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:04.691027 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2752/5552] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:04 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:12:05.697112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c7324-4b43-49ff-8523-051a0e51e0ee None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:05.697700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c7324-4b43-49ff-8523-051a0e51e0ee None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:05.697700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c7324-4b43-49ff-8523-051a0e51e0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:05.718233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-658c7324-4b43-49ff-8523-051a0e51e0ee None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:05.719485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-658c7324-4b43-49ff-8523-051a0e51e0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:05.727356 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2801/5553] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:05.734228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53767d-d529-4e83-9825-de9a4f71782a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:05.735149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53767d-d529-4e83-9825-de9a4f71782a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:05.735788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53767d-d529-4e83-9825-de9a4f71782a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:05.757809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8a53767d-d529-4e83-9825-de9a4f71782a None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:05.759298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a53767d-d529-4e83-9825-de9a4f71782a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:05.764628 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2753/5554] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:05 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:12:05.769737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536176a-6c0c-44dc-a9af-0ac7d06dfff1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:05.770325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536176a-6c0c-44dc-a9af-0ac7d06dfff1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:05.770770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536176a-6c0c-44dc-a9af-0ac7d06dfff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:05.844471 ncdv-master-1 devstack@keystone.service[104118]: WARNING keystone.server.flask.application [None req-0536176a-6c0c-44dc-a9af-0ac7d06dfff1 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.23: keystone.exception.Unauthorized: The request you have made requires authentication. Aug 20 21:12:05.847310 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2802/5555] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:05 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 78 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Aug 20 21:12:05.852183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54806bd0-3e5e-4e98-8f57-7582aef710f0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:05.852553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54806bd0-3e5e-4e98-8f57-7582aef710f0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:05.853146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54806bd0-3e5e-4e98-8f57-7582aef710f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:05.919709 ncdv-master-1 devstack@keystone.service[104119]: WARNING keystone.server.flask.application [None req-54806bd0-3e5e-4e98-8f57-7582aef710f0 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.23: keystone.exception.Unauthorized: The request you have made requires authentication. Aug 20 21:12:05.922569 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2754/5556] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:05 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 71 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Aug 20 21:12:05.927025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c5acd-622e-4002-b5ff-cb68c6abaab2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:05.928116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c5acd-622e-4002-b5ff-cb68c6abaab2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:05.928623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c5acd-622e-4002-b5ff-cb68c6abaab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:05.973592 ncdv-master-1 devstack@keystone.service[104118]: WARNING keystone.server.flask.application [None req-084c5acd-622e-4002-b5ff-cb68c6abaab2 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.23: keystone.exception.Unauthorized: The request you have made requires authentication. Aug 20 21:12:05.977097 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2803/5557] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:05 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 51 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Aug 20 21:12:07.697622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54806bd0-3e5e-4e98-8f57-7582aef710f0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:07.700204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-54806bd0-3e5e-4e98-8f57-7582aef710f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:07.713249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:INSERT=7,keystone:UPDATE=4,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:07.797070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54806bd0-3e5e-4e98-8f57-7582aef710f0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:07.802514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015829a-1e94-4bce-ae00-54e34bdfb79f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:07.803150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015829a-1e94-4bce-ae00-54e34bdfb79f service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:07.803422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015829a-1e94-4bce-ae00-54e34bdfb79f service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:07.803684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015829a-1e94-4bce-ae00-54e34bdfb79f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:07.805527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6015829a-1e94-4bce-ae00-54e34bdfb79f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:07.905641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015829a-1e94-4bce-ae00-54e34bdfb79f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:07.907468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015829a-1e94-4bce-ae00-54e34bdfb79f service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:07.949112 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2755/5558] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:12:15.960672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:UPDATE=3,keystone:INSERT=4,keystone:DELETE=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:16.028123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c5acd-622e-4002-b5ff-cb68c6abaab2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:16.052755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c5acd-622e-4002-b5ff-cb68c6abaab2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:16.061421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5165a2-c1b6-41fe-bc19-f163533d6436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:16.062165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5165a2-c1b6-41fe-bc19-f163533d6436 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:16.062496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5165a2-c1b6-41fe-bc19-f163533d6436 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:16.062766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5165a2-c1b6-41fe-bc19-f163533d6436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:16.068328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-de5165a2-c1b6-41fe-bc19-f163533d6436 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:16.296025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5165a2-c1b6-41fe-bc19-f163533d6436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:16.299366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5165a2-c1b6-41fe-bc19-f163533d6436 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:16.351683 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2804/5559] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:16 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:12:16.895225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5efab2-e0b9-44c9-9bc6-2b2573dcbfb4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:16.895730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5efab2-e0b9-44c9-9bc6-2b2573dcbfb4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:16.895821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5efab2-e0b9-44c9-9bc6-2b2573dcbfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:16.932856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4f5efab2-e0b9-44c9-9bc6-2b2573dcbfb4 None None] MFA Rules not processed for user `f127011ceab246a8834d50f01f426487`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:16.934098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4f5efab2-e0b9-44c9-9bc6-2b2573dcbfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:16.988769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d6750-d284-4343-996d-e362807d6d2d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:16.989766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d6750-d284-4343-996d-e362807d6d2d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:16.990228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d6750-d284-4343-996d-e362807d6d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:17.067354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e79d6750-d284-4343-996d-e362807d6d2d None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:17.069589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e79d6750-d284-4343-996d-e362807d6d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:17.079141 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2805/5560] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:16 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:12:17.082708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d6750-d284-4343-996d-e362807d6d2d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:17.104263 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2756/5561] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:17.113613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d6750-d284-4343-996d-e362807d6d2d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:17.127256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ca90f-64d8-4403-b6cd-c9d90c621ee0 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77743eed1b5f449bb12a00de89246722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a697d7497f2f4f6eb63189a800bed008', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:17.128103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ca90f-64d8-4403-b6cd-c9d90c621ee0 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:17.128693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ca90f-64d8-4403-b6cd-c9d90c621ee0 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:17.129039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ca90f-64d8-4403-b6cd-c9d90c621ee0 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] PATH_INFO: `/v3/users/77743eed1b5f449bb12a00de89246722/password` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:17.226546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-de6ca90f-64d8-4403-b6cd-c9d90c621ee0 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:17.247255 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2806/5562] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:12:17 2022] POST /identity/v3/users/77743eed1b5f449bb12a00de89246722/password => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:18.252117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d63c9b-eca7-4303-8ebc-d99d930bb78c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:18.252761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d63c9b-eca7-4303-8ebc-d99d930bb78c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:18.252946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d63c9b-eca7-4303-8ebc-d99d930bb78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:18.311239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-45d63c9b-eca7-4303-8ebc-d99d930bb78c None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:18.314578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-45d63c9b-eca7-4303-8ebc-d99d930bb78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:18.321344 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2757/5563] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:18 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:12:18.325928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cb41d-ae27-46d1-8043-50d7d5e33e94 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:18.326461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cb41d-ae27-46d1-8043-50d7d5e33e94 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:18.326461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cb41d-ae27-46d1-8043-50d7d5e33e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-730cb41d-ae27-46d1-8043-50d7d5e33e94 None None] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 20 21:12:18.366272 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 185, in authenticate_for_token Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application authenticate(auth_info, auth_context) Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 152, in authenticate Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = method.authenticate(auth_info.get_method_data(method_name)) Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 42, in authenticate Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application token = self._get_token_ref(auth_payload) Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 36, in _get_token_ref Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return PROVIDERS.token_provider_api.validate_token(token_id) Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application self.check_revocation(token) Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:12:18.367292 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:12:18.371149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Aug 20 21:12:18.371149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Aug 20 21:12:18.371149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:12:18.371149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:12:18.371149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Aug 20 21:12:18.371149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Aug 20 21:12:18.371149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:12:18.371149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:12:18.371149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Aug 20 21:12:18.371149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Aug 20 21:12:18.371149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Aug 20 21:12:18.371149 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:12:18.371149 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2807/5564] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:18 2022] POST /identity/v3/auth/tokens => generated 80 bytes in 45 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 21:12:18.373063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ba633-5140-4a74-8946-e7789a0eadc1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:18.373356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ba633-5140-4a74-8946-e7789a0eadc1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:18.373624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ba633-5140-4a74-8946-e7789a0eadc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:18.431347 ncdv-master-1 devstack@keystone.service[104119]: WARNING keystone.server.flask.application [None req-830ba633-5140-4a74-8946-e7789a0eadc1 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.23: keystone.exception.Unauthorized: The request you have made requires authentication. Aug 20 21:12:18.433691 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2758/5565] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:18 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 62 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Aug 20 21:12:18.438250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24ba66-0515-4a47-8f01-7a01be8b64bc None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:18.438791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24ba66-0515-4a47-8f01-7a01be8b64bc None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:18.440079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24ba66-0515-4a47-8f01-7a01be8b64bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:18.578010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1f24ba66-0515-4a47-8f01-7a01be8b64bc None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:18.579615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1f24ba66-0515-4a47-8f01-7a01be8b64bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:18.594872 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2808/5566] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:18.598246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ba633-5140-4a74-8946-e7789a0eadc1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:18.620887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ba633-5140-4a74-8946-e7789a0eadc1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:18.635251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f2f19-690e-4c1e-a41a-8f82395ee26f tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77743eed1b5f449bb12a00de89246722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a697d7497f2f4f6eb63189a800bed008', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:18.636191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f2f19-690e-4c1e-a41a-8f82395ee26f tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:18.636500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f2f19-690e-4c1e-a41a-8f82395ee26f tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:18.636767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f2f19-690e-4c1e-a41a-8f82395ee26f tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] PATH_INFO: `/v3/users/77743eed1b5f449bb12a00de89246722/password` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:18.678651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:18.731113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9a7f2f19-690e-4c1e-a41a-8f82395ee26f tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:18.759010 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2759/5567] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:12:18 2022] POST /identity/v3/users/77743eed1b5f449bb12a00de89246722/password => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:19.763546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e6d8f-5f99-4c5e-9b44-7e2b2d9b20cf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:19.764104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e6d8f-5f99-4c5e-9b44-7e2b2d9b20cf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:19.764104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e6d8f-5f99-4c5e-9b44-7e2b2d9b20cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:19.788718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-488e6d8f-5f99-4c5e-9b44-7e2b2d9b20cf None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:19.790098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-488e6d8f-5f99-4c5e-9b44-7e2b2d9b20cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:19.795655 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2809/5568] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:19 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:12:19.799987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4d931-59a6-4b69-85e1-6ef4a2f1b4d6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:19.800884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4d931-59a6-4b69-85e1-6ef4a2f1b4d6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:19.801367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4d931-59a6-4b69-85e1-6ef4a2f1b4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:19.831915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a9c4d931-59a6-4b69-85e1-6ef4a2f1b4d6 None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:19.834776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a9c4d931-59a6-4b69-85e1-6ef4a2f1b4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:19.845592 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2760/5569] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:19.850072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e6d8f-5f99-4c5e-9b44-7e2b2d9b20cf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:19.888590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e6d8f-5f99-4c5e-9b44-7e2b2d9b20cf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:19.900062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e472d43-f453-41dc-8e48-01b37dc710a7 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77743eed1b5f449bb12a00de89246722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a697d7497f2f4f6eb63189a800bed008', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:19.901021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e472d43-f453-41dc-8e48-01b37dc710a7 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:19.901337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e472d43-f453-41dc-8e48-01b37dc710a7 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:19.901712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e472d43-f453-41dc-8e48-01b37dc710a7 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] PATH_INFO: `/v3/users/77743eed1b5f449bb12a00de89246722/password` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:19.978394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5e472d43-f453-41dc-8e48-01b37dc710a7 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:20.007198 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2810/5570] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:12:19 2022] POST /identity/v3/users/77743eed1b5f449bb12a00de89246722/password => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:21.011418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ba725-566e-4044-86e7-48547e0ffc2f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:21.011984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ba725-566e-4044-86e7-48547e0ffc2f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:21.011984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ba725-566e-4044-86e7-48547e0ffc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:21.031459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cf1ba725-566e-4044-86e7-48547e0ffc2f None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:21.033790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cf1ba725-566e-4044-86e7-48547e0ffc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:21.036971 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2761/5571] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:21 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:12:21.041719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ebc86-7ea7-4d8c-9835-894e35e7d8f4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:21.042201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ebc86-7ea7-4d8c-9835-894e35e7d8f4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:21.042451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ebc86-7ea7-4d8c-9835-894e35e7d8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:21.062352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8e2ebc86-7ea7-4d8c-9835-894e35e7d8f4 None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:21.064240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8e2ebc86-7ea7-4d8c-9835-894e35e7d8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:21.071584 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2811/5572] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:21.075843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ba725-566e-4044-86e7-48547e0ffc2f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:21.088667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ba725-566e-4044-86e7-48547e0ffc2f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:21.099046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc3d56-6688-4cac-a051-f16da60f3224 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77743eed1b5f449bb12a00de89246722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a697d7497f2f4f6eb63189a800bed008', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:21.099718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc3d56-6688-4cac-a051-f16da60f3224 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:21.099994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc3d56-6688-4cac-a051-f16da60f3224 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:21.100262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc3d56-6688-4cac-a051-f16da60f3224 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] PATH_INFO: `/v3/users/77743eed1b5f449bb12a00de89246722/password` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:21.151996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64cc3d56-6688-4cac-a051-f16da60f3224 tempest-IdentityV3UsersTest-666433848 tempest-IdentityV3UsersTest-666433848-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:21.167906 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2762/5573] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:12:21 2022] POST /identity/v3/users/77743eed1b5f449bb12a00de89246722/password => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:21.945551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ebc86-7ea7-4d8c-9835-894e35e7d8f4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:21.950265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8e2ebc86-7ea7-4d8c-9835-894e35e7d8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:22.045352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:UPDATE=4,keystone:INSERT=4,keystone:DELETE=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:22.174303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e21038-77f3-48ce-b0b0-9fbc0ac5d859 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:22.174802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e21038-77f3-48ce-b0b0-9fbc0ac5d859 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:22.175017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e21038-77f3-48ce-b0b0-9fbc0ac5d859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:22.196252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ebc86-7ea7-4d8c-9835-894e35e7d8f4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:22.220183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112a758-e43c-499f-8d9d-d89618d910d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:22.223157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112a758-e43c-499f-8d9d-d89618d910d1 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:22.224067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112a758-e43c-499f-8d9d-d89618d910d1 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:22.224465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112a758-e43c-499f-8d9d-d89618d910d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:22.227540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d0e21038-77f3-48ce-b0b0-9fbc0ac5d859 None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:22.229392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d0e21038-77f3-48ce-b0b0-9fbc0ac5d859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:22.231555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7112a758-e43c-499f-8d9d-d89618d910d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:22.242877 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2763/5574] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:22 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 20 21:12:22.257653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112a758-e43c-499f-8d9d-d89618d910d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:22.259779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112a758-e43c-499f-8d9d-d89618d910d1 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:22.275337 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2812/5575] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:12:23.247570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede11d6-55de-4f48-ae34-9c255565433b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:23.248070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede11d6-55de-4f48-ae34-9c255565433b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:23.248161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede11d6-55de-4f48-ae34-9c255565433b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:23.268764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bede11d6-55de-4f48-ae34-9c255565433b None None] MFA Rules not processed for user `77743eed1b5f449bb12a00de89246722`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:23.269991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bede11d6-55de-4f48-ae34-9c255565433b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:23.277182 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2764/5576] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:23.283783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112a758-e43c-499f-8d9d-d89618d910d1 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:23.353098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112a758-e43c-499f-8d9d-d89618d910d1 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:23.358995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:23.359151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:23.359375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:23.359640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] PATH_INFO: `/v3/users/77743eed1b5f449bb12a00de89246722` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:23.360949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77743eed1b5f449bb12a00de89246722)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:23.362914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:23.427348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:23.437839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:23.449676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:23.465570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:23.466256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77743eed1b5f449bb12a00de89246722'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:23.472703 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2813/5577] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:12:23 2022] DELETE /identity/v3/users/77743eed1b5f449bb12a00de89246722 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:23.477521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab38b55-03fb-4f16-a68c-be781e984f00 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:23.478200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab38b55-03fb-4f16-a68c-be781e984f00 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:23.478683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab38b55-03fb-4f16-a68c-be781e984f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:23.502872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3ab38b55-03fb-4f16-a68c-be781e984f00 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:23.504543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3ab38b55-03fb-4f16-a68c-be781e984f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:23.584742 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2765/5578] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:23.593671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:23.595219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:23.641944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:23.692156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d08b8-47e7-4e62-983a-0fa70827b018 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:23.697722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37b488-009b-4963-8526-67b63dba8d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:23.698927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b488-009b-4963-8526-67b63dba8d3b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:23.699202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b488-009b-4963-8526-67b63dba8d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:23.699614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b488-009b-4963-8526-67b63dba8d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:23.714746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37b488-009b-4963-8526-67b63dba8d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:23.717368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37b488-009b-4963-8526-67b63dba8d3b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:23.729617 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2814/5579] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:12:24.389506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab38b55-03fb-4f16-a68c-be781e984f00 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:24.391719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3ab38b55-03fb-4f16-a68c-be781e984f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:24.411752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab38b55-03fb-4f16-a68c-be781e984f00 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:24.417265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec91d8-16d7-4ea5-af10-8c38172b6f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:24.418540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec91d8-16d7-4ea5-af10-8c38172b6f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:24.419029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec91d8-16d7-4ea5-af10-8c38172b6f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:24.419925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec91d8-16d7-4ea5-af10-8c38172b6f94 admin admin] PATH_INFO: `/v3/projects/a697d7497f2f4f6eb63189a800bed008` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:24.421485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec91d8-16d7-4ea5-af10-8c38172b6f94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a697d7497f2f4f6eb63189a800bed008)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:24.424494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec91d8-16d7-4ea5-af10-8c38172b6f94 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:24.505630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bec91d8-16d7-4ea5-af10-8c38172b6f94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a697d7497f2f4f6eb63189a800bed008', 'request_id': 'req-5bec91d8-16d7-4ea5-af10-8c38172b6f94'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:24.512561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bec91d8-16d7-4ea5-af10-8c38172b6f94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a697d7497f2f4f6eb63189a800bed008', 'request_id': 'req-5bec91d8-16d7-4ea5-af10-8c38172b6f94'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:24.522611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bec91d8-16d7-4ea5-af10-8c38172b6f94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a697d7497f2f4f6eb63189a800bed008', 'request_id': 'req-5bec91d8-16d7-4ea5-af10-8c38172b6f94'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:24.534250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=82,keystone:UPDATE=2,keystone:INSERT=5,keystone:DELETE=11 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:24.539821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bec91d8-16d7-4ea5-af10-8c38172b6f94 admin admin] The token cache is being invalidate because project a697d7497f2f4f6eb63189a800bed008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:12:24.540115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bec91d8-16d7-4ea5-af10-8c38172b6f94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:24.541170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bec91d8-16d7-4ea5-af10-8c38172b6f94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:24.543559 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2766/5580] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:12:24 2022] DELETE /identity/v3/projects/a697d7497f2f4f6eb63189a800bed008 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:24.549709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d766c-f011-4bfe-8ecc-63d8299a06d7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:24.550307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d766c-f011-4bfe-8ecc-63d8299a06d7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:24.551141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d766c-f011-4bfe-8ecc-63d8299a06d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:24.572793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b13d766c-f011-4bfe-8ecc-63d8299a06d7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:24.574028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b13d766c-f011-4bfe-8ecc-63d8299a06d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:24.646162 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2815/5581] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:24.653716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c113a8-8280-45c4-b1df-422de20cbe11 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:24.654004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c113a8-8280-45c4-b1df-422de20cbe11 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:24.654278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c113a8-8280-45c4-b1df-422de20cbe11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:24.687804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e9c113a8-8280-45c4-b1df-422de20cbe11 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:24.689539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e9c113a8-8280-45c4-b1df-422de20cbe11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:24.699050 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2767/5582] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:24.703933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d766c-f011-4bfe-8ecc-63d8299a06d7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:24.717644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d766c-f011-4bfe-8ecc-63d8299a06d7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:24.723227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0a820-e0db-4685-b6b0-de94ba62eb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:24.724095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0a820-e0db-4685-b6b0-de94ba62eb02 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:24.724463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0a820-e0db-4685-b6b0-de94ba62eb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:24.724796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0a820-e0db-4685-b6b0-de94ba62eb02 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:24.725512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0a820-e0db-4685-b6b0-de94ba62eb02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:12:24.725869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0a820-e0db-4685-b6b0-de94ba62eb02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:24.727632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0a820-e0db-4685-b6b0-de94ba62eb02 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:24.750284 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2816/5583] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:12:24.752199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c113a8-8280-45c4-b1df-422de20cbe11 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:24.766355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c113a8-8280-45c4-b1df-422de20cbe11 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:24.771755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223148e-8812-4535-872c-06b9f4d5bea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:24.772536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223148e-8812-4535-872c-06b9f4d5bea8 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:24.772879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223148e-8812-4535-872c-06b9f4d5bea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:24.773220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223148e-8812-4535-872c-06b9f4d5bea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:24.774093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223148e-8812-4535-872c-06b9f4d5bea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:24.776137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223148e-8812-4535-872c-06b9f4d5bea8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:24.814278 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2768/5584] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:12:24 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:12:24.818379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0a820-e0db-4685-b6b0-de94ba62eb02 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:24.841146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0a820-e0db-4685-b6b0-de94ba62eb02 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:24.973003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe72a7e-ce5a-4a88-8930-da97ba49e41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:24.974314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe72a7e-ce5a-4a88-8930-da97ba49e41d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:24.974594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe72a7e-ce5a-4a88-8930-da97ba49e41d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:24.974862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe72a7e-ce5a-4a88-8930-da97ba49e41d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:24.975508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe72a7e-ce5a-4a88-8930-da97ba49e41d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:24.983818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe72a7e-ce5a-4a88-8930-da97ba49e41d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:25.044618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223148e-8812-4535-872c-06b9f4d5bea8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:25.052353 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2817/5585] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:12:24 2022] POST /identity/v3/users => generated 296 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:12:25.057812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223148e-8812-4535-872c-06b9f4d5bea8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:25.065122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f5fba-af2a-47a5-be8f-29f0c12cc3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:25.074724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f5fba-af2a-47a5-be8f-29f0c12cc3fb admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:25.074724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f5fba-af2a-47a5-be8f-29f0c12cc3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:25.074724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f5fba-af2a-47a5-be8f-29f0c12cc3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:25.074724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f5fba-af2a-47a5-be8f-29f0c12cc3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:25.076747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f5fba-af2a-47a5-be8f-29f0c12cc3fb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:25.100607 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2769/5586] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:12:25.102623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe72a7e-ce5a-4a88-8930-da97ba49e41d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:25.127132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe72a7e-ce5a-4a88-8930-da97ba49e41d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:25.140490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37bf0d-b8ed-4e05-b29b-ae403a2b9215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:25.141524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37bf0d-b8ed-4e05-b29b-ae403a2b9215 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:25.142340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37bf0d-b8ed-4e05-b29b-ae403a2b9215 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:25.142623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37bf0d-b8ed-4e05-b29b-ae403a2b9215 admin admin] PATH_INFO: `/v3/projects/21ba9eb9e25748b9b8e6daef5401b45a/users/1dc3203fd29f4cde92adb7ab8506025f/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:25.164997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37bf0d-b8ed-4e05-b29b-ae403a2b9215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ba9eb9e25748b9b8e6daef5401b45a, user_id=1dc3203fd29f4cde92adb7ab8506025f, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:25.167736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37bf0d-b8ed-4e05-b29b-ae403a2b9215 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:25.203867 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2818/5587] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:12:25 2022] PUT /identity/v3/projects/21ba9eb9e25748b9b8e6daef5401b45a/users/1dc3203fd29f4cde92adb7ab8506025f/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:25.208228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f5fba-af2a-47a5-be8f-29f0c12cc3fb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:25.223173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f5fba-af2a-47a5-be8f-29f0c12cc3fb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:25.360200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf3584-5217-4979-ba54-b2059a745ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:25.361124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf3584-5217-4979-ba54-b2059a745ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:25.362939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf3584-5217-4979-ba54-b2059a745ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:25.363329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf3584-5217-4979-ba54-b2059a745ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:25.364008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf3584-5217-4979-ba54-b2059a745ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:25.365697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf3584-5217-4979-ba54-b2059a745ee5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:25.380945 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2770/5588] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:12:25.381788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37bf0d-b8ed-4e05-b29b-ae403a2b9215 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:25.393705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37bf0d-b8ed-4e05-b29b-ae403a2b9215 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:25.398981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2cc7b-4d7c-4e28-9b6d-1d1ab9552903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:25.399644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2cc7b-4d7c-4e28-9b6d-1d1ab9552903 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:25.399915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2cc7b-4d7c-4e28-9b6d-1d1ab9552903 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:25.400184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2cc7b-4d7c-4e28-9b6d-1d1ab9552903 admin admin] PATH_INFO: `/v3/projects/21ba9eb9e25748b9b8e6daef5401b45a/users/1dc3203fd29f4cde92adb7ab8506025f/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:25.402103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2cc7b-4d7c-4e28-9b6d-1d1ab9552903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ba9eb9e25748b9b8e6daef5401b45a, user_id=1dc3203fd29f4cde92adb7ab8506025f, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:25.405057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2cc7b-4d7c-4e28-9b6d-1d1ab9552903 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:25.424431 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2819/5589] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:12:25 2022] PUT /identity/v3/projects/21ba9eb9e25748b9b8e6daef5401b45a/users/1dc3203fd29f4cde92adb7ab8506025f/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:12:25.429595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2b063-6ce5-4360-8bd1-b4445732e2ae None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:25.430057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2b063-6ce5-4360-8bd1-b4445732e2ae None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:25.430270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2b063-6ce5-4360-8bd1-b4445732e2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:25.464146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8ba2b063-6ce5-4360-8bd1-b4445732e2ae None None] MFA Rules not processed for user `1dc3203fd29f4cde92adb7ab8506025f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:25.465512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8ba2b063-6ce5-4360-8bd1-b4445732e2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:25.558815 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2771/5590] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:25.559941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2cc7b-4d7c-4e28-9b6d-1d1ab9552903 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:25.578799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:25.660404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2cc7b-4d7c-4e28-9b6d-1d1ab9552903 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:25.667869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e19e2-8912-44d5-a760-9f0ce17bf58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:25.668899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e19e2-8912-44d5-a760-9f0ce17bf58d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:25.669420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e19e2-8912-44d5-a760-9f0ce17bf58d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:25.669878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e19e2-8912-44d5-a760-9f0ce17bf58d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:25.670723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e19e2-8912-44d5-a760-9f0ce17bf58d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:25.674307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e19e2-8912-44d5-a760-9f0ce17bf58d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:25.720217 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2820/5591] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:12:25 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:12:25.724558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2b063-6ce5-4360-8bd1-b4445732e2ae None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:25.738730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2b063-6ce5-4360-8bd1-b4445732e2ae None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:25.817262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:25.883310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bf8ae-7764-4404-9d0e-0663ba0b0426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:25.884598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bf8ae-7764-4404-9d0e-0663ba0b0426 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:25.884971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bf8ae-7764-4404-9d0e-0663ba0b0426 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:25.886284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bf8ae-7764-4404-9d0e-0663ba0b0426 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:25.887064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bf8ae-7764-4404-9d0e-0663ba0b0426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:25.889183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bf8ae-7764-4404-9d0e-0663ba0b0426 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:25.933236 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2772/5592] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:12:25 2022] POST /identity/v3/users => generated 290 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:12:25.936868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e19e2-8912-44d5-a760-9f0ce17bf58d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:25.958163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e19e2-8912-44d5-a760-9f0ce17bf58d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:25.963303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272f675-a075-45d0-9d75-41e845fd531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:25.964283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272f675-a075-45d0-9d75-41e845fd531f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:25.964788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272f675-a075-45d0-9d75-41e845fd531f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:25.965055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272f675-a075-45d0-9d75-41e845fd531f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:25.965750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272f675-a075-45d0-9d75-41e845fd531f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:25.967937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272f675-a075-45d0-9d75-41e845fd531f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:25.982188 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2821/5593] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:12:25.986925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bf8ae-7764-4404-9d0e-0663ba0b0426 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:25.999415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bf8ae-7764-4404-9d0e-0663ba0b0426 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:26.008296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f6256-5998-4623-802f-52ea5dc1bd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:26.013280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f6256-5998-4623-802f-52ea5dc1bd36 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:26.013280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f6256-5998-4623-802f-52ea5dc1bd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:26.013280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f6256-5998-4623-802f-52ea5dc1bd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:26.013280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f6256-5998-4623-802f-52ea5dc1bd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:26.013280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f6256-5998-4623-802f-52ea5dc1bd36 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:26.032899 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2773/5594] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:12:26.033177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272f675-a075-45d0-9d75-41e845fd531f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:26.049497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272f675-a075-45d0-9d75-41e845fd531f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:26.066084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbecab-3683-4a00-9057-7eaffc0069cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:26.066789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbecab-3683-4a00-9057-7eaffc0069cd admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:26.067065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbecab-3683-4a00-9057-7eaffc0069cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:26.067350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbecab-3683-4a00-9057-7eaffc0069cd admin admin] PATH_INFO: `/v3/projects/9e11d33493f348beb87881502d463a71/users/35657fdf3f95402b9dc0776c5df2a9e7/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:26.103873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbecab-3683-4a00-9057-7eaffc0069cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e11d33493f348beb87881502d463a71, user_id=35657fdf3f95402b9dc0776c5df2a9e7, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:26.110408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbecab-3683-4a00-9057-7eaffc0069cd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:26.148936 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2822/5595] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:12:26 2022] PUT /identity/v3/projects/9e11d33493f348beb87881502d463a71/users/35657fdf3f95402b9dc0776c5df2a9e7/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:12:26.159862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eced8a-0adf-46d5-8ada-f1319a064cf8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:26.160470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eced8a-0adf-46d5-8ada-f1319a064cf8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:26.160834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eced8a-0adf-46d5-8ada-f1319a064cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:26.236350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-54eced8a-0adf-46d5-8ada-f1319a064cf8 None None] MFA Rules not processed for user `35657fdf3f95402b9dc0776c5df2a9e7`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:26.237704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-54eced8a-0adf-46d5-8ada-f1319a064cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:26.400414 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2774/5596] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:26.415743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbecab-3683-4a00-9057-7eaffc0069cd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:26.420815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6adbecab-3683-4a00-9057-7eaffc0069cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:26.556686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbecab-3683-4a00-9057-7eaffc0069cd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:26.561873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5b0ac-333d-4788-9d28-bae78b73dfb7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:26.562566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5b0ac-333d-4788-9d28-bae78b73dfb7 service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:26.562843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5b0ac-333d-4788-9d28-bae78b73dfb7 service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:26.563260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5b0ac-333d-4788-9d28-bae78b73dfb7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:26.644024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5b0ac-333d-4788-9d28-bae78b73dfb7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:26.645737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5b0ac-333d-4788-9d28-bae78b73dfb7 service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:26.658277 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2823/5597] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:12:27.812121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eced8a-0adf-46d5-8ada-f1319a064cf8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:27.907613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eced8a-0adf-46d5-8ada-f1319a064cf8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:27.913514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:27.914483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:27.915288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:27.915832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] PATH_INFO: `/v3/users/1dc3203fd29f4cde92adb7ab8506025f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:27.917399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dc3203fd29f4cde92adb7ab8506025f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:27.919952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:28.041742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dc3203fd29f4cde92adb7ab8506025f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:28.054647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:28.068787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dc3203fd29f4cde92adb7ab8506025f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:28.114360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dc3203fd29f4cde92adb7ab8506025f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:28.115703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dc3203fd29f4cde92adb7ab8506025f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:28.117562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dc3203fd29f4cde92adb7ab8506025f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:28.150903 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2775/5598] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:12:27 2022] DELETE /identity/v3/users/1dc3203fd29f4cde92adb7ab8506025f => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:28.158136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2bc51-1ffc-4331-bb4e-45cbbd2d7099 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:28.160674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2bc51-1ffc-4331-bb4e-45cbbd2d7099 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:28.162086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2bc51-1ffc-4331-bb4e-45cbbd2d7099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:28.179440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:28.207591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a3e2bc51-1ffc-4331-bb4e-45cbbd2d7099 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:28.209250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a3e2bc51-1ffc-4331-bb4e-45cbbd2d7099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:28.341453 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2824/5599] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:28.353749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:28.355293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:28.479243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b98cd8-f407-45f0-8dd3-cb832d143f44 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:28.483397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf8fd0-3662-4ffc-ac17-935ff4e5743d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:28.484030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf8fd0-3662-4ffc-ac17-935ff4e5743d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:28.484517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf8fd0-3662-4ffc-ac17-935ff4e5743d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:28.484881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf8fd0-3662-4ffc-ac17-935ff4e5743d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:28.496295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf8fd0-3662-4ffc-ac17-935ff4e5743d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:28.497948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf8fd0-3662-4ffc-ac17-935ff4e5743d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:28.511679 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2776/5600] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:12:28.624336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2bc51-1ffc-4331-bb4e-45cbbd2d7099 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:28.629030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a3e2bc51-1ffc-4331-bb4e-45cbbd2d7099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:28.652740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2bc51-1ffc-4331-bb4e-45cbbd2d7099 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:28.659534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:28.660400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:28.660781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:28.661186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] PATH_INFO: `/v3/users/bb87b3c690584dbdbb0916d69e87065b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:28.662840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb87b3c690584dbdbb0916d69e87065b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:28.665377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:28.763066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb87b3c690584dbdbb0916d69e87065b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:28.781655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb87b3c690584dbdbb0916d69e87065b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:28.797294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb87b3c690584dbdbb0916d69e87065b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:28.811938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb87b3c690584dbdbb0916d69e87065b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:28.813324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb87b3c690584dbdbb0916d69e87065b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:28.822508 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2825/5601] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:12:28 2022] DELETE /identity/v3/users/bb87b3c690584dbdbb0916d69e87065b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:29.076763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf8fd0-3662-4ffc-ac17-935ff4e5743d service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:29.079649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d4bf8fd0-3662-4ffc-ac17-935ff4e5743d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:29.106019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf8fd0-3662-4ffc-ac17-935ff4e5743d service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:29.178230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:29.180206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:29.240663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:29.242463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:29.242846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:29.243870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] PATH_INFO: `/v3/projects/21ba9eb9e25748b9b8e6daef5401b45a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:29.249512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21ba9eb9e25748b9b8e6daef5401b45a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:29.256932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:29.263327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1c4fa-e0a4-434e-b5c0-3d29c1ae1703 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:29.268530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:29.269176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:29.269444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:29.269706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec admin admin] PATH_INFO: `/v3/projects/7af26a7f1fc742d39f13dbed6031f8b6` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:29.271250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=7af26a7f1fc742d39f13dbed6031f8b6)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:29.273708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:29.294709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:29.339506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7af26a7f1fc742d39f13dbed6031f8b6', 'request_id': 'req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.351576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21ba9eb9e25748b9b8e6daef5401b45a', 'request_id': 'req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.353139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7af26a7f1fc742d39f13dbed6031f8b6', 'request_id': 'req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.360860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21ba9eb9e25748b9b8e6daef5401b45a', 'request_id': 'req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.361437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7af26a7f1fc742d39f13dbed6031f8b6', 'request_id': 'req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.371303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21ba9eb9e25748b9b8e6daef5401b45a', 'request_id': 'req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.378439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec admin admin] The token cache is being invalidate because project 7af26a7f1fc742d39f13dbed6031f8b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:12:29.378957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.379821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4c3c4d43-f0c5-4f8a-8ca8-9c3e96e968ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.381714 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2826/5602] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:12:29 2022] DELETE /identity/v3/projects/7af26a7f1fc742d39f13dbed6031f8b6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:12:29.381973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:29.387515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc371c3-fcd8-4091-bf9c-ac84d99e28a5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:29.387781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc371c3-fcd8-4091-bf9c-ac84d99e28a5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:29.388338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc371c3-fcd8-4091-bf9c-ac84d99e28a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:29.394556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] The token cache is being invalidate because project 21ba9eb9e25748b9b8e6daef5401b45a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:12:29.395544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.399322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.401235 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2777/5603] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:12:29 2022] DELETE /identity/v3/projects/21ba9eb9e25748b9b8e6daef5401b45a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:29.413083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:29.419937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:29.422756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bbc371c3-fcd8-4091-bf9c-ac84d99e28a5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:29.423931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bbc371c3-fcd8-4091-bf9c-ac84d99e28a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:29.515564 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2827/5604] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:29.523421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c253ae-d39e-4831-ba64-ee26b42edf81 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:29.523707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c253ae-d39e-4831-ba64-ee26b42edf81 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:29.523959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c253ae-d39e-4831-ba64-ee26b42edf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:29.530770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bfbf5-ace9-4e15-9848-7c01a88d7f22 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:29.535859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:29.536829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:29.537111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:29.537387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] PATH_INFO: `/v3/users/35657fdf3f95402b9dc0776c5df2a9e7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:29.539457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35657fdf3f95402b9dc0776c5df2a9e7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:29.541468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:29.566794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-64c253ae-d39e-4831-ba64-ee26b42edf81 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:29.569722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-64c253ae-d39e-4831-ba64-ee26b42edf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:29.599830 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2828/5605] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:12:29.599830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c253ae-d39e-4831-ba64-ee26b42edf81 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:29.604234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c253ae-d39e-4831-ba64-ee26b42edf81 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:29.614546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801c04a-4c98-4b7c-a4e8-7686bd7533b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:29.615344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801c04a-4c98-4b7c-a4e8-7686bd7533b4 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:29.615542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801c04a-4c98-4b7c-a4e8-7686bd7533b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:29.615835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801c04a-4c98-4b7c-a4e8-7686bd7533b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:29.616554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801c04a-4c98-4b7c-a4e8-7686bd7533b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:12:29.616858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801c04a-4c98-4b7c-a4e8-7686bd7533b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:29.623658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801c04a-4c98-4b7c-a4e8-7686bd7533b4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:29.638418 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2829/5606] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:12:29.640585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35657fdf3f95402b9dc0776c5df2a9e7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.643697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801c04a-4c98-4b7c-a4e8-7686bd7533b4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:29.654402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35657fdf3f95402b9dc0776c5df2a9e7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.660753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801c04a-4c98-4b7c-a4e8-7686bd7533b4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:29.673931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720aea0-9569-4331-9ced-9f7d651fff0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:29.674573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720aea0-9569-4331-9ced-9f7d651fff0e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:29.674847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720aea0-9569-4331-9ced-9f7d651fff0e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:29.675141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720aea0-9569-4331-9ced-9f7d651fff0e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:29.675764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720aea0-9569-4331-9ced-9f7d651fff0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:29.677265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35657fdf3f95402b9dc0776c5df2a9e7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.677719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720aea0-9569-4331-9ced-9f7d651fff0e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:29.678345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35657fdf3f95402b9dc0776c5df2a9e7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.679024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35657fdf3f95402b9dc0776c5df2a9e7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:29.697200 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2778/5607] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:12:29 2022] DELETE /identity/v3/users/35657fdf3f95402b9dc0776c5df2a9e7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:29.722395 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2830/5608] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:12:29 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:12:29.726692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:29.728219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:29.873898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4e9ef-01bc-4cd4-a232-6a0d38607509 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:29.947200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:30.031910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4e62c-9135-4d1e-8967-786db8495823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:30.032692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4e62c-9135-4d1e-8967-786db8495823 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:30.033005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4e62c-9135-4d1e-8967-786db8495823 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:30.033278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4e62c-9135-4d1e-8967-786db8495823 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:30.033976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4e62c-9135-4d1e-8967-786db8495823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:30.036077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4e62c-9135-4d1e-8967-786db8495823 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:30.074377 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2779/5609] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:12:29 2022] POST /identity/v3/users => generated 293 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:12:30.091384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720aea0-9569-4331-9ced-9f7d651fff0e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:30.108462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720aea0-9569-4331-9ced-9f7d651fff0e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:30.115333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aa6fb-a287-438b-976d-c19a693d7caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:30.116633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aa6fb-a287-438b-976d-c19a693d7caa admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:30.117125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aa6fb-a287-438b-976d-c19a693d7caa admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:30.117588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aa6fb-a287-438b-976d-c19a693d7caa admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:30.118467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4aa6fb-a287-438b-976d-c19a693d7caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:30.120332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4aa6fb-a287-438b-976d-c19a693d7caa admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:30.135122 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2831/5610] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:12:30.139352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4e62c-9135-4d1e-8967-786db8495823 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:30.154419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4e62c-9135-4d1e-8967-786db8495823 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:30.169370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bebe10-3524-4bc5-a6dd-1161263382ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:30.169692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bebe10-3524-4bc5-a6dd-1161263382ec admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:30.169970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bebe10-3524-4bc5-a6dd-1161263382ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:30.170435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bebe10-3524-4bc5-a6dd-1161263382ec admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:30.171116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bebe10-3524-4bc5-a6dd-1161263382ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:30.174558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bebe10-3524-4bc5-a6dd-1161263382ec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:30.205962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aa6fb-a287-438b-976d-c19a693d7caa admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:30.207106 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2780/5611] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:12:30.215142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bebe10-3524-4bc5-a6dd-1161263382ec admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:30.219997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aa6fb-a287-438b-976d-c19a693d7caa admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:30.226356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c4bebe10-3524-4bc5-a6dd-1161263382ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:30.240034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37b43e-5d74-43cb-a2b2-84bcaa8716cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:30.240770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37b43e-5d74-43cb-a2b2-84bcaa8716cf admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:30.241046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37b43e-5d74-43cb-a2b2-84bcaa8716cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:30.241320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37b43e-5d74-43cb-a2b2-84bcaa8716cf admin admin] PATH_INFO: `/v3/projects/fd632566dc1d43a0bf3a7106b877773a/users/a0b8f53d77ba4aa88a1eab7d5cd0195d/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:30.260992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bebe10-3524-4bc5-a6dd-1161263382ec admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:30.272662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22aff9-f05a-4eff-8fe4-381d33a88606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:30.273326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22aff9-f05a-4eff-8fe4-381d33a88606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:30.273600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22aff9-f05a-4eff-8fe4-381d33a88606 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:30.273868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22aff9-f05a-4eff-8fe4-381d33a88606 admin admin] PATH_INFO: `/v3/projects/9e11d33493f348beb87881502d463a71` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:30.277103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22aff9-f05a-4eff-8fe4-381d33a88606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e11d33493f348beb87881502d463a71)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:30.280724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37b43e-5d74-43cb-a2b2-84bcaa8716cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd632566dc1d43a0bf3a7106b877773a, user_id=a0b8f53d77ba4aa88a1eab7d5cd0195d, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:30.284782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22aff9-f05a-4eff-8fe4-381d33a88606 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:30.287083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37b43e-5d74-43cb-a2b2-84bcaa8716cf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:30.318782 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2832/5612] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:12:30 2022] PUT /identity/v3/projects/fd632566dc1d43a0bf3a7106b877773a/users/a0b8f53d77ba4aa88a1eab7d5cd0195d/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:30.321059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1733c-30ac-44b4-8818-c80f90f0efbe None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:30.321341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1733c-30ac-44b4-8818-c80f90f0efbe None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:30.321613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1733c-30ac-44b4-8818-c80f90f0efbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:30.367074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c5f1733c-30ac-44b4-8818-c80f90f0efbe None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:30.372009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c5f1733c-30ac-44b4-8818-c80f90f0efbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:30.395658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c22aff9-f05a-4eff-8fe4-381d33a88606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e11d33493f348beb87881502d463a71', 'request_id': 'req-9c22aff9-f05a-4eff-8fe4-381d33a88606'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:30.414968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c22aff9-f05a-4eff-8fe4-381d33a88606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e11d33493f348beb87881502d463a71', 'request_id': 'req-9c22aff9-f05a-4eff-8fe4-381d33a88606'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:30.431011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c22aff9-f05a-4eff-8fe4-381d33a88606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e11d33493f348beb87881502d463a71', 'request_id': 'req-9c22aff9-f05a-4eff-8fe4-381d33a88606'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:30.435431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:30.452623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c22aff9-f05a-4eff-8fe4-381d33a88606 admin admin] The token cache is being invalidate because project 9e11d33493f348beb87881502d463a71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:12:30.452993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c22aff9-f05a-4eff-8fe4-381d33a88606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:30.454499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c22aff9-f05a-4eff-8fe4-381d33a88606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:30.456498 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2781/5613] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:12:30 2022] DELETE /identity/v3/projects/9e11d33493f348beb87881502d463a71 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:30.462199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97494bb6-2bc9-4ab0-9fef-f7cc5f4661c4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:30.462586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97494bb6-2bc9-4ab0-9fef-f7cc5f4661c4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:30.462908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97494bb6-2bc9-4ab0-9fef-f7cc5f4661c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:30.527046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-97494bb6-2bc9-4ab0-9fef-f7cc5f4661c4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:30.529134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-97494bb6-2bc9-4ab0-9fef-f7cc5f4661c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:30.535946 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2833/5614] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:30.543190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1733c-30ac-44b4-8818-c80f90f0efbe None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:30.549977 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2782/5615] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:30.550205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c5f1733c-30ac-44b4-8818-c80f90f0efbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:30.560361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb156e52-37e7-43e6-8d7a-39225d87f181 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:30.560686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb156e52-37e7-43e6-8d7a-39225d87f181 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:30.560930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb156e52-37e7-43e6-8d7a-39225d87f181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:30.617210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-eb156e52-37e7-43e6-8d7a-39225d87f181 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:30.619582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-eb156e52-37e7-43e6-8d7a-39225d87f181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:30.633138 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2783/5616] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:30.634507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb156e52-37e7-43e6-8d7a-39225d87f181 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:30.645447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb156e52-37e7-43e6-8d7a-39225d87f181 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:30.645669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1733c-30ac-44b4-8818-c80f90f0efbe None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:30.651234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4719f7-ba17-4bcd-9560-0f9e1f7c5de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:30.652433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf86bb-31f7-4c18-bd8c-78bdf1536254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:30.652582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4719f7-ba17-4bcd-9560-0f9e1f7c5de9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:30.653089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf86bb-31f7-4c18-bd8c-78bdf1536254 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:30.653211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4719f7-ba17-4bcd-9560-0f9e1f7c5de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:30.653358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf86bb-31f7-4c18-bd8c-78bdf1536254 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:30.653611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4719f7-ba17-4bcd-9560-0f9e1f7c5de9 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:30.653746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf86bb-31f7-4c18-bd8c-78bdf1536254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:30.654763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4719f7-ba17-4bcd-9560-0f9e1f7c5de9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:12:30.655624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4719f7-ba17-4bcd-9560-0f9e1f7c5de9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:30.657056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4719f7-ba17-4bcd-9560-0f9e1f7c5de9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:30.666796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf86bb-31f7-4c18-bd8c-78bdf1536254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:30.671532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf86bb-31f7-4c18-bd8c-78bdf1536254 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:30.679460 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2784/5617] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:12:30.693253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4719f7-ba17-4bcd-9560-0f9e1f7c5de9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:30.709527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4719f7-ba17-4bcd-9560-0f9e1f7c5de9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:30.712811 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2834/5618] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:12:30.721060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245ade0-014b-4da5-b371-2c3b2c595fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:30.722261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245ade0-014b-4da5-b371-2c3b2c595fcf admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:30.722732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245ade0-014b-4da5-b371-2c3b2c595fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:30.724122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245ade0-014b-4da5-b371-2c3b2c595fcf admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:30.724902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245ade0-014b-4da5-b371-2c3b2c595fcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:30.727471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245ade0-014b-4da5-b371-2c3b2c595fcf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:30.803672 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2785/5619] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:12:30 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:12:30.807653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf86bb-31f7-4c18-bd8c-78bdf1536254 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:30.826521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf86bb-31f7-4c18-bd8c-78bdf1536254 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:30.967585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6bda1-7f40-47dc-8398-09a58b599e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:30.968260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6bda1-7f40-47dc-8398-09a58b599e6a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:30.968557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6bda1-7f40-47dc-8398-09a58b599e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:30.969488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6bda1-7f40-47dc-8398-09a58b599e6a admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:30.974809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6bda1-7f40-47dc-8398-09a58b599e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:30.980694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6bda1-7f40-47dc-8398-09a58b599e6a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:31.041241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245ade0-014b-4da5-b371-2c3b2c595fcf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:31.041472 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2835/5620] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:12:30 2022] POST /identity/v3/users => generated 285 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:12:31.066547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245ade0-014b-4da5-b371-2c3b2c595fcf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:31.080770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc088a-0959-43ec-9469-f97cb2f6cd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:31.081519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc088a-0959-43ec-9469-f97cb2f6cd20 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:31.081857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc088a-0959-43ec-9469-f97cb2f6cd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:31.082264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc088a-0959-43ec-9469-f97cb2f6cd20 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:31.082976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc088a-0959-43ec-9469-f97cb2f6cd20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:31.085410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc088a-0959-43ec-9469-f97cb2f6cd20 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:31.118326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6bda1-7f40-47dc-8398-09a58b599e6a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:31.125356 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2786/5621] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:12:31.130845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6bda1-7f40-47dc-8398-09a58b599e6a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:31.138230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69f549-e324-4002-8d72-2e7107b435ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:31.138952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69f549-e324-4002-8d72-2e7107b435ce admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:31.139310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69f549-e324-4002-8d72-2e7107b435ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:31.139630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69f549-e324-4002-8d72-2e7107b435ce admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:31.140408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69f549-e324-4002-8d72-2e7107b435ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:31.143328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69f549-e324-4002-8d72-2e7107b435ce admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:31.161649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc088a-0959-43ec-9469-f97cb2f6cd20 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:31.167952 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2836/5622] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:12:31.174824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc088a-0959-43ec-9469-f97cb2f6cd20 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:31.179700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c2281-0543-4785-abff-101f64eebc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:31.180448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c2281-0543-4785-abff-101f64eebc36 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:31.180843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c2281-0543-4785-abff-101f64eebc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:31.181211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c2281-0543-4785-abff-101f64eebc36 admin admin] PATH_INFO: `/v3/projects/13380f881cdb4df5bcff2f04cf579d34/users/dbad940a59cc49b28af1629f165a9cee/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:31.203800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c2281-0543-4785-abff-101f64eebc36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13380f881cdb4df5bcff2f04cf579d34, user_id=dbad940a59cc49b28af1629f165a9cee, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:31.207565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c2281-0543-4785-abff-101f64eebc36 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:31.249410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d534a-6fd2-45a6-92e3-1c1bcfcc75d4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:31.249827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d534a-6fd2-45a6-92e3-1c1bcfcc75d4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:31.250236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d534a-6fd2-45a6-92e3-1c1bcfcc75d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:31.250438 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2787/5623] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:12:31 2022] PUT /identity/v3/projects/13380f881cdb4df5bcff2f04cf579d34/users/dbad940a59cc49b28af1629f165a9cee/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:31.263795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a81f0-4f0a-4977-9297-abf9ade9bd70 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:31.264081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a81f0-4f0a-4977-9297-abf9ade9bd70 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:31.264362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a81f0-4f0a-4977-9297-abf9ade9bd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:31.286014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:31.344052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-104d534a-6fd2-45a6-92e3-1c1bcfcc75d4 None None] MFA Rules not processed for user `a0b8f53d77ba4aa88a1eab7d5cd0195d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:31.346590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bf9a81f0-4f0a-4977-9297-abf9ade9bd70 None None] MFA Rules not processed for user `dbad940a59cc49b28af1629f165a9cee`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:31.348005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bf9a81f0-4f0a-4977-9297-abf9ade9bd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:31.349116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-104d534a-6fd2-45a6-92e3-1c1bcfcc75d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:31.426015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:31.464537 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2837/5624] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:12:31.471943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc69611-bf46-47f4-a760-ee0713a7812d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:31.473333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc69611-bf46-47f4-a760-ee0713a7812d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:31.473688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc69611-bf46-47f4-a760-ee0713a7812d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:31.491535 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2788/5625] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:12:31.497881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a81f0-4f0a-4977-9297-abf9ade9bd70 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:31.499210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1cc69611-bf46-47f4-a760-ee0713a7812d None None] MFA Rules not processed for user `a0b8f53d77ba4aa88a1eab7d5cd0195d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:31.500670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1cc69611-bf46-47f4-a760-ee0713a7812d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:31.517979 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2838/5626] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:12:31.526991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc69611-bf46-47f4-a760-ee0713a7812d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:31.529114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1cc69611-bf46-47f4-a760-ee0713a7812d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:31.615915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a81f0-4f0a-4977-9297-abf9ade9bd70 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:31.622221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586a6ba-e9df-47f8-a69a-e814901517ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:31.623042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586a6ba-e9df-47f8-a69a-e814901517ee admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:31.623320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586a6ba-e9df-47f8-a69a-e814901517ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:31.623595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586a6ba-e9df-47f8-a69a-e814901517ee admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:31.624316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586a6ba-e9df-47f8-a69a-e814901517ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:31.626498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586a6ba-e9df-47f8-a69a-e814901517ee admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:31.649814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc69611-bf46-47f4-a760-ee0713a7812d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:31.657358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fecce-14db-46a8-bcd1-73193707c4e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:31.658626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fecce-14db-46a8-bcd1-73193707c4e6 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:31.658899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fecce-14db-46a8-bcd1-73193707c4e6 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:31.659164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fecce-14db-46a8-bcd1-73193707c4e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:31.665121 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2789/5627] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:12:31 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:12:31.669372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586a6ba-e9df-47f8-a69a-e814901517ee admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:31.675092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fecce-14db-46a8-bcd1-73193707c4e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:31.676748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fecce-14db-46a8-bcd1-73193707c4e6 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:31.687086 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2839/5628] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:12:31.785650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586a6ba-e9df-47f8-a69a-e814901517ee admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:31.791148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0557c-81bd-442f-8fcc-6614fca8dce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:31.791863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0557c-81bd-442f-8fcc-6614fca8dce8 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:31.792274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0557c-81bd-442f-8fcc-6614fca8dce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:31.792656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0557c-81bd-442f-8fcc-6614fca8dce8 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:31.793422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0557c-81bd-442f-8fcc-6614fca8dce8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:31.795661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0557c-81bd-442f-8fcc-6614fca8dce8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:31.840247 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2790/5629] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:12:31 2022] POST /identity/v3/users => generated 291 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:12:31.844289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fecce-14db-46a8-bcd1-73193707c4e6 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:31.860882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fecce-14db-46a8-bcd1-73193707c4e6 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:31.871075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc3714-667e-45c3-8c21-b42f0c260dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:31.871814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc3714-667e-45c3-8c21-b42f0c260dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:31.872209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc3714-667e-45c3-8c21-b42f0c260dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:31.872620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc3714-667e-45c3-8c21-b42f0c260dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:31.873488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc3714-667e-45c3-8c21-b42f0c260dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:31.875538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc3714-667e-45c3-8c21-b42f0c260dc3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:31.890990 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2840/5630] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:12:31.895146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0557c-81bd-442f-8fcc-6614fca8dce8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:31.910035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0557c-81bd-442f-8fcc-6614fca8dce8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:31.917459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774af3e5-c2be-40a1-a8ab-9f073201c6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:31.918595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774af3e5-c2be-40a1-a8ab-9f073201c6c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:31.918744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774af3e5-c2be-40a1-a8ab-9f073201c6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:31.919194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774af3e5-c2be-40a1-a8ab-9f073201c6c0 admin admin] PATH_INFO: `/v3/projects/bad102793f01450b8304c759b0be8164/users/6c63bcba93e34a38a41a183b863d736c/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:31.923719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:31.950354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774af3e5-c2be-40a1-a8ab-9f073201c6c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad102793f01450b8304c759b0be8164, user_id=6c63bcba93e34a38a41a183b863d736c, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:31.953858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774af3e5-c2be-40a1-a8ab-9f073201c6c0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:31.975059 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2791/5631] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:12:31 2022] PUT /identity/v3/projects/bad102793f01450b8304c759b0be8164/users/6c63bcba93e34a38a41a183b863d736c/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:31.979227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc3714-667e-45c3-8c21-b42f0c260dc3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:31.992537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc3714-667e-45c3-8c21-b42f0c260dc3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:32.106714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacdc29c-5273-4958-bbda-508496f2e3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:32.107695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdc29c-5273-4958-bbda-508496f2e3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:32.107968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdc29c-5273-4958-bbda-508496f2e3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:32.108231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdc29c-5273-4958-bbda-508496f2e3f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:32.108913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacdc29c-5273-4958-bbda-508496f2e3f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:32.110637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacdc29c-5273-4958-bbda-508496f2e3f7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:32.133751 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2841/5632] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:12:32.138098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774af3e5-c2be-40a1-a8ab-9f073201c6c0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:32.148779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774af3e5-c2be-40a1-a8ab-9f073201c6c0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:32.172304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab95ba-f321-4f9a-948c-aba1d4fa364c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:32.175759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab95ba-f321-4f9a-948c-aba1d4fa364c admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:32.176243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab95ba-f321-4f9a-948c-aba1d4fa364c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:32.176554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab95ba-f321-4f9a-948c-aba1d4fa364c admin admin] PATH_INFO: `/v3/projects/bad102793f01450b8304c759b0be8164/users/6c63bcba93e34a38a41a183b863d736c/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:32.184576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab95ba-f321-4f9a-948c-aba1d4fa364c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad102793f01450b8304c759b0be8164, user_id=6c63bcba93e34a38a41a183b863d736c, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:32.187481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab95ba-f321-4f9a-948c-aba1d4fa364c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:32.216628 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2792/5633] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:12:32 2022] PUT /identity/v3/projects/bad102793f01450b8304c759b0be8164/users/6c63bcba93e34a38a41a183b863d736c/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:32.226875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1848d-6fcc-45a6-9a9f-c2cbffab6af5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:32.227145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1848d-6fcc-45a6-9a9f-c2cbffab6af5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:32.227406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1848d-6fcc-45a6-9a9f-c2cbffab6af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:32.281856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fdf1848d-6fcc-45a6-9a9f-c2cbffab6af5 None None] MFA Rules not processed for user `6c63bcba93e34a38a41a183b863d736c`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:32.293226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fdf1848d-6fcc-45a6-9a9f-c2cbffab6af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:32.340341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:32.398710 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2842/5634] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:32.407451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab95ba-f321-4f9a-948c-aba1d4fa364c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:32.507373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab95ba-f321-4f9a-948c-aba1d4fa364c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:32.512074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc1be1-bfc3-404a-9f63-138ca08f8d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:32.513137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc1be1-bfc3-404a-9f63-138ca08f8d52 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:32.522871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc1be1-bfc3-404a-9f63-138ca08f8d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:32.523539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc1be1-bfc3-404a-9f63-138ca08f8d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:32.536263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc1be1-bfc3-404a-9f63-138ca08f8d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:32.539961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc1be1-bfc3-404a-9f63-138ca08f8d52 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:32.557688 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2793/5635] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:12:32.924590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1848d-6fcc-45a6-9a9f-c2cbffab6af5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:32.926972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fdf1848d-6fcc-45a6-9a9f-c2cbffab6af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:33.046201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1848d-6fcc-45a6-9a9f-c2cbffab6af5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:33.053834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ea94d-a412-47cd-81e9-f3d3fdc9b420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:33.054596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ea94d-a412-47cd-81e9-f3d3fdc9b420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:33.054936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ea94d-a412-47cd-81e9-f3d3fdc9b420 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:33.055530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ea94d-a412-47cd-81e9-f3d3fdc9b420 admin admin] PATH_INFO: `/v3/users/a0b8f53d77ba4aa88a1eab7d5cd0195d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:33.056716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ea94d-a412-47cd-81e9-f3d3fdc9b420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0b8f53d77ba4aa88a1eab7d5cd0195d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:33.058887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ea94d-a412-47cd-81e9-f3d3fdc9b420 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:33.118178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d81ea94d-a412-47cd-81e9-f3d3fdc9b420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0b8f53d77ba4aa88a1eab7d5cd0195d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:33.130576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d81ea94d-a412-47cd-81e9-f3d3fdc9b420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0b8f53d77ba4aa88a1eab7d5cd0195d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:33.140535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d81ea94d-a412-47cd-81e9-f3d3fdc9b420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0b8f53d77ba4aa88a1eab7d5cd0195d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:33.190490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d81ea94d-a412-47cd-81e9-f3d3fdc9b420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0b8f53d77ba4aa88a1eab7d5cd0195d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:33.191675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d81ea94d-a412-47cd-81e9-f3d3fdc9b420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0b8f53d77ba4aa88a1eab7d5cd0195d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:33.212478 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2843/5636] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:12:32 2022] DELETE /identity/v3/users/a0b8f53d77ba4aa88a1eab7d5cd0195d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:33.563334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc1be1-bfc3-404a-9f63-138ca08f8d52 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:33.591792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc1be1-bfc3-404a-9f63-138ca08f8d52 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:33.687046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb64338-9c18-4ada-930f-0013871035e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:33.687630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb64338-9c18-4ada-930f-0013871035e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:33.687978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb64338-9c18-4ada-930f-0013871035e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:33.688252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb64338-9c18-4ada-930f-0013871035e8 admin admin] PATH_INFO: `/v3/projects/fd632566dc1d43a0bf3a7106b877773a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:33.689660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb64338-9c18-4ada-930f-0013871035e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd632566dc1d43a0bf3a7106b877773a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:33.693101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb64338-9c18-4ada-930f-0013871035e8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:33.810479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cdb64338-9c18-4ada-930f-0013871035e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd632566dc1d43a0bf3a7106b877773a', 'request_id': 'req-cdb64338-9c18-4ada-930f-0013871035e8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:33.818267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cdb64338-9c18-4ada-930f-0013871035e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd632566dc1d43a0bf3a7106b877773a', 'request_id': 'req-cdb64338-9c18-4ada-930f-0013871035e8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:33.831742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cdb64338-9c18-4ada-930f-0013871035e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd632566dc1d43a0bf3a7106b877773a', 'request_id': 'req-cdb64338-9c18-4ada-930f-0013871035e8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:33.881175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cdb64338-9c18-4ada-930f-0013871035e8 admin admin] The token cache is being invalidate because project fd632566dc1d43a0bf3a7106b877773a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:12:33.881680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cdb64338-9c18-4ada-930f-0013871035e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:33.882626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cdb64338-9c18-4ada-930f-0013871035e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:33.884677 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2794/5637] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:12:33 2022] DELETE /identity/v3/projects/fd632566dc1d43a0bf3a7106b877773a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:33.889809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b44eb-3330-4bf9-9abc-2c673a4e5075 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:33.890738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b44eb-3330-4bf9-9abc-2c673a4e5075 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:33.891337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b44eb-3330-4bf9-9abc-2c673a4e5075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:33.933788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c14b44eb-3330-4bf9-9abc-2c673a4e5075 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:33.935481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c14b44eb-3330-4bf9-9abc-2c673a4e5075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:34.021674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:34.047593 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2844/5638] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:34.058495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6f69d-429a-424b-903a-dcf9f02961dd None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:34.058969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6f69d-429a-424b-903a-dcf9f02961dd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:34.059291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6f69d-429a-424b-903a-dcf9f02961dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:34.080615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:34.096636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2fc6f69d-429a-424b-903a-dcf9f02961dd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:34.099374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2fc6f69d-429a-424b-903a-dcf9f02961dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:34.117770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b44eb-3330-4bf9-9abc-2c673a4e5075 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:34.125701 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2795/5639] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:34.131147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b44eb-3330-4bf9-9abc-2c673a4e5075 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:34.141214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463dc8b-45e8-4636-8069-d8bc6417a9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:34.145522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463dc8b-45e8-4636-8069-d8bc6417a9ac admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:34.146181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463dc8b-45e8-4636-8069-d8bc6417a9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:34.146688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463dc8b-45e8-4636-8069-d8bc6417a9ac admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:34.147603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463dc8b-45e8-4636-8069-d8bc6417a9ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:12:34.149260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463dc8b-45e8-4636-8069-d8bc6417a9ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:34.151409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463dc8b-45e8-4636-8069-d8bc6417a9ac admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:34.177611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6f69d-429a-424b-903a-dcf9f02961dd None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:34.190491 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2845/5640] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:12:34.193360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6f69d-429a-424b-903a-dcf9f02961dd None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:34.198854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d13ea-5df2-4743-9b95-9e4b00fe628b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:34.200001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d13ea-5df2-4743-9b95-9e4b00fe628b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:34.200278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d13ea-5df2-4743-9b95-9e4b00fe628b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:34.200580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d13ea-5df2-4743-9b95-9e4b00fe628b admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:34.201202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d13ea-5df2-4743-9b95-9e4b00fe628b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:34.208445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d13ea-5df2-4743-9b95-9e4b00fe628b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:34.271120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463dc8b-45e8-4636-8069-d8bc6417a9ac admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:34.276972 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2796/5641] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:12:34 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:12:34.306313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463dc8b-45e8-4636-8069-d8bc6417a9ac admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:34.413222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578dd03-e187-48ef-a972-ef708999f333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:34.415521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578dd03-e187-48ef-a972-ef708999f333 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:34.416097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578dd03-e187-48ef-a972-ef708999f333 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:34.416649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578dd03-e187-48ef-a972-ef708999f333 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:34.420918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578dd03-e187-48ef-a972-ef708999f333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:34.423465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578dd03-e187-48ef-a972-ef708999f333 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:34.461745 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2846/5642] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:12:34 2022] POST /identity/v3/users => generated 295 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:12:34.466744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d13ea-5df2-4743-9b95-9e4b00fe628b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:34.484221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d13ea-5df2-4743-9b95-9e4b00fe628b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:34.492297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901e956-ca9d-4223-857a-dfd64eca3709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:34.492959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901e956-ca9d-4223-857a-dfd64eca3709 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:34.493286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901e956-ca9d-4223-857a-dfd64eca3709 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:34.493549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901e956-ca9d-4223-857a-dfd64eca3709 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:34.494193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901e956-ca9d-4223-857a-dfd64eca3709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:34.495983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901e956-ca9d-4223-857a-dfd64eca3709 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:34.514367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578dd03-e187-48ef-a972-ef708999f333 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:34.514623 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2797/5643] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:12:34.528531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578dd03-e187-48ef-a972-ef708999f333 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:34.535808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea158e-6f8d-44a6-9ac2-4f12fb4a142b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:34.536822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea158e-6f8d-44a6-9ac2-4f12fb4a142b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:34.537091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea158e-6f8d-44a6-9ac2-4f12fb4a142b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:34.537296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea158e-6f8d-44a6-9ac2-4f12fb4a142b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:34.538052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea158e-6f8d-44a6-9ac2-4f12fb4a142b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:34.540187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea158e-6f8d-44a6-9ac2-4f12fb4a142b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:34.558428 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2847/5644] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:12:34.563700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901e956-ca9d-4223-857a-dfd64eca3709 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:34.577638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901e956-ca9d-4223-857a-dfd64eca3709 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:34.585463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5cc4b-da4d-4c5a-9c4c-7fa625e64a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:34.585463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5cc4b-da4d-4c5a-9c4c-7fa625e64a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:34.585463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5cc4b-da4d-4c5a-9c4c-7fa625e64a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:34.586979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5cc4b-da4d-4c5a-9c4c-7fa625e64a3f admin admin] PATH_INFO: `/v3/projects/094f627bb04848519808048a66b2010f/users/51ac29b4f090431c836479723ef1f71d/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:34.619119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e5cc4b-da4d-4c5a-9c4c-7fa625e64a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=094f627bb04848519808048a66b2010f, user_id=51ac29b4f090431c836479723ef1f71d, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:34.626631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e5cc4b-da4d-4c5a-9c4c-7fa625e64a3f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:34.648281 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2798/5645] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:12:34 2022] PUT /identity/v3/projects/094f627bb04848519808048a66b2010f/users/51ac29b4f090431c836479723ef1f71d/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:34.653852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecf45f-1014-4846-847c-f31402cbb5cd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:34.654270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecf45f-1014-4846-847c-f31402cbb5cd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:34.654843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecf45f-1014-4846-847c-f31402cbb5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:34.700473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-00ecf45f-1014-4846-847c-f31402cbb5cd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:34.703803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-00ecf45f-1014-4846-847c-f31402cbb5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:34.808243 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2848/5646] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:12:34.838170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5cc4b-da4d-4c5a-9c4c-7fa625e64a3f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:34.841250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-14e5cc4b-da4d-4c5a-9c4c-7fa625e64a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:34.976901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5cc4b-da4d-4c5a-9c4c-7fa625e64a3f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:34.983026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bb539-3512-4103-a275-78ad884f4695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:34.983675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bb539-3512-4103-a275-78ad884f4695 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:34.983950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bb539-3512-4103-a275-78ad884f4695 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:34.988638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bb539-3512-4103-a275-78ad884f4695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:35.009481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bb539-3512-4103-a275-78ad884f4695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:35.016038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bb539-3512-4103-a275-78ad884f4695 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:35.029076 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2799/5647] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:12:44.132763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ecf45f-1014-4846-847c-f31402cbb5cd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:44.134434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-00ecf45f-1014-4846-847c-f31402cbb5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:44.233293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:44.283022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ecf45f-1014-4846-847c-f31402cbb5cd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:44.293249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fde4b-2b4f-4a8a-a127-c68807bba294 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:44.294045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fde4b-2b4f-4a8a-a127-c68807bba294 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:44.294418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fde4b-2b4f-4a8a-a127-c68807bba294 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:44.294790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fde4b-2b4f-4a8a-a127-c68807bba294 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:44.299592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-396fde4b-2b4f-4a8a-a127-c68807bba294 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:44.335606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fde4b-2b4f-4a8a-a127-c68807bba294 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:44.337279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fde4b-2b4f-4a8a-a127-c68807bba294 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:44.358695 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2849/5648] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:12:44.389374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bb539-3512-4103-a275-78ad884f4695 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:44.391068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-eb7bb539-3512-4103-a275-78ad884f4695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:44.544783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bb539-3512-4103-a275-78ad884f4695 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:44.553705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489248b-372f-4a0f-9224-a9cca73c8199 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c63bcba93e34a38a41a183b863d736c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bad102793f01450b8304c759b0be8164', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:44.554456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489248b-372f-4a0f-9224-a9cca73c8199 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:44.554990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489248b-372f-4a0f-9224-a9cca73c8199 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:44.555358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489248b-372f-4a0f-9224-a9cca73c8199 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:44.556119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489248b-372f-4a0f-9224-a9cca73c8199 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:12:44.556591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489248b-372f-4a0f-9224-a9cca73c8199 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:44.558443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489248b-372f-4a0f-9224-a9cca73c8199 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:44.588484 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2800/5649] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:12:44.592668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fde4b-2b4f-4a8a-a127-c68807bba294 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:44.610350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fde4b-2b4f-4a8a-a127-c68807bba294 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:44.632946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7815831-68f3-4109-8a8a-b2acafc40f20 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c63bcba93e34a38a41a183b863d736c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bad102793f01450b8304c759b0be8164', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:44.633604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7815831-68f3-4109-8a8a-b2acafc40f20 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:44.634065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7815831-68f3-4109-8a8a-b2acafc40f20 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:44.634343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7815831-68f3-4109-8a8a-b2acafc40f20 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:44.635754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7815831-68f3-4109-8a8a-b2acafc40f20 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:44.638098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7815831-68f3-4109-8a8a-b2acafc40f20 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:44.745836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489248b-372f-4a0f-9224-a9cca73c8199 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:44.746040 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2850/5650] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:12:44 2022] POST /identity/v3/projects => generated 347 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:12:44.764806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489248b-372f-4a0f-9224-a9cca73c8199 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:44.867206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:44.937996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718dde8-ba99-4356-83b5-1d5bc66abc8c tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c63bcba93e34a38a41a183b863d736c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bad102793f01450b8304c759b0be8164', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:44.940769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718dde8-ba99-4356-83b5-1d5bc66abc8c tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:44.941234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718dde8-ba99-4356-83b5-1d5bc66abc8c tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:44.941655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718dde8-ba99-4356-83b5-1d5bc66abc8c tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:44.942864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718dde8-ba99-4356-83b5-1d5bc66abc8c tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:12:44.945798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718dde8-ba99-4356-83b5-1d5bc66abc8c tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:44.952868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718dde8-ba99-4356-83b5-1d5bc66abc8c tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:44.998834 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2801/5651] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:12:45.932819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7815831-68f3-4109-8a8a-b2acafc40f20 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:45.948242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7815831-68f3-4109-8a8a-b2acafc40f20 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:45.962878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d154f9-b40a-407d-b5c8-0793f817fb17 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c63bcba93e34a38a41a183b863d736c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bad102793f01450b8304c759b0be8164', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:45.963564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d154f9-b40a-407d-b5c8-0793f817fb17 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:45.964002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d154f9-b40a-407d-b5c8-0793f817fb17 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:45.964282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d154f9-b40a-407d-b5c8-0793f817fb17 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] PATH_INFO: `/v3/projects/d19fc51ff75b4f8e83fd798235844374` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:45.965669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d154f9-b40a-407d-b5c8-0793f817fb17 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d19fc51ff75b4f8e83fd798235844374)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:45.967651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d154f9-b40a-407d-b5c8-0793f817fb17 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:46.064615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7d154f9-b40a-407d-b5c8-0793f817fb17 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd19fc51ff75b4f8e83fd798235844374', 'request_id': 'req-a7d154f9-b40a-407d-b5c8-0793f817fb17'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:46.082377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7d154f9-b40a-407d-b5c8-0793f817fb17 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd19fc51ff75b4f8e83fd798235844374', 'request_id': 'req-a7d154f9-b40a-407d-b5c8-0793f817fb17'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:46.097091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7d154f9-b40a-407d-b5c8-0793f817fb17 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd19fc51ff75b4f8e83fd798235844374', 'request_id': 'req-a7d154f9-b40a-407d-b5c8-0793f817fb17'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:46.118772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7d154f9-b40a-407d-b5c8-0793f817fb17 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] The token cache is being invalidate because project d19fc51ff75b4f8e83fd798235844374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:12:46.119062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7d154f9-b40a-407d-b5c8-0793f817fb17 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:46.121112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7d154f9-b40a-407d-b5c8-0793f817fb17 tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:12:46.123278 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2851/5652] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:12:45 2022] DELETE /identity/v3/projects/d19fc51ff75b4f8e83fd798235844374 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:12:46.135038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718dde8-ba99-4356-83b5-1d5bc66abc8c tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:46.136893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1718dde8-ba99-4356-83b5-1d5bc66abc8c tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:46.333099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718dde8-ba99-4356-83b5-1d5bc66abc8c tempest-RoutersAdminTest-1836731016 tempest-RoutersAdminTest-1836731016-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:46.435642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b8812-e87b-4557-a620-3b16dfcd7f62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:46.436512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b8812-e87b-4557-a620-3b16dfcd7f62 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:46.436787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b8812-e87b-4557-a620-3b16dfcd7f62 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:46.437056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b8812-e87b-4557-a620-3b16dfcd7f62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:46.439346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-038b8812-e87b-4557-a620-3b16dfcd7f62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:46.587525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b8812-e87b-4557-a620-3b16dfcd7f62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:46.597185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b8812-e87b-4557-a620-3b16dfcd7f62 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:46.620900 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2802/5653] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:12:47.186775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a574ddf-eaac-461e-ab88-c27c15fd6891 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:47.187324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a574ddf-eaac-461e-ab88-c27c15fd6891 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:47.187407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a574ddf-eaac-461e-ab88-c27c15fd6891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:47.259073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7a574ddf-eaac-461e-ab88-c27c15fd6891 None None] MFA Rules not processed for user `51ac29b4f090431c836479723ef1f71d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:47.262038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7a574ddf-eaac-461e-ab88-c27c15fd6891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:47.471436 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2852/5654] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:47.490002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b8812-e87b-4557-a620-3b16dfcd7f62 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:47.495853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-038b8812-e87b-4557-a620-3b16dfcd7f62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:47.595595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:12:47.647810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b8812-e87b-4557-a620-3b16dfcd7f62 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:47.658420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0706a2d-e582-4dec-866e-d92cf0b7fd03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:47.659177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0706a2d-e582-4dec-866e-d92cf0b7fd03 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:47.659518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0706a2d-e582-4dec-866e-d92cf0b7fd03 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:47.659850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0706a2d-e582-4dec-866e-d92cf0b7fd03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:47.678372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0706a2d-e582-4dec-866e-d92cf0b7fd03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:47.685283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0706a2d-e582-4dec-866e-d92cf0b7fd03 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:47.700999 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2803/5655] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:12:52.290222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3d900-b377-4d2f-b81a-b15aabc7339a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:52.290728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3d900-b377-4d2f-b81a-b15aabc7339a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:52.290805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3d900-b377-4d2f-b81a-b15aabc7339a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:52.314746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f7a3d900-b377-4d2f-b81a-b15aabc7339a None None] MFA Rules not processed for user `51ac29b4f090431c836479723ef1f71d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:12:52.316152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f7a3d900-b377-4d2f-b81a-b15aabc7339a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:52.324869 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2853/5656] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:12:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:12:54.769749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0706a2d-e582-4dec-866e-d92cf0b7fd03 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:12:54.771919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f0706a2d-e582-4dec-866e-d92cf0b7fd03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:54.884392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0706a2d-e582-4dec-866e-d92cf0b7fd03 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:12:54.890324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b507c3-48b3-479c-9826-79d9d2c58fa3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:12:54.891214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b507c3-48b3-479c-9826-79d9d2c58fa3 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:12:54.891499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b507c3-48b3-479c-9826-79d9d2c58fa3 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:12:54.891772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b507c3-48b3-479c-9826-79d9d2c58fa3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:12:54.894319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a3b507c3-48b3-479c-9826-79d9d2c58fa3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:12:54.913176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b507c3-48b3-479c-9826-79d9d2c58fa3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:12:54.915852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b507c3-48b3-479c-9826-79d9d2c58fa3 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:12:54.930621 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2804/5657] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:12:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:13:00.501342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3d900-b377-4d2f-b81a-b15aabc7339a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:00.502953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f7a3d900-b377-4d2f-b81a-b15aabc7339a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:00.541551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:00.635478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3d900-b377-4d2f-b81a-b15aabc7339a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:00.646729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:00.647617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:00.649548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:00.650144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] PATH_INFO: `/v3/users/f127011ceab246a8834d50f01f426487` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:00.653137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f127011ceab246a8834d50f01f426487)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:00.658418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:00.735258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f127011ceab246a8834d50f01f426487'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:00.750392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f127011ceab246a8834d50f01f426487'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:00.762961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f127011ceab246a8834d50f01f426487'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:00.794394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f127011ceab246a8834d50f01f426487'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:00.795322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f127011ceab246a8834d50f01f426487'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:00.808407 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2854/5658] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:13:00 2022] DELETE /identity/v3/users/f127011ceab246a8834d50f01f426487 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:01.670040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b507c3-48b3-479c-9826-79d9d2c58fa3 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:01.686730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b507c3-48b3-479c-9826-79d9d2c58fa3 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:01.828633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:01.829284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:01.829556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:01.829819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] PATH_INFO: `/v3/projects/a265133763e7444e80adc0cd8541340c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:01.834026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a265133763e7444e80adc0cd8541340c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:01.835985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:01.917125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:01.918585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a265133763e7444e80adc0cd8541340c', 'request_id': 'req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:01.930109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a265133763e7444e80adc0cd8541340c', 'request_id': 'req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:01.947103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a265133763e7444e80adc0cd8541340c', 'request_id': 'req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:01.975289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] The token cache is being invalidate because project a265133763e7444e80adc0cd8541340c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:13:01.975765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:01.976895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:01.987046 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2805/5659] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:13:01 2022] DELETE /identity/v3/projects/a265133763e7444e80adc0cd8541340c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:01.988547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:01.990301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:02.025230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f43c3c-de7a-4dd7-9b19-42bfad0ae9bf admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:02.167932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:02.169529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:02.169973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:02.170986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] PATH_INFO: `/v3/users/7cec2ab9334d4317bd17951c53ef2560` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:02.175226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cec2ab9334d4317bd17951c53ef2560)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:02.181210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:02.258161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:02.268596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cec2ab9334d4317bd17951c53ef2560'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:02.285059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cec2ab9334d4317bd17951c53ef2560'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:02.301497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cec2ab9334d4317bd17951c53ef2560'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:02.326209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cec2ab9334d4317bd17951c53ef2560'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:02.326969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cec2ab9334d4317bd17951c53ef2560'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:02.340839 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2855/5660] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:13:01 2022] DELETE /identity/v3/users/7cec2ab9334d4317bd17951c53ef2560 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:03.327211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:03.330027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:03.357458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f385b0-4a6e-4ec5-be1a-d99991c6d4ba admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:03.494838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34c2bc-b8a7-49f6-8224-bbf6707181da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:03.495827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34c2bc-b8a7-49f6-8224-bbf6707181da admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:03.496263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34c2bc-b8a7-49f6-8224-bbf6707181da admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:03.496723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34c2bc-b8a7-49f6-8224-bbf6707181da admin admin] PATH_INFO: `/v3/projects/582064e0834345e2a13aec61f9f5bee2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:03.498180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34c2bc-b8a7-49f6-8224-bbf6707181da admin admin] RBAC: Authorizing `identity:delete_project(project_id=582064e0834345e2a13aec61f9f5bee2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:03.500629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34c2bc-b8a7-49f6-8224-bbf6707181da admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:03.566129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d34c2bc-b8a7-49f6-8224-bbf6707181da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '582064e0834345e2a13aec61f9f5bee2', 'request_id': 'req-8d34c2bc-b8a7-49f6-8224-bbf6707181da'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:03.574907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d34c2bc-b8a7-49f6-8224-bbf6707181da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '582064e0834345e2a13aec61f9f5bee2', 'request_id': 'req-8d34c2bc-b8a7-49f6-8224-bbf6707181da'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:03.600255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d34c2bc-b8a7-49f6-8224-bbf6707181da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '582064e0834345e2a13aec61f9f5bee2', 'request_id': 'req-8d34c2bc-b8a7-49f6-8224-bbf6707181da'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:03.616909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d34c2bc-b8a7-49f6-8224-bbf6707181da admin admin] The token cache is being invalidate because project 582064e0834345e2a13aec61f9f5bee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:13:03.617998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d34c2bc-b8a7-49f6-8224-bbf6707181da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:03.618885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d34c2bc-b8a7-49f6-8224-bbf6707181da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:03.621526 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2806/5661] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:13:03 2022] DELETE /identity/v3/projects/582064e0834345e2a13aec61f9f5bee2 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:12.331428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=14,keystone:DELETE=4,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:13.611526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=58,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:26.377102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:26.379713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:26.410983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1b03f-741c-4ece-9176-b33b32142bd1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:26.489041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:26.490018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:26.490359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:26.490698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] PATH_INFO: `/v3/users/dbad940a59cc49b28af1629f165a9cee` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:26.491966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbad940a59cc49b28af1629f165a9cee)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:26.494298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:26.616942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbad940a59cc49b28af1629f165a9cee'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:26.641828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbad940a59cc49b28af1629f165a9cee'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:26.652402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbad940a59cc49b28af1629f165a9cee'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:26.672116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbad940a59cc49b28af1629f165a9cee'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:26.673169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbad940a59cc49b28af1629f165a9cee'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:26.683386 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2856/5662] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:13:26 2022] DELETE /identity/v3/users/dbad940a59cc49b28af1629f165a9cee => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:26.685176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ed8fb-2a6e-4570-b5d9-ba112500a1be None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:26.685796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ed8fb-2a6e-4570-b5d9-ba112500a1be None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:26.685921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ed8fb-2a6e-4570-b5d9-ba112500a1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:26.711116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f75ed8fb-2a6e-4570-b5d9-ba112500a1be None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:26.712325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f75ed8fb-2a6e-4570-b5d9-ba112500a1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:26.801724 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2807/5663] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:13:26.811631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:26.813102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:26.880979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2eeffb-cc04-467d-beab-a57c87c67833 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:26.885125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9791f8-cf1a-4eaa-bf89-19a557e5248e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:26.886160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9791f8-cf1a-4eaa-bf89-19a557e5248e service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:26.886550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9791f8-cf1a-4eaa-bf89-19a557e5248e service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:26.886965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9791f8-cf1a-4eaa-bf89-19a557e5248e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:26.896950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9791f8-cf1a-4eaa-bf89-19a557e5248e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:26.898368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9791f8-cf1a-4eaa-bf89-19a557e5248e service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:26.908536 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2857/5664] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:13:27.471453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ed8fb-2a6e-4570-b5d9-ba112500a1be None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:27.473999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f75ed8fb-2a6e-4570-b5d9-ba112500a1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:27.495616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ed8fb-2a6e-4570-b5d9-ba112500a1be None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:27.502217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:27.503082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:27.510821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:27.511304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] PATH_INFO: `/v3/projects/13380f881cdb4df5bcff2f04cf579d34` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:27.513560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13380f881cdb4df5bcff2f04cf579d34)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:27.519095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:27.602063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13380f881cdb4df5bcff2f04cf579d34', 'request_id': 'req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:27.610308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13380f881cdb4df5bcff2f04cf579d34', 'request_id': 'req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:27.621798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13380f881cdb4df5bcff2f04cf579d34', 'request_id': 'req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:27.638737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] The token cache is being invalidate because project 13380f881cdb4df5bcff2f04cf579d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:13:27.639015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:27.640922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:27.643372 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2808/5665] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:13:27 2022] DELETE /identity/v3/projects/13380f881cdb4df5bcff2f04cf579d34 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:27.651697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9791f8-cf1a-4eaa-bf89-19a557e5248e service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:27.668533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9791f8-cf1a-4eaa-bf89-19a557e5248e service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:27.693246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:27.757213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f5ede-7a02-4ac1-9cd8-7363ee07028d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:27.758010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f5ede-7a02-4ac1-9cd8-7363ee07028d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:27.758347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f5ede-7a02-4ac1-9cd8-7363ee07028d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:27.758916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f5ede-7a02-4ac1-9cd8-7363ee07028d admin admin] PATH_INFO: `/v3/users/6c63bcba93e34a38a41a183b863d736c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:27.761097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f5ede-7a02-4ac1-9cd8-7363ee07028d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c63bcba93e34a38a41a183b863d736c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:27.763161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f5ede-7a02-4ac1-9cd8-7363ee07028d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:27.884824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-725f5ede-7a02-4ac1-9cd8-7363ee07028d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c63bcba93e34a38a41a183b863d736c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:27.896283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-725f5ede-7a02-4ac1-9cd8-7363ee07028d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c63bcba93e34a38a41a183b863d736c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:27.907598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-725f5ede-7a02-4ac1-9cd8-7363ee07028d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c63bcba93e34a38a41a183b863d736c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:27.929202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-725f5ede-7a02-4ac1-9cd8-7363ee07028d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c63bcba93e34a38a41a183b863d736c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:27.930169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-725f5ede-7a02-4ac1-9cd8-7363ee07028d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c63bcba93e34a38a41a183b863d736c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:27.941926 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2858/5666] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:13:27 2022] DELETE /identity/v3/users/6c63bcba93e34a38a41a183b863d736c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:28.370409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:28.372532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:28.388763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5ccd9-2e65-4d92-aa0c-02f322870f47 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:28.530936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:28.532238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:28.532747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:28.533160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa admin admin] PATH_INFO: `/v3/projects/bad102793f01450b8304c759b0be8164` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:28.534611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad102793f01450b8304c759b0be8164)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:28.537554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:28.588971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:28.632518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad102793f01450b8304c759b0be8164', 'request_id': 'req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:28.642938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad102793f01450b8304c759b0be8164', 'request_id': 'req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:28.663681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad102793f01450b8304c759b0be8164', 'request_id': 'req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:28.681614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa admin admin] The token cache is being invalidate because project bad102793f01450b8304c759b0be8164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:13:28.682154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:28.683024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c6c32342-b1af-48a9-8cc6-23fc20ef6afa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:28.685875 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2809/5667] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:13:28 2022] DELETE /identity/v3/projects/bad102793f01450b8304c759b0be8164 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:28.691503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb57f7-8f53-4aea-8cb0-dc3fd5e9e293 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:28.691949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb57f7-8f53-4aea-8cb0-dc3fd5e9e293 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:28.692068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb57f7-8f53-4aea-8cb0-dc3fd5e9e293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:28.767136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a8eb57f7-8f53-4aea-8cb0-dc3fd5e9e293 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:28.768341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a8eb57f7-8f53-4aea-8cb0-dc3fd5e9e293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:28.844450 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2859/5668] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:13:28.853565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b9d20-ed65-4aaf-aee0-88821120c232 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:28.853847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b9d20-ed65-4aaf-aee0-88821120c232 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:28.854120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b9d20-ed65-4aaf-aee0-88821120c232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:28.876547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e42b9d20-ed65-4aaf-aee0-88821120c232 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:28.877971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e42b9d20-ed65-4aaf-aee0-88821120c232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:28.887942 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2810/5669] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:13:28.892853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb57f7-8f53-4aea-8cb0-dc3fd5e9e293 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:28.903760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb57f7-8f53-4aea-8cb0-dc3fd5e9e293 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:28.918945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfdb36-bbfb-4641-b017-9fb0b75816a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:28.920495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfdb36-bbfb-4641-b017-9fb0b75816a1 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:28.921067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfdb36-bbfb-4641-b017-9fb0b75816a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:28.921741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfdb36-bbfb-4641-b017-9fb0b75816a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:28.922834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfdb36-bbfb-4641-b017-9fb0b75816a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:13:28.923351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfdb36-bbfb-4641-b017-9fb0b75816a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:28.926596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfdb36-bbfb-4641-b017-9fb0b75816a1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:28.944706 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2860/5670] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:13:28.949366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b9d20-ed65-4aaf-aee0-88821120c232 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:28.962359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b9d20-ed65-4aaf-aee0-88821120c232 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:28.969942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6099af5-ed63-49a2-94d4-5eeaeeb3119b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:28.972169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6099af5-ed63-49a2-94d4-5eeaeeb3119b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:28.972636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6099af5-ed63-49a2-94d4-5eeaeeb3119b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:28.972904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6099af5-ed63-49a2-94d4-5eeaeeb3119b admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:28.973556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6099af5-ed63-49a2-94d4-5eeaeeb3119b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:28.975705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6099af5-ed63-49a2-94d4-5eeaeeb3119b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:29.010498 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2811/5671] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:13:28 2022] POST /identity/v3/projects => generated 395 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:13:29.015582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfdb36-bbfb-4641-b017-9fb0b75816a1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:29.026856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfdb36-bbfb-4641-b017-9fb0b75816a1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:29.039673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:29.095589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1d662-b269-48ec-9455-422823274227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:29.096251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1d662-b269-48ec-9455-422823274227 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:29.096557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1d662-b269-48ec-9455-422823274227 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:29.096940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1d662-b269-48ec-9455-422823274227 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:29.097577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1d662-b269-48ec-9455-422823274227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:29.100286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1d662-b269-48ec-9455-422823274227 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:29.127023 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2861/5672] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:13:29 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:13:29.131129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6099af5-ed63-49a2-94d4-5eeaeeb3119b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:29.143405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6099af5-ed63-49a2-94d4-5eeaeeb3119b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:29.149565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55968d96-b527-4996-acb1-4a91161ad05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:29.150178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55968d96-b527-4996-acb1-4a91161ad05c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:29.150623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55968d96-b527-4996-acb1-4a91161ad05c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:29.150888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55968d96-b527-4996-acb1-4a91161ad05c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:29.151526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55968d96-b527-4996-acb1-4a91161ad05c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:29.153323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55968d96-b527-4996-acb1-4a91161ad05c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:29.168274 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2812/5673] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:13:29.169411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1d662-b269-48ec-9455-422823274227 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:29.181660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1d662-b269-48ec-9455-422823274227 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:29.189234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc9702-0049-4457-af68-b3b4aa2b4a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:29.190263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc9702-0049-4457-af68-b3b4aa2b4a78 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:29.190790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc9702-0049-4457-af68-b3b4aa2b4a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:29.192010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc9702-0049-4457-af68-b3b4aa2b4a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:29.193151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc9702-0049-4457-af68-b3b4aa2b4a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:29.195521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc9702-0049-4457-af68-b3b4aa2b4a78 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:29.214770 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2862/5674] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:13:29.219161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55968d96-b527-4996-acb1-4a91161ad05c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:29.252807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55968d96-b527-4996-acb1-4a91161ad05c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:29.262207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a876757-172e-4db4-b3c0-e3c63ee56a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:29.263217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a876757-172e-4db4-b3c0-e3c63ee56a4a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:29.265133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a876757-172e-4db4-b3c0-e3c63ee56a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:29.265715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a876757-172e-4db4-b3c0-e3c63ee56a4a admin admin] PATH_INFO: `/v3/projects/2cfdf3b6eea44d3893013f1b4ba56191/users/77333b157e014945bfe41255a6f3cecd/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:29.288875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a876757-172e-4db4-b3c0-e3c63ee56a4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cfdf3b6eea44d3893013f1b4ba56191, user_id=77333b157e014945bfe41255a6f3cecd, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:29.292685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a876757-172e-4db4-b3c0-e3c63ee56a4a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:29.338286 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2813/5675] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:13:29 2022] PUT /identity/v3/projects/2cfdf3b6eea44d3893013f1b4ba56191/users/77333b157e014945bfe41255a6f3cecd/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:29.343818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafbb52-27bf-4324-a0a1-843f24071c47 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:29.344113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafbb52-27bf-4324-a0a1-843f24071c47 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:29.344460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafbb52-27bf-4324-a0a1-843f24071c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:29.393968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cfafbb52-27bf-4324-a0a1-843f24071c47 None None] MFA Rules not processed for user `77333b157e014945bfe41255a6f3cecd`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:29.395340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cfafbb52-27bf-4324-a0a1-843f24071c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:29.490619 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2863/5676] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:13:29.498252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a876757-172e-4db4-b3c0-e3c63ee56a4a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:29.500950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0a876757-172e-4db4-b3c0-e3c63ee56a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:29.579753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a876757-172e-4db4-b3c0-e3c63ee56a4a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:29.584544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3772194-396f-4cf6-bc31-297e528b2eae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:29.585161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3772194-396f-4cf6-bc31-297e528b2eae service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:29.585435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3772194-396f-4cf6-bc31-297e528b2eae service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:29.585702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3772194-396f-4cf6-bc31-297e528b2eae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:29.598595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3772194-396f-4cf6-bc31-297e528b2eae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:29.600122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3772194-396f-4cf6-bc31-297e528b2eae service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:29.620976 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2814/5677] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:13:29 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:13:33.601986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafbb52-27bf-4324-a0a1-843f24071c47 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:33.604185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cfafbb52-27bf-4324-a0a1-843f24071c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:33.769599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafbb52-27bf-4324-a0a1-843f24071c47 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:33.785571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1b83a-b25f-45f1-b80e-069c674c7b94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:33.786213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1b83a-b25f-45f1-b80e-069c674c7b94 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:33.786483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1b83a-b25f-45f1-b80e-069c674c7b94 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:33.786746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1b83a-b25f-45f1-b80e-069c674c7b94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:33.788622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fdd1b83a-b25f-45f1-b80e-069c674c7b94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:33.821320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1b83a-b25f-45f1-b80e-069c674c7b94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:33.823278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1b83a-b25f-45f1-b80e-069c674c7b94 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:33.840650 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2864/5678] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:13:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:13:37.872437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3772194-396f-4cf6-bc31-297e528b2eae service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:37.874194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c3772194-396f-4cf6-bc31-297e528b2eae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:37.892672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3772194-396f-4cf6-bc31-297e528b2eae service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:37.905696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847ce1a-43bc-4550-9987-2ae6c038bb6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:37.906634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847ce1a-43bc-4550-9987-2ae6c038bb6a service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:37.907104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847ce1a-43bc-4550-9987-2ae6c038bb6a service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:37.907587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847ce1a-43bc-4550-9987-2ae6c038bb6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:37.914436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c847ce1a-43bc-4550-9987-2ae6c038bb6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:37.974380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:38.020036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847ce1a-43bc-4550-9987-2ae6c038bb6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:38.024976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847ce1a-43bc-4550-9987-2ae6c038bb6a service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:38.067009 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2815/5679] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:13:43.831834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:47.388070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1b83a-b25f-45f1-b80e-069c674c7b94 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:47.398769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1b83a-b25f-45f1-b80e-069c674c7b94 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:47.404115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:47.405533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:47.406430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:47.406894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] PATH_INFO: `/v3/users/77333b157e014945bfe41255a6f3cecd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:47.408326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=77333b157e014945bfe41255a6f3cecd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:47.411162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:47.636065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77333b157e014945bfe41255a6f3cecd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:47.661327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77333b157e014945bfe41255a6f3cecd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:47.677116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77333b157e014945bfe41255a6f3cecd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:47.698576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77333b157e014945bfe41255a6f3cecd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:47.699229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77333b157e014945bfe41255a6f3cecd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:47.714597 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2865/5680] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:13:47 2022] DELETE /identity/v3/users/77333b157e014945bfe41255a6f3cecd => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:47.716425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951350a-599c-43d7-8468-6553a528f588 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:47.716895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951350a-599c-43d7-8468-6553a528f588 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:47.717017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951350a-599c-43d7-8468-6553a528f588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:47.748081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5951350a-599c-43d7-8468-6553a528f588 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:47.749750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5951350a-599c-43d7-8468-6553a528f588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:47.838277 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2816/5681] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:13:47.849413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:47.851317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:47.966391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbb5b3-7456-44e5-91c8-8da451435f4b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:47.971562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733bb27-c49a-4f92-b477-f33d8fdb85d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:47.972193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733bb27-c49a-4f92-b477-f33d8fdb85d5 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:47.972673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733bb27-c49a-4f92-b477-f33d8fdb85d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:47.972941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733bb27-c49a-4f92-b477-f33d8fdb85d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:47.984606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733bb27-c49a-4f92-b477-f33d8fdb85d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:47.986332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733bb27-c49a-4f92-b477-f33d8fdb85d5 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:48.002003 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2866/5682] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:13:48.282073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951350a-599c-43d7-8468-6553a528f588 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:48.283896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5951350a-599c-43d7-8468-6553a528f588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:48.302371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951350a-599c-43d7-8468-6553a528f588 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:48.307465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:48.308286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:48.308599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:48.308893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d admin admin] PATH_INFO: `/v3/projects/2cfdf3b6eea44d3893013f1b4ba56191` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:48.310052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cfdf3b6eea44d3893013f1b4ba56191)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:48.312573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:48.387108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cfdf3b6eea44d3893013f1b4ba56191', 'request_id': 'req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:48.394843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cfdf3b6eea44d3893013f1b4ba56191', 'request_id': 'req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:48.408820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cfdf3b6eea44d3893013f1b4ba56191', 'request_id': 'req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:48.425975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d admin admin] The token cache is being invalidate because project 2cfdf3b6eea44d3893013f1b4ba56191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:13:48.426563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:48.427286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-711c8913-aa7a-4e6c-bc8f-32d8fe383c6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:48.429273 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2817/5683] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:13:48 2022] DELETE /identity/v3/projects/2cfdf3b6eea44d3893013f1b4ba56191 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:48.434021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927586c3-e1dc-441a-9933-3e0728381932 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:48.434540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927586c3-e1dc-441a-9933-3e0728381932 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:48.434540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927586c3-e1dc-441a-9933-3e0728381932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:48.461808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-927586c3-e1dc-441a-9933-3e0728381932 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:48.462938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-927586c3-e1dc-441a-9933-3e0728381932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:48.547993 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2867/5684] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:13:48.555161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15459d-3a38-4ad7-8481-800d7181ac06 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:48.558470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15459d-3a38-4ad7-8481-800d7181ac06 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:48.558900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15459d-3a38-4ad7-8481-800d7181ac06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:48.597559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2f15459d-3a38-4ad7-8481-800d7181ac06 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:48.598910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2f15459d-3a38-4ad7-8481-800d7181ac06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:48.634965 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2818/5685] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:13:48.640279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927586c3-e1dc-441a-9933-3e0728381932 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:48.658582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927586c3-e1dc-441a-9933-3e0728381932 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:48.684550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264c0eb-62df-4289-8433-17e6d785fc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:48.685225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264c0eb-62df-4289-8433-17e6d785fc90 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:48.685597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264c0eb-62df-4289-8433-17e6d785fc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:48.685930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264c0eb-62df-4289-8433-17e6d785fc90 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:48.687176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264c0eb-62df-4289-8433-17e6d785fc90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:13:48.687550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264c0eb-62df-4289-8433-17e6d785fc90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:48.690656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264c0eb-62df-4289-8433-17e6d785fc90 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:48.708169 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2868/5686] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:13:48.712485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15459d-3a38-4ad7-8481-800d7181ac06 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:48.725407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15459d-3a38-4ad7-8481-800d7181ac06 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:48.731837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d2083-313f-4f3b-8f74-627641cc9099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:48.733136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d2083-313f-4f3b-8f74-627641cc9099 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:48.733791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d2083-313f-4f3b-8f74-627641cc9099 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:48.734678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d2083-313f-4f3b-8f74-627641cc9099 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:48.736000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d2083-313f-4f3b-8f74-627641cc9099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:48.739629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d2083-313f-4f3b-8f74-627641cc9099 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:48.795517 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2819/5687] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:13:48 2022] POST /identity/v3/projects => generated 389 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:13:48.799622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264c0eb-62df-4289-8433-17e6d785fc90 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:48.805566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:48.813715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264c0eb-62df-4289-8433-17e6d785fc90 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:48.922882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41990a47-73af-46be-9706-cb2a8ecfb61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:48.925356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41990a47-73af-46be-9706-cb2a8ecfb61b admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:48.926330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41990a47-73af-46be-9706-cb2a8ecfb61b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:48.928515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41990a47-73af-46be-9706-cb2a8ecfb61b admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:48.929537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41990a47-73af-46be-9706-cb2a8ecfb61b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:48.931717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41990a47-73af-46be-9706-cb2a8ecfb61b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:48.987867 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2869/5688] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:13:48 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:13:48.991733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d2083-313f-4f3b-8f74-627641cc9099 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:49.002559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d2083-313f-4f3b-8f74-627641cc9099 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:49.007552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffd5ff-9a46-4e66-91df-561d27a1da60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:49.008302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffd5ff-9a46-4e66-91df-561d27a1da60 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:49.008696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffd5ff-9a46-4e66-91df-561d27a1da60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:49.009061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffd5ff-9a46-4e66-91df-561d27a1da60 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:49.009805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffd5ff-9a46-4e66-91df-561d27a1da60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:49.011550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffd5ff-9a46-4e66-91df-561d27a1da60 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:49.025006 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2820/5689] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:13:49.028852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41990a47-73af-46be-9706-cb2a8ecfb61b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:49.053543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41990a47-73af-46be-9706-cb2a8ecfb61b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:49.065758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c930ee5-861b-41e9-a4be-25ab3ef0cd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:49.066715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c930ee5-861b-41e9-a4be-25ab3ef0cd5a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:49.067137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c930ee5-861b-41e9-a4be-25ab3ef0cd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:49.067418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c930ee5-861b-41e9-a4be-25ab3ef0cd5a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:49.068068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c930ee5-861b-41e9-a4be-25ab3ef0cd5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:49.070090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c930ee5-861b-41e9-a4be-25ab3ef0cd5a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:49.085146 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2870/5690] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:13:49.089740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffd5ff-9a46-4e66-91df-561d27a1da60 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:49.117838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffd5ff-9a46-4e66-91df-561d27a1da60 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:49.136565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0827a6-1e32-49b8-8201-f025fd198848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:49.137267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0827a6-1e32-49b8-8201-f025fd198848 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:49.137714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0827a6-1e32-49b8-8201-f025fd198848 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:49.137827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0827a6-1e32-49b8-8201-f025fd198848 admin admin] PATH_INFO: `/v3/projects/2cc2d68895d84e55b1c5f1ae3385e27a/users/34c993b8f205458bbc4361b921b11980/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:49.401302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0827a6-1e32-49b8-8201-f025fd198848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc2d68895d84e55b1c5f1ae3385e27a, user_id=34c993b8f205458bbc4361b921b11980, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:49.405577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0827a6-1e32-49b8-8201-f025fd198848 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:49.429938 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2821/5691] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:13:49 2022] PUT /identity/v3/projects/2cc2d68895d84e55b1c5f1ae3385e27a/users/34c993b8f205458bbc4361b921b11980/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:13:49.538702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8a94d-51d1-4895-842f-ed3095fbce60 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:49.539504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8a94d-51d1-4895-842f-ed3095fbce60 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:49.539937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8a94d-51d1-4895-842f-ed3095fbce60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:49.595983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6cd8a94d-51d1-4895-842f-ed3095fbce60 None None] MFA Rules not processed for user `34c993b8f205458bbc4361b921b11980`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:49.597250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6cd8a94d-51d1-4895-842f-ed3095fbce60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:49.768208 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2871/5692] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:13:49.803165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0827a6-1e32-49b8-8201-f025fd198848 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:49.806076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5a0827a6-1e32-49b8-8201-f025fd198848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:49.821264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:49.972636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0827a6-1e32-49b8-8201-f025fd198848 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:49.988689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db654b6c-7a32-4907-a887-e7ea67d0fdc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:49.989787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db654b6c-7a32-4907-a887-e7ea67d0fdc4 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:49.990110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db654b6c-7a32-4907-a887-e7ea67d0fdc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:49.990383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db654b6c-7a32-4907-a887-e7ea67d0fdc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:50.013117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db654b6c-7a32-4907-a887-e7ea67d0fdc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:50.015169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db654b6c-7a32-4907-a887-e7ea67d0fdc4 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:50.048195 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2822/5693] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:13:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:13:50.339404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8a94d-51d1-4895-842f-ed3095fbce60 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:50.340898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6cd8a94d-51d1-4895-842f-ed3095fbce60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:50.457956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8a94d-51d1-4895-842f-ed3095fbce60 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:50.463149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:50.464223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:50.464613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:50.464945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] PATH_INFO: `/v3/users/8afc45f50dc54b84a8abf3b6e168426b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:50.466303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8afc45f50dc54b84a8abf3b6e168426b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:50.468735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:50.497435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:50.537752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8afc45f50dc54b84a8abf3b6e168426b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:50.549193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8afc45f50dc54b84a8abf3b6e168426b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:50.558410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8afc45f50dc54b84a8abf3b6e168426b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:50.580859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8afc45f50dc54b84a8abf3b6e168426b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:50.581971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8afc45f50dc54b84a8abf3b6e168426b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:50.597567 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2872/5694] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:13:50 2022] DELETE /identity/v3/users/8afc45f50dc54b84a8abf3b6e168426b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:51.017707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db654b6c-7a32-4907-a887-e7ea67d0fdc4 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:51.038547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db654b6c-7a32-4907-a887-e7ea67d0fdc4 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:51.130328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:51.130990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:51.131263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:51.131528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] PATH_INFO: `/v3/projects/ea18513aca0a4723b3ae3043624a52da` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:51.134136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea18513aca0a4723b3ae3043624a52da)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:51.137047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:51.228910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea18513aca0a4723b3ae3043624a52da', 'request_id': 'req-872f7630-1393-4b8a-b48c-55e28c1cfb2b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:51.270301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea18513aca0a4723b3ae3043624a52da', 'request_id': 'req-872f7630-1393-4b8a-b48c-55e28c1cfb2b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:51.289427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea18513aca0a4723b3ae3043624a52da', 'request_id': 'req-872f7630-1393-4b8a-b48c-55e28c1cfb2b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:51.308568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] The token cache is being invalidate because project ea18513aca0a4723b3ae3043624a52da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:13:51.309005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:51.310026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:51.311975 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2823/5695] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:13:51 2022] DELETE /identity/v3/projects/ea18513aca0a4723b3ae3043624a52da => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:51.316049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:51.317863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:51.345960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201916cd-03c2-4847-922b-b7b636e1c8d4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:51.491276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7b4da-d433-4f9a-9fd5-7b6f8ecf6de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:51.492701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7b4da-d433-4f9a-9fd5-7b6f8ecf6de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:51.493144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7b4da-d433-4f9a-9fd5-7b6f8ecf6de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:51.494992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7b4da-d433-4f9a-9fd5-7b6f8ecf6de9 admin admin] PATH_INFO: `/v3/users/0802ea75a81747e68e9bfa7d11ef50dc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:51.525793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7b4da-d433-4f9a-9fd5-7b6f8ecf6de9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0802ea75a81747e68e9bfa7d11ef50dc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:51.527709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7b4da-d433-4f9a-9fd5-7b6f8ecf6de9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:51.613557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fcf7b4da-d433-4f9a-9fd5-7b6f8ecf6de9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0802ea75a81747e68e9bfa7d11ef50dc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:51.627151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fcf7b4da-d433-4f9a-9fd5-7b6f8ecf6de9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0802ea75a81747e68e9bfa7d11ef50dc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:51.650239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fcf7b4da-d433-4f9a-9fd5-7b6f8ecf6de9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0802ea75a81747e68e9bfa7d11ef50dc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:51.679739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fcf7b4da-d433-4f9a-9fd5-7b6f8ecf6de9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0802ea75a81747e68e9bfa7d11ef50dc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:51.681401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fcf7b4da-d433-4f9a-9fd5-7b6f8ecf6de9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0802ea75a81747e68e9bfa7d11ef50dc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:51.683164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:51.693469 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2873/5696] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:13:51 2022] DELETE /identity/v3/users/0802ea75a81747e68e9bfa7d11ef50dc => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:52.378779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:52.380479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:52.402450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f7630-1393-4b8a-b48c-55e28c1cfb2b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:52.467442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:52.495937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389444c-3ea5-4e01-ad2a-b5c55542dd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:52.497113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389444c-3ea5-4e01-ad2a-b5c55542dd17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:52.497446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389444c-3ea5-4e01-ad2a-b5c55542dd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:52.497717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389444c-3ea5-4e01-ad2a-b5c55542dd17 admin admin] PATH_INFO: `/v3/projects/d5241d90bd18473790afe7eec33ec9d2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:52.513506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389444c-3ea5-4e01-ad2a-b5c55542dd17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5241d90bd18473790afe7eec33ec9d2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:52.515974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389444c-3ea5-4e01-ad2a-b5c55542dd17 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:52.590595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3389444c-3ea5-4e01-ad2a-b5c55542dd17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5241d90bd18473790afe7eec33ec9d2', 'request_id': 'req-3389444c-3ea5-4e01-ad2a-b5c55542dd17'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:52.599745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3389444c-3ea5-4e01-ad2a-b5c55542dd17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5241d90bd18473790afe7eec33ec9d2', 'request_id': 'req-3389444c-3ea5-4e01-ad2a-b5c55542dd17'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:52.613325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3389444c-3ea5-4e01-ad2a-b5c55542dd17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5241d90bd18473790afe7eec33ec9d2', 'request_id': 'req-3389444c-3ea5-4e01-ad2a-b5c55542dd17'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:52.658606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3389444c-3ea5-4e01-ad2a-b5c55542dd17 admin admin] The token cache is being invalidate because project d5241d90bd18473790afe7eec33ec9d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:13:52.662716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3389444c-3ea5-4e01-ad2a-b5c55542dd17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:52.663753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3389444c-3ea5-4e01-ad2a-b5c55542dd17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:52.668280 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2824/5697] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:13:52 2022] DELETE /identity/v3/projects/d5241d90bd18473790afe7eec33ec9d2 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:52.679579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfde79-7970-4be0-9f3f-6f5b6b0f4a61 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:52.680189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfde79-7970-4be0-9f3f-6f5b6b0f4a61 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:52.680271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfde79-7970-4be0-9f3f-6f5b6b0f4a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:52.726409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dfcfde79-7970-4be0-9f3f-6f5b6b0f4a61 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:52.727650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dfcfde79-7970-4be0-9f3f-6f5b6b0f4a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:52.893208 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2874/5698] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:13:52.914091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc960a8-b015-4232-96ba-398c975a5c19 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:52.914376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc960a8-b015-4232-96ba-398c975a5c19 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:52.914643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc960a8-b015-4232-96ba-398c975a5c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:52.958871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7bc960a8-b015-4232-96ba-398c975a5c19 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:52.973206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7bc960a8-b015-4232-96ba-398c975a5c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:52.981865 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2825/5699] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:13:52.987619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfde79-7970-4be0-9f3f-6f5b6b0f4a61 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:53.003594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfde79-7970-4be0-9f3f-6f5b6b0f4a61 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:53.021216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174f5ab-bdda-4194-8d13-043a2c58b18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:53.021666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174f5ab-bdda-4194-8d13-043a2c58b18d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:53.022133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174f5ab-bdda-4194-8d13-043a2c58b18d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:53.022596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174f5ab-bdda-4194-8d13-043a2c58b18d admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:53.024030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174f5ab-bdda-4194-8d13-043a2c58b18d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:13:53.026180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174f5ab-bdda-4194-8d13-043a2c58b18d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:53.038038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174f5ab-bdda-4194-8d13-043a2c58b18d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:53.072138 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2875/5700] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:13:53.073640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc960a8-b015-4232-96ba-398c975a5c19 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:53.091208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc960a8-b015-4232-96ba-398c975a5c19 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:53.098960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454bd1a-2429-428a-89e4-2c556ddf5840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:53.099613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454bd1a-2429-428a-89e4-2c556ddf5840 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:53.099884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454bd1a-2429-428a-89e4-2c556ddf5840 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:53.100284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454bd1a-2429-428a-89e4-2c556ddf5840 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:53.100921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454bd1a-2429-428a-89e4-2c556ddf5840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:53.105655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454bd1a-2429-428a-89e4-2c556ddf5840 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:53.186167 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2826/5701] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:13:53 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:13:53.187730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174f5ab-bdda-4194-8d13-043a2c58b18d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:53.200651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174f5ab-bdda-4194-8d13-043a2c58b18d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:53.305375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89c1fa-25b8-49cd-b4a0-05bb9f68bea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:53.306713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89c1fa-25b8-49cd-b4a0-05bb9f68bea2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:53.307139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89c1fa-25b8-49cd-b4a0-05bb9f68bea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:53.307582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89c1fa-25b8-49cd-b4a0-05bb9f68bea2 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:53.308445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89c1fa-25b8-49cd-b4a0-05bb9f68bea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:53.310563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89c1fa-25b8-49cd-b4a0-05bb9f68bea2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:53.339908 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2876/5702] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:13:53 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:13:53.344235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454bd1a-2429-428a-89e4-2c556ddf5840 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:53.358173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454bd1a-2429-428a-89e4-2c556ddf5840 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:53.363773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16520768-6509-4bcc-ad39-00314b22533f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:53.364743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16520768-6509-4bcc-ad39-00314b22533f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:53.365206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16520768-6509-4bcc-ad39-00314b22533f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:53.365692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16520768-6509-4bcc-ad39-00314b22533f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:53.366531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16520768-6509-4bcc-ad39-00314b22533f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:53.368671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16520768-6509-4bcc-ad39-00314b22533f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:53.385062 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2827/5703] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:13:53.388816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89c1fa-25b8-49cd-b4a0-05bb9f68bea2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:53.401140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89c1fa-25b8-49cd-b4a0-05bb9f68bea2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:53.436202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa740ed-f8fd-4310-a903-66cb43218907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:53.437885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa740ed-f8fd-4310-a903-66cb43218907 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:53.439761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa740ed-f8fd-4310-a903-66cb43218907 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:53.440213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa740ed-f8fd-4310-a903-66cb43218907 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:53.441069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa740ed-f8fd-4310-a903-66cb43218907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:53.445517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa740ed-f8fd-4310-a903-66cb43218907 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:53.465223 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2877/5704] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:13:53.488570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16520768-6509-4bcc-ad39-00314b22533f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:53.506336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16520768-6509-4bcc-ad39-00314b22533f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:53.522754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad17371-bf25-4493-b9b0-89d320797d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:53.523446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad17371-bf25-4493-b9b0-89d320797d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:53.523726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad17371-bf25-4493-b9b0-89d320797d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:53.523996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad17371-bf25-4493-b9b0-89d320797d13 admin admin] PATH_INFO: `/v3/projects/3b15cdd8d7fa4da79626028b15b99382/users/9671d01ad0d44cc99e0eecc966f7c3e6/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:53.556492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad17371-bf25-4493-b9b0-89d320797d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b15cdd8d7fa4da79626028b15b99382, user_id=9671d01ad0d44cc99e0eecc966f7c3e6, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:53.563233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad17371-bf25-4493-b9b0-89d320797d13 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:53.584972 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2828/5705] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:13:53 2022] PUT /identity/v3/projects/3b15cdd8d7fa4da79626028b15b99382/users/9671d01ad0d44cc99e0eecc966f7c3e6/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:53.598478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80c46d-dbdb-4360-b0c6-4c5879b4a0a2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:53.599433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80c46d-dbdb-4360-b0c6-4c5879b4a0a2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:53.599598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80c46d-dbdb-4360-b0c6-4c5879b4a0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:53.674309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cd80c46d-dbdb-4360-b0c6-4c5879b4a0a2 None None] MFA Rules not processed for user `9671d01ad0d44cc99e0eecc966f7c3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:53.684629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cd80c46d-dbdb-4360-b0c6-4c5879b4a0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:53.758412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:53.773665 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2878/5706] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:13:53.777740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad17371-bf25-4493-b9b0-89d320797d13 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:53.928538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:53.935338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad17371-bf25-4493-b9b0-89d320797d13 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:53.947147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015cbb5-90df-47f9-8cb0-4d1f06f91526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:53.948018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015cbb5-90df-47f9-8cb0-4d1f06f91526 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:53.948140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015cbb5-90df-47f9-8cb0-4d1f06f91526 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:53.948404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015cbb5-90df-47f9-8cb0-4d1f06f91526 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:53.949365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5015cbb5-90df-47f9-8cb0-4d1f06f91526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:53.951389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5015cbb5-90df-47f9-8cb0-4d1f06f91526 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:53.995542 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2829/5707] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:13:53 2022] POST /identity/v3/projects => generated 375 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:13:53.999821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80c46d-dbdb-4360-b0c6-4c5879b4a0a2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:54.012405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80c46d-dbdb-4360-b0c6-4c5879b4a0a2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:54.110926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549497d9-6291-4c16-8c34-82dc36d21c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:54.111591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549497d9-6291-4c16-8c34-82dc36d21c75 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:54.111863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549497d9-6291-4c16-8c34-82dc36d21c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:54.112151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549497d9-6291-4c16-8c34-82dc36d21c75 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:54.113184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549497d9-6291-4c16-8c34-82dc36d21c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:54.114765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549497d9-6291-4c16-8c34-82dc36d21c75 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:54.153165 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2879/5708] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:13:53 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:13:54.157522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015cbb5-90df-47f9-8cb0-4d1f06f91526 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:54.174868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015cbb5-90df-47f9-8cb0-4d1f06f91526 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:54.184284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fdb48-7234-44a5-b241-b90bf6a6ed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:54.186438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fdb48-7234-44a5-b241-b90bf6a6ed39 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:54.188576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fdb48-7234-44a5-b241-b90bf6a6ed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:54.189239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fdb48-7234-44a5-b241-b90bf6a6ed39 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:54.190168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fdb48-7234-44a5-b241-b90bf6a6ed39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:54.211259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fdb48-7234-44a5-b241-b90bf6a6ed39 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:54.251727 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2830/5709] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:13:54.256549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549497d9-6291-4c16-8c34-82dc36d21c75 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:54.272603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549497d9-6291-4c16-8c34-82dc36d21c75 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:54.286375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61aa20-b172-49fc-a12e-227b573612a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:54.287076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61aa20-b172-49fc-a12e-227b573612a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:54.287344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61aa20-b172-49fc-a12e-227b573612a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:54.287615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61aa20-b172-49fc-a12e-227b573612a6 admin admin] PATH_INFO: `/v3/projects/1786ebd38fb94e9596bd11d4cec5132a/users/7117e6dc0cf343c890d617e86b6ec5a8/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:54.352494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61aa20-b172-49fc-a12e-227b573612a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1786ebd38fb94e9596bd11d4cec5132a, user_id=7117e6dc0cf343c890d617e86b6ec5a8, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:54.358727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61aa20-b172-49fc-a12e-227b573612a6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:54.401518 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2880/5710] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:13:54 2022] PUT /identity/v3/projects/1786ebd38fb94e9596bd11d4cec5132a/users/7117e6dc0cf343c890d617e86b6ec5a8/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:54.405578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fdb48-7234-44a5-b241-b90bf6a6ed39 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:54.417112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fdb48-7234-44a5-b241-b90bf6a6ed39 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:54.638246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0245f-af2c-4e0b-9f5b-6d646ba93058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:54.639156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0245f-af2c-4e0b-9f5b-6d646ba93058 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:54.639755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0245f-af2c-4e0b-9f5b-6d646ba93058 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:54.643038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0245f-af2c-4e0b-9f5b-6d646ba93058 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:54.653278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0245f-af2c-4e0b-9f5b-6d646ba93058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:54.665546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0245f-af2c-4e0b-9f5b-6d646ba93058 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:54.698964 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2831/5711] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:13:54.703145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61aa20-b172-49fc-a12e-227b573612a6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:54.721622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61aa20-b172-49fc-a12e-227b573612a6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:54.733180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcd19b-dd68-48c7-9101-0662269f5fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:54.733868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcd19b-dd68-48c7-9101-0662269f5fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:54.734987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcd19b-dd68-48c7-9101-0662269f5fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:54.735275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcd19b-dd68-48c7-9101-0662269f5fe4 admin admin] PATH_INFO: `/v3/projects/1786ebd38fb94e9596bd11d4cec5132a/users/7117e6dc0cf343c890d617e86b6ec5a8/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:54.744176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcd19b-dd68-48c7-9101-0662269f5fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1786ebd38fb94e9596bd11d4cec5132a, user_id=7117e6dc0cf343c890d617e86b6ec5a8, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:54.748129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcd19b-dd68-48c7-9101-0662269f5fe4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:54.777915 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2881/5712] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:13:54 2022] PUT /identity/v3/projects/1786ebd38fb94e9596bd11d4cec5132a/users/7117e6dc0cf343c890d617e86b6ec5a8/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:54.791155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598dbb86-8d4d-4d90-9e64-a99e6a33856e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:54.792365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598dbb86-8d4d-4d90-9e64-a99e6a33856e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:54.792808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598dbb86-8d4d-4d90-9e64-a99e6a33856e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:54.869590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-598dbb86-8d4d-4d90-9e64-a99e6a33856e None None] MFA Rules not processed for user `7117e6dc0cf343c890d617e86b6ec5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:54.871003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-598dbb86-8d4d-4d90-9e64-a99e6a33856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:55.064882 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2832/5713] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:13:55.078961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141804c2-e4a8-43c6-b84d-6d277462134c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:55.079381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141804c2-e4a8-43c6-b84d-6d277462134c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:55.080347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141804c2-e4a8-43c6-b84d-6d277462134c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:55.111291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-141804c2-e4a8-43c6-b84d-6d277462134c None None] MFA Rules not processed for user `9671d01ad0d44cc99e0eecc966f7c3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:55.112695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-141804c2-e4a8-43c6-b84d-6d277462134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:55.193775 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2882/5714] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:13:55.209516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598dbb86-8d4d-4d90-9e64-a99e6a33856e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:55.211351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-598dbb86-8d4d-4d90-9e64-a99e6a33856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:55.324820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:55.411002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598dbb86-8d4d-4d90-9e64-a99e6a33856e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:55.428935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216b141-8cd9-44cb-b6d6-0d63bbafe027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:55.429579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216b141-8cd9-44cb-b6d6-0d63bbafe027 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:55.429850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216b141-8cd9-44cb-b6d6-0d63bbafe027 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:55.430117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216b141-8cd9-44cb-b6d6-0d63bbafe027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:55.454075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216b141-8cd9-44cb-b6d6-0d63bbafe027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:55.459445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216b141-8cd9-44cb-b6d6-0d63bbafe027 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:55.487706 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2833/5715] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:13:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:13:59.730460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141804c2-e4a8-43c6-b84d-6d277462134c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:13:59.732002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-141804c2-e4a8-43c6-b84d-6d277462134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:13:59.794864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:13:59.812966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141804c2-e4a8-43c6-b84d-6d277462134c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:13:59.817649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:13:59.818297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:59.818562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:59.819031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] PATH_INFO: `/v3/users/549ccd150d7b4a8a882e6fbda33129ea` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:59.820879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=549ccd150d7b4a8a882e6fbda33129ea)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:13:59.823344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:13:59.882684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '549ccd150d7b4a8a882e6fbda33129ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:59.892806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '549ccd150d7b4a8a882e6fbda33129ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:59.904110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '549ccd150d7b4a8a882e6fbda33129ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:59.921921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '549ccd150d7b4a8a882e6fbda33129ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:59.922661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '549ccd150d7b4a8a882e6fbda33129ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:13:59.931988 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2883/5716] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:13:59 2022] DELETE /identity/v3/users/549ccd150d7b4a8a882e6fbda33129ea => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:13:59.937172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db6da3-22df-43c7-b7e8-ef13906bdd19 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:13:59.937666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db6da3-22df-43c7-b7e8-ef13906bdd19 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:13:59.938278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db6da3-22df-43c7-b7e8-ef13906bdd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:13:59.963068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d9db6da3-22df-43c7-b7e8-ef13906bdd19 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:13:59.964297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d9db6da3-22df-43c7-b7e8-ef13906bdd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:00.040531 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2834/5717] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:00.060656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:00.065005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:00.187991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897660b1-99a9-4bd8-9c7a-b2e4875afd8c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:00.193400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81a337-0360-40c1-8600-0e3b542d0ef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:00.194424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81a337-0360-40c1-8600-0e3b542d0ef3 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:00.195408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81a337-0360-40c1-8600-0e3b542d0ef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:00.195882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81a337-0360-40c1-8600-0e3b542d0ef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:00.209977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81a337-0360-40c1-8600-0e3b542d0ef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:00.211916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81a337-0360-40c1-8600-0e3b542d0ef3 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:00.230457 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2884/5718] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:14:00.731749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db6da3-22df-43c7-b7e8-ef13906bdd19 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:00.733385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d9db6da3-22df-43c7-b7e8-ef13906bdd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:00.746059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db6da3-22df-43c7-b7e8-ef13906bdd19 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:00.751529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135de74-25e8-41be-b266-9e94624aae98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:00.752573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135de74-25e8-41be-b266-9e94624aae98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:00.753006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135de74-25e8-41be-b266-9e94624aae98 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:00.753585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135de74-25e8-41be-b266-9e94624aae98 admin admin] PATH_INFO: `/v3/projects/5f16924ecb4e466e94a815b44528130d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:00.755007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135de74-25e8-41be-b266-9e94624aae98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f16924ecb4e466e94a815b44528130d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:00.772468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135de74-25e8-41be-b266-9e94624aae98 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:00.830826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4135de74-25e8-41be-b266-9e94624aae98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f16924ecb4e466e94a815b44528130d', 'request_id': 'req-4135de74-25e8-41be-b266-9e94624aae98'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:00.839550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4135de74-25e8-41be-b266-9e94624aae98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f16924ecb4e466e94a815b44528130d', 'request_id': 'req-4135de74-25e8-41be-b266-9e94624aae98'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:00.849387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4135de74-25e8-41be-b266-9e94624aae98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f16924ecb4e466e94a815b44528130d', 'request_id': 'req-4135de74-25e8-41be-b266-9e94624aae98'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:00.862753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4135de74-25e8-41be-b266-9e94624aae98 admin admin] The token cache is being invalidate because project 5f16924ecb4e466e94a815b44528130d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:14:00.863080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4135de74-25e8-41be-b266-9e94624aae98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:00.863705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4135de74-25e8-41be-b266-9e94624aae98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:00.869211 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2835/5719] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:14:00 2022] DELETE /identity/v3/projects/5f16924ecb4e466e94a815b44528130d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:14:00.870917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914f990-2e2a-459f-b688-64dfb3f22812 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:00.871860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914f990-2e2a-459f-b688-64dfb3f22812 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:00.872272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914f990-2e2a-459f-b688-64dfb3f22812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:00.894684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e914f990-2e2a-459f-b688-64dfb3f22812 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:00.896132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e914f990-2e2a-459f-b688-64dfb3f22812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:00.979131 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2885/5720] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:14:00.986955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7c566-9fc6-4520-952b-8d5918b3810f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:00.987499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7c566-9fc6-4520-952b-8d5918b3810f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:00.987597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7c566-9fc6-4520-952b-8d5918b3810f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:01.013352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-faa7c566-9fc6-4520-952b-8d5918b3810f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:01.015557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-faa7c566-9fc6-4520-952b-8d5918b3810f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:01.024242 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2836/5721] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:14:01.028595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914f990-2e2a-459f-b688-64dfb3f22812 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:01.040106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914f990-2e2a-459f-b688-64dfb3f22812 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:01.045867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cda6ba-e887-440e-80b0-2569520c9125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:01.046693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cda6ba-e887-440e-80b0-2569520c9125 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:01.047026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cda6ba-e887-440e-80b0-2569520c9125 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:01.047605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cda6ba-e887-440e-80b0-2569520c9125 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:01.048196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cda6ba-e887-440e-80b0-2569520c9125 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:14:01.049202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cda6ba-e887-440e-80b0-2569520c9125 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:01.051602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cda6ba-e887-440e-80b0-2569520c9125 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:01.062074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:01.070880 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2886/5722] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:14:01.093259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7c566-9fc6-4520-952b-8d5918b3810f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:01.106860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7c566-9fc6-4520-952b-8d5918b3810f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:01.112415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26c73f-c31c-4ad1-b001-1f42e8ce8d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:01.113207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26c73f-c31c-4ad1-b001-1f42e8ce8d78 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:01.113474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26c73f-c31c-4ad1-b001-1f42e8ce8d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:01.113727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26c73f-c31c-4ad1-b001-1f42e8ce8d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:01.114495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26c73f-c31c-4ad1-b001-1f42e8ce8d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:01.116796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26c73f-c31c-4ad1-b001-1f42e8ce8d78 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:01.156567 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2837/5723] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:14:01 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:14:01.161370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cda6ba-e887-440e-80b0-2569520c9125 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:01.175078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cda6ba-e887-440e-80b0-2569520c9125 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:01.263368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96684ea8-79da-4deb-bd3b-8cf0dc8d0cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:01.264667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96684ea8-79da-4deb-bd3b-8cf0dc8d0cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:01.265073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96684ea8-79da-4deb-bd3b-8cf0dc8d0cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:01.265414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96684ea8-79da-4deb-bd3b-8cf0dc8d0cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:01.266236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96684ea8-79da-4deb-bd3b-8cf0dc8d0cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:01.268242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96684ea8-79da-4deb-bd3b-8cf0dc8d0cb2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:01.295407 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2887/5724] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:14:01 2022] POST /identity/v3/users => generated 291 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:14:01.300539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26c73f-c31c-4ad1-b001-1f42e8ce8d78 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:01.312041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26c73f-c31c-4ad1-b001-1f42e8ce8d78 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:01.317703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa50ff-4f59-49ea-86f8-44348c3f1bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:01.318719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa50ff-4f59-49ea-86f8-44348c3f1bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:01.319003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa50ff-4f59-49ea-86f8-44348c3f1bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:01.319253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa50ff-4f59-49ea-86f8-44348c3f1bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:01.320458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa50ff-4f59-49ea-86f8-44348c3f1bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:01.322089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa50ff-4f59-49ea-86f8-44348c3f1bd9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:01.334449 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2838/5725] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:14:01.338786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96684ea8-79da-4deb-bd3b-8cf0dc8d0cb2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:01.349786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96684ea8-79da-4deb-bd3b-8cf0dc8d0cb2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:01.354953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22037f55-c323-4df4-be9c-3e8482a4dc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:01.355644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22037f55-c323-4df4-be9c-3e8482a4dc5a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:01.355998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22037f55-c323-4df4-be9c-3e8482a4dc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:01.356325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22037f55-c323-4df4-be9c-3e8482a4dc5a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:01.357074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22037f55-c323-4df4-be9c-3e8482a4dc5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:01.359416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22037f55-c323-4df4-be9c-3e8482a4dc5a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:01.370746 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2888/5726] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:14:01.375256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa50ff-4f59-49ea-86f8-44348c3f1bd9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:01.384051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:01.389128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa50ff-4f59-49ea-86f8-44348c3f1bd9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:01.397757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae46e9-3d48-4a92-98d8-899fee129f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:01.398758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae46e9-3d48-4a92-98d8-899fee129f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:01.399251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae46e9-3d48-4a92-98d8-899fee129f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:01.399761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae46e9-3d48-4a92-98d8-899fee129f33 admin admin] PATH_INFO: `/v3/projects/527eae23dba049829730aa245da525d9/users/1e210ffc8c14496f8ac641f31afd534f/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:01.417322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae46e9-3d48-4a92-98d8-899fee129f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=527eae23dba049829730aa245da525d9, user_id=1e210ffc8c14496f8ac641f31afd534f, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:01.420981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae46e9-3d48-4a92-98d8-899fee129f33 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:01.446502 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2839/5727] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:14:01 2022] PUT /identity/v3/projects/527eae23dba049829730aa245da525d9/users/1e210ffc8c14496f8ac641f31afd534f/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:14:01.457798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2430b-8b78-461e-b23f-aaf1d21a7643 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:01.458146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2430b-8b78-461e-b23f-aaf1d21a7643 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:01.458517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2430b-8b78-461e-b23f-aaf1d21a7643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:01.487325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-43b2430b-8b78-461e-b23f-aaf1d21a7643 None None] MFA Rules not processed for user `1e210ffc8c14496f8ac641f31afd534f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:01.488630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-43b2430b-8b78-461e-b23f-aaf1d21a7643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:01.555010 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2889/5728] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:01.565536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae46e9-3d48-4a92-98d8-899fee129f33 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:01.567789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-60ae46e9-3d48-4a92-98d8-899fee129f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:01.631243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae46e9-3d48-4a92-98d8-899fee129f33 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:01.635004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940e20b-4efc-45a0-8c1c-9d320a488782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:01.635610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940e20b-4efc-45a0-8c1c-9d320a488782 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:01.635871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940e20b-4efc-45a0-8c1c-9d320a488782 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:01.636467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940e20b-4efc-45a0-8c1c-9d320a488782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:01.645449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a940e20b-4efc-45a0-8c1c-9d320a488782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:01.646839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a940e20b-4efc-45a0-8c1c-9d320a488782 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:01.658611 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2840/5729] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:14:02.674468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28361ddc-ba6a-4135-9f5d-079def5d17b6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:02.674989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28361ddc-ba6a-4135-9f5d-079def5d17b6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:02.674989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28361ddc-ba6a-4135-9f5d-079def5d17b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:02.694601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-28361ddc-ba6a-4135-9f5d-079def5d17b6 None None] MFA Rules not processed for user `51ac29b4f090431c836479723ef1f71d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:02.696187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-28361ddc-ba6a-4135-9f5d-079def5d17b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:02.757494 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2890/5730] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:09.940489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940e20b-4efc-45a0-8c1c-9d320a488782 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:10.037112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940e20b-4efc-45a0-8c1c-9d320a488782 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:10.041828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:10.042518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:10.042780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:10.043045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] PATH_INFO: `/v3/users/1e210ffc8c14496f8ac641f31afd534f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:10.044621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e210ffc8c14496f8ac641f31afd534f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:10.046975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:10.099911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e210ffc8c14496f8ac641f31afd534f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:10.108721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e210ffc8c14496f8ac641f31afd534f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:10.129359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e210ffc8c14496f8ac641f31afd534f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:10.130287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e210ffc8c14496f8ac641f31afd534f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:10.131038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e210ffc8c14496f8ac641f31afd534f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:10.145136 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2841/5731] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:14:09 2022] DELETE /identity/v3/users/1e210ffc8c14496f8ac641f31afd534f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:14:10.150317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b19940-fcad-4d3e-8b03-bd93031876f4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:10.151241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b19940-fcad-4d3e-8b03-bd93031876f4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:10.151585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b19940-fcad-4d3e-8b03-bd93031876f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:10.172102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-23b19940-fcad-4d3e-8b03-bd93031876f4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:10.173588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-23b19940-fcad-4d3e-8b03-bd93031876f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:10.182479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:10.248831 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2891/5732] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:10.258661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:10.260107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:10.265615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:10.327780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a6309-74f1-4988-b730-51e7e885ee4d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:10.334003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e004027-0f77-49fd-b5f7-d2efa0a1bb33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:10.335058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e004027-0f77-49fd-b5f7-d2efa0a1bb33 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:10.335484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e004027-0f77-49fd-b5f7-d2efa0a1bb33 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:10.336185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e004027-0f77-49fd-b5f7-d2efa0a1bb33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:10.349436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e004027-0f77-49fd-b5f7-d2efa0a1bb33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:10.351199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e004027-0f77-49fd-b5f7-d2efa0a1bb33 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:10.361598 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2842/5733] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:14:10.554369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b19940-fcad-4d3e-8b03-bd93031876f4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:10.556170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-23b19940-fcad-4d3e-8b03-bd93031876f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:10.606132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e004027-0f77-49fd-b5f7-d2efa0a1bb33 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:10.608525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4e004027-0f77-49fd-b5f7-d2efa0a1bb33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:10.638463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e004027-0f77-49fd-b5f7-d2efa0a1bb33 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:10.651370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74838014-623f-41f3-8195-94b221825d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:10.653217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74838014-623f-41f3-8195-94b221825d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:10.654518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74838014-623f-41f3-8195-94b221825d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:10.656423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74838014-623f-41f3-8195-94b221825d3e admin admin] PATH_INFO: `/v3/users/34c993b8f205458bbc4361b921b11980` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:10.658309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74838014-623f-41f3-8195-94b221825d3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=34c993b8f205458bbc4361b921b11980)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:10.664811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74838014-623f-41f3-8195-94b221825d3e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:10.678316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b19940-fcad-4d3e-8b03-bd93031876f4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:10.685815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf3a53-49e5-48ef-a853-92b52e719aec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:10.686667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf3a53-49e5-48ef-a853-92b52e719aec service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:10.687244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf3a53-49e5-48ef-a853-92b52e719aec service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:10.687753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf3a53-49e5-48ef-a853-92b52e719aec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:10.690968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dccf3a53-49e5-48ef-a853-92b52e719aec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:10.765472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-74838014-623f-41f3-8195-94b221825d3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34c993b8f205458bbc4361b921b11980'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:10.777818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-74838014-623f-41f3-8195-94b221825d3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34c993b8f205458bbc4361b921b11980'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:10.790349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf3a53-49e5-48ef-a853-92b52e719aec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:10.792080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf3a53-49e5-48ef-a853-92b52e719aec service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:10.795595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-74838014-623f-41f3-8195-94b221825d3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34c993b8f205458bbc4361b921b11980'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:10.796436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-74838014-623f-41f3-8195-94b221825d3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34c993b8f205458bbc4361b921b11980'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:10.796986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-74838014-623f-41f3-8195-94b221825d3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34c993b8f205458bbc4361b921b11980'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:10.804355 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2892/5734] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:14:10.805371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf3a53-49e5-48ef-a853-92b52e719aec service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:10.806678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dccf3a53-49e5-48ef-a853-92b52e719aec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:10.823264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf3a53-49e5-48ef-a853-92b52e719aec service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:10.823773 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2843/5735] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:14:10 2022] DELETE /identity/v3/users/34c993b8f205458bbc4361b921b11980 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:14:10.829337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaa0a9-46ce-4d96-bd02-48e6821fa190 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:10.829575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaa0a9-46ce-4d96-bd02-48e6821fa190 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:10.829860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaa0a9-46ce-4d96-bd02-48e6821fa190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:10.852566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-32eaa0a9-46ce-4d96-bd02-48e6821fa190 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:10.854770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-32eaa0a9-46ce-4d96-bd02-48e6821fa190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:10.892991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:10.910341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:10.911114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:10.911420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:10.911684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70 admin admin] PATH_INFO: `/v3/projects/527eae23dba049829730aa245da525d9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:10.913103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=527eae23dba049829730aa245da525d9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:10.915633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:10.947130 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2844/5736] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:10.951902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaa0a9-46ce-4d96-bd02-48e6821fa190 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:10.955583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-32eaa0a9-46ce-4d96-bd02-48e6821fa190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:10.977622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:10.979490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '527eae23dba049829730aa245da525d9', 'request_id': 'req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:10.990794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '527eae23dba049829730aa245da525d9', 'request_id': 'req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:11.001088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '527eae23dba049829730aa245da525d9', 'request_id': 'req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:11.020659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70 admin admin] The token cache is being invalidate because project 527eae23dba049829730aa245da525d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:14:11.021087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:11.023424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-31bf4e2b-b6ee-4e9f-8866-f9afa3883f70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:11.025717 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2893/5737] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:14:10 2022] DELETE /identity/v3/projects/527eae23dba049829730aa245da525d9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:14:11.031478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5589e-b6f1-4a7d-817a-df52853d25bb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.031833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5589e-b6f1-4a7d-817a-df52853d25bb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.032171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5589e-b6f1-4a7d-817a-df52853d25bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.035041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaa0a9-46ce-4d96-bd02-48e6821fa190 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:11.060108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-daa5589e-b6f1-4a7d-817a-df52853d25bb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:11.061574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-daa5589e-b6f1-4a7d-817a-df52853d25bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:11.114589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17225536-58de-41e1-a0dc-8d61252e8522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:11.115294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17225536-58de-41e1-a0dc-8d61252e8522 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.115634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17225536-58de-41e1-a0dc-8d61252e8522 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.115967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17225536-58de-41e1-a0dc-8d61252e8522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.118949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-17225536-58de-41e1-a0dc-8d61252e8522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:11.145411 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2894/5738] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:11.147052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68844bd-a0ca-4e11-82c2-a3a7835c922e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.147386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68844bd-a0ca-4e11-82c2-a3a7835c922e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.147653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68844bd-a0ca-4e11-82c2-a3a7835c922e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.169108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b68844bd-a0ca-4e11-82c2-a3a7835c922e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:11.170382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b68844bd-a0ca-4e11-82c2-a3a7835c922e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:11.178503 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2895/5739] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:11.183207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68844bd-a0ca-4e11-82c2-a3a7835c922e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:11.194221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68844bd-a0ca-4e11-82c2-a3a7835c922e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:11.199317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68431a51-2490-477e-b1a4-bcfd9055bace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:11.200024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68431a51-2490-477e-b1a4-bcfd9055bace admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.202704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68431a51-2490-477e-b1a4-bcfd9055bace admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.202704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68431a51-2490-477e-b1a4-bcfd9055bace admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.202704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68431a51-2490-477e-b1a4-bcfd9055bace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:14:11.202704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68431a51-2490-477e-b1a4-bcfd9055bace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:11.203717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68431a51-2490-477e-b1a4-bcfd9055bace admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:11.207470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17225536-58de-41e1-a0dc-8d61252e8522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:11.209321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17225536-58de-41e1-a0dc-8d61252e8522 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:11.215789 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2896/5740] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:14:11.220266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68431a51-2490-477e-b1a4-bcfd9055bace admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:11.228240 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2845/5741] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:14:11.239940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68431a51-2490-477e-b1a4-bcfd9055bace admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:11.245452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4db355-c9b9-4096-b66a-98e539e0ea0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:11.246317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4db355-c9b9-4096-b66a-98e539e0ea0e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.247027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4db355-c9b9-4096-b66a-98e539e0ea0e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.247500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4db355-c9b9-4096-b66a-98e539e0ea0e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.248467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4db355-c9b9-4096-b66a-98e539e0ea0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:11.250879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4db355-c9b9-4096-b66a-98e539e0ea0e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:11.288418 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2897/5742] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:14:11 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:14:11.289616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17225536-58de-41e1-a0dc-8d61252e8522 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:11.299649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17225536-58de-41e1-a0dc-8d61252e8522 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:11.364056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef01093-349d-427c-837a-ae5ed3f838e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:11.364839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01093-349d-427c-837a-ae5ed3f838e4 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.365116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01093-349d-427c-837a-ae5ed3f838e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.365374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01093-349d-427c-837a-ae5ed3f838e4 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.365990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef01093-349d-427c-837a-ae5ed3f838e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:11.368452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef01093-349d-427c-837a-ae5ed3f838e4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:11.394585 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2846/5743] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:14:11 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:14:11.398631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4db355-c9b9-4096-b66a-98e539e0ea0e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:11.410481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4db355-c9b9-4096-b66a-98e539e0ea0e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:11.416014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864673f-bdea-4557-a624-b2f852a57c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:11.416759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864673f-bdea-4557-a624-b2f852a57c3d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.417180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864673f-bdea-4557-a624-b2f852a57c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.417518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864673f-bdea-4557-a624-b2f852a57c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.418233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864673f-bdea-4557-a624-b2f852a57c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:11.420209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864673f-bdea-4557-a624-b2f852a57c3d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:11.426883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef01093-349d-427c-837a-ae5ed3f838e4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:11.428952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cef01093-349d-427c-837a-ae5ed3f838e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:11.452434 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2898/5744] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:14:11.453494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864673f-bdea-4557-a624-b2f852a57c3d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:11.464914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef01093-349d-427c-837a-ae5ed3f838e4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:11.465922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864673f-bdea-4557-a624-b2f852a57c3d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:11.472266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf640b-99dd-4bfc-ba3d-e58db8875d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:11.472896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf640b-99dd-4bfc-ba3d-e58db8875d7e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.473263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf640b-99dd-4bfc-ba3d-e58db8875d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.473594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf640b-99dd-4bfc-ba3d-e58db8875d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.473846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae893f-748d-4a04-85bb-79cb42d59b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:11.474604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf640b-99dd-4bfc-ba3d-e58db8875d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:11.475199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae893f-748d-4a04-85bb-79cb42d59b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.475497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae893f-748d-4a04-85bb-79cb42d59b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.475858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae893f-748d-4a04-85bb-79cb42d59b89 admin admin] PATH_INFO: `/v3/projects/2cc2d68895d84e55b1c5f1ae3385e27a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.476689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf640b-99dd-4bfc-ba3d-e58db8875d7e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:11.477152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae893f-748d-4a04-85bb-79cb42d59b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc2d68895d84e55b1c5f1ae3385e27a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:11.479853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae893f-748d-4a04-85bb-79cb42d59b89 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:11.490636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:11.494388 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2899/5745] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:14:11.495340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf640b-99dd-4bfc-ba3d-e58db8875d7e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:11.506693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf640b-99dd-4bfc-ba3d-e58db8875d7e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:11.511981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe3280-9497-48d5-97b9-3d9d9cc21402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:11.512604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe3280-9497-48d5-97b9-3d9d9cc21402 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.512927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe3280-9497-48d5-97b9-3d9d9cc21402 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.513316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe3280-9497-48d5-97b9-3d9d9cc21402 admin admin] PATH_INFO: `/v3/projects/318093183f51425a8903d639227711c5/users/c304cd7ed8ad4c64909d18e7625f7a78/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.526381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:11.533006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe3280-9497-48d5-97b9-3d9d9cc21402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=318093183f51425a8903d639227711c5, user_id=c304cd7ed8ad4c64909d18e7625f7a78, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:11.536136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe3280-9497-48d5-97b9-3d9d9cc21402 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:11.539200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-efae893f-748d-4a04-85bb-79cb42d59b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc2d68895d84e55b1c5f1ae3385e27a', 'request_id': 'req-efae893f-748d-4a04-85bb-79cb42d59b89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:11.549077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-efae893f-748d-4a04-85bb-79cb42d59b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc2d68895d84e55b1c5f1ae3385e27a', 'request_id': 'req-efae893f-748d-4a04-85bb-79cb42d59b89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:11.556977 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2900/5746] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:14:11 2022] PUT /identity/v3/projects/318093183f51425a8903d639227711c5/users/c304cd7ed8ad4c64909d18e7625f7a78/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:14:11.558370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-efae893f-748d-4a04-85bb-79cb42d59b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc2d68895d84e55b1c5f1ae3385e27a', 'request_id': 'req-efae893f-748d-4a04-85bb-79cb42d59b89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:11.566369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe3280-9497-48d5-97b9-3d9d9cc21402 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:11.569429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-efae893f-748d-4a04-85bb-79cb42d59b89 admin admin] The token cache is being invalidate because project 2cc2d68895d84e55b1c5f1ae3385e27a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:14:11.570185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-efae893f-748d-4a04-85bb-79cb42d59b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:11.570378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dfbe3280-9497-48d5-97b9-3d9d9cc21402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:11.571040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-efae893f-748d-4a04-85bb-79cb42d59b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:11.575811 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2847/5747] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:14:11 2022] DELETE /identity/v3/projects/2cc2d68895d84e55b1c5f1ae3385e27a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:14:11.575811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f214da7-5f80-4de5-9bf1-d9c4794a42cc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.575811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f214da7-5f80-4de5-9bf1-d9c4794a42cc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.575811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f214da7-5f80-4de5-9bf1-d9c4794a42cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.608639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0f214da7-5f80-4de5-9bf1-d9c4794a42cc None None] MFA Rules not processed for user `c304cd7ed8ad4c64909d18e7625f7a78`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:11.609871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0f214da7-5f80-4de5-9bf1-d9c4794a42cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:11.644410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe3280-9497-48d5-97b9-3d9d9cc21402 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:11.648169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27f653-e44a-4e2d-bc1c-68ed4ada9181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:11.648872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27f653-e44a-4e2d-bc1c-68ed4ada9181 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.649214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27f653-e44a-4e2d-bc1c-68ed4ada9181 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.649547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27f653-e44a-4e2d-bc1c-68ed4ada9181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.651353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1e27f653-e44a-4e2d-bc1c-68ed4ada9181 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:11.663394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27f653-e44a-4e2d-bc1c-68ed4ada9181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:11.664836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27f653-e44a-4e2d-bc1c-68ed4ada9181 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:11.674959 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2901/5748] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:14:11.676299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6054eed-8556-489f-88d3-51b37d1ef54a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.676703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6054eed-8556-489f-88d3-51b37d1ef54a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.676949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6054eed-8556-489f-88d3-51b37d1ef54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.680553 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2848/5749] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:11.696158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f214da7-5f80-4de5-9bf1-d9c4794a42cc None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:11.698407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0f214da7-5f80-4de5-9bf1-d9c4794a42cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:11.705937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b6054eed-8556-489f-88d3-51b37d1ef54a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:11.707874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b6054eed-8556-489f-88d3-51b37d1ef54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:11.729322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f214da7-5f80-4de5-9bf1-d9c4794a42cc None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:11.733531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9af31d-6eee-42e8-9fd3-4ea1f695cddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:11.734234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9af31d-6eee-42e8-9fd3-4ea1f695cddd service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.734510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9af31d-6eee-42e8-9fd3-4ea1f695cddd service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.734775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9af31d-6eee-42e8-9fd3-4ea1f695cddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.744313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9af31d-6eee-42e8-9fd3-4ea1f695cddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:11.745796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9af31d-6eee-42e8-9fd3-4ea1f695cddd service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:11.756524 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2849/5750] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:14:11.791090 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2902/5751] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:11.798019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9af31d-6eee-42e8-9fd3-4ea1f695cddd service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:11.799548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3bf1a-dbdf-450f-87d8-501170422f5a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.799720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cb9af31d-6eee-42e8-9fd3-4ea1f695cddd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:11.800161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3bf1a-dbdf-450f-87d8-501170422f5a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.800630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3bf1a-dbdf-450f-87d8-501170422f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.825557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c7a3bf1a-dbdf-450f-87d8-501170422f5a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:11.826930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c7a3bf1a-dbdf-450f-87d8-501170422f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:11.836327 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2903/5752] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:14:11.840519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3bf1a-dbdf-450f-87d8-501170422f5a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:11.850527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3bf1a-dbdf-450f-87d8-501170422f5a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:11.855999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9091951-8706-461e-8a42-b8b133cfecab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:11.856638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9091951-8706-461e-8a42-b8b133cfecab admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.856922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9091951-8706-461e-8a42-b8b133cfecab admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.857159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9091951-8706-461e-8a42-b8b133cfecab admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.859699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9091951-8706-461e-8a42-b8b133cfecab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:14:11.860139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9091951-8706-461e-8a42-b8b133cfecab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:11.861803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9091951-8706-461e-8a42-b8b133cfecab admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:11.876646 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2904/5753] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:14:11.881234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9091951-8706-461e-8a42-b8b133cfecab admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:11.893945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9091951-8706-461e-8a42-b8b133cfecab admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:11.903721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc39a0-1a38-4708-83b5-0278c2621d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:11.904583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc39a0-1a38-4708-83b5-0278c2621d66 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.904986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc39a0-1a38-4708-83b5-0278c2621d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.905519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc39a0-1a38-4708-83b5-0278c2621d66 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.906333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc39a0-1a38-4708-83b5-0278c2621d66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:11.909795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc39a0-1a38-4708-83b5-0278c2621d66 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:11.909942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9af31d-6eee-42e8-9fd3-4ea1f695cddd service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:11.918534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2263dd9-285c-4981-a3fb-47b4a93212b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:11.919527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2263dd9-285c-4981-a3fb-47b4a93212b2 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:11.919835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2263dd9-285c-4981-a3fb-47b4a93212b2 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:11.920103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2263dd9-285c-4981-a3fb-47b4a93212b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:11.922904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b2263dd9-285c-4981-a3fb-47b4a93212b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:11.944325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2263dd9-285c-4981-a3fb-47b4a93212b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:11.946856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2263dd9-285c-4981-a3fb-47b4a93212b2 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:11.948014 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2905/5754] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:14:11 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:14:11.961218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc39a0-1a38-4708-83b5-0278c2621d66 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:11.966012 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2850/5755] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:14:12.048919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:12.078871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc39a0-1a38-4708-83b5-0278c2621d66 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:12.101224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d44e8b-3b90-4807-9708-04bfe323666a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:12.102132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d44e8b-3b90-4807-9708-04bfe323666a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:12.102516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d44e8b-3b90-4807-9708-04bfe323666a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:12.102874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d44e8b-3b90-4807-9708-04bfe323666a admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:12.103830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d44e8b-3b90-4807-9708-04bfe323666a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:12.105962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d44e8b-3b90-4807-9708-04bfe323666a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:12.143915 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2906/5756] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:14:11 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:14:12.149123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2263dd9-285c-4981-a3fb-47b4a93212b2 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:12.157786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:12.168588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2263dd9-285c-4981-a3fb-47b4a93212b2 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:12.174941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a073e5-d553-4ac2-902e-651277a2dae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:12.175593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a073e5-d553-4ac2-902e-651277a2dae3 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:12.176089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a073e5-d553-4ac2-902e-651277a2dae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:12.176530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a073e5-d553-4ac2-902e-651277a2dae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:12.177419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a073e5-d553-4ac2-902e-651277a2dae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:12.179138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a073e5-d553-4ac2-902e-651277a2dae3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:12.199029 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2851/5757] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:14:12.203686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d44e8b-3b90-4807-9708-04bfe323666a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:12.214632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d44e8b-3b90-4807-9708-04bfe323666a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:12.219851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d84816-6a39-468c-bb0b-fc3bb6283732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:12.220498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d84816-6a39-468c-bb0b-fc3bb6283732 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:12.220788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d84816-6a39-468c-bb0b-fc3bb6283732 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:12.221038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d84816-6a39-468c-bb0b-fc3bb6283732 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:12.221779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d84816-6a39-468c-bb0b-fc3bb6283732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:12.229245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d84816-6a39-468c-bb0b-fc3bb6283732 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:12.248433 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2907/5758] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:14:12.251479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a073e5-d553-4ac2-902e-651277a2dae3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:12.253136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd409514-6473-4862-a22c-3170270c9bf6 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:12.255087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd409514-6473-4862-a22c-3170270c9bf6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:12.255087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd409514-6473-4862-a22c-3170270c9bf6 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:12.255464 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2908/5759] 10.222.0.23 () {54 vars in 921 bytes} [Sat Aug 20 21:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 20 21:14:12.261297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f52250-7964-4022-b400-5bb8433fd620 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:12.261573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f52250-7964-4022-b400-5bb8433fd620 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:12.261842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f52250-7964-4022-b400-5bb8433fd620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:12.265734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a073e5-d553-4ac2-902e-651277a2dae3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:12.270748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f45f03-f6ea-4494-98ee-919e3d029610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:12.272138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f45f03-f6ea-4494-98ee-919e3d029610 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:12.272648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f45f03-f6ea-4494-98ee-919e3d029610 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:12.273095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f45f03-f6ea-4494-98ee-919e3d029610 admin admin] PATH_INFO: `/v3/projects/b5f9932443eb4bcb82c0509cb7832074/users/480ae5fa63df4f1899a70ef9f4fcdddd/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:12.292148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f45f03-f6ea-4494-98ee-919e3d029610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5f9932443eb4bcb82c0509cb7832074, user_id=480ae5fa63df4f1899a70ef9f4fcdddd, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:12.295451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f45f03-f6ea-4494-98ee-919e3d029610 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:12.317306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-34f52250-7964-4022-b400-5bb8433fd620 None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:12.318106 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2852/5760] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:14:12 2022] PUT /identity/v3/projects/b5f9932443eb4bcb82c0509cb7832074/users/480ae5fa63df4f1899a70ef9f4fcdddd/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:14:12.320797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-34f52250-7964-4022-b400-5bb8433fd620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:12.329373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a593cc-e0a7-4fc7-a85c-eab84b393aaf None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:12.329664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a593cc-e0a7-4fc7-a85c-eab84b393aaf None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:12.329920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a593cc-e0a7-4fc7-a85c-eab84b393aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:12.361556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-03a593cc-e0a7-4fc7-a85c-eab84b393aaf None None] MFA Rules not processed for user `480ae5fa63df4f1899a70ef9f4fcdddd`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:12.363075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-03a593cc-e0a7-4fc7-a85c-eab84b393aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:12.414445 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2909/5761] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 21:14:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:14:12.426970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f52250-7964-4022-b400-5bb8433fd620 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:12.428552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-34f52250-7964-4022-b400-5bb8433fd620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:12.439154 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2853/5762] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:12.453413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a593cc-e0a7-4fc7-a85c-eab84b393aaf None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:12.458561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-03a593cc-e0a7-4fc7-a85c-eab84b393aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:12.511609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f52250-7964-4022-b400-5bb8433fd620 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:12.516320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed06b7-b8ee-4c7a-ae8a-b4f925d84486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:12.517013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed06b7-b8ee-4c7a-ae8a-b4f925d84486 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:12.517281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed06b7-b8ee-4c7a-ae8a-b4f925d84486 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:12.517542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed06b7-b8ee-4c7a-ae8a-b4f925d84486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:12.529833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed06b7-b8ee-4c7a-ae8a-b4f925d84486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:12.531630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed06b7-b8ee-4c7a-ae8a-b4f925d84486 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:12.546420 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2910/5763] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:14:12.557164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a593cc-e0a7-4fc7-a85c-eab84b393aaf None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:12.561811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a650aa-2cb0-4f8c-8ec7-6b281659c773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:12.562619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a650aa-2cb0-4f8c-8ec7-6b281659c773 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:12.563046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a650aa-2cb0-4f8c-8ec7-6b281659c773 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:12.563498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a650aa-2cb0-4f8c-8ec7-6b281659c773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:12.574763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a650aa-2cb0-4f8c-8ec7-6b281659c773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:12.577280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a650aa-2cb0-4f8c-8ec7-6b281659c773 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:12.591321 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2854/5764] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:14:13.487462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed06b7-b8ee-4c7a-ae8a-b4f925d84486 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:13.540915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:13.562429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed06b7-b8ee-4c7a-ae8a-b4f925d84486 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:13.568331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:13.569140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:13.569450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:13.570253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] PATH_INFO: `/v3/users/480ae5fa63df4f1899a70ef9f4fcdddd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:13.571662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] RBAC: Authorizing `identity:delete_user(user_id=480ae5fa63df4f1899a70ef9f4fcdddd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:13.573719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:13.629004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '480ae5fa63df4f1899a70ef9f4fcdddd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:13.641990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '480ae5fa63df4f1899a70ef9f4fcdddd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:13.653031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '480ae5fa63df4f1899a70ef9f4fcdddd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:13.679535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '480ae5fa63df4f1899a70ef9f4fcdddd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:13.680944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '480ae5fa63df4f1899a70ef9f4fcdddd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:13.699520 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2911/5765] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:14:13 2022] DELETE /identity/v3/users/480ae5fa63df4f1899a70ef9f4fcdddd => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:14:13.704353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c55f2d-649b-49ab-b937-c7bf9765d7d3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:13.704865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c55f2d-649b-49ab-b937-c7bf9765d7d3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:13.704958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c55f2d-649b-49ab-b937-c7bf9765d7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:13.729400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-34c55f2d-649b-49ab-b937-c7bf9765d7d3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:13.731527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-34c55f2d-649b-49ab-b937-c7bf9765d7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:13.799973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:13.812361 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2855/5766] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:13.823197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:13.824832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:13.918022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71c46c-4b11-4955-85f4-3dc6ac39a74f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:13.922329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54c76e-7d27-4fd5-b5f8-f2b69c56e8a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:13.923066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54c76e-7d27-4fd5-b5f8-f2b69c56e8a1 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:13.923337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54c76e-7d27-4fd5-b5f8-f2b69c56e8a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:13.923595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54c76e-7d27-4fd5-b5f8-f2b69c56e8a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:13.933559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54c76e-7d27-4fd5-b5f8-f2b69c56e8a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:13.936023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54c76e-7d27-4fd5-b5f8-f2b69c56e8a1 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:13.946930 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2912/5767] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:14:14.284312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c55f2d-649b-49ab-b937-c7bf9765d7d3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:14.286020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-34c55f2d-649b-49ab-b937-c7bf9765d7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:14.307157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c55f2d-649b-49ab-b937-c7bf9765d7d3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:14.321005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87062a82-8954-4fd7-bef7-a8ca75f55ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:14.322200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87062a82-8954-4fd7-bef7-a8ca75f55ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:14.322540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87062a82-8954-4fd7-bef7-a8ca75f55ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:14.322878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87062a82-8954-4fd7-bef7-a8ca75f55ae2 admin admin] PATH_INFO: `/v3/projects/b5f9932443eb4bcb82c0509cb7832074` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:14.324787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87062a82-8954-4fd7-bef7-a8ca75f55ae2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5f9932443eb4bcb82c0509cb7832074)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:14.330378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87062a82-8954-4fd7-bef7-a8ca75f55ae2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:14.419179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-87062a82-8954-4fd7-bef7-a8ca75f55ae2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5f9932443eb4bcb82c0509cb7832074', 'request_id': 'req-87062a82-8954-4fd7-bef7-a8ca75f55ae2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:14.427910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-87062a82-8954-4fd7-bef7-a8ca75f55ae2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5f9932443eb4bcb82c0509cb7832074', 'request_id': 'req-87062a82-8954-4fd7-bef7-a8ca75f55ae2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:14.443048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-87062a82-8954-4fd7-bef7-a8ca75f55ae2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5f9932443eb4bcb82c0509cb7832074', 'request_id': 'req-87062a82-8954-4fd7-bef7-a8ca75f55ae2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:14.459271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-87062a82-8954-4fd7-bef7-a8ca75f55ae2 admin admin] The token cache is being invalidate because project b5f9932443eb4bcb82c0509cb7832074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:14:14.459646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-87062a82-8954-4fd7-bef7-a8ca75f55ae2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:14.460496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-87062a82-8954-4fd7-bef7-a8ca75f55ae2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:14:14.462186 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2856/5768] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:14:14 2022] DELETE /identity/v3/projects/b5f9932443eb4bcb82c0509cb7832074 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:14:14.467567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1389e20-a1e0-4c13-82e1-41e135682ac0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:14.467993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1389e20-a1e0-4c13-82e1-41e135682ac0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:14.468366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1389e20-a1e0-4c13-82e1-41e135682ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:14.488960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f1389e20-a1e0-4c13-82e1-41e135682ac0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:14.490964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f1389e20-a1e0-4c13-82e1-41e135682ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:14.587895 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2913/5769] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:14.596249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327a548-3948-4b6b-a19b-cc197d9a9dc0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:14.596616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327a548-3948-4b6b-a19b-cc197d9a9dc0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:14.596989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327a548-3948-4b6b-a19b-cc197d9a9dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:14.620153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c327a548-3948-4b6b-a19b-cc197d9a9dc0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:14.621549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c327a548-3948-4b6b-a19b-cc197d9a9dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:14.634691 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2857/5770] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:14.639497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1389e20-a1e0-4c13-82e1-41e135682ac0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:14.646606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:14.650912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1389e20-a1e0-4c13-82e1-41e135682ac0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:14.658263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d36d9-a485-4f03-94d4-4af02bfb142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:14.658988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d36d9-a485-4f03-94d4-4af02bfb142a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:14.659430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d36d9-a485-4f03-94d4-4af02bfb142a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:14.660006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d36d9-a485-4f03-94d4-4af02bfb142a admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:14.661312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d36d9-a485-4f03-94d4-4af02bfb142a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:14:14.661437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d36d9-a485-4f03-94d4-4af02bfb142a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:14.663386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d36d9-a485-4f03-94d4-4af02bfb142a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:14.679937 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2914/5771] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:14:14.682225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327a548-3948-4b6b-a19b-cc197d9a9dc0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:14.693225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327a548-3948-4b6b-a19b-cc197d9a9dc0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:14.699010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641556c-0f15-414f-b772-2e55dfa3a12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:14.699741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641556c-0f15-414f-b772-2e55dfa3a12f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:14.700133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641556c-0f15-414f-b772-2e55dfa3a12f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:14.700533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641556c-0f15-414f-b772-2e55dfa3a12f admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:14.701328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641556c-0f15-414f-b772-2e55dfa3a12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:14.703328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641556c-0f15-414f-b772-2e55dfa3a12f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:14.736041 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2858/5772] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:14:14 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:14:14.740196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d36d9-a485-4f03-94d4-4af02bfb142a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:14.753352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d36d9-a485-4f03-94d4-4af02bfb142a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:14.837063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6601a-95f5-45a0-b11d-78fd89d70273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:14.837760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6601a-95f5-45a0-b11d-78fd89d70273 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:14.838189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6601a-95f5-45a0-b11d-78fd89d70273 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:14.838513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6601a-95f5-45a0-b11d-78fd89d70273 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:14.839246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6601a-95f5-45a0-b11d-78fd89d70273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:14.841293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6601a-95f5-45a0-b11d-78fd89d70273 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:14.870848 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2915/5773] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:14:14 2022] POST /identity/v3/users => generated 285 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:14:14.872101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641556c-0f15-414f-b772-2e55dfa3a12f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:14.885228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641556c-0f15-414f-b772-2e55dfa3a12f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:14.893035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9c6db-3c78-42d6-8658-21ce50142589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:14.893553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9c6db-3c78-42d6-8658-21ce50142589 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:14.893951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9c6db-3c78-42d6-8658-21ce50142589 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:14.894216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9c6db-3c78-42d6-8658-21ce50142589 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:14.894998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9c6db-3c78-42d6-8658-21ce50142589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:14.896773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9c6db-3c78-42d6-8658-21ce50142589 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:14.913037 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2859/5774] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:14:14.914192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6601a-95f5-45a0-b11d-78fd89d70273 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:14.925184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6601a-95f5-45a0-b11d-78fd89d70273 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:14.929787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c85630-8a7d-471b-bb51-f6085aaa9740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:14.930383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c85630-8a7d-471b-bb51-f6085aaa9740 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:14.930882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c85630-8a7d-471b-bb51-f6085aaa9740 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:14.930882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c85630-8a7d-471b-bb51-f6085aaa9740 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:14.931746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c85630-8a7d-471b-bb51-f6085aaa9740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:14.933609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c85630-8a7d-471b-bb51-f6085aaa9740 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:14.944596 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2916/5775] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:14:14.950307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9c6db-3c78-42d6-8658-21ce50142589 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:14.961685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9c6db-3c78-42d6-8658-21ce50142589 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:14.966867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ccddf-c73f-454b-8689-c6340dde6040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:14.967597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ccddf-c73f-454b-8689-c6340dde6040 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:14.967962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ccddf-c73f-454b-8689-c6340dde6040 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:14.968293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ccddf-c73f-454b-8689-c6340dde6040 admin admin] PATH_INFO: `/v3/projects/9d61c631d64e4757876fd6cd1b502f31/users/168b34153ba14981b627cb9d4b317fcf/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:14.986437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ccddf-c73f-454b-8689-c6340dde6040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d61c631d64e4757876fd6cd1b502f31, user_id=168b34153ba14981b627cb9d4b317fcf, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:14.989653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ccddf-c73f-454b-8689-c6340dde6040 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:15.007459 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2860/5776] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:14:14 2022] PUT /identity/v3/projects/9d61c631d64e4757876fd6cd1b502f31/users/168b34153ba14981b627cb9d4b317fcf/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:14:15.025169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e2ccc-b74b-4c24-9d22-f399ee3a7918 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:15.025717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e2ccc-b74b-4c24-9d22-f399ee3a7918 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:15.027431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e2ccc-b74b-4c24-9d22-f399ee3a7918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:15.082202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e37e2ccc-b74b-4c24-9d22-f399ee3a7918 None None] MFA Rules not processed for user `168b34153ba14981b627cb9d4b317fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:15.084992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e37e2ccc-b74b-4c24-9d22-f399ee3a7918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:15.175243 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2917/5777] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:15.186665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ccddf-c73f-454b-8689-c6340dde6040 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:15.189191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8b0ccddf-c73f-454b-8689-c6340dde6040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:15.284183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ccddf-c73f-454b-8689-c6340dde6040 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:15.290119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d131d55-9711-4a09-b541-b3713235fd4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:15.290616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d131d55-9711-4a09-b541-b3713235fd4b service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:15.291086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d131d55-9711-4a09-b541-b3713235fd4b service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:15.291370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d131d55-9711-4a09-b541-b3713235fd4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:15.302995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d131d55-9711-4a09-b541-b3713235fd4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:15.305497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d131d55-9711-4a09-b541-b3713235fd4b service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:15.317935 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2861/5778] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:14:25.172136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:25.321185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:14:51.445945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9d4fa-edbb-495f-8d40-21cc21e9cb7f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:51.446595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9d4fa-edbb-495f-8d40-21cc21e9cb7f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:51.446800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9d4fa-edbb-495f-8d40-21cc21e9cb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:51.467806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c1b9d4fa-edbb-495f-8d40-21cc21e9cb7f None None] MFA Rules not processed for user `9671d01ad0d44cc99e0eecc966f7c3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:14:51.470058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c1b9d4fa-edbb-495f-8d40-21cc21e9cb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:51.537336 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2918/5779] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:14:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:14:51.680361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d131d55-9711-4a09-b541-b3713235fd4b service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:51.683494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3d131d55-9711-4a09-b541-b3713235fd4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:51.705572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d131d55-9711-4a09-b541-b3713235fd4b service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:51.710741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfb3a6-5a76-4241-af16-3289bc4d5495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:51.711357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfb3a6-5a76-4241-af16-3289bc4d5495 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:51.711630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfb3a6-5a76-4241-af16-3289bc4d5495 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:51.711897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfb3a6-5a76-4241-af16-3289bc4d5495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:51.727596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0ebfb3a6-5a76-4241-af16-3289bc4d5495 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:51.852487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebfb3a6-5a76-4241-af16-3289bc4d5495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:51.854902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebfb3a6-5a76-4241-af16-3289bc4d5495 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:51.874266 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2862/5780] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:14:56.019375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9d4fa-edbb-495f-8d40-21cc21e9cb7f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:14:56.039130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9d4fa-edbb-495f-8d40-21cc21e9cb7f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:14:56.051612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56275a-8ef2-4476-b933-fb75dafa1d62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:14:56.052528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56275a-8ef2-4476-b933-fb75dafa1d62 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:14:56.055446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56275a-8ef2-4476-b933-fb75dafa1d62 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:14:56.055727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56275a-8ef2-4476-b933-fb75dafa1d62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:14:56.060409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ef56275a-8ef2-4476-b933-fb75dafa1d62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:14:56.109166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56275a-8ef2-4476-b933-fb75dafa1d62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:14:56.112360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56275a-8ef2-4476-b933-fb75dafa1d62 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:14:56.144935 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2919/5781] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:14:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:15:01.865925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=34 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:06.134523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=31 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:11.294240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfb3a6-5a76-4241-af16-3289bc4d5495 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:11.295699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0ebfb3a6-5a76-4241-af16-3289bc4d5495 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:11.341065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56275a-8ef2-4476-b933-fb75dafa1d62 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:11.342695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ef56275a-8ef2-4476-b933-fb75dafa1d62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:11.377560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfb3a6-5a76-4241-af16-3289bc4d5495 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:11.382583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:11.384017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:11.384289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:11.384604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] PATH_INFO: `/v3/users/51ac29b4f090431c836479723ef1f71d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:11.385743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ac29b4f090431c836479723ef1f71d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:11.389057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:11.411737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56275a-8ef2-4476-b933-fb75dafa1d62 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:11.416528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb45d9-ecaa-4c36-b67e-66271306077b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:11.417138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb45d9-ecaa-4c36-b67e-66271306077b service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:11.417603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb45d9-ecaa-4c36-b67e-66271306077b service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:11.417867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb45d9-ecaa-4c36-b67e-66271306077b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:11.420069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5ecb45d9-ecaa-4c36-b67e-66271306077b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:11.463289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ac29b4f090431c836479723ef1f71d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:11.480860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ac29b4f090431c836479723ef1f71d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:11.507942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ac29b4f090431c836479723ef1f71d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:11.509217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ac29b4f090431c836479723ef1f71d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:11.511527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ac29b4f090431c836479723ef1f71d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:11.529833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb45d9-ecaa-4c36-b67e-66271306077b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:11.531154 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2863/5782] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:15:11 2022] DELETE /identity/v3/users/51ac29b4f090431c836479723ef1f71d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:15:11.531630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb45d9-ecaa-4c36-b67e-66271306077b service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:11.589551 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2920/5783] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:15:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:15:11.915913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:11.917403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:12.017202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e0e44-0be4-4510-8192-219bc2981edb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:12.023008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0741f4-dc73-419f-be47-b5a79611b11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:12.023795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0741f4-dc73-419f-be47-b5a79611b11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:12.024070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0741f4-dc73-419f-be47-b5a79611b11b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:12.024339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0741f4-dc73-419f-be47-b5a79611b11b admin admin] PATH_INFO: `/v3/projects/094f627bb04848519808048a66b2010f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:12.025515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0741f4-dc73-419f-be47-b5a79611b11b admin admin] RBAC: Authorizing `identity:delete_project(project_id=094f627bb04848519808048a66b2010f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:12.027454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0741f4-dc73-419f-be47-b5a79611b11b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:12.063262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:12.098276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5e0741f4-dc73-419f-be47-b5a79611b11b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '094f627bb04848519808048a66b2010f', 'request_id': 'req-5e0741f4-dc73-419f-be47-b5a79611b11b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:12.105147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5e0741f4-dc73-419f-be47-b5a79611b11b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '094f627bb04848519808048a66b2010f', 'request_id': 'req-5e0741f4-dc73-419f-be47-b5a79611b11b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:12.117113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5e0741f4-dc73-419f-be47-b5a79611b11b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '094f627bb04848519808048a66b2010f', 'request_id': 'req-5e0741f4-dc73-419f-be47-b5a79611b11b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:12.130210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5e0741f4-dc73-419f-be47-b5a79611b11b admin admin] The token cache is being invalidate because project 094f627bb04848519808048a66b2010f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:15:12.130517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5e0741f4-dc73-419f-be47-b5a79611b11b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:12.131148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5e0741f4-dc73-419f-be47-b5a79611b11b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:12.136340 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2864/5784] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:15:11 2022] DELETE /identity/v3/projects/094f627bb04848519808048a66b2010f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:15:12.137831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31474b3d-31ef-4f72-83da-137f44ad1418 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:12.138358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31474b3d-31ef-4f72-83da-137f44ad1418 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:12.138805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31474b3d-31ef-4f72-83da-137f44ad1418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:12.163381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-31474b3d-31ef-4f72-83da-137f44ad1418 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:12.166024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-31474b3d-31ef-4f72-83da-137f44ad1418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:12.344958 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2921/5785] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:12.353827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f954300-94fb-48d9-92a3-16f94c2759bf None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:12.354359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f954300-94fb-48d9-92a3-16f94c2759bf None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:12.354359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f954300-94fb-48d9-92a3-16f94c2759bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:12.388247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4f954300-94fb-48d9-92a3-16f94c2759bf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:12.389262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4f954300-94fb-48d9-92a3-16f94c2759bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:12.401112 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2865/5786] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:12.402114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31474b3d-31ef-4f72-83da-137f44ad1418 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:12.419458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31474b3d-31ef-4f72-83da-137f44ad1418 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:12.425802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98af5f-33bd-499c-ab5b-eca71b1f7e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:12.427656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98af5f-33bd-499c-ab5b-eca71b1f7e4e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:12.428307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98af5f-33bd-499c-ab5b-eca71b1f7e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:12.428855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98af5f-33bd-499c-ab5b-eca71b1f7e4e admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:12.430756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98af5f-33bd-499c-ab5b-eca71b1f7e4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:15:12.431313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98af5f-33bd-499c-ab5b-eca71b1f7e4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:12.433670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98af5f-33bd-499c-ab5b-eca71b1f7e4e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:12.449984 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2922/5787] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:15:12.454098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f954300-94fb-48d9-92a3-16f94c2759bf None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:12.468573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f954300-94fb-48d9-92a3-16f94c2759bf None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:12.474298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8902fbc-fc0b-493a-87a7-9dfbb9076de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:12.474928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8902fbc-fc0b-493a-87a7-9dfbb9076de9 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:12.475179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8902fbc-fc0b-493a-87a7-9dfbb9076de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:12.475448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8902fbc-fc0b-493a-87a7-9dfbb9076de9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:12.476074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8902fbc-fc0b-493a-87a7-9dfbb9076de9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:12.478630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8902fbc-fc0b-493a-87a7-9dfbb9076de9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:12.531795 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2866/5788] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:15:12 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:15:12.535938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98af5f-33bd-499c-ab5b-eca71b1f7e4e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:12.546804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98af5f-33bd-499c-ab5b-eca71b1f7e4e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:12.592195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:12.621866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08c23e-8117-459d-a79c-517347fb377b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:12.622554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08c23e-8117-459d-a79c-517347fb377b admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:12.622897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08c23e-8117-459d-a79c-517347fb377b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:12.623384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08c23e-8117-459d-a79c-517347fb377b admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:12.624178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08c23e-8117-459d-a79c-517347fb377b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:12.626233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08c23e-8117-459d-a79c-517347fb377b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:12.657953 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2923/5789] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:15:12 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:15:12.662596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8902fbc-fc0b-493a-87a7-9dfbb9076de9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:12.678031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8902fbc-fc0b-493a-87a7-9dfbb9076de9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:12.688396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f9cf2-891c-4a68-b787-b6d62587f3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:12.689038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f9cf2-891c-4a68-b787-b6d62587f3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:12.689437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f9cf2-891c-4a68-b787-b6d62587f3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:12.689972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f9cf2-891c-4a68-b787-b6d62587f3a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:12.691269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f9cf2-891c-4a68-b787-b6d62587f3a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:12.692974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f9cf2-891c-4a68-b787-b6d62587f3a5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:12.706592 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2867/5790] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:15:12.710829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08c23e-8117-459d-a79c-517347fb377b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:12.732708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08c23e-8117-459d-a79c-517347fb377b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:12.745983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d206f57-22cd-4371-ae30-06412f68bfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:12.747361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d206f57-22cd-4371-ae30-06412f68bfcc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:12.748294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d206f57-22cd-4371-ae30-06412f68bfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:12.748787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d206f57-22cd-4371-ae30-06412f68bfcc admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:12.752015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d206f57-22cd-4371-ae30-06412f68bfcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:12.753950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d206f57-22cd-4371-ae30-06412f68bfcc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:12.773248 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2924/5791] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:15:12.777203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f9cf2-891c-4a68-b787-b6d62587f3a5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:12.792838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f9cf2-891c-4a68-b787-b6d62587f3a5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:12.801882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5829579-c5bc-42fe-ac13-e48f0158f6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:12.803083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5829579-c5bc-42fe-ac13-e48f0158f6ce admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:12.803370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5829579-c5bc-42fe-ac13-e48f0158f6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:12.803654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5829579-c5bc-42fe-ac13-e48f0158f6ce admin admin] PATH_INFO: `/v3/projects/da107d25a79c4ae3a25c836bc2d23e19/users/fa3184ee6bec45da840eb9182b9a5f4d/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:12.833366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5829579-c5bc-42fe-ac13-e48f0158f6ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=da107d25a79c4ae3a25c836bc2d23e19, user_id=fa3184ee6bec45da840eb9182b9a5f4d, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:12.836401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5829579-c5bc-42fe-ac13-e48f0158f6ce admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:12.867303 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2868/5792] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:15:12 2022] PUT /identity/v3/projects/da107d25a79c4ae3a25c836bc2d23e19/users/fa3184ee6bec45da840eb9182b9a5f4d/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:15:12.872673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123efc0d-4fe0-4e92-9d31-c23bd2f44e59 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:12.872944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123efc0d-4fe0-4e92-9d31-c23bd2f44e59 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:12.873153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123efc0d-4fe0-4e92-9d31-c23bd2f44e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:12.917524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-123efc0d-4fe0-4e92-9d31-c23bd2f44e59 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:12.918890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-123efc0d-4fe0-4e92-9d31-c23bd2f44e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:13.013232 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2925/5793] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:13.026941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5829579-c5bc-42fe-ac13-e48f0158f6ce admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:13.029170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a5829579-c5bc-42fe-ac13-e48f0158f6ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:13.123939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5829579-c5bc-42fe-ac13-e48f0158f6ce admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:13.127667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93945b19-9ee0-40c4-87b8-8fb849474d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:13.128499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93945b19-9ee0-40c4-87b8-8fb849474d74 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:13.129106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93945b19-9ee0-40c4-87b8-8fb849474d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:13.129668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93945b19-9ee0-40c4-87b8-8fb849474d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:13.142579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93945b19-9ee0-40c4-87b8-8fb849474d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:13.144143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93945b19-9ee0-40c4-87b8-8fb849474d74 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:13.159574 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2869/5794] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:15:13.299918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123efc0d-4fe0-4e92-9d31-c23bd2f44e59 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:13.301890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-123efc0d-4fe0-4e92-9d31-c23bd2f44e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:13.321266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123efc0d-4fe0-4e92-9d31-c23bd2f44e59 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:13.328515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:13.329459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:13.329459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:13.329691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] PATH_INFO: `/v3/users/c304cd7ed8ad4c64909d18e7625f7a78` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:13.330809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c304cd7ed8ad4c64909d18e7625f7a78)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:13.332827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:13.418826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c304cd7ed8ad4c64909d18e7625f7a78'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:13.431436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c304cd7ed8ad4c64909d18e7625f7a78'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:13.442580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c304cd7ed8ad4c64909d18e7625f7a78'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:13.459616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c304cd7ed8ad4c64909d18e7625f7a78'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:13.460651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c304cd7ed8ad4c64909d18e7625f7a78'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:13.471520 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2926/5795] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:15:13 2022] DELETE /identity/v3/users/c304cd7ed8ad4c64909d18e7625f7a78 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:15:13.475820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b18bec-951e-4db8-a5ba-100e8819ae60 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:13.476336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b18bec-951e-4db8-a5ba-100e8819ae60 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:13.476527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b18bec-951e-4db8-a5ba-100e8819ae60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:13.499417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-52b18bec-951e-4db8-a5ba-100e8819ae60 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:13.501083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-52b18bec-951e-4db8-a5ba-100e8819ae60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:13.541439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:13.580572 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2870/5796] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:13.589867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:13.592157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:13.635852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:13.687234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2474c05-9940-45cb-ae75-4593519950e9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:13.692029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14dcf0-06c6-4113-b833-b70b960d730c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:13.692830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14dcf0-06c6-4113-b833-b70b960d730c service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:13.693216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14dcf0-06c6-4113-b833-b70b960d730c service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:13.693435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14dcf0-06c6-4113-b833-b70b960d730c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:13.707134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14dcf0-06c6-4113-b833-b70b960d730c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:13.708967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14dcf0-06c6-4113-b833-b70b960d730c service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:13.722052 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2927/5797] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:15:14.103375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b18bec-951e-4db8-a5ba-100e8819ae60 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:14.107587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-52b18bec-951e-4db8-a5ba-100e8819ae60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:14.139264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b18bec-951e-4db8-a5ba-100e8819ae60 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:14.146913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e5017-923f-423f-a5cb-b739ea1b7458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:14.148172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e5017-923f-423f-a5cb-b739ea1b7458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:14.148817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e5017-923f-423f-a5cb-b739ea1b7458 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:14.149711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e5017-923f-423f-a5cb-b739ea1b7458 admin admin] PATH_INFO: `/v3/projects/318093183f51425a8903d639227711c5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:14.151363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e5017-923f-423f-a5cb-b739ea1b7458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=318093183f51425a8903d639227711c5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:14.154845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e5017-923f-423f-a5cb-b739ea1b7458 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:14.245631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c29e5017-923f-423f-a5cb-b739ea1b7458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '318093183f51425a8903d639227711c5', 'request_id': 'req-c29e5017-923f-423f-a5cb-b739ea1b7458'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:14.258351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c29e5017-923f-423f-a5cb-b739ea1b7458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '318093183f51425a8903d639227711c5', 'request_id': 'req-c29e5017-923f-423f-a5cb-b739ea1b7458'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:14.277109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c29e5017-923f-423f-a5cb-b739ea1b7458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '318093183f51425a8903d639227711c5', 'request_id': 'req-c29e5017-923f-423f-a5cb-b739ea1b7458'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:14.290403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c29e5017-923f-423f-a5cb-b739ea1b7458 admin admin] The token cache is being invalidate because project 318093183f51425a8903d639227711c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:15:14.290692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c29e5017-923f-423f-a5cb-b739ea1b7458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:14.292068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c29e5017-923f-423f-a5cb-b739ea1b7458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:14.294230 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2871/5798] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:15:14 2022] DELETE /identity/v3/projects/318093183f51425a8903d639227711c5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:15:14.299228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0357e-eef2-4ccc-a907-eb30328aff7a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:14.299575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0357e-eef2-4ccc-a907-eb30328aff7a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:14.299903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0357e-eef2-4ccc-a907-eb30328aff7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:14.322784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-24c0357e-eef2-4ccc-a907-eb30328aff7a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:14.324325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-24c0357e-eef2-4ccc-a907-eb30328aff7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:14.398179 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2928/5799] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:14.403541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302014f1-49d4-4ada-a265-c909350e3f6c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:14.403541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302014f1-49d4-4ada-a265-c909350e3f6c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:14.403844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302014f1-49d4-4ada-a265-c909350e3f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:14.426301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-302014f1-49d4-4ada-a265-c909350e3f6c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:14.427635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-302014f1-49d4-4ada-a265-c909350e3f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:14.440418 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2872/5800] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:14.441462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0357e-eef2-4ccc-a907-eb30328aff7a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:14.458819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0357e-eef2-4ccc-a907-eb30328aff7a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:14.470765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6143a-d505-4d7a-8c91-b92316f475ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:14.471450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6143a-d505-4d7a-8c91-b92316f475ed admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:14.471722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6143a-d505-4d7a-8c91-b92316f475ed admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:14.471981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6143a-d505-4d7a-8c91-b92316f475ed admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:14.474843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6143a-d505-4d7a-8c91-b92316f475ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:15:14.475148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6143a-d505-4d7a-8c91-b92316f475ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:14.478974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6143a-d505-4d7a-8c91-b92316f475ed admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:14.497992 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2929/5801] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:15:14.502369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302014f1-49d4-4ada-a265-c909350e3f6c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:14.513475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302014f1-49d4-4ada-a265-c909350e3f6c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:14.518508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ae51f-5d81-4146-b238-5f23a35383d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:14.519338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ae51f-5d81-4146-b238-5f23a35383d5 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:14.519760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ae51f-5d81-4146-b238-5f23a35383d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:14.520198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ae51f-5d81-4146-b238-5f23a35383d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:14.521031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ae51f-5d81-4146-b238-5f23a35383d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:14.523090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ae51f-5d81-4146-b238-5f23a35383d5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:14.581115 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2873/5802] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:15:14 2022] POST /identity/v3/projects => generated 361 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:15:14.664123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6143a-d505-4d7a-8c91-b92316f475ed admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:14.674605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6143a-d505-4d7a-8c91-b92316f475ed admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:14.774075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ab511-9ae4-4d87-be23-059d91b7da53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:14.776679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ab511-9ae4-4d87-be23-059d91b7da53 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:14.777154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ab511-9ae4-4d87-be23-059d91b7da53 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:14.777813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ab511-9ae4-4d87-be23-059d91b7da53 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:14.778792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ab511-9ae4-4d87-be23-059d91b7da53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:14.784226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ab511-9ae4-4d87-be23-059d91b7da53 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:14.825056 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2930/5803] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:15:14 2022] POST /identity/v3/users => generated 281 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:15:14.828149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ae51f-5d81-4146-b238-5f23a35383d5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:14.847377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ae51f-5d81-4146-b238-5f23a35383d5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:14.853878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886cc20-79a9-4240-833f-5262bdac31d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:14.854823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886cc20-79a9-4240-833f-5262bdac31d1 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:14.855327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886cc20-79a9-4240-833f-5262bdac31d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:14.855878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886cc20-79a9-4240-833f-5262bdac31d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:14.856924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886cc20-79a9-4240-833f-5262bdac31d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:14.859650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886cc20-79a9-4240-833f-5262bdac31d1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:14.876720 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2874/5804] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:15:14.880597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ab511-9ae4-4d87-be23-059d91b7da53 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:14.894037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ab511-9ae4-4d87-be23-059d91b7da53 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:14.901234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874788e-3008-4e87-90e5-902f225cd9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:14.902078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874788e-3008-4e87-90e5-902f225cd9af admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:14.903048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874788e-3008-4e87-90e5-902f225cd9af admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:14.903235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874788e-3008-4e87-90e5-902f225cd9af admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:14.905164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874788e-3008-4e87-90e5-902f225cd9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:14.919202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874788e-3008-4e87-90e5-902f225cd9af admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:14.926471 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2931/5805] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:15:14.926764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886cc20-79a9-4240-833f-5262bdac31d1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:14.941255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886cc20-79a9-4240-833f-5262bdac31d1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:14.955102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fe68d-a529-4c0d-81ff-6add64c8ba82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:14.957882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fe68d-a529-4c0d-81ff-6add64c8ba82 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:14.958734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fe68d-a529-4c0d-81ff-6add64c8ba82 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:14.959343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fe68d-a529-4c0d-81ff-6add64c8ba82 admin admin] PATH_INFO: `/v3/projects/22b1177941f74a25aabc4c5320fe9bc7/users/67daecb428214d7980cb9d5b3c3fb0fd/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:14.985937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fe68d-a529-4c0d-81ff-6add64c8ba82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b1177941f74a25aabc4c5320fe9bc7, user_id=67daecb428214d7980cb9d5b3c3fb0fd, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:14.988634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fe68d-a529-4c0d-81ff-6add64c8ba82 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:15.012453 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2875/5806] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:15:14 2022] PUT /identity/v3/projects/22b1177941f74a25aabc4c5320fe9bc7/users/67daecb428214d7980cb9d5b3c3fb0fd/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:15:15.021164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201bd00-fc79-42c8-99f6-cb26b45204f4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:15.021475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201bd00-fc79-42c8-99f6-cb26b45204f4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:15.021743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201bd00-fc79-42c8-99f6-cb26b45204f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:15.054640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5201bd00-fc79-42c8-99f6-cb26b45204f4 None None] MFA Rules not processed for user `67daecb428214d7980cb9d5b3c3fb0fd`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:15.055919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5201bd00-fc79-42c8-99f6-cb26b45204f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:15.095930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:15.186762 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2932/5807] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:15.201744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fe68d-a529-4c0d-81ff-6add64c8ba82 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:15.203772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-064fe68d-a529-4c0d-81ff-6add64c8ba82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:15.304143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fe68d-a529-4c0d-81ff-6add64c8ba82 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:15.308816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f3e2a-61bf-455a-9e2e-e6377b857ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:15.309434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f3e2a-61bf-455a-9e2e-e6377b857ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:15.309707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f3e2a-61bf-455a-9e2e-e6377b857ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:15.309980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f3e2a-61bf-455a-9e2e-e6377b857ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:15.319467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f3e2a-61bf-455a-9e2e-e6377b857ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:15.321355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f3e2a-61bf-455a-9e2e-e6377b857ac6 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:15.336746 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2876/5808] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:15:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:15:21.352846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2ecdc-2d66-49a1-8dfe-4438d6a54f88 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:21.353662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2ecdc-2d66-49a1-8dfe-4438d6a54f88 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:21.354188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2ecdc-2d66-49a1-8dfe-4438d6a54f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:21.401408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dfe2ecdc-2d66-49a1-8dfe-4438d6a54f88 None None] MFA Rules not processed for user `fa3184ee6bec45da840eb9182b9a5f4d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:21.403053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dfe2ecdc-2d66-49a1-8dfe-4438d6a54f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:21.515447 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2933/5809] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:21.540736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f3e2a-61bf-455a-9e2e-e6377b857ac6 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:21.545069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-413f3e2a-61bf-455a-9e2e-e6377b857ac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:21.573087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:21.806405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f3e2a-61bf-455a-9e2e-e6377b857ac6 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:21.815130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9d92e-8602-41d8-90f8-7515d18a12ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:21.815797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9d92e-8602-41d8-90f8-7515d18a12ed service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:21.816074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9d92e-8602-41d8-90f8-7515d18a12ed service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:21.816391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9d92e-8602-41d8-90f8-7515d18a12ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:21.827536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9d92e-8602-41d8-90f8-7515d18a12ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:21.830454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9d92e-8602-41d8-90f8-7515d18a12ed service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:21.861096 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2877/5810] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:15:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:15:25.300593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2ecdc-2d66-49a1-8dfe-4438d6a54f88 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:25.302049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dfe2ecdc-2d66-49a1-8dfe-4438d6a54f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:25.380485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2ecdc-2d66-49a1-8dfe-4438d6a54f88 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:25.394371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1a70f-141d-4743-b66c-c7874d747d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:25.395036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1a70f-141d-4743-b66c-c7874d747d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:25.395303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1a70f-141d-4743-b66c-c7874d747d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:25.395566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1a70f-141d-4743-b66c-c7874d747d91 admin admin] PATH_INFO: `/v3/users/168b34153ba14981b627cb9d4b317fcf` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:25.401136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1a70f-141d-4743-b66c-c7874d747d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=168b34153ba14981b627cb9d4b317fcf)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:25.406107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1a70f-141d-4743-b66c-c7874d747d91 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:25.494899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-01c1a70f-141d-4743-b66c-c7874d747d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '168b34153ba14981b627cb9d4b317fcf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:25.508135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-01c1a70f-141d-4743-b66c-c7874d747d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '168b34153ba14981b627cb9d4b317fcf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:25.510440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:25.519755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-01c1a70f-141d-4743-b66c-c7874d747d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '168b34153ba14981b627cb9d4b317fcf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:25.540166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393b137-9327-483e-ba20-e9f8167c4145 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:25.541018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393b137-9327-483e-ba20-e9f8167c4145 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:25.541768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393b137-9327-483e-ba20-e9f8167c4145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:25.547543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-01c1a70f-141d-4743-b66c-c7874d747d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '168b34153ba14981b627cb9d4b317fcf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:25.548733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-01c1a70f-141d-4743-b66c-c7874d747d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '168b34153ba14981b627cb9d4b317fcf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:25.560222 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2934/5811] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:15:25 2022] DELETE /identity/v3/users/168b34153ba14981b627cb9d4b317fcf => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:15:25.562460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737f8f8-e951-48cb-a13e-7cff349e5865 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:25.562968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737f8f8-e951-48cb-a13e-7cff349e5865 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:25.563250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737f8f8-e951-48cb-a13e-7cff349e5865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:25.572851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5393b137-9327-483e-ba20-e9f8167c4145 None None] MFA Rules not processed for user `fa3184ee6bec45da840eb9182b9a5f4d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:25.575249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5393b137-9327-483e-ba20-e9f8167c4145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:25.585715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d737f8f8-e951-48cb-a13e-7cff349e5865 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:25.587029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d737f8f8-e951-48cb-a13e-7cff349e5865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:25.647902 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2878/5812] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:15:25.685288 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2935/5813] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:25.707643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393b137-9327-483e-ba20-e9f8167c4145 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:25.709474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5393b137-9327-483e-ba20-e9f8167c4145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:25.798570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393b137-9327-483e-ba20-e9f8167c4145 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:25.802395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910c620-4fde-485e-879a-cc24dd1f914c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:25.803105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910c620-4fde-485e-879a-cc24dd1f914c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:25.803435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910c620-4fde-485e-879a-cc24dd1f914c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:25.803772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910c620-4fde-485e-879a-cc24dd1f914c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:25.815309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910c620-4fde-485e-879a-cc24dd1f914c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:25.817013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910c620-4fde-485e-879a-cc24dd1f914c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:25.829997 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2879/5814] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:15:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:15:26.263313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737f8f8-e951-48cb-a13e-7cff349e5865 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:26.265108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d737f8f8-e951-48cb-a13e-7cff349e5865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:26.289191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737f8f8-e951-48cb-a13e-7cff349e5865 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:26.306727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea537d23-ccb5-477a-a632-52fa19193cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:26.310861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea537d23-ccb5-477a-a632-52fa19193cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:26.311358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea537d23-ccb5-477a-a632-52fa19193cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:26.311771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea537d23-ccb5-477a-a632-52fa19193cdd admin admin] PATH_INFO: `/v3/projects/9d61c631d64e4757876fd6cd1b502f31` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:26.317041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea537d23-ccb5-477a-a632-52fa19193cdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d61c631d64e4757876fd6cd1b502f31)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:26.327293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea537d23-ccb5-477a-a632-52fa19193cdd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:26.455656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea537d23-ccb5-477a-a632-52fa19193cdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d61c631d64e4757876fd6cd1b502f31', 'request_id': 'req-ea537d23-ccb5-477a-a632-52fa19193cdd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:26.498205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea537d23-ccb5-477a-a632-52fa19193cdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d61c631d64e4757876fd6cd1b502f31', 'request_id': 'req-ea537d23-ccb5-477a-a632-52fa19193cdd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:26.505779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea537d23-ccb5-477a-a632-52fa19193cdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d61c631d64e4757876fd6cd1b502f31', 'request_id': 'req-ea537d23-ccb5-477a-a632-52fa19193cdd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:26.528821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea537d23-ccb5-477a-a632-52fa19193cdd admin admin] The token cache is being invalidate because project 9d61c631d64e4757876fd6cd1b502f31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:15:26.529374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea537d23-ccb5-477a-a632-52fa19193cdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:26.530594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea537d23-ccb5-477a-a632-52fa19193cdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:26.534759 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2936/5815] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:15:26 2022] DELETE /identity/v3/projects/9d61c631d64e4757876fd6cd1b502f31 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:15:26.539804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9bd3f-0d1b-4960-bb7d-828690f9fe53 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:26.540112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9bd3f-0d1b-4960-bb7d-828690f9fe53 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:26.540418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9bd3f-0d1b-4960-bb7d-828690f9fe53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:26.577072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6fb9bd3f-0d1b-4960-bb7d-828690f9fe53 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:26.578295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6fb9bd3f-0d1b-4960-bb7d-828690f9fe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:26.629836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:26.698578 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2880/5816] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:26.706668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49756485-dcb2-4b1d-8302-d1f3ad9ff3f6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:26.707240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49756485-dcb2-4b1d-8302-d1f3ad9ff3f6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:26.707687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49756485-dcb2-4b1d-8302-d1f3ad9ff3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:26.761533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-49756485-dcb2-4b1d-8302-d1f3ad9ff3f6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:26.768882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-49756485-dcb2-4b1d-8302-d1f3ad9ff3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:26.791575 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2937/5817] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:26.796059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9bd3f-0d1b-4960-bb7d-828690f9fe53 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:26.809701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9bd3f-0d1b-4960-bb7d-828690f9fe53 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:26.825486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439561b-42de-47ba-81cf-516c3a3a77fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:26.826129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439561b-42de-47ba-81cf-516c3a3a77fc admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:26.826403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439561b-42de-47ba-81cf-516c3a3a77fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:26.826807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439561b-42de-47ba-81cf-516c3a3a77fc admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:26.839585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439561b-42de-47ba-81cf-516c3a3a77fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:15:26.839982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439561b-42de-47ba-81cf-516c3a3a77fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:26.841692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439561b-42de-47ba-81cf-516c3a3a77fc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:26.880648 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2881/5818] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:15:26.885080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49756485-dcb2-4b1d-8302-d1f3ad9ff3f6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:26.904477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49756485-dcb2-4b1d-8302-d1f3ad9ff3f6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:26.920857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5695ca-95fc-404c-888e-3ddb7b0dc9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:26.921558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5695ca-95fc-404c-888e-3ddb7b0dc9df admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:26.921836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5695ca-95fc-404c-888e-3ddb7b0dc9df admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:26.922098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5695ca-95fc-404c-888e-3ddb7b0dc9df admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:26.922732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5695ca-95fc-404c-888e-3ddb7b0dc9df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:26.925112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5695ca-95fc-404c-888e-3ddb7b0dc9df admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:26.992801 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2938/5819] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:15:26 2022] POST /identity/v3/projects => generated 359 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:15:26.999282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439561b-42de-47ba-81cf-516c3a3a77fc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:27.015220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439561b-42de-47ba-81cf-516c3a3a77fc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:27.169938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f06180-5150-40fe-b689-922aba102090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:27.171062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f06180-5150-40fe-b689-922aba102090 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:27.171427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f06180-5150-40fe-b689-922aba102090 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:27.171777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f06180-5150-40fe-b689-922aba102090 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:27.172538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f06180-5150-40fe-b689-922aba102090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:27.175607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f06180-5150-40fe-b689-922aba102090 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:27.230651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5695ca-95fc-404c-888e-3ddb7b0dc9df admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:27.239718 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2882/5820] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:15:26 2022] POST /identity/v3/users => generated 280 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:15:27.255027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5695ca-95fc-404c-888e-3ddb7b0dc9df admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:27.261764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad276e-fb3c-484c-8c05-9f9a6a10793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:27.262977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad276e-fb3c-484c-8c05-9f9a6a10793a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:27.263496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad276e-fb3c-484c-8c05-9f9a6a10793a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:27.263943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad276e-fb3c-484c-8c05-9f9a6a10793a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:27.265039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad276e-fb3c-484c-8c05-9f9a6a10793a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:27.267507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad276e-fb3c-484c-8c05-9f9a6a10793a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:27.287898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f06180-5150-40fe-b689-922aba102090 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:27.297180 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2939/5821] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:15:27.305632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f06180-5150-40fe-b689-922aba102090 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:27.312140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f7238-1cf8-4156-8e4a-c48e5fc3e55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:27.313055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f7238-1cf8-4156-8e4a-c48e5fc3e55d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:27.313716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f7238-1cf8-4156-8e4a-c48e5fc3e55d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:27.314272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f7238-1cf8-4156-8e4a-c48e5fc3e55d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:27.315067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f7238-1cf8-4156-8e4a-c48e5fc3e55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:27.319208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f7238-1cf8-4156-8e4a-c48e5fc3e55d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:27.335596 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2883/5822] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:15:27.340148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad276e-fb3c-484c-8c05-9f9a6a10793a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:27.353135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad276e-fb3c-484c-8c05-9f9a6a10793a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:27.359193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31768cbc-3f57-426d-a0e9-bc4ff5c76d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:27.360108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31768cbc-3f57-426d-a0e9-bc4ff5c76d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:27.360395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31768cbc-3f57-426d-a0e9-bc4ff5c76d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:27.360676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31768cbc-3f57-426d-a0e9-bc4ff5c76d59 admin admin] PATH_INFO: `/v3/projects/0a4120a39edb45028aec4cd8f6d5967c/users/9a3e30cc83da46c9896b653ffe5bca5c/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:27.376535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:27.384234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31768cbc-3f57-426d-a0e9-bc4ff5c76d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4120a39edb45028aec4cd8f6d5967c, user_id=9a3e30cc83da46c9896b653ffe5bca5c, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:27.390172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31768cbc-3f57-426d-a0e9-bc4ff5c76d59 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:27.425904 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2940/5823] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:15:27 2022] PUT /identity/v3/projects/0a4120a39edb45028aec4cd8f6d5967c/users/9a3e30cc83da46c9896b653ffe5bca5c/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:15:27.428590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a8a5a-7884-45d4-a956-e819630864a1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:27.430962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a8a5a-7884-45d4-a956-e819630864a1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:27.432385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a8a5a-7884-45d4-a956-e819630864a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:27.488919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a21a8a5a-7884-45d4-a956-e819630864a1 None None] MFA Rules not processed for user `9a3e30cc83da46c9896b653ffe5bca5c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:27.490292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a21a8a5a-7884-45d4-a956-e819630864a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:27.594251 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2884/5824] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:27.604983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31768cbc-3f57-426d-a0e9-bc4ff5c76d59 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:27.606780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-31768cbc-3f57-426d-a0e9-bc4ff5c76d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:27.718515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31768cbc-3f57-426d-a0e9-bc4ff5c76d59 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:27.723062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce1fad-6d5d-4fff-bc89-8b7a614a0444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:27.724263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce1fad-6d5d-4fff-bc89-8b7a614a0444 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:27.724838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce1fad-6d5d-4fff-bc89-8b7a614a0444 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:27.725405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce1fad-6d5d-4fff-bc89-8b7a614a0444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:27.735324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce1fad-6d5d-4fff-bc89-8b7a614a0444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:27.737357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce1fad-6d5d-4fff-bc89-8b7a614a0444 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:27.748611 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2941/5825] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:15:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:15:29.411695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a8a5a-7884-45d4-a956-e819630864a1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:29.425709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a8a5a-7884-45d4-a956-e819630864a1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:29.429879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7c860-d631-4db4-b385-73ac21ed7a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:29.430687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7c860-d631-4db4-b385-73ac21ed7a4f service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:29.430959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7c860-d631-4db4-b385-73ac21ed7a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:29.431221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7c860-d631-4db4-b385-73ac21ed7a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:29.433251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-13c7c860-d631-4db4-b385-73ac21ed7a4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:29.447093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7c860-d631-4db4-b385-73ac21ed7a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:29.448660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7c860-d631-4db4-b385-73ac21ed7a4f service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:29.461320 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2885/5826] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:15:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:15:37.743938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=29,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:37.795951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce1fad-6d5d-4fff-bc89-8b7a614a0444 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:37.807757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce1fad-6d5d-4fff-bc89-8b7a614a0444 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:37.813044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab166f6-d7fd-49c6-a64c-a156ee4efc83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:37.813623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab166f6-d7fd-49c6-a64c-a156ee4efc83 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:37.813862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab166f6-d7fd-49c6-a64c-a156ee4efc83 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:37.814143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab166f6-d7fd-49c6-a64c-a156ee4efc83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:37.816206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0ab166f6-d7fd-49c6-a64c-a156ee4efc83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:37.830843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab166f6-d7fd-49c6-a64c-a156ee4efc83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:37.832412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab166f6-d7fd-49c6-a64c-a156ee4efc83 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:37.845008 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2942/5827] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:15:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:15:39.454154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:47.840178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:49.543475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe010a1-7982-46b6-bfe2-f21f5d2813ea None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:49.544063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe010a1-7982-46b6-bfe2-f21f5d2813ea None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:49.544063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe010a1-7982-46b6-bfe2-f21f5d2813ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:49.573268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1fe010a1-7982-46b6-bfe2-f21f5d2813ea None None] MFA Rules not processed for user `9671d01ad0d44cc99e0eecc966f7c3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:49.575648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1fe010a1-7982-46b6-bfe2-f21f5d2813ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:49.697467 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2886/5828] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:53.898829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab166f6-d7fd-49c6-a64c-a156ee4efc83 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:53.900540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0ab166f6-d7fd-49c6-a64c-a156ee4efc83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:54.001838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab166f6-d7fd-49c6-a64c-a156ee4efc83 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:54.009913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:54.010770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:54.011634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:54.012207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] PATH_INFO: `/v3/users/67daecb428214d7980cb9d5b3c3fb0fd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:54.013911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67daecb428214d7980cb9d5b3c3fb0fd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:54.016088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:54.084676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67daecb428214d7980cb9d5b3c3fb0fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:54.095736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67daecb428214d7980cb9d5b3c3fb0fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:54.106257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67daecb428214d7980cb9d5b3c3fb0fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:54.123495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67daecb428214d7980cb9d5b3c3fb0fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:54.124465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67daecb428214d7980cb9d5b3c3fb0fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:54.132879 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2943/5829] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:15:53 2022] DELETE /identity/v3/users/67daecb428214d7980cb9d5b3c3fb0fd => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:15:54.137571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0586da-ba8d-4f19-80d0-2b16dadc9cf4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:54.138080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0586da-ba8d-4f19-80d0-2b16dadc9cf4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:54.138080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0586da-ba8d-4f19-80d0-2b16dadc9cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:54.159711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ca0586da-ba8d-4f19-80d0-2b16dadc9cf4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:54.161163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ca0586da-ba8d-4f19-80d0-2b16dadc9cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:54.243604 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2887/5830] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:54.303911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:54.306430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:54.379659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eac9a6-95b1-47b1-b80a-a88233aec892 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:54.385053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed84344-5420-48e3-b845-a25a997ff926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:54.385714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84344-5420-48e3-b845-a25a997ff926 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:54.386047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84344-5420-48e3-b845-a25a997ff926 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:54.386379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84344-5420-48e3-b845-a25a997ff926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:54.396520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed84344-5420-48e3-b845-a25a997ff926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:54.398113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed84344-5420-48e3-b845-a25a997ff926 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:54.409905 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2944/5831] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:15:54.928057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0586da-ba8d-4f19-80d0-2b16dadc9cf4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:54.930171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ca0586da-ba8d-4f19-80d0-2b16dadc9cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:54.943490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0586da-ba8d-4f19-80d0-2b16dadc9cf4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:54.951316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8c8ae-a866-4854-95f7-007346be2660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:54.952311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8c8ae-a866-4854-95f7-007346be2660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:54.952807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8c8ae-a866-4854-95f7-007346be2660 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:54.953289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8c8ae-a866-4854-95f7-007346be2660 admin admin] PATH_INFO: `/v3/projects/22b1177941f74a25aabc4c5320fe9bc7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:54.954836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8c8ae-a866-4854-95f7-007346be2660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22b1177941f74a25aabc4c5320fe9bc7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:54.957427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8c8ae-a866-4854-95f7-007346be2660 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:55.027899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bfe8c8ae-a866-4854-95f7-007346be2660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22b1177941f74a25aabc4c5320fe9bc7', 'request_id': 'req-bfe8c8ae-a866-4854-95f7-007346be2660'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:55.034193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bfe8c8ae-a866-4854-95f7-007346be2660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22b1177941f74a25aabc4c5320fe9bc7', 'request_id': 'req-bfe8c8ae-a866-4854-95f7-007346be2660'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:55.042669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bfe8c8ae-a866-4854-95f7-007346be2660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22b1177941f74a25aabc4c5320fe9bc7', 'request_id': 'req-bfe8c8ae-a866-4854-95f7-007346be2660'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:55.054091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bfe8c8ae-a866-4854-95f7-007346be2660 admin admin] The token cache is being invalidate because project 22b1177941f74a25aabc4c5320fe9bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:15:55.054282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bfe8c8ae-a866-4854-95f7-007346be2660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:55.054934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bfe8c8ae-a866-4854-95f7-007346be2660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:15:55.056514 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2888/5832] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:15:54 2022] DELETE /identity/v3/projects/22b1177941f74a25aabc4c5320fe9bc7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:15:55.061834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc2aa1-9830-4984-b167-b38a2f6016bf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:55.062367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc2aa1-9830-4984-b167-b38a2f6016bf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:55.062367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc2aa1-9830-4984-b167-b38a2f6016bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:55.082879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-80cc2aa1-9830-4984-b167-b38a2f6016bf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:55.084577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-80cc2aa1-9830-4984-b167-b38a2f6016bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:55.107352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:55.170659 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2945/5833] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:15:55.180025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccd984-b2ca-454b-95ac-56a19527ba2a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:55.180630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccd984-b2ca-454b-95ac-56a19527ba2a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:55.180863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccd984-b2ca-454b-95ac-56a19527ba2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:55.201792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-72ccd984-b2ca-454b-95ac-56a19527ba2a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:55.203022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-72ccd984-b2ca-454b-95ac-56a19527ba2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:55.211114 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2889/5834] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:15:55.216101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc2aa1-9830-4984-b167-b38a2f6016bf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:55.240734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc2aa1-9830-4984-b167-b38a2f6016bf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:55.245605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f76156-68ab-4b33-9128-9cb7d71e3495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:55.246387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f76156-68ab-4b33-9128-9cb7d71e3495 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:55.246659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f76156-68ab-4b33-9128-9cb7d71e3495 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:55.246923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f76156-68ab-4b33-9128-9cb7d71e3495 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:55.247572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f76156-68ab-4b33-9128-9cb7d71e3495 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:15:55.247882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f76156-68ab-4b33-9128-9cb7d71e3495 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:55.249532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f76156-68ab-4b33-9128-9cb7d71e3495 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:55.268735 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2946/5835] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:15:55.273305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccd984-b2ca-454b-95ac-56a19527ba2a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:55.283890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccd984-b2ca-454b-95ac-56a19527ba2a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:55.292629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6e996-432a-46ef-a30b-a8bff1c48faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:55.293259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6e996-432a-46ef-a30b-a8bff1c48faf admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:55.293633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6e996-432a-46ef-a30b-a8bff1c48faf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:55.293788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6e996-432a-46ef-a30b-a8bff1c48faf admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:55.294606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b6e996-432a-46ef-a30b-a8bff1c48faf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:55.296724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b6e996-432a-46ef-a30b-a8bff1c48faf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:55.322981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:15:55.335661 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2890/5836] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:15:55 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:15:55.336971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f76156-68ab-4b33-9128-9cb7d71e3495 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:55.348278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f76156-68ab-4b33-9128-9cb7d71e3495 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:55.430015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe1925-16ff-424d-a7a6-e8d6f2fdfea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:55.430641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe1925-16ff-424d-a7a6-e8d6f2fdfea3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:55.430914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe1925-16ff-424d-a7a6-e8d6f2fdfea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:55.431188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe1925-16ff-424d-a7a6-e8d6f2fdfea3 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:55.431821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe1925-16ff-424d-a7a6-e8d6f2fdfea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:55.433920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe1925-16ff-424d-a7a6-e8d6f2fdfea3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:55.461126 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2947/5837] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:15:55 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:15:55.463390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6e996-432a-46ef-a30b-a8bff1c48faf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:55.485704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6e996-432a-46ef-a30b-a8bff1c48faf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:55.490352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c218bf3-558d-4b51-a6c2-f094ba6a4a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:55.490980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c218bf3-558d-4b51-a6c2-f094ba6a4a03 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:55.491462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c218bf3-558d-4b51-a6c2-f094ba6a4a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:55.491739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c218bf3-558d-4b51-a6c2-f094ba6a4a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:55.492401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c218bf3-558d-4b51-a6c2-f094ba6a4a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:55.494075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c218bf3-558d-4b51-a6c2-f094ba6a4a03 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:55.506847 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2891/5838] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:15:55.512181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe1925-16ff-424d-a7a6-e8d6f2fdfea3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:55.525974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe1925-16ff-424d-a7a6-e8d6f2fdfea3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:55.531852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badc10d-d152-4ce8-b360-900c5bb381c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:55.532510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badc10d-d152-4ce8-b360-900c5bb381c6 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:55.532916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badc10d-d152-4ce8-b360-900c5bb381c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:55.533191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badc10d-d152-4ce8-b360-900c5bb381c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:55.533981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1badc10d-d152-4ce8-b360-900c5bb381c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:55.538055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1badc10d-d152-4ce8-b360-900c5bb381c6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:55.554905 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2948/5839] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:15:55.559208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c218bf3-558d-4b51-a6c2-f094ba6a4a03 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:55.570388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c218bf3-558d-4b51-a6c2-f094ba6a4a03 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:55.576039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeb9af-efd8-4aa4-aca8-5d0750494f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:55.576746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeb9af-efd8-4aa4-aca8-5d0750494f56 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:55.577023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeb9af-efd8-4aa4-aca8-5d0750494f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:55.577292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeb9af-efd8-4aa4-aca8-5d0750494f56 admin admin] PATH_INFO: `/v3/projects/0fc396a4790b4a25b9cab5d510db1369/users/8e94def1933b4a978081db477f522bb4/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:55.598103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeb9af-efd8-4aa4-aca8-5d0750494f56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fc396a4790b4a25b9cab5d510db1369, user_id=8e94def1933b4a978081db477f522bb4, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:55.602826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeb9af-efd8-4aa4-aca8-5d0750494f56 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:55.631366 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2892/5840] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:15:55 2022] PUT /identity/v3/projects/0fc396a4790b4a25b9cab5d510db1369/users/8e94def1933b4a978081db477f522bb4/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:15:55.633486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670975b8-65f0-4104-b7a0-3cf2ba9dd002 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:55.633486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670975b8-65f0-4104-b7a0-3cf2ba9dd002 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:55.633718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670975b8-65f0-4104-b7a0-3cf2ba9dd002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:55.678511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-670975b8-65f0-4104-b7a0-3cf2ba9dd002 None None] MFA Rules not processed for user `8e94def1933b4a978081db477f522bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:15:55.679772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-670975b8-65f0-4104-b7a0-3cf2ba9dd002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:55.759991 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2949/5841] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:15:55.772494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeb9af-efd8-4aa4-aca8-5d0750494f56 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:15:55.774550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-04aeb9af-efd8-4aa4-aca8-5d0750494f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:15:55.839895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeb9af-efd8-4aa4-aca8-5d0750494f56 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:15:55.843779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030dede-3114-4f0f-b922-a3d383374792 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:15:55.846084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030dede-3114-4f0f-b922-a3d383374792 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:15:55.846084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030dede-3114-4f0f-b922-a3d383374792 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:15:55.846084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030dede-3114-4f0f-b922-a3d383374792 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:15:55.856938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030dede-3114-4f0f-b922-a3d383374792 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:15:55.858434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030dede-3114-4f0f-b922-a3d383374792 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:15:55.870112 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2893/5842] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:15:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:16:01.776516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670975b8-65f0-4104-b7a0-3cf2ba9dd002 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:01.780653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-670975b8-65f0-4104-b7a0-3cf2ba9dd002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:01.825176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:01.996866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670975b8-65f0-4104-b7a0-3cf2ba9dd002 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:02.021168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086eca67-3469-4996-b6cc-9be42f658d87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:02.021830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086eca67-3469-4996-b6cc-9be42f658d87 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:02.022077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086eca67-3469-4996-b6cc-9be42f658d87 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:02.022517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086eca67-3469-4996-b6cc-9be42f658d87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:02.025048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-086eca67-3469-4996-b6cc-9be42f658d87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:02.061152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086eca67-3469-4996-b6cc-9be42f658d87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:02.063224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086eca67-3469-4996-b6cc-9be42f658d87 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:02.091874 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2950/5843] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:16:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:16:05.864021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:06.798469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030dede-3114-4f0f-b922-a3d383374792 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:06.810909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030dede-3114-4f0f-b922-a3d383374792 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:06.815071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28c2d0-6ef9-4f22-9d72-f87f66e2b3f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:06.816601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28c2d0-6ef9-4f22-9d72-f87f66e2b3f7 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:06.816984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28c2d0-6ef9-4f22-9d72-f87f66e2b3f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:06.817351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28c2d0-6ef9-4f22-9d72-f87f66e2b3f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:06.819876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ab28c2d0-6ef9-4f22-9d72-f87f66e2b3f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:06.835114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28c2d0-6ef9-4f22-9d72-f87f66e2b3f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:06.836752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28c2d0-6ef9-4f22-9d72-f87f66e2b3f7 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:06.850139 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2894/5844] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:16:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:16:07.272787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102fee9-0005-4b11-9762-bf848389af38 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:07.273275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102fee9-0005-4b11-9762-bf848389af38 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:07.273275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102fee9-0005-4b11-9762-bf848389af38 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:07.280670 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2951/5845] 10.222.0.23 () {54 vars in 921 bytes} [Sat Aug 20 21:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:16:07.293035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2264b7e-fd88-4561-9789-b9e8e15145ba None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:07.294103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2264b7e-fd88-4561-9789-b9e8e15145ba None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:07.294270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2264b7e-fd88-4561-9789-b9e8e15145ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:07.355615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b2264b7e-fd88-4561-9789-b9e8e15145ba None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:07.358318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b2264b7e-fd88-4561-9789-b9e8e15145ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:07.380218 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2895/5846] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:16:07.390383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102fee9-0005-4b11-9762-bf848389af38 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:07.414373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102fee9-0005-4b11-9762-bf848389af38 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:07.419078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e395c22-3fd2-45f8-a45a-f728a6e186e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:07.420049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e395c22-3fd2-45f8-a45a-f728a6e186e5 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:07.420687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e395c22-3fd2-45f8-a45a-f728a6e186e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:07.421108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e395c22-3fd2-45f8-a45a-f728a6e186e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:07.436888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e395c22-3fd2-45f8-a45a-f728a6e186e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:07.438781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e395c22-3fd2-45f8-a45a-f728a6e186e5 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:07.479806 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2952/5847] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:16:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:16:10.250958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2264b7e-fd88-4561-9789-b9e8e15145ba None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:10.265240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2264b7e-fd88-4561-9789-b9e8e15145ba None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:10.269073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c5397-db22-45cb-a403-5f2fbca7a818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:10.269814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c5397-db22-45cb-a403-5f2fbca7a818 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:10.270145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c5397-db22-45cb-a403-5f2fbca7a818 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:10.276413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c5397-db22-45cb-a403-5f2fbca7a818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:10.276413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-721c5397-db22-45cb-a403-5f2fbca7a818 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:10.342952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c5397-db22-45cb-a403-5f2fbca7a818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:10.344398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c5397-db22-45cb-a403-5f2fbca7a818 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:10.356758 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2896/5848] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:16:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:16:15.222658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e395c22-3fd2-45f8-a45a-f728a6e186e5 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:15.225022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8e395c22-3fd2-45f8-a45a-f728a6e186e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:15.333474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e395c22-3fd2-45f8-a45a-f728a6e186e5 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:15.338798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c561bb-c88c-4009-a120-70e64a94d735 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:15.339408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c561bb-c88c-4009-a120-70e64a94d735 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:15.339675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c561bb-c88c-4009-a120-70e64a94d735 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:15.339935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c561bb-c88c-4009-a120-70e64a94d735 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:15.341683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-14c561bb-c88c-4009-a120-70e64a94d735 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:15.355991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c561bb-c88c-4009-a120-70e64a94d735 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:15.357615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c561bb-c88c-4009-a120-70e64a94d735 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:15.370548 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2953/5849] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:16:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:16:20.357746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=39 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:22.499332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c5397-db22-45cb-a403-5f2fbca7a818 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:22.500908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-721c5397-db22-45cb-a403-5f2fbca7a818 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:22.515498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c5397-db22-45cb-a403-5f2fbca7a818 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:22.520332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:22.520995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:22.521261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:22.521514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] PATH_INFO: `/v3/users/7c276ebf6fe14a9c8f50bd1d1f44fb58` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:22.522868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c276ebf6fe14a9c8f50bd1d1f44fb58)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:22.524822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:22.583722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c276ebf6fe14a9c8f50bd1d1f44fb58'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:22.604476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c276ebf6fe14a9c8f50bd1d1f44fb58'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:22.608603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c276ebf6fe14a9c8f50bd1d1f44fb58'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:22.609444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c276ebf6fe14a9c8f50bd1d1f44fb58'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:22.610081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c276ebf6fe14a9c8f50bd1d1f44fb58'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:22.625506 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2897/5850] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:16:22 2022] DELETE /identity/v3/users/7c276ebf6fe14a9c8f50bd1d1f44fb58 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:16:22.926766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c561bb-c88c-4009-a120-70e64a94d735 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:22.949640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c561bb-c88c-4009-a120-70e64a94d735 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:23.013980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:23.014621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:23.014907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:23.015164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] PATH_INFO: `/v3/projects/c8bfb40c6e1e496fb1a3703ddb50e8bb` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:23.016994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8bfb40c6e1e496fb1a3703ddb50e8bb)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:23.018904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:23.044888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:23.071657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8bfb40c6e1e496fb1a3703ddb50e8bb', 'request_id': 'req-d07d7dad-e6ad-4c09-8210-337f63617c81'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.080868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8bfb40c6e1e496fb1a3703ddb50e8bb', 'request_id': 'req-d07d7dad-e6ad-4c09-8210-337f63617c81'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.087492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8bfb40c6e1e496fb1a3703ddb50e8bb', 'request_id': 'req-d07d7dad-e6ad-4c09-8210-337f63617c81'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.100188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] The token cache is being invalidate because project c8bfb40c6e1e496fb1a3703ddb50e8bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:16:23.100604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.101289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.102903 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2954/5851] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:16:22 2022] DELETE /identity/v3/projects/c8bfb40c6e1e496fb1a3703ddb50e8bb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:16:23.107866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:23.109438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:23.125663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63bea7-cddb-490f-a1c5-171a310bc79d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:23.188734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040b103-e0de-4289-8d48-28ee3671b1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:23.189471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040b103-e0de-4289-8d48-28ee3671b1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:23.189746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040b103-e0de-4289-8d48-28ee3671b1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:23.190012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040b103-e0de-4289-8d48-28ee3671b1fc admin admin] PATH_INFO: `/v3/users/ae7d0d25bbf24edda377a9cc2e5a59e6` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:23.206026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040b103-e0de-4289-8d48-28ee3671b1fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae7d0d25bbf24edda377a9cc2e5a59e6)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:23.208016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040b103-e0de-4289-8d48-28ee3671b1fc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:23.260265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f040b103-e0de-4289-8d48-28ee3671b1fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae7d0d25bbf24edda377a9cc2e5a59e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.268325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f040b103-e0de-4289-8d48-28ee3671b1fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae7d0d25bbf24edda377a9cc2e5a59e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.275396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:23.281953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f040b103-e0de-4289-8d48-28ee3671b1fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae7d0d25bbf24edda377a9cc2e5a59e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.282998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f040b103-e0de-4289-8d48-28ee3671b1fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae7d0d25bbf24edda377a9cc2e5a59e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.283967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f040b103-e0de-4289-8d48-28ee3671b1fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae7d0d25bbf24edda377a9cc2e5a59e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.301301 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2898/5852] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:16:23 2022] DELETE /identity/v3/users/ae7d0d25bbf24edda377a9cc2e5a59e6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:16:23.492671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:23.494239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:23.510757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d7dad-e6ad-4c09-8210-337f63617c81 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:23.574999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca39293-68a4-48b8-8683-9cc4ea3e299b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:23.575639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca39293-68a4-48b8-8683-9cc4ea3e299b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:23.575909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca39293-68a4-48b8-8683-9cc4ea3e299b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:23.576172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca39293-68a4-48b8-8683-9cc4ea3e299b admin admin] PATH_INFO: `/v3/projects/57f3b0c432c4400f9c2a9a456b37dbf2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:23.587511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca39293-68a4-48b8-8683-9cc4ea3e299b admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f3b0c432c4400f9c2a9a456b37dbf2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:23.589900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca39293-68a4-48b8-8683-9cc4ea3e299b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:23.649566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7ca39293-68a4-48b8-8683-9cc4ea3e299b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f3b0c432c4400f9c2a9a456b37dbf2', 'request_id': 'req-7ca39293-68a4-48b8-8683-9cc4ea3e299b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.660799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7ca39293-68a4-48b8-8683-9cc4ea3e299b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f3b0c432c4400f9c2a9a456b37dbf2', 'request_id': 'req-7ca39293-68a4-48b8-8683-9cc4ea3e299b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.670011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7ca39293-68a4-48b8-8683-9cc4ea3e299b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f3b0c432c4400f9c2a9a456b37dbf2', 'request_id': 'req-7ca39293-68a4-48b8-8683-9cc4ea3e299b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.684713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7ca39293-68a4-48b8-8683-9cc4ea3e299b admin admin] The token cache is being invalidate because project 57f3b0c432c4400f9c2a9a456b37dbf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:16:23.685041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7ca39293-68a4-48b8-8683-9cc4ea3e299b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.685648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7ca39293-68a4-48b8-8683-9cc4ea3e299b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:23.687380 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2955/5853] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:16:23 2022] DELETE /identity/v3/projects/57f3b0c432c4400f9c2a9a456b37dbf2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:16:23.693849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2798bf-980e-4c3f-89d5-b07249ab7d67 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:23.696364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2798bf-980e-4c3f-89d5-b07249ab7d67 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:23.697032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2798bf-980e-4c3f-89d5-b07249ab7d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:23.732757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3e2798bf-980e-4c3f-89d5-b07249ab7d67 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:23.734284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3e2798bf-980e-4c3f-89d5-b07249ab7d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:23.812623 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2899/5854] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:16:23.822250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1183a8-2b7c-4027-a39a-53602bbca3df None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:23.822617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1183a8-2b7c-4027-a39a-53602bbca3df None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:23.823012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1183a8-2b7c-4027-a39a-53602bbca3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:23.851929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1d1183a8-2b7c-4027-a39a-53602bbca3df None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:23.855606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1d1183a8-2b7c-4027-a39a-53602bbca3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:23.868440 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2956/5855] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:16:23.870296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2798bf-980e-4c3f-89d5-b07249ab7d67 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:23.881335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2798bf-980e-4c3f-89d5-b07249ab7d67 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:23.887946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce681e-d5c5-4907-b7f4-5227a85fed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:23.888684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce681e-d5c5-4907-b7f4-5227a85fed51 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:23.888994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce681e-d5c5-4907-b7f4-5227a85fed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:23.889252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce681e-d5c5-4907-b7f4-5227a85fed51 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:23.889997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce681e-d5c5-4907-b7f4-5227a85fed51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:16:23.890371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce681e-d5c5-4907-b7f4-5227a85fed51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:23.892040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce681e-d5c5-4907-b7f4-5227a85fed51 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:23.907065 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2900/5856] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:16:23.911993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1183a8-2b7c-4027-a39a-53602bbca3df None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:23.935343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1183a8-2b7c-4027-a39a-53602bbca3df None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:23.951265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a0cf9-ea3e-4535-9f79-a17f7f59f883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:23.951895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a0cf9-ea3e-4535-9f79-a17f7f59f883 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:23.952162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a0cf9-ea3e-4535-9f79-a17f7f59f883 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:23.952439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a0cf9-ea3e-4535-9f79-a17f7f59f883 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:23.953506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a0cf9-ea3e-4535-9f79-a17f7f59f883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:23.955740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a0cf9-ea3e-4535-9f79-a17f7f59f883 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:24.006473 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2957/5857] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:16:23 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:16:24.007329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce681e-d5c5-4907-b7f4-5227a85fed51 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:24.019549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce681e-d5c5-4907-b7f4-5227a85fed51 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:24.110887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce0a2f-b3a4-4701-9b5d-6fc3a23bb806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:24.111655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce0a2f-b3a4-4701-9b5d-6fc3a23bb806 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:24.111925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce0a2f-b3a4-4701-9b5d-6fc3a23bb806 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:24.112277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce0a2f-b3a4-4701-9b5d-6fc3a23bb806 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:24.113093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce0a2f-b3a4-4701-9b5d-6fc3a23bb806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:24.115176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce0a2f-b3a4-4701-9b5d-6fc3a23bb806 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:24.147482 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2901/5858] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:16:24 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:16:24.149122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a0cf9-ea3e-4535-9f79-a17f7f59f883 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:24.156178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:24.165590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a0cf9-ea3e-4535-9f79-a17f7f59f883 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:24.184136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d291453-81c7-4bc7-838f-782035776918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:24.185034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d291453-81c7-4bc7-838f-782035776918 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:24.185578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d291453-81c7-4bc7-838f-782035776918 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:24.186056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d291453-81c7-4bc7-838f-782035776918 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:24.187085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d291453-81c7-4bc7-838f-782035776918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:24.188939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d291453-81c7-4bc7-838f-782035776918 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:24.202422 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2958/5859] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:16:24.206593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce0a2f-b3a4-4701-9b5d-6fc3a23bb806 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:24.218115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce0a2f-b3a4-4701-9b5d-6fc3a23bb806 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:24.223216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e62dd4-b333-4302-8ab6-78f1bc5d1065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:24.224461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e62dd4-b333-4302-8ab6-78f1bc5d1065 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:24.224461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e62dd4-b333-4302-8ab6-78f1bc5d1065 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:24.224461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e62dd4-b333-4302-8ab6-78f1bc5d1065 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:24.228128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e62dd4-b333-4302-8ab6-78f1bc5d1065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:24.228128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e62dd4-b333-4302-8ab6-78f1bc5d1065 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:24.241612 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2902/5860] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:16:24.246119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d291453-81c7-4bc7-838f-782035776918 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:24.258238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d291453-81c7-4bc7-838f-782035776918 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:24.263833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea3a7d-b1e6-412d-904e-1867a08bd64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:24.265206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea3a7d-b1e6-412d-904e-1867a08bd64b admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:24.265477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea3a7d-b1e6-412d-904e-1867a08bd64b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:24.265738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea3a7d-b1e6-412d-904e-1867a08bd64b admin admin] PATH_INFO: `/v3/projects/6dacfd0976654c86bff273c82e405b9a/users/37d0e7b8014b4a91a4b4eaaf2fd1ffcd/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:24.284560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea3a7d-b1e6-412d-904e-1867a08bd64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dacfd0976654c86bff273c82e405b9a, user_id=37d0e7b8014b4a91a4b4eaaf2fd1ffcd, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:24.287508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea3a7d-b1e6-412d-904e-1867a08bd64b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:24.305808 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2959/5861] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:16:24 2022] PUT /identity/v3/projects/6dacfd0976654c86bff273c82e405b9a/users/37d0e7b8014b4a91a4b4eaaf2fd1ffcd/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:16:24.316798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a07565-4b22-4ebe-8b8f-43712518296e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:24.317120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a07565-4b22-4ebe-8b8f-43712518296e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:24.317345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a07565-4b22-4ebe-8b8f-43712518296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:24.365126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c7a07565-4b22-4ebe-8b8f-43712518296e None None] MFA Rules not processed for user `37d0e7b8014b4a91a4b4eaaf2fd1ffcd`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:24.366402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c7a07565-4b22-4ebe-8b8f-43712518296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:24.422843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:24.469892 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2903/5862] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:16:24.475049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea3a7d-b1e6-412d-904e-1867a08bd64b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:24.554186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea3a7d-b1e6-412d-904e-1867a08bd64b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:24.559381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231adf76-1dbc-43df-8de4-808d95437a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:24.560319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231adf76-1dbc-43df-8de4-808d95437a04 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:24.560617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231adf76-1dbc-43df-8de4-808d95437a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:24.560878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231adf76-1dbc-43df-8de4-808d95437a04 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:24.561500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231adf76-1dbc-43df-8de4-808d95437a04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:24.563414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231adf76-1dbc-43df-8de4-808d95437a04 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:24.595509 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2960/5863] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:16:24 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:16:24.596690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a07565-4b22-4ebe-8b8f-43712518296e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:24.607918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a07565-4b22-4ebe-8b8f-43712518296e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:24.668633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f5508-6570-4fe6-b111-7a3b22a51d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:24.669148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f5508-6570-4fe6-b111-7a3b22a51d70 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:24.669416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f5508-6570-4fe6-b111-7a3b22a51d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:24.669677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f5508-6570-4fe6-b111-7a3b22a51d70 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:24.670603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f5508-6570-4fe6-b111-7a3b22a51d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:24.672551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f5508-6570-4fe6-b111-7a3b22a51d70 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:24.697710 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2904/5864] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:16:24 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:16:24.702267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231adf76-1dbc-43df-8de4-808d95437a04 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:24.713648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231adf76-1dbc-43df-8de4-808d95437a04 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:24.718855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a796652-5c95-4d44-b24c-40391964380e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:24.719587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a796652-5c95-4d44-b24c-40391964380e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:24.719748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a796652-5c95-4d44-b24c-40391964380e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:24.720017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a796652-5c95-4d44-b24c-40391964380e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:24.720706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a796652-5c95-4d44-b24c-40391964380e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:24.722271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a796652-5c95-4d44-b24c-40391964380e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:24.734274 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2961/5865] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:16:24.738778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f5508-6570-4fe6-b111-7a3b22a51d70 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:24.749813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f5508-6570-4fe6-b111-7a3b22a51d70 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:24.755128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cac3a-2a8f-43e1-97d1-44dd3cab980e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:24.755802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cac3a-2a8f-43e1-97d1-44dd3cab980e admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:24.756064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cac3a-2a8f-43e1-97d1-44dd3cab980e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:24.756480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cac3a-2a8f-43e1-97d1-44dd3cab980e admin admin] PATH_INFO: `/v3/projects/e858ef63ca444c9c934adf01aeb0c797/users/5b26b55a6d83450289b5cdeb2186b1ed/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:24.772811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cac3a-2a8f-43e1-97d1-44dd3cab980e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e858ef63ca444c9c934adf01aeb0c797, user_id=5b26b55a6d83450289b5cdeb2186b1ed, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:24.776323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cac3a-2a8f-43e1-97d1-44dd3cab980e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:24.792610 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2905/5866] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:16:24 2022] PUT /identity/v3/projects/e858ef63ca444c9c934adf01aeb0c797/users/5b26b55a6d83450289b5cdeb2186b1ed/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:16:24.796500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a796652-5c95-4d44-b24c-40391964380e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:24.807033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a796652-5c95-4d44-b24c-40391964380e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:24.870599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088b22c-435c-4500-abd5-c8578fa6169a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:24.871354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088b22c-435c-4500-abd5-c8578fa6169a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:24.871633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088b22c-435c-4500-abd5-c8578fa6169a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:24.871900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088b22c-435c-4500-abd5-c8578fa6169a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:24.872599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088b22c-435c-4500-abd5-c8578fa6169a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:24.874208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088b22c-435c-4500-abd5-c8578fa6169a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:24.885658 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2962/5867] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:16:24.890072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cac3a-2a8f-43e1-97d1-44dd3cab980e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:24.901710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cac3a-2a8f-43e1-97d1-44dd3cab980e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:24.907598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9d887-b9c2-4cec-bdf2-8dea6782d28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:24.908347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d887-b9c2-4cec-bdf2-8dea6782d28d admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:24.908720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d887-b9c2-4cec-bdf2-8dea6782d28d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:24.909101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d887-b9c2-4cec-bdf2-8dea6782d28d admin admin] PATH_INFO: `/v3/projects/e858ef63ca444c9c934adf01aeb0c797/users/5b26b55a6d83450289b5cdeb2186b1ed/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:24.911708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9d887-b9c2-4cec-bdf2-8dea6782d28d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e858ef63ca444c9c934adf01aeb0c797, user_id=5b26b55a6d83450289b5cdeb2186b1ed, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:24.914442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9d887-b9c2-4cec-bdf2-8dea6782d28d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:24.928581 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2906/5868] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:16:24 2022] PUT /identity/v3/projects/e858ef63ca444c9c934adf01aeb0c797/users/5b26b55a6d83450289b5cdeb2186b1ed/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:16:24.939019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c193e-db42-43df-a69e-d3eaf8786e5f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:24.939554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c193e-db42-43df-a69e-d3eaf8786e5f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:24.939554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c193e-db42-43df-a69e-d3eaf8786e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:24.970116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-804c193e-db42-43df-a69e-d3eaf8786e5f None None] MFA Rules not processed for user `5b26b55a6d83450289b5cdeb2186b1ed`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:24.971408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-804c193e-db42-43df-a69e-d3eaf8786e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:24.997724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:25.044625 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2963/5869] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:16:25.056996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9d887-b9c2-4cec-bdf2-8dea6782d28d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:25.058873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-07a9d887-b9c2-4cec-bdf2-8dea6782d28d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:25.142374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9d887-b9c2-4cec-bdf2-8dea6782d28d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:25.149228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fa86e-98b5-422e-af32-56e961729451 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:25.149925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fa86e-98b5-422e-af32-56e961729451 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:25.150263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fa86e-98b5-422e-af32-56e961729451 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:25.150595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fa86e-98b5-422e-af32-56e961729451 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:25.160406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fa86e-98b5-422e-af32-56e961729451 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:25.162211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fa86e-98b5-422e-af32-56e961729451 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:25.176125 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2907/5870] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:16:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:16:25.192198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1871fea-13ce-48cd-a034-bd242d1bafd1 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:25.192561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1871fea-13ce-48cd-a034-bd242d1bafd1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:25.192925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1871fea-13ce-48cd-a034-bd242d1bafd1 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:25.194878 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2964/5871] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:16:25.200112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fa86e-98b5-422e-af32-56e961729451 service nova] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:16:25.227498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fa86e-98b5-422e-af32-56e961729451 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:25.248774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fa86e-98b5-422e-af32-56e961729451 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:25.260324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68402cac-c2a3-4ee5-9819-ab3fab6607b5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26b55a6d83450289b5cdeb2186b1ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e858ef63ca444c9c934adf01aeb0c797', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:25.260324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68402cac-c2a3-4ee5-9819-ab3fab6607b5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:25.260324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68402cac-c2a3-4ee5-9819-ab3fab6607b5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:25.260873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68402cac-c2a3-4ee5-9819-ab3fab6607b5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] PATH_INFO: `/v3/projects/6dacfd0976654c86bff273c82e405b9a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:25.262377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68402cac-c2a3-4ee5-9819-ab3fab6607b5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorizing `identity:get_project(project_id=6dacfd0976654c86bff273c82e405b9a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:25.265163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68402cac-c2a3-4ee5-9819-ab3fab6607b5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:25.267879 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2908/5872] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:16:25 2022] GET /identity/v3/projects/6dacfd0976654c86bff273c82e405b9a => generated 381 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:16:25.460874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39d416-4f35-474e-bb3f-b6b2219fe129 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:25.461410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39d416-4f35-474e-bb3f-b6b2219fe129 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:25.461816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39d416-4f35-474e-bb3f-b6b2219fe129 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:25.463601 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2965/5873] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:16:25.477743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68402cac-c2a3-4ee5-9819-ab3fab6607b5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:16:25.489247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:25.512868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68402cac-c2a3-4ee5-9819-ab3fab6607b5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:25.528195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68402cac-c2a3-4ee5-9819-ab3fab6607b5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:25.534303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a961a6-11f5-4829-b176-cdab8e88ff57 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26b55a6d83450289b5cdeb2186b1ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e858ef63ca444c9c934adf01aeb0c797', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:25.535131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a961a6-11f5-4829-b176-cdab8e88ff57 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:25.535483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a961a6-11f5-4829-b176-cdab8e88ff57 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:25.535825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a961a6-11f5-4829-b176-cdab8e88ff57 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] PATH_INFO: `/v3/projects/6dacfd0976654c86bff273c82e405b9a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:25.537737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a961a6-11f5-4829-b176-cdab8e88ff57 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorizing `identity:get_project(project_id=6dacfd0976654c86bff273c82e405b9a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:25.540978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a961a6-11f5-4829-b176-cdab8e88ff57 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:25.543821 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2909/5874] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:16:25 2022] GET /identity/v3/projects/6dacfd0976654c86bff273c82e405b9a => generated 381 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:16:25.643300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d800a1-bf70-4923-a86e-4486842519a9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:25.643664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d800a1-bf70-4923-a86e-4486842519a9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:25.643997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d800a1-bf70-4923-a86e-4486842519a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:25.671430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-84d800a1-bf70-4923-a86e-4486842519a9 None None] MFA Rules not processed for user `37d0e7b8014b4a91a4b4eaaf2fd1ffcd`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:25.674870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-84d800a1-bf70-4923-a86e-4486842519a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:25.741642 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2966/5875] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:16:25.753975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a961a6-11f5-4829-b176-cdab8e88ff57 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:25.770937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a961a6-11f5-4829-b176-cdab8e88ff57 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:25.787146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e3766-66f1-4aba-8445-bbad74cd1137 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:25.787684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e3766-66f1-4aba-8445-bbad74cd1137 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:25.787956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e3766-66f1-4aba-8445-bbad74cd1137 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:25.788222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e3766-66f1-4aba-8445-bbad74cd1137 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:25.808875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e3766-66f1-4aba-8445-bbad74cd1137 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:25.810571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e3766-66f1-4aba-8445-bbad74cd1137 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:25.835903 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2910/5876] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:16:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:16:26.483659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d182621-59fe-4df4-ac6b-7093d2e5fbe6 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:26.484152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d182621-59fe-4df4-ac6b-7093d2e5fbe6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:26.484764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d182621-59fe-4df4-ac6b-7093d2e5fbe6 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:26.486708 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2967/5877] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:16:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:16:26.500148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e3766-66f1-4aba-8445-bbad74cd1137 service nova] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:16:26.502256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-942e3766-66f1-4aba-8445-bbad74cd1137 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:26.529430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e3766-66f1-4aba-8445-bbad74cd1137 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:26.559571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e3766-66f1-4aba-8445-bbad74cd1137 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:26.577896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef68d6-61ee-4d7c-8ae7-b779681d76af tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26b55a6d83450289b5cdeb2186b1ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e858ef63ca444c9c934adf01aeb0c797', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:26.578658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef68d6-61ee-4d7c-8ae7-b779681d76af tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:26.579017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef68d6-61ee-4d7c-8ae7-b779681d76af tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:26.580702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef68d6-61ee-4d7c-8ae7-b779681d76af tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] PATH_INFO: `/v3/projects/6dacfd0976654c86bff273c82e405b9a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:26.581979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef68d6-61ee-4d7c-8ae7-b779681d76af tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorizing `identity:get_project(project_id=6dacfd0976654c86bff273c82e405b9a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:26.584791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef68d6-61ee-4d7c-8ae7-b779681d76af tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:26.598166 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2911/5878] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:16:26 2022] GET /identity/v3/projects/6dacfd0976654c86bff273c82e405b9a => generated 381 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:16:26.764252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5079f8b-9c0e-4bf6-9c68-42d9207969a4 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:26.764575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5079f8b-9c0e-4bf6-9c68-42d9207969a4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:26.764850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5079f8b-9c0e-4bf6-9c68-42d9207969a4 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:26.774354 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2968/5879] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:16:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:16:26.780692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef68d6-61ee-4d7c-8ae7-b779681d76af tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:16:26.817055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef68d6-61ee-4d7c-8ae7-b779681d76af tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:26.838791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef68d6-61ee-4d7c-8ae7-b779681d76af tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:26.855080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8646e8-3629-417b-8d47-bb34e20cb8d6 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26b55a6d83450289b5cdeb2186b1ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e858ef63ca444c9c934adf01aeb0c797', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:26.855821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8646e8-3629-417b-8d47-bb34e20cb8d6 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:26.856095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8646e8-3629-417b-8d47-bb34e20cb8d6 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:26.856361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8646e8-3629-417b-8d47-bb34e20cb8d6 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] PATH_INFO: `/v3/projects/6dacfd0976654c86bff273c82e405b9a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:26.862368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8646e8-3629-417b-8d47-bb34e20cb8d6 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorizing `identity:get_project(project_id=6dacfd0976654c86bff273c82e405b9a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:26.864934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8646e8-3629-417b-8d47-bb34e20cb8d6 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:26.879136 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2912/5880] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:16:26 2022] GET /identity/v3/projects/6dacfd0976654c86bff273c82e405b9a => generated 381 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:16:26.952526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221935-2f46-4e3f-9b57-585bba6d0e7f None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:26.952833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221935-2f46-4e3f-9b57-585bba6d0e7f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:26.953105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221935-2f46-4e3f-9b57-585bba6d0e7f None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:26.961469 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2969/5881] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:16:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:16:26.981440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8646e8-3629-417b-8d47-bb34e20cb8d6 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:16:27.006723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8646e8-3629-417b-8d47-bb34e20cb8d6 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:27.029857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8646e8-3629-417b-8d47-bb34e20cb8d6 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:27.043796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696bb6f-e4e3-4420-b1f2-bff14c49cedf tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26b55a6d83450289b5cdeb2186b1ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e858ef63ca444c9c934adf01aeb0c797', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:27.044588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696bb6f-e4e3-4420-b1f2-bff14c49cedf tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:27.044906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696bb6f-e4e3-4420-b1f2-bff14c49cedf tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:27.045176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696bb6f-e4e3-4420-b1f2-bff14c49cedf tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] PATH_INFO: `/v3/projects/6dacfd0976654c86bff273c82e405b9a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:27.046495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696bb6f-e4e3-4420-b1f2-bff14c49cedf tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorizing `identity:get_project(project_id=6dacfd0976654c86bff273c82e405b9a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:27.048788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696bb6f-e4e3-4420-b1f2-bff14c49cedf tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:27.053829 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2913/5882] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:16:26 2022] GET /identity/v3/projects/6dacfd0976654c86bff273c82e405b9a => generated 381 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:16:27.167028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf5f08-8e2e-4682-a43e-d3d5d6aa640c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:27.167487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf5f08-8e2e-4682-a43e-d3d5d6aa640c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:27.167768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf5f08-8e2e-4682-a43e-d3d5d6aa640c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:27.241812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e4cf5f08-8e2e-4682-a43e-d3d5d6aa640c None None] MFA Rules not processed for user `37d0e7b8014b4a91a4b4eaaf2fd1ffcd`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:27.243858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e4cf5f08-8e2e-4682-a43e-d3d5d6aa640c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:27.266828 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2970/5883] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:16:28.185357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2833149-8b24-4ac6-8900-095bd9c15861 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:28.185924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2833149-8b24-4ac6-8900-095bd9c15861 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:28.185924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2833149-8b24-4ac6-8900-095bd9c15861 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:28.187508 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2914/5884] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:16:28.192554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf5f08-8e2e-4682-a43e-d3d5d6aa640c None None] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:16:28.209357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf5f08-8e2e-4682-a43e-d3d5d6aa640c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:28.220995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf5f08-8e2e-4682-a43e-d3d5d6aa640c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:28.225901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1cca1-cf27-4162-bff8-ec50f939d147 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26b55a6d83450289b5cdeb2186b1ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e858ef63ca444c9c934adf01aeb0c797', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:28.226576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1cca1-cf27-4162-bff8-ec50f939d147 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:28.226853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1cca1-cf27-4162-bff8-ec50f939d147 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:28.227120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1cca1-cf27-4162-bff8-ec50f939d147 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] PATH_INFO: `/v3/projects/6dacfd0976654c86bff273c82e405b9a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:28.228267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1cca1-cf27-4162-bff8-ec50f939d147 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorizing `identity:get_project(project_id=6dacfd0976654c86bff273c82e405b9a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:28.230813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1cca1-cf27-4162-bff8-ec50f939d147 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:28.232688 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2971/5885] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:16:28 2022] GET /identity/v3/projects/6dacfd0976654c86bff273c82e405b9a => generated 381 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:16:28.331297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58229471-d850-4287-9a05-a7869e95df4c None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:28.331676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58229471-d850-4287-9a05-a7869e95df4c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:28.332149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58229471-d850-4287-9a05-a7869e95df4c None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:28.333587 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2915/5886] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:16:28.339356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1cca1-cf27-4162-bff8-ec50f939d147 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:16:28.356193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1cca1-cf27-4162-bff8-ec50f939d147 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:28.369009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1cca1-cf27-4162-bff8-ec50f939d147 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:28.376511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88d8cc-573b-409a-844e-230996cb6ab5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26b55a6d83450289b5cdeb2186b1ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e858ef63ca444c9c934adf01aeb0c797', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:28.377923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88d8cc-573b-409a-844e-230996cb6ab5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:28.378328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88d8cc-573b-409a-844e-230996cb6ab5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:28.378668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88d8cc-573b-409a-844e-230996cb6ab5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] PATH_INFO: `/v3/projects/6dacfd0976654c86bff273c82e405b9a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:28.380282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88d8cc-573b-409a-844e-230996cb6ab5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorizing `identity:get_project(project_id=6dacfd0976654c86bff273c82e405b9a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:28.382747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88d8cc-573b-409a-844e-230996cb6ab5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:28.384996 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2972/5887] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:16:28 2022] GET /identity/v3/projects/6dacfd0976654c86bff273c82e405b9a => generated 381 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:16:28.436554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0360d03-6a66-431e-9d1f-5fbf4ea5ab78 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:28.436827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0360d03-6a66-431e-9d1f-5fbf4ea5ab78 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:28.437228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0360d03-6a66-431e-9d1f-5fbf4ea5ab78 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:28.439201 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2916/5888] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:16:28.445664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88d8cc-573b-409a-844e-230996cb6ab5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:16:28.463865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88d8cc-573b-409a-844e-230996cb6ab5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:28.477220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88d8cc-573b-409a-844e-230996cb6ab5 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:28.485785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd910a-c76b-4b01-b5ef-37888ca5dd53 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26b55a6d83450289b5cdeb2186b1ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e858ef63ca444c9c934adf01aeb0c797', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:28.486902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd910a-c76b-4b01-b5ef-37888ca5dd53 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:28.487370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd910a-c76b-4b01-b5ef-37888ca5dd53 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:28.487802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd910a-c76b-4b01-b5ef-37888ca5dd53 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] PATH_INFO: `/v3/projects/6dacfd0976654c86bff273c82e405b9a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:28.489339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd910a-c76b-4b01-b5ef-37888ca5dd53 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorizing `identity:get_project(project_id=6dacfd0976654c86bff273c82e405b9a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:28.491962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd910a-c76b-4b01-b5ef-37888ca5dd53 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:28.509039 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2973/5889] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:16:28 2022] GET /identity/v3/projects/6dacfd0976654c86bff273c82e405b9a => generated 381 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:16:28.606476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f25127-40f9-43b0-ba55-1e2cec49a470 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:28.611827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f25127-40f9-43b0-ba55-1e2cec49a470 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:28.612490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f25127-40f9-43b0-ba55-1e2cec49a470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:28.640574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c2f25127-40f9-43b0-ba55-1e2cec49a470 None None] MFA Rules not processed for user `37d0e7b8014b4a91a4b4eaaf2fd1ffcd`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:28.642228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c2f25127-40f9-43b0-ba55-1e2cec49a470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:28.668500 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2917/5890] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:16:29.417078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddda06c-ab82-4f3d-a175-eabfd1ee6c36 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:29.418108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddda06c-ab82-4f3d-a175-eabfd1ee6c36 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:29.418599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddda06c-ab82-4f3d-a175-eabfd1ee6c36 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:29.421123 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2974/5891] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:16:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:16:29.428283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f25127-40f9-43b0-ba55-1e2cec49a470 None None] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:16:29.453779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f25127-40f9-43b0-ba55-1e2cec49a470 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:29.474630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f25127-40f9-43b0-ba55-1e2cec49a470 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:29.492573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c20f35-2b6e-4873-b45a-64eb91737f44 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26b55a6d83450289b5cdeb2186b1ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e858ef63ca444c9c934adf01aeb0c797', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:29.493330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c20f35-2b6e-4873-b45a-64eb91737f44 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:29.493694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c20f35-2b6e-4873-b45a-64eb91737f44 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:29.493928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c20f35-2b6e-4873-b45a-64eb91737f44 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] PATH_INFO: `/v3/projects/6dacfd0976654c86bff273c82e405b9a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:29.497800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c20f35-2b6e-4873-b45a-64eb91737f44 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorizing `identity:get_project(project_id=6dacfd0976654c86bff273c82e405b9a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:29.506138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c20f35-2b6e-4873-b45a-64eb91737f44 tempest-QuotasAdminNegativeTest-1623101360 tempest-QuotasAdminNegativeTest-1623101360-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:29.519559 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2918/5892] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:16:29 2022] GET /identity/v3/projects/6dacfd0976654c86bff273c82e405b9a => generated 381 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:16:29.657774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddda06c-ab82-4f3d-a175-eabfd1ee6c36 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:29.778857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddda06c-ab82-4f3d-a175-eabfd1ee6c36 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:29.786401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:29.786739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:29.787532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:29.787863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] PATH_INFO: `/v3/users/37d0e7b8014b4a91a4b4eaaf2fd1ffcd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:29.789488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=37d0e7b8014b4a91a4b4eaaf2fd1ffcd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:29.791705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:29.896086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:29.913658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37d0e7b8014b4a91a4b4eaaf2fd1ffcd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:29.931192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37d0e7b8014b4a91a4b4eaaf2fd1ffcd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:29.943418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37d0e7b8014b4a91a4b4eaaf2fd1ffcd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:29.985997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37d0e7b8014b4a91a4b4eaaf2fd1ffcd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:29.987217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37d0e7b8014b4a91a4b4eaaf2fd1ffcd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:29.998772 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2975/5893] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:16:29 2022] DELETE /identity/v3/users/37d0e7b8014b4a91a4b4eaaf2fd1ffcd => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:16:30.004039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac775a1-ded6-4f7f-9e74-2a8e3884834e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:30.004398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac775a1-ded6-4f7f-9e74-2a8e3884834e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:30.004621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac775a1-ded6-4f7f-9e74-2a8e3884834e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:30.054707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bac775a1-ded6-4f7f-9e74-2a8e3884834e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:30.057420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bac775a1-ded6-4f7f-9e74-2a8e3884834e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:30.180090 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2919/5894] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:16:30.203408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:30.205053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:30.290962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab8d02-f12f-48b7-9d3b-e0cc5de958ed admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:30.295095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204dac6-2f1a-4854-85b4-d9f864429845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:30.296208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204dac6-2f1a-4854-85b4-d9f864429845 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:30.296682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204dac6-2f1a-4854-85b4-d9f864429845 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:30.297127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204dac6-2f1a-4854-85b4-d9f864429845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:30.310396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204dac6-2f1a-4854-85b4-d9f864429845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:30.313721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204dac6-2f1a-4854-85b4-d9f864429845 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:30.327841 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2976/5895] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:16:31.293453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac775a1-ded6-4f7f-9e74-2a8e3884834e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:31.294935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bac775a1-ded6-4f7f-9e74-2a8e3884834e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:31.315272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac775a1-ded6-4f7f-9e74-2a8e3884834e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:31.331965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:31.332956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:31.333381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:31.333993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] PATH_INFO: `/v3/projects/6dacfd0976654c86bff273c82e405b9a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:31.335804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dacfd0976654c86bff273c82e405b9a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:31.338358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:31.496212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dacfd0976654c86bff273c82e405b9a', 'request_id': 'req-125170b9-4a33-4c53-86e7-1f4c9fee4bab'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:31.503991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dacfd0976654c86bff273c82e405b9a', 'request_id': 'req-125170b9-4a33-4c53-86e7-1f4c9fee4bab'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:31.518920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dacfd0976654c86bff273c82e405b9a', 'request_id': 'req-125170b9-4a33-4c53-86e7-1f4c9fee4bab'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:31.548182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] The token cache is being invalidate because project 6dacfd0976654c86bff273c82e405b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:16:31.548795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:31.549777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:31.554911 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2920/5896] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:16:31 2022] DELETE /identity/v3/projects/6dacfd0976654c86bff273c82e405b9a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:16:31.561040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204dac6-2f1a-4854-85b4-d9f864429845 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:31.615807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204dac6-2f1a-4854-85b4-d9f864429845 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:31.763859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9313c-b9bc-438e-8872-b307f043d6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:31.767133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9313c-b9bc-438e-8872-b307f043d6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:31.767653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9313c-b9bc-438e-8872-b307f043d6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:31.768237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9313c-b9bc-438e-8872-b307f043d6cf admin admin] PATH_INFO: `/v3/users/5b26b55a6d83450289b5cdeb2186b1ed` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:31.771767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9313c-b9bc-438e-8872-b307f043d6cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b26b55a6d83450289b5cdeb2186b1ed)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:31.774205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9313c-b9bc-438e-8872-b307f043d6cf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:31.858342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49e9313c-b9bc-438e-8872-b307f043d6cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b26b55a6d83450289b5cdeb2186b1ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:31.874540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49e9313c-b9bc-438e-8872-b307f043d6cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b26b55a6d83450289b5cdeb2186b1ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:31.891137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:31.897801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49e9313c-b9bc-438e-8872-b307f043d6cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b26b55a6d83450289b5cdeb2186b1ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:31.930918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49e9313c-b9bc-438e-8872-b307f043d6cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b26b55a6d83450289b5cdeb2186b1ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:31.932895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49e9313c-b9bc-438e-8872-b307f043d6cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b26b55a6d83450289b5cdeb2186b1ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:31.946523 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2977/5897] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:16:31 2022] DELETE /identity/v3/users/5b26b55a6d83450289b5cdeb2186b1ed => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:16:32.599849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:32.605059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:32.633465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125170b9-4a33-4c53-86e7-1f4c9fee4bab admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:32.665645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0fa01-9507-482f-aaec-57fb5f46a43e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:32.667340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0fa01-9507-482f-aaec-57fb5f46a43e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:32.667762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0fa01-9507-482f-aaec-57fb5f46a43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:32.671071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:32.735577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-59b0fa01-9507-482f-aaec-57fb5f46a43e None None] MFA Rules not processed for user `fa3184ee6bec45da840eb9182b9a5f4d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:32.741005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-59b0fa01-9507-482f-aaec-57fb5f46a43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:32.752025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf0632-eae2-4935-8f48-4c81570d1ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:32.753538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf0632-eae2-4935-8f48-4c81570d1ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:32.754124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf0632-eae2-4935-8f48-4c81570d1ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:32.754596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf0632-eae2-4935-8f48-4c81570d1ef1 admin admin] PATH_INFO: `/v3/projects/e858ef63ca444c9c934adf01aeb0c797` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:32.759743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf0632-eae2-4935-8f48-4c81570d1ef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e858ef63ca444c9c934adf01aeb0c797)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:32.763482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf0632-eae2-4935-8f48-4c81570d1ef1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:32.839635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a1bf0632-eae2-4935-8f48-4c81570d1ef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e858ef63ca444c9c934adf01aeb0c797', 'request_id': 'req-a1bf0632-eae2-4935-8f48-4c81570d1ef1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:32.852433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a1bf0632-eae2-4935-8f48-4c81570d1ef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e858ef63ca444c9c934adf01aeb0c797', 'request_id': 'req-a1bf0632-eae2-4935-8f48-4c81570d1ef1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:32.874342 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2978/5898] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:16:32.890381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a1bf0632-eae2-4935-8f48-4c81570d1ef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e858ef63ca444c9c934adf01aeb0c797', 'request_id': 'req-a1bf0632-eae2-4935-8f48-4c81570d1ef1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:32.944845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a1bf0632-eae2-4935-8f48-4c81570d1ef1 admin admin] The token cache is being invalidate because project e858ef63ca444c9c934adf01aeb0c797 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:16:32.954385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a1bf0632-eae2-4935-8f48-4c81570d1ef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:32.957521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a1bf0632-eae2-4935-8f48-4c81570d1ef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:16:32.970137 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2921/5899] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:16:32 2022] DELETE /identity/v3/projects/e858ef63ca444c9c934adf01aeb0c797 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:16:32.975607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0161e2-7ba8-4096-a2af-7dfaade94a3f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:32.976036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0161e2-7ba8-4096-a2af-7dfaade94a3f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:32.976496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0161e2-7ba8-4096-a2af-7dfaade94a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:33.013733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1d0161e2-7ba8-4096-a2af-7dfaade94a3f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:33.014952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1d0161e2-7ba8-4096-a2af-7dfaade94a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:33.154943 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2979/5900] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:16:33.164135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395312d-699f-4c1c-9788-b075b086fe08 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:33.164442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395312d-699f-4c1c-9788-b075b086fe08 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:33.164719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395312d-699f-4c1c-9788-b075b086fe08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:33.203219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6395312d-699f-4c1c-9788-b075b086fe08 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:33.204733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6395312d-699f-4c1c-9788-b075b086fe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:33.214178 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2922/5901] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:16:33.218686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0161e2-7ba8-4096-a2af-7dfaade94a3f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:33.231216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0161e2-7ba8-4096-a2af-7dfaade94a3f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:33.235985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bcc98-f123-4d0a-a339-8b923ab1cf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:33.236654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bcc98-f123-4d0a-a339-8b923ab1cf01 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:33.236922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bcc98-f123-4d0a-a339-8b923ab1cf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:33.237185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bcc98-f123-4d0a-a339-8b923ab1cf01 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:33.238316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bcc98-f123-4d0a-a339-8b923ab1cf01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:16:33.238316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bcc98-f123-4d0a-a339-8b923ab1cf01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:33.240452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bcc98-f123-4d0a-a339-8b923ab1cf01 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:33.253701 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2980/5902] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:16:33.258600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395312d-699f-4c1c-9788-b075b086fe08 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:33.273586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395312d-699f-4c1c-9788-b075b086fe08 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:33.282683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a50b4c-8d3b-4b65-b14b-270faf4e55dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:33.283669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a50b4c-8d3b-4b65-b14b-270faf4e55dd admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:33.284090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a50b4c-8d3b-4b65-b14b-270faf4e55dd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:33.284654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a50b4c-8d3b-4b65-b14b-270faf4e55dd admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:33.286257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a50b4c-8d3b-4b65-b14b-270faf4e55dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:33.290654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a50b4c-8d3b-4b65-b14b-270faf4e55dd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:33.347752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bcc98-f123-4d0a-a339-8b923ab1cf01 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:33.363410 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2923/5903] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:16:33 2022] POST /identity/v3/projects => generated 411 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:16:33.366869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bcc98-f123-4d0a-a339-8b923ab1cf01 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:33.488131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c8974-48d7-4912-b157-2a16c3b2a164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:33.488807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c8974-48d7-4912-b157-2a16c3b2a164 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:33.490634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c8974-48d7-4912-b157-2a16c3b2a164 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:33.490926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c8974-48d7-4912-b157-2a16c3b2a164 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:33.491998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c8974-48d7-4912-b157-2a16c3b2a164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:33.494193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c8974-48d7-4912-b157-2a16c3b2a164 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:33.534227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:33.553406 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2981/5904] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:16:33 2022] POST /identity/v3/users => generated 306 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:16:33.555354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a50b4c-8d3b-4b65-b14b-270faf4e55dd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:33.576744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a50b4c-8d3b-4b65-b14b-270faf4e55dd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:33.589366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a41229-0e64-49d6-b6e3-5722a8a4c30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:33.590311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a41229-0e64-49d6-b6e3-5722a8a4c30a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:33.590735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a41229-0e64-49d6-b6e3-5722a8a4c30a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:33.591147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a41229-0e64-49d6-b6e3-5722a8a4c30a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:33.592031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a41229-0e64-49d6-b6e3-5722a8a4c30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:33.596422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a41229-0e64-49d6-b6e3-5722a8a4c30a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:33.614843 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2924/5905] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:16:33.618746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c8974-48d7-4912-b157-2a16c3b2a164 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:33.635736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c8974-48d7-4912-b157-2a16c3b2a164 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:33.642750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98a305-94f7-4a26-a905-d176c6e9968b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:33.644491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98a305-94f7-4a26-a905-d176c6e9968b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:33.645917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98a305-94f7-4a26-a905-d176c6e9968b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:33.646265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98a305-94f7-4a26-a905-d176c6e9968b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:33.647289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98a305-94f7-4a26-a905-d176c6e9968b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:33.649417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98a305-94f7-4a26-a905-d176c6e9968b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:33.671895 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2982/5906] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:16:33.676274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a41229-0e64-49d6-b6e3-5722a8a4c30a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:33.688340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a41229-0e64-49d6-b6e3-5722a8a4c30a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:33.694151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69fe14-0150-4fb1-8f01-b35eaf542f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:33.695649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69fe14-0150-4fb1-8f01-b35eaf542f2d admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:33.696020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69fe14-0150-4fb1-8f01-b35eaf542f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:33.696447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69fe14-0150-4fb1-8f01-b35eaf542f2d admin admin] PATH_INFO: `/v3/projects/2dd8b4e8ad82476da000932b66bbdd4a/users/f0ede2c171834b0bb86c6f702370cf85/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:33.718259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69fe14-0150-4fb1-8f01-b35eaf542f2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd8b4e8ad82476da000932b66bbdd4a, user_id=f0ede2c171834b0bb86c6f702370cf85, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:33.721741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69fe14-0150-4fb1-8f01-b35eaf542f2d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:33.742148 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2925/5907] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:16:33 2022] PUT /identity/v3/projects/2dd8b4e8ad82476da000932b66bbdd4a/users/f0ede2c171834b0bb86c6f702370cf85/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:16:33.747810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01436ff2-cb1a-4f1a-abda-fab8eb0218ba None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:33.748360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01436ff2-cb1a-4f1a-abda-fab8eb0218ba None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:33.748868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01436ff2-cb1a-4f1a-abda-fab8eb0218ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:33.776047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-01436ff2-cb1a-4f1a-abda-fab8eb0218ba None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:33.777738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-01436ff2-cb1a-4f1a-abda-fab8eb0218ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:33.865529 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2983/5908] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:16:33.877275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69fe14-0150-4fb1-8f01-b35eaf542f2d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:33.881218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0f69fe14-0150-4fb1-8f01-b35eaf542f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:34.035092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69fe14-0150-4fb1-8f01-b35eaf542f2d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:34.054069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5dabe6-6048-46fb-bc72-49bde4641047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:34.054715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5dabe6-6048-46fb-bc72-49bde4641047 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:34.055013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5dabe6-6048-46fb-bc72-49bde4641047 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:34.055279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5dabe6-6048-46fb-bc72-49bde4641047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:34.065078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:34.071771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5dabe6-6048-46fb-bc72-49bde4641047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:34.073309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5dabe6-6048-46fb-bc72-49bde4641047 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:34.091922 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2926/5909] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:16:35.311515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01436ff2-cb1a-4f1a-abda-fab8eb0218ba None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:35.313029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-01436ff2-cb1a-4f1a-abda-fab8eb0218ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:35.470977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01436ff2-cb1a-4f1a-abda-fab8eb0218ba None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:35.478195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f66c5-976a-41a8-af12-ed8f009c1e99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:35.479254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f66c5-976a-41a8-af12-ed8f009c1e99 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:35.479886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f66c5-976a-41a8-af12-ed8f009c1e99 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:35.480318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f66c5-976a-41a8-af12-ed8f009c1e99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:35.482597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-879f66c5-976a-41a8-af12-ed8f009c1e99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:35.503489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f66c5-976a-41a8-af12-ed8f009c1e99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:35.505948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f66c5-976a-41a8-af12-ed8f009c1e99 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:35.522988 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2984/5910] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:16:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:16:44.090552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:45.341934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5dabe6-6048-46fb-bc72-49bde4641047 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:45.343644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4a5dabe6-6048-46fb-bc72-49bde4641047 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:45.455153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5dabe6-6048-46fb-bc72-49bde4641047 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:45.462358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88347cb4-7872-46b6-9740-903615db4020 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:45.463060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88347cb4-7872-46b6-9740-903615db4020 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:45.463329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88347cb4-7872-46b6-9740-903615db4020 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:45.464164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88347cb4-7872-46b6-9740-903615db4020 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:45.465403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-88347cb4-7872-46b6-9740-903615db4020 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:45.501266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88347cb4-7872-46b6-9740-903615db4020 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:45.502884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88347cb4-7872-46b6-9740-903615db4020 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:45.513419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=65 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:16:45.531069 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2927/5911] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:16:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:16:49.358941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8e8a0-72f2-4005-8a45-00059ae1ca05 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:49.359540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8e8a0-72f2-4005-8a45-00059ae1ca05 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:49.359747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8e8a0-72f2-4005-8a45-00059ae1ca05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:49.401094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e4f8e8a0-72f2-4005-8a45-00059ae1ca05 None None] MFA Rules not processed for user `f0ede2c171834b0bb86c6f702370cf85`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:49.403540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e4f8e8a0-72f2-4005-8a45-00059ae1ca05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:49.517145 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2985/5912] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:49 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:16:49.517473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053323e-c9fb-4925-810a-06d7bdff485e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:49.517979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053323e-c9fb-4925-810a-06d7bdff485e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:49.518064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053323e-c9fb-4925-810a-06d7bdff485e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:49.602909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f053323e-c9fb-4925-810a-06d7bdff485e None None] MFA Rules not processed for user `f0ede2c171834b0bb86c6f702370cf85`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:49.605833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f053323e-c9fb-4925-810a-06d7bdff485e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:49.627341 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2928/5913] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:49 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:16:49.641669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8e8a0-72f2-4005-8a45-00059ae1ca05 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:16:49.668173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8e8a0-72f2-4005-8a45-00059ae1ca05 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:16:49.676464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e411e8c-5bea-4c66-b72e-323e95aa602d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:16:49.677384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e411e8c-5bea-4c66-b72e-323e95aa602d service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:49.677856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e411e8c-5bea-4c66-b72e-323e95aa602d service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:49.678993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e411e8c-5bea-4c66-b72e-323e95aa602d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:49.690166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e411e8c-5bea-4c66-b72e-323e95aa602d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:16:49.692894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e411e8c-5bea-4c66-b72e-323e95aa602d service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:16:49.706374 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2986/5914] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:16:49 2022] GET /identity/v3/auth/tokens => generated 3288 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:16:55.819851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b796c9b-789e-476d-9b89-d9f2ecb7d1d0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:16:55.820478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b796c9b-789e-476d-9b89-d9f2ecb7d1d0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:16:55.820587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b796c9b-789e-476d-9b89-d9f2ecb7d1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:16:55.861642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0b796c9b-789e-476d-9b89-d9f2ecb7d1d0 None None] MFA Rules not processed for user `9671d01ad0d44cc99e0eecc966f7c3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:16:55.863593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0b796c9b-789e-476d-9b89-d9f2ecb7d1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:16:55.999107 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2929/5915] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:16:59.700771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=38 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:05.970316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=62 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:16.313453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e411e8c-5bea-4c66-b72e-323e95aa602d service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:16.323684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0e411e8c-5bea-4c66-b72e-323e95aa602d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:16.359900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e411e8c-5bea-4c66-b72e-323e95aa602d service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:16.372902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e056270-5098-475f-8df9-43d6074efb64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:16.373532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e056270-5098-475f-8df9-43d6074efb64 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:16.376416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e056270-5098-475f-8df9-43d6074efb64 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:16.376867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e056270-5098-475f-8df9-43d6074efb64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:16.382023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4e056270-5098-475f-8df9-43d6074efb64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:16.412195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e056270-5098-475f-8df9-43d6074efb64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:16.417379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e056270-5098-475f-8df9-43d6074efb64 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:16.456108 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2987/5916] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:17:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:17:17.143246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b796c9b-789e-476d-9b89-d9f2ecb7d1d0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:17.168331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b796c9b-789e-476d-9b89-d9f2ecb7d1d0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:17.174808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd0c24-f4f5-498c-a45b-f3c2e167faba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:17.175717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd0c24-f4f5-498c-a45b-f3c2e167faba service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:17.176144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd0c24-f4f5-498c-a45b-f3c2e167faba service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:17.176660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd0c24-f4f5-498c-a45b-f3c2e167faba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:17.179462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9afd0c24-f4f5-498c-a45b-f3c2e167faba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:17.204353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd0c24-f4f5-498c-a45b-f3c2e167faba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:17.206489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd0c24-f4f5-498c-a45b-f3c2e167faba service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:17.225364 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2930/5917] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:17:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:17:26.449971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:27.214828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:31.038433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e056270-5098-475f-8df9-43d6074efb64 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:31.039981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4e056270-5098-475f-8df9-43d6074efb64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:31.063621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e056270-5098-475f-8df9-43d6074efb64 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:31.075277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:31.076489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:31.076677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:31.077360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] PATH_INFO: `/v3/users/9a3e30cc83da46c9896b653ffe5bca5c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:31.079422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a3e30cc83da46c9896b653ffe5bca5c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:31.083364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:31.190115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a3e30cc83da46c9896b653ffe5bca5c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:31.206403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a3e30cc83da46c9896b653ffe5bca5c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:31.219594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a3e30cc83da46c9896b653ffe5bca5c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:31.258493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a3e30cc83da46c9896b653ffe5bca5c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:31.258493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a3e30cc83da46c9896b653ffe5bca5c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:31.265950 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2988/5918] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:17:31 2022] DELETE /identity/v3/users/9a3e30cc83da46c9896b653ffe5bca5c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:31.271293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6d262-f303-43bb-8877-15b47d9abbcb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:31.272176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6d262-f303-43bb-8877-15b47d9abbcb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:31.272661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6d262-f303-43bb-8877-15b47d9abbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:31.297897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c6b6d262-f303-43bb-8877-15b47d9abbcb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:31.299540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c6b6d262-f303-43bb-8877-15b47d9abbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:31.406853 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2931/5919] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:31.422317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:31.424617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:31.535246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de54e36-94ee-417a-8c28-a455bbb815fe admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:31.539528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907dd14e-7e9d-47d2-a61b-6414a8c8cd8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:31.540667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dd14e-7e9d-47d2-a61b-6414a8c8cd8b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:31.540937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dd14e-7e9d-47d2-a61b-6414a8c8cd8b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:31.541209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dd14e-7e9d-47d2-a61b-6414a8c8cd8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:31.552864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907dd14e-7e9d-47d2-a61b-6414a8c8cd8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:31.554002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907dd14e-7e9d-47d2-a61b-6414a8c8cd8b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:31.568725 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2989/5920] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:17:32.444611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6d262-f303-43bb-8877-15b47d9abbcb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:32.446219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c6b6d262-f303-43bb-8877-15b47d9abbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:32.466515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6d262-f303-43bb-8877-15b47d9abbcb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:32.481057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:32.481709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:32.481979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:32.482242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab admin admin] PATH_INFO: `/v3/projects/0a4120a39edb45028aec4cd8f6d5967c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:32.489907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a4120a39edb45028aec4cd8f6d5967c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:32.496825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:32.566265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a4120a39edb45028aec4cd8f6d5967c', 'request_id': 'req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:32.576987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a4120a39edb45028aec4cd8f6d5967c', 'request_id': 'req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:32.592307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a4120a39edb45028aec4cd8f6d5967c', 'request_id': 'req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:32.612923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab admin admin] The token cache is being invalidate because project 0a4120a39edb45028aec4cd8f6d5967c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:17:32.613270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:32.613888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2633eed7-b6e9-4b94-bf60-5f27dd37f8ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:32.619730 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2932/5921] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:17:32 2022] DELETE /identity/v3/projects/0a4120a39edb45028aec4cd8f6d5967c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:32.621963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3998dbd-d269-4525-95f3-1b9135ddc731 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:32.622448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3998dbd-d269-4525-95f3-1b9135ddc731 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:32.622536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3998dbd-d269-4525-95f3-1b9135ddc731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:32.646321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b3998dbd-d269-4525-95f3-1b9135ddc731 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:32.648738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b3998dbd-d269-4525-95f3-1b9135ddc731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:32.754852 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2990/5922] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:32.756429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f5524-0abd-4f98-ad27-b13736946150 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:32.756797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f5524-0abd-4f98-ad27-b13736946150 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:32.757140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f5524-0abd-4f98-ad27-b13736946150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:32.778525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d40f5524-0abd-4f98-ad27-b13736946150 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:32.780022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d40f5524-0abd-4f98-ad27-b13736946150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:32.789175 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2933/5923] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:17:32.793518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3998dbd-d269-4525-95f3-1b9135ddc731 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:32.804170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3998dbd-d269-4525-95f3-1b9135ddc731 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:32.814793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebffdd-4d5d-4ecb-a998-3e380ae73861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:32.815743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebffdd-4d5d-4ecb-a998-3e380ae73861 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:32.816009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebffdd-4d5d-4ecb-a998-3e380ae73861 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:32.816267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebffdd-4d5d-4ecb-a998-3e380ae73861 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:32.816968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebffdd-4d5d-4ecb-a998-3e380ae73861 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:17:32.817301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebffdd-4d5d-4ecb-a998-3e380ae73861 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:32.819583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebffdd-4d5d-4ecb-a998-3e380ae73861 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:32.843115 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2991/5924] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:17:32.847714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f5524-0abd-4f98-ad27-b13736946150 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:32.859887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f5524-0abd-4f98-ad27-b13736946150 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:32.865215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088ca95-f661-4222-a2f5-4cc82375b0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:32.865828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088ca95-f661-4222-a2f5-4cc82375b0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:32.866998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088ca95-f661-4222-a2f5-4cc82375b0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:32.867272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088ca95-f661-4222-a2f5-4cc82375b0a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:32.868249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088ca95-f661-4222-a2f5-4cc82375b0a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:32.870222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088ca95-f661-4222-a2f5-4cc82375b0a1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:32.903730 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2934/5925] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:17:32 2022] POST /identity/v3/projects => generated 359 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:17:32.908719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebffdd-4d5d-4ecb-a998-3e380ae73861 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:32.921392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebffdd-4d5d-4ecb-a998-3e380ae73861 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:32.922634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:32.997848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba2e5e-0378-475d-b635-9628f3380317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:32.998608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba2e5e-0378-475d-b635-9628f3380317 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:32.998943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba2e5e-0378-475d-b635-9628f3380317 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:32.999274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba2e5e-0378-475d-b635-9628f3380317 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:32.999981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba2e5e-0378-475d-b635-9628f3380317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:33.002432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba2e5e-0378-475d-b635-9628f3380317 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:33.025802 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2992/5926] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:17:32 2022] POST /identity/v3/users => generated 280 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:17:33.029948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088ca95-f661-4222-a2f5-4cc82375b0a1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:33.041423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088ca95-f661-4222-a2f5-4cc82375b0a1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:33.048364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2bfd7-244d-47c4-87b1-bdf734ce96ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:33.049609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2bfd7-244d-47c4-87b1-bdf734ce96ca admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:33.049889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2bfd7-244d-47c4-87b1-bdf734ce96ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:33.050160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2bfd7-244d-47c4-87b1-bdf734ce96ca admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:33.050804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2bfd7-244d-47c4-87b1-bdf734ce96ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:33.053228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2bfd7-244d-47c4-87b1-bdf734ce96ca admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:33.066162 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2935/5927] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:17:33.070898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba2e5e-0378-475d-b635-9628f3380317 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:33.086006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba2e5e-0378-475d-b635-9628f3380317 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:33.092097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce54ee-ef69-4d5e-a8ec-9f2315e2f068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:33.094014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce54ee-ef69-4d5e-a8ec-9f2315e2f068 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:33.094381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce54ee-ef69-4d5e-a8ec-9f2315e2f068 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:33.094685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce54ee-ef69-4d5e-a8ec-9f2315e2f068 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:33.095400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce54ee-ef69-4d5e-a8ec-9f2315e2f068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:33.098235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce54ee-ef69-4d5e-a8ec-9f2315e2f068 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:33.110949 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2993/5928] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:17:33.115374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2bfd7-244d-47c4-87b1-bdf734ce96ca admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:33.126323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2bfd7-244d-47c4-87b1-bdf734ce96ca admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:33.131869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec384c5-0101-41c2-bbab-33ecaa158a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:33.132586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec384c5-0101-41c2-bbab-33ecaa158a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:33.132864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec384c5-0101-41c2-bbab-33ecaa158a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:33.133138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec384c5-0101-41c2-bbab-33ecaa158a65 admin admin] PATH_INFO: `/v3/projects/48ff6e1839864f38bc527c49bdc106ad/users/a07d9d869d314a1791cba121b2b8998d/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:33.150474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec384c5-0101-41c2-bbab-33ecaa158a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48ff6e1839864f38bc527c49bdc106ad, user_id=a07d9d869d314a1791cba121b2b8998d, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:33.153301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec384c5-0101-41c2-bbab-33ecaa158a65 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:33.170791 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2936/5929] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:17:33 2022] PUT /identity/v3/projects/48ff6e1839864f38bc527c49bdc106ad/users/a07d9d869d314a1791cba121b2b8998d/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:33.181761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c7ffd-7cb5-4eb5-9700-652692e16b7e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:33.182283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c7ffd-7cb5-4eb5-9700-652692e16b7e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:33.182583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c7ffd-7cb5-4eb5-9700-652692e16b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:33.213304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8e5c7ffd-7cb5-4eb5-9700-652692e16b7e None None] MFA Rules not processed for user `a07d9d869d314a1791cba121b2b8998d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:33.215576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8e5c7ffd-7cb5-4eb5-9700-652692e16b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:33.294161 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2994/5930] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:17:33.306506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec384c5-0101-41c2-bbab-33ecaa158a65 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:33.308608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3ec384c5-0101-41c2-bbab-33ecaa158a65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:33.334672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:33.380141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec384c5-0101-41c2-bbab-33ecaa158a65 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:33.386606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4dfa8-f5e7-44a3-92a0-7b381cca20cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:33.387353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4dfa8-f5e7-44a3-92a0-7b381cca20cb service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:33.387621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4dfa8-f5e7-44a3-92a0-7b381cca20cb service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:33.387876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4dfa8-f5e7-44a3-92a0-7b381cca20cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:33.396833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4dfa8-f5e7-44a3-92a0-7b381cca20cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:33.398282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4dfa8-f5e7-44a3-92a0-7b381cca20cb service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:33.408643 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2937/5931] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:17:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:17:34.722928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c7ffd-7cb5-4eb5-9700-652692e16b7e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:34.729608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8e5c7ffd-7cb5-4eb5-9700-652692e16b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:34.897611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c7ffd-7cb5-4eb5-9700-652692e16b7e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:34.909204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358113cf-389a-4b36-8b94-64a447ad114f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:34.909849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358113cf-389a-4b36-8b94-64a447ad114f service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:34.910116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358113cf-389a-4b36-8b94-64a447ad114f service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:34.910384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358113cf-389a-4b36-8b94-64a447ad114f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:34.914211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-358113cf-389a-4b36-8b94-64a447ad114f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:34.939376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:34.951978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358113cf-389a-4b36-8b94-64a447ad114f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:34.953985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358113cf-389a-4b36-8b94-64a447ad114f service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:34.994788 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2995/5932] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:17:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:17:42.896976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4dfa8-f5e7-44a3-92a0-7b381cca20cb service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:42.899401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8de4dfa8-f5e7-44a3-92a0-7b381cca20cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:43.000865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4dfa8-f5e7-44a3-92a0-7b381cca20cb service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:43.005847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f3191-1500-4507-9a21-d4fbf2951103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:43.006718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f3191-1500-4507-9a21-d4fbf2951103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:43.007161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f3191-1500-4507-9a21-d4fbf2951103 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:43.007588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f3191-1500-4507-9a21-d4fbf2951103 admin admin] PATH_INFO: `/v3/users/f0ede2c171834b0bb86c6f702370cf85` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:43.009357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f3191-1500-4507-9a21-d4fbf2951103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0ede2c171834b0bb86c6f702370cf85)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:43.011526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f3191-1500-4507-9a21-d4fbf2951103 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:43.078806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ac2f3191-1500-4507-9a21-d4fbf2951103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0ede2c171834b0bb86c6f702370cf85'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:43.089522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ac2f3191-1500-4507-9a21-d4fbf2951103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0ede2c171834b0bb86c6f702370cf85'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:43.105696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ac2f3191-1500-4507-9a21-d4fbf2951103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0ede2c171834b0bb86c6f702370cf85'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:43.106736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ac2f3191-1500-4507-9a21-d4fbf2951103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0ede2c171834b0bb86c6f702370cf85'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:43.107554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ac2f3191-1500-4507-9a21-d4fbf2951103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0ede2c171834b0bb86c6f702370cf85'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:43.126531 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2938/5933] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:17:42 2022] DELETE /identity/v3/users/f0ede2c171834b0bb86c6f702370cf85 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:43.644244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358113cf-389a-4b36-8b94-64a447ad114f service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:43.656609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358113cf-389a-4b36-8b94-64a447ad114f service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:43.782069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c2242-eb60-4792-a968-5b8b740ea96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:43.782792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c2242-eb60-4792-a968-5b8b740ea96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:43.783067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c2242-eb60-4792-a968-5b8b740ea96d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:43.783352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c2242-eb60-4792-a968-5b8b740ea96d admin admin] PATH_INFO: `/v3/projects/2dd8b4e8ad82476da000932b66bbdd4a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:43.784611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c2242-eb60-4792-a968-5b8b740ea96d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd8b4e8ad82476da000932b66bbdd4a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:43.786699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c2242-eb60-4792-a968-5b8b740ea96d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:43.893049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d84c2242-eb60-4792-a968-5b8b740ea96d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd8b4e8ad82476da000932b66bbdd4a', 'request_id': 'req-d84c2242-eb60-4792-a968-5b8b740ea96d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:43.905753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d84c2242-eb60-4792-a968-5b8b740ea96d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd8b4e8ad82476da000932b66bbdd4a', 'request_id': 'req-d84c2242-eb60-4792-a968-5b8b740ea96d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:43.919486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d84c2242-eb60-4792-a968-5b8b740ea96d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd8b4e8ad82476da000932b66bbdd4a', 'request_id': 'req-d84c2242-eb60-4792-a968-5b8b740ea96d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:43.941155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d84c2242-eb60-4792-a968-5b8b740ea96d admin admin] The token cache is being invalidate because project 2dd8b4e8ad82476da000932b66bbdd4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:17:43.941893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d84c2242-eb60-4792-a968-5b8b740ea96d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:43.942440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d84c2242-eb60-4792-a968-5b8b740ea96d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:43.946791 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2996/5934] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:17:43 2022] DELETE /identity/v3/projects/2dd8b4e8ad82476da000932b66bbdd4a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:43.950701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4472b-e1f1-45c8-90a0-84ace82e9c02 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:43.951203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4472b-e1f1-45c8-90a0-84ace82e9c02 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:43.951203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4472b-e1f1-45c8-90a0-84ace82e9c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:43.979352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-26e4472b-e1f1-45c8-90a0-84ace82e9c02 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:43.980651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-26e4472b-e1f1-45c8-90a0-84ace82e9c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:44.038159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:44.080819 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2939/5935] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:44.086470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50aa3a-fed3-4d3c-9b41-ead2d0e663c0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:44.088044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50aa3a-fed3-4d3c-9b41-ead2d0e663c0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:44.088588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50aa3a-fed3-4d3c-9b41-ead2d0e663c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:44.114940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-eb50aa3a-fed3-4d3c-9b41-ead2d0e663c0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:44.116515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-eb50aa3a-fed3-4d3c-9b41-ead2d0e663c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:44.132970 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2997/5936] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:44.137001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4472b-e1f1-45c8-90a0-84ace82e9c02 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:44.148008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4472b-e1f1-45c8-90a0-84ace82e9c02 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:44.154627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788a47d-b9d4-4613-9bff-f240aec1720a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:44.155433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788a47d-b9d4-4613-9bff-f240aec1720a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:44.155858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788a47d-b9d4-4613-9bff-f240aec1720a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:44.156288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788a47d-b9d4-4613-9bff-f240aec1720a admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:44.157127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788a47d-b9d4-4613-9bff-f240aec1720a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:17:44.157590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788a47d-b9d4-4613-9bff-f240aec1720a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:44.159471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788a47d-b9d4-4613-9bff-f240aec1720a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:44.174036 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2940/5937] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:17:44.178113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50aa3a-fed3-4d3c-9b41-ead2d0e663c0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:44.191790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50aa3a-fed3-4d3c-9b41-ead2d0e663c0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:44.199634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b02469-9278-4207-8a71-e7a949bbe7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:44.200435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b02469-9278-4207-8a71-e7a949bbe7c9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:44.200708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b02469-9278-4207-8a71-e7a949bbe7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:44.201011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b02469-9278-4207-8a71-e7a949bbe7c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:44.201792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b02469-9278-4207-8a71-e7a949bbe7c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:44.203946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b02469-9278-4207-8a71-e7a949bbe7c9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:44.244484 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2998/5938] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:17:44 2022] POST /identity/v3/projects => generated 363 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:17:44.246296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788a47d-b9d4-4613-9bff-f240aec1720a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:44.259245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788a47d-b9d4-4613-9bff-f240aec1720a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:44.334996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b96442-774c-48d4-b707-02e8b5f3f895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:44.337503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b96442-774c-48d4-b707-02e8b5f3f895 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:44.338141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b96442-774c-48d4-b707-02e8b5f3f895 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:44.339171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b96442-774c-48d4-b707-02e8b5f3f895 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:44.340303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b96442-774c-48d4-b707-02e8b5f3f895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:44.344482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b96442-774c-48d4-b707-02e8b5f3f895 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:44.384830 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2941/5939] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:17:44 2022] POST /identity/v3/users => generated 282 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:17:44.389144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b02469-9278-4207-8a71-e7a949bbe7c9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:44.409035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b02469-9278-4207-8a71-e7a949bbe7c9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:44.416601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd404871-d2a1-4b07-8152-3b8d015fa21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:44.417310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd404871-d2a1-4b07-8152-3b8d015fa21f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:44.417584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd404871-d2a1-4b07-8152-3b8d015fa21f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:44.417850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd404871-d2a1-4b07-8152-3b8d015fa21f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:44.418494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd404871-d2a1-4b07-8152-3b8d015fa21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:44.420400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd404871-d2a1-4b07-8152-3b8d015fa21f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:44.435474 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 2999/5940] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:17:44.443506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b96442-774c-48d4-b707-02e8b5f3f895 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:44.464115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b96442-774c-48d4-b707-02e8b5f3f895 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:44.470145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b1694-f25e-4a8f-8212-cc846ce14575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:44.470918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b1694-f25e-4a8f-8212-cc846ce14575 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:44.471196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b1694-f25e-4a8f-8212-cc846ce14575 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:44.471460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b1694-f25e-4a8f-8212-cc846ce14575 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:44.472105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b1694-f25e-4a8f-8212-cc846ce14575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:44.473791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b1694-f25e-4a8f-8212-cc846ce14575 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:44.491176 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2942/5941] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:17:44.495631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd404871-d2a1-4b07-8152-3b8d015fa21f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:44.511094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd404871-d2a1-4b07-8152-3b8d015fa21f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:44.520657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00726901-8ba6-4d5f-b437-93b21e5209a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:44.521510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00726901-8ba6-4d5f-b437-93b21e5209a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:44.521766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00726901-8ba6-4d5f-b437-93b21e5209a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:44.522033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00726901-8ba6-4d5f-b437-93b21e5209a9 admin admin] PATH_INFO: `/v3/projects/13a3c875257e4db7b3fb2560c9bb6392/users/f49e7f7982ad4d98b48519f7c1b43fdb/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:44.555964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00726901-8ba6-4d5f-b437-93b21e5209a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a3c875257e4db7b3fb2560c9bb6392, user_id=f49e7f7982ad4d98b48519f7c1b43fdb, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:44.560341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00726901-8ba6-4d5f-b437-93b21e5209a9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:44.590337 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3000/5942] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:17:44 2022] PUT /identity/v3/projects/13a3c875257e4db7b3fb2560c9bb6392/users/f49e7f7982ad4d98b48519f7c1b43fdb/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:44.595371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c541-beb1-4d10-a66d-705f0f715fcd None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:44.595652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c541-beb1-4d10-a66d-705f0f715fcd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:44.595971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c541-beb1-4d10-a66d-705f0f715fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:44.636556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8d00c541-beb1-4d10-a66d-705f0f715fcd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:44.637821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8d00c541-beb1-4d10-a66d-705f0f715fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:44.786423 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2943/5943] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:44.814078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00726901-8ba6-4d5f-b437-93b21e5209a9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:44.816111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-00726901-8ba6-4d5f-b437-93b21e5209a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:44.867354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:44.960655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00726901-8ba6-4d5f-b437-93b21e5209a9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:44.968250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1749a-1571-46c6-a242-a72ac23e01d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:44.969132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1749a-1571-46c6-a242-a72ac23e01d7 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:44.969548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1749a-1571-46c6-a242-a72ac23e01d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:44.969971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1749a-1571-46c6-a242-a72ac23e01d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:44.986867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1749a-1571-46c6-a242-a72ac23e01d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:44.988521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1749a-1571-46c6-a242-a72ac23e01d7 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:45.001420 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3001/5944] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:17:46.267391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00c541-beb1-4d10-a66d-705f0f715fcd None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:46.268865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8d00c541-beb1-4d10-a66d-705f0f715fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:46.365957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:46.396542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00c541-beb1-4d10-a66d-705f0f715fcd None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:46.403491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcac6f-9b81-48e4-b7fb-31e7eef93c21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:46.404123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcac6f-9b81-48e4-b7fb-31e7eef93c21 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:46.404419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcac6f-9b81-48e4-b7fb-31e7eef93c21 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:46.405054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcac6f-9b81-48e4-b7fb-31e7eef93c21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:46.408463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-51fcac6f-9b81-48e4-b7fb-31e7eef93c21 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:46.441584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcac6f-9b81-48e4-b7fb-31e7eef93c21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:46.443107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcac6f-9b81-48e4-b7fb-31e7eef93c21 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:46.460216 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2944/5945] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:17:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:17:46.914231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1749a-1571-46c6-a242-a72ac23e01d7 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:46.916972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-09d1749a-1571-46c6-a242-a72ac23e01d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:46.934779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1749a-1571-46c6-a242-a72ac23e01d7 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:46.940497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:46.941201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:46.941496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:46.942072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] PATH_INFO: `/v3/users/a07d9d869d314a1791cba121b2b8998d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:46.943813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07d9d869d314a1791cba121b2b8998d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:46.946636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:47.038688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07d9d869d314a1791cba121b2b8998d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:47.057374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07d9d869d314a1791cba121b2b8998d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:47.070687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07d9d869d314a1791cba121b2b8998d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:47.094318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07d9d869d314a1791cba121b2b8998d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:47.095123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07d9d869d314a1791cba121b2b8998d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:47.104423 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3002/5946] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:17:46 2022] DELETE /identity/v3/users/a07d9d869d314a1791cba121b2b8998d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:47.108655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaf533-bce1-4c17-a0fb-97b8af72d1ee None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:47.109295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaf533-bce1-4c17-a0fb-97b8af72d1ee None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:47.109740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaf533-bce1-4c17-a0fb-97b8af72d1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:47.166005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-feeaf533-bce1-4c17-a0fb-97b8af72d1ee None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:47.168289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-feeaf533-bce1-4c17-a0fb-97b8af72d1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:47.303936 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2945/5947] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:47.320110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:47.322142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:47.463029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b61ef8-83ca-414d-9f4e-f067db25b0f8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:47.477694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e77366-23bc-46c7-ab55-48ad22c4f168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:47.478570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e77366-23bc-46c7-ab55-48ad22c4f168 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:47.478903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e77366-23bc-46c7-ab55-48ad22c4f168 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:47.483134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e77366-23bc-46c7-ab55-48ad22c4f168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:47.513525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e77366-23bc-46c7-ab55-48ad22c4f168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:47.516481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e77366-23bc-46c7-ab55-48ad22c4f168 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:47.573870 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3003/5948] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:17:48.418204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaf533-bce1-4c17-a0fb-97b8af72d1ee None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:48.422319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-feeaf533-bce1-4c17-a0fb-97b8af72d1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:48.451626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaf533-bce1-4c17-a0fb-97b8af72d1ee None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:48.459209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:48.476169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:48.489077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:48.489853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470 admin admin] PATH_INFO: `/v3/projects/48ff6e1839864f38bc527c49bdc106ad` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:48.495728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48ff6e1839864f38bc527c49bdc106ad)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:48.499174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:48.643984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48ff6e1839864f38bc527c49bdc106ad', 'request_id': 'req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:48.655482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48ff6e1839864f38bc527c49bdc106ad', 'request_id': 'req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:48.675295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48ff6e1839864f38bc527c49bdc106ad', 'request_id': 'req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:48.716824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470 admin admin] The token cache is being invalidate because project 48ff6e1839864f38bc527c49bdc106ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:17:48.716824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:48.716824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64ce6b8e-6fef-4e5d-a8e7-5bf099952470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:48.717975 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2946/5949] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:17:48 2022] DELETE /identity/v3/projects/48ff6e1839864f38bc527c49bdc106ad => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:48.720440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c471e2c-a7c2-49d4-920e-b3693f5b6c3d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:48.720440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c471e2c-a7c2-49d4-920e-b3693f5b6c3d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:48.720440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c471e2c-a7c2-49d4-920e-b3693f5b6c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:48.755360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7c471e2c-a7c2-49d4-920e-b3693f5b6c3d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:48.756645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c471e2c-a7c2-49d4-920e-b3693f5b6c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:48.804473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:48.845115 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3004/5950] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:48.854657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef8ab8-b6e7-4a1f-953f-d152bca866c5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:48.855381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef8ab8-b6e7-4a1f-953f-d152bca866c5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:48.855862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef8ab8-b6e7-4a1f-953f-d152bca866c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:48.886392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-70ef8ab8-b6e7-4a1f-953f-d152bca866c5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:48.887734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-70ef8ab8-b6e7-4a1f-953f-d152bca866c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:48.900478 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2947/5951] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:48.901837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c471e2c-a7c2-49d4-920e-b3693f5b6c3d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:48.916146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c471e2c-a7c2-49d4-920e-b3693f5b6c3d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:48.922284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6dfb2-64d3-4533-b331-01fc65f46f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:48.924044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6dfb2-64d3-4533-b331-01fc65f46f73 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:48.924618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6dfb2-64d3-4533-b331-01fc65f46f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:48.925005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6dfb2-64d3-4533-b331-01fc65f46f73 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:48.925790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6dfb2-64d3-4533-b331-01fc65f46f73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:17:48.926348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6dfb2-64d3-4533-b331-01fc65f46f73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:48.928520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6dfb2-64d3-4533-b331-01fc65f46f73 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:48.947077 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3005/5952] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:17:48.952423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef8ab8-b6e7-4a1f-953f-d152bca866c5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:48.964563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef8ab8-b6e7-4a1f-953f-d152bca866c5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:48.969563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0072e-f791-4769-812d-7cb666d147a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:48.970279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0072e-f791-4769-812d-7cb666d147a2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:48.970607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0072e-f791-4769-812d-7cb666d147a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:48.972543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0072e-f791-4769-812d-7cb666d147a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:48.973682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0072e-f791-4769-812d-7cb666d147a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:48.975811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0072e-f791-4769-812d-7cb666d147a2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:49.026618 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2948/5953] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:17:48 2022] POST /identity/v3/projects => generated 367 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:17:49.028075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6dfb2-64d3-4533-b331-01fc65f46f73 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:49.042581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6dfb2-64d3-4533-b331-01fc65f46f73 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:49.115949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462b07f-28c3-485a-b603-95489f95a894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:49.116728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462b07f-28c3-485a-b603-95489f95a894 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:49.117084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462b07f-28c3-485a-b603-95489f95a894 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:49.117665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462b07f-28c3-485a-b603-95489f95a894 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:49.118271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462b07f-28c3-485a-b603-95489f95a894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:49.120411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462b07f-28c3-485a-b603-95489f95a894 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:49.151672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0072e-f791-4769-812d-7cb666d147a2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:49.152610 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3006/5954] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:17:49 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:17:49.163480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0072e-f791-4769-812d-7cb666d147a2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:49.169710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf304fc8-d292-4ee0-9d2b-0af35da2fd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:49.170336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf304fc8-d292-4ee0-9d2b-0af35da2fd10 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:49.170617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf304fc8-d292-4ee0-9d2b-0af35da2fd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:49.170878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf304fc8-d292-4ee0-9d2b-0af35da2fd10 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:49.171525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf304fc8-d292-4ee0-9d2b-0af35da2fd10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:49.173293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf304fc8-d292-4ee0-9d2b-0af35da2fd10 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:49.192188 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2949/5955] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:17:49.198430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462b07f-28c3-485a-b603-95489f95a894 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:49.215879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462b07f-28c3-485a-b603-95489f95a894 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:49.221507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564c6b8-2122-4a56-a1d7-3483db450915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:49.222465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564c6b8-2122-4a56-a1d7-3483db450915 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:49.229354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564c6b8-2122-4a56-a1d7-3483db450915 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:49.229354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564c6b8-2122-4a56-a1d7-3483db450915 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:49.229354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564c6b8-2122-4a56-a1d7-3483db450915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:49.229354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564c6b8-2122-4a56-a1d7-3483db450915 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:49.243757 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3007/5956] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:17:49.248751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf304fc8-d292-4ee0-9d2b-0af35da2fd10 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:49.264147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf304fc8-d292-4ee0-9d2b-0af35da2fd10 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:49.274647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f3bce-8816-44f6-a126-89258515686a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:49.276048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f3bce-8816-44f6-a126-89258515686a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:49.277051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f3bce-8816-44f6-a126-89258515686a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:49.277051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f3bce-8816-44f6-a126-89258515686a admin admin] PATH_INFO: `/v3/projects/cbb23f70213241e0b263b1a1a99f734b/users/19d2e1626a3642dca393315471be9ee4/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:49.298094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f3bce-8816-44f6-a126-89258515686a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb23f70213241e0b263b1a1a99f734b, user_id=19d2e1626a3642dca393315471be9ee4, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:49.301857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f3bce-8816-44f6-a126-89258515686a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:49.313858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:49.322797 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2950/5957] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:17:49 2022] PUT /identity/v3/projects/cbb23f70213241e0b263b1a1a99f734b/users/19d2e1626a3642dca393315471be9ee4/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:17:49.335662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d0079-9d07-4458-af34-67e39e4c3ad1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:49.336654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d0079-9d07-4458-af34-67e39e4c3ad1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:49.337038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d0079-9d07-4458-af34-67e39e4c3ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:49.374044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-162d0079-9d07-4458-af34-67e39e4c3ad1 None None] MFA Rules not processed for user `19d2e1626a3642dca393315471be9ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:49.375529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-162d0079-9d07-4458-af34-67e39e4c3ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:49.570375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f3bce-8816-44f6-a126-89258515686a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:49.571328 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3008/5958] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:49.745888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f3bce-8816-44f6-a126-89258515686a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:49.767068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28838449-4ccb-424a-acdc-e79a4a2907da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:49.768283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28838449-4ccb-424a-acdc-e79a4a2907da admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:49.768600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28838449-4ccb-424a-acdc-e79a4a2907da admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:49.768871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28838449-4ccb-424a-acdc-e79a4a2907da admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:49.769515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28838449-4ccb-424a-acdc-e79a4a2907da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:49.772481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28838449-4ccb-424a-acdc-e79a4a2907da admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:49.851396 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2951/5959] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:17:49 2022] POST /identity/v3/projects => generated 369 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:17:49.853077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d0079-9d07-4458-af34-67e39e4c3ad1 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:49.873736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d0079-9d07-4458-af34-67e39e4c3ad1 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:49.938482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:49.989276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e8a5c-b42a-4a14-aad4-ce01b83692b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:49.989920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e8a5c-b42a-4a14-aad4-ce01b83692b4 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:49.990595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e8a5c-b42a-4a14-aad4-ce01b83692b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:49.990595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e8a5c-b42a-4a14-aad4-ce01b83692b4 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:49.991845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e8a5c-b42a-4a14-aad4-ce01b83692b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:49.993158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e8a5c-b42a-4a14-aad4-ce01b83692b4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:50.039922 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3009/5960] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:17:49 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:17:50.045254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28838449-4ccb-424a-acdc-e79a4a2907da admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:50.062419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28838449-4ccb-424a-acdc-e79a4a2907da admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:50.071915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9b14e-0d25-4e8a-8679-442d1ada1305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:50.072567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9b14e-0d25-4e8a-8679-442d1ada1305 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:50.072844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9b14e-0d25-4e8a-8679-442d1ada1305 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:50.073107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9b14e-0d25-4e8a-8679-442d1ada1305 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:50.074243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9b14e-0d25-4e8a-8679-442d1ada1305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:50.076235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9b14e-0d25-4e8a-8679-442d1ada1305 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:50.098332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e8a5c-b42a-4a14-aad4-ce01b83692b4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:50.104942 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2952/5961] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:17:50.108951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e8a5c-b42a-4a14-aad4-ce01b83692b4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:50.116584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de9e45-a667-4214-a124-0ba3d7bd5af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:50.117216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de9e45-a667-4214-a124-0ba3d7bd5af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:50.117554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de9e45-a667-4214-a124-0ba3d7bd5af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:50.118102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de9e45-a667-4214-a124-0ba3d7bd5af8 admin admin] PATH_INFO: `/v3/projects/71e20a76e7454caa967768a84a07eccf/users/83733e301dd5432495e09c324234c1d3/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:50.134486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de9e45-a667-4214-a124-0ba3d7bd5af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e20a76e7454caa967768a84a07eccf, user_id=83733e301dd5432495e09c324234c1d3, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:50.137301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de9e45-a667-4214-a124-0ba3d7bd5af8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:50.164719 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3010/5962] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:17:50 2022] PUT /identity/v3/projects/71e20a76e7454caa967768a84a07eccf/users/83733e301dd5432495e09c324234c1d3/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:50.169181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9b14e-0d25-4e8a-8679-442d1ada1305 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:50.189967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9b14e-0d25-4e8a-8679-442d1ada1305 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:50.345152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160cc6a-3196-4840-a8cf-4b35198467c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:50.345855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160cc6a-3196-4840-a8cf-4b35198467c9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:50.346273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160cc6a-3196-4840-a8cf-4b35198467c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:50.346676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160cc6a-3196-4840-a8cf-4b35198467c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:50.347462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160cc6a-3196-4840-a8cf-4b35198467c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:50.349306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160cc6a-3196-4840-a8cf-4b35198467c9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:50.374792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de9e45-a667-4214-a124-0ba3d7bd5af8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:50.384484 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2953/5963] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:17:50.388828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de9e45-a667-4214-a124-0ba3d7bd5af8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:50.400938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32a410-fa4a-4ef3-b706-37929943ffc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:50.401680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32a410-fa4a-4ef3-b706-37929943ffc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:50.401956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32a410-fa4a-4ef3-b706-37929943ffc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:50.402220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32a410-fa4a-4ef3-b706-37929943ffc5 admin admin] PATH_INFO: `/v3/projects/71e20a76e7454caa967768a84a07eccf/users/83733e301dd5432495e09c324234c1d3/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:50.409061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32a410-fa4a-4ef3-b706-37929943ffc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e20a76e7454caa967768a84a07eccf, user_id=83733e301dd5432495e09c324234c1d3, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:50.411818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32a410-fa4a-4ef3-b706-37929943ffc5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:50.451826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959771c-1de0-4655-9a6c-584b7eb556fd None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:50.452601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959771c-1de0-4655-9a6c-584b7eb556fd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:50.452601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959771c-1de0-4655-9a6c-584b7eb556fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:50.461327 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3011/5964] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:17:50 2022] PUT /identity/v3/projects/71e20a76e7454caa967768a84a07eccf/users/83733e301dd5432495e09c324234c1d3/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:17:50.528591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4959771c-1de0-4655-9a6c-584b7eb556fd None None] MFA Rules not processed for user `83733e301dd5432495e09c324234c1d3`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:50.529420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4959771c-1de0-4655-9a6c-584b7eb556fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:50.658851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:50.705765 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2954/5965] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:50.718212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32a410-fa4a-4ef3-b706-37929943ffc5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:50.720710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4a32a410-fa4a-4ef3-b706-37929943ffc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:50.899331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32a410-fa4a-4ef3-b706-37929943ffc5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:50.913586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f014fc-e1d4-4afe-8d52-b6ccc5574e0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:50.914280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f014fc-e1d4-4afe-8d52-b6ccc5574e0a service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:50.914529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f014fc-e1d4-4afe-8d52-b6ccc5574e0a service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:50.914803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f014fc-e1d4-4afe-8d52-b6ccc5574e0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:50.937825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f014fc-e1d4-4afe-8d52-b6ccc5574e0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:50.939285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f014fc-e1d4-4afe-8d52-b6ccc5574e0a service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:50.959390 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3012/5966] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:17:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:17:53.792045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023d218-e17a-4f88-8609-85c43de6d67b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:53.792616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023d218-e17a-4f88-8609-85c43de6d67b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:53.793167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023d218-e17a-4f88-8609-85c43de6d67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:53.840174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f023d218-e17a-4f88-8609-85c43de6d67b None None] MFA Rules not processed for user `f49e7f7982ad4d98b48519f7c1b43fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:53.841589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f023d218-e17a-4f88-8609-85c43de6d67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:53.856342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f014fc-e1d4-4afe-8d52-b6ccc5574e0a service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:53.957164 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2955/5967] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:53.971788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e21e3-57fa-48cb-96a3-b3065e50dd7b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:53.972509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e21e3-57fa-48cb-96a3-b3065e50dd7b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:53.973023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e21e3-57fa-48cb-96a3-b3065e50dd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:54.028618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f014fc-e1d4-4afe-8d52-b6ccc5574e0a service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:54.050759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2a4e21e3-57fa-48cb-96a3-b3065e50dd7b None None] MFA Rules not processed for user `f49e7f7982ad4d98b48519f7c1b43fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:54.054397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0dd1b-fd30-4aa6-91ba-5eed5bad21e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:54.055250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0dd1b-fd30-4aa6-91ba-5eed5bad21e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:54.055958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0dd1b-fd30-4aa6-91ba-5eed5bad21e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:54.056395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0dd1b-fd30-4aa6-91ba-5eed5bad21e1 admin admin] PATH_INFO: `/v3/users/19d2e1626a3642dca393315471be9ee4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:54.057901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0dd1b-fd30-4aa6-91ba-5eed5bad21e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d2e1626a3642dca393315471be9ee4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:54.059385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2a4e21e3-57fa-48cb-96a3-b3065e50dd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:54.060501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0dd1b-fd30-4aa6-91ba-5eed5bad21e1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:54.085533 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2956/5968] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:54.100470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e21e3-57fa-48cb-96a3-b3065e50dd7b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:54.125629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2a4e21e3-57fa-48cb-96a3-b3065e50dd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:54.146159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:54.162700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ff0dd1b-fd30-4aa6-91ba-5eed5bad21e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d2e1626a3642dca393315471be9ee4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:54.185921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ff0dd1b-fd30-4aa6-91ba-5eed5bad21e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d2e1626a3642dca393315471be9ee4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:54.198609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ff0dd1b-fd30-4aa6-91ba-5eed5bad21e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d2e1626a3642dca393315471be9ee4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:54.214396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ff0dd1b-fd30-4aa6-91ba-5eed5bad21e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d2e1626a3642dca393315471be9ee4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:54.215263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4ff0dd1b-fd30-4aa6-91ba-5eed5bad21e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d2e1626a3642dca393315471be9ee4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:54.232763 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3013/5969] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:17:53 2022] DELETE /identity/v3/users/19d2e1626a3642dca393315471be9ee4 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:54.234281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448ecd9-289e-400b-9c7b-78083ae6f36f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:54.234561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448ecd9-289e-400b-9c7b-78083ae6f36f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:54.234827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448ecd9-289e-400b-9c7b-78083ae6f36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:54.304553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5448ecd9-289e-400b-9c7b-78083ae6f36f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:54.305890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5448ecd9-289e-400b-9c7b-78083ae6f36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:54.395797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e21e3-57fa-48cb-96a3-b3065e50dd7b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:54.433695 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3014/5970] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:54.441185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448ecd9-289e-400b-9c7b-78083ae6f36f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:54.444811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5448ecd9-289e-400b-9c7b-78083ae6f36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:54.495737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66842092-41cf-4c07-8461-91c8bb9bac80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:54.496280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66842092-41cf-4c07-8461-91c8bb9bac80 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:54.496600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66842092-41cf-4c07-8461-91c8bb9bac80 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:54.496866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66842092-41cf-4c07-8461-91c8bb9bac80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:54.498887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-66842092-41cf-4c07-8461-91c8bb9bac80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:54.518082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:54.550203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448ecd9-289e-400b-9c7b-78083ae6f36f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:54.555080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc13d8-3c32-4d4f-810a-fef10ce070f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:54.555848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc13d8-3c32-4d4f-810a-fef10ce070f6 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:54.556404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc13d8-3c32-4d4f-810a-fef10ce070f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:54.556744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc13d8-3c32-4d4f-810a-fef10ce070f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:54.582631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc13d8-3c32-4d4f-810a-fef10ce070f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:54.584789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc13d8-3c32-4d4f-810a-fef10ce070f6 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:54.603053 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3015/5971] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:17:54.609941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66842092-41cf-4c07-8461-91c8bb9bac80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:54.611677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66842092-41cf-4c07-8461-91c8bb9bac80 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:54.654465 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2957/5972] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:17:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:17:56.233759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc13d8-3c32-4d4f-810a-fef10ce070f6 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:56.236099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fffc13d8-3c32-4d4f-810a-fef10ce070f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:56.254269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc13d8-3c32-4d4f-810a-fef10ce070f6 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:56.259982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:56.261037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:56.261732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:56.262262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] PATH_INFO: `/v3/projects/cbb23f70213241e0b263b1a1a99f734b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:56.264227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb23f70213241e0b263b1a1a99f734b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:56.266236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:56.400172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:56.406396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb23f70213241e0b263b1a1a99f734b', 'request_id': 'req-816435d0-7358-4787-a968-b1105489c77b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:56.444758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb23f70213241e0b263b1a1a99f734b', 'request_id': 'req-816435d0-7358-4787-a968-b1105489c77b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:56.469279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb23f70213241e0b263b1a1a99f734b', 'request_id': 'req-816435d0-7358-4787-a968-b1105489c77b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:56.503192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] The token cache is being invalidate because project cbb23f70213241e0b263b1a1a99f734b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:17:56.503482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:56.506843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:56.514166 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3016/5973] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:17:56 2022] DELETE /identity/v3/projects/cbb23f70213241e0b263b1a1a99f734b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:56.515435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66842092-41cf-4c07-8461-91c8bb9bac80 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:56.536178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66842092-41cf-4c07-8461-91c8bb9bac80 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:56.750170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6400ab-86b9-4480-a7fa-676ca6aff902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:56.751002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6400ab-86b9-4480-a7fa-676ca6aff902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:56.751355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6400ab-86b9-4480-a7fa-676ca6aff902 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:56.751722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6400ab-86b9-4480-a7fa-676ca6aff902 admin admin] PATH_INFO: `/v3/users/83733e301dd5432495e09c324234c1d3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:56.752977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6400ab-86b9-4480-a7fa-676ca6aff902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83733e301dd5432495e09c324234c1d3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:56.755326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6400ab-86b9-4480-a7fa-676ca6aff902 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:56.856135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5e6400ab-86b9-4480-a7fa-676ca6aff902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83733e301dd5432495e09c324234c1d3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:56.875317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5e6400ab-86b9-4480-a7fa-676ca6aff902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83733e301dd5432495e09c324234c1d3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:56.899976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5e6400ab-86b9-4480-a7fa-676ca6aff902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83733e301dd5432495e09c324234c1d3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:56.902217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5e6400ab-86b9-4480-a7fa-676ca6aff902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83733e301dd5432495e09c324234c1d3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:56.904365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5e6400ab-86b9-4480-a7fa-676ca6aff902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83733e301dd5432495e09c324234c1d3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:56.950064 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2958/5974] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:17:56 2022] DELETE /identity/v3/users/83733e301dd5432495e09c324234c1d3 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:58.165804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:58.167972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:58.211049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816435d0-7358-4787-a968-b1105489c77b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:58.412838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61b0c9-8703-4192-a326-79e5f7ebafff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:58.413986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61b0c9-8703-4192-a326-79e5f7ebafff admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:58.414605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61b0c9-8703-4192-a326-79e5f7ebafff admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:58.415339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61b0c9-8703-4192-a326-79e5f7ebafff admin admin] PATH_INFO: `/v3/projects/71e20a76e7454caa967768a84a07eccf` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:58.418897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61b0c9-8703-4192-a326-79e5f7ebafff admin admin] RBAC: Authorizing `identity:delete_project(project_id=71e20a76e7454caa967768a84a07eccf)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:58.421176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61b0c9-8703-4192-a326-79e5f7ebafff admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:58.512521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-df61b0c9-8703-4192-a326-79e5f7ebafff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71e20a76e7454caa967768a84a07eccf', 'request_id': 'req-df61b0c9-8703-4192-a326-79e5f7ebafff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:58.578919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-df61b0c9-8703-4192-a326-79e5f7ebafff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71e20a76e7454caa967768a84a07eccf', 'request_id': 'req-df61b0c9-8703-4192-a326-79e5f7ebafff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:58.587931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-df61b0c9-8703-4192-a326-79e5f7ebafff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71e20a76e7454caa967768a84a07eccf', 'request_id': 'req-df61b0c9-8703-4192-a326-79e5f7ebafff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:58.602025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-df61b0c9-8703-4192-a326-79e5f7ebafff admin admin] The token cache is being invalidate because project 71e20a76e7454caa967768a84a07eccf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:17:58.602585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-df61b0c9-8703-4192-a326-79e5f7ebafff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:58.603505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-df61b0c9-8703-4192-a326-79e5f7ebafff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:58.608967 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3017/5975] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:17:58 2022] DELETE /identity/v3/projects/71e20a76e7454caa967768a84a07eccf => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:58.610457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18fa04-b0c9-476e-b9ed-0bdd28771233 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:58.610912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18fa04-b0c9-476e-b9ed-0bdd28771233 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:58.610995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18fa04-b0c9-476e-b9ed-0bdd28771233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:58.646142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3c18fa04-b0c9-476e-b9ed-0bdd28771233 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:58.647686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3c18fa04-b0c9-476e-b9ed-0bdd28771233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:58.694892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:58.787156 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2959/5976] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:58.790535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1ec57-f497-4a8e-8fff-e44e0a0a13f4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:58.790814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1ec57-f497-4a8e-8fff-e44e0a0a13f4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:58.791080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1ec57-f497-4a8e-8fff-e44e0a0a13f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:58.837665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b4b1ec57-f497-4a8e-8fff-e44e0a0a13f4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:58.838990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b4b1ec57-f497-4a8e-8fff-e44e0a0a13f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:58.854487 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3018/5977] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:17:58.855244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18fa04-b0c9-476e-b9ed-0bdd28771233 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:58.866493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18fa04-b0c9-476e-b9ed-0bdd28771233 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:58.872760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54046565-632f-4c40-8c84-b24cdfc9b2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:58.873383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54046565-632f-4c40-8c84-b24cdfc9b2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:58.873710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54046565-632f-4c40-8c84-b24cdfc9b2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:58.874435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54046565-632f-4c40-8c84-b24cdfc9b2d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:58.875460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54046565-632f-4c40-8c84-b24cdfc9b2d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:17:58.875885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54046565-632f-4c40-8c84-b24cdfc9b2d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:58.877757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54046565-632f-4c40-8c84-b24cdfc9b2d1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:58.896879 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2960/5978] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:17:58.906399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1ec57-f497-4a8e-8fff-e44e0a0a13f4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:58.920924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1ec57-f497-4a8e-8fff-e44e0a0a13f4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:58.928172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac59087-909a-4046-bc80-8da63dff2e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:58.929433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac59087-909a-4046-bc80-8da63dff2e8a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:58.929818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac59087-909a-4046-bc80-8da63dff2e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:58.930780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac59087-909a-4046-bc80-8da63dff2e8a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:58.931430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac59087-909a-4046-bc80-8da63dff2e8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:58.935287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac59087-909a-4046-bc80-8da63dff2e8a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:58.985633 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3019/5979] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:17:58 2022] POST /identity/v3/projects => generated 399 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:17:58.986355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54046565-632f-4c40-8c84-b24cdfc9b2d1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:59.002722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54046565-632f-4c40-8c84-b24cdfc9b2d1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:59.105353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7524cb-0d81-4f1d-b6a4-a5240d18435d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:59.106106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7524cb-0d81-4f1d-b6a4-a5240d18435d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:59.106481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7524cb-0d81-4f1d-b6a4-a5240d18435d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:59.106844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7524cb-0d81-4f1d-b6a4-a5240d18435d admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:59.107573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7524cb-0d81-4f1d-b6a4-a5240d18435d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:59.109638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7524cb-0d81-4f1d-b6a4-a5240d18435d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:59.139719 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2961/5980] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:17:58 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:17:59.144601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac59087-909a-4046-bc80-8da63dff2e8a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:59.162932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac59087-909a-4046-bc80-8da63dff2e8a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:59.170429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54801dfb-7968-4ad6-9fa9-8885daf06052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:59.174002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54801dfb-7968-4ad6-9fa9-8885daf06052 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:59.174002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54801dfb-7968-4ad6-9fa9-8885daf06052 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:59.174002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54801dfb-7968-4ad6-9fa9-8885daf06052 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:59.174002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54801dfb-7968-4ad6-9fa9-8885daf06052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:59.174825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7524cb-0d81-4f1d-b6a4-a5240d18435d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:59.176137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54801dfb-7968-4ad6-9fa9-8885daf06052 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:59.177240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cd7524cb-0d81-4f1d-b6a4-a5240d18435d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:59.195820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7524cb-0d81-4f1d-b6a4-a5240d18435d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:59.196596 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3020/5981] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:17:59.198550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54801dfb-7968-4ad6-9fa9-8885daf06052 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:59.210974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54801dfb-7968-4ad6-9fa9-8885daf06052 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:59.222420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c54b8-ba59-4c88-873f-0335148f5ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:59.223408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c54b8-ba59-4c88-873f-0335148f5ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:59.224999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c54b8-ba59-4c88-873f-0335148f5ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:59.225518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c54b8-ba59-4c88-873f-0335148f5ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:59.226440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c54b8-ba59-4c88-873f-0335148f5ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:59.229604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c54b8-ba59-4c88-873f-0335148f5ce1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:59.231018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:59.231690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:59.232024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:59.232303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] PATH_INFO: `/v3/users/fa3184ee6bec45da840eb9182b9a5f4d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:59.234494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa3184ee6bec45da840eb9182b9a5f4d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:59.241210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:59.251007 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3021/5982] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:17:59.252052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c54b8-ba59-4c88-873f-0335148f5ce1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:59.266249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c54b8-ba59-4c88-873f-0335148f5ce1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:59.272611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a05e3-a859-4f70-bb50-fb24389d59e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:59.274002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a05e3-a859-4f70-bb50-fb24389d59e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:59.274492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a05e3-a859-4f70-bb50-fb24389d59e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:59.274940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a05e3-a859-4f70-bb50-fb24389d59e6 admin admin] PATH_INFO: `/v3/projects/6d8a2f7c819f4b51a01d36e4c9bba070/users/edfdf7fc45b048679a51fbb6fd2fbdcb/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:59.309624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:59.314482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a05e3-a859-4f70-bb50-fb24389d59e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d8a2f7c819f4b51a01d36e4c9bba070, user_id=edfdf7fc45b048679a51fbb6fd2fbdcb, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:59.319371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a05e3-a859-4f70-bb50-fb24389d59e6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:59.327091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa3184ee6bec45da840eb9182b9a5f4d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:59.342611 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3022/5983] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:17:59 2022] PUT /identity/v3/projects/6d8a2f7c819f4b51a01d36e4c9bba070/users/edfdf7fc45b048679a51fbb6fd2fbdcb/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:17:59.349368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa3184ee6bec45da840eb9182b9a5f4d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:59.356691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51a3af-c9e5-4c6c-93b5-3812135a5153 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:59.357093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51a3af-c9e5-4c6c-93b5-3812135a5153 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:59.357608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51a3af-c9e5-4c6c-93b5-3812135a5153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:59.382106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa3184ee6bec45da840eb9182b9a5f4d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:59.382914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa3184ee6bec45da840eb9182b9a5f4d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:59.384507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa3184ee6bec45da840eb9182b9a5f4d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:59.403432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-da51a3af-c9e5-4c6c-93b5-3812135a5153 None None] MFA Rules not processed for user `edfdf7fc45b048679a51fbb6fd2fbdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:17:59.407060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-da51a3af-c9e5-4c6c-93b5-3812135a5153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:59.413685 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2962/5984] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:17:59 2022] DELETE /identity/v3/users/fa3184ee6bec45da840eb9182b9a5f4d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:17:59.514927 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3023/5985] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:17:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:17:59.515479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:59.516938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:59.548497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:59.574131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51a3af-c9e5-4c6c-93b5-3812135a5153 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:59.575663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-da51a3af-c9e5-4c6c-93b5-3812135a5153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:17:59.666440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439dfd1-1939-43c7-b9b7-53d4723d40c7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:59.677162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af55cb-c74a-4673-8704-f71aaef39c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:59.679870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af55cb-c74a-4673-8704-f71aaef39c41 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:59.680711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af55cb-c74a-4673-8704-f71aaef39c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:59.681194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af55cb-c74a-4673-8704-f71aaef39c41 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:59.682563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af55cb-c74a-4673-8704-f71aaef39c41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:59.685396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af55cb-c74a-4673-8704-f71aaef39c41 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:59.731182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51a3af-c9e5-4c6c-93b5-3812135a5153 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:59.751028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:59.751737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:59.752018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:59.752318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] PATH_INFO: `/v3/users/8e94def1933b4a978081db477f522bb4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:59.754281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e94def1933b4a978081db477f522bb4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:59.762820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:59.773141 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2963/5986] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:17:59 2022] POST /identity/v3/projects => generated 399 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:17:59.774353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af55cb-c74a-4673-8704-f71aaef39c41 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:59.906119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af55cb-c74a-4673-8704-f71aaef39c41 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:17:59.912948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e94def1933b4a978081db477f522bb4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:59.917589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec12837-960f-4c24-b603-519e8da3b2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:17:59.918222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec12837-960f-4c24-b603-519e8da3b2e7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:17:59.918497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec12837-960f-4c24-b603-519e8da3b2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:17:59.918768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec12837-960f-4c24-b603-519e8da3b2e7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:17:59.919408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec12837-960f-4c24-b603-519e8da3b2e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:17:59.921401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec12837-960f-4c24-b603-519e8da3b2e7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:17:59.945125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e94def1933b4a978081db477f522bb4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:59.956998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e94def1933b4a978081db477f522bb4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:59.987836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e94def1933b4a978081db477f522bb4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:59.988625 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2964/5987] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:17:59 2022] POST /identity/v3/users => generated 306 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:17:59.989275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec12837-960f-4c24-b603-519e8da3b2e7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:17:59.989407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e94def1933b4a978081db477f522bb4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:17:59.994078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:17:59.996578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2ec12837-960f-4c24-b603-519e8da3b2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:00.007720 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3024/5988] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:17:59 2022] DELETE /identity/v3/users/8e94def1933b4a978081db477f522bb4 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:18:00.009382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:00.011620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:00.019811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec12837-960f-4c24-b603-519e8da3b2e7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:00.108874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:00.109530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.109810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.110074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] PATH_INFO: `/v3/projects/da107d25a79c4ae3a25c836bc2d23e19` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.111240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da107d25a79c4ae3a25c836bc2d23e19)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:00.113284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:00.115243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ef872-5025-4228-90dc-9302ec3b7553 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:00.122026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996f526-88cf-4cc1-826e-d60f6251be58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:00.123322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996f526-88cf-4cc1-826e-d60f6251be58 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.123767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996f526-88cf-4cc1-826e-d60f6251be58 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.124193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996f526-88cf-4cc1-826e-d60f6251be58 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.125204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996f526-88cf-4cc1-826e-d60f6251be58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:00.127137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996f526-88cf-4cc1-826e-d60f6251be58 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:00.142622 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3025/5989] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:00.145275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910067e4-31c6-43ad-8a77-967013fa9ee5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.146170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910067e4-31c6-43ad-8a77-967013fa9ee5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.146672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910067e4-31c6-43ad-8a77-967013fa9ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.163749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:00.176740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da107d25a79c4ae3a25c836bc2d23e19', 'request_id': 'req-fc650bc1-5a00-4c81-b0a6-5287f3104622'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:00.181388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-910067e4-31c6-43ad-8a77-967013fa9ee5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:00.183604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-910067e4-31c6-43ad-8a77-967013fa9ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:00.184964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da107d25a79c4ae3a25c836bc2d23e19', 'request_id': 'req-fc650bc1-5a00-4c81-b0a6-5287f3104622'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:00.200048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da107d25a79c4ae3a25c836bc2d23e19', 'request_id': 'req-fc650bc1-5a00-4c81-b0a6-5287f3104622'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:00.217304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] The token cache is being invalidate because project da107d25a79c4ae3a25c836bc2d23e19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:00.217796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:00.218735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:00.221884 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2965/5990] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:17:59 2022] DELETE /identity/v3/projects/da107d25a79c4ae3a25c836bc2d23e19 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:00.239587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:00.244054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:00.273007 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3026/5991] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:00.274727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d7f1a-2417-4ec5-a747-0f1e99dc20ca None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.275195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d7f1a-2417-4ec5-a747-0f1e99dc20ca None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.275669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d7f1a-2417-4ec5-a747-0f1e99dc20ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.299340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fd2d7f1a-2417-4ec5-a747-0f1e99dc20ca None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:00.301479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fd2d7f1a-2417-4ec5-a747-0f1e99dc20ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:00.317424 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3027/5992] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:00.318367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d7f1a-2417-4ec5-a747-0f1e99dc20ca None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:00.320551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fd2d7f1a-2417-4ec5-a747-0f1e99dc20ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:00.330720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc650bc1-5a00-4c81-b0a6-5287f3104622 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:00.338288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3da2d-cd5f-4bbe-8c0b-64ead3eba300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:00.339029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3da2d-cd5f-4bbe-8c0b-64ead3eba300 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.339461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3da2d-cd5f-4bbe-8c0b-64ead3eba300 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.339736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3da2d-cd5f-4bbe-8c0b-64ead3eba300 admin admin] PATH_INFO: `/v3/projects/1ec8aa8d9ccd4a2695247dcd8cf0c252/users/107d61a9534245dbb661dc596c201bb5/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.358791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3da2d-cd5f-4bbe-8c0b-64ead3eba300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ec8aa8d9ccd4a2695247dcd8cf0c252, user_id=107d61a9534245dbb661dc596c201bb5, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:00.361991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3da2d-cd5f-4bbe-8c0b-64ead3eba300 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:00.392354 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2966/5993] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:00 2022] PUT /identity/v3/projects/1ec8aa8d9ccd4a2695247dcd8cf0c252/users/107d61a9534245dbb661dc596c201bb5/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:00.392354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897375f-1436-47c1-9fc4-e05e19071d77 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.392354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897375f-1436-47c1-9fc4-e05e19071d77 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.392354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897375f-1436-47c1-9fc4-e05e19071d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.423797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d7f1a-2417-4ec5-a747-0f1e99dc20ca None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:00.425654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4897375f-1436-47c1-9fc4-e05e19071d77 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:00.426963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4897375f-1436-47c1-9fc4-e05e19071d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:00.487712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:00.494110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd8cb3-8c2e-4dc0-bf09-bb2c9521eab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:00.494823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd8cb3-8c2e-4dc0-bf09-bb2c9521eab3 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.495161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd8cb3-8c2e-4dc0-bf09-bb2c9521eab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.495497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd8cb3-8c2e-4dc0-bf09-bb2c9521eab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.508511 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2967/5994] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:00.509733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897375f-1436-47c1-9fc4-e05e19071d77 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:00.520746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897375f-1436-47c1-9fc4-e05e19071d77 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:00.526837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b4166-d97b-4bc0-a9d7-501f4db590e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:00.527562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b4166-d97b-4bc0-a9d7-501f4db590e8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.527906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b4166-d97b-4bc0-a9d7-501f4db590e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.528240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b4166-d97b-4bc0-a9d7-501f4db590e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.528987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b4166-d97b-4bc0-a9d7-501f4db590e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:00.530679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b4166-d97b-4bc0-a9d7-501f4db590e8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:00.543893 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2968/5995] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:00.544921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b4166-d97b-4bc0-a9d7-501f4db590e8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:00.557194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b4166-d97b-4bc0-a9d7-501f4db590e8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:00.563332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536496cf-9cf0-4553-a817-588a10352575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:00.564026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536496cf-9cf0-4553-a817-588a10352575 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.564395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536496cf-9cf0-4553-a817-588a10352575 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.564732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536496cf-9cf0-4553-a817-588a10352575 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.565456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536496cf-9cf0-4553-a817-588a10352575 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:18:00.565824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536496cf-9cf0-4553-a817-588a10352575 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:00.567557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536496cf-9cf0-4553-a817-588a10352575 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:00.571066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd8cb3-8c2e-4dc0-bf09-bb2c9521eab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:00.572769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd8cb3-8c2e-4dc0-bf09-bb2c9521eab3 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:00.583593 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2969/5996] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:18:00.584450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536496cf-9cf0-4553-a817-588a10352575 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:00.584952 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3028/5997] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:00.588808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd8cb3-8c2e-4dc0-bf09-bb2c9521eab3 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:00.598094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536496cf-9cf0-4553-a817-588a10352575 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:00.601176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd8cb3-8c2e-4dc0-bf09-bb2c9521eab3 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:00.605772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c7adc-d66c-437b-a154-cebdc2ad036f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:00.606547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c7adc-d66c-437b-a154-cebdc2ad036f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.607210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c7adc-d66c-437b-a154-cebdc2ad036f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.607626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c7adc-d66c-437b-a154-cebdc2ad036f admin admin] PATH_INFO: `/v3/projects/1ec8aa8d9ccd4a2695247dcd8cf0c252/users/107d61a9534245dbb661dc596c201bb5/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.607825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067987be-155c-46a1-aee2-0a74a5565bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:00.608573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067987be-155c-46a1-aee2-0a74a5565bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.608939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067987be-155c-46a1-aee2-0a74a5565bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.609318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067987be-155c-46a1-aee2-0a74a5565bf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.609939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c7adc-d66c-437b-a154-cebdc2ad036f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ec8aa8d9ccd4a2695247dcd8cf0c252, user_id=107d61a9534245dbb661dc596c201bb5, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:00.610138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067987be-155c-46a1-aee2-0a74a5565bf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:00.612161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067987be-155c-46a1-aee2-0a74a5565bf2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:00.613460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c7adc-d66c-437b-a154-cebdc2ad036f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:00.628055 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2970/5998] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:00 2022] PUT /identity/v3/projects/1ec8aa8d9ccd4a2695247dcd8cf0c252/users/107d61a9534245dbb661dc596c201bb5/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:00.642515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd401f-cbe3-4e2b-bbe6-f643da5b256e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.643271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd401f-cbe3-4e2b-bbe6-f643da5b256e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.643694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd401f-cbe3-4e2b-bbe6-f643da5b256e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.661177 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3029/5999] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:00 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:00.666293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067987be-155c-46a1-aee2-0a74a5565bf2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:00.690367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:00.701267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a5fd401f-cbe3-4e2b-bbe6-f643da5b256e None None] MFA Rules not processed for user `107d61a9534245dbb661dc596c201bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:00.702926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a5fd401f-cbe3-4e2b-bbe6-f643da5b256e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:00.754790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067987be-155c-46a1-aee2-0a74a5565bf2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:00.760327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96addc82-a4d3-4ca7-923e-d460f5c55906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:00.760988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96addc82-a4d3-4ca7-923e-d460f5c55906 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.761372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96addc82-a4d3-4ca7-923e-d460f5c55906 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.762666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96addc82-a4d3-4ca7-923e-d460f5c55906 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.763412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96addc82-a4d3-4ca7-923e-d460f5c55906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:00.765751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96addc82-a4d3-4ca7-923e-d460f5c55906 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:00.803868 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3030/6000] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:00 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:00.806255 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2971/6001] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:00.812976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96addc82-a4d3-4ca7-923e-d460f5c55906 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:00.814447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd401f-cbe3-4e2b-bbe6-f643da5b256e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:00.816807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a5fd401f-cbe3-4e2b-bbe6-f643da5b256e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:00.831184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96addc82-a4d3-4ca7-923e-d460f5c55906 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:00.837339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21082e-4edc-4de1-98f9-df3ed7c86c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:00.837966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21082e-4edc-4de1-98f9-df3ed7c86c03 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.838240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21082e-4edc-4de1-98f9-df3ed7c86c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.838509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21082e-4edc-4de1-98f9-df3ed7c86c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.839154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21082e-4edc-4de1-98f9-df3ed7c86c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:00.840811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21082e-4edc-4de1-98f9-df3ed7c86c03 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:00.853331 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3031/6002] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:00.854144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21082e-4edc-4de1-98f9-df3ed7c86c03 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:00.859241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5f21082e-4edc-4de1-98f9-df3ed7c86c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:00.879582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21082e-4edc-4de1-98f9-df3ed7c86c03 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:00.885755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87e03a-677e-4eb3-9d67-a385f4992f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:00.886484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87e03a-677e-4eb3-9d67-a385f4992f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.886866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87e03a-677e-4eb3-9d67-a385f4992f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.887084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87e03a-677e-4eb3-9d67-a385f4992f44 admin admin] PATH_INFO: `/v3/projects/0fc396a4790b4a25b9cab5d510db1369` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.888323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87e03a-677e-4eb3-9d67-a385f4992f44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fc396a4790b4a25b9cab5d510db1369)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:00.890339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87e03a-677e-4eb3-9d67-a385f4992f44 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:00.893267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd401f-cbe3-4e2b-bbe6-f643da5b256e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:00.897030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab60fc-38cb-449f-b882-d825d9919e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:00.897705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab60fc-38cb-449f-b882-d825d9919e30 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:00.897990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab60fc-38cb-449f-b882-d825d9919e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:00.898332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab60fc-38cb-449f-b882-d825d9919e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:00.909785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab60fc-38cb-449f-b882-d825d9919e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:00.911313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab60fc-38cb-449f-b882-d825d9919e30 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:00.923788 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2972/6003] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:00 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:00.924286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab60fc-38cb-449f-b882-d825d9919e30 service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:00.936737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab60fc-38cb-449f-b882-d825d9919e30 service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:00.955412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:00.958228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a87e03a-677e-4eb3-9d67-a385f4992f44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fc396a4790b4a25b9cab5d510db1369', 'request_id': 'req-8a87e03a-677e-4eb3-9d67-a385f4992f44'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:00.969493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a87e03a-677e-4eb3-9d67-a385f4992f44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fc396a4790b4a25b9cab5d510db1369', 'request_id': 'req-8a87e03a-677e-4eb3-9d67-a385f4992f44'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:00.978699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a87e03a-677e-4eb3-9d67-a385f4992f44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fc396a4790b4a25b9cab5d510db1369', 'request_id': 'req-8a87e03a-677e-4eb3-9d67-a385f4992f44'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:01.002591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a87e03a-677e-4eb3-9d67-a385f4992f44 admin admin] The token cache is being invalidate because project 0fc396a4790b4a25b9cab5d510db1369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:01.002591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a87e03a-677e-4eb3-9d67-a385f4992f44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:01.003411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8a87e03a-677e-4eb3-9d67-a385f4992f44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:01.010223 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3032/6004] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:00 2022] DELETE /identity/v3/projects/0fc396a4790b4a25b9cab5d510db1369 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:01.012212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f50cfd-0843-49bd-b6a0-a8e619a83ae7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:01.012607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f50cfd-0843-49bd-b6a0-a8e619a83ae7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:01.013608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f50cfd-0843-49bd-b6a0-a8e619a83ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:01.028566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8274c-849c-4515-b6f5-22389b87e058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:01.029297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8274c-849c-4515-b6f5-22389b87e058 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:01.029741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8274c-849c-4515-b6f5-22389b87e058 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:01.030064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8274c-849c-4515-b6f5-22389b87e058 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:01.030895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea8274c-849c-4515-b6f5-22389b87e058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:01.032867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea8274c-849c-4515-b6f5-22389b87e058 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:01.041505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d3f50cfd-0843-49bd-b6a0-a8e619a83ae7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:01.043201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d3f50cfd-0843-49bd-b6a0-a8e619a83ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:01.050545 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2973/6005] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:01.052694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8274c-849c-4515-b6f5-22389b87e058 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:01.054871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6ea8274c-849c-4515-b6f5-22389b87e058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:01.069700 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3033/6006] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:01.070655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f50cfd-0843-49bd-b6a0-a8e619a83ae7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:01.072359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d3f50cfd-0843-49bd-b6a0-a8e619a83ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:01.090048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f50cfd-0843-49bd-b6a0-a8e619a83ae7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:01.096133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f758e3-f699-4b1d-8503-d72f186a300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:01.096933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f758e3-f699-4b1d-8503-d72f186a300d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:01.097554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f758e3-f699-4b1d-8503-d72f186a300d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:01.097885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f758e3-f699-4b1d-8503-d72f186a300d admin admin] PATH_INFO: `/v3/projects/aa2842b604df4797b6f5cfbdfb03f173/users/7013eec1908f407d9949d271287b1570/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:01.108028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:01.120637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f758e3-f699-4b1d-8503-d72f186a300d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa2842b604df4797b6f5cfbdfb03f173, user_id=7013eec1908f407d9949d271287b1570, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:01.124788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f758e3-f699-4b1d-8503-d72f186a300d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:01.144754 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3034/6007] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:01 2022] PUT /identity/v3/projects/aa2842b604df4797b6f5cfbdfb03f173/users/7013eec1908f407d9949d271287b1570/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:01.147030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcd86d-45dc-46f7-97ef-34d2e7c191bb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:01.147441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcd86d-45dc-46f7-97ef-34d2e7c191bb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:01.147795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcd86d-45dc-46f7-97ef-34d2e7c191bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:01.158039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8274c-849c-4515-b6f5-22389b87e058 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:01.181154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4fbcd86d-45dc-46f7-97ef-34d2e7c191bb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:01.183105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4fbcd86d-45dc-46f7-97ef-34d2e7c191bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:01.241339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d87b3d-97c0-44df-bc50-c3199ffbb8d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:01.242574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d87b3d-97c0-44df-bc50-c3199ffbb8d3 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:01.242574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d87b3d-97c0-44df-bc50-c3199ffbb8d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:01.243260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d87b3d-97c0-44df-bc50-c3199ffbb8d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:01.245858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-81d87b3d-97c0-44df-bc50-c3199ffbb8d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:01.278265 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3035/6008] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:01.279879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc2b56-c1a2-4cf0-86ab-03414cb70de7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:01.280313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc2b56-c1a2-4cf0-86ab-03414cb70de7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:01.280754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc2b56-c1a2-4cf0-86ab-03414cb70de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:01.307612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-00bc2b56-c1a2-4cf0-86ab-03414cb70de7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:01.309147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-00bc2b56-c1a2-4cf0-86ab-03414cb70de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:01.320464 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3036/6009] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:01.320464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc2b56-c1a2-4cf0-86ab-03414cb70de7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:01.325386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d87b3d-97c0-44df-bc50-c3199ffbb8d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:01.327970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d87b3d-97c0-44df-bc50-c3199ffbb8d3 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:01.331734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc2b56-c1a2-4cf0-86ab-03414cb70de7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:01.340356 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2974/6010] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:01.341030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b0dd2-343c-425f-b119-d34abecbcedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:01.341655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b0dd2-343c-425f-b119-d34abecbcedf admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:01.341919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b0dd2-343c-425f-b119-d34abecbcedf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:01.342177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b0dd2-343c-425f-b119-d34abecbcedf admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:01.343257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b0dd2-343c-425f-b119-d34abecbcedf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:18:01.343257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b0dd2-343c-425f-b119-d34abecbcedf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:01.344731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b0dd2-343c-425f-b119-d34abecbcedf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:01.345699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d87b3d-97c0-44df-bc50-c3199ffbb8d3 service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:01.347571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-81d87b3d-97c0-44df-bc50-c3199ffbb8d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:01.361798 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3037/6011] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:18:01.363502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b0dd2-343c-425f-b119-d34abecbcedf admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:01.379337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b0dd2-343c-425f-b119-d34abecbcedf admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:01.386324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc58f7-a743-46a5-b48b-054391ccc21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:01.387177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc58f7-a743-46a5-b48b-054391ccc21f admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:01.387445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc58f7-a743-46a5-b48b-054391ccc21f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:01.387709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc58f7-a743-46a5-b48b-054391ccc21f admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:01.388337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc58f7-a743-46a5-b48b-054391ccc21f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:01.390361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc58f7-a743-46a5-b48b-054391ccc21f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:01.456702 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3038/6012] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:01 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:01.460279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc58f7-a743-46a5-b48b-054391ccc21f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:01.502494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d87b3d-97c0-44df-bc50-c3199ffbb8d3 service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:01.622171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b056a77-a367-41bb-8b9c-46efdec4a4dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:01.624011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b056a77-a367-41bb-8b9c-46efdec4a4dd service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:01.624606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b056a77-a367-41bb-8b9c-46efdec4a4dd service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:01.625036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b056a77-a367-41bb-8b9c-46efdec4a4dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:01.667498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc58f7-a743-46a5-b48b-054391ccc21f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:01.668176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:01.678935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf00b1f-0c7e-42ae-93aa-14dbec43618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:01.680086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf00b1f-0c7e-42ae-93aa-14dbec43618f admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:01.680807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf00b1f-0c7e-42ae-93aa-14dbec43618f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:01.681268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf00b1f-0c7e-42ae-93aa-14dbec43618f admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:01.684659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf00b1f-0c7e-42ae-93aa-14dbec43618f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:01.694774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf00b1f-0c7e-42ae-93aa-14dbec43618f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:01.721130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:01.746672 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3039/6013] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:01 2022] POST /identity/v3/users => generated 286 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:01.748303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf00b1f-0c7e-42ae-93aa-14dbec43618f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:01.777754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf00b1f-0c7e-42ae-93aa-14dbec43618f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:01.827159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d23cd-1b79-4ae1-85b7-5d083b11339b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:01.827903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d23cd-1b79-4ae1-85b7-5d083b11339b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:01.828178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d23cd-1b79-4ae1-85b7-5d083b11339b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:01.828485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d23cd-1b79-4ae1-85b7-5d083b11339b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:01.829127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d23cd-1b79-4ae1-85b7-5d083b11339b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:01.848033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d23cd-1b79-4ae1-85b7-5d083b11339b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:01.861679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b056a77-a367-41bb-8b9c-46efdec4a4dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:01.863139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b056a77-a367-41bb-8b9c-46efdec4a4dd service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:01.887893 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3040/6014] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:01.892745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d23cd-1b79-4ae1-85b7-5d083b11339b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:01.908619 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2975/6015] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:01.924707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d23cd-1b79-4ae1-85b7-5d083b11339b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:01.941336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b06bd9-f19e-4c60-93bf-2b42632b37ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:01.942352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b06bd9-f19e-4c60-93bf-2b42632b37ef admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:01.946061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b06bd9-f19e-4c60-93bf-2b42632b37ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:01.947031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b06bd9-f19e-4c60-93bf-2b42632b37ef admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:01.948642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b06bd9-f19e-4c60-93bf-2b42632b37ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:01.950878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b06bd9-f19e-4c60-93bf-2b42632b37ef admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:01.986590 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3041/6016] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:01.988908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b056a77-a367-41bb-8b9c-46efdec4a4dd service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:02.003439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b056a77-a367-41bb-8b9c-46efdec4a4dd service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:02.016306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ad5a7-c5cd-46ec-84b6-baa852d12346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:02.018492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ad5a7-c5cd-46ec-84b6-baa852d12346 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:02.019374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ad5a7-c5cd-46ec-84b6-baa852d12346 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:02.020043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ad5a7-c5cd-46ec-84b6-baa852d12346 admin admin] PATH_INFO: `/v3/projects/ae34c5d4a3dc40168445a3822e12e7d6/users/a22eb4ad08ce4027bfa7f6f487a3534b/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:02.055477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ad5a7-c5cd-46ec-84b6-baa852d12346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae34c5d4a3dc40168445a3822e12e7d6, user_id=a22eb4ad08ce4027bfa7f6f487a3534b, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:02.060093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ad5a7-c5cd-46ec-84b6-baa852d12346 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:02.100482 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2976/6017] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:01 2022] PUT /identity/v3/projects/ae34c5d4a3dc40168445a3822e12e7d6/users/a22eb4ad08ce4027bfa7f6f487a3534b/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:02.108924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d9f9-65d8-4a69-a838-538c21967d99 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:02.109553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d9f9-65d8-4a69-a838-538c21967d99 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:02.110036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d9f9-65d8-4a69-a838-538c21967d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:02.218298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9473d9f9-65d8-4a69-a838-538c21967d99 None None] MFA Rules not processed for user `a22eb4ad08ce4027bfa7f6f487a3534b`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:02.221837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9473d9f9-65d8-4a69-a838-538c21967d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:02.355763 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3042/6018] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:02.386766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ad5a7-c5cd-46ec-84b6-baa852d12346 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:02.486458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ad5a7-c5cd-46ec-84b6-baa852d12346 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:02.493127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e4842-9000-4c9f-b149-73a0f7919d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:02.493783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e4842-9000-4c9f-b149-73a0f7919d3b service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:02.494100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e4842-9000-4c9f-b149-73a0f7919d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:02.494373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e4842-9000-4c9f-b149-73a0f7919d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:02.506549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e4842-9000-4c9f-b149-73a0f7919d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:02.508169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e4842-9000-4c9f-b149-73a0f7919d3b service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:02.521940 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2977/6019] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:03.098050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d9f9-65d8-4a69-a838-538c21967d99 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:03.100520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9473d9f9-65d8-4a69-a838-538c21967d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:03.273521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d9f9-65d8-4a69-a838-538c21967d99 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:03.298329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0ef50-e93a-442a-8c6d-39bcc814c0ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:03.299471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0ef50-e93a-442a-8c6d-39bcc814c0ab service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:03.299728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0ef50-e93a-442a-8c6d-39bcc814c0ab service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:03.300817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0ef50-e93a-442a-8c6d-39bcc814c0ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:03.303573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9de0ef50-e93a-442a-8c6d-39bcc814c0ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:03.323269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0ef50-e93a-442a-8c6d-39bcc814c0ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:03.325079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0ef50-e93a-442a-8c6d-39bcc814c0ab service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:03.340096 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3043/6020] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:03.656016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e4842-9000-4c9f-b149-73a0f7919d3b service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:03.658294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-702e4842-9000-4c9f-b149-73a0f7919d3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:03.815331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e4842-9000-4c9f-b149-73a0f7919d3b service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:03.823957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:03.825035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:03.825499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:03.825952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] PATH_INFO: `/v3/users/edfdf7fc45b048679a51fbb6fd2fbdcb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:03.827407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edfdf7fc45b048679a51fbb6fd2fbdcb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:03.829774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:03.886502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:03.907265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edfdf7fc45b048679a51fbb6fd2fbdcb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:03.918145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edfdf7fc45b048679a51fbb6fd2fbdcb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:03.936218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edfdf7fc45b048679a51fbb6fd2fbdcb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:03.937005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edfdf7fc45b048679a51fbb6fd2fbdcb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:03.937625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edfdf7fc45b048679a51fbb6fd2fbdcb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:03.954992 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2978/6021] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:03 2022] DELETE /identity/v3/users/edfdf7fc45b048679a51fbb6fd2fbdcb => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:03.960452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe450b-9bd1-40fe-a853-d7b4a5f17a70 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:03.960865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe450b-9bd1-40fe-a853-d7b4a5f17a70 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:03.961328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe450b-9bd1-40fe-a853-d7b4a5f17a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:03.991319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-48fe450b-9bd1-40fe-a853-d7b4a5f17a70 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:03.994567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-48fe450b-9bd1-40fe-a853-d7b4a5f17a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:04.074153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:04.111201 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3044/6022] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:04.132326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:04.135723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:04.267574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38115c-ca4a-4f12-a6b1-7afad63fe3a7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:04.279163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e77a8-59f9-46c2-b94c-525e9db99a1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:04.279534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e77a8-59f9-46c2-b94c-525e9db99a1f service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:04.279806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e77a8-59f9-46c2-b94c-525e9db99a1f service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:04.280072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e77a8-59f9-46c2-b94c-525e9db99a1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:04.296081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e77a8-59f9-46c2-b94c-525e9db99a1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:04.299054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e77a8-59f9-46c2-b94c-525e9db99a1f service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:04.317076 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2979/6023] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:05.048034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe450b-9bd1-40fe-a853-d7b4a5f17a70 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:05.049956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-48fe450b-9bd1-40fe-a853-d7b4a5f17a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:05.065299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe450b-9bd1-40fe-a853-d7b4a5f17a70 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:05.073154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:05.074266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:05.074527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:05.075017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] PATH_INFO: `/v3/projects/6d8a2f7c819f4b51a01d36e4c9bba070` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:05.076894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d8a2f7c819f4b51a01d36e4c9bba070)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:05.079671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:05.134979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d8a2f7c819f4b51a01d36e4c9bba070', 'request_id': 'req-a447e90f-1529-45df-8a1f-a423467ff780'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:05.150315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d8a2f7c819f4b51a01d36e4c9bba070', 'request_id': 'req-a447e90f-1529-45df-8a1f-a423467ff780'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:05.157405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d8a2f7c819f4b51a01d36e4c9bba070', 'request_id': 'req-a447e90f-1529-45df-8a1f-a423467ff780'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:05.169175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] The token cache is being invalidate because project 6d8a2f7c819f4b51a01d36e4c9bba070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:05.169550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:05.170232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:05.171755 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3045/6024] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:05 2022] DELETE /identity/v3/projects/6d8a2f7c819f4b51a01d36e4c9bba070 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:05.176608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e77a8-59f9-46c2-b94c-525e9db99a1f service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:05.189744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e77a8-59f9-46c2-b94c-525e9db99a1f service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:05.267160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a779a42-ac72-435b-88b2-7cad09f4fe80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:05.267957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a779a42-ac72-435b-88b2-7cad09f4fe80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:05.268610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a779a42-ac72-435b-88b2-7cad09f4fe80 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:05.269159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a779a42-ac72-435b-88b2-7cad09f4fe80 admin admin] PATH_INFO: `/v3/users/107d61a9534245dbb661dc596c201bb5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:05.270439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a779a42-ac72-435b-88b2-7cad09f4fe80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=107d61a9534245dbb661dc596c201bb5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:05.272662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a779a42-ac72-435b-88b2-7cad09f4fe80 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:05.357523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6a779a42-ac72-435b-88b2-7cad09f4fe80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '107d61a9534245dbb661dc596c201bb5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:05.370122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6a779a42-ac72-435b-88b2-7cad09f4fe80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '107d61a9534245dbb661dc596c201bb5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:05.378041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:05.417382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6a779a42-ac72-435b-88b2-7cad09f4fe80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '107d61a9534245dbb661dc596c201bb5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:05.419005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6a779a42-ac72-435b-88b2-7cad09f4fe80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '107d61a9534245dbb661dc596c201bb5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:05.420689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6a779a42-ac72-435b-88b2-7cad09f4fe80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '107d61a9534245dbb661dc596c201bb5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:05.455909 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2980/6025] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:05 2022] DELETE /identity/v3/users/107d61a9534245dbb661dc596c201bb5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:05.965534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:05.967194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:05.989012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447e90f-1529-45df-8a1f-a423467ff780 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:06.084366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ea096-1203-4369-b2c1-01573f7c27bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:06.084999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ea096-1203-4369-b2c1-01573f7c27bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:06.086135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ea096-1203-4369-b2c1-01573f7c27bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:06.086980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ea096-1203-4369-b2c1-01573f7c27bd admin admin] PATH_INFO: `/v3/projects/1ec8aa8d9ccd4a2695247dcd8cf0c252` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:06.089154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ea096-1203-4369-b2c1-01573f7c27bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ec8aa8d9ccd4a2695247dcd8cf0c252)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:06.092444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ea096-1203-4369-b2c1-01573f7c27bd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:06.182544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e03ea096-1203-4369-b2c1-01573f7c27bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ec8aa8d9ccd4a2695247dcd8cf0c252', 'request_id': 'req-e03ea096-1203-4369-b2c1-01573f7c27bd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:06.195812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e03ea096-1203-4369-b2c1-01573f7c27bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ec8aa8d9ccd4a2695247dcd8cf0c252', 'request_id': 'req-e03ea096-1203-4369-b2c1-01573f7c27bd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:06.211178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e03ea096-1203-4369-b2c1-01573f7c27bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ec8aa8d9ccd4a2695247dcd8cf0c252', 'request_id': 'req-e03ea096-1203-4369-b2c1-01573f7c27bd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:06.227930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e03ea096-1203-4369-b2c1-01573f7c27bd admin admin] The token cache is being invalidate because project 1ec8aa8d9ccd4a2695247dcd8cf0c252 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:06.228625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e03ea096-1203-4369-b2c1-01573f7c27bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:06.233607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e03ea096-1203-4369-b2c1-01573f7c27bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:06.236049 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3046/6026] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:05 2022] DELETE /identity/v3/projects/1ec8aa8d9ccd4a2695247dcd8cf0c252 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:06.241326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3b39-3dd7-4cf6-8836-68f4f605f867 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:06.241757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3b39-3dd7-4cf6-8836-68f4f605f867 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:06.241757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3b39-3dd7-4cf6-8836-68f4f605f867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:06.263573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9c2f3b39-3dd7-4cf6-8836-68f4f605f867 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:06.264845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9c2f3b39-3dd7-4cf6-8836-68f4f605f867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:06.338627 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2981/6027] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:06.348190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b23334-4c5b-4207-b7b2-403d7c63133c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:06.348533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b23334-4c5b-4207-b7b2-403d7c63133c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:06.348813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b23334-4c5b-4207-b7b2-403d7c63133c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:06.357985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:06.373134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e3b23334-4c5b-4207-b7b2-403d7c63133c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:06.375155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e3b23334-4c5b-4207-b7b2-403d7c63133c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:06.384165 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3047/6028] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:06.388628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f3b39-3dd7-4cf6-8836-68f4f605f867 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:06.401029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f3b39-3dd7-4cf6-8836-68f4f605f867 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:06.406554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409a190-ac1f-46f7-a88f-ee610810cb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:06.407271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409a190-ac1f-46f7-a88f-ee610810cb65 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:06.407757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409a190-ac1f-46f7-a88f-ee610810cb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:06.408012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409a190-ac1f-46f7-a88f-ee610810cb65 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:06.408793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409a190-ac1f-46f7-a88f-ee610810cb65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:18:06.409241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409a190-ac1f-46f7-a88f-ee610810cb65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:06.411105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409a190-ac1f-46f7-a88f-ee610810cb65 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:06.425583 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2982/6029] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:18:06.429809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b23334-4c5b-4207-b7b2-403d7c63133c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:06.449155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b23334-4c5b-4207-b7b2-403d7c63133c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:06.456710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2af2e-9689-46c6-b0f5-32cbbf36be51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:06.457520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2af2e-9689-46c6-b0f5-32cbbf36be51 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:06.457886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2af2e-9689-46c6-b0f5-32cbbf36be51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:06.458242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2af2e-9689-46c6-b0f5-32cbbf36be51 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:06.458992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2af2e-9689-46c6-b0f5-32cbbf36be51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:06.461322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2af2e-9689-46c6-b0f5-32cbbf36be51 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:06.500143 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3048/6030] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:06 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:06.501094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409a190-ac1f-46f7-a88f-ee610810cb65 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:06.540524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409a190-ac1f-46f7-a88f-ee610810cb65 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:06.641201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031480e3-e476-429e-ab56-bc75495a3dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:06.641757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031480e3-e476-429e-ab56-bc75495a3dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:06.642080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031480e3-e476-429e-ab56-bc75495a3dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:06.643168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031480e3-e476-429e-ab56-bc75495a3dc2 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:06.643372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031480e3-e476-429e-ab56-bc75495a3dc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:06.645343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031480e3-e476-429e-ab56-bc75495a3dc2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:06.697981 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2983/6031] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:06 2022] POST /identity/v3/users => generated 286 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:18:06.704299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2af2e-9689-46c6-b0f5-32cbbf36be51 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:06.721908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2af2e-9689-46c6-b0f5-32cbbf36be51 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:06.732578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376d454-5ce4-455b-9bbe-bc89fa938588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:06.736987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376d454-5ce4-455b-9bbe-bc89fa938588 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:06.737477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376d454-5ce4-455b-9bbe-bc89fa938588 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:06.737900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376d454-5ce4-455b-9bbe-bc89fa938588 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:06.738884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376d454-5ce4-455b-9bbe-bc89fa938588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:06.741424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376d454-5ce4-455b-9bbe-bc89fa938588 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:06.782504 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3049/6032] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:06.783149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031480e3-e476-429e-ab56-bc75495a3dc2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:06.799521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031480e3-e476-429e-ab56-bc75495a3dc2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:06.810499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132bd72c-5580-467d-8660-0a9a80bd4fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:06.811076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bd72c-5580-467d-8660-0a9a80bd4fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:06.811320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bd72c-5580-467d-8660-0a9a80bd4fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:06.812052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bd72c-5580-467d-8660-0a9a80bd4fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:06.812254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132bd72c-5580-467d-8660-0a9a80bd4fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:06.813888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132bd72c-5580-467d-8660-0a9a80bd4fb2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:06.832425 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2984/6033] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:18:06.840872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376d454-5ce4-455b-9bbe-bc89fa938588 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:06.857150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376d454-5ce4-455b-9bbe-bc89fa938588 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:06.862745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d73b7e-b575-4961-81c2-8f52893d8a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:06.863655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d73b7e-b575-4961-81c2-8f52893d8a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:06.864087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d73b7e-b575-4961-81c2-8f52893d8a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:06.864561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d73b7e-b575-4961-81c2-8f52893d8a91 admin admin] PATH_INFO: `/v3/projects/2bdf1d25b1e540e0bcf1c8ec5184c2ff/users/9262cad2fbc7453197bd2bb1ad70d159/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:06.893566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d73b7e-b575-4961-81c2-8f52893d8a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bdf1d25b1e540e0bcf1c8ec5184c2ff, user_id=9262cad2fbc7453197bd2bb1ad70d159, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:06.896547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d73b7e-b575-4961-81c2-8f52893d8a91 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:06.930675 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3050/6034] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:06 2022] PUT /identity/v3/projects/2bdf1d25b1e540e0bcf1c8ec5184c2ff/users/9262cad2fbc7453197bd2bb1ad70d159/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:06.933962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fdcbb-6caa-421a-ae8d-17327604671c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:06.934681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fdcbb-6caa-421a-ae8d-17327604671c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:06.934960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fdcbb-6caa-421a-ae8d-17327604671c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:06.995737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4e9fdcbb-6caa-421a-ae8d-17327604671c None None] MFA Rules not processed for user `9262cad2fbc7453197bd2bb1ad70d159`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:06.997629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4e9fdcbb-6caa-421a-ae8d-17327604671c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:07.042655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:07.155897 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2985/6035] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:07.173067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d73b7e-b575-4961-81c2-8f52893d8a91 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:07.177073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-74d73b7e-b575-4961-81c2-8f52893d8a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:07.389403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d73b7e-b575-4961-81c2-8f52893d8a91 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:07.398037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347a53c-6d95-4007-b021-4510ba94b7ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:07.398958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347a53c-6d95-4007-b021-4510ba94b7ab service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:07.399412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347a53c-6d95-4007-b021-4510ba94b7ab service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:07.399836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347a53c-6d95-4007-b021-4510ba94b7ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:07.429507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347a53c-6d95-4007-b021-4510ba94b7ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:07.431933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347a53c-6d95-4007-b021-4510ba94b7ab service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:07.473297 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3051/6036] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:07.494713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fdcbb-6caa-421a-ae8d-17327604671c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:07.611504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fdcbb-6caa-421a-ae8d-17327604671c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:07.617588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:07.618889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:07.619169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:07.619446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] PATH_INFO: `/v3/users/9262cad2fbc7453197bd2bb1ad70d159` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:07.621086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9262cad2fbc7453197bd2bb1ad70d159)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:07.623430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:07.699059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9262cad2fbc7453197bd2bb1ad70d159'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:07.716537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9262cad2fbc7453197bd2bb1ad70d159'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:07.732734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9262cad2fbc7453197bd2bb1ad70d159'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:07.733574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9262cad2fbc7453197bd2bb1ad70d159'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:07.734265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9262cad2fbc7453197bd2bb1ad70d159'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:07.753813 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2986/6037] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:07 2022] DELETE /identity/v3/users/9262cad2fbc7453197bd2bb1ad70d159 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:07.758402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8b8d9-442f-4be9-886e-149acf8c160e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:07.758848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8b8d9-442f-4be9-886e-149acf8c160e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:07.759124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8b8d9-442f-4be9-886e-149acf8c160e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:07.792146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-18a8b8d9-442f-4be9-886e-149acf8c160e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:07.793602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-18a8b8d9-442f-4be9-886e-149acf8c160e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:07.905501 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3052/6038] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:07.939824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:07.941278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:08.042567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:08.057055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfb64a-2c1a-4a09-af92-aa6c1d011685 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:08.062165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9a5c4-66a9-4a1a-b641-d1979148092c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:08.063155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9a5c4-66a9-4a1a-b641-d1979148092c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:08.063446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9a5c4-66a9-4a1a-b641-d1979148092c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:08.063768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9a5c4-66a9-4a1a-b641-d1979148092c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:08.087483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9a5c4-66a9-4a1a-b641-d1979148092c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:08.089269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9a5c4-66a9-4a1a-b641-d1979148092c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:08.134771 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2987/6039] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:09.343851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8b8d9-442f-4be9-886e-149acf8c160e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:09.345936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-18a8b8d9-442f-4be9-886e-149acf8c160e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:09.366356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8b8d9-442f-4be9-886e-149acf8c160e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:09.384745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a503c-bd8f-4f88-af2a-343bd5c2b116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:09.385820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a503c-bd8f-4f88-af2a-343bd5c2b116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:09.386411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a503c-bd8f-4f88-af2a-343bd5c2b116 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:09.386972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a503c-bd8f-4f88-af2a-343bd5c2b116 admin admin] PATH_INFO: `/v3/projects/2bdf1d25b1e540e0bcf1c8ec5184c2ff` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:09.395346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a503c-bd8f-4f88-af2a-343bd5c2b116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bdf1d25b1e540e0bcf1c8ec5184c2ff)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:09.398061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a503c-bd8f-4f88-af2a-343bd5c2b116 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:09.488347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:09.532001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e87a503c-bd8f-4f88-af2a-343bd5c2b116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bdf1d25b1e540e0bcf1c8ec5184c2ff', 'request_id': 'req-e87a503c-bd8f-4f88-af2a-343bd5c2b116'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:09.542949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e87a503c-bd8f-4f88-af2a-343bd5c2b116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bdf1d25b1e540e0bcf1c8ec5184c2ff', 'request_id': 'req-e87a503c-bd8f-4f88-af2a-343bd5c2b116'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:09.552224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e87a503c-bd8f-4f88-af2a-343bd5c2b116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bdf1d25b1e540e0bcf1c8ec5184c2ff', 'request_id': 'req-e87a503c-bd8f-4f88-af2a-343bd5c2b116'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:09.571269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e87a503c-bd8f-4f88-af2a-343bd5c2b116 admin admin] The token cache is being invalidate because project 2bdf1d25b1e540e0bcf1c8ec5184c2ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:09.571545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e87a503c-bd8f-4f88-af2a-343bd5c2b116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:09.573297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e87a503c-bd8f-4f88-af2a-343bd5c2b116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:09.575306 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3053/6040] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:09 2022] DELETE /identity/v3/projects/2bdf1d25b1e540e0bcf1c8ec5184c2ff => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:09.581783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f55a9-1941-4460-9781-c0d854f3dba6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:09.582763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f55a9-1941-4460-9781-c0d854f3dba6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:09.583713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f55a9-1941-4460-9781-c0d854f3dba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:09.607753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-616f55a9-1941-4460-9781-c0d854f3dba6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:09.609227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-616f55a9-1941-4460-9781-c0d854f3dba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:09.707260 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2988/6041] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:09.715771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb153df5-b650-4af9-9f2b-1c17fb495b3b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:09.716481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb153df5-b650-4af9-9f2b-1c17fb495b3b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:09.717323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb153df5-b650-4af9-9f2b-1c17fb495b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:09.773799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cb153df5-b650-4af9-9f2b-1c17fb495b3b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:09.777816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cb153df5-b650-4af9-9f2b-1c17fb495b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:09.788325 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3054/6042] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:09.793242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f55a9-1941-4460-9781-c0d854f3dba6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:09.817673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f55a9-1941-4460-9781-c0d854f3dba6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:09.831855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a7cdc-e47c-4965-aab4-dc417dd684dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:09.832591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a7cdc-e47c-4965-aab4-dc417dd684dd admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:09.832843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a7cdc-e47c-4965-aab4-dc417dd684dd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:09.833109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a7cdc-e47c-4965-aab4-dc417dd684dd admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:09.835417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a7cdc-e47c-4965-aab4-dc417dd684dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:18:09.836016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a7cdc-e47c-4965-aab4-dc417dd684dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:09.840042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a7cdc-e47c-4965-aab4-dc417dd684dd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:09.861794 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2989/6043] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:18:09.864445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb153df5-b650-4af9-9f2b-1c17fb495b3b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:09.876236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb153df5-b650-4af9-9f2b-1c17fb495b3b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:09.882741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0d40b-0619-48f6-af9c-19d9ba45c114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:09.883683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0d40b-0619-48f6-af9c-19d9ba45c114 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:09.883683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0d40b-0619-48f6-af9c-19d9ba45c114 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:09.884043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0d40b-0619-48f6-af9c-19d9ba45c114 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:09.884672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0d40b-0619-48f6-af9c-19d9ba45c114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:09.886810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0d40b-0619-48f6-af9c-19d9ba45c114 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:09.925481 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3055/6044] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:09 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:18:09.930207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a7cdc-e47c-4965-aab4-dc417dd684dd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:09.947000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a7cdc-e47c-4965-aab4-dc417dd684dd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:10.046443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab055393-bc13-4b44-99dc-f4a6d74d9546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:10.048226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab055393-bc13-4b44-99dc-f4a6d74d9546 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:10.048752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab055393-bc13-4b44-99dc-f4a6d74d9546 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:10.049227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab055393-bc13-4b44-99dc-f4a6d74d9546 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:10.050127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab055393-bc13-4b44-99dc-f4a6d74d9546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:10.053953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab055393-bc13-4b44-99dc-f4a6d74d9546 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:10.102466 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2990/6045] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:09 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:18:10.106209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0d40b-0619-48f6-af9c-19d9ba45c114 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:10.124707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0d40b-0619-48f6-af9c-19d9ba45c114 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:10.136805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30c7b1-4e6a-4048-8831-0c6d12befc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:10.136805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30c7b1-4e6a-4048-8831-0c6d12befc30 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:10.136805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30c7b1-4e6a-4048-8831-0c6d12befc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:10.136805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30c7b1-4e6a-4048-8831-0c6d12befc30 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:10.137901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30c7b1-4e6a-4048-8831-0c6d12befc30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:10.147836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30c7b1-4e6a-4048-8831-0c6d12befc30 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:10.172610 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3056/6046] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:10.173336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab055393-bc13-4b44-99dc-f4a6d74d9546 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:10.186509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab055393-bc13-4b44-99dc-f4a6d74d9546 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:10.208874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b71fbd-cc56-461a-9755-dbc30513ec34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:10.210045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b71fbd-cc56-461a-9755-dbc30513ec34 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:10.210619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b71fbd-cc56-461a-9755-dbc30513ec34 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:10.211607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b71fbd-cc56-461a-9755-dbc30513ec34 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:10.212847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b71fbd-cc56-461a-9755-dbc30513ec34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:10.214655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b71fbd-cc56-461a-9755-dbc30513ec34 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:10.270261 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2991/6047] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:10.271729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30c7b1-4e6a-4048-8831-0c6d12befc30 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:10.282914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30c7b1-4e6a-4048-8831-0c6d12befc30 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:10.287845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5ce8c-a994-48bb-8b4e-6d469be55c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:10.288834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5ce8c-a994-48bb-8b4e-6d469be55c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:10.289251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5ce8c-a994-48bb-8b4e-6d469be55c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:10.289605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5ce8c-a994-48bb-8b4e-6d469be55c81 admin admin] PATH_INFO: `/v3/projects/d807d627277140bda6d1cb7f2ef6c7fb/users/7ecba3c3de1647f88dee89c8259119ec/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:10.308483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5ce8c-a994-48bb-8b4e-6d469be55c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d807d627277140bda6d1cb7f2ef6c7fb, user_id=7ecba3c3de1647f88dee89c8259119ec, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:10.312660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5ce8c-a994-48bb-8b4e-6d469be55c81 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:10.342386 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3057/6048] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:10 2022] PUT /identity/v3/projects/d807d627277140bda6d1cb7f2ef6c7fb/users/7ecba3c3de1647f88dee89c8259119ec/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:18:10.345063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac657dfa-7dc3-4951-b9d0-0efd1bbadb8f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:10.345644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac657dfa-7dc3-4951-b9d0-0efd1bbadb8f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:10.345784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac657dfa-7dc3-4951-b9d0-0efd1bbadb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:10.392741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ac657dfa-7dc3-4951-b9d0-0efd1bbadb8f None None] MFA Rules not processed for user `7ecba3c3de1647f88dee89c8259119ec`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:10.394909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ac657dfa-7dc3-4951-b9d0-0efd1bbadb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:10.458372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:10.490254 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2992/6049] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:10.504703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5ce8c-a994-48bb-8b4e-6d469be55c81 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:10.507062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e0a5ce8c-a994-48bb-8b4e-6d469be55c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:10.663186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5ce8c-a994-48bb-8b4e-6d469be55c81 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:10.674843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8656c-b43f-4640-b4fb-5e96841f59e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:10.675651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8656c-b43f-4640-b4fb-5e96841f59e2 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:10.676610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8656c-b43f-4640-b4fb-5e96841f59e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:10.678378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8656c-b43f-4640-b4fb-5e96841f59e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:10.705550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8656c-b43f-4640-b4fb-5e96841f59e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:10.721390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8656c-b43f-4640-b4fb-5e96841f59e2 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:10.748640 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3058/6050] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:12.923793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac657dfa-7dc3-4951-b9d0-0efd1bbadb8f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:13.107840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac657dfa-7dc3-4951-b9d0-0efd1bbadb8f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:13.117549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08137449-ba03-4deb-8c02-bc3c3d9816fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:13.118213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08137449-ba03-4deb-8c02-bc3c3d9816fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:13.118484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08137449-ba03-4deb-8c02-bc3c3d9816fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:13.118747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08137449-ba03-4deb-8c02-bc3c3d9816fa admin admin] PATH_INFO: `/v3/users/7ecba3c3de1647f88dee89c8259119ec` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:13.119923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08137449-ba03-4deb-8c02-bc3c3d9816fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ecba3c3de1647f88dee89c8259119ec)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:13.121906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08137449-ba03-4deb-8c02-bc3c3d9816fa admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:13.285286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-08137449-ba03-4deb-8c02-bc3c3d9816fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ecba3c3de1647f88dee89c8259119ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:13.303181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a2a95-b57f-44c3-86e9-45b54a734a13 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:13.303894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a2a95-b57f-44c3-86e9-45b54a734a13 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:13.303894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a2a95-b57f-44c3-86e9-45b54a734a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:13.309757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-08137449-ba03-4deb-8c02-bc3c3d9816fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ecba3c3de1647f88dee89c8259119ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:13.346504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-014a2a95-b57f-44c3-86e9-45b54a734a13 None None] MFA Rules not processed for user `7013eec1908f407d9949d271287b1570`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:13.347998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-014a2a95-b57f-44c3-86e9-45b54a734a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:13.359618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-08137449-ba03-4deb-8c02-bc3c3d9816fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ecba3c3de1647f88dee89c8259119ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:13.360722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-08137449-ba03-4deb-8c02-bc3c3d9816fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ecba3c3de1647f88dee89c8259119ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:13.362901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-08137449-ba03-4deb-8c02-bc3c3d9816fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ecba3c3de1647f88dee89c8259119ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:13.390192 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2993/6051] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:12 2022] DELETE /identity/v3/users/7ecba3c3de1647f88dee89c8259119ec => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:13.393739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3276ad9-e3e4-4e0d-a94f-2aad9a2b0706 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:13.394017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3276ad9-e3e4-4e0d-a94f-2aad9a2b0706 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:13.394278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3276ad9-e3e4-4e0d-a94f-2aad9a2b0706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:13.412058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:13.431260 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3059/6052] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:18:13.447813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a3276ad9-e3e4-4e0d-a94f-2aad9a2b0706 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:13.449064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a3276ad9-e3e4-4e0d-a94f-2aad9a2b0706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:13.578868 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2994/6053] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:13.590800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a2a95-b57f-44c3-86e9-45b54a734a13 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:13.592553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-014a2a95-b57f-44c3-86e9-45b54a734a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:13.651121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3276ad9-e3e4-4e0d-a94f-2aad9a2b0706 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:13.652948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a3276ad9-e3e4-4e0d-a94f-2aad9a2b0706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:13.685599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:13.769164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a2a95-b57f-44c3-86e9-45b54a734a13 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:13.913145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3276ad9-e3e4-4e0d-a94f-2aad9a2b0706 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:13.917551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d98ac-e5ac-4a44-9991-d5c4217c87fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:13.918561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d98ac-e5ac-4a44-9991-d5c4217c87fd service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:13.919068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d98ac-e5ac-4a44-9991-d5c4217c87fd service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:13.919544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d98ac-e5ac-4a44-9991-d5c4217c87fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:13.926048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f56575-aebc-4710-b6fd-54ba7b289182 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:13.926759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f56575-aebc-4710-b6fd-54ba7b289182 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:13.926938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f56575-aebc-4710-b6fd-54ba7b289182 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:13.927199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f56575-aebc-4710-b6fd-54ba7b289182 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:13.943846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d98ac-e5ac-4a44-9991-d5c4217c87fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:13.947370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d98ac-e5ac-4a44-9991-d5c4217c87fd service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:13.975962 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3060/6054] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:14.083497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f56575-aebc-4710-b6fd-54ba7b289182 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:14.090291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f56575-aebc-4710-b6fd-54ba7b289182 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:14.112709 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2995/6055] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:18:15.034765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d98ac-e5ac-4a44-9991-d5c4217c87fd service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:15.036928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c18d98ac-e5ac-4a44-9991-d5c4217c87fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:15.063393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d98ac-e5ac-4a44-9991-d5c4217c87fd service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:15.080468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:15.081862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:15.082145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:15.082413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9 admin admin] PATH_INFO: `/v3/projects/d807d627277140bda6d1cb7f2ef6c7fb` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:15.089750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d807d627277140bda6d1cb7f2ef6c7fb)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:15.091976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:15.213980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd807d627277140bda6d1cb7f2ef6c7fb', 'request_id': 'req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:15.235030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd807d627277140bda6d1cb7f2ef6c7fb', 'request_id': 'req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:15.242429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd807d627277140bda6d1cb7f2ef6c7fb', 'request_id': 'req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:15.273480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9 admin admin] The token cache is being invalidate because project d807d627277140bda6d1cb7f2ef6c7fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:15.274104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:15.274463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0ebe74fc-ae74-44a1-bb95-cb55bd55a6f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:15.276691 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3061/6056] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:15 2022] DELETE /identity/v3/projects/d807d627277140bda6d1cb7f2ef6c7fb => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:18:15.285990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d02ec0-39a8-487a-b207-7c0cdfaf56d9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:15.287660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d02ec0-39a8-487a-b207-7c0cdfaf56d9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:15.287934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d02ec0-39a8-487a-b207-7c0cdfaf56d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:15.327081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-42d02ec0-39a8-487a-b207-7c0cdfaf56d9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:15.328484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-42d02ec0-39a8-487a-b207-7c0cdfaf56d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:15.466502 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2996/6057] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:15.468477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e398e-b043-4933-b19a-02fce14a7a16 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:15.468853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e398e-b043-4933-b19a-02fce14a7a16 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:15.469213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e398e-b043-4933-b19a-02fce14a7a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:15.508257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-588e398e-b043-4933-b19a-02fce14a7a16 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:15.509764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-588e398e-b043-4933-b19a-02fce14a7a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:15.522583 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3062/6058] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:15.524091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d02ec0-39a8-487a-b207-7c0cdfaf56d9 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:15.549240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d02ec0-39a8-487a-b207-7c0cdfaf56d9 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:15.558105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b948f-17a9-4f83-9072-f81a46d5df4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:15.558798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b948f-17a9-4f83-9072-f81a46d5df4a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:15.559078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b948f-17a9-4f83-9072-f81a46d5df4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:15.559389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b948f-17a9-4f83-9072-f81a46d5df4a admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:15.560111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b948f-17a9-4f83-9072-f81a46d5df4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:18:15.560449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b948f-17a9-4f83-9072-f81a46d5df4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:15.562197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b948f-17a9-4f83-9072-f81a46d5df4a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:15.583541 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2997/6059] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:18:15.587490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e398e-b043-4933-b19a-02fce14a7a16 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:15.602379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e398e-b043-4933-b19a-02fce14a7a16 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:15.609631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb0604-b78a-48f4-9a9d-f893ca0027a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:15.610312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb0604-b78a-48f4-9a9d-f893ca0027a4 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:15.610590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb0604-b78a-48f4-9a9d-f893ca0027a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:15.610859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb0604-b78a-48f4-9a9d-f893ca0027a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:15.611549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb0604-b78a-48f4-9a9d-f893ca0027a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:15.613942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb0604-b78a-48f4-9a9d-f893ca0027a4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:15.664010 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3063/6060] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:15 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:15.665474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b948f-17a9-4f83-9072-f81a46d5df4a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:15.679509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b948f-17a9-4f83-9072-f81a46d5df4a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:15.744542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:15.791616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aab793-2b64-4d7c-9b26-f11fe8dff8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:15.792718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aab793-2b64-4d7c-9b26-f11fe8dff8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:15.793166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aab793-2b64-4d7c-9b26-f11fe8dff8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:15.793687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aab793-2b64-4d7c-9b26-f11fe8dff8a7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:15.794665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aab793-2b64-4d7c-9b26-f11fe8dff8a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:15.797997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aab793-2b64-4d7c-9b26-f11fe8dff8a7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:15.844653 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2998/6061] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:15 2022] POST /identity/v3/users => generated 287 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:15.848519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb0604-b78a-48f4-9a9d-f893ca0027a4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:15.869663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb0604-b78a-48f4-9a9d-f893ca0027a4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:15.892872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5b1c1-2c1c-461f-860e-6d39ee1150e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:15.893565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5b1c1-2c1c-461f-860e-6d39ee1150e0 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:15.893907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5b1c1-2c1c-461f-860e-6d39ee1150e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:15.894242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5b1c1-2c1c-461f-860e-6d39ee1150e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:15.894948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5b1c1-2c1c-461f-860e-6d39ee1150e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:15.896653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5b1c1-2c1c-461f-860e-6d39ee1150e0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:15.907457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:15.929083 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3064/6062] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:18:15.933123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aab793-2b64-4d7c-9b26-f11fe8dff8a7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:15.949373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aab793-2b64-4d7c-9b26-f11fe8dff8a7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:15.963681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cfc41-0bbe-4ed5-bc46-8c944572fe46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:15.964317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cfc41-0bbe-4ed5-bc46-8c944572fe46 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:15.964627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cfc41-0bbe-4ed5-bc46-8c944572fe46 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:15.964897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cfc41-0bbe-4ed5-bc46-8c944572fe46 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:15.965552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cfc41-0bbe-4ed5-bc46-8c944572fe46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:15.967189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cfc41-0bbe-4ed5-bc46-8c944572fe46 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:16.009423 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 2999/6063] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:16.014026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5b1c1-2c1c-461f-860e-6d39ee1150e0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:16.026834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5b1c1-2c1c-461f-860e-6d39ee1150e0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:16.032055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f39f06-074f-4d83-be74-590af1ac78cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:16.032922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f39f06-074f-4d83-be74-590af1ac78cb admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:16.033293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f39f06-074f-4d83-be74-590af1ac78cb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:16.033675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f39f06-074f-4d83-be74-590af1ac78cb admin admin] PATH_INFO: `/v3/projects/bafce9b915bf41ac950461fa6751f1a1/users/c3f90ffe3ef64ffa869985abf7542002/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:16.051198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f39f06-074f-4d83-be74-590af1ac78cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bafce9b915bf41ac950461fa6751f1a1, user_id=c3f90ffe3ef64ffa869985abf7542002, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:16.055024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f39f06-074f-4d83-be74-590af1ac78cb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:16.083077 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3065/6064] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:16 2022] PUT /identity/v3/projects/bafce9b915bf41ac950461fa6751f1a1/users/c3f90ffe3ef64ffa869985abf7542002/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:16.094382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a173e3-b33d-4b9e-a1a5-9823faffd9ee None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:16.097627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a173e3-b33d-4b9e-a1a5-9823faffd9ee None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:16.098151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a173e3-b33d-4b9e-a1a5-9823faffd9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:16.164886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-99a173e3-b33d-4b9e-a1a5-9823faffd9ee None None] MFA Rules not processed for user `c3f90ffe3ef64ffa869985abf7542002`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:16.166284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-99a173e3-b33d-4b9e-a1a5-9823faffd9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:16.304040 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3000/6065] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:16.308876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f39f06-074f-4d83-be74-590af1ac78cb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:16.420623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f39f06-074f-4d83-be74-590af1ac78cb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:16.436127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef70ce-a580-41fb-827a-104397a7df6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:16.437479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef70ce-a580-41fb-827a-104397a7df6e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:16.437911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef70ce-a580-41fb-827a-104397a7df6e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:16.438514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef70ce-a580-41fb-827a-104397a7df6e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:16.439539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef70ce-a580-41fb-827a-104397a7df6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:16.442168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef70ce-a580-41fb-827a-104397a7df6e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:16.484885 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3066/6066] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:16 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:16.488717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a173e3-b33d-4b9e-a1a5-9823faffd9ee None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:16.531772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a173e3-b33d-4b9e-a1a5-9823faffd9ee None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:16.706447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a6efb-6805-40de-85de-c5e4398d13fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:16.716181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a6efb-6805-40de-85de-c5e4398d13fa admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:16.717300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a6efb-6805-40de-85de-c5e4398d13fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:16.717677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a6efb-6805-40de-85de-c5e4398d13fa admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:16.718765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a6efb-6805-40de-85de-c5e4398d13fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:16.722347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a6efb-6805-40de-85de-c5e4398d13fa admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:16.805248 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3001/6067] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:16 2022] POST /identity/v3/users => generated 293 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:16.809527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef70ce-a580-41fb-827a-104397a7df6e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:16.843044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef70ce-a580-41fb-827a-104397a7df6e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:16.857519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31c05e-0c3f-4e99-aef0-cd43d74e4383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:16.858572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31c05e-0c3f-4e99-aef0-cd43d74e4383 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:16.859326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31c05e-0c3f-4e99-aef0-cd43d74e4383 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:16.859600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31c05e-0c3f-4e99-aef0-cd43d74e4383 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:16.860285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31c05e-0c3f-4e99-aef0-cd43d74e4383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:16.862251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31c05e-0c3f-4e99-aef0-cd43d74e4383 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:16.878204 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3067/6068] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:16.882379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a6efb-6805-40de-85de-c5e4398d13fa admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:16.904488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a6efb-6805-40de-85de-c5e4398d13fa admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:16.909525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf49442-672c-4494-97da-deb4050cac1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:16.910206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf49442-672c-4494-97da-deb4050cac1f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:16.910482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf49442-672c-4494-97da-deb4050cac1f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:16.910751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf49442-672c-4494-97da-deb4050cac1f admin admin] PATH_INFO: `/v3/projects/d95df16633454f6ebbbf454796641782/users/05f7061da9dc4443ad876e52cb267e17/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:16.953593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf49442-672c-4494-97da-deb4050cac1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d95df16633454f6ebbbf454796641782, user_id=05f7061da9dc4443ad876e52cb267e17, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:16.959362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf49442-672c-4494-97da-deb4050cac1f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:16.987693 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3002/6069] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:16 2022] PUT /identity/v3/projects/d95df16633454f6ebbbf454796641782/users/05f7061da9dc4443ad876e52cb267e17/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:16.991717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31c05e-0c3f-4e99-aef0-cd43d74e4383 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:17.005392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31c05e-0c3f-4e99-aef0-cd43d74e4383 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:17.111482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0f760-188b-403c-8a44-1fe13a95837a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:17.111764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0f760-188b-403c-8a44-1fe13a95837a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:17.112036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0f760-188b-403c-8a44-1fe13a95837a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:17.120945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96cc56-e3db-4b9d-93f4-c82b1e358299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:17.121527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:17.122412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96cc56-e3db-4b9d-93f4-c82b1e358299 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:17.122882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96cc56-e3db-4b9d-93f4-c82b1e358299 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:17.123303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96cc56-e3db-4b9d-93f4-c82b1e358299 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:17.124267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96cc56-e3db-4b9d-93f4-c82b1e358299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:17.126828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96cc56-e3db-4b9d-93f4-c82b1e358299 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:17.142749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c9c0f760-188b-403c-8a44-1fe13a95837a None None] MFA Rules not processed for user `7013eec1908f407d9949d271287b1570`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:17.143465 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3068/6070] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:17.143851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c9c0f760-188b-403c-8a44-1fe13a95837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:17.144728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96cc56-e3db-4b9d-93f4-c82b1e358299 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:17.155528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96cc56-e3db-4b9d-93f4-c82b1e358299 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:17.160591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad83236-3eb2-4487-bce8-6380778ed641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:17.161466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad83236-3eb2-4487-bce8-6380778ed641 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:17.161897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad83236-3eb2-4487-bce8-6380778ed641 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:17.162359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad83236-3eb2-4487-bce8-6380778ed641 admin admin] PATH_INFO: `/v3/projects/d95df16633454f6ebbbf454796641782/users/05f7061da9dc4443ad876e52cb267e17/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:17.164699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad83236-3eb2-4487-bce8-6380778ed641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d95df16633454f6ebbbf454796641782, user_id=05f7061da9dc4443ad876e52cb267e17, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:17.167636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad83236-3eb2-4487-bce8-6380778ed641 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:17.191617 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3069/6071] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:17 2022] PUT /identity/v3/projects/d95df16633454f6ebbbf454796641782/users/05f7061da9dc4443ad876e52cb267e17/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:17.205205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21468163-b8be-4cfb-a72e-d248312f82d7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:17.205668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21468163-b8be-4cfb-a72e-d248312f82d7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:17.206091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21468163-b8be-4cfb-a72e-d248312f82d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:17.237459 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3003/6072] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:18:17.269196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-21468163-b8be-4cfb-a72e-d248312f82d7 None None] MFA Rules not processed for user `05f7061da9dc4443ad876e52cb267e17`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:17.271724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-21468163-b8be-4cfb-a72e-d248312f82d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:17.302623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:17.377269 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3070/6073] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:17.391914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0f760-188b-403c-8a44-1fe13a95837a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:17.394633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c9c0f760-188b-403c-8a44-1fe13a95837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:17.501431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0f760-188b-403c-8a44-1fe13a95837a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:17.583240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e467e-379d-4930-9c95-43a664e40487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:17.583958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e467e-379d-4930-9c95-43a664e40487 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:17.584234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e467e-379d-4930-9c95-43a664e40487 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:17.584533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e467e-379d-4930-9c95-43a664e40487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:17.685958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e467e-379d-4930-9c95-43a664e40487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:17.688462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e467e-379d-4930-9c95-43a664e40487 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:17.701198 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3004/6074] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:19.577218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21468163-b8be-4cfb-a72e-d248312f82d7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:19.697452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21468163-b8be-4cfb-a72e-d248312f82d7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:19.703608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:19.704309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:19.704773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:19.705223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] PATH_INFO: `/v3/users/c3f90ffe3ef64ffa869985abf7542002` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:19.706604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f90ffe3ef64ffa869985abf7542002)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:19.708963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:19.808907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f90ffe3ef64ffa869985abf7542002'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:19.824198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f90ffe3ef64ffa869985abf7542002'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:19.849226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f90ffe3ef64ffa869985abf7542002'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:19.898065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f90ffe3ef64ffa869985abf7542002'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:19.899380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f90ffe3ef64ffa869985abf7542002'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:19.921007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce82e93-0e7e-4367-a8a9-8b5f9d3b08f4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:19.921808 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3071/6075] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:19 2022] DELETE /identity/v3/users/c3f90ffe3ef64ffa869985abf7542002 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:19.922018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce82e93-0e7e-4367-a8a9-8b5f9d3b08f4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:19.922018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce82e93-0e7e-4367-a8a9-8b5f9d3b08f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:19.970237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3ce82e93-0e7e-4367-a8a9-8b5f9d3b08f4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:19.971411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3ce82e93-0e7e-4367-a8a9-8b5f9d3b08f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:20.010989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:20.128425 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3005/6076] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:18:20.141165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:20.142813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:20.188946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:20.232129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d5922-c97c-4a60-be68-4b76f499d2a6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:20.237634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0118a1-213f-4ad4-b4fb-8cd25a319b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:20.238355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0118a1-213f-4ad4-b4fb-8cd25a319b55 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:20.238630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0118a1-213f-4ad4-b4fb-8cd25a319b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:20.238896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0118a1-213f-4ad4-b4fb-8cd25a319b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:20.250538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0118a1-213f-4ad4-b4fb-8cd25a319b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:20.252322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0118a1-213f-4ad4-b4fb-8cd25a319b55 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:20.272337 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3072/6077] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:20.635088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce82e93-0e7e-4367-a8a9-8b5f9d3b08f4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:20.637194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3ce82e93-0e7e-4367-a8a9-8b5f9d3b08f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:20.660510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce82e93-0e7e-4367-a8a9-8b5f9d3b08f4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:20.667469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:20.668205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:20.668611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:20.669423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] PATH_INFO: `/v3/projects/bafce9b915bf41ac950461fa6751f1a1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:20.671553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bafce9b915bf41ac950461fa6751f1a1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:20.674241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:20.761760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bafce9b915bf41ac950461fa6751f1a1', 'request_id': 'req-4c942485-767b-4a6c-beae-8c1e65936f6c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:20.774578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bafce9b915bf41ac950461fa6751f1a1', 'request_id': 'req-4c942485-767b-4a6c-beae-8c1e65936f6c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:20.791068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bafce9b915bf41ac950461fa6751f1a1', 'request_id': 'req-4c942485-767b-4a6c-beae-8c1e65936f6c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:20.818262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] The token cache is being invalidate because project bafce9b915bf41ac950461fa6751f1a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:20.818625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:20.828426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:20.828426 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3006/6078] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:20 2022] DELETE /identity/v3/projects/bafce9b915bf41ac950461fa6751f1a1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:18:20.828809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0118a1-213f-4ad4-b4fb-8cd25a319b55 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:20.839948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0118a1-213f-4ad4-b4fb-8cd25a319b55 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:20.938209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd46fd-9266-4e31-8906-1b7a790ce570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:20.940157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd46fd-9266-4e31-8906-1b7a790ce570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:20.940579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd46fd-9266-4e31-8906-1b7a790ce570 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:20.940846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd46fd-9266-4e31-8906-1b7a790ce570 admin admin] PATH_INFO: `/v3/users/05f7061da9dc4443ad876e52cb267e17` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:20.942321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd46fd-9266-4e31-8906-1b7a790ce570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05f7061da9dc4443ad876e52cb267e17)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:20.945522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd46fd-9266-4e31-8906-1b7a790ce570 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:21.016752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d5fd46fd-9266-4e31-8906-1b7a790ce570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05f7061da9dc4443ad876e52cb267e17'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:21.026682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d5fd46fd-9266-4e31-8906-1b7a790ce570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05f7061da9dc4443ad876e52cb267e17'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:21.035505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d5fd46fd-9266-4e31-8906-1b7a790ce570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05f7061da9dc4443ad876e52cb267e17'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:21.048902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d5fd46fd-9266-4e31-8906-1b7a790ce570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05f7061da9dc4443ad876e52cb267e17'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:21.049775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d5fd46fd-9266-4e31-8906-1b7a790ce570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05f7061da9dc4443ad876e52cb267e17'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:21.058218 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3073/6079] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:20 2022] DELETE /identity/v3/users/05f7061da9dc4443ad876e52cb267e17 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:21.369574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:21.375323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:21.394418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c942485-767b-4a6c-beae-8c1e65936f6c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:21.484345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:21.485419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:21.485910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:21.486241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5 admin admin] PATH_INFO: `/v3/projects/d95df16633454f6ebbbf454796641782` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:21.487475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d95df16633454f6ebbbf454796641782)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:21.489633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:21.539606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:21.546846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd95df16633454f6ebbbf454796641782', 'request_id': 'req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:21.555730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd95df16633454f6ebbbf454796641782', 'request_id': 'req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:21.567746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd95df16633454f6ebbbf454796641782', 'request_id': 'req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:21.580886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5 admin admin] The token cache is being invalidate because project d95df16633454f6ebbbf454796641782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:21.581412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:21.582165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-11d80a14-e3f7-49ae-a5aa-33dcd9fafbc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:21.584099 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3007/6080] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:21 2022] DELETE /identity/v3/projects/d95df16633454f6ebbbf454796641782 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:21.591874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb16c4b-3772-4251-9d20-37266d877e2a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:21.592150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb16c4b-3772-4251-9d20-37266d877e2a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:21.592445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb16c4b-3772-4251-9d20-37266d877e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:21.614469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-feb16c4b-3772-4251-9d20-37266d877e2a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:21.615727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-feb16c4b-3772-4251-9d20-37266d877e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:21.687326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:21.703173 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3074/6081] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:21.711755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0687e-2fa2-467c-8a15-8d5ed7c2e85d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:21.712036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0687e-2fa2-467c-8a15-8d5ed7c2e85d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:21.712305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0687e-2fa2-467c-8a15-8d5ed7c2e85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:21.738321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6be0687e-2fa2-467c-8a15-8d5ed7c2e85d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:21.739751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6be0687e-2fa2-467c-8a15-8d5ed7c2e85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:21.748563 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3008/6082] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:21.752806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb16c4b-3772-4251-9d20-37266d877e2a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:21.763399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb16c4b-3772-4251-9d20-37266d877e2a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:21.769722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845a1ac-8f87-4f6e-b41f-b386778be09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:21.770365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845a1ac-8f87-4f6e-b41f-b386778be09e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:21.770659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845a1ac-8f87-4f6e-b41f-b386778be09e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:21.770906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845a1ac-8f87-4f6e-b41f-b386778be09e admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:21.771707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845a1ac-8f87-4f6e-b41f-b386778be09e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:18:21.772479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845a1ac-8f87-4f6e-b41f-b386778be09e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:21.774240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845a1ac-8f87-4f6e-b41f-b386778be09e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:21.788350 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3075/6083] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:18:21.792745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0687e-2fa2-467c-8a15-8d5ed7c2e85d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:21.807076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0687e-2fa2-467c-8a15-8d5ed7c2e85d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:21.812175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837a748-e00b-43dc-b088-2b0f5ee8dcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:21.813170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837a748-e00b-43dc-b088-2b0f5ee8dcb7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:21.813170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837a748-e00b-43dc-b088-2b0f5ee8dcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:21.813419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837a748-e00b-43dc-b088-2b0f5ee8dcb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:21.813999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837a748-e00b-43dc-b088-2b0f5ee8dcb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:21.816114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837a748-e00b-43dc-b088-2b0f5ee8dcb7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:21.857442 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3009/6084] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:21 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:21.858395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845a1ac-8f87-4f6e-b41f-b386778be09e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:21.871087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845a1ac-8f87-4f6e-b41f-b386778be09e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:21.978831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcbcdd-cb5c-43c0-a96a-2fb543798d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:21.979456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcbcdd-cb5c-43c0-a96a-2fb543798d39 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:21.979735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcbcdd-cb5c-43c0-a96a-2fb543798d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:21.980007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcbcdd-cb5c-43c0-a96a-2fb543798d39 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:21.980679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcbcdd-cb5c-43c0-a96a-2fb543798d39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:21.984857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcbcdd-cb5c-43c0-a96a-2fb543798d39 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:22.038068 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3076/6085] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:21 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:22.064590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837a748-e00b-43dc-b088-2b0f5ee8dcb7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:22.065707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837a748-e00b-43dc-b088-2b0f5ee8dcb7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:22.071400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67a089-c711-4bbb-acb8-d4126394629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:22.072150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67a089-c711-4bbb-acb8-d4126394629e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:22.072273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67a089-c711-4bbb-acb8-d4126394629e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:22.072726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67a089-c711-4bbb-acb8-d4126394629e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:22.073480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67a089-c711-4bbb-acb8-d4126394629e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:22.075128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67a089-c711-4bbb-acb8-d4126394629e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:22.096717 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3010/6086] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:22.100415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcbcdd-cb5c-43c0-a96a-2fb543798d39 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:22.120787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcbcdd-cb5c-43c0-a96a-2fb543798d39 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:22.137430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c085248-f913-4011-8107-b896f505caf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:22.139229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c085248-f913-4011-8107-b896f505caf9 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:22.139229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c085248-f913-4011-8107-b896f505caf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:22.139229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c085248-f913-4011-8107-b896f505caf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:22.140283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c085248-f913-4011-8107-b896f505caf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:22.149425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c085248-f913-4011-8107-b896f505caf9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:22.168012 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3077/6087] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:22.171788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67a089-c711-4bbb-acb8-d4126394629e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:22.193409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67a089-c711-4bbb-acb8-d4126394629e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:22.206347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9db72-54d0-44c4-abfa-da54294a4962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:22.207461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9db72-54d0-44c4-abfa-da54294a4962 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:22.207797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9db72-54d0-44c4-abfa-da54294a4962 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:22.208620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9db72-54d0-44c4-abfa-da54294a4962 admin admin] PATH_INFO: `/v3/projects/d4453a910474400cbd05eaafd72b9635/users/992dbf65374343d59e1f22e00cb75c0d/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:22.257823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9db72-54d0-44c4-abfa-da54294a4962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4453a910474400cbd05eaafd72b9635, user_id=992dbf65374343d59e1f22e00cb75c0d, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:22.261896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9db72-54d0-44c4-abfa-da54294a4962 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:22.284784 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3011/6088] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:22 2022] PUT /identity/v3/projects/d4453a910474400cbd05eaafd72b9635/users/992dbf65374343d59e1f22e00cb75c0d/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:22.289832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad1a8c-2d9f-4b3a-b3fe-497afe6c3caa None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:22.290114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad1a8c-2d9f-4b3a-b3fe-497afe6c3caa None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:22.290383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad1a8c-2d9f-4b3a-b3fe-497afe6c3caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:22.326957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bcad1a8c-2d9f-4b3a-b3fe-497afe6c3caa None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:22.328590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bcad1a8c-2d9f-4b3a-b3fe-497afe6c3caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:22.416991 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3078/6089] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:22.428507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9db72-54d0-44c4-abfa-da54294a4962 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:22.430766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e5c9db72-54d0-44c4-abfa-da54294a4962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:22.524252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9db72-54d0-44c4-abfa-da54294a4962 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:22.529413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1b72b-31b6-4d7b-add9-1b14d0f865c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:22.530337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1b72b-31b6-4d7b-add9-1b14d0f865c0 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:22.530803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1b72b-31b6-4d7b-add9-1b14d0f865c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:22.531670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1b72b-31b6-4d7b-add9-1b14d0f865c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:22.547860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1b72b-31b6-4d7b-add9-1b14d0f865c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:22.550911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1b72b-31b6-4d7b-add9-1b14d0f865c0 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:22.599351 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3012/6090] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:24.441589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1759b0-3cde-48ea-b2ed-eb44ec940008 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:24.442819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1759b0-3cde-48ea-b2ed-eb44ec940008 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:24.443315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1759b0-3cde-48ea-b2ed-eb44ec940008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:24.510774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cb1759b0-3cde-48ea-b2ed-eb44ec940008 None None] MFA Rules not processed for user `992dbf65374343d59e1f22e00cb75c0d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:24.515782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cb1759b0-3cde-48ea-b2ed-eb44ec940008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:24.657897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:24.680007 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3079/6091] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:24.696684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1b72b-31b6-4d7b-add9-1b14d0f865c0 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:24.699076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-77d1b72b-31b6-4d7b-add9-1b14d0f865c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:24.837903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1b72b-31b6-4d7b-add9-1b14d0f865c0 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:24.845530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af9221-5619-4b55-abed-03984fd298b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:24.846166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af9221-5619-4b55-abed-03984fd298b0 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:24.846491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af9221-5619-4b55-abed-03984fd298b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:24.846758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af9221-5619-4b55-abed-03984fd298b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:24.878055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af9221-5619-4b55-abed-03984fd298b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:24.879907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af9221-5619-4b55-abed-03984fd298b0 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:24.914643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a087-d448-4723-85ca-83dbc1284a07 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:24.915025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a087-d448-4723-85ca-83dbc1284a07 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:24.915413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a087-d448-4723-85ca-83dbc1284a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:24.918548 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3013/6092] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:25.012470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c345a087-d448-4723-85ca-83dbc1284a07 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:25.014088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c345a087-d448-4723-85ca-83dbc1284a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:25.113596 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3080/6093] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:18:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:25.185158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af9221-5619-4b55-abed-03984fd298b0 service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:25.186669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-15af9221-5619-4b55-abed-03984fd298b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:25.211611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:25.355045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af9221-5619-4b55-abed-03984fd298b0 service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:25.365960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0216c6c-c856-4bf3-924c-6b696b6b8ee3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:25.367022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0216c6c-c856-4bf3-924c-6b696b6b8ee3 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:25.367356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0216c6c-c856-4bf3-924c-6b696b6b8ee3 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:25.367824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0216c6c-c856-4bf3-924c-6b696b6b8ee3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:25.388069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0216c6c-c856-4bf3-924c-6b696b6b8ee3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:25.395772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0216c6c-c856-4bf3-924c-6b696b6b8ee3 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:25.421387 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3014/6094] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:18:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:18:28.496732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345a087-d448-4723-85ca-83dbc1284a07 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:28.514312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345a087-d448-4723-85ca-83dbc1284a07 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:28.520071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abc0bd-7e44-4b70-a7b5-54ec47f23911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:28.522767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abc0bd-7e44-4b70-a7b5-54ec47f23911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:28.523400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abc0bd-7e44-4b70-a7b5-54ec47f23911 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:28.523875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abc0bd-7e44-4b70-a7b5-54ec47f23911 admin admin] PATH_INFO: `/v3/users/992dbf65374343d59e1f22e00cb75c0d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:28.525371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abc0bd-7e44-4b70-a7b5-54ec47f23911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=992dbf65374343d59e1f22e00cb75c0d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:28.529021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abc0bd-7e44-4b70-a7b5-54ec47f23911 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:28.608502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8abc0bd-7e44-4b70-a7b5-54ec47f23911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '992dbf65374343d59e1f22e00cb75c0d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:28.619320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8abc0bd-7e44-4b70-a7b5-54ec47f23911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '992dbf65374343d59e1f22e00cb75c0d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:28.628615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8abc0bd-7e44-4b70-a7b5-54ec47f23911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '992dbf65374343d59e1f22e00cb75c0d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:28.643694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8abc0bd-7e44-4b70-a7b5-54ec47f23911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '992dbf65374343d59e1f22e00cb75c0d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:28.644090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f8abc0bd-7e44-4b70-a7b5-54ec47f23911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '992dbf65374343d59e1f22e00cb75c0d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:28.655310 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3081/6095] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:28 2022] DELETE /identity/v3/users/992dbf65374343d59e1f22e00cb75c0d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:29.125657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0216c6c-c856-4bf3-924c-6b696b6b8ee3 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:29.152726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0216c6c-c856-4bf3-924c-6b696b6b8ee3 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:29.273034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37467e8-b012-4dbe-8ae1-d7caf43354e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:29.274039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37467e8-b012-4dbe-8ae1-d7caf43354e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:29.274588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37467e8-b012-4dbe-8ae1-d7caf43354e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:29.275519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37467e8-b012-4dbe-8ae1-d7caf43354e6 admin admin] PATH_INFO: `/v3/projects/d4453a910474400cbd05eaafd72b9635` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:29.292982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37467e8-b012-4dbe-8ae1-d7caf43354e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4453a910474400cbd05eaafd72b9635)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:29.299460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37467e8-b012-4dbe-8ae1-d7caf43354e6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:29.425653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f37467e8-b012-4dbe-8ae1-d7caf43354e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4453a910474400cbd05eaafd72b9635', 'request_id': 'req-f37467e8-b012-4dbe-8ae1-d7caf43354e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:29.442754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f37467e8-b012-4dbe-8ae1-d7caf43354e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4453a910474400cbd05eaafd72b9635', 'request_id': 'req-f37467e8-b012-4dbe-8ae1-d7caf43354e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:29.456647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f37467e8-b012-4dbe-8ae1-d7caf43354e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4453a910474400cbd05eaafd72b9635', 'request_id': 'req-f37467e8-b012-4dbe-8ae1-d7caf43354e6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:29.469458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f37467e8-b012-4dbe-8ae1-d7caf43354e6 admin admin] The token cache is being invalidate because project d4453a910474400cbd05eaafd72b9635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:29.469751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f37467e8-b012-4dbe-8ae1-d7caf43354e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:29.470406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f37467e8-b012-4dbe-8ae1-d7caf43354e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:29.472676 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3015/6096] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:29 2022] DELETE /identity/v3/projects/d4453a910474400cbd05eaafd72b9635 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:29.482674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812db18-c400-420f-8241-4851e9de6181 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:29.483134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812db18-c400-420f-8241-4851e9de6181 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:29.483253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812db18-c400-420f-8241-4851e9de6181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:29.504823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9812db18-c400-420f-8241-4851e9de6181 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:29.506842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9812db18-c400-420f-8241-4851e9de6181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:29.546876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:29.595512 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3082/6097] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:29.598678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861d514-2bd8-48ab-950c-69e012ba01a8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:29.599790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861d514-2bd8-48ab-950c-69e012ba01a8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:29.600055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861d514-2bd8-48ab-950c-69e012ba01a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:29.627674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a861d514-2bd8-48ab-950c-69e012ba01a8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:29.629289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a861d514-2bd8-48ab-950c-69e012ba01a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:29.646688 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3016/6098] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:29.647693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812db18-c400-420f-8241-4851e9de6181 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:29.670752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812db18-c400-420f-8241-4851e9de6181 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:29.682932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e138e5c-8238-46bc-888f-48c40d6213fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:29.687160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e138e5c-8238-46bc-888f-48c40d6213fc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:29.687647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e138e5c-8238-46bc-888f-48c40d6213fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:29.688085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e138e5c-8238-46bc-888f-48c40d6213fc admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:29.688945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e138e5c-8238-46bc-888f-48c40d6213fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:18:29.689751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e138e5c-8238-46bc-888f-48c40d6213fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:29.693365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e138e5c-8238-46bc-888f-48c40d6213fc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:29.726983 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3083/6099] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:18:29.729943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861d514-2bd8-48ab-950c-69e012ba01a8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:29.741664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861d514-2bd8-48ab-950c-69e012ba01a8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:29.754080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef93cf-74a3-43f2-af91-dcd1e9eea8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:29.755005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef93cf-74a3-43f2-af91-dcd1e9eea8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:29.755279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef93cf-74a3-43f2-af91-dcd1e9eea8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:29.755613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef93cf-74a3-43f2-af91-dcd1e9eea8d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:29.756334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef93cf-74a3-43f2-af91-dcd1e9eea8d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:29.758389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef93cf-74a3-43f2-af91-dcd1e9eea8d2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:29.763877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:29.821264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e138e5c-8238-46bc-888f-48c40d6213fc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:29.822136 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3017/6100] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:29 2022] POST /identity/v3/projects => generated 389 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:18:29.836210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e138e5c-8238-46bc-888f-48c40d6213fc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:29.982334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b65a7-a806-4500-9393-295113f959ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:29.982964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b65a7-a806-4500-9393-295113f959ee admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:29.983232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b65a7-a806-4500-9393-295113f959ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:29.983494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b65a7-a806-4500-9393-295113f959ee admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:29.984121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b65a7-a806-4500-9393-295113f959ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:29.986083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b65a7-a806-4500-9393-295113f959ee admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:30.036155 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3084/6101] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:29 2022] POST /identity/v3/users => generated 295 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:30.040247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef93cf-74a3-43f2-af91-dcd1e9eea8d2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:30.064947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef93cf-74a3-43f2-af91-dcd1e9eea8d2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:30.093137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c8553-cc65-4055-bca6-c24cebe8464d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:30.094172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c8553-cc65-4055-bca6-c24cebe8464d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:30.094441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c8553-cc65-4055-bca6-c24cebe8464d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:30.094706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c8553-cc65-4055-bca6-c24cebe8464d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:30.095358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c8553-cc65-4055-bca6-c24cebe8464d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:30.097028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c8553-cc65-4055-bca6-c24cebe8464d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:30.124748 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3018/6102] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:30.128640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b65a7-a806-4500-9393-295113f959ee admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:30.148106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b65a7-a806-4500-9393-295113f959ee admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:30.153544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9fc27-6159-4544-b3cf-68073f1758f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:30.154357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9fc27-6159-4544-b3cf-68073f1758f9 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:30.154492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9fc27-6159-4544-b3cf-68073f1758f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:30.154689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9fc27-6159-4544-b3cf-68073f1758f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:30.155339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9fc27-6159-4544-b3cf-68073f1758f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:30.157023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9fc27-6159-4544-b3cf-68073f1758f9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:30.203079 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3085/6103] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:30.203280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c8553-cc65-4055-bca6-c24cebe8464d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:30.235018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c8553-cc65-4055-bca6-c24cebe8464d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:30.248502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b42dc-043a-487a-a85d-e1fde4c186fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:30.249292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b42dc-043a-487a-a85d-e1fde4c186fa admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:30.249592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b42dc-043a-487a-a85d-e1fde4c186fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:30.249865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b42dc-043a-487a-a85d-e1fde4c186fa admin admin] PATH_INFO: `/v3/projects/3a33ad1b28874d46b76b7bbe756d339e/users/85805da2da9a419bab802722aeea5f1a/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:30.292925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b42dc-043a-487a-a85d-e1fde4c186fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a33ad1b28874d46b76b7bbe756d339e, user_id=85805da2da9a419bab802722aeea5f1a, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:30.296499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b42dc-043a-487a-a85d-e1fde4c186fa admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:30.376435 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3019/6104] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:30 2022] PUT /identity/v3/projects/3a33ad1b28874d46b76b7bbe756d339e/users/85805da2da9a419bab802722aeea5f1a/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:18:30.379909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44458e-4055-459c-84a0-d9097e801d9c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:30.383095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44458e-4055-459c-84a0-d9097e801d9c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:30.383095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44458e-4055-459c-84a0-d9097e801d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:30.442488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fc44458e-4055-459c-84a0-d9097e801d9c None None] MFA Rules not processed for user `85805da2da9a419bab802722aeea5f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:30.444312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fc44458e-4055-459c-84a0-d9097e801d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:30.642661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b42dc-043a-487a-a85d-e1fde4c186fa admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:30.643216 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3086/6105] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:18:30.852982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b42dc-043a-487a-a85d-e1fde4c186fa admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:30.859698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0593b-3055-4d1d-b5ae-6196731a1383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:30.860322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0593b-3055-4d1d-b5ae-6196731a1383 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:30.860888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0593b-3055-4d1d-b5ae-6196731a1383 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:30.862022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0593b-3055-4d1d-b5ae-6196731a1383 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:30.862820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0593b-3055-4d1d-b5ae-6196731a1383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:30.864895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0593b-3055-4d1d-b5ae-6196731a1383 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:30.919123 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3020/6106] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:30 2022] POST /identity/v3/projects => generated 387 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:18:30.922724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44458e-4055-459c-84a0-d9097e801d9c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:30.941511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44458e-4055-459c-84a0-d9097e801d9c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:31.000815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:31.051301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc6a9c-5336-4cf0-ada6-66e6a68b7a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:31.052313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc6a9c-5336-4cf0-ada6-66e6a68b7a41 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:31.053741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc6a9c-5336-4cf0-ada6-66e6a68b7a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:31.054334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc6a9c-5336-4cf0-ada6-66e6a68b7a41 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:31.055458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc6a9c-5336-4cf0-ada6-66e6a68b7a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:31.058705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc6a9c-5336-4cf0-ada6-66e6a68b7a41 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:31.096350 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3087/6107] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:30 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:31.098044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0593b-3055-4d1d-b5ae-6196731a1383 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:31.115183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0593b-3055-4d1d-b5ae-6196731a1383 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:31.124293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537c8e1-5bed-4ba9-a029-7bea5f03e5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:31.126495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537c8e1-5bed-4ba9-a029-7bea5f03e5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:31.126775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537c8e1-5bed-4ba9-a029-7bea5f03e5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:31.127039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537c8e1-5bed-4ba9-a029-7bea5f03e5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:31.127696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537c8e1-5bed-4ba9-a029-7bea5f03e5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:31.129462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537c8e1-5bed-4ba9-a029-7bea5f03e5a7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:31.146137 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3021/6108] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:31.150497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc6a9c-5336-4cf0-ada6-66e6a68b7a41 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:31.162314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc6a9c-5336-4cf0-ada6-66e6a68b7a41 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:31.169121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fe83f-1c8b-431e-8925-5311485da16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:31.169755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fe83f-1c8b-431e-8925-5311485da16b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:31.170021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fe83f-1c8b-431e-8925-5311485da16b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:31.170282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fe83f-1c8b-431e-8925-5311485da16b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:31.170940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fe83f-1c8b-431e-8925-5311485da16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:31.172791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fe83f-1c8b-431e-8925-5311485da16b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:31.188275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537c8e1-5bed-4ba9-a029-7bea5f03e5a7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:31.199395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537c8e1-5bed-4ba9-a029-7bea5f03e5a7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:31.200399 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3088/6109] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:31.207248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73461a53-a7a8-41f8-a14c-f4d745e074d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:31.208724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73461a53-a7a8-41f8-a14c-f4d745e074d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:31.208995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73461a53-a7a8-41f8-a14c-f4d745e074d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:31.209259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73461a53-a7a8-41f8-a14c-f4d745e074d5 admin admin] PATH_INFO: `/v3/projects/f5b0577869a346ee95ef9ed22fdb036a/users/f83fcb7f7a3b4f809f6f284372c1c502/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:31.235170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73461a53-a7a8-41f8-a14c-f4d745e074d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b0577869a346ee95ef9ed22fdb036a, user_id=f83fcb7f7a3b4f809f6f284372c1c502, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:31.238083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73461a53-a7a8-41f8-a14c-f4d745e074d5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:31.284008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f39306-8730-4364-8c99-80ee5c509f30 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:31.284318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f39306-8730-4364-8c99-80ee5c509f30 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:31.285246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f39306-8730-4364-8c99-80ee5c509f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:31.287951 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3022/6110] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:31 2022] PUT /identity/v3/projects/f5b0577869a346ee95ef9ed22fdb036a/users/f83fcb7f7a3b4f809f6f284372c1c502/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:31.322427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f8f39306-8730-4364-8c99-80ee5c509f30 None None] MFA Rules not processed for user `f83fcb7f7a3b4f809f6f284372c1c502`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:31.323822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f8f39306-8730-4364-8c99-80ee5c509f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:31.421319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73461a53-a7a8-41f8-a14c-f4d745e074d5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:31.423567 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3089/6111] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:31.524699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73461a53-a7a8-41f8-a14c-f4d745e074d5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:31.534144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47dba1-89f4-4431-98a7-63c4d4c2a2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:31.535152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47dba1-89f4-4431-98a7-63c4d4c2a2a7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:31.535646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47dba1-89f4-4431-98a7-63c4d4c2a2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:31.536237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47dba1-89f4-4431-98a7-63c4d4c2a2a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:31.537244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47dba1-89f4-4431-98a7-63c4d4c2a2a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:31.539950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47dba1-89f4-4431-98a7-63c4d4c2a2a7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:31.546206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:31.584849 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3023/6112] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:31 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:31.589145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f39306-8730-4364-8c99-80ee5c509f30 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:31.608595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f39306-8730-4364-8c99-80ee5c509f30 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:31.705220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295429c8-2ddb-4561-a77a-d46e4bad73b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:31.706370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295429c8-2ddb-4561-a77a-d46e4bad73b9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:31.706903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295429c8-2ddb-4561-a77a-d46e4bad73b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:31.707264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295429c8-2ddb-4561-a77a-d46e4bad73b9 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:31.708593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295429c8-2ddb-4561-a77a-d46e4bad73b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:31.712226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295429c8-2ddb-4561-a77a-d46e4bad73b9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:31.761217 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3090/6113] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:31 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:31.763064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47dba1-89f4-4431-98a7-63c4d4c2a2a7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:31.785925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47dba1-89f4-4431-98a7-63c4d4c2a2a7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:31.795119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4745b79-6757-47e0-a6aa-1b4093c6e172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:31.795932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4745b79-6757-47e0-a6aa-1b4093c6e172 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:31.798746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4745b79-6757-47e0-a6aa-1b4093c6e172 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:31.799461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4745b79-6757-47e0-a6aa-1b4093c6e172 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:31.800732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4745b79-6757-47e0-a6aa-1b4093c6e172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:31.803349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4745b79-6757-47e0-a6aa-1b4093c6e172 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:31.820415 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3024/6114] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:31.824888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295429c8-2ddb-4561-a77a-d46e4bad73b9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:31.840629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295429c8-2ddb-4561-a77a-d46e4bad73b9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:31.846512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08da4d-7fc8-4034-8466-46eaf5601c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:31.847204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08da4d-7fc8-4034-8466-46eaf5601c39 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:31.847475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08da4d-7fc8-4034-8466-46eaf5601c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:31.847754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08da4d-7fc8-4034-8466-46eaf5601c39 admin admin] PATH_INFO: `/v3/projects/639616336005402688a4df83ca5be9f2/users/336d480048bf4a2eb28153f434634269/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:31.870745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08da4d-7fc8-4034-8466-46eaf5601c39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=639616336005402688a4df83ca5be9f2, user_id=336d480048bf4a2eb28153f434634269, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:31.874322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08da4d-7fc8-4034-8466-46eaf5601c39 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:31.907343 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3091/6115] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:31 2022] PUT /identity/v3/projects/639616336005402688a4df83ca5be9f2/users/336d480048bf4a2eb28153f434634269/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:31.911712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4745b79-6757-47e0-a6aa-1b4093c6e172 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:31.935440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4745b79-6757-47e0-a6aa-1b4093c6e172 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:32.074273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252267f-8ac4-476a-a9ca-4121a6b60d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:32.075229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252267f-8ac4-476a-a9ca-4121a6b60d5d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:32.075815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252267f-8ac4-476a-a9ca-4121a6b60d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:32.076498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252267f-8ac4-476a-a9ca-4121a6b60d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:32.077398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252267f-8ac4-476a-a9ca-4121a6b60d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:32.080235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252267f-8ac4-476a-a9ca-4121a6b60d5d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:32.099123 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3025/6116] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:32.103376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08da4d-7fc8-4034-8466-46eaf5601c39 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:32.120967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08da4d-7fc8-4034-8466-46eaf5601c39 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:32.132364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4b136-dccc-4934-b5f8-dfc04ac4b25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:32.132846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4b136-dccc-4934-b5f8-dfc04ac4b25a admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:32.133129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4b136-dccc-4934-b5f8-dfc04ac4b25a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:32.133397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4b136-dccc-4934-b5f8-dfc04ac4b25a admin admin] PATH_INFO: `/v3/projects/639616336005402688a4df83ca5be9f2/users/336d480048bf4a2eb28153f434634269/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:32.138734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4b136-dccc-4934-b5f8-dfc04ac4b25a admin admin] RBAC: Authorizing `identity:create_grant(project_id=639616336005402688a4df83ca5be9f2, user_id=336d480048bf4a2eb28153f434634269, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:32.144278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4b136-dccc-4934-b5f8-dfc04ac4b25a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:32.151169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:32.175703 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3092/6117] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:32 2022] PUT /identity/v3/projects/639616336005402688a4df83ca5be9f2/users/336d480048bf4a2eb28153f434634269/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:18:32.186136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43bf0-09d3-4d8b-b414-18343bff956e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:32.187317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43bf0-09d3-4d8b-b414-18343bff956e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:32.187795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43bf0-09d3-4d8b-b414-18343bff956e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:32.309698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a0e43bf0-09d3-4d8b-b414-18343bff956e None None] MFA Rules not processed for user `336d480048bf4a2eb28153f434634269`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:32.311869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a0e43bf0-09d3-4d8b-b414-18343bff956e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:32.520280 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3026/6118] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:18:32.521370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4b136-dccc-4934-b5f8-dfc04ac4b25a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:32.523365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b2e4b136-dccc-4934-b5f8-dfc04ac4b25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:32.783543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4b136-dccc-4934-b5f8-dfc04ac4b25a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:32.792562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa32742-7c45-441f-99ad-173265827386 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:32.794515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa32742-7c45-441f-99ad-173265827386 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:32.794859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa32742-7c45-441f-99ad-173265827386 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:32.795215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa32742-7c45-441f-99ad-173265827386 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:32.873930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa32742-7c45-441f-99ad-173265827386 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:32.875811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa32742-7c45-441f-99ad-173265827386 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:32.890025 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3093/6119] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:33.893537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43bf0-09d3-4d8b-b414-18343bff956e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:33.895320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a0e43bf0-09d3-4d8b-b414-18343bff956e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:34.013385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:34.020298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43bf0-09d3-4d8b-b414-18343bff956e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:34.040241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:34.041217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:34.041497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:34.043441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] PATH_INFO: `/v3/users/a22eb4ad08ce4027bfa7f6f487a3534b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:34.045046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a22eb4ad08ce4027bfa7f6f487a3534b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:34.047437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:34.150726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a22eb4ad08ce4027bfa7f6f487a3534b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:34.167067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a22eb4ad08ce4027bfa7f6f487a3534b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:34.208812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a22eb4ad08ce4027bfa7f6f487a3534b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:34.209752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a22eb4ad08ce4027bfa7f6f487a3534b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:34.210558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a22eb4ad08ce4027bfa7f6f487a3534b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:34.250109 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3027/6120] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:33 2022] DELETE /identity/v3/users/a22eb4ad08ce4027bfa7f6f487a3534b => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:34.259009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5273d-9787-4e2f-bf12-836fa53cf2ae None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:34.259576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5273d-9787-4e2f-bf12-836fa53cf2ae None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:34.260414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5273d-9787-4e2f-bf12-836fa53cf2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:34.284781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1dc5273d-9787-4e2f-bf12-836fa53cf2ae None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:34.286004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1dc5273d-9787-4e2f-bf12-836fa53cf2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:34.389027 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3094/6121] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:34.401412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:34.402981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:34.500766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d4071-34ce-478d-be4a-1b61a3d1ec5a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:34.505404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4fd3f-1791-45d1-afdf-5ce5e796e0fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:34.506024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4fd3f-1791-45d1-afdf-5ce5e796e0fd service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:34.506296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4fd3f-1791-45d1-afdf-5ce5e796e0fd service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:34.506561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4fd3f-1791-45d1-afdf-5ce5e796e0fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:34.517026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4fd3f-1791-45d1-afdf-5ce5e796e0fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:34.518589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4fd3f-1791-45d1-afdf-5ce5e796e0fd service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:34.530769 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3028/6122] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:35.187843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5273d-9787-4e2f-bf12-836fa53cf2ae None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:35.189685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1dc5273d-9787-4e2f-bf12-836fa53cf2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:35.230470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5273d-9787-4e2f-bf12-836fa53cf2ae None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:35.255520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff5d01-e603-4556-baa0-2328a620ecee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:35.261211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff5d01-e603-4556-baa0-2328a620ecee admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:35.262258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff5d01-e603-4556-baa0-2328a620ecee admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:35.262596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff5d01-e603-4556-baa0-2328a620ecee admin admin] PATH_INFO: `/v3/projects/ae34c5d4a3dc40168445a3822e12e7d6` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:35.264554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff5d01-e603-4556-baa0-2328a620ecee admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae34c5d4a3dc40168445a3822e12e7d6)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:35.266894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff5d01-e603-4556-baa0-2328a620ecee admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:35.309532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=1,keystone:SELECT=98,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:35.335148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c7ff5d01-e603-4556-baa0-2328a620ecee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae34c5d4a3dc40168445a3822e12e7d6', 'request_id': 'req-c7ff5d01-e603-4556-baa0-2328a620ecee'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:35.350939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c7ff5d01-e603-4556-baa0-2328a620ecee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae34c5d4a3dc40168445a3822e12e7d6', 'request_id': 'req-c7ff5d01-e603-4556-baa0-2328a620ecee'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:35.358529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c7ff5d01-e603-4556-baa0-2328a620ecee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae34c5d4a3dc40168445a3822e12e7d6', 'request_id': 'req-c7ff5d01-e603-4556-baa0-2328a620ecee'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:35.371279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c7ff5d01-e603-4556-baa0-2328a620ecee admin admin] The token cache is being invalidate because project ae34c5d4a3dc40168445a3822e12e7d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:35.371575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c7ff5d01-e603-4556-baa0-2328a620ecee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:35.372481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c7ff5d01-e603-4556-baa0-2328a620ecee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:35.374461 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3095/6123] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:35 2022] DELETE /identity/v3/projects/ae34c5d4a3dc40168445a3822e12e7d6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:35.379084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd1e8f-6e8c-4205-b414-edb63d998c33 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:35.380059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd1e8f-6e8c-4205-b414-edb63d998c33 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:35.380513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd1e8f-6e8c-4205-b414-edb63d998c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:35.430587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e5fd1e8f-6e8c-4205-b414-edb63d998c33 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:35.431866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e5fd1e8f-6e8c-4205-b414-edb63d998c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:35.593262 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3029/6124] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:35.602026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1df780-472f-4957-8a70-49c5b7142801 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:35.602731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1df780-472f-4957-8a70-49c5b7142801 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:35.603095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1df780-472f-4957-8a70-49c5b7142801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:35.676873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6b1df780-472f-4957-8a70-49c5b7142801 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:35.676873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6b1df780-472f-4957-8a70-49c5b7142801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:35.682911 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3096/6125] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:35.684286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd1e8f-6e8c-4205-b414-edb63d998c33 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:35.714363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd1e8f-6e8c-4205-b414-edb63d998c33 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:35.720701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d820b4b-109b-4316-94a8-6bbd2f1e6553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:35.722519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d820b4b-109b-4316-94a8-6bbd2f1e6553 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:35.723110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d820b4b-109b-4316-94a8-6bbd2f1e6553 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:35.723589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d820b4b-109b-4316-94a8-6bbd2f1e6553 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:35.724434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d820b4b-109b-4316-94a8-6bbd2f1e6553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:18:35.724958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d820b4b-109b-4316-94a8-6bbd2f1e6553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:35.735023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d820b4b-109b-4316-94a8-6bbd2f1e6553 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:35.785055 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3030/6126] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:18:35.791335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1df780-472f-4957-8a70-49c5b7142801 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:35.803127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1df780-472f-4957-8a70-49c5b7142801 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:35.808194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e926b-083c-4d03-806a-b576f3fe77c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:35.808848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e926b-083c-4d03-806a-b576f3fe77c2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:35.809117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e926b-083c-4d03-806a-b576f3fe77c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:35.809809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e926b-083c-4d03-806a-b576f3fe77c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:35.809995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e926b-083c-4d03-806a-b576f3fe77c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:35.812014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e926b-083c-4d03-806a-b576f3fe77c2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:35.849944 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3097/6127] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:35 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:35.854221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d820b4b-109b-4316-94a8-6bbd2f1e6553 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:35.864495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:35.871409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d820b4b-109b-4316-94a8-6bbd2f1e6553 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:35.979091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05417cd-119b-452f-91d0-890e725e9fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:35.979705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05417cd-119b-452f-91d0-890e725e9fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:35.980025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05417cd-119b-452f-91d0-890e725e9fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:35.980707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05417cd-119b-452f-91d0-890e725e9fe8 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:35.981490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05417cd-119b-452f-91d0-890e725e9fe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:35.983882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05417cd-119b-452f-91d0-890e725e9fe8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:36.026199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e926b-083c-4d03-806a-b576f3fe77c2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:36.027386 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3031/6128] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:35 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:18:36.039756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e926b-083c-4d03-806a-b576f3fe77c2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:36.058394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6c4df-0204-4057-8820-7629251a1205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:36.059096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6c4df-0204-4057-8820-7629251a1205 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:36.059400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6c4df-0204-4057-8820-7629251a1205 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:36.059675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6c4df-0204-4057-8820-7629251a1205 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:36.060397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6c4df-0204-4057-8820-7629251a1205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:36.062074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6c4df-0204-4057-8820-7629251a1205 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:36.093107 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3098/6129] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:18:36.097708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05417cd-119b-452f-91d0-890e725e9fe8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:36.126298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05417cd-119b-452f-91d0-890e725e9fe8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:36.141340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30571610-df1e-48b4-bfd7-495aed0ee1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:36.141978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30571610-df1e-48b4-bfd7-495aed0ee1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:36.142247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30571610-df1e-48b4-bfd7-495aed0ee1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:36.142508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30571610-df1e-48b4-bfd7-495aed0ee1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:36.143156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30571610-df1e-48b4-bfd7-495aed0ee1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:36.144834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30571610-df1e-48b4-bfd7-495aed0ee1e8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:36.167419 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3032/6130] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:36.173035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6c4df-0204-4057-8820-7629251a1205 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:36.201455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6c4df-0204-4057-8820-7629251a1205 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:36.211613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81cb86-9d53-404b-921a-391287a9e974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:36.219281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81cb86-9d53-404b-921a-391287a9e974 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:36.220328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81cb86-9d53-404b-921a-391287a9e974 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:36.220635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81cb86-9d53-404b-921a-391287a9e974 admin admin] PATH_INFO: `/v3/projects/4dae500113014803bf09fe30d45959ae/users/cd8cdc29b9884c249809cfc84a149801/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:36.224086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30571610-df1e-48b4-bfd7-495aed0ee1e8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:36.227023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-30571610-df1e-48b4-bfd7-495aed0ee1e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:36.288310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81cb86-9d53-404b-921a-391287a9e974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dae500113014803bf09fe30d45959ae, user_id=cd8cdc29b9884c249809cfc84a149801, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:36.293057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81cb86-9d53-404b-921a-391287a9e974 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:36.336003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30571610-df1e-48b4-bfd7-495aed0ee1e8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:36.348139 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3099/6131] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:36 2022] PUT /identity/v3/projects/4dae500113014803bf09fe30d45959ae/users/cd8cdc29b9884c249809cfc84a149801/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:36.372868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e8ac1-468b-4f2a-bf2f-ff1be87bc025 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:36.373153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e8ac1-468b-4f2a-bf2f-ff1be87bc025 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:36.373438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e8ac1-468b-4f2a-bf2f-ff1be87bc025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:36.430519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d8e94-d836-47b9-91c5-e962c0e32930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:36.431288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d8e94-d836-47b9-91c5-e962c0e32930 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:36.431659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d8e94-d836-47b9-91c5-e962c0e32930 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:36.432022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d8e94-d836-47b9-91c5-e962c0e32930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:36.440793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-563e8ac1-468b-4f2a-bf2f-ff1be87bc025 None None] MFA Rules not processed for user `cd8cdc29b9884c249809cfc84a149801`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:36.445129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-563e8ac1-468b-4f2a-bf2f-ff1be87bc025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:36.452796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-de5d8e94-d836-47b9-91c5-e962c0e32930 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:36.523310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:36.579836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d8e94-d836-47b9-91c5-e962c0e32930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:36.583945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d8e94-d836-47b9-91c5-e962c0e32930 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:36.611686 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3033/6132] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:36.733896 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3100/6133] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:18:36.838504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d8e94-d836-47b9-91c5-e962c0e32930 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:36.840834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-de5d8e94-d836-47b9-91c5-e962c0e32930 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:36.857159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d8e94-d836-47b9-91c5-e962c0e32930 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:36.862469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52cd13e-3795-46b2-bf44-73f24658595f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:36.863105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cd13e-3795-46b2-bf44-73f24658595f service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:36.863381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cd13e-3795-46b2-bf44-73f24658595f service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:36.863653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cd13e-3795-46b2-bf44-73f24658595f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:36.878475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52cd13e-3795-46b2-bf44-73f24658595f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:36.880814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52cd13e-3795-46b2-bf44-73f24658595f service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:36.908571 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3034/6134] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:38.298519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e8ac1-468b-4f2a-bf2f-ff1be87bc025 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:38.300186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-563e8ac1-468b-4f2a-bf2f-ff1be87bc025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:38.440654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:38.512035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e8ac1-468b-4f2a-bf2f-ff1be87bc025 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:38.517830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:38.518616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:38.521933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:38.521933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] PATH_INFO: `/v3/users/9671d01ad0d44cc99e0eecc966f7c3e6` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:38.521933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9671d01ad0d44cc99e0eecc966f7c3e6)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:38.523778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:38.599054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9671d01ad0d44cc99e0eecc966f7c3e6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:38.619623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9671d01ad0d44cc99e0eecc966f7c3e6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:38.639021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9671d01ad0d44cc99e0eecc966f7c3e6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:38.658898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9671d01ad0d44cc99e0eecc966f7c3e6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:38.659704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9671d01ad0d44cc99e0eecc966f7c3e6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:38.672319 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3101/6135] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:38 2022] DELETE /identity/v3/users/9671d01ad0d44cc99e0eecc966f7c3e6 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:38.676996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e7cab-4169-48df-ba31-32a7e9d31c06 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:38.677916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e7cab-4169-48df-ba31-32a7e9d31c06 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:38.678401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e7cab-4169-48df-ba31-32a7e9d31c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:38.707280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c81e7cab-4169-48df-ba31-32a7e9d31c06 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:38.716572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c81e7cab-4169-48df-ba31-32a7e9d31c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:38.912462 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3035/6136] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:38.945990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:38.949180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:39.109869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5738fdf-0e43-434e-8c5e-39b2cd26e1a6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:39.127093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8c11e-96a5-4398-9266-ba47f38174be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:39.127774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8c11e-96a5-4398-9266-ba47f38174be service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:39.128410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8c11e-96a5-4398-9266-ba47f38174be service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:39.128798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8c11e-96a5-4398-9266-ba47f38174be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:39.158606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8c11e-96a5-4398-9266-ba47f38174be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:39.160430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8c11e-96a5-4398-9266-ba47f38174be service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:39.189735 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3102/6137] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:39.734253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e7cab-4169-48df-ba31-32a7e9d31c06 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:39.740507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c81e7cab-4169-48df-ba31-32a7e9d31c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:39.763096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e7cab-4169-48df-ba31-32a7e9d31c06 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:39.772536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:39.773508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:39.773724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:39.773895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] PATH_INFO: `/v3/projects/3b15cdd8d7fa4da79626028b15b99382` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:39.775073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b15cdd8d7fa4da79626028b15b99382)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:39.778577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:39.929869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b15cdd8d7fa4da79626028b15b99382', 'request_id': 'req-f61f7c0a-7670-408a-a33b-91d639f03a92'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:39.962629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b15cdd8d7fa4da79626028b15b99382', 'request_id': 'req-f61f7c0a-7670-408a-a33b-91d639f03a92'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:39.988265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b15cdd8d7fa4da79626028b15b99382', 'request_id': 'req-f61f7c0a-7670-408a-a33b-91d639f03a92'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:40.014066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] The token cache is being invalidate because project 3b15cdd8d7fa4da79626028b15b99382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:40.014386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:40.016918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:40.027110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8c11e-96a5-4398-9266-ba47f38174be service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:40.027650 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3036/6138] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:39 2022] DELETE /identity/v3/projects/3b15cdd8d7fa4da79626028b15b99382 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:40.044602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8c11e-96a5-4398-9266-ba47f38174be service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:40.219914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a1d2c-ae13-42af-9ef0-d450e85566bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:40.221597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a1d2c-ae13-42af-9ef0-d450e85566bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:40.222174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a1d2c-ae13-42af-9ef0-d450e85566bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:40.222704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a1d2c-ae13-42af-9ef0-d450e85566bc admin admin] PATH_INFO: `/v3/users/7117e6dc0cf343c890d617e86b6ec5a8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:40.225125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a1d2c-ae13-42af-9ef0-d450e85566bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7117e6dc0cf343c890d617e86b6ec5a8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:40.228217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a1d2c-ae13-42af-9ef0-d450e85566bc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:40.237145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:40.344806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c40a1d2c-ae13-42af-9ef0-d450e85566bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7117e6dc0cf343c890d617e86b6ec5a8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:40.361596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c40a1d2c-ae13-42af-9ef0-d450e85566bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7117e6dc0cf343c890d617e86b6ec5a8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:40.372532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c40a1d2c-ae13-42af-9ef0-d450e85566bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7117e6dc0cf343c890d617e86b6ec5a8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:40.406171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c40a1d2c-ae13-42af-9ef0-d450e85566bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7117e6dc0cf343c890d617e86b6ec5a8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:40.407559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c40a1d2c-ae13-42af-9ef0-d450e85566bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7117e6dc0cf343c890d617e86b6ec5a8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:40.417874 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3103/6139] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:40 2022] DELETE /identity/v3/users/7117e6dc0cf343c890d617e86b6ec5a8 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:40.899592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:40.901557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:40.938106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f7c0a-7670-408a-a33b-91d639f03a92 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:41.095686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a43061-1ce2-403b-a91d-c8cda71a323a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:41.096451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a43061-1ce2-403b-a91d-c8cda71a323a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:41.096814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a43061-1ce2-403b-a91d-c8cda71a323a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:41.097139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a43061-1ce2-403b-a91d-c8cda71a323a admin admin] PATH_INFO: `/v3/projects/1786ebd38fb94e9596bd11d4cec5132a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:41.098923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a43061-1ce2-403b-a91d-c8cda71a323a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1786ebd38fb94e9596bd11d4cec5132a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:41.101139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a43061-1ce2-403b-a91d-c8cda71a323a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:41.105282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:41.230681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04a43061-1ce2-403b-a91d-c8cda71a323a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1786ebd38fb94e9596bd11d4cec5132a', 'request_id': 'req-04a43061-1ce2-403b-a91d-c8cda71a323a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:41.246967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04a43061-1ce2-403b-a91d-c8cda71a323a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1786ebd38fb94e9596bd11d4cec5132a', 'request_id': 'req-04a43061-1ce2-403b-a91d-c8cda71a323a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:41.266978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04a43061-1ce2-403b-a91d-c8cda71a323a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1786ebd38fb94e9596bd11d4cec5132a', 'request_id': 'req-04a43061-1ce2-403b-a91d-c8cda71a323a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:41.297473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04a43061-1ce2-403b-a91d-c8cda71a323a admin admin] The token cache is being invalidate because project 1786ebd38fb94e9596bd11d4cec5132a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:41.297892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04a43061-1ce2-403b-a91d-c8cda71a323a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:41.300309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04a43061-1ce2-403b-a91d-c8cda71a323a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:41.304824 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3037/6140] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:40 2022] DELETE /identity/v3/projects/1786ebd38fb94e9596bd11d4cec5132a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:41.309939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc31a54-d0aa-4f5b-afb5-bc7782a03b63 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:41.310791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc31a54-d0aa-4f5b-afb5-bc7782a03b63 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:41.311252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc31a54-d0aa-4f5b-afb5-bc7782a03b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:41.357889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-afc31a54-d0aa-4f5b-afb5-bc7782a03b63 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:41.359678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-afc31a54-d0aa-4f5b-afb5-bc7782a03b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:41.480724 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3104/6141] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:41.489527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793a2f2-180a-4ca6-8fdf-bf27ee47f623 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:41.492355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793a2f2-180a-4ca6-8fdf-bf27ee47f623 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:41.492925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793a2f2-180a-4ca6-8fdf-bf27ee47f623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:41.564055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a793a2f2-180a-4ca6-8fdf-bf27ee47f623 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:41.568468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a793a2f2-180a-4ca6-8fdf-bf27ee47f623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:41.610443 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3038/6142] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:41.610665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc31a54-d0aa-4f5b-afb5-bc7782a03b63 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:41.631676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc31a54-d0aa-4f5b-afb5-bc7782a03b63 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:41.637633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a72c1-6c87-43aa-8eb6-4a628b73fdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:41.638582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a72c1-6c87-43aa-8eb6-4a628b73fdc2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:41.638756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a72c1-6c87-43aa-8eb6-4a628b73fdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:41.639133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a72c1-6c87-43aa-8eb6-4a628b73fdc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:41.640050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a72c1-6c87-43aa-8eb6-4a628b73fdc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:18:41.640843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a72c1-6c87-43aa-8eb6-4a628b73fdc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:41.643150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a72c1-6c87-43aa-8eb6-4a628b73fdc2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:41.670543 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3105/6143] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:18:41.671876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793a2f2-180a-4ca6-8fdf-bf27ee47f623 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:41.685243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793a2f2-180a-4ca6-8fdf-bf27ee47f623 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:41.698950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f89069-97e5-4c61-b237-8bcdf2ca9163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:41.699455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f89069-97e5-4c61-b237-8bcdf2ca9163 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:41.699730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f89069-97e5-4c61-b237-8bcdf2ca9163 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:41.699999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f89069-97e5-4c61-b237-8bcdf2ca9163 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:41.700662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f89069-97e5-4c61-b237-8bcdf2ca9163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:41.702752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f89069-97e5-4c61-b237-8bcdf2ca9163 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:41.778752 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3039/6144] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:41 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:41.782967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a72c1-6c87-43aa-8eb6-4a628b73fdc2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:41.804342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a72c1-6c87-43aa-8eb6-4a628b73fdc2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:41.963948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280aca0e-de77-4b4e-936b-e8a86b321148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:41.964936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280aca0e-de77-4b4e-936b-e8a86b321148 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:41.965381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280aca0e-de77-4b4e-936b-e8a86b321148 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:41.967913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280aca0e-de77-4b4e-936b-e8a86b321148 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:41.968858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280aca0e-de77-4b4e-936b-e8a86b321148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:41.970973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280aca0e-de77-4b4e-936b-e8a86b321148 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:42.007843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:42.020056 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3106/6145] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:41 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:42.021136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f89069-97e5-4c61-b237-8bcdf2ca9163 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:42.042486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f89069-97e5-4c61-b237-8bcdf2ca9163 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:42.052526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d711010-4ae9-485a-96a2-4933d8cccbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:42.053158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d711010-4ae9-485a-96a2-4933d8cccbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:42.053749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d711010-4ae9-485a-96a2-4933d8cccbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:42.054314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d711010-4ae9-485a-96a2-4933d8cccbb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:42.055553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d711010-4ae9-485a-96a2-4933d8cccbb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:42.057479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d711010-4ae9-485a-96a2-4933d8cccbb7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:42.077371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280aca0e-de77-4b4e-936b-e8a86b321148 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:42.079482 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3040/6146] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:42.093198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280aca0e-de77-4b4e-936b-e8a86b321148 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:42.107311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57992ecc-ce55-4acb-a59f-47c8245fadef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:42.108206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57992ecc-ce55-4acb-a59f-47c8245fadef admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:42.108695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57992ecc-ce55-4acb-a59f-47c8245fadef admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:42.109138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57992ecc-ce55-4acb-a59f-47c8245fadef admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:42.110026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57992ecc-ce55-4acb-a59f-47c8245fadef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:42.111854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57992ecc-ce55-4acb-a59f-47c8245fadef admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:42.143030 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3107/6147] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:42.144533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d711010-4ae9-485a-96a2-4933d8cccbb7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:42.164689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d711010-4ae9-485a-96a2-4933d8cccbb7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:42.174877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ea991-1fc9-480d-819f-bd9dea13be34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:42.187417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ea991-1fc9-480d-819f-bd9dea13be34 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:42.187838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ea991-1fc9-480d-819f-bd9dea13be34 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:42.188230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ea991-1fc9-480d-819f-bd9dea13be34 admin admin] PATH_INFO: `/v3/projects/ac3755e014f24355852fe12d62830df2/users/3dcce2c34fff4f2f85f9e8de1bf689a1/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:42.224004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ea991-1fc9-480d-819f-bd9dea13be34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3755e014f24355852fe12d62830df2, user_id=3dcce2c34fff4f2f85f9e8de1bf689a1, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:42.227306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ea991-1fc9-480d-819f-bd9dea13be34 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:42.255758 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3041/6148] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:42 2022] PUT /identity/v3/projects/ac3755e014f24355852fe12d62830df2/users/3dcce2c34fff4f2f85f9e8de1bf689a1/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:42.265795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd8f7a-9215-437e-b0bc-15fb95cf6976 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:42.266077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd8f7a-9215-437e-b0bc-15fb95cf6976 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:42.266578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd8f7a-9215-437e-b0bc-15fb95cf6976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:42.337508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-90cd8f7a-9215-437e-b0bc-15fb95cf6976 None None] MFA Rules not processed for user `3dcce2c34fff4f2f85f9e8de1bf689a1`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:42.339336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-90cd8f7a-9215-437e-b0bc-15fb95cf6976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:42.515933 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3108/6149] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:42.524283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ea991-1fc9-480d-819f-bd9dea13be34 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:42.633786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ea991-1fc9-480d-819f-bd9dea13be34 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:42.639754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f0f69-3bf3-4642-a9d9-bfd17b5f7a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:42.640625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f0f69-3bf3-4642-a9d9-bfd17b5f7a9f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:42.641038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f0f69-3bf3-4642-a9d9-bfd17b5f7a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:42.641410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f0f69-3bf3-4642-a9d9-bfd17b5f7a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:42.642168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f0f69-3bf3-4642-a9d9-bfd17b5f7a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:42.659576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f0f69-3bf3-4642-a9d9-bfd17b5f7a9f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:42.771814 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3042/6150] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:42 2022] POST /identity/v3/projects => generated 385 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:42.775062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd8f7a-9215-437e-b0bc-15fb95cf6976 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:42.805449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd8f7a-9215-437e-b0bc-15fb95cf6976 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:42.969553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e74d90-d0c7-4e2e-be90-5faf75636c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:42.971564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74d90-d0c7-4e2e-be90-5faf75636c01 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:42.972283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74d90-d0c7-4e2e-be90-5faf75636c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:42.973423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74d90-d0c7-4e2e-be90-5faf75636c01 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:42.975103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e74d90-d0c7-4e2e-be90-5faf75636c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:42.980934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e74d90-d0c7-4e2e-be90-5faf75636c01 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:43.022176 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3109/6151] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:42 2022] POST /identity/v3/users => generated 299 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:43.025994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f0f69-3bf3-4642-a9d9-bfd17b5f7a9f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:43.065499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f0f69-3bf3-4642-a9d9-bfd17b5f7a9f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:43.080469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c80cba-c920-44d3-98f0-01b9dda9c4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:43.082360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c80cba-c920-44d3-98f0-01b9dda9c4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:43.082758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c80cba-c920-44d3-98f0-01b9dda9c4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:43.083624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c80cba-c920-44d3-98f0-01b9dda9c4a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:43.086324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c80cba-c920-44d3-98f0-01b9dda9c4a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:43.090103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c80cba-c920-44d3-98f0-01b9dda9c4a5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:43.104346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:43.119478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e74d90-d0c7-4e2e-be90-5faf75636c01 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:43.127564 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3043/6152] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:43.135094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e74d90-d0c7-4e2e-be90-5faf75636c01 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:43.149721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683d1cb-94a7-4e26-8efd-5f78e8e0de10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:43.150614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683d1cb-94a7-4e26-8efd-5f78e8e0de10 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:43.151035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683d1cb-94a7-4e26-8efd-5f78e8e0de10 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:43.151409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683d1cb-94a7-4e26-8efd-5f78e8e0de10 admin admin] PATH_INFO: `/v3/projects/1ca776c00fe44a32b9cd8b4b014cb548/users/2862b7ada04a4c0087ce809b0cb8340c/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:43.200476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683d1cb-94a7-4e26-8efd-5f78e8e0de10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca776c00fe44a32b9cd8b4b014cb548, user_id=2862b7ada04a4c0087ce809b0cb8340c, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:43.207125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683d1cb-94a7-4e26-8efd-5f78e8e0de10 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:43.235917 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3110/6153] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:43 2022] PUT /identity/v3/projects/1ca776c00fe44a32b9cd8b4b014cb548/users/2862b7ada04a4c0087ce809b0cb8340c/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:43.240321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c80cba-c920-44d3-98f0-01b9dda9c4a5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:43.257550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c80cba-c920-44d3-98f0-01b9dda9c4a5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:43.358096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0cf04-3539-41c0-ab03-5acb0af8c63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:43.358736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0cf04-3539-41c0-ab03-5acb0af8c63a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:43.359009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0cf04-3539-41c0-ab03-5acb0af8c63a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:43.359275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0cf04-3539-41c0-ab03-5acb0af8c63a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:43.360185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0cf04-3539-41c0-ab03-5acb0af8c63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:43.367717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0cf04-3539-41c0-ab03-5acb0af8c63a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:43.386429 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3044/6154] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:18:43.391921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683d1cb-94a7-4e26-8efd-5f78e8e0de10 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:43.410527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683d1cb-94a7-4e26-8efd-5f78e8e0de10 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:43.418302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61fa4c-e807-443a-a20a-06bba5db94b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:43.419051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61fa4c-e807-443a-a20a-06bba5db94b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:43.419559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61fa4c-e807-443a-a20a-06bba5db94b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:43.419837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61fa4c-e807-443a-a20a-06bba5db94b3 admin admin] PATH_INFO: `/v3/projects/1ca776c00fe44a32b9cd8b4b014cb548/users/2862b7ada04a4c0087ce809b0cb8340c/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:43.423706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61fa4c-e807-443a-a20a-06bba5db94b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca776c00fe44a32b9cd8b4b014cb548, user_id=2862b7ada04a4c0087ce809b0cb8340c, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:43.426793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61fa4c-e807-443a-a20a-06bba5db94b3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:43.452572 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3111/6155] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:43 2022] PUT /identity/v3/projects/1ca776c00fe44a32b9cd8b4b014cb548/users/2862b7ada04a4c0087ce809b0cb8340c/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:18:43.465943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda921e-5484-4f4b-8c6a-72461ee5ca5d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:43.466238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda921e-5484-4f4b-8c6a-72461ee5ca5d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:43.466606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda921e-5484-4f4b-8c6a-72461ee5ca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:43.526771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cdda921e-5484-4f4b-8c6a-72461ee5ca5d None None] MFA Rules not processed for user `2862b7ada04a4c0087ce809b0cb8340c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:43.528020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cdda921e-5484-4f4b-8c6a-72461ee5ca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:43.623259 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3045/6156] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:43.640830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61fa4c-e807-443a-a20a-06bba5db94b3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:43.643721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8f61fa4c-e807-443a-a20a-06bba5db94b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:43.785801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:43.934741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61fa4c-e807-443a-a20a-06bba5db94b3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:43.958382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23d14d-177a-4326-9e86-fef79574f8c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:43.959221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23d14d-177a-4326-9e86-fef79574f8c2 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:43.959495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23d14d-177a-4326-9e86-fef79574f8c2 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:43.959804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23d14d-177a-4326-9e86-fef79574f8c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:43.991666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23d14d-177a-4326-9e86-fef79574f8c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:43.992361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23d14d-177a-4326-9e86-fef79574f8c2 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:44.041716 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3112/6157] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:44.165830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e81fb-0fca-483c-a5b8-5b866da5315b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:44.166310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e81fb-0fca-483c-a5b8-5b866da5315b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:44.166432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e81fb-0fca-483c-a5b8-5b866da5315b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:44.192293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fe7e81fb-0fca-483c-a5b8-5b866da5315b None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:44.193600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fe7e81fb-0fca-483c-a5b8-5b866da5315b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:44.289110 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3046/6158] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:18:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:44.296765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23d14d-177a-4326-9e86-fef79574f8c2 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:44.298231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6b23d14d-177a-4326-9e86-fef79574f8c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:44.484895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23d14d-177a-4326-9e86-fef79574f8c2 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:44.500489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2df5bd-97a8-4d37-9e0d-14946548eee9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:44.502243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2df5bd-97a8-4d37-9e0d-14946548eee9 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:44.502709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2df5bd-97a8-4d37-9e0d-14946548eee9 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:44.503348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2df5bd-97a8-4d37-9e0d-14946548eee9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:44.530114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2df5bd-97a8-4d37-9e0d-14946548eee9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:44.536285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2df5bd-97a8-4d37-9e0d-14946548eee9 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:44.567304 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3113/6159] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:18:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:18:44.881992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e81fb-0fca-483c-a5b8-5b866da5315b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:44.943167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:45.047057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e81fb-0fca-483c-a5b8-5b866da5315b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:45.053587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe660a-0acf-42b1-8942-f269ddfa4df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:45.056518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe660a-0acf-42b1-8942-f269ddfa4df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:45.057518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe660a-0acf-42b1-8942-f269ddfa4df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:45.057979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe660a-0acf-42b1-8942-f269ddfa4df0 admin admin] PATH_INFO: `/v3/users/3dcce2c34fff4f2f85f9e8de1bf689a1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:45.059764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe660a-0acf-42b1-8942-f269ddfa4df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dcce2c34fff4f2f85f9e8de1bf689a1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:45.062063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe660a-0acf-42b1-8942-f269ddfa4df0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:45.173344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b5fe660a-0acf-42b1-8942-f269ddfa4df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dcce2c34fff4f2f85f9e8de1bf689a1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:45.196516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b5fe660a-0acf-42b1-8942-f269ddfa4df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dcce2c34fff4f2f85f9e8de1bf689a1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:45.218860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b5fe660a-0acf-42b1-8942-f269ddfa4df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dcce2c34fff4f2f85f9e8de1bf689a1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:45.219901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b5fe660a-0acf-42b1-8942-f269ddfa4df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dcce2c34fff4f2f85f9e8de1bf689a1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:45.220894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b5fe660a-0acf-42b1-8942-f269ddfa4df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dcce2c34fff4f2f85f9e8de1bf689a1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:45.257069 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3047/6160] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:44 2022] DELETE /identity/v3/users/3dcce2c34fff4f2f85f9e8de1bf689a1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:45.258003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cde59-8360-421a-b487-c596edeb8f05 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:45.258290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cde59-8360-421a-b487-c596edeb8f05 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:45.258563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cde59-8360-421a-b487-c596edeb8f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:45.300872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019807f-afc6-4f0a-b105-58888e0ba930 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:45.301195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019807f-afc6-4f0a-b105-58888e0ba930 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:45.301475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019807f-afc6-4f0a-b105-58888e0ba930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:45.311151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6d2cde59-8360-421a-b487-c596edeb8f05 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:45.312809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6d2cde59-8360-421a-b487-c596edeb8f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:45.340346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e019807f-afc6-4f0a-b105-58888e0ba930 None None] MFA Rules not processed for user `85805da2da9a419bab802722aeea5f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:45.341603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e019807f-afc6-4f0a-b105-58888e0ba930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:45.441354 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3114/6161] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:18:45.443118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14f0cd-68f1-4900-8c03-39b433147cfd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:45.443680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14f0cd-68f1-4900-8c03-39b433147cfd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:45.444617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14f0cd-68f1-4900-8c03-39b433147cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:45.450163 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3048/6162] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:18:45.457780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019807f-afc6-4f0a-b105-58888e0ba930 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:45.461529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e019807f-afc6-4f0a-b105-58888e0ba930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:45.480494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6e14f0cd-68f1-4900-8c03-39b433147cfd None None] MFA Rules not processed for user `f49e7f7982ad4d98b48519f7c1b43fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:45.481810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6e14f0cd-68f1-4900-8c03-39b433147cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:45.558663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019807f-afc6-4f0a-b105-58888e0ba930 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:45.559307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:45.564999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7af75-fcfb-43e9-b3ac-35bc644afae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:45.565738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7af75-fcfb-43e9-b3ac-35bc644afae5 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:45.566270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7af75-fcfb-43e9-b3ac-35bc644afae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:45.566411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7af75-fcfb-43e9-b3ac-35bc644afae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:45.577951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:45.581058 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3115/6163] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:45.583049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7af75-fcfb-43e9-b3ac-35bc644afae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:45.585479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7af75-fcfb-43e9-b3ac-35bc644afae5 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:45.618764 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3049/6164] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:46.178062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14f0cd-68f1-4900-8c03-39b433147cfd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:46.200027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14f0cd-68f1-4900-8c03-39b433147cfd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:46.209143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21e06f-c0ea-439f-b816-164007b818a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:46.210207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21e06f-c0ea-439f-b816-164007b818a1 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:46.210865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21e06f-c0ea-439f-b816-164007b818a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:46.219260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21e06f-c0ea-439f-b816-164007b818a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:46.222207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cd21e06f-c0ea-439f-b816-164007b818a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:46.251482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21e06f-c0ea-439f-b816-164007b818a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:46.253304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21e06f-c0ea-439f-b816-164007b818a1 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:46.272157 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3116/6165] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:46.977441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7af75-fcfb-43e9-b3ac-35bc644afae5 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:46.979288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f1f7af75-fcfb-43e9-b3ac-35bc644afae5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:47.004739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7af75-fcfb-43e9-b3ac-35bc644afae5 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:47.017632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:47.018293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:47.018562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:47.018825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] PATH_INFO: `/v3/projects/ac3755e014f24355852fe12d62830df2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:47.022792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac3755e014f24355852fe12d62830df2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:47.026892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:47.160351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac3755e014f24355852fe12d62830df2', 'request_id': 'req-ec6cf4ea-10d7-47d0-b78f-50301faccdae'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:47.189122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac3755e014f24355852fe12d62830df2', 'request_id': 'req-ec6cf4ea-10d7-47d0-b78f-50301faccdae'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:47.216716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac3755e014f24355852fe12d62830df2', 'request_id': 'req-ec6cf4ea-10d7-47d0-b78f-50301faccdae'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:47.252117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] The token cache is being invalidate because project ac3755e014f24355852fe12d62830df2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:47.252432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:47.268791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:47.270718 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3050/6166] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:46 2022] DELETE /identity/v3/projects/ac3755e014f24355852fe12d62830df2 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:47.275006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21e06f-c0ea-439f-b816-164007b818a1 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:47.295053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21e06f-c0ea-439f-b816-164007b818a1 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:47.449664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:47.450282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:47.450560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:47.451232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] PATH_INFO: `/v3/users/2862b7ada04a4c0087ce809b0cb8340c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:47.452457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2862b7ada04a4c0087ce809b0cb8340c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:47.455698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:47.578890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2862b7ada04a4c0087ce809b0cb8340c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:47.593182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2862b7ada04a4c0087ce809b0cb8340c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:47.623045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2862b7ada04a4c0087ce809b0cb8340c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:47.684795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2862b7ada04a4c0087ce809b0cb8340c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:47.686484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2862b7ada04a4c0087ce809b0cb8340c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:47.705613 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3117/6167] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:47 2022] DELETE /identity/v3/users/2862b7ada04a4c0087ce809b0cb8340c => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:48.420813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:48.426764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:48.521047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cf4ea-10d7-47d0-b78f-50301faccdae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:48.533975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:48.536580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:48.740516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:48.756156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:48.758478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:48.759103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:48.760211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf admin admin] PATH_INFO: `/v3/projects/1ca776c00fe44a32b9cd8b4b014cb548` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:48.762364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ca776c00fe44a32b9cd8b4b014cb548)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:48.766212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:48.787803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e115fe-ce42-488c-8f35-281c4d7c3352 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:48.795439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c29a50-9e69-4042-a3d7-2f025447ec9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:48.796768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c29a50-9e69-4042-a3d7-2f025447ec9d service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:48.797390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c29a50-9e69-4042-a3d7-2f025447ec9d service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:48.797811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c29a50-9e69-4042-a3d7-2f025447ec9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:48.801167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d6c29a50-9e69-4042-a3d7-2f025447ec9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:48.834149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c29a50-9e69-4042-a3d7-2f025447ec9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:48.836488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c29a50-9e69-4042-a3d7-2f025447ec9d service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:48.855210 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3118/6168] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:18:48.856606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c29a50-9e69-4042-a3d7-2f025447ec9d service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:48.861942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d6c29a50-9e69-4042-a3d7-2f025447ec9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:48.884819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ca776c00fe44a32b9cd8b4b014cb548', 'request_id': 'req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:48.885494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c29a50-9e69-4042-a3d7-2f025447ec9d service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:48.901455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ca776c00fe44a32b9cd8b4b014cb548', 'request_id': 'req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:48.918887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ca776c00fe44a32b9cd8b4b014cb548', 'request_id': 'req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:48.945101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf admin admin] The token cache is being invalidate because project 1ca776c00fe44a32b9cd8b4b014cb548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:48.945471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:48.946469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d4cc9fdc-24f5-42b1-92fd-7967640f6abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:48.953458 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3051/6169] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:48 2022] DELETE /identity/v3/projects/1ca776c00fe44a32b9cd8b4b014cb548 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:48.958275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a145fba-88ee-4b74-9c3b-c99e734e97f9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:48.958604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a145fba-88ee-4b74-9c3b-c99e734e97f9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:48.959086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a145fba-88ee-4b74-9c3b-c99e734e97f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:49.006305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:49.011092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a92ea-4017-417a-8946-57f203591e56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:49.012594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a92ea-4017-417a-8946-57f203591e56 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:49.020432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a92ea-4017-417a-8946-57f203591e56 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:49.021061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a92ea-4017-417a-8946-57f203591e56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:49.022497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5a145fba-88ee-4b74-9c3b-c99e734e97f9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:49.027351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5a145fba-88ee-4b74-9c3b-c99e734e97f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:49.028364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fd9a92ea-4017-417a-8946-57f203591e56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:49.052140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a92ea-4017-417a-8946-57f203591e56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:49.054073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a92ea-4017-417a-8946-57f203591e56 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:49.072649 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3119/6170] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:49.144876 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3052/6171] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:49.157557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc1ab-471f-4e5c-a770-a009f4ebdebd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:49.157557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc1ab-471f-4e5c-a770-a009f4ebdebd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:49.157557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc1ab-471f-4e5c-a770-a009f4ebdebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:49.189810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e5ebc1ab-471f-4e5c-a770-a009f4ebdebd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:49.192402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e5ebc1ab-471f-4e5c-a770-a009f4ebdebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:49.203452 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3120/6172] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:49.208766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a145fba-88ee-4b74-9c3b-c99e734e97f9 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:49.220186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a145fba-88ee-4b74-9c3b-c99e734e97f9 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:49.230228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee30d4f-cd9c-4358-b6ca-076d63c309fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:49.233189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee30d4f-cd9c-4358-b6ca-076d63c309fb admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:49.233557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee30d4f-cd9c-4358-b6ca-076d63c309fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:49.233916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee30d4f-cd9c-4358-b6ca-076d63c309fb admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:49.234639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee30d4f-cd9c-4358-b6ca-076d63c309fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:18:49.235013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee30d4f-cd9c-4358-b6ca-076d63c309fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:49.236773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee30d4f-cd9c-4358-b6ca-076d63c309fb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:49.250592 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3053/6173] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:18:49.255188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc1ab-471f-4e5c-a770-a009f4ebdebd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:49.265342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc1ab-471f-4e5c-a770-a009f4ebdebd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:49.271077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa3599-33cf-4a5d-8221-e986045d1b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:49.271692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa3599-33cf-4a5d-8221-e986045d1b0d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:49.271958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa3599-33cf-4a5d-8221-e986045d1b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:49.272462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa3599-33cf-4a5d-8221-e986045d1b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:49.272994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa3599-33cf-4a5d-8221-e986045d1b0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:49.274892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa3599-33cf-4a5d-8221-e986045d1b0d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:49.305870 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3121/6174] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:49 2022] POST /identity/v3/projects => generated 395 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:18:49.310080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee30d4f-cd9c-4358-b6ca-076d63c309fb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:49.323444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee30d4f-cd9c-4358-b6ca-076d63c309fb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:49.388219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147646a7-21d9-408d-99a5-4794d0308585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:49.389199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147646a7-21d9-408d-99a5-4794d0308585 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:49.389199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147646a7-21d9-408d-99a5-4794d0308585 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:49.389587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147646a7-21d9-408d-99a5-4794d0308585 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:49.390080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147646a7-21d9-408d-99a5-4794d0308585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:49.392198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147646a7-21d9-408d-99a5-4794d0308585 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:49.419052 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3054/6175] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:49 2022] POST /identity/v3/users => generated 298 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:18:49.423259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa3599-33cf-4a5d-8221-e986045d1b0d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:49.434490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa3599-33cf-4a5d-8221-e986045d1b0d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:49.438919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2aef4-84b7-4a42-a2c7-6f866dee82e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:49.439619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2aef4-84b7-4a42-a2c7-6f866dee82e5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:49.440154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2aef4-84b7-4a42-a2c7-6f866dee82e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:49.440291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2aef4-84b7-4a42-a2c7-6f866dee82e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:49.440948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2aef4-84b7-4a42-a2c7-6f866dee82e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:49.442572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2aef4-84b7-4a42-a2c7-6f866dee82e5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:49.454666 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3122/6176] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:49.459890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147646a7-21d9-408d-99a5-4794d0308585 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:49.470786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147646a7-21d9-408d-99a5-4794d0308585 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:49.475644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e5043-a9b5-4569-abaf-0892f17b3e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:49.476508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e5043-a9b5-4569-abaf-0892f17b3e95 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:49.476790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e5043-a9b5-4569-abaf-0892f17b3e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:49.477055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e5043-a9b5-4569-abaf-0892f17b3e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:49.477705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e5043-a9b5-4569-abaf-0892f17b3e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:49.479319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e5043-a9b5-4569-abaf-0892f17b3e95 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:49.490701 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3055/6177] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:49.495822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2aef4-84b7-4a42-a2c7-6f866dee82e5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:49.512316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2aef4-84b7-4a42-a2c7-6f866dee82e5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:49.522465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f7d2c-0bb7-46f3-8766-463242cc9788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:49.523215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f7d2c-0bb7-46f3-8766-463242cc9788 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:49.523636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f7d2c-0bb7-46f3-8766-463242cc9788 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:49.524071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f7d2c-0bb7-46f3-8766-463242cc9788 admin admin] PATH_INFO: `/v3/projects/691da569594e4b22b71c895a2b269f47/users/9b65cfa237a64065bb5836bb5d580ebc/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:49.541990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f7d2c-0bb7-46f3-8766-463242cc9788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=691da569594e4b22b71c895a2b269f47, user_id=9b65cfa237a64065bb5836bb5d580ebc, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:49.545239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f7d2c-0bb7-46f3-8766-463242cc9788 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:49.561914 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3123/6178] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:49 2022] PUT /identity/v3/projects/691da569594e4b22b71c895a2b269f47/users/9b65cfa237a64065bb5836bb5d580ebc/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:49.573737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b06190-95e4-4fc8-851b-baeb0b0ac856 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:49.573882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b06190-95e4-4fc8-851b-baeb0b0ac856 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:49.574255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b06190-95e4-4fc8-851b-baeb0b0ac856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:49.619079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d0b06190-95e4-4fc8-851b-baeb0b0ac856 None None] MFA Rules not processed for user `9b65cfa237a64065bb5836bb5d580ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:49.620319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d0b06190-95e4-4fc8-851b-baeb0b0ac856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:49.697425 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3056/6179] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:49 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:49.702859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f7d2c-0bb7-46f3-8766-463242cc9788 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:49.799196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f7d2c-0bb7-46f3-8766-463242cc9788 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:49.807372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cab387-c492-4408-bc1d-aaaf0f55d5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:49.808956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cab387-c492-4408-bc1d-aaaf0f55d5bc admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:49.810364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cab387-c492-4408-bc1d-aaaf0f55d5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:49.810364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cab387-c492-4408-bc1d-aaaf0f55d5bc admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:49.810364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cab387-c492-4408-bc1d-aaaf0f55d5bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:49.812972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cab387-c492-4408-bc1d-aaaf0f55d5bc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:49.836145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:49.860279 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3124/6180] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:49 2022] POST /identity/v3/projects => generated 395 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:49.864312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b06190-95e4-4fc8-851b-baeb0b0ac856 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:49.870137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:49.887577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b06190-95e4-4fc8-851b-baeb0b0ac856 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:49.963129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667f591-0390-4bad-b137-ce7432ff38cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:49.963748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667f591-0390-4bad-b137-ce7432ff38cf admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:49.964017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667f591-0390-4bad-b137-ce7432ff38cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:49.964278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667f591-0390-4bad-b137-ce7432ff38cf admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:49.965223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667f591-0390-4bad-b137-ce7432ff38cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:49.967133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667f591-0390-4bad-b137-ce7432ff38cf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:50.015333 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3057/6181] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:49 2022] POST /identity/v3/users => generated 304 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:50.019950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cab387-c492-4408-bc1d-aaaf0f55d5bc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:50.038919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cab387-c492-4408-bc1d-aaaf0f55d5bc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:50.051193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c4ae4-cb26-42f8-9885-4143247af5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:50.051832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c4ae4-cb26-42f8-9885-4143247af5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:50.052112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c4ae4-cb26-42f8-9885-4143247af5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:50.052411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c4ae4-cb26-42f8-9885-4143247af5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:50.055532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c4ae4-cb26-42f8-9885-4143247af5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:50.059595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c4ae4-cb26-42f8-9885-4143247af5f7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:50.076606 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3125/6182] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:50.078618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667f591-0390-4bad-b137-ce7432ff38cf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:50.094920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667f591-0390-4bad-b137-ce7432ff38cf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:50.103447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3439f9-19cf-41a9-bb40-d64ab5e94095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:50.104529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3439f9-19cf-41a9-bb40-d64ab5e94095 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:50.105239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3439f9-19cf-41a9-bb40-d64ab5e94095 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:50.105749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3439f9-19cf-41a9-bb40-d64ab5e94095 admin admin] PATH_INFO: `/v3/projects/8a704c3257944849bb3a9aa4d3c79e13/users/5b2e94a090c447318306be8c5cbfc503/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:50.132428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3439f9-19cf-41a9-bb40-d64ab5e94095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a704c3257944849bb3a9aa4d3c79e13, user_id=5b2e94a090c447318306be8c5cbfc503, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:50.135865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3439f9-19cf-41a9-bb40-d64ab5e94095 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:50.160856 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3058/6183] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:50 2022] PUT /identity/v3/projects/8a704c3257944849bb3a9aa4d3c79e13/users/5b2e94a090c447318306be8c5cbfc503/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:50.162233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c4ae4-cb26-42f8-9885-4143247af5f7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:50.173996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c4ae4-cb26-42f8-9885-4143247af5f7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:50.263014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6277aa7-2f73-4c50-832a-38944fa751cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:50.263637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6277aa7-2f73-4c50-832a-38944fa751cd admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:50.263912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6277aa7-2f73-4c50-832a-38944fa751cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:50.264366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6277aa7-2f73-4c50-832a-38944fa751cd admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:50.265086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6277aa7-2f73-4c50-832a-38944fa751cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:50.266747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6277aa7-2f73-4c50-832a-38944fa751cd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:50.279580 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3126/6184] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:50.283898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3439f9-19cf-41a9-bb40-d64ab5e94095 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:50.296075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3439f9-19cf-41a9-bb40-d64ab5e94095 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:50.302597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a11843-42c8-4032-902d-e92950369ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:50.304458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a11843-42c8-4032-902d-e92950369ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:50.304994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a11843-42c8-4032-902d-e92950369ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:50.305655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a11843-42c8-4032-902d-e92950369ce8 admin admin] PATH_INFO: `/v3/projects/8a704c3257944849bb3a9aa4d3c79e13/users/5b2e94a090c447318306be8c5cbfc503/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:50.308872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a11843-42c8-4032-902d-e92950369ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a704c3257944849bb3a9aa4d3c79e13, user_id=5b2e94a090c447318306be8c5cbfc503, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:50.312226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a11843-42c8-4032-902d-e92950369ce8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:50.327648 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3059/6185] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:50 2022] PUT /identity/v3/projects/8a704c3257944849bb3a9aa4d3c79e13/users/5b2e94a090c447318306be8c5cbfc503/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:18:50.339876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ded157-9082-4a38-9c95-774e4dbaac37 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:50.341386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ded157-9082-4a38-9c95-774e4dbaac37 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:50.342228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ded157-9082-4a38-9c95-774e4dbaac37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:50.377959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d0ded157-9082-4a38-9c95-774e4dbaac37 None None] MFA Rules not processed for user `5b2e94a090c447318306be8c5cbfc503`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:50.379629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d0ded157-9082-4a38-9c95-774e4dbaac37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:50.470611 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3127/6186] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:50 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:18:50.486196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6205b3b-2ff1-40de-9bb9-ac81a0ba5003 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:50.486757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6205b3b-2ff1-40de-9bb9-ac81a0ba5003 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:50.486757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6205b3b-2ff1-40de-9bb9-ac81a0ba5003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:50.515100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d6205b3b-2ff1-40de-9bb9-ac81a0ba5003 None None] MFA Rules not processed for user `9b65cfa237a64065bb5836bb5d580ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:50.516981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d6205b3b-2ff1-40de-9bb9-ac81a0ba5003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:50.595311 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3060/6187] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:50 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:50.606350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ded157-9082-4a38-9c95-774e4dbaac37 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:50.607979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d0ded157-9082-4a38-9c95-774e4dbaac37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:50.696471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:50.705989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ded157-9082-4a38-9c95-774e4dbaac37 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:50.717831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436ff2d-6871-480d-91bc-e83803439b42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:50.718890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ff2d-6871-480d-91bc-e83803439b42 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:50.719347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ff2d-6871-480d-91bc-e83803439b42 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:50.719808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ff2d-6871-480d-91bc-e83803439b42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:50.733175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436ff2d-6871-480d-91bc-e83803439b42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:50.737868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436ff2d-6871-480d-91bc-e83803439b42 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:50.746654 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3128/6188] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:50 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:52.990373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6205b3b-2ff1-40de-9bb9-ac81a0ba5003 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:52.992401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d6205b3b-2ff1-40de-9bb9-ac81a0ba5003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:53.061150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6205b3b-2ff1-40de-9bb9-ac81a0ba5003 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:53.065844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:53.066473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:53.066742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:53.067000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] PATH_INFO: `/v3/users/cd8cdc29b9884c249809cfc84a149801` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:53.068132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd8cdc29b9884c249809cfc84a149801)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:53.070519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:53.072059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:53.122388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd8cdc29b9884c249809cfc84a149801'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:53.130980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd8cdc29b9884c249809cfc84a149801'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:53.144401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd8cdc29b9884c249809cfc84a149801'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:53.145468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd8cdc29b9884c249809cfc84a149801'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:53.146211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd8cdc29b9884c249809cfc84a149801'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:53.161501 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3061/6189] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:52 2022] DELETE /identity/v3/users/cd8cdc29b9884c249809cfc84a149801 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:53.166365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead99e6b-c4ec-4ceb-a15b-24d4d8178244 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:53.166896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead99e6b-c4ec-4ceb-a15b-24d4d8178244 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:53.166896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead99e6b-c4ec-4ceb-a15b-24d4d8178244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:53.201386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ead99e6b-c4ec-4ceb-a15b-24d4d8178244 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:53.206107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ead99e6b-c4ec-4ceb-a15b-24d4d8178244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:53.297808 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3129/6190] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:18:53.308418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:53.310143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:53.385272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251be186-23e0-4de8-b5ab-155b67177689 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:53.389524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77ddaf-1f44-41db-b8ec-362d2d9308c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:53.390713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77ddaf-1f44-41db-b8ec-362d2d9308c0 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:53.391483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77ddaf-1f44-41db-b8ec-362d2d9308c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:53.392212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77ddaf-1f44-41db-b8ec-362d2d9308c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:53.403498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77ddaf-1f44-41db-b8ec-362d2d9308c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:53.404975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77ddaf-1f44-41db-b8ec-362d2d9308c0 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:53.414731 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3062/6191] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:18:53.648646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead99e6b-c4ec-4ceb-a15b-24d4d8178244 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:53.653008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ead99e6b-c4ec-4ceb-a15b-24d4d8178244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:53.692542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead99e6b-c4ec-4ceb-a15b-24d4d8178244 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:53.698666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae626eb4-4478-4c52-bf23-c6b943807e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:53.699378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae626eb4-4478-4c52-bf23-c6b943807e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:53.699656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae626eb4-4478-4c52-bf23-c6b943807e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:53.699923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae626eb4-4478-4c52-bf23-c6b943807e00 admin admin] PATH_INFO: `/v3/projects/4dae500113014803bf09fe30d45959ae` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:53.701130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae626eb4-4478-4c52-bf23-c6b943807e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dae500113014803bf09fe30d45959ae)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:53.703373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae626eb4-4478-4c52-bf23-c6b943807e00 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:53.771635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ae626eb4-4478-4c52-bf23-c6b943807e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dae500113014803bf09fe30d45959ae', 'request_id': 'req-ae626eb4-4478-4c52-bf23-c6b943807e00'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:53.786025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ae626eb4-4478-4c52-bf23-c6b943807e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dae500113014803bf09fe30d45959ae', 'request_id': 'req-ae626eb4-4478-4c52-bf23-c6b943807e00'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:53.802092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ae626eb4-4478-4c52-bf23-c6b943807e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dae500113014803bf09fe30d45959ae', 'request_id': 'req-ae626eb4-4478-4c52-bf23-c6b943807e00'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:53.823581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ae626eb4-4478-4c52-bf23-c6b943807e00 admin admin] The token cache is being invalidate because project 4dae500113014803bf09fe30d45959ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:53.824009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ae626eb4-4478-4c52-bf23-c6b943807e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:53.825911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ae626eb4-4478-4c52-bf23-c6b943807e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:53.833588 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3130/6192] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:53 2022] DELETE /identity/v3/projects/4dae500113014803bf09fe30d45959ae => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:53.835999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05d7dd-e262-4c41-aa98-7e784411c814 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:53.836272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05d7dd-e262-4c41-aa98-7e784411c814 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:53.836588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05d7dd-e262-4c41-aa98-7e784411c814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:53.858164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8d05d7dd-e262-4c41-aa98-7e784411c814 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:53.859399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8d05d7dd-e262-4c41-aa98-7e784411c814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:53.941139 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3063/6193] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:53.944694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c987ef-2dd6-4803-a93f-fac326759a94 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:53.945200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c987ef-2dd6-4803-a93f-fac326759a94 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:53.945683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c987ef-2dd6-4803-a93f-fac326759a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:53.975706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c3c987ef-2dd6-4803-a93f-fac326759a94 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:53.977224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c3c987ef-2dd6-4803-a93f-fac326759a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:53.986135 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3131/6194] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:53.990297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05d7dd-e262-4c41-aa98-7e784411c814 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:54.001188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05d7dd-e262-4c41-aa98-7e784411c814 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:54.006433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ec3ff-a18f-4909-af39-4e53670e7bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:54.007125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ec3ff-a18f-4909-af39-4e53670e7bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:54.007474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ec3ff-a18f-4909-af39-4e53670e7bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:54.007808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ec3ff-a18f-4909-af39-4e53670e7bc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:54.008689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ec3ff-a18f-4909-af39-4e53670e7bc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:18:54.009062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ec3ff-a18f-4909-af39-4e53670e7bc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:54.010885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ec3ff-a18f-4909-af39-4e53670e7bc8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:54.027399 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3064/6195] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:18:54.028548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c987ef-2dd6-4803-a93f-fac326759a94 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:54.037924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c987ef-2dd6-4803-a93f-fac326759a94 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:54.042786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db88b5-6c74-4f3e-a096-e2554bed997b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:54.043450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db88b5-6c74-4f3e-a096-e2554bed997b admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:54.043718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db88b5-6c74-4f3e-a096-e2554bed997b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:54.044001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db88b5-6c74-4f3e-a096-e2554bed997b admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:54.044651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db88b5-6c74-4f3e-a096-e2554bed997b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:54.046562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db88b5-6c74-4f3e-a096-e2554bed997b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:54.075740 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3132/6196] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:54 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:18:54.080104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ec3ff-a18f-4909-af39-4e53670e7bc8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:54.091341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ec3ff-a18f-4909-af39-4e53670e7bc8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:54.096385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:54.154965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cb89f-5eed-4fbd-8413-16781f9a3846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:54.155847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cb89f-5eed-4fbd-8413-16781f9a3846 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:54.155847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cb89f-5eed-4fbd-8413-16781f9a3846 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:54.156103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cb89f-5eed-4fbd-8413-16781f9a3846 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:54.156745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cb89f-5eed-4fbd-8413-16781f9a3846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:54.158698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cb89f-5eed-4fbd-8413-16781f9a3846 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:54.181460 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3065/6197] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:54 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:54.186869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db88b5-6c74-4f3e-a096-e2554bed997b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:54.198733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db88b5-6c74-4f3e-a096-e2554bed997b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:54.204150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034561a-109e-4134-a1c9-0e78e39c0953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:54.204838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034561a-109e-4134-a1c9-0e78e39c0953 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:54.205136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034561a-109e-4134-a1c9-0e78e39c0953 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:54.205375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034561a-109e-4134-a1c9-0e78e39c0953 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:54.206194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034561a-109e-4134-a1c9-0e78e39c0953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:54.207861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034561a-109e-4134-a1c9-0e78e39c0953 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:54.219928 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3133/6198] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:54.224063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cb89f-5eed-4fbd-8413-16781f9a3846 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:54.243320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cb89f-5eed-4fbd-8413-16781f9a3846 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:54.248537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907b829-31cb-4a4e-a0d2-13508721821e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:54.249299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907b829-31cb-4a4e-a0d2-13508721821e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:54.249633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907b829-31cb-4a4e-a0d2-13508721821e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:54.249962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907b829-31cb-4a4e-a0d2-13508721821e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:54.250677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907b829-31cb-4a4e-a0d2-13508721821e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:54.252341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907b829-31cb-4a4e-a0d2-13508721821e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:54.266973 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3066/6199] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:54.271166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034561a-109e-4134-a1c9-0e78e39c0953 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:54.285709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034561a-109e-4134-a1c9-0e78e39c0953 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:54.290786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1c8db-1a03-4943-9841-39aac112d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:54.291446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1c8db-1a03-4943-9841-39aac112d9ec admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:54.291714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1c8db-1a03-4943-9841-39aac112d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:54.291975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1c8db-1a03-4943-9841-39aac112d9ec admin admin] PATH_INFO: `/v3/projects/6e5011e2175d41d9abb76962b6c61221/users/9606cc0443d249c0aa70a3033476d841/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:54.307603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1c8db-1a03-4943-9841-39aac112d9ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5011e2175d41d9abb76962b6c61221, user_id=9606cc0443d249c0aa70a3033476d841, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:54.310477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1c8db-1a03-4943-9841-39aac112d9ec admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:54.325647 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3134/6200] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:54 2022] PUT /identity/v3/projects/6e5011e2175d41d9abb76962b6c61221/users/9606cc0443d249c0aa70a3033476d841/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:54.337001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be7310-a85a-49f0-8e24-a08847e0fb81 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:54.337347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be7310-a85a-49f0-8e24-a08847e0fb81 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:54.337554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be7310-a85a-49f0-8e24-a08847e0fb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:54.369259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b2be7310-a85a-49f0-8e24-a08847e0fb81 None None] MFA Rules not processed for user `9606cc0443d249c0aa70a3033476d841`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:54.370492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b2be7310-a85a-49f0-8e24-a08847e0fb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:54.444845 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3067/6201] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:54.449545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1c8db-1a03-4943-9841-39aac112d9ec admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:54.463908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:54.518520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1c8db-1a03-4943-9841-39aac112d9ec admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:54.523708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67fac3-4ceb-45ae-9f2f-a05f25fa4522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:54.524655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67fac3-4ceb-45ae-9f2f-a05f25fa4522 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:54.525561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67fac3-4ceb-45ae-9f2f-a05f25fa4522 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:54.525561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67fac3-4ceb-45ae-9f2f-a05f25fa4522 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:54.525801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67fac3-4ceb-45ae-9f2f-a05f25fa4522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:54.527709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67fac3-4ceb-45ae-9f2f-a05f25fa4522 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:54.562324 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3135/6202] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:54 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:54.566368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be7310-a85a-49f0-8e24-a08847e0fb81 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:54.585121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be7310-a85a-49f0-8e24-a08847e0fb81 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:54.654845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b75fb-8f1d-41da-afc3-6db06a1710f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:54.655482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b75fb-8f1d-41da-afc3-6db06a1710f5 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:54.655757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b75fb-8f1d-41da-afc3-6db06a1710f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:54.656071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b75fb-8f1d-41da-afc3-6db06a1710f5 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:54.656748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b75fb-8f1d-41da-afc3-6db06a1710f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:54.658699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b75fb-8f1d-41da-afc3-6db06a1710f5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:54.688920 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3068/6203] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:54 2022] POST /identity/v3/users => generated 297 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:54.692716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67fac3-4ceb-45ae-9f2f-a05f25fa4522 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:54.704480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67fac3-4ceb-45ae-9f2f-a05f25fa4522 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:54.709475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908cace-9ee6-4951-b83d-c974af65d4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:54.710131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908cace-9ee6-4951-b83d-c974af65d4ca admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:54.710574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908cace-9ee6-4951-b83d-c974af65d4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:54.710833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908cace-9ee6-4951-b83d-c974af65d4ca admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:54.711771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908cace-9ee6-4951-b83d-c974af65d4ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:54.713437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908cace-9ee6-4951-b83d-c974af65d4ca admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:54.736566 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3136/6204] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:54.737820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b75fb-8f1d-41da-afc3-6db06a1710f5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:54.752650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b75fb-8f1d-41da-afc3-6db06a1710f5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:54.765143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799a2cd-baf1-44ce-8ef4-d571c8098c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:54.765783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799a2cd-baf1-44ce-8ef4-d571c8098c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:54.766055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799a2cd-baf1-44ce-8ef4-d571c8098c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:54.766322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799a2cd-baf1-44ce-8ef4-d571c8098c17 admin admin] PATH_INFO: `/v3/projects/e7c6c29a71a84b0c9c121ad539a5e210/users/bf5f98edf7fb468e9fb6766f0a513fee/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:54.768943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:54.785104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799a2cd-baf1-44ce-8ef4-d571c8098c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c6c29a71a84b0c9c121ad539a5e210, user_id=bf5f98edf7fb468e9fb6766f0a513fee, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:54.787971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799a2cd-baf1-44ce-8ef4-d571c8098c17 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:54.805432 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3069/6205] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:54 2022] PUT /identity/v3/projects/e7c6c29a71a84b0c9c121ad539a5e210/users/bf5f98edf7fb468e9fb6766f0a513fee/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:54.810237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908cace-9ee6-4951-b83d-c974af65d4ca admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:54.841204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908cace-9ee6-4951-b83d-c974af65d4ca admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:54.911829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e2334-e227-4766-98c8-63f10930f0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:54.912996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e2334-e227-4766-98c8-63f10930f0db admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:54.913429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e2334-e227-4766-98c8-63f10930f0db admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:54.922366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e2334-e227-4766-98c8-63f10930f0db admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:54.923665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e2334-e227-4766-98c8-63f10930f0db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:54.927731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e2334-e227-4766-98c8-63f10930f0db admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:54.941535 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3137/6206] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:54.945817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799a2cd-baf1-44ce-8ef4-d571c8098c17 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:54.957601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799a2cd-baf1-44ce-8ef4-d571c8098c17 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:54.969173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f261b-0358-43aa-a85c-05b644064927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:54.970148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f261b-0358-43aa-a85c-05b644064927 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:54.970407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f261b-0358-43aa-a85c-05b644064927 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:54.970680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f261b-0358-43aa-a85c-05b644064927 admin admin] PATH_INFO: `/v3/projects/e7c6c29a71a84b0c9c121ad539a5e210/users/bf5f98edf7fb468e9fb6766f0a513fee/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:54.972835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f261b-0358-43aa-a85c-05b644064927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c6c29a71a84b0c9c121ad539a5e210, user_id=bf5f98edf7fb468e9fb6766f0a513fee, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:54.975813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f261b-0358-43aa-a85c-05b644064927 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:54.989312 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3070/6207] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:54 2022] PUT /identity/v3/projects/e7c6c29a71a84b0c9c121ad539a5e210/users/bf5f98edf7fb468e9fb6766f0a513fee/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:18:55.001026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c3050-8837-494a-960f-9c4f0dce6520 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:55.001308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c3050-8837-494a-960f-9c4f0dce6520 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:55.001577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c3050-8837-494a-960f-9c4f0dce6520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:55.033925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3e1c3050-8837-494a-960f-9c4f0dce6520 None None] MFA Rules not processed for user `bf5f98edf7fb468e9fb6766f0a513fee`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:55.035425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3e1c3050-8837-494a-960f-9c4f0dce6520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:55.136428 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3138/6208] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:55.145659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f261b-0358-43aa-a85c-05b644064927 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:55.148119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-060f261b-0358-43aa-a85c-05b644064927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:55.230011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f261b-0358-43aa-a85c-05b644064927 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:55.235697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079123e-b493-4271-b6e5-d3c75711fdc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:55.236529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079123e-b493-4271-b6e5-d3c75711fdc8 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:55.236806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079123e-b493-4271-b6e5-d3c75711fdc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:55.237072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079123e-b493-4271-b6e5-d3c75711fdc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:55.248328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079123e-b493-4271-b6e5-d3c75711fdc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:55.249907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079123e-b493-4271-b6e5-d3c75711fdc8 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:55.263452 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3071/6209] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:55.420645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c3050-8837-494a-960f-9c4f0dce6520 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:55.435593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c3050-8837-494a-960f-9c4f0dce6520 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:55.439279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0c608-7bc2-4a18-910d-4ea0c2c2dd04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:55.439961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0c608-7bc2-4a18-910d-4ea0c2c2dd04 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:55.440327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0c608-7bc2-4a18-910d-4ea0c2c2dd04 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:55.440855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0c608-7bc2-4a18-910d-4ea0c2c2dd04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:55.446099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:55.510589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0c608-7bc2-4a18-910d-4ea0c2c2dd04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:55.512031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0c608-7bc2-4a18-910d-4ea0c2c2dd04 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:55.529454 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3139/6210] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:57.075957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079123e-b493-4271-b6e5-d3c75711fdc8 service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:57.147909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079123e-b493-4271-b6e5-d3c75711fdc8 service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:57.153781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:57.154522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:57.154799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:57.155106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] PATH_INFO: `/v3/users/9606cc0443d249c0aa70a3033476d841` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:57.156293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9606cc0443d249c0aa70a3033476d841)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:57.158312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:57.214894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9606cc0443d249c0aa70a3033476d841'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:57.225050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9606cc0443d249c0aa70a3033476d841'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:57.245419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9606cc0443d249c0aa70a3033476d841'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:57.246180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9606cc0443d249c0aa70a3033476d841'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:57.246802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9606cc0443d249c0aa70a3033476d841'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:57.249150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:57.264874 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3072/6211] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:57 2022] DELETE /identity/v3/users/9606cc0443d249c0aa70a3033476d841 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:57.269690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba67b2-e0eb-43e3-8927-30a34c745c54 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:57.270216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba67b2-e0eb-43e3-8927-30a34c745c54 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:57.270216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba67b2-e0eb-43e3-8927-30a34c745c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:57.290946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8cba67b2-e0eb-43e3-8927-30a34c745c54 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:57.292166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8cba67b2-e0eb-43e3-8927-30a34c745c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:57.358310 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3140/6212] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:57.365111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:57.366770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:57.442842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571c0f6-6738-47c9-b6f8-3e85f98446bf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:57.448037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387ce2b-2a3d-4080-9b91-6d93df455f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:57.448762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387ce2b-2a3d-4080-9b91-6d93df455f55 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:57.449103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387ce2b-2a3d-4080-9b91-6d93df455f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:57.449441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387ce2b-2a3d-4080-9b91-6d93df455f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:57.461325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8387ce2b-2a3d-4080-9b91-6d93df455f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:57.462843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8387ce2b-2a3d-4080-9b91-6d93df455f55 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:57.475186 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3073/6213] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:18:58.304033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba67b2-e0eb-43e3-8927-30a34c745c54 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:58.306840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8cba67b2-e0eb-43e3-8927-30a34c745c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:58.325123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba67b2-e0eb-43e3-8927-30a34c745c54 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:58.338092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:58.339120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:58.339714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:58.340589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] PATH_INFO: `/v3/projects/6e5011e2175d41d9abb76962b6c61221` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:58.342103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e5011e2175d41d9abb76962b6c61221)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:58.344326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:58.437334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e5011e2175d41d9abb76962b6c61221', 'request_id': 'req-a098089f-85a1-4d73-848c-8780ea0fa621'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:58.449789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e5011e2175d41d9abb76962b6c61221', 'request_id': 'req-a098089f-85a1-4d73-848c-8780ea0fa621'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:58.460727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e5011e2175d41d9abb76962b6c61221', 'request_id': 'req-a098089f-85a1-4d73-848c-8780ea0fa621'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:58.479328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] The token cache is being invalidate because project 6e5011e2175d41d9abb76962b6c61221 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:58.479605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:58.481976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:58.483693 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3141/6214] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:58 2022] DELETE /identity/v3/projects/6e5011e2175d41d9abb76962b6c61221 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:58.487541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387ce2b-2a3d-4080-9b91-6d93df455f55 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:58.500588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387ce2b-2a3d-4080-9b91-6d93df455f55 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:58.575795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abd6df-18a0-41fd-8b24-844344985af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:58.576647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abd6df-18a0-41fd-8b24-844344985af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:58.576950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abd6df-18a0-41fd-8b24-844344985af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:58.577181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abd6df-18a0-41fd-8b24-844344985af0 admin admin] PATH_INFO: `/v3/users/bf5f98edf7fb468e9fb6766f0a513fee` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:58.578506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abd6df-18a0-41fd-8b24-844344985af0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf5f98edf7fb468e9fb6766f0a513fee)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:58.580497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abd6df-18a0-41fd-8b24-844344985af0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:58.645831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-37abd6df-18a0-41fd-8b24-844344985af0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf5f98edf7fb468e9fb6766f0a513fee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:58.670793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-37abd6df-18a0-41fd-8b24-844344985af0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf5f98edf7fb468e9fb6766f0a513fee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:58.698545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-37abd6df-18a0-41fd-8b24-844344985af0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf5f98edf7fb468e9fb6766f0a513fee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:58.700017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-37abd6df-18a0-41fd-8b24-844344985af0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf5f98edf7fb468e9fb6766f0a513fee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:58.700899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-37abd6df-18a0-41fd-8b24-844344985af0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf5f98edf7fb468e9fb6766f0a513fee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:58.731087 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3074/6215] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:18:58 2022] DELETE /identity/v3/users/bf5f98edf7fb468e9fb6766f0a513fee => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:59.262100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:59.263696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:59.278664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098089f-85a1-4d73-848c-8780ea0fa621 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:59.319335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:59.362187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:59.362862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:59.363134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:59.363398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5 admin admin] PATH_INFO: `/v3/projects/e7c6c29a71a84b0c9c121ad539a5e210` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:59.364554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c6c29a71a84b0c9c121ad539a5e210)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:59.366574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:59.416513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c6c29a71a84b0c9c121ad539a5e210', 'request_id': 'req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:59.424676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c6c29a71a84b0c9c121ad539a5e210', 'request_id': 'req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:59.430444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c6c29a71a84b0c9c121ad539a5e210', 'request_id': 'req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:59.441404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5 admin admin] The token cache is being invalidate because project e7c6c29a71a84b0c9c121ad539a5e210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:18:59.441686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:59.442314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b6e36a58-67ea-4fc4-97fc-250c7e0233d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:18:59.444244 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3142/6216] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:18:59 2022] DELETE /identity/v3/projects/e7c6c29a71a84b0c9c121ad539a5e210 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:18:59.449187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0eb16-fea4-4f83-ad05-17e12a7c9802 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:59.449468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0eb16-fea4-4f83-ad05-17e12a7c9802 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:59.449736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0eb16-fea4-4f83-ad05-17e12a7c9802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:59.469736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-02f0eb16-fea4-4f83-ad05-17e12a7c9802 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:59.470940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-02f0eb16-fea4-4f83-ad05-17e12a7c9802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:59.479690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:18:59.535636 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3075/6217] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:59.543933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0631bc-bb34-4c1f-9500-c837f2db6080 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:59.544475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0631bc-bb34-4c1f-9500-c837f2db6080 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:59.544750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0631bc-bb34-4c1f-9500-c837f2db6080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:59.564839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dd0631bc-bb34-4c1f-9500-c837f2db6080 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:59.565961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dd0631bc-bb34-4c1f-9500-c837f2db6080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:18:59.575775 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3143/6218] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:18:59.580140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0eb16-fea4-4f83-ad05-17e12a7c9802 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:59.590139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0eb16-fea4-4f83-ad05-17e12a7c9802 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:59.594842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a89c14-6909-471d-879d-08f5870edc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:59.595438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a89c14-6909-471d-879d-08f5870edc02 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:59.595703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a89c14-6909-471d-879d-08f5870edc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:59.595965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a89c14-6909-471d-879d-08f5870edc02 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:59.596635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a89c14-6909-471d-879d-08f5870edc02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:18:59.596927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a89c14-6909-471d-879d-08f5870edc02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:59.598507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a89c14-6909-471d-879d-08f5870edc02 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:59.611211 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3076/6219] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:18:59.615834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0631bc-bb34-4c1f-9500-c837f2db6080 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:59.625781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0631bc-bb34-4c1f-9500-c837f2db6080 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:59.630914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939d17f-c9cf-4f9e-aae7-e7cb093a6ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:59.631631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939d17f-c9cf-4f9e-aae7-e7cb093a6ead admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:59.631896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939d17f-c9cf-4f9e-aae7-e7cb093a6ead admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:59.632153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939d17f-c9cf-4f9e-aae7-e7cb093a6ead admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:59.632805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939d17f-c9cf-4f9e-aae7-e7cb093a6ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:59.634720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939d17f-c9cf-4f9e-aae7-e7cb093a6ead admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:59.663616 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3144/6220] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:18:59 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:59.667777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a89c14-6909-471d-879d-08f5870edc02 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:59.677996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a89c14-6909-471d-879d-08f5870edc02 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:59.734100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b89fe-27f5-49f5-ad71-993ef46c853b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:59.734708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b89fe-27f5-49f5-ad71-993ef46c853b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:59.734984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b89fe-27f5-49f5-ad71-993ef46c853b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:59.735249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b89fe-27f5-49f5-ad71-993ef46c853b admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:59.736037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b89fe-27f5-49f5-ad71-993ef46c853b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:59.737825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b89fe-27f5-49f5-ad71-993ef46c853b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:59.760827 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3077/6221] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:18:59 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:18:59.765186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939d17f-c9cf-4f9e-aae7-e7cb093a6ead admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:59.775939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939d17f-c9cf-4f9e-aae7-e7cb093a6ead admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:59.781052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd39008-ac51-4055-87c8-c57219f77d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:59.781821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd39008-ac51-4055-87c8-c57219f77d8a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:59.782090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd39008-ac51-4055-87c8-c57219f77d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:59.782385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd39008-ac51-4055-87c8-c57219f77d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:59.783019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd39008-ac51-4055-87c8-c57219f77d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:59.784717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd39008-ac51-4055-87c8-c57219f77d8a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:59.797594 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3145/6222] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:59.799608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b89fe-27f5-49f5-ad71-993ef46c853b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:59.810151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b89fe-27f5-49f5-ad71-993ef46c853b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:59.815008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcea5a2-9596-4d0a-98e2-e8f975e09289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:59.815534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcea5a2-9596-4d0a-98e2-e8f975e09289 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:59.815816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcea5a2-9596-4d0a-98e2-e8f975e09289 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:59.816062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcea5a2-9596-4d0a-98e2-e8f975e09289 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:59.816757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcea5a2-9596-4d0a-98e2-e8f975e09289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:59.818355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcea5a2-9596-4d0a-98e2-e8f975e09289 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:59.833085 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3078/6223] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:18:59.834601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd39008-ac51-4055-87c8-c57219f77d8a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:18:59.845119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd39008-ac51-4055-87c8-c57219f77d8a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:18:59.849928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8721b2f-773f-4f55-b942-35ff3127cfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:18:59.850758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8721b2f-773f-4f55-b942-35ff3127cfca admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:59.851020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8721b2f-773f-4f55-b942-35ff3127cfca admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:59.851279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8721b2f-773f-4f55-b942-35ff3127cfca admin admin] PATH_INFO: `/v3/projects/733b0c556e6f4f87b873a6b8e2b999f2/users/5fe4c53ad928499c873e88897cdb2d47/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:59.870907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8721b2f-773f-4f55-b942-35ff3127cfca admin admin] RBAC: Authorizing `identity:create_grant(project_id=733b0c556e6f4f87b873a6b8e2b999f2, user_id=5fe4c53ad928499c873e88897cdb2d47, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:18:59.873659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8721b2f-773f-4f55-b942-35ff3127cfca admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:18:59.888192 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3146/6224] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:18:59 2022] PUT /identity/v3/projects/733b0c556e6f4f87b873a6b8e2b999f2/users/5fe4c53ad928499c873e88897cdb2d47/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:18:59.900887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc985b-f394-416f-b4e1-2f902f93428d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:18:59.901172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc985b-f394-416f-b4e1-2f902f93428d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:18:59.901442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc985b-f394-416f-b4e1-2f902f93428d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:18:59.930258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4cfc985b-f394-416f-b4e1-2f902f93428d None None] MFA Rules not processed for user `5fe4c53ad928499c873e88897cdb2d47`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:18:59.931522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4cfc985b-f394-416f-b4e1-2f902f93428d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:00.002012 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3079/6225] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:00.007158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8721b2f-773f-4f55-b942-35ff3127cfca admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:00.069556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:00.075350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8721b2f-773f-4f55-b942-35ff3127cfca admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:00.082086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5afdb-88b3-4d56-8e6d-ed6c29356291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:00.083364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5afdb-88b3-4d56-8e6d-ed6c29356291 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:00.084174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5afdb-88b3-4d56-8e6d-ed6c29356291 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:00.084174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5afdb-88b3-4d56-8e6d-ed6c29356291 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:00.085306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5afdb-88b3-4d56-8e6d-ed6c29356291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:00.087184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5afdb-88b3-4d56-8e6d-ed6c29356291 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:00.118127 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3147/6226] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:19:00 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:00.119391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc985b-f394-416f-b4e1-2f902f93428d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:00.130461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc985b-f394-416f-b4e1-2f902f93428d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:00.135316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:00.197709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0d935-2626-4352-ad87-c877d66321e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:00.198338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0d935-2626-4352-ad87-c877d66321e9 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:00.198616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0d935-2626-4352-ad87-c877d66321e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:00.199025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0d935-2626-4352-ad87-c877d66321e9 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:00.199823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0d935-2626-4352-ad87-c877d66321e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:00.201860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0d935-2626-4352-ad87-c877d66321e9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:00.236462 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3080/6227] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:19:00 2022] POST /identity/v3/users => generated 290 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:00.240931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5afdb-88b3-4d56-8e6d-ed6c29356291 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:00.256989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5afdb-88b3-4d56-8e6d-ed6c29356291 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:00.272425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0fcda-330a-45d1-9401-95c2961b6d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:00.275188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0fcda-330a-45d1-9401-95c2961b6d9a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:00.275476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0fcda-330a-45d1-9401-95c2961b6d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:00.275880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0fcda-330a-45d1-9401-95c2961b6d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:00.287286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0fcda-330a-45d1-9401-95c2961b6d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:00.289119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0fcda-330a-45d1-9401-95c2961b6d9a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:00.303155 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3148/6228] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:00.308342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0d935-2626-4352-ad87-c877d66321e9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:00.322662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0d935-2626-4352-ad87-c877d66321e9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:00.332626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e244e-f1ba-42a8-a640-1e99896070af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:00.333383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e244e-f1ba-42a8-a640-1e99896070af admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:00.333666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e244e-f1ba-42a8-a640-1e99896070af admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:00.333904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e244e-f1ba-42a8-a640-1e99896070af admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:00.335542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e244e-f1ba-42a8-a640-1e99896070af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:00.337751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e244e-f1ba-42a8-a640-1e99896070af admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:00.351877 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3081/6229] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:00.355903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0fcda-330a-45d1-9401-95c2961b6d9a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:00.367699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0fcda-330a-45d1-9401-95c2961b6d9a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:00.372535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e07a9a-c258-43ff-88d8-e91e8d47f468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:00.373204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e07a9a-c258-43ff-88d8-e91e8d47f468 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:00.373475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e07a9a-c258-43ff-88d8-e91e8d47f468 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:00.373738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e07a9a-c258-43ff-88d8-e91e8d47f468 admin admin] PATH_INFO: `/v3/projects/6b447d3605ef4786b9cefdbcf4bbe13d/users/843a9ebbae524a38a9d457bd35e88171/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:00.390753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e07a9a-c258-43ff-88d8-e91e8d47f468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b447d3605ef4786b9cefdbcf4bbe13d, user_id=843a9ebbae524a38a9d457bd35e88171, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:00.393450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e07a9a-c258-43ff-88d8-e91e8d47f468 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:00.412651 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3149/6230] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:19:00 2022] PUT /identity/v3/projects/6b447d3605ef4786b9cefdbcf4bbe13d/users/843a9ebbae524a38a9d457bd35e88171/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:19:00.423178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc28db-5e11-4317-b559-c162a66e6e0f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:00.423454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc28db-5e11-4317-b559-c162a66e6e0f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:00.423742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc28db-5e11-4317-b559-c162a66e6e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:00.453588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d4dc28db-5e11-4317-b559-c162a66e6e0f None None] MFA Rules not processed for user `843a9ebbae524a38a9d457bd35e88171`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:00.454808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d4dc28db-5e11-4317-b559-c162a66e6e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:00.526741 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3082/6231] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:19:00.531369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e07a9a-c258-43ff-88d8-e91e8d47f468 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:00.597874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e07a9a-c258-43ff-88d8-e91e8d47f468 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:00.603287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b60fdb-05d2-4e4b-b177-08f196ce6656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:00.603925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b60fdb-05d2-4e4b-b177-08f196ce6656 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:00.604196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b60fdb-05d2-4e4b-b177-08f196ce6656 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:00.604628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b60fdb-05d2-4e4b-b177-08f196ce6656 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:00.605259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b60fdb-05d2-4e4b-b177-08f196ce6656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:00.607175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b60fdb-05d2-4e4b-b177-08f196ce6656 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:00.638182 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3150/6232] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:19:00 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:00.642407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc28db-5e11-4317-b559-c162a66e6e0f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:00.662143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc28db-5e11-4317-b559-c162a66e6e0f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:00.734492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1dac4-2e67-44f9-9a9f-825740cbeffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:00.734813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1dac4-2e67-44f9-9a9f-825740cbeffc admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:00.735058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1dac4-2e67-44f9-9a9f-825740cbeffc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:00.735290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1dac4-2e67-44f9-9a9f-825740cbeffc admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:00.735956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1dac4-2e67-44f9-9a9f-825740cbeffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:00.737920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1dac4-2e67-44f9-9a9f-825740cbeffc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:00.771485 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3083/6233] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:19:00 2022] POST /identity/v3/users => generated 296 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:00.772719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b60fdb-05d2-4e4b-b177-08f196ce6656 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:00.785769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b60fdb-05d2-4e4b-b177-08f196ce6656 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:00.793682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2df799-2181-4e91-b3c3-801f4b3e2525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:00.794404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2df799-2181-4e91-b3c3-801f4b3e2525 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:00.795181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2df799-2181-4e91-b3c3-801f4b3e2525 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:00.795491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2df799-2181-4e91-b3c3-801f4b3e2525 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:00.796280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2df799-2181-4e91-b3c3-801f4b3e2525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:00.798403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2df799-2181-4e91-b3c3-801f4b3e2525 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:00.816619 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3151/6234] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:00.819604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1dac4-2e67-44f9-9a9f-825740cbeffc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:00.830902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1dac4-2e67-44f9-9a9f-825740cbeffc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:00.837088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98a8cb-e429-4237-9139-efbcf6196820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:00.837738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98a8cb-e429-4237-9139-efbcf6196820 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:00.838012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98a8cb-e429-4237-9139-efbcf6196820 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:00.838276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98a8cb-e429-4237-9139-efbcf6196820 admin admin] PATH_INFO: `/v3/projects/06f1ee8a49484ef9b6c00dd868e82f6f/users/b5358a10fd2f4777b0125e34083017c1/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:00.841562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:00.865080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98a8cb-e429-4237-9139-efbcf6196820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f1ee8a49484ef9b6c00dd868e82f6f, user_id=b5358a10fd2f4777b0125e34083017c1, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:00.868270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98a8cb-e429-4237-9139-efbcf6196820 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:00.883804 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3084/6235] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:19:00 2022] PUT /identity/v3/projects/06f1ee8a49484ef9b6c00dd868e82f6f/users/b5358a10fd2f4777b0125e34083017c1/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:00.887796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2df799-2181-4e91-b3c3-801f4b3e2525 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:00.898787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2df799-2181-4e91-b3c3-801f4b3e2525 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:00.959873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3556b5-9a29-45d4-85c9-47a6cac2738f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:00.960510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3556b5-9a29-45d4-85c9-47a6cac2738f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:00.960778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3556b5-9a29-45d4-85c9-47a6cac2738f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:00.961037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3556b5-9a29-45d4-85c9-47a6cac2738f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:00.961654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3556b5-9a29-45d4-85c9-47a6cac2738f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:00.963228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3556b5-9a29-45d4-85c9-47a6cac2738f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:00.976599 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3152/6236] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:00.979915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98a8cb-e429-4237-9139-efbcf6196820 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:00.990326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98a8cb-e429-4237-9139-efbcf6196820 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:00.995199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210a41c-d0cf-4254-8889-f1d0d042d7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:00.995968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210a41c-d0cf-4254-8889-f1d0d042d7a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:00.996299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210a41c-d0cf-4254-8889-f1d0d042d7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:00.996752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210a41c-d0cf-4254-8889-f1d0d042d7a0 admin admin] PATH_INFO: `/v3/projects/06f1ee8a49484ef9b6c00dd868e82f6f/users/b5358a10fd2f4777b0125e34083017c1/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:00.998711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210a41c-d0cf-4254-8889-f1d0d042d7a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f1ee8a49484ef9b6c00dd868e82f6f, user_id=b5358a10fd2f4777b0125e34083017c1, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:01.001471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210a41c-d0cf-4254-8889-f1d0d042d7a0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:01.012851 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3085/6237] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:19:00 2022] PUT /identity/v3/projects/06f1ee8a49484ef9b6c00dd868e82f6f/users/b5358a10fd2f4777b0125e34083017c1/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:01.023692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bf49f-2c5d-4cc7-b7f4-216e26c0b858 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:01.023974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bf49f-2c5d-4cc7-b7f4-216e26c0b858 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:01.024248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bf49f-2c5d-4cc7-b7f4-216e26c0b858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:01.052890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3c8bf49f-2c5d-4cc7-b7f4-216e26c0b858 None None] MFA Rules not processed for user `b5358a10fd2f4777b0125e34083017c1`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:01.054141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3c8bf49f-2c5d-4cc7-b7f4-216e26c0b858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:01.058358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:01.127445 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3153/6238] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:01.139203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210a41c-d0cf-4254-8889-f1d0d042d7a0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:01.141010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4210a41c-d0cf-4254-8889-f1d0d042d7a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:01.205909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210a41c-d0cf-4254-8889-f1d0d042d7a0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:01.210906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bef652-f9fc-478c-9d1c-1f1baf89e059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:01.211512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bef652-f9fc-478c-9d1c-1f1baf89e059 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:01.211798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bef652-f9fc-478c-9d1c-1f1baf89e059 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:01.212044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bef652-f9fc-478c-9d1c-1f1baf89e059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:01.222488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bef652-f9fc-478c-9d1c-1f1baf89e059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:01.224492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bef652-f9fc-478c-9d1c-1f1baf89e059 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:01.235787 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3086/6239] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:19:01.279568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bf49f-2c5d-4cc7-b7f4-216e26c0b858 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:01.280979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3c8bf49f-2c5d-4cc7-b7f4-216e26c0b858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:01.295288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bf49f-2c5d-4cc7-b7f4-216e26c0b858 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:01.299068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b972367-8b79-4aa8-84f8-be25dcd38e24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:01.299582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b972367-8b79-4aa8-84f8-be25dcd38e24 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:01.299849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b972367-8b79-4aa8-84f8-be25dcd38e24 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:01.300134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b972367-8b79-4aa8-84f8-be25dcd38e24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:01.371367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b972367-8b79-4aa8-84f8-be25dcd38e24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:01.372785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b972367-8b79-4aa8-84f8-be25dcd38e24 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:01.384656 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3154/6240] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:19:03.340313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bef652-f9fc-478c-9d1c-1f1baf89e059 service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:03.407861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bef652-f9fc-478c-9d1c-1f1baf89e059 service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:03.412974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:03.413607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:03.413868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:03.414139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] PATH_INFO: `/v3/users/5fe4c53ad928499c873e88897cdb2d47` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:03.416745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fe4c53ad928499c873e88897cdb2d47)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:03.418670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:03.483931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fe4c53ad928499c873e88897cdb2d47'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:03.492052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fe4c53ad928499c873e88897cdb2d47'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:03.504446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fe4c53ad928499c873e88897cdb2d47'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:03.505354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fe4c53ad928499c873e88897cdb2d47'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:03.506172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fe4c53ad928499c873e88897cdb2d47'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:03.508225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:03.521619 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3087/6241] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:19:03 2022] DELETE /identity/v3/users/5fe4c53ad928499c873e88897cdb2d47 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:03.526448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861f1ee-26e5-4b5c-995d-f62ec9818d75 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:03.526977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861f1ee-26e5-4b5c-995d-f62ec9818d75 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:03.526977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861f1ee-26e5-4b5c-995d-f62ec9818d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:03.547039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5861f1ee-26e5-4b5c-995d-f62ec9818d75 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:03.548260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5861f1ee-26e5-4b5c-995d-f62ec9818d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:03.610097 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3155/6242] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:19:03.620965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:03.622437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:03.689220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e855b1-b07b-42d8-b989-9572997c1e29 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:03.692998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea9bf4-f7e4-45f5-b218-a40766f34754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:03.693607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea9bf4-f7e4-45f5-b218-a40766f34754 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:03.694237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea9bf4-f7e4-45f5-b218-a40766f34754 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:03.694696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea9bf4-f7e4-45f5-b218-a40766f34754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:03.704471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea9bf4-f7e4-45f5-b218-a40766f34754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:03.705928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea9bf4-f7e4-45f5-b218-a40766f34754 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:03.718362 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3088/6243] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:19:04.203526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861f1ee-26e5-4b5c-995d-f62ec9818d75 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:04.205129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5861f1ee-26e5-4b5c-995d-f62ec9818d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:04.220847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861f1ee-26e5-4b5c-995d-f62ec9818d75 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:04.244275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:04.245171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:04.245530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:04.246996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] PATH_INFO: `/v3/projects/733b0c556e6f4f87b873a6b8e2b999f2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:04.248213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=733b0c556e6f4f87b873a6b8e2b999f2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:04.250394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:04.299708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:04.312936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '733b0c556e6f4f87b873a6b8e2b999f2', 'request_id': 'req-13a71885-e40a-4586-9dc7-7138b92c2765'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:04.324623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '733b0c556e6f4f87b873a6b8e2b999f2', 'request_id': 'req-13a71885-e40a-4586-9dc7-7138b92c2765'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:04.332028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '733b0c556e6f4f87b873a6b8e2b999f2', 'request_id': 'req-13a71885-e40a-4586-9dc7-7138b92c2765'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:04.355101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] The token cache is being invalidate because project 733b0c556e6f4f87b873a6b8e2b999f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:19:04.355501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:04.356348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:04.360415 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3156/6244] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:19:04 2022] DELETE /identity/v3/projects/733b0c556e6f4f87b873a6b8e2b999f2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:04.362526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea9bf4-f7e4-45f5-b218-a40766f34754 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:04.377429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea9bf4-f7e4-45f5-b218-a40766f34754 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:04.472015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:04.472933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:04.473376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:04.473840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] PATH_INFO: `/v3/users/843a9ebbae524a38a9d457bd35e88171` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:04.475222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=843a9ebbae524a38a9d457bd35e88171)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:04.478501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:04.554109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '843a9ebbae524a38a9d457bd35e88171'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:04.570270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '843a9ebbae524a38a9d457bd35e88171'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:04.595256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '843a9ebbae524a38a9d457bd35e88171'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:04.596008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '843a9ebbae524a38a9d457bd35e88171'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:04.596725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '843a9ebbae524a38a9d457bd35e88171'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:04.614823 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3089/6245] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:19:04 2022] DELETE /identity/v3/users/843a9ebbae524a38a9d457bd35e88171 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:04.998620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:05.000147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:05.015276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a71885-e40a-4586-9dc7-7138b92c2765 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:05.083641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:05.084299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:05.084601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:05.084869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] PATH_INFO: `/v3/projects/6b447d3605ef4786b9cefdbcf4bbe13d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:05.085974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b447d3605ef4786b9cefdbcf4bbe13d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:05.088010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:05.168923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b447d3605ef4786b9cefdbcf4bbe13d', 'request_id': 'req-8db39443-7ffe-48c5-9f80-d5a0ab887754'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.183567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b447d3605ef4786b9cefdbcf4bbe13d', 'request_id': 'req-8db39443-7ffe-48c5-9f80-d5a0ab887754'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.195088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b447d3605ef4786b9cefdbcf4bbe13d', 'request_id': 'req-8db39443-7ffe-48c5-9f80-d5a0ab887754'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.209888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] The token cache is being invalidate because project 6b447d3605ef4786b9cefdbcf4bbe13d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:19:05.210250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.211251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.214456 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3157/6246] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:19:04 2022] DELETE /identity/v3/projects/6b447d3605ef4786b9cefdbcf4bbe13d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:05.219244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:05.221811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:05.264537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcc75c-a4a3-4e2d-8aee-e24aed8d23ac admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:05.278254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:05.345439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:05.346564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:05.346931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:05.347275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] PATH_INFO: `/v3/users/b5358a10fd2f4777b0125e34083017c1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:05.348719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5358a10fd2f4777b0125e34083017c1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:05.350865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:05.405651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5358a10fd2f4777b0125e34083017c1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.415967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5358a10fd2f4777b0125e34083017c1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.428907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5358a10fd2f4777b0125e34083017c1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.429557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5358a10fd2f4777b0125e34083017c1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.430270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5358a10fd2f4777b0125e34083017c1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.444618 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3090/6247] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:19:05 2022] DELETE /identity/v3/users/b5358a10fd2f4777b0125e34083017c1 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:05.724977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:05.726428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:05.740749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db39443-7ffe-48c5-9f80-d5a0ab887754 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:05.791929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:05.802239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c085f-164e-45e7-aed2-9a0c04581976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:05.802881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c085f-164e-45e7-aed2-9a0c04581976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:05.803159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c085f-164e-45e7-aed2-9a0c04581976 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:05.803421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c085f-164e-45e7-aed2-9a0c04581976 admin admin] PATH_INFO: `/v3/projects/06f1ee8a49484ef9b6c00dd868e82f6f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:05.804550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c085f-164e-45e7-aed2-9a0c04581976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06f1ee8a49484ef9b6c00dd868e82f6f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:05.806645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c085f-164e-45e7-aed2-9a0c04581976 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:05.853478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:05.855111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:05.862361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e9c085f-164e-45e7-aed2-9a0c04581976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06f1ee8a49484ef9b6c00dd868e82f6f', 'request_id': 'req-8e9c085f-164e-45e7-aed2-9a0c04581976'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.869906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8b900-ef0d-4b2c-82cd-f65a13c8f08b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:05.871355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e9c085f-164e-45e7-aed2-9a0c04581976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06f1ee8a49484ef9b6c00dd868e82f6f', 'request_id': 'req-8e9c085f-164e-45e7-aed2-9a0c04581976'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.878058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e9c085f-164e-45e7-aed2-9a0c04581976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06f1ee8a49484ef9b6c00dd868e82f6f', 'request_id': 'req-8e9c085f-164e-45e7-aed2-9a0c04581976'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.890093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e9c085f-164e-45e7-aed2-9a0c04581976 admin admin] The token cache is being invalidate because project 06f1ee8a49484ef9b6c00dd868e82f6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:19:05.890386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e9c085f-164e-45e7-aed2-9a0c04581976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.891030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e9c085f-164e-45e7-aed2-9a0c04581976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:05.892593 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3158/6248] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:19:05 2022] DELETE /identity/v3/projects/06f1ee8a49484ef9b6c00dd868e82f6f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:05.898952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddc244-403c-4a39-88d8-45bb070199e5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:05.899408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddc244-403c-4a39-88d8-45bb070199e5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:05.899743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddc244-403c-4a39-88d8-45bb070199e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:05.920309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-33ddc244-403c-4a39-88d8-45bb070199e5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:05.921789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-33ddc244-403c-4a39-88d8-45bb070199e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:05.933887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449c298-0e79-44f0-9199-e7634374bce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:05.934707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449c298-0e79-44f0-9199-e7634374bce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:05.935387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449c298-0e79-44f0-9199-e7634374bce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:05.935387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449c298-0e79-44f0-9199-e7634374bce4 admin admin] PATH_INFO: `/v3/users/9b65cfa237a64065bb5836bb5d580ebc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:05.936634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449c298-0e79-44f0-9199-e7634374bce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b65cfa237a64065bb5836bb5d580ebc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:05.938804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449c298-0e79-44f0-9199-e7634374bce4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:05.991739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:06.001475 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3159/6249] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:06.006935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6449c298-0e79-44f0-9199-e7634374bce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b65cfa237a64065bb5836bb5d580ebc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:06.011627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716358f8-2c00-4f0c-888e-99e4b7b25a12 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:06.012134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716358f8-2c00-4f0c-888e-99e4b7b25a12 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:06.012742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716358f8-2c00-4f0c-888e-99e4b7b25a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:06.018688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6449c298-0e79-44f0-9199-e7634374bce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b65cfa237a64065bb5836bb5d580ebc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:06.033386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6449c298-0e79-44f0-9199-e7634374bce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b65cfa237a64065bb5836bb5d580ebc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:06.033970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6449c298-0e79-44f0-9199-e7634374bce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b65cfa237a64065bb5836bb5d580ebc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:06.034415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-716358f8-2c00-4f0c-888e-99e4b7b25a12 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:06.034589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6449c298-0e79-44f0-9199-e7634374bce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b65cfa237a64065bb5836bb5d580ebc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:06.035634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-716358f8-2c00-4f0c-888e-99e4b7b25a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:06.044040 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3160/6250] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:06.047632 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3091/6251] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:19:05 2022] DELETE /identity/v3/users/9b65cfa237a64065bb5836bb5d580ebc => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:19:06.048525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716358f8-2c00-4f0c-888e-99e4b7b25a12 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:06.051891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18683a95-7431-44fb-816d-72cb0863ae56 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:06.052171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18683a95-7431-44fb-816d-72cb0863ae56 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:06.052501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18683a95-7431-44fb-816d-72cb0863ae56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:06.057914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716358f8-2c00-4f0c-888e-99e4b7b25a12 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:06.074806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-18683a95-7431-44fb-816d-72cb0863ae56 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:06.076315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-18683a95-7431-44fb-816d-72cb0863ae56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:06.122194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecafec-506d-4158-9a52-50cb5666a075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:06.123013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecafec-506d-4158-9a52-50cb5666a075 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:06.123355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecafec-506d-4158-9a52-50cb5666a075 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:06.123682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecafec-506d-4158-9a52-50cb5666a075 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:06.124561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecafec-506d-4158-9a52-50cb5666a075 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:19:06.124918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecafec-506d-4158-9a52-50cb5666a075 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:06.126877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecafec-506d-4158-9a52-50cb5666a075 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:06.140449 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3161/6252] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:19:06.140908 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3092/6253] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:19:06.144175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecafec-506d-4158-9a52-50cb5666a075 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:06.152741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18683a95-7431-44fb-816d-72cb0863ae56 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:06.154541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecafec-506d-4158-9a52-50cb5666a075 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:06.157677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-18683a95-7431-44fb-816d-72cb0863ae56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:06.159720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afffec-a6f1-459e-b037-26292f50ce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:06.160332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afffec-a6f1-459e-b037-26292f50ce77 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:06.160657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afffec-a6f1-459e-b037-26292f50ce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:06.160919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afffec-a6f1-459e-b037-26292f50ce77 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:06.161533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afffec-a6f1-459e-b037-26292f50ce77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:06.163422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afffec-a6f1-459e-b037-26292f50ce77 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:06.174006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:06.199912 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3162/6254] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:19:06 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:06.201165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afffec-a6f1-459e-b037-26292f50ce77 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:06.254256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18683a95-7431-44fb-816d-72cb0863ae56 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:06.283161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afffec-a6f1-459e-b037-26292f50ce77 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:06.288176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d11e4-4d91-486e-80c5-1d6e20efdd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:06.288740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d11e4-4d91-486e-80c5-1d6e20efdd7e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:06.289009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d11e4-4d91-486e-80c5-1d6e20efdd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:06.289369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d11e4-4d91-486e-80c5-1d6e20efdd7e admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:06.290168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d11e4-4d91-486e-80c5-1d6e20efdd7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:06.292263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d11e4-4d91-486e-80c5-1d6e20efdd7e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:06.304737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac56b7-cc0d-46d5-a770-ae9d4ffb6261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:06.305562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac56b7-cc0d-46d5-a770-ae9d4ffb6261 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:06.305934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac56b7-cc0d-46d5-a770-ae9d4ffb6261 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:06.306272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac56b7-cc0d-46d5-a770-ae9d4ffb6261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:06.315595 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3163/6255] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:19:06 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:06.315785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac56b7-cc0d-46d5-a770-ae9d4ffb6261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:06.317112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac56b7-cc0d-46d5-a770-ae9d4ffb6261 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:06.320041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d11e4-4d91-486e-80c5-1d6e20efdd7e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:06.327642 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3093/6256] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:19:06.330401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d11e4-4d91-486e-80c5-1d6e20efdd7e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:06.335094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64086a74-b68f-404c-b9ee-b8000b83f083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:06.335702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64086a74-b68f-404c-b9ee-b8000b83f083 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:06.335979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64086a74-b68f-404c-b9ee-b8000b83f083 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:06.336615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64086a74-b68f-404c-b9ee-b8000b83f083 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:06.337063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64086a74-b68f-404c-b9ee-b8000b83f083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:06.338682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64086a74-b68f-404c-b9ee-b8000b83f083 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:06.349557 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3164/6257] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:19:06.354498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac56b7-cc0d-46d5-a770-ae9d4ffb6261 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:06.366274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac56b7-cc0d-46d5-a770-ae9d4ffb6261 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:06.371733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00472683-b772-4228-be98-a292dca3349d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:06.372896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00472683-b772-4228-be98-a292dca3349d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:06.373238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00472683-b772-4228-be98-a292dca3349d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:06.373596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00472683-b772-4228-be98-a292dca3349d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:06.374325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00472683-b772-4228-be98-a292dca3349d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:06.376438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00472683-b772-4228-be98-a292dca3349d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:06.382831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:06.392888 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3094/6258] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:19:06.394984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64086a74-b68f-404c-b9ee-b8000b83f083 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:06.406695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64086a74-b68f-404c-b9ee-b8000b83f083 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:06.411519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c78fa-1b66-42f6-893a-5f458105e84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:06.412243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c78fa-1b66-42f6-893a-5f458105e84d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:06.412608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c78fa-1b66-42f6-893a-5f458105e84d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:06.412942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c78fa-1b66-42f6-893a-5f458105e84d admin admin] PATH_INFO: `/v3/projects/b1c62b660df4488f99b3fab48dc055d7/users/b034d3ed7cb54971b17aa876ac6dcb96/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:06.428829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c78fa-1b66-42f6-893a-5f458105e84d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c62b660df4488f99b3fab48dc055d7, user_id=b034d3ed7cb54971b17aa876ac6dcb96, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:06.431853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c78fa-1b66-42f6-893a-5f458105e84d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:06.447780 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3165/6259] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:19:06 2022] PUT /identity/v3/projects/b1c62b660df4488f99b3fab48dc055d7/users/b034d3ed7cb54971b17aa876ac6dcb96/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:19:06.466213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323decda-8dc6-41c0-b83d-29447f66d6d9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:06.466838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323decda-8dc6-41c0-b83d-29447f66d6d9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:06.466838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323decda-8dc6-41c0-b83d-29447f66d6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:06.497925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-323decda-8dc6-41c0-b83d-29447f66d6d9 None None] MFA Rules not processed for user `b034d3ed7cb54971b17aa876ac6dcb96`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:06.499448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-323decda-8dc6-41c0-b83d-29447f66d6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:06.569785 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3095/6260] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:06.581790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c78fa-1b66-42f6-893a-5f458105e84d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:06.585730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d72c78fa-1b66-42f6-893a-5f458105e84d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:06.613836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323decda-8dc6-41c0-b83d-29447f66d6d9 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:06.616151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-323decda-8dc6-41c0-b83d-29447f66d6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:06.717619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c78fa-1b66-42f6-893a-5f458105e84d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:06.721028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323decda-8dc6-41c0-b83d-29447f66d6d9 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:06.722323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5529b59-4f52-4f04-8ee8-1564c5f5a163 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:06.722936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5529b59-4f52-4f04-8ee8-1564c5f5a163 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:06.723205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5529b59-4f52-4f04-8ee8-1564c5f5a163 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:06.723521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5529b59-4f52-4f04-8ee8-1564c5f5a163 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:06.728232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:06.729107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:06.729382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:06.729830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] PATH_INFO: `/v3/projects/691da569594e4b22b71c895a2b269f47` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:06.731658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] RBAC: Authorizing `identity:delete_project(project_id=691da569594e4b22b71c895a2b269f47)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:06.732668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5529b59-4f52-4f04-8ee8-1564c5f5a163 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:06.733620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:06.734101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5529b59-4f52-4f04-8ee8-1564c5f5a163 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:06.748864 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3166/6261] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:19:06.797342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '691da569594e4b22b71c895a2b269f47', 'request_id': 'req-73f36791-3474-4cf5-8980-d2b17459279c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:06.803864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '691da569594e4b22b71c895a2b269f47', 'request_id': 'req-73f36791-3474-4cf5-8980-d2b17459279c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:06.814976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '691da569594e4b22b71c895a2b269f47', 'request_id': 'req-73f36791-3474-4cf5-8980-d2b17459279c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:06.828592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] The token cache is being invalidate because project 691da569594e4b22b71c895a2b269f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:19:06.828808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:06.829653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:06.831504 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3096/6262] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:19:06 2022] DELETE /identity/v3/projects/691da569594e4b22b71c895a2b269f47 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:06.837243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5529b59-4f52-4f04-8ee8-1564c5f5a163 service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:06.849035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5529b59-4f52-4f04-8ee8-1564c5f5a163 service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:06.898839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:06.917140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5051c0-d32d-4b48-b645-b918155f2fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:06.918132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5051c0-d32d-4b48-b645-b918155f2fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:06.918310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5051c0-d32d-4b48-b645-b918155f2fce admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:06.918587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5051c0-d32d-4b48-b645-b918155f2fce admin admin] PATH_INFO: `/v3/users/5b2e94a090c447318306be8c5cbfc503` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:06.919778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5051c0-d32d-4b48-b645-b918155f2fce admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b2e94a090c447318306be8c5cbfc503)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:06.921959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5051c0-d32d-4b48-b645-b918155f2fce admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:06.986170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cb5051c0-d32d-4b48-b645-b918155f2fce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b2e94a090c447318306be8c5cbfc503'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:06.997167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cb5051c0-d32d-4b48-b645-b918155f2fce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b2e94a090c447318306be8c5cbfc503'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:07.006032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cb5051c0-d32d-4b48-b645-b918155f2fce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b2e94a090c447318306be8c5cbfc503'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:07.019930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cb5051c0-d32d-4b48-b645-b918155f2fce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b2e94a090c447318306be8c5cbfc503'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:07.021000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cb5051c0-d32d-4b48-b645-b918155f2fce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b2e94a090c447318306be8c5cbfc503'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:07.027904 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3167/6263] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:19:06 2022] DELETE /identity/v3/users/5b2e94a090c447318306be8c5cbfc503 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:07.361410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:07.362943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:07.378305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f36791-3474-4cf5-8980-d2b17459279c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:07.397268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:07.441940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:07.442861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:07.443142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:07.443410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89 admin admin] PATH_INFO: `/v3/projects/8a704c3257944849bb3a9aa4d3c79e13` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:07.445155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a704c3257944849bb3a9aa4d3c79e13)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:07.447197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:07.495469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a704c3257944849bb3a9aa4d3c79e13', 'request_id': 'req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:07.501761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a704c3257944849bb3a9aa4d3c79e13', 'request_id': 'req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:07.510427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a704c3257944849bb3a9aa4d3c79e13', 'request_id': 'req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:07.521584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89 admin admin] The token cache is being invalidate because project 8a704c3257944849bb3a9aa4d3c79e13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:19:07.521831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:07.522571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8afa5ab8-63e6-4d88-8b1c-1fc5239c5a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:07.524472 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3097/6264] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:19:07 2022] DELETE /identity/v3/projects/8a704c3257944849bb3a9aa4d3c79e13 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:07.529358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54fa66-d803-4cbc-b500-baa9f39da6d8 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:07.529635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54fa66-d803-4cbc-b500-baa9f39da6d8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:07.529896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54fa66-d803-4cbc-b500-baa9f39da6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:07.553358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ed54fa66-d803-4cbc-b500-baa9f39da6d8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:07.555068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ed54fa66-d803-4cbc-b500-baa9f39da6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:07.617220 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3168/6265] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:19:07.625536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d6db0-2c8c-4501-9573-b4ba825c757c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:07.625813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d6db0-2c8c-4501-9573-b4ba825c757c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:07.626082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d6db0-2c8c-4501-9573-b4ba825c757c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:07.646805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ec4d6db0-2c8c-4501-9573-b4ba825c757c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:07.648023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ec4d6db0-2c8c-4501-9573-b4ba825c757c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:07.656241 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3098/6266] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:07.660816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed54fa66-d803-4cbc-b500-baa9f39da6d8 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:07.670826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed54fa66-d803-4cbc-b500-baa9f39da6d8 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:07.675425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79c7ee-7e2d-42dd-aab9-51507494512b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:07.676052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79c7ee-7e2d-42dd-aab9-51507494512b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:07.676325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79c7ee-7e2d-42dd-aab9-51507494512b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:07.676618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79c7ee-7e2d-42dd-aab9-51507494512b admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:07.677262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb79c7ee-7e2d-42dd-aab9-51507494512b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:19:07.677559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb79c7ee-7e2d-42dd-aab9-51507494512b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:07.679189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb79c7ee-7e2d-42dd-aab9-51507494512b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:07.691736 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3169/6267] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:19:07.697215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d6db0-2c8c-4501-9573-b4ba825c757c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:07.707198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d6db0-2c8c-4501-9573-b4ba825c757c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:07.713786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73faafd8-fbd7-4dfc-b502-db78cf464868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:07.714282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73faafd8-fbd7-4dfc-b502-db78cf464868 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:07.714643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73faafd8-fbd7-4dfc-b502-db78cf464868 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:07.714981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73faafd8-fbd7-4dfc-b502-db78cf464868 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:07.715673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73faafd8-fbd7-4dfc-b502-db78cf464868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:07.718364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73faafd8-fbd7-4dfc-b502-db78cf464868 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:07.755281 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3099/6268] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:19:07 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:07.763145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79c7ee-7e2d-42dd-aab9-51507494512b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:07.774088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79c7ee-7e2d-42dd-aab9-51507494512b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:07.824317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:07.835885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe879971-1ad8-4c7e-be7f-585f8daa652d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:07.836585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe879971-1ad8-4c7e-be7f-585f8daa652d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:07.836897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe879971-1ad8-4c7e-be7f-585f8daa652d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:07.837164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe879971-1ad8-4c7e-be7f-585f8daa652d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:07.837781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe879971-1ad8-4c7e-be7f-585f8daa652d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:07.839700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe879971-1ad8-4c7e-be7f-585f8daa652d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:07.862697 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3170/6269] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:19:07 2022] POST /identity/v3/users => generated 302 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:07.867371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73faafd8-fbd7-4dfc-b502-db78cf464868 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:07.878438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73faafd8-fbd7-4dfc-b502-db78cf464868 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:07.883467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e95d7e-aa41-4893-9da0-a52159843ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:07.884183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e95d7e-aa41-4893-9da0-a52159843ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:07.884613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e95d7e-aa41-4893-9da0-a52159843ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:07.884883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e95d7e-aa41-4893-9da0-a52159843ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:07.885518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e95d7e-aa41-4893-9da0-a52159843ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:07.887112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e95d7e-aa41-4893-9da0-a52159843ba3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:07.897381 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3100/6270] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:07.901911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe879971-1ad8-4c7e-be7f-585f8daa652d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:07.912544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe879971-1ad8-4c7e-be7f-585f8daa652d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:07.919440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b72aeb-2675-4638-8c79-df85fec64e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:07.920056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b72aeb-2675-4638-8c79-df85fec64e1f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:07.920327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b72aeb-2675-4638-8c79-df85fec64e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:07.920626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b72aeb-2675-4638-8c79-df85fec64e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:07.921261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b72aeb-2675-4638-8c79-df85fec64e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:07.922861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b72aeb-2675-4638-8c79-df85fec64e1f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:07.936620 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3171/6271] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:07.940984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e95d7e-aa41-4893-9da0-a52159843ba3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:07.951823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e95d7e-aa41-4893-9da0-a52159843ba3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:07.957011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c232b14-0625-4ad8-98f4-6fcc32e3117c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:07.957671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c232b14-0625-4ad8-98f4-6fcc32e3117c admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:07.957933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c232b14-0625-4ad8-98f4-6fcc32e3117c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:07.958208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c232b14-0625-4ad8-98f4-6fcc32e3117c admin admin] PATH_INFO: `/v3/projects/c7966ed44e1647089427f5fedadf54bc/users/917b352cca2d4ad2b358e271a75dfd8f/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:07.974178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c232b14-0625-4ad8-98f4-6fcc32e3117c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7966ed44e1647089427f5fedadf54bc, user_id=917b352cca2d4ad2b358e271a75dfd8f, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:07.977265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c232b14-0625-4ad8-98f4-6fcc32e3117c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:07.992094 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3101/6272] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:19:07 2022] PUT /identity/v3/projects/c7966ed44e1647089427f5fedadf54bc/users/917b352cca2d4ad2b358e271a75dfd8f/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:08.003374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0e956-b1ce-4432-95be-f2831c6b40c7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:08.003643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0e956-b1ce-4432-95be-f2831c6b40c7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:08.003930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0e956-b1ce-4432-95be-f2831c6b40c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:08.036460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7ae0e956-b1ce-4432-95be-f2831c6b40c7 None None] MFA Rules not processed for user `917b352cca2d4ad2b358e271a75dfd8f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:08.037409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7ae0e956-b1ce-4432-95be-f2831c6b40c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:08.108976 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3172/6273] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:08 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:19:08.113596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c232b14-0625-4ad8-98f4-6fcc32e3117c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:08.172242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:08.194426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c232b14-0625-4ad8-98f4-6fcc32e3117c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:08.208036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1e7a9-83c2-4597-ac37-f481cd757194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:08.208036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1e7a9-83c2-4597-ac37-f481cd757194 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:08.208036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1e7a9-83c2-4597-ac37-f481cd757194 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:08.208036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1e7a9-83c2-4597-ac37-f481cd757194 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:08.209083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f1e7a9-83c2-4597-ac37-f481cd757194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:08.209083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f1e7a9-83c2-4597-ac37-f481cd757194 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:08.253156 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3102/6274] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:19:08 2022] POST /identity/v3/projects => generated 401 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:08.257339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0e956-b1ce-4432-95be-f2831c6b40c7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:08.269507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0e956-b1ce-4432-95be-f2831c6b40c7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:08.340613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fb913-5e31-4530-b5f8-055b51f33d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:08.341387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fb913-5e31-4530-b5f8-055b51f33d9e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:08.341738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fb913-5e31-4530-b5f8-055b51f33d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:08.342097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fb913-5e31-4530-b5f8-055b51f33d9e admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:08.343127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fb913-5e31-4530-b5f8-055b51f33d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:08.345234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fb913-5e31-4530-b5f8-055b51f33d9e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:08.372994 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3173/6275] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:19:08 2022] POST /identity/v3/users => generated 307 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:08.377256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fb913-5e31-4530-b5f8-055b51f33d9e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:08.377404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1e7a9-83c2-4597-ac37-f481cd757194 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:08.379391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-35f1e7a9-83c2-4597-ac37-f481cd757194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:08.388138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fb913-5e31-4530-b5f8-055b51f33d9e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:08.393422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5463e8a-afd2-4c9b-ab46-2f9fdc770bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:08.394145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5463e8a-afd2-4c9b-ab46-2f9fdc770bcb admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:08.394296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5463e8a-afd2-4c9b-ab46-2f9fdc770bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:08.394716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5463e8a-afd2-4c9b-ab46-2f9fdc770bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:08.395356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5463e8a-afd2-4c9b-ab46-2f9fdc770bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:08.397193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1e7a9-83c2-4597-ac37-f481cd757194 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:08.397349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5463e8a-afd2-4c9b-ab46-2f9fdc770bcb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:08.403095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c779c-a924-4c52-a525-06e17409a07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:08.403745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c779c-a924-4c52-a525-06e17409a07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:08.404011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c779c-a924-4c52-a525-06e17409a07e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:08.404276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c779c-a924-4c52-a525-06e17409a07e admin admin] PATH_INFO: `/v3/users/b034d3ed7cb54971b17aa876ac6dcb96` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:08.405458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c779c-a924-4c52-a525-06e17409a07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b034d3ed7cb54971b17aa876ac6dcb96)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:08.407377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c779c-a924-4c52-a525-06e17409a07e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:08.411206 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3174/6276] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:08.415655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5463e8a-afd2-4c9b-ab46-2f9fdc770bcb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:08.427237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5463e8a-afd2-4c9b-ab46-2f9fdc770bcb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:08.433106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85723b-810e-4be6-ba02-2d67fa785d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:08.433785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85723b-810e-4be6-ba02-2d67fa785d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:08.434060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85723b-810e-4be6-ba02-2d67fa785d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:08.435032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85723b-810e-4be6-ba02-2d67fa785d9c admin admin] PATH_INFO: `/v3/projects/efca5e143abb4f91a05806d984e8e922/users/88713d199ed54e8fb76d83f1ffc18968/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:08.456222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85723b-810e-4be6-ba02-2d67fa785d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=efca5e143abb4f91a05806d984e8e922, user_id=88713d199ed54e8fb76d83f1ffc18968, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:08.459817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85723b-810e-4be6-ba02-2d67fa785d9c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:08.467377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-453c779c-a924-4c52-a525-06e17409a07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b034d3ed7cb54971b17aa876ac6dcb96'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:08.475832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-453c779c-a924-4c52-a525-06e17409a07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b034d3ed7cb54971b17aa876ac6dcb96'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:08.479890 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3175/6277] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:19:08 2022] PUT /identity/v3/projects/efca5e143abb4f91a05806d984e8e922/users/88713d199ed54e8fb76d83f1ffc18968/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:19:08.484192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85723b-810e-4be6-ba02-2d67fa785d9c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:08.498066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-453c779c-a924-4c52-a525-06e17409a07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b034d3ed7cb54971b17aa876ac6dcb96'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:08.499191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-453c779c-a924-4c52-a525-06e17409a07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b034d3ed7cb54971b17aa876ac6dcb96'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:08.499873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-453c779c-a924-4c52-a525-06e17409a07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b034d3ed7cb54971b17aa876ac6dcb96'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:08.502866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:08.516433 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3103/6278] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:19:08 2022] DELETE /identity/v3/users/b034d3ed7cb54971b17aa876ac6dcb96 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:08.521149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7228ea-4956-4065-86e7-149f0d4b89c2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:08.521663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7228ea-4956-4065-86e7-149f0d4b89c2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:08.522126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7228ea-4956-4065-86e7-149f0d4b89c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:08.545716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ba7228ea-4956-4065-86e7-149f0d4b89c2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:08.546967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ba7228ea-4956-4065-86e7-149f0d4b89c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:08.561587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85723b-810e-4be6-ba02-2d67fa785d9c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:08.622430 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3104/6279] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:08.632658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7228ea-4956-4065-86e7-149f0d4b89c2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:08.634364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ba7228ea-4956-4065-86e7-149f0d4b89c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:08.643144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2920d6a-6623-4981-98f6-32fe7bb07c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:08.643689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2920d6a-6623-4981-98f6-32fe7bb07c01 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:08.643969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2920d6a-6623-4981-98f6-32fe7bb07c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:08.644238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2920d6a-6623-4981-98f6-32fe7bb07c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:08.644917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2920d6a-6623-4981-98f6-32fe7bb07c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:08.646549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2920d6a-6623-4981-98f6-32fe7bb07c01 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:08.659807 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3176/6280] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:19:08.664259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2920d6a-6623-4981-98f6-32fe7bb07c01 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:08.666724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c2920d6a-6623-4981-98f6-32fe7bb07c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:08.681052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2920d6a-6623-4981-98f6-32fe7bb07c01 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:08.686016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f36b8-35f3-4d7c-a6bc-006d48aed0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:08.686700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f36b8-35f3-4d7c-a6bc-006d48aed0e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:08.686968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f36b8-35f3-4d7c-a6bc-006d48aed0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:08.687231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f36b8-35f3-4d7c-a6bc-006d48aed0e1 admin admin] PATH_INFO: `/v3/projects/efca5e143abb4f91a05806d984e8e922/users/88713d199ed54e8fb76d83f1ffc18968/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:08.690585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f36b8-35f3-4d7c-a6bc-006d48aed0e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efca5e143abb4f91a05806d984e8e922, user_id=88713d199ed54e8fb76d83f1ffc18968, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:08.691072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:08.693734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f36b8-35f3-4d7c-a6bc-006d48aed0e1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:08.706558 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3177/6281] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:19:08 2022] PUT /identity/v3/projects/efca5e143abb4f91a05806d984e8e922/users/88713d199ed54e8fb76d83f1ffc18968/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:19:08.710327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7228ea-4956-4065-86e7-149f0d4b89c2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:08.717206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dbd70-a767-42fb-9b80-fee517ad6414 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:08.717515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dbd70-a767-42fb-9b80-fee517ad6414 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:08.717811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dbd70-a767-42fb-9b80-fee517ad6414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:08.753041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5e2dbd70-a767-42fb-9b80-fee517ad6414 None None] MFA Rules not processed for user `88713d199ed54e8fb76d83f1ffc18968`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:08.754285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5e2dbd70-a767-42fb-9b80-fee517ad6414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:08.772151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b1840-3871-43df-ae56-b3b8001fd005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:08.772809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b1840-3871-43df-ae56-b3b8001fd005 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:08.773074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b1840-3871-43df-ae56-b3b8001fd005 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:08.773341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b1840-3871-43df-ae56-b3b8001fd005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:08.849252 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3178/6282] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:08 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:08.861822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dbd70-a767-42fb-9b80-fee517ad6414 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:08.863471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5e2dbd70-a767-42fb-9b80-fee517ad6414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:08.864424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b1840-3871-43df-ae56-b3b8001fd005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:08.866064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b1840-3871-43df-ae56-b3b8001fd005 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:08.878375 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3105/6283] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:19:08.902594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:08.958350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dbd70-a767-42fb-9b80-fee517ad6414 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:08.964827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819283d-8a70-4f3d-8597-7a6d3182098f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:08.965832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819283d-8a70-4f3d-8597-7a6d3182098f service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:08.965832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819283d-8a70-4f3d-8597-7a6d3182098f service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:08.966036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819283d-8a70-4f3d-8597-7a6d3182098f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:08.976941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819283d-8a70-4f3d-8597-7a6d3182098f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:08.978845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819283d-8a70-4f3d-8597-7a6d3182098f service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:08.989370 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3179/6284] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:08 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:19:09.188761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b1840-3871-43df-ae56-b3b8001fd005 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:09.190895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-660b1840-3871-43df-ae56-b3b8001fd005 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:09.207761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b1840-3871-43df-ae56-b3b8001fd005 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:09.231671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:09.232939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:09.232939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:09.233151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] PATH_INFO: `/v3/projects/b1c62b660df4488f99b3fab48dc055d7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:09.234984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c62b660df4488f99b3fab48dc055d7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:09.237104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:09.305291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c62b660df4488f99b3fab48dc055d7', 'request_id': 'req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:09.312392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c62b660df4488f99b3fab48dc055d7', 'request_id': 'req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:09.323686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c62b660df4488f99b3fab48dc055d7', 'request_id': 'req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:09.349441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] The token cache is being invalidate because project b1c62b660df4488f99b3fab48dc055d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:19:09.349735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:09.351068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:09.356261 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3106/6285] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:19:09 2022] DELETE /identity/v3/projects/b1c62b660df4488f99b3fab48dc055d7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:09.358086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f0791-7880-4189-b187-cd02cab8d686 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:09.358581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f0791-7880-4189-b187-cd02cab8d686 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:09.358740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f0791-7880-4189-b187-cd02cab8d686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:09.380503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fb2f0791-7880-4189-b187-cd02cab8d686 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:09.381866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fb2f0791-7880-4189-b187-cd02cab8d686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:09.465228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:09.467601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:09.483787 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3180/6286] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:09.492749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4f76b-2fc7-4e26-bdcf-66161be36934 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:09.493109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4f76b-2fc7-4e26-bdcf-66161be36934 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:09.493389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4f76b-2fc7-4e26-bdcf-66161be36934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:09.508458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:09.522326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-77a4f76b-2fc7-4e26-bdcf-66161be36934 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:09.524555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-77a4f76b-2fc7-4e26-bdcf-66161be36934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:09.538702 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3181/6287] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:09.539870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4f76b-2fc7-4e26-bdcf-66161be36934 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:09.549808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4f76b-2fc7-4e26-bdcf-66161be36934 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:09.556246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b9a33-2b1e-4aca-944e-ac41ee0f8b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:09.557508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b9a33-2b1e-4aca-944e-ac41ee0f8b6d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:09.557791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b9a33-2b1e-4aca-944e-ac41ee0f8b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:09.558053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b9a33-2b1e-4aca-944e-ac41ee0f8b6d admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:09.559370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b9a33-2b1e-4aca-944e-ac41ee0f8b6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:19:09.559683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b9a33-2b1e-4aca-944e-ac41ee0f8b6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:09.561770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b9a33-2b1e-4aca-944e-ac41ee0f8b6d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:09.567301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea0165-aefe-4d6e-a7ec-334f0c4dddfe admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:09.575833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2c128-2e8d-459b-a932-c57891d60089 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:09.576726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2c128-2e8d-459b-a932-c57891d60089 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:09.577812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2c128-2e8d-459b-a932-c57891d60089 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:09.578505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2c128-2e8d-459b-a932-c57891d60089 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:09.580477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3ab2c128-2e8d-459b-a932-c57891d60089 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:09.580627 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3182/6288] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:19:09.585154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b9a33-2b1e-4aca-944e-ac41ee0f8b6d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:09.596577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b9a33-2b1e-4aca-944e-ac41ee0f8b6d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:09.602608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc44d14-630e-42c1-af63-501e2c7a59ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:09.603225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc44d14-630e-42c1-af63-501e2c7a59ca admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:09.603497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc44d14-630e-42c1-af63-501e2c7a59ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:09.603763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc44d14-630e-42c1-af63-501e2c7a59ca admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:09.604401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc44d14-630e-42c1-af63-501e2c7a59ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:09.607405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc44d14-630e-42c1-af63-501e2c7a59ca admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:09.642833 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3183/6289] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:19:09 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:19:09.647692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc44d14-630e-42c1-af63-501e2c7a59ca admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:09.657151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2c128-2e8d-459b-a932-c57891d60089 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:09.659210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2c128-2e8d-459b-a932-c57891d60089 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:09.671055 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3107/6290] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:09 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:19:09.707809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2c128-2e8d-459b-a932-c57891d60089 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:09.710048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3ab2c128-2e8d-459b-a932-c57891d60089 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:09.741822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2c128-2e8d-459b-a932-c57891d60089 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:09.747855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc44d14-630e-42c1-af63-501e2c7a59ca admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:09.756573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59b0d9-e695-48a0-879f-8f18210ee892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:09.756866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d15cc4-6744-4a63-b039-0a7363762541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:09.757382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59b0d9-e695-48a0-879f-8f18210ee892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:09.757666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59b0d9-e695-48a0-879f-8f18210ee892 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:09.757785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d15cc4-6744-4a63-b039-0a7363762541 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:09.758129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59b0d9-e695-48a0-879f-8f18210ee892 admin admin] PATH_INFO: `/v3/users/917b352cca2d4ad2b358e271a75dfd8f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:09.758240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d15cc4-6744-4a63-b039-0a7363762541 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:09.758550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d15cc4-6744-4a63-b039-0a7363762541 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:09.760635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d15cc4-6744-4a63-b039-0a7363762541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:09.760797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59b0d9-e695-48a0-879f-8f18210ee892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=917b352cca2d4ad2b358e271a75dfd8f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:09.762270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59b0d9-e695-48a0-879f-8f18210ee892 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:09.762386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d15cc4-6744-4a63-b039-0a7363762541 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:09.799283 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3184/6291] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:19:09 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:09.804832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d15cc4-6744-4a63-b039-0a7363762541 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:09.822217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:09.829538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d15cc4-6744-4a63-b039-0a7363762541 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:09.837653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8cf80f-205c-4543-a056-cacee3fa2530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:09.839142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cf80f-205c-4543-a056-cacee3fa2530 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:09.840038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cf80f-205c-4543-a056-cacee3fa2530 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:09.840333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cf80f-205c-4543-a056-cacee3fa2530 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:09.842079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8cf80f-205c-4543-a056-cacee3fa2530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:09.854926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8cf80f-205c-4543-a056-cacee3fa2530 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:09.883857 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3185/6292] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:09.886635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8cf80f-205c-4543-a056-cacee3fa2530 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:09.889064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9a59b0d9-e695-48a0-879f-8f18210ee892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '917b352cca2d4ad2b358e271a75dfd8f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:09.908719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8cf80f-205c-4543-a056-cacee3fa2530 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:09.911605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9a59b0d9-e695-48a0-879f-8f18210ee892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '917b352cca2d4ad2b358e271a75dfd8f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:09.919366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3e73a-1a20-412e-98f9-27b38f4777c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:09.921668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3e73a-1a20-412e-98f9-27b38f4777c4 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:09.922701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3e73a-1a20-412e-98f9-27b38f4777c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:09.924304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3e73a-1a20-412e-98f9-27b38f4777c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:09.925517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3e73a-1a20-412e-98f9-27b38f4777c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:09.926752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9a59b0d9-e695-48a0-879f-8f18210ee892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '917b352cca2d4ad2b358e271a75dfd8f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:09.927910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9a59b0d9-e695-48a0-879f-8f18210ee892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '917b352cca2d4ad2b358e271a75dfd8f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:09.929277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3e73a-1a20-412e-98f9-27b38f4777c4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:09.937268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9a59b0d9-e695-48a0-879f-8f18210ee892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '917b352cca2d4ad2b358e271a75dfd8f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:09.953738 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3186/6293] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:09.957634 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3108/6294] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:19:09 2022] DELETE /identity/v3/users/917b352cca2d4ad2b358e271a75dfd8f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:19:09.958600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3e73a-1a20-412e-98f9-27b38f4777c4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:09.959095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d35d9-72f9-46e5-888b-96a5474fe7af None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:09.959376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d35d9-72f9-46e5-888b-96a5474fe7af None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:09.959644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d35d9-72f9-46e5-888b-96a5474fe7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:09.965653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4db3e73a-1a20-412e-98f9-27b38f4777c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:09.985636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c08d35d9-72f9-46e5-888b-96a5474fe7af None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:09.987498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c08d35d9-72f9-46e5-888b-96a5474fe7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:09.991400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3e73a-1a20-412e-98f9-27b38f4777c4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:10.056346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:10.093579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7f7f2-b264-45f4-a130-d9ea261cb8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:10.094252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7f7f2-b264-45f4-a130-d9ea261cb8c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:10.094599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7f7f2-b264-45f4-a130-d9ea261cb8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:10.094871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7f7f2-b264-45f4-a130-d9ea261cb8c7 admin admin] PATH_INFO: `/v3/projects/8cb608dad7694608b123c0c9ca95313b/users/db31ff6066d84eb29fc968a700665dc6/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:10.110282 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3109/6295] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:10.114819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7f7f2-b264-45f4-a130-d9ea261cb8c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cb608dad7694608b123c0c9ca95313b, user_id=db31ff6066d84eb29fc968a700665dc6, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:10.118134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7f7f2-b264-45f4-a130-d9ea261cb8c7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:10.136409 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3187/6296] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:19:09 2022] PUT /identity/v3/projects/8cb608dad7694608b123c0c9ca95313b/users/db31ff6066d84eb29fc968a700665dc6/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:10.138047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d35d9-72f9-46e5-888b-96a5474fe7af None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:10.139607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c08d35d9-72f9-46e5-888b-96a5474fe7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:10.150527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575625c-bc1e-4278-9d2c-c71932f00516 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:10.151115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575625c-bc1e-4278-9d2c-c71932f00516 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:10.151576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575625c-bc1e-4278-9d2c-c71932f00516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:10.197182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6575625c-bc1e-4278-9d2c-c71932f00516 None None] MFA Rules not processed for user `db31ff6066d84eb29fc968a700665dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:10.199290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6575625c-bc1e-4278-9d2c-c71932f00516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:10.216631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d35d9-72f9-46e5-888b-96a5474fe7af None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:10.283836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54adfd43-82ac-4c5a-9f93-4fbc381388ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:10.284627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adfd43-82ac-4c5a-9f93-4fbc381388ed service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:10.285428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adfd43-82ac-4c5a-9f93-4fbc381388ed service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:10.285873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adfd43-82ac-4c5a-9f93-4fbc381388ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:10.313112 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3188/6297] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:10.329823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575625c-bc1e-4278-9d2c-c71932f00516 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:10.331902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6575625c-bc1e-4278-9d2c-c71932f00516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:10.385704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54adfd43-82ac-4c5a-9f93-4fbc381388ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:10.387001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54adfd43-82ac-4c5a-9f93-4fbc381388ed service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:10.399004 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3110/6298] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:19:10.436526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575625c-bc1e-4278-9d2c-c71932f00516 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:10.441207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3dfc8-36a1-4c4c-8869-3d370a2c5f9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:10.442034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3dfc8-36a1-4c4c-8869-3d370a2c5f9b service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:10.442468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3dfc8-36a1-4c4c-8869-3d370a2c5f9b service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:10.442882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3dfc8-36a1-4c4c-8869-3d370a2c5f9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:10.455533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3dfc8-36a1-4c4c-8869-3d370a2c5f9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:10.457630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3dfc8-36a1-4c4c-8869-3d370a2c5f9b service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:10.463812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:10.475347 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3189/6299] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:19:10.816799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54adfd43-82ac-4c5a-9f93-4fbc381388ed service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:10.818339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-54adfd43-82ac-4c5a-9f93-4fbc381388ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:10.838785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54adfd43-82ac-4c5a-9f93-4fbc381388ed service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:10.846229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:10.847364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:10.847960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:10.848120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] PATH_INFO: `/v3/projects/c7966ed44e1647089427f5fedadf54bc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:10.854726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7966ed44e1647089427f5fedadf54bc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:10.856692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:10.912422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:10.921084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7966ed44e1647089427f5fedadf54bc', 'request_id': 'req-d620b541-c7a2-409e-a218-6ba41311a668'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:10.930064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7966ed44e1647089427f5fedadf54bc', 'request_id': 'req-d620b541-c7a2-409e-a218-6ba41311a668'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:10.944584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7966ed44e1647089427f5fedadf54bc', 'request_id': 'req-d620b541-c7a2-409e-a218-6ba41311a668'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:10.957015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] The token cache is being invalidate because project c7966ed44e1647089427f5fedadf54bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:19:10.957366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:10.958113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:10.959879 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3111/6300] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:19:10 2022] DELETE /identity/v3/projects/c7966ed44e1647089427f5fedadf54bc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:19:10.969718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3dfc8-36a1-4c4c-8869-3d370a2c5f9b service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:10.982914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3dfc8-36a1-4c4c-8869-3d370a2c5f9b service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:11.044176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051dd9b-5e0a-40bd-9952-cfaed869c7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:11.045227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051dd9b-5e0a-40bd-9952-cfaed869c7ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:11.045497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051dd9b-5e0a-40bd-9952-cfaed869c7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:11.045787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051dd9b-5e0a-40bd-9952-cfaed869c7ec admin admin] PATH_INFO: `/v3/users/88713d199ed54e8fb76d83f1ffc18968` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:11.046914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051dd9b-5e0a-40bd-9952-cfaed869c7ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=88713d199ed54e8fb76d83f1ffc18968)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:11.048884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051dd9b-5e0a-40bd-9952-cfaed869c7ec admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:11.100514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6051dd9b-5e0a-40bd-9952-cfaed869c7ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88713d199ed54e8fb76d83f1ffc18968'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:11.109095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6051dd9b-5e0a-40bd-9952-cfaed869c7ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88713d199ed54e8fb76d83f1ffc18968'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:11.115976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6051dd9b-5e0a-40bd-9952-cfaed869c7ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88713d199ed54e8fb76d83f1ffc18968'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:11.128283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6051dd9b-5e0a-40bd-9952-cfaed869c7ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88713d199ed54e8fb76d83f1ffc18968'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:11.128973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6051dd9b-5e0a-40bd-9952-cfaed869c7ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88713d199ed54e8fb76d83f1ffc18968'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:11.135154 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3190/6301] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:19:10 2022] DELETE /identity/v3/users/88713d199ed54e8fb76d83f1ffc18968 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:11.430131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:11.431713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:11.445788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620b541-c7a2-409e-a218-6ba41311a668 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:11.509689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b8621-36b9-4be8-bd7b-13c5f589cd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:11.510244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b8621-36b9-4be8-bd7b-13c5f589cd9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:11.510545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b8621-36b9-4be8-bd7b-13c5f589cd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:11.510807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b8621-36b9-4be8-bd7b-13c5f589cd9b admin admin] PATH_INFO: `/v3/projects/efca5e143abb4f91a05806d984e8e922` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:11.512073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b8621-36b9-4be8-bd7b-13c5f589cd9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=efca5e143abb4f91a05806d984e8e922)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:11.514080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b8621-36b9-4be8-bd7b-13c5f589cd9b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:11.567899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-548b8621-36b9-4be8-bd7b-13c5f589cd9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efca5e143abb4f91a05806d984e8e922', 'request_id': 'req-548b8621-36b9-4be8-bd7b-13c5f589cd9b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:11.575451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-548b8621-36b9-4be8-bd7b-13c5f589cd9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efca5e143abb4f91a05806d984e8e922', 'request_id': 'req-548b8621-36b9-4be8-bd7b-13c5f589cd9b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:11.587381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-548b8621-36b9-4be8-bd7b-13c5f589cd9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efca5e143abb4f91a05806d984e8e922', 'request_id': 'req-548b8621-36b9-4be8-bd7b-13c5f589cd9b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:11.598388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-548b8621-36b9-4be8-bd7b-13c5f589cd9b admin admin] The token cache is being invalidate because project efca5e143abb4f91a05806d984e8e922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:19:11.598666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-548b8621-36b9-4be8-bd7b-13c5f589cd9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:11.599432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-548b8621-36b9-4be8-bd7b-13c5f589cd9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:11.601667 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3112/6302] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:19:11 2022] DELETE /identity/v3/projects/efca5e143abb4f91a05806d984e8e922 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:11.607208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b86f8-9023-496d-b017-e3d986127809 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:11.607720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b86f8-9023-496d-b017-e3d986127809 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:11.607720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b86f8-9023-496d-b017-e3d986127809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:11.629452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b71b86f8-9023-496d-b017-e3d986127809 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:11.630634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b71b86f8-9023-496d-b017-e3d986127809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:11.694927 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3191/6303] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:11.703324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f19aad-7e2f-4770-912d-0be65f8450dd None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:11.703600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f19aad-7e2f-4770-912d-0be65f8450dd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:11.703864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f19aad-7e2f-4770-912d-0be65f8450dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:11.726705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f0f19aad-7e2f-4770-912d-0be65f8450dd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:11.727919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f0f19aad-7e2f-4770-912d-0be65f8450dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:11.736316 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3113/6304] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:11.740880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b86f8-9023-496d-b017-e3d986127809 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:11.750779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b86f8-9023-496d-b017-e3d986127809 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:11.755437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f3940-86e6-4a7e-a994-a34d7fb35213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:11.756044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f3940-86e6-4a7e-a994-a34d7fb35213 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:11.756312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f3940-86e6-4a7e-a994-a34d7fb35213 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:11.756629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f3940-86e6-4a7e-a994-a34d7fb35213 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:11.757353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f3940-86e6-4a7e-a994-a34d7fb35213 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:19:11.757566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f3940-86e6-4a7e-a994-a34d7fb35213 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:11.759213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f3940-86e6-4a7e-a994-a34d7fb35213 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:11.770982 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3192/6305] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:19:11.775578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f19aad-7e2f-4770-912d-0be65f8450dd None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:11.785456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f19aad-7e2f-4770-912d-0be65f8450dd None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:11.790424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47826ff-d5b8-4294-937c-e2ec3d3f4440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:11.790999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47826ff-d5b8-4294-937c-e2ec3d3f4440 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:11.791268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47826ff-d5b8-4294-937c-e2ec3d3f4440 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:11.791529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47826ff-d5b8-4294-937c-e2ec3d3f4440 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:11.792261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47826ff-d5b8-4294-937c-e2ec3d3f4440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:11.794230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47826ff-d5b8-4294-937c-e2ec3d3f4440 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:11.829421 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3114/6306] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:19:11 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:11.830857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f3940-86e6-4a7e-a994-a34d7fb35213 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:11.838487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:11.843089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f3940-86e6-4a7e-a994-a34d7fb35213 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:11.907273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9703118-ad17-4667-a7c8-814e8b704439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:11.907960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9703118-ad17-4667-a7c8-814e8b704439 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:11.908241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9703118-ad17-4667-a7c8-814e8b704439 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:11.908547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9703118-ad17-4667-a7c8-814e8b704439 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:11.909176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9703118-ad17-4667-a7c8-814e8b704439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:11.911130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9703118-ad17-4667-a7c8-814e8b704439 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:11.939521 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3193/6307] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:19:11 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:11.940792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47826ff-d5b8-4294-937c-e2ec3d3f4440 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:11.953632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47826ff-d5b8-4294-937c-e2ec3d3f4440 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:11.958777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01610ef4-e803-42d6-a327-8c167dcf5628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:11.959601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610ef4-e803-42d6-a327-8c167dcf5628 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:11.959875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610ef4-e803-42d6-a327-8c167dcf5628 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:11.960137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610ef4-e803-42d6-a327-8c167dcf5628 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:11.960812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01610ef4-e803-42d6-a327-8c167dcf5628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:11.963142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01610ef4-e803-42d6-a327-8c167dcf5628 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:11.976773 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3115/6308] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:11.980803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9703118-ad17-4667-a7c8-814e8b704439 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:11.992089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9703118-ad17-4667-a7c8-814e8b704439 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:11.997613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74b3ef-c7ac-45d4-8d75-3538011c28d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:11.998285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74b3ef-c7ac-45d4-8d75-3538011c28d0 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:11.998559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74b3ef-c7ac-45d4-8d75-3538011c28d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:11.998824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74b3ef-c7ac-45d4-8d75-3538011c28d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:11.999461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d74b3ef-c7ac-45d4-8d75-3538011c28d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:12.001096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d74b3ef-c7ac-45d4-8d75-3538011c28d0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:12.012789 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3194/6309] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:12.017081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01610ef4-e803-42d6-a327-8c167dcf5628 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:12.030182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01610ef4-e803-42d6-a327-8c167dcf5628 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:12.035566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516718d2-eec6-465e-bdb2-d4c5aea2aa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:12.036411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516718d2-eec6-465e-bdb2-d4c5aea2aa3b admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:12.036716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516718d2-eec6-465e-bdb2-d4c5aea2aa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:12.037006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516718d2-eec6-465e-bdb2-d4c5aea2aa3b admin admin] PATH_INFO: `/v3/projects/e8e32728adcb4350b93179a08535090a/users/15b92b4428014995a26cde9537a27f5d/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:12.056024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516718d2-eec6-465e-bdb2-d4c5aea2aa3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8e32728adcb4350b93179a08535090a, user_id=15b92b4428014995a26cde9537a27f5d, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:12.058566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516718d2-eec6-465e-bdb2-d4c5aea2aa3b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:12.067110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:12.074333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74b3ef-c7ac-45d4-8d75-3538011c28d0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:12.074837 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3116/6310] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:19:12 2022] PUT /identity/v3/projects/e8e32728adcb4350b93179a08535090a/users/15b92b4428014995a26cde9537a27f5d/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:12.076441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3d74b3ef-c7ac-45d4-8d75-3538011c28d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:12.079830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875cca5-4fdc-4665-a511-3ab22d3154e1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:12.080110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875cca5-4fdc-4665-a511-3ab22d3154e1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:12.080403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875cca5-4fdc-4665-a511-3ab22d3154e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:12.090405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74b3ef-c7ac-45d4-8d75-3538011c28d0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:12.099896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b875cca5-4fdc-4665-a511-3ab22d3154e1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:12.101144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b875cca5-4fdc-4665-a511-3ab22d3154e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:12.149058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799368d-7d9c-430b-92e4-3210d4c2f02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:12.149814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799368d-7d9c-430b-92e4-3210d4c2f02c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:12.150163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799368d-7d9c-430b-92e4-3210d4c2f02c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:12.150476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799368d-7d9c-430b-92e4-3210d4c2f02c admin admin] PATH_INFO: `/v3/users/db31ff6066d84eb29fc968a700665dc6` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:12.151618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799368d-7d9c-430b-92e4-3210d4c2f02c admin admin] RBAC: Authorizing `identity:delete_user(user_id=db31ff6066d84eb29fc968a700665dc6)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:12.153615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799368d-7d9c-430b-92e4-3210d4c2f02c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:12.159978 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3117/6311] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:12.169844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875cca5-4fdc-4665-a511-3ab22d3154e1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:12.171195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b875cca5-4fdc-4665-a511-3ab22d3154e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:12.209358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8799368d-7d9c-430b-92e4-3210d4c2f02c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db31ff6066d84eb29fc968a700665dc6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:12.219766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8799368d-7d9c-430b-92e4-3210d4c2f02c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db31ff6066d84eb29fc968a700665dc6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:12.234027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8799368d-7d9c-430b-92e4-3210d4c2f02c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db31ff6066d84eb29fc968a700665dc6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:12.234274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875cca5-4fdc-4665-a511-3ab22d3154e1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:12.237249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc98b7-7a01-4c2e-b772-f19c88eaa59a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:12.238023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc98b7-7a01-4c2e-b772-f19c88eaa59a service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:12.238250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc98b7-7a01-4c2e-b772-f19c88eaa59a service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:12.238523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc98b7-7a01-4c2e-b772-f19c88eaa59a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:12.245334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8799368d-7d9c-430b-92e4-3210d4c2f02c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db31ff6066d84eb29fc968a700665dc6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:12.246284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8799368d-7d9c-430b-92e4-3210d4c2f02c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db31ff6066d84eb29fc968a700665dc6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:12.249076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc98b7-7a01-4c2e-b772-f19c88eaa59a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:12.250539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc98b7-7a01-4c2e-b772-f19c88eaa59a service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:12.252942 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3195/6312] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:19:12 2022] DELETE /identity/v3/users/db31ff6066d84eb29fc968a700665dc6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:19:12.257405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e51ca-2f05-4396-92eb-2a82a0df903b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:12.257682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e51ca-2f05-4396-92eb-2a82a0df903b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:12.257958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e51ca-2f05-4396-92eb-2a82a0df903b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:12.260894 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3118/6313] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:19:12.265191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:12.280900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-211e51ca-2f05-4396-92eb-2a82a0df903b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:12.282140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-211e51ca-2f05-4396-92eb-2a82a0df903b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:12.347556 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3196/6314] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:12.359563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc98b7-7a01-4c2e-b772-f19c88eaa59a service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:12.361296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c7cc98b7-7a01-4c2e-b772-f19c88eaa59a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:12.375972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc98b7-7a01-4c2e-b772-f19c88eaa59a service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:12.439800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4db59-cff1-46cf-9fbe-9914cafd1381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:12.440668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4db59-cff1-46cf-9fbe-9914cafd1381 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:12.441057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4db59-cff1-46cf-9fbe-9914cafd1381 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:12.441199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4db59-cff1-46cf-9fbe-9914cafd1381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:12.453342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4db59-cff1-46cf-9fbe-9914cafd1381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:12.454953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4db59-cff1-46cf-9fbe-9914cafd1381 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:12.466704 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3119/6315] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:19:13.033451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e51ca-2f05-4396-92eb-2a82a0df903b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:13.037693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-211e51ca-2f05-4396-92eb-2a82a0df903b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:13.064594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e51ca-2f05-4396-92eb-2a82a0df903b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:13.092797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:13.093879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:13.094296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:13.094606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70 admin admin] PATH_INFO: `/v3/projects/8cb608dad7694608b123c0c9ca95313b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:13.096216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cb608dad7694608b123c0c9ca95313b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:13.098243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:13.181652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cb608dad7694608b123c0c9ca95313b', 'request_id': 'req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:13.193545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cb608dad7694608b123c0c9ca95313b', 'request_id': 'req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:13.205205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cb608dad7694608b123c0c9ca95313b', 'request_id': 'req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:13.231218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70 admin admin] The token cache is being invalidate because project 8cb608dad7694608b123c0c9ca95313b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:19:13.232141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:13.235472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dea6a372-26fd-4fbf-bcb9-7c16ac8d7e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:19:13.242692 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3197/6316] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:19:13 2022] DELETE /identity/v3/projects/8cb608dad7694608b123c0c9ca95313b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:13.258347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f417fe-fe52-45b9-9f07-e4d7414cae4f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:13.259110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f417fe-fe52-45b9-9f07-e4d7414cae4f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:13.259479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f417fe-fe52-45b9-9f07-e4d7414cae4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:13.284305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-01f417fe-fe52-45b9-9f07-e4d7414cae4f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:13.285687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-01f417fe-fe52-45b9-9f07-e4d7414cae4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:13.343091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:13.375259 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3120/6317] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:13.384832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe310b4-e479-4633-b0f9-9c8a300f1edd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:13.385164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe310b4-e479-4633-b0f9-9c8a300f1edd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:13.385530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe310b4-e479-4633-b0f9-9c8a300f1edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:13.414997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6fe310b4-e479-4633-b0f9-9c8a300f1edd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:13.416670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6fe310b4-e479-4633-b0f9-9c8a300f1edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:13.431139 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3198/6318] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:19:13.432665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f417fe-fe52-45b9-9f07-e4d7414cae4f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:13.455630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f417fe-fe52-45b9-9f07-e4d7414cae4f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:13.468048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a127f-3b97-4337-9d21-33f1c601364f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:13.480444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a127f-3b97-4337-9d21-33f1c601364f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:13.480444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a127f-3b97-4337-9d21-33f1c601364f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:13.480444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a127f-3b97-4337-9d21-33f1c601364f admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:13.480444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a127f-3b97-4337-9d21-33f1c601364f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:19:13.480444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a127f-3b97-4337-9d21-33f1c601364f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:13.480444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a127f-3b97-4337-9d21-33f1c601364f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:13.495309 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3121/6319] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:19:13.496336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe310b4-e479-4633-b0f9-9c8a300f1edd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:13.510012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe310b4-e479-4633-b0f9-9c8a300f1edd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:13.515378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6552ad-ff65-44a1-8840-f0b7cec6fca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:13.516068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6552ad-ff65-44a1-8840-f0b7cec6fca1 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:13.516657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6552ad-ff65-44a1-8840-f0b7cec6fca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:13.516995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6552ad-ff65-44a1-8840-f0b7cec6fca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:13.517866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6552ad-ff65-44a1-8840-f0b7cec6fca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:13.520007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6552ad-ff65-44a1-8840-f0b7cec6fca1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:13.564731 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3199/6320] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:19:13 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:19:13.565856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a127f-3b97-4337-9d21-33f1c601364f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:13.578910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a127f-3b97-4337-9d21-33f1c601364f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:13.665388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302dc8a-f8c2-4db3-85bb-a17a99023897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:13.666086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302dc8a-f8c2-4db3-85bb-a17a99023897 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:13.666354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302dc8a-f8c2-4db3-85bb-a17a99023897 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:13.666616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302dc8a-f8c2-4db3-85bb-a17a99023897 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:13.667248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302dc8a-f8c2-4db3-85bb-a17a99023897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:13.669842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302dc8a-f8c2-4db3-85bb-a17a99023897 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:13.700824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6552ad-ff65-44a1-8840-f0b7cec6fca1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:13.701022 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3122/6321] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:19:13 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:19:13.712527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6552ad-ff65-44a1-8840-f0b7cec6fca1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:13.728271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f0339-90aa-4091-a50e-7e7493261d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:13.729012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f0339-90aa-4091-a50e-7e7493261d05 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:13.729294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f0339-90aa-4091-a50e-7e7493261d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:13.729591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f0339-90aa-4091-a50e-7e7493261d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:13.730427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f0339-90aa-4091-a50e-7e7493261d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:13.732216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f0339-90aa-4091-a50e-7e7493261d05 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:13.746454 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3200/6322] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:13.747438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302dc8a-f8c2-4db3-85bb-a17a99023897 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:13.758473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302dc8a-f8c2-4db3-85bb-a17a99023897 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:13.763619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b018a9c-2323-4022-9844-5f09ed4250a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:13.764303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b018a9c-2323-4022-9844-5f09ed4250a9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:13.764667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b018a9c-2323-4022-9844-5f09ed4250a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:13.764993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b018a9c-2323-4022-9844-5f09ed4250a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:13.765700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b018a9c-2323-4022-9844-5f09ed4250a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:13.767383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b018a9c-2323-4022-9844-5f09ed4250a9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:13.782888 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3123/6323] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:13.783941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f0339-90aa-4091-a50e-7e7493261d05 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:13.794535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f0339-90aa-4091-a50e-7e7493261d05 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:13.799660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab0bca-71ef-4cf8-9ab3-767558547c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:13.800324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab0bca-71ef-4cf8-9ab3-767558547c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:13.800632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab0bca-71ef-4cf8-9ab3-767558547c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:13.800904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab0bca-71ef-4cf8-9ab3-767558547c98 admin admin] PATH_INFO: `/v3/projects/ff3eb7954b534dc89fefa2ec56d97fe0/users/daffa49c8fa64a3bb96f917fef222291/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:13.817910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab0bca-71ef-4cf8-9ab3-767558547c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3eb7954b534dc89fefa2ec56d97fe0, user_id=daffa49c8fa64a3bb96f917fef222291, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:13.821462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab0bca-71ef-4cf8-9ab3-767558547c98 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:13.837055 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3201/6324] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:19:13 2022] PUT /identity/v3/projects/ff3eb7954b534dc89fefa2ec56d97fe0/users/daffa49c8fa64a3bb96f917fef222291/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:19:13.848806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125e95a-4988-45f9-918f-0da943338ca2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:13.848806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125e95a-4988-45f9-918f-0da943338ca2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:13.849050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125e95a-4988-45f9-918f-0da943338ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:13.884304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9125e95a-4988-45f9-918f-0da943338ca2 None None] MFA Rules not processed for user `daffa49c8fa64a3bb96f917fef222291`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:13.885563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9125e95a-4988-45f9-918f-0da943338ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:13.981972 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3124/6325] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:13.986538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab0bca-71ef-4cf8-9ab3-767558547c98 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:14.019373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:14.066002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab0bca-71ef-4cf8-9ab3-767558547c98 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:14.089709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63147585-1fbe-441b-aa8a-032bb57d9530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:14.090470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63147585-1fbe-441b-aa8a-032bb57d9530 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:14.090884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63147585-1fbe-441b-aa8a-032bb57d9530 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:14.091250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63147585-1fbe-441b-aa8a-032bb57d9530 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:14.099465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63147585-1fbe-441b-aa8a-032bb57d9530 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:14.102017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63147585-1fbe-441b-aa8a-032bb57d9530 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:14.174152 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3202/6326] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:19:13 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:14.177882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125e95a-4988-45f9-918f-0da943338ca2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:14.190042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125e95a-4988-45f9-918f-0da943338ca2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:14.257981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:14.285579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a37607-6bde-4f5c-ad9a-0be7146b659c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:14.286309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a37607-6bde-4f5c-ad9a-0be7146b659c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:14.286633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a37607-6bde-4f5c-ad9a-0be7146b659c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:14.286901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a37607-6bde-4f5c-ad9a-0be7146b659c admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:14.287542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a37607-6bde-4f5c-ad9a-0be7146b659c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:14.289667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a37607-6bde-4f5c-ad9a-0be7146b659c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:14.317112 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3125/6327] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:19:14 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:19:14.321557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63147585-1fbe-441b-aa8a-032bb57d9530 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:14.332910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63147585-1fbe-441b-aa8a-032bb57d9530 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:14.338052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed5086-ff3b-464c-ad5f-540015d57348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:14.338723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed5086-ff3b-464c-ad5f-540015d57348 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:14.339015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed5086-ff3b-464c-ad5f-540015d57348 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:14.339320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed5086-ff3b-464c-ad5f-540015d57348 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:14.340458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed5086-ff3b-464c-ad5f-540015d57348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:14.341900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed5086-ff3b-464c-ad5f-540015d57348 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:14.358542 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3203/6328] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:19:14.359894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a37607-6bde-4f5c-ad9a-0be7146b659c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:14.373737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a37607-6bde-4f5c-ad9a-0be7146b659c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:14.378787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60c50c-8b1a-4248-b977-f1353d56f46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:14.379448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60c50c-8b1a-4248-b977-f1353d56f46e admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:14.379720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60c50c-8b1a-4248-b977-f1353d56f46e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:14.379989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60c50c-8b1a-4248-b977-f1353d56f46e admin admin] PATH_INFO: `/v3/projects/9141064c318849649c2ec7ac626d7c23/users/a713ae7d938a4cd3b57e2fd7203de82a/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:14.396788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60c50c-8b1a-4248-b977-f1353d56f46e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9141064c318849649c2ec7ac626d7c23, user_id=a713ae7d938a4cd3b57e2fd7203de82a, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:14.399487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60c50c-8b1a-4248-b977-f1353d56f46e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:14.414324 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3126/6329] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:19:14 2022] PUT /identity/v3/projects/9141064c318849649c2ec7ac626d7c23/users/a713ae7d938a4cd3b57e2fd7203de82a/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:14.418387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed5086-ff3b-464c-ad5f-540015d57348 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:14.429500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed5086-ff3b-464c-ad5f-540015d57348 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:14.508423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0ce0b-e910-4a9e-ad98-b1b79ac78118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:14.508957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0ce0b-e910-4a9e-ad98-b1b79ac78118 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:14.509200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0ce0b-e910-4a9e-ad98-b1b79ac78118 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:14.509464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0ce0b-e910-4a9e-ad98-b1b79ac78118 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:14.510100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0ce0b-e910-4a9e-ad98-b1b79ac78118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:14.511743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0ce0b-e910-4a9e-ad98-b1b79ac78118 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:14.522323 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3204/6330] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:19:14.526543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60c50c-8b1a-4248-b977-f1353d56f46e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:14.537239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60c50c-8b1a-4248-b977-f1353d56f46e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:14.543574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79fa56-93fe-4fe6-9189-b55e9cf31e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:14.544242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79fa56-93fe-4fe6-9189-b55e9cf31e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:14.544553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79fa56-93fe-4fe6-9189-b55e9cf31e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:14.544825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79fa56-93fe-4fe6-9189-b55e9cf31e72 admin admin] PATH_INFO: `/v3/projects/9141064c318849649c2ec7ac626d7c23/users/a713ae7d938a4cd3b57e2fd7203de82a/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:14.546985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79fa56-93fe-4fe6-9189-b55e9cf31e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9141064c318849649c2ec7ac626d7c23, user_id=a713ae7d938a4cd3b57e2fd7203de82a, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:14.549690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79fa56-93fe-4fe6-9189-b55e9cf31e72 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:14.561861 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3127/6331] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:19:14 2022] PUT /identity/v3/projects/9141064c318849649c2ec7ac626d7c23/users/a713ae7d938a4cd3b57e2fd7203de82a/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:19:14.573244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854b771-83bb-46d4-9dfe-6df3c0203c5c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:14.573583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854b771-83bb-46d4-9dfe-6df3c0203c5c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:14.573898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854b771-83bb-46d4-9dfe-6df3c0203c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:14.607727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b854b771-83bb-46d4-9dfe-6df3c0203c5c None None] MFA Rules not processed for user `a713ae7d938a4cd3b57e2fd7203de82a`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:14.609057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b854b771-83bb-46d4-9dfe-6df3c0203c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:14.744520 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3205/6332] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:14.756621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79fa56-93fe-4fe6-9189-b55e9cf31e72 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:14.759338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9b79fa56-93fe-4fe6-9189-b55e9cf31e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:14.890543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79fa56-93fe-4fe6-9189-b55e9cf31e72 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:14.901041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b1c57-dca4-46ae-bf86-2f2afbfe974a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:14.902121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b1c57-dca4-46ae-bf86-2f2afbfe974a service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:14.902856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b1c57-dca4-46ae-bf86-2f2afbfe974a service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:14.903291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b1c57-dca4-46ae-bf86-2f2afbfe974a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:14.993236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b1c57-dca4-46ae-bf86-2f2afbfe974a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:14.995343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b1c57-dca4-46ae-bf86-2f2afbfe974a service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:15.006981 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3128/6333] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:19:17.792135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3a9fa-367a-45a3-a625-40dac37a4c1a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:17.792798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3a9fa-367a-45a3-a625-40dac37a4c1a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:17.792798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3a9fa-367a-45a3-a625-40dac37a4c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:17.829979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:17.848077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a2a3a9fa-367a-45a3-a625-40dac37a4c1a None None] MFA Rules not processed for user `15b92b4428014995a26cde9537a27f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:17.851829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a2a3a9fa-367a-45a3-a625-40dac37a4c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:18.052150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b1c57-dca4-46ae-bf86-2f2afbfe974a service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:18.052949 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3206/6334] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:18.057588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c37b1c57-dca4-46ae-bf86-2f2afbfe974a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:18.130356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:18.214315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b1c57-dca4-46ae-bf86-2f2afbfe974a service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:18.236653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd9aad-c076-461c-8592-fbd05694f4ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:18.239119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd9aad-c076-461c-8592-fbd05694f4ce service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:18.239887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd9aad-c076-461c-8592-fbd05694f4ce service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:18.240558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd9aad-c076-461c-8592-fbd05694f4ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:18.251303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd9aad-c076-461c-8592-fbd05694f4ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:18.253710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd9aad-c076-461c-8592-fbd05694f4ce service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:18.268970 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3129/6335] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:19:18.986088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3a9fa-367a-45a3-a625-40dac37a4c1a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:19.000677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3a9fa-367a-45a3-a625-40dac37a4c1a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:19.004776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b04c9f-bbdb-4aca-9874-17819cb8c2ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:19.005416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b04c9f-bbdb-4aca-9874-17819cb8c2ec service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:19.005688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b04c9f-bbdb-4aca-9874-17819cb8c2ec service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:19.005962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b04c9f-bbdb-4aca-9874-17819cb8c2ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:19.031633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b04c9f-bbdb-4aca-9874-17819cb8c2ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:19.033242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b04c9f-bbdb-4aca-9874-17819cb8c2ec service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:19.075495 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3207/6336] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:19:19.096473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c763c6-a9c1-4c45-9ea2-fba51099b662 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:19.097741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c763c6-a9c1-4c45-9ea2-fba51099b662 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:19.098208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c763c6-a9c1-4c45-9ea2-fba51099b662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:19.138877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-37c763c6-a9c1-4c45-9ea2-fba51099b662 None None] MFA Rules not processed for user `15b92b4428014995a26cde9537a27f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:19.142954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-37c763c6-a9c1-4c45-9ea2-fba51099b662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:19.164304 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3130/6337] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:24.234970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c6adc-ab30-43af-9f33-e75bf28a7ae2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:24.235630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c6adc-ab30-43af-9f33-e75bf28a7ae2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:24.235630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c6adc-ab30-43af-9f33-e75bf28a7ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:24.280318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ed8c6adc-ab30-43af-9f33-e75bf28a7ae2 None None] MFA Rules not processed for user `7013eec1908f407d9949d271287b1570`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:24.284173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ed8c6adc-ab30-43af-9f33-e75bf28a7ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:24.412455 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3208/6338] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:27.474856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5737a-1909-41e6-9f1c-66c1146e82de None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:27.475396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5737a-1909-41e6-9f1c-66c1146e82de None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:27.475396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5737a-1909-41e6-9f1c-66c1146e82de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:27.500153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-33b5737a-1909-41e6-9f1c-66c1146e82de None None] MFA Rules not processed for user `daffa49c8fa64a3bb96f917fef222291`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:27.501521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-33b5737a-1909-41e6-9f1c-66c1146e82de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:27.510818 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3131/6339] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:34.401911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=67 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:37.493525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=30 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:19:39.593000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3d8af-5b7a-4b08-bf5d-b73f261faf97 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:39.593530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3d8af-5b7a-4b08-bf5d-b73f261faf97 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:39.593530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3d8af-5b7a-4b08-bf5d-b73f261faf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:39.619918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f9b3d8af-5b7a-4b08-bf5d-b73f261faf97 None None] MFA Rules not processed for user `15b92b4428014995a26cde9537a27f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:39.621239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f9b3d8af-5b7a-4b08-bf5d-b73f261faf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:39.628313 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3209/6340] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:44.155112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c1bb1-b059-4e8d-b287-da2a35e895c1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:44.155649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c1bb1-b059-4e8d-b287-da2a35e895c1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:44.155649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c1bb1-b059-4e8d-b287-da2a35e895c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:44.175595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8e5c1bb1-b059-4e8d-b287-da2a35e895c1 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:44.176838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8e5c1bb1-b059-4e8d-b287-da2a35e895c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:44.253879 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3132/6341] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:19:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:44.279821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3d8af-5b7a-4b08-bf5d-b73f261faf97 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:44.281940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f9b3d8af-5b7a-4b08-bf5d-b73f261faf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:44.367278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3d8af-5b7a-4b08-bf5d-b73f261faf97 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:44.372274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3245e-3868-484b-8edf-463c4afc0507 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:44.372941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3245e-3868-484b-8edf-463c4afc0507 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:44.373209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3245e-3868-484b-8edf-463c4afc0507 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:44.373472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3245e-3868-484b-8edf-463c4afc0507 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:44.381823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3245e-3868-484b-8edf-463c4afc0507 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:44.383767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3245e-3868-484b-8edf-463c4afc0507 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:44.391883 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3210/6342] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:19:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:19:52.687043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c1bb1-b059-4e8d-b287-da2a35e895c1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:52.688622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8e5c1bb1-b059-4e8d-b287-da2a35e895c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:52.722003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c1bb1-b059-4e8d-b287-da2a35e895c1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:52.725692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6d249-1f1e-4a5d-afe9-3701e9dc9f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:52.726751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6d249-1f1e-4a5d-afe9-3701e9dc9f87 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:52.727022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6d249-1f1e-4a5d-afe9-3701e9dc9f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:52.727271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6d249-1f1e-4a5d-afe9-3701e9dc9f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:52.729134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ace6d249-1f1e-4a5d-afe9-3701e9dc9f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:52.747472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6d249-1f1e-4a5d-afe9-3701e9dc9f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:52.748958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6d249-1f1e-4a5d-afe9-3701e9dc9f87 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:52.763995 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3133/6343] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:19:54.193275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19720f57-67e5-4064-935a-ac547e50bba6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:54.194202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19720f57-67e5-4064-935a-ac547e50bba6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:54.194202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19720f57-67e5-4064-935a-ac547e50bba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:54.215421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-19720f57-67e5-4064-935a-ac547e50bba6 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:54.216618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-19720f57-67e5-4064-935a-ac547e50bba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:54.224298 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3211/6344] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:19:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:54.238244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6d249-1f1e-4a5d-afe9-3701e9dc9f87 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:54.253181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6d249-1f1e-4a5d-afe9-3701e9dc9f87 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:54.258500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e146d-b70c-425c-b8aa-0cb83f1a2b4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:54.259183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e146d-b70c-425c-b8aa-0cb83f1a2b4a service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:54.259456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e146d-b70c-425c-b8aa-0cb83f1a2b4a service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:54.259719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e146d-b70c-425c-b8aa-0cb83f1a2b4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:54.268294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e146d-b70c-425c-b8aa-0cb83f1a2b4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:54.269960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e146d-b70c-425c-b8aa-0cb83f1a2b4a service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:54.283054 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3134/6345] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:19:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:19:54.730223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9df11b-2c50-4f4a-86b1-7a5ebccb11b6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:54.730815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9df11b-2c50-4f4a-86b1-7a5ebccb11b6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:54.731582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9df11b-2c50-4f4a-86b1-7a5ebccb11b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:54.755306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2c9df11b-2c50-4f4a-86b1-7a5ebccb11b6 None None] MFA Rules not processed for user `f49e7f7982ad4d98b48519f7c1b43fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:19:54.756826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2c9df11b-2c50-4f4a-86b1-7a5ebccb11b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:54.828475 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3212/6346] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:19:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:19:56.481305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e146d-b70c-425c-b8aa-0cb83f1a2b4a service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:19:56.494524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e146d-b70c-425c-b8aa-0cb83f1a2b4a service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:19:56.498930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f961f-0750-4276-bc70-a439ed963543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:19:56.499564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f961f-0750-4276-bc70-a439ed963543 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:19:56.499848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f961f-0750-4276-bc70-a439ed963543 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:19:56.500111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f961f-0750-4276-bc70-a439ed963543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:19:56.501902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ba1f961f-0750-4276-bc70-a439ed963543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:19:56.518765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f961f-0750-4276-bc70-a439ed963543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:19:56.520215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f961f-0750-4276-bc70-a439ed963543 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:19:56.536353 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3135/6347] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:19:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:20:04.821164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=63 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:20:06.528301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=55 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:20:09.342563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9df11b-2c50-4f4a-86b1-7a5ebccb11b6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:09.344149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2c9df11b-2c50-4f4a-86b1-7a5ebccb11b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:09.441098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9df11b-2c50-4f4a-86b1-7a5ebccb11b6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:09.446126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd623a0a-7147-4808-be35-fc2b80dde0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:09.446755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd623a0a-7147-4808-be35-fc2b80dde0f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:09.447018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd623a0a-7147-4808-be35-fc2b80dde0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:09.447279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd623a0a-7147-4808-be35-fc2b80dde0f6 admin admin] PATH_INFO: `/v3/users/15b92b4428014995a26cde9537a27f5d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:09.448447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd623a0a-7147-4808-be35-fc2b80dde0f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15b92b4428014995a26cde9537a27f5d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:09.450362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd623a0a-7147-4808-be35-fc2b80dde0f6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:09.500151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dd623a0a-7147-4808-be35-fc2b80dde0f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15b92b4428014995a26cde9537a27f5d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:09.508284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dd623a0a-7147-4808-be35-fc2b80dde0f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15b92b4428014995a26cde9537a27f5d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:09.516078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dd623a0a-7147-4808-be35-fc2b80dde0f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15b92b4428014995a26cde9537a27f5d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:09.528129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dd623a0a-7147-4808-be35-fc2b80dde0f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15b92b4428014995a26cde9537a27f5d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:09.528902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dd623a0a-7147-4808-be35-fc2b80dde0f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15b92b4428014995a26cde9537a27f5d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:09.535097 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3213/6348] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:20:09 2022] DELETE /identity/v3/users/15b92b4428014995a26cde9537a27f5d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:20:09.713752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f961f-0750-4276-bc70-a439ed963543 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:09.735936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f961f-0750-4276-bc70-a439ed963543 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:09.800566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:09.801218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:09.801478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:09.801743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7 admin admin] PATH_INFO: `/v3/projects/e8e32728adcb4350b93179a08535090a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:09.802881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8e32728adcb4350b93179a08535090a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:09.804899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:09.857502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8e32728adcb4350b93179a08535090a', 'request_id': 'req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:09.865218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8e32728adcb4350b93179a08535090a', 'request_id': 'req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:09.878783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8e32728adcb4350b93179a08535090a', 'request_id': 'req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:09.895190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7 admin admin] The token cache is being invalidate because project e8e32728adcb4350b93179a08535090a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:20:09.895480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:09.896125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d1ab3ed-899e-45a5-a23d-f6a23bb3efe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:09.898076 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3136/6349] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:20:09 2022] DELETE /identity/v3/projects/e8e32728adcb4350b93179a08535090a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:20:09.904482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f926887-a7a1-456f-addd-7309416e4d3f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:09.905018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f926887-a7a1-456f-addd-7309416e4d3f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:09.905512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f926887-a7a1-456f-addd-7309416e4d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:09.954178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5f926887-a7a1-456f-addd-7309416e4d3f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:09.955381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5f926887-a7a1-456f-addd-7309416e4d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:09.999184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:20:10.021831 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3214/6350] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:10.030463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbbcf0-bed8-4258-8408-ac524ec5030f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:10.030744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbbcf0-bed8-4258-8408-ac524ec5030f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:10.031044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbbcf0-bed8-4258-8408-ac524ec5030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:10.053127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-afdbbcf0-bed8-4258-8408-ac524ec5030f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:10.054996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-afdbbcf0-bed8-4258-8408-ac524ec5030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:10.063914 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3137/6351] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:10.069004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f926887-a7a1-456f-addd-7309416e4d3f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:10.079732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f926887-a7a1-456f-addd-7309416e4d3f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:10.085142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8de397-21f7-4316-afc1-3fdccdc5a8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:10.086111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8de397-21f7-4316-afc1-3fdccdc5a8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:10.086407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8de397-21f7-4316-afc1-3fdccdc5a8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:10.086675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8de397-21f7-4316-afc1-3fdccdc5a8e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:10.087325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8de397-21f7-4316-afc1-3fdccdc5a8e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:20:10.087619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8de397-21f7-4316-afc1-3fdccdc5a8e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:10.089300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8de397-21f7-4316-afc1-3fdccdc5a8e8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:10.104484 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3215/6352] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:20:10.109190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbbcf0-bed8-4258-8408-ac524ec5030f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:10.119491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbbcf0-bed8-4258-8408-ac524ec5030f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:10.124640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5a758-26a3-4d8c-8b66-5cc8714b5135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:10.125253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5a758-26a3-4d8c-8b66-5cc8714b5135 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:10.125520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5a758-26a3-4d8c-8b66-5cc8714b5135 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:10.125783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5a758-26a3-4d8c-8b66-5cc8714b5135 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:10.126413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5a758-26a3-4d8c-8b66-5cc8714b5135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:10.128522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5a758-26a3-4d8c-8b66-5cc8714b5135 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:10.173069 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3138/6353] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:20:10 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:20:10.179900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8de397-21f7-4316-afc1-3fdccdc5a8e8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:10.190856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8de397-21f7-4316-afc1-3fdccdc5a8e8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:10.263264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ccb59-901a-4627-ba5d-e4ce04780cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:10.264228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ccb59-901a-4627-ba5d-e4ce04780cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:10.264706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ccb59-901a-4627-ba5d-e4ce04780cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:10.265117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ccb59-901a-4627-ba5d-e4ce04780cb4 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:10.266135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ccb59-901a-4627-ba5d-e4ce04780cb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:10.270752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ccb59-901a-4627-ba5d-e4ce04780cb4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:10.304639 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3216/6354] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:20:10 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:20:10.309420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5a758-26a3-4d8c-8b66-5cc8714b5135 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:10.324601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5a758-26a3-4d8c-8b66-5cc8714b5135 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:10.331427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333b757-6cec-4d32-b469-18c73cdba13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:10.332308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333b757-6cec-4d32-b469-18c73cdba13e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:10.332747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333b757-6cec-4d32-b469-18c73cdba13e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:10.333215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333b757-6cec-4d32-b469-18c73cdba13e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:10.334328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333b757-6cec-4d32-b469-18c73cdba13e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:10.337002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333b757-6cec-4d32-b469-18c73cdba13e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:10.351821 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3139/6355] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:20:10.356781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ccb59-901a-4627-ba5d-e4ce04780cb4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:10.372271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ccb59-901a-4627-ba5d-e4ce04780cb4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:10.380325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3a67b-b128-47dc-b89c-302fe46ed44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:10.380950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3a67b-b128-47dc-b89c-302fe46ed44a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:10.381224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3a67b-b128-47dc-b89c-302fe46ed44a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:10.381548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3a67b-b128-47dc-b89c-302fe46ed44a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:10.382278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3a67b-b128-47dc-b89c-302fe46ed44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:10.384019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3a67b-b128-47dc-b89c-302fe46ed44a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:10.395574 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3217/6356] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:20:10.400039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333b757-6cec-4d32-b469-18c73cdba13e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:10.410961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333b757-6cec-4d32-b469-18c73cdba13e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:10.417320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72568a2-ea58-45e6-993b-fce090683f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:10.417978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72568a2-ea58-45e6-993b-fce090683f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:10.418248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72568a2-ea58-45e6-993b-fce090683f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:10.418512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72568a2-ea58-45e6-993b-fce090683f83 admin admin] PATH_INFO: `/v3/projects/2dd04dea4ef54d71a82c70e1b3f8745e/users/f6e784b7272d487bbb7b0f59226384e9/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:10.441698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72568a2-ea58-45e6-993b-fce090683f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd04dea4ef54d71a82c70e1b3f8745e, user_id=f6e784b7272d487bbb7b0f59226384e9, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:10.446007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72568a2-ea58-45e6-993b-fce090683f83 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:10.465998 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3140/6357] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:20:10 2022] PUT /identity/v3/projects/2dd04dea4ef54d71a82c70e1b3f8745e/users/f6e784b7272d487bbb7b0f59226384e9/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:20:10.471566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8bd2-5333-43ed-9202-4334c0b3a08f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:10.471845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8bd2-5333-43ed-9202-4334c0b3a08f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:10.472099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8bd2-5333-43ed-9202-4334c0b3a08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:10.493344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4d1c8bd2-5333-43ed-9202-4334c0b3a08f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:10.494480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4d1c8bd2-5333-43ed-9202-4334c0b3a08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:10.556657 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3218/6358] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:10.569333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72568a2-ea58-45e6-993b-fce090683f83 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:10.571600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b72568a2-ea58-45e6-993b-fce090683f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:10.603740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:20:10.634923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72568a2-ea58-45e6-993b-fce090683f83 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:10.638638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b3d8a-8dc0-454b-8c01-a27cba9521c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:10.639245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b3d8a-8dc0-454b-8c01-a27cba9521c1 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:10.639520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b3d8a-8dc0-454b-8c01-a27cba9521c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:10.639820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b3d8a-8dc0-454b-8c01-a27cba9521c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:10.649323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b3d8a-8dc0-454b-8c01-a27cba9521c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:10.650763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b3d8a-8dc0-454b-8c01-a27cba9521c1 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:10.660120 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3141/6359] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:20:11.301939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c8bd2-5333-43ed-9202-4334c0b3a08f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:11.305685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4d1c8bd2-5333-43ed-9202-4334c0b3a08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:11.378524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c8bd2-5333-43ed-9202-4334c0b3a08f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:11.382416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17f4e4-b901-426d-809c-5a80ec31b935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:11.383067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17f4e4-b901-426d-809c-5a80ec31b935 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:11.383346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17f4e4-b901-426d-809c-5a80ec31b935 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:11.383604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17f4e4-b901-426d-809c-5a80ec31b935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:11.385792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5e17f4e4-b901-426d-809c-5a80ec31b935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:11.388588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:20:11.500737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17f4e4-b901-426d-809c-5a80ec31b935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:11.502319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17f4e4-b901-426d-809c-5a80ec31b935 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:11.515337 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3219/6360] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:20:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:20:14.928547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a946b-53ec-4c22-8f0f-12f31c75529f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:14.929072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a946b-53ec-4c22-8f0f-12f31c75529f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:14.929198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a946b-53ec-4c22-8f0f-12f31c75529f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:14.962244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e42a946b-53ec-4c22-8f0f-12f31c75529f None None] MFA Rules not processed for user `f6e784b7272d487bbb7b0f59226384e9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:14.964427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e42a946b-53ec-4c22-8f0f-12f31c75529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:15.035045 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3142/6361] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:20:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:15.051744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17f4e4-b901-426d-809c-5a80ec31b935 service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:15.053242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5e17f4e4-b901-426d-809c-5a80ec31b935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:15.138747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17f4e4-b901-426d-809c-5a80ec31b935 service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:15.144937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfd666-aae7-4178-a357-fa200390fe20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:15.145674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfd666-aae7-4178-a357-fa200390fe20 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:15.146067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfd666-aae7-4178-a357-fa200390fe20 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:15.146376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfd666-aae7-4178-a357-fa200390fe20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:15.154723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfd666-aae7-4178-a357-fa200390fe20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:15.156306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfd666-aae7-4178-a357-fa200390fe20 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:15.165510 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3220/6362] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:20:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:20:16.591598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b8cff-6813-46af-96ff-103ed90228a6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:16.592196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b8cff-6813-46af-96ff-103ed90228a6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:16.592196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b8cff-6813-46af-96ff-103ed90228a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:16.612719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fb1b8cff-6813-46af-96ff-103ed90228a6 None None] MFA Rules not processed for user `f6e784b7272d487bbb7b0f59226384e9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:16.613850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fb1b8cff-6813-46af-96ff-103ed90228a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:16.624413 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3143/6363] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:20:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:17.029311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b925e6e-8bd4-457c-92b7-a2623dd4aa65 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:17.030257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b925e6e-8bd4-457c-92b7-a2623dd4aa65 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:17.030689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b925e6e-8bd4-457c-92b7-a2623dd4aa65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:17.058802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2b925e6e-8bd4-457c-92b7-a2623dd4aa65 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:17.061235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2b925e6e-8bd4-457c-92b7-a2623dd4aa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:17.172555 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3221/6364] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:20:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:17.189453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b8cff-6813-46af-96ff-103ed90228a6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:17.191274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fb1b8cff-6813-46af-96ff-103ed90228a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:17.309049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b8cff-6813-46af-96ff-103ed90228a6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:17.314399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89c7b0-4f4b-44ec-a903-ad40692dd952 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:17.314951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89c7b0-4f4b-44ec-a903-ad40692dd952 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:17.315225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89c7b0-4f4b-44ec-a903-ad40692dd952 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:17.315489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89c7b0-4f4b-44ec-a903-ad40692dd952 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:17.324622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89c7b0-4f4b-44ec-a903-ad40692dd952 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:17.326239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89c7b0-4f4b-44ec-a903-ad40692dd952 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:17.334687 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3144/6365] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:20:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:20:25.774312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b925e6e-8bd4-457c-92b7-a2623dd4aa65 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:25.808767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b925e6e-8bd4-457c-92b7-a2623dd4aa65 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:25.825627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79069a3-6edd-49d2-ad40-16f4f97657a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:25.826290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79069a3-6edd-49d2-ad40-16f4f97657a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:25.826803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79069a3-6edd-49d2-ad40-16f4f97657a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:25.827276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79069a3-6edd-49d2-ad40-16f4f97657a8 admin admin] PATH_INFO: `/v3/users/f6e784b7272d487bbb7b0f59226384e9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:25.829035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79069a3-6edd-49d2-ad40-16f4f97657a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e784b7272d487bbb7b0f59226384e9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:25.831235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79069a3-6edd-49d2-ad40-16f4f97657a8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:25.853312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:20:25.920804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c79069a3-6edd-49d2-ad40-16f4f97657a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e784b7272d487bbb7b0f59226384e9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:25.942086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c79069a3-6edd-49d2-ad40-16f4f97657a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e784b7272d487bbb7b0f59226384e9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:25.956403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c79069a3-6edd-49d2-ad40-16f4f97657a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e784b7272d487bbb7b0f59226384e9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:25.986970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c79069a3-6edd-49d2-ad40-16f4f97657a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e784b7272d487bbb7b0f59226384e9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:25.988200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c79069a3-6edd-49d2-ad40-16f4f97657a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e784b7272d487bbb7b0f59226384e9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:26.001433 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3222/6366] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:20:25 2022] DELETE /identity/v3/users/f6e784b7272d487bbb7b0f59226384e9 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:20:26.416197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89c7b0-4f4b-44ec-a903-ad40692dd952 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:26.437915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89c7b0-4f4b-44ec-a903-ad40692dd952 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:26.526912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:20:26.552942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbf37d-b807-49c8-92c2-470dbd87d1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:26.554528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbf37d-b807-49c8-92c2-470dbd87d1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:26.554967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbf37d-b807-49c8-92c2-470dbd87d1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:26.555237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbf37d-b807-49c8-92c2-470dbd87d1eb admin admin] PATH_INFO: `/v3/projects/2dd04dea4ef54d71a82c70e1b3f8745e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:26.557002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbf37d-b807-49c8-92c2-470dbd87d1eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd04dea4ef54d71a82c70e1b3f8745e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:26.560292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbf37d-b807-49c8-92c2-470dbd87d1eb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:26.625067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80cbf37d-b807-49c8-92c2-470dbd87d1eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd04dea4ef54d71a82c70e1b3f8745e', 'request_id': 'req-80cbf37d-b807-49c8-92c2-470dbd87d1eb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:26.634564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80cbf37d-b807-49c8-92c2-470dbd87d1eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd04dea4ef54d71a82c70e1b3f8745e', 'request_id': 'req-80cbf37d-b807-49c8-92c2-470dbd87d1eb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:26.646617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80cbf37d-b807-49c8-92c2-470dbd87d1eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd04dea4ef54d71a82c70e1b3f8745e', 'request_id': 'req-80cbf37d-b807-49c8-92c2-470dbd87d1eb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:26.667746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80cbf37d-b807-49c8-92c2-470dbd87d1eb admin admin] The token cache is being invalidate because project 2dd04dea4ef54d71a82c70e1b3f8745e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:20:26.668339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80cbf37d-b807-49c8-92c2-470dbd87d1eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:26.669161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80cbf37d-b807-49c8-92c2-470dbd87d1eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:20:26.671002 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3145/6367] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:20:26 2022] DELETE /identity/v3/projects/2dd04dea4ef54d71a82c70e1b3f8745e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:20:26.684487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024ef52-eed4-4707-951c-47af0c6ae256 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:26.685295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024ef52-eed4-4707-951c-47af0c6ae256 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:26.685787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024ef52-eed4-4707-951c-47af0c6ae256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:26.712012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d024ef52-eed4-4707-951c-47af0c6ae256 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:26.713549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d024ef52-eed4-4707-951c-47af0c6ae256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:26.799148 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3223/6368] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:26.807717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dd778-9d51-46cf-938d-b52056dc59ff None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:26.807985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dd778-9d51-46cf-938d-b52056dc59ff None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:26.808248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dd778-9d51-46cf-938d-b52056dc59ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:26.843325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ce8dd778-9d51-46cf-938d-b52056dc59ff None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:26.844614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ce8dd778-9d51-46cf-938d-b52056dc59ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:26.858175 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3146/6369] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:26.863687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024ef52-eed4-4707-951c-47af0c6ae256 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:26.900916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024ef52-eed4-4707-951c-47af0c6ae256 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:26.913107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a93d7c-74cb-4bcd-ae5d-aa1244fc263c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:26.914009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a93d7c-74cb-4bcd-ae5d-aa1244fc263c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:26.914961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a93d7c-74cb-4bcd-ae5d-aa1244fc263c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:26.916505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a93d7c-74cb-4bcd-ae5d-aa1244fc263c admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:26.917372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a93d7c-74cb-4bcd-ae5d-aa1244fc263c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:20:26.918368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a93d7c-74cb-4bcd-ae5d-aa1244fc263c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:26.921731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a93d7c-74cb-4bcd-ae5d-aa1244fc263c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:26.937538 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3224/6370] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:20:26.942001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8dd778-9d51-46cf-938d-b52056dc59ff None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:26.956617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8dd778-9d51-46cf-938d-b52056dc59ff None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:26.969283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2c1fd-e560-4388-b675-9607036c1c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:26.970752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2c1fd-e560-4388-b675-9607036c1c51 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:26.972604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2c1fd-e560-4388-b675-9607036c1c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:26.973329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2c1fd-e560-4388-b675-9607036c1c51 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:26.974156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2c1fd-e560-4388-b675-9607036c1c51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:26.978412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2c1fd-e560-4388-b675-9607036c1c51 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:27.036419 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3147/6371] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:20:26 2022] POST /identity/v3/projects => generated 367 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:20:27.038117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a93d7c-74cb-4bcd-ae5d-aa1244fc263c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:27.053819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a93d7c-74cb-4bcd-ae5d-aa1244fc263c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:27.160948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf55f37-3def-4a43-9cb6-a48c0a3ac39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:27.161586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf55f37-3def-4a43-9cb6-a48c0a3ac39d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:27.161857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf55f37-3def-4a43-9cb6-a48c0a3ac39d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:27.162177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf55f37-3def-4a43-9cb6-a48c0a3ac39d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:27.162959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf55f37-3def-4a43-9cb6-a48c0a3ac39d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:27.165910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf55f37-3def-4a43-9cb6-a48c0a3ac39d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:27.198105 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3225/6372] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:20:27 2022] POST /identity/v3/users => generated 284 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:20:27.202955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2c1fd-e560-4388-b675-9607036c1c51 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:27.223188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2c1fd-e560-4388-b675-9607036c1c51 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:27.242900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b1470-49be-4f30-819f-405248de4d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:27.243980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b1470-49be-4f30-819f-405248de4d2b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:27.245139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b1470-49be-4f30-819f-405248de4d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:27.245943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b1470-49be-4f30-819f-405248de4d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:27.248214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b1470-49be-4f30-819f-405248de4d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:27.250306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b1470-49be-4f30-819f-405248de4d2b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:27.276394 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3148/6373] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:20:27.276872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf55f37-3def-4a43-9cb6-a48c0a3ac39d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:27.285758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:20:27.294301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf55f37-3def-4a43-9cb6-a48c0a3ac39d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:27.302423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115882f9-ded7-4614-94a0-17e1dbdd5205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:27.303047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115882f9-ded7-4614-94a0-17e1dbdd5205 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:27.303321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115882f9-ded7-4614-94a0-17e1dbdd5205 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:27.303684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115882f9-ded7-4614-94a0-17e1dbdd5205 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:27.304338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115882f9-ded7-4614-94a0-17e1dbdd5205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:27.306086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115882f9-ded7-4614-94a0-17e1dbdd5205 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:27.323369 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3226/6374] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:20:27.326743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b1470-49be-4f30-819f-405248de4d2b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:27.358998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b1470-49be-4f30-819f-405248de4d2b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:27.369993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9537ac-c2d3-46f5-aea6-7d723b501fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:27.371244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9537ac-c2d3-46f5-aea6-7d723b501fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:27.371884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9537ac-c2d3-46f5-aea6-7d723b501fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:27.372365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9537ac-c2d3-46f5-aea6-7d723b501fdf admin admin] PATH_INFO: `/v3/projects/dac85ef1c85b4a06951f2df42dd98e50/users/f1dde492e29c4157bab5b7419a310838/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:27.415763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9537ac-c2d3-46f5-aea6-7d723b501fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac85ef1c85b4a06951f2df42dd98e50, user_id=f1dde492e29c4157bab5b7419a310838, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:27.419328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9537ac-c2d3-46f5-aea6-7d723b501fdf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:27.459135 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3149/6375] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:20:27 2022] PUT /identity/v3/projects/dac85ef1c85b4a06951f2df42dd98e50/users/f1dde492e29c4157bab5b7419a310838/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:20:27.464571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33de21-c5ed-4620-be3d-da615bd389a6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:27.464951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33de21-c5ed-4620-be3d-da615bd389a6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:27.465122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33de21-c5ed-4620-be3d-da615bd389a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:27.487002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7c33de21-c5ed-4620-be3d-da615bd389a6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:27.488276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c33de21-c5ed-4620-be3d-da615bd389a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:27.548454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0537839-a450-43c8-b4e3-a3d0efb2f8a8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:27.548739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0537839-a450-43c8-b4e3-a3d0efb2f8a8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:27.549006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0537839-a450-43c8-b4e3-a3d0efb2f8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:27.577207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a0537839-a450-43c8-b4e3-a3d0efb2f8a8 None None] MFA Rules not processed for user `7013eec1908f407d9949d271287b1570`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:27.578729 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3227/6376] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:27.579084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a0537839-a450-43c8-b4e3-a3d0efb2f8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:27.588731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33de21-c5ed-4620-be3d-da615bd389a6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:27.592288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c33de21-c5ed-4620-be3d-da615bd389a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:27.645166 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3150/6377] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:20:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:27.716889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33de21-c5ed-4620-be3d-da615bd389a6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:27.721869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac9522-4a7b-4120-94f9-1d94873ab22e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:27.722553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac9522-4a7b-4120-94f9-1d94873ab22e service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:27.722819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac9522-4a7b-4120-94f9-1d94873ab22e service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:27.723083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac9522-4a7b-4120-94f9-1d94873ab22e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:27.739272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac9522-4a7b-4120-94f9-1d94873ab22e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:27.740931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac9522-4a7b-4120-94f9-1d94873ab22e service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:27.752175 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3228/6378] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:20:32.123166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b425e9-2fbe-48fc-802e-8c81318d0156 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:32.126479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b425e9-2fbe-48fc-802e-8c81318d0156 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:32.127187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b425e9-2fbe-48fc-802e-8c81318d0156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:32.172474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b8b425e9-2fbe-48fc-802e-8c81318d0156 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:32.176027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b8b425e9-2fbe-48fc-802e-8c81318d0156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:32.190166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:20:32.295521 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3151/6379] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:20:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:32.313441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac9522-4a7b-4120-94f9-1d94873ab22e service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:32.315077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cdac9522-4a7b-4120-94f9-1d94873ab22e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:32.459255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac9522-4a7b-4120-94f9-1d94873ab22e service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:32.467303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1fa87-6609-4a2d-b8e3-4375cdfdfd75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:32.468331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1fa87-6609-4a2d-b8e3-4375cdfdfd75 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:32.468331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1fa87-6609-4a2d-b8e3-4375cdfdfd75 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:32.468825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1fa87-6609-4a2d-b8e3-4375cdfdfd75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:32.483663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1fa87-6609-4a2d-b8e3-4375cdfdfd75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:32.487214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1fa87-6609-4a2d-b8e3-4375cdfdfd75 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:32.508262 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3229/6380] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:20:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:20:33.390579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f99a1d-ef27-4ee9-a114-4a58c07208e7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:33.391144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f99a1d-ef27-4ee9-a114-4a58c07208e7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:33.391252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f99a1d-ef27-4ee9-a114-4a58c07208e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:33.412648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e8f99a1d-ef27-4ee9-a114-4a58c07208e7 None None] MFA Rules not processed for user `f49e7f7982ad4d98b48519f7c1b43fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:33.414623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e8f99a1d-ef27-4ee9-a114-4a58c07208e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:33.522601 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3152/6381] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:20:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:33.940893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1fa87-6609-4a2d-b8e3-4375cdfdfd75 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:33.944639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-79c1fa87-6609-4a2d-b8e3-4375cdfdfd75 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:33.985785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1fa87-6609-4a2d-b8e3-4375cdfdfd75 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:33.991833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71efcf-360e-49ee-a142-e39c30789ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:33.993349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71efcf-360e-49ee-a142-e39c30789ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:33.993868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71efcf-360e-49ee-a142-e39c30789ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:33.994397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71efcf-360e-49ee-a142-e39c30789ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:33.996536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-da71efcf-360e-49ee-a142-e39c30789ea3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:34.040906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71efcf-360e-49ee-a142-e39c30789ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:34.043685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71efcf-360e-49ee-a142-e39c30789ea3 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:34.070497 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3230/6382] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:20:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:20:36.812103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd500320-a5f3-419e-b173-caa10f2ef495 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:36.812617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd500320-a5f3-419e-b173-caa10f2ef495 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:36.812736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd500320-a5f3-419e-b173-caa10f2ef495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:36.903583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fd500320-a5f3-419e-b173-caa10f2ef495 None None] MFA Rules not processed for user `f1dde492e29c4157bab5b7419a310838`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:36.911828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fd500320-a5f3-419e-b173-caa10f2ef495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:37.156289 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3153/6383] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:20:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:37.175745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71efcf-360e-49ee-a142-e39c30789ea3 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:37.178115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-da71efcf-360e-49ee-a142-e39c30789ea3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:37.223584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:20:37.393589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71efcf-360e-49ee-a142-e39c30789ea3 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:37.405568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1e25d-6875-404a-85b3-eced0ec49542 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:37.406210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1e25d-6875-404a-85b3-eced0ec49542 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:37.406536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1e25d-6875-404a-85b3-eced0ec49542 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:37.406757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1e25d-6875-404a-85b3-eced0ec49542 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:37.431061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1e25d-6875-404a-85b3-eced0ec49542 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:37.433295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1e25d-6875-404a-85b3-eced0ec49542 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:37.458700 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3231/6384] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:20:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:20:38.880195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1aaf2c-9336-44b7-9617-91183703ecc3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:38.880785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1aaf2c-9336-44b7-9617-91183703ecc3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:38.880894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1aaf2c-9336-44b7-9617-91183703ecc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:38.918166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ba1aaf2c-9336-44b7-9617-91183703ecc3 None None] MFA Rules not processed for user `85805da2da9a419bab802722aeea5f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:38.919449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ba1aaf2c-9336-44b7-9617-91183703ecc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:39.044586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:20:39.063095 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3154/6385] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:20:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:42.412086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5a26-1594-4f82-9137-7a86649b993f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:42.412658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5a26-1594-4f82-9137-7a86649b993f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:42.413140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5a26-1594-4f82-9137-7a86649b993f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:42.437552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9d6a5a26-1594-4f82-9137-7a86649b993f None None] MFA Rules not processed for user `f1dde492e29c4157bab5b7419a310838`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:20:42.438875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9d6a5a26-1594-4f82-9137-7a86649b993f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:42.446599 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3232/6386] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:20:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:20:45.871353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1aaf2c-9336-44b7-9617-91183703ecc3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:20:45.890447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1aaf2c-9336-44b7-9617-91183703ecc3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:20:45.895268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811231bb-b422-4a90-bfeb-d99b2b5551d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:20:45.896394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811231bb-b422-4a90-bfeb-d99b2b5551d5 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:20:45.897478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811231bb-b422-4a90-bfeb-d99b2b5551d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:20:45.898474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811231bb-b422-4a90-bfeb-d99b2b5551d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:20:45.900922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-811231bb-b422-4a90-bfeb-d99b2b5551d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:20:45.920856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811231bb-b422-4a90-bfeb-d99b2b5551d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:20:45.922453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811231bb-b422-4a90-bfeb-d99b2b5551d5 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:20:45.938849 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3155/6387] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:20:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:20:52.431685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=31 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:20:55.928230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:06.644177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0245e1-2dc3-4f37-822e-3e3695968f7a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:06.645181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0245e1-2dc3-4f37-822e-3e3695968f7a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:06.645637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0245e1-2dc3-4f37-822e-3e3695968f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:06.701439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2a0245e1-2dc3-4f37-822e-3e3695968f7a None None] MFA Rules not processed for user `daffa49c8fa64a3bb96f917fef222291`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:06.703064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2a0245e1-2dc3-4f37-822e-3e3695968f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:06.829636 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3233/6388] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:08.824637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811231bb-b422-4a90-bfeb-d99b2b5551d5 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:08.826486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-811231bb-b422-4a90-bfeb-d99b2b5551d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:08.851733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811231bb-b422-4a90-bfeb-d99b2b5551d5 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:08.865020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9230d-e62c-4609-9d78-cf49a3203ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:08.867123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9230d-e62c-4609-9d78-cf49a3203ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:08.867880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9230d-e62c-4609-9d78-cf49a3203ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:08.869099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9230d-e62c-4609-9d78-cf49a3203ceb admin admin] PATH_INFO: `/v3/users/f49e7f7982ad4d98b48519f7c1b43fdb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:08.870959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9230d-e62c-4609-9d78-cf49a3203ceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f49e7f7982ad4d98b48519f7c1b43fdb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:08.887115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9230d-e62c-4609-9d78-cf49a3203ceb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:08.986599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a2d9230d-e62c-4609-9d78-cf49a3203ceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f49e7f7982ad4d98b48519f7c1b43fdb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:09.004644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a2d9230d-e62c-4609-9d78-cf49a3203ceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f49e7f7982ad4d98b48519f7c1b43fdb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:09.024626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a2d9230d-e62c-4609-9d78-cf49a3203ceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f49e7f7982ad4d98b48519f7c1b43fdb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:09.025528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a2d9230d-e62c-4609-9d78-cf49a3203ceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f49e7f7982ad4d98b48519f7c1b43fdb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:09.026328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a2d9230d-e62c-4609-9d78-cf49a3203ceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f49e7f7982ad4d98b48519f7c1b43fdb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:09.041443 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3156/6389] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:08 2022] DELETE /identity/v3/users/f49e7f7982ad4d98b48519f7c1b43fdb => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:09.244063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0245e1-2dc3-4f37-822e-3e3695968f7a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:09.267781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0245e1-2dc3-4f37-822e-3e3695968f7a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:09.339878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f58ab9-6782-427a-b774-9ca0013a9ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:09.340674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f58ab9-6782-427a-b774-9ca0013a9ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:09.341015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f58ab9-6782-427a-b774-9ca0013a9ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:09.341346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f58ab9-6782-427a-b774-9ca0013a9ecb admin admin] PATH_INFO: `/v3/projects/13a3c875257e4db7b3fb2560c9bb6392` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:09.342724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f58ab9-6782-427a-b774-9ca0013a9ecb admin admin] RBAC: Authorizing `identity:delete_project(project_id=13a3c875257e4db7b3fb2560c9bb6392)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:09.344841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f58ab9-6782-427a-b774-9ca0013a9ecb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:09.425595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-03f58ab9-6782-427a-b774-9ca0013a9ecb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13a3c875257e4db7b3fb2560c9bb6392', 'request_id': 'req-03f58ab9-6782-427a-b774-9ca0013a9ecb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:09.443019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-03f58ab9-6782-427a-b774-9ca0013a9ecb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13a3c875257e4db7b3fb2560c9bb6392', 'request_id': 'req-03f58ab9-6782-427a-b774-9ca0013a9ecb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:09.457342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-03f58ab9-6782-427a-b774-9ca0013a9ecb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13a3c875257e4db7b3fb2560c9bb6392', 'request_id': 'req-03f58ab9-6782-427a-b774-9ca0013a9ecb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:09.478502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-03f58ab9-6782-427a-b774-9ca0013a9ecb admin admin] The token cache is being invalidate because project 13a3c875257e4db7b3fb2560c9bb6392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:09.478803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-03f58ab9-6782-427a-b774-9ca0013a9ecb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:09.480092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-03f58ab9-6782-427a-b774-9ca0013a9ecb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:09.482422 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3234/6390] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:09 2022] DELETE /identity/v3/projects/13a3c875257e4db7b3fb2560c9bb6392 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:09.487873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76669483-9b06-4645-8089-764591487dba None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:09.488319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76669483-9b06-4645-8089-764591487dba None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:09.488833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76669483-9b06-4645-8089-764591487dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:09.516649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-76669483-9b06-4645-8089-764591487dba None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:09.517867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-76669483-9b06-4645-8089-764591487dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:09.607170 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3157/6391] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:09.608356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d412e-9420-4a41-8b69-49f1758bf835 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:09.608719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d412e-9420-4a41-8b69-49f1758bf835 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:09.608996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d412e-9420-4a41-8b69-49f1758bf835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:09.638629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2d6d412e-9420-4a41-8b69-49f1758bf835 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:09.641307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2d6d412e-9420-4a41-8b69-49f1758bf835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:09.651757 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3235/6392] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:09.656099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76669483-9b06-4645-8089-764591487dba None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:09.666932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76669483-9b06-4645-8089-764591487dba None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:09.671681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c943fe-f468-4032-ad0f-203fbb9cbd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:09.672288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c943fe-f468-4032-ad0f-203fbb9cbd89 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:09.672602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c943fe-f468-4032-ad0f-203fbb9cbd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:09.680826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c943fe-f468-4032-ad0f-203fbb9cbd89 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:09.682932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c943fe-f468-4032-ad0f-203fbb9cbd89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:09.683586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c943fe-f468-4032-ad0f-203fbb9cbd89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:09.686073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c943fe-f468-4032-ad0f-203fbb9cbd89 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:09.699166 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3158/6393] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:09.703656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d412e-9420-4a41-8b69-49f1758bf835 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:09.713331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d412e-9420-4a41-8b69-49f1758bf835 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:09.718237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffd413-1f90-4696-85af-ad60fc7ae270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:09.718854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffd413-1f90-4696-85af-ad60fc7ae270 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:09.719125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffd413-1f90-4696-85af-ad60fc7ae270 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:09.719390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffd413-1f90-4696-85af-ad60fc7ae270 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:09.720057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffd413-1f90-4696-85af-ad60fc7ae270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:09.722027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffd413-1f90-4696-85af-ad60fc7ae270 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:09.751185 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3236/6394] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:09 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:09.755516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c943fe-f468-4032-ad0f-203fbb9cbd89 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:09.766355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c943fe-f468-4032-ad0f-203fbb9cbd89 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:09.826963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cc228-0f09-42bf-a5bb-d7ffeb6f4c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:09.827596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cc228-0f09-42bf-a5bb-d7ffeb6f4c89 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:09.827851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cc228-0f09-42bf-a5bb-d7ffeb6f4c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:09.828116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cc228-0f09-42bf-a5bb-d7ffeb6f4c89 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:09.828794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8cc228-0f09-42bf-a5bb-d7ffeb6f4c89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:09.831072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8cc228-0f09-42bf-a5bb-d7ffeb6f4c89 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:09.841934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:09.868179 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3159/6395] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:09 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:09.872825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffd413-1f90-4696-85af-ad60fc7ae270 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:09.886529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffd413-1f90-4696-85af-ad60fc7ae270 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:09.891752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1d56d-88ec-4c21-80a1-9d91f7a7a54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:09.892386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1d56d-88ec-4c21-80a1-9d91f7a7a54c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:09.892670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1d56d-88ec-4c21-80a1-9d91f7a7a54c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:09.893830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1d56d-88ec-4c21-80a1-9d91f7a7a54c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:09.893830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1d56d-88ec-4c21-80a1-9d91f7a7a54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:09.895166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1d56d-88ec-4c21-80a1-9d91f7a7a54c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:09.896815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:09.917164 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3237/6396] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:09.921402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cc228-0f09-42bf-a5bb-d7ffeb6f4c89 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:09.933946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cc228-0f09-42bf-a5bb-d7ffeb6f4c89 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:09.939688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1036bb-68db-439e-90ac-b09ea8267b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:09.940531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1036bb-68db-439e-90ac-b09ea8267b2b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:09.940972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1036bb-68db-439e-90ac-b09ea8267b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:09.941381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1036bb-68db-439e-90ac-b09ea8267b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:09.942160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1036bb-68db-439e-90ac-b09ea8267b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:09.944316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1036bb-68db-439e-90ac-b09ea8267b2b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:09.960039 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3160/6397] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:09.964252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1d56d-88ec-4c21-80a1-9d91f7a7a54c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:09.975709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1d56d-88ec-4c21-80a1-9d91f7a7a54c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:09.986072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dbc42-3c72-4906-a1ba-4bc175ba5fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:09.986732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dbc42-3c72-4906-a1ba-4bc175ba5fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:09.987059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dbc42-3c72-4906-a1ba-4bc175ba5fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:09.987283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dbc42-3c72-4906-a1ba-4bc175ba5fe5 admin admin] PATH_INFO: `/v3/projects/0a81886143994c2f9e071a18e63536d2/users/d5c5056440254c628a3ac74e43d08699/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:10.003712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4dbc42-3c72-4906-a1ba-4bc175ba5fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a81886143994c2f9e071a18e63536d2, user_id=d5c5056440254c628a3ac74e43d08699, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:10.006538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4dbc42-3c72-4906-a1ba-4bc175ba5fe5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:10.028581 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3238/6398] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:09 2022] PUT /identity/v3/projects/0a81886143994c2f9e071a18e63536d2/users/d5c5056440254c628a3ac74e43d08699/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:10.039678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c765f-792e-471f-af22-821ebbfb6b1e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:10.039937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c765f-792e-471f-af22-821ebbfb6b1e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:10.040412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c765f-792e-471f-af22-821ebbfb6b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:10.079844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-471c765f-792e-471f-af22-821ebbfb6b1e None None] MFA Rules not processed for user `d5c5056440254c628a3ac74e43d08699`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:10.081213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-471c765f-792e-471f-af22-821ebbfb6b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:10.189089 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3161/6399] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:10 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:10.201588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dbc42-3c72-4906-a1ba-4bc175ba5fe5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:10.204602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4c4dbc42-3c72-4906-a1ba-4bc175ba5fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:10.325346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dbc42-3c72-4906-a1ba-4bc175ba5fe5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:10.332793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f46d9-5d86-40da-b713-65e5714244b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:10.333433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f46d9-5d86-40da-b713-65e5714244b1 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:10.333770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f46d9-5d86-40da-b713-65e5714244b1 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:10.334100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f46d9-5d86-40da-b713-65e5714244b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:10.343104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f46d9-5d86-40da-b713-65e5714244b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:10.344859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f46d9-5d86-40da-b713-65e5714244b1 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:10.356008 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3239/6400] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:10 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:10.395190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c765f-792e-471f-af22-821ebbfb6b1e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:10.467417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c765f-792e-471f-af22-821ebbfb6b1e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:10.472757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:10.473526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:10.473783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:10.474054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] PATH_INFO: `/v3/users/d5c5056440254c628a3ac74e43d08699` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:10.475269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5c5056440254c628a3ac74e43d08699)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:10.477334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:10.529130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5c5056440254c628a3ac74e43d08699'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:10.537451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5c5056440254c628a3ac74e43d08699'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:10.548736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:10.550359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5c5056440254c628a3ac74e43d08699'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:10.551155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5c5056440254c628a3ac74e43d08699'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:10.552105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5c5056440254c628a3ac74e43d08699'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:10.566675 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3162/6401] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:10 2022] DELETE /identity/v3/users/d5c5056440254c628a3ac74e43d08699 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:10.571966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfd4f1-8c29-4c07-8262-f5830462f628 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:10.572222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfd4f1-8c29-4c07-8262-f5830462f628 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:10.572515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfd4f1-8c29-4c07-8262-f5830462f628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:10.594961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6fdfd4f1-8c29-4c07-8262-f5830462f628 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:10.596214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6fdfd4f1-8c29-4c07-8262-f5830462f628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:10.679652 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3240/6402] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:10.689384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:10.691604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:10.759607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f1e95-5b95-48fe-9675-b2d50abb4f20 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:10.764397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b84e4-7544-41ef-b298-aa2b05e31dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:10.765025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b84e4-7544-41ef-b298-aa2b05e31dff service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:10.765293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b84e4-7544-41ef-b298-aa2b05e31dff service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:10.765554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b84e4-7544-41ef-b298-aa2b05e31dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:10.775534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b84e4-7544-41ef-b298-aa2b05e31dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:10.777011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b84e4-7544-41ef-b298-aa2b05e31dff service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:10.787419 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3163/6403] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:11.075165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfd4f1-8c29-4c07-8262-f5830462f628 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:11.076985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6fdfd4f1-8c29-4c07-8262-f5830462f628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:11.094290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfd4f1-8c29-4c07-8262-f5830462f628 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:11.100207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1983374-3788-444f-8cde-0df9b2beb5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:11.101016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1983374-3788-444f-8cde-0df9b2beb5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:11.101343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1983374-3788-444f-8cde-0df9b2beb5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:11.101675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1983374-3788-444f-8cde-0df9b2beb5cf admin admin] PATH_INFO: `/v3/projects/0a81886143994c2f9e071a18e63536d2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:11.103005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1983374-3788-444f-8cde-0df9b2beb5cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a81886143994c2f9e071a18e63536d2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:11.105106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1983374-3788-444f-8cde-0df9b2beb5cf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:11.175623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1983374-3788-444f-8cde-0df9b2beb5cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a81886143994c2f9e071a18e63536d2', 'request_id': 'req-c1983374-3788-444f-8cde-0df9b2beb5cf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:11.180135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:11.186830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1983374-3788-444f-8cde-0df9b2beb5cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a81886143994c2f9e071a18e63536d2', 'request_id': 'req-c1983374-3788-444f-8cde-0df9b2beb5cf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:11.196883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1983374-3788-444f-8cde-0df9b2beb5cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a81886143994c2f9e071a18e63536d2', 'request_id': 'req-c1983374-3788-444f-8cde-0df9b2beb5cf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:11.211778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1983374-3788-444f-8cde-0df9b2beb5cf admin admin] The token cache is being invalidate because project 0a81886143994c2f9e071a18e63536d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:11.212061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1983374-3788-444f-8cde-0df9b2beb5cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:11.212752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1983374-3788-444f-8cde-0df9b2beb5cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:11.216409 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3241/6404] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:11 2022] DELETE /identity/v3/projects/0a81886143994c2f9e071a18e63536d2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:11.224252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9ee7d-0287-4d58-877c-07a074e4fa4f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:11.224793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9ee7d-0287-4d58-877c-07a074e4fa4f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:11.224979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9ee7d-0287-4d58-877c-07a074e4fa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:11.270963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f4c9ee7d-0287-4d58-877c-07a074e4fa4f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:11.272518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f4c9ee7d-0287-4d58-877c-07a074e4fa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:11.399463 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3164/6405] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:11.401271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ae225-65f2-47f7-ad8a-a8e1e469ad36 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:11.401755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ae225-65f2-47f7-ad8a-a8e1e469ad36 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:11.401859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ae225-65f2-47f7-ad8a-a8e1e469ad36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:11.433710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-835ae225-65f2-47f7-ad8a-a8e1e469ad36 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:11.435199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-835ae225-65f2-47f7-ad8a-a8e1e469ad36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:11.444291 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3242/6406] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:11.448796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9ee7d-0287-4d58-877c-07a074e4fa4f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:11.459560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9ee7d-0287-4d58-877c-07a074e4fa4f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:11.465016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66141fb4-9fcb-4f3b-8a32-5b457faf562d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:11.465689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66141fb4-9fcb-4f3b-8a32-5b457faf562d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:11.465965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66141fb4-9fcb-4f3b-8a32-5b457faf562d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:11.466181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66141fb4-9fcb-4f3b-8a32-5b457faf562d admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:11.466841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66141fb4-9fcb-4f3b-8a32-5b457faf562d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:11.467209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66141fb4-9fcb-4f3b-8a32-5b457faf562d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:11.468910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66141fb4-9fcb-4f3b-8a32-5b457faf562d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:11.483116 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3165/6407] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:11.487536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ae225-65f2-47f7-ad8a-a8e1e469ad36 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:11.498425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ae225-65f2-47f7-ad8a-a8e1e469ad36 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:11.503541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d74cd-571c-4f80-9d72-72356ef3db86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:11.504273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d74cd-571c-4f80-9d72-72356ef3db86 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:11.504656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d74cd-571c-4f80-9d72-72356ef3db86 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:11.504998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d74cd-571c-4f80-9d72-72356ef3db86 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:11.505708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d74cd-571c-4f80-9d72-72356ef3db86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:11.507873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d74cd-571c-4f80-9d72-72356ef3db86 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:11.539130 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3243/6408] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:11 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:11.543657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66141fb4-9fcb-4f3b-8a32-5b457faf562d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:11.555426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66141fb4-9fcb-4f3b-8a32-5b457faf562d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:11.635333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc3ac1-fa12-4fce-8c4d-f89bafbef3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:11.636046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc3ac1-fa12-4fce-8c4d-f89bafbef3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:11.636322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc3ac1-fa12-4fce-8c4d-f89bafbef3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:11.636647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc3ac1-fa12-4fce-8c4d-f89bafbef3e3 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:11.637413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc3ac1-fa12-4fce-8c4d-f89bafbef3e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:11.639578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc3ac1-fa12-4fce-8c4d-f89bafbef3e3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:11.667626 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3166/6409] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:11 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:11.671713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d74cd-571c-4f80-9d72-72356ef3db86 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:11.685057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d74cd-571c-4f80-9d72-72356ef3db86 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:11.689898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63569c59-3c91-4d66-bc23-c9f80ba87b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:11.690646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63569c59-3c91-4d66-bc23-c9f80ba87b5b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:11.691666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63569c59-3c91-4d66-bc23-c9f80ba87b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:11.691666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63569c59-3c91-4d66-bc23-c9f80ba87b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:11.692278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63569c59-3c91-4d66-bc23-c9f80ba87b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:11.694906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63569c59-3c91-4d66-bc23-c9f80ba87b5b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:11.706842 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3244/6410] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:11.711751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc3ac1-fa12-4fce-8c4d-f89bafbef3e3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:11.731830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc3ac1-fa12-4fce-8c4d-f89bafbef3e3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:11.744737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babc1c2-36d7-475c-aeca-fb5e514e3844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:11.746356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babc1c2-36d7-475c-aeca-fb5e514e3844 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:11.747166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babc1c2-36d7-475c-aeca-fb5e514e3844 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:11.747508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babc1c2-36d7-475c-aeca-fb5e514e3844 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:11.748291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babc1c2-36d7-475c-aeca-fb5e514e3844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:11.750002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babc1c2-36d7-475c-aeca-fb5e514e3844 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:11.751847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:11.773035 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3167/6411] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:11.775670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63569c59-3c91-4d66-bc23-c9f80ba87b5b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:11.788724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63569c59-3c91-4d66-bc23-c9f80ba87b5b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:11.794752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d85cf-d22e-48ed-9aa5-4333ddb2bafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:11.795474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d85cf-d22e-48ed-9aa5-4333ddb2bafd admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:11.796165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d85cf-d22e-48ed-9aa5-4333ddb2bafd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:11.807722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d85cf-d22e-48ed-9aa5-4333ddb2bafd admin admin] PATH_INFO: `/v3/projects/8ab8ffa848e748bfb47492e71560dabe/users/0f51348fa8944a54a6a0b1b9c2a9319b/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:11.846784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d85cf-d22e-48ed-9aa5-4333ddb2bafd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab8ffa848e748bfb47492e71560dabe, user_id=0f51348fa8944a54a6a0b1b9c2a9319b, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:11.849526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d85cf-d22e-48ed-9aa5-4333ddb2bafd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:11.881951 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3245/6412] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:11 2022] PUT /identity/v3/projects/8ab8ffa848e748bfb47492e71560dabe/users/0f51348fa8944a54a6a0b1b9c2a9319b/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:11.883290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3f51a-d6a3-41a6-a702-f39989eb6101 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:11.883749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3f51a-d6a3-41a6-a702-f39989eb6101 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:11.884022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3f51a-d6a3-41a6-a702-f39989eb6101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:11.921301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-eff3f51a-d6a3-41a6-a702-f39989eb6101 None None] MFA Rules not processed for user `0f51348fa8944a54a6a0b1b9c2a9319b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:11.922437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-eff3f51a-d6a3-41a6-a702-f39989eb6101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:12.002611 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3168/6413] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:12.016161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d85cf-d22e-48ed-9aa5-4333ddb2bafd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:12.017936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4c1d85cf-d22e-48ed-9aa5-4333ddb2bafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:12.100753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d85cf-d22e-48ed-9aa5-4333ddb2bafd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:12.106673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17bde6-e8fd-405b-894f-94388832c089 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:12.107290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17bde6-e8fd-405b-894f-94388832c089 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:12.107563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17bde6-e8fd-405b-894f-94388832c089 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:12.107823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17bde6-e8fd-405b-894f-94388832c089 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:12.116947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17bde6-e8fd-405b-894f-94388832c089 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:12.118571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17bde6-e8fd-405b-894f-94388832c089 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:12.131742 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3246/6414] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:12.825595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3f51a-d6a3-41a6-a702-f39989eb6101 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:12.930665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3f51a-d6a3-41a6-a702-f39989eb6101 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:12.935984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:12.936748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:12.937104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:12.937470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] PATH_INFO: `/v3/users/0f51348fa8944a54a6a0b1b9c2a9319b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:12.938786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f51348fa8944a54a6a0b1b9c2a9319b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:12.940963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:13.019899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f51348fa8944a54a6a0b1b9c2a9319b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:13.029633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f51348fa8944a54a6a0b1b9c2a9319b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:13.046201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f51348fa8944a54a6a0b1b9c2a9319b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:13.048784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f51348fa8944a54a6a0b1b9c2a9319b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:13.050312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f51348fa8944a54a6a0b1b9c2a9319b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:13.068014 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3169/6415] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:12 2022] DELETE /identity/v3/users/0f51348fa8944a54a6a0b1b9c2a9319b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:13.070234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698f9a8-d551-4e71-a0d6-457d87c10453 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:13.070727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698f9a8-d551-4e71-a0d6-457d87c10453 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:13.070814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698f9a8-d551-4e71-a0d6-457d87c10453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:13.090382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a698f9a8-d551-4e71-a0d6-457d87c10453 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:13.091859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a698f9a8-d551-4e71-a0d6-457d87c10453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:13.188413 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3247/6416] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:13.216078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:13.217513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:13.234347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:13.305772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18de47-12c0-4c14-9129-3cdbd9c63599 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:13.310068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe47d6a-7552-4067-9a33-16df75d88b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:13.310749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe47d6a-7552-4067-9a33-16df75d88b3c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:13.311083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe47d6a-7552-4067-9a33-16df75d88b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:13.311728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe47d6a-7552-4067-9a33-16df75d88b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:13.324096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe47d6a-7552-4067-9a33-16df75d88b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:13.326028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe47d6a-7552-4067-9a33-16df75d88b3c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:13.337819 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3170/6417] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:13.723072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698f9a8-d551-4e71-a0d6-457d87c10453 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:13.724657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a698f9a8-d551-4e71-a0d6-457d87c10453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:13.732499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:13.739029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698f9a8-d551-4e71-a0d6-457d87c10453 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:13.746683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c41607-4c49-47cb-8709-3a8ef8e5eae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:13.747974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c41607-4c49-47cb-8709-3a8ef8e5eae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:13.748892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c41607-4c49-47cb-8709-3a8ef8e5eae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:13.749170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c41607-4c49-47cb-8709-3a8ef8e5eae2 admin admin] PATH_INFO: `/v3/projects/8ab8ffa848e748bfb47492e71560dabe` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:13.750597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c41607-4c49-47cb-8709-3a8ef8e5eae2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab8ffa848e748bfb47492e71560dabe)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:13.755416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c41607-4c49-47cb-8709-3a8ef8e5eae2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:13.815881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-66c41607-4c49-47cb-8709-3a8ef8e5eae2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab8ffa848e748bfb47492e71560dabe', 'request_id': 'req-66c41607-4c49-47cb-8709-3a8ef8e5eae2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:13.829150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-66c41607-4c49-47cb-8709-3a8ef8e5eae2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab8ffa848e748bfb47492e71560dabe', 'request_id': 'req-66c41607-4c49-47cb-8709-3a8ef8e5eae2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:13.836032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-66c41607-4c49-47cb-8709-3a8ef8e5eae2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab8ffa848e748bfb47492e71560dabe', 'request_id': 'req-66c41607-4c49-47cb-8709-3a8ef8e5eae2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:13.850014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-66c41607-4c49-47cb-8709-3a8ef8e5eae2 admin admin] The token cache is being invalidate because project 8ab8ffa848e748bfb47492e71560dabe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:13.851497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-66c41607-4c49-47cb-8709-3a8ef8e5eae2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:13.852175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-66c41607-4c49-47cb-8709-3a8ef8e5eae2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:13.853743 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3248/6418] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:13 2022] DELETE /identity/v3/projects/8ab8ffa848e748bfb47492e71560dabe => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:13.860234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6ff5a-3758-4b29-93b8-461549dbfc52 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:13.860759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6ff5a-3758-4b29-93b8-461549dbfc52 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:13.860945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6ff5a-3758-4b29-93b8-461549dbfc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:13.880891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ece6ff5a-3758-4b29-93b8-461549dbfc52 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:13.882360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ece6ff5a-3758-4b29-93b8-461549dbfc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:13.951653 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3171/6419] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:13.961513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8973e-fe79-46bb-84f0-973ce83d1c17 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:13.961513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8973e-fe79-46bb-84f0-973ce83d1c17 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:13.961513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8973e-fe79-46bb-84f0-973ce83d1c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:13.983601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-83b8973e-fe79-46bb-84f0-973ce83d1c17 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:13.984871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-83b8973e-fe79-46bb-84f0-973ce83d1c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:13.995379 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3249/6420] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:13.999649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6ff5a-3758-4b29-93b8-461549dbfc52 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:14.011908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6ff5a-3758-4b29-93b8-461549dbfc52 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:14.016878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6069bb0-ba34-4614-bd8a-78d918835045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:14.017499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6069bb0-ba34-4614-bd8a-78d918835045 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:14.017798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6069bb0-ba34-4614-bd8a-78d918835045 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:14.018063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6069bb0-ba34-4614-bd8a-78d918835045 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:14.018713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6069bb0-ba34-4614-bd8a-78d918835045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:14.019012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6069bb0-ba34-4614-bd8a-78d918835045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:14.020802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6069bb0-ba34-4614-bd8a-78d918835045 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:14.035022 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3172/6421] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:14.039465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8973e-fe79-46bb-84f0-973ce83d1c17 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:14.052330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8973e-fe79-46bb-84f0-973ce83d1c17 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:14.057496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cc911-58f1-420d-bd54-debd6e47815c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:14.058217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cc911-58f1-420d-bd54-debd6e47815c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:14.058493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cc911-58f1-420d-bd54-debd6e47815c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:14.058761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cc911-58f1-420d-bd54-debd6e47815c admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:14.059407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cc911-58f1-420d-bd54-debd6e47815c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:14.061400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cc911-58f1-420d-bd54-debd6e47815c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:14.093643 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3250/6422] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:14 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:14.097692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6069bb0-ba34-4614-bd8a-78d918835045 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:14.108835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6069bb0-ba34-4614-bd8a-78d918835045 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:14.181849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e31a04-087e-4598-a545-0da200a2288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:14.182597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e31a04-087e-4598-a545-0da200a2288c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:14.182926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e31a04-087e-4598-a545-0da200a2288c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:14.184021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e31a04-087e-4598-a545-0da200a2288c admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:14.184808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e31a04-087e-4598-a545-0da200a2288c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:14.187208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e31a04-087e-4598-a545-0da200a2288c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:14.213296 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3173/6423] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:14 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:14.218651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cc911-58f1-420d-bd54-debd6e47815c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:14.230115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cc911-58f1-420d-bd54-debd6e47815c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:14.236714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75829c-198f-43fd-92fe-00e290d2bb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:14.237385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75829c-198f-43fd-92fe-00e290d2bb8a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:14.237678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75829c-198f-43fd-92fe-00e290d2bb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:14.237943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75829c-198f-43fd-92fe-00e290d2bb8a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:14.238766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75829c-198f-43fd-92fe-00e290d2bb8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:14.241101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75829c-198f-43fd-92fe-00e290d2bb8a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:14.253048 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3251/6424] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:14.257622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e31a04-087e-4598-a545-0da200a2288c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:14.268880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e31a04-087e-4598-a545-0da200a2288c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:14.274939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cfe82-086a-40ff-8f5c-3d3a13c58f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:14.275570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cfe82-086a-40ff-8f5c-3d3a13c58f4f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:14.276004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cfe82-086a-40ff-8f5c-3d3a13c58f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:14.276273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cfe82-086a-40ff-8f5c-3d3a13c58f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:14.276964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cfe82-086a-40ff-8f5c-3d3a13c58f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:14.278595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cfe82-086a-40ff-8f5c-3d3a13c58f4f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:14.290565 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3174/6425] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:14.294946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75829c-198f-43fd-92fe-00e290d2bb8a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:14.306307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75829c-198f-43fd-92fe-00e290d2bb8a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:14.311029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a07fc4-d733-44d0-8079-a69cd831d573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:14.311701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a07fc4-d733-44d0-8079-a69cd831d573 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:14.311967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a07fc4-d733-44d0-8079-a69cd831d573 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:14.312227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a07fc4-d733-44d0-8079-a69cd831d573 admin admin] PATH_INFO: `/v3/projects/ce0bc22a432d44efb17402c8a4e39ed4/users/99a79da61566495e855ff1ba4a467cf2/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:14.329000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a07fc4-d733-44d0-8079-a69cd831d573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce0bc22a432d44efb17402c8a4e39ed4, user_id=99a79da61566495e855ff1ba4a467cf2, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:14.331697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a07fc4-d733-44d0-8079-a69cd831d573 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:14.346448 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3252/6426] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:14 2022] PUT /identity/v3/projects/ce0bc22a432d44efb17402c8a4e39ed4/users/99a79da61566495e855ff1ba4a467cf2/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:14.351842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02748927-87aa-4f93-a0f1-90eab336640d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:14.352181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02748927-87aa-4f93-a0f1-90eab336640d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:14.352582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02748927-87aa-4f93-a0f1-90eab336640d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:14.372330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-02748927-87aa-4f93-a0f1-90eab336640d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:14.373948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-02748927-87aa-4f93-a0f1-90eab336640d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:14.383979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:14.448348 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3175/6427] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:14.458152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a07fc4-d733-44d0-8079-a69cd831d573 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:14.461392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-28a07fc4-d733-44d0-8079-a69cd831d573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:14.525875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a07fc4-d733-44d0-8079-a69cd831d573 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:14.529538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ab05b-bbab-4ad3-80c1-7836305ab135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:14.530175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ab05b-bbab-4ad3-80c1-7836305ab135 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:14.530442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ab05b-bbab-4ad3-80c1-7836305ab135 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:14.530704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ab05b-bbab-4ad3-80c1-7836305ab135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:14.540544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ab05b-bbab-4ad3-80c1-7836305ab135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:14.541985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ab05b-bbab-4ad3-80c1-7836305ab135 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:14.551790 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3253/6428] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:16.757585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02748927-87aa-4f93-a0f1-90eab336640d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:16.759091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-02748927-87aa-4f93-a0f1-90eab336640d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:16.947741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ab05b-bbab-4ad3-80c1-7836305ab135 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:16.949524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9c5ab05b-bbab-4ad3-80c1-7836305ab135 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:16.970568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ab05b-bbab-4ad3-80c1-7836305ab135 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:16.990360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:16.991463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:16.991892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:16.992303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] PATH_INFO: `/v3/users/85805da2da9a419bab802722aeea5f1a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:16.994192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=85805da2da9a419bab802722aeea5f1a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:16.998558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:17.003658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02748927-87aa-4f93-a0f1-90eab336640d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:17.011968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b6d2a-7306-4b00-9f1b-a25a4934e4d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:17.013135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b6d2a-7306-4b00-9f1b-a25a4934e4d2 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:17.013407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b6d2a-7306-4b00-9f1b-a25a4934e4d2 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:17.013670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b6d2a-7306-4b00-9f1b-a25a4934e4d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:17.016792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-234b6d2a-7306-4b00-9f1b-a25a4934e4d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:17.052455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:17.078589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b6d2a-7306-4b00-9f1b-a25a4934e4d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:17.094289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b6d2a-7306-4b00-9f1b-a25a4934e4d2 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:17.116172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85805da2da9a419bab802722aeea5f1a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:17.133601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85805da2da9a419bab802722aeea5f1a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:17.144561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85805da2da9a419bab802722aeea5f1a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:17.150889 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3176/6429] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:17.159873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85805da2da9a419bab802722aeea5f1a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:17.160688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85805da2da9a419bab802722aeea5f1a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:17.170906 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3254/6430] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:16 2022] DELETE /identity/v3/users/85805da2da9a419bab802722aeea5f1a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:17.177819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774606a-45d8-4ce7-9a0b-a95e7e9ed386 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:17.178124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774606a-45d8-4ce7-9a0b-a95e7e9ed386 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:17.178403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774606a-45d8-4ce7-9a0b-a95e7e9ed386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:17.221615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3774606a-45d8-4ce7-9a0b-a95e7e9ed386 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:17.224413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3774606a-45d8-4ce7-9a0b-a95e7e9ed386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:17.376507 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3177/6431] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:17.403232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:17.405142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:17.495072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699d212-886a-4a6f-9f91-9039829f22ba admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:17.499637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c420a1-37f3-4027-9517-45aa97a9dd13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:17.500322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c420a1-37f3-4027-9517-45aa97a9dd13 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:17.500743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c420a1-37f3-4027-9517-45aa97a9dd13 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:17.501076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c420a1-37f3-4027-9517-45aa97a9dd13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:17.511267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c420a1-37f3-4027-9517-45aa97a9dd13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:17.512997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c420a1-37f3-4027-9517-45aa97a9dd13 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:17.531122 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3255/6432] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:17.714643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774606a-45d8-4ce7-9a0b-a95e7e9ed386 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:17.717060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3774606a-45d8-4ce7-9a0b-a95e7e9ed386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:17.733312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774606a-45d8-4ce7-9a0b-a95e7e9ed386 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:17.738805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:17.740082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:17.740082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:17.740082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] PATH_INFO: `/v3/projects/3a33ad1b28874d46b76b7bbe756d339e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:17.744084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a33ad1b28874d46b76b7bbe756d339e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:17.746033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:17.798325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:17.807981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a33ad1b28874d46b76b7bbe756d339e', 'request_id': 'req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:17.820214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a33ad1b28874d46b76b7bbe756d339e', 'request_id': 'req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:17.833458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a33ad1b28874d46b76b7bbe756d339e', 'request_id': 'req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:17.851998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] The token cache is being invalidate because project 3a33ad1b28874d46b76b7bbe756d339e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:17.851998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:17.852682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:17.855463 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3178/6433] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:17 2022] DELETE /identity/v3/projects/3a33ad1b28874d46b76b7bbe756d339e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:17.859550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c420a1-37f3-4027-9517-45aa97a9dd13 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:17.875206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c420a1-37f3-4027-9517-45aa97a9dd13 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:17.972162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:17.972926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:17.973251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:17.973534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] PATH_INFO: `/v3/users/f83fcb7f7a3b4f809f6f284372c1c502` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:17.975036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f83fcb7f7a3b4f809f6f284372c1c502)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:17.977473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:18.057861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:18.065962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f83fcb7f7a3b4f809f6f284372c1c502'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:18.080819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f83fcb7f7a3b4f809f6f284372c1c502'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:18.092284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f83fcb7f7a3b4f809f6f284372c1c502'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:18.118402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f83fcb7f7a3b4f809f6f284372c1c502'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:18.120535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f83fcb7f7a3b4f809f6f284372c1c502'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:18.131067 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3256/6434] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:17 2022] DELETE /identity/v3/users/f83fcb7f7a3b4f809f6f284372c1c502 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:18.711027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:18.712692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:18.735378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d352ffa-2e0b-4a0c-bc72-7c9565847bfc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:18.830163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:18.830979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:18.831196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:18.831794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] PATH_INFO: `/v3/projects/f5b0577869a346ee95ef9ed22fdb036a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:18.832783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5b0577869a346ee95ef9ed22fdb036a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:18.835301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:18.926060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5b0577869a346ee95ef9ed22fdb036a', 'request_id': 'req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:18.945380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5b0577869a346ee95ef9ed22fdb036a', 'request_id': 'req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:18.965881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5b0577869a346ee95ef9ed22fdb036a', 'request_id': 'req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:18.982499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] The token cache is being invalidate because project f5b0577869a346ee95ef9ed22fdb036a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:18.983031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:18.983811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:18.985664 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3179/6435] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:18 2022] DELETE /identity/v3/projects/f5b0577869a346ee95ef9ed22fdb036a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:18.989699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:18.991539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:19.007242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736133e6-d752-45cb-b691-f3cad6b0e088 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:19.096056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29837c77-1b80-4bd8-b121-e737ea472b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:19.096828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29837c77-1b80-4bd8-b121-e737ea472b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:19.097174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29837c77-1b80-4bd8-b121-e737ea472b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:19.097507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29837c77-1b80-4bd8-b121-e737ea472b3d admin admin] PATH_INFO: `/v3/users/336d480048bf4a2eb28153f434634269` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:19.098821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29837c77-1b80-4bd8-b121-e737ea472b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=336d480048bf4a2eb28153f434634269)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:19.101114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29837c77-1b80-4bd8-b121-e737ea472b3d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:19.193161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-29837c77-1b80-4bd8-b121-e737ea472b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336d480048bf4a2eb28153f434634269'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:19.204856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-29837c77-1b80-4bd8-b121-e737ea472b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336d480048bf4a2eb28153f434634269'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:19.242500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-29837c77-1b80-4bd8-b121-e737ea472b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336d480048bf4a2eb28153f434634269'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:19.274281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-29837c77-1b80-4bd8-b121-e737ea472b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336d480048bf4a2eb28153f434634269'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:19.275131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-29837c77-1b80-4bd8-b121-e737ea472b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336d480048bf4a2eb28153f434634269'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:19.287331 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3257/6436] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:18 2022] DELETE /identity/v3/users/336d480048bf4a2eb28153f434634269 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:19.620977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:19.622773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:19.651062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31bf5f-0b3d-4cff-83e1-e02fa0959b5c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:19.798486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:19.808423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:19.811072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:19.811451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:19.811827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] PATH_INFO: `/v3/projects/639616336005402688a4df83ca5be9f2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:19.813165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=639616336005402688a4df83ca5be9f2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:19.815422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:19.898250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '639616336005402688a4df83ca5be9f2', 'request_id': 'req-3a15dc41-0ae7-449b-8b2c-714bb47e4489'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:19.909962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '639616336005402688a4df83ca5be9f2', 'request_id': 'req-3a15dc41-0ae7-449b-8b2c-714bb47e4489'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:19.921757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '639616336005402688a4df83ca5be9f2', 'request_id': 'req-3a15dc41-0ae7-449b-8b2c-714bb47e4489'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:19.944161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] The token cache is being invalidate because project 639616336005402688a4df83ca5be9f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:19.944722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:19.945623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:19.949128 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3180/6437] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:19 2022] DELETE /identity/v3/projects/639616336005402688a4df83ca5be9f2 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:19.954614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99bf7d-5cf3-426c-b5af-343e7d6c0523 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:19.955002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99bf7d-5cf3-426c-b5af-343e7d6c0523 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:19.955967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99bf7d-5cf3-426c-b5af-343e7d6c0523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:19.991044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cd99bf7d-5cf3-426c-b5af-343e7d6c0523 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:19.992356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cd99bf7d-5cf3-426c-b5af-343e7d6c0523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:20.031959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:20.033736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:20.036572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:20.089029 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3258/6438] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:20.095245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb75d4-8386-409f-852f-f51e8e461b5d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:20.095850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb75d4-8386-409f-852f-f51e8e461b5d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:20.096726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb75d4-8386-409f-852f-f51e8e461b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:20.168934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9bfb75d4-8386-409f-852f-f51e8e461b5d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:20.170439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9bfb75d4-8386-409f-852f-f51e8e461b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:20.189065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15dc41-0ae7-449b-8b2c-714bb47e4489 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:20.205992 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3259/6439] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:20.209368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ade83-c657-482a-b760-9b77c5b47ad7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:20.211010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ade83-c657-482a-b760-9b77c5b47ad7 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:20.211430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ade83-c657-482a-b760-9b77c5b47ad7 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:20.211541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb75d4-8386-409f-852f-f51e8e461b5d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:20.212106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ade83-c657-482a-b760-9b77c5b47ad7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:20.214183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d32ade83-c657-482a-b760-9b77c5b47ad7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:20.229454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb75d4-8386-409f-852f-f51e8e461b5d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:20.253542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8998cf-600e-4590-8b13-67853e3c752f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:20.254829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8998cf-600e-4590-8b13-67853e3c752f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:20.255207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8998cf-600e-4590-8b13-67853e3c752f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:20.255573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8998cf-600e-4590-8b13-67853e3c752f admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:20.257885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ade83-c657-482a-b760-9b77c5b47ad7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:20.258385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8998cf-600e-4590-8b13-67853e3c752f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:20.258825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8998cf-600e-4590-8b13-67853e3c752f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:20.259509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ade83-c657-482a-b760-9b77c5b47ad7 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:20.263631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8998cf-600e-4590-8b13-67853e3c752f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:20.281062 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3260/6440] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:20.282014 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3181/6441] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:21:20.285612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8998cf-600e-4590-8b13-67853e3c752f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:20.296611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8998cf-600e-4590-8b13-67853e3c752f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:20.304980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429529c2-0940-4397-9b17-a5a4f47915ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:20.305842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429529c2-0940-4397-9b17-a5a4f47915ea admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:20.306185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429529c2-0940-4397-9b17-a5a4f47915ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:20.306517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429529c2-0940-4397-9b17-a5a4f47915ea admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:20.307246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429529c2-0940-4397-9b17-a5a4f47915ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:20.309995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429529c2-0940-4397-9b17-a5a4f47915ea admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:20.374656 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3261/6442] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:20 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:20.378993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ade83-c657-482a-b760-9b77c5b47ad7 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:20.407925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ade83-c657-482a-b760-9b77c5b47ad7 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:20.511417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691a75c-4c9c-43d2-883d-7d21eb3bd316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:20.513044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691a75c-4c9c-43d2-883d-7d21eb3bd316 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:20.513044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691a75c-4c9c-43d2-883d-7d21eb3bd316 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:20.513044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691a75c-4c9c-43d2-883d-7d21eb3bd316 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:20.513044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691a75c-4c9c-43d2-883d-7d21eb3bd316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:20.515217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691a75c-4c9c-43d2-883d-7d21eb3bd316 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:20.539763 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3182/6443] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:20 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:20.543758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429529c2-0940-4397-9b17-a5a4f47915ea admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:20.555372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429529c2-0940-4397-9b17-a5a4f47915ea admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:20.566381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac422ac4-a9fb-444c-a5b2-0444ccabf0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:20.567478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac422ac4-a9fb-444c-a5b2-0444ccabf0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:20.568202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac422ac4-a9fb-444c-a5b2-0444ccabf0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:20.568202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac422ac4-a9fb-444c-a5b2-0444ccabf0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:20.569326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac422ac4-a9fb-444c-a5b2-0444ccabf0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:20.570934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac422ac4-a9fb-444c-a5b2-0444ccabf0c2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:20.581936 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3262/6444] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:20.586313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691a75c-4c9c-43d2-883d-7d21eb3bd316 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:20.601893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:20.606406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691a75c-4c9c-43d2-883d-7d21eb3bd316 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:20.616047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c3bc3-e496-4eb1-8abf-cbf2ae4739b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:20.617119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c3bc3-e496-4eb1-8abf-cbf2ae4739b3 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:20.617497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c3bc3-e496-4eb1-8abf-cbf2ae4739b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:20.617866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c3bc3-e496-4eb1-8abf-cbf2ae4739b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:20.618590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c3bc3-e496-4eb1-8abf-cbf2ae4739b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:20.620856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c3bc3-e496-4eb1-8abf-cbf2ae4739b3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:20.633591 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3183/6445] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:20.637705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac422ac4-a9fb-444c-a5b2-0444ccabf0c2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:20.649660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac422ac4-a9fb-444c-a5b2-0444ccabf0c2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:20.655020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8e1e6-e4af-470f-ad36-e25b5d0d2da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:20.655770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8e1e6-e4af-470f-ad36-e25b5d0d2da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:20.656413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8e1e6-e4af-470f-ad36-e25b5d0d2da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:20.656413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8e1e6-e4af-470f-ad36-e25b5d0d2da7 admin admin] PATH_INFO: `/v3/projects/99f4196542c5464c86c83894720ebd3b/users/45bedd6ddcef4a9c942c5e0a63565481/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:20.672815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8e1e6-e4af-470f-ad36-e25b5d0d2da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99f4196542c5464c86c83894720ebd3b, user_id=45bedd6ddcef4a9c942c5e0a63565481, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:20.676491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8e1e6-e4af-470f-ad36-e25b5d0d2da7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:20.692215 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3263/6446] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:20 2022] PUT /identity/v3/projects/99f4196542c5464c86c83894720ebd3b/users/45bedd6ddcef4a9c942c5e0a63565481/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:20.698515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec11d5-4937-48d2-a3e7-a90384b7a271 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:20.698864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec11d5-4937-48d2-a3e7-a90384b7a271 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:20.699458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec11d5-4937-48d2-a3e7-a90384b7a271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:20.723290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4dec11d5-4937-48d2-a3e7-a90384b7a271 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:20.725069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4dec11d5-4937-48d2-a3e7-a90384b7a271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:20.797140 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3184/6447] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:20.808558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8e1e6-e4af-470f-ad36-e25b5d0d2da7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:20.810362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f6d8e1e6-e4af-470f-ad36-e25b5d0d2da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:20.878398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8e1e6-e4af-470f-ad36-e25b5d0d2da7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:20.884093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c2236-c6a4-4113-a53b-e8bd7406a079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:20.884827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c2236-c6a4-4113-a53b-e8bd7406a079 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:20.885228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c2236-c6a4-4113-a53b-e8bd7406a079 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:20.885502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c2236-c6a4-4113-a53b-e8bd7406a079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:20.898741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c2236-c6a4-4113-a53b-e8bd7406a079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:20.900402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c2236-c6a4-4113-a53b-e8bd7406a079 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:20.914346 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3264/6448] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:21.352703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dd99f-1d0f-4a7e-afb0-9fb1336161df None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:21.353240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dd99f-1d0f-4a7e-afb0-9fb1336161df None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:21.353240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dd99f-1d0f-4a7e-afb0-9fb1336161df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:21.418386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d41dd99f-1d0f-4a7e-afb0-9fb1336161df None None] MFA Rules not processed for user `99a79da61566495e855ff1ba4a467cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:21.419938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d41dd99f-1d0f-4a7e-afb0-9fb1336161df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:21.546146 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3185/6449] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:21.660537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c2236-c6a4-4113-a53b-e8bd7406a079 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:21.759329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:21.781085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c2236-c6a4-4113-a53b-e8bd7406a079 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:21.792619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c719947-b906-48a3-96be-9e4f3c74a0e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:21.792907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c719947-b906-48a3-96be-9e4f3c74a0e7 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:21.793031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c719947-b906-48a3-96be-9e4f3c74a0e7 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:21.793202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c719947-b906-48a3-96be-9e4f3c74a0e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:21.807587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c719947-b906-48a3-96be-9e4f3c74a0e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:21.809685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c719947-b906-48a3-96be-9e4f3c74a0e7 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:21.822892 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3265/6450] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:22.254514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc30954-cef4-46c2-8c39-069be006884c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:22.255181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc30954-cef4-46c2-8c39-069be006884c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:22.255674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc30954-cef4-46c2-8c39-069be006884c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:22.317469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1cc30954-cef4-46c2-8c39-069be006884c None None] MFA Rules not processed for user `45bedd6ddcef4a9c942c5e0a63565481`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:22.322170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1cc30954-cef4-46c2-8c39-069be006884c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:22.475863 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3186/6451] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:22.476840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c719947-b906-48a3-96be-9e4f3c74a0e7 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:22.489112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c719947-b906-48a3-96be-9e4f3c74a0e7 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:22.494264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac56183-94af-49c5-b707-36a01a7c96e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:22.501847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac56183-94af-49c5-b707-36a01a7c96e3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:22.506622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac56183-94af-49c5-b707-36a01a7c96e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:22.507385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac56183-94af-49c5-b707-36a01a7c96e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:22.507785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac56183-94af-49c5-b707-36a01a7c96e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:22.509819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac56183-94af-49c5-b707-36a01a7c96e3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:22.572116 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3266/6452] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:22 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:22.575982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc30954-cef4-46c2-8c39-069be006884c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:22.591450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc30954-cef4-46c2-8c39-069be006884c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:22.617824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:22.766945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49a2b8-4e90-437c-afdd-586718af4a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:22.768091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49a2b8-4e90-437c-afdd-586718af4a15 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:22.768482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49a2b8-4e90-437c-afdd-586718af4a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:22.768814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49a2b8-4e90-437c-afdd-586718af4a15 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:22.772361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49a2b8-4e90-437c-afdd-586718af4a15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:22.774744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49a2b8-4e90-437c-afdd-586718af4a15 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:22.829187 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3187/6453] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:22 2022] POST /identity/v3/users => generated 295 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:22.830458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac56183-94af-49c5-b707-36a01a7c96e3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:22.844823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac56183-94af-49c5-b707-36a01a7c96e3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:22.853161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76f9cb-b497-42b2-b180-6aa3439da9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:22.854392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76f9cb-b497-42b2-b180-6aa3439da9ab admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:22.854888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76f9cb-b497-42b2-b180-6aa3439da9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:22.855342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76f9cb-b497-42b2-b180-6aa3439da9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:22.856526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76f9cb-b497-42b2-b180-6aa3439da9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:22.858598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76f9cb-b497-42b2-b180-6aa3439da9ab admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:22.876563 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3267/6454] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:22.880747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49a2b8-4e90-437c-afdd-586718af4a15 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:22.892570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49a2b8-4e90-437c-afdd-586718af4a15 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:22.897589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12ef49-7967-4914-898b-55c400e83c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:22.898275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12ef49-7967-4914-898b-55c400e83c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:22.898546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12ef49-7967-4914-898b-55c400e83c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:22.898812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12ef49-7967-4914-898b-55c400e83c94 admin admin] PATH_INFO: `/v3/projects/2cc801be22724963afad7aba7c516d19/users/7eea1ee716fc4b30b2023ad90336fd85/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:22.927307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12ef49-7967-4914-898b-55c400e83c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc801be22724963afad7aba7c516d19, user_id=7eea1ee716fc4b30b2023ad90336fd85, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:22.930583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12ef49-7967-4914-898b-55c400e83c94 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:22.955308 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3188/6455] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:22 2022] PUT /identity/v3/projects/2cc801be22724963afad7aba7c516d19/users/7eea1ee716fc4b30b2023ad90336fd85/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:22.960995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76f9cb-b497-42b2-b180-6aa3439da9ab admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:22.979019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76f9cb-b497-42b2-b180-6aa3439da9ab admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:23.098774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de51e8-7a57-49c5-a865-788c8f6fe0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:23.099422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de51e8-7a57-49c5-a865-788c8f6fe0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:23.099725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de51e8-7a57-49c5-a865-788c8f6fe0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:23.100187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de51e8-7a57-49c5-a865-788c8f6fe0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:23.101103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de51e8-7a57-49c5-a865-788c8f6fe0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:23.102751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de51e8-7a57-49c5-a865-788c8f6fe0f4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:23.128438 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3268/6456] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:23.129851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12ef49-7967-4914-898b-55c400e83c94 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:23.145425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12ef49-7967-4914-898b-55c400e83c94 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:23.159187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912caf3e-55f2-4c77-b4b0-301889dff3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:23.159914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912caf3e-55f2-4c77-b4b0-301889dff3cd admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:23.160246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912caf3e-55f2-4c77-b4b0-301889dff3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:23.160628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912caf3e-55f2-4c77-b4b0-301889dff3cd admin admin] PATH_INFO: `/v3/projects/2cc801be22724963afad7aba7c516d19/users/7eea1ee716fc4b30b2023ad90336fd85/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:23.165997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912caf3e-55f2-4c77-b4b0-301889dff3cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc801be22724963afad7aba7c516d19, user_id=7eea1ee716fc4b30b2023ad90336fd85, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:23.175130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912caf3e-55f2-4c77-b4b0-301889dff3cd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:23.235661 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3189/6457] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:23 2022] PUT /identity/v3/projects/2cc801be22724963afad7aba7c516d19/users/7eea1ee716fc4b30b2023ad90336fd85/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:24.020159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de51e8-7a57-49c5-a865-788c8f6fe0f4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:24.022539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-04de51e8-7a57-49c5-a865-788c8f6fe0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:24.124215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de51e8-7a57-49c5-a865-788c8f6fe0f4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:24.235173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167d73f-84b1-4ea7-b523-1d4033fe8373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:24.235975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167d73f-84b1-4ea7-b523-1d4033fe8373 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:24.236356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167d73f-84b1-4ea7-b523-1d4033fe8373 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:24.237601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167d73f-84b1-4ea7-b523-1d4033fe8373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:24.240171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9167d73f-84b1-4ea7-b523-1d4033fe8373 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:24.284478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:24.305049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a4f3f-4fec-4325-9d55-0e6c8ecf8d9e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:24.306116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a4f3f-4fec-4325-9d55-0e6c8ecf8d9e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:24.306608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a4f3f-4fec-4325-9d55-0e6c8ecf8d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:24.343076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e26a4f3f-4fec-4325-9d55-0e6c8ecf8d9e None None] MFA Rules not processed for user `99a79da61566495e855ff1ba4a467cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:24.345465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e26a4f3f-4fec-4325-9d55-0e6c8ecf8d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:24.375842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167d73f-84b1-4ea7-b523-1d4033fe8373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:24.378709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167d73f-84b1-4ea7-b523-1d4033fe8373 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:24.401095 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3269/6458] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:24.403210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b51c06-484e-4b63-b884-874b3c168329 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:24.403723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b51c06-484e-4b63-b884-874b3c168329 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:24.404719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b51c06-484e-4b63-b884-874b3c168329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:24.462341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-99b51c06-484e-4b63-b884-874b3c168329 None None] MFA Rules not processed for user `7eea1ee716fc4b30b2023ad90336fd85`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:24.467474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-99b51c06-484e-4b63-b884-874b3c168329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:24.493315 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3190/6459] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:24.636546 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3270/6460] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:24.638247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113557ed-1963-43e4-96c0-496019ad07e2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:24.638528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113557ed-1963-43e4-96c0-496019ad07e2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:24.638818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113557ed-1963-43e4-96c0-496019ad07e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:24.681466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-113557ed-1963-43e4-96c0-496019ad07e2 None None] MFA Rules not processed for user `45bedd6ddcef4a9c942c5e0a63565481`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:24.682693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-113557ed-1963-43e4-96c0-496019ad07e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:24.815954 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3191/6461] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:24.827509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b51c06-484e-4b63-b884-874b3c168329 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:24.928310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b51c06-484e-4b63-b884-874b3c168329 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:24.937933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b37508-d0db-438b-95f5-e0530a1f3bbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:24.939309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b37508-d0db-438b-95f5-e0530a1f3bbf service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:24.940167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b37508-d0db-438b-95f5-e0530a1f3bbf service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:24.940683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b37508-d0db-438b-95f5-e0530a1f3bbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:24.959252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b37508-d0db-438b-95f5-e0530a1f3bbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:24.963461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b37508-d0db-438b-95f5-e0530a1f3bbf service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:24.985913 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3271/6462] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:29.452338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113557ed-1963-43e4-96c0-496019ad07e2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:29.454099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-113557ed-1963-43e4-96c0-496019ad07e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:29.482938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113557ed-1963-43e4-96c0-496019ad07e2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:29.490280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2b8f4-30ae-4c60-85d5-dc3f9d1b09a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:29.491353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2b8f4-30ae-4c60-85d5-dc3f9d1b09a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:29.491756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2b8f4-30ae-4c60-85d5-dc3f9d1b09a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:29.492082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2b8f4-30ae-4c60-85d5-dc3f9d1b09a0 admin admin] PATH_INFO: `/v3/users/811bc5f0350147f18454a1ac0127e47b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:29.493404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2b8f4-30ae-4c60-85d5-dc3f9d1b09a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=811bc5f0350147f18454a1ac0127e47b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:29.495493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2b8f4-30ae-4c60-85d5-dc3f9d1b09a0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:29.512331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:29.567495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-89e2b8f4-30ae-4c60-85d5-dc3f9d1b09a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '811bc5f0350147f18454a1ac0127e47b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:29.579974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-89e2b8f4-30ae-4c60-85d5-dc3f9d1b09a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '811bc5f0350147f18454a1ac0127e47b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:29.597267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-89e2b8f4-30ae-4c60-85d5-dc3f9d1b09a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '811bc5f0350147f18454a1ac0127e47b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:29.598056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-89e2b8f4-30ae-4c60-85d5-dc3f9d1b09a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '811bc5f0350147f18454a1ac0127e47b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:29.598666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-89e2b8f4-30ae-4c60-85d5-dc3f9d1b09a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '811bc5f0350147f18454a1ac0127e47b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:29.617576 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3192/6463] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:29 2022] DELETE /identity/v3/users/811bc5f0350147f18454a1ac0127e47b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:30.049960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b37508-d0db-438b-95f5-e0530a1f3bbf service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:30.064592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b37508-d0db-438b-95f5-e0530a1f3bbf service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:30.093470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:30.136320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:30.137014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:30.137291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:30.138365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e admin admin] PATH_INFO: `/v3/projects/ce96fc00ce214713947c7e949c7a7192` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:30.139991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce96fc00ce214713947c7e949c7a7192)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:30.142658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:30.205485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce96fc00ce214713947c7e949c7a7192', 'request_id': 'req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:30.218776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce96fc00ce214713947c7e949c7a7192', 'request_id': 'req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:30.241873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce96fc00ce214713947c7e949c7a7192', 'request_id': 'req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:30.258187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e admin admin] The token cache is being invalidate because project ce96fc00ce214713947c7e949c7a7192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:30.258559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:30.259365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-20a3f1d4-bcfd-48e5-8edc-babc63a8969e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:30.261264 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3272/6464] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:30 2022] DELETE /identity/v3/projects/ce96fc00ce214713947c7e949c7a7192 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:30.268930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d95993-621a-40d4-b66f-69b3dcc598e1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:30.269054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d95993-621a-40d4-b66f-69b3dcc598e1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:30.269299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d95993-621a-40d4-b66f-69b3dcc598e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:30.297741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e6d95993-621a-40d4-b66f-69b3dcc598e1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:30.298983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e6d95993-621a-40d4-b66f-69b3dcc598e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:30.457916 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3193/6465] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:30.466206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a133f6d-2a07-4a45-95c1-a85ffbc60009 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:30.466620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a133f6d-2a07-4a45-95c1-a85ffbc60009 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:30.466798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a133f6d-2a07-4a45-95c1-a85ffbc60009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:30.511711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8a133f6d-2a07-4a45-95c1-a85ffbc60009 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:30.513432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8a133f6d-2a07-4a45-95c1-a85ffbc60009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:30.526169 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3273/6466] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:30.530722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d95993-621a-40d4-b66f-69b3dcc598e1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:30.544708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d95993-621a-40d4-b66f-69b3dcc598e1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:30.551577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901da4a0-cb0c-413e-a356-929d192ad72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:30.564847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901da4a0-cb0c-413e-a356-929d192ad72b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:30.566324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901da4a0-cb0c-413e-a356-929d192ad72b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:30.566825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901da4a0-cb0c-413e-a356-929d192ad72b admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:30.568054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901da4a0-cb0c-413e-a356-929d192ad72b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:30.570518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901da4a0-cb0c-413e-a356-929d192ad72b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:30.574674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901da4a0-cb0c-413e-a356-929d192ad72b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:30.624928 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3194/6467] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:30.628991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a133f6d-2a07-4a45-95c1-a85ffbc60009 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:30.639972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a133f6d-2a07-4a45-95c1-a85ffbc60009 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:30.645170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add546fd-93c0-4cd3-b661-931427032911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:30.646349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add546fd-93c0-4cd3-b661-931427032911 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:30.646349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add546fd-93c0-4cd3-b661-931427032911 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:30.646629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add546fd-93c0-4cd3-b661-931427032911 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:30.648026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add546fd-93c0-4cd3-b661-931427032911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:30.650088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add546fd-93c0-4cd3-b661-931427032911 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:30.699502 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3274/6468] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:30 2022] POST /identity/v3/projects => generated 367 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:30.703664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901da4a0-cb0c-413e-a356-929d192ad72b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:30.718947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901da4a0-cb0c-413e-a356-929d192ad72b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:30.825060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f8a98-7b9b-4ab7-8952-739bd8cd90dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:30.825834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8a98-7b9b-4ab7-8952-739bd8cd90dd admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:30.826112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8a98-7b9b-4ab7-8952-739bd8cd90dd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:30.826443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8a98-7b9b-4ab7-8952-739bd8cd90dd admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:30.827073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f8a98-7b9b-4ab7-8952-739bd8cd90dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:30.829659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f8a98-7b9b-4ab7-8952-739bd8cd90dd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:30.865556 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3195/6469] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:30 2022] POST /identity/v3/users => generated 284 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:30.866739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add546fd-93c0-4cd3-b661-931427032911 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:30.896319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add546fd-93c0-4cd3-b661-931427032911 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:30.907036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c67ebc-3467-48c0-95f7-553a19be9fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:30.907036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c67ebc-3467-48c0-95f7-553a19be9fef admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:30.907036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c67ebc-3467-48c0-95f7-553a19be9fef admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:30.907036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c67ebc-3467-48c0-95f7-553a19be9fef admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:30.907390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c67ebc-3467-48c0-95f7-553a19be9fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:30.909874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c67ebc-3467-48c0-95f7-553a19be9fef admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:30.932680 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3275/6470] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:30.935118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f8a98-7b9b-4ab7-8952-739bd8cd90dd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:30.941964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:30.953798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f8a98-7b9b-4ab7-8952-739bd8cd90dd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:30.978595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d371ec2-ef96-4db0-bd48-6d09b7df01fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:30.978595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d371ec2-ef96-4db0-bd48-6d09b7df01fb admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:30.979797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d371ec2-ef96-4db0-bd48-6d09b7df01fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:30.979797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d371ec2-ef96-4db0-bd48-6d09b7df01fb admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:30.984132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d371ec2-ef96-4db0-bd48-6d09b7df01fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:30.986756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d371ec2-ef96-4db0-bd48-6d09b7df01fb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:31.019930 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3196/6471] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:31.021364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c67ebc-3467-48c0-95f7-553a19be9fef admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:31.045577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c67ebc-3467-48c0-95f7-553a19be9fef admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:31.052877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70bd5f7-0e05-4996-b689-72a2b51e2b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:31.054868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70bd5f7-0e05-4996-b689-72a2b51e2b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:31.055305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70bd5f7-0e05-4996-b689-72a2b51e2b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:31.055722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70bd5f7-0e05-4996-b689-72a2b51e2b80 admin admin] PATH_INFO: `/v3/projects/0de65fd605b54265b7dfd053748c654d/users/83ea757df4db405c9584eec28d4c9fd3/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:31.093111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70bd5f7-0e05-4996-b689-72a2b51e2b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de65fd605b54265b7dfd053748c654d, user_id=83ea757df4db405c9584eec28d4c9fd3, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:31.102600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70bd5f7-0e05-4996-b689-72a2b51e2b80 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:31.155113 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3276/6472] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:31 2022] PUT /identity/v3/projects/0de65fd605b54265b7dfd053748c654d/users/83ea757df4db405c9584eec28d4c9fd3/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:31.165839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ab7cb-4baf-4b43-b930-98f13df8150e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:31.166116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ab7cb-4baf-4b43-b930-98f13df8150e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:31.166387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ab7cb-4baf-4b43-b930-98f13df8150e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:31.239258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9a3ab7cb-4baf-4b43-b930-98f13df8150e None None] MFA Rules not processed for user `83ea757df4db405c9584eec28d4c9fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:31.240793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9a3ab7cb-4baf-4b43-b930-98f13df8150e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:31.462910 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3197/6473] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:31.464329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70bd5f7-0e05-4996-b689-72a2b51e2b80 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:31.570025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:31.581868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70bd5f7-0e05-4996-b689-72a2b51e2b80 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:31.593319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5718fa5-ef65-499f-8218-3da3efef3de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:31.600033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5718fa5-ef65-499f-8218-3da3efef3de5 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:31.601426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5718fa5-ef65-499f-8218-3da3efef3de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:31.601880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5718fa5-ef65-499f-8218-3da3efef3de5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:31.603637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5718fa5-ef65-499f-8218-3da3efef3de5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:31.606525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5718fa5-ef65-499f-8218-3da3efef3de5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:31.667954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ab7cb-4baf-4b43-b930-98f13df8150e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:31.668626 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3277/6474] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:31 2022] POST /identity/v3/projects => generated 367 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:31.684681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ab7cb-4baf-4b43-b930-98f13df8150e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:31.814090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2602c-daf4-4f45-9c71-a94b247bfac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:31.814886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2602c-daf4-4f45-9c71-a94b247bfac2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:31.814994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2602c-daf4-4f45-9c71-a94b247bfac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:31.815255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2602c-daf4-4f45-9c71-a94b247bfac2 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:31.815889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2602c-daf4-4f45-9c71-a94b247bfac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:31.817936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2602c-daf4-4f45-9c71-a94b247bfac2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:31.844632 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3198/6475] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:31 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:31.849138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5718fa5-ef65-499f-8218-3da3efef3de5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:31.864502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5718fa5-ef65-499f-8218-3da3efef3de5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:31.871406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0686058-dcb6-4ad7-bae4-75b308b215b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:31.872282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0686058-dcb6-4ad7-bae4-75b308b215b9 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:31.873389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0686058-dcb6-4ad7-bae4-75b308b215b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:31.874051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0686058-dcb6-4ad7-bae4-75b308b215b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:31.875162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0686058-dcb6-4ad7-bae4-75b308b215b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:31.876914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0686058-dcb6-4ad7-bae4-75b308b215b9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:31.904321 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3278/6476] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:31.904689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2602c-daf4-4f45-9c71-a94b247bfac2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:31.919171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2602c-daf4-4f45-9c71-a94b247bfac2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:31.932495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47cdd0-6e06-4294-a3e4-01187b80d7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:31.932495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47cdd0-6e06-4294-a3e4-01187b80d7a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:31.932495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47cdd0-6e06-4294-a3e4-01187b80d7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:31.933110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47cdd0-6e06-4294-a3e4-01187b80d7a2 admin admin] PATH_INFO: `/v3/projects/0f4516d917044c62af9d1ea8c2b9b6cb/users/64b40d6e43fd45209f05e5f78767ed56/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:31.956052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47cdd0-6e06-4294-a3e4-01187b80d7a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f4516d917044c62af9d1ea8c2b9b6cb, user_id=64b40d6e43fd45209f05e5f78767ed56, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:31.961946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47cdd0-6e06-4294-a3e4-01187b80d7a2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:31.992538 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3199/6477] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:31 2022] PUT /identity/v3/projects/0f4516d917044c62af9d1ea8c2b9b6cb/users/64b40d6e43fd45209f05e5f78767ed56/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:31.996405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0686058-dcb6-4ad7-bae4-75b308b215b9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:32.023237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0686058-dcb6-4ad7-bae4-75b308b215b9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:32.164280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebe966-43e9-4240-bf70-b5a32c660c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:32.164942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebe966-43e9-4240-bf70-b5a32c660c9a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:32.164942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebe966-43e9-4240-bf70-b5a32c660c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:32.165800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebe966-43e9-4240-bf70-b5a32c660c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:32.168507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebe966-43e9-4240-bf70-b5a32c660c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:32.172219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebe966-43e9-4240-bf70-b5a32c660c9a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:32.203169 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3279/6478] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:32.204738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47cdd0-6e06-4294-a3e4-01187b80d7a2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:32.225470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47cdd0-6e06-4294-a3e4-01187b80d7a2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:32.241446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551e1cd-6003-4e15-bc22-bfb4c1d468ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:32.242774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551e1cd-6003-4e15-bc22-bfb4c1d468ab admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:32.243171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551e1cd-6003-4e15-bc22-bfb4c1d468ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:32.243592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551e1cd-6003-4e15-bc22-bfb4c1d468ab admin admin] PATH_INFO: `/v3/projects/0f4516d917044c62af9d1ea8c2b9b6cb/users/64b40d6e43fd45209f05e5f78767ed56/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:32.249791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551e1cd-6003-4e15-bc22-bfb4c1d468ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f4516d917044c62af9d1ea8c2b9b6cb, user_id=64b40d6e43fd45209f05e5f78767ed56, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:32.258725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551e1cd-6003-4e15-bc22-bfb4c1d468ab admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:32.285969 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3200/6479] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:32 2022] PUT /identity/v3/projects/0f4516d917044c62af9d1ea8c2b9b6cb/users/64b40d6e43fd45209f05e5f78767ed56/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:32.298510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96e38a-622c-4c82-a55b-872f216ccc07 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:32.298794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96e38a-622c-4c82-a55b-872f216ccc07 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:32.299067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96e38a-622c-4c82-a55b-872f216ccc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:32.336479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ef96e38a-622c-4c82-a55b-872f216ccc07 None None] MFA Rules not processed for user `64b40d6e43fd45209f05e5f78767ed56`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:32.337822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ef96e38a-622c-4c82-a55b-872f216ccc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:32.499654 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3280/6480] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:32.503883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551e1cd-6003-4e15-bc22-bfb4c1d468ab admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:32.519556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551e1cd-6003-4e15-bc22-bfb4c1d468ab admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:32.531168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9933d2d-ca54-4a39-a9a8-5ac96db01157 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:32.531811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9933d2d-ca54-4a39-a9a8-5ac96db01157 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:32.532087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9933d2d-ca54-4a39-a9a8-5ac96db01157 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:32.532365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9933d2d-ca54-4a39-a9a8-5ac96db01157 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:32.533071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9933d2d-ca54-4a39-a9a8-5ac96db01157 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:32.534700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9933d2d-ca54-4a39-a9a8-5ac96db01157 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:32.569979 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3201/6481] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:21:32 2022] POST /identity/v3/domains => generated 276 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:32.574180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96e38a-622c-4c82-a55b-872f216ccc07 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:32.592149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96e38a-622c-4c82-a55b-872f216ccc07 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:32.702704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:32.733348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbb4ae-6d87-4f28-b201-355676fce339 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:32.734277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbb4ae-6d87-4f28-b201-355676fce339 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:32.734788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbb4ae-6d87-4f28-b201-355676fce339 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:32.735068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbb4ae-6d87-4f28-b201-355676fce339 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:32.736465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbb4ae-6d87-4f28-b201-355676fce339 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:32.750649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbb4ae-6d87-4f28-b201-355676fce339 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:32.794563 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3281/6482] 10.222.0.23 () {60 vars in 1147 bytes} [Sat Aug 20 21:21:32 2022] POST /identity/v3/groups => generated 280 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:32.798781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9933d2d-ca54-4a39-a9a8-5ac96db01157 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:32.815017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9933d2d-ca54-4a39-a9a8-5ac96db01157 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:32.827744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a3c9f-aa28-4860-aac6-5c4dc5dacbdf tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:32.828578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a3c9f-aa28-4860-aac6-5c4dc5dacbdf tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:32.828923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a3c9f-aa28-4860-aac6-5c4dc5dacbdf tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:32.829307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a3c9f-aa28-4860-aac6-5c4dc5dacbdf tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/4d7cf4517c2643c19ab8ab9e66c27e0c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:32.844407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a3c9f-aa28-4860-aac6-5c4dc5dacbdf tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:update_group(group_id=4d7cf4517c2643c19ab8ab9e66c27e0c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:32.845646 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-7b0a3c9f-aa28-4860-aac6-5c4dc5dacbdf tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:21:32.845646 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:21:32.845646 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:21:32.848467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a3c9f-aa28-4860-aac6-5c4dc5dacbdf tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:32.879611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:UPDATE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:32.894868 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3202/6483] 10.222.0.23 () {60 vars in 1247 bytes} [Sat Aug 20 21:21:32 2022] PATCH /identity/v3/groups/4d7cf4517c2643c19ab8ab9e66c27e0c => generated 291 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:32.896173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbb4ae-6d87-4f28-b201-355676fce339 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:32.914712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbb4ae-6d87-4f28-b201-355676fce339 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:32.931016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1a9ac-320e-49e5-bc0e-60a16f3a8b49 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:32.931683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1a9ac-320e-49e5-bc0e-60a16f3a8b49 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:32.931957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1a9ac-320e-49e5-bc0e-60a16f3a8b49 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:32.932221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1a9ac-320e-49e5-bc0e-60a16f3a8b49 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/4d7cf4517c2643c19ab8ab9e66c27e0c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:32.938981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1a9ac-320e-49e5-bc0e-60a16f3a8b49 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:get_group(group_id=4d7cf4517c2643c19ab8ab9e66c27e0c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:32.942125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1a9ac-320e-49e5-bc0e-60a16f3a8b49 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:32.947017 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3282/6484] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:32 2022] GET /identity/v3/groups/4d7cf4517c2643c19ab8ab9e66c27e0c => generated 291 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:32.950566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a3c9f-aa28-4860-aac6-5c4dc5dacbdf tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:32.968093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a3c9f-aa28-4860-aac6-5c4dc5dacbdf tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:32.985318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a22cf-e8e9-468c-bb18-a32146c060fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:32.986212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a22cf-e8e9-468c-bb18-a32146c060fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:32.987102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a22cf-e8e9-468c-bb18-a32146c060fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:32.987913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a22cf-e8e9-468c-bb18-a32146c060fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/4d7cf4517c2643c19ab8ab9e66c27e0c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:32.989671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a22cf-e8e9-468c-bb18-a32146c060fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:update_group(group_id=4d7cf4517c2643c19ab8ab9e66c27e0c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:32.992747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a22cf-e8e9-468c-bb18-a32146c060fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:33.025011 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3203/6485] 10.222.0.23 () {60 vars in 1246 bytes} [Sat Aug 20 21:21:32 2022] PATCH /identity/v3/groups/4d7cf4517c2643c19ab8ab9e66c27e0c => generated 307 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:33.030006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1a9ac-320e-49e5-bc0e-60a16f3a8b49 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:33.044766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1a9ac-320e-49e5-bc0e-60a16f3a8b49 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:33.051645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704001a-ad11-4e1e-9e20-1846c2667766 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:33.052700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704001a-ad11-4e1e-9e20-1846c2667766 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:33.053166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704001a-ad11-4e1e-9e20-1846c2667766 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:33.053772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704001a-ad11-4e1e-9e20-1846c2667766 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/4d7cf4517c2643c19ab8ab9e66c27e0c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:33.061003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704001a-ad11-4e1e-9e20-1846c2667766 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:delete_group(group_id=4d7cf4517c2643c19ab8ab9e66c27e0c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:33.065978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704001a-ad11-4e1e-9e20-1846c2667766 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:33.154304 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3283/6486] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:21:33 2022] DELETE /identity/v3/groups/4d7cf4517c2643c19ab8ab9e66c27e0c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:33.159634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a22cf-e8e9-468c-bb18-a32146c060fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:33.175158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a22cf-e8e9-468c-bb18-a32146c060fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:33.276957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c071090-b2fc-451c-b262-a52fd5d4cf57 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:33.277541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c071090-b2fc-451c-b262-a52fd5d4cf57 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:33.277823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c071090-b2fc-451c-b262-a52fd5d4cf57 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:33.278072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c071090-b2fc-451c-b262-a52fd5d4cf57 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:33.278699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c071090-b2fc-451c-b262-a52fd5d4cf57 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:33.287656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c071090-b2fc-451c-b262-a52fd5d4cf57 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:33.317141 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3204/6487] 10.222.0.23 () {60 vars in 1147 bytes} [Sat Aug 20 21:21:33 2022] POST /identity/v3/groups => generated 325 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:33.321846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704001a-ad11-4e1e-9e20-1846c2667766 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:33.334379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704001a-ad11-4e1e-9e20-1846c2667766 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:33.339223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b64ce4-bb75-4726-b717-d47b012ed776 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:33.339845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b64ce4-bb75-4726-b717-d47b012ed776 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:33.340133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b64ce4-bb75-4726-b717-d47b012ed776 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:33.340474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b64ce4-bb75-4726-b717-d47b012ed776 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:33.341112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b64ce4-bb75-4726-b717-d47b012ed776 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:33.343059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b64ce4-bb75-4726-b717-d47b012ed776 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:33.374045 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3284/6488] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:33 2022] POST /identity/v3/users => generated 319 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:33.378199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c071090-b2fc-451c-b262-a52fd5d4cf57 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:33.407099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c071090-b2fc-451c-b262-a52fd5d4cf57 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:33.412867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41991ea4-651c-4a5d-8605-c2e999e442fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:33.413839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41991ea4-651c-4a5d-8605-c2e999e442fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:33.414707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41991ea4-651c-4a5d-8605-c2e999e442fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:33.414707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41991ea4-651c-4a5d-8605-c2e999e442fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/5e4067393801418e8de5d0a066c1d516/users/504b9374c77a4b91aac712c39108a33e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:33.440858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41991ea4-651c-4a5d-8605-c2e999e442fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=5e4067393801418e8de5d0a066c1d516, user_id=504b9374c77a4b91aac712c39108a33e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:33.443777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41991ea4-651c-4a5d-8605-c2e999e442fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:33.471291 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3205/6489] 10.222.0.23 () {60 vars in 1360 bytes} [Sat Aug 20 21:21:33 2022] PUT /identity/v3/groups/5e4067393801418e8de5d0a066c1d516/users/504b9374c77a4b91aac712c39108a33e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:33.475668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b64ce4-bb75-4726-b717-d47b012ed776 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:33.486467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b64ce4-bb75-4726-b717-d47b012ed776 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:33.577156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a79b51-5db8-4155-ada5-9b476825ac74 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:33.577895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a79b51-5db8-4155-ada5-9b476825ac74 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:33.578175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a79b51-5db8-4155-ada5-9b476825ac74 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:33.578448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a79b51-5db8-4155-ada5-9b476825ac74 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:33.579139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a79b51-5db8-4155-ada5-9b476825ac74 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:33.581670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a79b51-5db8-4155-ada5-9b476825ac74 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:33.614807 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3285/6490] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:33 2022] POST /identity/v3/users => generated 323 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:33.619286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41991ea4-651c-4a5d-8605-c2e999e442fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:33.631369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41991ea4-651c-4a5d-8605-c2e999e442fc tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:33.636566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67ae4d-d87e-48d4-9c1f-aeb07327d43b tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:33.637504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67ae4d-d87e-48d4-9c1f-aeb07327d43b tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:33.637918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67ae4d-d87e-48d4-9c1f-aeb07327d43b tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:33.638327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67ae4d-d87e-48d4-9c1f-aeb07327d43b tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/5e4067393801418e8de5d0a066c1d516/users/ed9845a18ff649a781ccb74beeb2ff53` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:33.655967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67ae4d-d87e-48d4-9c1f-aeb07327d43b tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=5e4067393801418e8de5d0a066c1d516, user_id=ed9845a18ff649a781ccb74beeb2ff53)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:33.658073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67ae4d-d87e-48d4-9c1f-aeb07327d43b tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:33.693135 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3206/6491] 10.222.0.23 () {60 vars in 1360 bytes} [Sat Aug 20 21:21:33 2022] PUT /identity/v3/groups/5e4067393801418e8de5d0a066c1d516/users/ed9845a18ff649a781ccb74beeb2ff53 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:33.697882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a79b51-5db8-4155-ada5-9b476825ac74 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:33.710355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a79b51-5db8-4155-ada5-9b476825ac74 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:33.812559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:INSERT=7,keystone:DELETE=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:33.843371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe97c00-e760-4a23-b0b5-0b2f165c96ac tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:33.844013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97c00-e760-4a23-b0b5-0b2f165c96ac tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:33.844320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97c00-e760-4a23-b0b5-0b2f165c96ac tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:33.844585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97c00-e760-4a23-b0b5-0b2f165c96ac tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:33.845217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe97c00-e760-4a23-b0b5-0b2f165c96ac tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:33.849235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe97c00-e760-4a23-b0b5-0b2f165c96ac tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:33.910804 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3286/6492] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:33 2022] POST /identity/v3/users => generated 321 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:33.915007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67ae4d-d87e-48d4-9c1f-aeb07327d43b tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:33.938509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67ae4d-d87e-48d4-9c1f-aeb07327d43b tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:33.953489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce365b7-00d4-4aa0-9b4a-02f24c09d0f3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:33.954177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce365b7-00d4-4aa0-9b4a-02f24c09d0f3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:33.954457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce365b7-00d4-4aa0-9b4a-02f24c09d0f3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:33.954731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce365b7-00d4-4aa0-9b4a-02f24c09d0f3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/5e4067393801418e8de5d0a066c1d516/users/d0b5782ba70b433ba1ca3c2657e8b7b2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:33.997921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce365b7-00d4-4aa0-9b4a-02f24c09d0f3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=5e4067393801418e8de5d0a066c1d516, user_id=d0b5782ba70b433ba1ca3c2657e8b7b2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:34.000077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce365b7-00d4-4aa0-9b4a-02f24c09d0f3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:34.055048 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3207/6493] 10.222.0.23 () {60 vars in 1360 bytes} [Sat Aug 20 21:21:33 2022] PUT /identity/v3/groups/5e4067393801418e8de5d0a066c1d516/users/d0b5782ba70b433ba1ca3c2657e8b7b2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:34.059457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe97c00-e760-4a23-b0b5-0b2f165c96ac tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:34.071789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe97c00-e760-4a23-b0b5-0b2f165c96ac tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:34.184359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66d7fc-cb45-483b-9a6c-072ccb9b765d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:34.186580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66d7fc-cb45-483b-9a6c-072ccb9b765d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:34.187009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66d7fc-cb45-483b-9a6c-072ccb9b765d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:34.187306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66d7fc-cb45-483b-9a6c-072ccb9b765d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/5e4067393801418e8de5d0a066c1d516/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:34.189205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66d7fc-cb45-483b-9a6c-072ccb9b765d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=5e4067393801418e8de5d0a066c1d516)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:34.190642 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-ac66d7fc-cb45-483b-9a6c-072ccb9b765d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:21:34.190642 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:21:34.190642 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:21:34.192707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66d7fc-cb45-483b-9a6c-072ccb9b765d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:34.233461 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3287/6494] 10.222.0.23 () {58 vars in 1242 bytes} [Sat Aug 20 21:21:34 2022] GET /identity/v3/groups/5e4067393801418e8de5d0a066c1d516/users => generated 1079 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:34.237882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce365b7-00d4-4aa0-9b4a-02f24c09d0f3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:34.256271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce365b7-00d4-4aa0-9b4a-02f24c09d0f3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:34.274424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25908d48-64b9-4c7f-ae0d-f59a3006ee77 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:34.275085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25908d48-64b9-4c7f-ae0d-f59a3006ee77 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:34.275353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25908d48-64b9-4c7f-ae0d-f59a3006ee77 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:34.275625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25908d48-64b9-4c7f-ae0d-f59a3006ee77 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/5e4067393801418e8de5d0a066c1d516/users/504b9374c77a4b91aac712c39108a33e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:34.278428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25908d48-64b9-4c7f-ae0d-f59a3006ee77 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=5e4067393801418e8de5d0a066c1d516, user_id=504b9374c77a4b91aac712c39108a33e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:34.279804 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-25908d48-64b9-4c7f-ae0d-f59a3006ee77 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_user_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:21:34.279804 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:21:34.279804 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:21:34.281109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25908d48-64b9-4c7f-ae0d-f59a3006ee77 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:34.309259 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3208/6495] 10.222.0.23 () {58 vars in 1342 bytes} [Sat Aug 20 21:21:34 2022] HEAD /identity/v3/groups/5e4067393801418e8de5d0a066c1d516/users/504b9374c77a4b91aac712c39108a33e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:34.309924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66d7fc-cb45-483b-9a6c-072ccb9b765d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:34.333114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66d7fc-cb45-483b-9a6c-072ccb9b765d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:34.340454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42754912-957b-445b-97f2-1bf9ef4ec91d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:34.341115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42754912-957b-445b-97f2-1bf9ef4ec91d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:34.341391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42754912-957b-445b-97f2-1bf9ef4ec91d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:34.341661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42754912-957b-445b-97f2-1bf9ef4ec91d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/5e4067393801418e8de5d0a066c1d516/users/504b9374c77a4b91aac712c39108a33e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:34.343933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42754912-957b-445b-97f2-1bf9ef4ec91d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=5e4067393801418e8de5d0a066c1d516, user_id=504b9374c77a4b91aac712c39108a33e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:34.345920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42754912-957b-445b-97f2-1bf9ef4ec91d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:34.356890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-42754912-957b-445b-97f2-1bf9ef4ec91d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '504b9374c77a4b91aac712c39108a33e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:34.384010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25908d48-64b9-4c7f-ae0d-f59a3006ee77 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:34.385044 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3288/6496] 10.222.0.23 () {58 vars in 1344 bytes} [Sat Aug 20 21:21:34 2022] DELETE /identity/v3/groups/5e4067393801418e8de5d0a066c1d516/users/504b9374c77a4b91aac712c39108a33e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:34.395503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25908d48-64b9-4c7f-ae0d-f59a3006ee77 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:34.413480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:34.462049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75acde7-f6da-432d-9688-668082abf10f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:34.462049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75acde7-f6da-432d-9688-668082abf10f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:34.462319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75acde7-f6da-432d-9688-668082abf10f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:34.462586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75acde7-f6da-432d-9688-668082abf10f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/5e4067393801418e8de5d0a066c1d516/users/ed9845a18ff649a781ccb74beeb2ff53` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:34.464258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75acde7-f6da-432d-9688-668082abf10f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=5e4067393801418e8de5d0a066c1d516, user_id=ed9845a18ff649a781ccb74beeb2ff53)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:34.466611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75acde7-f6da-432d-9688-668082abf10f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:34.484434 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3209/6497] 10.222.0.23 () {58 vars in 1342 bytes} [Sat Aug 20 21:21:34 2022] HEAD /identity/v3/groups/5e4067393801418e8de5d0a066c1d516/users/ed9845a18ff649a781ccb74beeb2ff53 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:34.487598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42754912-957b-445b-97f2-1bf9ef4ec91d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:34.504603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42754912-957b-445b-97f2-1bf9ef4ec91d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:34.511456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d80fdc-3174-4fd8-8794-8bf6b5c3cfc7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:34.512110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d80fdc-3174-4fd8-8794-8bf6b5c3cfc7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:34.512398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d80fdc-3174-4fd8-8794-8bf6b5c3cfc7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:34.512770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d80fdc-3174-4fd8-8794-8bf6b5c3cfc7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/5e4067393801418e8de5d0a066c1d516/users/ed9845a18ff649a781ccb74beeb2ff53` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:34.514461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d80fdc-3174-4fd8-8794-8bf6b5c3cfc7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=5e4067393801418e8de5d0a066c1d516, user_id=ed9845a18ff649a781ccb74beeb2ff53)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:34.516402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d80fdc-3174-4fd8-8794-8bf6b5c3cfc7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:34.525199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91d80fdc-3174-4fd8-8794-8bf6b5c3cfc7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'ed9845a18ff649a781ccb74beeb2ff53'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:34.540920 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3289/6498] 10.222.0.23 () {58 vars in 1344 bytes} [Sat Aug 20 21:21:34 2022] DELETE /identity/v3/groups/5e4067393801418e8de5d0a066c1d516/users/ed9845a18ff649a781ccb74beeb2ff53 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:34.545576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75acde7-f6da-432d-9688-668082abf10f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:34.556221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75acde7-f6da-432d-9688-668082abf10f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:34.619616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb537f0-748a-443c-98b5-b1fe2aabb19e tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:34.620276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb537f0-748a-443c-98b5-b1fe2aabb19e tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:34.620595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb537f0-748a-443c-98b5-b1fe2aabb19e tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:34.620871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb537f0-748a-443c-98b5-b1fe2aabb19e tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/5e4067393801418e8de5d0a066c1d516/users/d0b5782ba70b433ba1ca3c2657e8b7b2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:34.622546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb537f0-748a-443c-98b5-b1fe2aabb19e tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=5e4067393801418e8de5d0a066c1d516, user_id=d0b5782ba70b433ba1ca3c2657e8b7b2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:34.624518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb537f0-748a-443c-98b5-b1fe2aabb19e tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:34.639005 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3210/6499] 10.222.0.23 () {58 vars in 1342 bytes} [Sat Aug 20 21:21:34 2022] HEAD /identity/v3/groups/5e4067393801418e8de5d0a066c1d516/users/d0b5782ba70b433ba1ca3c2657e8b7b2 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:34.643359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d80fdc-3174-4fd8-8794-8bf6b5c3cfc7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:34.658149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d80fdc-3174-4fd8-8794-8bf6b5c3cfc7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:34.663209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2917096-e33a-4f07-8605-1660cb768948 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:34.663905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2917096-e33a-4f07-8605-1660cb768948 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:34.664181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2917096-e33a-4f07-8605-1660cb768948 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:34.664504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2917096-e33a-4f07-8605-1660cb768948 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/5e4067393801418e8de5d0a066c1d516/users/d0b5782ba70b433ba1ca3c2657e8b7b2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:34.666177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2917096-e33a-4f07-8605-1660cb768948 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=5e4067393801418e8de5d0a066c1d516, user_id=d0b5782ba70b433ba1ca3c2657e8b7b2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:34.668098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2917096-e33a-4f07-8605-1660cb768948 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:34.676433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f2917096-e33a-4f07-8605-1660cb768948 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'd0b5782ba70b433ba1ca3c2657e8b7b2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:34.692587 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3290/6500] 10.222.0.23 () {58 vars in 1344 bytes} [Sat Aug 20 21:21:34 2022] DELETE /identity/v3/groups/5e4067393801418e8de5d0a066c1d516/users/d0b5782ba70b433ba1ca3c2657e8b7b2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:34.697356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb537f0-748a-443c-98b5-b1fe2aabb19e tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:34.707744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb537f0-748a-443c-98b5-b1fe2aabb19e tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:34.766417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec7d6d-63de-404b-a465-89bab76b56e2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:34.767086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec7d6d-63de-404b-a465-89bab76b56e2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:34.767364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec7d6d-63de-404b-a465-89bab76b56e2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:34.767738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec7d6d-63de-404b-a465-89bab76b56e2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/5e4067393801418e8de5d0a066c1d516/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:34.769034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec7d6d-63de-404b-a465-89bab76b56e2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=5e4067393801418e8de5d0a066c1d516)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:34.770059 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-77ec7d6d-63de-404b-a465-89bab76b56e2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:21:34.770059 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:21:34.770059 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:21:34.771307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec7d6d-63de-404b-a465-89bab76b56e2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:34.781780 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3211/6501] 10.222.0.23 () {58 vars in 1242 bytes} [Sat Aug 20 21:21:34 2022] GET /identity/v3/groups/5e4067393801418e8de5d0a066c1d516/users => generated 145 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:34.786447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2917096-e33a-4f07-8605-1660cb768948 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:34.797469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2917096-e33a-4f07-8605-1660cb768948 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:34.803006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e5857-b764-403b-b6a5-fb3696a7353f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:34.803683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5857-b764-403b-b6a5-fb3696a7353f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:34.803980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5857-b764-403b-b6a5-fb3696a7353f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:34.804289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5857-b764-403b-b6a5-fb3696a7353f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/users/d0b5782ba70b433ba1ca3c2657e8b7b2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:34.805618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e5857-b764-403b-b6a5-fb3696a7353f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:delete_user(user_id=d0b5782ba70b433ba1ca3c2657e8b7b2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:34.807713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e5857-b764-403b-b6a5-fb3696a7353f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:34.857196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1c1e5857-b764-403b-b6a5-fb3696a7353f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0b5782ba70b433ba1ca3c2657e8b7b2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:34.866525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1c1e5857-b764-403b-b6a5-fb3696a7353f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0b5782ba70b433ba1ca3c2657e8b7b2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:34.871375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=79,keystone:INSERT=7,keystone:DELETE=14 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:34.880063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1c1e5857-b764-403b-b6a5-fb3696a7353f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0b5782ba70b433ba1ca3c2657e8b7b2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:34.892598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1c1e5857-b764-403b-b6a5-fb3696a7353f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0b5782ba70b433ba1ca3c2657e8b7b2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:34.893440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1c1e5857-b764-403b-b6a5-fb3696a7353f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0b5782ba70b433ba1ca3c2657e8b7b2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:34.901034 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3291/6502] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:34 2022] DELETE /identity/v3/users/d0b5782ba70b433ba1ca3c2657e8b7b2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:34.905103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec7d6d-63de-404b-a465-89bab76b56e2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:34.906810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-77ec7d6d-63de-404b-a465-89bab76b56e2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:34.920167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec7d6d-63de-404b-a465-89bab76b56e2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:34.967636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:34.992994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:34.993744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:34.994087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:34.994420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/users/ed9845a18ff649a781ccb74beeb2ff53` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:34.995648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:delete_user(user_id=ed9845a18ff649a781ccb74beeb2ff53)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:34.997949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:35.052455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed9845a18ff649a781ccb74beeb2ff53'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:35.068643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed9845a18ff649a781ccb74beeb2ff53'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:35.083579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed9845a18ff649a781ccb74beeb2ff53'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:35.084497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed9845a18ff649a781ccb74beeb2ff53'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:35.085106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed9845a18ff649a781ccb74beeb2ff53'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:35.104148 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3212/6503] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:34 2022] DELETE /identity/v3/users/ed9845a18ff649a781ccb74beeb2ff53 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:35.105399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e5857-b764-403b-b6a5-fb3696a7353f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:35.120207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e5857-b764-403b-b6a5-fb3696a7353f tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:35.181654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:35.182522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:35.182951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:35.183451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/users/504b9374c77a4b91aac712c39108a33e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:35.185476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:delete_user(user_id=504b9374c77a4b91aac712c39108a33e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:35.188899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:35.254147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '504b9374c77a4b91aac712c39108a33e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:35.280778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '504b9374c77a4b91aac712c39108a33e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:35.293619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '504b9374c77a4b91aac712c39108a33e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:35.316650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '504b9374c77a4b91aac712c39108a33e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:35.317532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '504b9374c77a4b91aac712c39108a33e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:35.332057 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3292/6504] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:35 2022] DELETE /identity/v3/users/504b9374c77a4b91aac712c39108a33e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:35.332253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:35.333002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:35.348416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003052f-f78f-4956-93de-11f97c91897c tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:35.416645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d387aac-fc38-4b5d-88ff-7b39b30dd0cd tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:35.417509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d387aac-fc38-4b5d-88ff-7b39b30dd0cd tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:35.417848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d387aac-fc38-4b5d-88ff-7b39b30dd0cd tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:35.418186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d387aac-fc38-4b5d-88ff-7b39b30dd0cd tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/5e4067393801418e8de5d0a066c1d516` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:35.419456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d387aac-fc38-4b5d-88ff-7b39b30dd0cd tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:delete_group(group_id=5e4067393801418e8de5d0a066c1d516)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:35.421480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d387aac-fc38-4b5d-88ff-7b39b30dd0cd tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:35.485404 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3213/6505] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:21:35 2022] DELETE /identity/v3/groups/5e4067393801418e8de5d0a066c1d516 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:35.493531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:35.495820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:35.519593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98ef69-5806-4979-826b-22c4ced8c785 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:35.605149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:35.614457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779fbd4-c46e-4c52-ba76-3b07880a65a8 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:35.615328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779fbd4-c46e-4c52-ba76-3b07880a65a8 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:35.615665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779fbd4-c46e-4c52-ba76-3b07880a65a8 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:35.616004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779fbd4-c46e-4c52-ba76-3b07880a65a8 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:35.616910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779fbd4-c46e-4c52-ba76-3b07880a65a8 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:35.619362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779fbd4-c46e-4c52-ba76-3b07880a65a8 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:35.644541 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3293/6506] 10.222.0.23 () {60 vars in 1147 bytes} [Sat Aug 20 21:21:35 2022] POST /identity/v3/groups => generated 326 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:35.646219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d387aac-fc38-4b5d-88ff-7b39b30dd0cd tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:35.660595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d387aac-fc38-4b5d-88ff-7b39b30dd0cd tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:35.665852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dbade-0696-4bf1-beff-70b9490bf0b3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:35.666583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dbade-0696-4bf1-beff-70b9490bf0b3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:35.666984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dbade-0696-4bf1-beff-70b9490bf0b3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:35.667353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dbade-0696-4bf1-beff-70b9490bf0b3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:35.668113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dbade-0696-4bf1-beff-70b9490bf0b3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:35.670294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dbade-0696-4bf1-beff-70b9490bf0b3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:35.708221 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3214/6507] 10.222.0.23 () {60 vars in 1147 bytes} [Sat Aug 20 21:21:35 2022] POST /identity/v3/groups => generated 326 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:35.716751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779fbd4-c46e-4c52-ba76-3b07880a65a8 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:35.728558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779fbd4-c46e-4c52-ba76-3b07880a65a8 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:35.734213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9449c-2383-4ab4-9eed-cbd34e214b45 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:35.735303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9449c-2383-4ab4-9eed-cbd34e214b45 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:35.735698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9449c-2383-4ab4-9eed-cbd34e214b45 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:35.736035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9449c-2383-4ab4-9eed-cbd34e214b45 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:35.736762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9449c-2383-4ab4-9eed-cbd34e214b45 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:35.738676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9449c-2383-4ab4-9eed-cbd34e214b45 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:35.759129 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3294/6508] 10.222.0.23 () {60 vars in 1147 bytes} [Sat Aug 20 21:21:35 2022] POST /identity/v3/groups => generated 327 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:35.763461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dbade-0696-4bf1-beff-70b9490bf0b3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:35.770875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:35.777152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dbade-0696-4bf1-beff-70b9490bf0b3 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:35.785814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54106bf2-6486-4584-acb3-5ba114def7a7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:35.786687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54106bf2-6486-4584-acb3-5ba114def7a7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:35.787454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54106bf2-6486-4584-acb3-5ba114def7a7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:35.787927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54106bf2-6486-4584-acb3-5ba114def7a7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:35.789005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54106bf2-6486-4584-acb3-5ba114def7a7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:list_groups()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:35.790596 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-54106bf2-6486-4584-acb3-5ba114def7a7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_groups": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:21:35.790596 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:21:35.790596 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:21:35.792795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54106bf2-6486-4584-acb3-5ba114def7a7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:35.804202 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3215/6509] 10.222.0.23 () {58 vars in 1125 bytes} [Sat Aug 20 21:21:35 2022] GET /identity/v3/groups => generated 1489 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:35.805742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9449c-2383-4ab4-9eed-cbd34e214b45 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:35.817986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9449c-2383-4ab4-9eed-cbd34e214b45 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:35.823901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e585b1-1748-4144-9ce8-4479deaf1093 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:35.824886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e585b1-1748-4144-9ce8-4479deaf1093 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:35.825625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e585b1-1748-4144-9ce8-4479deaf1093 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:35.825625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e585b1-1748-4144-9ce8-4479deaf1093 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/0419af16aa3b4d3e9ea611627409ae3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:35.831954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e585b1-1748-4144-9ce8-4479deaf1093 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:delete_group(group_id=0419af16aa3b4d3e9ea611627409ae3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:35.834156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e585b1-1748-4144-9ce8-4479deaf1093 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:35.894012 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3295/6510] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:21:35 2022] DELETE /identity/v3/groups/0419af16aa3b4d3e9ea611627409ae3a => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:35.900005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54106bf2-6486-4584-acb3-5ba114def7a7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:35.923236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54106bf2-6486-4584-acb3-5ba114def7a7 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:36.006438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50ff00-7ac9-46d3-9a3b-88bb1f4f2216 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:36.006911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50ff00-7ac9-46d3-9a3b-88bb1f4f2216 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:36.007237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50ff00-7ac9-46d3-9a3b-88bb1f4f2216 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:36.007659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50ff00-7ac9-46d3-9a3b-88bb1f4f2216 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/16ae9705b0a8442a8af2c9f2c48bba94` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:36.013424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50ff00-7ac9-46d3-9a3b-88bb1f4f2216 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:delete_group(group_id=16ae9705b0a8442a8af2c9f2c48bba94)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:36.015637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50ff00-7ac9-46d3-9a3b-88bb1f4f2216 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:36.083571 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3216/6511] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:21:35 2022] DELETE /identity/v3/groups/16ae9705b0a8442a8af2c9f2c48bba94 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:36.085453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e585b1-1748-4144-9ce8-4479deaf1093 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:36.097022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e585b1-1748-4144-9ce8-4479deaf1093 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:36.162923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a89aa-c455-467d-a703-31d1cd52e392 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:36.163609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a89aa-c455-467d-a703-31d1cd52e392 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:36.163887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a89aa-c455-467d-a703-31d1cd52e392 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:36.164159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a89aa-c455-467d-a703-31d1cd52e392 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/b477d5803cc4494f8128c7222457b198` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:36.169857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a89aa-c455-467d-a703-31d1cd52e392 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:delete_group(group_id=b477d5803cc4494f8128c7222457b198)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:36.172040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a89aa-c455-467d-a703-31d1cd52e392 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:36.219238 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3296/6512] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:21:36 2022] DELETE /identity/v3/groups/b477d5803cc4494f8128c7222457b198 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:36.224960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50ff00-7ac9-46d3-9a3b-88bb1f4f2216 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:36.236602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50ff00-7ac9-46d3-9a3b-88bb1f4f2216 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:36.300838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757fe89-0d6f-4926-a3ee-02dfb79ac858 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:36.301664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757fe89-0d6f-4926-a3ee-02dfb79ac858 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:36.301941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757fe89-0d6f-4926-a3ee-02dfb79ac858 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:36.302351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757fe89-0d6f-4926-a3ee-02dfb79ac858 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:36.303915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757fe89-0d6f-4926-a3ee-02dfb79ac858 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:36.305906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757fe89-0d6f-4926-a3ee-02dfb79ac858 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:36.336667 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3217/6513] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:36 2022] POST /identity/v3/users => generated 321 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:36.340738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a89aa-c455-467d-a703-31d1cd52e392 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:36.351979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a89aa-c455-467d-a703-31d1cd52e392 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:36.357659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ab88c-7d44-4a24-8339-2cbaca95619d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:36.358269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ab88c-7d44-4a24-8339-2cbaca95619d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:36.358540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ab88c-7d44-4a24-8339-2cbaca95619d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:36.358796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ab88c-7d44-4a24-8339-2cbaca95619d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:36.359607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ab88c-7d44-4a24-8339-2cbaca95619d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:36.361594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ab88c-7d44-4a24-8339-2cbaca95619d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:36.378871 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3297/6514] 10.222.0.23 () {60 vars in 1147 bytes} [Sat Aug 20 21:21:36 2022] POST /identity/v3/groups => generated 327 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:36.383924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757fe89-0d6f-4926-a3ee-02dfb79ac858 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:36.394665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757fe89-0d6f-4926-a3ee-02dfb79ac858 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:36.400143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c14f8-0471-4cc5-b53e-f942fe777039 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:36.400890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c14f8-0471-4cc5-b53e-f942fe777039 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:36.401109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c14f8-0471-4cc5-b53e-f942fe777039 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:36.401367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c14f8-0471-4cc5-b53e-f942fe777039 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/7d07f7201cdb4e309330b8d5adba2a8a/users/3a59356d58ec4fcf86286c386462a56d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:36.421182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c14f8-0471-4cc5-b53e-f942fe777039 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=7d07f7201cdb4e309330b8d5adba2a8a, user_id=3a59356d58ec4fcf86286c386462a56d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:36.423307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c14f8-0471-4cc5-b53e-f942fe777039 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:36.446889 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3218/6515] 10.222.0.23 () {60 vars in 1360 bytes} [Sat Aug 20 21:21:36 2022] PUT /identity/v3/groups/7d07f7201cdb4e309330b8d5adba2a8a/users/3a59356d58ec4fcf86286c386462a56d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:36.451059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ab88c-7d44-4a24-8339-2cbaca95619d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:36.462183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ab88c-7d44-4a24-8339-2cbaca95619d tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:36.484557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:36.531782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbed7d-8e69-41be-9111-9f8f602f3355 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:36.532434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbed7d-8e69-41be-9111-9f8f602f3355 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:36.532688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbed7d-8e69-41be-9111-9f8f602f3355 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:36.532953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbed7d-8e69-41be-9111-9f8f602f3355 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:36.533573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbed7d-8e69-41be-9111-9f8f602f3355 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:36.535482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbed7d-8e69-41be-9111-9f8f602f3355 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:36.553047 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3298/6516] 10.222.0.23 () {60 vars in 1147 bytes} [Sat Aug 20 21:21:36 2022] POST /identity/v3/groups => generated 325 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:36.557810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c14f8-0471-4cc5-b53e-f942fe777039 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:36.563406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:36.570242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c14f8-0471-4cc5-b53e-f942fe777039 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:36.576614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad5df2-a89d-4cdd-86d8-cc153fbd78d2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:36.577241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad5df2-a89d-4cdd-86d8-cc153fbd78d2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:36.577712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad5df2-a89d-4cdd-86d8-cc153fbd78d2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:36.578138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad5df2-a89d-4cdd-86d8-cc153fbd78d2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/bee720d6d5034744b8c55c732e50e283/users/3a59356d58ec4fcf86286c386462a56d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:36.584623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad5df2-a89d-4cdd-86d8-cc153fbd78d2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=bee720d6d5034744b8c55c732e50e283, user_id=3a59356d58ec4fcf86286c386462a56d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:36.586338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad5df2-a89d-4cdd-86d8-cc153fbd78d2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:36.626567 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3219/6517] 10.222.0.23 () {60 vars in 1360 bytes} [Sat Aug 20 21:21:36 2022] PUT /identity/v3/groups/bee720d6d5034744b8c55c732e50e283/users/3a59356d58ec4fcf86286c386462a56d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:36.632116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbed7d-8e69-41be-9111-9f8f602f3355 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:36.644230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbed7d-8e69-41be-9111-9f8f602f3355 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:36.719591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880acbb8-1cf4-48be-97d5-fcdc99ef8091 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:36.720509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880acbb8-1cf4-48be-97d5-fcdc99ef8091 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:36.720895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880acbb8-1cf4-48be-97d5-fcdc99ef8091 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:36.721434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880acbb8-1cf4-48be-97d5-fcdc99ef8091 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/users/3a59356d58ec4fcf86286c386462a56d/groups` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:36.722821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880acbb8-1cf4-48be-97d5-fcdc99ef8091 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:list_groups_for_user(user_id=3a59356d58ec4fcf86286c386462a56d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:36.726533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880acbb8-1cf4-48be-97d5-fcdc99ef8091 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:36.747586 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3299/6518] 10.222.0.23 () {58 vars in 1242 bytes} [Sat Aug 20 21:21:36 2022] GET /identity/v3/users/3a59356d58ec4fcf86286c386462a56d/groups => generated 838 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:36.752024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad5df2-a89d-4cdd-86d8-cc153fbd78d2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:36.765116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad5df2-a89d-4cdd-86d8-cc153fbd78d2 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:36.770604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd8754-5319-47a5-934c-92ca6f6feed9 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:36.771524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd8754-5319-47a5-934c-92ca6f6feed9 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:36.771806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd8754-5319-47a5-934c-92ca6f6feed9 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:36.772139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd8754-5319-47a5-934c-92ca6f6feed9 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/bee720d6d5034744b8c55c732e50e283` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:36.773476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd8754-5319-47a5-934c-92ca6f6feed9 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:delete_group(group_id=bee720d6d5034744b8c55c732e50e283)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:36.775577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd8754-5319-47a5-934c-92ca6f6feed9 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:36.837467 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3220/6519] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:21:36 2022] DELETE /identity/v3/groups/bee720d6d5034744b8c55c732e50e283 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:36.842042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880acbb8-1cf4-48be-97d5-fcdc99ef8091 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:36.853223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880acbb8-1cf4-48be-97d5-fcdc99ef8091 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:36.910070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6179818-c48b-4b99-b724-04137d5f8420 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:36.910702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6179818-c48b-4b99-b724-04137d5f8420 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:36.910978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6179818-c48b-4b99-b724-04137d5f8420 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:36.911251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6179818-c48b-4b99-b724-04137d5f8420 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/groups/7d07f7201cdb4e309330b8d5adba2a8a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:36.912499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6179818-c48b-4b99-b724-04137d5f8420 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:delete_group(group_id=7d07f7201cdb4e309330b8d5adba2a8a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:36.914641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6179818-c48b-4b99-b724-04137d5f8420 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:36.981380 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3300/6520] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:21:36 2022] DELETE /identity/v3/groups/7d07f7201cdb4e309330b8d5adba2a8a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:36.986021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd8754-5319-47a5-934c-92ca6f6feed9 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:36.999690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd8754-5319-47a5-934c-92ca6f6feed9 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:37.067959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7f94c-f3f2-40ab-b987-97ab44f6d7d4 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:37.069280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7f94c-f3f2-40ab-b987-97ab44f6d7d4 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:37.069619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7f94c-f3f2-40ab-b987-97ab44f6d7d4 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:37.069943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7f94c-f3f2-40ab-b987-97ab44f6d7d4 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/users/3a59356d58ec4fcf86286c386462a56d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:37.071239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7f94c-f3f2-40ab-b987-97ab44f6d7d4 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:delete_user(user_id=3a59356d58ec4fcf86286c386462a56d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:37.073318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7f94c-f3f2-40ab-b987-97ab44f6d7d4 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:37.137849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51c7f94c-f3f2-40ab-b987-97ab44f6d7d4 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a59356d58ec4fcf86286c386462a56d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.150062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51c7f94c-f3f2-40ab-b987-97ab44f6d7d4 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a59356d58ec4fcf86286c386462a56d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.167391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51c7f94c-f3f2-40ab-b987-97ab44f6d7d4 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a59356d58ec4fcf86286c386462a56d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.168025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51c7f94c-f3f2-40ab-b987-97ab44f6d7d4 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a59356d58ec4fcf86286c386462a56d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.168769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51c7f94c-f3f2-40ab-b987-97ab44f6d7d4 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a59356d58ec4fcf86286c386462a56d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.194844 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3221/6521] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:36 2022] DELETE /identity/v3/users/3a59356d58ec4fcf86286c386462a56d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:37.195028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6179818-c48b-4b99-b724-04137d5f8420 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:37.199668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f6179818-c48b-4b99-b724-04137d5f8420 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:37.214305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6179818-c48b-4b99-b724-04137d5f8420 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:37.218270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:37.291216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441389d1-ec22-4ba8-a563-ffb874b4b1d0 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:37.292160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441389d1-ec22-4ba8-a563-ffb874b4b1d0 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:37.292562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441389d1-ec22-4ba8-a563-ffb874b4b1d0 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:37.292909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441389d1-ec22-4ba8-a563-ffb874b4b1d0 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/domains/0bbc946a4be34ab18b2c5ac49e333ef2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:37.296768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441389d1-ec22-4ba8-a563-ffb874b4b1d0 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=0bbc946a4be34ab18b2c5ac49e333ef2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:37.298461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441389d1-ec22-4ba8-a563-ffb874b4b1d0 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:37.334971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-441389d1-ec22-4ba8-a563-ffb874b4b1d0 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '0bbc946a4be34ab18b2c5ac49e333ef2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.336554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-441389d1-ec22-4ba8-a563-ffb874b4b1d0 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '0bbc946a4be34ab18b2c5ac49e333ef2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.376700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-441389d1-ec22-4ba8-a563-ffb874b4b1d0 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '0bbc946a4be34ab18b2c5ac49e333ef2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.377802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-441389d1-ec22-4ba8-a563-ffb874b4b1d0 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '0bbc946a4be34ab18b2c5ac49e333ef2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.381022 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3301/6522] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:21:37 2022] PATCH /identity/v3/domains/0bbc946a4be34ab18b2c5ac49e333ef2 => generated 277 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:37.386226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7f94c-f3f2-40ab-b987-97ab44f6d7d4 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:37.394196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:37.399900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7f94c-f3f2-40ab-b987-97ab44f6d7d4 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:37.517741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64b40d6e43fd45209f05e5f78767ed56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f4516d917044c62af9d1ea8c2b9b6cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:37.518685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:37.519102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:37.519498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] PATH_INFO: `/v3/domains/0bbc946a4be34ab18b2c5ac49e333ef2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:37.532395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=0bbc946a4be34ab18b2c5ac49e333ef2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:37.534332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:37.561035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '0bbc946a4be34ab18b2c5ac49e333ef2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.565221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '0bbc946a4be34ab18b2c5ac49e333ef2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.631740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bbc946a4be34ab18b2c5ac49e333ef2', 'request_id': 'req-8519dfc1-6841-432e-9f7b-293a577c0193'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.640735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bbc946a4be34ab18b2c5ac49e333ef2', 'request_id': 'req-8519dfc1-6841-432e-9f7b-293a577c0193'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.651842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bbc946a4be34ab18b2c5ac49e333ef2', 'request_id': 'req-8519dfc1-6841-432e-9f7b-293a577c0193'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.665443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] The token cache is being invalidate because project 0bbc946a4be34ab18b2c5ac49e333ef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:37.665795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.666452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.697313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '0bbc946a4be34ab18b2c5ac49e333ef2', 'request_id': 'req-8519dfc1-6841-432e-9f7b-293a577c0193'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.698924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '0bbc946a4be34ab18b2c5ac49e333ef2', 'request_id': 'req-8519dfc1-6841-432e-9f7b-293a577c0193'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.705907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8519dfc1-6841-432e-9f7b-293a577c0193 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '0bbc946a4be34ab18b2c5ac49e333ef2', 'request_id': 'req-8519dfc1-6841-432e-9f7b-293a577c0193'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.717294 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3222/6523] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:37 2022] DELETE /identity/v3/domains/0bbc946a4be34ab18b2c5ac49e333ef2 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:37.718286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441389d1-ec22-4ba8-a563-ffb874b4b1d0 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:37.720741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-441389d1-ec22-4ba8-a563-ffb874b4b1d0 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:37.799019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441389d1-ec22-4ba8-a563-ffb874b4b1d0 tempest-GroupsV3TestJSON-824504241 tempest-GroupsV3TestJSON-824504241-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:37.869248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:37.869248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:37.869248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:37.869248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] PATH_INFO: `/v3/users/83ea757df4db405c9584eec28d4c9fd3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:37.871881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83ea757df4db405c9584eec28d4c9fd3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:37.874859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:37.921521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:DELETE=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:37.960755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83ea757df4db405c9584eec28d4c9fd3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.975576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83ea757df4db405c9584eec28d4c9fd3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:37.989237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83ea757df4db405c9584eec28d4c9fd3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:38.007488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83ea757df4db405c9584eec28d4c9fd3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:38.008633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83ea757df4db405c9584eec28d4c9fd3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:38.023975 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3302/6524] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:37 2022] DELETE /identity/v3/users/83ea757df4db405c9584eec28d4c9fd3 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:38.029221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3583d-159f-4139-b5ba-abb316cb49c4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:38.029718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3583d-159f-4139-b5ba-abb316cb49c4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:38.029807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3583d-159f-4139-b5ba-abb316cb49c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:38.056918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-66b3583d-159f-4139-b5ba-abb316cb49c4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:38.058407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-66b3583d-159f-4139-b5ba-abb316cb49c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:38.122053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:38.150406 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3223/6525] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:38.162674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:38.162674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:38.228165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711471d-f3d5-45eb-a090-7ff70e8815e4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:38.231896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f4b6a-79a3-425d-8c73-813ba30340ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:38.232641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f4b6a-79a3-425d-8c73-813ba30340ed service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:38.232802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f4b6a-79a3-425d-8c73-813ba30340ed service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:38.233290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f4b6a-79a3-425d-8c73-813ba30340ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:38.243922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f4b6a-79a3-425d-8c73-813ba30340ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:38.245507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f4b6a-79a3-425d-8c73-813ba30340ed service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:38.257237 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3303/6526] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:38.655475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3583d-159f-4139-b5ba-abb316cb49c4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:38.659225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-66b3583d-159f-4139-b5ba-abb316cb49c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:38.673056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3583d-159f-4139-b5ba-abb316cb49c4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:38.681308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:38.683264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:38.683608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:38.683889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] PATH_INFO: `/v3/projects/0de65fd605b54265b7dfd053748c654d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:38.685364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de65fd605b54265b7dfd053748c654d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:38.687831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:38.768094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de65fd605b54265b7dfd053748c654d', 'request_id': 'req-71f75231-173e-43a0-9bd8-61eef9e42178'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:38.774752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de65fd605b54265b7dfd053748c654d', 'request_id': 'req-71f75231-173e-43a0-9bd8-61eef9e42178'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:38.785258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de65fd605b54265b7dfd053748c654d', 'request_id': 'req-71f75231-173e-43a0-9bd8-61eef9e42178'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:38.813799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] The token cache is being invalidate because project 0de65fd605b54265b7dfd053748c654d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:38.814085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:38.816455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:38.818886 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3224/6527] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:38 2022] DELETE /identity/v3/projects/0de65fd605b54265b7dfd053748c654d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:38.822928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f4b6a-79a3-425d-8c73-813ba30340ed service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:38.833241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f4b6a-79a3-425d-8c73-813ba30340ed service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:38.904862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:38.905655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:38.905928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:38.906200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] PATH_INFO: `/v3/users/64b40d6e43fd45209f05e5f78767ed56` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:38.907494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b40d6e43fd45209f05e5f78767ed56)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:38.911234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:38.976020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:38.981378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b40d6e43fd45209f05e5f78767ed56'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:38.994785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b40d6e43fd45209f05e5f78767ed56'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.007745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b40d6e43fd45209f05e5f78767ed56'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.034447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b40d6e43fd45209f05e5f78767ed56'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.035608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b40d6e43fd45209f05e5f78767ed56'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.051323 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3304/6528] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:38 2022] DELETE /identity/v3/users/64b40d6e43fd45209f05e5f78767ed56 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:39.400908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:39.403749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:39.426271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f75231-173e-43a0-9bd8-61eef9e42178 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:39.454689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:39.456546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:39.512686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd544c-11c9-4d35-b9e1-19510a19746f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:39.514263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd544c-11c9-4d35-b9e1-19510a19746f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:39.514631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd544c-11c9-4d35-b9e1-19510a19746f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:39.515066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd544c-11c9-4d35-b9e1-19510a19746f admin admin] PATH_INFO: `/v3/users/7013eec1908f407d9949d271287b1570` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:39.516609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd544c-11c9-4d35-b9e1-19510a19746f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7013eec1908f407d9949d271287b1570)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:39.519338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd544c-11c9-4d35-b9e1-19510a19746f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:39.610728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d343b78-bcff-4ed9-97cd-5f536350492b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:39.619577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-68cd544c-11c9-4d35-b9e1-19510a19746f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7013eec1908f407d9949d271287b1570'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.631668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184afedd-092a-4757-9acc-953a238a1677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:39.632825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184afedd-092a-4757-9acc-953a238a1677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:39.633245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184afedd-092a-4757-9acc-953a238a1677 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:39.633401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-68cd544c-11c9-4d35-b9e1-19510a19746f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7013eec1908f407d9949d271287b1570'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.633830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184afedd-092a-4757-9acc-953a238a1677 admin admin] PATH_INFO: `/v3/projects/0f4516d917044c62af9d1ea8c2b9b6cb` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:39.635316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184afedd-092a-4757-9acc-953a238a1677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f4516d917044c62af9d1ea8c2b9b6cb)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:39.637957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184afedd-092a-4757-9acc-953a238a1677 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:39.647857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:39.650390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-68cd544c-11c9-4d35-b9e1-19510a19746f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7013eec1908f407d9949d271287b1570'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.651335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-68cd544c-11c9-4d35-b9e1-19510a19746f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7013eec1908f407d9949d271287b1570'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.652294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-68cd544c-11c9-4d35-b9e1-19510a19746f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7013eec1908f407d9949d271287b1570'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.676269 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3225/6529] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:39 2022] DELETE /identity/v3/users/7013eec1908f407d9949d271287b1570 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:39.707964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-184afedd-092a-4757-9acc-953a238a1677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f4516d917044c62af9d1ea8c2b9b6cb', 'request_id': 'req-184afedd-092a-4757-9acc-953a238a1677'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.719678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-184afedd-092a-4757-9acc-953a238a1677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f4516d917044c62af9d1ea8c2b9b6cb', 'request_id': 'req-184afedd-092a-4757-9acc-953a238a1677'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.726987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-184afedd-092a-4757-9acc-953a238a1677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f4516d917044c62af9d1ea8c2b9b6cb', 'request_id': 'req-184afedd-092a-4757-9acc-953a238a1677'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.741206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-184afedd-092a-4757-9acc-953a238a1677 admin admin] The token cache is being invalidate because project 0f4516d917044c62af9d1ea8c2b9b6cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:39.741771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-184afedd-092a-4757-9acc-953a238a1677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.742823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-184afedd-092a-4757-9acc-953a238a1677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:39.744544 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3305/6530] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:39 2022] DELETE /identity/v3/projects/0f4516d917044c62af9d1ea8c2b9b6cb => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:39.750655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbf365-681d-44f6-9971-79698e1f03db None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:39.750937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbf365-681d-44f6-9971-79698e1f03db None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:39.751208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbf365-681d-44f6-9971-79698e1f03db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:39.773955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-23dbf365-681d-44f6-9971-79698e1f03db None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:39.775184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-23dbf365-681d-44f6-9971-79698e1f03db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:39.882284 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3226/6531] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:39.890092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0720a-a5c9-4923-b7e3-7275346e1f01 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:39.890653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0720a-a5c9-4923-b7e3-7275346e1f01 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:39.890876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0720a-a5c9-4923-b7e3-7275346e1f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:39.913201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fec0720a-a5c9-4923-b7e3-7275346e1f01 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:39.914518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fec0720a-a5c9-4923-b7e3-7275346e1f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:39.929435 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3306/6532] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:39.933945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbf365-681d-44f6-9971-79698e1f03db None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:39.946253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbf365-681d-44f6-9971-79698e1f03db None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:39.956860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebb629-d509-4438-8568-0656e73c7d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:39.958968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebb629-d509-4438-8568-0656e73c7d33 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:39.959476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebb629-d509-4438-8568-0656e73c7d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:39.959929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebb629-d509-4438-8568-0656e73c7d33 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:39.961743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebb629-d509-4438-8568-0656e73c7d33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:39.967733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebb629-d509-4438-8568-0656e73c7d33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:39.969693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebb629-d509-4438-8568-0656e73c7d33 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:39.990956 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3227/6533] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:39.995197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0720a-a5c9-4923-b7e3-7275346e1f01 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.010501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebb629-d509-4438-8568-0656e73c7d33 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.012704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d0ebb629-d509-4438-8568-0656e73c7d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:40.013871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0720a-a5c9-4923-b7e3-7275346e1f01 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.018705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c251c-de68-42fa-9d47-595a9b201681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.019326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c251c-de68-42fa-9d47-595a9b201681 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.020942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c251c-de68-42fa-9d47-595a9b201681 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.020942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c251c-de68-42fa-9d47-595a9b201681 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.020942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c251c-de68-42fa-9d47-595a9b201681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.023513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c251c-de68-42fa-9d47-595a9b201681 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.048701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebb629-d509-4438-8568-0656e73c7d33 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.056340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a9a60-5feb-4cd8-a616-6751d534a84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.057152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a9a60-5feb-4cd8-a616-6751d534a84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.057422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a9a60-5feb-4cd8-a616-6751d534a84f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.057686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a9a60-5feb-4cd8-a616-6751d534a84f admin admin] PATH_INFO: `/v3/projects/aa2842b604df4797b6f5cfbdfb03f173` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.058855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a9a60-5feb-4cd8-a616-6751d534a84f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa2842b604df4797b6f5cfbdfb03f173)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.061065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a9a60-5feb-4cd8-a616-6751d534a84f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.068809 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3307/6534] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:39 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:40.072927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c251c-de68-42fa-9d47-595a9b201681 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.101012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:40.124120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2b7a9a60-5feb-4cd8-a616-6751d534a84f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa2842b604df4797b6f5cfbdfb03f173', 'request_id': 'req-2b7a9a60-5feb-4cd8-a616-6751d534a84f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:40.133566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2b7a9a60-5feb-4cd8-a616-6751d534a84f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa2842b604df4797b6f5cfbdfb03f173', 'request_id': 'req-2b7a9a60-5feb-4cd8-a616-6751d534a84f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:40.143119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2b7a9a60-5feb-4cd8-a616-6751d534a84f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa2842b604df4797b6f5cfbdfb03f173', 'request_id': 'req-2b7a9a60-5feb-4cd8-a616-6751d534a84f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:40.152002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c251c-de68-42fa-9d47-595a9b201681 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.156191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2b7a9a60-5feb-4cd8-a616-6751d534a84f admin admin] The token cache is being invalidate because project aa2842b604df4797b6f5cfbdfb03f173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:40.156585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2b7a9a60-5feb-4cd8-a616-6751d534a84f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:40.157709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2b7a9a60-5feb-4cd8-a616-6751d534a84f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:40.159787 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3228/6535] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:40 2022] DELETE /identity/v3/projects/aa2842b604df4797b6f5cfbdfb03f173 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:40.164524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209676c7-0583-4a4d-84ff-ec711d1b6109 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.165035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209676c7-0583-4a4d-84ff-ec711d1b6109 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.165177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209676c7-0583-4a4d-84ff-ec711d1b6109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.192084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-209676c7-0583-4a4d-84ff-ec711d1b6109 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:40.194871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-209676c7-0583-4a4d-84ff-ec711d1b6109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:40.221227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e5ac5-0e91-46a9-90e2-c38fa6bd2c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.221857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e5ac5-0e91-46a9-90e2-c38fa6bd2c19 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.222128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e5ac5-0e91-46a9-90e2-c38fa6bd2c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.222393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e5ac5-0e91-46a9-90e2-c38fa6bd2c19 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.226698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e5ac5-0e91-46a9-90e2-c38fa6bd2c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.228970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e5ac5-0e91-46a9-90e2-c38fa6bd2c19 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.258736 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3308/6536] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:40 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:40.262478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e5ac5-0e91-46a9-90e2-c38fa6bd2c19 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.265204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-717e5ac5-0e91-46a9-90e2-c38fa6bd2c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:40.267769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:40.278992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e5ac5-0e91-46a9-90e2-c38fa6bd2c19 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.284488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5e28f-d952-43a5-8ec5-1cf4548f85e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.284642 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3229/6537] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:40.285487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5e28f-d952-43a5-8ec5-1cf4548f85e8 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.285855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5e28f-d952-43a5-8ec5-1cf4548f85e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.286219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5e28f-d952-43a5-8ec5-1cf4548f85e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.287039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5e28f-d952-43a5-8ec5-1cf4548f85e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.288850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5e28f-d952-43a5-8ec5-1cf4548f85e8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.293786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbfd8c-255e-472a-8bab-319c6dcc6108 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.294081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbfd8c-255e-472a-8bab-319c6dcc6108 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.294345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbfd8c-255e-472a-8bab-319c6dcc6108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.304076 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3309/6538] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:40.308775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5e28f-d952-43a5-8ec5-1cf4548f85e8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.319967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5e28f-d952-43a5-8ec5-1cf4548f85e8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.320642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c1bbfd8c-255e-472a-8bab-319c6dcc6108 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:40.321891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c1bbfd8c-255e-472a-8bab-319c6dcc6108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:40.325106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f8866-a97b-4e94-af07-9f09fafd9640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.325760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f8866-a97b-4e94-af07-9f09fafd9640 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.326130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f8866-a97b-4e94-af07-9f09fafd9640 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.326496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f8866-a97b-4e94-af07-9f09fafd9640 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.327199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f8866-a97b-4e94-af07-9f09fafd9640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.328945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f8866-a97b-4e94-af07-9f09fafd9640 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.330413 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3230/6539] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:40.334571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbfd8c-255e-472a-8bab-319c6dcc6108 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.346396 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3310/6540] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:40.346587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbfd8c-255e-472a-8bab-319c6dcc6108 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.347650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f8866-a97b-4e94-af07-9f09fafd9640 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.352215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426df8f-bf6e-435f-9b40-740069c03808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.353074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426df8f-bf6e-435f-9b40-740069c03808 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.353974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426df8f-bf6e-435f-9b40-740069c03808 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.353974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426df8f-bf6e-435f-9b40-740069c03808 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.354654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426df8f-bf6e-435f-9b40-740069c03808 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:40.355072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426df8f-bf6e-435f-9b40-740069c03808 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.356949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426df8f-bf6e-435f-9b40-740069c03808 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.361416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f8866-a97b-4e94-af07-9f09fafd9640 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.369337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005bab5-92f3-4676-8e47-9bf7bf3221ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.370018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005bab5-92f3-4676-8e47-9bf7bf3221ed admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.370287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005bab5-92f3-4676-8e47-9bf7bf3221ed admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.370552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005bab5-92f3-4676-8e47-9bf7bf3221ed admin admin] PATH_INFO: `/v3/projects/88e55c8bc2094b6b8e9381be96881cc1/users/b96a9c7e564b43dc8b0e0f59923cc51c/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.373918 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3231/6541] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:40.374765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426df8f-bf6e-435f-9b40-740069c03808 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.385255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426df8f-bf6e-435f-9b40-740069c03808 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.387487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005bab5-92f3-4676-8e47-9bf7bf3221ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e55c8bc2094b6b8e9381be96881cc1, user_id=b96a9c7e564b43dc8b0e0f59923cc51c, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.390401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005bab5-92f3-4676-8e47-9bf7bf3221ed admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.391880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2fd95-9616-47f7-b665-4d36bdeca27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.392628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2fd95-9616-47f7-b665-4d36bdeca27a admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.393000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2fd95-9616-47f7-b665-4d36bdeca27a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.393343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2fd95-9616-47f7-b665-4d36bdeca27a admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.394077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2fd95-9616-47f7-b665-4d36bdeca27a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.396231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2fd95-9616-47f7-b665-4d36bdeca27a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.415604 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3311/6542] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:40 2022] PUT /identity/v3/projects/88e55c8bc2094b6b8e9381be96881cc1/users/b96a9c7e564b43dc8b0e0f59923cc51c/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:40.419543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb784be-7659-4be4-bfd4-1bfe3850bf1c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.420518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb784be-7659-4be4-bfd4-1bfe3850bf1c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.420518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb784be-7659-4be4-bfd4-1bfe3850bf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.433628 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3232/6543] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:40 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:40.434763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2fd95-9616-47f7-b665-4d36bdeca27a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.454637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cfb784be-7659-4be4-bfd4-1bfe3850bf1c None None] MFA Rules not processed for user `b96a9c7e564b43dc8b0e0f59923cc51c`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:40.456285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cfb784be-7659-4be4-bfd4-1bfe3850bf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:40.511622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2fd95-9616-47f7-b665-4d36bdeca27a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.516330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804d5af-7235-4805-9a33-26fd7fc4c012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.516966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804d5af-7235-4805-9a33-26fd7fc4c012 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.517245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804d5af-7235-4805-9a33-26fd7fc4c012 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.517525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804d5af-7235-4805-9a33-26fd7fc4c012 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.518156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804d5af-7235-4805-9a33-26fd7fc4c012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.520092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804d5af-7235-4805-9a33-26fd7fc4c012 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.530587 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3312/6544] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:40.535464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb784be-7659-4be4-bfd4-1bfe3850bf1c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.541381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:40.548859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb784be-7659-4be4-bfd4-1bfe3850bf1c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.555113 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3233/6545] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:40 2022] POST /identity/v3/users => generated 283 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:40.556105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804d5af-7235-4805-9a33-26fd7fc4c012 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.556987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e180b-ca6b-43a5-a95e-33b19a0227c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.557608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e180b-ca6b-43a5-a95e-33b19a0227c0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.558120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e180b-ca6b-43a5-a95e-33b19a0227c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.558454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e180b-ca6b-43a5-a95e-33b19a0227c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.559173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e180b-ca6b-43a5-a95e-33b19a0227c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.561334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e180b-ca6b-43a5-a95e-33b19a0227c0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.567462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804d5af-7235-4805-9a33-26fd7fc4c012 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.573174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdec94-6e52-4376-809e-8a6bf3315522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.573570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdec94-6e52-4376-809e-8a6bf3315522 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.573841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdec94-6e52-4376-809e-8a6bf3315522 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.574114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdec94-6e52-4376-809e-8a6bf3315522 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.574747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdec94-6e52-4376-809e-8a6bf3315522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.576338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdec94-6e52-4376-809e-8a6bf3315522 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.588588 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3234/6546] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:40.592618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdec94-6e52-4376-809e-8a6bf3315522 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.599871 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3313/6547] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:40 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:40.601542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e180b-ca6b-43a5-a95e-33b19a0227c0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.605916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdec94-6e52-4376-809e-8a6bf3315522 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.676295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd3eaf-f4ab-468d-9bd4-06eacb88eeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.676750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e180b-ca6b-43a5-a95e-33b19a0227c0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.676935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd3eaf-f4ab-468d-9bd4-06eacb88eeae admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.677209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd3eaf-f4ab-468d-9bd4-06eacb88eeae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.677589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd3eaf-f4ab-468d-9bd4-06eacb88eeae admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.678473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd3eaf-f4ab-468d-9bd4-06eacb88eeae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.680213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd3eaf-f4ab-468d-9bd4-06eacb88eeae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.683380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30eda7-5abd-4d23-be53-6f8ee2c5a32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.684282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30eda7-5abd-4d23-be53-6f8ee2c5a32c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.684652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30eda7-5abd-4d23-be53-6f8ee2c5a32c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.685138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30eda7-5abd-4d23-be53-6f8ee2c5a32c admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.685961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30eda7-5abd-4d23-be53-6f8ee2c5a32c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.688014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30eda7-5abd-4d23-be53-6f8ee2c5a32c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.694120 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3235/6548] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:40.695732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd3eaf-f4ab-468d-9bd4-06eacb88eeae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.708809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd3eaf-f4ab-468d-9bd4-06eacb88eeae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.713907 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3314/6549] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:40 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:40.717168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db62ee-61a5-4843-8a15-bd3feefce8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.717852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db62ee-61a5-4843-8a15-bd3feefce8fb admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.718858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30eda7-5abd-4d23-be53-6f8ee2c5a32c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.719445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db62ee-61a5-4843-8a15-bd3feefce8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.719723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db62ee-61a5-4843-8a15-bd3feefce8fb admin admin] PATH_INFO: `/v3/projects/963b7fc23b044b378008315043ea307a/users/3658cbe4b8ff4d5094fb9dfa0f2cbdd2/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.730452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30eda7-5abd-4d23-be53-6f8ee2c5a32c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.736140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a0ebd-f03f-42ce-abd6-dfc8476f6f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.736969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a0ebd-f03f-42ce-abd6-dfc8476f6f35 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.737334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a0ebd-f03f-42ce-abd6-dfc8476f6f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.737693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a0ebd-f03f-42ce-abd6-dfc8476f6f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.738447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a0ebd-f03f-42ce-abd6-dfc8476f6f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.738886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db62ee-61a5-4843-8a15-bd3feefce8fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=963b7fc23b044b378008315043ea307a, user_id=3658cbe4b8ff4d5094fb9dfa0f2cbdd2, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.740352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a0ebd-f03f-42ce-abd6-dfc8476f6f35 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.741668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db62ee-61a5-4843-8a15-bd3feefce8fb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.753608 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3315/6550] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:40.758003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a0ebd-f03f-42ce-abd6-dfc8476f6f35 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.765066 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3236/6551] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:40 2022] PUT /identity/v3/projects/963b7fc23b044b378008315043ea307a/users/3658cbe4b8ff4d5094fb9dfa0f2cbdd2/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:40.768534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a0ebd-f03f-42ce-abd6-dfc8476f6f35 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.774117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d379e65-c964-408b-9209-1028f96943e3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.774398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d379e65-c964-408b-9209-1028f96943e3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.774671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d379e65-c964-408b-9209-1028f96943e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.784156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:40.811207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5d379e65-c964-408b-9209-1028f96943e3 None None] MFA Rules not processed for user `3658cbe4b8ff4d5094fb9dfa0f2cbdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:40.812802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5d379e65-c964-408b-9209-1028f96943e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:40.838358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a59989-165e-442d-946f-008a79ae544b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:40.839358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a59989-165e-442d-946f-008a79ae544b admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:40.839481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a59989-165e-442d-946f-008a79ae544b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:40.839762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a59989-165e-442d-946f-008a79ae544b admin admin] PATH_INFO: `/v3/projects/d670bba78f784854b63a071cfd797410/users/da1146d638ab42cdadcba16d5365ef9f/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:40.857339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a59989-165e-442d-946f-008a79ae544b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d670bba78f784854b63a071cfd797410, user_id=da1146d638ab42cdadcba16d5365ef9f, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:40.860733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a59989-165e-442d-946f-008a79ae544b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:40.882144 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3316/6552] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:40 2022] PUT /identity/v3/projects/d670bba78f784854b63a071cfd797410/users/da1146d638ab42cdadcba16d5365ef9f/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:40.883313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a59989-165e-442d-946f-008a79ae544b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.899815 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3237/6553] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:40.912232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d379e65-c964-408b-9209-1028f96943e3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:40.914993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5d379e65-c964-408b-9209-1028f96943e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:40.969744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:40.993460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d379e65-c964-408b-9209-1028f96943e3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:40.993679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a59989-165e-442d-946f-008a79ae544b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:41.000741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb8011-6c83-41cb-88ef-6144b0f4a9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:41.001468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb8011-6c83-41cb-88ef-6144b0f4a9bc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:41.001896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb8011-6c83-41cb-88ef-6144b0f4a9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:41.002177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb8011-6c83-41cb-88ef-6144b0f4a9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:41.003353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb8011-6c83-41cb-88ef-6144b0f4a9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:41.005519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb8011-6c83-41cb-88ef-6144b0f4a9bc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:41.019518 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3317/6554] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:41.024610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb8011-6c83-41cb-88ef-6144b0f4a9bc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:41.039902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb8011-6c83-41cb-88ef-6144b0f4a9bc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:41.045238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580a9ff-c522-4d50-a83f-00935df7fedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:41.046072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580a9ff-c522-4d50-a83f-00935df7fedd admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:41.046432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580a9ff-c522-4d50-a83f-00935df7fedd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:41.046790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580a9ff-c522-4d50-a83f-00935df7fedd admin admin] PATH_INFO: `/v3/projects/d670bba78f784854b63a071cfd797410/users/da1146d638ab42cdadcba16d5365ef9f/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:41.048516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7cf89-7487-4598-b386-60281ce5a8a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:41.049065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580a9ff-c522-4d50-a83f-00935df7fedd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d670bba78f784854b63a071cfd797410, user_id=da1146d638ab42cdadcba16d5365ef9f, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:41.049302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7cf89-7487-4598-b386-60281ce5a8a8 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:41.049579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7cf89-7487-4598-b386-60281ce5a8a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:41.050405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7cf89-7487-4598-b386-60281ce5a8a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:41.052021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580a9ff-c522-4d50-a83f-00935df7fedd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:41.072287 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3318/6555] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:41 2022] PUT /identity/v3/projects/d670bba78f784854b63a071cfd797410/users/da1146d638ab42cdadcba16d5365ef9f/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:41.084058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ef9bd-1f0e-43f7-89e3-f847d6ae3a80 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:41.084389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ef9bd-1f0e-43f7-89e3-f847d6ae3a80 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:41.084730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ef9bd-1f0e-43f7-89e3-f847d6ae3a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:41.128129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e26ef9bd-1f0e-43f7-89e3-f847d6ae3a80 None None] MFA Rules not processed for user `da1146d638ab42cdadcba16d5365ef9f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:41.129668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e26ef9bd-1f0e-43f7-89e3-f847d6ae3a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:41.151850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7cf89-7487-4598-b386-60281ce5a8a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:41.153779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7cf89-7487-4598-b386-60281ce5a8a8 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:41.172079 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3238/6556] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:41.236060 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3319/6557] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:41.241142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7cf89-7487-4598-b386-60281ce5a8a8 service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:41.247700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:41.365910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7cf89-7487-4598-b386-60281ce5a8a8 service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:41.372157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e4d37-705d-4e82-b0d7-caaecefbb9c1 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:41.372832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e4d37-705d-4e82-b0d7-caaecefbb9c1 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:41.373093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e4d37-705d-4e82-b0d7-caaecefbb9c1 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:41.373360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e4d37-705d-4e82-b0d7-caaecefbb9c1 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:41.374022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e4d37-705d-4e82-b0d7-caaecefbb9c1 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:41.376471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e4d37-705d-4e82-b0d7-caaecefbb9c1 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:41.414873 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3239/6558] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:21:41 2022] POST /identity/v3/domains => generated 276 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:41.416069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ef9bd-1f0e-43f7-89e3-f847d6ae3a80 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:41.427220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ef9bd-1f0e-43f7-89e3-f847d6ae3a80 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:41.501677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4d134-32cb-4940-8fc0-5a4f562feea4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:41.502304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4d134-32cb-4940-8fc0-5a4f562feea4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:41.502967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4d134-32cb-4940-8fc0-5a4f562feea4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:41.502967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4d134-32cb-4940-8fc0-5a4f562feea4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:41.504000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4d134-32cb-4940-8fc0-5a4f562feea4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:41.506579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4d134-32cb-4940-8fc0-5a4f562feea4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:41.558566 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3320/6559] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:41 2022] POST /identity/v3/projects => generated 403 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:41.559770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e4d37-705d-4e82-b0d7-caaecefbb9c1 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:41.579923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e4d37-705d-4e82-b0d7-caaecefbb9c1 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:41.692661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec591a67-2f59-457a-9147-7ef08fdb39b8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:41.693852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec591a67-2f59-457a-9147-7ef08fdb39b8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:41.694029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec591a67-2f59-457a-9147-7ef08fdb39b8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:41.694305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec591a67-2f59-457a-9147-7ef08fdb39b8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/groups` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:41.694977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec591a67-2f59-457a-9147-7ef08fdb39b8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:41.696981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec591a67-2f59-457a-9147-7ef08fdb39b8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:41.714907 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3240/6560] 10.222.0.23 () {60 vars in 1147 bytes} [Sat Aug 20 21:21:41 2022] POST /identity/v3/groups => generated 301 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:41.719090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4d134-32cb-4940-8fc0-5a4f562feea4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:41.730579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4d134-32cb-4940-8fc0-5a4f562feea4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:41.736216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e7ea3-c91c-4d6f-b0d9-afc252ed1dce tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:41.737305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e7ea3-c91c-4d6f-b0d9-afc252ed1dce tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:41.737416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e7ea3-c91c-4d6f-b0d9-afc252ed1dce tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:41.737641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e7ea3-c91c-4d6f-b0d9-afc252ed1dce tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:41.738314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e7ea3-c91c-4d6f-b0d9-afc252ed1dce tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:41.740526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e7ea3-c91c-4d6f-b0d9-afc252ed1dce tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:41.781206 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3321/6561] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:41 2022] POST /identity/v3/users => generated 442 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:41.782693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec591a67-2f59-457a-9147-7ef08fdb39b8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:41.796113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec591a67-2f59-457a-9147-7ef08fdb39b8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:41.802372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8da8f1-11d4-4cf6-93b8-d3fd1543bbea tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:41.803258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8da8f1-11d4-4cf6-93b8-d3fd1543bbea tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:41.803923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8da8f1-11d4-4cf6-93b8-d3fd1543bbea tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:41.804422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8da8f1-11d4-4cf6-93b8-d3fd1543bbea tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:41.805328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8da8f1-11d4-4cf6-93b8-d3fd1543bbea tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:41.807314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8da8f1-11d4-4cf6-93b8-d3fd1543bbea tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:41.829649 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3241/6562] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:21:41 2022] POST /identity/v3/roles => generated 240 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:41.831033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e7ea3-c91c-4d6f-b0d9-afc252ed1dce tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:41.844981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e7ea3-c91c-4d6f-b0d9-afc252ed1dce tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:41.850817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6756f-c54e-4bc5-a839-40b5c873908d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:41.851702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6756f-c54e-4bc5-a839-40b5c873908d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:41.852424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6756f-c54e-4bc5-a839-40b5c873908d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:41.852424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6756f-c54e-4bc5-a839-40b5c873908d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/groups/13f06b3cce4945b1987c5b942b37a61a/roles/5951e8c3837f499b9c56712c0b1b6787/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:41.856866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:41.860641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6756f-c54e-4bc5-a839-40b5c873908d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_grant(project_id=ae1fae1af07f4195a6c2a570f00eeb66, group_id=13f06b3cce4945b1987c5b942b37a61a, role_id=5951e8c3837f499b9c56712c0b1b6787)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:41.863720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6756f-c54e-4bc5-a839-40b5c873908d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:41.878644 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3322/6563] 10.222.0.23 () {60 vars in 1585 bytes} [Sat Aug 20 21:21:41 2022] PUT /identity/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/groups/13f06b3cce4945b1987c5b942b37a61a/roles/5951e8c3837f499b9c56712c0b1b6787/inherited_to_projects => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:41.883030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8da8f1-11d4-4cf6-93b8-d3fd1543bbea tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:41.893092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8da8f1-11d4-4cf6-93b8-d3fd1543bbea tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:41.959480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c3409-bb9e-4f2b-b230-11a10090f0cc tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:41.960660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c3409-bb9e-4f2b-b230-11a10090f0cc tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:41.961109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c3409-bb9e-4f2b-b230-11a10090f0cc tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:41.961490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c3409-bb9e-4f2b-b230-11a10090f0cc tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/groups/13f06b3cce4945b1987c5b942b37a61a/roles/5951e8c3837f499b9c56712c0b1b6787/inherited_to_projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:41.964183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c3409-bb9e-4f2b-b230-11a10090f0cc tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:check_grant(project_id=ae1fae1af07f4195a6c2a570f00eeb66, group_id=13f06b3cce4945b1987c5b942b37a61a, role_id=5951e8c3837f499b9c56712c0b1b6787)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:41.967030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c3409-bb9e-4f2b-b230-11a10090f0cc tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:41.976013 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3242/6564] 10.222.0.23 () {58 vars in 1567 bytes} [Sat Aug 20 21:21:41 2022] HEAD /identity/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/groups/13f06b3cce4945b1987c5b942b37a61a/roles/5951e8c3837f499b9c56712c0b1b6787/inherited_to_projects => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:41.980290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6756f-c54e-4bc5-a839-40b5c873908d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:41.995883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6756f-c54e-4bc5-a839-40b5c873908d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:42.003745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609ef0c-68c9-492c-993b-2af3aac6cb9e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:42.004583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609ef0c-68c9-492c-993b-2af3aac6cb9e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:42.004917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609ef0c-68c9-492c-993b-2af3aac6cb9e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:42.005257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609ef0c-68c9-492c-993b-2af3aac6cb9e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/groups/13f06b3cce4945b1987c5b942b37a61a/roles/5951e8c3837f499b9c56712c0b1b6787/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:42.008009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609ef0c-68c9-492c-993b-2af3aac6cb9e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=ae1fae1af07f4195a6c2a570f00eeb66, group_id=13f06b3cce4945b1987c5b942b37a61a, role_id=5951e8c3837f499b9c56712c0b1b6787)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:42.010742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609ef0c-68c9-492c-993b-2af3aac6cb9e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:42.017934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e609ef0c-68c9-492c-993b-2af3aac6cb9e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invalidating the token cache because role 5951e8c3837f499b9c56712c0b1b6787 was removed from group 13f06b3cce4945b1987c5b942b37a61a on project ae1fae1af07f4195a6c2a570f00eeb66. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:42.018350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e609ef0c-68c9-492c-993b-2af3aac6cb9e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:42.019037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e609ef0c-68c9-492c-993b-2af3aac6cb9e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:42.034127 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3323/6565] 10.222.0.23 () {58 vars in 1569 bytes} [Sat Aug 20 21:21:41 2022] DELETE /identity/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/groups/13f06b3cce4945b1987c5b942b37a61a/roles/5951e8c3837f499b9c56712c0b1b6787/inherited_to_projects => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:42.038591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c3409-bb9e-4f2b-b230-11a10090f0cc tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:42.040457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9c2c3409-bb9e-4f2b-b230-11a10090f0cc tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:42.053689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c3409-bb9e-4f2b-b230-11a10090f0cc tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:42.066560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:42.117448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9211610-795b-4ff2-8ea1-a72f4cd52dcf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:42.118239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9211610-795b-4ff2-8ea1-a72f4cd52dcf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:42.118619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9211610-795b-4ff2-8ea1-a72f4cd52dcf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:42.119007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9211610-795b-4ff2-8ea1-a72f4cd52dcf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/roles/5951e8c3837f499b9c56712c0b1b6787` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:42.122113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9211610-795b-4ff2-8ea1-a72f4cd52dcf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:delete_role(role_id=5951e8c3837f499b9c56712c0b1b6787)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:42.122950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9211610-795b-4ff2-8ea1-a72f4cd52dcf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:42.165188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9211610-795b-4ff2-8ea1-a72f4cd52dcf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invalidating the token cache because role 5951e8c3837f499b9c56712c0b1b6787 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:42.165558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9211610-795b-4ff2-8ea1-a72f4cd52dcf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:42.166319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9211610-795b-4ff2-8ea1-a72f4cd52dcf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:42.168724 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3243/6566] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:42 2022] DELETE /identity/v3/roles/5951e8c3837f499b9c56712c0b1b6787 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:42.173700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609ef0c-68c9-492c-993b-2af3aac6cb9e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:42.184625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609ef0c-68c9-492c-993b-2af3aac6cb9e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:42.253290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b281b-872f-4bf9-a07f-d18a40f72f3f tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:42.253956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b281b-872f-4bf9-a07f-d18a40f72f3f tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:42.254416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b281b-872f-4bf9-a07f-d18a40f72f3f tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:42.254508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b281b-872f-4bf9-a07f-d18a40f72f3f tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:42.255190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b281b-872f-4bf9-a07f-d18a40f72f3f tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:42.257082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b281b-872f-4bf9-a07f-d18a40f72f3f tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:42.278904 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3324/6567] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:21:42 2022] POST /identity/v3/roles => generated 240 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:42.282395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9211610-795b-4ff2-8ea1-a72f4cd52dcf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:42.283882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f9211610-795b-4ff2-8ea1-a72f4cd52dcf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:42.297978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9211610-795b-4ff2-8ea1-a72f4cd52dcf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:42.303633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a286379-35f4-42c2-bfcf-7b963b6acacf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:42.304581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a286379-35f4-42c2-bfcf-7b963b6acacf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:42.304919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a286379-35f4-42c2-bfcf-7b963b6acacf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:42.305260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a286379-35f4-42c2-bfcf-7b963b6acacf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/users/b7b2379860f4465aab4eb3399d8e5b25/roles/cd63de4be43a44aab8d77cbb89846444/inherited_to_projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:42.326787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a286379-35f4-42c2-bfcf-7b963b6acacf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_grant(project_id=ae1fae1af07f4195a6c2a570f00eeb66, user_id=b7b2379860f4465aab4eb3399d8e5b25, role_id=cd63de4be43a44aab8d77cbb89846444)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:42.330177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a286379-35f4-42c2-bfcf-7b963b6acacf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:42.348424 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3244/6568] 10.222.0.23 () {60 vars in 1582 bytes} [Sat Aug 20 21:21:42 2022] PUT /identity/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/users/b7b2379860f4465aab4eb3399d8e5b25/roles/cd63de4be43a44aab8d77cbb89846444/inherited_to_projects => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:42.351689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b281b-872f-4bf9-a07f-d18a40f72f3f tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:42.364237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b281b-872f-4bf9-a07f-d18a40f72f3f tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:42.445067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de62303-4b45-4c75-ab3b-6d6e04c5b8be tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:42.445871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de62303-4b45-4c75-ab3b-6d6e04c5b8be tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:42.446216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de62303-4b45-4c75-ab3b-6d6e04c5b8be tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:42.446562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de62303-4b45-4c75-ab3b-6d6e04c5b8be tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/users/b7b2379860f4465aab4eb3399d8e5b25/roles/cd63de4be43a44aab8d77cbb89846444/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:42.449179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de62303-4b45-4c75-ab3b-6d6e04c5b8be tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:check_grant(project_id=ae1fae1af07f4195a6c2a570f00eeb66, user_id=b7b2379860f4465aab4eb3399d8e5b25, role_id=cd63de4be43a44aab8d77cbb89846444)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:42.450728 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-3de62303-4b45-4c75-ab3b-6d6e04c5b8be tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:21:42.450728 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:21:42.450728 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:21:42.452990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de62303-4b45-4c75-ab3b-6d6e04c5b8be tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:42.464805 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3325/6569] 10.222.0.23 () {58 vars in 1564 bytes} [Sat Aug 20 21:21:42 2022] HEAD /identity/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/users/b7b2379860f4465aab4eb3399d8e5b25/roles/cd63de4be43a44aab8d77cbb89846444/inherited_to_projects => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:42.465842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a286379-35f4-42c2-bfcf-7b963b6acacf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:42.480065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a286379-35f4-42c2-bfcf-7b963b6acacf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:42.487388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c50d4-39ac-490d-ac45-f0b0f784194c tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:42.488208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c50d4-39ac-490d-ac45-f0b0f784194c tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:42.488614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c50d4-39ac-490d-ac45-f0b0f784194c tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:42.488984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c50d4-39ac-490d-ac45-f0b0f784194c tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/users/b7b2379860f4465aab4eb3399d8e5b25/roles/cd63de4be43a44aab8d77cbb89846444/inherited_to_projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:42.492466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c50d4-39ac-490d-ac45-f0b0f784194c tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=ae1fae1af07f4195a6c2a570f00eeb66, user_id=b7b2379860f4465aab4eb3399d8e5b25, role_id=cd63de4be43a44aab8d77cbb89846444)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:42.495511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c50d4-39ac-490d-ac45-f0b0f784194c tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:42.501376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fd8c50d4-39ac-490d-ac45-f0b0f784194c tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invalidating the token cache because role cd63de4be43a44aab8d77cbb89846444 was removed from user b7b2379860f4465aab4eb3399d8e5b25 on project ae1fae1af07f4195a6c2a570f00eeb66. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:42.501735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fd8c50d4-39ac-490d-ac45-f0b0f784194c tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:42.502642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fd8c50d4-39ac-490d-ac45-f0b0f784194c tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:42.522809 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3245/6570] 10.222.0.23 () {58 vars in 1566 bytes} [Sat Aug 20 21:21:42 2022] DELETE /identity/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/users/b7b2379860f4465aab4eb3399d8e5b25/roles/cd63de4be43a44aab8d77cbb89846444/inherited_to_projects => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:42.527791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de62303-4b45-4c75-ab3b-6d6e04c5b8be tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:42.530959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3de62303-4b45-4c75-ab3b-6d6e04c5b8be tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:42.546756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de62303-4b45-4c75-ab3b-6d6e04c5b8be tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:42.635772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc5c8a-b81b-4049-906b-438903aefd60 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:42.636524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc5c8a-b81b-4049-906b-438903aefd60 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:42.636829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc5c8a-b81b-4049-906b-438903aefd60 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:42.637095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc5c8a-b81b-4049-906b-438903aefd60 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/roles/cd63de4be43a44aab8d77cbb89846444` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:42.638563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc5c8a-b81b-4049-906b-438903aefd60 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:delete_role(role_id=cd63de4be43a44aab8d77cbb89846444)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:42.640209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc5c8a-b81b-4049-906b-438903aefd60 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:42.651615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:42.698752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-67cc5c8a-b81b-4049-906b-438903aefd60 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invalidating the token cache because role cd63de4be43a44aab8d77cbb89846444 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:42.699278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-67cc5c8a-b81b-4049-906b-438903aefd60 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:42.699827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-67cc5c8a-b81b-4049-906b-438903aefd60 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:42.706900 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3326/6571] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:42 2022] DELETE /identity/v3/roles/cd63de4be43a44aab8d77cbb89846444 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:42.708150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c50d4-39ac-490d-ac45-f0b0f784194c tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:42.721367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c50d4-39ac-490d-ac45-f0b0f784194c tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:42.811188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fb91e-05cb-44db-91f9-5c138b942baa tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:42.811806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fb91e-05cb-44db-91f9-5c138b942baa tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:42.812151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fb91e-05cb-44db-91f9-5c138b942baa tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:42.812535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fb91e-05cb-44db-91f9-5c138b942baa tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:42.813570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3fb91e-05cb-44db-91f9-5c138b942baa tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:42.815360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3fb91e-05cb-44db-91f9-5c138b942baa tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:42.836249 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3246/6572] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:21:42 2022] POST /identity/v3/roles => generated 240 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:42.841049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc5c8a-b81b-4049-906b-438903aefd60 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:42.843073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-67cc5c8a-b81b-4049-906b-438903aefd60 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:42.860731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc5c8a-b81b-4049-906b-438903aefd60 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:42.868357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bef15-c5ce-411b-8a99-c9798119c743 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:42.869098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bef15-c5ce-411b-8a99-c9798119c743 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:42.869371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bef15-c5ce-411b-8a99-c9798119c743 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:42.869608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bef15-c5ce-411b-8a99-c9798119c743 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/groups/13f06b3cce4945b1987c5b942b37a61a/roles/7e539fc3411e401d996f857e9936bb64/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:42.871799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bef15-c5ce-411b-8a99-c9798119c743 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=ea2d792531ea4de0a70794e75e951f79, group_id=13f06b3cce4945b1987c5b942b37a61a, role_id=7e539fc3411e401d996f857e9936bb64)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:42.877183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bef15-c5ce-411b-8a99-c9798119c743 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:42.878633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fb91e-05cb-44db-91f9-5c138b942baa tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:42.880346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2f3fb91e-05cb-44db-91f9-5c138b942baa tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:42.892924 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3327/6573] 10.222.0.23 () {60 vars in 1582 bytes} [Sat Aug 20 21:21:42 2022] PUT /identity/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/groups/13f06b3cce4945b1987c5b942b37a61a/roles/7e539fc3411e401d996f857e9936bb64/inherited_to_projects => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:42.897247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bef15-c5ce-411b-8a99-c9798119c743 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:42.941441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:42.970268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bef15-c5ce-411b-8a99-c9798119c743 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:42.975298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb14d3-0494-47f9-b972-9c59fdee4e99 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:42.975965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb14d3-0494-47f9-b972-9c59fdee4e99 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:42.976244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb14d3-0494-47f9-b972-9c59fdee4e99 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:42.976554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb14d3-0494-47f9-b972-9c59fdee4e99 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/groups/13f06b3cce4945b1987c5b942b37a61a/roles/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:42.976929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fb91e-05cb-44db-91f9-5c138b942baa tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:42.978251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb14d3-0494-47f9-b972-9c59fdee4e99 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=ea2d792531ea4de0a70794e75e951f79, group_id=13f06b3cce4945b1987c5b942b37a61a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:42.979289 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-f2fb14d3-0494-47f9-b972-9c59fdee4e99 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:21:42.979289 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:21:42.979289 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:21:42.981293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb14d3-0494-47f9-b972-9c59fdee4e99 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:42.997362 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3328/6574] 10.222.0.23 () {58 vars in 1464 bytes} [Sat Aug 20 21:21:42 2022] GET /identity/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/groups/13f06b3cce4945b1987c5b942b37a61a/roles/inherited_to_projects => generated 448 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:42.998631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb14d3-0494-47f9-b972-9c59fdee4e99 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.009494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb14d3-0494-47f9-b972-9c59fdee4e99 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.014643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fab414-f566-430f-b304-b1be9241b062 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.015386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fab414-f566-430f-b304-b1be9241b062 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.015636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fab414-f566-430f-b304-b1be9241b062 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.015909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fab414-f566-430f-b304-b1be9241b062 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/groups/13f06b3cce4945b1987c5b942b37a61a/roles/7e539fc3411e401d996f857e9936bb64/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.018181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fab414-f566-430f-b304-b1be9241b062 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=ea2d792531ea4de0a70794e75e951f79, group_id=13f06b3cce4945b1987c5b942b37a61a, role_id=7e539fc3411e401d996f857e9936bb64)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.020924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fab414-f566-430f-b304-b1be9241b062 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.027766 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3329/6575] 10.222.0.23 () {58 vars in 1564 bytes} [Sat Aug 20 21:21:42 2022] HEAD /identity/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/groups/13f06b3cce4945b1987c5b942b37a61a/roles/7e539fc3411e401d996f857e9936bb64/inherited_to_projects => generated 0 bytes in 29 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:43.031744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fab414-f566-430f-b304-b1be9241b062 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.035312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5651bf-95e1-4274-96a3-30f96618f372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.035948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5651bf-95e1-4274-96a3-30f96618f372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.036216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5651bf-95e1-4274-96a3-30f96618f372 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.036552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5651bf-95e1-4274-96a3-30f96618f372 admin admin] PATH_INFO: `/v3/users/3658cbe4b8ff4d5094fb9dfa0f2cbdd2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.037772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5651bf-95e1-4274-96a3-30f96618f372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3658cbe4b8ff4d5094fb9dfa0f2cbdd2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.039954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5651bf-95e1-4274-96a3-30f96618f372 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.041788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fab414-f566-430f-b304-b1be9241b062 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.046413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6e916-8865-4b21-9831-48554a7b8e8e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.047064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6e916-8865-4b21-9831-48554a7b8e8e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.047457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6e916-8865-4b21-9831-48554a7b8e8e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.047737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6e916-8865-4b21-9831-48554a7b8e8e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/groups/13f06b3cce4945b1987c5b942b37a61a/roles/7e539fc3411e401d996f857e9936bb64/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.049944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6e916-8865-4b21-9831-48554a7b8e8e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=ea2d792531ea4de0a70794e75e951f79, group_id=13f06b3cce4945b1987c5b942b37a61a, role_id=7e539fc3411e401d996f857e9936bb64)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.052845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6e916-8865-4b21-9831-48554a7b8e8e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.058464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91c6e916-8865-4b21-9831-48554a7b8e8e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invalidating the token cache because role 7e539fc3411e401d996f857e9936bb64 was removed from group 13f06b3cce4945b1987c5b942b37a61a on domain ea2d792531ea4de0a70794e75e951f79. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:43.058748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91c6e916-8865-4b21-9831-48554a7b8e8e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.059474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91c6e916-8865-4b21-9831-48554a7b8e8e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.075424 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3330/6576] 10.222.0.23 () {58 vars in 1566 bytes} [Sat Aug 20 21:21:43 2022] DELETE /identity/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/groups/13f06b3cce4945b1987c5b942b37a61a/roles/7e539fc3411e401d996f857e9936bb64/inherited_to_projects => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:43.079994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6e916-8865-4b21-9831-48554a7b8e8e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.081547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-91c6e916-8865-4b21-9831-48554a7b8e8e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:43.094817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cb5651bf-95e1-4274-96a3-30f96618f372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3658cbe4b8ff4d5094fb9dfa0f2cbdd2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.103287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cb5651bf-95e1-4274-96a3-30f96618f372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3658cbe4b8ff4d5094fb9dfa0f2cbdd2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.116562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cb5651bf-95e1-4274-96a3-30f96618f372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3658cbe4b8ff4d5094fb9dfa0f2cbdd2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.117262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cb5651bf-95e1-4274-96a3-30f96618f372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3658cbe4b8ff4d5094fb9dfa0f2cbdd2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.117925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cb5651bf-95e1-4274-96a3-30f96618f372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3658cbe4b8ff4d5094fb9dfa0f2cbdd2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.131532 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3247/6577] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:42 2022] DELETE /identity/v3/users/3658cbe4b8ff4d5094fb9dfa0f2cbdd2 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:43.138366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8a045-06a2-453e-a0ff-22031b744719 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.138650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8a045-06a2-453e-a0ff-22031b744719 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.138919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8a045-06a2-453e-a0ff-22031b744719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.153578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6e916-8865-4b21-9831-48554a7b8e8e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.158340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cde8a045-06a2-453e-a0ff-22031b744719 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:43.159942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cde8a045-06a2-453e-a0ff-22031b744719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:43.184961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:43.215992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560eb0aa-dfc0-4cb6-b616-97c0284c8e89 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.216800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eb0aa-dfc0-4cb6-b616-97c0284c8e89 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.217075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eb0aa-dfc0-4cb6-b616-97c0284c8e89 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.217345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eb0aa-dfc0-4cb6-b616-97c0284c8e89 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/roles/7e539fc3411e401d996f857e9936bb64` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.218729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560eb0aa-dfc0-4cb6-b616-97c0284c8e89 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:delete_role(role_id=7e539fc3411e401d996f857e9936bb64)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.220365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560eb0aa-dfc0-4cb6-b616-97c0284c8e89 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.224056 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3248/6578] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:43.237603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8a045-06a2-453e-a0ff-22031b744719 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.239222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cde8a045-06a2-453e-a0ff-22031b744719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:43.246387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:43.258207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-560eb0aa-dfc0-4cb6-b616-97c0284c8e89 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invalidating the token cache because role 7e539fc3411e401d996f857e9936bb64 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:43.258494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-560eb0aa-dfc0-4cb6-b616-97c0284c8e89 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.259180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-560eb0aa-dfc0-4cb6-b616-97c0284c8e89 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.261107 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3331/6579] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:43 2022] DELETE /identity/v3/roles/7e539fc3411e401d996f857e9936bb64 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:43.266142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560eb0aa-dfc0-4cb6-b616-97c0284c8e89 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.267504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-560eb0aa-dfc0-4cb6-b616-97c0284c8e89 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:43.312654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8a045-06a2-453e-a0ff-22031b744719 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.338659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560eb0aa-dfc0-4cb6-b616-97c0284c8e89 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.344762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e089f50-173c-4d9a-b127-486c8fc4c213 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.345395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e089f50-173c-4d9a-b127-486c8fc4c213 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.345673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e089f50-173c-4d9a-b127-486c8fc4c213 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.345944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e089f50-173c-4d9a-b127-486c8fc4c213 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.346610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e089f50-173c-4d9a-b127-486c8fc4c213 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.348327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e089f50-173c-4d9a-b127-486c8fc4c213 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.367777 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3332/6580] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:21:43 2022] POST /identity/v3/roles => generated 240 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:43.368795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e089f50-173c-4d9a-b127-486c8fc4c213 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.370052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6ccb6-19fe-4113-86ed-16d88fe612e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.370679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6ccb6-19fe-4113-86ed-16d88fe612e1 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.370949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6ccb6-19fe-4113-86ed-16d88fe612e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.371212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6ccb6-19fe-4113-86ed-16d88fe612e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.372983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-68c6ccb6-19fe-4113-86ed-16d88fe612e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:43.379410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e089f50-173c-4d9a-b127-486c8fc4c213 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.384213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad93070-4d3b-4033-8b5e-8145d7e251cf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.384896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad93070-4d3b-4033-8b5e-8145d7e251cf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.385169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad93070-4d3b-4033-8b5e-8145d7e251cf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.385432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad93070-4d3b-4033-8b5e-8145d7e251cf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/users/b7b2379860f4465aab4eb3399d8e5b25/roles/3335fa767db143e29e59227bb68abd0f/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.387644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad93070-4d3b-4033-8b5e-8145d7e251cf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=ea2d792531ea4de0a70794e75e951f79, user_id=b7b2379860f4465aab4eb3399d8e5b25, role_id=3335fa767db143e29e59227bb68abd0f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.390361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad93070-4d3b-4033-8b5e-8145d7e251cf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.406549 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3333/6581] 10.222.0.23 () {60 vars in 1579 bytes} [Sat Aug 20 21:21:43 2022] PUT /identity/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/users/b7b2379860f4465aab4eb3399d8e5b25/roles/3335fa767db143e29e59227bb68abd0f/inherited_to_projects => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:43.411031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad93070-4d3b-4033-8b5e-8145d7e251cf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.451179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6ccb6-19fe-4113-86ed-16d88fe612e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.452943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6ccb6-19fe-4113-86ed-16d88fe612e1 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.465870 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3249/6582] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:43.482989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad93070-4d3b-4033-8b5e-8145d7e251cf tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.488327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3635a7-67a9-4631-aff5-8b2c1100672e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.489027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3635a7-67a9-4631-aff5-8b2c1100672e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.489305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3635a7-67a9-4631-aff5-8b2c1100672e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.489580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3635a7-67a9-4631-aff5-8b2c1100672e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/users/b7b2379860f4465aab4eb3399d8e5b25/roles/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.491617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3635a7-67a9-4631-aff5-8b2c1100672e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=ea2d792531ea4de0a70794e75e951f79, user_id=b7b2379860f4465aab4eb3399d8e5b25)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.494308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3635a7-67a9-4631-aff5-8b2c1100672e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.507106 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3334/6583] 10.222.0.23 () {58 vars in 1461 bytes} [Sat Aug 20 21:21:43 2022] GET /identity/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/users/b7b2379860f4465aab4eb3399d8e5b25/roles/inherited_to_projects => generated 447 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:43.511665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6ccb6-19fe-4113-86ed-16d88fe612e1 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.523878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6ccb6-19fe-4113-86ed-16d88fe612e1 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.529232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a3bbc-44a3-41ab-aa56-5a52056bb5de tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.530251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a3bbc-44a3-41ab-aa56-5a52056bb5de tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.530597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a3bbc-44a3-41ab-aa56-5a52056bb5de tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.530938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a3bbc-44a3-41ab-aa56-5a52056bb5de tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/users/b7b2379860f4465aab4eb3399d8e5b25/roles/3335fa767db143e29e59227bb68abd0f/inherited_to_projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.533430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a3bbc-44a3-41ab-aa56-5a52056bb5de tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=ea2d792531ea4de0a70794e75e951f79, user_id=b7b2379860f4465aab4eb3399d8e5b25, role_id=3335fa767db143e29e59227bb68abd0f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.536436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a3bbc-44a3-41ab-aa56-5a52056bb5de tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.545989 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3250/6584] 10.222.0.23 () {58 vars in 1561 bytes} [Sat Aug 20 21:21:43 2022] HEAD /identity/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/users/b7b2379860f4465aab4eb3399d8e5b25/roles/3335fa767db143e29e59227bb68abd0f/inherited_to_projects => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:43.550992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3635a7-67a9-4631-aff5-8b2c1100672e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.565620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3635a7-67a9-4631-aff5-8b2c1100672e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.571128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ed1bc-7d73-436f-93c6-f75121309781 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.571976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ed1bc-7d73-436f-93c6-f75121309781 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.572285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ed1bc-7d73-436f-93c6-f75121309781 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.572665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ed1bc-7d73-436f-93c6-f75121309781 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/users/b7b2379860f4465aab4eb3399d8e5b25/roles/3335fa767db143e29e59227bb68abd0f/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.575069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ed1bc-7d73-436f-93c6-f75121309781 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=ea2d792531ea4de0a70794e75e951f79, user_id=b7b2379860f4465aab4eb3399d8e5b25, role_id=3335fa767db143e29e59227bb68abd0f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.578105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ed1bc-7d73-436f-93c6-f75121309781 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.581710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:43.584341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-000ed1bc-7d73-436f-93c6-f75121309781 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invalidating the token cache because role 3335fa767db143e29e59227bb68abd0f was removed from user b7b2379860f4465aab4eb3399d8e5b25 on domain ea2d792531ea4de0a70794e75e951f79. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:43.584794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-000ed1bc-7d73-436f-93c6-f75121309781 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.585550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-000ed1bc-7d73-436f-93c6-f75121309781 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.601924 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3335/6585] 10.222.0.23 () {58 vars in 1563 bytes} [Sat Aug 20 21:21:43 2022] DELETE /identity/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/users/b7b2379860f4465aab4eb3399d8e5b25/roles/3335fa767db143e29e59227bb68abd0f/inherited_to_projects => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:43.605865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a3bbc-44a3-41ab-aa56-5a52056bb5de tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.607796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ee8a3bbc-44a3-41ab-aa56-5a52056bb5de tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:43.621837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a3bbc-44a3-41ab-aa56-5a52056bb5de tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.684123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1b5c1-cb99-463f-9920-6ca7bb0a0d12 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.684802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b5c1-cb99-463f-9920-6ca7bb0a0d12 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.685062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b5c1-cb99-463f-9920-6ca7bb0a0d12 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.685331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b5c1-cb99-463f-9920-6ca7bb0a0d12 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/roles/3335fa767db143e29e59227bb68abd0f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.686735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1b5c1-cb99-463f-9920-6ca7bb0a0d12 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:delete_role(role_id=3335fa767db143e29e59227bb68abd0f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.688538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1b5c1-cb99-463f-9920-6ca7bb0a0d12 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.690067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:43.727759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c6d1b5c1-cb99-463f-9920-6ca7bb0a0d12 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invalidating the token cache because role 3335fa767db143e29e59227bb68abd0f has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:43.728620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c6d1b5c1-cb99-463f-9920-6ca7bb0a0d12 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.728944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c6d1b5c1-cb99-463f-9920-6ca7bb0a0d12 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.731398 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3251/6586] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:43 2022] DELETE /identity/v3/roles/3335fa767db143e29e59227bb68abd0f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:43.736961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ed1bc-7d73-436f-93c6-f75121309781 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.748252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ed1bc-7d73-436f-93c6-f75121309781 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.753876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1b5c1-cb99-463f-9920-6ca7bb0a0d12 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.755592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c6d1b5c1-cb99-463f-9920-6ca7bb0a0d12 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:43.826146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b0cd5-6d0e-4f71-b499-5a21dfe2a4d7 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.826760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b0cd5-6d0e-4f71-b499-5a21dfe2a4d7 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.827245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b0cd5-6d0e-4f71-b499-5a21dfe2a4d7 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.827514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b0cd5-6d0e-4f71-b499-5a21dfe2a4d7 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.828181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b0cd5-6d0e-4f71-b499-5a21dfe2a4d7 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.829835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b0cd5-6d0e-4f71-b499-5a21dfe2a4d7 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.831558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1b5c1-cb99-463f-9920-6ca7bb0a0d12 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.836867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83712af-e4bb-42c5-8a04-0349f0110edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.837508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83712af-e4bb-42c5-8a04-0349f0110edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.837777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83712af-e4bb-42c5-8a04-0349f0110edb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.838675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83712af-e4bb-42c5-8a04-0349f0110edb admin admin] PATH_INFO: `/v3/projects/963b7fc23b044b378008315043ea307a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.840115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83712af-e4bb-42c5-8a04-0349f0110edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=963b7fc23b044b378008315043ea307a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.842078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83712af-e4bb-42c5-8a04-0349f0110edb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.845599 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3336/6587] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:21:43 2022] POST /identity/v3/roles => generated 239 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:43.849813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b0cd5-6d0e-4f71-b499-5a21dfe2a4d7 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.851508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-706b0cd5-6d0e-4f71-b499-5a21dfe2a4d7 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:43.863674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b0cd5-6d0e-4f71-b499-5a21dfe2a4d7 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.868474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231e3df-27aa-4453-87ae-d99cb4c17473 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.869078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231e3df-27aa-4453-87ae-d99cb4c17473 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.869354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231e3df-27aa-4453-87ae-d99cb4c17473 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.869689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231e3df-27aa-4453-87ae-d99cb4c17473 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.870256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231e3df-27aa-4453-87ae-d99cb4c17473 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.872207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231e3df-27aa-4453-87ae-d99cb4c17473 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:43.900696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c83712af-e4bb-42c5-8a04-0349f0110edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '963b7fc23b044b378008315043ea307a', 'request_id': 'req-c83712af-e4bb-42c5-8a04-0349f0110edb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.907048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c83712af-e4bb-42c5-8a04-0349f0110edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '963b7fc23b044b378008315043ea307a', 'request_id': 'req-c83712af-e4bb-42c5-8a04-0349f0110edb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.910611 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3337/6588] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:43 2022] POST /identity/v3/projects => generated 409 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:43.914874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c83712af-e4bb-42c5-8a04-0349f0110edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '963b7fc23b044b378008315043ea307a', 'request_id': 'req-c83712af-e4bb-42c5-8a04-0349f0110edb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.915229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231e3df-27aa-4453-87ae-d99cb4c17473 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:43.925755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c83712af-e4bb-42c5-8a04-0349f0110edb admin admin] The token cache is being invalidate because project 963b7fc23b044b378008315043ea307a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:43.926174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c83712af-e4bb-42c5-8a04-0349f0110edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.926900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c83712af-e4bb-42c5-8a04-0349f0110edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:43.928669 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3252/6589] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:43 2022] DELETE /identity/v3/projects/963b7fc23b044b378008315043ea307a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:43.941885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c25f-3a29-4672-8afc-573af081ed4e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.942161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c25f-3a29-4672-8afc-573af081ed4e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.942427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c25f-3a29-4672-8afc-573af081ed4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.963778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4199c25f-3a29-4672-8afc-573af081ed4e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:43.965015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4199c25f-3a29-4672-8afc-573af081ed4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:43.986103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231e3df-27aa-4453-87ae-d99cb4c17473 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:43.992085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c5104-39a5-4ad9-8689-168975838bc4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:43.992085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c5104-39a5-4ad9-8689-168975838bc4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:43.992305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c5104-39a5-4ad9-8689-168975838bc4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:43.992507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c5104-39a5-4ad9-8689-168975838bc4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/users/b7b2379860f4465aab4eb3399d8e5b25/roles/74ef352eb7d5484d872e8460ea122323/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:43.995186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c5104-39a5-4ad9-8689-168975838bc4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=ea2d792531ea4de0a70794e75e951f79, user_id=b7b2379860f4465aab4eb3399d8e5b25, role_id=74ef352eb7d5484d872e8460ea122323)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:43.998000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c5104-39a5-4ad9-8689-168975838bc4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.013332 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3338/6590] 10.222.0.23 () {60 vars in 1579 bytes} [Sat Aug 20 21:21:43 2022] PUT /identity/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/users/b7b2379860f4465aab4eb3399d8e5b25/roles/74ef352eb7d5484d872e8460ea122323/inherited_to_projects => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:44.018230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c5104-39a5-4ad9-8689-168975838bc4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.019917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-687c5104-39a5-4ad9-8689-168975838bc4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:44.024509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:44.042607 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3253/6591] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:44.050731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c07da-a85d-49cb-b006-d7ea1b0d5338 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.051141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c07da-a85d-49cb-b006-d7ea1b0d5338 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.051519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c07da-a85d-49cb-b006-d7ea1b0d5338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.072319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c03c07da-a85d-49cb-b006-d7ea1b0d5338 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:44.073743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c03c07da-a85d-49cb-b006-d7ea1b0d5338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:44.087267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:44.105550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c5104-39a5-4ad9-8689-168975838bc4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.110886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafb74b-f7d9-4765-a136-4ba263eee318 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.111378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafb74b-f7d9-4765-a136-4ba263eee318 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.111654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafb74b-f7d9-4765-a136-4ba263eee318 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.111923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafb74b-f7d9-4765-a136-4ba263eee318 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.112853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafb74b-f7d9-4765-a136-4ba263eee318 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Adding query filter params (scope.project.id=ae1fae1af07f4195a6c2a570f00eeb66, user.id=b7b2379860f4465aab4eb3399d8e5b25) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:44.113188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafb74b-f7d9-4765-a136-4ba263eee318 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:44.115170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafb74b-f7d9-4765-a136-4ba263eee318 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.152503 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3254/6592] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:44.157466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c07da-a85d-49cb-b006-d7ea1b0d5338 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.169211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c07da-a85d-49cb-b006-d7ea1b0d5338 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.172931 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3339/6593] 10.222.0.23 () {58 vars in 1356 bytes} [Sat Aug 20 21:21:44 2022] GET /identity/v3/role_assignments?scope.project.id=ae1fae1af07f4195a6c2a570f00eeb66&user.id=b7b2379860f4465aab4eb3399d8e5b25&effective => generated 648 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:44.177084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafb74b-f7d9-4765-a136-4ba263eee318 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.178065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1652c-f3f9-4fdb-9734-1a215c9ed065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.178698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1652c-f3f9-4fdb-9734-1a215c9ed065 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.179248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1652c-f3f9-4fdb-9734-1a215c9ed065 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.179523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1652c-f3f9-4fdb-9734-1a215c9ed065 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.180176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1652c-f3f9-4fdb-9734-1a215c9ed065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:44.180517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1652c-f3f9-4fdb-9734-1a215c9ed065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:44.182132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1652c-f3f9-4fdb-9734-1a215c9ed065 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.191914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafb74b-f7d9-4765-a136-4ba263eee318 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.200325 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3255/6594] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:44.201373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bd166-34af-4456-82b5-8d4c8e6a67c4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.201523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1652c-f3f9-4fdb-9734-1a215c9ed065 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.201997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bd166-34af-4456-82b5-8d4c8e6a67c4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.202275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bd166-34af-4456-82b5-8d4c8e6a67c4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.202544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bd166-34af-4456-82b5-8d4c8e6a67c4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.203268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bd166-34af-4456-82b5-8d4c8e6a67c4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Adding query filter params (scope.project.id=26d543a25e214530b11e5701487f2dd0, user.id=b7b2379860f4465aab4eb3399d8e5b25) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:44.203567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bd166-34af-4456-82b5-8d4c8e6a67c4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:44.205474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bd166-34af-4456-82b5-8d4c8e6a67c4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.212941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1652c-f3f9-4fdb-9734-1a215c9ed065 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.219060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16128b70-1338-415b-806a-d4a52b1f9746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.219304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16128b70-1338-415b-806a-d4a52b1f9746 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.219533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16128b70-1338-415b-806a-d4a52b1f9746 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.219810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16128b70-1338-415b-806a-d4a52b1f9746 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.220447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16128b70-1338-415b-806a-d4a52b1f9746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:44.222594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16128b70-1338-415b-806a-d4a52b1f9746 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.260568 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3256/6595] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:44 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:44.262804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16128b70-1338-415b-806a-d4a52b1f9746 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.279539 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3340/6596] 10.222.0.23 () {58 vars in 1356 bytes} [Sat Aug 20 21:21:44 2022] GET /identity/v3/role_assignments?scope.project.id=26d543a25e214530b11e5701487f2dd0&user.id=b7b2379860f4465aab4eb3399d8e5b25&effective => generated 648 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:44.280816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bd166-34af-4456-82b5-8d4c8e6a67c4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.295354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bd166-34af-4456-82b5-8d4c8e6a67c4 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.359660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16128b70-1338-415b-806a-d4a52b1f9746 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.365185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f2151-d02e-4741-a782-dc628f38546e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.366132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f2151-d02e-4741-a782-dc628f38546e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.366591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f2151-d02e-4741-a782-dc628f38546e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.366972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f2151-d02e-4741-a782-dc628f38546e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/users/b7b2379860f4465aab4eb3399d8e5b25/roles/74ef352eb7d5484d872e8460ea122323/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.367948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9e3f2-602e-4482-a7b5-8b1fcf871755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.370458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9e3f2-602e-4482-a7b5-8b1fcf871755 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.370458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9e3f2-602e-4482-a7b5-8b1fcf871755 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.370688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f2151-d02e-4741-a782-dc628f38546e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=ea2d792531ea4de0a70794e75e951f79, user_id=b7b2379860f4465aab4eb3399d8e5b25, role_id=74ef352eb7d5484d872e8460ea122323)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:44.370788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9e3f2-602e-4482-a7b5-8b1fcf871755 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.370788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9e3f2-602e-4482-a7b5-8b1fcf871755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:44.372939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9e3f2-602e-4482-a7b5-8b1fcf871755 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.373243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f2151-d02e-4741-a782-dc628f38546e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.379810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc9f2151-d02e-4741-a782-dc628f38546e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invalidating the token cache because role 74ef352eb7d5484d872e8460ea122323 was removed from user b7b2379860f4465aab4eb3399d8e5b25 on domain ea2d792531ea4de0a70794e75e951f79. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:44.380220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc9f2151-d02e-4741-a782-dc628f38546e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:44.381224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bc9f2151-d02e-4741-a782-dc628f38546e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:44.403692 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3257/6597] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:44 2022] POST /identity/v3/users => generated 286 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:44.403893 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3341/6598] 10.222.0.23 () {58 vars in 1563 bytes} [Sat Aug 20 21:21:44 2022] DELETE /identity/v3/OS-INHERIT/domains/ea2d792531ea4de0a70794e75e951f79/users/b7b2379860f4465aab4eb3399d8e5b25/roles/74ef352eb7d5484d872e8460ea122323/inherited_to_projects => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:44.405631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f2151-d02e-4741-a782-dc628f38546e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.407215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9e3f2-602e-4482-a7b5-8b1fcf871755 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.409196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bc9f2151-d02e-4741-a782-dc628f38546e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:44.410978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-38a9e3f2-602e-4482-a7b5-8b1fcf871755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:44.434598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9e3f2-602e-4482-a7b5-8b1fcf871755 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.498789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:44.510379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:44.513140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e5917-4e48-4160-b949-eee7c4ae89f2 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.514041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e5917-4e48-4160-b949-eee7c4ae89f2 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.514299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e5917-4e48-4160-b949-eee7c4ae89f2 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.514567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e5917-4e48-4160-b949-eee7c4ae89f2 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.515304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e5917-4e48-4160-b949-eee7c4ae89f2 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Adding query filter params (scope.project.id=ae1fae1af07f4195a6c2a570f00eeb66, user.id=b7b2379860f4465aab4eb3399d8e5b25) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:44.515612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e5917-4e48-4160-b949-eee7c4ae89f2 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:44.516737 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-9d8e5917-4e48-4160-b949-eee7c4ae89f2 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:21:44.516737 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:21:44.516737 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:21:44.517831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e5917-4e48-4160-b949-eee7c4ae89f2 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.533691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f2151-d02e-4741-a782-dc628f38546e tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.539428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe346aac-d9e7-4adc-baa8-ce5ca502a4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.540055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe346aac-d9e7-4adc-baa8-ce5ca502a4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.540325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe346aac-d9e7-4adc-baa8-ce5ca502a4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.540619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe346aac-d9e7-4adc-baa8-ce5ca502a4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.541263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe346aac-d9e7-4adc-baa8-ce5ca502a4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:44.542899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe346aac-d9e7-4adc-baa8-ce5ca502a4a7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.557814 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3342/6599] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:44 2022] GET /identity/v3/roles => generated 1650 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:44.558928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe346aac-d9e7-4adc-baa8-ce5ca502a4a7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.566199 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3258/6600] 10.222.0.23 () {58 vars in 1356 bytes} [Sat Aug 20 21:21:44 2022] GET /identity/v3/role_assignments?scope.project.id=ae1fae1af07f4195a6c2a570f00eeb66&user.id=b7b2379860f4465aab4eb3399d8e5b25&effective => generated 228 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:44.569729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe346aac-d9e7-4adc-baa8-ce5ca502a4a7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.571240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e5917-4e48-4160-b949-eee7c4ae89f2 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.575021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bf28c-1f8b-419c-b25a-e52640336fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.575639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bf28c-1f8b-419c-b25a-e52640336fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.575911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bf28c-1f8b-419c-b25a-e52640336fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.576176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bf28c-1f8b-419c-b25a-e52640336fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.576851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5bf28c-1f8b-419c-b25a-e52640336fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:44.578479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5bf28c-1f8b-419c-b25a-e52640336fc8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.582103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e5917-4e48-4160-b949-eee7c4ae89f2 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.587989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f62e6-64ca-456a-9242-ce9d4794cfed tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.588650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f62e6-64ca-456a-9242-ce9d4794cfed tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.588929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f62e6-64ca-456a-9242-ce9d4794cfed tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.589214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f62e6-64ca-456a-9242-ce9d4794cfed tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.589931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f62e6-64ca-456a-9242-ce9d4794cfed tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Adding query filter params (scope.project.id=26d543a25e214530b11e5701487f2dd0, user.id=b7b2379860f4465aab4eb3399d8e5b25) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:44.590235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f62e6-64ca-456a-9242-ce9d4794cfed tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:44.594018 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3343/6601] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:44 2022] GET /identity/v3/roles => generated 1650 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:44.594183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f62e6-64ca-456a-9242-ce9d4794cfed tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.594894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bf28c-1f8b-419c-b25a-e52640336fc8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.605852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bf28c-1f8b-419c-b25a-e52640336fc8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.611588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737b541-213c-41fc-aa0e-0913450df581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.611588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737b541-213c-41fc-aa0e-0913450df581 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.611588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737b541-213c-41fc-aa0e-0913450df581 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.611925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737b541-213c-41fc-aa0e-0913450df581 admin admin] PATH_INFO: `/v3/projects/a692fbd7cb6f425bb640cae1fa3724b5/users/9e3f3d1d44284a83ac6c0be75af7088e/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.630684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737b541-213c-41fc-aa0e-0913450df581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a692fbd7cb6f425bb640cae1fa3724b5, user_id=9e3f3d1d44284a83ac6c0be75af7088e, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:44.633644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737b541-213c-41fc-aa0e-0913450df581 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.652594 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3344/6602] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:44 2022] PUT /identity/v3/projects/a692fbd7cb6f425bb640cae1fa3724b5/users/9e3f3d1d44284a83ac6c0be75af7088e/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:44.656214 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3259/6603] 10.222.0.23 () {58 vars in 1356 bytes} [Sat Aug 20 21:21:44 2022] GET /identity/v3/role_assignments?scope.project.id=26d543a25e214530b11e5701487f2dd0&user.id=b7b2379860f4465aab4eb3399d8e5b25&effective => generated 228 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:44.657476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737b541-213c-41fc-aa0e-0913450df581 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.662618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727ae4d-a11f-4a57-971e-8c35228f4345 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.663256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727ae4d-a11f-4a57-971e-8c35228f4345 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.663767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727ae4d-a11f-4a57-971e-8c35228f4345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.694669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d727ae4d-a11f-4a57-971e-8c35228f4345 None None] MFA Rules not processed for user `9e3f3d1d44284a83ac6c0be75af7088e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:44.695907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d727ae4d-a11f-4a57-971e-8c35228f4345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:44.728922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737b541-213c-41fc-aa0e-0913450df581 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.733913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.734588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.735002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.735285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/projects/26d543a25e214530b11e5701487f2dd0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.736664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:delete_project(project_id=26d543a25e214530b11e5701487f2dd0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:44.739114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.773323 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3260/6604] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:44.784255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727ae4d-a11f-4a57-971e-8c35228f4345 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.785720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d727ae4d-a11f-4a57-971e-8c35228f4345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:44.796235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26d543a25e214530b11e5701487f2dd0', 'request_id': 'req-40ded289-1448-4a03-a33a-76735a9f8422'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:44.806263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26d543a25e214530b11e5701487f2dd0', 'request_id': 'req-40ded289-1448-4a03-a33a-76735a9f8422'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:44.813971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26d543a25e214530b11e5701487f2dd0', 'request_id': 'req-40ded289-1448-4a03-a33a-76735a9f8422'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:44.828359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] The token cache is being invalidate because project 26d543a25e214530b11e5701487f2dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:44.828626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:44.829414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:44.831388 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3345/6605] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:44 2022] DELETE /identity/v3/projects/26d543a25e214530b11e5701487f2dd0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:44.835273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.836790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:44.843668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:44.872208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727ae4d-a11f-4a57-971e-8c35228f4345 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.886310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:44.915264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ded289-1448-4a03-a33a-76735a9f8422 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:44.920726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fdb8b-ef79-4426-916d-3ba7936e1650 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.921561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fdb8b-ef79-4426-916d-3ba7936e1650 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.922201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fdb8b-ef79-4426-916d-3ba7936e1650 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.922488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fdb8b-ef79-4426-916d-3ba7936e1650 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/roles/74ef352eb7d5484d872e8460ea122323` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.923945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fdb8b-ef79-4426-916d-3ba7936e1650 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:delete_role(role_id=74ef352eb7d5484d872e8460ea122323)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:44.925697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fdb8b-ef79-4426-916d-3ba7936e1650 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:44.946944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aea7b-cfab-4811-b655-5d4a5ad6f539 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:44.947558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aea7b-cfab-4811-b655-5d4a5ad6f539 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:44.947830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aea7b-cfab-4811-b655-5d4a5ad6f539 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:44.948151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aea7b-cfab-4811-b655-5d4a5ad6f539 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:44.949962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-290aea7b-cfab-4811-b655-5d4a5ad6f539 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:44.965277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-991fdb8b-ef79-4426-916d-3ba7936e1650 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invalidating the token cache because role 74ef352eb7d5484d872e8460ea122323 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:44.965610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-991fdb8b-ef79-4426-916d-3ba7936e1650 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:44.966239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-991fdb8b-ef79-4426-916d-3ba7936e1650 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:44.968336 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3346/6606] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:44 2022] DELETE /identity/v3/roles/74ef352eb7d5484d872e8460ea122323 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:44.973223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fdb8b-ef79-4426-916d-3ba7936e1650 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:44.975249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-991fdb8b-ef79-4426-916d-3ba7936e1650 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:45.021770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290aea7b-cfab-4811-b655-5d4a5ad6f539 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:45.023964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290aea7b-cfab-4811-b655-5d4a5ad6f539 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:45.036871 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3261/6607] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:45.059277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fdb8b-ef79-4426-916d-3ba7936e1650 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:45.065627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858c38d-4dfb-4670-abfb-6356164d3cb5 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:45.065935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858c38d-4dfb-4670-abfb-6356164d3cb5 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:45.066259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858c38d-4dfb-4670-abfb-6356164d3cb5 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:45.066607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858c38d-4dfb-4670-abfb-6356164d3cb5 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:45.067489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858c38d-4dfb-4670-abfb-6356164d3cb5 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:45.070125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858c38d-4dfb-4670-abfb-6356164d3cb5 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:45.094549 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3347/6608] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Aug 20 21:21:44 2022] POST /identity/v3/roles => generated 239 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:45.098648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aea7b-cfab-4811-b655-5d4a5ad6f539 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:45.117904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aea7b-cfab-4811-b655-5d4a5ad6f539 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:45.123230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47875a12-ca52-45a7-a102-609a5dfb9971 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:45.123936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47875a12-ca52-45a7-a102-609a5dfb9971 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:45.124284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47875a12-ca52-45a7-a102-609a5dfb9971 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:45.124655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47875a12-ca52-45a7-a102-609a5dfb9971 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:45.125716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47875a12-ca52-45a7-a102-609a5dfb9971 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:45.127702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47875a12-ca52-45a7-a102-609a5dfb9971 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:45.177255 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3262/6609] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:45 2022] POST /identity/v3/projects => generated 409 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:45.178306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858c38d-4dfb-4670-abfb-6356164d3cb5 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:45.190092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858c38d-4dfb-4670-abfb-6356164d3cb5 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:45.277208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ecb35-f1a9-453c-8c8a-05ff07aa3fba tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:45.277208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ecb35-f1a9-453c-8c8a-05ff07aa3fba tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:45.277608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ecb35-f1a9-453c-8c8a-05ff07aa3fba tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:45.277948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ecb35-f1a9-453c-8c8a-05ff07aa3fba tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/users/b7b2379860f4465aab4eb3399d8e5b25/roles/861c96743c1d4f73b1ce006ec564b3ad/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:45.278151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47875a12-ca52-45a7-a102-609a5dfb9971 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:45.279874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-47875a12-ca52-45a7-a102-609a5dfb9971 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:45.280425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ecb35-f1a9-453c-8c8a-05ff07aa3fba tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:create_grant(project_id=ae1fae1af07f4195a6c2a570f00eeb66, user_id=b7b2379860f4465aab4eb3399d8e5b25, role_id=861c96743c1d4f73b1ce006ec564b3ad)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:45.283157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ecb35-f1a9-453c-8c8a-05ff07aa3fba tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:45.296926 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3348/6610] 10.222.0.23 () {60 vars in 1582 bytes} [Sat Aug 20 21:21:45 2022] PUT /identity/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/users/b7b2379860f4465aab4eb3399d8e5b25/roles/861c96743c1d4f73b1ce006ec564b3ad/inherited_to_projects => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:45.301524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ecb35-f1a9-453c-8c8a-05ff07aa3fba tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:45.332252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:45.348927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47875a12-ca52-45a7-a102-609a5dfb9971 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:45.352780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:45.367293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ecb35-f1a9-453c-8c8a-05ff07aa3fba tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:45.373318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776b5ac-a8cd-4e22-90f1-638840b96c8b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:45.373929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776b5ac-a8cd-4e22-90f1-638840b96c8b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:45.374208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776b5ac-a8cd-4e22-90f1-638840b96c8b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:45.374478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776b5ac-a8cd-4e22-90f1-638840b96c8b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:45.375228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776b5ac-a8cd-4e22-90f1-638840b96c8b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Adding query filter params (scope.project.id=7eeff2339f204d98af0142bb575314b4, user.id=b7b2379860f4465aab4eb3399d8e5b25) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:45.375529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776b5ac-a8cd-4e22-90f1-638840b96c8b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:45.377837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776b5ac-a8cd-4e22-90f1-638840b96c8b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:45.411476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb0831-06f2-408a-855c-81ccaad2760f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:45.412279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb0831-06f2-408a-855c-81ccaad2760f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:45.412594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb0831-06f2-408a-855c-81ccaad2760f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:45.412866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb0831-06f2-408a-855c-81ccaad2760f admin admin] PATH_INFO: `/v3/users/9e3f3d1d44284a83ac6c0be75af7088e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:45.414015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb0831-06f2-408a-855c-81ccaad2760f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e3f3d1d44284a83ac6c0be75af7088e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:45.415930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb0831-06f2-408a-855c-81ccaad2760f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:45.434880 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3349/6611] 10.222.0.23 () {58 vars in 1356 bytes} [Sat Aug 20 21:21:45 2022] GET /identity/v3/role_assignments?scope.project.id=7eeff2339f204d98af0142bb575314b4&user.id=b7b2379860f4465aab4eb3399d8e5b25&effective => generated 649 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:45.435811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776b5ac-a8cd-4e22-90f1-638840b96c8b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:45.446778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776b5ac-a8cd-4e22-90f1-638840b96c8b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:45.451927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134b36a-1bcf-4ae6-967d-ef441708871d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:45.452622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134b36a-1bcf-4ae6-967d-ef441708871d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:45.452905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134b36a-1bcf-4ae6-967d-ef441708871d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:45.453214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134b36a-1bcf-4ae6-967d-ef441708871d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/users/b7b2379860f4465aab4eb3399d8e5b25/roles/861c96743c1d4f73b1ce006ec564b3ad/inherited_to_projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:45.455573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134b36a-1bcf-4ae6-967d-ef441708871d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=ae1fae1af07f4195a6c2a570f00eeb66, user_id=b7b2379860f4465aab4eb3399d8e5b25, role_id=861c96743c1d4f73b1ce006ec564b3ad)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:45.458304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134b36a-1bcf-4ae6-967d-ef441708871d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:45.464412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3134b36a-1bcf-4ae6-967d-ef441708871d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invalidating the token cache because role 861c96743c1d4f73b1ce006ec564b3ad was removed from user b7b2379860f4465aab4eb3399d8e5b25 on project ae1fae1af07f4195a6c2a570f00eeb66. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:45.464697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3134b36a-1bcf-4ae6-967d-ef441708871d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.465371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3134b36a-1bcf-4ae6-967d-ef441708871d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.467617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6dbb0831-06f2-408a-855c-81ccaad2760f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e3f3d1d44284a83ac6c0be75af7088e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.476487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6dbb0831-06f2-408a-855c-81ccaad2760f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e3f3d1d44284a83ac6c0be75af7088e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.478175 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3350/6612] 10.222.0.23 () {58 vars in 1566 bytes} [Sat Aug 20 21:21:45 2022] DELETE /identity/v3/OS-INHERIT/projects/ae1fae1af07f4195a6c2a570f00eeb66/users/b7b2379860f4465aab4eb3399d8e5b25/roles/861c96743c1d4f73b1ce006ec564b3ad/inherited_to_projects => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:45.485026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134b36a-1bcf-4ae6-967d-ef441708871d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:45.487077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3134b36a-1bcf-4ae6-967d-ef441708871d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:45.491918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6dbb0831-06f2-408a-855c-81ccaad2760f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e3f3d1d44284a83ac6c0be75af7088e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.492731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6dbb0831-06f2-408a-855c-81ccaad2760f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e3f3d1d44284a83ac6c0be75af7088e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.493414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6dbb0831-06f2-408a-855c-81ccaad2760f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e3f3d1d44284a83ac6c0be75af7088e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.508878 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3263/6613] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:45 2022] DELETE /identity/v3/users/9e3f3d1d44284a83ac6c0be75af7088e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:45.513313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e7576-b504-46af-bf88-556235f64934 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:45.513685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e7576-b504-46af-bf88-556235f64934 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:45.514066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e7576-b504-46af-bf88-556235f64934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:45.533917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-638e7576-b504-46af-bf88-556235f64934 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:45.535310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-638e7576-b504-46af-bf88-556235f64934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:45.560144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134b36a-1bcf-4ae6-967d-ef441708871d tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:45.597527 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3264/6614] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:45.608143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e7576-b504-46af-bf88-556235f64934 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:45.609799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-638e7576-b504-46af-bf88-556235f64934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:45.622321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24320779-eb93-46a9-8a0a-2057a5543c02 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:45.623324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24320779-eb93-46a9-8a0a-2057a5543c02 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:45.623440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24320779-eb93-46a9-8a0a-2057a5543c02 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:45.623683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24320779-eb93-46a9-8a0a-2057a5543c02 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:45.624426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24320779-eb93-46a9-8a0a-2057a5543c02 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Adding query filter params (scope.project.id=7eeff2339f204d98af0142bb575314b4, user.id=b7b2379860f4465aab4eb3399d8e5b25) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:45.624738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24320779-eb93-46a9-8a0a-2057a5543c02 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:45.626609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24320779-eb93-46a9-8a0a-2057a5543c02 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:45.638519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:45.647666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:45.682320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e7576-b504-46af-bf88-556235f64934 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:45.686894 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3351/6615] 10.222.0.23 () {58 vars in 1356 bytes} [Sat Aug 20 21:21:45 2022] GET /identity/v3/role_assignments?scope.project.id=7eeff2339f204d98af0142bb575314b4&user.id=b7b2379860f4465aab4eb3399d8e5b25&effective => generated 228 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:45.687026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b641f10-30a0-4b84-8cee-b32928e4a522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:45.687246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b641f10-30a0-4b84-8cee-b32928e4a522 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:45.687485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b641f10-30a0-4b84-8cee-b32928e4a522 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:45.687756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b641f10-30a0-4b84-8cee-b32928e4a522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:45.687877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24320779-eb93-46a9-8a0a-2057a5543c02 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:45.689617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-24320779-eb93-46a9-8a0a-2057a5543c02 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:45.697728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b641f10-30a0-4b84-8cee-b32928e4a522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:45.699158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b641f10-30a0-4b84-8cee-b32928e4a522 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:45.702573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24320779-eb93-46a9-8a0a-2057a5543c02 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:45.708222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:45.708752 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3265/6616] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:45.709523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:45.709999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:45.710364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/projects/7eeff2339f204d98af0142bb575314b4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:45.711587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7eeff2339f204d98af0142bb575314b4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:45.713525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:45.764852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eeff2339f204d98af0142bb575314b4', 'request_id': 'req-cbe0c591-e3da-43b2-b875-a728337ad17b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.773495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eeff2339f204d98af0142bb575314b4', 'request_id': 'req-cbe0c591-e3da-43b2-b875-a728337ad17b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.779938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eeff2339f204d98af0142bb575314b4', 'request_id': 'req-cbe0c591-e3da-43b2-b875-a728337ad17b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.791581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] The token cache is being invalidate because project 7eeff2339f204d98af0142bb575314b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:45.791866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.792562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.794143 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3352/6617] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:45 2022] DELETE /identity/v3/projects/7eeff2339f204d98af0142bb575314b4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:45.798509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b641f10-30a0-4b84-8cee-b32928e4a522 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:45.809516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b641f10-30a0-4b84-8cee-b32928e4a522 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:45.880569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e772e-9168-4b9f-9da8-8a03062b66ab tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:45.881736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e772e-9168-4b9f-9da8-8a03062b66ab tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:45.882160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e772e-9168-4b9f-9da8-8a03062b66ab tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:45.882431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e772e-9168-4b9f-9da8-8a03062b66ab tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/roles/861c96743c1d4f73b1ce006ec564b3ad` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:45.883951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e772e-9168-4b9f-9da8-8a03062b66ab tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:delete_role(role_id=861c96743c1d4f73b1ce006ec564b3ad)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:45.885592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e772e-9168-4b9f-9da8-8a03062b66ab tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:45.931934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3d7e772e-9168-4b9f-9da8-8a03062b66ab tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invalidating the token cache because role 861c96743c1d4f73b1ce006ec564b3ad has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:45.932395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3d7e772e-9168-4b9f-9da8-8a03062b66ab tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.933251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3d7e772e-9168-4b9f-9da8-8a03062b66ab tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:45.935295 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3266/6618] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:45 2022] DELETE /identity/v3/roles/861c96743c1d4f73b1ce006ec564b3ad => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:45.940880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:45.942450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:45.958881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0c591-e3da-43b2-b875-a728337ad17b tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:46.022413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e772e-9168-4b9f-9da8-8a03062b66ab tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:46.023985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3d7e772e-9168-4b9f-9da8-8a03062b66ab tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:46.035964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:46.036872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.037147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.037532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/users/b7b2379860f4465aab4eb3399d8e5b25` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.038622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:delete_user(user_id=b7b2379860f4465aab4eb3399d8e5b25)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:46.041482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:46.090401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7b2379860f4465aab4eb3399d8e5b25'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.094073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:46.097294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e772e-9168-4b9f-9da8-8a03062b66ab tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:46.098959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7b2379860f4465aab4eb3399d8e5b25'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.102441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3b820-dab5-4b17-afb2-58e50757143f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:46.103103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3b820-dab5-4b17-afb2-58e50757143f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.103371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3b820-dab5-4b17-afb2-58e50757143f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.103636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3b820-dab5-4b17-afb2-58e50757143f admin admin] PATH_INFO: `/v3/projects/a692fbd7cb6f425bb640cae1fa3724b5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.104807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3b820-dab5-4b17-afb2-58e50757143f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a692fbd7cb6f425bb640cae1fa3724b5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:46.106786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3b820-dab5-4b17-afb2-58e50757143f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:46.108811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7b2379860f4465aab4eb3399d8e5b25'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.127096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7b2379860f4465aab4eb3399d8e5b25'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.127999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7b2379860f4465aab4eb3399d8e5b25'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.134918 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3353/6619] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:45 2022] DELETE /identity/v3/users/b7b2379860f4465aab4eb3399d8e5b25 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:46.139371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:46.140961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:46.152268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:46.163200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a9d3b820-dab5-4b17-afb2-58e50757143f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a692fbd7cb6f425bb640cae1fa3724b5', 'request_id': 'req-a9d3b820-dab5-4b17-afb2-58e50757143f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.170066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a9d3b820-dab5-4b17-afb2-58e50757143f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a692fbd7cb6f425bb640cae1fa3724b5', 'request_id': 'req-a9d3b820-dab5-4b17-afb2-58e50757143f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.180821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a9d3b820-dab5-4b17-afb2-58e50757143f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a692fbd7cb6f425bb640cae1fa3724b5', 'request_id': 'req-a9d3b820-dab5-4b17-afb2-58e50757143f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.193030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a9d3b820-dab5-4b17-afb2-58e50757143f admin admin] The token cache is being invalidate because project a692fbd7cb6f425bb640cae1fa3724b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:46.193613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a9d3b820-dab5-4b17-afb2-58e50757143f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.194743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a9d3b820-dab5-4b17-afb2-58e50757143f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.196569 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3267/6620] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:46 2022] DELETE /identity/v3/projects/a692fbd7cb6f425bb640cae1fa3724b5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:46.201235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec6e40-7551-4c51-8329-b3598dcd5a31 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.201510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec6e40-7551-4c51-8329-b3598dcd5a31 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.201778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec6e40-7551-4c51-8329-b3598dcd5a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.215539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7a0ef-9042-485c-bc42-969ea227fff6 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:46.221618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-95ec6e40-7551-4c51-8329-b3598dcd5a31 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:46.222804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-95ec6e40-7551-4c51-8329-b3598dcd5a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:46.280716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5274430-5307-40fe-bd21-bced42cd6e61 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:46.281331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5274430-5307-40fe-bd21-bced42cd6e61 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.281604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5274430-5307-40fe-bd21-bced42cd6e61 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.281866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5274430-5307-40fe-bd21-bced42cd6e61 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/groups/13f06b3cce4945b1987c5b942b37a61a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.283017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5274430-5307-40fe-bd21-bced42cd6e61 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:delete_group(group_id=13f06b3cce4945b1987c5b942b37a61a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:46.284976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5274430-5307-40fe-bd21-bced42cd6e61 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:46.287887 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3268/6621] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:46.295631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3a8f0-08f1-41f5-a41a-2b7119f46b0e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.295913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3a8f0-08f1-41f5-a41a-2b7119f46b0e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.296186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3a8f0-08f1-41f5-a41a-2b7119f46b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.316651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e0f3a8f0-08f1-41f5-a41a-2b7119f46b0e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:46.317950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e0f3a8f0-08f1-41f5-a41a-2b7119f46b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:46.326338 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3269/6622] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:46.329715 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3354/6623] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:21:46 2022] DELETE /identity/v3/groups/13f06b3cce4945b1987c5b942b37a61a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:46.330813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3a8f0-08f1-41f5-a41a-2b7119f46b0e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:46.333369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5274430-5307-40fe-bd21-bced42cd6e61 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:46.335378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a5274430-5307-40fe-bd21-bced42cd6e61 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:46.340267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3a8f0-08f1-41f5-a41a-2b7119f46b0e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:46.375879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:46.396366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655edf7-164d-42af-b593-7811b47fa00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:46.397009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655edf7-164d-42af-b593-7811b47fa00b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.397281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655edf7-164d-42af-b593-7811b47fa00b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.397545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655edf7-164d-42af-b593-7811b47fa00b admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.398188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655edf7-164d-42af-b593-7811b47fa00b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:46.398489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655edf7-164d-42af-b593-7811b47fa00b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:46.400182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655edf7-164d-42af-b593-7811b47fa00b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:46.401321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5274430-5307-40fe-bd21-bced42cd6e61 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:46.405885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:46.406511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.406836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.407045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/projects/ae1fae1af07f4195a6c2a570f00eeb66` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.408165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ae1fae1af07f4195a6c2a570f00eeb66)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:46.410100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:46.415292 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3270/6624] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:46.416293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655edf7-164d-42af-b593-7811b47fa00b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:46.426636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655edf7-164d-42af-b593-7811b47fa00b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:46.431708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb9f53-57a3-40db-837e-e7a5d81bc599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:46.432292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb9f53-57a3-40db-837e-e7a5d81bc599 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.432607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb9f53-57a3-40db-837e-e7a5d81bc599 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.432879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb9f53-57a3-40db-837e-e7a5d81bc599 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.433493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb9f53-57a3-40db-837e-e7a5d81bc599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:46.435415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb9f53-57a3-40db-837e-e7a5d81bc599 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:46.459726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae1fae1af07f4195a6c2a570f00eeb66', 'request_id': 'req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.472001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae1fae1af07f4195a6c2a570f00eeb66', 'request_id': 'req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.472238 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3271/6625] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:46 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:46.476715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb9f53-57a3-40db-837e-e7a5d81bc599 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:46.480959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae1fae1af07f4195a6c2a570f00eeb66', 'request_id': 'req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.499796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] The token cache is being invalidate because project ae1fae1af07f4195a6c2a570f00eeb66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:46.500090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.500873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.503179 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3355/6626] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:46 2022] DELETE /identity/v3/projects/ae1fae1af07f4195a6c2a570f00eeb66 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:46.507132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:46.508603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:46.520810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:46.559797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb9f53-57a3-40db-837e-e7a5d81bc599 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:46.564563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e36068-2881-4969-bec8-3a9bc88c73fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:46.565171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e36068-2881-4969-bec8-3a9bc88c73fa admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.565444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e36068-2881-4969-bec8-3a9bc88c73fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.565709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e36068-2881-4969-bec8-3a9bc88c73fa admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.566344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e36068-2881-4969-bec8-3a9bc88c73fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:46.568752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e36068-2881-4969-bec8-3a9bc88c73fa admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:46.577068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd1f5f-e36f-4d68-a40b-2e365eef726a tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:46.595126 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3272/6627] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:46 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:46.596132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e36068-2881-4969-bec8-3a9bc88c73fa admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:46.597820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f4e36068-2881-4969-bec8-3a9bc88c73fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:46.610120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e36068-2881-4969-bec8-3a9bc88c73fa admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:46.614796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cc1d5-049e-4de4-ab3e-c9631e50ed68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:46.615588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cc1d5-049e-4de4-ab3e-c9631e50ed68 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.615758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cc1d5-049e-4de4-ab3e-c9631e50ed68 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.616016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cc1d5-049e-4de4-ab3e-c9631e50ed68 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.616679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cc1d5-049e-4de4-ab3e-c9631e50ed68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:46.618306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cc1d5-049e-4de4-ab3e-c9631e50ed68 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:46.628399 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3273/6628] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:46.632821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cc1d5-049e-4de4-ab3e-c9631e50ed68 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:46.639790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41ff4d-8667-4a2a-9b70-f335dec0d448 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:46.640481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41ff4d-8667-4a2a-9b70-f335dec0d448 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.640759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41ff4d-8667-4a2a-9b70-f335dec0d448 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.641029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41ff4d-8667-4a2a-9b70-f335dec0d448 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/domains/ea2d792531ea4de0a70794e75e951f79` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.642376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41ff4d-8667-4a2a-9b70-f335dec0d448 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=ea2d792531ea4de0a70794e75e951f79)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:46.642493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cc1d5-049e-4de4-ab3e-c9631e50ed68 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:46.644006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41ff4d-8667-4a2a-9b70-f335dec0d448 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:46.647368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7955ce2-f7dd-4d88-b7ea-0c3901fe5915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:46.647990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7955ce2-f7dd-4d88-b7ea-0c3901fe5915 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.648237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7955ce2-f7dd-4d88-b7ea-0c3901fe5915 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.648536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7955ce2-f7dd-4d88-b7ea-0c3901fe5915 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.649172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7955ce2-f7dd-4d88-b7ea-0c3901fe5915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:46.650755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7955ce2-f7dd-4d88-b7ea-0c3901fe5915 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:46.661406 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3274/6629] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:46.663851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:46.667434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7955ce2-f7dd-4d88-b7ea-0c3901fe5915 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:46.668432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9e41ff4d-8667-4a2a-9b70-f335dec0d448 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'ea2d792531ea4de0a70794e75e951f79'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.669460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9e41ff4d-8667-4a2a-9b70-f335dec0d448 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'ea2d792531ea4de0a70794e75e951f79'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.679401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7955ce2-f7dd-4d88-b7ea-0c3901fe5915 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:46.702361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9e41ff4d-8667-4a2a-9b70-f335dec0d448 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'ea2d792531ea4de0a70794e75e951f79'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.703138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9e41ff4d-8667-4a2a-9b70-f335dec0d448 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'ea2d792531ea4de0a70794e75e951f79'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.705965 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3356/6630] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:21:46 2022] PATCH /identity/v3/domains/ea2d792531ea4de0a70794e75e951f79 => generated 277 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:46.710084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41ff4d-8667-4a2a-9b70-f335dec0d448 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:46.711430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9e41ff4d-8667-4a2a-9b70-f335dec0d448 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:46.743336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f81659-8c59-48ec-bb16-f602f342e54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:46.744135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f81659-8c59-48ec-bb16-f602f342e54e admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.744505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f81659-8c59-48ec-bb16-f602f342e54e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.744929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f81659-8c59-48ec-bb16-f602f342e54e admin admin] PATH_INFO: `/v3/projects/a8358b7d0c724ef68af314a9962b06b8/users/a95c7ee657cb4d9eba2773ac55db8c0b/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.760323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f81659-8c59-48ec-bb16-f602f342e54e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8358b7d0c724ef68af314a9962b06b8, user_id=a95c7ee657cb4d9eba2773ac55db8c0b, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:46.763126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f81659-8c59-48ec-bb16-f602f342e54e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:46.778980 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3275/6631] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:46 2022] PUT /identity/v3/projects/a8358b7d0c724ef68af314a9962b06b8/users/a95c7ee657cb4d9eba2773ac55db8c0b/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:46.789260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a4baf-3ae8-4f7c-b990-b529d7cd0380 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.789539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a4baf-3ae8-4f7c-b990-b529d7cd0380 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.789804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a4baf-3ae8-4f7c-b990-b529d7cd0380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.796616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41ff4d-8667-4a2a-9b70-f335dec0d448 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:46.823530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-663a4baf-3ae8-4f7c-b990-b529d7cd0380 None None] MFA Rules not processed for user `a95c7ee657cb4d9eba2773ac55db8c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:46.824894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-663a4baf-3ae8-4f7c-b990-b529d7cd0380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:46.876065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da1146d638ab42cdadcba16d5365ef9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd670bba78f784854b63a071cfd797410', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:46.876840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:46.877114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:46.877389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] PATH_INFO: `/v3/domains/ea2d792531ea4de0a70794e75e951f79` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:46.889865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=ea2d792531ea4de0a70794e75e951f79)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:46.891561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:46.911724 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3276/6632] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:46.912878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a4baf-3ae8-4f7c-b990-b529d7cd0380 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:46.914282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-663a4baf-3ae8-4f7c-b990-b529d7cd0380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:46.917671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'ea2d792531ea4de0a70794e75e951f79'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.927929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'ea2d792531ea4de0a70794e75e951f79'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.929506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:46.982493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea2d792531ea4de0a70794e75e951f79', 'request_id': 'req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:46.987578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:UPDATE=1,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:46.994171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea2d792531ea4de0a70794e75e951f79', 'request_id': 'req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:47.000924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a4baf-3ae8-4f7c-b990-b529d7cd0380 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.001986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea2d792531ea4de0a70794e75e951f79', 'request_id': 'req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:47.021326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] The token cache is being invalidate because project ea2d792531ea4de0a70794e75e951f79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:47.021827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:47.022942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:47.038146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'ea2d792531ea4de0a70794e75e951f79', 'request_id': 'req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:47.039040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'ea2d792531ea4de0a70794e75e951f79', 'request_id': 'req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:47.043679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'ea2d792531ea4de0a70794e75e951f79', 'request_id': 'req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:47.050103 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3357/6633] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:46 2022] DELETE /identity/v3/domains/ea2d792531ea4de0a70794e75e951f79 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:47.054102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.055602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:47.077960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881b59c-8361-4050-8a9d-caa8d09f0651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.078609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881b59c-8361-4050-8a9d-caa8d09f0651 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.078881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881b59c-8361-4050-8a9d-caa8d09f0651 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.079147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881b59c-8361-4050-8a9d-caa8d09f0651 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.079843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881b59c-8361-4050-8a9d-caa8d09f0651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.082053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881b59c-8361-4050-8a9d-caa8d09f0651 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.116260 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3277/6634] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:46 2022] POST /identity/v3/projects => generated 389 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:47.120297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881b59c-8361-4050-8a9d-caa8d09f0651 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.122122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b881b59c-8361-4050-8a9d-caa8d09f0651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:47.132308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90358a87-b3ed-46e9-86ce-efbcf5ba89f8 tempest-InheritsV3TestJSON-1592390601 tempest-InheritsV3TestJSON-1592390601-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.199681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881b59c-8361-4050-8a9d-caa8d09f0651 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.202246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e1476-064d-4940-8e42-2c428a9a5a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.202943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e1476-064d-4940-8e42-2c428a9a5a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.203247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e1476-064d-4940-8e42-2c428a9a5a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.203516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e1476-064d-4940-8e42-2c428a9a5a9c admin admin] PATH_INFO: `/v3/users/b96a9c7e564b43dc8b0e0f59923cc51c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.204465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe6802-2652-4ec7-9cfe-f78a7cacc4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.204726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e1476-064d-4940-8e42-2c428a9a5a9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b96a9c7e564b43dc8b0e0f59923cc51c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.205067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe6802-2652-4ec7-9cfe-f78a7cacc4cf admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.205337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe6802-2652-4ec7-9cfe-f78a7cacc4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.205594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe6802-2652-4ec7-9cfe-f78a7cacc4cf admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.206202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe6802-2652-4ec7-9cfe-f78a7cacc4cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.206706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e1476-064d-4940-8e42-2c428a9a5a9c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.208123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe6802-2652-4ec7-9cfe-f78a7cacc4cf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.235654 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3278/6635] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:47 2022] POST /identity/v3/users => generated 301 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:47.237321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe6802-2652-4ec7-9cfe-f78a7cacc4cf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.248449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe6802-2652-4ec7-9cfe-f78a7cacc4cf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.253336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d60087-b5e6-447d-b9bb-eb79d8c27579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.253936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d60087-b5e6-447d-b9bb-eb79d8c27579 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.254198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d60087-b5e6-447d-b9bb-eb79d8c27579 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.254461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d60087-b5e6-447d-b9bb-eb79d8c27579 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.255092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d60087-b5e6-447d-b9bb-eb79d8c27579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.256830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d60087-b5e6-447d-b9bb-eb79d8c27579 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.263174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-770e1476-064d-4940-8e42-2c428a9a5a9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b96a9c7e564b43dc8b0e0f59923cc51c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:47.271153 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3279/6636] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:47.271410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-770e1476-064d-4940-8e42-2c428a9a5a9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b96a9c7e564b43dc8b0e0f59923cc51c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:47.272452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d60087-b5e6-447d-b9bb-eb79d8c27579 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.279234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-770e1476-064d-4940-8e42-2c428a9a5a9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b96a9c7e564b43dc8b0e0f59923cc51c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:47.282318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d60087-b5e6-447d-b9bb-eb79d8c27579 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.286162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:47.287096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04ada9-0217-4586-99b0-b5c09746d7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.287741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04ada9-0217-4586-99b0-b5c09746d7cf admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.288015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04ada9-0217-4586-99b0-b5c09746d7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.288278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04ada9-0217-4586-99b0-b5c09746d7cf admin admin] PATH_INFO: `/v3/projects/7afd2624d27b458a9daf105b30949fc4/users/7f82a4ad32804c89b962a0d0a5810bcf/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.295724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-770e1476-064d-4940-8e42-2c428a9a5a9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b96a9c7e564b43dc8b0e0f59923cc51c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:47.296642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-770e1476-064d-4940-8e42-2c428a9a5a9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b96a9c7e564b43dc8b0e0f59923cc51c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:47.303156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04ada9-0217-4586-99b0-b5c09746d7cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afd2624d27b458a9daf105b30949fc4, user_id=7f82a4ad32804c89b962a0d0a5810bcf, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.303671 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3358/6637] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:47 2022] DELETE /identity/v3/users/b96a9c7e564b43dc8b0e0f59923cc51c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:47.305917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04ada9-0217-4586-99b0-b5c09746d7cf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.308120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739f3a0-5637-46b2-bbb3-753c0b37b701 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.308419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739f3a0-5637-46b2-bbb3-753c0b37b701 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.308692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739f3a0-5637-46b2-bbb3-753c0b37b701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.315117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:47.322237 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3280/6638] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:47 2022] PUT /identity/v3/projects/7afd2624d27b458a9daf105b30949fc4/users/7f82a4ad32804c89b962a0d0a5810bcf/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:47.326848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04ada9-0217-4586-99b0-b5c09746d7cf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.328978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ec04ada9-0217-4586-99b0-b5c09746d7cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:47.329602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5739f3a0-5637-46b2-bbb3-753c0b37b701 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:47.330766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5739f3a0-5637-46b2-bbb3-753c0b37b701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:47.390633 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3359/6639] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:47.393894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04ada9-0217-4586-99b0-b5c09746d7cf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.398913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21692d-e60e-4ecf-878f-7fea1202f458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.399520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21692d-e60e-4ecf-878f-7fea1202f458 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.399820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21692d-e60e-4ecf-878f-7fea1202f458 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.400046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21692d-e60e-4ecf-878f-7fea1202f458 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.400712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21692d-e60e-4ecf-878f-7fea1202f458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.401071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739f3a0-5637-46b2-bbb3-753c0b37b701 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.402352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21692d-e60e-4ecf-878f-7fea1202f458 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.402460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5739f3a0-5637-46b2-bbb3-753c0b37b701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:47.414353 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3281/6640] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:47.417697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21692d-e60e-4ecf-878f-7fea1202f458 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.428419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21692d-e60e-4ecf-878f-7fea1202f458 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.433115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af7c45-4677-4633-bb17-09b65780fc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.433758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af7c45-4677-4633-bb17-09b65780fc1f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.434030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af7c45-4677-4633-bb17-09b65780fc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.434553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af7c45-4677-4633-bb17-09b65780fc1f admin admin] PATH_INFO: `/v3/projects/7afd2624d27b458a9daf105b30949fc4/users/7f82a4ad32804c89b962a0d0a5810bcf/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.436233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af7c45-4677-4633-bb17-09b65780fc1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afd2624d27b458a9daf105b30949fc4, user_id=7f82a4ad32804c89b962a0d0a5810bcf, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.438896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af7c45-4677-4633-bb17-09b65780fc1f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.452723 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3282/6641] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:47 2022] PUT /identity/v3/projects/7afd2624d27b458a9daf105b30949fc4/users/7f82a4ad32804c89b962a0d0a5810bcf/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:47.465552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2513dfb-3631-47ba-b480-435da54913bd None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.465797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2513dfb-3631-47ba-b480-435da54913bd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.466043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2513dfb-3631-47ba-b480-435da54913bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.471058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739f3a0-5637-46b2-bbb3-753c0b37b701 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.496202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a2513dfb-3631-47ba-b480-435da54913bd None None] MFA Rules not processed for user `7f82a4ad32804c89b962a0d0a5810bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:47.497544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a2513dfb-3631-47ba-b480-435da54913bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:47.530685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be5f06-c189-49d6-af66-4d03f476417e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.531361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be5f06-c189-49d6-af66-4d03f476417e service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.531589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be5f06-c189-49d6-af66-4d03f476417e service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.531868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be5f06-c189-49d6-af66-4d03f476417e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.574647 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3283/6642] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:47.579151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2513dfb-3631-47ba-b480-435da54913bd None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.588596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2513dfb-3631-47ba-b480-435da54913bd None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.593294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a6145-f00a-441d-8178-3fee4172ff52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f82a4ad32804c89b962a0d0a5810bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7afd2624d27b458a9daf105b30949fc4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.593910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a6145-f00a-441d-8178-3fee4172ff52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.594240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a6145-f00a-441d-8178-3fee4172ff52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.594468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a6145-f00a-441d-8178-3fee4172ff52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] PATH_INFO: `/v3/projects/a8358b7d0c724ef68af314a9962b06b8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.595410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:47.595751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79a6145-f00a-441d-8178-3fee4172ff52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorizing `identity:get_project(project_id=a8358b7d0c724ef68af314a9962b06b8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.597948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be5f06-c189-49d6-af66-4d03f476417e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.598132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79a6145-f00a-441d-8178-3fee4172ff52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.599537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be5f06-c189-49d6-af66-4d03f476417e service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.600127 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3284/6643] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:47 2022] GET /identity/v3/projects/a8358b7d0c724ef68af314a9962b06b8 => generated 389 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:47.603957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a6145-f00a-441d-8178-3fee4172ff52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.610869 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3360/6644] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:47.614443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a6145-f00a-441d-8178-3fee4172ff52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.620100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e69d0-d644-4636-9bd7-3c3e3effbfdb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f82a4ad32804c89b962a0d0a5810bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7afd2624d27b458a9daf105b30949fc4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.620922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e69d0-d644-4636-9bd7-3c3e3effbfdb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.621192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e69d0-d644-4636-9bd7-3c3e3effbfdb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.621455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e69d0-d644-4636-9bd7-3c3e3effbfdb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.622062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e69d0-d644-4636-9bd7-3c3e3effbfdb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.624006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e69d0-d644-4636-9bd7-3c3e3effbfdb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.660662 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3285/6645] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:21:47 2022] POST /identity/v3/projects => generated 339 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:47.662166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be5f06-c189-49d6-af66-4d03f476417e service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.674056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be5f06-c189-49d6-af66-4d03f476417e service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.737438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a8cea-0b55-4395-adea-0c8590d0433f tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f82a4ad32804c89b962a0d0a5810bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7afd2624d27b458a9daf105b30949fc4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.738059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a8cea-0b55-4395-adea-0c8590d0433f tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.738346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a8cea-0b55-4395-adea-0c8590d0433f tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.738626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a8cea-0b55-4395-adea-0c8590d0433f tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.739261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a8cea-0b55-4395-adea-0c8590d0433f tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.741288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a8cea-0b55-4395-adea-0c8590d0433f tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.773056 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3361/6646] 10.222.0.23 () {58 vars in 1131 bytes} [Sat Aug 20 21:21:47 2022] GET /identity/v3/projects => generated 6530 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:47.774099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e69d0-d644-4636-9bd7-3c3e3effbfdb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.785454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e69d0-d644-4636-9bd7-3c3e3effbfdb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.790605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49eec4-edb3-41d4-b9af-b7b302584b52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f82a4ad32804c89b962a0d0a5810bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7afd2624d27b458a9daf105b30949fc4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.791255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49eec4-edb3-41d4-b9af-b7b302584b52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.791533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49eec4-edb3-41d4-b9af-b7b302584b52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.791877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49eec4-edb3-41d4-b9af-b7b302584b52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] PATH_INFO: `/v3/projects/a8358b7d0c724ef68af314a9962b06b8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.792967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49eec4-edb3-41d4-b9af-b7b302584b52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorizing `identity:get_project(project_id=a8358b7d0c724ef68af314a9962b06b8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.795114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49eec4-edb3-41d4-b9af-b7b302584b52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.797013 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3286/6647] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:47 2022] GET /identity/v3/projects/a8358b7d0c724ef68af314a9962b06b8 => generated 389 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:47.800951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a8cea-0b55-4395-adea-0c8590d0433f tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.812223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a8cea-0b55-4395-adea-0c8590d0433f tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.817461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90960f81-8dea-47ab-bba6-81c836782213 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f82a4ad32804c89b962a0d0a5810bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7afd2624d27b458a9daf105b30949fc4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.818576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90960f81-8dea-47ab-bba6-81c836782213 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.818816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90960f81-8dea-47ab-bba6-81c836782213 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.819090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90960f81-8dea-47ab-bba6-81c836782213 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] PATH_INFO: `/v3/projects/c45064d1354043dcab6f565611d2d01f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.820628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90960f81-8dea-47ab-bba6-81c836782213 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorizing `identity:get_project(project_id=c45064d1354043dcab6f565611d2d01f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.822695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90960f81-8dea-47ab-bba6-81c836782213 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.824532 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3362/6648] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:47 2022] GET /identity/v3/projects/c45064d1354043dcab6f565611d2d01f => generated 339 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:47.829548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49eec4-edb3-41d4-b9af-b7b302584b52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.840482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49eec4-edb3-41d4-b9af-b7b302584b52 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.845850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316d860-b58a-47d7-ac3d-53f6fbbcdf2d tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f82a4ad32804c89b962a0d0a5810bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7afd2624d27b458a9daf105b30949fc4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.846485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316d860-b58a-47d7-ac3d-53f6fbbcdf2d tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.846752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316d860-b58a-47d7-ac3d-53f6fbbcdf2d tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.847035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316d860-b58a-47d7-ac3d-53f6fbbcdf2d tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.849321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316d860-b58a-47d7-ac3d-53f6fbbcdf2d tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.851254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316d860-b58a-47d7-ac3d-53f6fbbcdf2d tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.875098 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3287/6649] 10.222.0.23 () {58 vars in 1131 bytes} [Sat Aug 20 21:21:47 2022] GET /identity/v3/projects => generated 6530 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:47.876474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90960f81-8dea-47ab-bba6-81c836782213 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.887932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90960f81-8dea-47ab-bba6-81c836782213 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.893074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77992b6b-a136-499c-9689-3088fb4aa1e1 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f82a4ad32804c89b962a0d0a5810bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7afd2624d27b458a9daf105b30949fc4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.893835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77992b6b-a136-499c-9689-3088fb4aa1e1 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.894189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77992b6b-a136-499c-9689-3088fb4aa1e1 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.894455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77992b6b-a136-499c-9689-3088fb4aa1e1 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.895112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77992b6b-a136-499c-9689-3088fb4aa1e1 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:47.895405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77992b6b-a136-499c-9689-3088fb4aa1e1 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.897768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77992b6b-a136-499c-9689-3088fb4aa1e1 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.901013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316d860-b58a-47d7-ac3d-53f6fbbcdf2d tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.903845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b316d860-b58a-47d7-ac3d-53f6fbbcdf2d tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:47.927252 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3363/6650] 10.222.0.23 () {58 vars in 1166 bytes} [Sat Aug 20 21:21:47 2022] GET /identity/v3/projects?domain_id=default => generated 6199 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:47.931463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77992b6b-a136-499c-9689-3088fb4aa1e1 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.939999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:47.944998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77992b6b-a136-499c-9689-3088fb4aa1e1 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.951274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d0951-6dcb-481a-af21-9288c31629dc tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f82a4ad32804c89b962a0d0a5810bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7afd2624d27b458a9daf105b30949fc4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.952044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d0951-6dcb-481a-af21-9288c31629dc tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.952323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d0951-6dcb-481a-af21-9288c31629dc tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.952641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d0951-6dcb-481a-af21-9288c31629dc tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.953336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d0951-6dcb-481a-af21-9288c31629dc tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.955322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d0951-6dcb-481a-af21-9288c31629dc tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.974475 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3364/6651] 10.222.0.23 () {58 vars in 1131 bytes} [Sat Aug 20 21:21:47 2022] GET /identity/v3/projects => generated 6530 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:47.979261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d0951-6dcb-481a-af21-9288c31629dc tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:47.989260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d0951-6dcb-481a-af21-9288c31629dc tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.989933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316d860-b58a-47d7-ac3d-53f6fbbcdf2d tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:47.993990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09609e88-be2a-4a76-b0a3-d80a699030bb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f82a4ad32804c89b962a0d0a5810bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7afd2624d27b458a9daf105b30949fc4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.994590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09609e88-be2a-4a76-b0a3-d80a699030bb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.994867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09609e88-be2a-4a76-b0a3-d80a699030bb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.994961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:47.995137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09609e88-be2a-4a76-b0a3-d80a699030bb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.995528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:47.995796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09609e88-be2a-4a76-b0a3-d80a699030bb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:47.995900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:47.996059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] PATH_INFO: `/v3/projects/88e55c8bc2094b6b8e9381be96881cc1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:47.996149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09609e88-be2a-4a76-b0a3-d80a699030bb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.997266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e55c8bc2094b6b8e9381be96881cc1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:47.997986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09609e88-be2a-4a76-b0a3-d80a699030bb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:47.999229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:48.017459 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3365/6652] 10.222.0.23 () {58 vars in 1166 bytes} [Sat Aug 20 21:21:47 2022] GET /identity/v3/projects?domain_id=default => generated 6199 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:48.022608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09609e88-be2a-4a76-b0a3-d80a699030bb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:48.032836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09609e88-be2a-4a76-b0a3-d80a699030bb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:48.038271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a3d91-48d4-43cd-b7a5-6366ec4c4d3a tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f82a4ad32804c89b962a0d0a5810bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7afd2624d27b458a9daf105b30949fc4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:48.038874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a3d91-48d4-43cd-b7a5-6366ec4c4d3a tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:48.039153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a3d91-48d4-43cd-b7a5-6366ec4c4d3a tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:48.039455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a3d91-48d4-43cd-b7a5-6366ec4c4d3a tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:48.040095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a3d91-48d4-43cd-b7a5-6366ec4c4d3a tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:48.042034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a3d91-48d4-43cd-b7a5-6366ec4c4d3a tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:48.054887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e55c8bc2094b6b8e9381be96881cc1', 'request_id': 'req-d1981073-8e90-4089-a2ec-fc332c9825ee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.065856 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3366/6653] 10.222.0.23 () {58 vars in 1131 bytes} [Sat Aug 20 21:21:48 2022] GET /identity/v3/projects => generated 6169 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:48.066037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e55c8bc2094b6b8e9381be96881cc1', 'request_id': 'req-d1981073-8e90-4089-a2ec-fc332c9825ee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.067614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a3d91-48d4-43cd-b7a5-6366ec4c4d3a tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:48.073543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e55c8bc2094b6b8e9381be96881cc1', 'request_id': 'req-d1981073-8e90-4089-a2ec-fc332c9825ee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.079252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a3d91-48d4-43cd-b7a5-6366ec4c4d3a tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:48.084684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] The token cache is being invalidate because project 88e55c8bc2094b6b8e9381be96881cc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:48.084971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.085713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.087372 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3288/6654] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:47 2022] DELETE /identity/v3/projects/88e55c8bc2094b6b8e9381be96881cc1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:48.091355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:48.092807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:48.137150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729923f-db43-41f2-8310-fdac55dd6534 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f82a4ad32804c89b962a0d0a5810bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7afd2624d27b458a9daf105b30949fc4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:48.137766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729923f-db43-41f2-8310-fdac55dd6534 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:48.138404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729923f-db43-41f2-8310-fdac55dd6534 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:48.138404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729923f-db43-41f2-8310-fdac55dd6534 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:48.138976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729923f-db43-41f2-8310-fdac55dd6534 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Adding query filter params (name=tempest-ListProjectsStaticTestJSON-1790527647) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:48.139271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729923f-db43-41f2-8310-fdac55dd6534 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:48.141160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729923f-db43-41f2-8310-fdac55dd6534 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:48.153131 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3367/6655] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Aug 20 21:21:48 2022] GET /identity/v3/projects?name=tempest-ListProjectsStaticTestJSON-1790527647 => generated 537 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:48.157898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1981073-8e90-4089-a2ec-fc332c9825ee admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:48.158714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729923f-db43-41f2-8310-fdac55dd6534 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:48.160960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0729923f-db43-41f2-8310-fdac55dd6534 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:48.164515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3812a-bed6-4978-bc57-5760ea3ff89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:48.165199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3812a-bed6-4978-bc57-5760ea3ff89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:48.165644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3812a-bed6-4978-bc57-5760ea3ff89f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:48.165879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3812a-bed6-4978-bc57-5760ea3ff89f admin admin] PATH_INFO: `/v3/users/da1146d638ab42cdadcba16d5365ef9f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:48.167024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3812a-bed6-4978-bc57-5760ea3ff89f admin admin] RBAC: Authorizing `identity:delete_user(user_id=da1146d638ab42cdadcba16d5365ef9f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:48.169137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3812a-bed6-4978-bc57-5760ea3ff89f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:48.176956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729923f-db43-41f2-8310-fdac55dd6534 tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:48.182442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f82a4ad32804c89b962a0d0a5810bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7afd2624d27b458a9daf105b30949fc4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:48.183068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:48.183335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:48.183601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] PATH_INFO: `/v3/projects/c45064d1354043dcab6f565611d2d01f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:48.184727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c45064d1354043dcab6f565611d2d01f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:48.186622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:48.213340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:48.232980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bc3812a-bed6-4978-bc57-5760ea3ff89f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da1146d638ab42cdadcba16d5365ef9f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.245077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c45064d1354043dcab6f565611d2d01f', 'request_id': 'req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.246066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bc3812a-bed6-4978-bc57-5760ea3ff89f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da1146d638ab42cdadcba16d5365ef9f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.250703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:48.255847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c45064d1354043dcab6f565611d2d01f', 'request_id': 'req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.263654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c45064d1354043dcab6f565611d2d01f', 'request_id': 'req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.267328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bc3812a-bed6-4978-bc57-5760ea3ff89f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da1146d638ab42cdadcba16d5365ef9f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.268563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bc3812a-bed6-4978-bc57-5760ea3ff89f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da1146d638ab42cdadcba16d5365ef9f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.269117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bc3812a-bed6-4978-bc57-5760ea3ff89f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da1146d638ab42cdadcba16d5365ef9f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.278984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] The token cache is being invalidate because project c45064d1354043dcab6f565611d2d01f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:48.279267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.279919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.282507 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3368/6656] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:48 2022] DELETE /identity/v3/projects/c45064d1354043dcab6f565611d2d01f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:48.283007 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3289/6657] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:48 2022] DELETE /identity/v3/users/da1146d638ab42cdadcba16d5365ef9f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:48.289460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:48.290831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:48.374548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea548e0-bba4-4ceb-b31f-0ba723be6eeb tempest-ListProjectsStaticTestJSON-1652373742 tempest-ListProjectsStaticTestJSON-1652373742-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:48.443574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:48.444209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:48.444525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:48.444794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] PATH_INFO: `/v3/users/a95c7ee657cb4d9eba2773ac55db8c0b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:48.446247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a95c7ee657cb4d9eba2773ac55db8c0b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:48.448408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:48.523660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a95c7ee657cb4d9eba2773ac55db8c0b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.536263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a95c7ee657cb4d9eba2773ac55db8c0b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.542847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:48.547363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a95c7ee657cb4d9eba2773ac55db8c0b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.562802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a95c7ee657cb4d9eba2773ac55db8c0b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.563583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a95c7ee657cb4d9eba2773ac55db8c0b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.572725 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3369/6658] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:48 2022] DELETE /identity/v3/users/a95c7ee657cb4d9eba2773ac55db8c0b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:48.578544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57920c2e-b885-448e-8373-601fe7243f75 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:48.578827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57920c2e-b885-448e-8373-601fe7243f75 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:48.579093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57920c2e-b885-448e-8373-601fe7243f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:48.603921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-57920c2e-b885-448e-8373-601fe7243f75 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:48.605208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-57920c2e-b885-448e-8373-601fe7243f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:48.617613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:48.619933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:48.688837 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3290/6659] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:48.706233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5cb0b-9d06-417b-9d81-cf2743b35868 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:48.712963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57920c2e-b885-448e-8373-601fe7243f75 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:48.713377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcba308-00d5-4033-b8bf-8e2290b5c122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:48.714494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-57920c2e-b885-448e-8373-601fe7243f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:48.714633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcba308-00d5-4033-b8bf-8e2290b5c122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:48.714871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcba308-00d5-4033-b8bf-8e2290b5c122 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:48.715140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcba308-00d5-4033-b8bf-8e2290b5c122 admin admin] PATH_INFO: `/v3/projects/d670bba78f784854b63a071cfd797410` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:48.716302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcba308-00d5-4033-b8bf-8e2290b5c122 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d670bba78f784854b63a071cfd797410)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:48.718868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcba308-00d5-4033-b8bf-8e2290b5c122 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:48.786697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-edcba308-00d5-4033-b8bf-8e2290b5c122 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd670bba78f784854b63a071cfd797410', 'request_id': 'req-edcba308-00d5-4033-b8bf-8e2290b5c122'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.786899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57920c2e-b885-448e-8373-601fe7243f75 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:48.815651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-edcba308-00d5-4033-b8bf-8e2290b5c122 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd670bba78f784854b63a071cfd797410', 'request_id': 'req-edcba308-00d5-4033-b8bf-8e2290b5c122'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.822695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-edcba308-00d5-4033-b8bf-8e2290b5c122 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd670bba78f784854b63a071cfd797410', 'request_id': 'req-edcba308-00d5-4033-b8bf-8e2290b5c122'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.838480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-edcba308-00d5-4033-b8bf-8e2290b5c122 admin admin] The token cache is being invalidate because project d670bba78f784854b63a071cfd797410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:48.838793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-edcba308-00d5-4033-b8bf-8e2290b5c122 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.839427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-edcba308-00d5-4033-b8bf-8e2290b5c122 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:48.841420 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3370/6660] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:48 2022] DELETE /identity/v3/projects/d670bba78f784854b63a071cfd797410 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:48.846508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee382d8-5dde-46e2-a684-da941ea5beaa None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:48.847038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee382d8-5dde-46e2-a684-da941ea5beaa None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:48.847155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bc110-2c9e-46e1-8c7b-464a07a808fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:48.847708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bc110-2c9e-46e1-8c7b-464a07a808fe service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:48.847984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bc110-2c9e-46e1-8c7b-464a07a808fe service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:48.848241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee382d8-5dde-46e2-a684-da941ea5beaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:48.848395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bc110-2c9e-46e1-8c7b-464a07a808fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:48.851102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-844bc110-2c9e-46e1-8c7b-464a07a808fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:48.870801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1ee382d8-5dde-46e2-a684-da941ea5beaa None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:48.872512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1ee382d8-5dde-46e2-a684-da941ea5beaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:48.903090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:48.929328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bc110-2c9e-46e1-8c7b-464a07a808fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:48.930903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bc110-2c9e-46e1-8c7b-464a07a808fe service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:48.941920 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3291/6661] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:48.948284 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3371/6662] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:48.955965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6c13d-26d6-4abd-bca2-d03407c24f46 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:48.956214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6c13d-26d6-4abd-bca2-d03407c24f46 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:48.956573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6c13d-26d6-4abd-bca2-d03407c24f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:48.976865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0ef6c13d-26d6-4abd-bca2-d03407c24f46 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:48.978098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0ef6c13d-26d6-4abd-bca2-d03407c24f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:48.986686 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3292/6663] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:48.991598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee382d8-5dde-46e2-a684-da941ea5beaa None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:49.001667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee382d8-5dde-46e2-a684-da941ea5beaa None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:49.006814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b81fd1-1922-468d-a188-31bdffd426aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:49.007426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b81fd1-1922-468d-a188-31bdffd426aa admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:49.007716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b81fd1-1922-468d-a188-31bdffd426aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:49.008011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b81fd1-1922-468d-a188-31bdffd426aa admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:49.008661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b81fd1-1922-468d-a188-31bdffd426aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:49.008952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b81fd1-1922-468d-a188-31bdffd426aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:49.010850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b81fd1-1922-468d-a188-31bdffd426aa admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:49.012763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:49.026328 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3372/6664] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:49.030672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6c13d-26d6-4abd-bca2-d03407c24f46 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:49.041324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6c13d-26d6-4abd-bca2-d03407c24f46 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:49.046052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc60078-fa44-45d1-92c1-e716d0f06485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:49.046710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc60078-fa44-45d1-92c1-e716d0f06485 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:49.047079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc60078-fa44-45d1-92c1-e716d0f06485 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:49.047336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc60078-fa44-45d1-92c1-e716d0f06485 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:49.048318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc60078-fa44-45d1-92c1-e716d0f06485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:49.050723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc60078-fa44-45d1-92c1-e716d0f06485 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:49.083293 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3293/6665] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:49 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:49.088406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b81fd1-1922-468d-a188-31bdffd426aa admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:49.100023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b81fd1-1922-468d-a188-31bdffd426aa admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:49.162977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f31c4-2ac5-4df7-b5ba-7cd8a5effc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:49.163654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f31c4-2ac5-4df7-b5ba-7cd8a5effc26 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:49.163869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f31c4-2ac5-4df7-b5ba-7cd8a5effc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:49.164587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f31c4-2ac5-4df7-b5ba-7cd8a5effc26 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:49.164891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f31c4-2ac5-4df7-b5ba-7cd8a5effc26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:49.166837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f31c4-2ac5-4df7-b5ba-7cd8a5effc26 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:49.192970 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3373/6666] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:49 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:49.194614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc60078-fa44-45d1-92c1-e716d0f06485 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:49.205074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc60078-fa44-45d1-92c1-e716d0f06485 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:49.209795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89ee60-99ff-4989-b2a7-f340e3d39a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:49.210397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89ee60-99ff-4989-b2a7-f340e3d39a13 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:49.210673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89ee60-99ff-4989-b2a7-f340e3d39a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:49.210940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89ee60-99ff-4989-b2a7-f340e3d39a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:49.211579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89ee60-99ff-4989-b2a7-f340e3d39a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:49.213216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89ee60-99ff-4989-b2a7-f340e3d39a13 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:49.226447 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3294/6667] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:49.227755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f31c4-2ac5-4df7-b5ba-7cd8a5effc26 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:49.254161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f31c4-2ac5-4df7-b5ba-7cd8a5effc26 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:49.262183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9c43e-fef5-4c42-8d28-8911d6a2870e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:49.263248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9c43e-fef5-4c42-8d28-8911d6a2870e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:49.263248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9c43e-fef5-4c42-8d28-8911d6a2870e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:49.263453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9c43e-fef5-4c42-8d28-8911d6a2870e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:49.264122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9c43e-fef5-4c42-8d28-8911d6a2870e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:49.265801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9c43e-fef5-4c42-8d28-8911d6a2870e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:49.278015 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3374/6668] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:49.282248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89ee60-99ff-4989-b2a7-f340e3d39a13 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:49.293890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89ee60-99ff-4989-b2a7-f340e3d39a13 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:49.300430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4829450-26a2-41a4-a160-74a30a12a7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:49.301266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4829450-26a2-41a4-a160-74a30a12a7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:49.301576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4829450-26a2-41a4-a160-74a30a12a7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:49.301849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4829450-26a2-41a4-a160-74a30a12a7d6 admin admin] PATH_INFO: `/v3/projects/a2b33e1eb4eb494384961b9293a64c71/users/82ac9008f360490b923e189fb11c79a1/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:49.322012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4829450-26a2-41a4-a160-74a30a12a7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b33e1eb4eb494384961b9293a64c71, user_id=82ac9008f360490b923e189fb11c79a1, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:49.324873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4829450-26a2-41a4-a160-74a30a12a7d6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:49.341603 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3295/6669] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:49 2022] PUT /identity/v3/projects/a2b33e1eb4eb494384961b9293a64c71/users/82ac9008f360490b923e189fb11c79a1/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:49.353586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1080b8a-88da-431b-a5e5-279bbafa79f4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:49.353917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1080b8a-88da-431b-a5e5-279bbafa79f4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:49.354159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1080b8a-88da-431b-a5e5-279bbafa79f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:49.386854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f1080b8a-88da-431b-a5e5-279bbafa79f4 None None] MFA Rules not processed for user `82ac9008f360490b923e189fb11c79a1`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:49.388195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f1080b8a-88da-431b-a5e5-279bbafa79f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:49.404615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4829450-26a2-41a4-a160-74a30a12a7d6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:49.406378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f4829450-26a2-41a4-a160-74a30a12a7d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:49.462061 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3375/6670] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:49.466631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1080b8a-88da-431b-a5e5-279bbafa79f4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:49.481763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4829450-26a2-41a4-a160-74a30a12a7d6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:49.486689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:49.487325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:49.487679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:49.487946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] PATH_INFO: `/v3/projects/a8358b7d0c724ef68af314a9962b06b8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:49.489105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8358b7d0c724ef68af314a9962b06b8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:49.491225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:49.531538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1080b8a-88da-431b-a5e5-279bbafa79f4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:49.537802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:49.545438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8358b7d0c724ef68af314a9962b06b8', 'request_id': 'req-d69c405a-4040-4f95-82d0-59c68e2d5616'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:49.552154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8358b7d0c724ef68af314a9962b06b8', 'request_id': 'req-d69c405a-4040-4f95-82d0-59c68e2d5616'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:49.562492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8358b7d0c724ef68af314a9962b06b8', 'request_id': 'req-d69c405a-4040-4f95-82d0-59c68e2d5616'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:49.573911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] The token cache is being invalidate because project a8358b7d0c724ef68af314a9962b06b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:49.574196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:49.574873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:49.576440 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3296/6671] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:49 2022] DELETE /identity/v3/projects/a8358b7d0c724ef68af314a9962b06b8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:49.580314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:49.581636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:49.588625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:49.596403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215675e-5ba9-4139-9cd1-c47f4adc4224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:49.597014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215675e-5ba9-4139-9cd1-c47f4adc4224 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:49.597287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215675e-5ba9-4139-9cd1-c47f4adc4224 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:49.597719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215675e-5ba9-4139-9cd1-c47f4adc4224 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:49.598447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215675e-5ba9-4139-9cd1-c47f4adc4224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:49.601157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215675e-5ba9-4139-9cd1-c47f4adc4224 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:49.628416 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3376/6672] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:49 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:49.633085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215675e-5ba9-4139-9cd1-c47f4adc4224 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:49.634681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b215675e-5ba9-4139-9cd1-c47f4adc4224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:49.652296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c405a-4040-4f95-82d0-59c68e2d5616 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:49.698490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215675e-5ba9-4139-9cd1-c47f4adc4224 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:49.703149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b082797-d021-4c4b-a2a7-191326b8a3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:49.703761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b082797-d021-4c4b-a2a7-191326b8a3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:49.704032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b082797-d021-4c4b-a2a7-191326b8a3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:49.704298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b082797-d021-4c4b-a2a7-191326b8a3d1 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:49.704961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b082797-d021-4c4b-a2a7-191326b8a3d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:49.706882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b082797-d021-4c4b-a2a7-191326b8a3d1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:49.710697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:49.711352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:49.711648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:49.711915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] PATH_INFO: `/v3/users/7f82a4ad32804c89b962a0d0a5810bcf` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:49.713017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f82a4ad32804c89b962a0d0a5810bcf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:49.714929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:49.728732 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3377/6673] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:49 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:49.732717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b082797-d021-4c4b-a2a7-191326b8a3d1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:49.743552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b082797-d021-4c4b-a2a7-191326b8a3d1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:49.749443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06485694-8065-49d5-8308-bc259a6838b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:49.750133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06485694-8065-49d5-8308-bc259a6838b9 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:49.750501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06485694-8065-49d5-8308-bc259a6838b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:49.751052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06485694-8065-49d5-8308-bc259a6838b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:49.751429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06485694-8065-49d5-8308-bc259a6838b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:49.753059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06485694-8065-49d5-8308-bc259a6838b9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:49.763801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f82a4ad32804c89b962a0d0a5810bcf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:49.765302 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3378/6674] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:49.769740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06485694-8065-49d5-8308-bc259a6838b9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:49.773174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f82a4ad32804c89b962a0d0a5810bcf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:49.780779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06485694-8065-49d5-8308-bc259a6838b9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:49.785614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd35a8d-c476-44e7-b403-9f1baa349b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:49.786077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f82a4ad32804c89b962a0d0a5810bcf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:49.786268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35a8d-c476-44e7-b403-9f1baa349b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:49.786584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35a8d-c476-44e7-b403-9f1baa349b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:49.786849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35a8d-c476-44e7-b403-9f1baa349b84 admin admin] PATH_INFO: `/v3/projects/532c743f753040f39578019e11be8ed7/users/0e3b1ab4f4c1483fbfdad1ab6e896d4d/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:49.787004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f82a4ad32804c89b962a0d0a5810bcf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:49.788035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f82a4ad32804c89b962a0d0a5810bcf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:49.800673 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3297/6675] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:49 2022] DELETE /identity/v3/users/7f82a4ad32804c89b962a0d0a5810bcf => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:49.803283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd35a8d-c476-44e7-b403-9f1baa349b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=532c743f753040f39578019e11be8ed7, user_id=0e3b1ab4f4c1483fbfdad1ab6e896d4d, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:49.805994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd35a8d-c476-44e7-b403-9f1baa349b84 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:49.822802 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3379/6676] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:49 2022] PUT /identity/v3/projects/532c743f753040f39578019e11be8ed7/users/0e3b1ab4f4c1483fbfdad1ab6e896d4d/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:49.827498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:49.829114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:49.870016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:49.914851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b833e41-b0e6-4c64-9d97-9319509ff230 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:50.021429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57790a-a034-4ada-98f2-8962d196aeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:50.022238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57790a-a034-4ada-98f2-8962d196aeff admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.022509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57790a-a034-4ada-98f2-8962d196aeff admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.022852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57790a-a034-4ada-98f2-8962d196aeff admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.023496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57790a-a034-4ada-98f2-8962d196aeff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:50.025229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57790a-a034-4ada-98f2-8962d196aeff admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:50.037546 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3298/6677] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:50.041734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd35a8d-c476-44e7-b403-9f1baa349b84 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:50.067875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd35a8d-c476-44e7-b403-9f1baa349b84 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:50.073482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aed208-816e-48ec-a9b9-d3b9062b2ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:50.074405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aed208-816e-48ec-a9b9-d3b9062b2ac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.074742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aed208-816e-48ec-a9b9-d3b9062b2ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.075077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aed208-816e-48ec-a9b9-d3b9062b2ac9 admin admin] PATH_INFO: `/v3/projects/532c743f753040f39578019e11be8ed7/users/0e3b1ab4f4c1483fbfdad1ab6e896d4d/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.077161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aed208-816e-48ec-a9b9-d3b9062b2ac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=532c743f753040f39578019e11be8ed7, user_id=0e3b1ab4f4c1483fbfdad1ab6e896d4d, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:50.080058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aed208-816e-48ec-a9b9-d3b9062b2ac9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:50.091094 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3380/6678] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:50 2022] PUT /identity/v3/projects/532c743f753040f39578019e11be8ed7/users/0e3b1ab4f4c1483fbfdad1ab6e896d4d/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:50.101998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349e41b-e915-4e5f-b0df-486f121998fc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.102287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349e41b-e915-4e5f-b0df-486f121998fc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.102563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349e41b-e915-4e5f-b0df-486f121998fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.135835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8349e41b-e915-4e5f-b0df-486f121998fc None None] MFA Rules not processed for user `0e3b1ab4f4c1483fbfdad1ab6e896d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:50.137084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8349e41b-e915-4e5f-b0df-486f121998fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:50.214657 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3299/6679] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:50.219702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aed208-816e-48ec-a9b9-d3b9062b2ac9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:50.237969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aed208-816e-48ec-a9b9-d3b9062b2ac9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:50.244284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53f52a-3557-4441-a6b9-b708744c7245 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:50.245100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53f52a-3557-4441-a6b9-b708744c7245 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.245441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53f52a-3557-4441-a6b9-b708744c7245 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.245766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53f52a-3557-4441-a6b9-b708744c7245 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.246586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53f52a-3557-4441-a6b9-b708744c7245 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:50.248714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53f52a-3557-4441-a6b9-b708744c7245 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:50.284443 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3381/6680] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:50 2022] POST /identity/v3/projects => generated 360 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:50.288756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349e41b-e915-4e5f-b0df-486f121998fc None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:50.299556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349e41b-e915-4e5f-b0df-486f121998fc None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:50.363039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88612ec7-fc78-4b76-9f20-e0e2a10b0282 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:50.363529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88612ec7-fc78-4b76-9f20-e0e2a10b0282 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.363839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88612ec7-fc78-4b76-9f20-e0e2a10b0282 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.364106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88612ec7-fc78-4b76-9f20-e0e2a10b0282 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.364884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88612ec7-fc78-4b76-9f20-e0e2a10b0282 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:50.366875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88612ec7-fc78-4b76-9f20-e0e2a10b0282 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:50.376733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:50.398152 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3300/6681] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:50 2022] POST /identity/v3/users => generated 413 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:50.399892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53f52a-3557-4441-a6b9-b708744c7245 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:50.411424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53f52a-3557-4441-a6b9-b708744c7245 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:50.417492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15502b09-0d77-4b94-901a-8c124586aa2d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:50.418141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15502b09-0d77-4b94-901a-8c124586aa2d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.418404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15502b09-0d77-4b94-901a-8c124586aa2d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.418678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15502b09-0d77-4b94-901a-8c124586aa2d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/users/f5ba158780c1465c9e1d9b25699171ed` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.431138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:50.435202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15502b09-0d77-4b94-901a-8c124586aa2d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:get_user(user_id=f5ba158780c1465c9e1d9b25699171ed)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:50.438544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15502b09-0d77-4b94-901a-8c124586aa2d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:50.440497 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3382/6682] 10.222.0.23 () {58 vars in 1221 bytes} [Sat Aug 20 21:21:50 2022] GET /identity/v3/users/f5ba158780c1465c9e1d9b25699171ed => generated 413 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:50.445445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88612ec7-fc78-4b76-9f20-e0e2a10b0282 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:50.457268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88612ec7-fc78-4b76-9f20-e0e2a10b0282 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:50.462632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:50.463504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.463809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.464071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/users/f5ba158780c1465c9e1d9b25699171ed` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.465399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_user(user_id=f5ba158780c1465c9e1d9b25699171ed)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:50.467528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:50.514122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15502b09-0d77-4b94-901a-8c124586aa2d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:50.516488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-15502b09-0d77-4b94-901a-8c124586aa2d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:50.521642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ba158780c1465c9e1d9b25699171ed'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.532341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ba158780c1465c9e1d9b25699171ed'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.549241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ba158780c1465c9e1d9b25699171ed'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.550085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ba158780c1465c9e1d9b25699171ed'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.550852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ba158780c1465c9e1d9b25699171ed'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.571272 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3301/6683] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:50 2022] DELETE /identity/v3/users/f5ba158780c1465c9e1d9b25699171ed => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:50.572294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:50.574581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:50.589657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15502b09-0d77-4b94-901a-8c124586aa2d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:50.639608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e0966-495f-4199-9723-58c68dc7c91f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:50.644360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:50.645030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.645327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.645584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/e8ef0d6064ad428596e4f259ed263235` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.646857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e8ef0d6064ad428596e4f259ed263235)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:50.647377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e34233-e503-4e48-b928-18db12777f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:50.648163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e34233-e503-4e48-b928-18db12777f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.648465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e34233-e503-4e48-b928-18db12777f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.648736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e34233-e503-4e48-b928-18db12777f2a admin admin] PATH_INFO: `/v3/projects/7afd2624d27b458a9daf105b30949fc4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.648861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:50.649892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e34233-e503-4e48-b928-18db12777f2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7afd2624d27b458a9daf105b30949fc4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:50.651913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e34233-e503-4e48-b928-18db12777f2a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:50.699063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8ef0d6064ad428596e4f259ed263235', 'request_id': 'req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.702015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-36e34233-e503-4e48-b928-18db12777f2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7afd2624d27b458a9daf105b30949fc4', 'request_id': 'req-36e34233-e503-4e48-b928-18db12777f2a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.705399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8ef0d6064ad428596e4f259ed263235', 'request_id': 'req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.711006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-36e34233-e503-4e48-b928-18db12777f2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7afd2624d27b458a9daf105b30949fc4', 'request_id': 'req-36e34233-e503-4e48-b928-18db12777f2a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.714122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8ef0d6064ad428596e4f259ed263235', 'request_id': 'req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.716582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=5,keystone:SELECT=79,keystone:DELETE=16 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:50.717275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-36e34233-e503-4e48-b928-18db12777f2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7afd2624d27b458a9daf105b30949fc4', 'request_id': 'req-36e34233-e503-4e48-b928-18db12777f2a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.727760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project e8ef0d6064ad428596e4f259ed263235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:50.728286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.729001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.730131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-36e34233-e503-4e48-b928-18db12777f2a admin admin] The token cache is being invalidate because project 7afd2624d27b458a9daf105b30949fc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:50.730420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-36e34233-e503-4e48-b928-18db12777f2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.730599 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3302/6684] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:50 2022] DELETE /identity/v3/projects/e8ef0d6064ad428596e4f259ed263235 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:50.732946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-36e34233-e503-4e48-b928-18db12777f2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:50.734843 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3383/6685] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:50 2022] DELETE /identity/v3/projects/7afd2624d27b458a9daf105b30949fc4 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:50.738231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:50.738231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:50.739528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645c7b8-b2a5-4cf7-b942-89c0b37923c4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.739919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645c7b8-b2a5-4cf7-b942-89c0b37923c4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.740187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645c7b8-b2a5-4cf7-b942-89c0b37923c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.764506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c645c7b8-b2a5-4cf7-b942-89c0b37923c4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:50.765750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c645c7b8-b2a5-4cf7-b942-89c0b37923c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:50.796572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:50.804114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5c46d-6b8a-4af5-8d2b-37fc198300f6 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:50.827922 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3384/6686] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:50.836671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e38a52-8bf8-4243-aa8f-90f6047c1091 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.836958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e38a52-8bf8-4243-aa8f-90f6047c1091 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.837378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e38a52-8bf8-4243-aa8f-90f6047c1091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.862544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-32e38a52-8bf8-4243-aa8f-90f6047c1091 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:50.863759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-32e38a52-8bf8-4243-aa8f-90f6047c1091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:50.866388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1758e0-8dca-4847-a8a4-bfb13aaf6bbc tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:50.867009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1758e0-8dca-4847-a8a4-bfb13aaf6bbc tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.867289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1758e0-8dca-4847-a8a4-bfb13aaf6bbc tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.867560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1758e0-8dca-4847-a8a4-bfb13aaf6bbc tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.868216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1758e0-8dca-4847-a8a4-bfb13aaf6bbc tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:50.870010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1758e0-8dca-4847-a8a4-bfb13aaf6bbc tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:50.872442 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3385/6687] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:50.876628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e38a52-8bf8-4243-aa8f-90f6047c1091 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:50.886164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e38a52-8bf8-4243-aa8f-90f6047c1091 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:50.887364 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3303/6688] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:50 2022] POST /identity/v3/projects => generated 348 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:50.891340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1758e0-8dca-4847-a8a4-bfb13aaf6bbc tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:50.893152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1c1758e0-8dca-4847-a8a4-bfb13aaf6bbc tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:50.949658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806369a-f1a7-47e7-aec7-ef678d4dec58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:50.950277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806369a-f1a7-47e7-aec7-ef678d4dec58 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.950551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806369a-f1a7-47e7-aec7-ef678d4dec58 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.950819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806369a-f1a7-47e7-aec7-ef678d4dec58 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.951464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806369a-f1a7-47e7-aec7-ef678d4dec58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:50.951765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806369a-f1a7-47e7-aec7-ef678d4dec58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:50.953402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806369a-f1a7-47e7-aec7-ef678d4dec58 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:50.965814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1758e0-8dca-4847-a8a4-bfb13aaf6bbc tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:50.970480 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3386/6689] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:50.971596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806369a-f1a7-47e7-aec7-ef678d4dec58 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:50.972553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225bc5-052a-449d-b1bd-70cb77ae52e3 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:50.973208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225bc5-052a-449d-b1bd-70cb77ae52e3 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.973484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225bc5-052a-449d-b1bd-70cb77ae52e3 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.973748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225bc5-052a-449d-b1bd-70cb77ae52e3 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.975256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c225bc5-052a-449d-b1bd-70cb77ae52e3 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Adding query filter params (is_domain=True) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:50.975594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c225bc5-052a-449d-b1bd-70cb77ae52e3 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:50.978267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c225bc5-052a-449d-b1bd-70cb77ae52e3 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:50.982015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806369a-f1a7-47e7-aec7-ef678d4dec58 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:50.988196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf74ac4-7aa5-45eb-bbc2-ee8460c312d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:50.988866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf74ac4-7aa5-45eb-bbc2-ee8460c312d7 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:50.989136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf74ac4-7aa5-45eb-bbc2-ee8460c312d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:50.989396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf74ac4-7aa5-45eb-bbc2-ee8460c312d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:50.990118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf74ac4-7aa5-45eb-bbc2-ee8460c312d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:50.992098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf74ac4-7aa5-45eb-bbc2-ee8460c312d7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.006068 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3304/6690] 10.222.0.23 () {58 vars in 1160 bytes} [Sat Aug 20 21:21:50 2022] GET /identity/v3/projects?is_domain=True => generated 1020 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:51.007949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225bc5-052a-449d-b1bd-70cb77ae52e3 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.026741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225bc5-052a-449d-b1bd-70cb77ae52e3 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.029116 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3387/6691] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:50 2022] POST /identity/v3/projects => generated 391 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:51.034872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf74ac4-7aa5-45eb-bbc2-ee8460c312d7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.061329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:51.122649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8c9c7-9c5f-45b1-b69c-abbc3b730271 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.123986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8c9c7-9c5f-45b1-b69c-abbc3b730271 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.123986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8c9c7-9c5f-45b1-b69c-abbc3b730271 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.123986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8c9c7-9c5f-45b1-b69c-abbc3b730271 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.124425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8c9c7-9c5f-45b1-b69c-abbc3b730271 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.126051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8c9c7-9c5f-45b1-b69c-abbc3b730271 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.131842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf74ac4-7aa5-45eb-bbc2-ee8460c312d7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.138376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd76050-122c-4037-b4e9-2a9881fc4cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.138997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76050-122c-4037-b4e9-2a9881fc4cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.139264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76050-122c-4037-b4e9-2a9881fc4cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.139530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76050-122c-4037-b4e9-2a9881fc4cf9 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.143490 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3305/6692] 10.222.0.23 () {58 vars in 1128 bytes} [Sat Aug 20 21:21:51 2022] GET /identity/v3/domains => generated 829 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:51.143626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd76050-122c-4037-b4e9-2a9881fc4cf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.143626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd76050-122c-4037-b4e9-2a9881fc4cf9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.144429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8c9c7-9c5f-45b1-b69c-abbc3b730271 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.156350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8c9c7-9c5f-45b1-b69c-abbc3b730271 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.162561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.163214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.163490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.163775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/a90de58cad314e2d9d3692aa45087c71` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.167138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:update_project(project_id=a90de58cad314e2d9d3692aa45087c71)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.168230 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:21:51.168230 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:21:51.168230 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:21:51.169291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.178969 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3388/6693] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:51 2022] POST /identity/v3/users => generated 296 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:51.184511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd76050-122c-4037-b4e9-2a9881fc4cf9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.188653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'a90de58cad314e2d9d3692aa45087c71'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.188653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'a90de58cad314e2d9d3692aa45087c71'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.200715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd76050-122c-4037-b4e9-2a9881fc4cf9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.224889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'a90de58cad314e2d9d3692aa45087c71'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.225663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'a90de58cad314e2d9d3692aa45087c71'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.229397 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3306/6694] 10.222.0.23 () {60 vars in 1252 bytes} [Sat Aug 20 21:21:51 2022] PATCH /identity/v3/projects/a90de58cad314e2d9d3692aa45087c71 => generated 362 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:51.233071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.234575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:51.253300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:51.288181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6d0b5-072c-46b8-8aaa-06c36d436789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.288964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6d0b5-072c-46b8-8aaa-06c36d436789 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.289308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6d0b5-072c-46b8-8aaa-06c36d436789 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.289645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6d0b5-072c-46b8-8aaa-06c36d436789 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.290413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6d0b5-072c-46b8-8aaa-06c36d436789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.292104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6d0b5-072c-46b8-8aaa-06c36d436789 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.304659 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3389/6695] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:51.308592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6d0b5-072c-46b8-8aaa-06c36d436789 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.310820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-82f6d0b5-072c-46b8-8aaa-06c36d436789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:51.323224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbeadb-c442-4605-837f-d5940d9a3f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.328208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6d0b5-072c-46b8-8aaa-06c36d436789 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.328350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.329014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.330900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.330900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/a90de58cad314e2d9d3692aa45087c71` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.334971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb406202-3978-4a89-968d-dd19da3a6821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.335962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb406202-3978-4a89-968d-dd19da3a6821 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.336846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb406202-3978-4a89-968d-dd19da3a6821 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.337321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb406202-3978-4a89-968d-dd19da3a6821 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.338148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb406202-3978-4a89-968d-dd19da3a6821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.341069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb406202-3978-4a89-968d-dd19da3a6821 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.343314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a90de58cad314e2d9d3692aa45087c71)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.345120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.353166 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3390/6696] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:51.357467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb406202-3978-4a89-968d-dd19da3a6821 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.370933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb406202-3978-4a89-968d-dd19da3a6821 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.376498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee23d1-baca-42a7-b581-f8e5b17bc6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.377474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee23d1-baca-42a7-b581-f8e5b17bc6f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.377752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee23d1-baca-42a7-b581-f8e5b17bc6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.378021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee23d1-baca-42a7-b581-f8e5b17bc6f0 admin admin] PATH_INFO: `/v3/projects/6975eca6b6fd4765b4898ee19b0d8bcb/users/ff3a1ea6523c48a09682006ad929e44d/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.380270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a90de58cad314e2d9d3692aa45087c71'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.384919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a90de58cad314e2d9d3692aa45087c71'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.397056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee23d1-baca-42a7-b581-f8e5b17bc6f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6975eca6b6fd4765b4898ee19b0d8bcb, user_id=ff3a1ea6523c48a09682006ad929e44d, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.399657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee23d1-baca-42a7-b581-f8e5b17bc6f0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.420033 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3391/6697] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:51 2022] PUT /identity/v3/projects/6975eca6b6fd4765b4898ee19b0d8bcb/users/ff3a1ea6523c48a09682006ad929e44d/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:51.435143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26c38f-4f62-4019-aaf7-5d69ea88cad6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.435512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26c38f-4f62-4019-aaf7-5d69ea88cad6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.435890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26c38f-4f62-4019-aaf7-5d69ea88cad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.459113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a90de58cad314e2d9d3692aa45087c71', 'request_id': 'req-10c4f48a-4dea-464d-81f7-2a7588f5fe78'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.469235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7e26c38f-4f62-4019-aaf7-5d69ea88cad6 None None] MFA Rules not processed for user `ff3a1ea6523c48a09682006ad929e44d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:51.470148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a90de58cad314e2d9d3692aa45087c71', 'request_id': 'req-10c4f48a-4dea-464d-81f7-2a7588f5fe78'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.470591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7e26c38f-4f62-4019-aaf7-5d69ea88cad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:51.475052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:DELETE=4,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:51.482346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a90de58cad314e2d9d3692aa45087c71', 'request_id': 'req-10c4f48a-4dea-464d-81f7-2a7588f5fe78'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.497951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project a90de58cad314e2d9d3692aa45087c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:51.498745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.499888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.511659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a90de58cad314e2d9d3692aa45087c71', 'request_id': 'req-10c4f48a-4dea-464d-81f7-2a7588f5fe78'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.512563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a90de58cad314e2d9d3692aa45087c71', 'request_id': 'req-10c4f48a-4dea-464d-81f7-2a7588f5fe78'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.519993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a90de58cad314e2d9d3692aa45087c71', 'request_id': 'req-10c4f48a-4dea-464d-81f7-2a7588f5fe78'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.527738 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3307/6698] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:51 2022] DELETE /identity/v3/projects/a90de58cad314e2d9d3692aa45087c71 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:51.532804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.534204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:51.553620 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3392/6699] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:51.562714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26c38f-4f62-4019-aaf7-5d69ea88cad6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.574041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26c38f-4f62-4019-aaf7-5d69ea88cad6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.580069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474d35a-2583-4480-b25f-96ae3120c890 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.581080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474d35a-2583-4480-b25f-96ae3120c890 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.581357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474d35a-2583-4480-b25f-96ae3120c890 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.581631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474d35a-2583-4480-b25f-96ae3120c890 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.582942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474d35a-2583-4480-b25f-96ae3120c890 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:create_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.584435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474d35a-2583-4480-b25f-96ae3120c890 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.624360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4f48a-4dea-464d-81f7-2a7588f5fe78 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.629804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd75ff-dc8b-4b59-8dd1-7792286e7840 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.630693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd75ff-dc8b-4b59-8dd1-7792286e7840 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.631199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd75ff-dc8b-4b59-8dd1-7792286e7840 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.631666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd75ff-dc8b-4b59-8dd1-7792286e7840 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.632249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd75ff-dc8b-4b59-8dd1-7792286e7840 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.634252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd75ff-dc8b-4b59-8dd1-7792286e7840 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.669069 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3393/6700] 10.222.0.23 () {60 vars in 1314 bytes} [Sat Aug 20 21:21:51 2022] POST /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials => generated 789 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:51.673546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474d35a-2583-4480-b25f-96ae3120c890 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.676444 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3308/6701] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:51 2022] POST /identity/v3/projects => generated 360 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:51.680464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd75ff-dc8b-4b59-8dd1-7792286e7840 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.684043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474d35a-2583-4480-b25f-96ae3120c890 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.697709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:51.748925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1329a-dd7e-4d1d-98e6-1a8443d32207 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.749762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1329a-dd7e-4d1d-98e6-1a8443d32207 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.750041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1329a-dd7e-4d1d-98e6-1a8443d32207 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.750316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1329a-dd7e-4d1d-98e6-1a8443d32207 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/cf331b90b0324b6f979d82e759842ba9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.755686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd75ff-dc8b-4b59-8dd1-7792286e7840 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.760963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9b531-0319-4747-85d2-fc74b7f8cdb7 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.761276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1329a-dd7e-4d1d-98e6-1a8443d32207 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:get_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d, application_credential_id=cf331b90b0324b6f979d82e759842ba9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.761610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9b531-0319-4747-85d2-fc74b7f8cdb7 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.761893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9b531-0319-4747-85d2-fc74b7f8cdb7 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.762164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9b531-0319-4747-85d2-fc74b7f8cdb7 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/ed3c4eda3c6740078b41ccc6da6e1ecf` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.763313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9b531-0319-4747-85d2-fc74b7f8cdb7 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:get_project(project_id=ed3c4eda3c6740078b41ccc6da6e1ecf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.763435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1329a-dd7e-4d1d-98e6-1a8443d32207 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.765719 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3394/6702] 10.222.0.23 () {58 vars in 1392 bytes} [Sat Aug 20 21:21:51 2022] GET /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/cf331b90b0324b6f979d82e759842ba9 => generated 689 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:51.769447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9b531-0319-4747-85d2-fc74b7f8cdb7 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.769447 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3309/6703] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:51 2022] GET /identity/v3/projects/ed3c4eda3c6740078b41ccc6da6e1ecf => generated 360 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:51.771423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3efb3-776a-46e5-8beb-73c697eb7dc8 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.771762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3efb3-776a-46e5-8beb-73c697eb7dc8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.772135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3efb3-776a-46e5-8beb-73c697eb7dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.772579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9b531-0319-4747-85d2-fc74b7f8cdb7 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.783319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9b531-0319-4747-85d2-fc74b7f8cdb7 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.787408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-70b3efb3-776a-46e5-8beb-73c697eb7dc8 None None] MFA Rules not processed for user `ff3a1ea6523c48a09682006ad929e44d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:51.788669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-70b3efb3-776a-46e5-8beb-73c697eb7dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:51.788946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.789577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.789849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.790116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/ed3c4eda3c6740078b41ccc6da6e1ecf` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.791274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ed3c4eda3c6740078b41ccc6da6e1ecf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.793357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.845442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:51.847336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed3c4eda3c6740078b41ccc6da6e1ecf', 'request_id': 'req-5be3413c-f71e-41dc-9a99-ae7cb29dc293'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.856229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed3c4eda3c6740078b41ccc6da6e1ecf', 'request_id': 'req-5be3413c-f71e-41dc-9a99-ae7cb29dc293'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.865945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed3c4eda3c6740078b41ccc6da6e1ecf', 'request_id': 'req-5be3413c-f71e-41dc-9a99-ae7cb29dc293'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.878083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project ed3c4eda3c6740078b41ccc6da6e1ecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:51.878389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.879292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:51.881552 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3310/6704] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:51 2022] DELETE /identity/v3/projects/ed3c4eda3c6740078b41ccc6da6e1ecf => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:51.887128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.888815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:51.910065 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3395/6705] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:51 2022] POST /identity/v3/auth/tokens => generated 3425 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Aug 20 21:21:51.914432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3efb3-776a-46e5-8beb-73c697eb7dc8 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:51.960041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3413c-f71e-41dc-9a99-ae7cb29dc293 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.964885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ab0af-1115-43f2-bfcd-e7fc21f45bcf tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.965506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ab0af-1115-43f2-bfcd-e7fc21f45bcf tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.965788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ab0af-1115-43f2-bfcd-e7fc21f45bcf tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.966058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ab0af-1115-43f2-bfcd-e7fc21f45bcf tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.966690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ab0af-1115-43f2-bfcd-e7fc21f45bcf tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.968594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ab0af-1115-43f2-bfcd-e7fc21f45bcf tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:51.988276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3efb3-776a-46e5-8beb-73c697eb7dc8 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:51.993747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f6d9e-df0c-41e2-b7e4-2193e840c375 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:51.994403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f6d9e-df0c-41e2-b7e4-2193e840c375 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:51.994812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f6d9e-df0c-41e2-b7e4-2193e840c375 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:51.995082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f6d9e-df0c-41e2-b7e4-2193e840c375 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/cf331b90b0324b6f979d82e759842ba9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:51.996205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f6d9e-df0c-41e2-b7e4-2193e840c375 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d, application_credential_id=cf331b90b0324b6f979d82e759842ba9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:51.997847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f6d9e-df0c-41e2-b7e4-2193e840c375 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.005372 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3311/6706] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:51 2022] POST /identity/v3/projects => generated 361 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:52.006624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ab0af-1115-43f2-bfcd-e7fc21f45bcf tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.018855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:52.025403 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3396/6707] 10.222.0.23 () {58 vars in 1395 bytes} [Sat Aug 20 21:21:51 2022] DELETE /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/cf331b90b0324b6f979d82e759842ba9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:52.031702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f6d9e-df0c-41e2-b7e4-2193e840c375 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.035105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ca3f6d9e-df0c-41e2-b7e4-2193e840c375 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:52.049811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f6d9e-df0c-41e2-b7e4-2193e840c375 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.080459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ab0af-1115-43f2-bfcd-e7fc21f45bcf tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.087076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a596faa-49a5-485c-8eff-1dde3f332577 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.088407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a596faa-49a5-485c-8eff-1dde3f332577 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.088696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a596faa-49a5-485c-8eff-1dde3f332577 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.088944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a596faa-49a5-485c-8eff-1dde3f332577 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/c39f48e78fe34ed88951aceaf2b2d93c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.090120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a596faa-49a5-485c-8eff-1dde3f332577 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:get_project(project_id=c39f48e78fe34ed88951aceaf2b2d93c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.092349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a596faa-49a5-485c-8eff-1dde3f332577 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.094089 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3312/6708] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:52 2022] GET /identity/v3/projects/c39f48e78fe34ed88951aceaf2b2d93c => generated 361 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:52.098589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a596faa-49a5-485c-8eff-1dde3f332577 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.108951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a596faa-49a5-485c-8eff-1dde3f332577 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.114045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.114728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.115102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.115415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/c39f48e78fe34ed88951aceaf2b2d93c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.116650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c39f48e78fe34ed88951aceaf2b2d93c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.116766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e656e-514e-4839-a798-a8b4abacfe79 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.117295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e656e-514e-4839-a798-a8b4abacfe79 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.117567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e656e-514e-4839-a798-a8b4abacfe79 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.117835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e656e-514e-4839-a798-a8b4abacfe79 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.118490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e656e-514e-4839-a798-a8b4abacfe79 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:create_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.118618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.119636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e656e-514e-4839-a798-a8b4abacfe79 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.166212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c39f48e78fe34ed88951aceaf2b2d93c', 'request_id': 'req-bde8ad19-82e1-458e-b49a-6c537a305313'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.173625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c39f48e78fe34ed88951aceaf2b2d93c', 'request_id': 'req-bde8ad19-82e1-458e-b49a-6c537a305313'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.177611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:52.181461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c39f48e78fe34ed88951aceaf2b2d93c', 'request_id': 'req-bde8ad19-82e1-458e-b49a-6c537a305313'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.199474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project c39f48e78fe34ed88951aceaf2b2d93c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:52.199871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.200673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.202299 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3313/6709] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:52 2022] DELETE /identity/v3/projects/c39f48e78fe34ed88951aceaf2b2d93c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:52.207285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.207789 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3397/6710] 10.222.0.23 () {60 vars in 1315 bytes} [Sat Aug 20 21:21:52 2022] POST /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials => generated 921 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:52.209150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:52.211956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e656e-514e-4839-a798-a8b4abacfe79 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.213623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e95e656e-514e-4839-a798-a8b4abacfe79 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:52.226544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e656e-514e-4839-a798-a8b4abacfe79 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.281772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8ad19-82e1-458e-b49a-6c537a305313 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.286500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ffee0-e1b5-44ec-b4a6-0d317da59a07 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.286672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb50b2e-be79-4516-b6cb-f5cc1a03aff4 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.287187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ffee0-e1b5-44ec-b4a6-0d317da59a07 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.287442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ffee0-e1b5-44ec-b4a6-0d317da59a07 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.287555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb50b2e-be79-4516-b6cb-f5cc1a03aff4 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.287697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ffee0-e1b5-44ec-b4a6-0d317da59a07 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.287790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb50b2e-be79-4516-b6cb-f5cc1a03aff4 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.287973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb50b2e-be79-4516-b6cb-f5cc1a03aff4 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/89283c9b401a49618470a77dcea81bd5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.288315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ffee0-e1b5-44ec-b4a6-0d317da59a07 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.290201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ffee0-e1b5-44ec-b4a6-0d317da59a07 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.299974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb50b2e-be79-4516-b6cb-f5cc1a03aff4 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d, application_credential_id=89283c9b401a49618470a77dcea81bd5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.301522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb50b2e-be79-4516-b6cb-f5cc1a03aff4 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.324413 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3314/6711] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:21:52 2022] POST /identity/v3/projects => generated 346 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:52.326183 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3398/6712] 10.222.0.23 () {58 vars in 1395 bytes} [Sat Aug 20 21:21:52 2022] DELETE /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/89283c9b401a49618470a77dcea81bd5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:52.329191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ffee0-e1b5-44ec-b4a6-0d317da59a07 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.331075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb50b2e-be79-4516-b6cb-f5cc1a03aff4 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.341385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb50b2e-be79-4516-b6cb-f5cc1a03aff4 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.350957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:52.393922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ffee0-e1b5-44ec-b4a6-0d317da59a07 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.398855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e9cab-3e52-44a6-8fd4-fcf482a626d0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.399494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e9cab-3e52-44a6-8fd4-fcf482a626d0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.399768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e9cab-3e52-44a6-8fd4-fcf482a626d0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.400037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e9cab-3e52-44a6-8fd4-fcf482a626d0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/557493d6dc254c1290ec1e0891eacd9e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.400811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dbbc8a-6c18-4a1a-88b7-ddb30c68a49d tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.401239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e9cab-3e52-44a6-8fd4-fcf482a626d0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:get_project(project_id=557493d6dc254c1290ec1e0891eacd9e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.401453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbbc8a-6c18-4a1a-88b7-ddb30c68a49d tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.401717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbbc8a-6c18-4a1a-88b7-ddb30c68a49d tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.401980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbbc8a-6c18-4a1a-88b7-ddb30c68a49d tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.402591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dbbc8a-6c18-4a1a-88b7-ddb30c68a49d tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:create_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.403416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e9cab-3e52-44a6-8fd4-fcf482a626d0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.403729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dbbc8a-6c18-4a1a-88b7-ddb30c68a49d tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.405142 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3315/6713] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:52 2022] GET /identity/v3/projects/557493d6dc254c1290ec1e0891eacd9e => generated 346 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:52.409276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e9cab-3e52-44a6-8fd4-fcf482a626d0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.419840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e9cab-3e52-44a6-8fd4-fcf482a626d0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.424612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.425224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.425498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.425765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/557493d6dc254c1290ec1e0891eacd9e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.426901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_project(project_id=557493d6dc254c1290ec1e0891eacd9e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.428950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.476569 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3399/6714] 10.222.0.23 () {60 vars in 1315 bytes} [Sat Aug 20 21:21:52 2022] POST /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials => generated 812 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:52.480671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dbbc8a-6c18-4a1a-88b7-ddb30c68a49d tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.488809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557493d6dc254c1290ec1e0891eacd9e', 'request_id': 'req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.490251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dbbc8a-6c18-4a1a-88b7-ddb30c68a49d tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.495805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557493d6dc254c1290ec1e0891eacd9e', 'request_id': 'req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.504427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557493d6dc254c1290ec1e0891eacd9e', 'request_id': 'req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.511339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:52.516079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project 557493d6dc254c1290ec1e0891eacd9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:52.516383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.518559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.520419 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3316/6715] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:52 2022] DELETE /identity/v3/projects/557493d6dc254c1290ec1e0891eacd9e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:52.525794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.527918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:52.548013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec8fa3-15f4-46a6-8bf6-f2f128ec80d0 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.548696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec8fa3-15f4-46a6-8bf6-f2f128ec80d0 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.548966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec8fa3-15f4-46a6-8bf6-f2f128ec80d0 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.549253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec8fa3-15f4-46a6-8bf6-f2f128ec80d0 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/295fb45eecee40f3b09c3e632b04133b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.558349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec8fa3-15f4-46a6-8bf6-f2f128ec80d0 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d, application_credential_id=295fb45eecee40f3b09c3e632b04133b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.559820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec8fa3-15f4-46a6-8bf6-f2f128ec80d0 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.580342 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3400/6716] 10.222.0.23 () {58 vars in 1395 bytes} [Sat Aug 20 21:21:52 2022] DELETE /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/295fb45eecee40f3b09c3e632b04133b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:52.585585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec8fa3-15f4-46a6-8bf6-f2f128ec80d0 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.587141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a9ec8fa3-15f4-46a6-8bf6-f2f128ec80d0 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:52.597563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c51eb1-b57d-4e96-a8f0-dcd51b616972 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.598478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec8fa3-15f4-46a6-8bf6-f2f128ec80d0 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.602873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e93446-8f4b-427c-b757-cc44f0529788 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.603010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa6b13-dc5f-4bf7-a8dd-296f50562ab8 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.603498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa6b13-dc5f-4bf7-a8dd-296f50562ab8 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.603616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e93446-8f4b-427c-b757-cc44f0529788 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.603768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e93446-8f4b-427c-b757-cc44f0529788 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.603863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa6b13-dc5f-4bf7-a8dd-296f50562ab8 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.604036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e93446-8f4b-427c-b757-cc44f0529788 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.604133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa6b13-dc5f-4bf7-a8dd-296f50562ab8 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.604728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa6b13-dc5f-4bf7-a8dd-296f50562ab8 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:create_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.604845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e93446-8f4b-427c-b757-cc44f0529788 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.605877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa6b13-dc5f-4bf7-a8dd-296f50562ab8 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.606455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e93446-8f4b-427c-b757-cc44f0529788 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.623269 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3317/6717] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:21:52 2022] POST /identity/v3/domains => generated 277 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:52.627185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e93446-8f4b-427c-b757-cc44f0529788 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.653443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:52.681972 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3401/6718] 10.222.0.23 () {60 vars in 1314 bytes} [Sat Aug 20 21:21:52 2022] POST /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials => generated 787 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:52.686306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa6b13-dc5f-4bf7-a8dd-296f50562ab8 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.697541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e93446-8f4b-427c-b757-cc44f0529788 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.702364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184cfa2-0948-4c1e-a79b-7470684c8242 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.702988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184cfa2-0948-4c1e-a79b-7470684c8242 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.703273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184cfa2-0948-4c1e-a79b-7470684c8242 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.703543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184cfa2-0948-4c1e-a79b-7470684c8242 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.704170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184cfa2-0948-4c1e-a79b-7470684c8242 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.707102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184cfa2-0948-4c1e-a79b-7470684c8242 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.710401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa6b13-dc5f-4bf7-a8dd-296f50562ab8 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.754360 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3318/6719] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:52 2022] POST /identity/v3/projects => generated 403 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:52.755226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184cfa2-0948-4c1e-a79b-7470684c8242 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.772320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9483fe-05b4-43b0-835d-658410c08465 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.773004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9483fe-05b4-43b0-835d-658410c08465 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.773279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9483fe-05b4-43b0-835d-658410c08465 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.773547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9483fe-05b4-43b0-835d-658410c08465 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.774496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9483fe-05b4-43b0-835d-658410c08465 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:create_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.775701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9483fe-05b4-43b0-835d-658410c08465 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.823040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184cfa2-0948-4c1e-a79b-7470684c8242 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.827872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce246124-bdca-4a73-9e43-0bdfa7145908 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.828533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce246124-bdca-4a73-9e43-0bdfa7145908 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.828811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce246124-bdca-4a73-9e43-0bdfa7145908 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.829187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce246124-bdca-4a73-9e43-0bdfa7145908 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/47e07a10dc174899b3e564e7e4506fc3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.832768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce246124-bdca-4a73-9e43-0bdfa7145908 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:get_project(project_id=47e07a10dc174899b3e564e7e4506fc3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.835326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce246124-bdca-4a73-9e43-0bdfa7145908 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.840099 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3319/6720] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:52 2022] GET /identity/v3/projects/47e07a10dc174899b3e564e7e4506fc3 => generated 403 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:52.841088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce246124-bdca-4a73-9e43-0bdfa7145908 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.851990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce246124-bdca-4a73-9e43-0bdfa7145908 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.856935 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3402/6721] 10.222.0.23 () {60 vars in 1314 bytes} [Sat Aug 20 21:21:52 2022] POST /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials => generated 788 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:52.857066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.857478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.857753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.858018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9483fe-05b4-43b0-835d-658410c08465 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.858111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/47e07a10dc174899b3e564e7e4506fc3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.859341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_project(project_id=47e07a10dc174899b3e564e7e4506fc3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.861456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.867512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9483fe-05b4-43b0-835d-658410c08465 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.868663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:52.912341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47e07a10dc174899b3e564e7e4506fc3', 'request_id': 'req-b391567a-298c-4440-8ed0-d6fc981d6816'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.918903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47e07a10dc174899b3e564e7e4506fc3', 'request_id': 'req-b391567a-298c-4440-8ed0-d6fc981d6816'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.920983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8940e3-c822-4c88-a1d4-7302107c79fe tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:52.921632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8940e3-c822-4c88-a1d4-7302107c79fe tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:52.921906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8940e3-c822-4c88-a1d4-7302107c79fe tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:52.922207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8940e3-c822-4c88-a1d4-7302107c79fe tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:52.922819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8940e3-c822-4c88-a1d4-7302107c79fe tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:list_application_credentials(user_id=ff3a1ea6523c48a09682006ad929e44d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:52.924600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8940e3-c822-4c88-a1d4-7302107c79fe tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:52.927412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47e07a10dc174899b3e564e7e4506fc3', 'request_id': 'req-b391567a-298c-4440-8ed0-d6fc981d6816'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.937562 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3403/6722] 10.222.0.23 () {58 vars in 1293 bytes} [Sat Aug 20 21:21:52 2022] GET /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials => generated 1499 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:52.939234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project 47e07a10dc174899b3e564e7e4506fc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:52.939520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.940171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:52.941719 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3320/6723] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:52 2022] DELETE /identity/v3/projects/47e07a10dc174899b3e564e7e4506fc3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:52.945236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8940e3-c822-4c88-a1d4-7302107c79fe tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.946037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:52.946918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5e8940e3-c822-4c88-a1d4-7302107c79fe tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:52.947542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:52.958242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8940e3-c822-4c88-a1d4-7302107c79fe tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:52.975303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:53.012164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ac6aa-0146-4e52-82fb-927a43c0f211 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.012757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ac6aa-0146-4e52-82fb-927a43c0f211 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.013032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ac6aa-0146-4e52-82fb-927a43c0f211 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.013234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391567a-298c-4440-8ed0-d6fc981d6816 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:53.013361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ac6aa-0146-4e52-82fb-927a43c0f211 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/6e8722a2e6124e60ba0e66e82e574753` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.018156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85903da-7782-4a40-89ad-47f3a55d5733 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.018788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85903da-7782-4a40-89ad-47f3a55d5733 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.019067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85903da-7782-4a40-89ad-47f3a55d5733 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.019336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85903da-7782-4a40-89ad-47f3a55d5733 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/domains/f9a396369f8d4c58be30774ddeec61fe` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.020589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85903da-7782-4a40-89ad-47f3a55d5733 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f9a396369f8d4c58be30774ddeec61fe)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.021836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ac6aa-0146-4e52-82fb-927a43c0f211 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d, application_credential_id=6e8722a2e6124e60ba0e66e82e574753)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.022480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85903da-7782-4a40-89ad-47f3a55d5733 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.023327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ac6aa-0146-4e52-82fb-927a43c0f211 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.043197 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3404/6724] 10.222.0.23 () {58 vars in 1395 bytes} [Sat Aug 20 21:21:52 2022] DELETE /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/6e8722a2e6124e60ba0e66e82e574753 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:53.043477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c85903da-7782-4a40-89ad-47f3a55d5733 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f9a396369f8d4c58be30774ddeec61fe'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.044192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c85903da-7782-4a40-89ad-47f3a55d5733 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f9a396369f8d4c58be30774ddeec61fe'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.047522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ac6aa-0146-4e52-82fb-927a43c0f211 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:53.049386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a37ac6aa-0146-4e52-82fb-927a43c0f211 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:53.061454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ac6aa-0146-4e52-82fb-927a43c0f211 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:53.071458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c85903da-7782-4a40-89ad-47f3a55d5733 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f9a396369f8d4c58be30774ddeec61fe'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.072086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c85903da-7782-4a40-89ad-47f3a55d5733 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f9a396369f8d4c58be30774ddeec61fe'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.074551 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3321/6725] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:21:52 2022] PATCH /identity/v3/domains/f9a396369f8d4c58be30774ddeec61fe => generated 278 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:53.078502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85903da-7782-4a40-89ad-47f3a55d5733 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:53.079836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c85903da-7782-4a40-89ad-47f3a55d5733 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:53.118754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9bda9-2c3f-4f31-bed6-9bea710886fd tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.119524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9bda9-2c3f-4f31-bed6-9bea710886fd tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.119812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9bda9-2c3f-4f31-bed6-9bea710886fd tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.120086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9bda9-2c3f-4f31-bed6-9bea710886fd tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/918d43ba3d7045499e016d56516b51e9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.129727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9bda9-2c3f-4f31-bed6-9bea710886fd tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d, application_credential_id=918d43ba3d7045499e016d56516b51e9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.131921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9bda9-2c3f-4f31-bed6-9bea710886fd tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.147536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85903da-7782-4a40-89ad-47f3a55d5733 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:53.152886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.153685 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3405/6726] 10.222.0.23 () {58 vars in 1395 bytes} [Sat Aug 20 21:21:53 2022] DELETE /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/918d43ba3d7045499e016d56516b51e9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:53.153800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.153913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.154216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/domains/f9a396369f8d4c58be30774ddeec61fe` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.156617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:53.159054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9bda9-2c3f-4f31-bed6-9bea710886fd tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:53.161253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fae9bda9-2c3f-4f31-bed6-9bea710886fd tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:53.171189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f9a396369f8d4c58be30774ddeec61fe)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.173165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.174232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9bda9-2c3f-4f31-bed6-9bea710886fd tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:53.178949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4cf82-4f14-4a5f-a21c-855643965467 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.179606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4cf82-4f14-4a5f-a21c-855643965467 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.180013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4cf82-4f14-4a5f-a21c-855643965467 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.180316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4cf82-4f14-4a5f-a21c-855643965467 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.180999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4cf82-4f14-4a5f-a21c-855643965467 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:create_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.182197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4cf82-4f14-4a5f-a21c-855643965467 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.196924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f9a396369f8d4c58be30774ddeec61fe'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.201019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f9a396369f8d4c58be30774ddeec61fe'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.251791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a396369f8d4c58be30774ddeec61fe', 'request_id': 'req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.260529 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3406/6727] 10.222.0.23 () {60 vars in 1314 bytes} [Sat Aug 20 21:21:53 2022] POST /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials => generated 789 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:53.260703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a396369f8d4c58be30774ddeec61fe', 'request_id': 'req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.261749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4cf82-4f14-4a5f-a21c-855643965467 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:53.269940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a396369f8d4c58be30774ddeec61fe', 'request_id': 'req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.272023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4cf82-4f14-4a5f-a21c-855643965467 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:53.280939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project f9a396369f8d4c58be30774ddeec61fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:53.281232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.281864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.284792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:53.291532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f9a396369f8d4c58be30774ddeec61fe', 'request_id': 'req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.292322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f9a396369f8d4c58be30774ddeec61fe', 'request_id': 'req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.296360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f9a396369f8d4c58be30774ddeec61fe', 'request_id': 'req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.302391 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3322/6728] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:53 2022] DELETE /identity/v3/domains/f9a396369f8d4c58be30774ddeec61fe => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:53.307526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:53.308963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:53.330493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb7337-f1f6-4e67-8118-da5b0e8cd863 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.331135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb7337-f1f6-4e67-8118-da5b0e8cd863 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.331435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb7337-f1f6-4e67-8118-da5b0e8cd863 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.331710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb7337-f1f6-4e67-8118-da5b0e8cd863 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.332360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb7337-f1f6-4e67-8118-da5b0e8cd863 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:create_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.333625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb7337-f1f6-4e67-8118-da5b0e8cd863 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.378554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2ecbb-7b71-4822-bfd5-ad2e654ed05e tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:53.383512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7da15f-6211-4fae-90d3-5e6df0f4da25 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.384129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7da15f-6211-4fae-90d3-5e6df0f4da25 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.384433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7da15f-6211-4fae-90d3-5e6df0f4da25 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.384705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7da15f-6211-4fae-90d3-5e6df0f4da25 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.385359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7da15f-6211-4fae-90d3-5e6df0f4da25 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.387031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7da15f-6211-4fae-90d3-5e6df0f4da25 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.406855 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3323/6729] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:21:53 2022] POST /identity/v3/domains => generated 275 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:53.408633 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3407/6730] 10.222.0.23 () {60 vars in 1314 bytes} [Sat Aug 20 21:21:53 2022] POST /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials => generated 787 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:53.410774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb7337-f1f6-4e67-8118-da5b0e8cd863 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:53.410908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7da15f-6211-4fae-90d3-5e6df0f4da25 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:53.412652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bc7da15f-6211-4fae-90d3-5e6df0f4da25 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:53.422701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb7337-f1f6-4e67-8118-da5b0e8cd863 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:53.455893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:53.491424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7da15f-6211-4fae-90d3-5e6df0f4da25 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:53.495898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76161a52-1456-4481-9ce3-ed203f329ed9 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.496592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76161a52-1456-4481-9ce3-ed203f329ed9 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.498580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76161a52-1456-4481-9ce3-ed203f329ed9 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.498580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76161a52-1456-4481-9ce3-ed203f329ed9 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.498580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76161a52-1456-4481-9ce3-ed203f329ed9 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Adding query filter params (name=tempest-application_credential-89909970) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:53.498580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76161a52-1456-4481-9ce3-ed203f329ed9 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:list_application_credentials(user_id=ff3a1ea6523c48a09682006ad929e44d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.498904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80224a-7229-43d5-92d5-8b8f4b22705f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.498904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80224a-7229-43d5-92d5-8b8f4b22705f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.499379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80224a-7229-43d5-92d5-8b8f4b22705f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.499618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80224a-7229-43d5-92d5-8b8f4b22705f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.499831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76161a52-1456-4481-9ce3-ed203f329ed9 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.500452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80224a-7229-43d5-92d5-8b8f4b22705f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.502512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80224a-7229-43d5-92d5-8b8f4b22705f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.512024 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3324/6731] 10.222.0.23 () {58 vars in 1382 bytes} [Sat Aug 20 21:21:53 2022] GET /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials?name=tempest-application_credential-89909970 => generated 883 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:53.516623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76161a52-1456-4481-9ce3-ed203f329ed9 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:53.529255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76161a52-1456-4481-9ce3-ed203f329ed9 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:53.537018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d14c88-3e18-417f-bec2-ca7eb3fea7c5 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.539993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d14c88-3e18-417f-bec2-ca7eb3fea7c5 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.540506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d14c88-3e18-417f-bec2-ca7eb3fea7c5 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.541025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d14c88-3e18-417f-bec2-ca7eb3fea7c5 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/266cddb3dfa1410ea38bf52107f9ad9a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.557535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d14c88-3e18-417f-bec2-ca7eb3fea7c5 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d, application_credential_id=266cddb3dfa1410ea38bf52107f9ad9a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.558642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d14c88-3e18-417f-bec2-ca7eb3fea7c5 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.562263 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3408/6732] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:53 2022] POST /identity/v3/projects => generated 409 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:53.567289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80224a-7229-43d5-92d5-8b8f4b22705f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:53.584562 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3325/6733] 10.222.0.23 () {58 vars in 1395 bytes} [Sat Aug 20 21:21:53 2022] DELETE /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/266cddb3dfa1410ea38bf52107f9ad9a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:53.588896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d14c88-3e18-417f-bec2-ca7eb3fea7c5 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:53.598192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d14c88-3e18-417f-bec2-ca7eb3fea7c5 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:53.620810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:53.642297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80224a-7229-43d5-92d5-8b8f4b22705f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:53.648913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71494516-ffc7-4b73-ad0a-7486aaf8b1c4 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.649645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71494516-ffc7-4b73-ad0a-7486aaf8b1c4 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.649840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71494516-ffc7-4b73-ad0a-7486aaf8b1c4 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.650072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71494516-ffc7-4b73-ad0a-7486aaf8b1c4 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.650703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71494516-ffc7-4b73-ad0a-7486aaf8b1c4 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.653205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71494516-ffc7-4b73-ad0a-7486aaf8b1c4 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.654067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7c997-7ec0-4255-8f72-1a61e2715ef6 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff3a1ea6523c48a09682006ad929e44d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6975eca6b6fd4765b4898ee19b0d8bcb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.654719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7c997-7ec0-4255-8f72-1a61e2715ef6 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.654994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7c997-7ec0-4255-8f72-1a61e2715ef6 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.655264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7c997-7ec0-4255-8f72-1a61e2715ef6 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/e9965f56805745b1b8e1a10a9e6eade5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.664782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7c997-7ec0-4255-8f72-1a61e2715ef6 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=ff3a1ea6523c48a09682006ad929e44d, application_credential_id=e9965f56805745b1b8e1a10a9e6eade5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.666293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7c997-7ec0-4255-8f72-1a61e2715ef6 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.689441 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3326/6734] 10.222.0.23 () {58 vars in 1395 bytes} [Sat Aug 20 21:21:53 2022] DELETE /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d/application_credentials/e9965f56805745b1b8e1a10a9e6eade5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:53.696614 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3409/6735] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:53 2022] POST /identity/v3/projects => generated 404 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:53.696802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7c997-7ec0-4255-8f72-1a61e2715ef6 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:53.697652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71494516-ffc7-4b73-ad0a-7486aaf8b1c4 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:53.698092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dbe7c997-7ec0-4255-8f72-1a61e2715ef6 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:53.767602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71494516-ffc7-4b73-ad0a-7486aaf8b1c4 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:53.773286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.774281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.774736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.775152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/f57c21a63baf42619ff71ac3702a1653` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.776633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f57c21a63baf42619ff71ac3702a1653)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.779045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.787404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7c997-7ec0-4255-8f72-1a61e2715ef6 tempest-ApplicationCredentialsV3Test-750675967 tempest-ApplicationCredentialsV3Test-750675967-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:53.810350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:53.873360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f57c21a63baf42619ff71ac3702a1653', 'request_id': 'req-02518ca3-82dd-40ed-88c7-5f88b15a2b50'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.886311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f57c21a63baf42619ff71ac3702a1653', 'request_id': 'req-02518ca3-82dd-40ed-88c7-5f88b15a2b50'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.897248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f57c21a63baf42619ff71ac3702a1653', 'request_id': 'req-02518ca3-82dd-40ed-88c7-5f88b15a2b50'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.917490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d22d3-f17c-4741-bbc2-fa352f1a97a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:53.918163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d22d3-f17c-4741-bbc2-fa352f1a97a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:53.918591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d22d3-f17c-4741-bbc2-fa352f1a97a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:53.919550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d22d3-f17c-4741-bbc2-fa352f1a97a0 admin admin] PATH_INFO: `/v3/users/ff3a1ea6523c48a09682006ad929e44d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:53.920982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d22d3-f17c-4741-bbc2-fa352f1a97a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff3a1ea6523c48a09682006ad929e44d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:53.922050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project f57c21a63baf42619ff71ac3702a1653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:53.922373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.923076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:53.924588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d22d3-f17c-4741-bbc2-fa352f1a97a0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:53.925082 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3410/6736] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:53 2022] DELETE /identity/v3/projects/f57c21a63baf42619ff71ac3702a1653 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:53.931075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:53.932622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:54.002327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-662d22d3-f17c-4741-bbc2-fa352f1a97a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff3a1ea6523c48a09682006ad929e44d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.007641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:54.013750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-662d22d3-f17c-4741-bbc2-fa352f1a97a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff3a1ea6523c48a09682006ad929e44d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.021685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02518ca3-82dd-40ed-88c7-5f88b15a2b50 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:54.028813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:54.029458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:54.029733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:54.030002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/a83e1a079f6b488ca601a1f3af6370fb` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:54.031353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a83e1a079f6b488ca601a1f3af6370fb)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:54.033527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:54.044832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-662d22d3-f17c-4741-bbc2-fa352f1a97a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff3a1ea6523c48a09682006ad929e44d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.045779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-662d22d3-f17c-4741-bbc2-fa352f1a97a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff3a1ea6523c48a09682006ad929e44d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.046558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-662d22d3-f17c-4741-bbc2-fa352f1a97a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff3a1ea6523c48a09682006ad929e44d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.062967 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3327/6737] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:53 2022] DELETE /identity/v3/users/ff3a1ea6523c48a09682006ad929e44d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:54.067933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b6b12-4c26-48df-a377-9573b9a98f69 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:54.068620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b6b12-4c26-48df-a377-9573b9a98f69 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:54.068876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b6b12-4c26-48df-a377-9573b9a98f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:54.089355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-874b6b12-4c26-48df-a377-9573b9a98f69 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:54.090532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-874b6b12-4c26-48df-a377-9573b9a98f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:54.090669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a83e1a079f6b488ca601a1f3af6370fb', 'request_id': 'req-49bfd371-3486-4b82-8b25-9f4f174c731a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.101055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a83e1a079f6b488ca601a1f3af6370fb', 'request_id': 'req-49bfd371-3486-4b82-8b25-9f4f174c731a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.106752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a83e1a079f6b488ca601a1f3af6370fb', 'request_id': 'req-49bfd371-3486-4b82-8b25-9f4f174c731a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.118010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project a83e1a079f6b488ca601a1f3af6370fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:54.118297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.118954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.120506 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3411/6738] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:53 2022] DELETE /identity/v3/projects/a83e1a079f6b488ca601a1f3af6370fb => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:54.124478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:54.125865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:54.155450 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3328/6739] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:54.165929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b6b12-4c26-48df-a377-9573b9a98f69 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:54.167517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-874b6b12-4c26-48df-a377-9573b9a98f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:54.195166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfd371-3486-4b82-8b25-9f4f174c731a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:54.201054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb78fe7-1234-44b6-8f8f-07c24f83fa09 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:54.201842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb78fe7-1234-44b6-8f8f-07c24f83fa09 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:54.202120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb78fe7-1234-44b6-8f8f-07c24f83fa09 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:54.202437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb78fe7-1234-44b6-8f8f-07c24f83fa09 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/domains/66e472dc7aad4ab9acc5edcd46d49dda` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:54.203706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb78fe7-1234-44b6-8f8f-07c24f83fa09 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=66e472dc7aad4ab9acc5edcd46d49dda)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:54.205361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb78fe7-1234-44b6-8f8f-07c24f83fa09 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:54.211126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:54.236175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ccb78fe7-1234-44b6-8f8f-07c24f83fa09 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '66e472dc7aad4ab9acc5edcd46d49dda'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.236937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ccb78fe7-1234-44b6-8f8f-07c24f83fa09 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '66e472dc7aad4ab9acc5edcd46d49dda'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.251938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b6b12-4c26-48df-a377-9573b9a98f69 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:54.269702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ccb78fe7-1234-44b6-8f8f-07c24f83fa09 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '66e472dc7aad4ab9acc5edcd46d49dda'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.270439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ccb78fe7-1234-44b6-8f8f-07c24f83fa09 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '66e472dc7aad4ab9acc5edcd46d49dda'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.280504 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3412/6740] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:21:54 2022] PATCH /identity/v3/domains/66e472dc7aad4ab9acc5edcd46d49dda => generated 276 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:54.281616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb78fe7-1234-44b6-8f8f-07c24f83fa09 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:54.283341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ccb78fe7-1234-44b6-8f8f-07c24f83fa09 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:54.311822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af0cf0-7bab-4f11-91d2-afa5ec485d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:54.312489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af0cf0-7bab-4f11-91d2-afa5ec485d51 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:54.312755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af0cf0-7bab-4f11-91d2-afa5ec485d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:54.313018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af0cf0-7bab-4f11-91d2-afa5ec485d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:54.314834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-24af0cf0-7bab-4f11-91d2-afa5ec485d51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:54.341044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:54.370304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb78fe7-1234-44b6-8f8f-07c24f83fa09 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:54.375373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:54.376096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:54.376468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:54.376764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/domains/66e472dc7aad4ab9acc5edcd46d49dda` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:54.389692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=66e472dc7aad4ab9acc5edcd46d49dda)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:54.391650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:54.397872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af0cf0-7bab-4f11-91d2-afa5ec485d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:54.399715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af0cf0-7bab-4f11-91d2-afa5ec485d51 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:54.419593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '66e472dc7aad4ab9acc5edcd46d49dda'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.424466 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3329/6741] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:21:54.433760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '66e472dc7aad4ab9acc5edcd46d49dda'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.481883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66e472dc7aad4ab9acc5edcd46d49dda', 'request_id': 'req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.489475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66e472dc7aad4ab9acc5edcd46d49dda', 'request_id': 'req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.495824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66e472dc7aad4ab9acc5edcd46d49dda', 'request_id': 'req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.509613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project 66e472dc7aad4ab9acc5edcd46d49dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:54.509900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.510570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.518639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '66e472dc7aad4ab9acc5edcd46d49dda', 'request_id': 'req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.519294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '66e472dc7aad4ab9acc5edcd46d49dda', 'request_id': 'req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.522935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '66e472dc7aad4ab9acc5edcd46d49dda', 'request_id': 'req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.529580 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3413/6742] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:54 2022] DELETE /identity/v3/domains/66e472dc7aad4ab9acc5edcd46d49dda => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:54.534786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af0cf0-7bab-4f11-91d2-afa5ec485d51 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:54.546574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af0cf0-7bab-4f11-91d2-afa5ec485d51 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:54.607253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2d3b4-76a3-4e60-a155-59d15cc0f39a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:54.607873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2d3b4-76a3-4e60-a155-59d15cc0f39a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:54.608155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2d3b4-76a3-4e60-a155-59d15cc0f39a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:54.608420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2d3b4-76a3-4e60-a155-59d15cc0f39a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:54.609054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2d3b4-76a3-4e60-a155-59d15cc0f39a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:54.610925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2d3b4-76a3-4e60-a155-59d15cc0f39a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:54.648912 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3330/6743] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:54 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:54.653589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:54.655140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:54.669273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1db1c8-a0e7-4ffa-ba3c-bcbafe8b1f29 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:54.704313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2d3b4-76a3-4e60-a155-59d15cc0f39a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:54.706704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-04f2d3b4-76a3-4e60-a155-59d15cc0f39a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:54.733326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:54.747144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd608d2-5199-4b14-aa60-dacbfba2a19d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:54.747923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd608d2-5199-4b14-aa60-dacbfba2a19d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:54.748203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd608d2-5199-4b14-aa60-dacbfba2a19d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:54.748505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd608d2-5199-4b14-aa60-dacbfba2a19d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/f5c19fe85c2d471f880840d84996cb42` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:54.749653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd608d2-5199-4b14-aa60-dacbfba2a19d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:get_project(project_id=f5c19fe85c2d471f880840d84996cb42)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:54.751787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd608d2-5199-4b14-aa60-dacbfba2a19d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:54.753656 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3414/6744] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:54 2022] GET /identity/v3/projects/f5c19fe85c2d471f880840d84996cb42 => generated 377 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:54.758148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd608d2-5199-4b14-aa60-dacbfba2a19d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:54.768538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd608d2-5199-4b14-aa60-dacbfba2a19d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:54.773588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81044b3-c713-4b92-a655-1316c464526f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:54.774178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81044b3-c713-4b92-a655-1316c464526f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:54.774455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81044b3-c713-4b92-a655-1316c464526f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:54.774724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81044b3-c713-4b92-a655-1316c464526f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:54.775358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81044b3-c713-4b92-a655-1316c464526f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:54.777247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81044b3-c713-4b92-a655-1316c464526f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:54.792428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2d3b4-76a3-4e60-a155-59d15cc0f39a tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:54.799005 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3415/6745] 10.222.0.23 () {58 vars in 1131 bytes} [Sat Aug 20 21:21:54 2022] GET /identity/v3/projects => generated 6185 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:54.799984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81044b3-c713-4b92-a655-1316c464526f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:54.800617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993f387-7dea-410b-8e64-3059d7c95c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:54.801274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f387-7dea-410b-8e64-3059d7c95c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:54.801549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f387-7dea-410b-8e64-3059d7c95c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:54.801819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f387-7dea-410b-8e64-3059d7c95c8e admin admin] PATH_INFO: `/v3/projects/6975eca6b6fd4765b4898ee19b0d8bcb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:54.803018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993f387-7dea-410b-8e64-3059d7c95c8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6975eca6b6fd4765b4898ee19b0d8bcb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:54.805167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993f387-7dea-410b-8e64-3059d7c95c8e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:54.810215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81044b3-c713-4b92-a655-1316c464526f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:54.814889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:54.815518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:54.815798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:54.816063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/f5c19fe85c2d471f880840d84996cb42` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:54.817225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f5c19fe85c2d471f880840d84996cb42)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:54.819578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:54.828226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:54.858265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c993f387-7dea-410b-8e64-3059d7c95c8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6975eca6b6fd4765b4898ee19b0d8bcb', 'request_id': 'req-c993f387-7dea-410b-8e64-3059d7c95c8e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.865575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c993f387-7dea-410b-8e64-3059d7c95c8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6975eca6b6fd4765b4898ee19b0d8bcb', 'request_id': 'req-c993f387-7dea-410b-8e64-3059d7c95c8e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.874736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c993f387-7dea-410b-8e64-3059d7c95c8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6975eca6b6fd4765b4898ee19b0d8bcb', 'request_id': 'req-c993f387-7dea-410b-8e64-3059d7c95c8e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.878016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5c19fe85c2d471f880840d84996cb42', 'request_id': 'req-e6c96e8e-e1df-4445-8f9a-94ac3c730963'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.886405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c993f387-7dea-410b-8e64-3059d7c95c8e admin admin] The token cache is being invalidate because project 6975eca6b6fd4765b4898ee19b0d8bcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:54.886748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c993f387-7dea-410b-8e64-3059d7c95c8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.887367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5c19fe85c2d471f880840d84996cb42', 'request_id': 'req-e6c96e8e-e1df-4445-8f9a-94ac3c730963'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.887721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c993f387-7dea-410b-8e64-3059d7c95c8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.889384 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3331/6746] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:54 2022] DELETE /identity/v3/projects/6975eca6b6fd4765b4898ee19b0d8bcb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:54.894493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5c19fe85c2d471f880840d84996cb42', 'request_id': 'req-e6c96e8e-e1df-4445-8f9a-94ac3c730963'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.895995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9b998-06ae-4bde-8892-b44f12423e2f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:54.897781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9b998-06ae-4bde-8892-b44f12423e2f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:54.898063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9b998-06ae-4bde-8892-b44f12423e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:54.906445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project f5c19fe85c2d471f880840d84996cb42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:54.906798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.907555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:54.909173 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3416/6747] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:54 2022] DELETE /identity/v3/projects/f5c19fe85c2d471f880840d84996cb42 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:54.914075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:54.915565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:54.920203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1eb9b998-06ae-4bde-8892-b44f12423e2f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:54.921400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1eb9b998-06ae-4bde-8892-b44f12423e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:54.986735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c96e8e-e1df-4445-8f9a-94ac3c730963 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:54.988352 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3332/6748] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:21:54.993543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6898a09-0f62-4571-908d-e5abc02f3e1b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:54.994169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6898a09-0f62-4571-908d-e5abc02f3e1b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:54.994439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6898a09-0f62-4571-908d-e5abc02f3e1b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:54.994705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6898a09-0f62-4571-908d-e5abc02f3e1b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:54.995333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6898a09-0f62-4571-908d-e5abc02f3e1b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.003618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6898a09-0f62-4571-908d-e5abc02f3e1b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.003947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bbf83-b64c-4ecd-9b7c-6aaf47a02cea None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.003947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bbf83-b64c-4ecd-9b7c-6aaf47a02cea None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.003947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bbf83-b64c-4ecd-9b7c-6aaf47a02cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.024288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-378bbf83-b64c-4ecd-9b7c-6aaf47a02cea None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:55.025760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-378bbf83-b64c-4ecd-9b7c-6aaf47a02cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:55.035524 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3333/6749] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:55.039630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378bbf83-b64c-4ecd-9b7c-6aaf47a02cea None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.044605 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3417/6750] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:21:54 2022] POST /identity/v3/projects => generated 346 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:55.048584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6898a09-0f62-4571-908d-e5abc02f3e1b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.052916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378bbf83-b64c-4ecd-9b7c-6aaf47a02cea None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.097562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:55.119566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6898a09-0f62-4571-908d-e5abc02f3e1b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.125057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e701b76-85ff-44e6-9cb3-970d55874c02 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.125815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e701b76-85ff-44e6-9cb3-970d55874c02 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.126093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e701b76-85ff-44e6-9cb3-970d55874c02 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.126222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0563a6-dac4-483f-a026-51631a0d2a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.126403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e701b76-85ff-44e6-9cb3-970d55874c02 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/37c45c678c1c413ebf61b01dddce09ad` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.127417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0563a6-dac4-483f-a026-51631a0d2a04 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.127655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e701b76-85ff-44e6-9cb3-970d55874c02 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:update_project(project_id=37c45c678c1c413ebf61b01dddce09ad)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.127769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0563a6-dac4-483f-a026-51631a0d2a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.127948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0563a6-dac4-483f-a026-51631a0d2a04 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.128633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0563a6-dac4-483f-a026-51631a0d2a04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:55.128798 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-5e701b76-85ff-44e6-9cb3-970d55874c02 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:21:55.128798 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:21:55.128798 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:21:55.129706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0563a6-dac4-483f-a026-51631a0d2a04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.129987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e701b76-85ff-44e6-9cb3-970d55874c02 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.131488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0563a6-dac4-483f-a026-51631a0d2a04 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.149534 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3334/6751] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:55.151881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0563a6-dac4-483f-a026-51631a0d2a04 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.163428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:55.164110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0563a6-dac4-483f-a026-51631a0d2a04 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.171028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af91a3-ee2d-43f8-b15c-a3ae512df4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.172012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af91a3-ee2d-43f8-b15c-a3ae512df4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.172356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af91a3-ee2d-43f8-b15c-a3ae512df4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.172721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af91a3-ee2d-43f8-b15c-a3ae512df4f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.173704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af91a3-ee2d-43f8-b15c-a3ae512df4f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.174329 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3418/6752] 10.222.0.23 () {60 vars in 1252 bytes} [Sat Aug 20 21:21:55 2022] PATCH /identity/v3/projects/37c45c678c1c413ebf61b01dddce09ad => generated 360 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:55.175332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e701b76-85ff-44e6-9cb3-970d55874c02 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.176634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af91a3-ee2d-43f8-b15c-a3ae512df4f1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.189197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e701b76-85ff-44e6-9cb3-970d55874c02 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.196905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec3bbc-1344-4901-8eef-7ce248c09d46 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.197553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec3bbc-1344-4901-8eef-7ce248c09d46 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.197825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec3bbc-1344-4901-8eef-7ce248c09d46 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.198087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec3bbc-1344-4901-8eef-7ce248c09d46 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/37c45c678c1c413ebf61b01dddce09ad` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.216343 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3335/6753] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:55 2022] POST /identity/v3/projects => generated 391 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:55.217428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af91a3-ee2d-43f8-b15c-a3ae512df4f1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.220212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec3bbc-1344-4901-8eef-7ce248c09d46 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:get_project(project_id=37c45c678c1c413ebf61b01dddce09ad)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.224862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec3bbc-1344-4901-8eef-7ce248c09d46 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.227059 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3419/6754] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:55 2022] GET /identity/v3/projects/37c45c678c1c413ebf61b01dddce09ad => generated 347 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:55.232891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec3bbc-1344-4901-8eef-7ce248c09d46 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.246009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec3bbc-1344-4901-8eef-7ce248c09d46 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.309435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af91a3-ee2d-43f8-b15c-a3ae512df4f1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.314498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc2d85-7e2c-4b56-bca9-fc5de6865a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.315040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc2d85-7e2c-4b56-bca9-fc5de6865a16 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.315311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc2d85-7e2c-4b56-bca9-fc5de6865a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.315573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc2d85-7e2c-4b56-bca9-fc5de6865a16 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.316187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc2d85-7e2c-4b56-bca9-fc5de6865a16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.318623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc2d85-7e2c-4b56-bca9-fc5de6865a16 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.319228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.319932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.320287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.320645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/37c45c678c1c413ebf61b01dddce09ad` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.321847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_project(project_id=37c45c678c1c413ebf61b01dddce09ad)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.323836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.345996 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3336/6755] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:55 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:55.347124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc2d85-7e2c-4b56-bca9-fc5de6865a16 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.360325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc2d85-7e2c-4b56-bca9-fc5de6865a16 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.366534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a10ae-dd23-417c-b12b-0e98de86439f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.367190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a10ae-dd23-417c-b12b-0e98de86439f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.367478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a10ae-dd23-417c-b12b-0e98de86439f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.367748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a10ae-dd23-417c-b12b-0e98de86439f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.368483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a10ae-dd23-417c-b12b-0e98de86439f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.370107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a10ae-dd23-417c-b12b-0e98de86439f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.379211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c45c678c1c413ebf61b01dddce09ad', 'request_id': 'req-91e8709a-b375-4b1c-871d-b5c2afbc6c03'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:55.384574 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3337/6756] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:55.385601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a10ae-dd23-417c-b12b-0e98de86439f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.389433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c45c678c1c413ebf61b01dddce09ad', 'request_id': 'req-91e8709a-b375-4b1c-871d-b5c2afbc6c03'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:55.396804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c45c678c1c413ebf61b01dddce09ad', 'request_id': 'req-91e8709a-b375-4b1c-871d-b5c2afbc6c03'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:55.397576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a10ae-dd23-417c-b12b-0e98de86439f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.410349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project 37c45c678c1c413ebf61b01dddce09ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:55.410630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:55.411418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:55.413458 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3420/6757] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:55 2022] DELETE /identity/v3/projects/37c45c678c1c413ebf61b01dddce09ad => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:55.418410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.419977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:55.459581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06feb647-64ec-42bd-af2d-ac83ed7e7c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.460196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06feb647-64ec-42bd-af2d-ac83ed7e7c99 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.460520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06feb647-64ec-42bd-af2d-ac83ed7e7c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.460787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06feb647-64ec-42bd-af2d-ac83ed7e7c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.461420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06feb647-64ec-42bd-af2d-ac83ed7e7c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.463121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06feb647-64ec-42bd-af2d-ac83ed7e7c99 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.474948 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3338/6758] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:55.480048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06feb647-64ec-42bd-af2d-ac83ed7e7c99 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.481949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-06feb647-64ec-42bd-af2d-ac83ed7e7c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:55.494698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06feb647-64ec-42bd-af2d-ac83ed7e7c99 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.502418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8709a-b375-4b1c-871d-b5c2afbc6c03 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.502546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12756f00-ffff-4dd9-bb48-f6bc3332956b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.503216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12756f00-ffff-4dd9-bb48-f6bc3332956b admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.503497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12756f00-ffff-4dd9-bb48-f6bc3332956b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.503828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12756f00-ffff-4dd9-bb48-f6bc3332956b admin admin] PATH_INFO: `/v3/projects/2eee1e4238484dd9a017174f71e9983f/users/d0cc23cdd2024684b73e97e65816bfbc/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.509429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b504997-4304-49d2-a663-3f71c63ebde5 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.510291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b504997-4304-49d2-a663-3f71c63ebde5 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.510460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b504997-4304-49d2-a663-3f71c63ebde5 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.510627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b504997-4304-49d2-a663-3f71c63ebde5 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.511334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b504997-4304-49d2-a663-3f71c63ebde5 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.513866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b504997-4304-49d2-a663-3f71c63ebde5 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.521188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12756f00-ffff-4dd9-bb48-f6bc3332956b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eee1e4238484dd9a017174f71e9983f, user_id=d0cc23cdd2024684b73e97e65816bfbc, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.524095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12756f00-ffff-4dd9-bb48-f6bc3332956b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.539581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:55.543111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:55.555179 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3339/6759] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:55 2022] PUT /identity/v3/projects/2eee1e4238484dd9a017174f71e9983f/users/d0cc23cdd2024684b73e97e65816bfbc/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:55.558143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57b93f-521b-4a22-8461-93a282081fc1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.558506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57b93f-521b-4a22-8461-93a282081fc1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.558873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57b93f-521b-4a22-8461-93a282081fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.564772 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3421/6760] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:55 2022] POST /identity/v3/projects => generated 359 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:55.567069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b504997-4304-49d2-a663-3f71c63ebde5 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.603673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-de57b93f-521b-4a22-8461-93a282081fc1 None None] MFA Rules not processed for user `d0cc23cdd2024684b73e97e65816bfbc`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:55.605115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-de57b93f-521b-4a22-8461-93a282081fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:55.659774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b504997-4304-49d2-a663-3f71c63ebde5 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.664813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53e49b-30cb-471f-a848-c7b47c853210 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.665468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53e49b-30cb-471f-a848-c7b47c853210 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.665746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53e49b-30cb-471f-a848-c7b47c853210 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.666052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53e49b-30cb-471f-a848-c7b47c853210 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/12d508555bc64afaa27b6a5f66fab3c2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.667377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53e49b-30cb-471f-a848-c7b47c853210 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:update_project(project_id=12d508555bc64afaa27b6a5f66fab3c2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.669612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53e49b-30cb-471f-a848-c7b47c853210 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.682508 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3340/6761] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:55.687103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57b93f-521b-4a22-8461-93a282081fc1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.731950 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3422/6762] 10.222.0.23 () {60 vars in 1252 bytes} [Sat Aug 20 21:21:55 2022] PATCH /identity/v3/projects/12d508555bc64afaa27b6a5f66fab3c2 => generated 371 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:55.733071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53e49b-30cb-471f-a848-c7b47c853210 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.744260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53e49b-30cb-471f-a848-c7b47c853210 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.749743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424c0fb-b155-433a-bf87-45eea20909fa tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.750375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424c0fb-b155-433a-bf87-45eea20909fa tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.750999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424c0fb-b155-433a-bf87-45eea20909fa tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.750999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424c0fb-b155-433a-bf87-45eea20909fa tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/12d508555bc64afaa27b6a5f66fab3c2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.761284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57b93f-521b-4a22-8461-93a282081fc1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.762657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424c0fb-b155-433a-bf87-45eea20909fa tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:get_project(project_id=12d508555bc64afaa27b6a5f66fab3c2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.764927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424c0fb-b155-433a-bf87-45eea20909fa tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.766556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82434c1-deb8-4e0c-b1a6-ac0e6af55764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.767125 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3423/6763] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:55 2022] GET /identity/v3/projects/12d508555bc64afaa27b6a5f66fab3c2 => generated 358 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:55.768646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82434c1-deb8-4e0c-b1a6-ac0e6af55764 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.769082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82434c1-deb8-4e0c-b1a6-ac0e6af55764 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.769567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82434c1-deb8-4e0c-b1a6-ac0e6af55764 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.770359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82434c1-deb8-4e0c-b1a6-ac0e6af55764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.771406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424c0fb-b155-433a-bf87-45eea20909fa tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.772661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82434c1-deb8-4e0c-b1a6-ac0e6af55764 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.782156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424c0fb-b155-433a-bf87-45eea20909fa tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.787887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.788757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.789035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.789300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/12d508555bc64afaa27b6a5f66fab3c2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.790880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_project(project_id=12d508555bc64afaa27b6a5f66fab3c2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.795472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.812133 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3341/6764] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:55 2022] POST /identity/v3/projects => generated 389 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:55.816113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82434c1-deb8-4e0c-b1a6-ac0e6af55764 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.849942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12d508555bc64afaa27b6a5f66fab3c2', 'request_id': 'req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:55.866311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12d508555bc64afaa27b6a5f66fab3c2', 'request_id': 'req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:55.872940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12d508555bc64afaa27b6a5f66fab3c2', 'request_id': 'req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:55.887627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project 12d508555bc64afaa27b6a5f66fab3c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:55.888061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:55.888781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:55.890423 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3424/6765] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:55 2022] DELETE /identity/v3/projects/12d508555bc64afaa27b6a5f66fab3c2 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:55.895531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82434c1-deb8-4e0c-b1a6-ac0e6af55764 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.895716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:55.897621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:55.916462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:55.943000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:UPDATE=1,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:55.966450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e87d2-40d3-4b1f-9ed1-a26739c70e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.967754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e87d2-40d3-4b1f-9ed1-a26739c70e8b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.968182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e87d2-40d3-4b1f-9ed1-a26739c70e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.968489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e87d2-40d3-4b1f-9ed1-a26739c70e8b admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.969133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e87d2-40d3-4b1f-9ed1-a26739c70e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.971056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e87d2-40d3-4b1f-9ed1-a26739c70e8b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.983553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a73c0d-e16c-4c01-8324-0cc5accc7c0b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:55.989226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f73e26-0ab3-419c-a23e-91ac399ac48b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:55.990239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f73e26-0ab3-419c-a23e-91ac399ac48b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:55.990585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f73e26-0ab3-419c-a23e-91ac399ac48b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:55.991324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f73e26-0ab3-419c-a23e-91ac399ac48b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:55.991507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f73e26-0ab3-419c-a23e-91ac399ac48b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:55.993402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f73e26-0ab3-419c-a23e-91ac399ac48b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:55.995666 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3342/6766] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:55 2022] POST /identity/v3/users => generated 301 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:55.999684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e87d2-40d3-4b1f-9ed1-a26739c70e8b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.001386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8c2e87d2-40d3-4b1f-9ed1-a26739c70e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:56.013675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e87d2-40d3-4b1f-9ed1-a26739c70e8b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.018687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3785605-4e5d-485d-ad54-8362ee74c864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.019274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3785605-4e5d-485d-ad54-8362ee74c864 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.019547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3785605-4e5d-485d-ad54-8362ee74c864 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.019816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3785605-4e5d-485d-ad54-8362ee74c864 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.020506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3785605-4e5d-485d-ad54-8362ee74c864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:56.022137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3785605-4e5d-485d-ad54-8362ee74c864 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:56.030758 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3425/6767] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:55 2022] POST /identity/v3/projects => generated 354 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:56.034467 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3343/6768] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:21:56.035223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f73e26-0ab3-419c-a23e-91ac399ac48b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.039284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3785605-4e5d-485d-ad54-8362ee74c864 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.050585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3785605-4e5d-485d-ad54-8362ee74c864 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.107285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f73e26-0ab3-419c-a23e-91ac399ac48b tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.112957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f7491-7823-4f3a-9e9e-a712dc8055e2 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.113683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f7491-7823-4f3a-9e9e-a712dc8055e2 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.113959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f7491-7823-4f3a-9e9e-a712dc8055e2 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.114225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f7491-7823-4f3a-9e9e-a712dc8055e2 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/2eadc49d35c94ee183f779e82c7a153f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.115448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f7491-7823-4f3a-9e9e-a712dc8055e2 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:update_project(project_id=2eadc49d35c94ee183f779e82c7a153f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:56.117808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f7491-7823-4f3a-9e9e-a712dc8055e2 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:56.119938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a334e-091e-4aaf-8dbc-afb6e66824c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.120745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a334e-091e-4aaf-8dbc-afb6e66824c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.121070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a334e-091e-4aaf-8dbc-afb6e66824c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.121332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a334e-091e-4aaf-8dbc-afb6e66824c2 admin admin] PATH_INFO: `/v3/projects/89b6248c80f843dd955b76655003e603/users/e267750534414123a0586062eb93739f/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.137990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a334e-091e-4aaf-8dbc-afb6e66824c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b6248c80f843dd955b76655003e603, user_id=e267750534414123a0586062eb93739f, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:56.140700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a334e-091e-4aaf-8dbc-afb6e66824c2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:56.155947 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3426/6769] 10.222.0.23 () {60 vars in 1252 bytes} [Sat Aug 20 21:21:56 2022] PATCH /identity/v3/projects/2eadc49d35c94ee183f779e82c7a153f => generated 368 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:56.157568 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3344/6770] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:56 2022] PUT /identity/v3/projects/89b6248c80f843dd955b76655003e603/users/e267750534414123a0586062eb93739f/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:56.160568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f7491-7823-4f3a-9e9e-a712dc8055e2 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.161973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a334e-091e-4aaf-8dbc-afb6e66824c2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.173215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f7491-7823-4f3a-9e9e-a712dc8055e2 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.239621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a334e-091e-4aaf-8dbc-afb6e66824c2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.241999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd68f6-b722-4844-bfe9-d092159e3ed0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.242755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd68f6-b722-4844-bfe9-d092159e3ed0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.243010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd68f6-b722-4844-bfe9-d092159e3ed0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.243254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd68f6-b722-4844-bfe9-d092159e3ed0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/2eadc49d35c94ee183f779e82c7a153f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.245498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9c0c9-00c5-4a1c-b29e-95dc782c05ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.246311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9c0c9-00c5-4a1c-b29e-95dc782c05ea admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.246582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9c0c9-00c5-4a1c-b29e-95dc782c05ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.247008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9c0c9-00c5-4a1c-b29e-95dc782c05ea admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.247629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9c0c9-00c5-4a1c-b29e-95dc782c05ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:56.249602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9c0c9-00c5-4a1c-b29e-95dc782c05ea admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:56.254431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd68f6-b722-4844-bfe9-d092159e3ed0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:get_project(project_id=2eadc49d35c94ee183f779e82c7a153f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:56.256602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd68f6-b722-4844-bfe9-d092159e3ed0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:56.258634 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3427/6771] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:21:56 2022] GET /identity/v3/projects/2eadc49d35c94ee183f779e82c7a153f => generated 355 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:21:56.259128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd68f6-b722-4844-bfe9-d092159e3ed0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.260924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8bdd68f6-b722-4844-bfe9-d092159e3ed0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:56.263467 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3345/6772] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:56.264279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9c0c9-00c5-4a1c-b29e-95dc782c05ea admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.274891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd68f6-b722-4844-bfe9-d092159e3ed0 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.275617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9c0c9-00c5-4a1c-b29e-95dc782c05ea admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.279470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.280091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.280363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.280660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] PATH_INFO: `/v3/users/45bedd6ddcef4a9c942c5e0a63565481` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.280847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.281480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.281747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.281858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=45bedd6ddcef4a9c942c5e0a63565481)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:56.282051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/projects/2eadc49d35c94ee183f779e82c7a153f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.283186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2eadc49d35c94ee183f779e82c7a153f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:56.283634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:56.285156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:56.303346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:56.312591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:UPDATE=1,keystone:DELETE=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:56.335199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45bedd6ddcef4a9c942c5e0a63565481'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.336549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eadc49d35c94ee183f779e82c7a153f', 'request_id': 'req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.342790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45bedd6ddcef4a9c942c5e0a63565481'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.345153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eadc49d35c94ee183f779e82c7a153f', 'request_id': 'req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.350240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45bedd6ddcef4a9c942c5e0a63565481'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.354498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eadc49d35c94ee183f779e82c7a153f', 'request_id': 'req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.363392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45bedd6ddcef4a9c942c5e0a63565481'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.364362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45bedd6ddcef4a9c942c5e0a63565481'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.367338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] The token cache is being invalidate because project 2eadc49d35c94ee183f779e82c7a153f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:56.367821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.368891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.371454 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3346/6773] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:56 2022] DELETE /identity/v3/projects/2eadc49d35c94ee183f779e82c7a153f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:56.371611 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3428/6774] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:56 2022] DELETE /identity/v3/users/45bedd6ddcef4a9c942c5e0a63565481 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:56.376088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.377516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:56.382056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.383559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:56.455237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce6938-8fa0-4ce3-9003-cc9fbd5afb27 tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.466676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4bc39-d11c-404e-8251-523d4a2a91ff admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.474292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44b259-0aff-4f05-ace6-fed2dd623c0d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.475048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44b259-0aff-4f05-ace6-fed2dd623c0d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.475406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44b259-0aff-4f05-ace6-fed2dd623c0d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.475747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44b259-0aff-4f05-ace6-fed2dd623c0d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/domains/66e472dc7aad4ab9acc5edcd46d49dda` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.483737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44b259-0aff-4f05-ace6-fed2dd623c0d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-8e44b259-0aff-4f05-ace6-fed2dd623c0d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Could not find domain: 66e472dc7aad4ab9acc5edcd46d49dda.: keystone.exception.DomainNotFound: Could not find domain: 66e472dc7aad4ab9acc5edcd46d49dda. Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 66e472dc7aad4ab9acc5edcd46d49dda. Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:21:56.488792 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:21:56.490090 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 66e472dc7aad4ab9acc5edcd46d49dda. Aug 20 21:21:56.491151 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:21:56.492257 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3429/6775] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:21:56 2022] PATCH /identity/v3/domains/66e472dc7aad4ab9acc5edcd46d49dda => generated 112 bytes in 109 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:21:56.494564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44b259-0aff-4f05-ace6-fed2dd623c0d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.496504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8e44b259-0aff-4f05-ace6-fed2dd623c0d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:56.513727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44b259-0aff-4f05-ace6-fed2dd623c0d tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.519413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c565283-0819-4278-a9a7-27fe6cddf16f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e3b1ab4f4c1483fbfdad1ab6e896d4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '532c743f753040f39578019e11be8ed7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.520084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c565283-0819-4278-a9a7-27fe6cddf16f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.520361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c565283-0819-4278-a9a7-27fe6cddf16f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.520825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c565283-0819-4278-a9a7-27fe6cddf16f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] PATH_INFO: `/v3/domains/f9a396369f8d4c58be30774ddeec61fe` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.525357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7baf4-107b-4ec6-a397-1f2b7ab786b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.526109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7baf4-107b-4ec6-a397-1f2b7ab786b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.526583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7baf4-107b-4ec6-a397-1f2b7ab786b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.527024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7baf4-107b-4ec6-a397-1f2b7ab786b9 admin admin] PATH_INFO: `/v3/projects/89b6248c80f843dd955b76655003e603/users/e267750534414123a0586062eb93739f/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.529034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c565283-0819-4278-a9a7-27fe6cddf16f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:21:56.529703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7baf4-107b-4ec6-a397-1f2b7ab786b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b6248c80f843dd955b76655003e603, user_id=e267750534414123a0586062eb93739f, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:56.532914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7baf4-107b-4ec6-a397-1f2b7ab786b9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-5c565283-0819-4278-a9a7-27fe6cddf16f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Could not find domain: f9a396369f8d4c58be30774ddeec61fe.: keystone.exception.DomainNotFound: Could not find domain: f9a396369f8d4c58be30774ddeec61fe. Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f9a396369f8d4c58be30774ddeec61fe. Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:21:56.533943 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:21:56.535071 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: f9a396369f8d4c58be30774ddeec61fe. Aug 20 21:21:56.536110 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:21:56.537818 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3430/6776] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:21:56 2022] PATCH /identity/v3/domains/f9a396369f8d4c58be30774ddeec61fe => generated 112 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Aug 20 21:21:56.542000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c565283-0819-4278-a9a7-27fe6cddf16f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.543959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5c565283-0819-4278-a9a7-27fe6cddf16f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:56.547238 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3347/6777] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:56 2022] PUT /identity/v3/projects/89b6248c80f843dd955b76655003e603/users/e267750534414123a0586062eb93739f/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:56.557648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39002b6-c837-497d-9af8-d6453b06067b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.558009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39002b6-c837-497d-9af8-d6453b06067b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.558604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39002b6-c837-497d-9af8-d6453b06067b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.568509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c565283-0819-4278-a9a7-27fe6cddf16f tempest-ProjectsTestJSON-2089547408 tempest-ProjectsTestJSON-2089547408-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.595549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d39002b6-c837-497d-9af8-d6453b06067b None None] MFA Rules not processed for user `e267750534414123a0586062eb93739f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:56.597298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d39002b6-c837-497d-9af8-d6453b06067b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:56.636754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:56.639053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.639854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.640198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.640564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] PATH_INFO: `/v3/users/82ac9008f360490b923e189fb11c79a1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.641784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] RBAC: Authorizing `identity:delete_user(user_id=82ac9008f360490b923e189fb11c79a1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:56.643988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:56.676175 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3348/6778] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:56 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:56.677346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39002b6-c837-497d-9af8-d6453b06067b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.679701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d39002b6-c837-497d-9af8-d6453b06067b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:56.687655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:56.696911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39002b6-c837-497d-9af8-d6453b06067b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.700313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82ac9008f360490b923e189fb11c79a1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.702157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.702803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.703075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.703344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4 admin admin] PATH_INFO: `/v3/projects/99f4196542c5464c86c83894720ebd3b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.704661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99f4196542c5464c86c83894720ebd3b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:56.706788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:56.708597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82ac9008f360490b923e189fb11c79a1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.715785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82ac9008f360490b923e189fb11c79a1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.727670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82ac9008f360490b923e189fb11c79a1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.728361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82ac9008f360490b923e189fb11c79a1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.734302 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3431/6779] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:56 2022] DELETE /identity/v3/users/82ac9008f360490b923e189fb11c79a1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:21:56.735216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.736705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:56.757123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99f4196542c5464c86c83894720ebd3b', 'request_id': 'req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.763583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99f4196542c5464c86c83894720ebd3b', 'request_id': 'req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.772318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99f4196542c5464c86c83894720ebd3b', 'request_id': 'req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.784095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4 admin admin] The token cache is being invalidate because project 99f4196542c5464c86c83894720ebd3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:56.784431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.785075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-26ed19ec-0e4f-472b-bbbb-c2d5f4412ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.793456 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3349/6780] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:56 2022] DELETE /identity/v3/projects/99f4196542c5464c86c83894720ebd3b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:56.793456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff07cd9-942f-4bff-aeca-5063811e2a7d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.793456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff07cd9-942f-4bff-aeca-5063811e2a7d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.793456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff07cd9-942f-4bff-aeca-5063811e2a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.803127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9feeb49-0aef-4b68-b6fe-5ee39a75b44a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.807804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-dff07cd9-942f-4bff-aeca-5063811e2a7d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:56.809035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dff07cd9-942f-4bff-aeca-5063811e2a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:56.857307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129ed62-2dfa-478d-9021-424982fb557b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.857890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129ed62-2dfa-478d-9021-424982fb557b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.858159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129ed62-2dfa-478d-9021-424982fb557b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.858442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129ed62-2dfa-478d-9021-424982fb557b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.860401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1129ed62-2dfa-478d-9021-424982fb557b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:56.872031 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3350/6781] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:56.872986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff07cd9-942f-4bff-aeca-5063811e2a7d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.874317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dff07cd9-942f-4bff-aeca-5063811e2a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:56.886374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff07cd9-942f-4bff-aeca-5063811e2a7d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.891180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.891837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.892135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.892418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] PATH_INFO: `/v3/users/7eea1ee716fc4b30b2023ad90336fd85` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.893655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eea1ee716fc4b30b2023ad90336fd85)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:56.895575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:56.925734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129ed62-2dfa-478d-9021-424982fb557b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:56.927472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129ed62-2dfa-478d-9021-424982fb557b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:56.938203 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3432/6782] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:56 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:56.939034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129ed62-2dfa-478d-9021-424982fb557b service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:56.940481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1129ed62-2dfa-478d-9021-424982fb557b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:56.947139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eea1ee716fc4b30b2023ad90336fd85'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.952893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:56.956148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129ed62-2dfa-478d-9021-424982fb557b service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:56.960951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8238ff7-dfa4-4050-af95-da780a2a9e3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:56.961647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8238ff7-dfa4-4050-af95-da780a2a9e3f service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:56.962010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8238ff7-dfa4-4050-af95-da780a2a9e3f service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:56.962567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8238ff7-dfa4-4050-af95-da780a2a9e3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:56.965507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eea1ee716fc4b30b2023ad90336fd85'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.981144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8238ff7-dfa4-4050-af95-da780a2a9e3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:56.982305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eea1ee716fc4b30b2023ad90336fd85'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.982765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8238ff7-dfa4-4050-af95-da780a2a9e3f service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:56.983155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eea1ee716fc4b30b2023ad90336fd85'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.983997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eea1ee716fc4b30b2023ad90336fd85'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:56.993957 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3433/6783] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:57.008560 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3351/6784] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:56 2022] DELETE /identity/v3/users/7eea1ee716fc4b30b2023ad90336fd85 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:57.209616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8238ff7-dfa4-4050-af95-da780a2a9e3f service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:57.220484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8238ff7-dfa4-4050-af95-da780a2a9e3f service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:57.227056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:57.308500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:57.309155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:57.309424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:57.309700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] PATH_INFO: `/v3/projects/2cc801be22724963afad7aba7c516d19` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:57.311991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc801be22724963afad7aba7c516d19)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:57.314272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:57.375845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc801be22724963afad7aba7c516d19', 'request_id': 'req-c52a81d5-2007-4f9f-8fef-c84d75304ecc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:57.391447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc801be22724963afad7aba7c516d19', 'request_id': 'req-c52a81d5-2007-4f9f-8fef-c84d75304ecc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:57.406987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc801be22724963afad7aba7c516d19', 'request_id': 'req-c52a81d5-2007-4f9f-8fef-c84d75304ecc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:57.424356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] The token cache is being invalidate because project 2cc801be22724963afad7aba7c516d19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:57.425241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:57.427418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:57.443982 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3434/6785] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:57 2022] DELETE /identity/v3/projects/2cc801be22724963afad7aba7c516d19 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:57.503859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:57.505740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:57.523777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab7ff7-fe1a-45a2-a744-3d0a11e81ccf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:57.617773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:57.618731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:57.619312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:57.619734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] PATH_INFO: `/v3/projects/a2b33e1eb4eb494384961b9293a64c71` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:57.621827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b33e1eb4eb494384961b9293a64c71)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:57.624048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:57.732470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b33e1eb4eb494384961b9293a64c71', 'request_id': 'req-34d19108-a53f-40bd-a94b-8b0c527b0004'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:57.741857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b33e1eb4eb494384961b9293a64c71', 'request_id': 'req-34d19108-a53f-40bd-a94b-8b0c527b0004'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:57.757281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b33e1eb4eb494384961b9293a64c71', 'request_id': 'req-34d19108-a53f-40bd-a94b-8b0c527b0004'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:57.795208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] The token cache is being invalidate because project a2b33e1eb4eb494384961b9293a64c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:57.795647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:57.796462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:57.798616 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3352/6786] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:57 2022] DELETE /identity/v3/projects/a2b33e1eb4eb494384961b9293a64c71 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:57.802728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:57.808510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:57.823445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a81d5-2007-4f9f-8fef-c84d75304ecc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:57.896243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c8715-5252-44d7-8604-542583f3e553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:57.897803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c8715-5252-44d7-8604-542583f3e553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:57.898678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c8715-5252-44d7-8604-542583f3e553 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:57.899481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c8715-5252-44d7-8604-542583f3e553 admin admin] PATH_INFO: `/v3/users/0e3b1ab4f4c1483fbfdad1ab6e896d4d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:57.901155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c8715-5252-44d7-8604-542583f3e553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e3b1ab4f4c1483fbfdad1ab6e896d4d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:57.905685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c8715-5252-44d7-8604-542583f3e553 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:57.968187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:57.985618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-126c8715-5252-44d7-8604-542583f3e553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e3b1ab4f4c1483fbfdad1ab6e896d4d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:57.995866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-126c8715-5252-44d7-8604-542583f3e553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e3b1ab4f4c1483fbfdad1ab6e896d4d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:58.013230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-126c8715-5252-44d7-8604-542583f3e553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e3b1ab4f4c1483fbfdad1ab6e896d4d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:58.028118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-126c8715-5252-44d7-8604-542583f3e553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e3b1ab4f4c1483fbfdad1ab6e896d4d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:58.028918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-126c8715-5252-44d7-8604-542583f3e553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e3b1ab4f4c1483fbfdad1ab6e896d4d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:58.035825 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3435/6787] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:21:57 2022] DELETE /identity/v3/users/0e3b1ab4f4c1483fbfdad1ab6e896d4d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:58.602721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:58.604261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:58.631012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d19108-a53f-40bd-a94b-8b0c527b0004 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:58.728324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:58.747412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:58.748439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:58.748725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:58.748987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc admin admin] PATH_INFO: `/v3/projects/532c743f753040f39578019e11be8ed7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:58.750221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=532c743f753040f39578019e11be8ed7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:58.752154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:58.824310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '532c743f753040f39578019e11be8ed7', 'request_id': 'req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:58.831528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '532c743f753040f39578019e11be8ed7', 'request_id': 'req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:58.843733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '532c743f753040f39578019e11be8ed7', 'request_id': 'req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:58.857822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc admin admin] The token cache is being invalidate because project 532c743f753040f39578019e11be8ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:21:58.858157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:58.858892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9140d823-e74c-4f63-a78c-3e2b8d1d53bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:21:58.860743 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3353/6788] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:21:58 2022] DELETE /identity/v3/projects/532c743f753040f39578019e11be8ed7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:58.868016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d21256-71f7-43bb-88c6-e5d8abe99397 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:58.868988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d21256-71f7-43bb-88c6-e5d8abe99397 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:58.868988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d21256-71f7-43bb-88c6-e5d8abe99397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:58.895991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f6d21256-71f7-43bb-88c6-e5d8abe99397 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:58.897325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f6d21256-71f7-43bb-88c6-e5d8abe99397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:59.018066 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3436/6789] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:59.028512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8a1ae-2f11-400f-8388-8803e907437e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:59.029524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8a1ae-2f11-400f-8388-8803e907437e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:59.029822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8a1ae-2f11-400f-8388-8803e907437e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:59.062011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-41d8a1ae-2f11-400f-8388-8803e907437e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:59.064469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-41d8a1ae-2f11-400f-8388-8803e907437e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:59.077878 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3354/6790] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:59.083514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d21256-71f7-43bb-88c6-e5d8abe99397 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:59.095736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d21256-71f7-43bb-88c6-e5d8abe99397 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:59.104226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa93233-7a4f-4572-81a8-dd94f6680789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:59.105169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa93233-7a4f-4572-81a8-dd94f6680789 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:59.105596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa93233-7a4f-4572-81a8-dd94f6680789 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:59.106012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa93233-7a4f-4572-81a8-dd94f6680789 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:59.107433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa93233-7a4f-4572-81a8-dd94f6680789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:21:59.107731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa93233-7a4f-4572-81a8-dd94f6680789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:59.113534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa93233-7a4f-4572-81a8-dd94f6680789 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:59.127309 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3437/6791] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:21:59.131760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8a1ae-2f11-400f-8388-8803e907437e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:59.144100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8a1ae-2f11-400f-8388-8803e907437e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:59.149923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82877d6-ca76-42a5-80d9-299246123ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:59.150285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82877d6-ca76-42a5-80d9-299246123ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:59.150644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82877d6-ca76-42a5-80d9-299246123ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:59.150997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82877d6-ca76-42a5-80d9-299246123ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:59.151691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82877d6-ca76-42a5-80d9-299246123ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:59.153764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82877d6-ca76-42a5-80d9-299246123ac4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:59.198550 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3355/6792] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:21:59 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:59.202362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa93233-7a4f-4572-81a8-dd94f6680789 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:59.217176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa93233-7a4f-4572-81a8-dd94f6680789 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:59.307403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca9322-de5a-42ce-9e81-66d358b6d1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:59.308905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca9322-de5a-42ce-9e81-66d358b6d1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:59.308905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca9322-de5a-42ce-9e81-66d358b6d1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:59.308905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca9322-de5a-42ce-9e81-66d358b6d1e5 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:59.309606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca9322-de5a-42ce-9e81-66d358b6d1e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:59.311230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca9322-de5a-42ce-9e81-66d358b6d1e5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:59.344235 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3438/6793] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:21:59 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:21:59.346685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82877d6-ca76-42a5-80d9-299246123ac4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:59.363209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82877d6-ca76-42a5-80d9-299246123ac4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:59.371499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b56516-820d-4c0c-be88-a6ed9d4c4b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:59.372203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b56516-820d-4c0c-be88-a6ed9d4c4b9e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:59.372557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b56516-820d-4c0c-be88-a6ed9d4c4b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:59.372824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b56516-820d-4c0c-be88-a6ed9d4c4b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:59.374865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b56516-820d-4c0c-be88-a6ed9d4c4b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:59.376607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b56516-820d-4c0c-be88-a6ed9d4c4b9e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:59.394346 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3356/6794] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:59.395584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca9322-de5a-42ce-9e81-66d358b6d1e5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:59.407269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca9322-de5a-42ce-9e81-66d358b6d1e5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:59.416901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ae654-d245-4a64-b48d-4ad65c9458af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:59.417794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ae654-d245-4a64-b48d-4ad65c9458af admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:59.418069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ae654-d245-4a64-b48d-4ad65c9458af admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:59.419076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ae654-d245-4a64-b48d-4ad65c9458af admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:59.419741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ae654-d245-4a64-b48d-4ad65c9458af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:59.421495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ae654-d245-4a64-b48d-4ad65c9458af admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:59.434111 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3439/6795] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:21:59.438599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b56516-820d-4c0c-be88-a6ed9d4c4b9e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:59.451130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b56516-820d-4c0c-be88-a6ed9d4c4b9e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:59.464337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bfcd8-439a-4c32-abad-b4636e6af50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:59.465652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bfcd8-439a-4c32-abad-b4636e6af50d admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:59.466095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bfcd8-439a-4c32-abad-b4636e6af50d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:59.466645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bfcd8-439a-4c32-abad-b4636e6af50d admin admin] PATH_INFO: `/v3/projects/5b092a844a4a4f18b29596fed8f7969c/users/93adb1bb96d340b1b74701cd2d2b11a1/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:59.485465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9bfcd8-439a-4c32-abad-b4636e6af50d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b092a844a4a4f18b29596fed8f7969c, user_id=93adb1bb96d340b1b74701cd2d2b11a1, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:59.491069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9bfcd8-439a-4c32-abad-b4636e6af50d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:59.511815 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3357/6796] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:21:59 2022] PUT /identity/v3/projects/5b092a844a4a4f18b29596fed8f7969c/users/93adb1bb96d340b1b74701cd2d2b11a1/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:21:59.525038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483590b5-5cc0-465c-82f7-971b5645735a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:59.525385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483590b5-5cc0-465c-82f7-971b5645735a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:59.525907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483590b5-5cc0-465c-82f7-971b5645735a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:59.552138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bfcd8-439a-4c32-abad-b4636e6af50d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:59.554151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9c9bfcd8-439a-4c32-abad-b4636e6af50d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:59.582643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-483590b5-5cc0-465c-82f7-971b5645735a None None] MFA Rules not processed for user `93adb1bb96d340b1b74701cd2d2b11a1`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:21:59.584921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-483590b5-5cc0-465c-82f7-971b5645735a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:59.612421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:59.658061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bfcd8-439a-4c32-abad-b4636e6af50d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:59.672176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497f686-b27c-4612-98a3-cc581423485d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:59.673001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497f686-b27c-4612-98a3-cc581423485d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:59.673273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497f686-b27c-4612-98a3-cc581423485d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:59.673534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497f686-b27c-4612-98a3-cc581423485d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:59.675554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9497f686-b27c-4612-98a3-cc581423485d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:21:59.691408 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3440/6797] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:21:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:21:59.696995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483590b5-5cc0-465c-82f7-971b5645735a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:59.707569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483590b5-5cc0-465c-82f7-971b5645735a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:59.713251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3ab0b-538a-41b0-9f14-2e0a73554fdc tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93adb1bb96d340b1b74701cd2d2b11a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b092a844a4a4f18b29596fed8f7969c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:59.714075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3ab0b-538a-41b0-9f14-2e0a73554fdc tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:59.714185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3ab0b-538a-41b0-9f14-2e0a73554fdc tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:59.714423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3ab0b-538a-41b0-9f14-2e0a73554fdc tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] PATH_INFO: `/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/application_credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:59.715164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3ab0b-538a-41b0-9f14-2e0a73554fdc tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorizing `identity:create_application_credential(user_id=93adb1bb96d340b1b74701cd2d2b11a1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:59.717888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3ab0b-538a-41b0-9f14-2e0a73554fdc tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:59.730479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:21:59.782277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497f686-b27c-4612-98a3-cc581423485d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:59.783858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497f686-b27c-4612-98a3-cc581423485d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:59.805410 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3358/6798] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:21:59 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:21:59.843047 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3441/6799] 10.222.0.23 () {60 vars in 1315 bytes} [Sat Aug 20 21:21:59 2022] POST /identity/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/application_credentials => generated 920 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:21:59.848792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497f686-b27c-4612-98a3-cc581423485d service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:59.861629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497f686-b27c-4612-98a3-cc581423485d service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:59.866748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf52b5-36dd-4ed8-abe9-08a1c048f3c8 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93adb1bb96d340b1b74701cd2d2b11a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b092a844a4a4f18b29596fed8f7969c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:59.867707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf52b5-36dd-4ed8-abe9-08a1c048f3c8 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:59.870518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf52b5-36dd-4ed8-abe9-08a1c048f3c8 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:59.870818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf52b5-36dd-4ed8-abe9-08a1c048f3c8 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] PATH_INFO: `/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/access_rules/02baec5dbbf3449997e01e41245b1b9b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:59.872150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf52b5-36dd-4ed8-abe9-08a1c048f3c8 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorizing `identity:delete_access_rule(user_id=93adb1bb96d340b1b74701cd2d2b11a1, access_rule_id=02baec5dbbf3449997e01e41245b1b9b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:59.873583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf52b5-36dd-4ed8-abe9-08a1c048f3c8 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-c9bf52b5-36dd-4ed8-abe9-08a1c048f3c8 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] May not delete access rule in use: keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 256, in delete_access_rule Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application session.delete(ref) Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application next(self.gen) Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application yield resource Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application next(self.gen) Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.session.rollback() Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.force_reraise() Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 20 21:21:59.927268 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise self.value Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application session.commit() Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._prepare_impl() Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.session.flush() Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._flush(objects) Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 20 21:21:59.928334 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application compat.raise_( Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application flush_context.execute() Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rec.execute(self) Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 577, in execute Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.dependency_processor.process_deletes(uow, states) Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 552, in process_deletes Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self._synchronize( Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 610, in _synchronize Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application sync.clear(dest, self.mapper, self.prop.synchronize_pairs) Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/sync.py", line 86, in clear Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise AssertionError( Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application AssertionError: Dependency rule tried to blank-out primary key column 'application_credential_access_rule.access_rule_id' on instance '' Aug 20 21:21:59.929467 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 763, in delete Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application PROVIDERS.application_credential_api.delete_access_rule( Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:21:59.930387 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:21:59.931326 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/core.py", line 252, in delete_access_rule Aug 20 21:21:59.931326 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application self.driver.delete_access_rule(access_rule_id) Aug 20 21:21:59.931326 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 258, in delete_access_rule Aug 20 21:21:59.931326 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Aug 20 21:21:59.931326 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Aug 20 21:21:59.931326 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:21:59.934033 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3359/6800] 10.222.0.23 () {58 vars in 1362 bytes} [Sat Aug 20 21:21:59 2022] DELETE /identity/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/access_rules/02baec5dbbf3449997e01e41245b1b9b => generated 89 bytes in 86 msecs (HTTP/1.1 403) 5 headers in 183 bytes (1 switches on core 0) Aug 20 21:21:59.938346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3ab0b-538a-41b0-9f14-2e0a73554fdc tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:21:59.951645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3ab0b-538a-41b0-9f14-2e0a73554fdc tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:21:59.958064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c78d6-a282-41fc-a505-50e76803fbee tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93adb1bb96d340b1b74701cd2d2b11a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b092a844a4a4f18b29596fed8f7969c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:21:59.958845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c78d6-a282-41fc-a505-50e76803fbee tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:21:59.959248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c78d6-a282-41fc-a505-50e76803fbee tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:21:59.959626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c78d6-a282-41fc-a505-50e76803fbee tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] PATH_INFO: `/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/application_credentials/01b9bb52b223468ba8e968467cdf779c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:21:59.975151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c78d6-a282-41fc-a505-50e76803fbee tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorizing `identity:delete_application_credential(user_id=93adb1bb96d340b1b74701cd2d2b11a1, application_credential_id=01b9bb52b223468ba8e968467cdf779c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:21:59.978853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c78d6-a282-41fc-a505-50e76803fbee tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:00.016156 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3442/6801] 10.222.0.23 () {58 vars in 1395 bytes} [Sat Aug 20 21:21:59 2022] DELETE /identity/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/application_credentials/01b9bb52b223468ba8e968467cdf779c => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:00.017402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf52b5-36dd-4ed8-abe9-08a1c048f3c8 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:00.036709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf52b5-36dd-4ed8-abe9-08a1c048f3c8 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:00.042457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f11e6-e44f-494d-a1cf-553de5cc2f36 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93adb1bb96d340b1b74701cd2d2b11a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b092a844a4a4f18b29596fed8f7969c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:00.043220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f11e6-e44f-494d-a1cf-553de5cc2f36 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:00.043572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f11e6-e44f-494d-a1cf-553de5cc2f36 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:00.043849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f11e6-e44f-494d-a1cf-553de5cc2f36 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] PATH_INFO: `/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/access_rules` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:00.045550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f11e6-e44f-494d-a1cf-553de5cc2f36 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorizing `identity:list_access_rules(user_id=93adb1bb96d340b1b74701cd2d2b11a1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:00.047128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f11e6-e44f-494d-a1cf-553de5cc2f36 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:00.054544 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3360/6802] 10.222.0.23 () {58 vars in 1260 bytes} [Sat Aug 20 21:22:00 2022] GET /identity/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/access_rules => generated 369 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:22:00.058058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c78d6-a282-41fc-a505-50e76803fbee tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:00.069778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c78d6-a282-41fc-a505-50e76803fbee tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:00.075003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ab3ef-33ad-49f9-9d74-fd0b0b6b4f81 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93adb1bb96d340b1b74701cd2d2b11a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b092a844a4a4f18b29596fed8f7969c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:00.076084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ab3ef-33ad-49f9-9d74-fd0b0b6b4f81 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:00.076558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ab3ef-33ad-49f9-9d74-fd0b0b6b4f81 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:00.077159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ab3ef-33ad-49f9-9d74-fd0b0b6b4f81 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] PATH_INFO: `/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/access_rules/02baec5dbbf3449997e01e41245b1b9b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:00.078976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ab3ef-33ad-49f9-9d74-fd0b0b6b4f81 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorizing `identity:delete_access_rule(user_id=93adb1bb96d340b1b74701cd2d2b11a1, access_rule_id=02baec5dbbf3449997e01e41245b1b9b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:00.080774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ab3ef-33ad-49f9-9d74-fd0b0b6b4f81 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:00.111032 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3443/6803] 10.222.0.23 () {58 vars in 1362 bytes} [Sat Aug 20 21:22:00 2022] DELETE /identity/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/access_rules/02baec5dbbf3449997e01e41245b1b9b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:00.115785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f11e6-e44f-494d-a1cf-553de5cc2f36 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:00.127090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f11e6-e44f-494d-a1cf-553de5cc2f36 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:00.141950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb1578-6d16-415b-8b02-22bf9cd85130 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93adb1bb96d340b1b74701cd2d2b11a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b092a844a4a4f18b29596fed8f7969c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:00.142614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1578-6d16-415b-8b02-22bf9cd85130 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:00.142893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1578-6d16-415b-8b02-22bf9cd85130 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:00.143160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1578-6d16-415b-8b02-22bf9cd85130 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] PATH_INFO: `/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/access_rules` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:00.151370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb1578-6d16-415b-8b02-22bf9cd85130 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorizing `identity:list_access_rules(user_id=93adb1bb96d340b1b74701cd2d2b11a1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:00.152636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb1578-6d16-415b-8b02-22bf9cd85130 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:00.158192 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3361/6804] 10.222.0.23 () {58 vars in 1260 bytes} [Sat Aug 20 21:22:00 2022] GET /identity/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/access_rules => generated 158 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:22:00.163985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ab3ef-33ad-49f9-9d74-fd0b0b6b4f81 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:00.175156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ab3ef-33ad-49f9-9d74-fd0b0b6b4f81 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:00.180951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11483b09-bfa9-4a8a-86c5-36aef619e674 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93adb1bb96d340b1b74701cd2d2b11a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b092a844a4a4f18b29596fed8f7969c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:00.181824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11483b09-bfa9-4a8a-86c5-36aef619e674 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:00.182444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11483b09-bfa9-4a8a-86c5-36aef619e674 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:00.182924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11483b09-bfa9-4a8a-86c5-36aef619e674 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] PATH_INFO: `/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/application_credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:00.183797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11483b09-bfa9-4a8a-86c5-36aef619e674 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorizing `identity:create_application_credential(user_id=93adb1bb96d340b1b74701cd2d2b11a1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:00.185418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11483b09-bfa9-4a8a-86c5-36aef619e674 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:00.271497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:00.323960 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3444/6805] 10.222.0.23 () {60 vars in 1315 bytes} [Sat Aug 20 21:22:00 2022] POST /identity/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/application_credentials => generated 921 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:00.325872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb1578-6d16-415b-8b02-22bf9cd85130 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:00.343529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb1578-6d16-415b-8b02-22bf9cd85130 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:00.351966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80f233-0b15-4ddf-bb2a-078d179360e5 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93adb1bb96d340b1b74701cd2d2b11a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b092a844a4a4f18b29596fed8f7969c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:00.352688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80f233-0b15-4ddf-bb2a-078d179360e5 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:00.353110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80f233-0b15-4ddf-bb2a-078d179360e5 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:00.353313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80f233-0b15-4ddf-bb2a-078d179360e5 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] PATH_INFO: `/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/access_rules` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:00.354567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80f233-0b15-4ddf-bb2a-078d179360e5 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorizing `identity:list_access_rules(user_id=93adb1bb96d340b1b74701cd2d2b11a1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:00.356124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80f233-0b15-4ddf-bb2a-078d179360e5 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:00.367358 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3362/6806] 10.222.0.23 () {58 vars in 1260 bytes} [Sat Aug 20 21:22:00 2022] GET /identity/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/access_rules => generated 369 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:22:00.368592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11483b09-bfa9-4a8a-86c5-36aef619e674 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:00.379968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11483b09-bfa9-4a8a-86c5-36aef619e674 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:00.385330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb95bc-88a5-4112-835b-7ed86db8437f tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93adb1bb96d340b1b74701cd2d2b11a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b092a844a4a4f18b29596fed8f7969c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:00.385504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb95bc-88a5-4112-835b-7ed86db8437f tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:00.385699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb95bc-88a5-4112-835b-7ed86db8437f tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:00.386319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb95bc-88a5-4112-835b-7ed86db8437f tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] PATH_INFO: `/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/application_credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:00.386627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb95bc-88a5-4112-835b-7ed86db8437f tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorizing `identity:create_application_credential(user_id=93adb1bb96d340b1b74701cd2d2b11a1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:00.387926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb95bc-88a5-4112-835b-7ed86db8437f tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:00.477799 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3445/6807] 10.222.0.23 () {60 vars in 1315 bytes} [Sat Aug 20 21:22:00 2022] POST /identity/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/application_credentials => generated 921 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:22:00.479094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80f233-0b15-4ddf-bb2a-078d179360e5 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:00.489831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80f233-0b15-4ddf-bb2a-078d179360e5 tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:00.500837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c229cb-9ca1-4bce-8a5f-ae009d38d52a tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93adb1bb96d340b1b74701cd2d2b11a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b092a844a4a4f18b29596fed8f7969c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:00.501512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c229cb-9ca1-4bce-8a5f-ae009d38d52a tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:00.501788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c229cb-9ca1-4bce-8a5f-ae009d38d52a tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:00.502097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c229cb-9ca1-4bce-8a5f-ae009d38d52a tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] PATH_INFO: `/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/access_rules/aa4e1456f24340f2a29cfbff5f44f391` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:00.503779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c229cb-9ca1-4bce-8a5f-ae009d38d52a tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorizing `identity:get_access_rule(user_id=93adb1bb96d340b1b74701cd2d2b11a1, access_rule_id=aa4e1456f24340f2a29cfbff5f44f391)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:00.506086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c229cb-9ca1-4bce-8a5f-ae009d38d52a tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:00.514510 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3363/6808] 10.222.0.23 () {58 vars in 1359 bytes} [Sat Aug 20 21:22:00 2022] GET /identity/v3/users/93adb1bb96d340b1b74701cd2d2b11a1/access_rules/aa4e1456f24340f2a29cfbff5f44f391 => generated 229 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:22:00.519726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb95bc-88a5-4112-835b-7ed86db8437f tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:00.665923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb95bc-88a5-4112-835b-7ed86db8437f tempest-AccessRulesV3Test-1790277548 tempest-AccessRulesV3Test-1790277548-project] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:00.684635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:00.685081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:00.685367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:00.685618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] PATH_INFO: `/v3/users/93adb1bb96d340b1b74701cd2d2b11a1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:00.686793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93adb1bb96d340b1b74701cd2d2b11a1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:00.688828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:00.786537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93adb1bb96d340b1b74701cd2d2b11a1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:00.803411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93adb1bb96d340b1b74701cd2d2b11a1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:00.810067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:INSERT=8,keystone:DELETE=8 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:00.813165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93adb1bb96d340b1b74701cd2d2b11a1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:00.927052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93adb1bb96d340b1b74701cd2d2b11a1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:00.929750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93adb1bb96d340b1b74701cd2d2b11a1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:00.944415 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3446/6809] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:00 2022] DELETE /identity/v3/users/93adb1bb96d340b1b74701cd2d2b11a1 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:00.948868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fd6f0-da33-49f2-98fd-3778aa62b8db None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:00.949260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fd6f0-da33-49f2-98fd-3778aa62b8db None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:00.949370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fd6f0-da33-49f2-98fd-3778aa62b8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:00.988661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a74fd6f0-da33-49f2-98fd-3778aa62b8db None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:00.990245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a74fd6f0-da33-49f2-98fd-3778aa62b8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:01.135536 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3364/6810] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:01.153178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:01.155124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:01.285286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402cadb-63f6-4eec-8ed9-cb65c08057a2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:01.291876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c143c88-15f8-4259-bcee-360846784cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:01.292556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c143c88-15f8-4259-bcee-360846784cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:01.292834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c143c88-15f8-4259-bcee-360846784cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:01.293100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c143c88-15f8-4259-bcee-360846784cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:01.314612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c143c88-15f8-4259-bcee-360846784cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:01.316083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c143c88-15f8-4259-bcee-360846784cc2 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:01.349036 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3447/6811] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:01.712578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fd6f0-da33-49f2-98fd-3778aa62b8db None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:01.714141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a74fd6f0-da33-49f2-98fd-3778aa62b8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:01.728939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fd6f0-da33-49f2-98fd-3778aa62b8db None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:01.734275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb7818-6231-437c-84eb-b8f825fdf208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:01.734958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb7818-6231-437c-84eb-b8f825fdf208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:01.735237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb7818-6231-437c-84eb-b8f825fdf208 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:01.735559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb7818-6231-437c-84eb-b8f825fdf208 admin admin] PATH_INFO: `/v3/projects/5b092a844a4a4f18b29596fed8f7969c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:01.736909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb7818-6231-437c-84eb-b8f825fdf208 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b092a844a4a4f18b29596fed8f7969c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:01.738831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb7818-6231-437c-84eb-b8f825fdf208 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:01.789093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-49eb7818-6231-437c-84eb-b8f825fdf208 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b092a844a4a4f18b29596fed8f7969c', 'request_id': 'req-49eb7818-6231-437c-84eb-b8f825fdf208'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:01.796702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-49eb7818-6231-437c-84eb-b8f825fdf208 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b092a844a4a4f18b29596fed8f7969c', 'request_id': 'req-49eb7818-6231-437c-84eb-b8f825fdf208'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:01.805472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-49eb7818-6231-437c-84eb-b8f825fdf208 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b092a844a4a4f18b29596fed8f7969c', 'request_id': 'req-49eb7818-6231-437c-84eb-b8f825fdf208'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:01.825537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-49eb7818-6231-437c-84eb-b8f825fdf208 admin admin] The token cache is being invalidate because project 5b092a844a4a4f18b29596fed8f7969c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:01.825830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-49eb7818-6231-437c-84eb-b8f825fdf208 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:01.826492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-49eb7818-6231-437c-84eb-b8f825fdf208 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:01.828194 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3365/6812] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:01 2022] DELETE /identity/v3/projects/5b092a844a4a4f18b29596fed8f7969c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:01.833937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961d257-1125-4bdd-89c4-e32c9fb0e027 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:01.834188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961d257-1125-4bdd-89c4-e32c9fb0e027 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:01.834488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961d257-1125-4bdd-89c4-e32c9fb0e027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:01.857093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c961d257-1125-4bdd-89c4-e32c9fb0e027 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:01.858341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c961d257-1125-4bdd-89c4-e32c9fb0e027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:01.927549 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3448/6813] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:01.935650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86301605-82fb-462d-9849-9f8c1dfddb1f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:01.935927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86301605-82fb-462d-9849-9f8c1dfddb1f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:01.936200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86301605-82fb-462d-9849-9f8c1dfddb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:01.956665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-86301605-82fb-462d-9849-9f8c1dfddb1f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:01.958424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-86301605-82fb-462d-9849-9f8c1dfddb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:01.970027 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3366/6814] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:01.974579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961d257-1125-4bdd-89c4-e32c9fb0e027 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:01.985594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961d257-1125-4bdd-89c4-e32c9fb0e027 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:01.992671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a88fbe-7832-49f0-8b86-3be414b90632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:01.993414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a88fbe-7832-49f0-8b86-3be414b90632 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:01.993688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a88fbe-7832-49f0-8b86-3be414b90632 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:01.993951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a88fbe-7832-49f0-8b86-3be414b90632 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:01.994597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a88fbe-7832-49f0-8b86-3be414b90632 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:22:01.994893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a88fbe-7832-49f0-8b86-3be414b90632 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:01.996596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a88fbe-7832-49f0-8b86-3be414b90632 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:02.009402 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3449/6815] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:22:02.013804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86301605-82fb-462d-9849-9f8c1dfddb1f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:02.018282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:02.025629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86301605-82fb-462d-9849-9f8c1dfddb1f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:02.032963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665558d7-c499-4fe8-82de-cbc87de21b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:02.033908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665558d7-c499-4fe8-82de-cbc87de21b61 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:02.034309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665558d7-c499-4fe8-82de-cbc87de21b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:02.035706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665558d7-c499-4fe8-82de-cbc87de21b61 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:02.037179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665558d7-c499-4fe8-82de-cbc87de21b61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:02.039894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665558d7-c499-4fe8-82de-cbc87de21b61 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:02.100227 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3367/6816] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:02 2022] POST /identity/v3/projects => generated 399 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:22:02.104724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a88fbe-7832-49f0-8b86-3be414b90632 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:02.119163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a88fbe-7832-49f0-8b86-3be414b90632 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:02.181616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:02.201351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f2bb6-efd8-4293-950b-dc36d94f07f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:02.202236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f2bb6-efd8-4293-950b-dc36d94f07f0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:02.202521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f2bb6-efd8-4293-950b-dc36d94f07f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:02.202914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f2bb6-efd8-4293-950b-dc36d94f07f0 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:02.203559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f2bb6-efd8-4293-950b-dc36d94f07f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:02.206081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f2bb6-efd8-4293-950b-dc36d94f07f0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:02.251982 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3450/6817] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:02 2022] POST /identity/v3/users => generated 306 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:02.256232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665558d7-c499-4fe8-82de-cbc87de21b61 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:02.278807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665558d7-c499-4fe8-82de-cbc87de21b61 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:02.286683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea060d-0b84-4d0e-9732-31a19cf72c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:02.287581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea060d-0b84-4d0e-9732-31a19cf72c51 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:02.287821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea060d-0b84-4d0e-9732-31a19cf72c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:02.289315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea060d-0b84-4d0e-9732-31a19cf72c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:02.290013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea060d-0b84-4d0e-9732-31a19cf72c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:02.291990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea060d-0b84-4d0e-9732-31a19cf72c51 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:02.312150 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3368/6818] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:02.318976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f2bb6-efd8-4293-950b-dc36d94f07f0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:02.335015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f2bb6-efd8-4293-950b-dc36d94f07f0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:02.343463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fbde1-327f-495a-9ced-c33d734882b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:02.344244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fbde1-327f-495a-9ced-c33d734882b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:02.344665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fbde1-327f-495a-9ced-c33d734882b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:02.345026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fbde1-327f-495a-9ced-c33d734882b8 admin admin] PATH_INFO: `/v3/projects/65810b808fa34419a5b9171507f2d783/users/36553f4883674161aad97ee233aa9da9/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:02.373363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fbde1-327f-495a-9ced-c33d734882b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65810b808fa34419a5b9171507f2d783, user_id=36553f4883674161aad97ee233aa9da9, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:02.376182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fbde1-327f-495a-9ced-c33d734882b8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:02.410552 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3451/6819] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:02 2022] PUT /identity/v3/projects/65810b808fa34419a5b9171507f2d783/users/36553f4883674161aad97ee233aa9da9/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:02.414826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea060d-0b84-4d0e-9732-31a19cf72c51 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:02.425241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea060d-0b84-4d0e-9732-31a19cf72c51 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:02.493214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b0dfb-fc78-4142-97db-2c9d131089d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:02.493956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b0dfb-fc78-4142-97db-2c9d131089d0 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:02.494332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b0dfb-fc78-4142-97db-2c9d131089d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:02.494664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b0dfb-fc78-4142-97db-2c9d131089d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:02.495367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b0dfb-fc78-4142-97db-2c9d131089d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:02.497141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b0dfb-fc78-4142-97db-2c9d131089d0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:02.512201 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3369/6820] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:02.513464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fbde1-327f-495a-9ced-c33d734882b8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:02.525878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fbde1-327f-495a-9ced-c33d734882b8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:02.535463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254896c-ed66-4118-9a16-45b805c5e7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:02.536387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254896c-ed66-4118-9a16-45b805c5e7a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:02.539543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254896c-ed66-4118-9a16-45b805c5e7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:02.540094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254896c-ed66-4118-9a16-45b805c5e7a6 admin admin] PATH_INFO: `/v3/projects/65810b808fa34419a5b9171507f2d783/users/36553f4883674161aad97ee233aa9da9/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:02.542402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254896c-ed66-4118-9a16-45b805c5e7a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65810b808fa34419a5b9171507f2d783, user_id=36553f4883674161aad97ee233aa9da9, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:02.546571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254896c-ed66-4118-9a16-45b805c5e7a6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:02.566713 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3452/6821] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:02 2022] PUT /identity/v3/projects/65810b808fa34419a5b9171507f2d783/users/36553f4883674161aad97ee233aa9da9/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:02.572582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b0c74-3395-474e-9109-3a2189e0689b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:02.572994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b0c74-3395-474e-9109-3a2189e0689b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:02.573357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b0c74-3395-474e-9109-3a2189e0689b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:02.611801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-451b0c74-3395-474e-9109-3a2189e0689b None None] MFA Rules not processed for user `36553f4883674161aad97ee233aa9da9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:02.613310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-451b0c74-3395-474e-9109-3a2189e0689b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:02.715233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254896c-ed66-4118-9a16-45b805c5e7a6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:02.722416 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3370/6822] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:02 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:02.828670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254896c-ed66-4118-9a16-45b805c5e7a6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:02.838704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4427bc9-e68d-4577-b755-70fb5608c4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:02.840125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4427bc9-e68d-4577-b755-70fb5608c4c2 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:02.841086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4427bc9-e68d-4577-b755-70fb5608c4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:02.841901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4427bc9-e68d-4577-b755-70fb5608c4c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:02.842605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4427bc9-e68d-4577-b755-70fb5608c4c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:02.845218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4427bc9-e68d-4577-b755-70fb5608c4c2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:02.893589 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3453/6823] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:02 2022] POST /identity/v3/projects => generated 399 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:02.898034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b0c74-3395-474e-9109-3a2189e0689b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:02.910741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b0c74-3395-474e-9109-3a2189e0689b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:02.986361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:03.000880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaa8e0-8ac6-4764-bb12-c947a43d5d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:03.001422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaa8e0-8ac6-4764-bb12-c947a43d5d67 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:03.001696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaa8e0-8ac6-4764-bb12-c947a43d5d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:03.001987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaa8e0-8ac6-4764-bb12-c947a43d5d67 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:03.002610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbaa8e0-8ac6-4764-bb12-c947a43d5d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:03.004641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbaa8e0-8ac6-4764-bb12-c947a43d5d67 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:03.031085 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3371/6824] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:02 2022] POST /identity/v3/users => generated 300 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:03.035885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4427bc9-e68d-4577-b755-70fb5608c4c2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:03.047838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4427bc9-e68d-4577-b755-70fb5608c4c2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:03.056731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e7047-978a-4a95-a2f6-d1b9315d31ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:03.058457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e7047-978a-4a95-a2f6-d1b9315d31ec admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:03.060705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e7047-978a-4a95-a2f6-d1b9315d31ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:03.061177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e7047-978a-4a95-a2f6-d1b9315d31ec admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:03.062130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e7047-978a-4a95-a2f6-d1b9315d31ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:03.064011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e7047-978a-4a95-a2f6-d1b9315d31ec admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:03.081576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaa8e0-8ac6-4764-bb12-c947a43d5d67 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:03.090704 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3454/6825] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:03.095143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaa8e0-8ac6-4764-bb12-c947a43d5d67 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:03.101037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcba808-2367-4a65-8bd0-449d94954024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:03.101801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcba808-2367-4a65-8bd0-449d94954024 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:03.102073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcba808-2367-4a65-8bd0-449d94954024 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:03.102336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcba808-2367-4a65-8bd0-449d94954024 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:03.102975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcba808-2367-4a65-8bd0-449d94954024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:03.105482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcba808-2367-4a65-8bd0-449d94954024 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:03.119093 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3372/6826] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:22:03.123551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e7047-978a-4a95-a2f6-d1b9315d31ec admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:03.138798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e7047-978a-4a95-a2f6-d1b9315d31ec admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:03.143736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fe73b-6d89-44cd-b56a-e200abc72dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:03.144459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fe73b-6d89-44cd-b56a-e200abc72dce admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:03.144735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fe73b-6d89-44cd-b56a-e200abc72dce admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:03.145005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fe73b-6d89-44cd-b56a-e200abc72dce admin admin] PATH_INFO: `/v3/projects/84632be37e0f4da9bc4ab1e9de264dcd/users/902edbebd6104bacb46cb4f78945a994/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:03.162702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fe73b-6d89-44cd-b56a-e200abc72dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=84632be37e0f4da9bc4ab1e9de264dcd, user_id=902edbebd6104bacb46cb4f78945a994, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:03.165863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fe73b-6d89-44cd-b56a-e200abc72dce admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:03.191033 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3455/6827] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:03 2022] PUT /identity/v3/projects/84632be37e0f4da9bc4ab1e9de264dcd/users/902edbebd6104bacb46cb4f78945a994/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:03.298382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997b017-aca5-4a10-9d54-f520ca900e9f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:03.298752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997b017-aca5-4a10-9d54-f520ca900e9f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:03.298931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997b017-aca5-4a10-9d54-f520ca900e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:03.363344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3997b017-aca5-4a10-9d54-f520ca900e9f None None] MFA Rules not processed for user `902edbebd6104bacb46cb4f78945a994`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:03.365854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3997b017-aca5-4a10-9d54-f520ca900e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:03.446072 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3373/6828] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:03.461907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fe73b-6d89-44cd-b56a-e200abc72dce admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:03.463870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-846fe73b-6d89-44cd-b56a-e200abc72dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:03.499154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997b017-aca5-4a10-9d54-f520ca900e9f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:03.500850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3997b017-aca5-4a10-9d54-f520ca900e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:03.523863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:03.548265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fe73b-6d89-44cd-b56a-e200abc72dce admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:03.553928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc9f83-b632-4f84-a0b7-aa4a173048cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:03.554659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc9f83-b632-4f84-a0b7-aa4a173048cb service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:03.555002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc9f83-b632-4f84-a0b7-aa4a173048cb service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:03.555359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc9f83-b632-4f84-a0b7-aa4a173048cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:03.582123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997b017-aca5-4a10-9d54-f520ca900e9f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:03.589752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:03.590681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:03.591008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:03.591344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] PATH_INFO: `/v3/users/d0cc23cdd2024684b73e97e65816bfbc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:03.592665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0cc23cdd2024684b73e97e65816bfbc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:03.594703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:03.628221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc9f83-b632-4f84-a0b7-aa4a173048cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:03.630276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc9f83-b632-4f84-a0b7-aa4a173048cb service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:03.642997 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3456/6829] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:03 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:03.659532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0cc23cdd2024684b73e97e65816bfbc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:03.668774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0cc23cdd2024684b73e97e65816bfbc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:03.676321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:03.685959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0cc23cdd2024684b73e97e65816bfbc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:03.687155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0cc23cdd2024684b73e97e65816bfbc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:03.688553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0cc23cdd2024684b73e97e65816bfbc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:03.703807 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3374/6830] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:03 2022] DELETE /identity/v3/users/d0cc23cdd2024684b73e97e65816bfbc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:03.708276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61404c1-3504-4a0b-8268-9cb7756f5f1b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:03.708787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61404c1-3504-4a0b-8268-9cb7756f5f1b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:03.709267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61404c1-3504-4a0b-8268-9cb7756f5f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:03.731950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c61404c1-3504-4a0b-8268-9cb7756f5f1b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:03.733304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c61404c1-3504-4a0b-8268-9cb7756f5f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:03.812772 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3457/6831] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:03.822321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:03.823857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:03.918157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca8511-dc26-4f8f-88a6-0f0f46368067 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:03.922851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e07fb9-a019-4b37-8ba2-f395e602b28d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:03.923868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e07fb9-a019-4b37-8ba2-f395e602b28d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:03.924306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e07fb9-a019-4b37-8ba2-f395e602b28d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:03.924750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e07fb9-a019-4b37-8ba2-f395e602b28d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:03.935014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e07fb9-a019-4b37-8ba2-f395e602b28d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:03.937021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e07fb9-a019-4b37-8ba2-f395e602b28d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:03.948608 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3375/6832] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:04.135550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61404c1-3504-4a0b-8268-9cb7756f5f1b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:04.137111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c61404c1-3504-4a0b-8268-9cb7756f5f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:04.151713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61404c1-3504-4a0b-8268-9cb7756f5f1b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:04.157321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77f881-d926-4606-b5e4-04f457987cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:04.158136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f881-d926-4606-b5e4-04f457987cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:04.158516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f881-d926-4606-b5e4-04f457987cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:04.158899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f881-d926-4606-b5e4-04f457987cf5 admin admin] PATH_INFO: `/v3/users/36553f4883674161aad97ee233aa9da9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:04.160181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77f881-d926-4606-b5e4-04f457987cf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36553f4883674161aad97ee233aa9da9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:04.163186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77f881-d926-4606-b5e4-04f457987cf5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:04.181612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e07fb9-a019-4b37-8ba2-f395e602b28d service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:04.184051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-27e07fb9-a019-4b37-8ba2-f395e602b28d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:04.204949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e07fb9-a019-4b37-8ba2-f395e602b28d service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:04.219467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:04.220260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:04.220675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:04.221049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] PATH_INFO: `/v3/projects/2eee1e4238484dd9a017174f71e9983f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:04.222584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eee1e4238484dd9a017174f71e9983f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:04.224852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:04.246567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0e77f881-d926-4606-b5e4-04f457987cf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36553f4883674161aad97ee233aa9da9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.257663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0e77f881-d926-4606-b5e4-04f457987cf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36553f4883674161aad97ee233aa9da9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.266828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0e77f881-d926-4606-b5e4-04f457987cf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36553f4883674161aad97ee233aa9da9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.283123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0e77f881-d926-4606-b5e4-04f457987cf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36553f4883674161aad97ee233aa9da9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.283869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0e77f881-d926-4606-b5e4-04f457987cf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36553f4883674161aad97ee233aa9da9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.285580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:04.291348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eee1e4238484dd9a017174f71e9983f', 'request_id': 'req-d227b263-c096-429f-b84f-da8343afc98e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.292766 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3458/6833] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:04 2022] DELETE /identity/v3/users/36553f4883674161aad97ee233aa9da9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:04.297721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4079c8-1dd6-4685-8b6b-e85d357282ec None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:04.298538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4079c8-1dd6-4685-8b6b-e85d357282ec None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:04.298820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4079c8-1dd6-4685-8b6b-e85d357282ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:04.299330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eee1e4238484dd9a017174f71e9983f', 'request_id': 'req-d227b263-c096-429f-b84f-da8343afc98e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.311280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eee1e4238484dd9a017174f71e9983f', 'request_id': 'req-d227b263-c096-429f-b84f-da8343afc98e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.323978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] The token cache is being invalidate because project 2eee1e4238484dd9a017174f71e9983f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:04.324355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.325155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.327447 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3376/6834] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:04 2022] DELETE /identity/v3/projects/2eee1e4238484dd9a017174f71e9983f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:04.327565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bb4079c8-1dd6-4685-8b6b-e85d357282ec None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:04.328827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bb4079c8-1dd6-4685-8b6b-e85d357282ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:04.330707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:04.332203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:04.401350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227b263-c096-429f-b84f-da8343afc98e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:04.404228 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3459/6835] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:04.406125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:04.406775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:04.407135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:04.407401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] PATH_INFO: `/v3/users/e267750534414123a0586062eb93739f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:04.408696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e267750534414123a0586062eb93739f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:04.410792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:04.415231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4079c8-1dd6-4685-8b6b-e85d357282ec None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:04.416270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:04.417343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bb4079c8-1dd6-4685-8b6b-e85d357282ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:04.480049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e267750534414123a0586062eb93739f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.484769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4079c8-1dd6-4685-8b6b-e85d357282ec None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:04.489732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e267750534414123a0586062eb93739f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.490162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43efcff-f254-43c5-88db-12624c776717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:04.490801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43efcff-f254-43c5-88db-12624c776717 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:04.491081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43efcff-f254-43c5-88db-12624c776717 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:04.491364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43efcff-f254-43c5-88db-12624c776717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:04.501395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43efcff-f254-43c5-88db-12624c776717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:04.502873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43efcff-f254-43c5-88db-12624c776717 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:04.512991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e267750534414123a0586062eb93739f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.514651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e267750534414123a0586062eb93739f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.516089 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3460/6836] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:22:04.516359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e267750534414123a0586062eb93739f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:04.533269 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3377/6837] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:04 2022] DELETE /identity/v3/users/e267750534414123a0586062eb93739f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:04.956333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43efcff-f254-43c5-88db-12624c776717 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:04.976676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43efcff-f254-43c5-88db-12624c776717 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:05.022960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:05.024898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:05.099343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b595e-9fd3-478a-871a-7e4bafc1c315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:05.099982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b595e-9fd3-478a-871a-7e4bafc1c315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:05.100249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b595e-9fd3-478a-871a-7e4bafc1c315 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:05.100615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b595e-9fd3-478a-871a-7e4bafc1c315 admin admin] PATH_INFO: `/v3/projects/89b6248c80f843dd955b76655003e603` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:05.101792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b595e-9fd3-478a-871a-7e4bafc1c315 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89b6248c80f843dd955b76655003e603)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:05.103706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b595e-9fd3-478a-871a-7e4bafc1c315 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:05.115263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f682a26-de08-4db5-93d9-4b25c22911c1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:05.119926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:05.120772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:05.121041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:05.121379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] PATH_INFO: `/v3/projects/65810b808fa34419a5b9171507f2d783` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:05.122822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65810b808fa34419a5b9171507f2d783)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:05.124906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:05.153841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:05.158386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a82b595e-9fd3-478a-871a-7e4bafc1c315 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89b6248c80f843dd955b76655003e603', 'request_id': 'req-a82b595e-9fd3-478a-871a-7e4bafc1c315'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.177579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a82b595e-9fd3-478a-871a-7e4bafc1c315 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89b6248c80f843dd955b76655003e603', 'request_id': 'req-a82b595e-9fd3-478a-871a-7e4bafc1c315'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.183142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65810b808fa34419a5b9171507f2d783', 'request_id': 'req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.188217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a82b595e-9fd3-478a-871a-7e4bafc1c315 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89b6248c80f843dd955b76655003e603', 'request_id': 'req-a82b595e-9fd3-478a-871a-7e4bafc1c315'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.190175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65810b808fa34419a5b9171507f2d783', 'request_id': 'req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.201683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a82b595e-9fd3-478a-871a-7e4bafc1c315 admin admin] The token cache is being invalidate because project 89b6248c80f843dd955b76655003e603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:05.202206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a82b595e-9fd3-478a-871a-7e4bafc1c315 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.202486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65810b808fa34419a5b9171507f2d783', 'request_id': 'req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.203379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a82b595e-9fd3-478a-871a-7e4bafc1c315 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.205212 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3461/6838] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:04 2022] DELETE /identity/v3/projects/89b6248c80f843dd955b76655003e603 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:05.211399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfec85-1872-4884-afac-ec5921d671cf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:05.211870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfec85-1872-4884-afac-ec5921d671cf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:05.212286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfec85-1872-4884-afac-ec5921d671cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:05.217907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] The token cache is being invalidate because project 65810b808fa34419a5b9171507f2d783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:05.218209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.218840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.220342 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3378/6839] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:05 2022] DELETE /identity/v3/projects/65810b808fa34419a5b9171507f2d783 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:05.224657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:05.236301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:05.237117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-10dfec85-1872-4884-afac-ec5921d671cf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:05.238454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-10dfec85-1872-4884-afac-ec5921d671cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:05.254333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:05.316928 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3462/6840] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:05.325634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82207e64-1d52-400b-8396-5f2b6656ede6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:05.325934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82207e64-1d52-400b-8396-5f2b6656ede6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:05.326177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82207e64-1d52-400b-8396-5f2b6656ede6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:05.343947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff452a5-e99e-4bd3-82c6-1474bee2cae5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:05.348353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-82207e64-1d52-400b-8396-5f2b6656ede6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:05.349363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:05.350384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:05.350384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:05.350619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-82207e64-1d52-400b-8396-5f2b6656ede6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:05.350720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] PATH_INFO: `/v3/users/902edbebd6104bacb46cb4f78945a994` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:05.351656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=902edbebd6104bacb46cb4f78945a994)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:05.353614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:05.365829 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3463/6841] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:05.366958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82207e64-1d52-400b-8396-5f2b6656ede6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:05.378699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82207e64-1d52-400b-8396-5f2b6656ede6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:05.383726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584ed3b-cd25-452b-9697-c5e63f2038b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:05.384515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584ed3b-cd25-452b-9697-c5e63f2038b0 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:05.384779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584ed3b-cd25-452b-9697-c5e63f2038b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:05.385248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584ed3b-cd25-452b-9697-c5e63f2038b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:05.385816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584ed3b-cd25-452b-9697-c5e63f2038b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:22:05.386174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584ed3b-cd25-452b-9697-c5e63f2038b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:05.387874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584ed3b-cd25-452b-9697-c5e63f2038b0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:05.403557 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3464/6842] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:22:05.407992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584ed3b-cd25-452b-9697-c5e63f2038b0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:05.419357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584ed3b-cd25-452b-9697-c5e63f2038b0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:05.423113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902edbebd6104bacb46cb4f78945a994'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.428566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14268b19-1f0a-4478-bd84-8aaa485fe1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:05.429402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14268b19-1f0a-4478-bd84-8aaa485fe1b9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:05.429821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14268b19-1f0a-4478-bd84-8aaa485fe1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:05.430122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14268b19-1f0a-4478-bd84-8aaa485fe1b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:05.430903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14268b19-1f0a-4478-bd84-8aaa485fe1b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:05.433016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14268b19-1f0a-4478-bd84-8aaa485fe1b9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:05.433947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902edbebd6104bacb46cb4f78945a994'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.449033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902edbebd6104bacb46cb4f78945a994'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.450038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902edbebd6104bacb46cb4f78945a994'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.450790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902edbebd6104bacb46cb4f78945a994'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:05.466455 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3379/6843] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:05 2022] DELETE /identity/v3/users/902edbebd6104bacb46cb4f78945a994 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:05.475042 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3465/6844] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:05 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:05.476995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:05.478586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:05.558531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73227b7-b98c-4e01-951c-9e9c7ad5c8a7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:05.607138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:05.636689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f773d-9547-4e75-94a5-cbd686d2b2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:05.637953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f773d-9547-4e75-94a5-cbd686d2b2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:05.638236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f773d-9547-4e75-94a5-cbd686d2b2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:05.638498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f773d-9547-4e75-94a5-cbd686d2b2f6 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:05.639144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f773d-9547-4e75-94a5-cbd686d2b2f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:05.641265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f773d-9547-4e75-94a5-cbd686d2b2f6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:05.665986 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3380/6845] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:05 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:22:05.670261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14268b19-1f0a-4478-bd84-8aaa485fe1b9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:05.681582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14268b19-1f0a-4478-bd84-8aaa485fe1b9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:05.686755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd0996-d45f-47d2-9df2-cb53cae1e80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:05.687371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0996-d45f-47d2-9df2-cb53cae1e80a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:05.687672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0996-d45f-47d2-9df2-cb53cae1e80a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:05.687940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0996-d45f-47d2-9df2-cb53cae1e80a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:05.688639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd0996-d45f-47d2-9df2-cb53cae1e80a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:05.690240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd0996-d45f-47d2-9df2-cb53cae1e80a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:05.701409 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3466/6846] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:22:05.705810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f773d-9547-4e75-94a5-cbd686d2b2f6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:05.717031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f773d-9547-4e75-94a5-cbd686d2b2f6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:05.722214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbce15-935e-4933-9af0-83ec869f7e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:05.722883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbce15-935e-4933-9af0-83ec869f7e4d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:05.723265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbce15-935e-4933-9af0-83ec869f7e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:05.723627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbce15-935e-4933-9af0-83ec869f7e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:05.724382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbce15-935e-4933-9af0-83ec869f7e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:05.726066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbce15-935e-4933-9af0-83ec869f7e4d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:05.740201 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3381/6847] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:05.741559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd0996-d45f-47d2-9df2-cb53cae1e80a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:05.751678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd0996-d45f-47d2-9df2-cb53cae1e80a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:05.756825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41345b7-a8da-435d-9fb3-45005a00bdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:05.757529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41345b7-a8da-435d-9fb3-45005a00bdd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:05.757798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41345b7-a8da-435d-9fb3-45005a00bdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:05.758055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41345b7-a8da-435d-9fb3-45005a00bdd0 admin admin] PATH_INFO: `/v3/projects/49e9b1754e8848608ba4a6a97939d829/users/342516fd19894a5596132f475b49fedd/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:05.773403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41345b7-a8da-435d-9fb3-45005a00bdd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e9b1754e8848608ba4a6a97939d829, user_id=342516fd19894a5596132f475b49fedd, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:05.776082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41345b7-a8da-435d-9fb3-45005a00bdd0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:05.791154 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3467/6848] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:05 2022] PUT /identity/v3/projects/49e9b1754e8848608ba4a6a97939d829/users/342516fd19894a5596132f475b49fedd/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:05.801981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91da4e2-7d01-4913-b9fa-859118d491f1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:05.802265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91da4e2-7d01-4913-b9fa-859118d491f1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:05.802548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91da4e2-7d01-4913-b9fa-859118d491f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:05.832090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b91da4e2-7d01-4913-b9fa-859118d491f1 None None] MFA Rules not processed for user `342516fd19894a5596132f475b49fedd`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:05.833625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b91da4e2-7d01-4913-b9fa-859118d491f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:05.904624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41345b7-a8da-435d-9fb3-45005a00bdd0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:05.907219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a41345b7-a8da-435d-9fb3-45005a00bdd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:05.963573 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3382/6849] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:05.966540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91da4e2-7d01-4913-b9fa-859118d491f1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:06.021488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:06.030031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41345b7-a8da-435d-9fb3-45005a00bdd0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:06.037932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc290a-4518-4908-b2d4-8b00d63e84c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:06.037932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc290a-4518-4908-b2d4-8b00d63e84c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.037932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc290a-4518-4908-b2d4-8b00d63e84c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.038552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc290a-4518-4908-b2d4-8b00d63e84c3 admin admin] PATH_INFO: `/v3/projects/84632be37e0f4da9bc4ab1e9de264dcd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.041742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc290a-4518-4908-b2d4-8b00d63e84c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84632be37e0f4da9bc4ab1e9de264dcd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:06.043717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc290a-4518-4908-b2d4-8b00d63e84c3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:06.082160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91da4e2-7d01-4913-b9fa-859118d491f1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:06.087256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52492ecc-eff8-4b1f-8152-ad9fe2e69466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:06.087935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52492ecc-eff8-4b1f-8152-ad9fe2e69466 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.088208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52492ecc-eff8-4b1f-8152-ad9fe2e69466 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.088489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52492ecc-eff8-4b1f-8152-ad9fe2e69466 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.089133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52492ecc-eff8-4b1f-8152-ad9fe2e69466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:06.091106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52492ecc-eff8-4b1f-8152-ad9fe2e69466 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:06.100097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d8bc290a-4518-4908-b2d4-8b00d63e84c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84632be37e0f4da9bc4ab1e9de264dcd', 'request_id': 'req-d8bc290a-4518-4908-b2d4-8b00d63e84c3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:06.110682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d8bc290a-4518-4908-b2d4-8b00d63e84c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84632be37e0f4da9bc4ab1e9de264dcd', 'request_id': 'req-d8bc290a-4518-4908-b2d4-8b00d63e84c3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:06.119153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d8bc290a-4518-4908-b2d4-8b00d63e84c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84632be37e0f4da9bc4ab1e9de264dcd', 'request_id': 'req-d8bc290a-4518-4908-b2d4-8b00d63e84c3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:06.123444 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3383/6850] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:05 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:22:06.127792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52492ecc-eff8-4b1f-8152-ad9fe2e69466 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:06.131674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d8bc290a-4518-4908-b2d4-8b00d63e84c3 admin admin] The token cache is being invalidate because project 84632be37e0f4da9bc4ab1e9de264dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:06.131889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d8bc290a-4518-4908-b2d4-8b00d63e84c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:06.132589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d8bc290a-4518-4908-b2d4-8b00d63e84c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:06.134163 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3468/6851] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:05 2022] DELETE /identity/v3/projects/84632be37e0f4da9bc4ab1e9de264dcd => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:06.139211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5028cc6-89de-4bbc-b990-60dcd82aad06 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.139553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5028cc6-89de-4bbc-b990-60dcd82aad06 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.139834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5028cc6-89de-4bbc-b990-60dcd82aad06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.160441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f5028cc6-89de-4bbc-b990-60dcd82aad06 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:06.161703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f5028cc6-89de-4bbc-b990-60dcd82aad06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:06.171183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:06.201815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52492ecc-eff8-4b1f-8152-ad9fe2e69466 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:06.207878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252812d-a9b1-4844-9929-6b1605d43d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:06.207878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252812d-a9b1-4844-9929-6b1605d43d0d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.207878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252812d-a9b1-4844-9929-6b1605d43d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.208480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252812d-a9b1-4844-9929-6b1605d43d0d admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.209121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252812d-a9b1-4844-9929-6b1605d43d0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:06.211116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252812d-a9b1-4844-9929-6b1605d43d0d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:06.236261 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3384/6852] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:06 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:22:06.240093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252812d-a9b1-4844-9929-6b1605d43d0d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:06.241859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9252812d-a9b1-4844-9929-6b1605d43d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:06.246574 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3469/6853] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:22:06.254666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252812d-a9b1-4844-9929-6b1605d43d0d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:06.254894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7acfa8-ee0c-4e85-a62e-d74b74c4a009 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.255359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7acfa8-ee0c-4e85-a62e-d74b74c4a009 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.255693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7acfa8-ee0c-4e85-a62e-d74b74c4a009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.260337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397face-925b-42cd-a8e8-e9d9144bf661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:06.261125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397face-925b-42cd-a8e8-e9d9144bf661 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.261474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397face-925b-42cd-a8e8-e9d9144bf661 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.261790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397face-925b-42cd-a8e8-e9d9144bf661 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.262357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397face-925b-42cd-a8e8-e9d9144bf661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:06.264102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397face-925b-42cd-a8e8-e9d9144bf661 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:06.275053 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3385/6854] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:22:06.279090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397face-925b-42cd-a8e8-e9d9144bf661 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:06.283021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3a7acfa8-ee0c-4e85-a62e-d74b74c4a009 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:06.284946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3a7acfa8-ee0c-4e85-a62e-d74b74c4a009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:06.290435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397face-925b-42cd-a8e8-e9d9144bf661 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:06.294098 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3470/6855] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:22:06.296147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ff28-d60f-4594-ba7b-c7a98fb67df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:06.297403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ff28-d60f-4594-ba7b-c7a98fb67df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.297403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ff28-d60f-4594-ba7b-c7a98fb67df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.297736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ff28-d60f-4594-ba7b-c7a98fb67df1 admin admin] PATH_INFO: `/v3/projects/a94b3ecb87b04e70a5b7cd568720c666/users/a2d698f2ce5144c5a5e8bbc8f92efb68/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.298064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7acfa8-ee0c-4e85-a62e-d74b74c4a009 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:06.307965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7acfa8-ee0c-4e85-a62e-d74b74c4a009 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:06.314237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c6d5b-73a5-444d-a694-6612211dd417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:06.314968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c6d5b-73a5-444d-a694-6612211dd417 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.315314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c6d5b-73a5-444d-a694-6612211dd417 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.315675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c6d5b-73a5-444d-a694-6612211dd417 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.316934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c6d5b-73a5-444d-a694-6612211dd417 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:22:06.316934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c6d5b-73a5-444d-a694-6612211dd417 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:06.317255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170ff28-d60f-4594-ba7b-c7a98fb67df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94b3ecb87b04e70a5b7cd568720c666, user_id=a2d698f2ce5144c5a5e8bbc8f92efb68, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:06.319186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c6d5b-73a5-444d-a694-6612211dd417 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:06.320810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170ff28-d60f-4594-ba7b-c7a98fb67df1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:06.333623 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3471/6856] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:22:06.338985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c6d5b-73a5-444d-a694-6612211dd417 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:06.352343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c6d5b-73a5-444d-a694-6612211dd417 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:06.352841 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3386/6857] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:06 2022] PUT /identity/v3/projects/a94b3ecb87b04e70a5b7cd568720c666/users/a2d698f2ce5144c5a5e8bbc8f92efb68/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:06.356603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ff28-d60f-4594-ba7b-c7a98fb67df1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:06.463207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd202d-7765-4a3d-bb09-0c5f5ca451c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:06.464450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd202d-7765-4a3d-bb09-0c5f5ca451c4 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.464566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd202d-7765-4a3d-bb09-0c5f5ca451c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.464933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd202d-7765-4a3d-bb09-0c5f5ca451c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.465944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd202d-7765-4a3d-bb09-0c5f5ca451c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:06.468146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd202d-7765-4a3d-bb09-0c5f5ca451c4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:06.478280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ff28-d60f-4594-ba7b-c7a98fb67df1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:06.483757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:06.486152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc5a32-6c6f-43a6-8f55-481281b4043a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:06.486847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc5a32-6c6f-43a6-8f55-481281b4043a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.487176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc5a32-6c6f-43a6-8f55-481281b4043a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.487960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc5a32-6c6f-43a6-8f55-481281b4043a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.488832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc5a32-6c6f-43a6-8f55-481281b4043a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:06.496304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc5a32-6c6f-43a6-8f55-481281b4043a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:06.517733 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3387/6858] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:06.519228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc5a32-6c6f-43a6-8f55-481281b4043a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:06.522662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-81cc5a32-6c6f-43a6-8f55-481281b4043a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:06.543866 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3472/6859] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:06 2022] POST /identity/v3/projects => generated 385 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:22:06.543866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd202d-7765-4a3d-bb09-0c5f5ca451c4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:06.649970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd202d-7765-4a3d-bb09-0c5f5ca451c4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:06.664267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c85454-80f0-4e05-b76e-12a68bcd47fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:06.665327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c85454-80f0-4e05-b76e-12a68bcd47fa admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.665819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c85454-80f0-4e05-b76e-12a68bcd47fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.666285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c85454-80f0-4e05-b76e-12a68bcd47fa admin admin] PATH_INFO: `/v3/projects/a94b3ecb87b04e70a5b7cd568720c666/users/a2d698f2ce5144c5a5e8bbc8f92efb68/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.669462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c85454-80f0-4e05-b76e-12a68bcd47fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94b3ecb87b04e70a5b7cd568720c666, user_id=a2d698f2ce5144c5a5e8bbc8f92efb68, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:06.672362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c85454-80f0-4e05-b76e-12a68bcd47fa admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:06.684682 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3473/6860] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:06 2022] PUT /identity/v3/projects/a94b3ecb87b04e70a5b7cd568720c666/users/a2d698f2ce5144c5a5e8bbc8f92efb68/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:06.685826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c85454-80f0-4e05-b76e-12a68bcd47fa admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:06.686136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc5a32-6c6f-43a6-8f55-481281b4043a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:06.725617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:06.807984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c85454-80f0-4e05-b76e-12a68bcd47fa admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:06.822874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2c60b-18d5-436b-97de-1396e78e3264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:06.823743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2c60b-18d5-436b-97de-1396e78e3264 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.824158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2c60b-18d5-436b-97de-1396e78e3264 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.824605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2c60b-18d5-436b-97de-1396e78e3264 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.825374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2c60b-18d5-436b-97de-1396e78e3264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:06.829423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2c60b-18d5-436b-97de-1396e78e3264 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:06.856004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b9287-5931-4b5c-878d-a29d25f42b9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:06.857152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b9287-5931-4b5c-878d-a29d25f42b9f service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.857745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b9287-5931-4b5c-878d-a29d25f42b9f service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.858174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b9287-5931-4b5c-878d-a29d25f42b9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.869635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-707b9287-5931-4b5c-878d-a29d25f42b9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:06.872678 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3474/6861] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:06 2022] POST /identity/v3/users => generated 299 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:06.876169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342ce61-72ed-4588-9060-29133f4ce9fe None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.877799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342ce61-72ed-4588-9060-29133f4ce9fe None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.878275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342ce61-72ed-4588-9060-29133f4ce9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.901832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b9287-5931-4b5c-878d-a29d25f42b9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:06.903690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b9287-5931-4b5c-878d-a29d25f42b9f service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:06.921763 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3388/6862] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:06.922548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b9287-5931-4b5c-878d-a29d25f42b9f service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:06.933968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b9287-5931-4b5c-878d-a29d25f42b9f service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:06.936822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d342ce61-72ed-4588-9060-29133f4ce9fe None None] MFA Rules not processed for user `a2d698f2ce5144c5a5e8bbc8f92efb68`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:06.938653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d342ce61-72ed-4588-9060-29133f4ce9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:06.939011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c68012-eeb8-4bee-b39f-d8539ad8f3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:06.940088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c68012-eeb8-4bee-b39f-d8539ad8f3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.940088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c68012-eeb8-4bee-b39f-d8539ad8f3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.940263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c68012-eeb8-4bee-b39f-d8539ad8f3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:06.940875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c68012-eeb8-4bee-b39f-d8539ad8f3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:06.942639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c68012-eeb8-4bee-b39f-d8539ad8f3f8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:06.955850 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3389/6863] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:06.960648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c68012-eeb8-4bee-b39f-d8539ad8f3f8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:06.980017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c68012-eeb8-4bee-b39f-d8539ad8f3f8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:06.985775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34284dc-0054-47a8-851f-9c6124f9b7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:06.986739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34284dc-0054-47a8-851f-9c6124f9b7ec admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:06.987298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34284dc-0054-47a8-851f-9c6124f9b7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:06.987711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34284dc-0054-47a8-851f-9c6124f9b7ec admin admin] PATH_INFO: `/v3/projects/4b5d18aaad9e4443ab368285945532e2/users/63b25c3a7f86446aa5919f2140812531/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:07.008480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34284dc-0054-47a8-851f-9c6124f9b7ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5d18aaad9e4443ab368285945532e2, user_id=63b25c3a7f86446aa5919f2140812531, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:07.011427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34284dc-0054-47a8-851f-9c6124f9b7ec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:07.030275 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3390/6864] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:06 2022] PUT /identity/v3/projects/4b5d18aaad9e4443ab368285945532e2/users/63b25c3a7f86446aa5919f2140812531/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:07.031393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:07.031692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34284dc-0054-47a8-851f-9c6124f9b7ec admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:07.039793 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3475/6865] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:07.046592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342ce61-72ed-4588-9060-29133f4ce9fe None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:07.048026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d342ce61-72ed-4588-9060-29133f4ce9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:07.104185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34284dc-0054-47a8-851f-9c6124f9b7ec admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:07.108861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41102-aa27-4f5a-b2c9-a2c96845ae49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:07.109472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41102-aa27-4f5a-b2c9-a2c96845ae49 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:07.109741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41102-aa27-4f5a-b2c9-a2c96845ae49 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:07.110003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41102-aa27-4f5a-b2c9-a2c96845ae49 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:07.110640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d41102-aa27-4f5a-b2c9-a2c96845ae49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:07.112469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d41102-aa27-4f5a-b2c9-a2c96845ae49 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:07.118394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342ce61-72ed-4588-9060-29133f4ce9fe None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:07.126298 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3391/6866] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:22:07.127266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41102-aa27-4f5a-b2c9-a2c96845ae49 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:07.138495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41102-aa27-4f5a-b2c9-a2c96845ae49 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:07.145264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b904a42-00a5-4b91-a663-8e9ce8208e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:07.146058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b904a42-00a5-4b91-a663-8e9ce8208e34 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:07.146330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b904a42-00a5-4b91-a663-8e9ce8208e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:07.146597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b904a42-00a5-4b91-a663-8e9ce8208e34 admin admin] PATH_INFO: `/v3/projects/4b5d18aaad9e4443ab368285945532e2/users/63b25c3a7f86446aa5919f2140812531/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:07.148567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b904a42-00a5-4b91-a663-8e9ce8208e34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5d18aaad9e4443ab368285945532e2, user_id=63b25c3a7f86446aa5919f2140812531, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:07.151215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b904a42-00a5-4b91-a663-8e9ce8208e34 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:07.164443 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3392/6867] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:07 2022] PUT /identity/v3/projects/4b5d18aaad9e4443ab368285945532e2/users/63b25c3a7f86446aa5919f2140812531/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:07.174292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309f6ef-1d35-4a40-8be8-57741aba50a3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:07.174565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309f6ef-1d35-4a40-8be8-57741aba50a3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:07.174861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309f6ef-1d35-4a40-8be8-57741aba50a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:07.177684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a8301-6bbc-4fdb-97e1-200dd800b575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:07.178300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a8301-6bbc-4fdb-97e1-200dd800b575 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:07.178574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a8301-6bbc-4fdb-97e1-200dd800b575 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:07.178840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a8301-6bbc-4fdb-97e1-200dd800b575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:07.180713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dc4a8301-6bbc-4fdb-97e1-200dd800b575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:07.208876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8309f6ef-1d35-4a40-8be8-57741aba50a3 None None] MFA Rules not processed for user `63b25c3a7f86446aa5919f2140812531`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:07.210202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8309f6ef-1d35-4a40-8be8-57741aba50a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:07.251990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:07.258596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a8301-6bbc-4fdb-97e1-200dd800b575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:07.261319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a8301-6bbc-4fdb-97e1-200dd800b575 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:07.272747 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3476/6868] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:22:07.296863 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3393/6869] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:22:07.301789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a8301-6bbc-4fdb-97e1-200dd800b575 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:07.313946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a8301-6bbc-4fdb-97e1-200dd800b575 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:07.376110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c45bbe-8e17-4cd1-b2b2-d847182b042f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:07.376865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c45bbe-8e17-4cd1-b2b2-d847182b042f admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:07.377155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c45bbe-8e17-4cd1-b2b2-d847182b042f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:07.377485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c45bbe-8e17-4cd1-b2b2-d847182b042f admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:07.378160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c45bbe-8e17-4cd1-b2b2-d847182b042f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:07.380186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c45bbe-8e17-4cd1-b2b2-d847182b042f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:07.400517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:07.418718 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3477/6870] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:07 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:07.420356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309f6ef-1d35-4a40-8be8-57741aba50a3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:07.434591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309f6ef-1d35-4a40-8be8-57741aba50a3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:07.523943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877049a-9530-4bfc-8c0a-179c35652ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:07.524595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877049a-9530-4bfc-8c0a-179c35652ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:07.525047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877049a-9530-4bfc-8c0a-179c35652ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:07.525323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877049a-9530-4bfc-8c0a-179c35652ba7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:07.525968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877049a-9530-4bfc-8c0a-179c35652ba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:07.527998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877049a-9530-4bfc-8c0a-179c35652ba7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:07.553473 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3394/6871] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:07 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:07.557526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c45bbe-8e17-4cd1-b2b2-d847182b042f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:07.570380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c45bbe-8e17-4cd1-b2b2-d847182b042f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:07.575959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9acf1-42e6-4530-b123-7141d3551ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:07.576971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9acf1-42e6-4530-b123-7141d3551ead admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:07.577246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9acf1-42e6-4530-b123-7141d3551ead admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:07.577507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9acf1-42e6-4530-b123-7141d3551ead admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:07.578149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9acf1-42e6-4530-b123-7141d3551ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:07.579996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9acf1-42e6-4530-b123-7141d3551ead admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:07.595302 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3478/6872] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:07.596294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877049a-9530-4bfc-8c0a-179c35652ba7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:07.610394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877049a-9530-4bfc-8c0a-179c35652ba7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:07.615264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9acf1-42e6-4530-b123-7141d3551ead admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:07.617395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-89f9acf1-42e6-4530-b123-7141d3551ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:07.621172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f456c-049b-4cf3-8565-d2539ac04795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:07.623038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f456c-049b-4cf3-8565-d2539ac04795 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:07.624665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f456c-049b-4cf3-8565-d2539ac04795 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:07.624975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f456c-049b-4cf3-8565-d2539ac04795 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:07.625694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f456c-049b-4cf3-8565-d2539ac04795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:07.631656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f456c-049b-4cf3-8565-d2539ac04795 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:07.656656 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3395/6873] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:07.660674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f456c-049b-4cf3-8565-d2539ac04795 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:07.679350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f456c-049b-4cf3-8565-d2539ac04795 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:07.697161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ca6d3-e901-4f40-aaa8-c160b4cbf935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:07.706729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca6d3-e901-4f40-aaa8-c160b4cbf935 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:07.707119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca6d3-e901-4f40-aaa8-c160b4cbf935 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:07.707322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca6d3-e901-4f40-aaa8-c160b4cbf935 admin admin] PATH_INFO: `/v3/projects/cd5b2d6c768446b0aa211dd06edf9344/users/d0b6a62edc0d4c068cd320a4f71a2235/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:07.727924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9acf1-42e6-4530-b123-7141d3551ead admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:07.734338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ca6d3-e901-4f40-aaa8-c160b4cbf935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd5b2d6c768446b0aa211dd06edf9344, user_id=d0b6a62edc0d4c068cd320a4f71a2235, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:07.737570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ca6d3-e901-4f40-aaa8-c160b4cbf935 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:07.740523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097b4dd-feb5-4a17-bbf9-e972d22a9b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:07.741225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097b4dd-feb5-4a17-bbf9-e972d22a9b39 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:07.741504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097b4dd-feb5-4a17-bbf9-e972d22a9b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:07.741772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097b4dd-feb5-4a17-bbf9-e972d22a9b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:07.787315 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3396/6874] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:07 2022] PUT /identity/v3/projects/cd5b2d6c768446b0aa211dd06edf9344/users/d0b6a62edc0d4c068cd320a4f71a2235/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:07.790715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa0d1b-6161-4625-8054-68355cf4153d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:07.790994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa0d1b-6161-4625-8054-68355cf4153d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:07.791284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa0d1b-6161-4625-8054-68355cf4153d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:07.842148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-57fa0d1b-6161-4625-8054-68355cf4153d None None] MFA Rules not processed for user `d0b6a62edc0d4c068cd320a4f71a2235`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:07.843421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-57fa0d1b-6161-4625-8054-68355cf4153d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:07.848866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097b4dd-feb5-4a17-bbf9-e972d22a9b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:07.851129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097b4dd-feb5-4a17-bbf9-e972d22a9b39 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:07.870324 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3479/6875] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:22:07.924036 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3397/6876] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:22:07.950589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097b4dd-feb5-4a17-bbf9-e972d22a9b39 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:07.952143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3097b4dd-feb5-4a17-bbf9-e972d22a9b39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:08.025583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097b4dd-feb5-4a17-bbf9-e972d22a9b39 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:08.086691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2d737-97b9-45e1-b4e1-27045ea52ee4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:08.087342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2d737-97b9-45e1-b4e1-27045ea52ee4 service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:08.087615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2d737-97b9-45e1-b4e1-27045ea52ee4 service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:08.087880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2d737-97b9-45e1-b4e1-27045ea52ee4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:08.092510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:08.158648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2d737-97b9-45e1-b4e1-27045ea52ee4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:08.160226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2d737-97b9-45e1-b4e1-27045ea52ee4 service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:08.171631 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3480/6877] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:08.436605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa0d1b-6161-4625-8054-68355cf4153d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:08.455789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:08.508680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa0d1b-6161-4625-8054-68355cf4153d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:08.514221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:08.514742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:08.515012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:08.515305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] PATH_INFO: `/v3/users/63b25c3a7f86446aa5919f2140812531` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:08.516496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] RBAC: Authorizing `identity:delete_user(user_id=63b25c3a7f86446aa5919f2140812531)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:08.518425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:08.571148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63b25c3a7f86446aa5919f2140812531'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:08.579856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63b25c3a7f86446aa5919f2140812531'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:08.593786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63b25c3a7f86446aa5919f2140812531'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:08.594656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63b25c3a7f86446aa5919f2140812531'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:08.595329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63b25c3a7f86446aa5919f2140812531'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:08.608300 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3398/6878] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:08 2022] DELETE /identity/v3/users/63b25c3a7f86446aa5919f2140812531 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:08.612986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271aa27-3121-4e28-b38c-d3299c495a3a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:08.613453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271aa27-3121-4e28-b38c-d3299c495a3a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:08.613651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271aa27-3121-4e28-b38c-d3299c495a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:08.633982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8271aa27-3121-4e28-b38c-d3299c495a3a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:08.635226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8271aa27-3121-4e28-b38c-d3299c495a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:08.701154 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3481/6879] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:08.713937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:08.716232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:08.780188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96a94b-fdc7-48c2-8422-82ec68f3a28f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:08.785058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adf1bf-e344-4f27-ac51-376e4033435c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:08.785547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adf1bf-e344-4f27-ac51-376e4033435c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:08.785824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adf1bf-e344-4f27-ac51-376e4033435c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:08.786127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adf1bf-e344-4f27-ac51-376e4033435c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:08.797078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7adf1bf-e344-4f27-ac51-376e4033435c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:08.798576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7adf1bf-e344-4f27-ac51-376e4033435c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:08.809997 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3399/6880] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:09.440303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271aa27-3121-4e28-b38c-d3299c495a3a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:09.441938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8271aa27-3121-4e28-b38c-d3299c495a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:09.457713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271aa27-3121-4e28-b38c-d3299c495a3a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:09.463906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:09.464743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:09.465015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:09.465280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] PATH_INFO: `/v3/projects/4b5d18aaad9e4443ab368285945532e2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:09.466418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b5d18aaad9e4443ab368285945532e2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:09.468456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:09.595775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b5d18aaad9e4443ab368285945532e2', 'request_id': 'req-240aabe5-0f5b-4187-8042-ad6f58c30bd3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:09.617692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b5d18aaad9e4443ab368285945532e2', 'request_id': 'req-240aabe5-0f5b-4187-8042-ad6f58c30bd3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:09.648341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b5d18aaad9e4443ab368285945532e2', 'request_id': 'req-240aabe5-0f5b-4187-8042-ad6f58c30bd3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:09.663311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] The token cache is being invalidate because project 4b5d18aaad9e4443ab368285945532e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:09.664023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:09.665008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:09.667146 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3482/6881] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:09 2022] DELETE /identity/v3/projects/4b5d18aaad9e4443ab368285945532e2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:09.671490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adf1bf-e344-4f27-ac51-376e4033435c service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:09.683905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adf1bf-e344-4f27-ac51-376e4033435c service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:09.723295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:09.757967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c13d5-c5fd-4fdc-a8a3-b68700e520f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:09.759216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c13d5-c5fd-4fdc-a8a3-b68700e520f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:09.760024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c13d5-c5fd-4fdc-a8a3-b68700e520f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:09.760504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c13d5-c5fd-4fdc-a8a3-b68700e520f1 admin admin] PATH_INFO: `/v3/users/d0b6a62edc0d4c068cd320a4f71a2235` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:09.762739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c13d5-c5fd-4fdc-a8a3-b68700e520f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0b6a62edc0d4c068cd320a4f71a2235)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:09.765766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c13d5-c5fd-4fdc-a8a3-b68700e520f1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:09.831292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ff9c13d5-c5fd-4fdc-a8a3-b68700e520f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0b6a62edc0d4c068cd320a4f71a2235'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:09.846689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ff9c13d5-c5fd-4fdc-a8a3-b68700e520f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0b6a62edc0d4c068cd320a4f71a2235'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:09.866422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ff9c13d5-c5fd-4fdc-a8a3-b68700e520f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0b6a62edc0d4c068cd320a4f71a2235'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:09.870530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ff9c13d5-c5fd-4fdc-a8a3-b68700e520f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0b6a62edc0d4c068cd320a4f71a2235'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:09.871265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ff9c13d5-c5fd-4fdc-a8a3-b68700e520f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0b6a62edc0d4c068cd320a4f71a2235'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:09.894517 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3400/6882] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:09 2022] DELETE /identity/v3/users/d0b6a62edc0d4c068cd320a4f71a2235 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:11.100921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:11.103184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:11.124499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aabe5-0f5b-4187-8042-ad6f58c30bd3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:11.148515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:11.196074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:11.197191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:11.197462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:11.197870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea admin admin] PATH_INFO: `/v3/projects/cd5b2d6c768446b0aa211dd06edf9344` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:11.199326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd5b2d6c768446b0aa211dd06edf9344)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:11.201420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:11.275006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd5b2d6c768446b0aa211dd06edf9344', 'request_id': 'req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:11.286349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd5b2d6c768446b0aa211dd06edf9344', 'request_id': 'req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:11.299470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd5b2d6c768446b0aa211dd06edf9344', 'request_id': 'req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:11.312690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea admin admin] The token cache is being invalidate because project cd5b2d6c768446b0aa211dd06edf9344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:11.312955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:11.313630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a22aa7d2-b3ec-42a2-aa62-eed1523a9dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:11.315489 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3483/6883] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:11 2022] DELETE /identity/v3/projects/cd5b2d6c768446b0aa211dd06edf9344 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:11.322290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf33069-aae8-481a-818d-6d6085b15278 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:11.322835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf33069-aae8-481a-818d-6d6085b15278 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:11.322835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf33069-aae8-481a-818d-6d6085b15278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:11.342684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-acf33069-aae8-481a-818d-6d6085b15278 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:11.344101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-acf33069-aae8-481a-818d-6d6085b15278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:11.430232 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3401/6884] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:11.432295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56f9ff-da24-4456-b114-9111d8d932a2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:11.432635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56f9ff-da24-4456-b114-9111d8d932a2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:11.432908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56f9ff-da24-4456-b114-9111d8d932a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:11.458224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7c56f9ff-da24-4456-b114-9111d8d932a2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:11.459722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c56f9ff-da24-4456-b114-9111d8d932a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:11.471397 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3484/6885] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:11.472289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf33069-aae8-481a-818d-6d6085b15278 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:11.483996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf33069-aae8-481a-818d-6d6085b15278 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:11.490300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6c817-32f6-4694-b175-f245a909eb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:11.490933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6c817-32f6-4694-b175-f245a909eb64 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:11.491507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6c817-32f6-4694-b175-f245a909eb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:11.491507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6c817-32f6-4694-b175-f245a909eb64 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:11.492099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6c817-32f6-4694-b175-f245a909eb64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:22:11.492493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6c817-32f6-4694-b175-f245a909eb64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:11.494119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6c817-32f6-4694-b175-f245a909eb64 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:11.510428 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3402/6886] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:22:11.511742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56f9ff-da24-4456-b114-9111d8d932a2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:11.523523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56f9ff-da24-4456-b114-9111d8d932a2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:11.529887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249dde8-7b63-4518-a8dd-7c918ac36312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:11.530836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249dde8-7b63-4518-a8dd-7c918ac36312 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:11.531296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249dde8-7b63-4518-a8dd-7c918ac36312 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:11.531783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249dde8-7b63-4518-a8dd-7c918ac36312 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:11.532970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249dde8-7b63-4518-a8dd-7c918ac36312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:11.534998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249dde8-7b63-4518-a8dd-7c918ac36312 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:11.568673 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3485/6887] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:11 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:11.573184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6c817-32f6-4694-b175-f245a909eb64 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:11.589588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6c817-32f6-4694-b175-f245a909eb64 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:11.643188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:11.669793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e07be-5fb2-4d83-adc5-6dcce4587ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:11.671248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e07be-5fb2-4d83-adc5-6dcce4587ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:11.671609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e07be-5fb2-4d83-adc5-6dcce4587ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:11.672537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e07be-5fb2-4d83-adc5-6dcce4587ef4 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:11.673814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e07be-5fb2-4d83-adc5-6dcce4587ef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:11.675856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e07be-5fb2-4d83-adc5-6dcce4587ef4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:11.708239 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3403/6888] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:11 2022] POST /identity/v3/users => generated 283 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:22:11.712480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249dde8-7b63-4518-a8dd-7c918ac36312 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:11.723973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249dde8-7b63-4518-a8dd-7c918ac36312 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:11.729441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23215f-822f-42ba-96e4-901d11cd56d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:11.730075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23215f-822f-42ba-96e4-901d11cd56d5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:11.730343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23215f-822f-42ba-96e4-901d11cd56d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:11.730594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23215f-822f-42ba-96e4-901d11cd56d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:11.731234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23215f-822f-42ba-96e4-901d11cd56d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:11.732878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23215f-822f-42ba-96e4-901d11cd56d5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:11.743436 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3486/6889] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:22:11.747497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e07be-5fb2-4d83-adc5-6dcce4587ef4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:11.758519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e07be-5fb2-4d83-adc5-6dcce4587ef4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:11.763630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9c5c1-0b85-45a1-b6b0-485ae1ed60a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:11.764237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9c5c1-0b85-45a1-b6b0-485ae1ed60a2 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:11.764542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9c5c1-0b85-45a1-b6b0-485ae1ed60a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:11.764929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9c5c1-0b85-45a1-b6b0-485ae1ed60a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:11.765578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9c5c1-0b85-45a1-b6b0-485ae1ed60a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:11.767229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9c5c1-0b85-45a1-b6b0-485ae1ed60a2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:11.785104 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3404/6890] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:22:11.786918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23215f-822f-42ba-96e4-901d11cd56d5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:11.797351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23215f-822f-42ba-96e4-901d11cd56d5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:11.801995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bc569-a9b6-4f61-88eb-fdcfab15ea81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:11.802652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bc569-a9b6-4f61-88eb-fdcfab15ea81 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:11.802925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bc569-a9b6-4f61-88eb-fdcfab15ea81 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:11.803215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bc569-a9b6-4f61-88eb-fdcfab15ea81 admin admin] PATH_INFO: `/v3/projects/d08fa98915e040cd98d7f9509ccf9a7a/users/ce7a1d3b3481480b8c1bbdc1b4390220/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:11.818743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bc569-a9b6-4f61-88eb-fdcfab15ea81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d08fa98915e040cd98d7f9509ccf9a7a, user_id=ce7a1d3b3481480b8c1bbdc1b4390220, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:11.821428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bc569-a9b6-4f61-88eb-fdcfab15ea81 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:11.835963 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3487/6891] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:11 2022] PUT /identity/v3/projects/d08fa98915e040cd98d7f9509ccf9a7a/users/ce7a1d3b3481480b8c1bbdc1b4390220/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:11.847560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825be8b-0a69-4750-a9ca-ac42052b5f90 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:11.847753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825be8b-0a69-4750-a9ca-ac42052b5f90 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:11.848019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825be8b-0a69-4750-a9ca-ac42052b5f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:11.879419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1825be8b-0a69-4750-a9ca-ac42052b5f90 None None] MFA Rules not processed for user `ce7a1d3b3481480b8c1bbdc1b4390220`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:11.880931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1825be8b-0a69-4750-a9ca-ac42052b5f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:11.951912 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3405/6892] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:11.963886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bc569-a9b6-4f61-88eb-fdcfab15ea81 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:11.965671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-270bc569-a9b6-4f61-88eb-fdcfab15ea81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:12.031073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bc569-a9b6-4f61-88eb-fdcfab15ea81 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:12.035089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b5c8c-3535-4bf3-8356-83afea443635 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:12.035758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b5c8c-3535-4bf3-8356-83afea443635 service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:12.036051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b5c8c-3535-4bf3-8356-83afea443635 service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:12.036289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b5c8c-3535-4bf3-8356-83afea443635 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:12.043869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:12.046342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b5c8c-3535-4bf3-8356-83afea443635 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:12.047880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b5c8c-3535-4bf3-8356-83afea443635 service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:12.059402 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3488/6893] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:12.449671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825be8b-0a69-4750-a9ca-ac42052b5f90 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:12.522104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825be8b-0a69-4750-a9ca-ac42052b5f90 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:12.535519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:12.536581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:12.536867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:12.537121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] PATH_INFO: `/v3/users/ce7a1d3b3481480b8c1bbdc1b4390220` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:12.538296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce7a1d3b3481480b8c1bbdc1b4390220)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:12.540269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:12.600488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce7a1d3b3481480b8c1bbdc1b4390220'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:12.608485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce7a1d3b3481480b8c1bbdc1b4390220'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:12.609741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:12.624396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce7a1d3b3481480b8c1bbdc1b4390220'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:12.625238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce7a1d3b3481480b8c1bbdc1b4390220'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:12.625935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce7a1d3b3481480b8c1bbdc1b4390220'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:12.638383 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3406/6894] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:12 2022] DELETE /identity/v3/users/ce7a1d3b3481480b8c1bbdc1b4390220 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:12.642988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9c4d7-e67a-4afa-8044-b6e569aabecc None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:12.643479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9c4d7-e67a-4afa-8044-b6e569aabecc None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:12.643572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9c4d7-e67a-4afa-8044-b6e569aabecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:12.663803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-84d9c4d7-e67a-4afa-8044-b6e569aabecc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:12.665367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-84d9c4d7-e67a-4afa-8044-b6e569aabecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:12.742786 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3489/6895] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:12.752846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:12.754301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:12.820959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55125a34-9019-43e4-9b0e-71bd4ecd55ff admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:12.824858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6813632-1098-476b-b971-e38d9dd89e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:12.825460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6813632-1098-476b-b971-e38d9dd89e91 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:12.825744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6813632-1098-476b-b971-e38d9dd89e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:12.826002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6813632-1098-476b-b971-e38d9dd89e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:12.836046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6813632-1098-476b-b971-e38d9dd89e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:12.837518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6813632-1098-476b-b971-e38d9dd89e91 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:12.847524 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3407/6896] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:13.138007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9c4d7-e67a-4afa-8044-b6e569aabecc None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:13.142077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-84d9c4d7-e67a-4afa-8044-b6e569aabecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:13.158045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9c4d7-e67a-4afa-8044-b6e569aabecc None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:13.164565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b676eda-657c-40fa-b24c-3b31bbb0c525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:13.165233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b676eda-657c-40fa-b24c-3b31bbb0c525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:13.165488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b676eda-657c-40fa-b24c-3b31bbb0c525 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:13.165755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b676eda-657c-40fa-b24c-3b31bbb0c525 admin admin] PATH_INFO: `/v3/projects/d08fa98915e040cd98d7f9509ccf9a7a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:13.166921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b676eda-657c-40fa-b24c-3b31bbb0c525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d08fa98915e040cd98d7f9509ccf9a7a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:13.169653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b676eda-657c-40fa-b24c-3b31bbb0c525 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:13.262353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9b676eda-657c-40fa-b24c-3b31bbb0c525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd08fa98915e040cd98d7f9509ccf9a7a', 'request_id': 'req-9b676eda-657c-40fa-b24c-3b31bbb0c525'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:13.278218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9b676eda-657c-40fa-b24c-3b31bbb0c525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd08fa98915e040cd98d7f9509ccf9a7a', 'request_id': 'req-9b676eda-657c-40fa-b24c-3b31bbb0c525'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:13.288261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9b676eda-657c-40fa-b24c-3b31bbb0c525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd08fa98915e040cd98d7f9509ccf9a7a', 'request_id': 'req-9b676eda-657c-40fa-b24c-3b31bbb0c525'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:13.305724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9b676eda-657c-40fa-b24c-3b31bbb0c525 admin admin] The token cache is being invalidate because project d08fa98915e040cd98d7f9509ccf9a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:13.306098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9b676eda-657c-40fa-b24c-3b31bbb0c525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:13.306874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9b676eda-657c-40fa-b24c-3b31bbb0c525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:13.310095 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3490/6897] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:13 2022] DELETE /identity/v3/projects/d08fa98915e040cd98d7f9509ccf9a7a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:13.314216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5bbad-fb77-4398-be6a-aa68fd7b9110 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:13.314506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5bbad-fb77-4398-be6a-aa68fd7b9110 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:13.314804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5bbad-fb77-4398-be6a-aa68fd7b9110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:13.336452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-96c5bbad-fb77-4398-be6a-aa68fd7b9110 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:13.352751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-96c5bbad-fb77-4398-be6a-aa68fd7b9110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:13.502597 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3408/6898] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:22:13.511810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d1a19-8696-4b58-8288-fea5138e1374 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:13.512085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d1a19-8696-4b58-8288-fea5138e1374 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:13.512354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d1a19-8696-4b58-8288-fea5138e1374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:13.549688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-885d1a19-8696-4b58-8288-fea5138e1374 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:13.551031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-885d1a19-8696-4b58-8288-fea5138e1374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:13.574890 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3491/6899] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:13.579592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5bbad-fb77-4398-be6a-aa68fd7b9110 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:13.595609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5bbad-fb77-4398-be6a-aa68fd7b9110 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:13.605029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4531b-94f9-483a-9fb7-2946dc91741e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:13.606290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4531b-94f9-483a-9fb7-2946dc91741e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:13.607571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4531b-94f9-483a-9fb7-2946dc91741e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:13.608131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4531b-94f9-483a-9fb7-2946dc91741e admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:13.609219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4531b-94f9-483a-9fb7-2946dc91741e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:22:13.609826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4531b-94f9-483a-9fb7-2946dc91741e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:13.611781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4531b-94f9-483a-9fb7-2946dc91741e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:13.632250 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3409/6900] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:22:13.632950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d1a19-8696-4b58-8288-fea5138e1374 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:13.664061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d1a19-8696-4b58-8288-fea5138e1374 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:13.669041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e40ec-a616-43e3-b93a-265cfabd89bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:13.669663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e40ec-a616-43e3-b93a-265cfabd89bc admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:13.669936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e40ec-a616-43e3-b93a-265cfabd89bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:13.670202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e40ec-a616-43e3-b93a-265cfabd89bc admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:13.670841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e40ec-a616-43e3-b93a-265cfabd89bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:13.672789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e40ec-a616-43e3-b93a-265cfabd89bc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:13.718744 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3492/6901] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:13 2022] POST /identity/v3/projects => generated 399 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:22:13.722799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4531b-94f9-483a-9fb7-2946dc91741e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:13.735080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4531b-94f9-483a-9fb7-2946dc91741e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:13.813256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe5db6-d794-494d-bf2b-171a7d469338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:13.814036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe5db6-d794-494d-bf2b-171a7d469338 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:13.814314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe5db6-d794-494d-bf2b-171a7d469338 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:13.814588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe5db6-d794-494d-bf2b-171a7d469338 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:13.815231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe5db6-d794-494d-bf2b-171a7d469338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:13.821258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe5db6-d794-494d-bf2b-171a7d469338 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:13.830221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:13.857556 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3410/6902] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:13 2022] POST /identity/v3/users => generated 300 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:13.859803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e40ec-a616-43e3-b93a-265cfabd89bc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:13.893104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e40ec-a616-43e3-b93a-265cfabd89bc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:13.909329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b36ed-620e-434b-920e-552a2b26c57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:13.910281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b36ed-620e-434b-920e-552a2b26c57d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:13.911145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b36ed-620e-434b-920e-552a2b26c57d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:13.912286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b36ed-620e-434b-920e-552a2b26c57d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:13.913201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b36ed-620e-434b-920e-552a2b26c57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:13.918374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b36ed-620e-434b-920e-552a2b26c57d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:13.967096 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3493/6903] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:13.969181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe5db6-d794-494d-bf2b-171a7d469338 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:13.982957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe5db6-d794-494d-bf2b-171a7d469338 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:13.998340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472ccd9-b4af-4f4b-9c5d-bf12527a0542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:13.999891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472ccd9-b4af-4f4b-9c5d-bf12527a0542 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:14.000413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472ccd9-b4af-4f4b-9c5d-bf12527a0542 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:14.001554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472ccd9-b4af-4f4b-9c5d-bf12527a0542 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:14.002453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472ccd9-b4af-4f4b-9c5d-bf12527a0542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:14.005034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472ccd9-b4af-4f4b-9c5d-bf12527a0542 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:14.027913 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3411/6904] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:14.029486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b36ed-620e-434b-920e-552a2b26c57d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:14.046458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b36ed-620e-434b-920e-552a2b26c57d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:14.054624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d56499-dc0b-41dd-8217-75618e2bab61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:14.055672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d56499-dc0b-41dd-8217-75618e2bab61 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:14.056124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d56499-dc0b-41dd-8217-75618e2bab61 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:14.056598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d56499-dc0b-41dd-8217-75618e2bab61 admin admin] PATH_INFO: `/v3/projects/fe2ca77984cf44b9a1f256de78fca212/users/283d1c6c26b84779ac49fc55d18505eb/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:14.079514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d56499-dc0b-41dd-8217-75618e2bab61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2ca77984cf44b9a1f256de78fca212, user_id=283d1c6c26b84779ac49fc55d18505eb, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:14.082574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d56499-dc0b-41dd-8217-75618e2bab61 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:14.101149 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3494/6905] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:14 2022] PUT /identity/v3/projects/fe2ca77984cf44b9a1f256de78fca212/users/283d1c6c26b84779ac49fc55d18505eb/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:14.111472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fe133-8d8b-4bee-9469-55266b64394f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:14.111782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fe133-8d8b-4bee-9469-55266b64394f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:14.112222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fe133-8d8b-4bee-9469-55266b64394f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:14.147723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e60fe133-8d8b-4bee-9469-55266b64394f None None] MFA Rules not processed for user `283d1c6c26b84779ac49fc55d18505eb`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:14.149276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e60fe133-8d8b-4bee-9469-55266b64394f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:14.238649 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3412/6906] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:14 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:14.243508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d56499-dc0b-41dd-8217-75618e2bab61 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:14.272433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:14.342930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d56499-dc0b-41dd-8217-75618e2bab61 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:14.348132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40936f-5032-4d6d-b7f6-790755d783e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:14.349224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40936f-5032-4d6d-b7f6-790755d783e9 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:14.349795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40936f-5032-4d6d-b7f6-790755d783e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:14.350220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40936f-5032-4d6d-b7f6-790755d783e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:14.350993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40936f-5032-4d6d-b7f6-790755d783e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:14.353210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40936f-5032-4d6d-b7f6-790755d783e9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:14.397196 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3495/6907] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:14 2022] POST /identity/v3/projects => generated 399 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:14.402097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fe133-8d8b-4bee-9469-55266b64394f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:14.413971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fe133-8d8b-4bee-9469-55266b64394f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:14.523154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1c52d-76a4-45fa-91f3-53d48c3d5376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:14.523990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1c52d-76a4-45fa-91f3-53d48c3d5376 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:14.524345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1c52d-76a4-45fa-91f3-53d48c3d5376 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:14.525214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1c52d-76a4-45fa-91f3-53d48c3d5376 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:14.526252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1c52d-76a4-45fa-91f3-53d48c3d5376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:14.528092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1c52d-76a4-45fa-91f3-53d48c3d5376 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:14.563430 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3413/6908] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:14 2022] POST /identity/v3/users => generated 306 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:14.567573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40936f-5032-4d6d-b7f6-790755d783e9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:14.579176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40936f-5032-4d6d-b7f6-790755d783e9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:14.584083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f3f54-ee06-4938-ab5e-ed4a11b753d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:14.584750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f3f54-ee06-4938-ab5e-ed4a11b753d5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:14.585031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f3f54-ee06-4938-ab5e-ed4a11b753d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:14.585296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f3f54-ee06-4938-ab5e-ed4a11b753d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:14.585976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f3f54-ee06-4938-ab5e-ed4a11b753d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:14.587579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f3f54-ee06-4938-ab5e-ed4a11b753d5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:14.600845 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3496/6909] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:14.604868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1c52d-76a4-45fa-91f3-53d48c3d5376 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:14.615641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1c52d-76a4-45fa-91f3-53d48c3d5376 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:14.620342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e927007-eb68-4a83-a02c-baaab537c3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:14.621044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e927007-eb68-4a83-a02c-baaab537c3f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:14.621316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e927007-eb68-4a83-a02c-baaab537c3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:14.621584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e927007-eb68-4a83-a02c-baaab537c3f2 admin admin] PATH_INFO: `/v3/projects/4b9fcf5fb1e247e99042f16c7a43086a/users/659a3524dcd54ecd876e354f18b25930/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:14.637583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e927007-eb68-4a83-a02c-baaab537c3f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b9fcf5fb1e247e99042f16c7a43086a, user_id=659a3524dcd54ecd876e354f18b25930, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:14.640502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e927007-eb68-4a83-a02c-baaab537c3f2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:14.655018 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3414/6910] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:14 2022] PUT /identity/v3/projects/4b9fcf5fb1e247e99042f16c7a43086a/users/659a3524dcd54ecd876e354f18b25930/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:14.658791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f3f54-ee06-4938-ab5e-ed4a11b753d5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:14.670234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f3f54-ee06-4938-ab5e-ed4a11b753d5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:14.738187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963e439-f83e-43c6-8cbe-5079ac7cfeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:14.738795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963e439-f83e-43c6-8cbe-5079ac7cfeef admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:14.739146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963e439-f83e-43c6-8cbe-5079ac7cfeef admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:14.739402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963e439-f83e-43c6-8cbe-5079ac7cfeef admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:14.740038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963e439-f83e-43c6-8cbe-5079ac7cfeef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:14.741688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963e439-f83e-43c6-8cbe-5079ac7cfeef admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:14.752096 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3497/6911] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:14.757448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e927007-eb68-4a83-a02c-baaab537c3f2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:14.769231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e927007-eb68-4a83-a02c-baaab537c3f2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:14.774481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c96e9f-be3e-4267-bea5-763172e4970a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:14.775183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c96e9f-be3e-4267-bea5-763172e4970a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:14.775451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c96e9f-be3e-4267-bea5-763172e4970a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:14.775933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c96e9f-be3e-4267-bea5-763172e4970a admin admin] PATH_INFO: `/v3/projects/4b9fcf5fb1e247e99042f16c7a43086a/users/659a3524dcd54ecd876e354f18b25930/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:14.778173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c96e9f-be3e-4267-bea5-763172e4970a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b9fcf5fb1e247e99042f16c7a43086a, user_id=659a3524dcd54ecd876e354f18b25930, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:14.780870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c96e9f-be3e-4267-bea5-763172e4970a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:14.804592 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3415/6912] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:14 2022] PUT /identity/v3/projects/4b9fcf5fb1e247e99042f16c7a43086a/users/659a3524dcd54ecd876e354f18b25930/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:14.811867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfff5e-7112-445b-b307-cda152c3e45b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:14.812147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfff5e-7112-445b-b307-cda152c3e45b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:14.812433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfff5e-7112-445b-b307-cda152c3e45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:14.849749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-67cfff5e-7112-445b-b307-cda152c3e45b None None] MFA Rules not processed for user `659a3524dcd54ecd876e354f18b25930`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:14.851241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-67cfff5e-7112-445b-b307-cda152c3e45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:14.954323 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3498/6913] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:14 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:14.966410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c96e9f-be3e-4267-bea5-763172e4970a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:14.968905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-73c96e9f-be3e-4267-bea5-763172e4970a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:15.011939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:15.053919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c96e9f-be3e-4267-bea5-763172e4970a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:15.059886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b7f91-4618-455e-8340-a12dbc69bb9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:15.060668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b7f91-4618-455e-8340-a12dbc69bb9d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:15.061064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b7f91-4618-455e-8340-a12dbc69bb9d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:15.061429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b7f91-4618-455e-8340-a12dbc69bb9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:15.136309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b7f91-4618-455e-8340-a12dbc69bb9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:15.137939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b7f91-4618-455e-8340-a12dbc69bb9d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:15.151891 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3416/6914] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:14 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:16.512269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfff5e-7112-445b-b307-cda152c3e45b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:16.515103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-67cfff5e-7112-445b-b307-cda152c3e45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:16.529750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:16.609970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfff5e-7112-445b-b307-cda152c3e45b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:16.619744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77377f7e-a42e-427c-917b-110dc8529312 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:16.620409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77377f7e-a42e-427c-917b-110dc8529312 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:16.621296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77377f7e-a42e-427c-917b-110dc8529312 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:16.621296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77377f7e-a42e-427c-917b-110dc8529312 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:16.648114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77377f7e-a42e-427c-917b-110dc8529312 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:16.651479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77377f7e-a42e-427c-917b-110dc8529312 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:16.672965 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3499/6915] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:16 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:19.294346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b7f91-4618-455e-8340-a12dbc69bb9d service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:19.295940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c74b7f91-4618-455e-8340-a12dbc69bb9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:19.452770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b7f91-4618-455e-8340-a12dbc69bb9d service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:19.461654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:19.463517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:19.464634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:19.465097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] PATH_INFO: `/v3/users/342516fd19894a5596132f475b49fedd` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:19.467640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=342516fd19894a5596132f475b49fedd)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:19.471467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:19.578579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '342516fd19894a5596132f475b49fedd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:19.603382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '342516fd19894a5596132f475b49fedd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:19.618918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '342516fd19894a5596132f475b49fedd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:19.620010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '342516fd19894a5596132f475b49fedd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:19.620859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '342516fd19894a5596132f475b49fedd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:19.628151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:19.643888 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3417/6916] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:19 2022] DELETE /identity/v3/users/342516fd19894a5596132f475b49fedd => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:19.645858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cb40d-509b-43df-9548-15606672f38d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:19.646406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cb40d-509b-43df-9548-15606672f38d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:19.646525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cb40d-509b-43df-9548-15606672f38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:19.682514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2c6cb40d-509b-43df-9548-15606672f38d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:19.684640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2c6cb40d-509b-43df-9548-15606672f38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:19.853099 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3500/6917] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:19.872721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:19.879818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:20.036668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3c4b2-f8db-4b86-b038-8d929ce937fd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:20.043221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999361b-6e08-4714-bd89-41d5f081c03c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:20.043860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999361b-6e08-4714-bd89-41d5f081c03c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:20.044288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999361b-6e08-4714-bd89-41d5f081c03c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:20.044585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999361b-6e08-4714-bd89-41d5f081c03c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:20.079739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999361b-6e08-4714-bd89-41d5f081c03c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:20.083927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999361b-6e08-4714-bd89-41d5f081c03c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:20.096545 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3418/6918] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:20.404071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6cb40d-509b-43df-9548-15606672f38d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:20.407099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2c6cb40d-509b-43df-9548-15606672f38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:20.440163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6cb40d-509b-43df-9548-15606672f38d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:20.463718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:20.464420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:20.464710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:20.464976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] PATH_INFO: `/v3/projects/49e9b1754e8848608ba4a6a97939d829` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:20.466173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49e9b1754e8848608ba4a6a97939d829)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:20.468764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:20.564431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49e9b1754e8848608ba4a6a97939d829', 'request_id': 'req-db73051b-b0a8-44aa-ab41-b7fb208cfff5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:20.581415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49e9b1754e8848608ba4a6a97939d829', 'request_id': 'req-db73051b-b0a8-44aa-ab41-b7fb208cfff5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:20.592419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49e9b1754e8848608ba4a6a97939d829', 'request_id': 'req-db73051b-b0a8-44aa-ab41-b7fb208cfff5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:20.609729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] The token cache is being invalidate because project 49e9b1754e8848608ba4a6a97939d829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:20.610261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:20.611140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:20.614951 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3501/6919] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:20 2022] DELETE /identity/v3/projects/49e9b1754e8848608ba4a6a97939d829 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:20.619612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999361b-6e08-4714-bd89-41d5f081c03c service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:20.631467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999361b-6e08-4714-bd89-41d5f081c03c service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:20.753698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84a505-44ba-4c91-88ed-61298d70646c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:20.755233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84a505-44ba-4c91-88ed-61298d70646c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:20.755664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84a505-44ba-4c91-88ed-61298d70646c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:20.756240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84a505-44ba-4c91-88ed-61298d70646c admin admin] PATH_INFO: `/v3/users/a2d698f2ce5144c5a5e8bbc8f92efb68` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:20.757970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84a505-44ba-4c91-88ed-61298d70646c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2d698f2ce5144c5a5e8bbc8f92efb68)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:20.763782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84a505-44ba-4c91-88ed-61298d70646c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:20.841012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-dc84a505-44ba-4c91-88ed-61298d70646c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2d698f2ce5144c5a5e8bbc8f92efb68'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:20.858252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-dc84a505-44ba-4c91-88ed-61298d70646c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2d698f2ce5144c5a5e8bbc8f92efb68'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:20.876308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-dc84a505-44ba-4c91-88ed-61298d70646c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2d698f2ce5144c5a5e8bbc8f92efb68'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:20.877433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-dc84a505-44ba-4c91-88ed-61298d70646c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2d698f2ce5144c5a5e8bbc8f92efb68'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:20.878138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-dc84a505-44ba-4c91-88ed-61298d70646c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2d698f2ce5144c5a5e8bbc8f92efb68'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:20.896423 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3419/6920] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:20 2022] DELETE /identity/v3/users/a2d698f2ce5144c5a5e8bbc8f92efb68 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:22.623486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:22.625096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:22.641546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73051b-b0a8-44aa-ab41-b7fb208cfff5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:22.665897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:22.786915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:22.788709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:22.788970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:22.789788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686 admin admin] PATH_INFO: `/v3/projects/a94b3ecb87b04e70a5b7cd568720c666` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:22.792442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a94b3ecb87b04e70a5b7cd568720c666)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:22.795344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:22.863712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a94b3ecb87b04e70a5b7cd568720c666', 'request_id': 'req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:22.874826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a94b3ecb87b04e70a5b7cd568720c666', 'request_id': 'req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:22.884764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a94b3ecb87b04e70a5b7cd568720c666', 'request_id': 'req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:22.901017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686 admin admin] The token cache is being invalidate because project a94b3ecb87b04e70a5b7cd568720c666 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:22.901479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:22.902244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0118e0d0-3f30-4bd3-9f00-6f9b0fe86686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:22.904280 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3502/6921] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:22 2022] DELETE /identity/v3/projects/a94b3ecb87b04e70a5b7cd568720c666 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:22.911146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a125229-4430-4271-93af-e129c2a517c0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:22.911681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a125229-4430-4271-93af-e129c2a517c0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:22.912093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a125229-4430-4271-93af-e129c2a517c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:22.938233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2a125229-4430-4271-93af-e129c2a517c0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:22.940390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2a125229-4430-4271-93af-e129c2a517c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:22.961038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:23.085911 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3420/6922] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:23.094519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b9be4-effd-463c-8943-b0e6269b9be2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:23.094808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b9be4-effd-463c-8943-b0e6269b9be2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:23.095254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b9be4-effd-463c-8943-b0e6269b9be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:23.140136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b31b9be4-effd-463c-8943-b0e6269b9be2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:23.141551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b31b9be4-effd-463c-8943-b0e6269b9be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:23.164827 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3503/6923] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:23.170495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a125229-4430-4271-93af-e129c2a517c0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:23.223652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a125229-4430-4271-93af-e129c2a517c0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:23.248401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa82e75-814d-4bd8-81fc-2ccbfc7cbb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:23.249680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82e75-814d-4bd8-81fc-2ccbfc7cbb45 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:23.250210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82e75-814d-4bd8-81fc-2ccbfc7cbb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:23.250706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82e75-814d-4bd8-81fc-2ccbfc7cbb45 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:23.252405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa82e75-814d-4bd8-81fc-2ccbfc7cbb45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:22:23.254658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa82e75-814d-4bd8-81fc-2ccbfc7cbb45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:23.259815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa82e75-814d-4bd8-81fc-2ccbfc7cbb45 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:23.281947 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3421/6924] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:22:23.286532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b9be4-effd-463c-8943-b0e6269b9be2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:23.298994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b9be4-effd-463c-8943-b0e6269b9be2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:23.313499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5e863-7213-4ff3-95b2-91237e265234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:23.314338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5e863-7213-4ff3-95b2-91237e265234 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:23.314765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5e863-7213-4ff3-95b2-91237e265234 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:23.315180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5e863-7213-4ff3-95b2-91237e265234 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:23.315952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5e863-7213-4ff3-95b2-91237e265234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:23.318927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5e863-7213-4ff3-95b2-91237e265234 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:23.385210 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3504/6925] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:23 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:23.390474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa82e75-814d-4bd8-81fc-2ccbfc7cbb45 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:23.409464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa82e75-814d-4bd8-81fc-2ccbfc7cbb45 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:23.505247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c718b0-9616-4f14-b972-876bf19ffa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:23.506004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c718b0-9616-4f14-b972-876bf19ffa48 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:23.506571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c718b0-9616-4f14-b972-876bf19ffa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:23.506926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c718b0-9616-4f14-b972-876bf19ffa48 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:23.507817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c718b0-9616-4f14-b972-876bf19ffa48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:23.509918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c718b0-9616-4f14-b972-876bf19ffa48 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:23.560915 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3422/6926] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:23 2022] POST /identity/v3/users => generated 283 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:23.562107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5e863-7213-4ff3-95b2-91237e265234 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:23.585677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5e863-7213-4ff3-95b2-91237e265234 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:23.605068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291e71e-9e95-4aec-a8b8-f8e642905e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:23.606175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291e71e-9e95-4aec-a8b8-f8e642905e7d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:23.606657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291e71e-9e95-4aec-a8b8-f8e642905e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:23.607257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291e71e-9e95-4aec-a8b8-f8e642905e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:23.608800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291e71e-9e95-4aec-a8b8-f8e642905e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:23.610975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291e71e-9e95-4aec-a8b8-f8e642905e7d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:23.630709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c718b0-9616-4f14-b972-876bf19ffa48 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:23.654742 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3505/6927] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:22:23.665464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c718b0-9616-4f14-b972-876bf19ffa48 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:23.681552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b63db1-0f33-4571-855d-ce9905830ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:23.682191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b63db1-0f33-4571-855d-ce9905830ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:23.682465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b63db1-0f33-4571-855d-ce9905830ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:23.682755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b63db1-0f33-4571-855d-ce9905830ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:23.690592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b63db1-0f33-4571-855d-ce9905830ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:23.691541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b63db1-0f33-4571-855d-ce9905830ed7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:23.748805 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3423/6928] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:22:23.757117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291e71e-9e95-4aec-a8b8-f8e642905e7d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:23.789173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291e71e-9e95-4aec-a8b8-f8e642905e7d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:23.796631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ddcc3-90c9-4470-a2cb-272e07cba9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:23.797903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ddcc3-90c9-4470-a2cb-272e07cba9a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:23.798367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ddcc3-90c9-4470-a2cb-272e07cba9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:23.798829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ddcc3-90c9-4470-a2cb-272e07cba9a2 admin admin] PATH_INFO: `/v3/projects/7eae0a6322fc432aa4458bbdec2fd318/users/fa409247f8c34b3b93d4b03f5018d734/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:23.826383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ddcc3-90c9-4470-a2cb-272e07cba9a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eae0a6322fc432aa4458bbdec2fd318, user_id=fa409247f8c34b3b93d4b03f5018d734, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:23.829614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ddcc3-90c9-4470-a2cb-272e07cba9a2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:23.851122 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3506/6929] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:23 2022] PUT /identity/v3/projects/7eae0a6322fc432aa4458bbdec2fd318/users/fa409247f8c34b3b93d4b03f5018d734/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:23.861399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37c4a0-4999-4d34-b158-4212c6f59817 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:23.861782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37c4a0-4999-4d34-b158-4212c6f59817 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:23.861944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37c4a0-4999-4d34-b158-4212c6f59817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:23.900934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5c37c4a0-4999-4d34-b158-4212c6f59817 None None] MFA Rules not processed for user `fa409247f8c34b3b93d4b03f5018d734`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:23.902172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5c37c4a0-4999-4d34-b158-4212c6f59817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:23.984123 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3424/6930] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:23.989385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ddcc3-90c9-4470-a2cb-272e07cba9a2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:24.057428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:24.082599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ddcc3-90c9-4470-a2cb-272e07cba9a2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:24.088647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6010a-0f10-42ed-9b42-fbc45fb1e7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:24.089663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6010a-0f10-42ed-9b42-fbc45fb1e7af admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:24.090024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6010a-0f10-42ed-9b42-fbc45fb1e7af admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:24.090303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6010a-0f10-42ed-9b42-fbc45fb1e7af admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:24.091826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6010a-0f10-42ed-9b42-fbc45fb1e7af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:24.094229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6010a-0f10-42ed-9b42-fbc45fb1e7af admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:24.133660 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3507/6931] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:23 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:24.137935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37c4a0-4999-4d34-b158-4212c6f59817 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:24.151163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37c4a0-4999-4d34-b158-4212c6f59817 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:24.163869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:24.234786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b80782-2a0f-4146-a641-1651cb64163b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:24.239624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b80782-2a0f-4146-a641-1651cb64163b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:24.241626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b80782-2a0f-4146-a641-1651cb64163b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:24.243121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b80782-2a0f-4146-a641-1651cb64163b admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:24.244113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b80782-2a0f-4146-a641-1651cb64163b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:24.247817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b80782-2a0f-4146-a641-1651cb64163b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:24.309954 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3425/6932] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:24 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:22:24.314961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6010a-0f10-42ed-9b42-fbc45fb1e7af admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:24.329662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6010a-0f10-42ed-9b42-fbc45fb1e7af admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:24.338652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f3479-be3b-4546-9428-0c77f01daf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:24.339699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f3479-be3b-4546-9428-0c77f01daf08 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:24.340254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f3479-be3b-4546-9428-0c77f01daf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:24.340703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f3479-be3b-4546-9428-0c77f01daf08 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:24.342838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f3479-be3b-4546-9428-0c77f01daf08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:24.346574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f3479-be3b-4546-9428-0c77f01daf08 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:24.363659 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3508/6933] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:24.368837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b80782-2a0f-4146-a641-1651cb64163b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:24.387346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b80782-2a0f-4146-a641-1651cb64163b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:24.393945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6da3e-b438-486a-88f0-32454020fda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:24.394742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6da3e-b438-486a-88f0-32454020fda3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:24.395154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6da3e-b438-486a-88f0-32454020fda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:24.395428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6da3e-b438-486a-88f0-32454020fda3 admin admin] PATH_INFO: `/v3/projects/77900bcbef8b4d70a9584a7734065585/users/bd6e2e37785a47e484c37d5b9fbce68a/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:24.440479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6da3e-b438-486a-88f0-32454020fda3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77900bcbef8b4d70a9584a7734065585, user_id=bd6e2e37785a47e484c37d5b9fbce68a, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:24.444406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6da3e-b438-486a-88f0-32454020fda3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:24.476598 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3426/6934] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:24 2022] PUT /identity/v3/projects/77900bcbef8b4d70a9584a7734065585/users/bd6e2e37785a47e484c37d5b9fbce68a/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:24.477869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f3479-be3b-4546-9428-0c77f01daf08 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:24.493419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f3479-be3b-4546-9428-0c77f01daf08 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:24.623357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a8e5c-1e4f-4f62-b283-85e0f0f47f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:24.624141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a8e5c-1e4f-4f62-b283-85e0f0f47f19 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:24.624440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a8e5c-1e4f-4f62-b283-85e0f0f47f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:24.624709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a8e5c-1e4f-4f62-b283-85e0f0f47f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:24.625353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a8e5c-1e4f-4f62-b283-85e0f0f47f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:24.626986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a8e5c-1e4f-4f62-b283-85e0f0f47f19 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:24.652141 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3509/6935] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:24.653710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6da3e-b438-486a-88f0-32454020fda3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:24.675601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6da3e-b438-486a-88f0-32454020fda3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:24.682441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6eb1a8-1f2a-473b-88b4-4907691d3276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:24.683347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6eb1a8-1f2a-473b-88b4-4907691d3276 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:24.683707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6eb1a8-1f2a-473b-88b4-4907691d3276 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:24.684095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6eb1a8-1f2a-473b-88b4-4907691d3276 admin admin] PATH_INFO: `/v3/projects/77900bcbef8b4d70a9584a7734065585/users/bd6e2e37785a47e484c37d5b9fbce68a/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:24.689158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6eb1a8-1f2a-473b-88b4-4907691d3276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77900bcbef8b4d70a9584a7734065585, user_id=bd6e2e37785a47e484c37d5b9fbce68a, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:24.695205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6eb1a8-1f2a-473b-88b4-4907691d3276 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:24.728004 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3427/6936] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:24 2022] PUT /identity/v3/projects/77900bcbef8b4d70a9584a7734065585/users/bd6e2e37785a47e484c37d5b9fbce68a/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:24.740278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c084f-c25d-4a5a-83fd-43c671e3931f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:24.740806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c084f-c25d-4a5a-83fd-43c671e3931f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:24.740898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c084f-c25d-4a5a-83fd-43c671e3931f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:24.827344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-095c084f-c25d-4a5a-83fd-43c671e3931f None None] MFA Rules not processed for user `bd6e2e37785a47e484c37d5b9fbce68a`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:24.829799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-095c084f-c25d-4a5a-83fd-43c671e3931f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:25.008666 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3510/6937] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:25.030029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6eb1a8-1f2a-473b-88b4-4907691d3276 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:25.031875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2a6eb1a8-1f2a-473b-88b4-4907691d3276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:25.154570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6eb1a8-1f2a-473b-88b4-4907691d3276 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:25.160026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec996e-77e5-442e-99fb-9beedb9cd2f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:25.161283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec996e-77e5-442e-99fb-9beedb9cd2f4 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:25.163222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec996e-77e5-442e-99fb-9beedb9cd2f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:25.163676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec996e-77e5-442e-99fb-9beedb9cd2f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:25.183946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec996e-77e5-442e-99fb-9beedb9cd2f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:25.185818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec996e-77e5-442e-99fb-9beedb9cd2f4 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:25.204588 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3428/6938] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:29.582218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c084f-c25d-4a5a-83fd-43c671e3931f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:29.598468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c084f-c25d-4a5a-83fd-43c671e3931f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:29.606899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046246b-3305-4275-951d-1b746741be34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:29.609228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046246b-3305-4275-951d-1b746741be34 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:29.615089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046246b-3305-4275-951d-1b746741be34 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:29.615380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046246b-3305-4275-951d-1b746741be34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:29.693491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:29.716219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046246b-3305-4275-951d-1b746741be34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:29.717749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046246b-3305-4275-951d-1b746741be34 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:29.729922 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3511/6939] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:30.572082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49da0b8-6fba-44f5-b7f6-507e44f00575 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:30.572597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49da0b8-6fba-44f5-b7f6-507e44f00575 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:30.572597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49da0b8-6fba-44f5-b7f6-507e44f00575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:30.602896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f49da0b8-6fba-44f5-b7f6-507e44f00575 None None] MFA Rules not processed for user `daffa49c8fa64a3bb96f917fef222291`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:30.604501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f49da0b8-6fba-44f5-b7f6-507e44f00575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:30.731198 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3429/6940] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:30.925739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046246b-3305-4275-951d-1b746741be34 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:30.928473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b046246b-3305-4275-951d-1b746741be34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:31.030464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046246b-3305-4275-951d-1b746741be34 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:31.039129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35db681-6974-43ec-81e0-bba1fda80846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:31.040035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35db681-6974-43ec-81e0-bba1fda80846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:31.040421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35db681-6974-43ec-81e0-bba1fda80846 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:31.041068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35db681-6974-43ec-81e0-bba1fda80846 admin admin] PATH_INFO: `/v3/users/f1dde492e29c4157bab5b7419a310838` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:31.042840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35db681-6974-43ec-81e0-bba1fda80846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1dde492e29c4157bab5b7419a310838)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:31.046339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35db681-6974-43ec-81e0-bba1fda80846 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:31.137506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a35db681-6974-43ec-81e0-bba1fda80846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1dde492e29c4157bab5b7419a310838'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:31.151511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a35db681-6974-43ec-81e0-bba1fda80846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1dde492e29c4157bab5b7419a310838'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:31.166325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a35db681-6974-43ec-81e0-bba1fda80846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1dde492e29c4157bab5b7419a310838'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:31.181071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a35db681-6974-43ec-81e0-bba1fda80846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1dde492e29c4157bab5b7419a310838'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:31.181827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a35db681-6974-43ec-81e0-bba1fda80846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1dde492e29c4157bab5b7419a310838'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:31.192578 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3512/6941] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:30 2022] DELETE /identity/v3/users/f1dde492e29c4157bab5b7419a310838 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:31.677495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49da0b8-6fba-44f5-b7f6-507e44f00575 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:31.692417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49da0b8-6fba-44f5-b7f6-507e44f00575 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:31.705972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:31.782946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:31.783943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:31.784223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:31.784527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] PATH_INFO: `/v3/users/fa409247f8c34b3b93d4b03f5018d734` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:31.785676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa409247f8c34b3b93d4b03f5018d734)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:31.788620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:31.867223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa409247f8c34b3b93d4b03f5018d734'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:31.881456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa409247f8c34b3b93d4b03f5018d734'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:31.908354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa409247f8c34b3b93d4b03f5018d734'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:31.909608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa409247f8c34b3b93d4b03f5018d734'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:31.910227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa409247f8c34b3b93d4b03f5018d734'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:31.937656 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3430/6942] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:31 2022] DELETE /identity/v3/users/fa409247f8c34b3b93d4b03f5018d734 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:31.939200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b8134-d8c6-409c-9044-3e8f5b67b9d6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:31.939712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b8134-d8c6-409c-9044-3e8f5b67b9d6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:31.939815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b8134-d8c6-409c-9044-3e8f5b67b9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:31.967782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-947b8134-d8c6-409c-9044-3e8f5b67b9d6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:31.969083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-947b8134-d8c6-409c-9044-3e8f5b67b9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:32.045749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:32.046388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:32.048521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:32.075959 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3513/6943] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:22:32.100072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b8134-d8c6-409c-9044-3e8f5b67b9d6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:32.101615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-947b8134-d8c6-409c-9044-3e8f5b67b9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:32.202781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b8134-d8c6-409c-9044-3e8f5b67b9d6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:32.205933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f67a5c-ded9-4ebc-93d5-16e57ec932ae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:32.207901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7543c9-6d77-408f-ba20-611e36e810bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:32.209266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7543c9-6d77-408f-ba20-611e36e810bf service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:32.209539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7543c9-6d77-408f-ba20-611e36e810bf service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:32.209980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7543c9-6d77-408f-ba20-611e36e810bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:32.216731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:32.217536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:32.217940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:32.218212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] PATH_INFO: `/v3/projects/dac85ef1c85b4a06951f2df42dd98e50` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:32.219371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dac85ef1c85b4a06951f2df42dd98e50)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:32.221630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:32.239759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7543c9-6d77-408f-ba20-611e36e810bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:32.241575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7543c9-6d77-408f-ba20-611e36e810bf service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:32.261028 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3514/6944] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:22:32.284128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:32.298555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dac85ef1c85b4a06951f2df42dd98e50', 'request_id': 'req-091b8e33-03ea-479d-b210-05636dab9a5d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:32.308532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dac85ef1c85b4a06951f2df42dd98e50', 'request_id': 'req-091b8e33-03ea-479d-b210-05636dab9a5d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:32.325937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dac85ef1c85b4a06951f2df42dd98e50', 'request_id': 'req-091b8e33-03ea-479d-b210-05636dab9a5d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:32.348577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] The token cache is being invalidate because project dac85ef1c85b4a06951f2df42dd98e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:32.349200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:32.350135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:32.354317 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3431/6945] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:32 2022] DELETE /identity/v3/projects/dac85ef1c85b4a06951f2df42dd98e50 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:32.358574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444f061-157d-4c70-97ab-fc224e8d343b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:32.358689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444f061-157d-4c70-97ab-fc224e8d343b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:32.358900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444f061-157d-4c70-97ab-fc224e8d343b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:32.403286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e444f061-157d-4c70-97ab-fc224e8d343b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:32.405192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e444f061-157d-4c70-97ab-fc224e8d343b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:32.421728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:32.423804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:32.541700 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3515/6946] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:32.544476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444f061-157d-4c70-97ab-fc224e8d343b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:32.545994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e444f061-157d-4c70-97ab-fc224e8d343b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:32.551883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b8e33-03ea-479d-b210-05636dab9a5d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:32.565151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7f23d-bead-4fbc-a65b-656ce6c61514 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:32.565746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7f23d-bead-4fbc-a65b-656ce6c61514 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:32.566022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7f23d-bead-4fbc-a65b-656ce6c61514 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:32.566309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7f23d-bead-4fbc-a65b-656ce6c61514 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:32.568989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d5e7f23d-bead-4fbc-a65b-656ce6c61514 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:32.572557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444f061-157d-4c70-97ab-fc224e8d343b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:32.585850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7f23d-bead-4fbc-a65b-656ce6c61514 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:32.587556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7f23d-bead-4fbc-a65b-656ce6c61514 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:32.589849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266de37-e48d-465e-a782-033688f5dd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:32.590515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266de37-e48d-465e-a782-033688f5dd8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:32.590783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266de37-e48d-465e-a782-033688f5dd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:32.591048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266de37-e48d-465e-a782-033688f5dd8f admin admin] PATH_INFO: `/v3/users/283d1c6c26b84779ac49fc55d18505eb` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:32.596512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266de37-e48d-465e-a782-033688f5dd8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=283d1c6c26b84779ac49fc55d18505eb)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:32.599688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266de37-e48d-465e-a782-033688f5dd8f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:32.603020 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3432/6947] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:22:32.605118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aa857-f9b0-4e6e-bce2-154092be7519 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:32.605316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aa857-f9b0-4e6e-bce2-154092be7519 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:32.605778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aa857-f9b0-4e6e-bce2-154092be7519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:32.638763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6a9aa857-f9b0-4e6e-bce2-154092be7519 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:32.641239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6a9aa857-f9b0-4e6e-bce2-154092be7519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:32.665263 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3433/6948] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:32.666369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aa857-f9b0-4e6e-bce2-154092be7519 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:32.680728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aa857-f9b0-4e6e-bce2-154092be7519 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:32.685989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0f9ce-d9cd-4dc9-83cd-b5edf0924fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:32.686613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0f9ce-d9cd-4dc9-83cd-b5edf0924fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:32.686950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0f9ce-d9cd-4dc9-83cd-b5edf0924fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:32.687214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0f9ce-d9cd-4dc9-83cd-b5edf0924fc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:32.687868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0f9ce-d9cd-4dc9-83cd-b5edf0924fc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:22:32.688170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0f9ce-d9cd-4dc9-83cd-b5edf0924fc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:32.690051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0f9ce-d9cd-4dc9-83cd-b5edf0924fc1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:32.708026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1266de37-e48d-465e-a782-033688f5dd8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '283d1c6c26b84779ac49fc55d18505eb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:32.708193 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3434/6949] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:22:32.709180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0f9ce-d9cd-4dc9-83cd-b5edf0924fc1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:32.719993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0f9ce-d9cd-4dc9-83cd-b5edf0924fc1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:32.726852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a773efa-bb05-4498-84c3-611f8f71cb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:32.727485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a773efa-bb05-4498-84c3-611f8f71cb27 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:32.728572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a773efa-bb05-4498-84c3-611f8f71cb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:32.728572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a773efa-bb05-4498-84c3-611f8f71cb27 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:32.729048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a773efa-bb05-4498-84c3-611f8f71cb27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:32.731409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a773efa-bb05-4498-84c3-611f8f71cb27 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:32.734897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1266de37-e48d-465e-a782-033688f5dd8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '283d1c6c26b84779ac49fc55d18505eb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:32.745379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1266de37-e48d-465e-a782-033688f5dd8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '283d1c6c26b84779ac49fc55d18505eb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:32.769664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1266de37-e48d-465e-a782-033688f5dd8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '283d1c6c26b84779ac49fc55d18505eb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:32.770578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1266de37-e48d-465e-a782-033688f5dd8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '283d1c6c26b84779ac49fc55d18505eb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:32.772833 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3435/6950] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:32 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:32.777250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a773efa-bb05-4498-84c3-611f8f71cb27 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:32.781106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0a773efa-bb05-4498-84c3-611f8f71cb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:32.788450 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3516/6951] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:32 2022] DELETE /identity/v3/users/283d1c6c26b84779ac49fc55d18505eb => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:32.792875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a15d78-3528-4c7c-a4ed-5956f6e6c634 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:32.793408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a15d78-3528-4c7c-a4ed-5956f6e6c634 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:32.793892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a15d78-3528-4c7c-a4ed-5956f6e6c634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:32.822400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-07a15d78-3528-4c7c-a4ed-5956f6e6c634 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:32.824661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-07a15d78-3528-4c7c-a4ed-5956f6e6c634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:32.837003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:32.924284 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3517/6952] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:22:32.925195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a15d78-3528-4c7c-a4ed-5956f6e6c634 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:32.926706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-07a15d78-3528-4c7c-a4ed-5956f6e6c634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:32.940064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a15d78-3528-4c7c-a4ed-5956f6e6c634 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:32.944208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a773efa-bb05-4498-84c3-611f8f71cb27 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:32.946925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:32.947812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:32.948091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:32.948356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] PATH_INFO: `/v3/projects/7eae0a6322fc432aa4458bbdec2fd318` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:32.949536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eae0a6322fc432aa4458bbdec2fd318)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:32.951458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:32.961404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc460d34-e3af-461f-a167-43a7898c175f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:32.962366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc460d34-e3af-461f-a167-43a7898c175f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:32.962939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc460d34-e3af-461f-a167-43a7898c175f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:32.963409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc460d34-e3af-461f-a167-43a7898c175f admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:32.964294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc460d34-e3af-461f-a167-43a7898c175f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:32.966708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc460d34-e3af-461f-a167-43a7898c175f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:33.014421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:33.024902 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3436/6953] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:32 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:22:33.025924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc460d34-e3af-461f-a167-43a7898c175f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:33.026192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eae0a6322fc432aa4458bbdec2fd318', 'request_id': 'req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.029034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cc460d34-e3af-461f-a167-43a7898c175f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:33.046603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eae0a6322fc432aa4458bbdec2fd318', 'request_id': 'req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.069368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eae0a6322fc432aa4458bbdec2fd318', 'request_id': 'req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.096592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] The token cache is being invalidate because project 7eae0a6322fc432aa4458bbdec2fd318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:33.096805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.103323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.104527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc460d34-e3af-461f-a167-43a7898c175f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:33.106807 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3518/6954] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:32 2022] DELETE /identity/v3/projects/7eae0a6322fc432aa4458bbdec2fd318 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:33.108881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:33.112999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:33.228018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a25eb7-943f-4908-846f-69e62f824f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:33.228936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a25eb7-943f-4908-846f-69e62f824f4c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:33.233174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a25eb7-943f-4908-846f-69e62f824f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:33.233596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a25eb7-943f-4908-846f-69e62f824f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:33.236213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f3a25eb7-943f-4908-846f-69e62f824f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:33.238947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e3cca-6e95-4f58-9867-d37c20f2dcaf admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:33.252226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a10c8d-8c19-480d-b276-45d999151aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:33.253173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a10c8d-8c19-480d-b276-45d999151aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:33.253651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a10c8d-8c19-480d-b276-45d999151aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:33.254153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a10c8d-8c19-480d-b276-45d999151aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:33.254873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a10c8d-8c19-480d-b276-45d999151aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:33.256693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a10c8d-8c19-480d-b276-45d999151aa2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:33.259186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a25eb7-943f-4908-846f-69e62f824f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:33.263547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a25eb7-943f-4908-846f-69e62f824f4c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:33.283589 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3519/6955] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:33.283899 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3437/6956] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:33.285980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a10c8d-8c19-480d-b276-45d999151aa2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:33.291418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-27a10c8d-8c19-480d-b276-45d999151aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:33.293389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a25eb7-943f-4908-846f-69e62f824f4c service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:33.306203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a25eb7-943f-4908-846f-69e62f824f4c service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:33.314003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a10c8d-8c19-480d-b276-45d999151aa2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:33.316496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed8c7a-347b-4aa6-9628-59fd379c698d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:33.317645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed8c7a-347b-4aa6-9628-59fd379c698d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:33.317920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed8c7a-347b-4aa6-9628-59fd379c698d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:33.318961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed8c7a-347b-4aa6-9628-59fd379c698d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:33.320325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed8c7a-347b-4aa6-9628-59fd379c698d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:33.322297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed8c7a-347b-4aa6-9628-59fd379c698d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:33.333381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:33.334055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:33.334335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:33.334607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] PATH_INFO: `/v3/users/bd6e2e37785a47e484c37d5b9fbce68a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:33.335932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd6e2e37785a47e484c37d5b9fbce68a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:33.337800 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3438/6957] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:33.341719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:33.343290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed8c7a-347b-4aa6-9628-59fd379c698d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:33.374032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed8c7a-347b-4aa6-9628-59fd379c698d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:33.381285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795bcd0-da45-496a-a3b3-62cf63e7fd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:33.382214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795bcd0-da45-496a-a3b3-62cf63e7fd7b admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:33.382502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795bcd0-da45-496a-a3b3-62cf63e7fd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:33.382942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795bcd0-da45-496a-a3b3-62cf63e7fd7b admin admin] PATH_INFO: `/v3/projects/4951d1fc525642ea97bcb1b772a5ea16/users/fdcfdca4137f498588b19aebb1b4053f/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:33.384167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:33.408484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795bcd0-da45-496a-a3b3-62cf63e7fd7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4951d1fc525642ea97bcb1b772a5ea16, user_id=fdcfdca4137f498588b19aebb1b4053f, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:33.413143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795bcd0-da45-496a-a3b3-62cf63e7fd7b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:33.430696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd6e2e37785a47e484c37d5b9fbce68a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.441781 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3439/6958] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:33 2022] PUT /identity/v3/projects/4951d1fc525642ea97bcb1b772a5ea16/users/fdcfdca4137f498588b19aebb1b4053f/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:33.448916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd6e2e37785a47e484c37d5b9fbce68a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.451420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1618c9-9464-4f4f-8b8a-6ddad75bada0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:33.451698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1618c9-9464-4f4f-8b8a-6ddad75bada0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:33.451972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1618c9-9464-4f4f-8b8a-6ddad75bada0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:33.461904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd6e2e37785a47e484c37d5b9fbce68a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.484456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd6e2e37785a47e484c37d5b9fbce68a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.485539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd6e2e37785a47e484c37d5b9fbce68a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.495789 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3520/6959] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:33 2022] DELETE /identity/v3/users/bd6e2e37785a47e484c37d5b9fbce68a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:33.498744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1b1618c9-9464-4f4f-8b8a-6ddad75bada0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:33.500188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1b1618c9-9464-4f4f-8b8a-6ddad75bada0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:33.637338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:33.638601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:33.677901 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3440/6960] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:33.692248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1618c9-9464-4f4f-8b8a-6ddad75bada0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:33.695540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1b1618c9-9464-4f4f-8b8a-6ddad75bada0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:33.744399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:33.752085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae39cb-40a8-4133-94ea-c092630ce069 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:33.760066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:33.761361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:33.761644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:33.762029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] PATH_INFO: `/v3/projects/fe2ca77984cf44b9a1f256de78fca212` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:33.763688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe2ca77984cf44b9a1f256de78fca212)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:33.766363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:33.827295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1618c9-9464-4f4f-8b8a-6ddad75bada0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:33.844699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe2ca77984cf44b9a1f256de78fca212', 'request_id': 'req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.860398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe2ca77984cf44b9a1f256de78fca212', 'request_id': 'req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.869582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe2ca77984cf44b9a1f256de78fca212', 'request_id': 'req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.888119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbdbe4-a089-41dd-a150-300a3d5019e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:33.888784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbdbe4-a089-41dd-a150-300a3d5019e1 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:33.889070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbdbe4-a089-41dd-a150-300a3d5019e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:33.889365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbdbe4-a089-41dd-a150-300a3d5019e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:33.893391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] The token cache is being invalidate because project fe2ca77984cf44b9a1f256de78fca212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:33.893823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.895075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:33.898759 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3521/6961] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:33 2022] DELETE /identity/v3/projects/fe2ca77984cf44b9a1f256de78fca212 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:33.902661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:33.905390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:33.967812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:34.001782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d155af-1fcd-4fe5-8cc1-85ad1f55e9ff admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:34.007532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:34.008199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:34.008508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:34.008784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] PATH_INFO: `/v3/users/659a3524dcd54ecd876e354f18b25930` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:34.009970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=659a3524dcd54ecd876e354f18b25930)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:34.012108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:34.031800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbdbe4-a089-41dd-a150-300a3d5019e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:34.036228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbdbe4-a089-41dd-a150-300a3d5019e1 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:34.066268 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3441/6962] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:34.080050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbdbe4-a089-41dd-a150-300a3d5019e1 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:34.082096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-87fbdbe4-a089-41dd-a150-300a3d5019e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:34.093679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '659a3524dcd54ecd876e354f18b25930'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.103463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbdbe4-a089-41dd-a150-300a3d5019e1 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:34.114109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593656d-7d83-4e5d-835e-18082209b44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:34.114964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593656d-7d83-4e5d-835e-18082209b44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:34.115335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593656d-7d83-4e5d-835e-18082209b44e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:34.115461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '659a3524dcd54ecd876e354f18b25930'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.115767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593656d-7d83-4e5d-835e-18082209b44e admin admin] PATH_INFO: `/v3/users/99a79da61566495e855ff1ba4a467cf2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:34.117145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593656d-7d83-4e5d-835e-18082209b44e admin admin] RBAC: Authorizing `identity:delete_user(user_id=99a79da61566495e855ff1ba4a467cf2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:34.119508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593656d-7d83-4e5d-835e-18082209b44e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:34.129271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '659a3524dcd54ecd876e354f18b25930'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.164618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '659a3524dcd54ecd876e354f18b25930'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.166731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '659a3524dcd54ecd876e354f18b25930'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.180136 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3522/6963] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:33 2022] DELETE /identity/v3/users/659a3524dcd54ecd876e354f18b25930 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:34.183251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:34.183251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:34.196944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4593656d-7d83-4e5d-835e-18082209b44e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99a79da61566495e855ff1ba4a467cf2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.228497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4593656d-7d83-4e5d-835e-18082209b44e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99a79da61566495e855ff1ba4a467cf2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.269514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4593656d-7d83-4e5d-835e-18082209b44e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99a79da61566495e855ff1ba4a467cf2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.270577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4593656d-7d83-4e5d-835e-18082209b44e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99a79da61566495e855ff1ba4a467cf2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.271358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4593656d-7d83-4e5d-835e-18082209b44e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99a79da61566495e855ff1ba4a467cf2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.311002 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3442/6964] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:34 2022] DELETE /identity/v3/users/99a79da61566495e855ff1ba4a467cf2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:34.321075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211515e4-2990-4622-8b6b-73ed4be037c2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:34.416790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:34.417759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:34.418053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:34.418182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] PATH_INFO: `/v3/projects/77900bcbef8b4d70a9584a7734065585` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:34.420263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77900bcbef8b4d70a9584a7734065585)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:34.423640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:34.427478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:34.507166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77900bcbef8b4d70a9584a7734065585', 'request_id': 'req-664d6887-321c-4366-864d-8f08ae70fdf9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.517998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77900bcbef8b4d70a9584a7734065585', 'request_id': 'req-664d6887-321c-4366-864d-8f08ae70fdf9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.526759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77900bcbef8b4d70a9584a7734065585', 'request_id': 'req-664d6887-321c-4366-864d-8f08ae70fdf9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.542125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] The token cache is being invalidate because project 77900bcbef8b4d70a9584a7734065585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:34.542308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.543230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.545384 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3523/6965] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:34 2022] DELETE /identity/v3/projects/77900bcbef8b4d70a9584a7734065585 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:34.551427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069064d4-1eac-4430-8467-510546568393 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:34.552262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069064d4-1eac-4430-8467-510546568393 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:34.552567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069064d4-1eac-4430-8467-510546568393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:34.566235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:34.582112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-069064d4-1eac-4430-8467-510546568393 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:34.584497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-069064d4-1eac-4430-8467-510546568393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:34.632911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:34.634360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:34.727969 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3443/6966] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:34.730882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19148f73-5129-4cf0-b1d4-9f95dd4e0629 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:34.731558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19148f73-5129-4cf0-b1d4-9f95dd4e0629 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:34.732151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19148f73-5129-4cf0-b1d4-9f95dd4e0629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:34.775179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d6887-321c-4366-864d-8f08ae70fdf9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:34.782652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:34.783843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:34.784126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:34.784458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] PATH_INFO: `/v3/projects/4b9fcf5fb1e247e99042f16c7a43086a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:34.785817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b9fcf5fb1e247e99042f16c7a43086a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:34.789571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:34.794296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-19148f73-5129-4cf0-b1d4-9f95dd4e0629 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:34.795652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-19148f73-5129-4cf0-b1d4-9f95dd4e0629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:34.823556 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3444/6967] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:34.828104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19148f73-5129-4cf0-b1d4-9f95dd4e0629 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:34.830365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-19148f73-5129-4cf0-b1d4-9f95dd4e0629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:34.864239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19148f73-5129-4cf0-b1d4-9f95dd4e0629 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:34.864792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b9fcf5fb1e247e99042f16c7a43086a', 'request_id': 'req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.878088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b9fcf5fb1e247e99042f16c7a43086a', 'request_id': 'req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.888305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b9fcf5fb1e247e99042f16c7a43086a', 'request_id': 'req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.909078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] The token cache is being invalidate because project 4b9fcf5fb1e247e99042f16c7a43086a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:34.909481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.910993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:34.914101 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3524/6968] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:34 2022] DELETE /identity/v3/projects/4b9fcf5fb1e247e99042f16c7a43086a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:34.915367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:34.917010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:34.953429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:34.980779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:34.981852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:34.982280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:34.982693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] PATH_INFO: `/v3/projects/ce0bc22a432d44efb17402c8a4e39ed4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:34.984027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce0bc22a432d44efb17402c8a4e39ed4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:34.986171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:35.015921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1798d9a-d03f-4e08-bf09-b91b6c57ad4a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.035921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a477c9-f74c-4186-8ed2-2574e199bc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.036333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a477c9-f74c-4186-8ed2-2574e199bc84 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.037195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a477c9-f74c-4186-8ed2-2574e199bc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.037444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a477c9-f74c-4186-8ed2-2574e199bc84 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.038400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a477c9-f74c-4186-8ed2-2574e199bc84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:22:35.039067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a477c9-f74c-4186-8ed2-2574e199bc84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:35.040816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a477c9-f74c-4186-8ed2-2574e199bc84 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:35.066550 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3525/6969] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:22:35.069327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357ac40-7624-4c06-b9a6-80ca8127411a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.069605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357ac40-7624-4c06-b9a6-80ca8127411a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.069877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357ac40-7624-4c06-b9a6-80ca8127411a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.077750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce0bc22a432d44efb17402c8a4e39ed4', 'request_id': 'req-e41e5652-e6b7-431a-88a8-04ec50304936'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:35.087565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce0bc22a432d44efb17402c8a4e39ed4', 'request_id': 'req-e41e5652-e6b7-431a-88a8-04ec50304936'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:35.095706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b357ac40-7624-4c06-b9a6-80ca8127411a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:35.097185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b357ac40-7624-4c06-b9a6-80ca8127411a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:35.101370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce0bc22a432d44efb17402c8a4e39ed4', 'request_id': 'req-e41e5652-e6b7-431a-88a8-04ec50304936'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:35.121904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] The token cache is being invalidate because project ce0bc22a432d44efb17402c8a4e39ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:35.122547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:35.123623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:35.125652 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3445/6970] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:34 2022] DELETE /identity/v3/projects/ce0bc22a432d44efb17402c8a4e39ed4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:35.126768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:35.128282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:35.194762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:35.198594 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3526/6971] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:35.200483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a046d-1ed6-4bcc-a531-8f7a39cde861 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.201085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a046d-1ed6-4bcc-a531-8f7a39cde861 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.201393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a046d-1ed6-4bcc-a531-8f7a39cde861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.253922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-089a046d-1ed6-4bcc-a531-8f7a39cde861 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:35.255568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-089a046d-1ed6-4bcc-a531-8f7a39cde861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:35.272624 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3527/6972] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:35.274666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297aecc-3d20-444b-a1ed-f4d194b99247 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.278437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297aecc-3d20-444b-a1ed-f4d194b99247 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.278841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297aecc-3d20-444b-a1ed-f4d194b99247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.288201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e5652-e6b7-431a-88a8-04ec50304936 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.294091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c35d4b-d045-4596-96ed-d1ddaf0c27fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.294091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c35d4b-d045-4596-96ed-d1ddaf0c27fe admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.294414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c35d4b-d045-4596-96ed-d1ddaf0c27fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.294506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c35d4b-d045-4596-96ed-d1ddaf0c27fe admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.295152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c35d4b-d045-4596-96ed-d1ddaf0c27fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:35.298511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c35d4b-d045-4596-96ed-d1ddaf0c27fe admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:35.314692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4297aecc-3d20-444b-a1ed-f4d194b99247 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:35.317915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4297aecc-3d20-444b-a1ed-f4d194b99247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:35.332430 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3528/6973] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:35.333465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8397602-7b72-4ae4-8aae-b4f4f8ef46ba None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.333584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8397602-7b72-4ae4-8aae-b4f4f8ef46ba None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.333846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8397602-7b72-4ae4-8aae-b4f4f8ef46ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.351440 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3446/6974] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:35 2022] POST /identity/v3/projects => generated 383 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:35.351440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c35d4b-d045-4596-96ed-d1ddaf0c27fe admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:35.360812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e8397602-7b72-4ae4-8aae-b4f4f8ef46ba None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:35.362291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e8397602-7b72-4ae4-8aae-b4f4f8ef46ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:35.443745 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3529/6975] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:35.444922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8397602-7b72-4ae4-8aae-b4f4f8ef46ba None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:35.457225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8397602-7b72-4ae4-8aae-b4f4f8ef46ba None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.464404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778fa8f-c19c-4a6d-aa80-ec6bd40705cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.465139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778fa8f-c19c-4a6d-aa80-ec6bd40705cc admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.465511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778fa8f-c19c-4a6d-aa80-ec6bd40705cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.465997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778fa8f-c19c-4a6d-aa80-ec6bd40705cc admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.467516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778fa8f-c19c-4a6d-aa80-ec6bd40705cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:35.470034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778fa8f-c19c-4a6d-aa80-ec6bd40705cc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:35.498966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c35d4b-d045-4596-96ed-d1ddaf0c27fe admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.507345 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3530/6976] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:35 2022] POST /identity/v3/users => generated 292 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:35.508647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778fa8f-c19c-4a6d-aa80-ec6bd40705cc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:35.513724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e1048-895e-4461-8541-9d522041bab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.514365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e1048-895e-4461-8541-9d522041bab1 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.514924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e1048-895e-4461-8541-9d522041bab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.514924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e1048-895e-4461-8541-9d522041bab1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.515533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e1048-895e-4461-8541-9d522041bab1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:22:35.516261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e1048-895e-4461-8541-9d522041bab1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:35.517626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e1048-895e-4461-8541-9d522041bab1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:35.526815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778fa8f-c19c-4a6d-aa80-ec6bd40705cc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.540764 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3447/6977] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:22:35.542580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e1048-895e-4461-8541-9d522041bab1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:35.544901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565cc66-1f98-467d-8bc2-74a08c9f086a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.545671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565cc66-1f98-467d-8bc2-74a08c9f086a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.546097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565cc66-1f98-467d-8bc2-74a08c9f086a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.546205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565cc66-1f98-467d-8bc2-74a08c9f086a admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.546995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565cc66-1f98-467d-8bc2-74a08c9f086a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:22:35.547295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565cc66-1f98-467d-8bc2-74a08c9f086a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:35.549320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565cc66-1f98-467d-8bc2-74a08c9f086a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:35.557603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e1048-895e-4461-8541-9d522041bab1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.569707 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3531/6978] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:22:35.569707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565cc66-1f98-467d-8bc2-74a08c9f086a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:35.572637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33843b-cb2c-426b-b3d1-12160c334c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.573032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33843b-cb2c-426b-b3d1-12160c334c07 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.573308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33843b-cb2c-426b-b3d1-12160c334c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.573573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33843b-cb2c-426b-b3d1-12160c334c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.574224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33843b-cb2c-426b-b3d1-12160c334c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:35.575843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33843b-cb2c-426b-b3d1-12160c334c07 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:35.581964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565cc66-1f98-467d-8bc2-74a08c9f086a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.590824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f231b-4136-462e-9bdb-6b8a48023d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.591461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f231b-4136-462e-9bdb-6b8a48023d8d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.591732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f231b-4136-462e-9bdb-6b8a48023d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.592003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f231b-4136-462e-9bdb-6b8a48023d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.592668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f231b-4136-462e-9bdb-6b8a48023d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:35.594942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f231b-4136-462e-9bdb-6b8a48023d8d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:35.602919 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3448/6979] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:35.606202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33843b-cb2c-426b-b3d1-12160c334c07 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:35.609900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:35.628932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33843b-cb2c-426b-b3d1-12160c334c07 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.641511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c14af6-4f4a-48e2-a238-98f44cbf9684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.642318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c14af6-4f4a-48e2-a238-98f44cbf9684 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.642428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c14af6-4f4a-48e2-a238-98f44cbf9684 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.642688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c14af6-4f4a-48e2-a238-98f44cbf9684 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.643314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c14af6-4f4a-48e2-a238-98f44cbf9684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:35.645712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c14af6-4f4a-48e2-a238-98f44cbf9684 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:35.646906 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3532/6980] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:35 2022] POST /identity/v3/projects => generated 393 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:35.655668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f231b-4136-462e-9bdb-6b8a48023d8d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:35.705864 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3449/6981] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:35 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:35.707335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c14af6-4f4a-48e2-a238-98f44cbf9684 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:35.791133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c14af6-4f4a-48e2-a238-98f44cbf9684 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.796670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc27dc6-9142-4b8c-8a2d-058ce56feaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.797487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc27dc6-9142-4b8c-8a2d-058ce56feaf3 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.797878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc27dc6-9142-4b8c-8a2d-058ce56feaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.798231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc27dc6-9142-4b8c-8a2d-058ce56feaf3 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.799219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc27dc6-9142-4b8c-8a2d-058ce56feaf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:35.802222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc27dc6-9142-4b8c-8a2d-058ce56feaf3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:35.810852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f231b-4136-462e-9bdb-6b8a48023d8d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.823010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c9f95-4dac-46eb-a680-2eacee6644ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.824795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c9f95-4dac-46eb-a680-2eacee6644ac admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.825255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c9f95-4dac-46eb-a680-2eacee6644ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.825662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c9f95-4dac-46eb-a680-2eacee6644ac admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.826468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c9f95-4dac-46eb-a680-2eacee6644ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:35.830055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c9f95-4dac-46eb-a680-2eacee6644ac admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:35.843698 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3450/6982] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:35 2022] POST /identity/v3/users => generated 297 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:35.848588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc27dc6-9142-4b8c-8a2d-058ce56feaf3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:35.862856 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3533/6983] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:35.866002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c9f95-4dac-46eb-a680-2eacee6644ac admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:35.866285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:35.879501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc27dc6-9142-4b8c-8a2d-058ce56feaf3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.893892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81618bc5-644e-4c38-82aa-267007b38f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.895405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c9f95-4dac-46eb-a680-2eacee6644ac admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.896693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81618bc5-644e-4c38-82aa-267007b38f0a admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.897693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81618bc5-644e-4c38-82aa-267007b38f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.898857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81618bc5-644e-4c38-82aa-267007b38f0a admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.899523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81618bc5-644e-4c38-82aa-267007b38f0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:35.902834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81618bc5-644e-4c38-82aa-267007b38f0a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:35.911558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c469e8-103f-4ad6-98b0-b3be10f539e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.912202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c469e8-103f-4ad6-98b0-b3be10f539e4 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.912507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c469e8-103f-4ad6-98b0-b3be10f539e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.912783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c469e8-103f-4ad6-98b0-b3be10f539e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.913952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c469e8-103f-4ad6-98b0-b3be10f539e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:35.916995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c469e8-103f-4ad6-98b0-b3be10f539e4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:35.941089 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3534/6984] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:35.941089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c469e8-103f-4ad6-98b0-b3be10f539e4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:35.956007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c469e8-103f-4ad6-98b0-b3be10f539e4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.960350 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3451/6985] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:35 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:35.961366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81618bc5-644e-4c38-82aa-267007b38f0a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:35.963337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590fde8-877f-4994-84da-8a79afae44b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.964354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590fde8-877f-4994-84da-8a79afae44b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:35.964929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590fde8-877f-4994-84da-8a79afae44b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:35.965365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590fde8-877f-4994-84da-8a79afae44b2 admin admin] PATH_INFO: `/v3/projects/09830dddcf0a4c3bb415fd0491a26dd9/users/8cb96a00e1bb4d2ba4b38687af4f2020/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:35.987020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81618bc5-644e-4c38-82aa-267007b38f0a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:35.997162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9056c-b6f0-4394-a57f-c65a06ca2d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:35.997502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590fde8-877f-4994-84da-8a79afae44b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09830dddcf0a4c3bb415fd0491a26dd9, user_id=8cb96a00e1bb4d2ba4b38687af4f2020, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:35.998469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9056c-b6f0-4394-a57f-c65a06ca2d20 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:36.006631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9056c-b6f0-4394-a57f-c65a06ca2d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:36.007177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9056c-b6f0-4394-a57f-c65a06ca2d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:36.007572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590fde8-877f-4994-84da-8a79afae44b2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:36.008957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9056c-b6f0-4394-a57f-c65a06ca2d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:36.010760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9056c-b6f0-4394-a57f-c65a06ca2d20 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:36.035151 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3452/6986] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:36.040590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9056c-b6f0-4394-a57f-c65a06ca2d20 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:36.045800 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3535/6987] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:35 2022] PUT /identity/v3/projects/09830dddcf0a4c3bb415fd0491a26dd9/users/8cb96a00e1bb4d2ba4b38687af4f2020/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:36.056723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590fde8-877f-4994-84da-8a79afae44b2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:36.069744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9056c-b6f0-4394-a57f-c65a06ca2d20 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:36.178923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590fde8-877f-4994-84da-8a79afae44b2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:36.187627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c373da8-1adc-4fab-a0ea-c598c5b4eb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:36.188911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c373da8-1adc-4fab-a0ea-c598c5b4eb0b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:36.188911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c373da8-1adc-4fab-a0ea-c598c5b4eb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:36.189593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eca82f-622f-4217-85bc-fa9d9b14e7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:36.190302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eca82f-622f-4217-85bc-fa9d9b14e7da admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:36.190571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eca82f-622f-4217-85bc-fa9d9b14e7da admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:36.190839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eca82f-622f-4217-85bc-fa9d9b14e7da admin admin] PATH_INFO: `/v3/projects/913a24b8cde54148aded4d0150b70f75/users/2a890d5ec4d44762ba61f5b977734a97/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:36.191122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c373da8-1adc-4fab-a0ea-c598c5b4eb0b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:36.191960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c373da8-1adc-4fab-a0ea-c598c5b4eb0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:36.195662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c373da8-1adc-4fab-a0ea-c598c5b4eb0b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:36.217196 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3453/6988] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:36.218857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7897d-624f-43e1-874f-25e3b222ab8a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:36.219165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7897d-624f-43e1-874f-25e3b222ab8a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:36.219470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7897d-624f-43e1-874f-25e3b222ab8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:36.220028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eca82f-622f-4217-85bc-fa9d9b14e7da admin admin] RBAC: Authorizing `identity:create_grant(project_id=913a24b8cde54148aded4d0150b70f75, user_id=2a890d5ec4d44762ba61f5b977734a97, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:36.225116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eca82f-622f-4217-85bc-fa9d9b14e7da admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:36.246535 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3536/6989] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:36 2022] PUT /identity/v3/projects/913a24b8cde54148aded4d0150b70f75/users/2a890d5ec4d44762ba61f5b977734a97/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:36.248004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eca82f-622f-4217-85bc-fa9d9b14e7da admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:36.286242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1ab7897d-624f-43e1-874f-25e3b222ab8a None None] MFA Rules not processed for user `8cb96a00e1bb4d2ba4b38687af4f2020`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:36.288173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1ab7897d-624f-43e1-874f-25e3b222ab8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:36.329695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:36.383720 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3454/6990] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:36.383720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe02df4-1df1-431e-b83e-ecd3dc1ac0dc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:36.383720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe02df4-1df1-431e-b83e-ecd3dc1ac0dc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:36.383720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe02df4-1df1-431e-b83e-ecd3dc1ac0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:36.387265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eca82f-622f-4217-85bc-fa9d9b14e7da admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:36.408890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e980ca5-8093-4ed0-ad27-43ba362d653e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:36.412214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e980ca5-8093-4ed0-ad27-43ba362d653e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:36.412661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e980ca5-8093-4ed0-ad27-43ba362d653e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:36.412945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e980ca5-8093-4ed0-ad27-43ba362d653e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:36.413629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e980ca5-8093-4ed0-ad27-43ba362d653e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:36.416966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e980ca5-8093-4ed0-ad27-43ba362d653e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:36.428836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-afe02df4-1df1-431e-b83e-ecd3dc1ac0dc None None] MFA Rules not processed for user `2a890d5ec4d44762ba61f5b977734a97`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:36.430270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-afe02df4-1df1-431e-b83e-ecd3dc1ac0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:36.438171 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3537/6991] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:36.444837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e980ca5-8093-4ed0-ad27-43ba362d653e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:36.466565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e980ca5-8093-4ed0-ad27-43ba362d653e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:36.474546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3a6e9-1aa8-4752-8499-1c7c8959c1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:36.475552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3a6e9-1aa8-4752-8499-1c7c8959c1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:36.475949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3a6e9-1aa8-4752-8499-1c7c8959c1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:36.476200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3a6e9-1aa8-4752-8499-1c7c8959c1f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:36.477362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f3a6e9-1aa8-4752-8499-1c7c8959c1f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:36.479853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f3a6e9-1aa8-4752-8499-1c7c8959c1f7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:36.503682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:36.539734 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3455/6992] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:36.540804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe02df4-1df1-431e-b83e-ecd3dc1ac0dc None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:36.554526 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3538/6993] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:36 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:36.558812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3a6e9-1aa8-4752-8499-1c7c8959c1f7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:36.559871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe02df4-1df1-431e-b83e-ecd3dc1ac0dc None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:36.699274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9544e-28f8-4bc8-acfa-c16c1fc5dafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:36.700415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3a6e9-1aa8-4752-8499-1c7c8959c1f7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:36.701468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9544e-28f8-4bc8-acfa-c16c1fc5dafe admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:36.702318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9544e-28f8-4bc8-acfa-c16c1fc5dafe admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:36.703112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9544e-28f8-4bc8-acfa-c16c1fc5dafe admin admin] PATH_INFO: `/v3/projects/ce224716983049beac8cb9d463715e5d/users/ae2c339c41354a159fc8881b56ac45ec/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:36.710871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c65d6c-14f6-4dec-91eb-1f70e729d97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:36.711752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c65d6c-14f6-4dec-91eb-1f70e729d97d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:36.714449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c65d6c-14f6-4dec-91eb-1f70e729d97d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:36.716431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c65d6c-14f6-4dec-91eb-1f70e729d97d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:36.720095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c65d6c-14f6-4dec-91eb-1f70e729d97d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:36.724464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c65d6c-14f6-4dec-91eb-1f70e729d97d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:36.742165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9544e-28f8-4bc8-acfa-c16c1fc5dafe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce224716983049beac8cb9d463715e5d, user_id=ae2c339c41354a159fc8881b56ac45ec, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:36.748874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9544e-28f8-4bc8-acfa-c16c1fc5dafe admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:36.769483 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3539/6994] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:36 2022] POST /identity/v3/users => generated 299 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:22:36.770304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c65d6c-14f6-4dec-91eb-1f70e729d97d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:36.776456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-32c65d6c-14f6-4dec-91eb-1f70e729d97d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:36.778357 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3456/6995] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:36 2022] PUT /identity/v3/projects/ce224716983049beac8cb9d463715e5d/users/ae2c339c41354a159fc8881b56ac45ec/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:36.779446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9544e-28f8-4bc8-acfa-c16c1fc5dafe admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:36.885076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9544e-28f8-4bc8-acfa-c16c1fc5dafe admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:36.887486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c65d6c-14f6-4dec-91eb-1f70e729d97d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:36.890604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f27c3-a2ea-468f-8c4f-2766176b5530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:36.891379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f27c3-a2ea-468f-8c4f-2766176b5530 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:36.891657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f27c3-a2ea-468f-8c4f-2766176b5530 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:36.891928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f27c3-a2ea-468f-8c4f-2766176b5530 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:36.892740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f27c3-a2ea-468f-8c4f-2766176b5530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:36.894370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f27c3-a2ea-468f-8c4f-2766176b5530 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:36.907371 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3457/6996] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:36.908977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c9276-401d-4008-8efb-9e5dac2f8535 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:36.909403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c9276-401d-4008-8efb-9e5dac2f8535 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:36.909683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c9276-401d-4008-8efb-9e5dac2f8535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:36.934693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-604c9276-401d-4008-8efb-9e5dac2f8535 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:36.935916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-604c9276-401d-4008-8efb-9e5dac2f8535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:36.946997 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3458/6997] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:36.947930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c9276-401d-4008-8efb-9e5dac2f8535 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:36.949591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-604c9276-401d-4008-8efb-9e5dac2f8535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:36.952815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c24f9-ae48-4757-a9cb-0d50b4e7bb31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:36.953677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c24f9-ae48-4757-a9cb-0d50b4e7bb31 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:36.954135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c24f9-ae48-4757-a9cb-0d50b4e7bb31 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:36.954584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c24f9-ae48-4757-a9cb-0d50b4e7bb31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:36.967168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:37.026005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c24f9-ae48-4757-a9cb-0d50b4e7bb31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:37.027466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c24f9-ae48-4757-a9cb-0d50b4e7bb31 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:37.037515 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3540/6998] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:36 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:37.038332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c24f9-ae48-4757-a9cb-0d50b4e7bb31 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:37.044640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:37.053715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c24f9-ae48-4757-a9cb-0d50b4e7bb31 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:37.061849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef91d0-f066-4dbd-933d-91ea263fce37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:37.062414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef91d0-f066-4dbd-933d-91ea263fce37 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:37.062679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef91d0-f066-4dbd-933d-91ea263fce37 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:37.062957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef91d0-f066-4dbd-933d-91ea263fce37 admin admin] PATH_INFO: `/v3/projects/b82a71abe5724eeab4f25037996c9b0a/users/53410cc3fb44400e8b6fdf146494f5cf/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:37.074690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c9276-401d-4008-8efb-9e5dac2f8535 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:37.081208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef91d0-f066-4dbd-933d-91ea263fce37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b82a71abe5724eeab4f25037996c9b0a, user_id=53410cc3fb44400e8b6fdf146494f5cf, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:37.084334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef91d0-f066-4dbd-933d-91ea263fce37 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:37.088710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a258b07-ed8e-4e49-83c1-2ca652d91e41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:37.089475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a258b07-ed8e-4e49-83c1-2ca652d91e41 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:37.089797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a258b07-ed8e-4e49-83c1-2ca652d91e41 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:37.090044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a258b07-ed8e-4e49-83c1-2ca652d91e41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:37.092288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2a258b07-ed8e-4e49-83c1-2ca652d91e41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:37.103188 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3541/6999] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:37 2022] PUT /identity/v3/projects/b82a71abe5724eeab4f25037996c9b0a/users/53410cc3fb44400e8b6fdf146494f5cf/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:37.106198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef91d0-f066-4dbd-933d-91ea263fce37 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:37.110761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a258b07-ed8e-4e49-83c1-2ca652d91e41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:37.112590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a258b07-ed8e-4e49-83c1-2ca652d91e41 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:37.129666 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3459/7000] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:37.130568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a258b07-ed8e-4e49-83c1-2ca652d91e41 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:37.143383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a258b07-ed8e-4e49-83c1-2ca652d91e41 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:37.175262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef91d0-f066-4dbd-933d-91ea263fce37 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:37.180563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e9cea-d170-41a8-8183-3b0263ab333f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:37.181271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e9cea-d170-41a8-8183-3b0263ab333f service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:37.181667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e9cea-d170-41a8-8183-3b0263ab333f service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:37.182066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e9cea-d170-41a8-8183-3b0263ab333f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:37.219339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fd5a0-3671-46c7-846d-f19e44295b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:37.221482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fd5a0-3671-46c7-846d-f19e44295b59 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:37.221921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fd5a0-3671-46c7-846d-f19e44295b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:37.222269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fd5a0-3671-46c7-846d-f19e44295b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:37.224537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fd5a0-3671-46c7-846d-f19e44295b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:37.226317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fd5a0-3671-46c7-846d-f19e44295b59 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:37.241840 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3460/7001] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:37.246430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fd5a0-3671-46c7-846d-f19e44295b59 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:37.260688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fd5a0-3671-46c7-846d-f19e44295b59 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:37.267351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bda1c8-df1f-44eb-8be9-4c62dd21c7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:37.268352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bda1c8-df1f-44eb-8be9-4c62dd21c7ed admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:37.268840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bda1c8-df1f-44eb-8be9-4c62dd21c7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:37.269252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bda1c8-df1f-44eb-8be9-4c62dd21c7ed admin admin] PATH_INFO: `/v3/projects/b82a71abe5724eeab4f25037996c9b0a/users/53410cc3fb44400e8b6fdf146494f5cf/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:37.272911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bda1c8-df1f-44eb-8be9-4c62dd21c7ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=b82a71abe5724eeab4f25037996c9b0a, user_id=53410cc3fb44400e8b6fdf146494f5cf, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:37.276945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bda1c8-df1f-44eb-8be9-4c62dd21c7ed admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:37.282845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e9cea-d170-41a8-8183-3b0263ab333f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:37.285619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e9cea-d170-41a8-8183-3b0263ab333f service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:37.290081 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3461/7002] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:37 2022] PUT /identity/v3/projects/b82a71abe5724eeab4f25037996c9b0a/users/53410cc3fb44400e8b6fdf146494f5cf/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:37.305722 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3542/7003] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:37.305842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fad67a-4992-4e97-b258-cb907b15a50b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:37.306308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fad67a-4992-4e97-b258-cb907b15a50b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:37.306651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fad67a-4992-4e97-b258-cb907b15a50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:37.348700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-87fad67a-4992-4e97-b258-cb907b15a50b None None] MFA Rules not processed for user `53410cc3fb44400e8b6fdf146494f5cf`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:37.350036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-87fad67a-4992-4e97-b258-cb907b15a50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:37.446968 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3462/7004] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:37.469577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e9cea-d170-41a8-8183-3b0263ab333f service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:37.487487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e9cea-d170-41a8-8183-3b0263ab333f service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:37.549899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:37.578055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5743d-c90d-4add-90ee-74348606bee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:37.579392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5743d-c90d-4add-90ee-74348606bee5 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:37.579392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5743d-c90d-4add-90ee-74348606bee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:37.579392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5743d-c90d-4add-90ee-74348606bee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:37.591549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5743d-c90d-4add-90ee-74348606bee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:37.593314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5743d-c90d-4add-90ee-74348606bee5 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:37.604881 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3543/7005] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:45.021452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fad67a-4992-4e97-b258-cb907b15a50b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:45.132427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:45.140103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fad67a-4992-4e97-b258-cb907b15a50b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:45.148134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:45.148971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:45.149342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:45.149709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] PATH_INFO: `/v3/users/2a890d5ec4d44762ba61f5b977734a97` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:45.151244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a890d5ec4d44762ba61f5b977734a97)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:45.153729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:45.222120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a890d5ec4d44762ba61f5b977734a97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:45.234732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a890d5ec4d44762ba61f5b977734a97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:45.255012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a890d5ec4d44762ba61f5b977734a97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:45.256039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a890d5ec4d44762ba61f5b977734a97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:45.256711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a890d5ec4d44762ba61f5b977734a97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:45.271182 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3463/7006] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:45 2022] DELETE /identity/v3/users/2a890d5ec4d44762ba61f5b977734a97 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:45.275798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f4021-551f-4bba-b700-b61dcb78c27e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:45.276293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f4021-551f-4bba-b700-b61dcb78c27e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:45.276406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f4021-551f-4bba-b700-b61dcb78c27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:45.295409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d37f4021-551f-4bba-b700-b61dcb78c27e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:45.296874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d37f4021-551f-4bba-b700-b61dcb78c27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:45.358151 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3544/7007] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:45.368828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:45.370387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:45.436658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f23c9-1937-4086-9cd2-4cfa6df72749 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:45.441004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c123e-7d52-4919-bc55-585ad01bf38b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:45.442048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c123e-7d52-4919-bc55-585ad01bf38b service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:45.442418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c123e-7d52-4919-bc55-585ad01bf38b service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:45.442694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c123e-7d52-4919-bc55-585ad01bf38b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:45.453375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c123e-7d52-4919-bc55-585ad01bf38b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:45.454823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c123e-7d52-4919-bc55-585ad01bf38b service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:45.466038 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3464/7008] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:45.593147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2a5e6-4b47-41d1-b762-dadb6855656c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:45.593440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2a5e6-4b47-41d1-b762-dadb6855656c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:45.593698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2a5e6-4b47-41d1-b762-dadb6855656c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:45.627274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2bd2a5e6-4b47-41d1-b762-dadb6855656c None None] MFA Rules not processed for user `fdcfdca4137f498588b19aebb1b4053f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:45.628535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2bd2a5e6-4b47-41d1-b762-dadb6855656c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:45.662787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c123e-7d52-4919-bc55-585ad01bf38b service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:45.664355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2e4c123e-7d52-4919-bc55-585ad01bf38b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:45.685685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c123e-7d52-4919-bc55-585ad01bf38b service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:45.691897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fddcacf-ae69-4098-bf60-2e5f4010127f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:45.691897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddcacf-ae69-4098-bf60-2e5f4010127f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:45.691897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddcacf-ae69-4098-bf60-2e5f4010127f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:45.691897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddcacf-ae69-4098-bf60-2e5f4010127f admin admin] PATH_INFO: `/v3/projects/913a24b8cde54148aded4d0150b70f75` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:45.693569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fddcacf-ae69-4098-bf60-2e5f4010127f admin admin] RBAC: Authorizing `identity:delete_project(project_id=913a24b8cde54148aded4d0150b70f75)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:45.695739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fddcacf-ae69-4098-bf60-2e5f4010127f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:45.743154 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3545/7009] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:45.744857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2a5e6-4b47-41d1-b762-dadb6855656c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:45.746153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2bd2a5e6-4b47-41d1-b762-dadb6855656c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:45.763458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2a5e6-4b47-41d1-b762-dadb6855656c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:45.771904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5fddcacf-ae69-4098-bf60-2e5f4010127f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913a24b8cde54148aded4d0150b70f75', 'request_id': 'req-5fddcacf-ae69-4098-bf60-2e5f4010127f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:45.781788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5fddcacf-ae69-4098-bf60-2e5f4010127f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913a24b8cde54148aded4d0150b70f75', 'request_id': 'req-5fddcacf-ae69-4098-bf60-2e5f4010127f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:45.795185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5fddcacf-ae69-4098-bf60-2e5f4010127f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913a24b8cde54148aded4d0150b70f75', 'request_id': 'req-5fddcacf-ae69-4098-bf60-2e5f4010127f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:45.808419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:45.815975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5fddcacf-ae69-4098-bf60-2e5f4010127f admin admin] The token cache is being invalidate because project 913a24b8cde54148aded4d0150b70f75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:45.816538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5fddcacf-ae69-4098-bf60-2e5f4010127f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:45.817871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5fddcacf-ae69-4098-bf60-2e5f4010127f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:45.822049 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3465/7010] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:45 2022] DELETE /identity/v3/projects/913a24b8cde54148aded4d0150b70f75 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:45.829059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edb76d-a9c4-44be-96b3-a1576beff74e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:45.829342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edb76d-a9c4-44be-96b3-a1576beff74e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:45.829593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edb76d-a9c4-44be-96b3-a1576beff74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:45.859526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-03edb76d-a9c4-44be-96b3-a1576beff74e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:45.861037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-03edb76d-a9c4-44be-96b3-a1576beff74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:45.869986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ee55f-e827-4e53-b3b1-735b7b4ba637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:45.871944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ee55f-e827-4e53-b3b1-735b7b4ba637 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:45.872602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ee55f-e827-4e53-b3b1-735b7b4ba637 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:45.873232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ee55f-e827-4e53-b3b1-735b7b4ba637 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:45.874072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ee55f-e827-4e53-b3b1-735b7b4ba637 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:45.877664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ee55f-e827-4e53-b3b1-735b7b4ba637 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:45.897680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:45.944797 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3546/7011] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:45 2022] POST /identity/v3/projects => generated 379 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:45.948748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ee55f-e827-4e53-b3b1-735b7b4ba637 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:45.953230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-025ee55f-e827-4e53-b3b1-735b7b4ba637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:45.990272 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3466/7012] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:45.990272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6356e-5796-4795-af0e-cd6f50041343 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:45.990272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6356e-5796-4795-af0e-cd6f50041343 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:46.004899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6356e-5796-4795-af0e-cd6f50041343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:46.056608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9ea6356e-5796-4795-af0e-cd6f50041343 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:46.057818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9ea6356e-5796-4795-af0e-cd6f50041343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:46.112125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ee55f-e827-4e53-b3b1-735b7b4ba637 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:46.127433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a62b2-e565-4421-9e9d-994eab8eaa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:46.128876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a62b2-e565-4421-9e9d-994eab8eaa95 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:46.129203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a62b2-e565-4421-9e9d-994eab8eaa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:46.129449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a62b2-e565-4421-9e9d-994eab8eaa95 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:46.130029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a62b2-e565-4421-9e9d-994eab8eaa95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:46.137149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a62b2-e565-4421-9e9d-994eab8eaa95 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:46.162762 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3467/7013] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:46.163810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6356e-5796-4795-af0e-cd6f50041343 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:46.165880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9ea6356e-5796-4795-af0e-cd6f50041343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:46.189581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6356e-5796-4795-af0e-cd6f50041343 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:46.191418 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3547/7014] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:45 2022] POST /identity/v3/users => generated 296 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:46.192499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a62b2-e565-4421-9e9d-994eab8eaa95 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:46.212708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a62b2-e565-4421-9e9d-994eab8eaa95 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:46.213661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:46.214580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:46.215442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:46.215882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] PATH_INFO: `/v3/users/8cb96a00e1bb4d2ba4b38687af4f2020` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:46.217988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb96a00e1bb4d2ba4b38687af4f2020)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:46.224311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:46.228641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fab16-af91-4e7e-a75f-afe650634748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:46.229382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fab16-af91-4e7e-a75f-afe650634748 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:46.229722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fab16-af91-4e7e-a75f-afe650634748 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:46.229997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fab16-af91-4e7e-a75f-afe650634748 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:46.230650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fab16-af91-4e7e-a75f-afe650634748 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:22:46.231316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fab16-af91-4e7e-a75f-afe650634748 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:46.235096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fab16-af91-4e7e-a75f-afe650634748 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:46.264461 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3548/7015] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:22:46.273907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fab16-af91-4e7e-a75f-afe650634748 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:46.294817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fab16-af91-4e7e-a75f-afe650634748 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:46.306198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd246c1f-3a6a-43ae-82e1-c96f2313875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:46.306829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd246c1f-3a6a-43ae-82e1-c96f2313875f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:46.307671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd246c1f-3a6a-43ae-82e1-c96f2313875f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:46.308177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd246c1f-3a6a-43ae-82e1-c96f2313875f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:46.309390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd246c1f-3a6a-43ae-82e1-c96f2313875f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:46.311568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd246c1f-3a6a-43ae-82e1-c96f2313875f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:46.332015 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3549/7016] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:46.335182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd246c1f-3a6a-43ae-82e1-c96f2313875f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:46.351950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb96a00e1bb4d2ba4b38687af4f2020'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:46.366094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd246c1f-3a6a-43ae-82e1-c96f2313875f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:46.370598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb96a00e1bb4d2ba4b38687af4f2020'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:46.390439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd95fee-1fa1-4d05-a308-412b959c64d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:46.391942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd95fee-1fa1-4d05-a308-412b959c64d0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:46.396044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd95fee-1fa1-4d05-a308-412b959c64d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:46.396911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd95fee-1fa1-4d05-a308-412b959c64d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:46.397649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd95fee-1fa1-4d05-a308-412b959c64d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:46.399888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd95fee-1fa1-4d05-a308-412b959c64d0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:46.405156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb96a00e1bb4d2ba4b38687af4f2020'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:46.413809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb96a00e1bb4d2ba4b38687af4f2020'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:46.416646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb96a00e1bb4d2ba4b38687af4f2020'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:46.449167 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3468/7017] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:46 2022] DELETE /identity/v3/users/8cb96a00e1bb4d2ba4b38687af4f2020 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:46.450336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:46.453754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:46.463637 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3550/7018] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:46 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:46.465169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abb26e-0986-4500-85d3-9066a370f084 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:46.465467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abb26e-0986-4500-85d3-9066a370f084 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:46.465719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abb26e-0986-4500-85d3-9066a370f084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:46.497646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:46.505856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c5abb26e-0986-4500-85d3-9066a370f084 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:46.507106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c5abb26e-0986-4500-85d3-9066a370f084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:46.638063 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3551/7019] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:46.638732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abb26e-0986-4500-85d3-9066a370f084 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:46.642597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c5abb26e-0986-4500-85d3-9066a370f084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:46.668970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52951196-8e1a-4b44-a62b-4c841b744683 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:46.671271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abb26e-0986-4500-85d3-9066a370f084 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:46.678516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdcb3b-f9c3-43cb-ad0a-f6996d35a9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:46.679071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdcb3b-f9c3-43cb-ad0a-f6996d35a9ba admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:46.679424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdcb3b-f9c3-43cb-ad0a-f6996d35a9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:46.679600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdcb3b-f9c3-43cb-ad0a-f6996d35a9ba admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:46.680242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdcb3b-f9c3-43cb-ad0a-f6996d35a9ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:46.682290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdcb3b-f9c3-43cb-ad0a-f6996d35a9ba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:46.696126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d902d-6ea5-400e-b9f9-1cf69abd6653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:46.697068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d902d-6ea5-400e-b9f9-1cf69abd6653 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:46.697447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d902d-6ea5-400e-b9f9-1cf69abd6653 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:46.697819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d902d-6ea5-400e-b9f9-1cf69abd6653 admin admin] PATH_INFO: `/v3/projects/9308366863164a3fb83124eacba73b87/users/e736d061cd08493ea325f9bfa29605a3/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:46.738354 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3552/7020] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:46 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:46.740290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdcb3b-f9c3-43cb-ad0a-f6996d35a9ba admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:46.742591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d902d-6ea5-400e-b9f9-1cf69abd6653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9308366863164a3fb83124eacba73b87, user_id=e736d061cd08493ea325f9bfa29605a3, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:46.743072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9dbdcb3b-f9c3-43cb-ad0a-f6996d35a9ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:46.757006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d902d-6ea5-400e-b9f9-1cf69abd6653 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:46.775248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:46.802298 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3469/7021] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:46 2022] PUT /identity/v3/projects/9308366863164a3fb83124eacba73b87/users/e736d061cd08493ea325f9bfa29605a3/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:46.802298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d902d-6ea5-400e-b9f9-1cf69abd6653 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:46.865852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdcb3b-f9c3-43cb-ad0a-f6996d35a9ba admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:46.958737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d902d-6ea5-400e-b9f9-1cf69abd6653 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:46.973099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4760a8f-7833-454c-a229-f73ad6d2785a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:46.976591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760a8f-7833-454c-a229-f73ad6d2785a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:46.977136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760a8f-7833-454c-a229-f73ad6d2785a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:46.977609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760a8f-7833-454c-a229-f73ad6d2785a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:46.978434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4760a8f-7833-454c-a229-f73ad6d2785a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:46.983542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4760a8f-7833-454c-a229-f73ad6d2785a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:47.009943 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3470/7022] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:47.011157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4760a8f-7833-454c-a229-f73ad6d2785a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:47.030954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4760a8f-7833-454c-a229-f73ad6d2785a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:47.032306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90afff-12d5-47b7-9768-406974b9a3f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:47.033509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90afff-12d5-47b7-9768-406974b9a3f1 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:47.034015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90afff-12d5-47b7-9768-406974b9a3f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:47.034436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90afff-12d5-47b7-9768-406974b9a3f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:47.046850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea7638-0bc0-4e5e-b64d-c93aba18410a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:47.052043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea7638-0bc0-4e5e-b64d-c93aba18410a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:47.052546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea7638-0bc0-4e5e-b64d-c93aba18410a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:47.052962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea7638-0bc0-4e5e-b64d-c93aba18410a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:47.053763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea7638-0bc0-4e5e-b64d-c93aba18410a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:47.060675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea7638-0bc0-4e5e-b64d-c93aba18410a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:47.074502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90afff-12d5-47b7-9768-406974b9a3f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:47.076495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90afff-12d5-47b7-9768-406974b9a3f1 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:47.082856 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3471/7023] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:47.086641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea7638-0bc0-4e5e-b64d-c93aba18410a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:47.095427 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3553/7024] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:47.097320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90afff-12d5-47b7-9768-406974b9a3f1 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:47.114797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea7638-0bc0-4e5e-b64d-c93aba18410a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:47.122629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90afff-12d5-47b7-9768-406974b9a3f1 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:47.130007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b43549-6ae1-4d31-bd81-1fb21c1dee3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:47.132039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b43549-6ae1-4d31-bd81-1fb21c1dee3d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:47.132039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b43549-6ae1-4d31-bd81-1fb21c1dee3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:47.132593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b43549-6ae1-4d31-bd81-1fb21c1dee3d admin admin] PATH_INFO: `/v3/projects/9308366863164a3fb83124eacba73b87/users/e736d061cd08493ea325f9bfa29605a3/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:47.135770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b43549-6ae1-4d31-bd81-1fb21c1dee3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9308366863164a3fb83124eacba73b87, user_id=e736d061cd08493ea325f9bfa29605a3, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:47.136834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8cb6e-f60b-4f0d-904a-61b5118f89d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:47.138284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8cb6e-f60b-4f0d-904a-61b5118f89d5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:47.140656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8cb6e-f60b-4f0d-904a-61b5118f89d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:47.141739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b43549-6ae1-4d31-bd81-1fb21c1dee3d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:47.142504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8cb6e-f60b-4f0d-904a-61b5118f89d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:47.147761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8cb6e-f60b-4f0d-904a-61b5118f89d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:47.157261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8cb6e-f60b-4f0d-904a-61b5118f89d5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:47.160620 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3554/7025] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:47 2022] PUT /identity/v3/projects/9308366863164a3fb83124eacba73b87/users/e736d061cd08493ea325f9bfa29605a3/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:47.200196 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3472/7026] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:47.203695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b43549-6ae1-4d31-bd81-1fb21c1dee3d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:47.376404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b43549-6ae1-4d31-bd81-1fb21c1dee3d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:47.388428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9488307-8f20-4b12-ba98-6a43efcab477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:47.389438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9488307-8f20-4b12-ba98-6a43efcab477 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:47.390329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9488307-8f20-4b12-ba98-6a43efcab477 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:47.390793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9488307-8f20-4b12-ba98-6a43efcab477 admin admin] PATH_INFO: `/v3/projects/442e84781df044bc8039c8b08b14450f/users/c682ee55fb054fde811f6e123677f962/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:47.429771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9488307-8f20-4b12-ba98-6a43efcab477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=442e84781df044bc8039c8b08b14450f, user_id=c682ee55fb054fde811f6e123677f962, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:47.434120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9488307-8f20-4b12-ba98-6a43efcab477 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:47.475308 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3555/7027] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:47 2022] PUT /identity/v3/projects/442e84781df044bc8039c8b08b14450f/users/c682ee55fb054fde811f6e123677f962/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:47.486142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b7233-5046-422a-85fd-dc68a5dd670c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:47.486425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b7233-5046-422a-85fd-dc68a5dd670c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:47.486695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b7233-5046-422a-85fd-dc68a5dd670c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:47.557631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-261b7233-5046-422a-85fd-dc68a5dd670c None None] MFA Rules not processed for user `c682ee55fb054fde811f6e123677f962`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:47.558988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-261b7233-5046-422a-85fd-dc68a5dd670c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:47.731079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:47.736952 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3473/7028] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:47.760884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9488307-8f20-4b12-ba98-6a43efcab477 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:47.763723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e9488307-8f20-4b12-ba98-6a43efcab477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:47.951044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:47.963056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9488307-8f20-4b12-ba98-6a43efcab477 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:47.972616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7b0a1-76ff-4e5e-902d-b32dc884cf83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:47.981031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7b0a1-76ff-4e5e-902d-b32dc884cf83 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:47.981560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7b0a1-76ff-4e5e-902d-b32dc884cf83 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:47.982970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7b0a1-76ff-4e5e-902d-b32dc884cf83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:48.012033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b7233-5046-422a-85fd-dc68a5dd670c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:48.013516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-261b7233-5046-422a-85fd-dc68a5dd670c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:48.028141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7b0a1-76ff-4e5e-902d-b32dc884cf83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:48.029851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7b0a1-76ff-4e5e-902d-b32dc884cf83 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:48.051190 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3556/7029] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:48.191089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b7233-5046-422a-85fd-dc68a5dd670c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:48.207759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:48.209040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:48.209592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:48.210329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] PATH_INFO: `/v3/projects/09830dddcf0a4c3bb415fd0491a26dd9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:48.211844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] RBAC: Authorizing `identity:delete_project(project_id=09830dddcf0a4c3bb415fd0491a26dd9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:48.214078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:48.292561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7b0a1-76ff-4e5e-902d-b32dc884cf83 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:48.338825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7b0a1-76ff-4e5e-902d-b32dc884cf83 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:48.402826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09830dddcf0a4c3bb415fd0491a26dd9', 'request_id': 'req-de6b88d1-7a15-4450-85c5-89c96548673d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.424913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09830dddcf0a4c3bb415fd0491a26dd9', 'request_id': 'req-de6b88d1-7a15-4450-85c5-89c96548673d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.450369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09830dddcf0a4c3bb415fd0491a26dd9', 'request_id': 'req-de6b88d1-7a15-4450-85c5-89c96548673d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.476332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] The token cache is being invalidate because project 09830dddcf0a4c3bb415fd0491a26dd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:48.476332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.484067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.490393 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3474/7030] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:48 2022] DELETE /identity/v3/projects/09830dddcf0a4c3bb415fd0491a26dd9 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:48.494193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:48.505181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:48.533292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c5659-61af-48ce-a26f-349a24dfb111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:48.534524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c5659-61af-48ce-a26f-349a24dfb111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:48.534815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c5659-61af-48ce-a26f-349a24dfb111 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:48.535149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c5659-61af-48ce-a26f-349a24dfb111 admin admin] PATH_INFO: `/v3/users/c682ee55fb054fde811f6e123677f962` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:48.540664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c5659-61af-48ce-a26f-349a24dfb111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c682ee55fb054fde811f6e123677f962)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:48.545561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c5659-61af-48ce-a26f-349a24dfb111 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:48.643124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b88d1-7a15-4450-85c5-89c96548673d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:48.649028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:48.649815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:48.650091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:48.650641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] PATH_INFO: `/v3/users/53410cc3fb44400e8b6fdf146494f5cf` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:48.651516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] RBAC: Authorizing `identity:delete_user(user_id=53410cc3fb44400e8b6fdf146494f5cf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:48.678918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:48.697236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-406c5659-61af-48ce-a26f-349a24dfb111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c682ee55fb054fde811f6e123677f962'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.712938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-406c5659-61af-48ce-a26f-349a24dfb111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c682ee55fb054fde811f6e123677f962'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.728144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-406c5659-61af-48ce-a26f-349a24dfb111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c682ee55fb054fde811f6e123677f962'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.758590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-406c5659-61af-48ce-a26f-349a24dfb111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c682ee55fb054fde811f6e123677f962'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.759990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-406c5659-61af-48ce-a26f-349a24dfb111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c682ee55fb054fde811f6e123677f962'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.768710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:48.783804 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3557/7031] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:48 2022] DELETE /identity/v3/users/c682ee55fb054fde811f6e123677f962 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:48.790226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0a308-d57a-4c74-b66c-e8d762063ae9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:48.790510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0a308-d57a-4c74-b66c-e8d762063ae9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:48.790781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0a308-d57a-4c74-b66c-e8d762063ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:48.811830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53410cc3fb44400e8b6fdf146494f5cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.826223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53410cc3fb44400e8b6fdf146494f5cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.841762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7be0a308-d57a-4c74-b66c-e8d762063ae9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:48.843146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7be0a308-d57a-4c74-b66c-e8d762063ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:48.869048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53410cc3fb44400e8b6fdf146494f5cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.869904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53410cc3fb44400e8b6fdf146494f5cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.871799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53410cc3fb44400e8b6fdf146494f5cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:48.905602 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3475/7032] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:22:48 2022] DELETE /identity/v3/users/53410cc3fb44400e8b6fdf146494f5cf => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:48.996064 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3558/7033] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:49.040588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:49.042266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:49.178007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa077b-a960-4e2f-9627-4f485a55aece admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:49.185486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efde29-2c14-45d6-8c3a-508b61244ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:49.186398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efde29-2c14-45d6-8c3a-508b61244ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:49.186821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efde29-2c14-45d6-8c3a-508b61244ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:49.187240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efde29-2c14-45d6-8c3a-508b61244ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:49.317820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efde29-2c14-45d6-8c3a-508b61244ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:49.320067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efde29-2c14-45d6-8c3a-508b61244ab7 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:49.337987 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3476/7034] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:49.678520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0a308-d57a-4c74-b66c-e8d762063ae9 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:49.685265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7be0a308-d57a-4c74-b66c-e8d762063ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:49.711539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0a308-d57a-4c74-b66c-e8d762063ae9 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:49.729859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb943093-2f30-455c-bb75-e0f2e26ae23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:49.731018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb943093-2f30-455c-bb75-e0f2e26ae23e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:49.731232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb943093-2f30-455c-bb75-e0f2e26ae23e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:49.731614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb943093-2f30-455c-bb75-e0f2e26ae23e admin admin] PATH_INFO: `/v3/projects/b82a71abe5724eeab4f25037996c9b0a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:49.733791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb943093-2f30-455c-bb75-e0f2e26ae23e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b82a71abe5724eeab4f25037996c9b0a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:49.750186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb943093-2f30-455c-bb75-e0f2e26ae23e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:49.762141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:49.866137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fb943093-2f30-455c-bb75-e0f2e26ae23e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b82a71abe5724eeab4f25037996c9b0a', 'request_id': 'req-fb943093-2f30-455c-bb75-e0f2e26ae23e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:49.887301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fb943093-2f30-455c-bb75-e0f2e26ae23e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b82a71abe5724eeab4f25037996c9b0a', 'request_id': 'req-fb943093-2f30-455c-bb75-e0f2e26ae23e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:49.894194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fb943093-2f30-455c-bb75-e0f2e26ae23e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b82a71abe5724eeab4f25037996c9b0a', 'request_id': 'req-fb943093-2f30-455c-bb75-e0f2e26ae23e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:49.943831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fb943093-2f30-455c-bb75-e0f2e26ae23e admin admin] The token cache is being invalidate because project b82a71abe5724eeab4f25037996c9b0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:49.944397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fb943093-2f30-455c-bb75-e0f2e26ae23e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:49.945071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fb943093-2f30-455c-bb75-e0f2e26ae23e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:49.950173 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3559/7035] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:49 2022] DELETE /identity/v3/projects/b82a71abe5724eeab4f25037996c9b0a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:49.952333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb2a92-3f93-4390-9821-23bd1a6a028f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:49.952333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb2a92-3f93-4390-9821-23bd1a6a028f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:49.952558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb2a92-3f93-4390-9821-23bd1a6a028f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:49.988984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c8bb2a92-3f93-4390-9821-23bd1a6a028f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:49.990053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c8bb2a92-3f93-4390-9821-23bd1a6a028f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:50.107872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:50.127859 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3477/7036] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:50.131121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd120e9b-3ed1-4275-b405-a9627a058d23 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:50.131401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd120e9b-3ed1-4275-b405-a9627a058d23 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:50.131833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd120e9b-3ed1-4275-b405-a9627a058d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:50.179330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dd120e9b-3ed1-4275-b405-a9627a058d23 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:50.183368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dd120e9b-3ed1-4275-b405-a9627a058d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:50.197987 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3560/7037] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:50.202263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb2a92-3f93-4390-9821-23bd1a6a028f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:50.226322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb2a92-3f93-4390-9821-23bd1a6a028f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:50.250403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831d77f-288f-41d8-b99e-1bc7aa978aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:50.251038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831d77f-288f-41d8-b99e-1bc7aa978aea admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:50.251311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831d77f-288f-41d8-b99e-1bc7aa978aea admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:50.251580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831d77f-288f-41d8-b99e-1bc7aa978aea admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:50.252235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831d77f-288f-41d8-b99e-1bc7aa978aea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:22:50.252620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831d77f-288f-41d8-b99e-1bc7aa978aea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:50.254414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831d77f-288f-41d8-b99e-1bc7aa978aea admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:50.300762 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3478/7038] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:22:50.305852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd120e9b-3ed1-4275-b405-a9627a058d23 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:50.318832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd120e9b-3ed1-4275-b405-a9627a058d23 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:50.325552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8f345-a850-489c-92a3-2a1679882a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:50.326376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8f345-a850-489c-92a3-2a1679882a06 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:50.331199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8f345-a850-489c-92a3-2a1679882a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:50.331631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8f345-a850-489c-92a3-2a1679882a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:50.332553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8f345-a850-489c-92a3-2a1679882a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:50.334939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8f345-a850-489c-92a3-2a1679882a06 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:50.379757 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3561/7039] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:50 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:50.381029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831d77f-288f-41d8-b99e-1bc7aa978aea admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:50.401741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831d77f-288f-41d8-b99e-1bc7aa978aea admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:50.490106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b3e32-4749-42b5-b360-e6dc9e5fc602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:50.490864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b3e32-4749-42b5-b360-e6dc9e5fc602 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:50.491245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b3e32-4749-42b5-b360-e6dc9e5fc602 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:50.491565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b3e32-4749-42b5-b360-e6dc9e5fc602 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:50.492265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b3e32-4749-42b5-b360-e6dc9e5fc602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:50.494885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b3e32-4749-42b5-b360-e6dc9e5fc602 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:50.528224 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3479/7040] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:50 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:50.531810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8f345-a850-489c-92a3-2a1679882a06 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:50.546900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8f345-a850-489c-92a3-2a1679882a06 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:50.557457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a40da6-8e0b-4e42-a25c-a628e0f4ebe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:50.558426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a40da6-8e0b-4e42-a25c-a628e0f4ebe3 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:50.560822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a40da6-8e0b-4e42-a25c-a628e0f4ebe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:50.562890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a40da6-8e0b-4e42-a25c-a628e0f4ebe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:50.564176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a40da6-8e0b-4e42-a25c-a628e0f4ebe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:50.566428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a40da6-8e0b-4e42-a25c-a628e0f4ebe3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:50.587096 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3562/7041] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:50.591334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b3e32-4749-42b5-b360-e6dc9e5fc602 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:50.610397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b3e32-4749-42b5-b360-e6dc9e5fc602 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:50.615765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0584679-33a7-4fe9-918f-6129ae02c7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:50.616418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0584679-33a7-4fe9-918f-6129ae02c7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:50.616696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0584679-33a7-4fe9-918f-6129ae02c7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:50.616981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0584679-33a7-4fe9-918f-6129ae02c7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:50.617903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0584679-33a7-4fe9-918f-6129ae02c7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:50.619535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0584679-33a7-4fe9-918f-6129ae02c7e4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:50.640449 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3480/7042] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:50.646559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a40da6-8e0b-4e42-a25c-a628e0f4ebe3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:50.664196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a40da6-8e0b-4e42-a25c-a628e0f4ebe3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:50.671594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921e292-f2ee-4330-8b75-9128f036d571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:50.672331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921e292-f2ee-4330-8b75-9128f036d571 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:50.672638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921e292-f2ee-4330-8b75-9128f036d571 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:50.672948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921e292-f2ee-4330-8b75-9128f036d571 admin admin] PATH_INFO: `/v3/projects/b10ddde6a64f4c809559c9682a131753/users/9354193110e841e5a383fb521ba80b9b/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:50.695125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921e292-f2ee-4330-8b75-9128f036d571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10ddde6a64f4c809559c9682a131753, user_id=9354193110e841e5a383fb521ba80b9b, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:50.698403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921e292-f2ee-4330-8b75-9128f036d571 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:50.736909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d45955-3e3e-4eb3-98f7-de40fa1435e1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:50.737275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d45955-3e3e-4eb3-98f7-de40fa1435e1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:50.737606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d45955-3e3e-4eb3-98f7-de40fa1435e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:50.741543 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3563/7043] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:50 2022] PUT /identity/v3/projects/b10ddde6a64f4c809559c9682a131753/users/9354193110e841e5a383fb521ba80b9b/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:50.790412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a3d45955-3e3e-4eb3-98f7-de40fa1435e1 None None] MFA Rules not processed for user `9354193110e841e5a383fb521ba80b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:50.791709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a3d45955-3e3e-4eb3-98f7-de40fa1435e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:50.914363 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3481/7044] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:50.974075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921e292-f2ee-4330-8b75-9128f036d571 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:50.977708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7921e292-f2ee-4330-8b75-9128f036d571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:51.076929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921e292-f2ee-4330-8b75-9128f036d571 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:51.084600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ebe9b-8473-4670-9a33-2981e5205945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:51.085960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ebe9b-8473-4670-9a33-2981e5205945 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:51.086644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ebe9b-8473-4670-9a33-2981e5205945 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:51.088107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ebe9b-8473-4670-9a33-2981e5205945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:51.111409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ebe9b-8473-4670-9a33-2981e5205945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:51.117380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ebe9b-8473-4670-9a33-2981e5205945 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:51.149705 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3564/7045] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:22:51.212460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d45955-3e3e-4eb3-98f7-de40fa1435e1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:51.213869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a3d45955-3e3e-4eb3-98f7-de40fa1435e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:51.258053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07238a21-3f16-4cf5-9c0b-7f28d2fd9744 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:51.258404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07238a21-3f16-4cf5-9c0b-7f28d2fd9744 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:51.258585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07238a21-3f16-4cf5-9c0b-7f28d2fd9744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:51.319446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-07238a21-3f16-4cf5-9c0b-7f28d2fd9744 None None] MFA Rules not processed for user `ae2c339c41354a159fc8881b56ac45ec`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:51.320455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:51.320702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-07238a21-3f16-4cf5-9c0b-7f28d2fd9744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:51.328168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d45955-3e3e-4eb3-98f7-de40fa1435e1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:51.338332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c703c-ea28-49cc-b4a6-06ddaab304d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:51.339500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c703c-ea28-49cc-b4a6-06ddaab304d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:51.340016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c703c-ea28-49cc-b4a6-06ddaab304d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:51.340585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c703c-ea28-49cc-b4a6-06ddaab304d6 admin admin] PATH_INFO: `/v3/projects/442e84781df044bc8039c8b08b14450f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:51.342128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c703c-ea28-49cc-b4a6-06ddaab304d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=442e84781df044bc8039c8b08b14450f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:51.344299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c703c-ea28-49cc-b4a6-06ddaab304d6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:51.367210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:51.421650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-773c703c-ea28-49cc-b4a6-06ddaab304d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '442e84781df044bc8039c8b08b14450f', 'request_id': 'req-773c703c-ea28-49cc-b4a6-06ddaab304d6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:51.430932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-773c703c-ea28-49cc-b4a6-06ddaab304d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '442e84781df044bc8039c8b08b14450f', 'request_id': 'req-773c703c-ea28-49cc-b4a6-06ddaab304d6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:51.446445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-773c703c-ea28-49cc-b4a6-06ddaab304d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '442e84781df044bc8039c8b08b14450f', 'request_id': 'req-773c703c-ea28-49cc-b4a6-06ddaab304d6'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:51.468870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-773c703c-ea28-49cc-b4a6-06ddaab304d6 admin admin] The token cache is being invalidate because project 442e84781df044bc8039c8b08b14450f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:22:51.469269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-773c703c-ea28-49cc-b4a6-06ddaab304d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:51.470027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-773c703c-ea28-49cc-b4a6-06ddaab304d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:22:51.475146 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3482/7046] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:22:51 2022] DELETE /identity/v3/projects/442e84781df044bc8039c8b08b14450f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:51.490101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6480cbf-c782-4709-a48e-8349d0f06f35 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:51.490485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6480cbf-c782-4709-a48e-8349d0f06f35 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:51.490691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6480cbf-c782-4709-a48e-8349d0f06f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:51.498251 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3565/7047] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:51.499212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07238a21-3f16-4cf5-9c0b-7f28d2fd9744 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:51.507110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-07238a21-3f16-4cf5-9c0b-7f28d2fd9744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:51.534350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07238a21-3f16-4cf5-9c0b-7f28d2fd9744 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:51.534681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a6480cbf-c782-4709-a48e-8349d0f06f35 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:51.536584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a6480cbf-c782-4709-a48e-8349d0f06f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:51.653408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509002a2-958e-4e87-a425-9411e9b0f602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:51.654042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509002a2-958e-4e87-a425-9411e9b0f602 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:51.654316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509002a2-958e-4e87-a425-9411e9b0f602 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:51.654586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509002a2-958e-4e87-a425-9411e9b0f602 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:51.655215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509002a2-958e-4e87-a425-9411e9b0f602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:51.657189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509002a2-958e-4e87-a425-9411e9b0f602 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:51.669947 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3483/7048] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:51.673394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99c109-0dd4-4e0e-a574-01e4527a9c27 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:51.673680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99c109-0dd4-4e0e-a574-01e4527a9c27 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:51.673953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99c109-0dd4-4e0e-a574-01e4527a9c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:51.720905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ac99c109-0dd4-4e0e-a574-01e4527a9c27 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:51.722167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ac99c109-0dd4-4e0e-a574-01e4527a9c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:51.731582 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3566/7049] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:51 2022] POST /identity/v3/projects => generated 389 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:51.736849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509002a2-958e-4e87-a425-9411e9b0f602 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:51.740725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-509002a2-958e-4e87-a425-9411e9b0f602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:51.861703 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3484/7050] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:51.863961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99c109-0dd4-4e0e-a574-01e4527a9c27 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:51.894669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99c109-0dd4-4e0e-a574-01e4527a9c27 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:51.897855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509002a2-958e-4e87-a425-9411e9b0f602 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:51.911969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872bce4-cf4c-496d-ad7c-f3c097c4f5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:51.912652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872bce4-cf4c-496d-ad7c-f3c097c4f5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:51.912933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872bce4-cf4c-496d-ad7c-f3c097c4f5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:51.913205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872bce4-cf4c-496d-ad7c-f3c097c4f5c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:51.916417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872bce4-cf4c-496d-ad7c-f3c097c4f5c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:22:51.916632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872bce4-cf4c-496d-ad7c-f3c097c4f5c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:51.920108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ed675-6c86-4a71-9b10-9f3b9f23a067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:51.921146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ed675-6c86-4a71-9b10-9f3b9f23a067 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:51.921433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872bce4-cf4c-496d-ad7c-f3c097c4f5c0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:51.921873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ed675-6c86-4a71-9b10-9f3b9f23a067 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:51.922338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ed675-6c86-4a71-9b10-9f3b9f23a067 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:51.923173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ed675-6c86-4a71-9b10-9f3b9f23a067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:51.932078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ed675-6c86-4a71-9b10-9f3b9f23a067 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:51.958751 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3485/7051] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:22:51.962605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872bce4-cf4c-496d-ad7c-f3c097c4f5c0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:52.005258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872bce4-cf4c-496d-ad7c-f3c097c4f5c0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:52.005505 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3567/7052] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:51 2022] POST /identity/v3/users => generated 301 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:52.007588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ed675-6c86-4a71-9b10-9f3b9f23a067 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:52.020183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45099e9-2b3b-43c3-82eb-f46e4a7fab9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:52.020992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45099e9-2b3b-43c3-82eb-f46e4a7fab9b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:52.021274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45099e9-2b3b-43c3-82eb-f46e4a7fab9b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:52.021545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45099e9-2b3b-43c3-82eb-f46e4a7fab9b admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:52.024764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45099e9-2b3b-43c3-82eb-f46e4a7fab9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:52.030050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45099e9-2b3b-43c3-82eb-f46e4a7fab9b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:52.036961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ed675-6c86-4a71-9b10-9f3b9f23a067 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:52.051643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03355888-34a1-4ec7-a35a-fd6402614f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:52.052264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03355888-34a1-4ec7-a35a-fd6402614f87 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:52.052613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03355888-34a1-4ec7-a35a-fd6402614f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:52.052886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03355888-34a1-4ec7-a35a-fd6402614f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:52.053538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03355888-34a1-4ec7-a35a-fd6402614f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:52.067369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03355888-34a1-4ec7-a35a-fd6402614f87 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:52.076099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:52.091302 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3568/7053] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:52.101877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03355888-34a1-4ec7-a35a-fd6402614f87 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:52.112349 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3486/7054] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:22:51 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:52.114559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45099e9-2b3b-43c3-82eb-f46e4a7fab9b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:52.131910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03355888-34a1-4ec7-a35a-fd6402614f87 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:52.212342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:52.359860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45099e9-2b3b-43c3-82eb-f46e4a7fab9b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:52.360513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40159c1-c399-4d79-aa0e-04c4a13e7165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:52.364167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40159c1-c399-4d79-aa0e-04c4a13e7165 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:52.364664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40159c1-c399-4d79-aa0e-04c4a13e7165 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:52.364940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40159c1-c399-4d79-aa0e-04c4a13e7165 admin admin] PATH_INFO: `/v3/projects/22d2200123db45adaa25c8e9676c9057/users/1e417d6da9c94de49b6cf49fe3a3fab0/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:52.373080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d440535-c2e2-4dad-8f45-e03eab705b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:52.375157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440535-c2e2-4dad-8f45-e03eab705b72 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:52.375443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440535-c2e2-4dad-8f45-e03eab705b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:52.375719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440535-c2e2-4dad-8f45-e03eab705b72 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:52.378098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d440535-c2e2-4dad-8f45-e03eab705b72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:52.386154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d440535-c2e2-4dad-8f45-e03eab705b72 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:52.419518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40159c1-c399-4d79-aa0e-04c4a13e7165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d2200123db45adaa25c8e9676c9057, user_id=1e417d6da9c94de49b6cf49fe3a3fab0, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:52.426665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40159c1-c399-4d79-aa0e-04c4a13e7165 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:52.441850 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3487/7055] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:22:52 2022] POST /identity/v3/users => generated 292 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:22:52.448155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d440535-c2e2-4dad-8f45-e03eab705b72 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:52.459352 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3569/7056] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:52 2022] PUT /identity/v3/projects/22d2200123db45adaa25c8e9676c9057/users/1e417d6da9c94de49b6cf49fe3a3fab0/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:52.463283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40159c1-c399-4d79-aa0e-04c4a13e7165 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:52.477095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d440535-c2e2-4dad-8f45-e03eab705b72 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:52.629662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265afa3-54cf-4e5c-8088-969e300bae1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:52.633559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265afa3-54cf-4e5c-8088-969e300bae1c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:52.635167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265afa3-54cf-4e5c-8088-969e300bae1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:52.635167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265afa3-54cf-4e5c-8088-969e300bae1c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:52.635167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265afa3-54cf-4e5c-8088-969e300bae1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:52.640730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265afa3-54cf-4e5c-8088-969e300bae1c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:52.653138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40159c1-c399-4d79-aa0e-04c4a13e7165 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:52.668781 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3488/7057] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:22:52.669006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eee426-1e19-4321-baee-750ac9d48a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:52.669006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eee426-1e19-4321-baee-750ac9d48a6a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:52.669006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eee426-1e19-4321-baee-750ac9d48a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:52.669006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eee426-1e19-4321-baee-750ac9d48a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:52.669297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eee426-1e19-4321-baee-750ac9d48a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:52.669297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eee426-1e19-4321-baee-750ac9d48a6a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:52.670289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265afa3-54cf-4e5c-8088-969e300bae1c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:52.703454 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3570/7058] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:52.704011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265afa3-54cf-4e5c-8088-969e300bae1c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:52.707529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eee426-1e19-4321-baee-750ac9d48a6a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:52.721917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aabfb6-cc7d-4cda-9562-73de2889fe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:52.721917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aabfb6-cc7d-4cda-9562-73de2889fe2d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:52.723882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aabfb6-cc7d-4cda-9562-73de2889fe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:52.724196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aabfb6-cc7d-4cda-9562-73de2889fe2d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:52.724916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aabfb6-cc7d-4cda-9562-73de2889fe2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:52.729741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aabfb6-cc7d-4cda-9562-73de2889fe2d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:52.738771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eee426-1e19-4321-baee-750ac9d48a6a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:52.752654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd2830-9f4f-4fd4-8103-2eb48ca87850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:52.753425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd2830-9f4f-4fd4-8103-2eb48ca87850 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:52.753851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd2830-9f4f-4fd4-8103-2eb48ca87850 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:52.754125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd2830-9f4f-4fd4-8103-2eb48ca87850 admin admin] PATH_INFO: `/v3/projects/22d2200123db45adaa25c8e9676c9057/users/1e417d6da9c94de49b6cf49fe3a3fab0/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:52.756510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd2830-9f4f-4fd4-8103-2eb48ca87850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d2200123db45adaa25c8e9676c9057, user_id=1e417d6da9c94de49b6cf49fe3a3fab0, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:52.757278 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3489/7059] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:22:52.759301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd2830-9f4f-4fd4-8103-2eb48ca87850 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:52.761500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aabfb6-cc7d-4cda-9562-73de2889fe2d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:52.781890 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3571/7060] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:52 2022] PUT /identity/v3/projects/22d2200123db45adaa25c8e9676c9057/users/1e417d6da9c94de49b6cf49fe3a3fab0/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:22:52.793965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aabfb6-cc7d-4cda-9562-73de2889fe2d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:52.906221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206673b2-1dd0-4390-adb1-703235fbbd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:52.907307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206673b2-1dd0-4390-adb1-703235fbbd83 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:52.907767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206673b2-1dd0-4390-adb1-703235fbbd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:52.908181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206673b2-1dd0-4390-adb1-703235fbbd83 admin admin] PATH_INFO: `/v3/projects/3dafa2f5aa284027af9b66c48c35bd32/users/47f61fd7b2384ba7ba84a5c1f8b913c4/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:52.927677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:22:52.936646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206673b2-1dd0-4390-adb1-703235fbbd83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dafa2f5aa284027af9b66c48c35bd32, user_id=47f61fd7b2384ba7ba84a5c1f8b913c4, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:52.939887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206673b2-1dd0-4390-adb1-703235fbbd83 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:52.982601 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3490/7061] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:22:52 2022] PUT /identity/v3/projects/3dafa2f5aa284027af9b66c48c35bd32/users/47f61fd7b2384ba7ba84a5c1f8b913c4/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:22:52.984827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e6148-3edc-442f-b13d-d7fdda51180e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:52.984827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e6148-3edc-442f-b13d-d7fdda51180e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:52.984827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e6148-3edc-442f-b13d-d7fdda51180e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:53.062071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9f7e6148-3edc-442f-b13d-d7fdda51180e None None] MFA Rules not processed for user `47f61fd7b2384ba7ba84a5c1f8b913c4`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:22:53.063702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9f7e6148-3edc-442f-b13d-d7fdda51180e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:53.229493 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3572/7062] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:22:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:22:53.323619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206673b2-1dd0-4390-adb1-703235fbbd83 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:22:53.326348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-206673b2-1dd0-4390-adb1-703235fbbd83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:22:53.474175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206673b2-1dd0-4390-adb1-703235fbbd83 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:22:53.490184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e07a3-0240-4ac0-9a11-befbf3948cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:22:53.490912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e07a3-0240-4ac0-9a11-befbf3948cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:22:53.491184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e07a3-0240-4ac0-9a11-befbf3948cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:22:53.491454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e07a3-0240-4ac0-9a11-befbf3948cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:22:53.531053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e07a3-0240-4ac0-9a11-befbf3948cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:22:53.532597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e07a3-0240-4ac0-9a11-befbf3948cc6 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:22:53.563500 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3491/7063] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:22:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:00.111260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67260a45-c6e2-4072-bd22-01205a5cd708 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:00.111790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67260a45-c6e2-4072-bd22-01205a5cd708 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:00.111957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67260a45-c6e2-4072-bd22-01205a5cd708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:00.197808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-67260a45-c6e2-4072-bd22-01205a5cd708 None None] MFA Rules not processed for user `e736d061cd08493ea325f9bfa29605a3`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:00.200433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-67260a45-c6e2-4072-bd22-01205a5cd708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:00.223671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:00.398541 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3573/7064] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:00.411970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e07a3-0240-4ac0-9a11-befbf3948cc6 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:00.413623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f63e07a3-0240-4ac0-9a11-befbf3948cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:00.631596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e07a3-0240-4ac0-9a11-befbf3948cc6 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:00.636108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973778f7-4e5f-47c3-81d6-aa0df8b2d13a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:00.636982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973778f7-4e5f-47c3-81d6-aa0df8b2d13a service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:00.637644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973778f7-4e5f-47c3-81d6-aa0df8b2d13a service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:00.638076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973778f7-4e5f-47c3-81d6-aa0df8b2d13a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:00.641549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-973778f7-4e5f-47c3-81d6-aa0df8b2d13a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:00.766779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973778f7-4e5f-47c3-81d6-aa0df8b2d13a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:00.768545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973778f7-4e5f-47c3-81d6-aa0df8b2d13a service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:00.782337 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3492/7065] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:06.607263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb93cba-599c-4a30-86d2-b96dcd2c9a7c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:06.607263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb93cba-599c-4a30-86d2-b96dcd2c9a7c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:06.607263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb93cba-599c-4a30-86d2-b96dcd2c9a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:06.700305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-feb93cba-599c-4a30-86d2-b96dcd2c9a7c None None] MFA Rules not processed for user `1e417d6da9c94de49b6cf49fe3a3fab0`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:06.701587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-feb93cba-599c-4a30-86d2-b96dcd2c9a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:06.901495 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3574/7066] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:06 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:06.912726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973778f7-4e5f-47c3-81d6-aa0df8b2d13a service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:06.916158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-973778f7-4e5f-47c3-81d6-aa0df8b2d13a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:06.938110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973778f7-4e5f-47c3-81d6-aa0df8b2d13a service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:06.965673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468c5a1-802c-4dcf-ad00-c02a740799b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:06.975993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468c5a1-802c-4dcf-ad00-c02a740799b3 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:06.978450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468c5a1-802c-4dcf-ad00-c02a740799b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:06.978969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468c5a1-802c-4dcf-ad00-c02a740799b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:06.993409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5468c5a1-802c-4dcf-ad00-c02a740799b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:07.172265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:07.293275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468c5a1-802c-4dcf-ad00-c02a740799b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:07.294802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468c5a1-802c-4dcf-ad00-c02a740799b3 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:07.371728 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3493/7067] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:12.047793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb93cba-599c-4a30-86d2-b96dcd2c9a7c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:12.049290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-feb93cba-599c-4a30-86d2-b96dcd2c9a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:12.257030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb93cba-599c-4a30-86d2-b96dcd2c9a7c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:12.281411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c027d01-41a0-458d-b4d1-64774ddd2c24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:12.282560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c027d01-41a0-458d-b4d1-64774ddd2c24 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:12.283136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c027d01-41a0-458d-b4d1-64774ddd2c24 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:12.283902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c027d01-41a0-458d-b4d1-64774ddd2c24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:12.295445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4c027d01-41a0-458d-b4d1-64774ddd2c24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:12.332070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c027d01-41a0-458d-b4d1-64774ddd2c24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:12.336285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c027d01-41a0-458d-b4d1-64774ddd2c24 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:12.380705 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3575/7068] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:16.936725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468c5a1-802c-4dcf-ad00-c02a740799b3 service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:16.937962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5468c5a1-802c-4dcf-ad00-c02a740799b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:16.976343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468c5a1-802c-4dcf-ad00-c02a740799b3 service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:17.002568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63edcaea-7fb2-4caf-a82b-c150f25fc894 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:17.003468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edcaea-7fb2-4caf-a82b-c150f25fc894 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:17.003784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edcaea-7fb2-4caf-a82b-c150f25fc894 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:17.004060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edcaea-7fb2-4caf-a82b-c150f25fc894 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:17.014093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-63edcaea-7fb2-4caf-a82b-c150f25fc894 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:17.044404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63edcaea-7fb2-4caf-a82b-c150f25fc894 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:17.046250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63edcaea-7fb2-4caf-a82b-c150f25fc894 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:17.076997 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3494/7069] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:20.641368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c027d01-41a0-458d-b4d1-64774ddd2c24 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:20.643534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4c027d01-41a0-458d-b4d1-64774ddd2c24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:20.651967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:20.762302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c027d01-41a0-458d-b4d1-64774ddd2c24 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:20.767641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:20.768829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:20.769322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:20.769817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] PATH_INFO: `/v3/users/9354193110e841e5a383fb521ba80b9b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:20.771429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] RBAC: Authorizing `identity:delete_user(user_id=9354193110e841e5a383fb521ba80b9b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:20.773725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:20.864548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9354193110e841e5a383fb521ba80b9b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:20.874700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9354193110e841e5a383fb521ba80b9b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:20.883734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9354193110e841e5a383fb521ba80b9b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:20.899347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9354193110e841e5a383fb521ba80b9b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:20.900299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9354193110e841e5a383fb521ba80b9b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:20.910740 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3576/7070] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:23:20 2022] DELETE /identity/v3/users/9354193110e841e5a383fb521ba80b9b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:20.918452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbd3ff-3450-4cb9-b733-ff26f8968caa None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:20.919038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbd3ff-3450-4cb9-b733-ff26f8968caa None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:20.919138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbd3ff-3450-4cb9-b733-ff26f8968caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:20.947434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-31bbd3ff-3450-4cb9-b733-ff26f8968caa None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:20.949476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-31bbd3ff-3450-4cb9-b733-ff26f8968caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:21.064639 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3495/7071] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:23:21.092665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:21.094184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:21.170820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8367e-52f5-43d2-9b1c-0a293cd9ccab admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:21.176600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbf302-e136-465a-8b70-49d31013e3a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:21.177434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf302-e136-465a-8b70-49d31013e3a1 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:21.177897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf302-e136-465a-8b70-49d31013e3a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:21.178166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf302-e136-465a-8b70-49d31013e3a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:21.188136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccbf302-e136-465a-8b70-49d31013e3a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:21.189936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccbf302-e136-465a-8b70-49d31013e3a1 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:21.213327 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3577/7072] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:21.459471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bbd3ff-3450-4cb9-b733-ff26f8968caa None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:21.462334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-31bbd3ff-3450-4cb9-b733-ff26f8968caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:21.480560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bbd3ff-3450-4cb9-b733-ff26f8968caa None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:21.486337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccf2b2-70e3-4d8b-a5fd-2e43bc442784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:21.487290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccf2b2-70e3-4d8b-a5fd-2e43bc442784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:21.487801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccf2b2-70e3-4d8b-a5fd-2e43bc442784 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:21.488350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccf2b2-70e3-4d8b-a5fd-2e43bc442784 admin admin] PATH_INFO: `/v3/users/fdcfdca4137f498588b19aebb1b4053f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:21.489902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccf2b2-70e3-4d8b-a5fd-2e43bc442784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdcfdca4137f498588b19aebb1b4053f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:21.492149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccf2b2-70e3-4d8b-a5fd-2e43bc442784 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:21.576200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a8ccf2b2-70e3-4d8b-a5fd-2e43bc442784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdcfdca4137f498588b19aebb1b4053f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:21.590160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a8ccf2b2-70e3-4d8b-a5fd-2e43bc442784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdcfdca4137f498588b19aebb1b4053f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:21.608295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a8ccf2b2-70e3-4d8b-a5fd-2e43bc442784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdcfdca4137f498588b19aebb1b4053f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:21.609392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a8ccf2b2-70e3-4d8b-a5fd-2e43bc442784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdcfdca4137f498588b19aebb1b4053f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:21.610432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a8ccf2b2-70e3-4d8b-a5fd-2e43bc442784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdcfdca4137f498588b19aebb1b4053f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:21.629053 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3496/7073] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:23:21 2022] DELETE /identity/v3/users/fdcfdca4137f498588b19aebb1b4053f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:21.739811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbf302-e136-465a-8b70-49d31013e3a1 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:21.742246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-eccbf302-e136-465a-8b70-49d31013e3a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:21.761114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbf302-e136-465a-8b70-49d31013e3a1 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:21.795828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:21.880685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdb695-ed11-4f25-873c-dbcc77496c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:21.881707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdb695-ed11-4f25-873c-dbcc77496c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:21.881983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdb695-ed11-4f25-873c-dbcc77496c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:21.882252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdb695-ed11-4f25-873c-dbcc77496c24 admin admin] PATH_INFO: `/v3/projects/b10ddde6a64f4c809559c9682a131753` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:21.883883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdb695-ed11-4f25-873c-dbcc77496c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10ddde6a64f4c809559c9682a131753)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:21.903239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdb695-ed11-4f25-873c-dbcc77496c24 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:21.986888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f5bdb695-ed11-4f25-873c-dbcc77496c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10ddde6a64f4c809559c9682a131753', 'request_id': 'req-f5bdb695-ed11-4f25-873c-dbcc77496c24'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:22.003892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f5bdb695-ed11-4f25-873c-dbcc77496c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10ddde6a64f4c809559c9682a131753', 'request_id': 'req-f5bdb695-ed11-4f25-873c-dbcc77496c24'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:22.016435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f5bdb695-ed11-4f25-873c-dbcc77496c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10ddde6a64f4c809559c9682a131753', 'request_id': 'req-f5bdb695-ed11-4f25-873c-dbcc77496c24'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:22.036039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f5bdb695-ed11-4f25-873c-dbcc77496c24 admin admin] The token cache is being invalidate because project b10ddde6a64f4c809559c9682a131753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:23:22.036465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f5bdb695-ed11-4f25-873c-dbcc77496c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:22.037211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f5bdb695-ed11-4f25-873c-dbcc77496c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:22.041126 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3578/7074] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:23:21 2022] DELETE /identity/v3/projects/b10ddde6a64f4c809559c9682a131753 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:22.046459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765cf6d-3a5a-4db3-aee8-e0ba71490c0c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:22.046883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765cf6d-3a5a-4db3-aee8-e0ba71490c0c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:22.047120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765cf6d-3a5a-4db3-aee8-e0ba71490c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:22.083056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1765cf6d-3a5a-4db3-aee8-e0ba71490c0c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:22.084951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1765cf6d-3a5a-4db3-aee8-e0ba71490c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:22.123353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:22.191571 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3497/7075] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:22.199912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a1899-ea74-45cd-a71e-5b3d7c13fc4e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:22.200221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a1899-ea74-45cd-a71e-5b3d7c13fc4e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:22.200519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a1899-ea74-45cd-a71e-5b3d7c13fc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:22.243065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-782a1899-ea74-45cd-a71e-5b3d7c13fc4e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:22.244302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-782a1899-ea74-45cd-a71e-5b3d7c13fc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:22.256221 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3579/7076] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:22.260697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765cf6d-3a5a-4db3-aee8-e0ba71490c0c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:22.274873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765cf6d-3a5a-4db3-aee8-e0ba71490c0c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:22.280932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19920df5-98c6-428e-ae84-1fe0c8d57da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:22.281837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19920df5-98c6-428e-ae84-1fe0c8d57da8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:22.282356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19920df5-98c6-428e-ae84-1fe0c8d57da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:22.283062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19920df5-98c6-428e-ae84-1fe0c8d57da8 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:22.284603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19920df5-98c6-428e-ae84-1fe0c8d57da8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:23:22.285432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19920df5-98c6-428e-ae84-1fe0c8d57da8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:22.287529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19920df5-98c6-428e-ae84-1fe0c8d57da8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:22.303867 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3498/7077] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:23:22.308707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a1899-ea74-45cd-a71e-5b3d7c13fc4e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:22.320677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a1899-ea74-45cd-a71e-5b3d7c13fc4e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:22.325001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4381a02-3dd4-4190-9695-1602c9c14772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:22.325708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4381a02-3dd4-4190-9695-1602c9c14772 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:22.326048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4381a02-3dd4-4190-9695-1602c9c14772 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:22.326391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4381a02-3dd4-4190-9695-1602c9c14772 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:22.327104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4381a02-3dd4-4190-9695-1602c9c14772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:22.329136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4381a02-3dd4-4190-9695-1602c9c14772 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:22.362529 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3580/7078] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:23:22 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:22.366953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19920df5-98c6-428e-ae84-1fe0c8d57da8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:22.381344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19920df5-98c6-428e-ae84-1fe0c8d57da8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:22.449852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac6c93-de7d-4603-85bc-da1e474ff869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:22.451030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac6c93-de7d-4603-85bc-da1e474ff869 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:22.451030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac6c93-de7d-4603-85bc-da1e474ff869 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:22.451247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac6c93-de7d-4603-85bc-da1e474ff869 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:22.451803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac6c93-de7d-4603-85bc-da1e474ff869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:22.455129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac6c93-de7d-4603-85bc-da1e474ff869 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:22.479630 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3499/7079] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:23:22 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:22.484065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4381a02-3dd4-4190-9695-1602c9c14772 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:22.498285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4381a02-3dd4-4190-9695-1602c9c14772 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:22.504491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589707a4-1c3a-4c11-b152-bf2a121be1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:22.504875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589707a4-1c3a-4c11-b152-bf2a121be1ba admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:22.505142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589707a4-1c3a-4c11-b152-bf2a121be1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:22.505402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589707a4-1c3a-4c11-b152-bf2a121be1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:22.506044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589707a4-1c3a-4c11-b152-bf2a121be1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:22.507704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589707a4-1c3a-4c11-b152-bf2a121be1ba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:22.519986 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3581/7080] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:23:22.524202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac6c93-de7d-4603-85bc-da1e474ff869 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:22.535234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac6c93-de7d-4603-85bc-da1e474ff869 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:22.541997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7b381-0421-4cfc-b692-7d467614638c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:22.542411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7b381-0421-4cfc-b692-7d467614638c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:22.542723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7b381-0421-4cfc-b692-7d467614638c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:22.542981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7b381-0421-4cfc-b692-7d467614638c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:22.544497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7b381-0421-4cfc-b692-7d467614638c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:22.546307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7b381-0421-4cfc-b692-7d467614638c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:22.564082 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3500/7081] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:22.565757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589707a4-1c3a-4c11-b152-bf2a121be1ba admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:22.582118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589707a4-1c3a-4c11-b152-bf2a121be1ba admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:22.593424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d77813-23fc-44ec-8811-450dfde7ab7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:22.594242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d77813-23fc-44ec-8811-450dfde7ab7e admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:22.595841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d77813-23fc-44ec-8811-450dfde7ab7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:22.596120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d77813-23fc-44ec-8811-450dfde7ab7e admin admin] PATH_INFO: `/v3/projects/86fdb0fc3bc24270a68fe7e7c4cbb653/users/ed95105bb93f4098b5a9e023c0057896/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:22.640446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d77813-23fc-44ec-8811-450dfde7ab7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=86fdb0fc3bc24270a68fe7e7c4cbb653, user_id=ed95105bb93f4098b5a9e023c0057896, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:22.644400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d77813-23fc-44ec-8811-450dfde7ab7e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:22.670875 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3582/7082] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:23:22 2022] PUT /identity/v3/projects/86fdb0fc3bc24270a68fe7e7c4cbb653/users/ed95105bb93f4098b5a9e023c0057896/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:22.681777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfc939-653e-4330-bad2-8dab5d646a66 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:22.682061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfc939-653e-4330-bad2-8dab5d646a66 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:22.682331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfc939-653e-4330-bad2-8dab5d646a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:22.730879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d77813-23fc-44ec-8811-450dfde7ab7e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:22.732892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a1d77813-23fc-44ec-8811-450dfde7ab7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:22.753564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-abdfc939-653e-4330-bad2-8dab5d646a66 None None] MFA Rules not processed for user `ed95105bb93f4098b5a9e023c0057896`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:22.764306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-abdfc939-653e-4330-bad2-8dab5d646a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:22.924083 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3501/7083] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:22.924882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:22.941242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfc939-653e-4330-bad2-8dab5d646a66 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:22.944415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-abdfc939-653e-4330-bad2-8dab5d646a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:22.973744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d77813-23fc-44ec-8811-450dfde7ab7e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:22.991289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:22.994742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:22.995553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:22.995866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:22.996159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] PATH_INFO: `/v3/projects/4951d1fc525642ea97bcb1b772a5ea16` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:23.002380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4951d1fc525642ea97bcb1b772a5ea16)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:23.005940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:23.051427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfc939-653e-4330-bad2-8dab5d646a66 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:23.057419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0dfdd2-db38-4553-be22-2a4c3dded7b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:23.058362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dfdd2-db38-4553-be22-2a4c3dded7b7 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:23.059392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dfdd2-db38-4553-be22-2a4c3dded7b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:23.059868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dfdd2-db38-4553-be22-2a4c3dded7b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:23.073222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0dfdd2-db38-4553-be22-2a4c3dded7b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:23.075211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0dfdd2-db38-4553-be22-2a4c3dded7b7 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:23.087047 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3502/7084] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:23.134960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4951d1fc525642ea97bcb1b772a5ea16', 'request_id': 'req-11ccf37f-e9b2-446b-a706-db000b6b846d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:23.148167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4951d1fc525642ea97bcb1b772a5ea16', 'request_id': 'req-11ccf37f-e9b2-446b-a706-db000b6b846d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:23.162078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4951d1fc525642ea97bcb1b772a5ea16', 'request_id': 'req-11ccf37f-e9b2-446b-a706-db000b6b846d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:23.201287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] The token cache is being invalidate because project 4951d1fc525642ea97bcb1b772a5ea16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:23:23.204358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:23.207450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:23.217688 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3583/7085] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:23:22 2022] DELETE /identity/v3/projects/4951d1fc525642ea97bcb1b772a5ea16 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:23.222249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0dfdd2-db38-4553-be22-2a4c3dded7b7 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:23.239444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0dfdd2-db38-4553-be22-2a4c3dded7b7 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:23.351501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c97d37-671b-4615-b687-21696745de8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:23.353064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c97d37-671b-4615-b687-21696745de8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:23.354065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c97d37-671b-4615-b687-21696745de8d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:23.354642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c97d37-671b-4615-b687-21696745de8d admin admin] PATH_INFO: `/v3/users/e736d061cd08493ea325f9bfa29605a3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:23.357292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c97d37-671b-4615-b687-21696745de8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e736d061cd08493ea325f9bfa29605a3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:23.360063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c97d37-671b-4615-b687-21696745de8d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:23.489641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-96c97d37-671b-4615-b687-21696745de8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e736d061cd08493ea325f9bfa29605a3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:23.510756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-96c97d37-671b-4615-b687-21696745de8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e736d061cd08493ea325f9bfa29605a3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:23.536754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-96c97d37-671b-4615-b687-21696745de8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e736d061cd08493ea325f9bfa29605a3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:23.537918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-96c97d37-671b-4615-b687-21696745de8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e736d061cd08493ea325f9bfa29605a3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:23.538683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-96c97d37-671b-4615-b687-21696745de8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e736d061cd08493ea325f9bfa29605a3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:23.554966 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3503/7086] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:23:23 2022] DELETE /identity/v3/users/e736d061cd08493ea325f9bfa29605a3 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:23:23.935627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:23.937441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:23.968009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccf37f-e9b2-446b-a706-db000b6b846d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:24.081269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0416d-c049-4ade-a48c-1d5a599ab958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:24.082271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0416d-c049-4ade-a48c-1d5a599ab958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:24.083451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0416d-c049-4ade-a48c-1d5a599ab958 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:24.083730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0416d-c049-4ade-a48c-1d5a599ab958 admin admin] PATH_INFO: `/v3/projects/9308366863164a3fb83124eacba73b87` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:24.085135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0416d-c049-4ade-a48c-1d5a599ab958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9308366863164a3fb83124eacba73b87)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:24.087168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0416d-c049-4ade-a48c-1d5a599ab958 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:24.179756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e5c0416d-c049-4ade-a48c-1d5a599ab958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9308366863164a3fb83124eacba73b87', 'request_id': 'req-e5c0416d-c049-4ade-a48c-1d5a599ab958'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:24.194076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e5c0416d-c049-4ade-a48c-1d5a599ab958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9308366863164a3fb83124eacba73b87', 'request_id': 'req-e5c0416d-c049-4ade-a48c-1d5a599ab958'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:24.224082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e5c0416d-c049-4ade-a48c-1d5a599ab958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9308366863164a3fb83124eacba73b87', 'request_id': 'req-e5c0416d-c049-4ade-a48c-1d5a599ab958'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:24.241916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e5c0416d-c049-4ade-a48c-1d5a599ab958 admin admin] The token cache is being invalidate because project 9308366863164a3fb83124eacba73b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:23:24.241916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e5c0416d-c049-4ade-a48c-1d5a599ab958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:24.242637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e5c0416d-c049-4ade-a48c-1d5a599ab958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:24.251306 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3584/7087] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:23:23 2022] DELETE /identity/v3/projects/9308366863164a3fb83124eacba73b87 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:24.252653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69748f4-150a-4c4d-88a9-df9ed9088066 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:24.253261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69748f4-150a-4c4d-88a9-df9ed9088066 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:24.253677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69748f4-150a-4c4d-88a9-df9ed9088066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:24.297443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d69748f4-150a-4c4d-88a9-df9ed9088066 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:24.298700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d69748f4-150a-4c4d-88a9-df9ed9088066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:24.389448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:24.418723 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3504/7088] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:23:24.427539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc5f8d-d34b-41cb-bdb6-a37681f75e88 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:24.427829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc5f8d-d34b-41cb-bdb6-a37681f75e88 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:24.428070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc5f8d-d34b-41cb-bdb6-a37681f75e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:24.457391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1edc5f8d-d34b-41cb-bdb6-a37681f75e88 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:24.458652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1edc5f8d-d34b-41cb-bdb6-a37681f75e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:24.477007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69748f4-150a-4c4d-88a9-df9ed9088066 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:24.483757 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3585/7089] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:24.495417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69748f4-150a-4c4d-88a9-df9ed9088066 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:24.501826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e0ca4-1912-49af-b7e8-7d6b94afee15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:24.502458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0ca4-1912-49af-b7e8-7d6b94afee15 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:24.502717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0ca4-1912-49af-b7e8-7d6b94afee15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:24.503118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0ca4-1912-49af-b7e8-7d6b94afee15 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:24.504260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e0ca4-1912-49af-b7e8-7d6b94afee15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:23:24.504533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e0ca4-1912-49af-b7e8-7d6b94afee15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:24.506542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e0ca4-1912-49af-b7e8-7d6b94afee15 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:24.526651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc5f8d-d34b-41cb-bdb6-a37681f75e88 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:24.534705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:24.539086 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3505/7090] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:23:24.540828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc5f8d-d34b-41cb-bdb6-a37681f75e88 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:24.553890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a57d66-3251-4467-a736-2f16e4111ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:24.554583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a57d66-3251-4467-a736-2f16e4111ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:24.554859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a57d66-3251-4467-a736-2f16e4111ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:24.555121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a57d66-3251-4467-a736-2f16e4111ff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:24.555829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a57d66-3251-4467-a736-2f16e4111ff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:24.558410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a57d66-3251-4467-a736-2f16e4111ff3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:24.618795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e0ca4-1912-49af-b7e8-7d6b94afee15 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:24.630610 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3586/7091] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:23:24 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:24.636982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e0ca4-1912-49af-b7e8-7d6b94afee15 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:24.828201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f8f26-fd0b-4394-9979-6a7fe7db730d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:24.829252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f8f26-fd0b-4394-9979-6a7fe7db730d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:24.829808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f8f26-fd0b-4394-9979-6a7fe7db730d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:24.830498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f8f26-fd0b-4394-9979-6a7fe7db730d admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:24.831625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f8f26-fd0b-4394-9979-6a7fe7db730d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:24.846916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f8f26-fd0b-4394-9979-6a7fe7db730d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:24.880436 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3506/7092] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:23:24 2022] POST /identity/v3/users => generated 283 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:24.886422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a57d66-3251-4467-a736-2f16e4111ff3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:24.901441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a57d66-3251-4467-a736-2f16e4111ff3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:24.911692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcb48c-02ca-46dd-9f23-707c7f8955df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:24.912301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcb48c-02ca-46dd-9f23-707c7f8955df admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:24.912597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcb48c-02ca-46dd-9f23-707c7f8955df admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:24.915054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcb48c-02ca-46dd-9f23-707c7f8955df admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:24.916463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcb48c-02ca-46dd-9f23-707c7f8955df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:24.919233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcb48c-02ca-46dd-9f23-707c7f8955df admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:24.957236 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3587/7093] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:24.959145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f8f26-fd0b-4394-9979-6a7fe7db730d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:24.986386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f8f26-fd0b-4394-9979-6a7fe7db730d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:25.006497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a326ae7-2411-4ebd-9e02-1d5f1ff2d1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:25.007760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a326ae7-2411-4ebd-9e02-1d5f1ff2d1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:25.008276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a326ae7-2411-4ebd-9e02-1d5f1ff2d1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:25.008819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a326ae7-2411-4ebd-9e02-1d5f1ff2d1f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:25.009697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a326ae7-2411-4ebd-9e02-1d5f1ff2d1f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:25.011706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a326ae7-2411-4ebd-9e02-1d5f1ff2d1f5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:25.037280 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3507/7094] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:23:25.038707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcb48c-02ca-46dd-9f23-707c7f8955df admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:25.057281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcb48c-02ca-46dd-9f23-707c7f8955df admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:25.085429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8162c39-f941-4d37-899a-eb7876ead3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:25.085429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8162c39-f941-4d37-899a-eb7876ead3c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:25.085429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8162c39-f941-4d37-899a-eb7876ead3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:25.086590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8162c39-f941-4d37-899a-eb7876ead3c1 admin admin] PATH_INFO: `/v3/projects/02bc53e8ace84e8582091742eed1bc76/users/4e6a5f35d172410481a8416b7bdaafac/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:25.134271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8162c39-f941-4d37-899a-eb7876ead3c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02bc53e8ace84e8582091742eed1bc76, user_id=4e6a5f35d172410481a8416b7bdaafac, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:25.175102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8162c39-f941-4d37-899a-eb7876ead3c1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:25.210384 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3588/7095] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:23:25 2022] PUT /identity/v3/projects/02bc53e8ace84e8582091742eed1bc76/users/4e6a5f35d172410481a8416b7bdaafac/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:25.215844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848c2d0-25b7-498f-b79d-6ea64a491b60 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:25.216301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848c2d0-25b7-498f-b79d-6ea64a491b60 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:25.216626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848c2d0-25b7-498f-b79d-6ea64a491b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:25.260621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d848c2d0-25b7-498f-b79d-6ea64a491b60 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:25.262525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d848c2d0-25b7-498f-b79d-6ea64a491b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:25.403867 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3508/7096] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:23:25.428459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8162c39-f941-4d37-899a-eb7876ead3c1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:25.433497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b8162c39-f941-4d37-899a-eb7876ead3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:25.553556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848c2d0-25b7-498f-b79d-6ea64a491b60 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:25.555120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d848c2d0-25b7-498f-b79d-6ea64a491b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:25.584925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848c2d0-25b7-498f-b79d-6ea64a491b60 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:25.605575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:25.606314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:25.607678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:25.607678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] PATH_INFO: `/v3/users/ae2c339c41354a159fc8881b56ac45ec` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:25.615427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae2c339c41354a159fc8881b56ac45ec)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:25.617673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:25.661303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8162c39-f941-4d37-899a-eb7876ead3c1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:25.676629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736f24a-ad3a-4c35-9e5d-a91c52b56053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:25.676629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736f24a-ad3a-4c35-9e5d-a91c52b56053 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:25.676629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736f24a-ad3a-4c35-9e5d-a91c52b56053 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:25.676629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736f24a-ad3a-4c35-9e5d-a91c52b56053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:25.702768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5736f24a-ad3a-4c35-9e5d-a91c52b56053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:25.707016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5736f24a-ad3a-4c35-9e5d-a91c52b56053 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:25.725155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:25.731398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae2c339c41354a159fc8881b56ac45ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:25.733357 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3589/7097] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:25.753835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae2c339c41354a159fc8881b56ac45ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:25.775540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae2c339c41354a159fc8881b56ac45ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:25.777495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae2c339c41354a159fc8881b56ac45ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:25.779095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae2c339c41354a159fc8881b56ac45ec'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:25.815553 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3509/7098] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:23:25 2022] DELETE /identity/v3/users/ae2c339c41354a159fc8881b56ac45ec => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:26.175331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736f24a-ad3a-4c35-9e5d-a91c52b56053 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:26.192712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736f24a-ad3a-4c35-9e5d-a91c52b56053 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:26.392705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:26.393172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:26.393990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:26.394319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] PATH_INFO: `/v3/projects/ce224716983049beac8cb9d463715e5d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:26.397952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce224716983049beac8cb9d463715e5d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:26.404078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:26.522980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce224716983049beac8cb9d463715e5d', 'request_id': 'req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:26.526222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:26.539142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce224716983049beac8cb9d463715e5d', 'request_id': 'req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:26.553230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce224716983049beac8cb9d463715e5d', 'request_id': 'req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:26.603222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] The token cache is being invalidate because project ce224716983049beac8cb9d463715e5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:23:26.605227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:26.608447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:26.614958 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3590/7099] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:23:26 2022] DELETE /identity/v3/projects/ce224716983049beac8cb9d463715e5d => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:26.618685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:26.621306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:26.655560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5544a-9cef-493a-b1e0-a0fde3a5a7ea admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:26.730344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c888f8-ddc5-435e-8abf-5aa2c279f11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:26.731314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c888f8-ddc5-435e-8abf-5aa2c279f11e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:26.731764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c888f8-ddc5-435e-8abf-5aa2c279f11e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:26.732250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c888f8-ddc5-435e-8abf-5aa2c279f11e admin admin] PATH_INFO: `/v3/users/1e417d6da9c94de49b6cf49fe3a3fab0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:26.733737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c888f8-ddc5-435e-8abf-5aa2c279f11e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e417d6da9c94de49b6cf49fe3a3fab0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:26.735935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c888f8-ddc5-435e-8abf-5aa2c279f11e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:26.823228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a5c888f8-ddc5-435e-8abf-5aa2c279f11e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e417d6da9c94de49b6cf49fe3a3fab0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:26.843761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a5c888f8-ddc5-435e-8abf-5aa2c279f11e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e417d6da9c94de49b6cf49fe3a3fab0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:26.862368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a5c888f8-ddc5-435e-8abf-5aa2c279f11e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e417d6da9c94de49b6cf49fe3a3fab0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:26.864025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a5c888f8-ddc5-435e-8abf-5aa2c279f11e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e417d6da9c94de49b6cf49fe3a3fab0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:26.864869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a5c888f8-ddc5-435e-8abf-5aa2c279f11e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e417d6da9c94de49b6cf49fe3a3fab0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:26.888046 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3510/7100] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:23:26 2022] DELETE /identity/v3/users/1e417d6da9c94de49b6cf49fe3a3fab0 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:27.544112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:27.547157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:27.609012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e0f92-94a2-483f-90a5-93f2a2eb91d3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:27.730279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27375286-fcf4-4d65-8afb-da0666c80bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:27.731493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375286-fcf4-4d65-8afb-da0666c80bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:27.731900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375286-fcf4-4d65-8afb-da0666c80bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:27.732291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375286-fcf4-4d65-8afb-da0666c80bb8 admin admin] PATH_INFO: `/v3/projects/22d2200123db45adaa25c8e9676c9057` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:27.734748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27375286-fcf4-4d65-8afb-da0666c80bb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d2200123db45adaa25c8e9676c9057)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:27.737157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27375286-fcf4-4d65-8afb-da0666c80bb8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:27.829138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-27375286-fcf4-4d65-8afb-da0666c80bb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d2200123db45adaa25c8e9676c9057', 'request_id': 'req-27375286-fcf4-4d65-8afb-da0666c80bb8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:27.842707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-27375286-fcf4-4d65-8afb-da0666c80bb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d2200123db45adaa25c8e9676c9057', 'request_id': 'req-27375286-fcf4-4d65-8afb-da0666c80bb8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:27.859057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-27375286-fcf4-4d65-8afb-da0666c80bb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d2200123db45adaa25c8e9676c9057', 'request_id': 'req-27375286-fcf4-4d65-8afb-da0666c80bb8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:27.877988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-27375286-fcf4-4d65-8afb-da0666c80bb8 admin admin] The token cache is being invalidate because project 22d2200123db45adaa25c8e9676c9057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:23:27.879802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-27375286-fcf4-4d65-8afb-da0666c80bb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:27.881124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-27375286-fcf4-4d65-8afb-da0666c80bb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:27.883762 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3591/7101] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:23:27 2022] DELETE /identity/v3/projects/22d2200123db45adaa25c8e9676c9057 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:27.889589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa010e80-2c52-49c5-a223-d8c3a78b7834 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:27.890146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa010e80-2c52-49c5-a223-d8c3a78b7834 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:27.890146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa010e80-2c52-49c5-a223-d8c3a78b7834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:27.911327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-aa010e80-2c52-49c5-a223-d8c3a78b7834 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:27.912947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-aa010e80-2c52-49c5-a223-d8c3a78b7834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:27.974372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:28.006148 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3511/7102] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:28.015333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19f187-7220-40da-9b1c-36b408ef81d4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:28.015333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19f187-7220-40da-9b1c-36b408ef81d4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:28.015333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19f187-7220-40da-9b1c-36b408ef81d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:28.040464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6e19f187-7220-40da-9b1c-36b408ef81d4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:28.042525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6e19f187-7220-40da-9b1c-36b408ef81d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:28.056694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa010e80-2c52-49c5-a223-d8c3a78b7834 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:28.067396 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3592/7103] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:28.070088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa010e80-2c52-49c5-a223-d8c3a78b7834 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:28.080532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b97806-953d-42fc-b8f8-34ef9a0cca4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:28.081819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b97806-953d-42fc-b8f8-34ef9a0cca4a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:28.082243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b97806-953d-42fc-b8f8-34ef9a0cca4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:28.082798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b97806-953d-42fc-b8f8-34ef9a0cca4a admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:28.084822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b97806-953d-42fc-b8f8-34ef9a0cca4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:23:28.085598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b97806-953d-42fc-b8f8-34ef9a0cca4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:28.087778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b97806-953d-42fc-b8f8-34ef9a0cca4a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:28.106923 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3512/7104] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:23:28.111525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19f187-7220-40da-9b1c-36b408ef81d4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:28.129073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19f187-7220-40da-9b1c-36b408ef81d4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:28.155167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4105ab-53f3-4d29-86cb-4c083702889a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:28.156006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4105ab-53f3-4d29-86cb-4c083702889a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:28.158996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4105ab-53f3-4d29-86cb-4c083702889a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:28.172763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4105ab-53f3-4d29-86cb-4c083702889a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:28.176760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4105ab-53f3-4d29-86cb-4c083702889a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:28.182435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4105ab-53f3-4d29-86cb-4c083702889a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:28.256995 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3593/7105] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:23:28 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:28.260602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b97806-953d-42fc-b8f8-34ef9a0cca4a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:28.282721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b97806-953d-42fc-b8f8-34ef9a0cca4a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:28.405162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab217b1-c830-4a30-9585-e8f3d06ae541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:28.406207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab217b1-c830-4a30-9585-e8f3d06ae541 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:28.406590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab217b1-c830-4a30-9585-e8f3d06ae541 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:28.407127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab217b1-c830-4a30-9585-e8f3d06ae541 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:28.408174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab217b1-c830-4a30-9585-e8f3d06ae541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:28.411183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab217b1-c830-4a30-9585-e8f3d06ae541 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:28.443475 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3513/7106] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:23:28 2022] POST /identity/v3/users => generated 291 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:28.445613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4105ab-53f3-4d29-86cb-4c083702889a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:28.466993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4105ab-53f3-4d29-86cb-4c083702889a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:28.475261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6131c1e-6986-468b-8f25-212e30721b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:28.475900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6131c1e-6986-468b-8f25-212e30721b2b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:28.476417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6131c1e-6986-468b-8f25-212e30721b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:28.476693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6131c1e-6986-468b-8f25-212e30721b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:28.478773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6131c1e-6986-468b-8f25-212e30721b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:28.480495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6131c1e-6986-468b-8f25-212e30721b2b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:28.495566 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3594/7107] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:28.496940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab217b1-c830-4a30-9585-e8f3d06ae541 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:28.509517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab217b1-c830-4a30-9585-e8f3d06ae541 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:28.515404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503df30-961c-411f-b616-d283f79a418a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:28.516437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503df30-961c-411f-b616-d283f79a418a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:28.516956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503df30-961c-411f-b616-d283f79a418a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:28.517452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503df30-961c-411f-b616-d283f79a418a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:28.518441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503df30-961c-411f-b616-d283f79a418a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:28.520332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503df30-961c-411f-b616-d283f79a418a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:28.538254 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3514/7108] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:28.539836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6131c1e-6986-468b-8f25-212e30721b2b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:28.553344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6131c1e-6986-468b-8f25-212e30721b2b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:28.559376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02156a97-44d9-4530-9e0a-d4844ec07b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:28.560858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02156a97-44d9-4530-9e0a-d4844ec07b0a admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:28.561328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02156a97-44d9-4530-9e0a-d4844ec07b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:28.561747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02156a97-44d9-4530-9e0a-d4844ec07b0a admin admin] PATH_INFO: `/v3/projects/3812d5019f224d889ea890ba9045245a/users/398f9812de35475c93fb6cfab8f6bf72/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:28.584665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02156a97-44d9-4530-9e0a-d4844ec07b0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3812d5019f224d889ea890ba9045245a, user_id=398f9812de35475c93fb6cfab8f6bf72, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:28.588727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02156a97-44d9-4530-9e0a-d4844ec07b0a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:28.618020 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3595/7109] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:23:28 2022] PUT /identity/v3/projects/3812d5019f224d889ea890ba9045245a/users/398f9812de35475c93fb6cfab8f6bf72/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:28.619901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b44ba-dd39-4082-bc3c-4abd6a986911 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:28.620337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b44ba-dd39-4082-bc3c-4abd6a986911 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:28.620777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b44ba-dd39-4082-bc3c-4abd6a986911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:28.665263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ae6b44ba-dd39-4082-bc3c-4abd6a986911 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:28.666899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ae6b44ba-dd39-4082-bc3c-4abd6a986911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:28.757254 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3515/7110] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:23:28.781313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02156a97-44d9-4530-9e0a-d4844ec07b0a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:28.783603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-02156a97-44d9-4530-9e0a-d4844ec07b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:28.799025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:28.874758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02156a97-44d9-4530-9e0a-d4844ec07b0a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:28.882036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2298e-38bc-4531-abd5-3a4276c773ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:28.882702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2298e-38bc-4531-abd5-3a4276c773ad service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:28.882975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2298e-38bc-4531-abd5-3a4276c773ad service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:28.883889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2298e-38bc-4531-abd5-3a4276c773ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:28.896109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2298e-38bc-4531-abd5-3a4276c773ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:28.897679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2298e-38bc-4531-abd5-3a4276c773ad service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:28.914273 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3596/7111] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:32.958647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b44ba-dd39-4082-bc3c-4abd6a986911 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:32.960144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ae6b44ba-dd39-4082-bc3c-4abd6a986911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:32.976773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b44ba-dd39-4082-bc3c-4abd6a986911 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:32.989156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:32.989826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:32.990105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:32.990373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] PATH_INFO: `/v3/users/47f61fd7b2384ba7ba84a5c1f8b913c4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:32.991540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f61fd7b2384ba7ba84a5c1f8b913c4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:32.993745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:33.079431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:33.088399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f61fd7b2384ba7ba84a5c1f8b913c4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:33.113483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f61fd7b2384ba7ba84a5c1f8b913c4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:33.130929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f61fd7b2384ba7ba84a5c1f8b913c4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:33.131826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f61fd7b2384ba7ba84a5c1f8b913c4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:33.132778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f61fd7b2384ba7ba84a5c1f8b913c4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:33.151893 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3516/7112] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:23:32 2022] DELETE /identity/v3/users/47f61fd7b2384ba7ba84a5c1f8b913c4 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:33.157440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d406231-46ab-4690-b692-6fcc62bc3667 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:33.158109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d406231-46ab-4690-b692-6fcc62bc3667 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:33.158109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d406231-46ab-4690-b692-6fcc62bc3667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:33.195910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3d406231-46ab-4690-b692-6fcc62bc3667 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:33.197203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3d406231-46ab-4690-b692-6fcc62bc3667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:33.341606 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3597/7113] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:33.393574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:33.394614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:33.600225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d60e8d-d7cc-4e15-b54e-3c5c4fa7dc6d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:33.607532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c96590-2550-4ee6-9fc3-d97dc930b61a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:33.609031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c96590-2550-4ee6-9fc3-d97dc930b61a service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:33.609031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c96590-2550-4ee6-9fc3-d97dc930b61a service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:33.609031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c96590-2550-4ee6-9fc3-d97dc930b61a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:33.633602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c96590-2550-4ee6-9fc3-d97dc930b61a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:33.636085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c96590-2550-4ee6-9fc3-d97dc930b61a service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:33.670187 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3517/7114] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:35.217039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d406231-46ab-4690-b692-6fcc62bc3667 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:35.219239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3d406231-46ab-4690-b692-6fcc62bc3667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:35.244585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d406231-46ab-4690-b692-6fcc62bc3667 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:35.269812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:35.270788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:35.271095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:35.271363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0 admin admin] PATH_INFO: `/v3/projects/3dafa2f5aa284027af9b66c48c35bd32` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:35.272793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dafa2f5aa284027af9b66c48c35bd32)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:35.275094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:35.364861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c96590-2550-4ee6-9fc3-d97dc930b61a service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:35.366477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-30c96590-2550-4ee6-9fc3-d97dc930b61a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:35.405340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c96590-2550-4ee6-9fc3-d97dc930b61a service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:35.410071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dafa2f5aa284027af9b66c48c35bd32', 'request_id': 'req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:35.441367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dafa2f5aa284027af9b66c48c35bd32', 'request_id': 'req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:35.467526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dafa2f5aa284027af9b66c48c35bd32', 'request_id': 'req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:35.510372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0 admin admin] The token cache is being invalidate because project 3dafa2f5aa284027af9b66c48c35bd32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:23:35.510799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:35.513043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dcab6fab-9e19-4ee8-aa3c-9e080d8f90f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:35.516857 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3598/7115] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:23:35 2022] DELETE /identity/v3/projects/3dafa2f5aa284027af9b66c48c35bd32 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:35.522494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed886c-9247-4bbf-b83f-7d20d091b79c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:35.522872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed886c-9247-4bbf-b83f-7d20d091b79c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:35.523140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed886c-9247-4bbf-b83f-7d20d091b79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:35.555577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5cb78-f281-4cb1-8ff4-51a611ae3f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:35.556251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5cb78-f281-4cb1-8ff4-51a611ae3f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:35.556625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5cb78-f281-4cb1-8ff4-51a611ae3f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:35.556903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5cb78-f281-4cb1-8ff4-51a611ae3f22 admin admin] PATH_INFO: `/v3/users/daffa49c8fa64a3bb96f917fef222291` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:35.559014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5cb78-f281-4cb1-8ff4-51a611ae3f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daffa49c8fa64a3bb96f917fef222291)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:35.561218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5cb78-f281-4cb1-8ff4-51a611ae3f22 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:35.613849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dfed886c-9247-4bbf-b83f-7d20d091b79c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:35.615862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dfed886c-9247-4bbf-b83f-7d20d091b79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:35.643618 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3599/7116] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:35.646446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45370414-6182-4574-a3e7-a7c7127c798d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:35.646982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45370414-6182-4574-a3e7-a7c7127c798d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:35.647525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45370414-6182-4574-a3e7-a7c7127c798d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:35.656473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3db5cb78-f281-4cb1-8ff4-51a611ae3f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daffa49c8fa64a3bb96f917fef222291'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:35.676973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3db5cb78-f281-4cb1-8ff4-51a611ae3f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daffa49c8fa64a3bb96f917fef222291'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:35.687853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:35.698447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3db5cb78-f281-4cb1-8ff4-51a611ae3f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daffa49c8fa64a3bb96f917fef222291'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:35.707199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-45370414-6182-4574-a3e7-a7c7127c798d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:35.708865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3db5cb78-f281-4cb1-8ff4-51a611ae3f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daffa49c8fa64a3bb96f917fef222291'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:35.709853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3db5cb78-f281-4cb1-8ff4-51a611ae3f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daffa49c8fa64a3bb96f917fef222291'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:35.711905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-45370414-6182-4574-a3e7-a7c7127c798d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:35.731511 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3518/7117] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:23:35 2022] DELETE /identity/v3/users/daffa49c8fa64a3bb96f917fef222291 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:23:35.735766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee5870-829b-42f1-9ab3-f521ff9f1bd4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:35.736511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee5870-829b-42f1-9ab3-f521ff9f1bd4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:35.736511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee5870-829b-42f1-9ab3-f521ff9f1bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:35.747435 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3600/7118] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:35.748930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45370414-6182-4574-a3e7-a7c7127c798d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:35.765488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-30ee5870-829b-42f1-9ab3-f521ff9f1bd4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:35.767319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-30ee5870-829b-42f1-9ab3-f521ff9f1bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:35.773901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:35.784697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45370414-6182-4574-a3e7-a7c7127c798d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:35.848321 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3519/7119] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:23:35.858058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee5870-829b-42f1-9ab3-f521ff9f1bd4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:35.859600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-30ee5870-829b-42f1-9ab3-f521ff9f1bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:35.864216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed71d2a-670e-46a8-bb86-6a332d29f374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:35.865337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed71d2a-670e-46a8-bb86-6a332d29f374 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:35.866333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed71d2a-670e-46a8-bb86-6a332d29f374 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:35.867104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed71d2a-670e-46a8-bb86-6a332d29f374 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:35.868347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed71d2a-670e-46a8-bb86-6a332d29f374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:23:35.868896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed71d2a-670e-46a8-bb86-6a332d29f374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:35.871773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed71d2a-670e-46a8-bb86-6a332d29f374 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:35.888525 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3601/7120] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:23:35.898069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed71d2a-670e-46a8-bb86-6a332d29f374 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:35.924101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed71d2a-670e-46a8-bb86-6a332d29f374 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:35.932829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee5870-829b-42f1-9ab3-f521ff9f1bd4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:35.934277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce55f8e-cd37-4742-9d08-d8dd8ad21d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:35.935152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce55f8e-cd37-4742-9d08-d8dd8ad21d03 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:35.935563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce55f8e-cd37-4742-9d08-d8dd8ad21d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:35.935981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce55f8e-cd37-4742-9d08-d8dd8ad21d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:35.937038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdff12-76f7-4ef6-951c-3d5d0ff2838a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:35.937535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce55f8e-cd37-4742-9d08-d8dd8ad21d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:35.937842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdff12-76f7-4ef6-951c-3d5d0ff2838a service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:35.938215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdff12-76f7-4ef6-951c-3d5d0ff2838a service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:35.938580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdff12-76f7-4ef6-951c-3d5d0ff2838a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:35.941349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce55f8e-cd37-4742-9d08-d8dd8ad21d03 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:35.953321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdff12-76f7-4ef6-951c-3d5d0ff2838a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:35.955208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdff12-76f7-4ef6-951c-3d5d0ff2838a service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:35.968498 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3520/7121] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:23:35.983993 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3602/7122] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:23:35 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:35.988314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdff12-76f7-4ef6-951c-3d5d0ff2838a service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:36.001956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdff12-76f7-4ef6-951c-3d5d0ff2838a service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:36.076867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b3f0e-806f-448f-9b6d-3740ccc8a780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:36.077646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b3f0e-806f-448f-9b6d-3740ccc8a780 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:36.077998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b3f0e-806f-448f-9b6d-3740ccc8a780 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:36.078336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b3f0e-806f-448f-9b6d-3740ccc8a780 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:36.079766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b3f0e-806f-448f-9b6d-3740ccc8a780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:36.082002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b3f0e-806f-448f-9b6d-3740ccc8a780 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:36.113070 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3521/7123] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:23:35 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:36.115063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce55f8e-cd37-4742-9d08-d8dd8ad21d03 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:36.129876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce55f8e-cd37-4742-9d08-d8dd8ad21d03 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:36.136823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b21c9-817b-40d7-9a3e-c537daab476c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:36.137639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b21c9-817b-40d7-9a3e-c537daab476c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:36.138097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b21c9-817b-40d7-9a3e-c537daab476c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:36.138705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b21c9-817b-40d7-9a3e-c537daab476c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:36.139918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b21c9-817b-40d7-9a3e-c537daab476c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:36.142236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b21c9-817b-40d7-9a3e-c537daab476c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:36.172700 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3603/7124] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:36.177076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b3f0e-806f-448f-9b6d-3740ccc8a780 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:36.201104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b3f0e-806f-448f-9b6d-3740ccc8a780 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:36.218252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978dbd1-751a-4e6b-ada1-145938950259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:36.220013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978dbd1-751a-4e6b-ada1-145938950259 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:36.220013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978dbd1-751a-4e6b-ada1-145938950259 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:36.220239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978dbd1-751a-4e6b-ada1-145938950259 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:36.221218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978dbd1-751a-4e6b-ada1-145938950259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:36.227413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978dbd1-751a-4e6b-ada1-145938950259 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:36.244430 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3522/7125] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:36.248017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b21c9-817b-40d7-9a3e-c537daab476c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:36.249424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978dbd1-751a-4e6b-ada1-145938950259 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:36.251651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-305b21c9-817b-40d7-9a3e-c537daab476c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:36.258681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:36.271933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978dbd1-751a-4e6b-ada1-145938950259 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:36.275546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b21c9-817b-40d7-9a3e-c537daab476c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:36.277492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d254df-629c-4a2a-b397-17d3b5fad59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:36.278325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d254df-629c-4a2a-b397-17d3b5fad59e admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:36.278683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d254df-629c-4a2a-b397-17d3b5fad59e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:36.279046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d254df-629c-4a2a-b397-17d3b5fad59e admin admin] PATH_INFO: `/v3/projects/9f376335767945ff99a527fca520a078/users/b02f273155964b468f710fd5b819c2ef/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:36.290096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:36.291747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:36.293376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:36.293982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] PATH_INFO: `/v3/projects/ff3eb7954b534dc89fefa2ec56d97fe0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:36.296321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff3eb7954b534dc89fefa2ec56d97fe0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:36.299459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:36.302872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d254df-629c-4a2a-b397-17d3b5fad59e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f376335767945ff99a527fca520a078, user_id=b02f273155964b468f710fd5b819c2ef, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:36.306039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d254df-629c-4a2a-b397-17d3b5fad59e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:36.336477 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3523/7126] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:23:36 2022] PUT /identity/v3/projects/9f376335767945ff99a527fca520a078/users/b02f273155964b468f710fd5b819c2ef/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:36.347359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a206de-2036-4250-966f-6314532acc65 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:36.347637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a206de-2036-4250-966f-6314532acc65 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:36.347903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a206de-2036-4250-966f-6314532acc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:36.421126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff3eb7954b534dc89fefa2ec56d97fe0', 'request_id': 'req-6bb20ea8-54ac-4683-be7f-7efffb741d68'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:36.423447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-89a206de-2036-4250-966f-6314532acc65 None None] MFA Rules not processed for user `b02f273155964b468f710fd5b819c2ef`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:36.424933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-89a206de-2036-4250-966f-6314532acc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:36.443280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff3eb7954b534dc89fefa2ec56d97fe0', 'request_id': 'req-6bb20ea8-54ac-4683-be7f-7efffb741d68'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:36.468936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff3eb7954b534dc89fefa2ec56d97fe0', 'request_id': 'req-6bb20ea8-54ac-4683-be7f-7efffb741d68'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:36.510691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] The token cache is being invalidate because project ff3eb7954b534dc89fefa2ec56d97fe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:23:36.511050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:36.513316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:36.526852 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3604/7127] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:23:36 2022] DELETE /identity/v3/projects/ff3eb7954b534dc89fefa2ec56d97fe0 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:36.527924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:36.532586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:36.641895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:36.654070 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3524/7128] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:23:36.675422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb20ea8-54ac-4683-be7f-7efffb741d68 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:36.690498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:36.691290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:36.691629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:36.691962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] PATH_INFO: `/v3/users/a713ae7d938a4cd3b57e2fd7203de82a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:36.694588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a713ae7d938a4cd3b57e2fd7203de82a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:36.696717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:36.728465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a206de-2036-4250-966f-6314532acc65 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:36.730332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-89a206de-2036-4250-966f-6314532acc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:36.812745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a713ae7d938a4cd3b57e2fd7203de82a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:36.833878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a713ae7d938a4cd3b57e2fd7203de82a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:36.855588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a713ae7d938a4cd3b57e2fd7203de82a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:36.878047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a206de-2036-4250-966f-6314532acc65 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:36.885447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a713ae7d938a4cd3b57e2fd7203de82a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:36.887568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a713ae7d938a4cd3b57e2fd7203de82a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:36.890204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec7eb1-4ce5-4664-8303-1505159096a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:36.890850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7eb1-4ce5-4664-8303-1505159096a1 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:36.891158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7eb1-4ce5-4664-8303-1505159096a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:36.891594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7eb1-4ce5-4664-8303-1505159096a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:36.896394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-32ec7eb1-4ce5-4664-8303-1505159096a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:36.909635 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3605/7129] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:23:36 2022] DELETE /identity/v3/users/a713ae7d938a4cd3b57e2fd7203de82a => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:36.931097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec7eb1-4ce5-4664-8303-1505159096a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:36.932770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec7eb1-4ce5-4664-8303-1505159096a1 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:36.957333 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3525/7130] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:23:38.037760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:38.040818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:38.282066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a211d13-810f-461a-b3d6-b84a917ea762 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:38.294957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:38.295622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:38.295891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:38.296169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] PATH_INFO: `/v3/projects/9141064c318849649c2ec7ac626d7c23` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:38.305332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9141064c318849649c2ec7ac626d7c23)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:38.307311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:38.433267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9141064c318849649c2ec7ac626d7c23', 'request_id': 'req-52697db4-8fba-4dd4-9636-86297d81f542'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:38.453291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9141064c318849649c2ec7ac626d7c23', 'request_id': 'req-52697db4-8fba-4dd4-9636-86297d81f542'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:38.477693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9141064c318849649c2ec7ac626d7c23', 'request_id': 'req-52697db4-8fba-4dd4-9636-86297d81f542'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:38.513140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:38.526171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] The token cache is being invalidate because project 9141064c318849649c2ec7ac626d7c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:23:38.528501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:38.529651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:38.539874 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3606/7131] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:23:38 2022] DELETE /identity/v3/projects/9141064c318849649c2ec7ac626d7c23 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:38.541675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26229c9a-8915-4d67-a7a9-685c28be7995 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:38.542131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26229c9a-8915-4d67-a7a9-685c28be7995 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:38.542244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26229c9a-8915-4d67-a7a9-685c28be7995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:38.593261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-26229c9a-8915-4d67-a7a9-685c28be7995 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:38.596013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-26229c9a-8915-4d67-a7a9-685c28be7995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:38.788648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:38.796783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:38.853753 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3526/7132] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:38.862172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa660c17-c4a2-4ef1-b8a9-f8863f1ba936 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:38.863681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa660c17-c4a2-4ef1-b8a9-f8863f1ba936 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:38.863972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa660c17-c4a2-4ef1-b8a9-f8863f1ba936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:38.940483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-aa660c17-c4a2-4ef1-b8a9-f8863f1ba936 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:38.949816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-aa660c17-c4a2-4ef1-b8a9-f8863f1ba936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:38.976681 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3527/7133] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:23:38.977775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a25ac-74bd-4f77-8c2c-7c66161deb17 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:38.978056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a25ac-74bd-4f77-8c2c-7c66161deb17 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:38.978317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a25ac-74bd-4f77-8c2c-7c66161deb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:38.992532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52697db4-8fba-4dd4-9636-86297d81f542 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:39.000913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:39.013092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:39.013713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:39.013967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:39.014462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] PATH_INFO: `/v3/users/b02f273155964b468f710fd5b819c2ef` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:39.016875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b02f273155964b468f710fd5b819c2ef)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:39.025222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:39.041069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-516a25ac-74bd-4f77-8c2c-7c66161deb17 None None] MFA Rules not processed for user `4e6a5f35d172410481a8416b7bdaafac`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:39.042351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-516a25ac-74bd-4f77-8c2c-7c66161deb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:39.129836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b02f273155964b468f710fd5b819c2ef'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:39.148947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b02f273155964b468f710fd5b819c2ef'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:39.166350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b02f273155964b468f710fd5b819c2ef'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:39.201916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b02f273155964b468f710fd5b819c2ef'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:39.205100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b02f273155964b468f710fd5b819c2ef'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:39.226863 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3607/7134] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:23:38 2022] DELETE /identity/v3/users/b02f273155964b468f710fd5b819c2ef => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:39.227072 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3528/7135] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:39.227199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:39.227295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c41f4a-600e-41e2-8929-9dc476cb5502 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:39.227295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c41f4a-600e-41e2-8929-9dc476cb5502 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:39.227295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c41f4a-600e-41e2-8929-9dc476cb5502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:39.227524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:39.276847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-53c41f4a-600e-41e2-8929-9dc476cb5502 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:39.278691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-53c41f4a-600e-41e2-8929-9dc476cb5502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:39.375050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38057d40-53a6-43de-9419-71d9a42cd4a1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:39.381075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c13af2-b73e-4c19-9ba8-e5666abe7ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:39.381712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c13af2-b73e-4c19-9ba8-e5666abe7ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:39.381985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c13af2-b73e-4c19-9ba8-e5666abe7ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:39.382251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c13af2-b73e-4c19-9ba8-e5666abe7ff5 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:39.382913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c13af2-b73e-4c19-9ba8-e5666abe7ff5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:23:39.383419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c13af2-b73e-4c19-9ba8-e5666abe7ff5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:39.385417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c13af2-b73e-4c19-9ba8-e5666abe7ff5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:39.419292 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3608/7136] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:23:39.420733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63d206-3933-4200-84fa-e371ce49ea26 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:39.421014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63d206-3933-4200-84fa-e371ce49ea26 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:39.421289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63d206-3933-4200-84fa-e371ce49ea26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:39.445420 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3529/7137] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:39.446222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c41f4a-600e-41e2-8929-9dc476cb5502 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:39.459482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c41f4a-600e-41e2-8929-9dc476cb5502 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:39.467591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3b63d206-3933-4200-84fa-e371ce49ea26 None None] MFA Rules not processed for user `4e6a5f35d172410481a8416b7bdaafac`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:39.469012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3b63d206-3933-4200-84fa-e371ce49ea26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:39.475448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:39.482634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9d9e4-92be-48bf-a61f-55cfbdc2cbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:39.484231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9d9e4-92be-48bf-a61f-55cfbdc2cbbc admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:39.491321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9d9e4-92be-48bf-a61f-55cfbdc2cbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:39.491598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9d9e4-92be-48bf-a61f-55cfbdc2cbbc admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:39.492430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9d9e4-92be-48bf-a61f-55cfbdc2cbbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:39.498949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9d9e4-92be-48bf-a61f-55cfbdc2cbbc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:39.571825 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3530/7138] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:23:39 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:39.572834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9d9e4-92be-48bf-a61f-55cfbdc2cbbc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:39.577056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3db9d9e4-92be-48bf-a61f-55cfbdc2cbbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:39.629283 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3609/7139] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:39.630230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63d206-3933-4200-84fa-e371ce49ea26 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:39.660472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63d206-3933-4200-84fa-e371ce49ea26 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:39.744968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9d9e4-92be-48bf-a61f-55cfbdc2cbbc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:39.751635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19eb2f-0611-48b5-aa2d-d544f0b78142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:39.752272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19eb2f-0611-48b5-aa2d-d544f0b78142 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:39.752842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19eb2f-0611-48b5-aa2d-d544f0b78142 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:39.753240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19eb2f-0611-48b5-aa2d-d544f0b78142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:39.786850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfe921-7e23-4144-a4c6-d2270f1cc701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:39.787496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfe921-7e23-4144-a4c6-d2270f1cc701 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:39.787768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfe921-7e23-4144-a4c6-d2270f1cc701 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:39.788034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfe921-7e23-4144-a4c6-d2270f1cc701 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:39.788744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfe921-7e23-4144-a4c6-d2270f1cc701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:39.790639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfe921-7e23-4144-a4c6-d2270f1cc701 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:39.827397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:39.839117 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3610/7140] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:23:39 2022] POST /identity/v3/users => generated 294 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:39.842351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfe921-7e23-4144-a4c6-d2270f1cc701 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:39.842351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-49bfe921-7e23-4144-a4c6-d2270f1cc701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:39.931247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19eb2f-0611-48b5-aa2d-d544f0b78142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:39.932977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19eb2f-0611-48b5-aa2d-d544f0b78142 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:39.952964 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3531/7141] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:39.955474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19eb2f-0611-48b5-aa2d-d544f0b78142 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:39.979271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19eb2f-0611-48b5-aa2d-d544f0b78142 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:39.991512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58d2cf-5edc-40af-87d0-a0e9e661fda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:39.992407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58d2cf-5edc-40af-87d0-a0e9e661fda8 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:39.996817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58d2cf-5edc-40af-87d0-a0e9e661fda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:39.997232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58d2cf-5edc-40af-87d0-a0e9e661fda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:39.998206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58d2cf-5edc-40af-87d0-a0e9e661fda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:40.000894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58d2cf-5edc-40af-87d0-a0e9e661fda8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:40.007176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfe921-7e23-4144-a4c6-d2270f1cc701 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:40.031557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b19f44-0de6-4872-b461-e294a212b315 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:40.032008 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3532/7142] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:40.032196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b19f44-0de6-4872-b461-e294a212b315 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:40.032770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b19f44-0de6-4872-b461-e294a212b315 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:40.033040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b19f44-0de6-4872-b461-e294a212b315 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:40.034227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58d2cf-5edc-40af-87d0-a0e9e661fda8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:40.039742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-28b19f44-0de6-4872-b461-e294a212b315 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:40.057860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58d2cf-5edc-40af-87d0-a0e9e661fda8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:40.075548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a388b6-d45c-44e6-aa28-b0b2c9c66297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:40.078825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a388b6-d45c-44e6-aa28-b0b2c9c66297 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:40.080927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a388b6-d45c-44e6-aa28-b0b2c9c66297 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:40.081968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a388b6-d45c-44e6-aa28-b0b2c9c66297 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:40.082981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a388b6-d45c-44e6-aa28-b0b2c9c66297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:40.087019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a388b6-d45c-44e6-aa28-b0b2c9c66297 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:40.115410 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3533/7143] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:40.120108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a388b6-d45c-44e6-aa28-b0b2c9c66297 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:40.151461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a388b6-d45c-44e6-aa28-b0b2c9c66297 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:40.166139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f6006-d331-4d29-8930-8a4cb27e5cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:40.166952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f6006-d331-4d29-8930-8a4cb27e5cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:40.168415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f6006-d331-4d29-8930-8a4cb27e5cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:40.168776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f6006-d331-4d29-8930-8a4cb27e5cc6 admin admin] PATH_INFO: `/v3/projects/e670ad3c4f114e188e152858d4756db7/users/22a2f69aa5c4469181b411c0ab26f6aa/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:40.205242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:40.208630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f6006-d331-4d29-8930-8a4cb27e5cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e670ad3c4f114e188e152858d4756db7, user_id=22a2f69aa5c4469181b411c0ab26f6aa, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:40.214943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f6006-d331-4d29-8930-8a4cb27e5cc6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:40.250242 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3534/7144] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:23:40 2022] PUT /identity/v3/projects/e670ad3c4f114e188e152858d4756db7/users/22a2f69aa5c4469181b411c0ab26f6aa/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:40.266139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b19f44-0de6-4872-b461-e294a212b315 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:40.266706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3e01f-11c3-4173-9f59-9b6d1533338a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:40.267358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3e01f-11c3-4173-9f59-9b6d1533338a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:40.267894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3e01f-11c3-4173-9f59-9b6d1533338a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:40.278657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b19f44-0de6-4872-b461-e294a212b315 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:40.306146 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3611/7145] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:40.377273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-64b3e01f-11c3-4173-9f59-9b6d1533338a None None] MFA Rules not processed for user `22a2f69aa5c4469181b411c0ab26f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:40.382169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-64b3e01f-11c3-4173-9f59-9b6d1533338a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:40.579805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b19f44-0de6-4872-b461-e294a212b315 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:40.582244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-28b19f44-0de6-4872-b461-e294a212b315 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:40.609720 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3535/7146] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:40.616234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3e01f-11c3-4173-9f59-9b6d1533338a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:40.621409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b19f44-0de6-4872-b461-e294a212b315 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:40.771255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3e01f-11c3-4173-9f59-9b6d1533338a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:40.782916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84021167-959b-4b05-95c8-08b39cf79af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:40.784433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84021167-959b-4b05-95c8-08b39cf79af5 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:40.785673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84021167-959b-4b05-95c8-08b39cf79af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:40.786033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84021167-959b-4b05-95c8-08b39cf79af5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:40.789116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84021167-959b-4b05-95c8-08b39cf79af5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:40.793687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84021167-959b-4b05-95c8-08b39cf79af5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:40.820495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:40.821115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:40.821384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:40.821653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735 admin admin] PATH_INFO: `/v3/projects/9f376335767945ff99a527fca520a078` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:40.822820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f376335767945ff99a527fca520a078)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:40.823677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:40.824874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:40.856313 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3536/7147] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:23:40 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:40.858500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84021167-959b-4b05-95c8-08b39cf79af5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:40.930891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f376335767945ff99a527fca520a078', 'request_id': 'req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:40.944097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f376335767945ff99a527fca520a078', 'request_id': 'req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:40.956451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f376335767945ff99a527fca520a078', 'request_id': 'req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:40.985470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735 admin admin] The token cache is being invalidate because project 9f376335767945ff99a527fca520a078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:23:40.986023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:40.987861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-24fce7c8-3b0b-4f48-b0d4-fb0dfaa4b735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:40.991092 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3612/7148] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:23:40 2022] DELETE /identity/v3/projects/9f376335767945ff99a527fca520a078 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:23:41.000101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b308e4-0296-4933-a7ae-76ac1ad6d78f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:41.000657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b308e4-0296-4933-a7ae-76ac1ad6d78f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:41.000975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b308e4-0296-4933-a7ae-76ac1ad6d78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:41.022033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84021167-959b-4b05-95c8-08b39cf79af5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:41.037890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-06b308e4-0296-4933-a7ae-76ac1ad6d78f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:41.039428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-06b308e4-0296-4933-a7ae-76ac1ad6d78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:41.126771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db5aef-f0aa-4334-8ee6-8bd523860d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:41.127726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db5aef-f0aa-4334-8ee6-8bd523860d01 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:41.128570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db5aef-f0aa-4334-8ee6-8bd523860d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:41.128570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db5aef-f0aa-4334-8ee6-8bd523860d01 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:41.130471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db5aef-f0aa-4334-8ee6-8bd523860d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:41.135898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db5aef-f0aa-4334-8ee6-8bd523860d01 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:41.152308 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3613/7149] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:41.162821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b308e4-0296-4933-a7ae-76ac1ad6d78f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:41.164819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-06b308e4-0296-4933-a7ae-76ac1ad6d78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:41.190459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:41.212737 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3537/7150] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:23:40 2022] POST /identity/v3/users => generated 300 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:41.218151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5672553-2069-449b-8087-746ff1999ea7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:41.218489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5672553-2069-449b-8087-746ff1999ea7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:41.218707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5672553-2069-449b-8087-746ff1999ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:41.296710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b308e4-0296-4933-a7ae-76ac1ad6d78f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:41.302414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144c593-5061-4e40-8fe2-83ca415df0a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:41.303599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144c593-5061-4e40-8fe2-83ca415df0a8 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:41.304084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144c593-5061-4e40-8fe2-83ca415df0a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:41.304740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144c593-5061-4e40-8fe2-83ca415df0a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:41.307502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1144c593-5061-4e40-8fe2-83ca415df0a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:41.322974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e5672553-2069-449b-8087-746ff1999ea7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:41.324249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e5672553-2069-449b-8087-746ff1999ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:41.333941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144c593-5061-4e40-8fe2-83ca415df0a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:41.335443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144c593-5061-4e40-8fe2-83ca415df0a8 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:41.339920 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3538/7151] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:41.339920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5672553-2069-449b-8087-746ff1999ea7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:41.341145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e5672553-2069-449b-8087-746ff1999ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:41.356197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5672553-2069-449b-8087-746ff1999ea7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:41.357731 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3614/7152] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:41.362512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144c593-5061-4e40-8fe2-83ca415df0a8 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:41.375911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144c593-5061-4e40-8fe2-83ca415df0a8 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:41.384790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c6ef3-dfe1-4c13-81bd-e326899c9f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:41.385426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c6ef3-dfe1-4c13-81bd-e326899c9f54 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:41.385876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c6ef3-dfe1-4c13-81bd-e326899c9f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:41.386141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c6ef3-dfe1-4c13-81bd-e326899c9f54 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:41.386789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c6ef3-dfe1-4c13-81bd-e326899c9f54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:23:41.387084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c6ef3-dfe1-4c13-81bd-e326899c9f54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:41.389194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c6ef3-dfe1-4c13-81bd-e326899c9f54 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:41.395959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a53b0-97bd-41af-a362-6f6ab0a33c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:41.396641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a53b0-97bd-41af-a362-6f6ab0a33c75 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:41.396925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a53b0-97bd-41af-a362-6f6ab0a33c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:41.397202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a53b0-97bd-41af-a362-6f6ab0a33c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:41.406738 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3615/7153] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:23:41.407083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a53b0-97bd-41af-a362-6f6ab0a33c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:41.408986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c6ef3-dfe1-4c13-81bd-e326899c9f54 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:41.422389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c6ef3-dfe1-4c13-81bd-e326899c9f54 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:41.423914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a53b0-97bd-41af-a362-6f6ab0a33c75 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:41.428270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bfe5e-9435-49b1-a8a7-60205e7cc998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:41.429400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfe5e-9435-49b1-a8a7-60205e7cc998 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:41.429861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfe5e-9435-49b1-a8a7-60205e7cc998 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:41.430520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfe5e-9435-49b1-a8a7-60205e7cc998 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:41.431388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bfe5e-9435-49b1-a8a7-60205e7cc998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:41.436025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bfe5e-9435-49b1-a8a7-60205e7cc998 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:41.461194 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3539/7154] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:41.462555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a53b0-97bd-41af-a362-6f6ab0a33c75 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:41.484047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a53b0-97bd-41af-a362-6f6ab0a33c75 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:41.486266 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3616/7155] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:23:41 2022] POST /identity/v3/projects => generated 359 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:41.487692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bfe5e-9435-49b1-a8a7-60205e7cc998 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:41.655778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60fdf2-cab6-4fe1-a3c7-1a854e8d3111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:41.657117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60fdf2-cab6-4fe1-a3c7-1a854e8d3111 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:41.658009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60fdf2-cab6-4fe1-a3c7-1a854e8d3111 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:41.658433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60fdf2-cab6-4fe1-a3c7-1a854e8d3111 admin admin] PATH_INFO: `/v3/projects/a5420998c3564a49ad3fb8537112c319/users/71f2d974bc9a4b7eb6831d5780866090/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:41.661224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bfe5e-9435-49b1-a8a7-60205e7cc998 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:41.672010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9efc3-bfbc-4c04-8b54-ab74600fec34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:41.675416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9efc3-bfbc-4c04-8b54-ab74600fec34 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:41.675569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9efc3-bfbc-4c04-8b54-ab74600fec34 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:41.675844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9efc3-bfbc-4c04-8b54-ab74600fec34 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:41.682508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f9efc3-bfbc-4c04-8b54-ab74600fec34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:41.690312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f9efc3-bfbc-4c04-8b54-ab74600fec34 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:41.712019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60fdf2-cab6-4fe1-a3c7-1a854e8d3111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5420998c3564a49ad3fb8537112c319, user_id=71f2d974bc9a4b7eb6831d5780866090, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:41.716076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60fdf2-cab6-4fe1-a3c7-1a854e8d3111 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:41.729497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:41.756124 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3540/7156] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:23:41 2022] PUT /identity/v3/projects/a5420998c3564a49ad3fb8537112c319/users/71f2d974bc9a4b7eb6831d5780866090/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:41.759146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd490e-dbb0-4b5f-927a-7ab6176e4239 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:41.760259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd490e-dbb0-4b5f-927a-7ab6176e4239 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:41.762169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd490e-dbb0-4b5f-927a-7ab6176e4239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:41.781739 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3617/7157] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:23:41 2022] POST /identity/v3/users => generated 280 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:41.785548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79422f83-91ec-4c53-9421-b392fda08edf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:41.788561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79422f83-91ec-4c53-9421-b392fda08edf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:41.788845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79422f83-91ec-4c53-9421-b392fda08edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:41.856529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cbbd490e-dbb0-4b5f-927a-7ab6176e4239 None None] MFA Rules not processed for user `398f9812de35475c93fb6cfab8f6bf72`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:41.857643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cbbd490e-dbb0-4b5f-927a-7ab6176e4239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:41.863283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-79422f83-91ec-4c53-9421-b392fda08edf None None] MFA Rules not processed for user `71f2d974bc9a4b7eb6831d5780866090`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:41.866129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-79422f83-91ec-4c53-9421-b392fda08edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:42.042069 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3541/7158] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:42.047506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd490e-dbb0-4b5f-927a-7ab6176e4239 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:42.049128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:42.062717 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3618/7159] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:41 2022] POST /identity/v3/auth/tokens => generated 3269 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:42.063870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79422f83-91ec-4c53-9421-b392fda08edf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:42.067187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-79422f83-91ec-4c53-9421-b392fda08edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:42.292170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd490e-dbb0-4b5f-927a-7ab6176e4239 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:42.306014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca82f8-e592-4b20-933c-8fbdf5a5ac62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:42.306914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca82f8-e592-4b20-933c-8fbdf5a5ac62 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:42.307544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca82f8-e592-4b20-933c-8fbdf5a5ac62 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:42.308234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca82f8-e592-4b20-933c-8fbdf5a5ac62 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:42.309074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca82f8-e592-4b20-933c-8fbdf5a5ac62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:42.313445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca82f8-e592-4b20-933c-8fbdf5a5ac62 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:42.315837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79422f83-91ec-4c53-9421-b392fda08edf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:42.331828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad8b80-4120-4265-8924-91c4189c9189 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:42.333469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad8b80-4120-4265-8924-91c4189c9189 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:42.334036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad8b80-4120-4265-8924-91c4189c9189 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:42.334535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad8b80-4120-4265-8924-91c4189c9189 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:42.349294 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3542/7160] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:42.350357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca82f8-e592-4b20-933c-8fbdf5a5ac62 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:42.352690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dfca82f8-e592-4b20-933c-8fbdf5a5ac62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:42.355653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad8b80-4120-4265-8924-91c4189c9189 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:42.357658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad8b80-4120-4265-8924-91c4189c9189 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:42.379889 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3619/7161] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:42.381306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad8b80-4120-4265-8924-91c4189c9189 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:42.397487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad8b80-4120-4265-8924-91c4189c9189 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:42.412157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6c52e-5185-4d16-ac12-021507d65e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:42.413029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6c52e-5185-4d16-ac12-021507d65e77 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:42.414969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6c52e-5185-4d16-ac12-021507d65e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:42.415352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6c52e-5185-4d16-ac12-021507d65e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:42.416114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6c52e-5185-4d16-ac12-021507d65e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:42.419245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6c52e-5185-4d16-ac12-021507d65e77 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:42.445189 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3620/7162] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:42.449648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6c52e-5185-4d16-ac12-021507d65e77 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:42.475147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6c52e-5185-4d16-ac12-021507d65e77 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:42.496783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d70c9-3232-41f4-8ce1-4e8a9d5ec177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:42.497671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d70c9-3232-41f4-8ce1-4e8a9d5ec177 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:42.498291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d70c9-3232-41f4-8ce1-4e8a9d5ec177 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:42.499809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d70c9-3232-41f4-8ce1-4e8a9d5ec177 admin admin] PATH_INFO: `/v3/projects/da32113170f44bce89f6d7db9702150a/users/423adc69e1be461b9a4392e251114e97/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:42.526613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d70c9-3232-41f4-8ce1-4e8a9d5ec177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da32113170f44bce89f6d7db9702150a, user_id=423adc69e1be461b9a4392e251114e97, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:42.529979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d70c9-3232-41f4-8ce1-4e8a9d5ec177 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:42.564844 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3621/7163] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:23:42 2022] PUT /identity/v3/projects/da32113170f44bce89f6d7db9702150a/users/423adc69e1be461b9a4392e251114e97/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:23:42.576087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c4e1a-03e1-42cd-aef9-6cc877646f90 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:42.576687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c4e1a-03e1-42cd-aef9-6cc877646f90 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:42.578885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c4e1a-03e1-42cd-aef9-6cc877646f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:42.589575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca82f8-e592-4b20-933c-8fbdf5a5ac62 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:42.641482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:42.661056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5b1c4e1a-03e1-42cd-aef9-6cc877646f90 None None] MFA Rules not processed for user `423adc69e1be461b9a4392e251114e97`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:42.663937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5b1c4e1a-03e1-42cd-aef9-6cc877646f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:42.782596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c40171-9204-4e78-968f-3760cdefec71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:42.783282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c40171-9204-4e78-968f-3760cdefec71 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:42.783607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c40171-9204-4e78-968f-3760cdefec71 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:42.783912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c40171-9204-4e78-968f-3760cdefec71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:42.890415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c40171-9204-4e78-968f-3760cdefec71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:42.894309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c40171-9204-4e78-968f-3760cdefec71 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:42.914198 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3543/7164] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:23:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 678 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:23:42.932956 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3622/7165] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:42.962173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c40171-9204-4e78-968f-3760cdefec71 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:42.963682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a7c40171-9204-4e78-968f-3760cdefec71 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:43.104966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c40171-9204-4e78-968f-3760cdefec71 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:43.108832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2412162-0987-40e2-b29b-9994ec5c9279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:43.109592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2412162-0987-40e2-b29b-9994ec5c9279 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:43.110039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2412162-0987-40e2-b29b-9994ec5c9279 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:43.110357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2412162-0987-40e2-b29b-9994ec5c9279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:43.133718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2412162-0987-40e2-b29b-9994ec5c9279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:43.135448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2412162-0987-40e2-b29b-9994ec5c9279 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:43.150742 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3544/7166] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:42 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:44.038528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c4e1a-03e1-42cd-aef9-6cc877646f90 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:44.138195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:44.197921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c4e1a-03e1-42cd-aef9-6cc877646f90 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:44.214759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:44.216421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:44.216864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:44.217320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] PATH_INFO: `/v3/users/22a2f69aa5c4469181b411c0ab26f6aa` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:44.218932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=22a2f69aa5c4469181b411c0ab26f6aa)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:44.221268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:44.328043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22a2f69aa5c4469181b411c0ab26f6aa'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:44.344937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22a2f69aa5c4469181b411c0ab26f6aa'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:44.358215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22a2f69aa5c4469181b411c0ab26f6aa'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:44.385904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22a2f69aa5c4469181b411c0ab26f6aa'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:44.386723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22a2f69aa5c4469181b411c0ab26f6aa'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:44.410599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3bd48-66da-4238-9c80-a32f6525f94e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:44.411177 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3623/7167] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:23:44 2022] DELETE /identity/v3/users/22a2f69aa5c4469181b411c0ab26f6aa => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:44.411903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3bd48-66da-4238-9c80-a32f6525f94e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:44.412426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3bd48-66da-4238-9c80-a32f6525f94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:44.470697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1dd3bd48-66da-4238-9c80-a32f6525f94e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:44.472563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1dd3bd48-66da-4238-9c80-a32f6525f94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:44.600213 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3545/7168] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:44.657921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:44.659415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:44.793323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028afb1-26e6-429e-bae3-4b484edc7f3e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:44.806871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c24d91-d9d3-4c3a-a236-36fe0c61373b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:44.808092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24d91-d9d3-4c3a-a236-36fe0c61373b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:44.808395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24d91-d9d3-4c3a-a236-36fe0c61373b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:44.808669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24d91-d9d3-4c3a-a236-36fe0c61373b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:44.835566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c24d91-d9d3-4c3a-a236-36fe0c61373b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:44.836854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c24d91-d9d3-4c3a-a236-36fe0c61373b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:44.861445 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3624/7169] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:47.088182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3bd48-66da-4238-9c80-a32f6525f94e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:47.091302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1dd3bd48-66da-4238-9c80-a32f6525f94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:47.110417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3bd48-66da-4238-9c80-a32f6525f94e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:47.137582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:47.138763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:47.139225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:47.139666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] PATH_INFO: `/v3/projects/e670ad3c4f114e188e152858d4756db7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:47.141216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=e670ad3c4f114e188e152858d4756db7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:47.143551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:47.145586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:47.288190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e670ad3c4f114e188e152858d4756db7', 'request_id': 'req-8a0850d6-b0cb-4278-a1e4-8875867382ca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:47.300978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e670ad3c4f114e188e152858d4756db7', 'request_id': 'req-8a0850d6-b0cb-4278-a1e4-8875867382ca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:47.325126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e670ad3c4f114e188e152858d4756db7', 'request_id': 'req-8a0850d6-b0cb-4278-a1e4-8875867382ca'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:47.345931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] The token cache is being invalidate because project e670ad3c4f114e188e152858d4756db7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:23:47.346497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:47.347407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:47.361993 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3546/7170] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:23:47 2022] DELETE /identity/v3/projects/e670ad3c4f114e188e152858d4756db7 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:47.366358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c24d91-d9d3-4c3a-a236-36fe0c61373b service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:47.380232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c24d91-d9d3-4c3a-a236-36fe0c61373b service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:47.520749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4e2be-8848-4aed-860a-5cfa090029e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:47.521787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4e2be-8848-4aed-860a-5cfa090029e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:47.522210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4e2be-8848-4aed-860a-5cfa090029e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:47.522652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4e2be-8848-4aed-860a-5cfa090029e0 admin admin] PATH_INFO: `/v3/users/71f2d974bc9a4b7eb6831d5780866090` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:47.524135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4e2be-8848-4aed-860a-5cfa090029e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71f2d974bc9a4b7eb6831d5780866090)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:47.526425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4e2be-8848-4aed-860a-5cfa090029e0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:47.531421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:47.626444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-22d4e2be-8848-4aed-860a-5cfa090029e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71f2d974bc9a4b7eb6831d5780866090'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:47.666178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-22d4e2be-8848-4aed-860a-5cfa090029e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71f2d974bc9a4b7eb6831d5780866090'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:47.691945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-22d4e2be-8848-4aed-860a-5cfa090029e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71f2d974bc9a4b7eb6831d5780866090'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:47.727273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-22d4e2be-8848-4aed-860a-5cfa090029e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71f2d974bc9a4b7eb6831d5780866090'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:47.728173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-22d4e2be-8848-4aed-860a-5cfa090029e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71f2d974bc9a4b7eb6831d5780866090'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:47.740963 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3625/7171] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:23:47 2022] DELETE /identity/v3/users/71f2d974bc9a4b7eb6831d5780866090 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:48.599330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:48.603181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:48.643339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0850d6-b0cb-4278-a1e4-8875867382ca admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:48.794036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453f83e-2826-410d-9f65-a29facab989e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:48.795058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453f83e-2826-410d-9f65-a29facab989e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:48.795522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453f83e-2826-410d-9f65-a29facab989e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:48.804405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453f83e-2826-410d-9f65-a29facab989e admin admin] PATH_INFO: `/v3/projects/a5420998c3564a49ad3fb8537112c319` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:48.811761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453f83e-2826-410d-9f65-a29facab989e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5420998c3564a49ad3fb8537112c319)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:48.814237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453f83e-2826-410d-9f65-a29facab989e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:48.925278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a453f83e-2826-410d-9f65-a29facab989e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5420998c3564a49ad3fb8537112c319', 'request_id': 'req-a453f83e-2826-410d-9f65-a29facab989e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:48.943093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a453f83e-2826-410d-9f65-a29facab989e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5420998c3564a49ad3fb8537112c319', 'request_id': 'req-a453f83e-2826-410d-9f65-a29facab989e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:48.956707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a453f83e-2826-410d-9f65-a29facab989e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5420998c3564a49ad3fb8537112c319', 'request_id': 'req-a453f83e-2826-410d-9f65-a29facab989e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:48.973873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a453f83e-2826-410d-9f65-a29facab989e admin admin] The token cache is being invalidate because project a5420998c3564a49ad3fb8537112c319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:23:48.974405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a453f83e-2826-410d-9f65-a29facab989e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:48.975624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a453f83e-2826-410d-9f65-a29facab989e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:48.978330 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3547/7172] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:23:48 2022] DELETE /identity/v3/projects/a5420998c3564a49ad3fb8537112c319 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:48.983333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aebec-7ad6-4b9b-a449-8636d0f32be5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:48.983827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aebec-7ad6-4b9b-a449-8636d0f32be5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:48.983926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aebec-7ad6-4b9b-a449-8636d0f32be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:49.023287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d50aebec-7ad6-4b9b-a449-8636d0f32be5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:49.024835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d50aebec-7ad6-4b9b-a449-8636d0f32be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:49.149392 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3626/7173] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:49.156787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8d7f4-cf44-4d35-81f7-17322632eb4b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:49.157108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8d7f4-cf44-4d35-81f7-17322632eb4b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:49.157336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8d7f4-cf44-4d35-81f7-17322632eb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:49.180119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cea8d7f4-cf44-4d35-81f7-17322632eb4b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:49.181556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cea8d7f4-cf44-4d35-81f7-17322632eb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:49.195857 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3548/7174] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:49.197145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aebec-7ad6-4b9b-a449-8636d0f32be5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:49.214623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aebec-7ad6-4b9b-a449-8636d0f32be5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:49.233922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6a6a5-8245-48d4-9eab-d3b0f833aec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:49.234617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6a6a5-8245-48d4-9eab-d3b0f833aec2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:49.235531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6a6a5-8245-48d4-9eab-d3b0f833aec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:49.235814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6a6a5-8245-48d4-9eab-d3b0f833aec2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:49.241299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6a6a5-8245-48d4-9eab-d3b0f833aec2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:23:49.241740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6a6a5-8245-48d4-9eab-d3b0f833aec2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:49.243722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6a6a5-8245-48d4-9eab-d3b0f833aec2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:49.262277 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3627/7175] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:23:49.266749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8d7f4-cf44-4d35-81f7-17322632eb4b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:49.282288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8d7f4-cf44-4d35-81f7-17322632eb4b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:49.303806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee1c03-e1fd-4049-9fa7-07952fa9e290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:49.304700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee1c03-e1fd-4049-9fa7-07952fa9e290 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:49.304986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee1c03-e1fd-4049-9fa7-07952fa9e290 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:49.305315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee1c03-e1fd-4049-9fa7-07952fa9e290 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:49.306950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee1c03-e1fd-4049-9fa7-07952fa9e290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:49.312247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee1c03-e1fd-4049-9fa7-07952fa9e290 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:49.332145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:49.368011 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3549/7176] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:23:49 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:23:49.371519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6a6a5-8245-48d4-9eab-d3b0f833aec2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:49.389255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6a6a5-8245-48d4-9eab-d3b0f833aec2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:49.501502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d72c4-9c75-431e-a99b-f1a7976182e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:49.502636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d72c4-9c75-431e-a99b-f1a7976182e6 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:49.503187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d72c4-9c75-431e-a99b-f1a7976182e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:49.503768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d72c4-9c75-431e-a99b-f1a7976182e6 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:49.504669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d72c4-9c75-431e-a99b-f1a7976182e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:49.507788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d72c4-9c75-431e-a99b-f1a7976182e6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:49.542891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:49.562376 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3628/7177] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:23:49 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:49.563717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee1c03-e1fd-4049-9fa7-07952fa9e290 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:49.583485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee1c03-e1fd-4049-9fa7-07952fa9e290 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:49.591265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fdf2f-f66d-4469-a575-5d149d8b9f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:49.592054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fdf2f-f66d-4469-a575-5d149d8b9f08 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:49.592449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fdf2f-f66d-4469-a575-5d149d8b9f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:49.592979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fdf2f-f66d-4469-a575-5d149d8b9f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:49.594971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fdf2f-f66d-4469-a575-5d149d8b9f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:49.600878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fdf2f-f66d-4469-a575-5d149d8b9f08 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:49.642999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d72c4-9c75-431e-a99b-f1a7976182e6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:49.644218 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3550/7178] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:49.666700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d72c4-9c75-431e-a99b-f1a7976182e6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:49.672015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319430fe-24b5-4c95-9e38-a707f408c996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:49.672732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319430fe-24b5-4c95-9e38-a707f408c996 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:49.673076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319430fe-24b5-4c95-9e38-a707f408c996 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:49.673411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319430fe-24b5-4c95-9e38-a707f408c996 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:49.675047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319430fe-24b5-4c95-9e38-a707f408c996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:49.676789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319430fe-24b5-4c95-9e38-a707f408c996 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:49.712420 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3629/7179] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:49.716254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fdf2f-f66d-4469-a575-5d149d8b9f08 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:49.741906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fdf2f-f66d-4469-a575-5d149d8b9f08 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:49.752451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da28abb-4b9c-4699-946c-228328cd0095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:49.754504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da28abb-4b9c-4699-946c-228328cd0095 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:49.754811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da28abb-4b9c-4699-946c-228328cd0095 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:49.755086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da28abb-4b9c-4699-946c-228328cd0095 admin admin] PATH_INFO: `/v3/projects/5e7dba88703b48f38215fde76a0fb80a/users/15c7508ede01463b9494100198a8d3a0/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:49.784796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da28abb-4b9c-4699-946c-228328cd0095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e7dba88703b48f38215fde76a0fb80a, user_id=15c7508ede01463b9494100198a8d3a0, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:49.792187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da28abb-4b9c-4699-946c-228328cd0095 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:49.822089 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3551/7180] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:23:49 2022] PUT /identity/v3/projects/5e7dba88703b48f38215fde76a0fb80a/users/15c7508ede01463b9494100198a8d3a0/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:23:49.822733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd972d-1afd-4c7f-85fc-90025b3c5890 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:49.825259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd972d-1afd-4c7f-85fc-90025b3c5890 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:49.826001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd972d-1afd-4c7f-85fc-90025b3c5890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:49.872770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-57dd972d-1afd-4c7f-85fc-90025b3c5890 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:49.874192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-57dd972d-1afd-4c7f-85fc-90025b3c5890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:50.006520 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3630/7181] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:50.033031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da28abb-4b9c-4699-946c-228328cd0095 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:50.035720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8da28abb-4b9c-4699-946c-228328cd0095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:50.164950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da28abb-4b9c-4699-946c-228328cd0095 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:50.171836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad7a2b-1fd5-426c-aa6a-b220f55d510e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:50.172780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad7a2b-1fd5-426c-aa6a-b220f55d510e service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:50.174051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad7a2b-1fd5-426c-aa6a-b220f55d510e service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:50.174353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad7a2b-1fd5-426c-aa6a-b220f55d510e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:50.192546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad7a2b-1fd5-426c-aa6a-b220f55d510e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:50.194597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad7a2b-1fd5-426c-aa6a-b220f55d510e service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:50.211273 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3552/7182] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:54.580009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd972d-1afd-4c7f-85fc-90025b3c5890 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:54.581813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-57dd972d-1afd-4c7f-85fc-90025b3c5890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:54.613118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd972d-1afd-4c7f-85fc-90025b3c5890 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:54.621439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912e758-5956-49db-95db-e0a272b2153b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:54.622199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912e758-5956-49db-95db-e0a272b2153b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:54.622562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912e758-5956-49db-95db-e0a272b2153b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:54.622955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912e758-5956-49db-95db-e0a272b2153b admin admin] PATH_INFO: `/v3/users/398f9812de35475c93fb6cfab8f6bf72` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:54.624449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912e758-5956-49db-95db-e0a272b2153b admin admin] RBAC: Authorizing `identity:delete_user(user_id=398f9812de35475c93fb6cfab8f6bf72)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:54.626652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912e758-5956-49db-95db-e0a272b2153b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:54.718838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4912e758-5956-49db-95db-e0a272b2153b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '398f9812de35475c93fb6cfab8f6bf72'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:54.731260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4912e758-5956-49db-95db-e0a272b2153b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '398f9812de35475c93fb6cfab8f6bf72'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:54.753886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4912e758-5956-49db-95db-e0a272b2153b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '398f9812de35475c93fb6cfab8f6bf72'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:54.775392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4912e758-5956-49db-95db-e0a272b2153b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '398f9812de35475c93fb6cfab8f6bf72'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:54.776250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4912e758-5956-49db-95db-e0a272b2153b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '398f9812de35475c93fb6cfab8f6bf72'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:54.785103 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3631/7183] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:23:54 2022] DELETE /identity/v3/users/398f9812de35475c93fb6cfab8f6bf72 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:55.110741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad7a2b-1fd5-426c-aa6a-b220f55d510e service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:55.133181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad7a2b-1fd5-426c-aa6a-b220f55d510e service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:55.244611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:55.245344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:55.245766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:55.246040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3 admin admin] PATH_INFO: `/v3/projects/3812d5019f224d889ea890ba9045245a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:55.250442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3812d5019f224d889ea890ba9045245a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:55.256110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:55.383419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3812d5019f224d889ea890ba9045245a', 'request_id': 'req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:55.404954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3812d5019f224d889ea890ba9045245a', 'request_id': 'req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:55.414967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3812d5019f224d889ea890ba9045245a', 'request_id': 'req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:55.417497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:55.434786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3 admin admin] The token cache is being invalidate because project 3812d5019f224d889ea890ba9045245a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:23:55.435670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:55.436716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f655ab4-a6c6-44b9-b12f-d5d4d57dd3f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:55.441797 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3553/7184] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:23:55 2022] DELETE /identity/v3/projects/3812d5019f224d889ea890ba9045245a => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:55.447059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e43c05-b372-4e1e-9ab4-b19b46edb853 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:55.447882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e43c05-b372-4e1e-9ab4-b19b46edb853 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:55.448311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e43c05-b372-4e1e-9ab4-b19b46edb853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:55.494830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b9e43c05-b372-4e1e-9ab4-b19b46edb853 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:55.496593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b9e43c05-b372-4e1e-9ab4-b19b46edb853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:55.692031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:55.708292 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3632/7185] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:55.718115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ca1fb-8119-4971-a667-9b5504e8dc74 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:55.718115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ca1fb-8119-4971-a667-9b5504e8dc74 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:55.718396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ca1fb-8119-4971-a667-9b5504e8dc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:55.762545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-861ca1fb-8119-4971-a667-9b5504e8dc74 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:55.763844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-861ca1fb-8119-4971-a667-9b5504e8dc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:55.787638 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3554/7186] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:55.792519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e43c05-b372-4e1e-9ab4-b19b46edb853 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:55.813736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e43c05-b372-4e1e-9ab4-b19b46edb853 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:55.826316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c232a1b-260a-41a9-a83c-e82c5eec0241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:55.826960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c232a1b-260a-41a9-a83c-e82c5eec0241 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:55.827231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c232a1b-260a-41a9-a83c-e82c5eec0241 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:55.827497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c232a1b-260a-41a9-a83c-e82c5eec0241 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:55.836353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c232a1b-260a-41a9-a83c-e82c5eec0241 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:23:55.836734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c232a1b-260a-41a9-a83c-e82c5eec0241 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:55.838353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c232a1b-260a-41a9-a83c-e82c5eec0241 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:55.882522 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3633/7187] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:23:55.887004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ca1fb-8119-4971-a667-9b5504e8dc74 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:55.903309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ca1fb-8119-4971-a667-9b5504e8dc74 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:55.930337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552f496-5d1c-4ba5-b17d-cd2896d7f3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:55.931476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552f496-5d1c-4ba5-b17d-cd2896d7f3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:55.931929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552f496-5d1c-4ba5-b17d-cd2896d7f3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:55.932266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552f496-5d1c-4ba5-b17d-cd2896d7f3e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:55.932946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552f496-5d1c-4ba5-b17d-cd2896d7f3e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:55.938845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552f496-5d1c-4ba5-b17d-cd2896d7f3e5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:56.013090 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3555/7188] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:23:55 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:56.014856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c232a1b-260a-41a9-a83c-e82c5eec0241 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:56.027321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c232a1b-260a-41a9-a83c-e82c5eec0241 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:56.112024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff26ffe-4f9b-4210-acfa-bf88e6840b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:56.112696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff26ffe-4f9b-4210-acfa-bf88e6840b93 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:56.113023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff26ffe-4f9b-4210-acfa-bf88e6840b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:56.113301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff26ffe-4f9b-4210-acfa-bf88e6840b93 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:56.114391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff26ffe-4f9b-4210-acfa-bf88e6840b93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:56.117689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff26ffe-4f9b-4210-acfa-bf88e6840b93 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:56.160533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552f496-5d1c-4ba5-b17d-cd2896d7f3e5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:56.167959 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3634/7189] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:23:56 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:23:56.173251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552f496-5d1c-4ba5-b17d-cd2896d7f3e5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:56.182671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32e395-9d41-46a0-9299-b5d36e81b74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:56.183304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e395-9d41-46a0-9299-b5d36e81b74a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:56.183585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e395-9d41-46a0-9299-b5d36e81b74a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:56.183895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e395-9d41-46a0-9299-b5d36e81b74a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:56.184724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32e395-9d41-46a0-9299-b5d36e81b74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:56.186635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32e395-9d41-46a0-9299-b5d36e81b74a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:56.214723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff26ffe-4f9b-4210-acfa-bf88e6840b93 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:56.230313 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3556/7190] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:56.235652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff26ffe-4f9b-4210-acfa-bf88e6840b93 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:56.241984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c61bdf-b6cf-4bd0-a182-cad40f16539e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:56.242613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c61bdf-b6cf-4bd0-a182-cad40f16539e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:56.242888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c61bdf-b6cf-4bd0-a182-cad40f16539e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:56.243255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c61bdf-b6cf-4bd0-a182-cad40f16539e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:56.244255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c61bdf-b6cf-4bd0-a182-cad40f16539e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:56.245907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c61bdf-b6cf-4bd0-a182-cad40f16539e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:56.262863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32e395-9d41-46a0-9299-b5d36e81b74a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:56.270956 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3635/7191] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:56.276294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32e395-9d41-46a0-9299-b5d36e81b74a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:56.281890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63943009-0ccb-436d-b439-cc18d64df4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:56.282820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63943009-0ccb-436d-b439-cc18d64df4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:56.283102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63943009-0ccb-436d-b439-cc18d64df4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:56.283510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63943009-0ccb-436d-b439-cc18d64df4c1 admin admin] PATH_INFO: `/v3/projects/0dd98141e3a44c969aa5cd38d24e0dbd/users/d5c97cfa7d924c3491f9e78b5051f0f8/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:56.305519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63943009-0ccb-436d-b439-cc18d64df4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd98141e3a44c969aa5cd38d24e0dbd, user_id=d5c97cfa7d924c3491f9e78b5051f0f8, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:56.309539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63943009-0ccb-436d-b439-cc18d64df4c1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:56.331063 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3557/7192] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:23:56 2022] PUT /identity/v3/projects/0dd98141e3a44c969aa5cd38d24e0dbd/users/d5c97cfa7d924c3491f9e78b5051f0f8/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:56.336259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4aeb2c-62e8-453b-856a-847280c2c337 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:56.336584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4aeb2c-62e8-453b-856a-847280c2c337 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:56.336882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4aeb2c-62e8-453b-856a-847280c2c337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:56.369099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6c4aeb2c-62e8-453b-856a-847280c2c337 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:56.370463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6c4aeb2c-62e8-453b-856a-847280c2c337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:56.462309 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3636/7193] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:56.476336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63943009-0ccb-436d-b439-cc18d64df4c1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:56.479674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-63943009-0ccb-436d-b439-cc18d64df4c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:56.568248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63943009-0ccb-436d-b439-cc18d64df4c1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:56.572707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f457b50-cb32-4182-8ba8-feeec38ea9ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:56.573651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f457b50-cb32-4182-8ba8-feeec38ea9ad service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:56.574074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f457b50-cb32-4182-8ba8-feeec38ea9ad service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:56.574487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f457b50-cb32-4182-8ba8-feeec38ea9ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:56.589051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f457b50-cb32-4182-8ba8-feeec38ea9ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:56.591231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f457b50-cb32-4182-8ba8-feeec38ea9ad service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:56.606741 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3558/7194] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:56.664323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80862b69-01f5-4830-8158-cebf96f84826 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:56.664708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80862b69-01f5-4830-8158-cebf96f84826 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:56.664978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80862b69-01f5-4830-8158-cebf96f84826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:56.698073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-80862b69-01f5-4830-8158-cebf96f84826 None None] MFA Rules not processed for user `4e6a5f35d172410481a8416b7bdaafac`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:56.701371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-80862b69-01f5-4830-8158-cebf96f84826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:56.857559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f457b50-cb32-4182-8ba8-feeec38ea9ad service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:56.859274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5f457b50-cb32-4182-8ba8-feeec38ea9ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:56.882132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f457b50-cb32-4182-8ba8-feeec38ea9ad service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:56.895869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:56.896553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:56.896875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:56.899115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] PATH_INFO: `/v3/users/ed95105bb93f4098b5a9e023c0057896` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:56.899588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed95105bb93f4098b5a9e023c0057896)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:56.901828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:56.944573 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3637/7195] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:57.006756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed95105bb93f4098b5a9e023c0057896'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:57.024659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed95105bb93f4098b5a9e023c0057896'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:57.051140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed95105bb93f4098b5a9e023c0057896'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:57.054106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed95105bb93f4098b5a9e023c0057896'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:57.054928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed95105bb93f4098b5a9e023c0057896'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:57.071033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:57.079411 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3559/7196] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:23:56 2022] DELETE /identity/v3/users/ed95105bb93f4098b5a9e023c0057896 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:57.084802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fbfd1-401a-4af6-994b-6b7c3cf2ec01 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:57.085207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fbfd1-401a-4af6-994b-6b7c3cf2ec01 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:57.085663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fbfd1-401a-4af6-994b-6b7c3cf2ec01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:57.118069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-278fbfd1-401a-4af6-994b-6b7c3cf2ec01 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:57.121687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-278fbfd1-401a-4af6-994b-6b7c3cf2ec01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:57.128653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:57.233528 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3638/7197] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:57.273104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:57.274344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:57.366829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981e7a0-68db-4ad6-8589-21726cda42c1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:57.370801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875fe25-b7fa-414b-be71-24cbcfa3c458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:57.372056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875fe25-b7fa-414b-be71-24cbcfa3c458 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:57.372713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875fe25-b7fa-414b-be71-24cbcfa3c458 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:57.373033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875fe25-b7fa-414b-be71-24cbcfa3c458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:57.404493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875fe25-b7fa-414b-be71-24cbcfa3c458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:57.408235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875fe25-b7fa-414b-be71-24cbcfa3c458 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:57.422768 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3560/7198] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:57.787704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fbfd1-401a-4af6-994b-6b7c3cf2ec01 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:57.789174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-278fbfd1-401a-4af6-994b-6b7c3cf2ec01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:57.825910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fbfd1-401a-4af6-994b-6b7c3cf2ec01 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:57.832496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f7534-d1c8-4f7c-b88a-397405369fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:57.833270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f7534-d1c8-4f7c-b88a-397405369fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:57.833608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f7534-d1c8-4f7c-b88a-397405369fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:57.833968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f7534-d1c8-4f7c-b88a-397405369fd1 admin admin] PATH_INFO: `/v3/projects/86fdb0fc3bc24270a68fe7e7c4cbb653` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:57.836413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f7534-d1c8-4f7c-b88a-397405369fd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86fdb0fc3bc24270a68fe7e7c4cbb653)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:57.838199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f7534-d1c8-4f7c-b88a-397405369fd1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:57.895922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-467f7534-d1c8-4f7c-b88a-397405369fd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86fdb0fc3bc24270a68fe7e7c4cbb653', 'request_id': 'req-467f7534-d1c8-4f7c-b88a-397405369fd1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:57.905470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-467f7534-d1c8-4f7c-b88a-397405369fd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86fdb0fc3bc24270a68fe7e7c4cbb653', 'request_id': 'req-467f7534-d1c8-4f7c-b88a-397405369fd1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:57.913232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-467f7534-d1c8-4f7c-b88a-397405369fd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86fdb0fc3bc24270a68fe7e7c4cbb653', 'request_id': 'req-467f7534-d1c8-4f7c-b88a-397405369fd1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:57.929318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-467f7534-d1c8-4f7c-b88a-397405369fd1 admin admin] The token cache is being invalidate because project 86fdb0fc3bc24270a68fe7e7c4cbb653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:23:57.929703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-467f7534-d1c8-4f7c-b88a-397405369fd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:57.930609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-467f7534-d1c8-4f7c-b88a-397405369fd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:23:57.932493 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3639/7199] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:23:57 2022] DELETE /identity/v3/projects/86fdb0fc3bc24270a68fe7e7c4cbb653 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:57.937282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45c046-abb9-44c9-861b-fa0b3722c373 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:57.938103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45c046-abb9-44c9-861b-fa0b3722c373 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:57.938562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45c046-abb9-44c9-861b-fa0b3722c373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:57.967872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4a45c046-abb9-44c9-861b-fa0b3722c373 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:57.969647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4a45c046-abb9-44c9-861b-fa0b3722c373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:58.045324 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3561/7200] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:23:58.053720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020602b-fb77-4444-8487-17a9b04fa1a0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:58.054070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020602b-fb77-4444-8487-17a9b04fa1a0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:58.054420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020602b-fb77-4444-8487-17a9b04fa1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:58.078543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5020602b-fb77-4444-8487-17a9b04fa1a0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:58.080116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5020602b-fb77-4444-8487-17a9b04fa1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:58.089981 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3640/7201] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:58.094125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45c046-abb9-44c9-861b-fa0b3722c373 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:58.118970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45c046-abb9-44c9-861b-fa0b3722c373 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:58.129516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4655f82-b770-4f0c-b0ba-1473b481a5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:58.130147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4655f82-b770-4f0c-b0ba-1473b481a5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:58.130889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4655f82-b770-4f0c-b0ba-1473b481a5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:58.130889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4655f82-b770-4f0c-b0ba-1473b481a5f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:58.131459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4655f82-b770-4f0c-b0ba-1473b481a5f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:23:58.131754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4655f82-b770-4f0c-b0ba-1473b481a5f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:58.133399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4655f82-b770-4f0c-b0ba-1473b481a5f2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:58.179191 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3562/7202] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:23:58.180003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020602b-fb77-4444-8487-17a9b04fa1a0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:58.191278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020602b-fb77-4444-8487-17a9b04fa1a0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:58.206551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086acf5-36a7-4fd7-8588-b9dde95f9bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:58.207749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086acf5-36a7-4fd7-8588-b9dde95f9bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:58.207966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086acf5-36a7-4fd7-8588-b9dde95f9bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:58.208226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086acf5-36a7-4fd7-8588-b9dde95f9bf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:58.208952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086acf5-36a7-4fd7-8588-b9dde95f9bf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:58.213000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086acf5-36a7-4fd7-8588-b9dde95f9bf0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:58.280658 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3641/7203] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:23:58 2022] POST /identity/v3/projects => generated 361 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:58.285778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4655f82-b770-4f0c-b0ba-1473b481a5f2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:58.314005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4655f82-b770-4f0c-b0ba-1473b481a5f2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:58.462404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dd2f2-5a77-4b09-a4c8-41e241d47a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:58.463175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dd2f2-5a77-4b09-a4c8-41e241d47a7f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:58.463595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dd2f2-5a77-4b09-a4c8-41e241d47a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:58.463999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dd2f2-5a77-4b09-a4c8-41e241d47a7f admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:58.465075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dd2f2-5a77-4b09-a4c8-41e241d47a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:58.467560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dd2f2-5a77-4b09-a4c8-41e241d47a7f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:58.511757 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3563/7204] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:23:58 2022] POST /identity/v3/users => generated 281 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:23:58.515818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086acf5-36a7-4fd7-8588-b9dde95f9bf0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:58.530266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086acf5-36a7-4fd7-8588-b9dde95f9bf0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:58.541890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeda682-7305-48a9-9da0-d11d1346c7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:58.542735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeda682-7305-48a9-9da0-d11d1346c7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:58.542842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeda682-7305-48a9-9da0-d11d1346c7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:58.543089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeda682-7305-48a9-9da0-d11d1346c7c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:58.543729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeda682-7305-48a9-9da0-d11d1346c7c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:58.545981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dd2f2-5a77-4b09-a4c8-41e241d47a7f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:58.547728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a83dd2f2-5a77-4b09-a4c8-41e241d47a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:58.559765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeda682-7305-48a9-9da0-d11d1346c7c7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:58.610572 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3642/7205] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:23:58.611762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeda682-7305-48a9-9da0-d11d1346c7c7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:58.629586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:58.649991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeda682-7305-48a9-9da0-d11d1346c7c7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:58.668600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42041f4-383f-4dab-ae38-b797292ff8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:58.670908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42041f4-383f-4dab-ae38-b797292ff8dc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:58.671220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42041f4-383f-4dab-ae38-b797292ff8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:58.671594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42041f4-383f-4dab-ae38-b797292ff8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:58.673307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42041f4-383f-4dab-ae38-b797292ff8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:58.678208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42041f4-383f-4dab-ae38-b797292ff8dc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:58.719617 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3643/7206] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:23:58.727358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42041f4-383f-4dab-ae38-b797292ff8dc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:58.762580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42041f4-383f-4dab-ae38-b797292ff8dc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:58.769604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9cf77-2b40-465c-95a9-ba4ef432a948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:58.770881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9cf77-2b40-465c-95a9-ba4ef432a948 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:58.771507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9cf77-2b40-465c-95a9-ba4ef432a948 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:58.771930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9cf77-2b40-465c-95a9-ba4ef432a948 admin admin] PATH_INFO: `/v3/projects/b16c88efcdbe4c4cbcf4cb1b00ec5579/users/8d92757a1ebe4019829c27b685ece14c/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:58.781097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dd2f2-5a77-4b09-a4c8-41e241d47a7f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:58.794766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9cf77-2b40-465c-95a9-ba4ef432a948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16c88efcdbe4c4cbcf4cb1b00ec5579, user_id=8d92757a1ebe4019829c27b685ece14c, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:58.796708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6993713-3c45-4c97-92dd-a8dd05cf5578 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:58.797343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6993713-3c45-4c97-92dd-a8dd05cf5578 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:58.797623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6993713-3c45-4c97-92dd-a8dd05cf5578 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:58.797894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6993713-3c45-4c97-92dd-a8dd05cf5578 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:58.799822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9cf77-2b40-465c-95a9-ba4ef432a948 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:58.805041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b6993713-3c45-4c97-92dd-a8dd05cf5578 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:58.832813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6993713-3c45-4c97-92dd-a8dd05cf5578 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:58.837881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6993713-3c45-4c97-92dd-a8dd05cf5578 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:58.841592 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3644/7207] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:23:58 2022] PUT /identity/v3/projects/b16c88efcdbe4c4cbcf4cb1b00ec5579/users/8d92757a1ebe4019829c27b685ece14c/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:23:58.853672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fecd91-5540-4593-86a9-6ea6a7853ac7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:58.854449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fecd91-5540-4593-86a9-6ea6a7853ac7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:58.855064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fecd91-5540-4593-86a9-6ea6a7853ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:58.869178 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3564/7208] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:23:58.927391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a4fecd91-5540-4593-86a9-6ea6a7853ac7 None None] MFA Rules not processed for user `8d92757a1ebe4019829c27b685ece14c`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:58.929220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a4fecd91-5540-4593-86a9-6ea6a7853ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:58.942578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:23:59.002561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6993713-3c45-4c97-92dd-a8dd05cf5578 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:59.027814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6993713-3c45-4c97-92dd-a8dd05cf5578 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:59.030238 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3645/7209] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:59.031738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320248cc-c433-4524-a3b1-071d630f0800 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:59.032018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320248cc-c433-4524-a3b1-071d630f0800 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:59.032284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320248cc-c433-4524-a3b1-071d630f0800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:59.055179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-320248cc-c433-4524-a3b1-071d630f0800 None None] MFA Rules not processed for user `4e6a5f35d172410481a8416b7bdaafac`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:23:59.056442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-320248cc-c433-4524-a3b1-071d630f0800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:59.143439 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3646/7210] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:23:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:23:59.144566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320248cc-c433-4524-a3b1-071d630f0800 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:59.145889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-320248cc-c433-4524-a3b1-071d630f0800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:59.156796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68630644-5eab-4903-b67f-16d90fc471d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:59.157737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68630644-5eab-4903-b67f-16d90fc471d7 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:59.158254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68630644-5eab-4903-b67f-16d90fc471d7 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:59.158686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68630644-5eab-4903-b67f-16d90fc471d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:59.160785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-68630644-5eab-4903-b67f-16d90fc471d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:59.190220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68630644-5eab-4903-b67f-16d90fc471d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:59.192017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68630644-5eab-4903-b67f-16d90fc471d7 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:59.219359 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3565/7211] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:23:59.275867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320248cc-c433-4524-a3b1-071d630f0800 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:59.287419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52181b-0f50-4470-b445-bb0c1af89347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:59.288069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52181b-0f50-4470-b445-bb0c1af89347 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:59.288298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52181b-0f50-4470-b445-bb0c1af89347 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:59.288591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52181b-0f50-4470-b445-bb0c1af89347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:59.317138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52181b-0f50-4470-b445-bb0c1af89347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:23:59.332499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52181b-0f50-4470-b445-bb0c1af89347 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:23:59.367544 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3647/7212] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:23:59.950682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68630644-5eab-4903-b67f-16d90fc471d7 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:23:59.956587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-68630644-5eab-4903-b67f-16d90fc471d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:23:59.975750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68630644-5eab-4903-b67f-16d90fc471d7 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:23:59.983453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd3e90-ba58-4764-b973-bb68cf3c1636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:23:59.984069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd3e90-ba58-4764-b973-bb68cf3c1636 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:23:59.984440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd3e90-ba58-4764-b973-bb68cf3c1636 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:23:59.984634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd3e90-ba58-4764-b973-bb68cf3c1636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:23:59.991780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dbdd3e90-ba58-4764-b973-bb68cf3c1636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:00.015062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd3e90-ba58-4764-b973-bb68cf3c1636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:00.017292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd3e90-ba58-4764-b973-bb68cf3c1636 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:00.039638 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3566/7213] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:23:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:01.337317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc17a3-4888-45ce-8c3f-b9c5f4b2a6fe None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:01.338628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc17a3-4888-45ce-8c3f-b9c5f4b2a6fe None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:01.339231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc17a3-4888-45ce-8c3f-b9c5f4b2a6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:01.386059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c7bc17a3-4888-45ce-8c3f-b9c5f4b2a6fe None None] MFA Rules not processed for user `15c7508ede01463b9494100198a8d3a0`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:01.388286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c7bc17a3-4888-45ce-8c3f-b9c5f4b2a6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:01.496832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:01.540140 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3648/7214] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:01.546490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd3e90-ba58-4764-b973-bb68cf3c1636 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:01.547786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dbdd3e90-ba58-4764-b973-bb68cf3c1636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:01.693935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd3e90-ba58-4764-b973-bb68cf3c1636 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:01.701547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42f3a2-3276-4aaa-9b96-7d5fbd8c3099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:01.702950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42f3a2-3276-4aaa-9b96-7d5fbd8c3099 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:01.703532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42f3a2-3276-4aaa-9b96-7d5fbd8c3099 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:01.704145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42f3a2-3276-4aaa-9b96-7d5fbd8c3099 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:01.705453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42f3a2-3276-4aaa-9b96-7d5fbd8c3099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:01.708050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42f3a2-3276-4aaa-9b96-7d5fbd8c3099 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:01.781482 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3567/7215] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:01 2022] POST /identity/v3/projects => generated 371 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:24:01.786070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc17a3-4888-45ce-8c3f-b9c5f4b2a6fe None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:01.798545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc17a3-4888-45ce-8c3f-b9c5f4b2a6fe None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:01.901011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1146940-b180-464b-bf57-386e4d5bf657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:01.901795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1146940-b180-464b-bf57-386e4d5bf657 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:01.902329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1146940-b180-464b-bf57-386e4d5bf657 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:01.902885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1146940-b180-464b-bf57-386e4d5bf657 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:01.903737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1146940-b180-464b-bf57-386e4d5bf657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:01.906241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1146940-b180-464b-bf57-386e4d5bf657 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:01.952640 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3649/7216] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:01 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:24:01.957238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42f3a2-3276-4aaa-9b96-7d5fbd8c3099 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:01.975051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42f3a2-3276-4aaa-9b96-7d5fbd8c3099 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:01.988789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545885a6-eca2-42bd-98ca-11acb6995b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:01.990720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545885a6-eca2-42bd-98ca-11acb6995b5d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:01.990991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545885a6-eca2-42bd-98ca-11acb6995b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:01.992263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545885a6-eca2-42bd-98ca-11acb6995b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:01.994454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545885a6-eca2-42bd-98ca-11acb6995b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:01.995861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545885a6-eca2-42bd-98ca-11acb6995b5d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:02.003746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:02.016828 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3568/7217] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:02.021048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1146940-b180-464b-bf57-386e4d5bf657 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:02.043381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1146940-b180-464b-bf57-386e4d5bf657 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:02.050686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf50e19-38fd-4d6e-b96f-f6f6ab804f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:02.052897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf50e19-38fd-4d6e-b96f-f6f6ab804f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:02.053527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf50e19-38fd-4d6e-b96f-f6f6ab804f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:02.054018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf50e19-38fd-4d6e-b96f-f6f6ab804f45 admin admin] PATH_INFO: `/v3/projects/3c8dee3911674124a15a41dfe181dca1/users/b9a588f471204b44aa11ea64a50aaa39/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:02.075341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf50e19-38fd-4d6e-b96f-f6f6ab804f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c8dee3911674124a15a41dfe181dca1, user_id=b9a588f471204b44aa11ea64a50aaa39, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:02.079815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf50e19-38fd-4d6e-b96f-f6f6ab804f45 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:02.098411 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3650/7218] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:02 2022] PUT /identity/v3/projects/3c8dee3911674124a15a41dfe181dca1/users/b9a588f471204b44aa11ea64a50aaa39/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:02.103575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545885a6-eca2-42bd-98ca-11acb6995b5d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:02.120597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545885a6-eca2-42bd-98ca-11acb6995b5d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:02.245714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2f70f-f093-4f32-b66f-826539f5c826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:02.246672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2f70f-f093-4f32-b66f-826539f5c826 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:02.247171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2f70f-f093-4f32-b66f-826539f5c826 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:02.247642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2f70f-f093-4f32-b66f-826539f5c826 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:02.248773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2f70f-f093-4f32-b66f-826539f5c826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:02.250886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2f70f-f093-4f32-b66f-826539f5c826 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:02.264166 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3569/7219] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:02.270741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf50e19-38fd-4d6e-b96f-f6f6ab804f45 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:02.287682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf50e19-38fd-4d6e-b96f-f6f6ab804f45 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:02.303139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c891a1-86b7-4948-8bf2-8a9f355d36ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:02.303955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c891a1-86b7-4948-8bf2-8a9f355d36ec admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:02.304301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c891a1-86b7-4948-8bf2-8a9f355d36ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:02.304735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c891a1-86b7-4948-8bf2-8a9f355d36ec admin admin] PATH_INFO: `/v3/projects/3c8dee3911674124a15a41dfe181dca1/users/b9a588f471204b44aa11ea64a50aaa39/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:02.308694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c891a1-86b7-4948-8bf2-8a9f355d36ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c8dee3911674124a15a41dfe181dca1, user_id=b9a588f471204b44aa11ea64a50aaa39, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:02.313444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c891a1-86b7-4948-8bf2-8a9f355d36ec admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:02.336019 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3651/7220] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:02 2022] PUT /identity/v3/projects/3c8dee3911674124a15a41dfe181dca1/users/b9a588f471204b44aa11ea64a50aaa39/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:04.263123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2f70f-f093-4f32-b66f-826539f5c826 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:04.264880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-45a2f70f-f093-4f32-b66f-826539f5c826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:04.465885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2f70f-f093-4f32-b66f-826539f5c826 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:04.641152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ffc14-64fb-457d-a9dc-3fec650b3861 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:04.641786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ffc14-64fb-457d-a9dc-3fec650b3861 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:04.642058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ffc14-64fb-457d-a9dc-3fec650b3861 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:04.642329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ffc14-64fb-457d-a9dc-3fec650b3861 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:04.649668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-653ffc14-64fb-457d-a9dc-3fec650b3861 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:04.682484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ffc14-64fb-457d-a9dc-3fec650b3861 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:04.684556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ffc14-64fb-457d-a9dc-3fec650b3861 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:04.711565 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3570/7221] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:05.761363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c891a1-86b7-4948-8bf2-8a9f355d36ec admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:05.764884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c6c891a1-86b7-4948-8bf2-8a9f355d36ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:05.920497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c891a1-86b7-4948-8bf2-8a9f355d36ec admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:05.931910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:05.932688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:05.933058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:05.933423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] PATH_INFO: `/v3/users/423adc69e1be461b9a4392e251114e97` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:05.936227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] RBAC: Authorizing `identity:delete_user(user_id=423adc69e1be461b9a4392e251114e97)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:05.938844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:06.049552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:06.060883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '423adc69e1be461b9a4392e251114e97'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:06.084571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '423adc69e1be461b9a4392e251114e97'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:06.097367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '423adc69e1be461b9a4392e251114e97'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:06.132209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '423adc69e1be461b9a4392e251114e97'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:06.134618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '423adc69e1be461b9a4392e251114e97'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:06.153516 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3652/7222] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:05 2022] DELETE /identity/v3/users/423adc69e1be461b9a4392e251114e97 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:06.157792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd994b20-85a8-45fd-8e17-482d5c1433c8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:06.158890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd994b20-85a8-45fd-8e17-482d5c1433c8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:06.159432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd994b20-85a8-45fd-8e17-482d5c1433c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:06.199468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bd994b20-85a8-45fd-8e17-482d5c1433c8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:06.200871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bd994b20-85a8-45fd-8e17-482d5c1433c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:06.283604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:06.389238 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3571/7223] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:06.452901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:06.454376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:06.622229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31dd39-6b46-4d33-814f-88fa65e5376b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:06.627036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cee91-a87c-48a6-bac1-8e86286d218b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:06.628911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cee91-a87c-48a6-bac1-8e86286d218b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:06.629267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cee91-a87c-48a6-bac1-8e86286d218b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:06.630111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cee91-a87c-48a6-bac1-8e86286d218b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:06.646041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cee91-a87c-48a6-bac1-8e86286d218b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:06.648003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cee91-a87c-48a6-bac1-8e86286d218b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:06.663318 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3653/7224] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:08.025487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd994b20-85a8-45fd-8e17-482d5c1433c8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:08.036621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bd994b20-85a8-45fd-8e17-482d5c1433c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:08.076920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd994b20-85a8-45fd-8e17-482d5c1433c8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:08.097086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04120b-68dd-4f9b-891c-ba83559315e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:08.097994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04120b-68dd-4f9b-891c-ba83559315e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:08.098383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04120b-68dd-4f9b-891c-ba83559315e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:08.100307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04120b-68dd-4f9b-891c-ba83559315e4 admin admin] PATH_INFO: `/v3/projects/da32113170f44bce89f6d7db9702150a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:08.107150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04120b-68dd-4f9b-891c-ba83559315e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da32113170f44bce89f6d7db9702150a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:08.116249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04120b-68dd-4f9b-891c-ba83559315e4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:08.196917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd04120b-68dd-4f9b-891c-ba83559315e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da32113170f44bce89f6d7db9702150a', 'request_id': 'req-cd04120b-68dd-4f9b-891c-ba83559315e4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:08.205229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd04120b-68dd-4f9b-891c-ba83559315e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da32113170f44bce89f6d7db9702150a', 'request_id': 'req-cd04120b-68dd-4f9b-891c-ba83559315e4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:08.219097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd04120b-68dd-4f9b-891c-ba83559315e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da32113170f44bce89f6d7db9702150a', 'request_id': 'req-cd04120b-68dd-4f9b-891c-ba83559315e4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:08.265870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd04120b-68dd-4f9b-891c-ba83559315e4 admin admin] The token cache is being invalidate because project da32113170f44bce89f6d7db9702150a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:08.269153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd04120b-68dd-4f9b-891c-ba83559315e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:08.270142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd04120b-68dd-4f9b-891c-ba83559315e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:08.272642 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3572/7225] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:08 2022] DELETE /identity/v3/projects/da32113170f44bce89f6d7db9702150a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:08.277311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0b259-e446-487b-af9c-52255a19fc3d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:08.277879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0b259-e446-487b-af9c-52255a19fc3d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:08.278002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0b259-e446-487b-af9c-52255a19fc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:08.360407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-02e0b259-e446-487b-af9c-52255a19fc3d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:08.360407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-02e0b259-e446-487b-af9c-52255a19fc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:08.519270 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3654/7226] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:08.520938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9bc7e-6e92-46bb-a76b-5b6da48eb744 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:08.521224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9bc7e-6e92-46bb-a76b-5b6da48eb744 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:08.521488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9bc7e-6e92-46bb-a76b-5b6da48eb744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:08.577674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cfc9bc7e-6e92-46bb-a76b-5b6da48eb744 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:08.579353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cfc9bc7e-6e92-46bb-a76b-5b6da48eb744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:08.589631 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3573/7227] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:08.594104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0b259-e446-487b-af9c-52255a19fc3d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:08.610540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0b259-e446-487b-af9c-52255a19fc3d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:08.626143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12dd9f3-660a-441c-8825-564de7ee9d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:08.629049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dd9f3-660a-441c-8825-564de7ee9d9a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:08.629304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dd9f3-660a-441c-8825-564de7ee9d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:08.631356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dd9f3-660a-441c-8825-564de7ee9d9a admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:08.632162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12dd9f3-660a-441c-8825-564de7ee9d9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:24:08.632524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12dd9f3-660a-441c-8825-564de7ee9d9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:08.645593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12dd9f3-660a-441c-8825-564de7ee9d9a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:08.676510 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3655/7228] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:24:08.681090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9bc7e-6e92-46bb-a76b-5b6da48eb744 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:08.700224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9bc7e-6e92-46bb-a76b-5b6da48eb744 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:08.710877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a8dff-239f-4bcf-b95c-e08a250976df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:08.711977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a8dff-239f-4bcf-b95c-e08a250976df admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:08.712446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a8dff-239f-4bcf-b95c-e08a250976df admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:08.712912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a8dff-239f-4bcf-b95c-e08a250976df admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:08.714044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a8dff-239f-4bcf-b95c-e08a250976df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:08.716852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a8dff-239f-4bcf-b95c-e08a250976df admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:08.771369 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3574/7229] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:08 2022] POST /identity/v3/projects => generated 371 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:08.776699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12dd9f3-660a-441c-8825-564de7ee9d9a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:08.795764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12dd9f3-660a-441c-8825-564de7ee9d9a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:08.902566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:08.927839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021aa70-68d2-4409-a88d-ab38b6108cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:08.928667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021aa70-68d2-4409-a88d-ab38b6108cde admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:08.929013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021aa70-68d2-4409-a88d-ab38b6108cde admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:08.929346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021aa70-68d2-4409-a88d-ab38b6108cde admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:08.931673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021aa70-68d2-4409-a88d-ab38b6108cde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:08.935992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021aa70-68d2-4409-a88d-ab38b6108cde admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:09.004314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a8dff-239f-4bcf-b95c-e08a250976df admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:09.019991 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3656/7230] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:08 2022] POST /identity/v3/users => generated 286 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:09.022222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a8dff-239f-4bcf-b95c-e08a250976df admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:09.038986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566153d-c7d2-470e-83f5-1463ca5cf914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:09.039708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566153d-c7d2-470e-83f5-1463ca5cf914 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:09.039954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566153d-c7d2-470e-83f5-1463ca5cf914 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:09.040227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566153d-c7d2-470e-83f5-1463ca5cf914 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:09.040910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566153d-c7d2-470e-83f5-1463ca5cf914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:09.042515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566153d-c7d2-470e-83f5-1463ca5cf914 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:09.076239 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3575/7231] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:09.078722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021aa70-68d2-4409-a88d-ab38b6108cde admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:09.095189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021aa70-68d2-4409-a88d-ab38b6108cde admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:09.110354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966206a4-317a-4463-aed5-4df6470cab8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:09.111916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966206a4-317a-4463-aed5-4df6470cab8d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:09.111916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966206a4-317a-4463-aed5-4df6470cab8d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:09.111916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966206a4-317a-4463-aed5-4df6470cab8d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:09.112520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966206a4-317a-4463-aed5-4df6470cab8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:09.116870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966206a4-317a-4463-aed5-4df6470cab8d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:09.157792 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3657/7232] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:09.161833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566153d-c7d2-470e-83f5-1463ca5cf914 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:09.194674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566153d-c7d2-470e-83f5-1463ca5cf914 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:09.227100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130d3cb-b851-48d4-ad52-d71623a9e8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:09.227965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130d3cb-b851-48d4-ad52-d71623a9e8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:09.228257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130d3cb-b851-48d4-ad52-d71623a9e8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:09.228563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130d3cb-b851-48d4-ad52-d71623a9e8a4 admin admin] PATH_INFO: `/v3/projects/84b1bc843d2447a5b94cf85a252a46c4/users/0fbe125061c746859d2680a9f26d0bae/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:09.271978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130d3cb-b851-48d4-ad52-d71623a9e8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b1bc843d2447a5b94cf85a252a46c4, user_id=0fbe125061c746859d2680a9f26d0bae, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:09.275503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130d3cb-b851-48d4-ad52-d71623a9e8a4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:09.306245 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3576/7233] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:09 2022] PUT /identity/v3/projects/84b1bc843d2447a5b94cf85a252a46c4/users/0fbe125061c746859d2680a9f26d0bae/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:09.318828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ce41b-4fb6-4bed-b7a3-badbfd934f29 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:09.319711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ce41b-4fb6-4bed-b7a3-badbfd934f29 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:09.320561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ce41b-4fb6-4bed-b7a3-badbfd934f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:09.407499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6e0ce41b-4fb6-4bed-b7a3-badbfd934f29 None None] MFA Rules not processed for user `0fbe125061c746859d2680a9f26d0bae`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:09.409446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6e0ce41b-4fb6-4bed-b7a3-badbfd934f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:09.530380 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3658/7234] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:09.536310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130d3cb-b851-48d4-ad52-d71623a9e8a4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:09.615088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:09.634699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130d3cb-b851-48d4-ad52-d71623a9e8a4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:09.641796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ddfd6-84e9-4625-9a45-dc251f4facea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:09.642832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ddfd6-84e9-4625-9a45-dc251f4facea admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:09.642832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ddfd6-84e9-4625-9a45-dc251f4facea admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:09.643657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ddfd6-84e9-4625-9a45-dc251f4facea admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:09.644617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ddfd6-84e9-4625-9a45-dc251f4facea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:09.646680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ddfd6-84e9-4625-9a45-dc251f4facea admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:09.685393 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3577/7235] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:09 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:09.686181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ce41b-4fb6-4bed-b7a3-badbfd934f29 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:09.699459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ce41b-4fb6-4bed-b7a3-badbfd934f29 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:09.765014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27460ec-23ba-489e-94e8-c807d13ab547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:09.765884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27460ec-23ba-489e-94e8-c807d13ab547 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:09.766849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27460ec-23ba-489e-94e8-c807d13ab547 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:09.767633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27460ec-23ba-489e-94e8-c807d13ab547 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:09.768550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27460ec-23ba-489e-94e8-c807d13ab547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:09.770843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27460ec-23ba-489e-94e8-c807d13ab547 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:09.799853 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3659/7236] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:09 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:24:09.803532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ddfd6-84e9-4625-9a45-dc251f4facea admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:09.818251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ddfd6-84e9-4625-9a45-dc251f4facea admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:09.834899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe07ae-9789-4c20-983e-8da6b72b7a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:09.836262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe07ae-9789-4c20-983e-8da6b72b7a77 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:09.836586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe07ae-9789-4c20-983e-8da6b72b7a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:09.837951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe07ae-9789-4c20-983e-8da6b72b7a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:09.838616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe07ae-9789-4c20-983e-8da6b72b7a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:09.842543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe07ae-9789-4c20-983e-8da6b72b7a77 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:09.860522 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3578/7237] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:09.864969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27460ec-23ba-489e-94e8-c807d13ab547 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:09.877693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27460ec-23ba-489e-94e8-c807d13ab547 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:09.890188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aee8e-028c-46a6-b357-a5a03e63e671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:09.891351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aee8e-028c-46a6-b357-a5a03e63e671 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:09.891813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aee8e-028c-46a6-b357-a5a03e63e671 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:09.892093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aee8e-028c-46a6-b357-a5a03e63e671 admin admin] PATH_INFO: `/v3/projects/357d6e9cf415455a8aee074611508cb8/users/bf909ae111a04261bbd1ad3cdda8dcf5/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:09.915456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0aee8e-028c-46a6-b357-a5a03e63e671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=357d6e9cf415455a8aee074611508cb8, user_id=bf909ae111a04261bbd1ad3cdda8dcf5, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:09.919964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0aee8e-028c-46a6-b357-a5a03e63e671 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:09.937205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a71af-cd52-42d8-b5ac-7fadb1cac487 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:09.937479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a71af-cd52-42d8-b5ac-7fadb1cac487 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:09.937805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a71af-cd52-42d8-b5ac-7fadb1cac487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:09.958471 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3660/7238] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:09 2022] PUT /identity/v3/projects/357d6e9cf415455a8aee074611508cb8/users/bf909ae111a04261bbd1ad3cdda8dcf5/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:09.962333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aee8e-028c-46a6-b357-a5a03e63e671 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:09.994552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:10.011697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-574a71af-cd52-42d8-b5ac-7fadb1cac487 None None] MFA Rules not processed for user `d5c97cfa7d924c3491f9e78b5051f0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:10.013146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-574a71af-cd52-42d8-b5ac-7fadb1cac487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:10.087326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aee8e-028c-46a6-b357-a5a03e63e671 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:10.110229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ad6d7-4267-4180-8c03-4345a743c9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:10.111429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ad6d7-4267-4180-8c03-4345a743c9fe admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:10.111609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ad6d7-4267-4180-8c03-4345a743c9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:10.111882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ad6d7-4267-4180-8c03-4345a743c9fe admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:10.113080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ad6d7-4267-4180-8c03-4345a743c9fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:10.114736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ad6d7-4267-4180-8c03-4345a743c9fe admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:10.147967 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3661/7239] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:10.149195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ad6d7-4267-4180-8c03-4345a743c9fe admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:10.171391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ad6d7-4267-4180-8c03-4345a743c9fe admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:10.176968 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3579/7240] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:10.189504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ae437-61fa-48c6-a46d-b8a3c16e808f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:10.190183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ae437-61fa-48c6-a46d-b8a3c16e808f admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:10.190453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ae437-61fa-48c6-a46d-b8a3c16e808f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:10.190715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ae437-61fa-48c6-a46d-b8a3c16e808f admin admin] PATH_INFO: `/v3/projects/357d6e9cf415455a8aee074611508cb8/users/bf909ae111a04261bbd1ad3cdda8dcf5/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:10.192544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a71af-cd52-42d8-b5ac-7fadb1cac487 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:10.194336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-574a71af-cd52-42d8-b5ac-7fadb1cac487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:10.201770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ae437-61fa-48c6-a46d-b8a3c16e808f admin admin] RBAC: Authorizing `identity:create_grant(project_id=357d6e9cf415455a8aee074611508cb8, user_id=bf909ae111a04261bbd1ad3cdda8dcf5, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:10.204875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ae437-61fa-48c6-a46d-b8a3c16e808f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:10.253547 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3662/7241] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:10 2022] PUT /identity/v3/projects/357d6e9cf415455a8aee074611508cb8/users/bf909ae111a04261bbd1ad3cdda8dcf5/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:10.255446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b99bc-1f29-49b7-9bd5-c34b179833ac None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:10.255877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b99bc-1f29-49b7-9bd5-c34b179833ac None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:10.256288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b99bc-1f29-49b7-9bd5-c34b179833ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:10.301952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7c4b99bc-1f29-49b7-9bd5-c34b179833ac None None] MFA Rules not processed for user `bf909ae111a04261bbd1ad3cdda8dcf5`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:10.303448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c4b99bc-1f29-49b7-9bd5-c34b179833ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:10.400931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a71af-cd52-42d8-b5ac-7fadb1cac487 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:10.441699 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3663/7242] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:10.452933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b99bc-1f29-49b7-9bd5-c34b179833ac None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:10.454369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c4b99bc-1f29-49b7-9bd5-c34b179833ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:10.458224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:10.544556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e5c55-5ab1-4f9e-b601-de5cd6faec46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:10.545331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e5c55-5ab1-4f9e-b601-de5cd6faec46 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:10.545627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e5c55-5ab1-4f9e-b601-de5cd6faec46 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:10.545886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e5c55-5ab1-4f9e-b601-de5cd6faec46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:10.625457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b99bc-1f29-49b7-9bd5-c34b179833ac None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:10.637768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0977669-418d-47d3-838e-5f9803bd9692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:10.639270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0977669-418d-47d3-838e-5f9803bd9692 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:10.639548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0977669-418d-47d3-838e-5f9803bd9692 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:10.641278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0977669-418d-47d3-838e-5f9803bd9692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:10.670147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0977669-418d-47d3-838e-5f9803bd9692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:10.673506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0977669-418d-47d3-838e-5f9803bd9692 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:10.689329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e5c55-5ab1-4f9e-b601-de5cd6faec46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:10.691623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e5c55-5ab1-4f9e-b601-de5cd6faec46 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:10.706965 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3580/7243] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:10.719516 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3664/7244] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:24:11.512826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e5c55-5ab1-4f9e-b601-de5cd6faec46 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:11.706141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e5c55-5ab1-4f9e-b601-de5cd6faec46 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:11.718566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:11.719592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:11.721301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:11.721723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] PATH_INFO: `/v3/users/0fbe125061c746859d2680a9f26d0bae` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:11.723634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fbe125061c746859d2680a9f26d0bae)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:11.729297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:11.873250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fbe125061c746859d2680a9f26d0bae'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:11.894947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fbe125061c746859d2680a9f26d0bae'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:11.922342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fbe125061c746859d2680a9f26d0bae'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:11.926281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fbe125061c746859d2680a9f26d0bae'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:11.928108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fbe125061c746859d2680a9f26d0bae'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:11.932583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:11.971216 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3581/7245] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:11 2022] DELETE /identity/v3/users/0fbe125061c746859d2680a9f26d0bae => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:11.977735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531a518-4390-4f05-91d8-1458a0ab80b8 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:11.978305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531a518-4390-4f05-91d8-1458a0ab80b8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:11.978305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531a518-4390-4f05-91d8-1458a0ab80b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:12.021750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5531a518-4390-4f05-91d8-1458a0ab80b8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:12.025111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5531a518-4390-4f05-91d8-1458a0ab80b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:12.119565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:12.203057 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3665/7246] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:12.224227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:12.227392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:12.383438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03fc57-2143-4520-a448-a21294dcf082 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:12.387318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b4999-1cf9-41b3-8bc2-0a4f0dd92df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:12.387988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b4999-1cf9-41b3-8bc2-0a4f0dd92df9 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:12.388262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b4999-1cf9-41b3-8bc2-0a4f0dd92df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:12.388595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b4999-1cf9-41b3-8bc2-0a4f0dd92df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:12.400123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b4999-1cf9-41b3-8bc2-0a4f0dd92df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:12.402343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b4999-1cf9-41b3-8bc2-0a4f0dd92df9 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:12.428620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25681f0-3ae3-46f4-bfbd-51b412bc5254 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:12.429237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25681f0-3ae3-46f4-bfbd-51b412bc5254 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:12.429618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25681f0-3ae3-46f4-bfbd-51b412bc5254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:12.446647 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3582/7247] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:12.455065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e25681f0-3ae3-46f4-bfbd-51b412bc5254 None None] MFA Rules not processed for user `d5c97cfa7d924c3491f9e78b5051f0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:12.456543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e25681f0-3ae3-46f4-bfbd-51b412bc5254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:12.587136 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3666/7248] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:24:14.468291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b4999-1cf9-41b3-8bc2-0a4f0dd92df9 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:14.469936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-561b4999-1cf9-41b3-8bc2-0a4f0dd92df9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:14.497279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b4999-1cf9-41b3-8bc2-0a4f0dd92df9 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:14.511865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:14.513055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:14.513776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:14.514459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d admin admin] PATH_INFO: `/v3/projects/84b1bc843d2447a5b94cf85a252a46c4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:14.518151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=84b1bc843d2447a5b94cf85a252a46c4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:14.538139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:14.675332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84b1bc843d2447a5b94cf85a252a46c4', 'request_id': 'req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:14.688141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84b1bc843d2447a5b94cf85a252a46c4', 'request_id': 'req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:14.707002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84b1bc843d2447a5b94cf85a252a46c4', 'request_id': 'req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:14.731847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d admin admin] The token cache is being invalidate because project 84b1bc843d2447a5b94cf85a252a46c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:14.732970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:14.738258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5cfcd24-6b59-4b84-8c2f-11ece7c7e57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:14.740643 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3583/7249] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:14 2022] DELETE /identity/v3/projects/84b1bc843d2447a5b94cf85a252a46c4 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:14.745091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25681f0-3ae3-46f4-bfbd-51b412bc5254 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:14.771538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25681f0-3ae3-46f4-bfbd-51b412bc5254 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:15.001152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:15.003506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:15.004005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:15.004533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] PATH_INFO: `/v3/users/bf909ae111a04261bbd1ad3cdda8dcf5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:15.008751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf909ae111a04261bbd1ad3cdda8dcf5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:15.014322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:15.118468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf909ae111a04261bbd1ad3cdda8dcf5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:15.146610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf909ae111a04261bbd1ad3cdda8dcf5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:15.157901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf909ae111a04261bbd1ad3cdda8dcf5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:15.178443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf909ae111a04261bbd1ad3cdda8dcf5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:15.194361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf909ae111a04261bbd1ad3cdda8dcf5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:15.198246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:15.211201 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3667/7250] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:14 2022] DELETE /identity/v3/users/bf909ae111a04261bbd1ad3cdda8dcf5 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:16.174824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4932e-5d8f-4dc8-a1b4-25a13826d752 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:16.175419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4932e-5d8f-4dc8-a1b4-25a13826d752 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:16.175717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4932e-5d8f-4dc8-a1b4-25a13826d752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:16.245109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e2c4932e-5d8f-4dc8-a1b4-25a13826d752 None None] MFA Rules not processed for user `b9a588f471204b44aa11ea64a50aaa39`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:16.246503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e2c4932e-5d8f-4dc8-a1b4-25a13826d752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:16.311856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:16.316123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:16.413600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:16.418925 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3584/7251] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:24:16.436983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4932e-5d8f-4dc8-a1b4-25a13826d752 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:16.441305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e2c4932e-5d8f-4dc8-a1b4-25a13826d752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:16.514479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd2d50-8059-487c-a042-3a8f6ab2a8bc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:16.526129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e045798-28a8-46d6-b1c5-8acaebc92060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:16.527638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e045798-28a8-46d6-b1c5-8acaebc92060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:16.528837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e045798-28a8-46d6-b1c5-8acaebc92060 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:16.529197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e045798-28a8-46d6-b1c5-8acaebc92060 admin admin] PATH_INFO: `/v3/projects/357d6e9cf415455a8aee074611508cb8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:16.531728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e045798-28a8-46d6-b1c5-8acaebc92060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=357d6e9cf415455a8aee074611508cb8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:16.535458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e045798-28a8-46d6-b1c5-8acaebc92060 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:16.657086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7e045798-28a8-46d6-b1c5-8acaebc92060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '357d6e9cf415455a8aee074611508cb8', 'request_id': 'req-7e045798-28a8-46d6-b1c5-8acaebc92060'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:16.658062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4932e-5d8f-4dc8-a1b4-25a13826d752 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:16.674221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7e045798-28a8-46d6-b1c5-8acaebc92060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '357d6e9cf415455a8aee074611508cb8', 'request_id': 'req-7e045798-28a8-46d6-b1c5-8acaebc92060'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:16.738623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7e045798-28a8-46d6-b1c5-8acaebc92060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '357d6e9cf415455a8aee074611508cb8', 'request_id': 'req-7e045798-28a8-46d6-b1c5-8acaebc92060'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:16.782076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7e045798-28a8-46d6-b1c5-8acaebc92060 admin admin] The token cache is being invalidate because project 357d6e9cf415455a8aee074611508cb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:16.782366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7e045798-28a8-46d6-b1c5-8acaebc92060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:16.784434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7e045798-28a8-46d6-b1c5-8acaebc92060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:16.786886 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3668/7252] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:16 2022] DELETE /identity/v3/projects/357d6e9cf415455a8aee074611508cb8 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:16.791787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c748d-d43c-40cf-980c-86d0eb8408bb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:16.792066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c748d-d43c-40cf-980c-86d0eb8408bb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:16.792339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c748d-d43c-40cf-980c-86d0eb8408bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:16.827948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea1790-0cf6-4381-9484-3f9bcd3a1823 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:16.828783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea1790-0cf6-4381-9484-3f9bcd3a1823 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:16.829080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea1790-0cf6-4381-9484-3f9bcd3a1823 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:16.829386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea1790-0cf6-4381-9484-3f9bcd3a1823 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:16.831671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-daea1790-0cf6-4381-9484-3f9bcd3a1823 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:16.832895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-242c748d-d43c-40cf-980c-86d0eb8408bb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:16.834792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-242c748d-d43c-40cf-980c-86d0eb8408bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:16.936791 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3669/7253] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:16.947288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a35ad7-0275-4d80-a19e-3b458b2a78bf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:16.947765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a35ad7-0275-4d80-a19e-3b458b2a78bf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:16.948356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a35ad7-0275-4d80-a19e-3b458b2a78bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:16.989467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-62a35ad7-0275-4d80-a19e-3b458b2a78bf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:16.993978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-62a35ad7-0275-4d80-a19e-3b458b2a78bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:17.002173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea1790-0cf6-4381-9484-3f9bcd3a1823 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:17.005253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea1790-0cf6-4381-9484-3f9bcd3a1823 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:17.043225 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3670/7254] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:17.044420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a35ad7-0275-4d80-a19e-3b458b2a78bf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:17.058392 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3585/7255] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:24:17.071414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a35ad7-0275-4d80-a19e-3b458b2a78bf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:17.081211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6afc3c-c986-4480-be21-a9295bc1ac31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:17.083165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6afc3c-c986-4480-be21-a9295bc1ac31 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:17.083165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6afc3c-c986-4480-be21-a9295bc1ac31 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:17.086736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6afc3c-c986-4480-be21-a9295bc1ac31 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:17.086736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6afc3c-c986-4480-be21-a9295bc1ac31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:24:17.087216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6afc3c-c986-4480-be21-a9295bc1ac31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:17.089046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6afc3c-c986-4480-be21-a9295bc1ac31 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:17.123657 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3671/7256] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:24:17.124407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea1790-0cf6-4381-9484-3f9bcd3a1823 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:17.148250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea1790-0cf6-4381-9484-3f9bcd3a1823 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:17.156600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31427677-0d0a-470c-aeec-50c69668ad48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:17.157544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31427677-0d0a-470c-aeec-50c69668ad48 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:17.158303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31427677-0d0a-470c-aeec-50c69668ad48 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:17.158744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31427677-0d0a-470c-aeec-50c69668ad48 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:17.159651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31427677-0d0a-470c-aeec-50c69668ad48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:17.162825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31427677-0d0a-470c-aeec-50c69668ad48 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:17.205181 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3586/7257] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:17 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:24:17.209489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6afc3c-c986-4480-be21-a9295bc1ac31 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:17.229137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6afc3c-c986-4480-be21-a9295bc1ac31 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:17.238954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:17.347216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3ffd2-737c-4857-901b-47308c9c6c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:17.347960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3ffd2-737c-4857-901b-47308c9c6c59 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:17.348611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3ffd2-737c-4857-901b-47308c9c6c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:17.349084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3ffd2-737c-4857-901b-47308c9c6c59 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:17.349950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3ffd2-737c-4857-901b-47308c9c6c59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:17.352595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3ffd2-737c-4857-901b-47308c9c6c59 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:17.386354 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3672/7258] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:17 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:17.387116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31427677-0d0a-470c-aeec-50c69668ad48 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:17.400041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31427677-0d0a-470c-aeec-50c69668ad48 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:17.405085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3fc4b-3b26-4350-bf6a-9a7abd1950f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:17.405701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3fc4b-3b26-4350-bf6a-9a7abd1950f2 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:17.405977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3fc4b-3b26-4350-bf6a-9a7abd1950f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:17.406374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3fc4b-3b26-4350-bf6a-9a7abd1950f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:17.407021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3fc4b-3b26-4350-bf6a-9a7abd1950f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:17.409043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3fc4b-3b26-4350-bf6a-9a7abd1950f2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:17.425005 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3587/7259] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:24:17.429552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3ffd2-737c-4857-901b-47308c9c6c59 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:17.448133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3ffd2-737c-4857-901b-47308c9c6c59 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:17.456393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d6037-c1de-44b4-9860-bdd2e54d0ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:17.457268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d6037-c1de-44b4-9860-bdd2e54d0ecf admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:17.457637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d6037-c1de-44b4-9860-bdd2e54d0ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:17.458022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d6037-c1de-44b4-9860-bdd2e54d0ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:17.458817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d6037-c1de-44b4-9860-bdd2e54d0ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:17.461598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d6037-c1de-44b4-9860-bdd2e54d0ecf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:17.486775 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3673/7260] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:17.492042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3fc4b-3b26-4350-bf6a-9a7abd1950f2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:17.506696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:17.514626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3fc4b-3b26-4350-bf6a-9a7abd1950f2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:17.526940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87013bfe-c8d1-48c2-b1f4-c493262919be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:17.527683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87013bfe-c8d1-48c2-b1f4-c493262919be admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:17.530760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87013bfe-c8d1-48c2-b1f4-c493262919be admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:17.531040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87013bfe-c8d1-48c2-b1f4-c493262919be admin admin] PATH_INFO: `/v3/projects/f9048abdb2b14af7a1679baaf15f5e70/users/1bfe439b9fdc42afabf253a5c758e8da/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:17.762588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87013bfe-c8d1-48c2-b1f4-c493262919be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9048abdb2b14af7a1679baaf15f5e70, user_id=1bfe439b9fdc42afabf253a5c758e8da, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:17.769264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87013bfe-c8d1-48c2-b1f4-c493262919be admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:17.782579 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3588/7261] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:17 2022] PUT /identity/v3/projects/f9048abdb2b14af7a1679baaf15f5e70/users/1bfe439b9fdc42afabf253a5c758e8da/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:17.793381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec4228-7c10-430a-bb8c-ff76a02ca96a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:17.793653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec4228-7c10-430a-bb8c-ff76a02ca96a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:17.793916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec4228-7c10-430a-bb8c-ff76a02ca96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:17.845938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-31ec4228-7c10-430a-bb8c-ff76a02ca96a None None] MFA Rules not processed for user `1bfe439b9fdc42afabf253a5c758e8da`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:17.847306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-31ec4228-7c10-430a-bb8c-ff76a02ca96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:17.973639 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3674/7262] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:17.977356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87013bfe-c8d1-48c2-b1f4-c493262919be admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:18.137595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87013bfe-c8d1-48c2-b1f4-c493262919be admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:18.142908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e120b-8b85-4e9a-92c8-59d33e85236d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:18.143618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e120b-8b85-4e9a-92c8-59d33e85236d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:18.143971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e120b-8b85-4e9a-92c8-59d33e85236d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:18.144336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e120b-8b85-4e9a-92c8-59d33e85236d admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:18.145394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e120b-8b85-4e9a-92c8-59d33e85236d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:18.147724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e120b-8b85-4e9a-92c8-59d33e85236d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:18.195518 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3589/7263] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:17 2022] POST /identity/v3/projects => generated 375 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:18.198485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec4228-7c10-430a-bb8c-ff76a02ca96a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:18.212735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec4228-7c10-430a-bb8c-ff76a02ca96a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:18.396481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75df937-6e91-4cd9-88e6-1d0a1f3652f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:18.397297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75df937-6e91-4cd9-88e6-1d0a1f3652f3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:18.397711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75df937-6e91-4cd9-88e6-1d0a1f3652f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:18.398078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75df937-6e91-4cd9-88e6-1d0a1f3652f3 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:18.398837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75df937-6e91-4cd9-88e6-1d0a1f3652f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:18.401151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75df937-6e91-4cd9-88e6-1d0a1f3652f3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:18.466205 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3675/7264] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:18 2022] POST /identity/v3/users => generated 294 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:18.467738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e120b-8b85-4e9a-92c8-59d33e85236d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:18.482270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e120b-8b85-4e9a-92c8-59d33e85236d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:18.487673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d717e-db59-4998-aafa-dbf3c92f8d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:18.488315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d717e-db59-4998-aafa-dbf3c92f8d10 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:18.488777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d717e-db59-4998-aafa-dbf3c92f8d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:18.489053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d717e-db59-4998-aafa-dbf3c92f8d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:18.489703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d717e-db59-4998-aafa-dbf3c92f8d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:18.492058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d717e-db59-4998-aafa-dbf3c92f8d10 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:18.511353 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3590/7265] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:18.512446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75df937-6e91-4cd9-88e6-1d0a1f3652f3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:18.536480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75df937-6e91-4cd9-88e6-1d0a1f3652f3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:18.546732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed7e7d-1de2-4715-83e9-762b1fe4f1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:18.547792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed7e7d-1de2-4715-83e9-762b1fe4f1d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:18.548313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed7e7d-1de2-4715-83e9-762b1fe4f1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:18.548823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed7e7d-1de2-4715-83e9-762b1fe4f1d8 admin admin] PATH_INFO: `/v3/projects/1cf3825fb870465ebf0c4772a3fc18be/users/a8b029738b664eed81586ba14e9f7591/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:18.552105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:18.574440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed7e7d-1de2-4715-83e9-762b1fe4f1d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf3825fb870465ebf0c4772a3fc18be, user_id=a8b029738b664eed81586ba14e9f7591, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:18.583487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed7e7d-1de2-4715-83e9-762b1fe4f1d8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:18.608090 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3676/7266] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:18 2022] PUT /identity/v3/projects/1cf3825fb870465ebf0c4772a3fc18be/users/a8b029738b664eed81586ba14e9f7591/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:24:18.614939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d717e-db59-4998-aafa-dbf3c92f8d10 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:18.644214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d717e-db59-4998-aafa-dbf3c92f8d10 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:18.734212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e5b10-0c39-4918-b8ec-2caac75e6172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:18.734829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e5b10-0c39-4918-b8ec-2caac75e6172 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:18.735101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e5b10-0c39-4918-b8ec-2caac75e6172 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:18.735366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e5b10-0c39-4918-b8ec-2caac75e6172 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:18.737572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e5b10-0c39-4918-b8ec-2caac75e6172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:18.739628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e5b10-0c39-4918-b8ec-2caac75e6172 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:18.766541 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3591/7267] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:18.768161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed7e7d-1de2-4715-83e9-762b1fe4f1d8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:18.781158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed7e7d-1de2-4715-83e9-762b1fe4f1d8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:18.786471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d0085-f142-47e3-a40c-38e812fcccfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:18.787749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d0085-f142-47e3-a40c-38e812fcccfc admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:18.788027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d0085-f142-47e3-a40c-38e812fcccfc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:18.788296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d0085-f142-47e3-a40c-38e812fcccfc admin admin] PATH_INFO: `/v3/projects/1cf3825fb870465ebf0c4772a3fc18be/users/a8b029738b664eed81586ba14e9f7591/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:18.790946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d0085-f142-47e3-a40c-38e812fcccfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf3825fb870465ebf0c4772a3fc18be, user_id=a8b029738b664eed81586ba14e9f7591, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:18.794075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d0085-f142-47e3-a40c-38e812fcccfc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:18.818314 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3677/7268] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:18 2022] PUT /identity/v3/projects/1cf3825fb870465ebf0c4772a3fc18be/users/a8b029738b664eed81586ba14e9f7591/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:24:18.821052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c2c7e-0e89-41ee-9b56-ddaf607dff31 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:18.821360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c2c7e-0e89-41ee-9b56-ddaf607dff31 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:18.821635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c2c7e-0e89-41ee-9b56-ddaf607dff31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:18.867042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2d1c2c7e-0e89-41ee-9b56-ddaf607dff31 None None] MFA Rules not processed for user `a8b029738b664eed81586ba14e9f7591`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:18.868986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2d1c2c7e-0e89-41ee-9b56-ddaf607dff31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:18.939161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:19.074680 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3592/7269] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:24:19.092963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d0085-f142-47e3-a40c-38e812fcccfc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:19.094750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-363d0085-f142-47e3-a40c-38e812fcccfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:19.194357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d0085-f142-47e3-a40c-38e812fcccfc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:19.200686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18519da0-5f67-435b-895f-0e616b76b01b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:19.202659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18519da0-5f67-435b-895f-0e616b76b01b service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:19.204865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18519da0-5f67-435b-895f-0e616b76b01b service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:19.205474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18519da0-5f67-435b-895f-0e616b76b01b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:19.321413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18519da0-5f67-435b-895f-0e616b76b01b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:19.323180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18519da0-5f67-435b-895f-0e616b76b01b service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:19.372582 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3678/7270] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:20.567220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32499dfe-5fa8-4524-ac44-8cd81907a96e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:20.568510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32499dfe-5fa8-4524-ac44-8cd81907a96e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:20.568981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32499dfe-5fa8-4524-ac44-8cd81907a96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:20.591276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-32499dfe-5fa8-4524-ac44-8cd81907a96e None None] MFA Rules not processed for user `15c7508ede01463b9494100198a8d3a0`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:20.592776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-32499dfe-5fa8-4524-ac44-8cd81907a96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:20.668142 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3593/7271] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:21.103043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18519da0-5f67-435b-895f-0e616b76b01b service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:21.187250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18519da0-5f67-435b-895f-0e616b76b01b service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:21.194045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:21.195104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:21.195535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:21.195968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] PATH_INFO: `/v3/users/1bfe439b9fdc42afabf253a5c758e8da` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:21.198008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bfe439b9fdc42afabf253a5c758e8da)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:21.200173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:21.261257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:21.302694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bfe439b9fdc42afabf253a5c758e8da'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:21.311959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bfe439b9fdc42afabf253a5c758e8da'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:21.330344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bfe439b9fdc42afabf253a5c758e8da'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:21.365433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bfe439b9fdc42afabf253a5c758e8da'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:21.366734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bfe439b9fdc42afabf253a5c758e8da'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:21.381629 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3679/7272] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:21 2022] DELETE /identity/v3/users/1bfe439b9fdc42afabf253a5c758e8da => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:21.383198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dbc5d-67a2-44d6-8de9-85cc87940c1f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:21.383474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dbc5d-67a2-44d6-8de9-85cc87940c1f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:21.383760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dbc5d-67a2-44d6-8de9-85cc87940c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:21.413342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c90dbc5d-67a2-44d6-8de9-85cc87940c1f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:21.416987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c90dbc5d-67a2-44d6-8de9-85cc87940c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:21.606392 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3594/7273] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:24:21.614816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:21.618079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:21.719528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96026ca2-daae-4259-85c2-cb3d2125ff9c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:21.726231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1106c8b-187e-4ccf-b953-5789aa52c4fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:21.726875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1106c8b-187e-4ccf-b953-5789aa52c4fc service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:21.727149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1106c8b-187e-4ccf-b953-5789aa52c4fc service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:21.727419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1106c8b-187e-4ccf-b953-5789aa52c4fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:21.738943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1106c8b-187e-4ccf-b953-5789aa52c4fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:21.740493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1106c8b-187e-4ccf-b953-5789aa52c4fc service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:21.753847 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3680/7274] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:22.228841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90dbc5d-67a2-44d6-8de9-85cc87940c1f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:22.228841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c90dbc5d-67a2-44d6-8de9-85cc87940c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:22.242313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fad12-0e16-4c8c-af22-23a6b9161ac0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:22.242796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fad12-0e16-4c8c-af22-23a6b9161ac0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:22.242796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fad12-0e16-4c8c-af22-23a6b9161ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:22.250587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90dbc5d-67a2-44d6-8de9-85cc87940c1f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:22.257761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:22.258876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:22.259340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:22.259826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] PATH_INFO: `/v3/projects/f9048abdb2b14af7a1679baaf15f5e70` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:22.261621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9048abdb2b14af7a1679baaf15f5e70)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:22.264669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:22.297507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b41fad12-0e16-4c8c-af22-23a6b9161ac0 None None] MFA Rules not processed for user `4e6a5f35d172410481a8416b7bdaafac`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:22.301152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b41fad12-0e16-4c8c-af22-23a6b9161ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:22.375953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9048abdb2b14af7a1679baaf15f5e70', 'request_id': 'req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:22.385917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9048abdb2b14af7a1679baaf15f5e70', 'request_id': 'req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:22.397817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9048abdb2b14af7a1679baaf15f5e70', 'request_id': 'req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:22.415575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] The token cache is being invalidate because project f9048abdb2b14af7a1679baaf15f5e70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:22.415796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:22.416516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:22.418347 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3595/7276] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:22 2022] DELETE /identity/v3/projects/f9048abdb2b14af7a1679baaf15f5e70 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:22.422828 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3681/7276] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:22.423203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:22.425217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:22.431210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:22.543866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc8acf-9187-44e5-9e44-9e6e6cfc5a13 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:22.558795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c49621-6bef-4d44-af16-cc95302f1276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:22.559467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c49621-6bef-4d44-af16-cc95302f1276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:22.559738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c49621-6bef-4d44-af16-cc95302f1276 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:22.560011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c49621-6bef-4d44-af16-cc95302f1276 admin admin] PATH_INFO: `/v3/users/a8b029738b664eed81586ba14e9f7591` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:22.566972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c49621-6bef-4d44-af16-cc95302f1276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8b029738b664eed81586ba14e9f7591)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:22.569347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c49621-6bef-4d44-af16-cc95302f1276 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:22.660970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9c49621-6bef-4d44-af16-cc95302f1276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8b029738b664eed81586ba14e9f7591'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:22.681238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9c49621-6bef-4d44-af16-cc95302f1276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8b029738b664eed81586ba14e9f7591'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:22.725693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9c49621-6bef-4d44-af16-cc95302f1276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8b029738b664eed81586ba14e9f7591'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:22.726642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9c49621-6bef-4d44-af16-cc95302f1276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8b029738b664eed81586ba14e9f7591'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:22.727374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9c49621-6bef-4d44-af16-cc95302f1276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8b029738b664eed81586ba14e9f7591'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:22.742916 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3596/7277] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:22 2022] DELETE /identity/v3/users/a8b029738b664eed81586ba14e9f7591 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:23.967886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fad12-0e16-4c8c-af22-23a6b9161ac0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:23.994121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fad12-0e16-4c8c-af22-23a6b9161ac0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:24.173873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:24.187431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549e280-0d78-47de-abc0-791d6a827a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:24.188647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549e280-0d78-47de-abc0-791d6a827a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:24.189244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549e280-0d78-47de-abc0-791d6a827a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:24.189748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549e280-0d78-47de-abc0-791d6a827a17 admin admin] PATH_INFO: `/v3/projects/1cf3825fb870465ebf0c4772a3fc18be` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:24.193639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549e280-0d78-47de-abc0-791d6a827a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf3825fb870465ebf0c4772a3fc18be)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:24.196469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549e280-0d78-47de-abc0-791d6a827a17 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:24.292984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7549e280-0d78-47de-abc0-791d6a827a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf3825fb870465ebf0c4772a3fc18be', 'request_id': 'req-7549e280-0d78-47de-abc0-791d6a827a17'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:24.318101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7549e280-0d78-47de-abc0-791d6a827a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf3825fb870465ebf0c4772a3fc18be', 'request_id': 'req-7549e280-0d78-47de-abc0-791d6a827a17'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:24.342590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7549e280-0d78-47de-abc0-791d6a827a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf3825fb870465ebf0c4772a3fc18be', 'request_id': 'req-7549e280-0d78-47de-abc0-791d6a827a17'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:24.368684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7549e280-0d78-47de-abc0-791d6a827a17 admin admin] The token cache is being invalidate because project 1cf3825fb870465ebf0c4772a3fc18be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:24.369754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7549e280-0d78-47de-abc0-791d6a827a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:24.370670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7549e280-0d78-47de-abc0-791d6a827a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:24.372837 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3682/7278] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:23 2022] DELETE /identity/v3/projects/1cf3825fb870465ebf0c4772a3fc18be => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:24.388142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8059e37-20df-4252-a886-f443320fe3bf None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:24.388668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8059e37-20df-4252-a886-f443320fe3bf None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:24.388766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8059e37-20df-4252-a886-f443320fe3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:24.427301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d8059e37-20df-4252-a886-f443320fe3bf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:24.428579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d8059e37-20df-4252-a886-f443320fe3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:24.573819 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3597/7279] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:24:24.582171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827a519-ef71-49a2-a95c-ebeb6e854987 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:24.582946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827a519-ef71-49a2-a95c-ebeb6e854987 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:24.584944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827a519-ef71-49a2-a95c-ebeb6e854987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:24.630636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8827a519-ef71-49a2-a95c-ebeb6e854987 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:24.631905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8827a519-ef71-49a2-a95c-ebeb6e854987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:24.646971 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3683/7280] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:24.647346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8059e37-20df-4252-a886-f443320fe3bf None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:24.663773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8059e37-20df-4252-a886-f443320fe3bf None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:24.670334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dabae6-bd0b-4ee9-9ea6-77b94f2a4171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:24.671048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dabae6-bd0b-4ee9-9ea6-77b94f2a4171 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:24.671412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dabae6-bd0b-4ee9-9ea6-77b94f2a4171 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:24.671859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dabae6-bd0b-4ee9-9ea6-77b94f2a4171 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:24.672686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dabae6-bd0b-4ee9-9ea6-77b94f2a4171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:24:24.674059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dabae6-bd0b-4ee9-9ea6-77b94f2a4171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:24.678972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dabae6-bd0b-4ee9-9ea6-77b94f2a4171 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:24.713228 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3598/7281] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:24:24.718015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827a519-ef71-49a2-a95c-ebeb6e854987 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:24.730352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827a519-ef71-49a2-a95c-ebeb6e854987 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:24.737384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd746e-8a59-4819-807f-61179db97485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:24.738260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd746e-8a59-4819-807f-61179db97485 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:24.738813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd746e-8a59-4819-807f-61179db97485 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:24.739324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd746e-8a59-4819-807f-61179db97485 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:24.740134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd746e-8a59-4819-807f-61179db97485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:24.742341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd746e-8a59-4819-807f-61179db97485 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:24.794210 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3684/7282] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:24 2022] POST /identity/v3/projects => generated 367 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:24.794460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dabae6-bd0b-4ee9-9ea6-77b94f2a4171 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:24.808132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dabae6-bd0b-4ee9-9ea6-77b94f2a4171 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:24.822673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:24.960025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c3da3-668b-4254-afe9-8bddb4a34404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:24.961451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c3da3-668b-4254-afe9-8bddb4a34404 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:24.961863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c3da3-668b-4254-afe9-8bddb4a34404 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:24.962452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c3da3-668b-4254-afe9-8bddb4a34404 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:24.963216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c3da3-668b-4254-afe9-8bddb4a34404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:24.966101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c3da3-668b-4254-afe9-8bddb4a34404 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:25.011975 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3599/7283] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:24 2022] POST /identity/v3/users => generated 284 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:24:25.018046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd746e-8a59-4819-807f-61179db97485 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:25.043199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd746e-8a59-4819-807f-61179db97485 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:25.058958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a8d65-aac7-467b-91be-def2ce171b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:25.059771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a8d65-aac7-467b-91be-def2ce171b52 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:25.060215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a8d65-aac7-467b-91be-def2ce171b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:25.060840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a8d65-aac7-467b-91be-def2ce171b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:25.061785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a8d65-aac7-467b-91be-def2ce171b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:25.066461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a8d65-aac7-467b-91be-def2ce171b52 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:25.095722 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3685/7284] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:25.099548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c3da3-668b-4254-afe9-8bddb4a34404 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:25.112721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c3da3-668b-4254-afe9-8bddb4a34404 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:25.128165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342618e3-73df-477f-be3c-8fd5eeda0448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:25.128909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342618e3-73df-477f-be3c-8fd5eeda0448 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:25.129250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342618e3-73df-477f-be3c-8fd5eeda0448 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:25.129514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342618e3-73df-477f-be3c-8fd5eeda0448 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:25.130148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342618e3-73df-477f-be3c-8fd5eeda0448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:25.131741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342618e3-73df-477f-be3c-8fd5eeda0448 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:25.146488 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3600/7285] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:25.147994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a8d65-aac7-467b-91be-def2ce171b52 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:25.159448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a8d65-aac7-467b-91be-def2ce171b52 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:25.164760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdb450-9ebf-49bc-880a-d66c660a0086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:25.165871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdb450-9ebf-49bc-880a-d66c660a0086 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:25.166374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdb450-9ebf-49bc-880a-d66c660a0086 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:25.166859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdb450-9ebf-49bc-880a-d66c660a0086 admin admin] PATH_INFO: `/v3/projects/136f346faf7646a9a8aa645af34e11ca/users/07e0d36c35f8448cb9d993bef8ac1dad/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:25.187410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdb450-9ebf-49bc-880a-d66c660a0086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=136f346faf7646a9a8aa645af34e11ca, user_id=07e0d36c35f8448cb9d993bef8ac1dad, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:25.190495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdb450-9ebf-49bc-880a-d66c660a0086 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:25.219644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5473e07-1e33-4511-ad65-9d853bf41797 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:25.220457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5473e07-1e33-4511-ad65-9d853bf41797 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:25.220986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5473e07-1e33-4511-ad65-9d853bf41797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:25.224460 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3686/7286] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:25 2022] PUT /identity/v3/projects/136f346faf7646a9a8aa645af34e11ca/users/07e0d36c35f8448cb9d993bef8ac1dad/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:24:25.257712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a5473e07-1e33-4511-ad65-9d853bf41797 None None] MFA Rules not processed for user `07e0d36c35f8448cb9d993bef8ac1dad`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:25.259472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a5473e07-1e33-4511-ad65-9d853bf41797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:25.394688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdb450-9ebf-49bc-880a-d66c660a0086 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:25.408779 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3601/7287] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:25.503062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdb450-9ebf-49bc-880a-d66c660a0086 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:25.509205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d42d3-3e24-4212-91b3-247f89735d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:25.510115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d42d3-3e24-4212-91b3-247f89735d0c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:25.510560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d42d3-3e24-4212-91b3-247f89735d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:25.511012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d42d3-3e24-4212-91b3-247f89735d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:25.511775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d42d3-3e24-4212-91b3-247f89735d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:25.514274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d42d3-3e24-4212-91b3-247f89735d0c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:25.560971 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3687/7288] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:25 2022] POST /identity/v3/projects => generated 369 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:25.567249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5473e07-1e33-4511-ad65-9d853bf41797 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:25.585488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5473e07-1e33-4511-ad65-9d853bf41797 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:25.653830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb601f0-ba40-40e3-aad5-54d948bb5cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:25.654888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb601f0-ba40-40e3-aad5-54d948bb5cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:25.655364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb601f0-ba40-40e3-aad5-54d948bb5cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:25.655828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb601f0-ba40-40e3-aad5-54d948bb5cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:25.656684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb601f0-ba40-40e3-aad5-54d948bb5cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:25.658902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb601f0-ba40-40e3-aad5-54d948bb5cc7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:25.702915 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3602/7289] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:25 2022] POST /identity/v3/users => generated 291 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:25.704566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d42d3-3e24-4212-91b3-247f89735d0c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:25.712964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:25.723012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d42d3-3e24-4212-91b3-247f89735d0c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:25.738173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2cf4f6-f5c2-47c5-9ad9-6e5ee90925d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:25.738962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cf4f6-f5c2-47c5-9ad9-6e5ee90925d8 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:25.739329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cf4f6-f5c2-47c5-9ad9-6e5ee90925d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:25.739690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cf4f6-f5c2-47c5-9ad9-6e5ee90925d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:25.740476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2cf4f6-f5c2-47c5-9ad9-6e5ee90925d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:25.742184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2cf4f6-f5c2-47c5-9ad9-6e5ee90925d8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:25.792425 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3688/7290] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:25.795427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb601f0-ba40-40e3-aad5-54d948bb5cc7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:25.818284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb601f0-ba40-40e3-aad5-54d948bb5cc7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:25.837494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af12994-3918-41a9-a65e-faef736071fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:25.838437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af12994-3918-41a9-a65e-faef736071fe admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:25.841334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af12994-3918-41a9-a65e-faef736071fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:25.841833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af12994-3918-41a9-a65e-faef736071fe admin admin] PATH_INFO: `/v3/projects/73c322df244b4b35a3d01f0c12569317/users/799404050a1b4118910975f0c27a6b74/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:25.846522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:25.893554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af12994-3918-41a9-a65e-faef736071fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c322df244b4b35a3d01f0c12569317, user_id=799404050a1b4118910975f0c27a6b74, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:25.900524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af12994-3918-41a9-a65e-faef736071fe admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:25.954752 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3603/7291] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:25 2022] PUT /identity/v3/projects/73c322df244b4b35a3d01f0c12569317/users/799404050a1b4118910975f0c27a6b74/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:25.956506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2cf4f6-f5c2-47c5-9ad9-6e5ee90925d8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:25.993348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2cf4f6-f5c2-47c5-9ad9-6e5ee90925d8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:26.143850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b8a92-966a-4707-9bf7-bd8b59cba313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:26.145874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b8a92-966a-4707-9bf7-bd8b59cba313 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:26.146629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b8a92-966a-4707-9bf7-bd8b59cba313 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:26.147250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b8a92-966a-4707-9bf7-bd8b59cba313 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:26.148758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b8a92-966a-4707-9bf7-bd8b59cba313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:26.151097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b8a92-966a-4707-9bf7-bd8b59cba313 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:26.171439 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3689/7292] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:26.173491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af12994-3918-41a9-a65e-faef736071fe admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:26.191182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af12994-3918-41a9-a65e-faef736071fe admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:26.200302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f64b3d-3ddf-4f2f-9608-5f3ca22098af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:26.201126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f64b3d-3ddf-4f2f-9608-5f3ca22098af admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:26.201457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f64b3d-3ddf-4f2f-9608-5f3ca22098af admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:26.201846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f64b3d-3ddf-4f2f-9608-5f3ca22098af admin admin] PATH_INFO: `/v3/projects/73c322df244b4b35a3d01f0c12569317/users/799404050a1b4118910975f0c27a6b74/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:26.204293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f64b3d-3ddf-4f2f-9608-5f3ca22098af admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c322df244b4b35a3d01f0c12569317, user_id=799404050a1b4118910975f0c27a6b74, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:26.207494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f64b3d-3ddf-4f2f-9608-5f3ca22098af admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:26.233726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dfccf-61cf-45e3-930e-dbc1422c80b2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:26.234125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dfccf-61cf-45e3-930e-dbc1422c80b2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:26.234486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dfccf-61cf-45e3-930e-dbc1422c80b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:26.234905 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3604/7293] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:26 2022] PUT /identity/v3/projects/73c322df244b4b35a3d01f0c12569317/users/799404050a1b4118910975f0c27a6b74/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:26.280009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-aa3dfccf-61cf-45e3-930e-dbc1422c80b2 None None] MFA Rules not processed for user `799404050a1b4118910975f0c27a6b74`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:26.281642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aa3dfccf-61cf-45e3-930e-dbc1422c80b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:26.447113 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3690/7294] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:26.461563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f64b3d-3ddf-4f2f-9608-5f3ca22098af admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:26.463856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-93f64b3d-3ddf-4f2f-9608-5f3ca22098af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:26.551498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f64b3d-3ddf-4f2f-9608-5f3ca22098af admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:26.557523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18741994-bd88-462c-a746-21f4cd95540c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:26.558188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18741994-bd88-462c-a746-21f4cd95540c service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:26.558945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18741994-bd88-462c-a746-21f4cd95540c service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:26.559306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18741994-bd88-462c-a746-21f4cd95540c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:26.588839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18741994-bd88-462c-a746-21f4cd95540c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:26.591919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18741994-bd88-462c-a746-21f4cd95540c service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:26.605991 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3605/7295] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:26.656745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3dfccf-61cf-45e3-930e-dbc1422c80b2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:26.658790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aa3dfccf-61cf-45e3-930e-dbc1422c80b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:26.676034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3dfccf-61cf-45e3-930e-dbc1422c80b2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:26.681879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a6cc0-768b-43b3-b2a7-bfb6b69158e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:26.682718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a6cc0-768b-43b3-b2a7-bfb6b69158e0 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:26.682971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a6cc0-768b-43b3-b2a7-bfb6b69158e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:26.684136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a6cc0-768b-43b3-b2a7-bfb6b69158e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:26.756689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a6cc0-768b-43b3-b2a7-bfb6b69158e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:26.758521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a6cc0-768b-43b3-b2a7-bfb6b69158e0 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:26.769928 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3691/7296] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:30.717258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18741994-bd88-462c-a746-21f4cd95540c service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:30.793092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18741994-bd88-462c-a746-21f4cd95540c service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:30.799148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:30.799970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:30.800358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:30.801018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] PATH_INFO: `/v3/users/07e0d36c35f8448cb9d993bef8ac1dad` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:30.803783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e0d36c35f8448cb9d993bef8ac1dad)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:30.805916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:30.859641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e0d36c35f8448cb9d993bef8ac1dad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:30.870443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e0d36c35f8448cb9d993bef8ac1dad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:30.885801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e0d36c35f8448cb9d993bef8ac1dad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:30.886628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e0d36c35f8448cb9d993bef8ac1dad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:30.887356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e0d36c35f8448cb9d993bef8ac1dad'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:30.890275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:30.907503 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3606/7297] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:30 2022] DELETE /identity/v3/users/07e0d36c35f8448cb9d993bef8ac1dad => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:30.912253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da11658-3587-402a-bc54-48fe02fc0743 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:30.913809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da11658-3587-402a-bc54-48fe02fc0743 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:30.914401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da11658-3587-402a-bc54-48fe02fc0743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:30.937636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0da11658-3587-402a-bc54-48fe02fc0743 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:30.939104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0da11658-3587-402a-bc54-48fe02fc0743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:30.946691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:31.059726 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3692/7298] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:31.141221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:31.143121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:31.253551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bde20-0e3d-43a5-86d7-b69b4a998a4a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:31.268319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafec406-7ee0-4562-84b2-bca154095a26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:31.269282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafec406-7ee0-4562-84b2-bca154095a26 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:31.269648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafec406-7ee0-4562-84b2-bca154095a26 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:31.270854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafec406-7ee0-4562-84b2-bca154095a26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:31.285269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafec406-7ee0-4562-84b2-bca154095a26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:31.290493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafec406-7ee0-4562-84b2-bca154095a26 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:31.305003 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3607/7299] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:31.985380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da11658-3587-402a-bc54-48fe02fc0743 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:31.987843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0da11658-3587-402a-bc54-48fe02fc0743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:32.002594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da11658-3587-402a-bc54-48fe02fc0743 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:32.009439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:32.010106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:32.010383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:32.010649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] PATH_INFO: `/v3/projects/136f346faf7646a9a8aa645af34e11ca` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:32.012031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=136f346faf7646a9a8aa645af34e11ca)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:32.014054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:32.078938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '136f346faf7646a9a8aa645af34e11ca', 'request_id': 'req-19903ea1-0741-41ac-b355-4067be270739'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.089327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '136f346faf7646a9a8aa645af34e11ca', 'request_id': 'req-19903ea1-0741-41ac-b355-4067be270739'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.096582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '136f346faf7646a9a8aa645af34e11ca', 'request_id': 'req-19903ea1-0741-41ac-b355-4067be270739'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.111267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] The token cache is being invalidate because project 136f346faf7646a9a8aa645af34e11ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:32.111987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.113042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.115110 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3693/7300] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:31 2022] DELETE /identity/v3/projects/136f346faf7646a9a8aa645af34e11ca => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:24:32.119286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafec406-7ee0-4562-84b2-bca154095a26 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:32.130398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafec406-7ee0-4562-84b2-bca154095a26 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:32.205843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d271f-a267-427a-b2e7-dcc0d880fe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:32.206973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d271f-a267-427a-b2e7-dcc0d880fe34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:32.207536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d271f-a267-427a-b2e7-dcc0d880fe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:32.207972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d271f-a267-427a-b2e7-dcc0d880fe34 admin admin] PATH_INFO: `/v3/users/799404050a1b4118910975f0c27a6b74` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:32.209296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d271f-a267-427a-b2e7-dcc0d880fe34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=799404050a1b4118910975f0c27a6b74)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:32.211307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d271f-a267-427a-b2e7-dcc0d880fe34 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:32.292111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e60d271f-a267-427a-b2e7-dcc0d880fe34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '799404050a1b4118910975f0c27a6b74'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.314956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e60d271f-a267-427a-b2e7-dcc0d880fe34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '799404050a1b4118910975f0c27a6b74'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.338916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e60d271f-a267-427a-b2e7-dcc0d880fe34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '799404050a1b4118910975f0c27a6b74'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.341357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e60d271f-a267-427a-b2e7-dcc0d880fe34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '799404050a1b4118910975f0c27a6b74'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.342597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e60d271f-a267-427a-b2e7-dcc0d880fe34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '799404050a1b4118910975f0c27a6b74'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.360139 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3608/7301] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:32 2022] DELETE /identity/v3/users/799404050a1b4118910975f0c27a6b74 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:32.687528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:32.689631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:32.719362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19903ea1-0741-41ac-b355-4067be270739 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:32.808143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e0fa6-8b7f-4811-98b3-21662ce03e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:32.808817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e0fa6-8b7f-4811-98b3-21662ce03e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:32.809085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e0fa6-8b7f-4811-98b3-21662ce03e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:32.809345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e0fa6-8b7f-4811-98b3-21662ce03e03 admin admin] PATH_INFO: `/v3/projects/73c322df244b4b35a3d01f0c12569317` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:32.812857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e0fa6-8b7f-4811-98b3-21662ce03e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73c322df244b4b35a3d01f0c12569317)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:32.816801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e0fa6-8b7f-4811-98b3-21662ce03e03 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:32.871270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:32.884959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-095e0fa6-8b7f-4811-98b3-21662ce03e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73c322df244b4b35a3d01f0c12569317', 'request_id': 'req-095e0fa6-8b7f-4811-98b3-21662ce03e03'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.912692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-095e0fa6-8b7f-4811-98b3-21662ce03e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73c322df244b4b35a3d01f0c12569317', 'request_id': 'req-095e0fa6-8b7f-4811-98b3-21662ce03e03'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.923862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-095e0fa6-8b7f-4811-98b3-21662ce03e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73c322df244b4b35a3d01f0c12569317', 'request_id': 'req-095e0fa6-8b7f-4811-98b3-21662ce03e03'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.941367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-095e0fa6-8b7f-4811-98b3-21662ce03e03 admin admin] The token cache is being invalidate because project 73c322df244b4b35a3d01f0c12569317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:32.943297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-095e0fa6-8b7f-4811-98b3-21662ce03e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.944207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-095e0fa6-8b7f-4811-98b3-21662ce03e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:32.952496 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3694/7302] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:32 2022] DELETE /identity/v3/projects/73c322df244b4b35a3d01f0c12569317 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:32.953291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4623b-3774-45b3-b608-7dc379d301de None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:32.953737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4623b-3774-45b3-b608-7dc379d301de None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:32.954025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4623b-3774-45b3-b608-7dc379d301de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:33.002358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7cb4623b-3774-45b3-b608-7dc379d301de None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:33.004341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7cb4623b-3774-45b3-b608-7dc379d301de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:33.020130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:33.095844 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3609/7303] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:33.097729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ef51c-00e2-4bc8-a97d-205b13934294 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:33.098215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ef51c-00e2-4bc8-a97d-205b13934294 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:33.098335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ef51c-00e2-4bc8-a97d-205b13934294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:33.123709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-074ef51c-00e2-4bc8-a97d-205b13934294 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:33.125263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-074ef51c-00e2-4bc8-a97d-205b13934294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:33.136961 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3695/7304] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:33.141402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4623b-3774-45b3-b608-7dc379d301de None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:33.153825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4623b-3774-45b3-b608-7dc379d301de None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:33.160895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3046fec-c8a2-48c3-a81c-23b711222c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:33.161804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3046fec-c8a2-48c3-a81c-23b711222c08 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:33.162238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3046fec-c8a2-48c3-a81c-23b711222c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:33.162662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3046fec-c8a2-48c3-a81c-23b711222c08 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:33.163612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3046fec-c8a2-48c3-a81c-23b711222c08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:24:33.164094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3046fec-c8a2-48c3-a81c-23b711222c08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:33.165903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3046fec-c8a2-48c3-a81c-23b711222c08 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:33.181052 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3610/7305] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:24:33.185343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ef51c-00e2-4bc8-a97d-205b13934294 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:33.196147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ef51c-00e2-4bc8-a97d-205b13934294 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:33.202799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123e7b7-c76f-4038-a9d2-fcff622e7e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:33.203839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123e7b7-c76f-4038-a9d2-fcff622e7e12 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:33.204527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123e7b7-c76f-4038-a9d2-fcff622e7e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:33.205630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123e7b7-c76f-4038-a9d2-fcff622e7e12 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:33.206541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123e7b7-c76f-4038-a9d2-fcff622e7e12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:33.209229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123e7b7-c76f-4038-a9d2-fcff622e7e12 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:33.256659 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3696/7306] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:33 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:33.261212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3046fec-c8a2-48c3-a81c-23b711222c08 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:33.276597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3046fec-c8a2-48c3-a81c-23b711222c08 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:33.364588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972686cd-a7c9-4742-a1e3-987738e7e7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:33.365484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972686cd-a7c9-4742-a1e3-987738e7e7c9 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:33.366357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972686cd-a7c9-4742-a1e3-987738e7e7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:33.366913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972686cd-a7c9-4742-a1e3-987738e7e7c9 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:33.367699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972686cd-a7c9-4742-a1e3-987738e7e7c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:33.370561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972686cd-a7c9-4742-a1e3-987738e7e7c9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:33.398762 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3611/7307] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:33 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:33.402930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123e7b7-c76f-4038-a9d2-fcff622e7e12 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:33.415722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123e7b7-c76f-4038-a9d2-fcff622e7e12 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:33.433351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36893b5-6694-4be5-b555-881d8d70c6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:33.434531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36893b5-6694-4be5-b555-881d8d70c6fd admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:33.435621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36893b5-6694-4be5-b555-881d8d70c6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:33.436058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36893b5-6694-4be5-b555-881d8d70c6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:33.436904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36893b5-6694-4be5-b555-881d8d70c6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:33.439609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36893b5-6694-4be5-b555-881d8d70c6fd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:33.469468 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3697/7308] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:33.473913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972686cd-a7c9-4742-a1e3-987738e7e7c9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:33.488315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972686cd-a7c9-4742-a1e3-987738e7e7c9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:33.496707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f615fe-9956-44db-b88d-9d8e31986be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:33.498139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f615fe-9956-44db-b88d-9d8e31986be7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:33.498576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f615fe-9956-44db-b88d-9d8e31986be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:33.498988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f615fe-9956-44db-b88d-9d8e31986be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:33.500225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f615fe-9956-44db-b88d-9d8e31986be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:33.502587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f615fe-9956-44db-b88d-9d8e31986be7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:33.516804 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3612/7309] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:24:33.521893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36893b5-6694-4be5-b555-881d8d70c6fd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:33.537484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36893b5-6694-4be5-b555-881d8d70c6fd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:33.549197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ce36d-ffde-4e5e-aba4-d911adb3f86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:33.550577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ce36d-ffde-4e5e-aba4-d911adb3f86f admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:33.551454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ce36d-ffde-4e5e-aba4-d911adb3f86f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:33.551974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ce36d-ffde-4e5e-aba4-d911adb3f86f admin admin] PATH_INFO: `/v3/projects/9353fd4f6493493194ef7eb5d841d32c/users/4851ba2c7afe47dda62349bff3dcb4b1/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:33.583464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ce36d-ffde-4e5e-aba4-d911adb3f86f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9353fd4f6493493194ef7eb5d841d32c, user_id=4851ba2c7afe47dda62349bff3dcb4b1, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:33.587587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ce36d-ffde-4e5e-aba4-d911adb3f86f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:33.611355 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3698/7310] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:33 2022] PUT /identity/v3/projects/9353fd4f6493493194ef7eb5d841d32c/users/4851ba2c7afe47dda62349bff3dcb4b1/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:33.623050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3de03-d6e6-489c-bd34-470ed94f3466 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:33.623569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3de03-d6e6-489c-bd34-470ed94f3466 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:33.624519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3de03-d6e6-489c-bd34-470ed94f3466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:33.664891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-52c3de03-d6e6-489c-bd34-470ed94f3466 None None] MFA Rules not processed for user `4851ba2c7afe47dda62349bff3dcb4b1`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:33.667464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-52c3de03-d6e6-489c-bd34-470ed94f3466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:33.782776 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3613/7311] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:33.783154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ce36d-ffde-4e5e-aba4-d911adb3f86f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:33.889928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ce36d-ffde-4e5e-aba4-d911adb3f86f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:33.896232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047cfe4-d4a3-493e-be4c-0ae47f3a5f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:33.897227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047cfe4-d4a3-493e-be4c-0ae47f3a5f36 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:33.897715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047cfe4-d4a3-493e-be4c-0ae47f3a5f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:33.898201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047cfe4-d4a3-493e-be4c-0ae47f3a5f36 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:33.899051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047cfe4-d4a3-493e-be4c-0ae47f3a5f36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:33.901475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047cfe4-d4a3-493e-be4c-0ae47f3a5f36 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:33.940471 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3699/7312] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:33 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:33.945828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3de03-d6e6-489c-bd34-470ed94f3466 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:33.959147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3de03-d6e6-489c-bd34-470ed94f3466 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:33.966355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:34.062168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0181f0c-cc89-4a0d-a8f6-ecd7e66a466e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:34.063678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0181f0c-cc89-4a0d-a8f6-ecd7e66a466e admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:34.063971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0181f0c-cc89-4a0d-a8f6-ecd7e66a466e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:34.064413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0181f0c-cc89-4a0d-a8f6-ecd7e66a466e admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:34.065740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0181f0c-cc89-4a0d-a8f6-ecd7e66a466e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:34.070493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0181f0c-cc89-4a0d-a8f6-ecd7e66a466e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:34.100120 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3614/7313] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:33 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:34.104701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047cfe4-d4a3-493e-be4c-0ae47f3a5f36 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:34.122549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047cfe4-d4a3-493e-be4c-0ae47f3a5f36 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:34.130195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603d740-c2d6-4317-8071-8d6674311b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:34.130831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603d740-c2d6-4317-8071-8d6674311b23 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:34.131108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603d740-c2d6-4317-8071-8d6674311b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:34.131385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603d740-c2d6-4317-8071-8d6674311b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:34.132691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603d740-c2d6-4317-8071-8d6674311b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:34.134908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603d740-c2d6-4317-8071-8d6674311b23 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:34.154205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0181f0c-cc89-4a0d-a8f6-ecd7e66a466e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:34.155164 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3700/7314] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:24:34.166592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0181f0c-cc89-4a0d-a8f6-ecd7e66a466e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:34.173210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb96605-8f15-45fb-ad64-b22ff0c6b9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:34.175013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb96605-8f15-45fb-ad64-b22ff0c6b9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:34.175126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb96605-8f15-45fb-ad64-b22ff0c6b9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:34.175375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb96605-8f15-45fb-ad64-b22ff0c6b9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:34.176195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb96605-8f15-45fb-ad64-b22ff0c6b9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:34.177846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb96605-8f15-45fb-ad64-b22ff0c6b9f0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:34.196184 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3615/7315] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:34.196807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603d740-c2d6-4317-8071-8d6674311b23 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:34.210398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603d740-c2d6-4317-8071-8d6674311b23 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:34.216274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2cdb9-6e3e-436d-ba92-aa39612a1611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:34.217523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2cdb9-6e3e-436d-ba92-aa39612a1611 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:34.217876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2cdb9-6e3e-436d-ba92-aa39612a1611 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:34.218712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2cdb9-6e3e-436d-ba92-aa39612a1611 admin admin] PATH_INFO: `/v3/projects/21950527e86d42af8f1acdc6e594e32e/users/60dad64ca8ff493899114e57171fdffc/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:34.257164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2cdb9-6e3e-436d-ba92-aa39612a1611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21950527e86d42af8f1acdc6e594e32e, user_id=60dad64ca8ff493899114e57171fdffc, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:34.261544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2cdb9-6e3e-436d-ba92-aa39612a1611 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:34.282918 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3701/7316] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:34 2022] PUT /identity/v3/projects/21950527e86d42af8f1acdc6e594e32e/users/60dad64ca8ff493899114e57171fdffc/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:24:34.396945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f4f49-0ebc-40d9-b441-bb8d9caa8629 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:34.397160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f4f49-0ebc-40d9-b441-bb8d9caa8629 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:34.397399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f4f49-0ebc-40d9-b441-bb8d9caa8629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:34.433373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c86f4f49-0ebc-40d9-b441-bb8d9caa8629 None None] MFA Rules not processed for user `60dad64ca8ff493899114e57171fdffc`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:34.434788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c86f4f49-0ebc-40d9-b441-bb8d9caa8629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:34.544312 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3616/7317] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:34.548934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2cdb9-6e3e-436d-ba92-aa39612a1611 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:34.569449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:34.697874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2cdb9-6e3e-436d-ba92-aa39612a1611 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:34.704287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c59f2c-d1a8-4bc9-849f-e530eaed6ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:34.705480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c59f2c-d1a8-4bc9-849f-e530eaed6ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:34.706381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c59f2c-d1a8-4bc9-849f-e530eaed6ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:34.706934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c59f2c-d1a8-4bc9-849f-e530eaed6ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:34.707953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c59f2c-d1a8-4bc9-849f-e530eaed6ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:34.712057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c59f2c-d1a8-4bc9-849f-e530eaed6ef4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:34.764832 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3702/7318] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:34 2022] POST /identity/v3/projects => generated 387 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:24:34.769929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f4f49-0ebc-40d9-b441-bb8d9caa8629 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:34.784230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f4f49-0ebc-40d9-b441-bb8d9caa8629 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:34.960815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38733b26-0741-4c8f-ba65-e6c9168c786e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:34.961431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38733b26-0741-4c8f-ba65-e6c9168c786e admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:34.961710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38733b26-0741-4c8f-ba65-e6c9168c786e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:34.961984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38733b26-0741-4c8f-ba65-e6c9168c786e admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:34.962675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38733b26-0741-4c8f-ba65-e6c9168c786e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:34.964842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38733b26-0741-4c8f-ba65-e6c9168c786e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:35.001041 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3617/7319] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:34 2022] POST /identity/v3/users => generated 300 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:35.006121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c59f2c-d1a8-4bc9-849f-e530eaed6ef4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:35.038692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c59f2c-d1a8-4bc9-849f-e530eaed6ef4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:35.053712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bf939-9e38-4440-add4-27e413cf474a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:35.054395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bf939-9e38-4440-add4-27e413cf474a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:35.054939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bf939-9e38-4440-add4-27e413cf474a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:35.054939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bf939-9e38-4440-add4-27e413cf474a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:35.059100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8bf939-9e38-4440-add4-27e413cf474a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:35.060883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8bf939-9e38-4440-add4-27e413cf474a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:35.092435 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3703/7320] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:24:35.096720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38733b26-0741-4c8f-ba65-e6c9168c786e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:35.117669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38733b26-0741-4c8f-ba65-e6c9168c786e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:35.141221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcc59e-0d6a-4530-b9c2-de32f45c8cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:35.142333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcc59e-0d6a-4530-b9c2-de32f45c8cae admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:35.142633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcc59e-0d6a-4530-b9c2-de32f45c8cae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:35.144148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcc59e-0d6a-4530-b9c2-de32f45c8cae admin admin] PATH_INFO: `/v3/projects/bc1df643b93a415dbd4b4cac2f0b2080/users/e7cb7eaff170455b8cb6aa3ef5d045df/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:35.151154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:35.186377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcc59e-0d6a-4530-b9c2-de32f45c8cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc1df643b93a415dbd4b4cac2f0b2080, user_id=e7cb7eaff170455b8cb6aa3ef5d045df, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:35.195415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcc59e-0d6a-4530-b9c2-de32f45c8cae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:35.219396 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3618/7321] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:35 2022] PUT /identity/v3/projects/bc1df643b93a415dbd4b4cac2f0b2080/users/e7cb7eaff170455b8cb6aa3ef5d045df/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:35.224002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bf939-9e38-4440-add4-27e413cf474a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:35.255294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bf939-9e38-4440-add4-27e413cf474a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:35.400415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f58a5-d90c-4540-800d-36389328ce13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:35.401294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f58a5-d90c-4540-800d-36389328ce13 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:35.401762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f58a5-d90c-4540-800d-36389328ce13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:35.402232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f58a5-d90c-4540-800d-36389328ce13 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:35.403108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f58a5-d90c-4540-800d-36389328ce13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:35.406291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f58a5-d90c-4540-800d-36389328ce13 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:35.421700 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3704/7322] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:35.429527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcc59e-0d6a-4530-b9c2-de32f45c8cae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:35.450886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcc59e-0d6a-4530-b9c2-de32f45c8cae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:35.470212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566987f0-bc03-47fd-ae42-6307d6ddfa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:35.471070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566987f0-bc03-47fd-ae42-6307d6ddfa89 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:35.472024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566987f0-bc03-47fd-ae42-6307d6ddfa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:35.472350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566987f0-bc03-47fd-ae42-6307d6ddfa89 admin admin] PATH_INFO: `/v3/projects/bc1df643b93a415dbd4b4cac2f0b2080/users/e7cb7eaff170455b8cb6aa3ef5d045df/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:35.475456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566987f0-bc03-47fd-ae42-6307d6ddfa89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc1df643b93a415dbd4b4cac2f0b2080, user_id=e7cb7eaff170455b8cb6aa3ef5d045df, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:35.479018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566987f0-bc03-47fd-ae42-6307d6ddfa89 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:35.495909 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3619/7323] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:35 2022] PUT /identity/v3/projects/bc1df643b93a415dbd4b4cac2f0b2080/users/e7cb7eaff170455b8cb6aa3ef5d045df/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:35.506468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff77ae-9261-4f89-91fe-b504b1b57c32 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:35.506745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff77ae-9261-4f89-91fe-b504b1b57c32 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:35.507522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff77ae-9261-4f89-91fe-b504b1b57c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:35.570181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a3ff77ae-9261-4f89-91fe-b504b1b57c32 None None] MFA Rules not processed for user `e7cb7eaff170455b8cb6aa3ef5d045df`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:35.572151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a3ff77ae-9261-4f89-91fe-b504b1b57c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:35.681259 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3705/7324] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:35.694984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566987f0-bc03-47fd-ae42-6307d6ddfa89 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:35.697289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-566987f0-bc03-47fd-ae42-6307d6ddfa89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:35.841114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566987f0-bc03-47fd-ae42-6307d6ddfa89 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:35.853191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429e1b4-a89c-4ceb-83d4-fc44f92d3c56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:35.853993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429e1b4-a89c-4ceb-83d4-fc44f92d3c56 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:35.854375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429e1b4-a89c-4ceb-83d4-fc44f92d3c56 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:35.854742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429e1b4-a89c-4ceb-83d4-fc44f92d3c56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:35.874041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429e1b4-a89c-4ceb-83d4-fc44f92d3c56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:35.876299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429e1b4-a89c-4ceb-83d4-fc44f92d3c56 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:35.889370 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3620/7325] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:36.859047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff77ae-9261-4f89-91fe-b504b1b57c32 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:36.860739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a3ff77ae-9261-4f89-91fe-b504b1b57c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:36.880781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff77ae-9261-4f89-91fe-b504b1b57c32 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:36.888171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2f43c-4314-44dc-9d18-6efb4e05e553 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:36.889165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2f43c-4314-44dc-9d18-6efb4e05e553 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:36.890067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2f43c-4314-44dc-9d18-6efb4e05e553 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:36.890588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2f43c-4314-44dc-9d18-6efb4e05e553 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:36.900331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:36.973040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2f43c-4314-44dc-9d18-6efb4e05e553 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:36.975079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2f43c-4314-44dc-9d18-6efb4e05e553 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:36.992178 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3706/7326] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:39.569098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429e1b4-a89c-4ceb-83d4-fc44f92d3c56 service cinder] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:39.571069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c429e1b4-a89c-4ceb-83d4-fc44f92d3c56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:39.688167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429e1b4-a89c-4ceb-83d4-fc44f92d3c56 service cinder] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:39.696288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb76888-4cb9-4754-b951-c38552288814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:39.697154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb76888-4cb9-4754-b951-c38552288814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:39.697640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb76888-4cb9-4754-b951-c38552288814 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:39.698085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb76888-4cb9-4754-b951-c38552288814 admin admin] PATH_INFO: `/v3/users/d5c97cfa7d924c3491f9e78b5051f0f8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:39.700069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb76888-4cb9-4754-b951-c38552288814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5c97cfa7d924c3491f9e78b5051f0f8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:39.702491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb76888-4cb9-4754-b951-c38552288814 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:39.781334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecb76888-4cb9-4754-b951-c38552288814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5c97cfa7d924c3491f9e78b5051f0f8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:39.790771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecb76888-4cb9-4754-b951-c38552288814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5c97cfa7d924c3491f9e78b5051f0f8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:39.807002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecb76888-4cb9-4754-b951-c38552288814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5c97cfa7d924c3491f9e78b5051f0f8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:39.808142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecb76888-4cb9-4754-b951-c38552288814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5c97cfa7d924c3491f9e78b5051f0f8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:39.808662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecb76888-4cb9-4754-b951-c38552288814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5c97cfa7d924c3491f9e78b5051f0f8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:39.814293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:39.833111 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3621/7327] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:39 2022] DELETE /identity/v3/users/d5c97cfa7d924c3491f9e78b5051f0f8 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:40.114391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2f43c-4314-44dc-9d18-6efb4e05e553 service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:40.126509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2f43c-4314-44dc-9d18-6efb4e05e553 service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:40.229460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:40.230275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:40.230891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:40.231361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2 admin admin] PATH_INFO: `/v3/projects/0dd98141e3a44c969aa5cd38d24e0dbd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:40.232982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dd98141e3a44c969aa5cd38d24e0dbd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:40.235395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:40.377188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dd98141e3a44c969aa5cd38d24e0dbd', 'request_id': 'req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:40.409678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dd98141e3a44c969aa5cd38d24e0dbd', 'request_id': 'req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:40.421617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dd98141e3a44c969aa5cd38d24e0dbd', 'request_id': 'req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:40.441983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2 admin admin] The token cache is being invalidate because project 0dd98141e3a44c969aa5cd38d24e0dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:40.444108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:40.445018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1629a0cf-4b3f-4edf-a1ac-ea65108bb6c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:40.447875 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3707/7328] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:40 2022] DELETE /identity/v3/projects/0dd98141e3a44c969aa5cd38d24e0dbd => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:40.453367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89f9fc-0449-4b82-b3d1-04cef6a35e8c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:40.453638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89f9fc-0449-4b82-b3d1-04cef6a35e8c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:40.453795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89f9fc-0449-4b82-b3d1-04cef6a35e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:40.492301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4d89f9fc-0449-4b82-b3d1-04cef6a35e8c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:40.493936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4d89f9fc-0449-4b82-b3d1-04cef6a35e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:40.692464 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3622/7329] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:40.695005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0f6e0-b830-464f-b7e2-d26327255d4b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:40.696101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0f6e0-b830-464f-b7e2-d26327255d4b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:40.696658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0f6e0-b830-464f-b7e2-d26327255d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:40.733004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1eb0f6e0-b830-464f-b7e2-d26327255d4b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:40.735419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1eb0f6e0-b830-464f-b7e2-d26327255d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:40.746953 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3708/7330] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:40.750928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89f9fc-0449-4b82-b3d1-04cef6a35e8c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:40.773441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89f9fc-0449-4b82-b3d1-04cef6a35e8c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:40.787958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbc74d-150c-4fcd-a01e-419ca2221b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:40.788663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbc74d-150c-4fcd-a01e-419ca2221b90 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:40.788947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbc74d-150c-4fcd-a01e-419ca2221b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:40.789211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbc74d-150c-4fcd-a01e-419ca2221b90 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:40.789877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbc74d-150c-4fcd-a01e-419ca2221b90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:24:40.790182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbc74d-150c-4fcd-a01e-419ca2221b90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:40.792279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbc74d-150c-4fcd-a01e-419ca2221b90 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:40.819757 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3623/7331] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:24:40.824316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0f6e0-b830-464f-b7e2-d26327255d4b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:40.843647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0f6e0-b830-464f-b7e2-d26327255d4b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:40.854419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516d554-489e-41fb-8cab-3bea2820c37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:40.856206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516d554-489e-41fb-8cab-3bea2820c37e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:40.856581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516d554-489e-41fb-8cab-3bea2820c37e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:40.857293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516d554-489e-41fb-8cab-3bea2820c37e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:40.859275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516d554-489e-41fb-8cab-3bea2820c37e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:40.865969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516d554-489e-41fb-8cab-3bea2820c37e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:40.921274 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3709/7332] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:40 2022] POST /identity/v3/projects => generated 397 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:40.925109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbc74d-150c-4fcd-a01e-419ca2221b90 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:40.940091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbc74d-150c-4fcd-a01e-419ca2221b90 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:41.086196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0f5a6-f673-431a-a3fe-93d9e188b0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:41.087107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0f5a6-f673-431a-a3fe-93d9e188b0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:41.089500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0f5a6-f673-431a-a3fe-93d9e188b0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:41.090037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0f5a6-f673-431a-a3fe-93d9e188b0a2 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:41.090834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0f5a6-f673-431a-a3fe-93d9e188b0a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:41.093569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0f5a6-f673-431a-a3fe-93d9e188b0a2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:41.132987 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3624/7333] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:40 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:41.138724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516d554-489e-41fb-8cab-3bea2820c37e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:41.170223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516d554-489e-41fb-8cab-3bea2820c37e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:41.187601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039d481-ca63-4dbe-bcb5-b02396c16387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:41.189420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039d481-ca63-4dbe-bcb5-b02396c16387 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:41.190445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039d481-ca63-4dbe-bcb5-b02396c16387 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:41.190949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039d481-ca63-4dbe-bcb5-b02396c16387 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:41.191872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039d481-ca63-4dbe-bcb5-b02396c16387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:41.195012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039d481-ca63-4dbe-bcb5-b02396c16387 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:41.199395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:41.226490 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3710/7334] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:41.227186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0f5a6-f673-431a-a3fe-93d9e188b0a2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:41.250965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0f5a6-f673-431a-a3fe-93d9e188b0a2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:41.258195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888c137-153f-4d4e-bdc7-5f60db72e59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:41.259816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888c137-153f-4d4e-bdc7-5f60db72e59f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:41.260317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888c137-153f-4d4e-bdc7-5f60db72e59f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:41.260788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888c137-153f-4d4e-bdc7-5f60db72e59f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:41.261782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888c137-153f-4d4e-bdc7-5f60db72e59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:41.264579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888c137-153f-4d4e-bdc7-5f60db72e59f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:41.283452 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3625/7335] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:41.284404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039d481-ca63-4dbe-bcb5-b02396c16387 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:41.301159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039d481-ca63-4dbe-bcb5-b02396c16387 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:41.310545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b2da3-58c6-49f7-b40f-4054e5ce1510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:41.311287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b2da3-58c6-49f7-b40f-4054e5ce1510 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:41.311566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b2da3-58c6-49f7-b40f-4054e5ce1510 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:41.312812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b2da3-58c6-49f7-b40f-4054e5ce1510 admin admin] PATH_INFO: `/v3/projects/8072f03884004cddb2e3a429846e3257/users/23301df1799c4676af4675f58c32a49b/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:41.338922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b2da3-58c6-49f7-b40f-4054e5ce1510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8072f03884004cddb2e3a429846e3257, user_id=23301df1799c4676af4675f58c32a49b, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:41.342336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b2da3-58c6-49f7-b40f-4054e5ce1510 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:41.373139 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3711/7336] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:41 2022] PUT /identity/v3/projects/8072f03884004cddb2e3a429846e3257/users/23301df1799c4676af4675f58c32a49b/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:41.379668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d374c-c148-4bb8-bba4-72ec9cfb9808 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:41.379959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d374c-c148-4bb8-bba4-72ec9cfb9808 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:41.380232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d374c-c148-4bb8-bba4-72ec9cfb9808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:41.404639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-294d374c-c148-4bb8-bba4-72ec9cfb9808 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:41.405938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-294d374c-c148-4bb8-bba4-72ec9cfb9808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:41.509966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:41.537956 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3626/7337] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:41.597517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b2da3-58c6-49f7-b40f-4054e5ce1510 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:41.599831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5f6b2da3-58c6-49f7-b40f-4054e5ce1510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:41.707904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d374c-c148-4bb8-bba4-72ec9cfb9808 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:41.710145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-294d374c-c148-4bb8-bba4-72ec9cfb9808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:41.737848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d374c-c148-4bb8-bba4-72ec9cfb9808 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:41.765830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:41.767038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:41.767571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:41.770740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] PATH_INFO: `/v3/users/4851ba2c7afe47dda62349bff3dcb4b1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:41.772176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4851ba2c7afe47dda62349bff3dcb4b1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:41.774221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:41.823189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b2da3-58c6-49f7-b40f-4054e5ce1510 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:41.831291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca24a37-2009-4cf8-b658-255d3d3faffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:41.833034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca24a37-2009-4cf8-b658-255d3d3faffe service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:41.834487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca24a37-2009-4cf8-b658-255d3d3faffe service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:41.835584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca24a37-2009-4cf8-b658-255d3d3faffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:41.862223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca24a37-2009-4cf8-b658-255d3d3faffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:41.865460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca24a37-2009-4cf8-b658-255d3d3faffe service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:41.874871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4851ba2c7afe47dda62349bff3dcb4b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:41.880551 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3712/7338] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:41.886417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4851ba2c7afe47dda62349bff3dcb4b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:41.901078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4851ba2c7afe47dda62349bff3dcb4b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:41.902991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4851ba2c7afe47dda62349bff3dcb4b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:41.903784 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4851ba2c7afe47dda62349bff3dcb4b1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:41.925536 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3627/7339] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:41 2022] DELETE /identity/v3/users/4851ba2c7afe47dda62349bff3dcb4b1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:24:41.927579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7542a1e-42c3-48a8-9771-2ef31cf916d7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:41.928068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7542a1e-42c3-48a8-9771-2ef31cf916d7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:41.928340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7542a1e-42c3-48a8-9771-2ef31cf916d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:41.951058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e7542a1e-42c3-48a8-9771-2ef31cf916d7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:41.952517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e7542a1e-42c3-48a8-9771-2ef31cf916d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:42.093181 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3713/7340] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:42.109499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:42.109499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:42.124610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7542a1e-42c3-48a8-9771-2ef31cf916d7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:42.137170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e7542a1e-42c3-48a8-9771-2ef31cf916d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:42.292478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7542a1e-42c3-48a8-9771-2ef31cf916d7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:42.303300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e81e2a-408a-4651-a371-2da040838a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:42.303993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e81e2a-408a-4651-a371-2da040838a49 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:42.304332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e81e2a-408a-4651-a371-2da040838a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:42.304973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e81e2a-408a-4651-a371-2da040838a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:42.307206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-14e81e2a-408a-4651-a371-2da040838a49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:42.308530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309a727-3345-44d1-9d2f-442530523f3b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:42.316099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eb09a-ed4c-4d07-9b4b-ad83260f18d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:42.316492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eb09a-ed4c-4d07-9b4b-ad83260f18d7 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:42.316771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eb09a-ed4c-4d07-9b4b-ad83260f18d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:42.317036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eb09a-ed4c-4d07-9b4b-ad83260f18d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:42.333139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8eb09a-ed4c-4d07-9b4b-ad83260f18d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:42.334715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8eb09a-ed4c-4d07-9b4b-ad83260f18d7 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:42.346390 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3628/7341] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:42.378915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:42.427682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e81e2a-408a-4651-a371-2da040838a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:42.429991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e81e2a-408a-4651-a371-2da040838a49 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:42.443944 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3714/7342] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:42.821250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eb09a-ed4c-4d07-9b4b-ad83260f18d7 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:42.822748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-eb8eb09a-ed4c-4d07-9b4b-ad83260f18d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:42.836860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eb09a-ed4c-4d07-9b4b-ad83260f18d7 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:42.851000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df097dd-813c-4961-94cb-f60e18113b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:42.851664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df097dd-813c-4961-94cb-f60e18113b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:42.851905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df097dd-813c-4961-94cb-f60e18113b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:42.852173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df097dd-813c-4961-94cb-f60e18113b37 admin admin] PATH_INFO: `/v3/projects/9353fd4f6493493194ef7eb5d841d32c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:42.853471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df097dd-813c-4961-94cb-f60e18113b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9353fd4f6493493194ef7eb5d841d32c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:42.855418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df097dd-813c-4961-94cb-f60e18113b37 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:42.909652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0df097dd-813c-4961-94cb-f60e18113b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9353fd4f6493493194ef7eb5d841d32c', 'request_id': 'req-0df097dd-813c-4961-94cb-f60e18113b37'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:42.916348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0df097dd-813c-4961-94cb-f60e18113b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9353fd4f6493493194ef7eb5d841d32c', 'request_id': 'req-0df097dd-813c-4961-94cb-f60e18113b37'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:42.925769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0df097dd-813c-4961-94cb-f60e18113b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9353fd4f6493493194ef7eb5d841d32c', 'request_id': 'req-0df097dd-813c-4961-94cb-f60e18113b37'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:42.931201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:42.939620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0df097dd-813c-4961-94cb-f60e18113b37 admin admin] The token cache is being invalidate because project 9353fd4f6493493194ef7eb5d841d32c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:42.941105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0df097dd-813c-4961-94cb-f60e18113b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:42.941885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0df097dd-813c-4961-94cb-f60e18113b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:42.944234 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3629/7343] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:42 2022] DELETE /identity/v3/projects/9353fd4f6493493194ef7eb5d841d32c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:42.948020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e81e2a-408a-4651-a371-2da040838a49 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:42.959384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e81e2a-408a-4651-a371-2da040838a49 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:43.037821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:43.038882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:43.039622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:43.039622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] PATH_INFO: `/v3/users/60dad64ca8ff493899114e57171fdffc` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:43.040947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=60dad64ca8ff493899114e57171fdffc)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:43.043226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:43.072209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1e4fb-9bb1-4226-ae3a-1c11271167ca None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:43.072526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1e4fb-9bb1-4226-ae3a-1c11271167ca None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:43.072803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1e4fb-9bb1-4226-ae3a-1c11271167ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:43.097225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-dee1e4fb-9bb1-4226-ae3a-1c11271167ca None None] MFA Rules not processed for user `4e6a5f35d172410481a8416b7bdaafac`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:43.098157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dee1e4fb-9bb1-4226-ae3a-1c11271167ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:43.110331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60dad64ca8ff493899114e57171fdffc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:43.121826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60dad64ca8ff493899114e57171fdffc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:43.134756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60dad64ca8ff493899114e57171fdffc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:43.154807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60dad64ca8ff493899114e57171fdffc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:43.155712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60dad64ca8ff493899114e57171fdffc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:43.163153 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3715/7344] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:42 2022] DELETE /identity/v3/users/60dad64ca8ff493899114e57171fdffc => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:43.173417 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3630/7345] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:24:43.892940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:43.913021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:44.068408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:44.124781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725b44d-b47c-4a85-ae73-47e2aad259ff admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:44.135394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:44.137299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:44.139486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:44.139789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] PATH_INFO: `/v3/projects/21950527e86d42af8f1acdc6e594e32e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:44.146635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] RBAC: Authorizing `identity:delete_project(project_id=21950527e86d42af8f1acdc6e594e32e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:44.150848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:44.259392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21950527e86d42af8f1acdc6e594e32e', 'request_id': 'req-68b62683-f0fc-466d-b01e-f771b0d4311f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:44.273999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21950527e86d42af8f1acdc6e594e32e', 'request_id': 'req-68b62683-f0fc-466d-b01e-f771b0d4311f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:44.288294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21950527e86d42af8f1acdc6e594e32e', 'request_id': 'req-68b62683-f0fc-466d-b01e-f771b0d4311f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:44.309993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] The token cache is being invalidate because project 21950527e86d42af8f1acdc6e594e32e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:44.310287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:44.311204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:44.313909 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3716/7346] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:43 2022] DELETE /identity/v3/projects/21950527e86d42af8f1acdc6e594e32e => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:44.317213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1e4fb-9bb1-4226-ae3a-1c11271167ca None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:44.333253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1e4fb-9bb1-4226-ae3a-1c11271167ca None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:44.425636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8aafd-e1d9-406f-9fef-44224f511428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:44.426678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8aafd-e1d9-406f-9fef-44224f511428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:44.427153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8aafd-e1d9-406f-9fef-44224f511428 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:44.428010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8aafd-e1d9-406f-9fef-44224f511428 admin admin] PATH_INFO: `/v3/users/e7cb7eaff170455b8cb6aa3ef5d045df` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:44.429742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8aafd-e1d9-406f-9fef-44224f511428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7cb7eaff170455b8cb6aa3ef5d045df)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:44.433645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8aafd-e1d9-406f-9fef-44224f511428 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:44.560610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-30c8aafd-e1d9-406f-9fef-44224f511428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7cb7eaff170455b8cb6aa3ef5d045df'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:44.587520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-30c8aafd-e1d9-406f-9fef-44224f511428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7cb7eaff170455b8cb6aa3ef5d045df'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:44.613914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-30c8aafd-e1d9-406f-9fef-44224f511428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7cb7eaff170455b8cb6aa3ef5d045df'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:44.615415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-30c8aafd-e1d9-406f-9fef-44224f511428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7cb7eaff170455b8cb6aa3ef5d045df'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:44.616591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-30c8aafd-e1d9-406f-9fef-44224f511428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7cb7eaff170455b8cb6aa3ef5d045df'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:44.640815 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3631/7347] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:44 2022] DELETE /identity/v3/users/e7cb7eaff170455b8cb6aa3ef5d045df => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:24:46.260929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:46.262489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:46.283569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b62683-f0fc-466d-b01e-f771b0d4311f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:46.410717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1369d3b-655c-452b-a6c5-b099b49f3083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:46.411931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1369d3b-655c-452b-a6c5-b099b49f3083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:46.412483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1369d3b-655c-452b-a6c5-b099b49f3083 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:46.412905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1369d3b-655c-452b-a6c5-b099b49f3083 admin admin] PATH_INFO: `/v3/projects/bc1df643b93a415dbd4b4cac2f0b2080` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:46.418546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1369d3b-655c-452b-a6c5-b099b49f3083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc1df643b93a415dbd4b4cac2f0b2080)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:46.421093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1369d3b-655c-452b-a6c5-b099b49f3083 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:46.515022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1369d3b-655c-452b-a6c5-b099b49f3083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc1df643b93a415dbd4b4cac2f0b2080', 'request_id': 'req-c1369d3b-655c-452b-a6c5-b099b49f3083'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:46.525405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1369d3b-655c-452b-a6c5-b099b49f3083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc1df643b93a415dbd4b4cac2f0b2080', 'request_id': 'req-c1369d3b-655c-452b-a6c5-b099b49f3083'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:46.534366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1369d3b-655c-452b-a6c5-b099b49f3083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc1df643b93a415dbd4b4cac2f0b2080', 'request_id': 'req-c1369d3b-655c-452b-a6c5-b099b49f3083'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:46.551231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1369d3b-655c-452b-a6c5-b099b49f3083 admin admin] The token cache is being invalidate because project bc1df643b93a415dbd4b4cac2f0b2080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:46.551806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1369d3b-655c-452b-a6c5-b099b49f3083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:46.552805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c1369d3b-655c-452b-a6c5-b099b49f3083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:46.554498 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3717/7348] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:46 2022] DELETE /identity/v3/projects/bc1df643b93a415dbd4b4cac2f0b2080 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:46.561490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de0596-ef43-4580-b5e5-66b740dc0b5c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:46.562116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de0596-ef43-4580-b5e5-66b740dc0b5c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:46.562116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de0596-ef43-4580-b5e5-66b740dc0b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:46.595044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c2de0596-ef43-4580-b5e5-66b740dc0b5c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:46.596935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c2de0596-ef43-4580-b5e5-66b740dc0b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:46.624508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:46.750165 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3632/7349] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:46.759465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1ea4f-e073-486f-9245-75561d0988b2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:46.759787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1ea4f-e073-486f-9245-75561d0988b2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:46.760107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1ea4f-e073-486f-9245-75561d0988b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:46.776195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:46.810558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-67e1ea4f-e073-486f-9245-75561d0988b2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:46.812053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-67e1ea4f-e073-486f-9245-75561d0988b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:46.825347 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3718/7350] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:46.829705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de0596-ef43-4580-b5e5-66b740dc0b5c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:46.844460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de0596-ef43-4580-b5e5-66b740dc0b5c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:46.860184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4bc38-bddb-4266-a300-e57958e72945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:46.860917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4bc38-bddb-4266-a300-e57958e72945 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:46.861233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4bc38-bddb-4266-a300-e57958e72945 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:46.861632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4bc38-bddb-4266-a300-e57958e72945 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:46.862367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4bc38-bddb-4266-a300-e57958e72945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:24:46.862804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4bc38-bddb-4266-a300-e57958e72945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:46.864519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4bc38-bddb-4266-a300-e57958e72945 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:46.889693 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3633/7351] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:24:46.889912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1ea4f-e073-486f-9245-75561d0988b2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:46.903587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1ea4f-e073-486f-9245-75561d0988b2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:46.919698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127728fc-b8e3-42c2-86a3-a597b857dd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:46.920812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127728fc-b8e3-42c2-86a3-a597b857dd2e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:46.921356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127728fc-b8e3-42c2-86a3-a597b857dd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:46.921811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127728fc-b8e3-42c2-86a3-a597b857dd2e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:46.922725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127728fc-b8e3-42c2-86a3-a597b857dd2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:46.925276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127728fc-b8e3-42c2-86a3-a597b857dd2e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:46.966800 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3719/7352] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:46 2022] POST /identity/v3/projects => generated 407 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:46.973107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4bc38-bddb-4266-a300-e57958e72945 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:47.002194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4bc38-bddb-4266-a300-e57958e72945 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:47.171706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec85b58-feb2-4cdf-83b3-cc32e699c151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:47.172345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec85b58-feb2-4cdf-83b3-cc32e699c151 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:47.172686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec85b58-feb2-4cdf-83b3-cc32e699c151 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:47.174229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec85b58-feb2-4cdf-83b3-cc32e699c151 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:47.174917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec85b58-feb2-4cdf-83b3-cc32e699c151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:47.177138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec85b58-feb2-4cdf-83b3-cc32e699c151 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:47.214268 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3634/7353] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:46 2022] POST /identity/v3/users => generated 304 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:47.218542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127728fc-b8e3-42c2-86a3-a597b857dd2e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:47.257199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127728fc-b8e3-42c2-86a3-a597b857dd2e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:47.264001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228c9c0-c643-458d-b462-6e245393392e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:47.269265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228c9c0-c643-458d-b462-6e245393392e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:47.270428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228c9c0-c643-458d-b462-6e245393392e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:47.271029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228c9c0-c643-458d-b462-6e245393392e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:47.271903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228c9c0-c643-458d-b462-6e245393392e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:47.274504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228c9c0-c643-458d-b462-6e245393392e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:47.296282 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3720/7354] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:47.301132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec85b58-feb2-4cdf-83b3-cc32e699c151 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:47.327195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec85b58-feb2-4cdf-83b3-cc32e699c151 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:47.339401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b00e2-468b-49a1-bad1-b87a496c5032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:47.349040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b00e2-468b-49a1-bad1-b87a496c5032 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:47.349897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b00e2-468b-49a1-bad1-b87a496c5032 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:47.350448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b00e2-468b-49a1-bad1-b87a496c5032 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:47.351979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b00e2-468b-49a1-bad1-b87a496c5032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:47.354262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b00e2-468b-49a1-bad1-b87a496c5032 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:47.376235 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3635/7355] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:47.380451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228c9c0-c643-458d-b462-6e245393392e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:47.394504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228c9c0-c643-458d-b462-6e245393392e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:47.402833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ab8af-497e-4a73-b051-2417c1635ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:47.405047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ab8af-497e-4a73-b051-2417c1635ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:47.405518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ab8af-497e-4a73-b051-2417c1635ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:47.405974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ab8af-497e-4a73-b051-2417c1635ad5 admin admin] PATH_INFO: `/v3/projects/b7de2505c08c422e99c2e2bda2f50e2b/users/f6a97852b65a48e28d96305d172edf40/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:47.431816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ab8af-497e-4a73-b051-2417c1635ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7de2505c08c422e99c2e2bda2f50e2b, user_id=f6a97852b65a48e28d96305d172edf40, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:47.436455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ab8af-497e-4a73-b051-2417c1635ad5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:47.491974 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3721/7356] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:47 2022] PUT /identity/v3/projects/b7de2505c08c422e99c2e2bda2f50e2b/users/f6a97852b65a48e28d96305d172edf40/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:24:47.504400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81834e1-098f-4629-b408-31b93de35796 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:47.504711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81834e1-098f-4629-b408-31b93de35796 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:47.505037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81834e1-098f-4629-b408-31b93de35796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:47.552128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f81834e1-098f-4629-b408-31b93de35796 None None] MFA Rules not processed for user `f6a97852b65a48e28d96305d172edf40`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:47.554094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f81834e1-098f-4629-b408-31b93de35796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:47.663839 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3636/7357] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:47 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:47.669502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ab8af-497e-4a73-b051-2417c1635ad5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:47.794136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ab8af-497e-4a73-b051-2417c1635ad5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:47.801488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478cb5c-1528-4b0f-892e-97454139d441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:47.802594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478cb5c-1528-4b0f-892e-97454139d441 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:47.803147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478cb5c-1528-4b0f-892e-97454139d441 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:47.803651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478cb5c-1528-4b0f-892e-97454139d441 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:47.804636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478cb5c-1528-4b0f-892e-97454139d441 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:47.808078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478cb5c-1528-4b0f-892e-97454139d441 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:47.872073 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3722/7358] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:47 2022] POST /identity/v3/projects => generated 407 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:47.876675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81834e1-098f-4629-b408-31b93de35796 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:47.891591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81834e1-098f-4629-b408-31b93de35796 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:47.933718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:48.072263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb3447-6dcc-4d63-a637-e09bd37042e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:48.076248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb3447-6dcc-4d63-a637-e09bd37042e7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:48.076720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb3447-6dcc-4d63-a637-e09bd37042e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:48.077202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb3447-6dcc-4d63-a637-e09bd37042e7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:48.077961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb3447-6dcc-4d63-a637-e09bd37042e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:48.086110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb3447-6dcc-4d63-a637-e09bd37042e7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:48.133866 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3637/7359] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:47 2022] POST /identity/v3/users => generated 310 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:24:48.138690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478cb5c-1528-4b0f-892e-97454139d441 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:48.161117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478cb5c-1528-4b0f-892e-97454139d441 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:48.177272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428a19d-3438-415a-8b7d-c7ae36beaa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:48.177954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428a19d-3438-415a-8b7d-c7ae36beaa62 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:48.178237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428a19d-3438-415a-8b7d-c7ae36beaa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:48.178507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428a19d-3438-415a-8b7d-c7ae36beaa62 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:48.179211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428a19d-3438-415a-8b7d-c7ae36beaa62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:48.180906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428a19d-3438-415a-8b7d-c7ae36beaa62 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:48.207541 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3723/7360] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:48.212796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb3447-6dcc-4d63-a637-e09bd37042e7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:48.245919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb3447-6dcc-4d63-a637-e09bd37042e7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:48.258842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47cd15-ce4f-432d-93d7-b8ce6c905fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:48.259782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47cd15-ce4f-432d-93d7-b8ce6c905fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:48.260202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47cd15-ce4f-432d-93d7-b8ce6c905fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:48.260640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47cd15-ce4f-432d-93d7-b8ce6c905fc2 admin admin] PATH_INFO: `/v3/projects/10d02626f99c45cfbaf7b871f7c281fd/users/0d02f97dc0ac4f61b7e7f2d35bfac9c8/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:48.309388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47cd15-ce4f-432d-93d7-b8ce6c905fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d02626f99c45cfbaf7b871f7c281fd, user_id=0d02f97dc0ac4f61b7e7f2d35bfac9c8, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:48.314033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47cd15-ce4f-432d-93d7-b8ce6c905fc2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:48.363791 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3638/7361] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:48 2022] PUT /identity/v3/projects/10d02626f99c45cfbaf7b871f7c281fd/users/0d02f97dc0ac4f61b7e7f2d35bfac9c8/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:48.368675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428a19d-3438-415a-8b7d-c7ae36beaa62 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:48.381417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428a19d-3438-415a-8b7d-c7ae36beaa62 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:48.513467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a9cb3-5ae1-46ae-bc35-2834c7035f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:48.514581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9cb3-5ae1-46ae-bc35-2834c7035f3c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:48.515098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9cb3-5ae1-46ae-bc35-2834c7035f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:48.516323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9cb3-5ae1-46ae-bc35-2834c7035f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:48.518059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9cb3-5ae1-46ae-bc35-2834c7035f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:48.522801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9cb3-5ae1-46ae-bc35-2834c7035f3c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:48.550472 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3724/7362] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:48.554873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47cd15-ce4f-432d-93d7-b8ce6c905fc2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:48.573417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47cd15-ce4f-432d-93d7-b8ce6c905fc2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:48.591752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3764f55-8576-48fd-948f-69e552438be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:48.592613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3764f55-8576-48fd-948f-69e552438be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:48.592994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3764f55-8576-48fd-948f-69e552438be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:48.593385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3764f55-8576-48fd-948f-69e552438be2 admin admin] PATH_INFO: `/v3/projects/10d02626f99c45cfbaf7b871f7c281fd/users/0d02f97dc0ac4f61b7e7f2d35bfac9c8/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:48.596475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3764f55-8576-48fd-948f-69e552438be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d02626f99c45cfbaf7b871f7c281fd, user_id=0d02f97dc0ac4f61b7e7f2d35bfac9c8, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:48.600112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3764f55-8576-48fd-948f-69e552438be2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:48.616855 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3639/7363] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:48 2022] PUT /identity/v3/projects/10d02626f99c45cfbaf7b871f7c281fd/users/0d02f97dc0ac4f61b7e7f2d35bfac9c8/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:48.630100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174ab0b-6778-465d-ad98-442f0e6f1676 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:48.630461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174ab0b-6778-465d-ad98-442f0e6f1676 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:48.630652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174ab0b-6778-465d-ad98-442f0e6f1676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:48.670220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:48.683607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4174ab0b-6778-465d-ad98-442f0e6f1676 None None] MFA Rules not processed for user `0d02f97dc0ac4f61b7e7f2d35bfac9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:48.685519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4174ab0b-6778-465d-ad98-442f0e6f1676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:48.804449 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3725/7364] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:48 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:48.817102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3764f55-8576-48fd-948f-69e552438be2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:48.820689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f3764f55-8576-48fd-948f-69e552438be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:48.915893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3764f55-8576-48fd-948f-69e552438be2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:48.922219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316e229-0ad6-455b-97f8-926f9aa8f51b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:48.923583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316e229-0ad6-455b-97f8-926f9aa8f51b service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:48.924108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316e229-0ad6-455b-97f8-926f9aa8f51b service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:48.925478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316e229-0ad6-455b-97f8-926f9aa8f51b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:48.939423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316e229-0ad6-455b-97f8-926f9aa8f51b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:48.941934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316e229-0ad6-455b-97f8-926f9aa8f51b service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:48.955889 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3640/7365] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:48 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:49.393258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174ab0b-6778-465d-ad98-442f0e6f1676 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:49.452704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174ab0b-6778-465d-ad98-442f0e6f1676 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:49.462460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd8437-14f6-48b6-9496-363d993f2ce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:49.463099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd8437-14f6-48b6-9496-363d993f2ce4 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:49.463380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd8437-14f6-48b6-9496-363d993f2ce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:49.463644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd8437-14f6-48b6-9496-363d993f2ce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:49.628426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd8437-14f6-48b6-9496-363d993f2ce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:49.630111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd8437-14f6-48b6-9496-363d993f2ce4 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:49.663634 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3726/7366] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:49 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:52.022827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393b122-9dcb-4283-a55e-ada2c230091f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:52.023366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393b122-9dcb-4283-a55e-ada2c230091f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:52.023605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393b122-9dcb-4283-a55e-ada2c230091f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:52.083750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e393b122-9dcb-4283-a55e-ada2c230091f None None] MFA Rules not processed for user `23301df1799c4676af4675f58c32a49b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:52.091623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e393b122-9dcb-4283-a55e-ada2c230091f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:52.113154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd8437-14f6-48b6-9496-363d993f2ce4 service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:52.202916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:52.209741 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3641/7367] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:52 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:52.226095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfddc3-d9b6-494b-86b0-6bcf3975e19c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:52.226597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfddc3-d9b6-494b-86b0-6bcf3975e19c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:52.226970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfddc3-d9b6-494b-86b0-6bcf3975e19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:52.251497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd8437-14f6-48b6-9496-363d993f2ce4 service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:52.258951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-70bfddc3-d9b6-494b-86b0-6bcf3975e19c None None] MFA Rules not processed for user `23301df1799c4676af4675f58c32a49b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:52.260887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-70bfddc3-d9b6-494b-86b0-6bcf3975e19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:52.273832 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3642/7368] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:52 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:52.277955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13233807-166e-4e1c-a67f-10b0d142e03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:52.278724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13233807-166e-4e1c-a67f-10b0d142e03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:52.279193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13233807-166e-4e1c-a67f-10b0d142e03d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:52.279579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13233807-166e-4e1c-a67f-10b0d142e03d admin admin] PATH_INFO: `/v3/users/f6a97852b65a48e28d96305d172edf40` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:52.280917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13233807-166e-4e1c-a67f-10b0d142e03d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6a97852b65a48e28d96305d172edf40)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:52.282139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfddc3-d9b6-494b-86b0-6bcf3975e19c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:52.284060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-70bfddc3-d9b6-494b-86b0-6bcf3975e19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:52.288846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13233807-166e-4e1c-a67f-10b0d142e03d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:52.361803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:52.377209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13233807-166e-4e1c-a67f-10b0d142e03d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6a97852b65a48e28d96305d172edf40'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:52.392115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13233807-166e-4e1c-a67f-10b0d142e03d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6a97852b65a48e28d96305d172edf40'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:52.406092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13233807-166e-4e1c-a67f-10b0d142e03d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6a97852b65a48e28d96305d172edf40'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:52.420706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13233807-166e-4e1c-a67f-10b0d142e03d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6a97852b65a48e28d96305d172edf40'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:52.422090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13233807-166e-4e1c-a67f-10b0d142e03d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6a97852b65a48e28d96305d172edf40'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:52.424522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfddc3-d9b6-494b-86b0-6bcf3975e19c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:52.430617 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3727/7369] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:52 2022] DELETE /identity/v3/users/f6a97852b65a48e28d96305d172edf40 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:24:52.431155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c910c-382c-4554-935e-3bb812c9f8a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:52.431841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c910c-382c-4554-935e-3bb812c9f8a4 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:52.432118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c910c-382c-4554-935e-3bb812c9f8a4 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:52.432806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c910c-382c-4554-935e-3bb812c9f8a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:52.434650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1c6c910c-382c-4554-935e-3bb812c9f8a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:52.435233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2dc5b-cdeb-455d-939b-c9fdec1ddefa None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:52.436532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2dc5b-cdeb-455d-939b-c9fdec1ddefa None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:52.437025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2dc5b-cdeb-455d-939b-c9fdec1ddefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:52.448246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c910c-382c-4554-935e-3bb812c9f8a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:52.449925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c910c-382c-4554-935e-3bb812c9f8a4 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:52.459677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-45d2dc5b-cdeb-455d-939b-c9fdec1ddefa None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:52.461037 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3643/7370] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:52 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:52.461824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-45d2dc5b-cdeb-455d-939b-c9fdec1ddefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:52.498406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c910c-382c-4554-935e-3bb812c9f8a4 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:52.500178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1c6c910c-382c-4554-935e-3bb812c9f8a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:52.549530 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3728/7371] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:52.560914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2dc5b-cdeb-455d-939b-c9fdec1ddefa None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:52.562753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-45d2dc5b-cdeb-455d-939b-c9fdec1ddefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:52.591758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c910c-382c-4554-935e-3bb812c9f8a4 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:52.637638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2dc5b-cdeb-455d-939b-c9fdec1ddefa None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:52.642487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0223974-6f11-4a12-b096-277317440023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:52.643210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0223974-6f11-4a12-b096-277317440023 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:52.644286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0223974-6f11-4a12-b096-277317440023 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:52.644636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0223974-6f11-4a12-b096-277317440023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:52.656220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0223974-6f11-4a12-b096-277317440023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:52.658027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0223974-6f11-4a12-b096-277317440023 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:52.667344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0c29b-e7ca-45e3-b0fb-5fccebf30f04 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:52.667968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0c29b-e7ca-45e3-b0fb-5fccebf30f04 service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:52.668240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0c29b-e7ca-45e3-b0fb-5fccebf30f04 service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:52.668556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0c29b-e7ca-45e3-b0fb-5fccebf30f04 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:52.671225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5bc0c29b-e7ca-45e3-b0fb-5fccebf30f04 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:52.676519 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3729/7372] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:24:52.753250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:52.801522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0c29b-e7ca-45e3-b0fb-5fccebf30f04 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:52.803176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0c29b-e7ca-45e3-b0fb-5fccebf30f04 service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:52.823308 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3644/7373] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:24:53.177243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0223974-6f11-4a12-b096-277317440023 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:53.182994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d0223974-6f11-4a12-b096-277317440023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:53.226949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0223974-6f11-4a12-b096-277317440023 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:53.265103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:53.270839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:53.271385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:53.272429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] PATH_INFO: `/v3/projects/b7de2505c08c422e99c2e2bda2f50e2b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:53.274974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7de2505c08c422e99c2e2bda2f50e2b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:53.277371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:53.362603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7de2505c08c422e99c2e2bda2f50e2b', 'request_id': 'req-ed04ceba-5a72-46fb-9d35-34f295e33f4e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:53.368430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:53.377499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7de2505c08c422e99c2e2bda2f50e2b', 'request_id': 'req-ed04ceba-5a72-46fb-9d35-34f295e33f4e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:53.386615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7de2505c08c422e99c2e2bda2f50e2b', 'request_id': 'req-ed04ceba-5a72-46fb-9d35-34f295e33f4e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:53.412118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] The token cache is being invalidate because project b7de2505c08c422e99c2e2bda2f50e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:53.416042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:53.417798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:53.421242 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3730/7374] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:53 2022] DELETE /identity/v3/projects/b7de2505c08c422e99c2e2bda2f50e2b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:53.425654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0c29b-e7ca-45e3-b0fb-5fccebf30f04 service glance] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:53.445841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0c29b-e7ca-45e3-b0fb-5fccebf30f04 service glance] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:53.619745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04831b3-fee2-40e6-b2c2-2fe5af7fcce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:53.620437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04831b3-fee2-40e6-b2c2-2fe5af7fcce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:53.620712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04831b3-fee2-40e6-b2c2-2fe5af7fcce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:53.620972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04831b3-fee2-40e6-b2c2-2fe5af7fcce0 admin admin] PATH_INFO: `/v3/users/0d02f97dc0ac4f61b7e7f2d35bfac9c8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:53.622116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04831b3-fee2-40e6-b2c2-2fe5af7fcce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d02f97dc0ac4f61b7e7f2d35bfac9c8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:53.624299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04831b3-fee2-40e6-b2c2-2fe5af7fcce0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:53.737644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f04831b3-fee2-40e6-b2c2-2fe5af7fcce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d02f97dc0ac4f61b7e7f2d35bfac9c8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:53.753797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f04831b3-fee2-40e6-b2c2-2fe5af7fcce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d02f97dc0ac4f61b7e7f2d35bfac9c8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:53.791059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f04831b3-fee2-40e6-b2c2-2fe5af7fcce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d02f97dc0ac4f61b7e7f2d35bfac9c8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:53.792363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f04831b3-fee2-40e6-b2c2-2fe5af7fcce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d02f97dc0ac4f61b7e7f2d35bfac9c8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:53.793305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f04831b3-fee2-40e6-b2c2-2fe5af7fcce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d02f97dc0ac4f61b7e7f2d35bfac9c8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:53.825462 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3645/7375] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:53 2022] DELETE /identity/v3/users/0d02f97dc0ac4f61b7e7f2d35bfac9c8 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:54.558052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:54.560554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:54.590297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04ceba-5a72-46fb-9d35-34f295e33f4e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:54.669422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c3136-7a84-47bc-946d-d88a0589d591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:54.672507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c3136-7a84-47bc-946d-d88a0589d591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:54.672979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c3136-7a84-47bc-946d-d88a0589d591 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:54.673260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c3136-7a84-47bc-946d-d88a0589d591 admin admin] PATH_INFO: `/v3/projects/10d02626f99c45cfbaf7b871f7c281fd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:54.676176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c3136-7a84-47bc-946d-d88a0589d591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d02626f99c45cfbaf7b871f7c281fd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:54.678182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c3136-7a84-47bc-946d-d88a0589d591 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:54.783032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-132c3136-7a84-47bc-946d-d88a0589d591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d02626f99c45cfbaf7b871f7c281fd', 'request_id': 'req-132c3136-7a84-47bc-946d-d88a0589d591'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:54.799356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-132c3136-7a84-47bc-946d-d88a0589d591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d02626f99c45cfbaf7b871f7c281fd', 'request_id': 'req-132c3136-7a84-47bc-946d-d88a0589d591'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:54.808470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-132c3136-7a84-47bc-946d-d88a0589d591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d02626f99c45cfbaf7b871f7c281fd', 'request_id': 'req-132c3136-7a84-47bc-946d-d88a0589d591'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:54.823496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-132c3136-7a84-47bc-946d-d88a0589d591 admin admin] The token cache is being invalidate because project 10d02626f99c45cfbaf7b871f7c281fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:54.824132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-132c3136-7a84-47bc-946d-d88a0589d591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:54.825152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-132c3136-7a84-47bc-946d-d88a0589d591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:54.828179 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3731/7376] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:54 2022] DELETE /identity/v3/projects/10d02626f99c45cfbaf7b871f7c281fd => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:54.843775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4f804-7711-4fc8-a323-2e06adfb4b8a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:54.844620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4f804-7711-4fc8-a323-2e06adfb4b8a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:54.844620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4f804-7711-4fc8-a323-2e06adfb4b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:54.894671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-95d4f804-7711-4fc8-a323-2e06adfb4b8a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:54.899698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-95d4f804-7711-4fc8-a323-2e06adfb4b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:54.988126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:55.020490 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3646/7377] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:55.029948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b32c4a-2385-4601-97e1-82763616ab26 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:55.030556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b32c4a-2385-4601-97e1-82763616ab26 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:55.031069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b32c4a-2385-4601-97e1-82763616ab26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:55.062958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-38b32c4a-2385-4601-97e1-82763616ab26 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:55.064494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-38b32c4a-2385-4601-97e1-82763616ab26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:55.074389 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3732/7378] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:55.079506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4f804-7711-4fc8-a323-2e06adfb4b8a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:55.092089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4f804-7711-4fc8-a323-2e06adfb4b8a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:55.105594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549efe4-8a81-4c0f-8910-f4d2e031f9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:55.106227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549efe4-8a81-4c0f-8910-f4d2e031f9f7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:55.106501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549efe4-8a81-4c0f-8910-f4d2e031f9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:55.106766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549efe4-8a81-4c0f-8910-f4d2e031f9f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:55.107418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549efe4-8a81-4c0f-8910-f4d2e031f9f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:24:55.107720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549efe4-8a81-4c0f-8910-f4d2e031f9f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:55.109396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549efe4-8a81-4c0f-8910-f4d2e031f9f7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:55.132950 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3647/7379] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:24:55.133957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b32c4a-2385-4601-97e1-82763616ab26 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:55.149787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b32c4a-2385-4601-97e1-82763616ab26 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:55.168674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de978ae-a6dc-443c-aec7-9ed33bf5fa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:55.169455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de978ae-a6dc-443c-aec7-9ed33bf5fa28 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:55.169831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de978ae-a6dc-443c-aec7-9ed33bf5fa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:55.170201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de978ae-a6dc-443c-aec7-9ed33bf5fa28 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:55.170943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de978ae-a6dc-443c-aec7-9ed33bf5fa28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:55.173176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de978ae-a6dc-443c-aec7-9ed33bf5fa28 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:55.234477 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3733/7380] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:55 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:24:55.239145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549efe4-8a81-4c0f-8910-f4d2e031f9f7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:55.260690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549efe4-8a81-4c0f-8910-f4d2e031f9f7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:55.363270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc069ca-d1ae-4f49-8ba5-0f550d64c6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:55.364051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc069ca-d1ae-4f49-8ba5-0f550d64c6d7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:55.364482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc069ca-d1ae-4f49-8ba5-0f550d64c6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:55.364919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc069ca-d1ae-4f49-8ba5-0f550d64c6d7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:55.365657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc069ca-d1ae-4f49-8ba5-0f550d64c6d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:55.367745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc069ca-d1ae-4f49-8ba5-0f550d64c6d7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:55.395222 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3648/7381] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:55 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:55.400531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de978ae-a6dc-443c-aec7-9ed33bf5fa28 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:55.419688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de978ae-a6dc-443c-aec7-9ed33bf5fa28 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:55.425528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cd221-2b91-426f-b3f8-291328f99589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:55.426153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cd221-2b91-426f-b3f8-291328f99589 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:55.426609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cd221-2b91-426f-b3f8-291328f99589 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:55.426733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cd221-2b91-426f-b3f8-291328f99589 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:55.427317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cd221-2b91-426f-b3f8-291328f99589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:55.428974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cd221-2b91-426f-b3f8-291328f99589 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:55.456467 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3734/7382] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:55.456608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc069ca-d1ae-4f49-8ba5-0f550d64c6d7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:55.470943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc069ca-d1ae-4f49-8ba5-0f550d64c6d7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:55.477563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c55ec-fab4-4bb3-ad57-b7b25584c9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:55.478342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c55ec-fab4-4bb3-ad57-b7b25584c9ec admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:55.478713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c55ec-fab4-4bb3-ad57-b7b25584c9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:55.479079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c55ec-fab4-4bb3-ad57-b7b25584c9ec admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:55.480203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c55ec-fab4-4bb3-ad57-b7b25584c9ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:55.482290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c55ec-fab4-4bb3-ad57-b7b25584c9ec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:55.498367 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3649/7383] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:55.503581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cd221-2b91-426f-b3f8-291328f99589 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:55.519148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cd221-2b91-426f-b3f8-291328f99589 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:55.530595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7c5b5-3ff6-452e-96fc-aac1e6828c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:55.531956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7c5b5-3ff6-452e-96fc-aac1e6828c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:55.532867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7c5b5-3ff6-452e-96fc-aac1e6828c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:55.533908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7c5b5-3ff6-452e-96fc-aac1e6828c6a admin admin] PATH_INFO: `/v3/projects/5d198f8fab70437c9336b67e9ab1e098/users/cb8eb39e4501407abf0f30adebbeb8f4/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:55.585322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7c5b5-3ff6-452e-96fc-aac1e6828c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d198f8fab70437c9336b67e9ab1e098, user_id=cb8eb39e4501407abf0f30adebbeb8f4, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:55.590436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7c5b5-3ff6-452e-96fc-aac1e6828c6a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:55.623775 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3735/7384] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:55 2022] PUT /identity/v3/projects/5d198f8fab70437c9336b67e9ab1e098/users/cb8eb39e4501407abf0f30adebbeb8f4/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:55.624358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe1bb4-ebc8-42f6-9c5a-9efdc8681025 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:55.624673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe1bb4-ebc8-42f6-9c5a-9efdc8681025 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:55.624941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe1bb4-ebc8-42f6-9c5a-9efdc8681025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:55.742274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ccbe1bb4-ebc8-42f6-9c5a-9efdc8681025 None None] MFA Rules not processed for user `cb8eb39e4501407abf0f30adebbeb8f4`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:55.745974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ccbe1bb4-ebc8-42f6-9c5a-9efdc8681025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:55.913048 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3650/7385] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:55.918527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7c5b5-3ff6-452e-96fc-aac1e6828c6a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:55.930779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:56.061073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7c5b5-3ff6-452e-96fc-aac1e6828c6a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:56.077821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568c0a1-65ab-44cb-9046-00ae4caabe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:56.078490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568c0a1-65ab-44cb-9046-00ae4caabe04 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:56.078795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568c0a1-65ab-44cb-9046-00ae4caabe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:56.080932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568c0a1-65ab-44cb-9046-00ae4caabe04 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:56.081805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568c0a1-65ab-44cb-9046-00ae4caabe04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:56.084639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568c0a1-65ab-44cb-9046-00ae4caabe04 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:56.126503 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3736/7386] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:24:55 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:24:56.130477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe1bb4-ebc8-42f6-9c5a-9efdc8681025 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:56.150742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe1bb4-ebc8-42f6-9c5a-9efdc8681025 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:56.217241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:56.242317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d9ede-d3d6-4282-bb1c-6dd9e195c767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:56.242944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d9ede-d3d6-4282-bb1c-6dd9e195c767 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:56.243216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d9ede-d3d6-4282-bb1c-6dd9e195c767 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:56.243484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d9ede-d3d6-4282-bb1c-6dd9e195c767 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:56.244247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d9ede-d3d6-4282-bb1c-6dd9e195c767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:56.246234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d9ede-d3d6-4282-bb1c-6dd9e195c767 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:56.311354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568c0a1-65ab-44cb-9046-00ae4caabe04 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:56.314731 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3651/7387] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:24:56 2022] POST /identity/v3/users => generated 300 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:24:56.328572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568c0a1-65ab-44cb-9046-00ae4caabe04 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:56.336681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5d85d-0bc6-4949-a9f4-82ab29a96901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:56.337688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5d85d-0bc6-4949-a9f4-82ab29a96901 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:56.338026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5d85d-0bc6-4949-a9f4-82ab29a96901 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:56.338397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5d85d-0bc6-4949-a9f4-82ab29a96901 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:56.339796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5d85d-0bc6-4949-a9f4-82ab29a96901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:56.342611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5d85d-0bc6-4949-a9f4-82ab29a96901 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:56.386351 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3737/7388] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:24:56.388525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d9ede-d3d6-4282-bb1c-6dd9e195c767 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:56.405714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d9ede-d3d6-4282-bb1c-6dd9e195c767 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:56.412329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6befa8e-dde5-4884-93bc-8f3ee0c9caba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:56.413200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6befa8e-dde5-4884-93bc-8f3ee0c9caba admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:56.413730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6befa8e-dde5-4884-93bc-8f3ee0c9caba admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:56.414095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6befa8e-dde5-4884-93bc-8f3ee0c9caba admin admin] PATH_INFO: `/v3/projects/c9fde8001da24853bfe369a979e40230/users/7c89f994f7ee42ffa7d42d1dbd2ec8cf/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:56.446551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6befa8e-dde5-4884-93bc-8f3ee0c9caba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9fde8001da24853bfe369a979e40230, user_id=7c89f994f7ee42ffa7d42d1dbd2ec8cf, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:56.452621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6befa8e-dde5-4884-93bc-8f3ee0c9caba admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:56.489226 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3652/7389] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:56 2022] PUT /identity/v3/projects/c9fde8001da24853bfe369a979e40230/users/7c89f994f7ee42ffa7d42d1dbd2ec8cf/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:56.489949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5d85d-0bc6-4949-a9f4-82ab29a96901 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:56.501008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5d85d-0bc6-4949-a9f4-82ab29a96901 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:56.513509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6befa8e-dde5-4884-93bc-8f3ee0c9caba admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:56.525119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a6befa8e-dde5-4884-93bc-8f3ee0c9caba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:56.677822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc7dd8-a906-42a4-a786-6f68a304dbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:56.678421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc7dd8-a906-42a4-a786-6f68a304dbfe admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:56.678696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc7dd8-a906-42a4-a786-6f68a304dbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:56.678957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc7dd8-a906-42a4-a786-6f68a304dbfe admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:56.679591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc7dd8-a906-42a4-a786-6f68a304dbfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:56.690223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc7dd8-a906-42a4-a786-6f68a304dbfe admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:56.726131 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3738/7390] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:24:56.730095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc7dd8-a906-42a4-a786-6f68a304dbfe admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:56.746334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc7dd8-a906-42a4-a786-6f68a304dbfe admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:56.750671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6befa8e-dde5-4884-93bc-8f3ee0c9caba admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:56.755762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bda441-6e34-4637-89ac-fb12bcd758e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:56.756401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bda441-6e34-4637-89ac-fb12bcd758e1 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:56.766035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bda441-6e34-4637-89ac-fb12bcd758e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:56.767281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bda441-6e34-4637-89ac-fb12bcd758e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:56.767857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b917d1-d573-4db9-b60b-1d70ef6230c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:56.768604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b917d1-d573-4db9-b60b-1d70ef6230c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:56.768879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b917d1-d573-4db9-b60b-1d70ef6230c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:56.769147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b917d1-d573-4db9-b60b-1d70ef6230c9 admin admin] PATH_INFO: `/v3/projects/c9fde8001da24853bfe369a979e40230/users/7c89f994f7ee42ffa7d42d1dbd2ec8cf/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:56.770156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e2bda441-6e34-4637-89ac-fb12bcd758e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:56.772192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b917d1-d573-4db9-b60b-1d70ef6230c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9fde8001da24853bfe369a979e40230, user_id=7c89f994f7ee42ffa7d42d1dbd2ec8cf, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:56.775638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b917d1-d573-4db9-b60b-1d70ef6230c9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:56.796434 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3739/7391] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:24:56 2022] PUT /identity/v3/projects/c9fde8001da24853bfe369a979e40230/users/7c89f994f7ee42ffa7d42d1dbd2ec8cf/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:24:56.806378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375bb15-2fda-4b1c-9ab9-a19f9a1fc16a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:56.806911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375bb15-2fda-4b1c-9ab9-a19f9a1fc16a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:56.807314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375bb15-2fda-4b1c-9ab9-a19f9a1fc16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:56.898608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f375bb15-2fda-4b1c-9ab9-a19f9a1fc16a None None] MFA Rules not processed for user `7c89f994f7ee42ffa7d42d1dbd2ec8cf`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:56.899869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f375bb15-2fda-4b1c-9ab9-a19f9a1fc16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:57.025963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bda441-6e34-4637-89ac-fb12bcd758e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:57.032217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bda441-6e34-4637-89ac-fb12bcd758e1 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:57.069221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:57.079781 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3653/7392] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:57.124525 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3740/7393] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:24:57.155917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bda441-6e34-4637-89ac-fb12bcd758e1 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:57.157491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e2bda441-6e34-4637-89ac-fb12bcd758e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:57.279184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:57.288941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bda441-6e34-4637-89ac-fb12bcd758e1 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:57.370250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d1b55-9c01-4288-be26-892be9ad93d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:57.370823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d1b55-9c01-4288-be26-892be9ad93d6 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:57.371103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d1b55-9c01-4288-be26-892be9ad93d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:57.371799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d1b55-9c01-4288-be26-892be9ad93d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:57.386599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d1b55-9c01-4288-be26-892be9ad93d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:57.388500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d1b55-9c01-4288-be26-892be9ad93d6 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:57.426582 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3654/7394] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:58.452509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375bb15-2fda-4b1c-9ab9-a19f9a1fc16a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:58.559276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375bb15-2fda-4b1c-9ab9-a19f9a1fc16a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:58.570011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:58.570708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:58.572083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:58.572083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] PATH_INFO: `/v3/users/cb8eb39e4501407abf0f30adebbeb8f4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:58.572935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb8eb39e4501407abf0f30adebbeb8f4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:58.575937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:58.665582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb8eb39e4501407abf0f30adebbeb8f4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:58.686851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb8eb39e4501407abf0f30adebbeb8f4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:58.699473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb8eb39e4501407abf0f30adebbeb8f4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:58.714673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb8eb39e4501407abf0f30adebbeb8f4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:58.715415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb8eb39e4501407abf0f30adebbeb8f4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:58.723253 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3741/7395] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:58 2022] DELETE /identity/v3/users/cb8eb39e4501407abf0f30adebbeb8f4 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:58.727642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3546db-315a-48a7-bafa-0bb4af224594 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:58.728145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3546db-315a-48a7-bafa-0bb4af224594 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:58.728259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3546db-315a-48a7-bafa-0bb4af224594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:58.753601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1a3546db-315a-48a7-bafa-0bb4af224594 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:58.761142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1a3546db-315a-48a7-bafa-0bb4af224594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:58.848211 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3655/7396] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:58.866498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:58.868015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:58.961276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6ce4e-1ec1-4e4a-bd3e-945c4547a07d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:58.966100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a462e4d-f827-4946-8778-9aa7a2043fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:58.966296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a462e4d-f827-4946-8778-9aa7a2043fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:58.966507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a462e4d-f827-4946-8778-9aa7a2043fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:58.966764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a462e4d-f827-4946-8778-9aa7a2043fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:58.978141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a462e4d-f827-4946-8778-9aa7a2043fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:58.979613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a462e4d-f827-4946-8778-9aa7a2043fe4 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:58.990444 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3742/7397] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:24:59.750023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a462e4d-f827-4946-8778-9aa7a2043fe4 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:24:59.750898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded28451-4bb2-492e-b0d2-33cfe75df73f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:59.751405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded28451-4bb2-492e-b0d2-33cfe75df73f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:59.751519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded28451-4bb2-492e-b0d2-33cfe75df73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:59.753941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5a462e4d-f827-4946-8778-9aa7a2043fe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:59.786677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a462e4d-f827-4946-8778-9aa7a2043fe4 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:24:59.809550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ded28451-4bb2-492e-b0d2-33cfe75df73f None None] MFA Rules not processed for user `4e6a5f35d172410481a8416b7bdaafac`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:24:59.811459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ded28451-4bb2-492e-b0d2-33cfe75df73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:24:59.814835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:24:59.815513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:24:59.815807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:24:59.816186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] PATH_INFO: `/v3/projects/5d198f8fab70437c9336b67e9ab1e098` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:24:59.818631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d198f8fab70437c9336b67e9ab1e098)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:24:59.824793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:24:59.843763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:24:59.918294 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3656/7398] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:24:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:24:59.932686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d198f8fab70437c9336b67e9ab1e098', 'request_id': 'req-8b898ab1-fbed-4628-8a06-5726dbf36aa8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:59.946089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d198f8fab70437c9336b67e9ab1e098', 'request_id': 'req-8b898ab1-fbed-4628-8a06-5726dbf36aa8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:59.958559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d198f8fab70437c9336b67e9ab1e098', 'request_id': 'req-8b898ab1-fbed-4628-8a06-5726dbf36aa8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:59.979300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] The token cache is being invalidate because project 5d198f8fab70437c9336b67e9ab1e098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:24:59.979693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:59.980624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:24:59.982613 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3743/7399] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:24:59 2022] DELETE /identity/v3/projects/5d198f8fab70437c9336b67e9ab1e098 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:24:59.987303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded28451-4bb2-492e-b0d2-33cfe75df73f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:00.002326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded28451-4bb2-492e-b0d2-33cfe75df73f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:00.071661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:00.082060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e1b30-e385-4b8c-9ab0-dc2d6ab9e71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:00.083102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e1b30-e385-4b8c-9ab0-dc2d6ab9e71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:00.083610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e1b30-e385-4b8c-9ab0-dc2d6ab9e71e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:00.083945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e1b30-e385-4b8c-9ab0-dc2d6ab9e71e admin admin] PATH_INFO: `/v3/users/7c89f994f7ee42ffa7d42d1dbd2ec8cf` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:00.085365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e1b30-e385-4b8c-9ab0-dc2d6ab9e71e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c89f994f7ee42ffa7d42d1dbd2ec8cf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:00.087771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e1b30-e385-4b8c-9ab0-dc2d6ab9e71e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:00.153366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f52e1b30-e385-4b8c-9ab0-dc2d6ab9e71e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c89f994f7ee42ffa7d42d1dbd2ec8cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:00.164837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f52e1b30-e385-4b8c-9ab0-dc2d6ab9e71e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c89f994f7ee42ffa7d42d1dbd2ec8cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:00.180946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f52e1b30-e385-4b8c-9ab0-dc2d6ab9e71e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c89f994f7ee42ffa7d42d1dbd2ec8cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:00.181291 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f52e1b30-e385-4b8c-9ab0-dc2d6ab9e71e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c89f994f7ee42ffa7d42d1dbd2ec8cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:00.182119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f52e1b30-e385-4b8c-9ab0-dc2d6ab9e71e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c89f994f7ee42ffa7d42d1dbd2ec8cf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:00.216503 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3657/7400] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:24:59 2022] DELETE /identity/v3/users/7c89f994f7ee42ffa7d42d1dbd2ec8cf => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:00.930350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:00.933243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:00.956033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b898ab1-fbed-4628-8a06-5726dbf36aa8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:01.145375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217e1d3-05e1-43aa-9401-b73b0250e08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:01.146167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217e1d3-05e1-43aa-9401-b73b0250e08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:01.146615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217e1d3-05e1-43aa-9401-b73b0250e08d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:01.146957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217e1d3-05e1-43aa-9401-b73b0250e08d admin admin] PATH_INFO: `/v3/projects/c9fde8001da24853bfe369a979e40230` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:01.149177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217e1d3-05e1-43aa-9401-b73b0250e08d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9fde8001da24853bfe369a979e40230)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:01.152718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217e1d3-05e1-43aa-9401-b73b0250e08d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:01.229984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c217e1d3-05e1-43aa-9401-b73b0250e08d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9fde8001da24853bfe369a979e40230', 'request_id': 'req-c217e1d3-05e1-43aa-9401-b73b0250e08d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:01.244802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c217e1d3-05e1-43aa-9401-b73b0250e08d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9fde8001da24853bfe369a979e40230', 'request_id': 'req-c217e1d3-05e1-43aa-9401-b73b0250e08d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:01.252721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c217e1d3-05e1-43aa-9401-b73b0250e08d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9fde8001da24853bfe369a979e40230', 'request_id': 'req-c217e1d3-05e1-43aa-9401-b73b0250e08d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:01.278242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c217e1d3-05e1-43aa-9401-b73b0250e08d admin admin] The token cache is being invalidate because project c9fde8001da24853bfe369a979e40230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:01.278534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c217e1d3-05e1-43aa-9401-b73b0250e08d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:01.279362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c217e1d3-05e1-43aa-9401-b73b0250e08d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:01.281336 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3744/7401] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:00 2022] DELETE /identity/v3/projects/c9fde8001da24853bfe369a979e40230 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:01.286883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f2f5c-aebd-4e19-904b-f76d8d86d1e4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:01.287449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f2f5c-aebd-4e19-904b-f76d8d86d1e4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:01.287558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f2f5c-aebd-4e19-904b-f76d8d86d1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:01.321398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-998f2f5c-aebd-4e19-904b-f76d8d86d1e4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:01.323233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-998f2f5c-aebd-4e19-904b-f76d8d86d1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:01.412240 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3658/7402] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:01.420432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ef47d-8d71-4e2c-8f72-537f24c5b709 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:01.420811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ef47d-8d71-4e2c-8f72-537f24c5b709 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:01.421091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ef47d-8d71-4e2c-8f72-537f24c5b709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:01.449152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fe5ef47d-8d71-4e2c-8f72-537f24c5b709 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:01.450470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fe5ef47d-8d71-4e2c-8f72-537f24c5b709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:01.462999 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3745/7403] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:01.465053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f2f5c-aebd-4e19-904b-f76d8d86d1e4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:01.480002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f2f5c-aebd-4e19-904b-f76d8d86d1e4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:01.485269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc816888-a512-41de-a1d1-fca5e8ed6054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:01.485893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc816888-a512-41de-a1d1-fca5e8ed6054 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:01.486168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc816888-a512-41de-a1d1-fca5e8ed6054 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:01.486437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc816888-a512-41de-a1d1-fca5e8ed6054 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:01.487121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc816888-a512-41de-a1d1-fca5e8ed6054 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:25:01.487422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc816888-a512-41de-a1d1-fca5e8ed6054 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:01.489105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc816888-a512-41de-a1d1-fca5e8ed6054 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:01.501798 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3659/7404] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:25:01.506312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ef47d-8d71-4e2c-8f72-537f24c5b709 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:01.517644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ef47d-8d71-4e2c-8f72-537f24c5b709 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:01.522490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00340f09-fefe-4890-ace5-335f750eaba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:01.523105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00340f09-fefe-4890-ace5-335f750eaba8 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:01.523670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00340f09-fefe-4890-ace5-335f750eaba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:01.523670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00340f09-fefe-4890-ace5-335f750eaba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:01.524255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00340f09-fefe-4890-ace5-335f750eaba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:01.526256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00340f09-fefe-4890-ace5-335f750eaba8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:01.546526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:01.566260 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3746/7405] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:01 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:01.568679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc816888-a512-41de-a1d1-fca5e8ed6054 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:01.589555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc816888-a512-41de-a1d1-fca5e8ed6054 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:01.682450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3170eb-0665-4329-9ccf-7f34f3f01821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:01.683371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3170eb-0665-4329-9ccf-7f34f3f01821 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:01.683849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3170eb-0665-4329-9ccf-7f34f3f01821 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:01.684222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3170eb-0665-4329-9ccf-7f34f3f01821 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:01.684934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3170eb-0665-4329-9ccf-7f34f3f01821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:01.686994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3170eb-0665-4329-9ccf-7f34f3f01821 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:01.697778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:01.717588 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3660/7406] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:01 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:01.722556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00340f09-fefe-4890-ace5-335f750eaba8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:01.734623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00340f09-fefe-4890-ace5-335f750eaba8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:01.739852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fbd19-414f-41b7-abaa-1b9f7a1f9345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:01.740443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fbd19-414f-41b7-abaa-1b9f7a1f9345 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:01.740716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fbd19-414f-41b7-abaa-1b9f7a1f9345 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:01.740980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fbd19-414f-41b7-abaa-1b9f7a1f9345 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:01.741613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fbd19-414f-41b7-abaa-1b9f7a1f9345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:01.743202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fbd19-414f-41b7-abaa-1b9f7a1f9345 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:01.754329 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3747/7407] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:01.758354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3170eb-0665-4329-9ccf-7f34f3f01821 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:01.768993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3170eb-0665-4329-9ccf-7f34f3f01821 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:01.773568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7e592-b1d0-40b6-a26c-1629d15a6dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:01.774215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7e592-b1d0-40b6-a26c-1629d15a6dca admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:01.774438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7e592-b1d0-40b6-a26c-1629d15a6dca admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:01.774745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7e592-b1d0-40b6-a26c-1629d15a6dca admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:01.775369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7e592-b1d0-40b6-a26c-1629d15a6dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:01.777006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7e592-b1d0-40b6-a26c-1629d15a6dca admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:01.788958 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3661/7408] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:01.793267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fbd19-414f-41b7-abaa-1b9f7a1f9345 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:01.803670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fbd19-414f-41b7-abaa-1b9f7a1f9345 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:01.808582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ba28c-2b24-4482-9424-62d02c490c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:01.809437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ba28c-2b24-4482-9424-62d02c490c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:01.809776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ba28c-2b24-4482-9424-62d02c490c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:01.810111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ba28c-2b24-4482-9424-62d02c490c92 admin admin] PATH_INFO: `/v3/projects/6454739c5f9040d8971d0dac17af821f/users/163e3b6f11a04710af64abe98b121ccf/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:01.827068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ba28c-2b24-4482-9424-62d02c490c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6454739c5f9040d8971d0dac17af821f, user_id=163e3b6f11a04710af64abe98b121ccf, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:01.830018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ba28c-2b24-4482-9424-62d02c490c92 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:01.846459 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3748/7409] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:01 2022] PUT /identity/v3/projects/6454739c5f9040d8971d0dac17af821f/users/163e3b6f11a04710af64abe98b121ccf/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:01.856902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853a57c-0fd2-4158-8184-ed66096a98d5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:01.857186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853a57c-0fd2-4158-8184-ed66096a98d5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:01.857452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853a57c-0fd2-4158-8184-ed66096a98d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:01.888674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d853a57c-0fd2-4158-8184-ed66096a98d5 None None] MFA Rules not processed for user `163e3b6f11a04710af64abe98b121ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:01.889907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d853a57c-0fd2-4158-8184-ed66096a98d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:01.980748 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3662/7410] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:01.992808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ba28c-2b24-4482-9424-62d02c490c92 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:01.994238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dc1ba28c-2b24-4482-9424-62d02c490c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:02.079290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ba28c-2b24-4482-9424-62d02c490c92 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:02.086298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779e196-17a9-4512-82eb-2bdf94f72f7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:02.086965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779e196-17a9-4512-82eb-2bdf94f72f7e service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:02.087218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779e196-17a9-4512-82eb-2bdf94f72f7e service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:02.087682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779e196-17a9-4512-82eb-2bdf94f72f7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:02.101109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779e196-17a9-4512-82eb-2bdf94f72f7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:02.102901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779e196-17a9-4512-82eb-2bdf94f72f7e service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:02.116365 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3749/7411] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:03.978760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853a57c-0fd2-4158-8184-ed66096a98d5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:04.089331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853a57c-0fd2-4158-8184-ed66096a98d5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:04.094186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:04.094838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:04.095105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:04.095371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] PATH_INFO: `/v3/users/163e3b6f11a04710af64abe98b121ccf` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:04.096554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] RBAC: Authorizing `identity:delete_user(user_id=163e3b6f11a04710af64abe98b121ccf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:04.098691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:04.166827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '163e3b6f11a04710af64abe98b121ccf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:04.178023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '163e3b6f11a04710af64abe98b121ccf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:04.194337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '163e3b6f11a04710af64abe98b121ccf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:04.195434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '163e3b6f11a04710af64abe98b121ccf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:04.197464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '163e3b6f11a04710af64abe98b121ccf'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:04.200920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:04.233633 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3663/7412] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:03 2022] DELETE /identity/v3/users/163e3b6f11a04710af64abe98b121ccf => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:04.238757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e943a-38e3-484c-a40b-0e5588698645 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:04.239208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e943a-38e3-484c-a40b-0e5588698645 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:04.239327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e943a-38e3-484c-a40b-0e5588698645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:04.279969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8c2e943a-38e3-484c-a40b-0e5588698645 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:04.285080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8c2e943a-38e3-484c-a40b-0e5588698645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:04.424070 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3750/7413] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:04.449244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:04.459352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:04.582806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678649e8-c02d-46d9-b671-8cb1e53f89df admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:04.588247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb33cf-a217-4db0-832c-c6b214733138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:04.588247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb33cf-a217-4db0-832c-c6b214733138 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:04.588746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb33cf-a217-4db0-832c-c6b214733138 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:04.589087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb33cf-a217-4db0-832c-c6b214733138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:04.599414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb33cf-a217-4db0-832c-c6b214733138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:04.602021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb33cf-a217-4db0-832c-c6b214733138 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:04.615077 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3664/7414] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:05.348083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e943a-38e3-484c-a40b-0e5588698645 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:05.351536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8c2e943a-38e3-484c-a40b-0e5588698645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:05.375965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e943a-38e3-484c-a40b-0e5588698645 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:05.387161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600e508-0eb1-4c2b-b6cf-8b85461bec23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:05.388680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600e508-0eb1-4c2b-b6cf-8b85461bec23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:05.389139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600e508-0eb1-4c2b-b6cf-8b85461bec23 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:05.389432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600e508-0eb1-4c2b-b6cf-8b85461bec23 admin admin] PATH_INFO: `/v3/projects/6454739c5f9040d8971d0dac17af821f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:05.391021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600e508-0eb1-4c2b-b6cf-8b85461bec23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6454739c5f9040d8971d0dac17af821f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:05.393090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600e508-0eb1-4c2b-b6cf-8b85461bec23 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:05.493843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8600e508-0eb1-4c2b-b6cf-8b85461bec23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6454739c5f9040d8971d0dac17af821f', 'request_id': 'req-8600e508-0eb1-4c2b-b6cf-8b85461bec23'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:05.503870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8600e508-0eb1-4c2b-b6cf-8b85461bec23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6454739c5f9040d8971d0dac17af821f', 'request_id': 'req-8600e508-0eb1-4c2b-b6cf-8b85461bec23'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:05.507238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:05.515392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8600e508-0eb1-4c2b-b6cf-8b85461bec23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6454739c5f9040d8971d0dac17af821f', 'request_id': 'req-8600e508-0eb1-4c2b-b6cf-8b85461bec23'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:05.530994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8600e508-0eb1-4c2b-b6cf-8b85461bec23 admin admin] The token cache is being invalidate because project 6454739c5f9040d8971d0dac17af821f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:05.531720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8600e508-0eb1-4c2b-b6cf-8b85461bec23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:05.532981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8600e508-0eb1-4c2b-b6cf-8b85461bec23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:05.535771 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3751/7415] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:05 2022] DELETE /identity/v3/projects/6454739c5f9040d8971d0dac17af821f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:05.553037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337324fb-83eb-4dd3-bb1d-979b06c75968 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:05.555098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337324fb-83eb-4dd3-bb1d-979b06c75968 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:05.556006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337324fb-83eb-4dd3-bb1d-979b06c75968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:05.590755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-337324fb-83eb-4dd3-bb1d-979b06c75968 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:05.592600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-337324fb-83eb-4dd3-bb1d-979b06c75968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:05.723961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b751b6e-bb39-4b3e-aa1f-912fb2137914 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:05.724343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b751b6e-bb39-4b3e-aa1f-912fb2137914 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:05.724736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b751b6e-bb39-4b3e-aa1f-912fb2137914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:05.731608 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3665/7416] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:05.751162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9b751b6e-bb39-4b3e-aa1f-912fb2137914 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:05.752579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9b751b6e-bb39-4b3e-aa1f-912fb2137914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:05.761221 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3752/7417] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:05.765416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337324fb-83eb-4dd3-bb1d-979b06c75968 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:05.779674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337324fb-83eb-4dd3-bb1d-979b06c75968 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:05.784864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cbcd6-0394-428a-b927-546cbb2e72f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:05.785465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cbcd6-0394-428a-b927-546cbb2e72f6 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:05.785754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cbcd6-0394-428a-b927-546cbb2e72f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:05.786131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cbcd6-0394-428a-b927-546cbb2e72f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:05.786774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2cbcd6-0394-428a-b927-546cbb2e72f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:25:05.787108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2cbcd6-0394-428a-b927-546cbb2e72f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:05.788747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2cbcd6-0394-428a-b927-546cbb2e72f6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:05.802721 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3666/7418] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:25:05.807069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b751b6e-bb39-4b3e-aa1f-912fb2137914 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:05.817869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b751b6e-bb39-4b3e-aa1f-912fb2137914 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:05.823267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0564140-029e-44a3-9033-45984b720626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:05.823811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0564140-029e-44a3-9033-45984b720626 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:05.824079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0564140-029e-44a3-9033-45984b720626 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:05.824338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0564140-029e-44a3-9033-45984b720626 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:05.825151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0564140-029e-44a3-9033-45984b720626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:05.827097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0564140-029e-44a3-9033-45984b720626 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:05.863626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cbcd6-0394-428a-b927-546cbb2e72f6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:05.871549 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3753/7419] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:05 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:05.876808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cbcd6-0394-428a-b927-546cbb2e72f6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:05.954405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982d687-d192-4bca-b293-9fbf71837b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:05.956092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982d687-d192-4bca-b293-9fbf71837b26 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:05.956511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982d687-d192-4bca-b293-9fbf71837b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:05.956969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982d687-d192-4bca-b293-9fbf71837b26 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:05.957905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982d687-d192-4bca-b293-9fbf71837b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:05.960183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982d687-d192-4bca-b293-9fbf71837b26 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:05.992232 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3667/7420] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:05 2022] POST /identity/v3/users => generated 291 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:05.996234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0564140-029e-44a3-9033-45984b720626 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:06.008440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0564140-029e-44a3-9033-45984b720626 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:06.014158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42a6eb-a38b-430d-9153-0923a1c1219a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:06.014799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42a6eb-a38b-430d-9153-0923a1c1219a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:06.015172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42a6eb-a38b-430d-9153-0923a1c1219a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:06.015540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42a6eb-a38b-430d-9153-0923a1c1219a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:06.016423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42a6eb-a38b-430d-9153-0923a1c1219a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:06.018170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42a6eb-a38b-430d-9153-0923a1c1219a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:06.033424 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3754/7421] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:25:06.035090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982d687-d192-4bca-b293-9fbf71837b26 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:06.046968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982d687-d192-4bca-b293-9fbf71837b26 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:06.053571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18985def-35b6-47d9-b736-73749559b425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:06.053571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18985def-35b6-47d9-b736-73749559b425 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:06.054534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18985def-35b6-47d9-b736-73749559b425 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:06.055892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18985def-35b6-47d9-b736-73749559b425 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:06.057395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18985def-35b6-47d9-b736-73749559b425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:06.059103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18985def-35b6-47d9-b736-73749559b425 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:06.060750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:06.077219 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3668/7422] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:06.081245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42a6eb-a38b-430d-9153-0923a1c1219a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:06.094661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42a6eb-a38b-430d-9153-0923a1c1219a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:06.103716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db3640-fa83-4667-bd6a-5c247ea6441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:06.104927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db3640-fa83-4667-bd6a-5c247ea6441a admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:06.105514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db3640-fa83-4667-bd6a-5c247ea6441a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:06.105833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db3640-fa83-4667-bd6a-5c247ea6441a admin admin] PATH_INFO: `/v3/projects/87a82f2c07b24e2d88831e593d4a09e2/users/86230b04dcab49a0a40c49df6c403c0f/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:06.131144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db3640-fa83-4667-bd6a-5c247ea6441a admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a82f2c07b24e2d88831e593d4a09e2, user_id=86230b04dcab49a0a40c49df6c403c0f, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:06.135764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db3640-fa83-4667-bd6a-5c247ea6441a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:06.156410 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3755/7423] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:06 2022] PUT /identity/v3/projects/87a82f2c07b24e2d88831e593d4a09e2/users/86230b04dcab49a0a40c49df6c403c0f/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:06.166994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4877-3dc0-4455-9cc1-d443aeb27ee6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:06.167267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4877-3dc0-4455-9cc1-d443aeb27ee6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:06.167530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4877-3dc0-4455-9cc1-d443aeb27ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:06.203005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6c9a4877-3dc0-4455-9cc1-d443aeb27ee6 None None] MFA Rules not processed for user `86230b04dcab49a0a40c49df6c403c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:06.204359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6c9a4877-3dc0-4455-9cc1-d443aeb27ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:06.278643 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3669/7424] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:06.283445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db3640-fa83-4667-bd6a-5c247ea6441a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:06.364027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db3640-fa83-4667-bd6a-5c247ea6441a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:06.369147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d35c0-54b7-4037-a3d8-17a9109b887d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:06.369976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d35c0-54b7-4037-a3d8-17a9109b887d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:06.370079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d35c0-54b7-4037-a3d8-17a9109b887d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:06.370337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d35c0-54b7-4037-a3d8-17a9109b887d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:06.370967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d35c0-54b7-4037-a3d8-17a9109b887d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:06.373656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d35c0-54b7-4037-a3d8-17a9109b887d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:06.408410 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3756/7425] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:06 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:06.412521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a4877-3dc0-4455-9cc1-d443aeb27ee6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:06.424554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a4877-3dc0-4455-9cc1-d443aeb27ee6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:06.501535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d116a-831a-4430-9c6d-65af40f62573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:06.502148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d116a-831a-4430-9c6d-65af40f62573 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:06.502416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d116a-831a-4430-9c6d-65af40f62573 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:06.502677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d116a-831a-4430-9c6d-65af40f62573 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:06.503294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d116a-831a-4430-9c6d-65af40f62573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:06.505323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d116a-831a-4430-9c6d-65af40f62573 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:06.533713 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3670/7426] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:06 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:06.537979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d35c0-54b7-4037-a3d8-17a9109b887d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:06.549520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d35c0-54b7-4037-a3d8-17a9109b887d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:06.555317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcf295-0f40-4156-8cc1-5abf4ab7c64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:06.555971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcf295-0f40-4156-8cc1-5abf4ab7c64a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:06.556226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcf295-0f40-4156-8cc1-5abf4ab7c64a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:06.556530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcf295-0f40-4156-8cc1-5abf4ab7c64a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:06.557184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddcf295-0f40-4156-8cc1-5abf4ab7c64a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:06.558825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddcf295-0f40-4156-8cc1-5abf4ab7c64a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:06.574676 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3757/7427] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:25:06.579121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d116a-831a-4430-9c6d-65af40f62573 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:06.591308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d116a-831a-4430-9c6d-65af40f62573 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:06.596713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c17e9d-7b38-417a-80df-0a11807cebe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:06.597435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c17e9d-7b38-417a-80df-0a11807cebe9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:06.597775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c17e9d-7b38-417a-80df-0a11807cebe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:06.598114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c17e9d-7b38-417a-80df-0a11807cebe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:06.598840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c17e9d-7b38-417a-80df-0a11807cebe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:06.600570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c17e9d-7b38-417a-80df-0a11807cebe9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:06.618130 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3671/7428] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:06.621975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcf295-0f40-4156-8cc1-5abf4ab7c64a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:06.641056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcf295-0f40-4156-8cc1-5abf4ab7c64a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:06.649803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e0e8a-c7db-4496-81e6-251761a201ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:06.650681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e0e8a-c7db-4496-81e6-251761a201ae admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:06.650958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e0e8a-c7db-4496-81e6-251761a201ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:06.651227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e0e8a-c7db-4496-81e6-251761a201ae admin admin] PATH_INFO: `/v3/projects/472f4e95f5ad4592b00b2cc4300d44ef/users/3aba077cc532481c98e1acc409d2e45a/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:06.668078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e0e8a-c7db-4496-81e6-251761a201ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=472f4e95f5ad4592b00b2cc4300d44ef, user_id=3aba077cc532481c98e1acc409d2e45a, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:06.670986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e0e8a-c7db-4496-81e6-251761a201ae admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:06.686097 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3758/7429] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:06 2022] PUT /identity/v3/projects/472f4e95f5ad4592b00b2cc4300d44ef/users/3aba077cc532481c98e1acc409d2e45a/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:06.697494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa656d-a44b-493f-af7e-e840b7fd48e4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:06.697774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa656d-a44b-493f-af7e-e840b7fd48e4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:06.698040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa656d-a44b-493f-af7e-e840b7fd48e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:06.733648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-33aa656d-a44b-493f-af7e-e840b7fd48e4 None None] MFA Rules not processed for user `3aba077cc532481c98e1acc409d2e45a`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:06.735376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-33aa656d-a44b-493f-af7e-e840b7fd48e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:06.806936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:06.826938 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3672/7430] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:06.828228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e0e8a-c7db-4496-81e6-251761a201ae admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:06.896733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:06.938162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e0e8a-c7db-4496-81e6-251761a201ae admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:06.945734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5096e8-fa76-40da-b44d-8d0d09ce5a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:06.946558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5096e8-fa76-40da-b44d-8d0d09ce5a6c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:06.946709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5096e8-fa76-40da-b44d-8d0d09ce5a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:06.946944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5096e8-fa76-40da-b44d-8d0d09ce5a6c admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:06.947560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5096e8-fa76-40da-b44d-8d0d09ce5a6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:06.950186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5096e8-fa76-40da-b44d-8d0d09ce5a6c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:07.001122 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3759/7431] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:06 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:07.001780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9a795-d7be-4539-bb05-de9bfb6289f1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:07.002072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9a795-d7be-4539-bb05-de9bfb6289f1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:07.002346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9a795-d7be-4539-bb05-de9bfb6289f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:07.004879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5096e8-fa76-40da-b44d-8d0d09ce5a6c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:07.023550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-64c9a795-d7be-4539-bb05-de9bfb6289f1 None None] MFA Rules not processed for user `23301df1799c4676af4675f58c32a49b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:07.024799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-64c9a795-d7be-4539-bb05-de9bfb6289f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:07.075642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5096e8-fa76-40da-b44d-8d0d09ce5a6c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:07.082713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8b1b7-c9fb-4601-940c-c0ac9ad82276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:07.083437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8b1b7-c9fb-4601-940c-c0ac9ad82276 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:07.084316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8b1b7-c9fb-4601-940c-c0ac9ad82276 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:07.084316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8b1b7-c9fb-4601-940c-c0ac9ad82276 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:07.084667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8b1b7-c9fb-4601-940c-c0ac9ad82276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:07.086550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8b1b7-c9fb-4601-940c-c0ac9ad82276 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:07.088448 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3673/7432] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:07.126228 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3760/7433] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:07 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:25:07.130374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9a795-d7be-4539-bb05-de9bfb6289f1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:07.147079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9a795-d7be-4539-bb05-de9bfb6289f1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:07.148017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a4bdd-a707-4f75-8bba-fc409e357e4e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:07.148272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a4bdd-a707-4f75-8bba-fc409e357e4e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:07.148595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a4bdd-a707-4f75-8bba-fc409e357e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:07.152580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f52e9-64ed-4199-8997-0e018681b20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:07.153414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f52e9-64ed-4199-8997-0e018681b20b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:07.153761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f52e9-64ed-4199-8997-0e018681b20b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:07.154260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f52e9-64ed-4199-8997-0e018681b20b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:07.154931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f52e9-64ed-4199-8997-0e018681b20b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:07.156608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f52e9-64ed-4199-8997-0e018681b20b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:07.169456 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3674/7434] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:07.172484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-945a4bdd-a707-4f75-8bba-fc409e357e4e None None] MFA Rules not processed for user `23301df1799c4676af4675f58c32a49b`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:07.173646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f52e9-64ed-4199-8997-0e018681b20b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:07.173895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-945a4bdd-a707-4f75-8bba-fc409e357e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:07.181633 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3761/7435] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:25:07.184516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f52e9-64ed-4199-8997-0e018681b20b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:07.189734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a969329-e708-41f3-a4b3-50e0dc83685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:07.190402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a969329-e708-41f3-a4b3-50e0dc83685c admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:07.190838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a969329-e708-41f3-a4b3-50e0dc83685c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:07.191174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a969329-e708-41f3-a4b3-50e0dc83685c admin admin] PATH_INFO: `/v3/projects/c2a386adbfad49cca4834f904edb03c6/users/d1644e4a3b394d4cb97345e86d7afbaf/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:07.209568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a969329-e708-41f3-a4b3-50e0dc83685c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2a386adbfad49cca4834f904edb03c6, user_id=d1644e4a3b394d4cb97345e86d7afbaf, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:07.212305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a969329-e708-41f3-a4b3-50e0dc83685c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:07.220880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b2ab1-3315-499f-ba9c-69086114a6e0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:07.221159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b2ab1-3315-499f-ba9c-69086114a6e0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:07.221426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b2ab1-3315-499f-ba9c-69086114a6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:07.233397 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3675/7436] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:07 2022] PUT /identity/v3/projects/c2a386adbfad49cca4834f904edb03c6/users/d1644e4a3b394d4cb97345e86d7afbaf/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:25:07.237309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a969329-e708-41f3-a4b3-50e0dc83685c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:07.254102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e82b2ab1-3315-499f-ba9c-69086114a6e0 None None] MFA Rules not processed for user `23301df1799c4676af4675f58c32a49b`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:07.255334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e82b2ab1-3315-499f-ba9c-69086114a6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:07.311577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a969329-e708-41f3-a4b3-50e0dc83685c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:07.317215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b55b16-ee34-45fc-b6ad-efe81b4f9a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:07.318197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b55b16-ee34-45fc-b6ad-efe81b4f9a88 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:07.318519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b55b16-ee34-45fc-b6ad-efe81b4f9a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:07.318774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b55b16-ee34-45fc-b6ad-efe81b4f9a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:07.320035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b55b16-ee34-45fc-b6ad-efe81b4f9a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:07.321888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b55b16-ee34-45fc-b6ad-efe81b4f9a88 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:07.329677 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3762/7437] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:07.336324 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3676/7438] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:25:07.337812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b2ab1-3315-499f-ba9c-69086114a6e0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:07.351876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b2ab1-3315-499f-ba9c-69086114a6e0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:07.354471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8358d-7077-47ef-8ea9-6fd027e15002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:07.355148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8358d-7077-47ef-8ea9-6fd027e15002 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:07.355430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8358d-7077-47ef-8ea9-6fd027e15002 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:07.355714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8358d-7077-47ef-8ea9-6fd027e15002 admin admin] PATH_INFO: `/v3/projects/c2a386adbfad49cca4834f904edb03c6/users/d1644e4a3b394d4cb97345e86d7afbaf/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:07.357640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8358d-7077-47ef-8ea9-6fd027e15002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2a386adbfad49cca4834f904edb03c6, user_id=d1644e4a3b394d4cb97345e86d7afbaf, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:07.360354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8358d-7077-47ef-8ea9-6fd027e15002 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:07.377940 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3763/7439] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:07 2022] PUT /identity/v3/projects/c2a386adbfad49cca4834f904edb03c6/users/d1644e4a3b394d4cb97345e86d7afbaf/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:07.383062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c43c85-4b6d-4d10-a9cb-ae94776dcacc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:07.383343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f51f8-19a0-4397-baf1-f5208f3d8df8 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:07.383639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f51f8-19a0-4397-baf1-f5208f3d8df8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:07.383809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c43c85-4b6d-4d10-a9cb-ae94776dcacc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:07.383961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f51f8-19a0-4397-baf1-f5208f3d8df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:07.384120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c43c85-4b6d-4d10-a9cb-ae94776dcacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:07.414970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-63c43c85-4b6d-4d10-a9cb-ae94776dcacc None None] MFA Rules not processed for user `23301df1799c4676af4675f58c32a49b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:07.416422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-126f51f8-19a0-4397-baf1-f5208f3d8df8 None None] MFA Rules not processed for user `d1644e4a3b394d4cb97345e86d7afbaf`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:07.416841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-63c43c85-4b6d-4d10-a9cb-ae94776dcacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:07.417617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-126f51f8-19a0-4397-baf1-f5208f3d8df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:07.439983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:07.507977 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3764/7441] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:07.508206 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3677/7441] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:25:07.520344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c43c85-4b6d-4d10-a9cb-ae94776dcacc None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:07.522061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-63c43c85-4b6d-4d10-a9cb-ae94776dcacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:07.536518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:07.552918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8be88-45db-411c-b007-6a7fbb3dc49f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:07.553225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8be88-45db-411c-b007-6a7fbb3dc49f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:07.553486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8be88-45db-411c-b007-6a7fbb3dc49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:07.581784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c8d8be88-45db-411c-b007-6a7fbb3dc49f None None] MFA Rules not processed for user `23301df1799c4676af4675f58c32a49b`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:07.583202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c8d8be88-45db-411c-b007-6a7fbb3dc49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:07.601678 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3765/7442] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:25:07.603234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c43c85-4b6d-4d10-a9cb-ae94776dcacc None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:07.606925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f39cdf-e1a4-4014-846b-538f0984f72b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:07.607669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f39cdf-e1a4-4014-846b-538f0984f72b service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:07.607845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f39cdf-e1a4-4014-846b-538f0984f72b service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:07.608060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f39cdf-e1a4-4014-846b-538f0984f72b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:07.677481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f39cdf-e1a4-4014-846b-538f0984f72b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:07.679249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f39cdf-e1a4-4014-846b-538f0984f72b service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:07.690008 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3678/7443] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:25:10.907412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8be88-45db-411c-b007-6a7fbb3dc49f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:10.972011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8be88-45db-411c-b007-6a7fbb3dc49f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:10.977003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:10.977626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:10.977895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:10.978150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] PATH_INFO: `/v3/users/86230b04dcab49a0a40c49df6c403c0f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:10.979281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86230b04dcab49a0a40c49df6c403c0f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:10.981321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:11.038712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86230b04dcab49a0a40c49df6c403c0f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:11.048116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86230b04dcab49a0a40c49df6c403c0f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:11.055857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86230b04dcab49a0a40c49df6c403c0f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:11.069871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86230b04dcab49a0a40c49df6c403c0f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:11.070672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86230b04dcab49a0a40c49df6c403c0f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:11.077179 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3766/7444] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:10 2022] DELETE /identity/v3/users/86230b04dcab49a0a40c49df6c403c0f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:11.082170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffb0b6-756b-4da3-bd8b-a172926aea23 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:11.082692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffb0b6-756b-4da3-bd8b-a172926aea23 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:11.082826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffb0b6-756b-4da3-bd8b-a172926aea23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:11.120635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b3ffb0b6-756b-4da3-bd8b-a172926aea23 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:11.121977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b3ffb0b6-756b-4da3-bd8b-a172926aea23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:11.183501 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3679/7445] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:11.193782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:11.195235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:11.234923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:11.270488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2a134-a846-4c2f-b7f1-fe8553277b49 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:11.275430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14817058-652b-4355-8812-0cef4b491d5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:11.276212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14817058-652b-4355-8812-0cef4b491d5c service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:11.276526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14817058-652b-4355-8812-0cef4b491d5c service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:11.276792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14817058-652b-4355-8812-0cef4b491d5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:11.290538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14817058-652b-4355-8812-0cef4b491d5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:11.293462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14817058-652b-4355-8812-0cef4b491d5c service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:11.308648 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3767/7446] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:11.920778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffb0b6-756b-4da3-bd8b-a172926aea23 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:11.922399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b3ffb0b6-756b-4da3-bd8b-a172926aea23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:11.950880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffb0b6-756b-4da3-bd8b-a172926aea23 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:11.960388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:11.961114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:11.961570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:11.961920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] PATH_INFO: `/v3/projects/87a82f2c07b24e2d88831e593d4a09e2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:11.975203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87a82f2c07b24e2d88831e593d4a09e2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:11.980851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:12.068990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87a82f2c07b24e2d88831e593d4a09e2', 'request_id': 'req-ee697a19-9d52-4a16-98a7-105d1168d3a4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:12.078524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87a82f2c07b24e2d88831e593d4a09e2', 'request_id': 'req-ee697a19-9d52-4a16-98a7-105d1168d3a4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:12.080681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:12.097039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87a82f2c07b24e2d88831e593d4a09e2', 'request_id': 'req-ee697a19-9d52-4a16-98a7-105d1168d3a4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:12.112262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] The token cache is being invalidate because project 87a82f2c07b24e2d88831e593d4a09e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:12.112593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:12.113727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:12.115776 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3680/7447] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:11 2022] DELETE /identity/v3/projects/87a82f2c07b24e2d88831e593d4a09e2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:12.120933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14817058-652b-4355-8812-0cef4b491d5c service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:12.137244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14817058-652b-4355-8812-0cef4b491d5c service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:12.242891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:12.243562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:12.243842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:12.244112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] PATH_INFO: `/v3/users/3aba077cc532481c98e1acc409d2e45a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:12.245740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aba077cc532481c98e1acc409d2e45a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:12.249792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:12.323520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aba077cc532481c98e1acc409d2e45a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:12.333948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aba077cc532481c98e1acc409d2e45a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:12.341923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aba077cc532481c98e1acc409d2e45a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:12.357190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aba077cc532481c98e1acc409d2e45a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:12.358172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aba077cc532481c98e1acc409d2e45a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:12.367555 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3768/7448] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:12 2022] DELETE /identity/v3/users/3aba077cc532481c98e1acc409d2e45a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:12.986621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:12.988926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:13.008395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee697a19-9d52-4a16-98a7-105d1168d3a4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:13.083914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:13.084811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:13.085358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:13.085858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] PATH_INFO: `/v3/projects/472f4e95f5ad4592b00b2cc4300d44ef` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:13.087547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=472f4e95f5ad4592b00b2cc4300d44ef)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:13.090134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:13.162465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '472f4e95f5ad4592b00b2cc4300d44ef', 'request_id': 'req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:13.172475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '472f4e95f5ad4592b00b2cc4300d44ef', 'request_id': 'req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:13.182814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '472f4e95f5ad4592b00b2cc4300d44ef', 'request_id': 'req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:13.195873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] The token cache is being invalidate because project 472f4e95f5ad4592b00b2cc4300d44ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:13.196342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:13.199355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:13.201500 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3681/7449] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:12 2022] DELETE /identity/v3/projects/472f4e95f5ad4592b00b2cc4300d44ef => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:13.205411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:13.207759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:13.222685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d687c-ca54-4571-8f0f-675de186d663 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:13.287955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:13.302524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c72d4-905b-4ca1-bbde-978e33619487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:13.304170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c72d4-905b-4ca1-bbde-978e33619487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:13.304684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c72d4-905b-4ca1-bbde-978e33619487 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:13.305136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c72d4-905b-4ca1-bbde-978e33619487 admin admin] PATH_INFO: `/v3/users/d1644e4a3b394d4cb97345e86d7afbaf` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:13.307292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c72d4-905b-4ca1-bbde-978e33619487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1644e4a3b394d4cb97345e86d7afbaf)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:13.309723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c72d4-905b-4ca1-bbde-978e33619487 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:13.422421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9b7c72d4-905b-4ca1-bbde-978e33619487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1644e4a3b394d4cb97345e86d7afbaf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:13.436242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9b7c72d4-905b-4ca1-bbde-978e33619487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1644e4a3b394d4cb97345e86d7afbaf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:13.450854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9b7c72d4-905b-4ca1-bbde-978e33619487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1644e4a3b394d4cb97345e86d7afbaf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:13.473711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9b7c72d4-905b-4ca1-bbde-978e33619487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1644e4a3b394d4cb97345e86d7afbaf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:13.475234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9b7c72d4-905b-4ca1-bbde-978e33619487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1644e4a3b394d4cb97345e86d7afbaf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:13.488443 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3769/7450] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:13 2022] DELETE /identity/v3/users/d1644e4a3b394d4cb97345e86d7afbaf => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:14.309149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:14.310696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:14.325075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc1439-e0d2-4460-9ba0-fda24a7672dd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:14.428607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:14.429693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:14.430153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:14.430578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee admin admin] PATH_INFO: `/v3/projects/c2a386adbfad49cca4834f904edb03c6` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:14.432088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2a386adbfad49cca4834f904edb03c6)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:14.434827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:14.457705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:14.506948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2a386adbfad49cca4834f904edb03c6', 'request_id': 'req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:14.516352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2a386adbfad49cca4834f904edb03c6', 'request_id': 'req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:14.528112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2a386adbfad49cca4834f904edb03c6', 'request_id': 'req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:14.541806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee admin admin] The token cache is being invalidate because project c2a386adbfad49cca4834f904edb03c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:14.542422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:14.543454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3e8f9c17-cce4-438f-85e2-f3fc89cb0dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:14.545580 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3682/7451] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:14 2022] DELETE /identity/v3/projects/c2a386adbfad49cca4834f904edb03c6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:14.550550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff22c40-9882-49b8-8f29-f5fd4907b673 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:14.551091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff22c40-9882-49b8-8f29-f5fd4907b673 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:14.551091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff22c40-9882-49b8-8f29-f5fd4907b673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:14.586193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5ff22c40-9882-49b8-8f29-f5fd4907b673 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:14.587454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5ff22c40-9882-49b8-8f29-f5fd4907b673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:14.716884 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3770/7452] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:14.719653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7609d-106f-4c77-86a9-dfaee1639bdd None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:14.720173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7609d-106f-4c77-86a9-dfaee1639bdd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:14.720689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7609d-106f-4c77-86a9-dfaee1639bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:14.768222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0ba7609d-106f-4c77-86a9-dfaee1639bdd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:14.769727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0ba7609d-106f-4c77-86a9-dfaee1639bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:14.804613 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3683/7453] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:14.808807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff22c40-9882-49b8-8f29-f5fd4907b673 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:14.821095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff22c40-9882-49b8-8f29-f5fd4907b673 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:14.825776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d26026-4cf5-41d2-8fc0-a6129f62374a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:14.826199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d26026-4cf5-41d2-8fc0-a6129f62374a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:14.826483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d26026-4cf5-41d2-8fc0-a6129f62374a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:14.826748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d26026-4cf5-41d2-8fc0-a6129f62374a admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:14.827398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d26026-4cf5-41d2-8fc0-a6129f62374a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:25:14.827691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d26026-4cf5-41d2-8fc0-a6129f62374a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:14.829446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d26026-4cf5-41d2-8fc0-a6129f62374a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:14.850744 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3771/7454] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:25:14.855226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7609d-106f-4c77-86a9-dfaee1639bdd None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:14.869328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7609d-106f-4c77-86a9-dfaee1639bdd None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:14.875534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bafdc6-3c5d-42bf-8ce3-f579e37c76a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:14.876187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bafdc6-3c5d-42bf-8ce3-f579e37c76a2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:14.876498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bafdc6-3c5d-42bf-8ce3-f579e37c76a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:14.876761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bafdc6-3c5d-42bf-8ce3-f579e37c76a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:14.877393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bafdc6-3c5d-42bf-8ce3-f579e37c76a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:14.880704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bafdc6-3c5d-42bf-8ce3-f579e37c76a2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:14.949620 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3684/7455] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:14 2022] POST /identity/v3/projects => generated 381 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:14.955453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d26026-4cf5-41d2-8fc0-a6129f62374a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:14.968359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d26026-4cf5-41d2-8fc0-a6129f62374a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:15.074861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51841374-bf20-466b-a2db-f8444a482734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:15.075529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51841374-bf20-466b-a2db-f8444a482734 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:15.075806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51841374-bf20-466b-a2db-f8444a482734 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:15.076551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51841374-bf20-466b-a2db-f8444a482734 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:15.076763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51841374-bf20-466b-a2db-f8444a482734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:15.079085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51841374-bf20-466b-a2db-f8444a482734 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:15.088957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:15.109963 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3772/7456] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:14 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:15.114055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bafdc6-3c5d-42bf-8ce3-f579e37c76a2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:15.127940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bafdc6-3c5d-42bf-8ce3-f579e37c76a2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:15.136755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfc28f-7a71-45da-9a5b-33a85cf65859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:15.137368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfc28f-7a71-45da-9a5b-33a85cf65859 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:15.137645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfc28f-7a71-45da-9a5b-33a85cf65859 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:15.137911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfc28f-7a71-45da-9a5b-33a85cf65859 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:15.138562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfc28f-7a71-45da-9a5b-33a85cf65859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:15.140364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfc28f-7a71-45da-9a5b-33a85cf65859 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:15.161792 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3685/7457] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:15.163315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51841374-bf20-466b-a2db-f8444a482734 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:15.181728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51841374-bf20-466b-a2db-f8444a482734 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:15.189393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec9187-2bd5-44cf-a41a-b8c572bf98a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:15.190297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9187-2bd5-44cf-a41a-b8c572bf98a4 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:15.190796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9187-2bd5-44cf-a41a-b8c572bf98a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:15.191141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9187-2bd5-44cf-a41a-b8c572bf98a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:15.191893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec9187-2bd5-44cf-a41a-b8c572bf98a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:15.193969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec9187-2bd5-44cf-a41a-b8c572bf98a4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:15.215734 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3773/7458] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:15.216648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfc28f-7a71-45da-9a5b-33a85cf65859 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:15.252996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfc28f-7a71-45da-9a5b-33a85cf65859 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:15.261330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638c968-f3e1-44de-9dcf-3243b2324e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:15.262012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638c968-f3e1-44de-9dcf-3243b2324e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:15.262969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638c968-f3e1-44de-9dcf-3243b2324e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:15.262969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638c968-f3e1-44de-9dcf-3243b2324e2a admin admin] PATH_INFO: `/v3/projects/d7717e18950144c78daf9b1ce66e88ab/users/6638b8f466b64d2fb88afb871f9fd589/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:15.294556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638c968-f3e1-44de-9dcf-3243b2324e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7717e18950144c78daf9b1ce66e88ab, user_id=6638b8f466b64d2fb88afb871f9fd589, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:15.301882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638c968-f3e1-44de-9dcf-3243b2324e2a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:15.325071 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3686/7459] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:15 2022] PUT /identity/v3/projects/d7717e18950144c78daf9b1ce66e88ab/users/6638b8f466b64d2fb88afb871f9fd589/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:25:15.331292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af9d89-000f-45e2-8676-4fb333f170ff None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:15.331623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af9d89-000f-45e2-8676-4fb333f170ff None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:15.331837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af9d89-000f-45e2-8676-4fb333f170ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:15.364618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b8af9d89-000f-45e2-8676-4fb333f170ff None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:15.368848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b8af9d89-000f-45e2-8676-4fb333f170ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:15.479276 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3774/7460] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:15.521851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638c968-f3e1-44de-9dcf-3243b2324e2a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:15.523690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f638c968-f3e1-44de-9dcf-3243b2324e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:15.603677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638c968-f3e1-44de-9dcf-3243b2324e2a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:15.607539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f26b10-cf68-4558-9e68-078f9bf85c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:15.608273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f26b10-cf68-4558-9e68-078f9bf85c92 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:15.608663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f26b10-cf68-4558-9e68-078f9bf85c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:15.608998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f26b10-cf68-4558-9e68-078f9bf85c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:15.620171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f26b10-cf68-4558-9e68-078f9bf85c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:15.622033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f26b10-cf68-4558-9e68-078f9bf85c92 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:15.636321 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3687/7461] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:18.750312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120752a-38c2-4843-a8b7-fd9f85d56e46 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:18.751490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120752a-38c2-4843-a8b7-fd9f85d56e46 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:18.751490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120752a-38c2-4843-a8b7-fd9f85d56e46 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:18.752319 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3775/7462] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:25:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:25:18.760298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a67199-adb9-4781-a7b8-b16da076e7c2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:18.760807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a67199-adb9-4781-a7b8-b16da076e7c2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:18.760960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a67199-adb9-4781-a7b8-b16da076e7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:18.775632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120752a-38c2-4843-a8b7-fd9f85d56e46 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:18.777698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2120752a-38c2-4843-a8b7-fd9f85d56e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:18.824033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120752a-38c2-4843-a8b7-fd9f85d56e46 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:18.834293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:18.836051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:18.836769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:18.845232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] PATH_INFO: `/v3/users/23301df1799c4676af4675f58c32a49b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:18.848127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=23301df1799c4676af4675f58c32a49b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:18.850223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:18.868086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-23a67199-adb9-4781-a7b8-b16da076e7c2 None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:18.870178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-23a67199-adb9-4781-a7b8-b16da076e7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:18.885549 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3688/7463] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:25:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:18.889161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a96365-b36e-46bd-9ed6-509f424e46f7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:18.889505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a96365-b36e-46bd-9ed6-509f424e46f7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:18.889781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a96365-b36e-46bd-9ed6-509f424e46f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:18.922812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:18.979350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c5a96365-b36e-46bd-9ed6-509f424e46f7 None None] MFA Rules not processed for user `6638b8f466b64d2fb88afb871f9fd589`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:18.982233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c5a96365-b36e-46bd-9ed6-509f424e46f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:18.987481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23301df1799c4676af4675f58c32a49b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:19.009591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23301df1799c4676af4675f58c32a49b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:19.023080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23301df1799c4676af4675f58c32a49b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:19.054095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23301df1799c4676af4675f58c32a49b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:19.054863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23301df1799c4676af4675f58c32a49b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:19.086227 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3776/7464] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:18 2022] DELETE /identity/v3/users/23301df1799c4676af4675f58c32a49b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:25:19.086227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:19.088318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:19.144181 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3689/7465] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:19.156173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a96365-b36e-46bd-9ed6-509f424e46f7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:19.160572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c5a96365-b36e-46bd-9ed6-509f424e46f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:19.295232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91764e17-5f59-41b9-be42-a995005f3c8c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:19.301260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcfb73-ab0b-4730-a058-e88f67f9300d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:19.302358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcfb73-ab0b-4730-a058-e88f67f9300d service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:19.303878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcfb73-ab0b-4730-a058-e88f67f9300d service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:19.304342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcfb73-ab0b-4730-a058-e88f67f9300d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:19.307309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8cfcfb73-ab0b-4730-a058-e88f67f9300d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:19.319987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:19.323250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfcfb73-ab0b-4730-a058-e88f67f9300d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:19.325887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfcfb73-ab0b-4730-a058-e88f67f9300d service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:19.336712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a96365-b36e-46bd-9ed6-509f424e46f7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:19.339493 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3777/7466] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:19.467987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f22b50-b404-41b7-8a82-b88e169afd44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:19.469119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f22b50-b404-41b7-8a82-b88e169afd44 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:19.469457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f22b50-b404-41b7-8a82-b88e169afd44 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:19.469726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f22b50-b404-41b7-8a82-b88e169afd44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:19.561225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f22b50-b404-41b7-8a82-b88e169afd44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:19.563330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f22b50-b404-41b7-8a82-b88e169afd44 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:19.592677 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3690/7467] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:19.593098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcfb73-ab0b-4730-a058-e88f67f9300d service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:19.595563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8cfcfb73-ab0b-4730-a058-e88f67f9300d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:19.713410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcfb73-ab0b-4730-a058-e88f67f9300d service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:19.721336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:19.722482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:19.722864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:19.723822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481 admin admin] PATH_INFO: `/v3/projects/8072f03884004cddb2e3a429846e3257` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:19.725463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8072f03884004cddb2e3a429846e3257)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:19.728478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:19.807307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8072f03884004cddb2e3a429846e3257', 'request_id': 'req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:19.838956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8072f03884004cddb2e3a429846e3257', 'request_id': 'req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:19.848186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8072f03884004cddb2e3a429846e3257', 'request_id': 'req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:19.879853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481 admin admin] The token cache is being invalidate because project 8072f03884004cddb2e3a429846e3257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:19.880194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:19.880981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2c8cc642-6e2a-44a2-93b4-96bfc52e8481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:19.883327 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3778/7468] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:19 2022] DELETE /identity/v3/projects/8072f03884004cddb2e3a429846e3257 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:19.889591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926618f0-b4e8-4cc1-8c87-f19990387c9a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:19.890085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926618f0-b4e8-4cc1-8c87-f19990387c9a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:19.890172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926618f0-b4e8-4cc1-8c87-f19990387c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:19.901401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:19.917134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-926618f0-b4e8-4cc1-8c87-f19990387c9a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:19.919215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-926618f0-b4e8-4cc1-8c87-f19990387c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:20.034755 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3691/7469] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:20.044761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd0fa3-fa3b-4021-b1db-21ff7e859d15 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:20.045241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd0fa3-fa3b-4021-b1db-21ff7e859d15 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:20.045693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd0fa3-fa3b-4021-b1db-21ff7e859d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:20.093828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bcbd0fa3-fa3b-4021-b1db-21ff7e859d15 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:20.099451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bcbd0fa3-fa3b-4021-b1db-21ff7e859d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:20.114051 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3779/7470] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:20.118269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926618f0-b4e8-4cc1-8c87-f19990387c9a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:20.129663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926618f0-b4e8-4cc1-8c87-f19990387c9a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:20.134997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1a8ed-3745-4c69-bc56-47c5ad45adb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:20.135667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1a8ed-3745-4c69-bc56-47c5ad45adb7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:20.135936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1a8ed-3745-4c69-bc56-47c5ad45adb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:20.136353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1a8ed-3745-4c69-bc56-47c5ad45adb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:20.136969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1a8ed-3745-4c69-bc56-47c5ad45adb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:25:20.137616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1a8ed-3745-4c69-bc56-47c5ad45adb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:20.139572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1a8ed-3745-4c69-bc56-47c5ad45adb7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:20.161703 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3692/7471] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:25:20.165755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd0fa3-fa3b-4021-b1db-21ff7e859d15 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:20.189872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd0fa3-fa3b-4021-b1db-21ff7e859d15 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:20.196061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e9f77-45ac-4287-ac0c-9a1e9cfec969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:20.197026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e9f77-45ac-4287-ac0c-9a1e9cfec969 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:20.197365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e9f77-45ac-4287-ac0c-9a1e9cfec969 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:20.197693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e9f77-45ac-4287-ac0c-9a1e9cfec969 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:20.198467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e9f77-45ac-4287-ac0c-9a1e9cfec969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:20.201325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e9f77-45ac-4287-ac0c-9a1e9cfec969 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:20.236482 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3780/7472] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:20 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:20.239349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1a8ed-3745-4c69-bc56-47c5ad45adb7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:20.252999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1a8ed-3745-4c69-bc56-47c5ad45adb7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:20.376538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed0882-45d1-425c-8b1a-66591e7151bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:20.377424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed0882-45d1-425c-8b1a-66591e7151bd admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:20.377803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed0882-45d1-425c-8b1a-66591e7151bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:20.378174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed0882-45d1-425c-8b1a-66591e7151bd admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:20.378900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed0882-45d1-425c-8b1a-66591e7151bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:20.383684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed0882-45d1-425c-8b1a-66591e7151bd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:20.416293 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3693/7473] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:20 2022] POST /identity/v3/users => generated 287 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:25:20.423106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e9f77-45ac-4287-ac0c-9a1e9cfec969 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:20.441117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e9f77-45ac-4287-ac0c-9a1e9cfec969 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:20.456356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c62346-291a-4a12-8eb0-6ca13cf744b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:20.457008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c62346-291a-4a12-8eb0-6ca13cf744b2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:20.457283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c62346-291a-4a12-8eb0-6ca13cf744b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:20.457550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c62346-291a-4a12-8eb0-6ca13cf744b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:20.458194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c62346-291a-4a12-8eb0-6ca13cf744b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:20.460523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c62346-291a-4a12-8eb0-6ca13cf744b2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:20.489268 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3781/7474] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:20.489670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed0882-45d1-425c-8b1a-66591e7151bd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:20.511875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed0882-45d1-425c-8b1a-66591e7151bd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:20.525330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a9a41-c3f5-47c9-b726-6ccc19595489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:20.526219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a9a41-c3f5-47c9-b726-6ccc19595489 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:20.527326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a9a41-c3f5-47c9-b726-6ccc19595489 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:20.527707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a9a41-c3f5-47c9-b726-6ccc19595489 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:20.529967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a9a41-c3f5-47c9-b726-6ccc19595489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:20.532277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a9a41-c3f5-47c9-b726-6ccc19595489 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:20.563491 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3694/7475] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:20.569425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c62346-291a-4a12-8eb0-6ca13cf744b2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:20.588712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c62346-291a-4a12-8eb0-6ca13cf744b2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:20.597129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14c936-ee26-4967-a48c-9d4668a06ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:20.599018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14c936-ee26-4967-a48c-9d4668a06ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:20.599285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14c936-ee26-4967-a48c-9d4668a06ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:20.606596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14c936-ee26-4967-a48c-9d4668a06ed0 admin admin] PATH_INFO: `/v3/projects/794c95c5336549e9914787c32be79ae1/users/d84dca25404141318e5f118ab80e6a86/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:20.631459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14c936-ee26-4967-a48c-9d4668a06ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=794c95c5336549e9914787c32be79ae1, user_id=d84dca25404141318e5f118ab80e6a86, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:20.634402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14c936-ee26-4967-a48c-9d4668a06ed0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:20.656457 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3782/7476] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:20 2022] PUT /identity/v3/projects/794c95c5336549e9914787c32be79ae1/users/d84dca25404141318e5f118ab80e6a86/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:20.661591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d2dfd-1d2a-440d-82be-3732684656cb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:20.661997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d2dfd-1d2a-440d-82be-3732684656cb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:20.662367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d2dfd-1d2a-440d-82be-3732684656cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:20.697731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fd8d2dfd-1d2a-440d-82be-3732684656cb None None] MFA Rules not processed for user `d84dca25404141318e5f118ab80e6a86`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:20.698915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fd8d2dfd-1d2a-440d-82be-3732684656cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:20.780203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:20.786272 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3695/7477] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:20.800423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859ce9a-e5ba-4fbd-9782-568fde4e67bf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:20.800709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859ce9a-e5ba-4fbd-9782-568fde4e67bf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:20.800982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859ce9a-e5ba-4fbd-9782-568fde4e67bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:20.822345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d859ce9a-e5ba-4fbd-9782-568fde4e67bf None None] MFA Rules not processed for user `d84dca25404141318e5f118ab80e6a86`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:20.823561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d859ce9a-e5ba-4fbd-9782-568fde4e67bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:20.831719 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3783/7478] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:20.842946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d2dfd-1d2a-440d-82be-3732684656cb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:20.845248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fd8d2dfd-1d2a-440d-82be-3732684656cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:20.962076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d2dfd-1d2a-440d-82be-3732684656cb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:20.988551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b731492-f2c4-4f2d-92b2-e62497341801 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:20.989241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b731492-f2c4-4f2d-92b2-e62497341801 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:20.989665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b731492-f2c4-4f2d-92b2-e62497341801 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:20.989989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b731492-f2c4-4f2d-92b2-e62497341801 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:21.001408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b731492-f2c4-4f2d-92b2-e62497341801 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:21.005754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b731492-f2c4-4f2d-92b2-e62497341801 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:21.020841 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3696/7479] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:22.359549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859ce9a-e5ba-4fbd-9782-568fde4e67bf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:22.362380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d859ce9a-e5ba-4fbd-9782-568fde4e67bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:22.383076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:22.446813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859ce9a-e5ba-4fbd-9782-568fde4e67bf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:22.451683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1207e-2b02-46f3-83a0-19fc93ff899b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:22.452390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1207e-2b02-46f3-83a0-19fc93ff899b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:22.452672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1207e-2b02-46f3-83a0-19fc93ff899b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:22.453045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1207e-2b02-46f3-83a0-19fc93ff899b admin admin] PATH_INFO: `/v3/users/6638b8f466b64d2fb88afb871f9fd589` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:22.454554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1207e-2b02-46f3-83a0-19fc93ff899b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6638b8f466b64d2fb88afb871f9fd589)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:22.456578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1207e-2b02-46f3-83a0-19fc93ff899b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:22.514598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8ab1207e-2b02-46f3-83a0-19fc93ff899b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6638b8f466b64d2fb88afb871f9fd589'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:22.523604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8ab1207e-2b02-46f3-83a0-19fc93ff899b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6638b8f466b64d2fb88afb871f9fd589'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:22.530729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8ab1207e-2b02-46f3-83a0-19fc93ff899b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6638b8f466b64d2fb88afb871f9fd589'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:22.543797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8ab1207e-2b02-46f3-83a0-19fc93ff899b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6638b8f466b64d2fb88afb871f9fd589'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:22.544552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8ab1207e-2b02-46f3-83a0-19fc93ff899b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6638b8f466b64d2fb88afb871f9fd589'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:22.550870 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3784/7480] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:22 2022] DELETE /identity/v3/users/6638b8f466b64d2fb88afb871f9fd589 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:22.810535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b731492-f2c4-4f2d-92b2-e62497341801 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:22.828254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b731492-f2c4-4f2d-92b2-e62497341801 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:22.923966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50724488-3efd-4015-a7e8-d3877416e040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:22.924949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50724488-3efd-4015-a7e8-d3877416e040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:22.925432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50724488-3efd-4015-a7e8-d3877416e040 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:22.925853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50724488-3efd-4015-a7e8-d3877416e040 admin admin] PATH_INFO: `/v3/projects/d7717e18950144c78daf9b1ce66e88ab` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:22.927013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50724488-3efd-4015-a7e8-d3877416e040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7717e18950144c78daf9b1ce66e88ab)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:22.929338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50724488-3efd-4015-a7e8-d3877416e040 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:22.955032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e15c6-771b-4714-b5ee-f1b7cd8d4643 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:22.955846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e15c6-771b-4714-b5ee-f1b7cd8d4643 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:22.956568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e15c6-771b-4714-b5ee-f1b7cd8d4643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:22.977907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2c1e15c6-771b-4714-b5ee-f1b7cd8d4643 None None] MFA Rules not processed for user `4e6a5f35d172410481a8416b7bdaafac`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:22.979118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2c1e15c6-771b-4714-b5ee-f1b7cd8d4643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:23.009648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-50724488-3efd-4015-a7e8-d3877416e040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7717e18950144c78daf9b1ce66e88ab', 'request_id': 'req-50724488-3efd-4015-a7e8-d3877416e040'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:23.017491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-50724488-3efd-4015-a7e8-d3877416e040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7717e18950144c78daf9b1ce66e88ab', 'request_id': 'req-50724488-3efd-4015-a7e8-d3877416e040'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:23.027336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-50724488-3efd-4015-a7e8-d3877416e040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7717e18950144c78daf9b1ce66e88ab', 'request_id': 'req-50724488-3efd-4015-a7e8-d3877416e040'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:23.040302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-50724488-3efd-4015-a7e8-d3877416e040 admin admin] The token cache is being invalidate because project d7717e18950144c78daf9b1ce66e88ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:23.040716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-50724488-3efd-4015-a7e8-d3877416e040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:23.041327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-50724488-3efd-4015-a7e8-d3877416e040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:23.043226 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3697/7481] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:22 2022] DELETE /identity/v3/projects/d7717e18950144c78daf9b1ce66e88ab => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:23.049193 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3785/7482] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:25:23.054832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b336d6-a07c-41d8-8830-83f32fdd70b0 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:23.055397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b336d6-a07c-41d8-8830-83f32fdd70b0 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:23.055813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b336d6-a07c-41d8-8830-83f32fdd70b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:23.077793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e3b336d6-a07c-41d8-8830-83f32fdd70b0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:23.079439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e3b336d6-a07c-41d8-8830-83f32fdd70b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:23.090208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:23.149391 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3698/7483] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:23.157309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c395d-24d0-406f-aa58-ad294bda8c7b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:23.157513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c395d-24d0-406f-aa58-ad294bda8c7b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:23.157788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c395d-24d0-406f-aa58-ad294bda8c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:23.179610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-039c395d-24d0-406f-aa58-ad294bda8c7b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:23.180867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-039c395d-24d0-406f-aa58-ad294bda8c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:23.190326 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3786/7484] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:25:23.196183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b336d6-a07c-41d8-8830-83f32fdd70b0 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:23.207320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b336d6-a07c-41d8-8830-83f32fdd70b0 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:23.212017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fa5e1-682c-4d45-b16f-7a3df38e26ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:23.212762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fa5e1-682c-4d45-b16f-7a3df38e26ad admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:23.213038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fa5e1-682c-4d45-b16f-7a3df38e26ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:23.213308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fa5e1-682c-4d45-b16f-7a3df38e26ad admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:23.213951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fa5e1-682c-4d45-b16f-7a3df38e26ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:25:23.214252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fa5e1-682c-4d45-b16f-7a3df38e26ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:23.215859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fa5e1-682c-4d45-b16f-7a3df38e26ad admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:23.235323 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3699/7485] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:25:23.238725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c395d-24d0-406f-aa58-ad294bda8c7b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:23.249783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c395d-24d0-406f-aa58-ad294bda8c7b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:23.257093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034c8ea-75f6-4fd8-86b0-faf505c41d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:23.257991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034c8ea-75f6-4fd8-86b0-faf505c41d84 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:23.258324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034c8ea-75f6-4fd8-86b0-faf505c41d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:23.258658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034c8ea-75f6-4fd8-86b0-faf505c41d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:23.259903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034c8ea-75f6-4fd8-86b0-faf505c41d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:23.261421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034c8ea-75f6-4fd8-86b0-faf505c41d84 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:23.296548 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3787/7486] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:23 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:23.301714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fa5e1-682c-4d45-b16f-7a3df38e26ad admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:23.315153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fa5e1-682c-4d45-b16f-7a3df38e26ad admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:23.390525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7e1da-863f-4dad-9f51-fc7e153f1626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:23.391591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7e1da-863f-4dad-9f51-fc7e153f1626 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:23.391919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7e1da-863f-4dad-9f51-fc7e153f1626 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:23.392190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7e1da-863f-4dad-9f51-fc7e153f1626 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:23.392887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7e1da-863f-4dad-9f51-fc7e153f1626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:23.395068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7e1da-863f-4dad-9f51-fc7e153f1626 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:23.423911 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3700/7487] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:23 2022] POST /identity/v3/users => generated 282 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:25:23.427948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034c8ea-75f6-4fd8-86b0-faf505c41d84 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:23.446978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034c8ea-75f6-4fd8-86b0-faf505c41d84 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:23.452710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b74c2a-c75f-4bdc-9f6a-e8b177907b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:23.454137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b74c2a-c75f-4bdc-9f6a-e8b177907b1e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:23.454476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b74c2a-c75f-4bdc-9f6a-e8b177907b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:23.454807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b74c2a-c75f-4bdc-9f6a-e8b177907b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:23.455517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b74c2a-c75f-4bdc-9f6a-e8b177907b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:23.457426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b74c2a-c75f-4bdc-9f6a-e8b177907b1e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:23.460781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:23.484858 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3788/7488] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:23.488741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7e1da-863f-4dad-9f51-fc7e153f1626 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:23.502579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7e1da-863f-4dad-9f51-fc7e153f1626 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:23.509000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f03e7-181c-45c6-a651-70f6f048d489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:23.509301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f03e7-181c-45c6-a651-70f6f048d489 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:23.509576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f03e7-181c-45c6-a651-70f6f048d489 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:23.509840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f03e7-181c-45c6-a651-70f6f048d489 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:23.511522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f03e7-181c-45c6-a651-70f6f048d489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:23.512666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f03e7-181c-45c6-a651-70f6f048d489 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:23.526217 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3701/7489] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:25:23.530330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b74c2a-c75f-4bdc-9f6a-e8b177907b1e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:23.541816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b74c2a-c75f-4bdc-9f6a-e8b177907b1e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:23.548118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c92f0-7fc1-4d10-80b8-1b2b4728f258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:23.549484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c92f0-7fc1-4d10-80b8-1b2b4728f258 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:23.549762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c92f0-7fc1-4d10-80b8-1b2b4728f258 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:23.550022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c92f0-7fc1-4d10-80b8-1b2b4728f258 admin admin] PATH_INFO: `/v3/projects/b61501771f9d458db280f1b0d2f141c7/users/f73c15c26f5b4dc2a5505b2cd37177bd/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:23.568931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c92f0-7fc1-4d10-80b8-1b2b4728f258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b61501771f9d458db280f1b0d2f141c7, user_id=f73c15c26f5b4dc2a5505b2cd37177bd, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:23.571621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c92f0-7fc1-4d10-80b8-1b2b4728f258 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:23.590731 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3789/7490] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:23 2022] PUT /identity/v3/projects/b61501771f9d458db280f1b0d2f141c7/users/f73c15c26f5b4dc2a5505b2cd37177bd/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:23.596025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3623b32-34af-46e2-86e3-324e06d097e8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:23.596518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3623b32-34af-46e2-86e3-324e06d097e8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:23.597142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3623b32-34af-46e2-86e3-324e06d097e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:23.634000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c3623b32-34af-46e2-86e3-324e06d097e8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:23.635610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c3623b32-34af-46e2-86e3-324e06d097e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:23.708786 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3702/7491] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:25:23.718859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c92f0-7fc1-4d10-80b8-1b2b4728f258 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:23.720878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-326c92f0-7fc1-4d10-80b8-1b2b4728f258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:23.800587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c92f0-7fc1-4d10-80b8-1b2b4728f258 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:23.805047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285f357-164f-4711-9443-07a1ca8959e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:23.806268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285f357-164f-4711-9443-07a1ca8959e1 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:23.806641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285f357-164f-4711-9443-07a1ca8959e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:23.807487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285f357-164f-4711-9443-07a1ca8959e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:23.819021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285f357-164f-4711-9443-07a1ca8959e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:23.820663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285f357-164f-4711-9443-07a1ca8959e1 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:23.843059 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3790/7492] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:30.011864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3623b32-34af-46e2-86e3-324e06d097e8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:30.013367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c3623b32-34af-46e2-86e3-324e06d097e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:30.027574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3623b32-34af-46e2-86e3-324e06d097e8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:30.033380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:30.034282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:30.034558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:30.034829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] PATH_INFO: `/v3/users/8d92757a1ebe4019829c27b685ece14c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:30.036002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d92757a1ebe4019829c27b685ece14c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:30.037998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:30.058307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:30.102932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d92757a1ebe4019829c27b685ece14c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:30.114038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d92757a1ebe4019829c27b685ece14c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:30.132752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d92757a1ebe4019829c27b685ece14c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:30.133856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d92757a1ebe4019829c27b685ece14c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:30.134448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d92757a1ebe4019829c27b685ece14c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:30.151565 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3703/7493] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:30 2022] DELETE /identity/v3/users/8d92757a1ebe4019829c27b685ece14c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:30.156145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adece687-392f-4b11-af7f-ecfc56d6aaac None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:30.156704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adece687-392f-4b11-af7f-ecfc56d6aaac None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:30.156704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adece687-392f-4b11-af7f-ecfc56d6aaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:30.181586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-adece687-392f-4b11-af7f-ecfc56d6aaac None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:30.183194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-adece687-392f-4b11-af7f-ecfc56d6aaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:30.258071 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3791/7494] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:30.270891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:30.274630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:30.346038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f696bf-5819-4055-b173-4d62b16f0a06 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:30.349803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77714a46-a0a2-421d-980f-5e8b9918bfcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:30.350533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77714a46-a0a2-421d-980f-5e8b9918bfcb service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:30.350913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77714a46-a0a2-421d-980f-5e8b9918bfcb service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:30.351252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77714a46-a0a2-421d-980f-5e8b9918bfcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:30.362195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77714a46-a0a2-421d-980f-5e8b9918bfcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:30.363812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77714a46-a0a2-421d-980f-5e8b9918bfcb service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:30.377005 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3704/7495] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:30.584440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adece687-392f-4b11-af7f-ecfc56d6aaac None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:30.586045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-adece687-392f-4b11-af7f-ecfc56d6aaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:30.602531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adece687-392f-4b11-af7f-ecfc56d6aaac None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:30.609492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057ffce-8751-4650-8ccf-656aa9388407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:30.610541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057ffce-8751-4650-8ccf-656aa9388407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:30.611247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057ffce-8751-4650-8ccf-656aa9388407 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:30.611247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057ffce-8751-4650-8ccf-656aa9388407 admin admin] PATH_INFO: `/v3/projects/b16c88efcdbe4c4cbcf4cb1b00ec5579` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:30.612589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057ffce-8751-4650-8ccf-656aa9388407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16c88efcdbe4c4cbcf4cb1b00ec5579)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:30.614828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057ffce-8751-4650-8ccf-656aa9388407 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:30.714075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f057ffce-8751-4650-8ccf-656aa9388407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16c88efcdbe4c4cbcf4cb1b00ec5579', 'request_id': 'req-f057ffce-8751-4650-8ccf-656aa9388407'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:30.722689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f057ffce-8751-4650-8ccf-656aa9388407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16c88efcdbe4c4cbcf4cb1b00ec5579', 'request_id': 'req-f057ffce-8751-4650-8ccf-656aa9388407'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:30.729668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f057ffce-8751-4650-8ccf-656aa9388407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16c88efcdbe4c4cbcf4cb1b00ec5579', 'request_id': 'req-f057ffce-8751-4650-8ccf-656aa9388407'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:30.747594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:30.753392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f057ffce-8751-4650-8ccf-656aa9388407 admin admin] The token cache is being invalidate because project b16c88efcdbe4c4cbcf4cb1b00ec5579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:30.753911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f057ffce-8751-4650-8ccf-656aa9388407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:30.754987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f057ffce-8751-4650-8ccf-656aa9388407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:30.757282 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3792/7496] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:30 2022] DELETE /identity/v3/projects/b16c88efcdbe4c4cbcf4cb1b00ec5579 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:30.762083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22382eb-37df-49df-ac85-fc5b80e9c37d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:30.762529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22382eb-37df-49df-ac85-fc5b80e9c37d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:30.762845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22382eb-37df-49df-ac85-fc5b80e9c37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:30.785958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b22382eb-37df-49df-ac85-fc5b80e9c37d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:30.787872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b22382eb-37df-49df-ac85-fc5b80e9c37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:30.884759 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3705/7497] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:30.988868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f00b0-3ff8-4ec4-87e3-fd243cc2938a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:30.988868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f00b0-3ff8-4ec4-87e3-fd243cc2938a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:30.989084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f00b0-3ff8-4ec4-87e3-fd243cc2938a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:31.027475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e29f00b0-3ff8-4ec4-87e3-fd243cc2938a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:31.030505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e29f00b0-3ff8-4ec4-87e3-fd243cc2938a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:31.041145 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3793/7498] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:31.046523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22382eb-37df-49df-ac85-fc5b80e9c37d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:31.060738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22382eb-37df-49df-ac85-fc5b80e9c37d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:31.067689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd997d2f-3391-4662-9f3a-7a58144557cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:31.068333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd997d2f-3391-4662-9f3a-7a58144557cc admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:31.068638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd997d2f-3391-4662-9f3a-7a58144557cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:31.068907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd997d2f-3391-4662-9f3a-7a58144557cc admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:31.069567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd997d2f-3391-4662-9f3a-7a58144557cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:25:31.069869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd997d2f-3391-4662-9f3a-7a58144557cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:31.071698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd997d2f-3391-4662-9f3a-7a58144557cc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:31.106286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f00b0-3ff8-4ec4-87e3-fd243cc2938a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:31.112532 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3706/7499] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:25:31.117795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f00b0-3ff8-4ec4-87e3-fd243cc2938a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:31.123216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d477ed-245e-4394-afca-d93a43196010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:31.123997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d477ed-245e-4394-afca-d93a43196010 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:31.124339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d477ed-245e-4394-afca-d93a43196010 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:31.124761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d477ed-245e-4394-afca-d93a43196010 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:31.125661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d477ed-245e-4394-afca-d93a43196010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:31.127809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d477ed-245e-4394-afca-d93a43196010 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:31.166149 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3794/7500] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:31 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:31.170512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd997d2f-3391-4662-9f3a-7a58144557cc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:31.182783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd997d2f-3391-4662-9f3a-7a58144557cc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:31.209854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:31.272469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb0358-c779-4bd0-bddf-1a0b78eeb899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:31.273569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb0358-c779-4bd0-bddf-1a0b78eeb899 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:31.273847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb0358-c779-4bd0-bddf-1a0b78eeb899 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:31.274116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb0358-c779-4bd0-bddf-1a0b78eeb899 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:31.274912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb0358-c779-4bd0-bddf-1a0b78eeb899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:31.276919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb0358-c779-4bd0-bddf-1a0b78eeb899 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:31.306087 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3707/7501] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:31 2022] POST /identity/v3/users => generated 289 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:31.310291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d477ed-245e-4394-afca-d93a43196010 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:31.321868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d477ed-245e-4394-afca-d93a43196010 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:31.328048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064894c4-1ab2-4e29-accc-8432bb71f194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:31.328968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064894c4-1ab2-4e29-accc-8432bb71f194 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:31.329248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064894c4-1ab2-4e29-accc-8432bb71f194 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:31.329518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064894c4-1ab2-4e29-accc-8432bb71f194 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:31.330302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064894c4-1ab2-4e29-accc-8432bb71f194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:31.332012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064894c4-1ab2-4e29-accc-8432bb71f194 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:31.344063 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3795/7502] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:31.349557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb0358-c779-4bd0-bddf-1a0b78eeb899 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:31.360497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb0358-c779-4bd0-bddf-1a0b78eeb899 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:31.365676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1c850-7482-4421-8303-2ecce3696baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:31.366703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1c850-7482-4421-8303-2ecce3696baa admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:31.366962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1c850-7482-4421-8303-2ecce3696baa admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:31.367233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1c850-7482-4421-8303-2ecce3696baa admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:31.367888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1c850-7482-4421-8303-2ecce3696baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:31.369554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1c850-7482-4421-8303-2ecce3696baa admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:31.387036 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3708/7503] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:25:31.392108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064894c4-1ab2-4e29-accc-8432bb71f194 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:31.403891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064894c4-1ab2-4e29-accc-8432bb71f194 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:31.409133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fce48c-306a-4f56-a28d-06d01c2b77c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:31.409832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fce48c-306a-4f56-a28d-06d01c2b77c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:31.410112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fce48c-306a-4f56-a28d-06d01c2b77c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:31.410383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fce48c-306a-4f56-a28d-06d01c2b77c6 admin admin] PATH_INFO: `/v3/projects/2ffb763c88a34fdd9b8f94bb6a69f6c5/users/84ea902e716c4e2087a0a08dc761e359/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:31.427598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fce48c-306a-4f56-a28d-06d01c2b77c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffb763c88a34fdd9b8f94bb6a69f6c5, user_id=84ea902e716c4e2087a0a08dc761e359, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:31.431008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fce48c-306a-4f56-a28d-06d01c2b77c6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:31.447177 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3796/7504] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:31 2022] PUT /identity/v3/projects/2ffb763c88a34fdd9b8f94bb6a69f6c5/users/84ea902e716c4e2087a0a08dc761e359/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:31.458616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a734f-f4f9-49db-811c-a800477163c5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:31.458896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a734f-f4f9-49db-811c-a800477163c5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:31.459163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a734f-f4f9-49db-811c-a800477163c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:31.495633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b45a734f-f4f9-49db-811c-a800477163c5 None None] MFA Rules not processed for user `84ea902e716c4e2087a0a08dc761e359`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:31.496959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b45a734f-f4f9-49db-811c-a800477163c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:31.589704 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3709/7505] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:31.598113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fce48c-306a-4f56-a28d-06d01c2b77c6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:31.600351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-86fce48c-306a-4f56-a28d-06d01c2b77c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:31.686534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fce48c-306a-4f56-a28d-06d01c2b77c6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:31.691890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2303399-b0f9-4dbd-9e89-858b203e9e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:31.693087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2303399-b0f9-4dbd-9e89-858b203e9e0a service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:31.693592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2303399-b0f9-4dbd-9e89-858b203e9e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:31.693976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2303399-b0f9-4dbd-9e89-858b203e9e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:31.710714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2303399-b0f9-4dbd-9e89-858b203e9e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:31.713468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2303399-b0f9-4dbd-9e89-858b203e9e0a service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:31.726480 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3797/7506] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:31.908008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9086136-e2c1-48a4-9f44-9a62704a8d52 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:31.908328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9086136-e2c1-48a4-9f44-9a62704a8d52 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:31.908718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9086136-e2c1-48a4-9f44-9a62704a8d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:31.948872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c9086136-e2c1-48a4-9f44-9a62704a8d52 None None] MFA Rules not processed for user `f73c15c26f5b4dc2a5505b2cd37177bd`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:31.949753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c9086136-e2c1-48a4-9f44-9a62704a8d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:32.051846 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3710/7507] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:32.057155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2303399-b0f9-4dbd-9e89-858b203e9e0a service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:32.058737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a2303399-b0f9-4dbd-9e89-858b203e9e0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:32.143234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2303399-b0f9-4dbd-9e89-858b203e9e0a service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:32.149692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f908fa-34ce-4131-8857-40745f2e8e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:32.151109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f908fa-34ce-4131-8857-40745f2e8e44 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:32.151652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f908fa-34ce-4131-8857-40745f2e8e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:32.152112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f908fa-34ce-4131-8857-40745f2e8e44 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:32.153393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f908fa-34ce-4131-8857-40745f2e8e44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:32.156310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f908fa-34ce-4131-8857-40745f2e8e44 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:32.186238 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3798/7508] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:32 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:32.188276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9086136-e2c1-48a4-9f44-9a62704a8d52 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:32.203345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9086136-e2c1-48a4-9f44-9a62704a8d52 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:32.208919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3245d-2fc7-4ce2-8e1c-8fc74e296e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:32.209096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3245d-2fc7-4ce2-8e1c-8fc74e296e0c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:32.209449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3245d-2fc7-4ce2-8e1c-8fc74e296e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:32.212323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3245d-2fc7-4ce2-8e1c-8fc74e296e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:32.212323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3245d-2fc7-4ce2-8e1c-8fc74e296e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:32.212323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3245d-2fc7-4ce2-8e1c-8fc74e296e0c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:32.225184 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3711/7509] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:25:32.229556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f908fa-34ce-4131-8857-40745f2e8e44 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:32.241414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f908fa-34ce-4131-8857-40745f2e8e44 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:32.247072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cccb3-554e-4ad8-acc2-eaf10cc9b188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:32.247992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cccb3-554e-4ad8-acc2-eaf10cc9b188 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:32.248116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cccb3-554e-4ad8-acc2-eaf10cc9b188 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:32.248331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cccb3-554e-4ad8-acc2-eaf10cc9b188 admin admin] PATH_INFO: `/v3/system/users/38e7417404af4a1fa487fdc14a810aea/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:32.267967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cccb3-554e-4ad8-acc2-eaf10cc9b188 admin admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=38e7417404af4a1fa487fdc14a810aea, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:32.272085 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-3f4cccb3-554e-4ad8-acc2-eaf10cc9b188 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:25:32.272085 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:25:32.272085 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:25:32.272085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cccb3-554e-4ad8-acc2-eaf10cc9b188 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:32.279659 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3799/7510] 10.222.0.23 () {60 vars in 1378 bytes} [Sat Aug 20 21:25:32 2022] PUT /identity/v3/system/users/38e7417404af4a1fa487fdc14a810aea/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:25:32.291709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028f404-512d-4fdf-bc35-47d9d56c5454 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:32.292307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028f404-512d-4fdf-bc35-47d9d56c5454 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:32.292672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028f404-512d-4fdf-bc35-47d9d56c5454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:32.319729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:32.329238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6028f404-512d-4fdf-bc35-47d9d56c5454 None None] MFA Rules not processed for user `38e7417404af4a1fa487fdc14a810aea`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:32.330900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6028f404-512d-4fdf-bc35-47d9d56c5454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:32.423169 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3712/7511] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:32 2022] POST /identity/v3/auth/tokens => generated 2415 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 20 21:25:32.426947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cccb3-554e-4ad8-acc2-eaf10cc9b188 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:32.450361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:32.579898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cccb3-554e-4ad8-acc2-eaf10cc9b188 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:32.628769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c87a7d-a549-4d63-af91-bc0d0b0acbde None tempest-ImageQuotaTest-1116144757-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38e7417404af4a1fa487fdc14a810aea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:32.629660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c87a7d-a549-4d63-af91-bc0d0b0acbde None tempest-ImageQuotaTest-1116144757-system-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:32.630106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c87a7d-a549-4d63-af91-bc0d0b0acbde None tempest-ImageQuotaTest-1116144757-system-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:32.630525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c87a7d-a549-4d63-af91-bc0d0b0acbde None tempest-ImageQuotaTest-1116144757-system-admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:32.631712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c87a7d-a549-4d63-af91-bc0d0b0acbde None tempest-ImageQuotaTest-1116144757-system-admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:32.633423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c87a7d-a549-4d63-af91-bc0d0b0acbde None tempest-ImageQuotaTest-1116144757-system-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:32.642974 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3800/7512] 10.222.0.23 () {58 vars in 1110 bytes} [Sat Aug 20 21:25:32 2022] GET /identity/v3/services => generated 2191 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:32.647540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028f404-512d-4fdf-bc35-47d9d56c5454 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:32.785314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028f404-512d-4fdf-bc35-47d9d56c5454 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:32.835716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:32.845446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f381010-50e0-40d3-a3d5-0ed358970155 None tempest-ImageQuotaTest-1116144757-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38e7417404af4a1fa487fdc14a810aea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:32.846098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f381010-50e0-40d3-a3d5-0ed358970155 None tempest-ImageQuotaTest-1116144757-system-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:32.846369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f381010-50e0-40d3-a3d5-0ed358970155 None tempest-ImageQuotaTest-1116144757-system-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:32.846662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f381010-50e0-40d3-a3d5-0ed358970155 None tempest-ImageQuotaTest-1116144757-system-admin] PATH_INFO: `/v3/limits` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:32.847349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f381010-50e0-40d3-a3d5-0ed358970155 None tempest-ImageQuotaTest-1116144757-system-admin] RBAC: Authorizing `identity:create_limits()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:32.848713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f381010-50e0-40d3-a3d5-0ed358970155 None tempest-ImageQuotaTest-1116144757-system-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-9f381010-50e0-40d3-a3d5-0ed358970155 None tempest-ImageQuotaTest-1116144757-system-admin] Unable to create a limit that has no corresponding registered limit.: keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/limits.py", line 110, in post Aug 20 21:25:32.888327 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application refs = PROVIDERS.unified_limit_api.create_limits(limits) Aug 20 21:25:32.895002 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:25:32.895002 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:25:32.895002 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/core.py", line 116, in create_limits Aug 20 21:25:32.895002 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.create_limits(limits) Aug 20 21:25:32.895002 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 20 21:25:32.895002 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 20 21:25:32.895002 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 260, in create_limits Aug 20 21:25:32.895002 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application target = self._check_and_fill_registered_limit_id(limit) Aug 20 21:25:32.895002 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 249, in _check_and_fill_registered_limit_id Aug 20 21:25:32.895002 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.NoLimitReference Aug 20 21:25:32.895002 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Aug 20 21:25:32.895002 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:25:32.895002 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3713/7513] 10.222.0.23 () {60 vars in 1126 bytes} [Sat Aug 20 21:25:32 2022] POST /identity/v3/limits => generated 124 bytes in 243 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Aug 20 21:25:32.895973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c87a7d-a549-4d63-af91-bc0d0b0acbde None tempest-ImageQuotaTest-1116144757-system-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:33.053661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:33.096433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c87a7d-a549-4d63-af91-bc0d0b0acbde None tempest-ImageQuotaTest-1116144757-system-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:33.149581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c990431-6418-46e8-ae43-bfb20b825979 None tempest-ImageQuotaTest-1116144757-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38e7417404af4a1fa487fdc14a810aea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:33.153058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c990431-6418-46e8-ae43-bfb20b825979 None tempest-ImageQuotaTest-1116144757-system-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:33.157166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c990431-6418-46e8-ae43-bfb20b825979 None tempest-ImageQuotaTest-1116144757-system-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:33.157166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c990431-6418-46e8-ae43-bfb20b825979 None tempest-ImageQuotaTest-1116144757-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:33.158047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c990431-6418-46e8-ae43-bfb20b825979 None tempest-ImageQuotaTest-1116144757-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:33.158983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c990431-6418-46e8-ae43-bfb20b825979 None tempest-ImageQuotaTest-1116144757-system-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:33.172011 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3801/7514] 10.222.0.23 () {58 vars in 1137 bytes} [Sat Aug 20 21:25:32 2022] GET /identity/v3/registered_limits => generated 129 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:25:33.178688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f381010-50e0-40d3-a3d5-0ed358970155 None tempest-ImageQuotaTest-1116144757-system-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:33.373307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f381010-50e0-40d3-a3d5-0ed358970155 None tempest-ImageQuotaTest-1116144757-system-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:33.463756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151233ba-87b1-4b73-92d0-7f55725f324b None tempest-ImageQuotaTest-1116144757-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38e7417404af4a1fa487fdc14a810aea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:33.465043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151233ba-87b1-4b73-92d0-7f55725f324b None tempest-ImageQuotaTest-1116144757-system-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:33.465463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151233ba-87b1-4b73-92d0-7f55725f324b None tempest-ImageQuotaTest-1116144757-system-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:33.465791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151233ba-87b1-4b73-92d0-7f55725f324b None tempest-ImageQuotaTest-1116144757-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:33.466499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151233ba-87b1-4b73-92d0-7f55725f324b None tempest-ImageQuotaTest-1116144757-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:33.469355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151233ba-87b1-4b73-92d0-7f55725f324b None tempest-ImageQuotaTest-1116144757-system-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:33.475988 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3714/7515] 10.222.0.23 () {58 vars in 1137 bytes} [Sat Aug 20 21:25:33 2022] GET /identity/v3/registered_limits => generated 129 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:25:37.687399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c990431-6418-46e8-ae43-bfb20b825979 None tempest-ImageQuotaTest-1116144757-system-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:37.709680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c990431-6418-46e8-ae43-bfb20b825979 None tempest-ImageQuotaTest-1116144757-system-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:37.719480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:37.720532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:37.721001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:37.721491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] PATH_INFO: `/v3/users/84ea902e716c4e2087a0a08dc761e359` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:37.723045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] RBAC: Authorizing `identity:delete_user(user_id=84ea902e716c4e2087a0a08dc761e359)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:37.731655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:37.850950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84ea902e716c4e2087a0a08dc761e359'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:37.867701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84ea902e716c4e2087a0a08dc761e359'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:37.877814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84ea902e716c4e2087a0a08dc761e359'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:37.891641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84ea902e716c4e2087a0a08dc761e359'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:37.892505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84ea902e716c4e2087a0a08dc761e359'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:37.899822 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3802/7516] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:37 2022] DELETE /identity/v3/users/84ea902e716c4e2087a0a08dc761e359 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:37.905043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd59ca-05b6-4a54-bbf3-c9cde76c6b84 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:37.905670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd59ca-05b6-4a54-bbf3-c9cde76c6b84 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:37.906252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd59ca-05b6-4a54-bbf3-c9cde76c6b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:37.926372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-54bd59ca-05b6-4a54-bbf3-c9cde76c6b84 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:37.927578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-54bd59ca-05b6-4a54-bbf3-c9cde76c6b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:37.978428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:37.993366 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3715/7517] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:38.004045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:38.005670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:38.072949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71339f-edef-430a-af8e-64dd3bcd726a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:38.077016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870216bf-3147-47e6-b4c2-1d177a477361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:38.077989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870216bf-3147-47e6-b4c2-1d177a477361 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:38.078175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870216bf-3147-47e6-b4c2-1d177a477361 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:38.079557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870216bf-3147-47e6-b4c2-1d177a477361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:38.090051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870216bf-3147-47e6-b4c2-1d177a477361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:38.091591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870216bf-3147-47e6-b4c2-1d177a477361 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:38.105940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd59ca-05b6-4a54-bbf3-c9cde76c6b84 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:38.107585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-54bd59ca-05b6-4a54-bbf3-c9cde76c6b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:38.111300 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3803/7518] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:38.125815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd59ca-05b6-4a54-bbf3-c9cde76c6b84 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:38.131067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:38.131713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:38.131985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:38.132284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] PATH_INFO: `/v3/users/f73c15c26f5b4dc2a5505b2cd37177bd` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:38.133498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f73c15c26f5b4dc2a5505b2cd37177bd)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:38.135427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:38.189920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f73c15c26f5b4dc2a5505b2cd37177bd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.199584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f73c15c26f5b4dc2a5505b2cd37177bd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.218098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f73c15c26f5b4dc2a5505b2cd37177bd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.218918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f73c15c26f5b4dc2a5505b2cd37177bd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.219546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f73c15c26f5b4dc2a5505b2cd37177bd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.237441 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3716/7519] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:38 2022] DELETE /identity/v3/users/f73c15c26f5b4dc2a5505b2cd37177bd => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:38.353369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870216bf-3147-47e6-b4c2-1d177a477361 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:38.354851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-870216bf-3147-47e6-b4c2-1d177a477361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:38.364493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:38.368512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870216bf-3147-47e6-b4c2-1d177a477361 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:38.438214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:38.438916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:38.439251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:38.439525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e admin admin] PATH_INFO: `/v3/projects/2ffb763c88a34fdd9b8f94bb6a69f6c5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:38.440820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ffb763c88a34fdd9b8f94bb6a69f6c5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:38.443453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:38.450956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:38.452610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:38.468029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e8d2d-cf59-42c0-9c1e-b344a55a73a1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:38.473456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:38.474169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:38.475159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:38.475433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] PATH_INFO: `/v3/projects/b61501771f9d458db280f1b0d2f141c7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:38.477099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b61501771f9d458db280f1b0d2f141c7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:38.479346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:38.499639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ffb763c88a34fdd9b8f94bb6a69f6c5', 'request_id': 'req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.507632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ffb763c88a34fdd9b8f94bb6a69f6c5', 'request_id': 'req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.514022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ffb763c88a34fdd9b8f94bb6a69f6c5', 'request_id': 'req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.525892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e admin admin] The token cache is being invalidate because project 2ffb763c88a34fdd9b8f94bb6a69f6c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:38.526182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.526804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d892ff7c-c3d8-425c-8eb2-a41ccf3e670e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.528339 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3804/7520] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:38 2022] DELETE /identity/v3/projects/2ffb763c88a34fdd9b8f94bb6a69f6c5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:38.529849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b61501771f9d458db280f1b0d2f141c7', 'request_id': 'req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.533180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba271ea-eba5-4297-aa99-00a8bdd31e69 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:38.533515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba271ea-eba5-4297-aa99-00a8bdd31e69 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:38.533840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba271ea-eba5-4297-aa99-00a8bdd31e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:38.536692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b61501771f9d458db280f1b0d2f141c7', 'request_id': 'req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.545042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b61501771f9d458db280f1b0d2f141c7', 'request_id': 'req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.554237 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fba271ea-eba5-4297-aa99-00a8bdd31e69 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:38.555645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fba271ea-eba5-4297-aa99-00a8bdd31e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:38.555799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] The token cache is being invalidate because project b61501771f9d458db280f1b0d2f141c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:38.555937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.556623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.558156 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3717/7521] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:38 2022] DELETE /identity/v3/projects/b61501771f9d458db280f1b0d2f141c7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:25:38.561996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:38.563608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:38.621019 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3805/7522] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:25:38.629143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:38.633059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39051ca6-1274-4571-8c8f-031da938b896 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:38.633303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39051ca6-1274-4571-8c8f-031da938b896 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:38.633552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39051ca6-1274-4571-8c8f-031da938b896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:38.646023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5de1ad-0ca2-4fb7-b35b-e8ef41f5897a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:38.651235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebe8b4-d8a1-4601-b89f-9490e79320de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:38.651876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebe8b4-d8a1-4601-b89f-9490e79320de admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:38.652148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebe8b4-d8a1-4601-b89f-9490e79320de admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:38.652450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebe8b4-d8a1-4601-b89f-9490e79320de admin admin] PATH_INFO: `/v3/users/38e7417404af4a1fa487fdc14a810aea` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:38.653593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ebe8b4-d8a1-4601-b89f-9490e79320de admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e7417404af4a1fa487fdc14a810aea)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:38.654088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-39051ca6-1274-4571-8c8f-031da938b896 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:38.655296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-39051ca6-1274-4571-8c8f-031da938b896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:38.655629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ebe8b4-d8a1-4601-b89f-9490e79320de admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:38.664102 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3806/7523] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:38.668434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39051ca6-1274-4571-8c8f-031da938b896 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:38.678628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39051ca6-1274-4571-8c8f-031da938b896 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:38.683446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58627e5-0f7a-4b55-89ea-0d57bf3eeeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:38.684068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58627e5-0f7a-4b55-89ea-0d57bf3eeeba admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:38.684334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58627e5-0f7a-4b55-89ea-0d57bf3eeeba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:38.684655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58627e5-0f7a-4b55-89ea-0d57bf3eeeba admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:38.685300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58627e5-0f7a-4b55-89ea-0d57bf3eeeba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:25:38.685594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58627e5-0f7a-4b55-89ea-0d57bf3eeeba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:38.687249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58627e5-0f7a-4b55-89ea-0d57bf3eeeba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:38.703298 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3807/7524] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:25:38.707531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58627e5-0f7a-4b55-89ea-0d57bf3eeeba admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:38.718239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58627e5-0f7a-4b55-89ea-0d57bf3eeeba admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:38.723131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f78aa0-7506-4659-b15d-45cdc63e27ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:38.723736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f78aa0-7506-4659-b15d-45cdc63e27ee admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:38.724052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f78aa0-7506-4659-b15d-45cdc63e27ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:38.724269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f78aa0-7506-4659-b15d-45cdc63e27ee admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:38.724941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f78aa0-7506-4659-b15d-45cdc63e27ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:38.726829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f78aa0-7506-4659-b15d-45cdc63e27ee admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:38.734391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d7ebe8b4-d8a1-4601-b89f-9490e79320de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e7417404af4a1fa487fdc14a810aea'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.742920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:38.744425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d7ebe8b4-d8a1-4601-b89f-9490e79320de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e7417404af4a1fa487fdc14a810aea'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.759724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d7ebe8b4-d8a1-4601-b89f-9490e79320de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e7417404af4a1fa487fdc14a810aea'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.760512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d7ebe8b4-d8a1-4601-b89f-9490e79320de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e7417404af4a1fa487fdc14a810aea'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.764044 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3808/7525] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:38 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:25:38.764494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d7ebe8b4-d8a1-4601-b89f-9490e79320de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e7417404af4a1fa487fdc14a810aea'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:38.764944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f78aa0-7506-4659-b15d-45cdc63e27ee admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:38.766874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e4f78aa0-7506-4659-b15d-45cdc63e27ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:38.779267 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3718/7526] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:38 2022] DELETE /identity/v3/users/38e7417404af4a1fa487fdc14a810aea => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:25:38.877877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f78aa0-7506-4659-b15d-45cdc63e27ee admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.020700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b061a1-5eeb-47f4-a0d0-baafe1c3e12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.023273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b061a1-5eeb-47f4-a0d0-baafe1c3e12d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.023565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b061a1-5eeb-47f4-a0d0-baafe1c3e12d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.023897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b061a1-5eeb-47f4-a0d0-baafe1c3e12d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.025103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b061a1-5eeb-47f4-a0d0-baafe1c3e12d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.029577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b061a1-5eeb-47f4-a0d0-baafe1c3e12d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.063640 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3809/7527] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:38 2022] POST /identity/v3/users => generated 289 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:25:39.064930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebe8b4-d8a1-4601-b89f-9490e79320de admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.075781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebe8b4-d8a1-4601-b89f-9490e79320de admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.080754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dede8-9876-4689-8aaf-7140eaa9c23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.081362 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dede8-9876-4689-8aaf-7140eaa9c23d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.081635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dede8-9876-4689-8aaf-7140eaa9c23d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.081900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dede8-9876-4689-8aaf-7140eaa9c23d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.082542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dede8-9876-4689-8aaf-7140eaa9c23d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.084491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dede8-9876-4689-8aaf-7140eaa9c23d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.096418 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3719/7528] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:39.099761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b061a1-5eeb-47f4-a0d0-baafe1c3e12d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.110270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b061a1-5eeb-47f4-a0d0-baafe1c3e12d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.115160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76c489-dc94-427c-9e4e-84cd8bb32d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.115764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76c489-dc94-427c-9e4e-84cd8bb32d05 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.116037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76c489-dc94-427c-9e4e-84cd8bb32d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.116297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76c489-dc94-427c-9e4e-84cd8bb32d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.116965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76c489-dc94-427c-9e4e-84cd8bb32d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.118565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76c489-dc94-427c-9e4e-84cd8bb32d05 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.130193 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3810/7529] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:39.134644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dede8-9876-4689-8aaf-7140eaa9c23d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.145392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dede8-9876-4689-8aaf-7140eaa9c23d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.150777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc366fb-a6d5-4c2f-82d9-bd0c43a89ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.151548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc366fb-a6d5-4c2f-82d9-bd0c43a89ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.151930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc366fb-a6d5-4c2f-82d9-bd0c43a89ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.152397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc366fb-a6d5-4c2f-82d9-bd0c43a89ad3 admin admin] PATH_INFO: `/v3/projects/b760d0406a084d629d1fe443b8367aea/users/3989c1a9738c4c0bb453f2664781791e/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.171228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc366fb-a6d5-4c2f-82d9-bd0c43a89ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b760d0406a084d629d1fe443b8367aea, user_id=3989c1a9738c4c0bb453f2664781791e, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.174283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc366fb-a6d5-4c2f-82d9-bd0c43a89ad3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.189176 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3720/7530] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:39 2022] PUT /identity/v3/projects/b760d0406a084d629d1fe443b8367aea/users/3989c1a9738c4c0bb453f2664781791e/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:39.199363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9f782-8b05-46f4-b0f8-636a976f6945 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.200103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9f782-8b05-46f4-b0f8-636a976f6945 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.200103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9f782-8b05-46f4-b0f8-636a976f6945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.201457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc366fb-a6d5-4c2f-82d9-bd0c43a89ad3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.203369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ecc366fb-a6d5-4c2f-82d9-bd0c43a89ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:39.257184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e1c9f782-8b05-46f4-b0f8-636a976f6945 None None] MFA Rules not processed for user `3989c1a9738c4c0bb453f2664781791e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:39.258257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e1c9f782-8b05-46f4-b0f8-636a976f6945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:39.291977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc366fb-a6d5-4c2f-82d9-bd0c43a89ad3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.299022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00cbec-b042-42f0-b7eb-fead5d1f6622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.299667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00cbec-b042-42f0-b7eb-fead5d1f6622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.300011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00cbec-b042-42f0-b7eb-fead5d1f6622 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.300292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00cbec-b042-42f0-b7eb-fead5d1f6622 admin admin] PATH_INFO: `/v3/projects/None` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.309516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea00cbec-b042-42f0-b7eb-fead5d1f6622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=None)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.311303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea00cbec-b042-42f0-b7eb-fead5d1f6622 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-ea00cbec-b042-42f0-b7eb-fead5d1f6622 admin admin] Could not find project: None.: keystone.exception.ProjectNotFound: Could not find project: None. Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:25:39.323142 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:25:39.324618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Aug 20 21:25:39.324618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Aug 20 21:25:39.324618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:25:39.324618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:25:39.324618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:25:39.324618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:25:39.324618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: None. Aug 20 21:25:39.324618 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:25:39.324618 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3721/7531] 10.222.0.23 () {58 vars in 1149 bytes} [Sat Aug 20 21:25:39 2022] DELETE /identity/v3/projects/None => generated 85 bytes in 123 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 20 21:25:39.331294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfb3a1-783b-4122-835f-38e12101d61f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.331849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfb3a1-783b-4122-835f-38e12101d61f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.332206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfb3a1-783b-4122-835f-38e12101d61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.343187 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3811/7532] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:39.356960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d9cfb3a1-783b-4122-835f-38e12101d61f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:39.358309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d9cfb3a1-783b-4122-835f-38e12101d61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:39.362632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9f782-8b05-46f4-b0f8-636a976f6945 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.364011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e1c9f782-8b05-46f4-b0f8-636a976f6945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:39.367765 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3722/7533] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:25:39.376088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3467f-b0b2-4c15-b131-aa531d2552b8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.376386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3467f-b0b2-4c15-b131-aa531d2552b8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.376683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3467f-b0b2-4c15-b131-aa531d2552b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.379755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:39.397793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-16d3467f-b0b2-4c15-b131-aa531d2552b8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:39.398998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-16d3467f-b0b2-4c15-b131-aa531d2552b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:39.409045 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3723/7534] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:25:39.414010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3467f-b0b2-4c15-b131-aa531d2552b8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.419788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:39.425183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3467f-b0b2-4c15-b131-aa531d2552b8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.431719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ee929-8ab7-4840-8fc2-7790385f0405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.432964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ee929-8ab7-4840-8fc2-7790385f0405 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.433638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ee929-8ab7-4840-8fc2-7790385f0405 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.433921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ee929-8ab7-4840-8fc2-7790385f0405 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.436309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ee929-8ab7-4840-8fc2-7790385f0405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:25:39.436309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ee929-8ab7-4840-8fc2-7790385f0405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.437820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ee929-8ab7-4840-8fc2-7790385f0405 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.441948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9f782-8b05-46f4-b0f8-636a976f6945 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.446140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba253f6d-ef95-4f83-baf6-71e00532d4f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.446879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba253f6d-ef95-4f83-baf6-71e00532d4f1 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.447213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba253f6d-ef95-4f83-baf6-71e00532d4f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.447540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba253f6d-ef95-4f83-baf6-71e00532d4f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.452406 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3724/7535] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:25:39.456129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ee929-8ab7-4840-8fc2-7790385f0405 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.459477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba253f6d-ef95-4f83-baf6-71e00532d4f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.461381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba253f6d-ef95-4f83-baf6-71e00532d4f1 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.466199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ee929-8ab7-4840-8fc2-7790385f0405 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.471441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35740ac0-11bc-4cc1-9877-3c537053b6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.472046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35740ac0-11bc-4cc1-9877-3c537053b6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.472318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35740ac0-11bc-4cc1-9877-3c537053b6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.472686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35740ac0-11bc-4cc1-9877-3c537053b6d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.473309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35740ac0-11bc-4cc1-9877-3c537053b6d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.475300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35740ac0-11bc-4cc1-9877-3c537053b6d2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.476566 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3812/7536] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:25:39.489456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba253f6d-ef95-4f83-baf6-71e00532d4f1 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.501488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba253f6d-ef95-4f83-baf6-71e00532d4f1 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.506701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf88807-c51f-4418-b5ff-15ed95f6442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.507211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf88807-c51f-4418-b5ff-15ed95f6442b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.507477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf88807-c51f-4418-b5ff-15ed95f6442b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.507740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf88807-c51f-4418-b5ff-15ed95f6442b admin admin] PATH_INFO: `/v3/users/3989c1a9738c4c0bb453f2664781791e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.508957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf88807-c51f-4418-b5ff-15ed95f6442b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3989c1a9738c4c0bb453f2664781791e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.510897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf88807-c51f-4418-b5ff-15ed95f6442b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.514021 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3725/7537] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:39 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:39.519718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35740ac0-11bc-4cc1-9877-3c537053b6d2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.565788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4bf88807-c51f-4418-b5ff-15ed95f6442b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3989c1a9738c4c0bb453f2664781791e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:39.577745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4bf88807-c51f-4418-b5ff-15ed95f6442b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3989c1a9738c4c0bb453f2664781791e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:39.588497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4bf88807-c51f-4418-b5ff-15ed95f6442b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3989c1a9738c4c0bb453f2664781791e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:39.606989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35740ac0-11bc-4cc1-9877-3c537053b6d2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.609293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4bf88807-c51f-4418-b5ff-15ed95f6442b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3989c1a9738c4c0bb453f2664781791e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:39.610098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4bf88807-c51f-4418-b5ff-15ed95f6442b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3989c1a9738c4c0bb453f2664781791e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:39.614307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85335f4-a0a6-40ad-889d-0de0414d839a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.615118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85335f4-a0a6-40ad-889d-0de0414d839a admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.615287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85335f4-a0a6-40ad-889d-0de0414d839a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.615709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85335f4-a0a6-40ad-889d-0de0414d839a admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.616464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85335f4-a0a6-40ad-889d-0de0414d839a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.618472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85335f4-a0a6-40ad-889d-0de0414d839a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.619286 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3813/7538] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:39 2022] DELETE /identity/v3/users/3989c1a9738c4c0bb453f2664781791e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:39.623956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d54f6-7809-44bb-bc0c-3962bf03a8c3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.624347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d54f6-7809-44bb-bc0c-3962bf03a8c3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.625336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d54f6-7809-44bb-bc0c-3962bf03a8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.648048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6a3d54f6-7809-44bb-bc0c-3962bf03a8c3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:39.649462 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3726/7539] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:39 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:39.649635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6a3d54f6-7809-44bb-bc0c-3962bf03a8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:39.653457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85335f4-a0a6-40ad-889d-0de0414d839a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.655175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f85335f4-a0a6-40ad-889d-0de0414d839a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:39.671189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85335f4-a0a6-40ad-889d-0de0414d839a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.724263 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3814/7540] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:25:39.735368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d54f6-7809-44bb-bc0c-3962bf03a8c3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.737148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6a3d54f6-7809-44bb-bc0c-3962bf03a8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:39.741839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b7f67-b1c3-4dad-81d1-c802a7369947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.742531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b7f67-b1c3-4dad-81d1-c802a7369947 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.742893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b7f67-b1c3-4dad-81d1-c802a7369947 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.743215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b7f67-b1c3-4dad-81d1-c802a7369947 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.743937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b7f67-b1c3-4dad-81d1-c802a7369947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.745880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b7f67-b1c3-4dad-81d1-c802a7369947 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.757058 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3727/7541] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:39.761357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b7f67-b1c3-4dad-81d1-c802a7369947 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.772939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b7f67-b1c3-4dad-81d1-c802a7369947 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.777841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bde98f-dbf0-4c70-9f51-669a9acfaf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.778440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bde98f-dbf0-4c70-9f51-669a9acfaf7c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.778719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bde98f-dbf0-4c70-9f51-669a9acfaf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.778983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bde98f-dbf0-4c70-9f51-669a9acfaf7c admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.779610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bde98f-dbf0-4c70-9f51-669a9acfaf7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.781676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bde98f-dbf0-4c70-9f51-669a9acfaf7c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.783369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:39.793207 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3728/7542] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:39.798555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bde98f-dbf0-4c70-9f51-669a9acfaf7c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.803001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d54f6-7809-44bb-bc0c-3962bf03a8c3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.806649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe711b-b700-461b-921d-2841ef720294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.807250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe711b-b700-461b-921d-2841ef720294 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.807519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe711b-b700-461b-921d-2841ef720294 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.807782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe711b-b700-461b-921d-2841ef720294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.808521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bde98f-dbf0-4c70-9f51-669a9acfaf7c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:39.813583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c80e6-2a72-413c-8a3c-986da62062de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:39.814488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c80e6-2a72-413c-8a3c-986da62062de admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.814760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c80e6-2a72-413c-8a3c-986da62062de admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.815074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c80e6-2a72-413c-8a3c-986da62062de admin admin] PATH_INFO: `/v3/projects/3181d5ac7c3743019399a71b6a4c7b62/users/4478c47d7f02468e84729c5fbfca0160/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.817850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe711b-b700-461b-921d-2841ef720294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.819228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe711b-b700-461b-921d-2841ef720294 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.829049 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3815/7543] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:25:39.831212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c80e6-2a72-413c-8a3c-986da62062de admin admin] RBAC: Authorizing `identity:create_grant(project_id=3181d5ac7c3743019399a71b6a4c7b62, user_id=4478c47d7f02468e84729c5fbfca0160, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:39.834051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c80e6-2a72-413c-8a3c-986da62062de admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:39.848545 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3729/7544] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:39 2022] PUT /identity/v3/projects/3181d5ac7c3743019399a71b6a4c7b62/users/4478c47d7f02468e84729c5fbfca0160/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:25:39.853747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4b61-0271-4d1e-ad20-66a524f56355 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:39.854026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4b61-0271-4d1e-ad20-66a524f56355 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:39.854310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4b61-0271-4d1e-ad20-66a524f56355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:39.873801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8aee4b61-0271-4d1e-ad20-66a524f56355 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:39.875113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8aee4b61-0271-4d1e-ad20-66a524f56355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:39.940431 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3816/7545] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:25:39.950778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c80e6-2a72-413c-8a3c-986da62062de admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:39.952676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-374c80e6-2a72-413c-8a3c-986da62062de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:39.981485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:40.034381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c80e6-2a72-413c-8a3c-986da62062de admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:40.038182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13433859-dda8-4fe3-90a5-1799a000587f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:40.038694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13433859-dda8-4fe3-90a5-1799a000587f service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:40.038962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13433859-dda8-4fe3-90a5-1799a000587f service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:40.039223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13433859-dda8-4fe3-90a5-1799a000587f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:40.049392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13433859-dda8-4fe3-90a5-1799a000587f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:40.050843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13433859-dda8-4fe3-90a5-1799a000587f service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:40.061384 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3730/7546] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:40.193061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee4b61-0271-4d1e-ad20-66a524f56355 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:40.194705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8aee4b61-0271-4d1e-ad20-66a524f56355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:40.207513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee4b61-0271-4d1e-ad20-66a524f56355 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:40.213628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:40.214322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:40.214612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:40.215004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6 admin admin] PATH_INFO: `/v3/projects/b760d0406a084d629d1fe443b8367aea` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:40.216524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b760d0406a084d629d1fe443b8367aea)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:40.218535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:40.273303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b760d0406a084d629d1fe443b8367aea', 'request_id': 'req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:40.282481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b760d0406a084d629d1fe443b8367aea', 'request_id': 'req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:40.288681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b760d0406a084d629d1fe443b8367aea', 'request_id': 'req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:40.301469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6 admin admin] The token cache is being invalidate because project b760d0406a084d629d1fe443b8367aea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:40.301760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:40.302421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d1d76365-6b8c-4d0d-af85-8f38c6fc33d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:40.304067 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3817/7547] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:40 2022] DELETE /identity/v3/projects/b760d0406a084d629d1fe443b8367aea => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:40.309165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e95a5-e307-49d7-b96d-2b7b281718f5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:40.309676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e95a5-e307-49d7-b96d-2b7b281718f5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:40.309783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e95a5-e307-49d7-b96d-2b7b281718f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:40.334863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-279e95a5-e307-49d7-b96d-2b7b281718f5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:40.336209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-279e95a5-e307-49d7-b96d-2b7b281718f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:40.400442 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3731/7548] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:40.408265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb3b7f-a61e-4573-97cf-125fb2171b51 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:40.408657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb3b7f-a61e-4573-97cf-125fb2171b51 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:40.408993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb3b7f-a61e-4573-97cf-125fb2171b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:40.432733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-acdb3b7f-a61e-4573-97cf-125fb2171b51 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:40.434069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-acdb3b7f-a61e-4573-97cf-125fb2171b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:40.445140 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3818/7549] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:40.449033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e95a5-e307-49d7-b96d-2b7b281718f5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:40.459611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e95a5-e307-49d7-b96d-2b7b281718f5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:40.464971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043f752-5bee-4bcf-a614-568eb0591a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:40.466005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043f752-5bee-4bcf-a614-568eb0591a50 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:40.466005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043f752-5bee-4bcf-a614-568eb0591a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:40.466322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043f752-5bee-4bcf-a614-568eb0591a50 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:40.466889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043f752-5bee-4bcf-a614-568eb0591a50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:25:40.467226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043f752-5bee-4bcf-a614-568eb0591a50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:40.469226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043f752-5bee-4bcf-a614-568eb0591a50 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:40.484060 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3732/7550] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:25:40.489824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb3b7f-a61e-4573-97cf-125fb2171b51 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:40.501743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb3b7f-a61e-4573-97cf-125fb2171b51 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:40.506464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff4485-501d-42bc-b25b-5b3f70c70286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:40.507086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff4485-501d-42bc-b25b-5b3f70c70286 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:40.507358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff4485-501d-42bc-b25b-5b3f70c70286 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:40.507624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff4485-501d-42bc-b25b-5b3f70c70286 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:40.508250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff4485-501d-42bc-b25b-5b3f70c70286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:40.510485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff4485-501d-42bc-b25b-5b3f70c70286 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:40.540982 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3819/7551] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:40 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:40.547868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043f752-5bee-4bcf-a614-568eb0591a50 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:40.560776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043f752-5bee-4bcf-a614-568eb0591a50 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:40.622034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbd355-c174-49f7-b6dd-8748dd4ed87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:40.622554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbd355-c174-49f7-b6dd-8748dd4ed87e admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:40.622826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbd355-c174-49f7-b6dd-8748dd4ed87e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:40.623091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbd355-c174-49f7-b6dd-8748dd4ed87e admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:40.623676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbd355-c174-49f7-b6dd-8748dd4ed87e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:40.625884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbd355-c174-49f7-b6dd-8748dd4ed87e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:40.648397 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3733/7552] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:40 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:40.652539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff4485-501d-42bc-b25b-5b3f70c70286 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:40.663344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff4485-501d-42bc-b25b-5b3f70c70286 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:40.668136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714b084-9338-455f-8152-a65982018bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:40.668800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714b084-9338-455f-8152-a65982018bdd admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:40.669070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714b084-9338-455f-8152-a65982018bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:40.669331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714b084-9338-455f-8152-a65982018bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:40.669961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714b084-9338-455f-8152-a65982018bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:40.671921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714b084-9338-455f-8152-a65982018bdd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:40.682837 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3820/7553] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:25:40.686918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbd355-c174-49f7-b6dd-8748dd4ed87e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:40.698775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbd355-c174-49f7-b6dd-8748dd4ed87e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:40.703962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eac763-6dfe-424d-8f87-0ea7efc993da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:40.704602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eac763-6dfe-424d-8f87-0ea7efc993da admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:40.704880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eac763-6dfe-424d-8f87-0ea7efc993da admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:40.705150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eac763-6dfe-424d-8f87-0ea7efc993da admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:40.705791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eac763-6dfe-424d-8f87-0ea7efc993da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:40.707410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eac763-6dfe-424d-8f87-0ea7efc993da admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:40.720417 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3734/7554] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:40.723376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714b084-9338-455f-8152-a65982018bdd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:40.735725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714b084-9338-455f-8152-a65982018bdd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:40.740547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fd26c-164d-4641-96f9-9294fd4f1da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:40.741451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fd26c-164d-4641-96f9-9294fd4f1da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:40.741451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fd26c-164d-4641-96f9-9294fd4f1da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:40.741731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fd26c-164d-4641-96f9-9294fd4f1da9 admin admin] PATH_INFO: `/v3/projects/b9d8d8a0277c456e94566c8cf5eeabbf/users/d7273b732dce4e14ba38201faf89f2f6/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:40.750104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:40.759162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fd26c-164d-4641-96f9-9294fd4f1da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d8d8a0277c456e94566c8cf5eeabbf, user_id=d7273b732dce4e14ba38201faf89f2f6, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:40.762124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fd26c-164d-4641-96f9-9294fd4f1da9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:40.781818 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3821/7555] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:40 2022] PUT /identity/v3/projects/b9d8d8a0277c456e94566c8cf5eeabbf/users/d7273b732dce4e14ba38201faf89f2f6/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:40.795314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2c33a-03b4-4f96-a262-648cafff8229 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:40.795314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2c33a-03b4-4f96-a262-648cafff8229 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:40.795314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2c33a-03b4-4f96-a262-648cafff8229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:40.833527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e4f2c33a-03b4-4f96-a262-648cafff8229 None None] MFA Rules not processed for user `d7273b732dce4e14ba38201faf89f2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:40.834763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e4f2c33a-03b4-4f96-a262-648cafff8229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:40.853580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:40.930040 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3735/7556] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:40 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:40.934328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fd26c-164d-4641-96f9-9294fd4f1da9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:41.010417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fd26c-164d-4641-96f9-9294fd4f1da9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:41.016032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e51fd-1145-4b20-8055-4943a9a4cfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:41.017340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e51fd-1145-4b20-8055-4943a9a4cfad admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:41.017340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e51fd-1145-4b20-8055-4943a9a4cfad admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:41.017340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e51fd-1145-4b20-8055-4943a9a4cfad admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:41.018894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e51fd-1145-4b20-8055-4943a9a4cfad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:41.020078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e51fd-1145-4b20-8055-4943a9a4cfad admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:41.055209 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3822/7557] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:40 2022] POST /identity/v3/projects => generated 405 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:41.059123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2c33a-03b4-4f96-a262-648cafff8229 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:41.072520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2c33a-03b4-4f96-a262-648cafff8229 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:41.145196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f79a29-0f78-4df6-a00c-0d6c9765dee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:41.145823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f79a29-0f78-4df6-a00c-0d6c9765dee9 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:41.146162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f79a29-0f78-4df6-a00c-0d6c9765dee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:41.146494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f79a29-0f78-4df6-a00c-0d6c9765dee9 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:41.147490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f79a29-0f78-4df6-a00c-0d6c9765dee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:41.152849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f79a29-0f78-4df6-a00c-0d6c9765dee9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:41.180446 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3736/7558] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:41 2022] POST /identity/v3/users => generated 309 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:41.183983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e51fd-1145-4b20-8055-4943a9a4cfad admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:41.200198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e51fd-1145-4b20-8055-4943a9a4cfad admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:41.206188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0a5ed-e890-44b5-8324-f8c4a563a67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:41.206994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0a5ed-e890-44b5-8324-f8c4a563a67e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:41.207269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0a5ed-e890-44b5-8324-f8c4a563a67e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:41.207532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0a5ed-e890-44b5-8324-f8c4a563a67e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:41.208170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0a5ed-e890-44b5-8324-f8c4a563a67e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:41.210103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0a5ed-e890-44b5-8324-f8c4a563a67e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:41.238151 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3823/7559] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:41.239468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f79a29-0f78-4df6-a00c-0d6c9765dee9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:41.250778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f79a29-0f78-4df6-a00c-0d6c9765dee9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:41.257238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f6872-3ad4-4376-9a0d-76199e090f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:41.257921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6872-3ad4-4376-9a0d-76199e090f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:41.258196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6872-3ad4-4376-9a0d-76199e090f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:41.258462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6872-3ad4-4376-9a0d-76199e090f70 admin admin] PATH_INFO: `/v3/projects/81b34d5dd79a4bc9abbe6b4f81464538/users/eab561b3fcef4dd8b944a9bd4bcce5a4/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:41.276322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f6872-3ad4-4376-9a0d-76199e090f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b34d5dd79a4bc9abbe6b4f81464538, user_id=eab561b3fcef4dd8b944a9bd4bcce5a4, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:41.279530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f6872-3ad4-4376-9a0d-76199e090f70 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:41.299441 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3737/7560] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:41 2022] PUT /identity/v3/projects/81b34d5dd79a4bc9abbe6b4f81464538/users/eab561b3fcef4dd8b944a9bd4bcce5a4/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:41.304137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0a5ed-e890-44b5-8324-f8c4a563a67e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:41.315436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0a5ed-e890-44b5-8324-f8c4a563a67e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:41.398208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ee9a3-282c-4904-a06d-89d6d9c91b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:41.399000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ee9a3-282c-4904-a06d-89d6d9c91b3f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:41.399373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ee9a3-282c-4904-a06d-89d6d9c91b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:41.399741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ee9a3-282c-4904-a06d-89d6d9c91b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:41.400678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ee9a3-282c-4904-a06d-89d6d9c91b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:41.402705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ee9a3-282c-4904-a06d-89d6d9c91b3f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:41.417383 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3824/7561] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:25:41.421622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f6872-3ad4-4376-9a0d-76199e090f70 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:41.433893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f6872-3ad4-4376-9a0d-76199e090f70 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:41.439257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02130e54-2c20-43c7-a2d1-6254c1d5cac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:41.440115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02130e54-2c20-43c7-a2d1-6254c1d5cac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:41.440441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02130e54-2c20-43c7-a2d1-6254c1d5cac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:41.440723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02130e54-2c20-43c7-a2d1-6254c1d5cac2 admin admin] PATH_INFO: `/v3/projects/81b34d5dd79a4bc9abbe6b4f81464538/users/eab561b3fcef4dd8b944a9bd4bcce5a4/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:41.443507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02130e54-2c20-43c7-a2d1-6254c1d5cac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b34d5dd79a4bc9abbe6b4f81464538, user_id=eab561b3fcef4dd8b944a9bd4bcce5a4, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:41.446325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02130e54-2c20-43c7-a2d1-6254c1d5cac2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:41.466812 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3738/7562] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:41 2022] PUT /identity/v3/projects/81b34d5dd79a4bc9abbe6b4f81464538/users/eab561b3fcef4dd8b944a9bd4bcce5a4/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:25:41.469497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ad6fa-804d-4901-a817-4da3c9dba32b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:41.469810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ad6fa-804d-4901-a817-4da3c9dba32b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:41.470081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ad6fa-804d-4901-a817-4da3c9dba32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:41.507659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-972ad6fa-804d-4901-a817-4da3c9dba32b None None] MFA Rules not processed for user `eab561b3fcef4dd8b944a9bd4bcce5a4`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:41.509602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-972ad6fa-804d-4901-a817-4da3c9dba32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:41.569323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:41.606155 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3825/7563] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:41 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:41.618516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02130e54-2c20-43c7-a2d1-6254c1d5cac2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:41.620969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-02130e54-2c20-43c7-a2d1-6254c1d5cac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:41.693221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02130e54-2c20-43c7-a2d1-6254c1d5cac2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:41.697232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869d3cf-3e6b-438f-84cb-638f13c5937a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:41.697925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869d3cf-3e6b-438f-84cb-638f13c5937a service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:41.698255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869d3cf-3e6b-438f-84cb-638f13c5937a service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:41.698605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869d3cf-3e6b-438f-84cb-638f13c5937a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:41.711606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869d3cf-3e6b-438f-84cb-638f13c5937a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:41.713968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869d3cf-3e6b-438f-84cb-638f13c5937a service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:41.733918 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3739/7564] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:41 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:41.823686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ad6fa-804d-4901-a817-4da3c9dba32b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:41.893559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ad6fa-804d-4901-a817-4da3c9dba32b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:41.901210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:41.902369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:41.902369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:41.902573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] PATH_INFO: `/v3/users/d7273b732dce4e14ba38201faf89f2f6` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:41.904114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7273b732dce4e14ba38201faf89f2f6)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:41.906618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:42.051063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7273b732dce4e14ba38201faf89f2f6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:42.092993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7273b732dce4e14ba38201faf89f2f6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:42.114316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7273b732dce4e14ba38201faf89f2f6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:42.163735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7273b732dce4e14ba38201faf89f2f6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:42.164734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7273b732dce4e14ba38201faf89f2f6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:42.181265 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3826/7565] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:41 2022] DELETE /identity/v3/users/d7273b732dce4e14ba38201faf89f2f6 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:42.183561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238170c5-3c7a-43d5-a315-9f6ee8a9ddb3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:42.183750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238170c5-3c7a-43d5-a315-9f6ee8a9ddb3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:42.184026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238170c5-3c7a-43d5-a315-9f6ee8a9ddb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:42.206975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-238170c5-3c7a-43d5-a315-9f6ee8a9ddb3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:42.208231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-238170c5-3c7a-43d5-a315-9f6ee8a9ddb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:42.221785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:42.334866 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3740/7566] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:42.350001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:42.351506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:42.435177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e776a9-ff69-4bc6-abf2-3b0b7e970630 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:42.440505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d5fad-2f03-4c78-a7c1-f98aec94a927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:42.441239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d5fad-2f03-4c78-a7c1-f98aec94a927 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:42.441515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d5fad-2f03-4c78-a7c1-f98aec94a927 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:42.441844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d5fad-2f03-4c78-a7c1-f98aec94a927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:42.453282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d5fad-2f03-4c78-a7c1-f98aec94a927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:42.455243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d5fad-2f03-4c78-a7c1-f98aec94a927 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:42.467908 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3827/7567] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:42.794502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238170c5-3c7a-43d5-a315-9f6ee8a9ddb3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:42.796406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-238170c5-3c7a-43d5-a315-9f6ee8a9ddb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:42.809823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238170c5-3c7a-43d5-a315-9f6ee8a9ddb3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:42.816754 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472aa5a-fcde-4f37-b097-878dfb850ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:42.818246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472aa5a-fcde-4f37-b097-878dfb850ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:42.818528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472aa5a-fcde-4f37-b097-878dfb850ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:42.819106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472aa5a-fcde-4f37-b097-878dfb850ed2 admin admin] PATH_INFO: `/v3/projects/b9d8d8a0277c456e94566c8cf5eeabbf` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:42.820627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472aa5a-fcde-4f37-b097-878dfb850ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9d8d8a0277c456e94566c8cf5eeabbf)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:42.823408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472aa5a-fcde-4f37-b097-878dfb850ed2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:42.891965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3472aa5a-fcde-4f37-b097-878dfb850ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9d8d8a0277c456e94566c8cf5eeabbf', 'request_id': 'req-3472aa5a-fcde-4f37-b097-878dfb850ed2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:42.899333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3472aa5a-fcde-4f37-b097-878dfb850ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9d8d8a0277c456e94566c8cf5eeabbf', 'request_id': 'req-3472aa5a-fcde-4f37-b097-878dfb850ed2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:42.911608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3472aa5a-fcde-4f37-b097-878dfb850ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9d8d8a0277c456e94566c8cf5eeabbf', 'request_id': 'req-3472aa5a-fcde-4f37-b097-878dfb850ed2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:42.927777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3472aa5a-fcde-4f37-b097-878dfb850ed2 admin admin] The token cache is being invalidate because project b9d8d8a0277c456e94566c8cf5eeabbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:42.928002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3472aa5a-fcde-4f37-b097-878dfb850ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:42.929236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3472aa5a-fcde-4f37-b097-878dfb850ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:42.930929 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3741/7568] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:42 2022] DELETE /identity/v3/projects/b9d8d8a0277c456e94566c8cf5eeabbf => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:42.935153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d5fad-2f03-4c78-a7c1-f98aec94a927 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:42.941508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:42.947526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d5fad-2f03-4c78-a7c1-f98aec94a927 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:43.017721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:43.018380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:43.018645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:43.018914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] PATH_INFO: `/v3/users/eab561b3fcef4dd8b944a9bd4bcce5a4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:43.021509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eab561b3fcef4dd8b944a9bd4bcce5a4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:43.023463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:43.086219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eab561b3fcef4dd8b944a9bd4bcce5a4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:43.099663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eab561b3fcef4dd8b944a9bd4bcce5a4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:43.112997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eab561b3fcef4dd8b944a9bd4bcce5a4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:43.128418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eab561b3fcef4dd8b944a9bd4bcce5a4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:43.129486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eab561b3fcef4dd8b944a9bd4bcce5a4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:43.136146 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3828/7569] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:42 2022] DELETE /identity/v3/users/eab561b3fcef4dd8b944a9bd4bcce5a4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:43.377254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58689899-6c45-4cfa-8609-fc30b0a94910 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:43.377531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58689899-6c45-4cfa-8609-fc30b0a94910 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:43.377797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58689899-6c45-4cfa-8609-fc30b0a94910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:43.415959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-58689899-6c45-4cfa-8609-fc30b0a94910 None None] MFA Rules not processed for user `4e6a5f35d172410481a8416b7bdaafac`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:43.418225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-58689899-6c45-4cfa-8609-fc30b0a94910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:43.563686 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3742/7570] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:43.578812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:43.580328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:43.761498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23202322-c103-4b42-9c7c-7e0c8fe74e7b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:43.769780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1778f9-ac86-4924-8b37-60f887462659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:43.771115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1778f9-ac86-4924-8b37-60f887462659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:43.771748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1778f9-ac86-4924-8b37-60f887462659 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:43.772239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1778f9-ac86-4924-8b37-60f887462659 admin admin] PATH_INFO: `/v3/projects/81b34d5dd79a4bc9abbe6b4f81464538` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:43.774635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1778f9-ac86-4924-8b37-60f887462659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81b34d5dd79a4bc9abbe6b4f81464538)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:43.777136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1778f9-ac86-4924-8b37-60f887462659 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:43.823329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:43.852433 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d1778f9-ac86-4924-8b37-60f887462659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81b34d5dd79a4bc9abbe6b4f81464538', 'request_id': 'req-6d1778f9-ac86-4924-8b37-60f887462659'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:43.861353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d1778f9-ac86-4924-8b37-60f887462659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81b34d5dd79a4bc9abbe6b4f81464538', 'request_id': 'req-6d1778f9-ac86-4924-8b37-60f887462659'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:43.868232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d1778f9-ac86-4924-8b37-60f887462659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81b34d5dd79a4bc9abbe6b4f81464538', 'request_id': 'req-6d1778f9-ac86-4924-8b37-60f887462659'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:43.882380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d1778f9-ac86-4924-8b37-60f887462659 admin admin] The token cache is being invalidate because project 81b34d5dd79a4bc9abbe6b4f81464538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:43.882723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d1778f9-ac86-4924-8b37-60f887462659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:43.883341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d1778f9-ac86-4924-8b37-60f887462659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:43.889390 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3829/7571] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:43 2022] DELETE /identity/v3/projects/81b34d5dd79a4bc9abbe6b4f81464538 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:43.890388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767afd9-d021-4ff6-aca7-ca5136615caf None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:43.890854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767afd9-d021-4ff6-aca7-ca5136615caf None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:43.891060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767afd9-d021-4ff6-aca7-ca5136615caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:43.919667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0767afd9-d021-4ff6-aca7-ca5136615caf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:43.920967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0767afd9-d021-4ff6-aca7-ca5136615caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:43.967255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:43.996754 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3743/7572] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:25:44.006610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a95d4-a2e3-430b-9d4e-be6e5ff9bb03 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:44.007144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a95d4-a2e3-430b-9d4e-be6e5ff9bb03 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:44.007394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a95d4-a2e3-430b-9d4e-be6e5ff9bb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:44.028553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-977a95d4-a2e3-430b-9d4e-be6e5ff9bb03 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:44.029890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-977a95d4-a2e3-430b-9d4e-be6e5ff9bb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:44.043716 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3830/7573] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:44.045496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767afd9-d021-4ff6-aca7-ca5136615caf None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:44.058103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767afd9-d021-4ff6-aca7-ca5136615caf None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:44.062997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071691f-1f20-4a56-a190-7f5d2cf3c890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:44.063682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071691f-1f20-4a56-a190-7f5d2cf3c890 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:44.064029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071691f-1f20-4a56-a190-7f5d2cf3c890 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:44.064411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071691f-1f20-4a56-a190-7f5d2cf3c890 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:44.065137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071691f-1f20-4a56-a190-7f5d2cf3c890 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:25:44.065501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071691f-1f20-4a56-a190-7f5d2cf3c890 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:44.067196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071691f-1f20-4a56-a190-7f5d2cf3c890 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:44.081726 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3744/7574] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:25:44.085026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a95d4-a2e3-430b-9d4e-be6e5ff9bb03 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:44.095040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a95d4-a2e3-430b-9d4e-be6e5ff9bb03 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:44.101360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d585b78-23ca-4663-ade7-c1e8f621ab63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:44.101988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d585b78-23ca-4663-ade7-c1e8f621ab63 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:44.102258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d585b78-23ca-4663-ade7-c1e8f621ab63 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:44.102525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d585b78-23ca-4663-ade7-c1e8f621ab63 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:44.103170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d585b78-23ca-4663-ade7-c1e8f621ab63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:44.105802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d585b78-23ca-4663-ade7-c1e8f621ab63 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:44.139974 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3831/7575] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:44 2022] POST /identity/v3/projects => generated 357 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:44.144360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071691f-1f20-4a56-a190-7f5d2cf3c890 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:44.156292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071691f-1f20-4a56-a190-7f5d2cf3c890 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:44.245578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7696722-0073-499f-aebd-42cc7714e89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:44.246679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7696722-0073-499f-aebd-42cc7714e89e admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:44.246679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7696722-0073-499f-aebd-42cc7714e89e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:44.247074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7696722-0073-499f-aebd-42cc7714e89e admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:44.247890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7696722-0073-499f-aebd-42cc7714e89e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:44.250275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7696722-0073-499f-aebd-42cc7714e89e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:44.276493 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3745/7576] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:44 2022] POST /identity/v3/users => generated 279 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:44.278600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d585b78-23ca-4663-ade7-c1e8f621ab63 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:44.290907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d585b78-23ca-4663-ade7-c1e8f621ab63 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:44.297521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb8319-90d2-4640-aead-4e3420237573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:44.298406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb8319-90d2-4640-aead-4e3420237573 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:44.298660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb8319-90d2-4640-aead-4e3420237573 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:44.299085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb8319-90d2-4640-aead-4e3420237573 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:44.299879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb8319-90d2-4640-aead-4e3420237573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:44.302178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb8319-90d2-4640-aead-4e3420237573 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:44.318628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7696722-0073-499f-aebd-42cc7714e89e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:44.319559 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3832/7577] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:44.329431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7696722-0073-499f-aebd-42cc7714e89e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:44.335477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9743478-7347-469d-b34a-6cc55364d209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:44.336103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9743478-7347-469d-b34a-6cc55364d209 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:44.336398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9743478-7347-469d-b34a-6cc55364d209 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:44.336690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9743478-7347-469d-b34a-6cc55364d209 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:44.337342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9743478-7347-469d-b34a-6cc55364d209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:44.338953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9743478-7347-469d-b34a-6cc55364d209 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:44.351799 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3746/7578] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:44.356214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb8319-90d2-4640-aead-4e3420237573 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:44.368881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb8319-90d2-4640-aead-4e3420237573 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:44.375312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6bb0a-1b1c-4ab8-ac61-97a4f122e1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:44.376134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6bb0a-1b1c-4ab8-ac61-97a4f122e1af admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:44.376301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6bb0a-1b1c-4ab8-ac61-97a4f122e1af admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:44.377057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6bb0a-1b1c-4ab8-ac61-97a4f122e1af admin admin] PATH_INFO: `/v3/projects/ae8b7d3ef3ce47768f487793d85fa4f9/users/c7ca0328a59f4fdb8c0d67631de50e89/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:44.395984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6bb0a-1b1c-4ab8-ac61-97a4f122e1af admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae8b7d3ef3ce47768f487793d85fa4f9, user_id=c7ca0328a59f4fdb8c0d67631de50e89, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:44.398765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6bb0a-1b1c-4ab8-ac61-97a4f122e1af admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:44.415795 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3833/7579] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:44 2022] PUT /identity/v3/projects/ae8b7d3ef3ce47768f487793d85fa4f9/users/c7ca0328a59f4fdb8c0d67631de50e89/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:25:44.428498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d530f-4ba7-457e-8b08-91b5aa18d315 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:44.429315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d530f-4ba7-457e-8b08-91b5aa18d315 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:44.430807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d530f-4ba7-457e-8b08-91b5aa18d315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:44.466177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-da4d530f-4ba7-457e-8b08-91b5aa18d315 None None] MFA Rules not processed for user `c7ca0328a59f4fdb8c0d67631de50e89`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:44.467532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-da4d530f-4ba7-457e-8b08-91b5aa18d315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:44.553865 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3747/7580] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:44.558689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6bb0a-1b1c-4ab8-ac61-97a4f122e1af admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:44.638620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6bb0a-1b1c-4ab8-ac61-97a4f122e1af admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:44.644954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffda44-776a-429d-b640-d7cb44e0026a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:44.645567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffda44-776a-429d-b640-d7cb44e0026a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:44.645834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffda44-776a-429d-b640-d7cb44e0026a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:44.646094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffda44-776a-429d-b640-d7cb44e0026a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:44.646712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffda44-776a-429d-b640-d7cb44e0026a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:44.649372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffda44-776a-429d-b640-d7cb44e0026a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:44.691381 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3834/7581] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:44 2022] POST /identity/v3/projects => generated 357 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:44.692034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d530f-4ba7-457e-8b08-91b5aa18d315 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:44.705475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d530f-4ba7-457e-8b08-91b5aa18d315 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:44.788695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:44.830577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1090473-8ece-49ed-8c74-0e8aa8744482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:44.831346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1090473-8ece-49ed-8c74-0e8aa8744482 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:44.832053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1090473-8ece-49ed-8c74-0e8aa8744482 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:44.833009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1090473-8ece-49ed-8c74-0e8aa8744482 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:44.833667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1090473-8ece-49ed-8c74-0e8aa8744482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:44.836448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1090473-8ece-49ed-8c74-0e8aa8744482 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:44.892205 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3748/7582] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:44 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:25:44.895965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffda44-776a-429d-b640-d7cb44e0026a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:44.910086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffda44-776a-429d-b640-d7cb44e0026a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:44.921348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce252602-94aa-455a-97d9-cf80ab7ee706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:44.922545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce252602-94aa-455a-97d9-cf80ab7ee706 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:44.924728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce252602-94aa-455a-97d9-cf80ab7ee706 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:44.925431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce252602-94aa-455a-97d9-cf80ab7ee706 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:44.925946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce252602-94aa-455a-97d9-cf80ab7ee706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:44.930511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce252602-94aa-455a-97d9-cf80ab7ee706 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:44.960969 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3835/7583] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:44.962158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1090473-8ece-49ed-8c74-0e8aa8744482 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:44.980752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1090473-8ece-49ed-8c74-0e8aa8744482 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:44.998198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7ca83-13fc-4f6b-8dfb-db888b92a791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:44.998911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7ca83-13fc-4f6b-8dfb-db888b92a791 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:44.999429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7ca83-13fc-4f6b-8dfb-db888b92a791 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:44.999713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7ca83-13fc-4f6b-8dfb-db888b92a791 admin admin] PATH_INFO: `/v3/projects/01c0e0281d85444693de7259a6de0a37/users/96238a87cb2c4366b27318192dce0464/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:45.034625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7ca83-13fc-4f6b-8dfb-db888b92a791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c0e0281d85444693de7259a6de0a37, user_id=96238a87cb2c4366b27318192dce0464, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:45.039308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7ca83-13fc-4f6b-8dfb-db888b92a791 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:45.063010 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3749/7584] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:44 2022] PUT /identity/v3/projects/01c0e0281d85444693de7259a6de0a37/users/96238a87cb2c4366b27318192dce0464/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:25:45.064312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce252602-94aa-455a-97d9-cf80ab7ee706 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:45.082991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce252602-94aa-455a-97d9-cf80ab7ee706 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:45.116886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:45.167272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9680cec-d58a-47bc-8e3b-83436cf29eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:45.168055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680cec-d58a-47bc-8e3b-83436cf29eee admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:45.168557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680cec-d58a-47bc-8e3b-83436cf29eee admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:45.168843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680cec-d58a-47bc-8e3b-83436cf29eee admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:45.169625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9680cec-d58a-47bc-8e3b-83436cf29eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:45.171601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9680cec-d58a-47bc-8e3b-83436cf29eee admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:45.188417 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3836/7585] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:45.189803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7ca83-13fc-4f6b-8dfb-db888b92a791 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:45.201716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7ca83-13fc-4f6b-8dfb-db888b92a791 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:45.207748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcd752-fd94-4cf9-8b42-5ff7781b6c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:45.208425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcd752-fd94-4cf9-8b42-5ff7781b6c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:45.208807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcd752-fd94-4cf9-8b42-5ff7781b6c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:45.209018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcd752-fd94-4cf9-8b42-5ff7781b6c3d admin admin] PATH_INFO: `/v3/projects/01c0e0281d85444693de7259a6de0a37/users/96238a87cb2c4366b27318192dce0464/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:45.211141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcd752-fd94-4cf9-8b42-5ff7781b6c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c0e0281d85444693de7259a6de0a37, user_id=96238a87cb2c4366b27318192dce0464, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:45.213825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcd752-fd94-4cf9-8b42-5ff7781b6c3d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:45.250983 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3750/7586] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:45 2022] PUT /identity/v3/projects/01c0e0281d85444693de7259a6de0a37/users/96238a87cb2c4366b27318192dce0464/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:45.260407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad802398-f47e-4466-a29f-4268e2796728 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:45.260677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad802398-f47e-4466-a29f-4268e2796728 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:45.261216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad802398-f47e-4466-a29f-4268e2796728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:45.317179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ad802398-f47e-4466-a29f-4268e2796728 None None] MFA Rules not processed for user `96238a87cb2c4366b27318192dce0464`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:45.319131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ad802398-f47e-4466-a29f-4268e2796728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:45.461294 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3837/7587] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:45 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:45.474644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcd752-fd94-4cf9-8b42-5ff7781b6c3d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:45.477635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e9fcd752-fd94-4cf9-8b42-5ff7781b6c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:45.577041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcd752-fd94-4cf9-8b42-5ff7781b6c3d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:45.581821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18701df6-bbbb-4dd7-89d8-66097368a3b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:45.582510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18701df6-bbbb-4dd7-89d8-66097368a3b6 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:45.582845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18701df6-bbbb-4dd7-89d8-66097368a3b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:45.583200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18701df6-bbbb-4dd7-89d8-66097368a3b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:45.609837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18701df6-bbbb-4dd7-89d8-66097368a3b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:45.611685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18701df6-bbbb-4dd7-89d8-66097368a3b6 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:45.629235 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3751/7588] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:45 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:45.757464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad802398-f47e-4466-a29f-4268e2796728 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:45.774803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad802398-f47e-4466-a29f-4268e2796728 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:45.792112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d69fd-a50b-4da7-9f36-a9a862685251 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:45.793967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d69fd-a50b-4da7-9f36-a9a862685251 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:45.794629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d69fd-a50b-4da7-9f36-a9a862685251 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:45.795101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d69fd-a50b-4da7-9f36-a9a862685251 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:45.923157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d69fd-a50b-4da7-9f36-a9a862685251 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:45.925102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d69fd-a50b-4da7-9f36-a9a862685251 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:45.964430 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3838/7589] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:25:46.191882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5261407-68f2-42d0-8235-59c44a324abb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:46.193285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5261407-68f2-42d0-8235-59c44a324abb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:46.194078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5261407-68f2-42d0-8235-59c44a324abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:46.259115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b5261407-68f2-42d0-8235-59c44a324abb None None] MFA Rules not processed for user `4478c47d7f02468e84729c5fbfca0160`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:46.265999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b5261407-68f2-42d0-8235-59c44a324abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:46.384528 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3752/7590] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:46.388826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d69fd-a50b-4da7-9f36-a9a862685251 service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:46.391190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5f6d69fd-a50b-4da7-9f36-a9a862685251 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:46.482995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:46.504856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d69fd-a50b-4da7-9f36-a9a862685251 service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:46.513518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c516a-c937-4f16-850e-3adc19e06a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:46.514791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c516a-c937-4f16-850e-3adc19e06a65 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:46.516953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c516a-c937-4f16-850e-3adc19e06a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:46.517280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c516a-c937-4f16-850e-3adc19e06a65 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:46.518009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c516a-c937-4f16-850e-3adc19e06a65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:46.522028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c516a-c937-4f16-850e-3adc19e06a65 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:46.554055 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3839/7591] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:46 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:46.558276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5261407-68f2-42d0-8235-59c44a324abb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:46.570246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5261407-68f2-42d0-8235-59c44a324abb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:46.589120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:46.638785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135227cb-c705-46f9-8f19-02051e3c894d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:46.639423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135227cb-c705-46f9-8f19-02051e3c894d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:46.640000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135227cb-c705-46f9-8f19-02051e3c894d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:46.640274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135227cb-c705-46f9-8f19-02051e3c894d admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:46.640977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135227cb-c705-46f9-8f19-02051e3c894d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:46.643298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135227cb-c705-46f9-8f19-02051e3c894d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:46.668453 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3753/7592] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:46 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:46.671867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c516a-c937-4f16-850e-3adc19e06a65 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:46.683502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c516a-c937-4f16-850e-3adc19e06a65 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:46.688736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe2ead-ad4c-4f68-8f48-25dc32be221b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:46.689488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe2ead-ad4c-4f68-8f48-25dc32be221b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:46.689746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe2ead-ad4c-4f68-8f48-25dc32be221b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:46.690139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe2ead-ad4c-4f68-8f48-25dc32be221b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:46.690918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe2ead-ad4c-4f68-8f48-25dc32be221b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:46.694588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe2ead-ad4c-4f68-8f48-25dc32be221b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:46.707965 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3840/7593] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:46.710604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135227cb-c705-46f9-8f19-02051e3c894d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:46.721221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135227cb-c705-46f9-8f19-02051e3c894d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:46.725912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83c35e-c2a4-445c-8b1d-7a988ebb3c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:46.726581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83c35e-c2a4-445c-8b1d-7a988ebb3c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:46.726852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83c35e-c2a4-445c-8b1d-7a988ebb3c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:46.727116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83c35e-c2a4-445c-8b1d-7a988ebb3c06 admin admin] PATH_INFO: `/v3/projects/4fe987d796744e8dbcfd6577ba9b61fe/users/3654b89398f64723970ca4eb07b09dba/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:46.743418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83c35e-c2a4-445c-8b1d-7a988ebb3c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe987d796744e8dbcfd6577ba9b61fe, user_id=3654b89398f64723970ca4eb07b09dba, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:46.746368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83c35e-c2a4-445c-8b1d-7a988ebb3c06 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:46.761264 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3754/7594] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:46 2022] PUT /identity/v3/projects/4fe987d796744e8dbcfd6577ba9b61fe/users/3654b89398f64723970ca4eb07b09dba/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:46.765386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe2ead-ad4c-4f68-8f48-25dc32be221b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:46.776477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe2ead-ad4c-4f68-8f48-25dc32be221b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:46.839053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42987e7-fcf6-4565-a9b6-014330b3017a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:46.839746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42987e7-fcf6-4565-a9b6-014330b3017a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:46.839998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42987e7-fcf6-4565-a9b6-014330b3017a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:46.840303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42987e7-fcf6-4565-a9b6-014330b3017a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:46.840971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42987e7-fcf6-4565-a9b6-014330b3017a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:46.842574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42987e7-fcf6-4565-a9b6-014330b3017a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:46.856896 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3841/7595] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:25:46.859204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83c35e-c2a4-445c-8b1d-7a988ebb3c06 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:46.871709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83c35e-c2a4-445c-8b1d-7a988ebb3c06 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:46.878494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2329d-a10d-4ddc-990a-76d5b5301983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:46.879151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2329d-a10d-4ddc-990a-76d5b5301983 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:46.879422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2329d-a10d-4ddc-990a-76d5b5301983 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:46.879690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2329d-a10d-4ddc-990a-76d5b5301983 admin admin] PATH_INFO: `/v3/projects/4fe987d796744e8dbcfd6577ba9b61fe/users/3654b89398f64723970ca4eb07b09dba/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:46.881651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2329d-a10d-4ddc-990a-76d5b5301983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe987d796744e8dbcfd6577ba9b61fe, user_id=3654b89398f64723970ca4eb07b09dba, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:46.884619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2329d-a10d-4ddc-990a-76d5b5301983 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:46.896976 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3755/7596] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:46 2022] PUT /identity/v3/projects/4fe987d796744e8dbcfd6577ba9b61fe/users/3654b89398f64723970ca4eb07b09dba/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:25:47.138233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42987e7-fcf6-4565-a9b6-014330b3017a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:47.140294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a42987e7-fcf6-4565-a9b6-014330b3017a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:47.248123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42987e7-fcf6-4565-a9b6-014330b3017a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:47.311283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202ae19-9b22-42f7-9fbe-a4862e78e12b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:47.312234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202ae19-9b22-42f7-9fbe-a4862e78e12b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:47.312824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202ae19-9b22-42f7-9fbe-a4862e78e12b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:47.313191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202ae19-9b22-42f7-9fbe-a4862e78e12b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:47.315246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f202ae19-9b22-42f7-9fbe-a4862e78e12b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:47.332071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202ae19-9b22-42f7-9fbe-a4862e78e12b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:47.334010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202ae19-9b22-42f7-9fbe-a4862e78e12b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:47.347483 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3842/7597] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:25:50.007584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2329d-a10d-4ddc-990a-76d5b5301983 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:50.141968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2329d-a10d-4ddc-990a-76d5b5301983 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:50.159222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:50.159908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:50.160188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:50.160580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] PATH_INFO: `/v3/users/c7ca0328a59f4fdb8c0d67631de50e89` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:50.162338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ca0328a59f4fdb8c0d67631de50e89)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:50.166471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:50.323410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ca0328a59f4fdb8c0d67631de50e89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:50.340023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ca0328a59f4fdb8c0d67631de50e89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:50.360208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ca0328a59f4fdb8c0d67631de50e89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:50.361057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ca0328a59f4fdb8c0d67631de50e89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:50.361724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ca0328a59f4fdb8c0d67631de50e89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:50.370855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:50.387514 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3756/7598] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:50 2022] DELETE /identity/v3/users/c7ca0328a59f4fdb8c0d67631de50e89 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:50.389431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1d953-4a18-41de-a941-af01af487c97 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:50.390111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1d953-4a18-41de-a941-af01af487c97 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:50.390111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1d953-4a18-41de-a941-af01af487c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:50.413617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2df1d953-4a18-41de-a941-af01af487c97 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:50.414853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2df1d953-4a18-41de-a941-af01af487c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:50.424056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:50.499854 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3843/7599] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:50.507994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:50.509628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:50.590970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b70d2-b6e5-459e-80f3-69722561aa63 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:50.597124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336c5f1-a986-4ec6-b8bb-eb17d76f33e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:50.598521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336c5f1-a986-4ec6-b8bb-eb17d76f33e5 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:50.599034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336c5f1-a986-4ec6-b8bb-eb17d76f33e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:50.599518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336c5f1-a986-4ec6-b8bb-eb17d76f33e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:50.611648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336c5f1-a986-4ec6-b8bb-eb17d76f33e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:50.613608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336c5f1-a986-4ec6-b8bb-eb17d76f33e5 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:50.629585 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3757/7600] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:50.971078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1d953-4a18-41de-a941-af01af487c97 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:50.972257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2df1d953-4a18-41de-a941-af01af487c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:50.996619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1d953-4a18-41de-a941-af01af487c97 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:51.003893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:51.005150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:51.005425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:51.005690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] PATH_INFO: `/v3/projects/ae8b7d3ef3ce47768f487793d85fa4f9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:51.018291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae8b7d3ef3ce47768f487793d85fa4f9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:51.023155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:51.143645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae8b7d3ef3ce47768f487793d85fa4f9', 'request_id': 'req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:51.172955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae8b7d3ef3ce47768f487793d85fa4f9', 'request_id': 'req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:51.185921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae8b7d3ef3ce47768f487793d85fa4f9', 'request_id': 'req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:51.200242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] The token cache is being invalidate because project ae8b7d3ef3ce47768f487793d85fa4f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:51.200729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:51.201465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:51.204932 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3844/7601] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:50 2022] DELETE /identity/v3/projects/ae8b7d3ef3ce47768f487793d85fa4f9 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:51.209898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336c5f1-a986-4ec6-b8bb-eb17d76f33e5 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:51.225348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336c5f1-a986-4ec6-b8bb-eb17d76f33e5 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:51.367166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea16017-ac1e-497d-82dd-b952f25ed6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:51.368024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea16017-ac1e-497d-82dd-b952f25ed6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:51.368446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea16017-ac1e-497d-82dd-b952f25ed6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:51.368831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea16017-ac1e-497d-82dd-b952f25ed6d5 admin admin] PATH_INFO: `/v3/users/96238a87cb2c4366b27318192dce0464` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:51.370300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea16017-ac1e-497d-82dd-b952f25ed6d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96238a87cb2c4366b27318192dce0464)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:51.375251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea16017-ac1e-497d-82dd-b952f25ed6d5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:51.505941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ea16017-ac1e-497d-82dd-b952f25ed6d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96238a87cb2c4366b27318192dce0464'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:51.517338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ea16017-ac1e-497d-82dd-b952f25ed6d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96238a87cb2c4366b27318192dce0464'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:51.543481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ea16017-ac1e-497d-82dd-b952f25ed6d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96238a87cb2c4366b27318192dce0464'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:51.544433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ea16017-ac1e-497d-82dd-b952f25ed6d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96238a87cb2c4366b27318192dce0464'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:51.545123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ea16017-ac1e-497d-82dd-b952f25ed6d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96238a87cb2c4366b27318192dce0464'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:51.559941 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3758/7602] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:51 2022] DELETE /identity/v3/users/96238a87cb2c4366b27318192dce0464 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:52.201051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:52.202678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:52.231336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db51258-eca0-4e98-a5a9-89b3f6d8cc47 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:52.329453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912906b7-95a2-4f56-b79a-5e99240f040a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:52.331382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912906b7-95a2-4f56-b79a-5e99240f040a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:52.332037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912906b7-95a2-4f56-b79a-5e99240f040a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:52.332453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912906b7-95a2-4f56-b79a-5e99240f040a admin admin] PATH_INFO: `/v3/projects/01c0e0281d85444693de7259a6de0a37` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:52.333897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912906b7-95a2-4f56-b79a-5e99240f040a admin admin] RBAC: Authorizing `identity:delete_project(project_id=01c0e0281d85444693de7259a6de0a37)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:52.336178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912906b7-95a2-4f56-b79a-5e99240f040a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:52.395717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:52.405111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-912906b7-95a2-4f56-b79a-5e99240f040a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01c0e0281d85444693de7259a6de0a37', 'request_id': 'req-912906b7-95a2-4f56-b79a-5e99240f040a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:52.421672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-912906b7-95a2-4f56-b79a-5e99240f040a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01c0e0281d85444693de7259a6de0a37', 'request_id': 'req-912906b7-95a2-4f56-b79a-5e99240f040a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:52.430524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-912906b7-95a2-4f56-b79a-5e99240f040a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01c0e0281d85444693de7259a6de0a37', 'request_id': 'req-912906b7-95a2-4f56-b79a-5e99240f040a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:52.468597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-912906b7-95a2-4f56-b79a-5e99240f040a admin admin] The token cache is being invalidate because project 01c0e0281d85444693de7259a6de0a37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:52.468889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-912906b7-95a2-4f56-b79a-5e99240f040a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:52.469584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-912906b7-95a2-4f56-b79a-5e99240f040a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:52.471157 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3845/7603] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:52 2022] DELETE /identity/v3/projects/01c0e0281d85444693de7259a6de0a37 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:52.476331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de486c0-8848-45a9-a4e2-ab56901a6751 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:52.476845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de486c0-8848-45a9-a4e2-ab56901a6751 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:52.477112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de486c0-8848-45a9-a4e2-ab56901a6751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:52.503254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6de486c0-8848-45a9-a4e2-ab56901a6751 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:52.504639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6de486c0-8848-45a9-a4e2-ab56901a6751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:52.517815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:52.587173 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3759/7604] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:52.596425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48bc8c-346b-4054-a388-54e6ea4caaab None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:52.596702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48bc8c-346b-4054-a388-54e6ea4caaab None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:52.597010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48bc8c-346b-4054-a388-54e6ea4caaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:52.620416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1e48bc8c-346b-4054-a388-54e6ea4caaab None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:52.623191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1e48bc8c-346b-4054-a388-54e6ea4caaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:52.635742 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3846/7605] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:52.636970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de486c0-8848-45a9-a4e2-ab56901a6751 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:52.653018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de486c0-8848-45a9-a4e2-ab56901a6751 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:52.659524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6c4c1-0937-489f-b5dd-25e6980af88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:52.660149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6c4c1-0937-489f-b5dd-25e6980af88e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:52.660596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6c4c1-0937-489f-b5dd-25e6980af88e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:52.661068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6c4c1-0937-489f-b5dd-25e6980af88e admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:52.661864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6c4c1-0937-489f-b5dd-25e6980af88e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:25:52.662216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6c4c1-0937-489f-b5dd-25e6980af88e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:52.664033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6c4c1-0937-489f-b5dd-25e6980af88e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:52.679478 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3760/7606] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:25:52.683460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48bc8c-346b-4054-a388-54e6ea4caaab None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:52.697608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48bc8c-346b-4054-a388-54e6ea4caaab None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:52.708702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1610e9-cacc-457c-b5e3-2084641d2e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:52.709500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1610e9-cacc-457c-b5e3-2084641d2e93 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:52.709820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1610e9-cacc-457c-b5e3-2084641d2e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:52.710087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1610e9-cacc-457c-b5e3-2084641d2e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:52.710727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1610e9-cacc-457c-b5e3-2084641d2e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:52.713119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1610e9-cacc-457c-b5e3-2084641d2e93 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:52.750601 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3847/7607] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:52 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:52.753496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6c4c1-0937-489f-b5dd-25e6980af88e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:52.769209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6c4c1-0937-489f-b5dd-25e6980af88e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:52.840548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a272d4-49e1-4b0c-a443-b6523e9d990c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:52.841282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a272d4-49e1-4b0c-a443-b6523e9d990c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:52.841614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a272d4-49e1-4b0c-a443-b6523e9d990c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:52.841975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a272d4-49e1-4b0c-a443-b6523e9d990c admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:52.842727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a272d4-49e1-4b0c-a443-b6523e9d990c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:52.845008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a272d4-49e1-4b0c-a443-b6523e9d990c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:52.869928 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3761/7608] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:52 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:25:52.874428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1610e9-cacc-457c-b5e3-2084641d2e93 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:52.884806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1610e9-cacc-457c-b5e3-2084641d2e93 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:52.890942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd8aa2-0417-4833-9e2c-cde87386abad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:52.890942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8aa2-0417-4833-9e2c-cde87386abad admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:52.890942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8aa2-0417-4833-9e2c-cde87386abad admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:52.890942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8aa2-0417-4833-9e2c-cde87386abad admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:52.892437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd8aa2-0417-4833-9e2c-cde87386abad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:52.893683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd8aa2-0417-4833-9e2c-cde87386abad admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:52.907158 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3848/7609] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:52.908748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a272d4-49e1-4b0c-a443-b6523e9d990c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:52.922893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a272d4-49e1-4b0c-a443-b6523e9d990c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:52.929284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef7f96-9edd-400a-99ba-04802037bbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:52.930318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef7f96-9edd-400a-99ba-04802037bbec admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:52.930533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef7f96-9edd-400a-99ba-04802037bbec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:52.930838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef7f96-9edd-400a-99ba-04802037bbec admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:52.931484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef7f96-9edd-400a-99ba-04802037bbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:52.933255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef7f96-9edd-400a-99ba-04802037bbec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:52.947355 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3762/7610] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:25:52.951413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd8aa2-0417-4833-9e2c-cde87386abad admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:52.963263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd8aa2-0417-4833-9e2c-cde87386abad admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:53.001374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42940eb-a8b1-4bba-aba8-5484e676d8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:53.002118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42940eb-a8b1-4bba-aba8-5484e676d8d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:53.002544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42940eb-a8b1-4bba-aba8-5484e676d8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:53.002768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42940eb-a8b1-4bba-aba8-5484e676d8d4 admin admin] PATH_INFO: `/v3/projects/db6a0c50dade4d858840cc3fa1003dd9/users/586239933daf47c8a64e95a5616fc85a/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:53.021372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42940eb-a8b1-4bba-aba8-5484e676d8d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db6a0c50dade4d858840cc3fa1003dd9, user_id=586239933daf47c8a64e95a5616fc85a, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:53.024088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42940eb-a8b1-4bba-aba8-5484e676d8d4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:53.051999 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3849/7611] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:52 2022] PUT /identity/v3/projects/db6a0c50dade4d858840cc3fa1003dd9/users/586239933daf47c8a64e95a5616fc85a/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:53.053847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07d962-f981-4aa7-a0a5-fa397deac566 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:53.054131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07d962-f981-4aa7-a0a5-fa397deac566 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:53.054400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07d962-f981-4aa7-a0a5-fa397deac566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:53.085541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8d07d962-f981-4aa7-a0a5-fa397deac566 None None] MFA Rules not processed for user `586239933daf47c8a64e95a5616fc85a`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:53.086493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8d07d962-f981-4aa7-a0a5-fa397deac566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:53.221784 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3763/7612] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:53.239804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42940eb-a8b1-4bba-aba8-5484e676d8d4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:53.245521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a42940eb-a8b1-4bba-aba8-5484e676d8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:53.319061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e9e2c-5f9c-4b4e-94f7-badd3ca2e423 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:53.319358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e9e2c-5f9c-4b4e-94f7-badd3ca2e423 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:53.319660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e9e2c-5f9c-4b4e-94f7-badd3ca2e423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:53.350160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:53.356592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42940eb-a8b1-4bba-aba8-5484e676d8d4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:53.361711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311f645-db1f-4d6d-86d0-14ea619b0582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:53.362341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311f645-db1f-4d6d-86d0-14ea619b0582 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:53.362613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311f645-db1f-4d6d-86d0-14ea619b0582 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:53.362898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311f645-db1f-4d6d-86d0-14ea619b0582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:53.366549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ce5e9e2c-5f9c-4b4e-94f7-badd3ca2e423 None None] MFA Rules not processed for user `3654b89398f64723970ca4eb07b09dba`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:53.369170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ce5e9e2c-5f9c-4b4e-94f7-badd3ca2e423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:53.373346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311f645-db1f-4d6d-86d0-14ea619b0582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:53.375502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311f645-db1f-4d6d-86d0-14ea619b0582 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:53.386537 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3850/7613] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:25:53.484257 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3764/7614] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:53.493563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311f645-db1f-4d6d-86d0-14ea619b0582 service cinder] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:53.496365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2311f645-db1f-4d6d-86d0-14ea619b0582 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:53.593271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311f645-db1f-4d6d-86d0-14ea619b0582 service cinder] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:53.597176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a080e1-509a-408a-aeea-9fadb47605dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:53.598091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a080e1-509a-408a-aeea-9fadb47605dc service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:53.598380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a080e1-509a-408a-aeea-9fadb47605dc service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:53.598641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a080e1-509a-408a-aeea-9fadb47605dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:53.600575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-20a080e1-509a-408a-aeea-9fadb47605dc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:53.630696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:53.694809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a080e1-509a-408a-aeea-9fadb47605dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:53.696533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a080e1-509a-408a-aeea-9fadb47605dc service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:53.710979 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3851/7615] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:54.487192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e9e2c-5f9c-4b4e-94f7-badd3ca2e423 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:54.489319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ce5e9e2c-5f9c-4b4e-94f7-badd3ca2e423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:54.550936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e9e2c-5f9c-4b4e-94f7-badd3ca2e423 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:54.555294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35b189-e864-4098-b487-3427c5105c72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:54.555909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35b189-e864-4098-b487-3427c5105c72 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:54.556178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35b189-e864-4098-b487-3427c5105c72 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:54.556474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35b189-e864-4098-b487-3427c5105c72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:54.558193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bc35b189-e864-4098-b487-3427c5105c72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:54.576058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35b189-e864-4098-b487-3427c5105c72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:54.577826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35b189-e864-4098-b487-3427c5105c72 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:54.590414 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3765/7616] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:55.125192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a080e1-509a-408a-aeea-9fadb47605dc service glance] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:55.190279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a080e1-509a-408a-aeea-9fadb47605dc service glance] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:55.196042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:55.196690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:55.196945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:55.197212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] PATH_INFO: `/v3/users/586239933daf47c8a64e95a5616fc85a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:55.198432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=586239933daf47c8a64e95a5616fc85a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:55.200343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:55.256487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '586239933daf47c8a64e95a5616fc85a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:55.267578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '586239933daf47c8a64e95a5616fc85a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:55.276822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '586239933daf47c8a64e95a5616fc85a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:55.292050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '586239933daf47c8a64e95a5616fc85a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:55.293069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '586239933daf47c8a64e95a5616fc85a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:55.301118 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3852/7617] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:25:55 2022] DELETE /identity/v3/users/586239933daf47c8a64e95a5616fc85a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:55.305816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3c683-9eae-447e-9ac8-38a0d650708b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:55.306101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3c683-9eae-447e-9ac8-38a0d650708b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:55.306390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3c683-9eae-447e-9ac8-38a0d650708b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:55.325800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c5a3c683-9eae-447e-9ac8-38a0d650708b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:55.327023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c5a3c683-9eae-447e-9ac8-38a0d650708b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:55.398641 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3766/7618] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:55.408461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:55.410260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:55.458280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:55.479608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a46983-40d6-4b73-a7c2-5b7fa6317a1b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:55.483538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f27e90-6f22-487e-aba5-af35765259e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:55.484206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f27e90-6f22-487e-aba5-af35765259e5 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:55.484505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f27e90-6f22-487e-aba5-af35765259e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:55.484767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f27e90-6f22-487e-aba5-af35765259e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:55.494831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f27e90-6f22-487e-aba5-af35765259e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:55.496258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f27e90-6f22-487e-aba5-af35765259e5 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:55.506708 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3853/7619] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:25:55.846401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3c683-9eae-447e-9ac8-38a0d650708b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:55.848271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c5a3c683-9eae-447e-9ac8-38a0d650708b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:55.872422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3c683-9eae-447e-9ac8-38a0d650708b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:55.884026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66550db7-fe2d-4a7b-81f5-ac11f1213817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:55.884717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66550db7-fe2d-4a7b-81f5-ac11f1213817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:55.884993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66550db7-fe2d-4a7b-81f5-ac11f1213817 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:55.885262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66550db7-fe2d-4a7b-81f5-ac11f1213817 admin admin] PATH_INFO: `/v3/projects/db6a0c50dade4d858840cc3fa1003dd9` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:55.886461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66550db7-fe2d-4a7b-81f5-ac11f1213817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db6a0c50dade4d858840cc3fa1003dd9)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:55.888442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66550db7-fe2d-4a7b-81f5-ac11f1213817 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:55.952816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:55.966744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-66550db7-fe2d-4a7b-81f5-ac11f1213817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db6a0c50dade4d858840cc3fa1003dd9', 'request_id': 'req-66550db7-fe2d-4a7b-81f5-ac11f1213817'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:55.982270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-66550db7-fe2d-4a7b-81f5-ac11f1213817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db6a0c50dade4d858840cc3fa1003dd9', 'request_id': 'req-66550db7-fe2d-4a7b-81f5-ac11f1213817'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:55.992208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-66550db7-fe2d-4a7b-81f5-ac11f1213817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db6a0c50dade4d858840cc3fa1003dd9', 'request_id': 'req-66550db7-fe2d-4a7b-81f5-ac11f1213817'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:56.007367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-66550db7-fe2d-4a7b-81f5-ac11f1213817 admin admin] The token cache is being invalidate because project db6a0c50dade4d858840cc3fa1003dd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:25:56.007936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-66550db7-fe2d-4a7b-81f5-ac11f1213817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:56.008819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-66550db7-fe2d-4a7b-81f5-ac11f1213817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:25:56.010626 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3767/7620] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:25:55 2022] DELETE /identity/v3/projects/db6a0c50dade4d858840cc3fa1003dd9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:56.015825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5f2ba-25a5-4f43-a78a-e31029372796 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:56.016111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5f2ba-25a5-4f43-a78a-e31029372796 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:56.016379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5f2ba-25a5-4f43-a78a-e31029372796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:56.036547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4ac5f2ba-25a5-4f43-a78a-e31029372796 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:56.037855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4ac5f2ba-25a5-4f43-a78a-e31029372796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:56.157364 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3854/7621] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:56.164893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e28adc-3ee0-41f0-b5f5-8f8de03d98f8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:56.165105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e28adc-3ee0-41f0-b5f5-8f8de03d98f8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:56.165295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e28adc-3ee0-41f0-b5f5-8f8de03d98f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:56.196535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-68e28adc-3ee0-41f0-b5f5-8f8de03d98f8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:56.199204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-68e28adc-3ee0-41f0-b5f5-8f8de03d98f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:56.211882 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3768/7622] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:56.213698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5f2ba-25a5-4f43-a78a-e31029372796 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:56.225888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5f2ba-25a5-4f43-a78a-e31029372796 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:56.232300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3758104-6560-4781-b0f4-015c1bdfc893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:56.233316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3758104-6560-4781-b0f4-015c1bdfc893 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:56.233667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3758104-6560-4781-b0f4-015c1bdfc893 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:56.234283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3758104-6560-4781-b0f4-015c1bdfc893 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:56.235061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3758104-6560-4781-b0f4-015c1bdfc893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:25:56.235562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3758104-6560-4781-b0f4-015c1bdfc893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:56.237368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3758104-6560-4781-b0f4-015c1bdfc893 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:56.252902 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3855/7623] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:25:56.257978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e28adc-3ee0-41f0-b5f5-8f8de03d98f8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:56.269250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e28adc-3ee0-41f0-b5f5-8f8de03d98f8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:56.274368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a3ee6-3210-4222-b5cb-cd74bf41c4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:56.274993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a3ee6-3210-4222-b5cb-cd74bf41c4bb admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:56.275266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a3ee6-3210-4222-b5cb-cd74bf41c4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:56.275557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a3ee6-3210-4222-b5cb-cd74bf41c4bb admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:56.276184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a3ee6-3210-4222-b5cb-cd74bf41c4bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:56.278296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a3ee6-3210-4222-b5cb-cd74bf41c4bb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:56.311178 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3769/7624] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:25:56 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:25:56.316537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3758104-6560-4781-b0f4-015c1bdfc893 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:56.330163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3758104-6560-4781-b0f4-015c1bdfc893 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:56.421504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db611add-d80a-4c80-a3f9-b37f230d3df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:56.422367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db611add-d80a-4c80-a3f9-b37f230d3df0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:56.422685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db611add-d80a-4c80-a3f9-b37f230d3df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:56.423025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db611add-d80a-4c80-a3f9-b37f230d3df0 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:56.423765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db611add-d80a-4c80-a3f9-b37f230d3df0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:56.425984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db611add-d80a-4c80-a3f9-b37f230d3df0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:56.468034 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3856/7625] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:25:56 2022] POST /identity/v3/users => generated 285 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:25:56.472315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a3ee6-3210-4222-b5cb-cd74bf41c4bb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:56.486109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a3ee6-3210-4222-b5cb-cd74bf41c4bb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:56.491313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897b358-aec1-4827-9a4e-3c5baa87799b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:56.492010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897b358-aec1-4827-9a4e-3c5baa87799b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:56.492288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897b358-aec1-4827-9a4e-3c5baa87799b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:56.492588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897b358-aec1-4827-9a4e-3c5baa87799b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:56.493579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897b358-aec1-4827-9a4e-3c5baa87799b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:56.494818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897b358-aec1-4827-9a4e-3c5baa87799b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:56.509701 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3770/7626] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:56.511831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db611add-d80a-4c80-a3f9-b37f230d3df0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:56.535156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db611add-d80a-4c80-a3f9-b37f230d3df0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:56.545466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68757bd9-29d1-4158-9a12-4967d06d0358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:56.546487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68757bd9-29d1-4158-9a12-4967d06d0358 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:56.546946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68757bd9-29d1-4158-9a12-4967d06d0358 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:56.547308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68757bd9-29d1-4158-9a12-4967d06d0358 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:56.548050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68757bd9-29d1-4158-9a12-4967d06d0358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:56.549792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68757bd9-29d1-4158-9a12-4967d06d0358 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:56.564627 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3857/7627] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:25:56.566958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897b358-aec1-4827-9a4e-3c5baa87799b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:56.579816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897b358-aec1-4827-9a4e-3c5baa87799b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:56.585206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067290b-e390-4020-8b3e-c828d4a7c8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:56.585868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067290b-e390-4020-8b3e-c828d4a7c8db admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:56.586139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067290b-e390-4020-8b3e-c828d4a7c8db admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:56.586407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067290b-e390-4020-8b3e-c828d4a7c8db admin admin] PATH_INFO: `/v3/projects/d932bfc150b44522b0c11f441d91f868/users/aac0e96e43f54ed58c639416dd18bf89/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:56.608560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067290b-e390-4020-8b3e-c828d4a7c8db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d932bfc150b44522b0c11f441d91f868, user_id=aac0e96e43f54ed58c639416dd18bf89, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:56.614055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067290b-e390-4020-8b3e-c828d4a7c8db admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:56.645424 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3771/7628] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:25:56 2022] PUT /identity/v3/projects/d932bfc150b44522b0c11f441d91f868/users/aac0e96e43f54ed58c639416dd18bf89/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:25:56.649438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ee488-6952-4d6d-b893-02017eb8320d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:56.649842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ee488-6952-4d6d-b893-02017eb8320d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:56.650229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ee488-6952-4d6d-b893-02017eb8320d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:56.712662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-915ee488-6952-4d6d-b893-02017eb8320d None None] MFA Rules not processed for user `aac0e96e43f54ed58c639416dd18bf89`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:25:56.714490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-915ee488-6952-4d6d-b893-02017eb8320d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:56.787113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:25:56.853075 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3858/7629] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:25:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:25:56.863967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067290b-e390-4020-8b3e-c828d4a7c8db admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:25:56.866616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5067290b-e390-4020-8b3e-c828d4a7c8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:25:56.975679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067290b-e390-4020-8b3e-c828d4a7c8db admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:25:56.986264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582c6ab-4714-4676-b480-b3da8433d911 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:25:56.987197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582c6ab-4714-4676-b480-b3da8433d911 service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:25:56.987621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582c6ab-4714-4676-b480-b3da8433d911 service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:25:56.988170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582c6ab-4714-4676-b480-b3da8433d911 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:25:56.998094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582c6ab-4714-4676-b480-b3da8433d911 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:25:56.999664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582c6ab-4714-4676-b480-b3da8433d911 service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:25:57.012362 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3772/7630] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:25:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:26:00.385427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ee488-6952-4d6d-b893-02017eb8320d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:00.386890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-915ee488-6952-4d6d-b893-02017eb8320d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:00.492515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ee488-6952-4d6d-b893-02017eb8320d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:00.498931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113ebd1-29d1-4532-a1d7-a762dd70854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:00.499981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113ebd1-29d1-4532-a1d7-a762dd70854a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:00.500524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113ebd1-29d1-4532-a1d7-a762dd70854a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:00.500896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113ebd1-29d1-4532-a1d7-a762dd70854a admin admin] PATH_INFO: `/v3/users/d84dca25404141318e5f118ab80e6a86` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:00.502315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113ebd1-29d1-4532-a1d7-a762dd70854a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84dca25404141318e5f118ab80e6a86)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:00.504478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113ebd1-29d1-4532-a1d7-a762dd70854a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:00.574322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9113ebd1-29d1-4532-a1d7-a762dd70854a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84dca25404141318e5f118ab80e6a86'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:00.591125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582c6ab-4714-4676-b480-b3da8433d911 service glance] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:00.596511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9113ebd1-29d1-4532-a1d7-a762dd70854a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84dca25404141318e5f118ab80e6a86'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:00.609773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9113ebd1-29d1-4532-a1d7-a762dd70854a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84dca25404141318e5f118ab80e6a86'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:00.609959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582c6ab-4714-4676-b480-b3da8433d911 service glance] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:00.614545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b7979-9f05-448c-8785-fa42e5eb6cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:00.615281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b7979-9f05-448c-8785-fa42e5eb6cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:00.615558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b7979-9f05-448c-8785-fa42e5eb6cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:00.615824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b7979-9f05-448c-8785-fa42e5eb6cb5 admin admin] PATH_INFO: `/v3/users/aac0e96e43f54ed58c639416dd18bf89` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:00.616982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b7979-9f05-448c-8785-fa42e5eb6cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aac0e96e43f54ed58c639416dd18bf89)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:00.618909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b7979-9f05-448c-8785-fa42e5eb6cb5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:00.629224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9113ebd1-29d1-4532-a1d7-a762dd70854a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84dca25404141318e5f118ab80e6a86'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:00.630603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9113ebd1-29d1-4532-a1d7-a762dd70854a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84dca25404141318e5f118ab80e6a86'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:00.648005 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3859/7631] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:00 2022] DELETE /identity/v3/users/d84dca25404141318e5f118ab80e6a86 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:00.649998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d091f79-ccb8-4f0c-8259-1142cf869d45 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:00.650440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d091f79-ccb8-4f0c-8259-1142cf869d45 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:00.650901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d091f79-ccb8-4f0c-8259-1142cf869d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:00.673478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5d091f79-ccb8-4f0c-8259-1142cf869d45 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:00.675032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5d091f79-ccb8-4f0c-8259-1142cf869d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:00.687715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-840b7979-9f05-448c-8785-fa42e5eb6cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aac0e96e43f54ed58c639416dd18bf89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:00.702730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-840b7979-9f05-448c-8785-fa42e5eb6cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aac0e96e43f54ed58c639416dd18bf89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:00.706858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=13,keystone:SELECT=83,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:00.729454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-840b7979-9f05-448c-8785-fa42e5eb6cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aac0e96e43f54ed58c639416dd18bf89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:00.730452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-840b7979-9f05-448c-8785-fa42e5eb6cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aac0e96e43f54ed58c639416dd18bf89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:00.731063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-840b7979-9f05-448c-8785-fa42e5eb6cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aac0e96e43f54ed58c639416dd18bf89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:00.751051 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3773/7632] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:00 2022] DELETE /identity/v3/users/aac0e96e43f54ed58c639416dd18bf89 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:00.755042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:00.755819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113da64-364f-49d8-b617-61fdd4d524cf None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:00.756327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113da64-364f-49d8-b617-61fdd4d524cf None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:00.756647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113da64-364f-49d8-b617-61fdd4d524cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:00.774857 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3860/7633] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:00.781795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6113da64-364f-49d8-b617-61fdd4d524cf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:00.783032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6113da64-364f-49d8-b617-61fdd4d524cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:00.790712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d091f79-ccb8-4f0c-8259-1142cf869d45 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:00.793161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5d091f79-ccb8-4f0c-8259-1142cf869d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:00.893242 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3774/7634] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:00.896046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d091f79-ccb8-4f0c-8259-1142cf869d45 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:00.903048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113da64-364f-49d8-b617-61fdd4d524cf None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:00.988424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113da64-364f-49d8-b617-61fdd4d524cf None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:00.990408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dc440-e71d-41dc-a5a6-b3814e37a7b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:00.990994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dc440-e71d-41dc-a5a6-b3814e37a7b4 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:00.991265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dc440-e71d-41dc-a5a6-b3814e37a7b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:00.991530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dc440-e71d-41dc-a5a6-b3814e37a7b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:00.992475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf9a7e-aac2-484e-a570-f3990f689621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:00.993159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf9a7e-aac2-484e-a570-f3990f689621 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:00.993440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf9a7e-aac2-484e-a570-f3990f689621 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:00.993706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf9a7e-aac2-484e-a570-f3990f689621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:01.004458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf9a7e-aac2-484e-a570-f3990f689621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:01.006007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf9a7e-aac2-484e-a570-f3990f689621 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:01.016931 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3775/7635] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:26:01.029301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77dc440-e71d-41dc-a5a6-b3814e37a7b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:01.031109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77dc440-e71d-41dc-a5a6-b3814e37a7b4 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:01.055572 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3861/7636] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:26:01.573248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf9a7e-aac2-484e-a570-f3990f689621 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:01.575202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1abf9a7e-aac2-484e-a570-f3990f689621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:01.592009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf9a7e-aac2-484e-a570-f3990f689621 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:01.598094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:01.599060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:01.599425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:01.599864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c admin admin] PATH_INFO: `/v3/projects/794c95c5336549e9914787c32be79ae1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:01.601504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c admin admin] RBAC: Authorizing `identity:delete_project(project_id=794c95c5336549e9914787c32be79ae1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:01.604178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:01.682669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '794c95c5336549e9914787c32be79ae1', 'request_id': 'req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:01.706047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '794c95c5336549e9914787c32be79ae1', 'request_id': 'req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:01.717738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '794c95c5336549e9914787c32be79ae1', 'request_id': 'req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:01.720753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dc440-e71d-41dc-a5a6-b3814e37a7b4 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:01.722901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b77dc440-e71d-41dc-a5a6-b3814e37a7b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:01.738802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:01.741598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dc440-e71d-41dc-a5a6-b3814e37a7b4 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:01.748826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c admin admin] The token cache is being invalidate because project 794c95c5336549e9914787c32be79ae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:01.749153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:01.751058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20832c4c-f7ae-4d41-8d0e-e88cfcdad59c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:01.758588 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3776/7637] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:01 2022] DELETE /identity/v3/projects/794c95c5336549e9914787c32be79ae1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:01.761049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781f27f-73bc-4af8-8042-d86274fbd418 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:01.761327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781f27f-73bc-4af8-8042-d86274fbd418 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:01.761838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781f27f-73bc-4af8-8042-d86274fbd418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:01.793375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b781f27f-73bc-4af8-8042-d86274fbd418 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:01.794577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b781f27f-73bc-4af8-8042-d86274fbd418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:01.832801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:01.834136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:01.834418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:01.834692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e admin admin] PATH_INFO: `/v3/projects/d932bfc150b44522b0c11f441d91f868` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:01.835935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d932bfc150b44522b0c11f441d91f868)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:01.838023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:01.877710 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3777/7638] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:01.886356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1546974-8168-45bc-ae55-8305ea330a21 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:01.886629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1546974-8168-45bc-ae55-8305ea330a21 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:01.886898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1546974-8168-45bc-ae55-8305ea330a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:01.893398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd932bfc150b44522b0c11f441d91f868', 'request_id': 'req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:01.901661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd932bfc150b44522b0c11f441d91f868', 'request_id': 'req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:01.904766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:01.909975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d1546974-8168-45bc-ae55-8305ea330a21 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:01.910159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd932bfc150b44522b0c11f441d91f868', 'request_id': 'req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:01.911245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d1546974-8168-45bc-ae55-8305ea330a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:01.925767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e admin admin] The token cache is being invalidate because project d932bfc150b44522b0c11f441d91f868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:01.926046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:01.926794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bce02aab-f7c8-432c-9683-ac3dc7f9cb7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:01.928727 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3862/7639] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:01 2022] DELETE /identity/v3/projects/d932bfc150b44522b0c11f441d91f868 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:26:01.933737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806f3aa-7c04-4f76-9d05-a46017bf7097 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:01.933940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806f3aa-7c04-4f76-9d05-a46017bf7097 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:01.934210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806f3aa-7c04-4f76-9d05-a46017bf7097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:01.954901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7806f3aa-7c04-4f76-9d05-a46017bf7097 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:01.956179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7806f3aa-7c04-4f76-9d05-a46017bf7097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:01.985903 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3778/7640] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:01.987329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1546974-8168-45bc-ae55-8305ea330a21 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.012233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1546974-8168-45bc-ae55-8305ea330a21 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.018460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91687e-aeb8-4316-9e62-7bc4dee67173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.019458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91687e-aeb8-4316-9e62-7bc4dee67173 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.019760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91687e-aeb8-4316-9e62-7bc4dee67173 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.020035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91687e-aeb8-4316-9e62-7bc4dee67173 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.020849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91687e-aeb8-4316-9e62-7bc4dee67173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:02.021148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91687e-aeb8-4316-9e62-7bc4dee67173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.022861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91687e-aeb8-4316-9e62-7bc4dee67173 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.033228 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3863/7641] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:02.041400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b43d1-6e00-4ccd-8967-4121e381ef96 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.041961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b43d1-6e00-4ccd-8967-4121e381ef96 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.042415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b43d1-6e00-4ccd-8967-4121e381ef96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.048985 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3779/7642] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:02.052310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91687e-aeb8-4316-9e62-7bc4dee67173 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.064802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91687e-aeb8-4316-9e62-7bc4dee67173 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.070854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d6e56-7821-4c97-a010-5c10a739c5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.071660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d6e56-7821-4c97-a010-5c10a739c5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.073322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d6e56-7821-4c97-a010-5c10a739c5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.073322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d6e56-7821-4c97-a010-5c10a739c5d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.073322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d6e56-7821-4c97-a010-5c10a739c5d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.073606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b56b43d1-6e00-4ccd-8967-4121e381ef96 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:02.074502 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b56b43d1-6e00-4ccd-8967-4121e381ef96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:02.075371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d6e56-7821-4c97-a010-5c10a739c5d1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.084384 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3864/7643] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:02.088271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b43d1-6e00-4ccd-8967-4121e381ef96 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.098863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b43d1-6e00-4ccd-8967-4121e381ef96 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.107486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735f79d-c136-45aa-86f9-ce76bb0f69d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.109228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735f79d-c136-45aa-86f9-ce76bb0f69d0 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.109925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735f79d-c136-45aa-86f9-ce76bb0f69d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.110194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735f79d-c136-45aa-86f9-ce76bb0f69d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.112360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735f79d-c136-45aa-86f9-ce76bb0f69d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:02.112742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735f79d-c136-45aa-86f9-ce76bb0f69d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.114331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735f79d-c136-45aa-86f9-ce76bb0f69d0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.121148 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3780/7644] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:02 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:02.122212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d6e56-7821-4c97-a010-5c10a739c5d1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.132561 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3865/7645] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:26:02.136142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735f79d-c136-45aa-86f9-ce76bb0f69d0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.149853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735f79d-c136-45aa-86f9-ce76bb0f69d0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.222764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d6e56-7821-4c97-a010-5c10a739c5d1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.227916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98bf44-884a-428c-a86d-b5a57416a5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.228983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98bf44-884a-428c-a86d-b5a57416a5d7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.229339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98bf44-884a-428c-a86d-b5a57416a5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.229638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ef2db-1e2c-4d31-9337-f0c5ec6829c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.229799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98bf44-884a-428c-a86d-b5a57416a5d7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.230513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98bf44-884a-428c-a86d-b5a57416a5d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.230700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ef2db-1e2c-4d31-9337-f0c5ec6829c3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.231115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ef2db-1e2c-4d31-9337-f0c5ec6829c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.231524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ef2db-1e2c-4d31-9337-f0c5ec6829c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.232405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ef2db-1e2c-4d31-9337-f0c5ec6829c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.233006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98bf44-884a-428c-a86d-b5a57416a5d7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.235896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ef2db-1e2c-4d31-9337-f0c5ec6829c3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.247802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:02.264530 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3781/7646] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:02 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:02.266668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98bf44-884a-428c-a86d-b5a57416a5d7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.276544 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3866/7647] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:02 2022] POST /identity/v3/projects => generated 391 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:02.277121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98bf44-884a-428c-a86d-b5a57416a5d7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.279076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ef2db-1e2c-4d31-9337-f0c5ec6829c3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.346447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:02.354893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41c256-53dc-4cb0-bb0d-2e612fdd027d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.355495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41c256-53dc-4cb0-bb0d-2e612fdd027d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.355779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41c256-53dc-4cb0-bb0d-2e612fdd027d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.356048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41c256-53dc-4cb0-bb0d-2e612fdd027d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.357064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41c256-53dc-4cb0-bb0d-2e612fdd027d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.358906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41c256-53dc-4cb0-bb0d-2e612fdd027d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.359992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ef2db-1e2c-4d31-9337-f0c5ec6829c3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.367156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b8bcf-900c-49d1-a656-4477a1fdeef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.368268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b8bcf-900c-49d1-a656-4477a1fdeef5 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.368634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b8bcf-900c-49d1-a656-4477a1fdeef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.368973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b8bcf-900c-49d1-a656-4477a1fdeef5 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.369685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b8bcf-900c-49d1-a656-4477a1fdeef5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.370408 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3782/7648] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:26:02.372476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b8bcf-900c-49d1-a656-4477a1fdeef5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.374664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41c256-53dc-4cb0-bb0d-2e612fdd027d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.386614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41c256-53dc-4cb0-bb0d-2e612fdd027d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.393399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b64f2-f330-4a49-99e3-72273088b062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.394366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b64f2-f330-4a49-99e3-72273088b062 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.394641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b64f2-f330-4a49-99e3-72273088b062 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.394907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b64f2-f330-4a49-99e3-72273088b062 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.395626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b64f2-f330-4a49-99e3-72273088b062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.397459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b64f2-f330-4a49-99e3-72273088b062 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.406370 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3867/7649] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:02 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:02.408304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b8bcf-900c-49d1-a656-4477a1fdeef5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.410142 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3783/7650] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:02.414294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b64f2-f330-4a49-99e3-72273088b062 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.425606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b64f2-f330-4a49-99e3-72273088b062 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.430719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c192a4-ad96-4137-8303-eb18f7867714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.431064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b8bcf-900c-49d1-a656-4477a1fdeef5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.431507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c192a4-ad96-4137-8303-eb18f7867714 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.431787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c192a4-ad96-4137-8303-eb18f7867714 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.432053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c192a4-ad96-4137-8303-eb18f7867714 admin admin] PATH_INFO: `/v3/projects/0562065ffa3e4c27a03fd19b49c252eb/users/2bd3b3a2aad74b62aa8af337703f4527/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.436509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414920c6-5674-4044-9097-5f3621a23c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.437372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414920c6-5674-4044-9097-5f3621a23c41 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.437825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414920c6-5674-4044-9097-5f3621a23c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.438235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414920c6-5674-4044-9097-5f3621a23c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.439179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414920c6-5674-4044-9097-5f3621a23c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.441214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414920c6-5674-4044-9097-5f3621a23c41 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.453062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c192a4-ad96-4137-8303-eb18f7867714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0562065ffa3e4c27a03fd19b49c252eb, user_id=2bd3b3a2aad74b62aa8af337703f4527, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.457222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c192a4-ad96-4137-8303-eb18f7867714 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.457548 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3868/7651] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:02.458750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414920c6-5674-4044-9097-5f3621a23c41 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.470638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414920c6-5674-4044-9097-5f3621a23c41 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.486713 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3784/7652] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:02 2022] PUT /identity/v3/projects/0562065ffa3e4c27a03fd19b49c252eb/users/2bd3b3a2aad74b62aa8af337703f4527/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:26:02.488983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903e37b-0fa0-4c1f-b193-26dd0979de55 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.489509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903e37b-0fa0-4c1f-b193-26dd0979de55 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.489876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903e37b-0fa0-4c1f-b193-26dd0979de55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.525057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c903e37b-0fa0-4c1f-b193-26dd0979de55 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:02.526413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c903e37b-0fa0-4c1f-b193-26dd0979de55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:02.541494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd9452-d0a3-4694-ba82-115d4f7fcb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.542184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd9452-d0a3-4694-ba82-115d4f7fcb89 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.542525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd9452-d0a3-4694-ba82-115d4f7fcb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.542852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd9452-d0a3-4694-ba82-115d4f7fcb89 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.543598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd9452-d0a3-4694-ba82-115d4f7fcb89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.545293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd9452-d0a3-4694-ba82-115d4f7fcb89 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.556427 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3869/7653] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:02.560481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd9452-d0a3-4694-ba82-115d4f7fcb89 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.571365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd9452-d0a3-4694-ba82-115d4f7fcb89 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.576271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8d535-9d0f-4b9f-ae08-8bc28b9bd73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.577378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8d535-9d0f-4b9f-ae08-8bc28b9bd73b admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.577378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8d535-9d0f-4b9f-ae08-8bc28b9bd73b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.577681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8d535-9d0f-4b9f-ae08-8bc28b9bd73b admin admin] PATH_INFO: `/v3/projects/fa42503d942c47bdb3f3366e3fea0241/users/f2c2c4da3c954f96af706644f238ed97/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.595962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab8d535-9d0f-4b9f-ae08-8bc28b9bd73b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa42503d942c47bdb3f3366e3fea0241, user_id=f2c2c4da3c954f96af706644f238ed97, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.601277 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3785/7654] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:02.601824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab8d535-9d0f-4b9f-ae08-8bc28b9bd73b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.609698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903e37b-0fa0-4c1f-b193-26dd0979de55 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.612613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c903e37b-0fa0-4c1f-b193-26dd0979de55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:02.614404 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3870/7655] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:02 2022] PUT /identity/v3/projects/fa42503d942c47bdb3f3366e3fea0241/users/f2c2c4da3c954f96af706644f238ed97/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:02.625059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0d97-c46b-4fec-97e2-a645787ef980 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.625335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0d97-c46b-4fec-97e2-a645787ef980 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.625601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0d97-c46b-4fec-97e2-a645787ef980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.670396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-434f0d97-c46b-4fec-97e2-a645787ef980 None None] MFA Rules not processed for user `f2c2c4da3c954f96af706644f238ed97`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:02.671976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-434f0d97-c46b-4fec-97e2-a645787ef980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:02.701967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903e37b-0fa0-4c1f-b193-26dd0979de55 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.719152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:02.758084 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3871/7656] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:02.762835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f0d97-c46b-4fec-97e2-a645787ef980 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.771553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b1c03-44ff-40f2-9aa6-4ad9b8f27ff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.772208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b1c03-44ff-40f2-9aa6-4ad9b8f27ff9 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.772546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b1c03-44ff-40f2-9aa6-4ad9b8f27ff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.772816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b1c03-44ff-40f2-9aa6-4ad9b8f27ff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.802417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:02.844320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f0d97-c46b-4fec-97e2-a645787ef980 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:02.846534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b1c03-44ff-40f2-9aa6-4ad9b8f27ff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.848403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b1c03-44ff-40f2-9aa6-4ad9b8f27ff9 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.849838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073326bf-5ea1-4550-9a88-26545f4b5644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:02.850287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073326bf-5ea1-4550-9a88-26545f4b5644 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:02.850562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073326bf-5ea1-4550-9a88-26545f4b5644 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:02.850826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073326bf-5ea1-4550-9a88-26545f4b5644 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:02.851498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073326bf-5ea1-4550-9a88-26545f4b5644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:02.853449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073326bf-5ea1-4550-9a88-26545f4b5644 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:02.869826 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3786/7657] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:26:02.919044 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3872/7658] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:02 2022] POST /identity/v3/projects => generated 393 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:26:02.920238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b1c03-44ff-40f2-9aa6-4ad9b8f27ff9 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:02.934305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b1c03-44ff-40f2-9aa6-4ad9b8f27ff9 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:03.026950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db24a0-6c59-4335-9e1b-7c5590cfb7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:03.027367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db24a0-6c59-4335-9e1b-7c5590cfb7ec admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:03.027900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db24a0-6c59-4335-9e1b-7c5590cfb7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:03.028169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db24a0-6c59-4335-9e1b-7c5590cfb7ec admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:03.029553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db24a0-6c59-4335-9e1b-7c5590cfb7ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:03.032718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db24a0-6c59-4335-9e1b-7c5590cfb7ec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:03.082090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073326bf-5ea1-4550-9a88-26545f4b5644 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:03.082447 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3787/7659] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:02 2022] POST /identity/v3/users => generated 303 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:03.107124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073326bf-5ea1-4550-9a88-26545f4b5644 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:03.113398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b664b55-046d-4d9e-8504-359c4b0350a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:03.114253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b664b55-046d-4d9e-8504-359c4b0350a1 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:03.115019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b664b55-046d-4d9e-8504-359c4b0350a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:03.115469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b664b55-046d-4d9e-8504-359c4b0350a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:03.116291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b664b55-046d-4d9e-8504-359c4b0350a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:03.118791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b664b55-046d-4d9e-8504-359c4b0350a1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:03.136717 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3873/7660] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:03.142617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db24a0-6c59-4335-9e1b-7c5590cfb7ec admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:03.156006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db24a0-6c59-4335-9e1b-7c5590cfb7ec admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:03.162697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a5702-07f4-490f-b050-c9e13da6c95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:03.163369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a5702-07f4-490f-b050-c9e13da6c95a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:03.163637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a5702-07f4-490f-b050-c9e13da6c95a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:03.164148 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a5702-07f4-490f-b050-c9e13da6c95a admin admin] PATH_INFO: `/v3/projects/0f72fd57efba4913982a47f8fa34871f/users/95695afba1af4de1ba73879c85f41d3b/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:03.187775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a5702-07f4-490f-b050-c9e13da6c95a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f72fd57efba4913982a47f8fa34871f, user_id=95695afba1af4de1ba73879c85f41d3b, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:03.199378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a5702-07f4-490f-b050-c9e13da6c95a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:03.222515 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3788/7661] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:03 2022] PUT /identity/v3/projects/0f72fd57efba4913982a47f8fa34871f/users/95695afba1af4de1ba73879c85f41d3b/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:03.234025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b664b55-046d-4d9e-8504-359c4b0350a1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:03.246370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b664b55-046d-4d9e-8504-359c4b0350a1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:03.365505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca7282-27c5-4610-8f71-a73f8c97bcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:03.366301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca7282-27c5-4610-8f71-a73f8c97bcba admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:03.366668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca7282-27c5-4610-8f71-a73f8c97bcba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:03.367027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca7282-27c5-4610-8f71-a73f8c97bcba admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:03.367846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca7282-27c5-4610-8f71-a73f8c97bcba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:03.369707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca7282-27c5-4610-8f71-a73f8c97bcba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:03.382140 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3874/7662] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:03.386227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a5702-07f4-490f-b050-c9e13da6c95a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:03.398175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a5702-07f4-490f-b050-c9e13da6c95a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:03.403746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20ef92-5b87-44f3-98f6-5661493098d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:03.404720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20ef92-5b87-44f3-98f6-5661493098d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:03.405477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20ef92-5b87-44f3-98f6-5661493098d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:03.405931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20ef92-5b87-44f3-98f6-5661493098d3 admin admin] PATH_INFO: `/v3/projects/0f72fd57efba4913982a47f8fa34871f/users/95695afba1af4de1ba73879c85f41d3b/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:03.408778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20ef92-5b87-44f3-98f6-5661493098d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f72fd57efba4913982a47f8fa34871f, user_id=95695afba1af4de1ba73879c85f41d3b, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:03.411542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20ef92-5b87-44f3-98f6-5661493098d3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:03.424067 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3789/7663] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:03 2022] PUT /identity/v3/projects/0f72fd57efba4913982a47f8fa34871f/users/95695afba1af4de1ba73879c85f41d3b/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:03.435563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c29645-8217-4317-ab16-0c99c47dd591 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:03.435839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c29645-8217-4317-ab16-0c99c47dd591 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:03.436144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c29645-8217-4317-ab16-0c99c47dd591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:03.479406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-15c29645-8217-4317-ab16-0c99c47dd591 None None] MFA Rules not processed for user `95695afba1af4de1ba73879c85f41d3b`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:03.480686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-15c29645-8217-4317-ab16-0c99c47dd591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:03.578092 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3875/7664] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:03 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:03.583382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20ef92-5b87-44f3-98f6-5661493098d3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:03.591683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0a20ef92-5b87-44f3-98f6-5661493098d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:03.624758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:03.762270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20ef92-5b87-44f3-98f6-5661493098d3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:03.777216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a934225-ee8b-4f2c-8c9c-9da8cbc87993 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:03.777894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a934225-ee8b-4f2c-8c9c-9da8cbc87993 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:03.778227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a934225-ee8b-4f2c-8c9c-9da8cbc87993 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:03.778504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a934225-ee8b-4f2c-8c9c-9da8cbc87993 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:03.919181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a934225-ee8b-4f2c-8c9c-9da8cbc87993 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:03.921035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a934225-ee8b-4f2c-8c9c-9da8cbc87993 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:03.946627 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3790/7665] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:26:05.106358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c29645-8217-4317-ab16-0c99c47dd591 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:05.108031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-15c29645-8217-4317-ab16-0c99c47dd591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:05.148532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:05.208781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c29645-8217-4317-ab16-0c99c47dd591 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:05.218752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:05.219700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:05.228866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:05.229637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] PATH_INFO: `/v3/users/4478c47d7f02468e84729c5fbfca0160` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:05.231600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4478c47d7f02468e84729c5fbfca0160)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:05.234093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:05.319728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4478c47d7f02468e84729c5fbfca0160'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:05.339412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4478c47d7f02468e84729c5fbfca0160'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:05.363667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4478c47d7f02468e84729c5fbfca0160'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:05.397485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4478c47d7f02468e84729c5fbfca0160'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:05.398233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4478c47d7f02468e84729c5fbfca0160'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:05.416205 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3876/7666] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:05 2022] DELETE /identity/v3/users/4478c47d7f02468e84729c5fbfca0160 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:05.826142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a934225-ee8b-4f2c-8c9c-9da8cbc87993 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:05.840678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a934225-ee8b-4f2c-8c9c-9da8cbc87993 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:05.922615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:05.923246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:05.923728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:05.923997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] PATH_INFO: `/v3/projects/3181d5ac7c3743019399a71b6a4c7b62` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:05.925436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3181d5ac7c3743019399a71b6a4c7b62)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:05.927342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:06.004967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3181d5ac7c3743019399a71b6a4c7b62', 'request_id': 'req-84908c0d-96db-4bf2-819f-b693f7f7f41a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:06.027897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3181d5ac7c3743019399a71b6a4c7b62', 'request_id': 'req-84908c0d-96db-4bf2-819f-b693f7f7f41a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:06.030263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:06.041551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3181d5ac7c3743019399a71b6a4c7b62', 'request_id': 'req-84908c0d-96db-4bf2-819f-b693f7f7f41a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:06.057871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] The token cache is being invalidate because project 3181d5ac7c3743019399a71b6a4c7b62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:06.058274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:06.059132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:06.064153 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3791/7667] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:05 2022] DELETE /identity/v3/projects/3181d5ac7c3743019399a71b6a4c7b62 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:06.065732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:06.067525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:06.087107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912c558-4633-4555-b99c-5bf8ea5f57cc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:06.174603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f66f83-c195-4e45-b5bf-6bca8b78d816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:06.175297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f66f83-c195-4e45-b5bf-6bca8b78d816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:06.175517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f66f83-c195-4e45-b5bf-6bca8b78d816 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:06.175776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f66f83-c195-4e45-b5bf-6bca8b78d816 admin admin] PATH_INFO: `/v3/users/3654b89398f64723970ca4eb07b09dba` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:06.176993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f66f83-c195-4e45-b5bf-6bca8b78d816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3654b89398f64723970ca4eb07b09dba)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:06.178963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f66f83-c195-4e45-b5bf-6bca8b78d816 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:06.247822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d3f66f83-c195-4e45-b5bf-6bca8b78d816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3654b89398f64723970ca4eb07b09dba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:06.251850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:06.262792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d3f66f83-c195-4e45-b5bf-6bca8b78d816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3654b89398f64723970ca4eb07b09dba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:06.278226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d3f66f83-c195-4e45-b5bf-6bca8b78d816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3654b89398f64723970ca4eb07b09dba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:06.301539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d3f66f83-c195-4e45-b5bf-6bca8b78d816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3654b89398f64723970ca4eb07b09dba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:06.302189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d3f66f83-c195-4e45-b5bf-6bca8b78d816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3654b89398f64723970ca4eb07b09dba'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:06.310202 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3877/7668] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:06 2022] DELETE /identity/v3/users/3654b89398f64723970ca4eb07b09dba => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:06.820351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:06.821931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:06.841927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84908c0d-96db-4bf2-819f-b693f7f7f41a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:06.923266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd259a13-3903-4a77-939f-0c757589ea76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:06.923915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd259a13-3903-4a77-939f-0c757589ea76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:06.924422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd259a13-3903-4a77-939f-0c757589ea76 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:06.925522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd259a13-3903-4a77-939f-0c757589ea76 admin admin] PATH_INFO: `/v3/projects/4fe987d796744e8dbcfd6577ba9b61fe` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:06.927359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd259a13-3903-4a77-939f-0c757589ea76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fe987d796744e8dbcfd6577ba9b61fe)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:06.929747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd259a13-3903-4a77-939f-0c757589ea76 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:07.002111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd259a13-3903-4a77-939f-0c757589ea76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fe987d796744e8dbcfd6577ba9b61fe', 'request_id': 'req-cd259a13-3903-4a77-939f-0c757589ea76'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:07.012669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd259a13-3903-4a77-939f-0c757589ea76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fe987d796744e8dbcfd6577ba9b61fe', 'request_id': 'req-cd259a13-3903-4a77-939f-0c757589ea76'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:07.030092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd259a13-3903-4a77-939f-0c757589ea76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fe987d796744e8dbcfd6577ba9b61fe', 'request_id': 'req-cd259a13-3903-4a77-939f-0c757589ea76'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:07.052815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd259a13-3903-4a77-939f-0c757589ea76 admin admin] The token cache is being invalidate because project 4fe987d796744e8dbcfd6577ba9b61fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:07.053363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd259a13-3903-4a77-939f-0c757589ea76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:07.054216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd259a13-3903-4a77-939f-0c757589ea76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:07.056281 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3792/7669] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:06 2022] DELETE /identity/v3/projects/4fe987d796744e8dbcfd6577ba9b61fe => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:09.745893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ec246-4802-4813-b284-114cad930f86 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:09.746462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ec246-4802-4813-b284-114cad930f86 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:09.746462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ec246-4802-4813-b284-114cad930f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:09.776809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b48ec246-4802-4813-b284-114cad930f86 None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:09.778367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b48ec246-4802-4813-b284-114cad930f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:09.867126 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3878/7670] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:09.881034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4498b5-133d-4086-90b1-4a3d2fd1eebc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:09.882301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4498b5-133d-4086-90b1-4a3d2fd1eebc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:09.882901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4498b5-133d-4086-90b1-4a3d2fd1eebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:09.907511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-dc4498b5-133d-4086-90b1-4a3d2fd1eebc None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:09.909126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dc4498b5-133d-4086-90b1-4a3d2fd1eebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:09.917021 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3793/7671] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:09.928558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ec246-4802-4813-b284-114cad930f86 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:09.930101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b48ec246-4802-4813-b284-114cad930f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:09.969076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4498b5-133d-4086-90b1-4a3d2fd1eebc None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:09.971208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dc4498b5-133d-4086-90b1-4a3d2fd1eebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:10.029383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ec246-4802-4813-b284-114cad930f86 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:10.035697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697c5bb-e1f1-453a-91bd-5b5cc6c00de0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:10.036453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697c5bb-e1f1-453a-91bd-5b5cc6c00de0 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:10.036823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697c5bb-e1f1-453a-91bd-5b5cc6c00de0 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:10.037167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697c5bb-e1f1-453a-91bd-5b5cc6c00de0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:10.047250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697c5bb-e1f1-453a-91bd-5b5cc6c00de0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:10.049225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697c5bb-e1f1-453a-91bd-5b5cc6c00de0 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:10.054142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4498b5-133d-4086-90b1-4a3d2fd1eebc None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:10.058007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11afd2-6e12-4a16-8a35-0ebedeac3678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:10.059503 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3879/7672] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:26:10.061171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11afd2-6e12-4a16-8a35-0ebedeac3678 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:10.061422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11afd2-6e12-4a16-8a35-0ebedeac3678 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:10.061676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11afd2-6e12-4a16-8a35-0ebedeac3678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:10.063807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-eb11afd2-6e12-4a16-8a35-0ebedeac3678 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:10.091648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:10.145107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11afd2-6e12-4a16-8a35-0ebedeac3678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:10.146634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11afd2-6e12-4a16-8a35-0ebedeac3678 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:10.158098 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3794/7673] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:09 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:26:14.574357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861a0ee-84c2-40b0-aee5-837eac407d53 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:14.574908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861a0ee-84c2-40b0-aee5-837eac407d53 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:14.575108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861a0ee-84c2-40b0-aee5-837eac407d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:14.620103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1861a0ee-84c2-40b0-aee5-837eac407d53 None None] MFA Rules not processed for user `f2c2c4da3c954f96af706644f238ed97`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:14.621350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1861a0ee-84c2-40b0-aee5-837eac407d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:14.712060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:14.756719 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3880/7674] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:20.152995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=20 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:20.703580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11afd2-6e12-4a16-8a35-0ebedeac3678 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:20.721398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11afd2-6e12-4a16-8a35-0ebedeac3678 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:20.728025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bd89a-c132-4f7c-8887-da2f031e34e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:20.728926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bd89a-c132-4f7c-8887-da2f031e34e7 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:20.729199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bd89a-c132-4f7c-8887-da2f031e34e7 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:20.729464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bd89a-c132-4f7c-8887-da2f031e34e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:20.731309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b88bd89a-c132-4f7c-8887-da2f031e34e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:20.747582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bd89a-c132-4f7c-8887-da2f031e34e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:20.749324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bd89a-c132-4f7c-8887-da2f031e34e7 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:20.765922 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3795/7675] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:26:23.067944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bebd2c-512b-40c6-9539-68c8b7a50659 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:23.068488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bebd2c-512b-40c6-9539-68c8b7a50659 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:23.068488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bebd2c-512b-40c6-9539-68c8b7a50659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:23.091674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c5bebd2c-512b-40c6-9539-68c8b7a50659 None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:23.093170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c5bebd2c-512b-40c6-9539-68c8b7a50659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:23.101207 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3881/7676] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:30.755670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:33.085436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=9 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:36.506464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bd89a-c132-4f7c-8887-da2f031e34e7 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:36.507883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b88bd89a-c132-4f7c-8887-da2f031e34e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:36.581815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bd89a-c132-4f7c-8887-da2f031e34e7 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:36.586702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb1f86-d7c3-4b20-9b88-a8ee862e97e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:36.587341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb1f86-d7c3-4b20-9b88-a8ee862e97e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:36.587609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb1f86-d7c3-4b20-9b88-a8ee862e97e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:36.587878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb1f86-d7c3-4b20-9b88-a8ee862e97e5 admin admin] PATH_INFO: `/v3/users/4e6a5f35d172410481a8416b7bdaafac` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:36.589065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb1f86-d7c3-4b20-9b88-a8ee862e97e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e6a5f35d172410481a8416b7bdaafac)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:36.590980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb1f86-d7c3-4b20-9b88-a8ee862e97e5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:36.639156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2feb1f86-d7c3-4b20-9b88-a8ee862e97e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e6a5f35d172410481a8416b7bdaafac'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:36.646930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2feb1f86-d7c3-4b20-9b88-a8ee862e97e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e6a5f35d172410481a8416b7bdaafac'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:36.658833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2feb1f86-d7c3-4b20-9b88-a8ee862e97e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e6a5f35d172410481a8416b7bdaafac'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:36.659594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2feb1f86-d7c3-4b20-9b88-a8ee862e97e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e6a5f35d172410481a8416b7bdaafac'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:36.660076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2feb1f86-d7c3-4b20-9b88-a8ee862e97e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e6a5f35d172410481a8416b7bdaafac'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:36.672331 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3796/7677] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:36 2022] DELETE /identity/v3/users/4e6a5f35d172410481a8416b7bdaafac => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:36.936484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bebd2c-512b-40c6-9539-68c8b7a50659 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:36.948709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bebd2c-512b-40c6-9539-68c8b7a50659 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:37.011558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89240906-a428-43c6-8a29-1dab544289cc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:37.011770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89240906-a428-43c6-8a29-1dab544289cc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:37.012014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89240906-a428-43c6-8a29-1dab544289cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:37.017384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:37.018136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:37.018416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:37.018641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82 admin admin] PATH_INFO: `/v3/projects/02bc53e8ace84e8582091742eed1bc76` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:37.020528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02bc53e8ace84e8582091742eed1bc76)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:37.023142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:37.032129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-89240906-a428-43c6-8a29-1dab544289cc None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:37.033359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-89240906-a428-43c6-8a29-1dab544289cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:37.072725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02bc53e8ace84e8582091742eed1bc76', 'request_id': 'req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:37.081374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02bc53e8ace84e8582091742eed1bc76', 'request_id': 'req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:37.087822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02bc53e8ace84e8582091742eed1bc76', 'request_id': 'req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:37.093794 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3797/7678] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:37.100269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82 admin admin] The token cache is being invalidate because project 02bc53e8ace84e8582091742eed1bc76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:37.100597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:37.101241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6dc8a0bf-41f5-476e-90fc-ed58c6d9be82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:37.102822 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3882/7679] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:36 2022] DELETE /identity/v3/projects/02bc53e8ace84e8582091742eed1bc76 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:37.108059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868a4c8-2943-474b-9c46-922ba7197837 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:37.118506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868a4c8-2943-474b-9c46-922ba7197837 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:37.118506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868a4c8-2943-474b-9c46-922ba7197837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:37.133249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3868a4c8-2943-474b-9c46-922ba7197837 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:37.134427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3868a4c8-2943-474b-9c46-922ba7197837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:37.153682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:37.199366 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3798/7680] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:37.208233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b771c-5b5d-4977-90ec-799cc4b2463e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:37.209171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b771c-5b5d-4977-90ec-799cc4b2463e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:37.209171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b771c-5b5d-4977-90ec-799cc4b2463e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:37.230686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4e6b771c-5b5d-4977-90ec-799cc4b2463e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:37.231877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4e6b771c-5b5d-4977-90ec-799cc4b2463e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:37.241168 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3883/7681] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:37.245841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868a4c8-2943-474b-9c46-922ba7197837 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:37.257158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868a4c8-2943-474b-9c46-922ba7197837 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:37.262417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040efcc-f122-46e9-b5a3-069ecee1fa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:37.262960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040efcc-f122-46e9-b5a3-069ecee1fa50 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:37.263235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040efcc-f122-46e9-b5a3-069ecee1fa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:37.263503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040efcc-f122-46e9-b5a3-069ecee1fa50 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:37.264141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040efcc-f122-46e9-b5a3-069ecee1fa50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:37.264517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040efcc-f122-46e9-b5a3-069ecee1fa50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:37.266679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040efcc-f122-46e9-b5a3-069ecee1fa50 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:37.278699 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3799/7682] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:37.283325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b771c-5b5d-4977-90ec-799cc4b2463e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:37.292721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b771c-5b5d-4977-90ec-799cc4b2463e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:37.297564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8a32f-c6f7-4641-a230-eab37459bd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:37.298153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8a32f-c6f7-4641-a230-eab37459bd2a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:37.298616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8a32f-c6f7-4641-a230-eab37459bd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:37.298739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8a32f-c6f7-4641-a230-eab37459bd2a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:37.299400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8a32f-c6f7-4641-a230-eab37459bd2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:37.301523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8a32f-c6f7-4641-a230-eab37459bd2a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:37.334043 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3884/7683] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:37 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:37.335403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040efcc-f122-46e9-b5a3-069ecee1fa50 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:37.346011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040efcc-f122-46e9-b5a3-069ecee1fa50 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:37.407891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8f83d-f3aa-410d-8da9-c0063a6f0559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:37.408537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8f83d-f3aa-410d-8da9-c0063a6f0559 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:37.408814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8f83d-f3aa-410d-8da9-c0063a6f0559 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:37.409101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8f83d-f3aa-410d-8da9-c0063a6f0559 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:37.409698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8f83d-f3aa-410d-8da9-c0063a6f0559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:37.411628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8f83d-f3aa-410d-8da9-c0063a6f0559 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:37.436109 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3800/7684] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:37 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:37.437589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8a32f-c6f7-4641-a230-eab37459bd2a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:37.448819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8a32f-c6f7-4641-a230-eab37459bd2a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:37.454073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da143b62-09aa-4a5a-9f78-abf2fbe1d865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:37.454677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da143b62-09aa-4a5a-9f78-abf2fbe1d865 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:37.454947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da143b62-09aa-4a5a-9f78-abf2fbe1d865 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:37.455209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da143b62-09aa-4a5a-9f78-abf2fbe1d865 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:37.455845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da143b62-09aa-4a5a-9f78-abf2fbe1d865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:37.457496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da143b62-09aa-4a5a-9f78-abf2fbe1d865 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:37.472417 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3885/7685] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:37.473248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8f83d-f3aa-410d-8da9-c0063a6f0559 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:37.483570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8f83d-f3aa-410d-8da9-c0063a6f0559 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:37.489639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd06d34-00a9-4e60-83b4-df52b147bd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:37.490488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd06d34-00a9-4e60-83b4-df52b147bd10 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:37.490823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd06d34-00a9-4e60-83b4-df52b147bd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:37.491225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd06d34-00a9-4e60-83b4-df52b147bd10 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:37.491928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd06d34-00a9-4e60-83b4-df52b147bd10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:37.493746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd06d34-00a9-4e60-83b4-df52b147bd10 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:37.508118 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3801/7686] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:26:37.509270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da143b62-09aa-4a5a-9f78-abf2fbe1d865 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:37.519492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da143b62-09aa-4a5a-9f78-abf2fbe1d865 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:37.524124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab9946-1055-4756-aa8e-67b5b38684c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:37.525058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab9946-1055-4756-aa8e-67b5b38684c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:37.525058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab9946-1055-4756-aa8e-67b5b38684c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:37.525319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab9946-1055-4756-aa8e-67b5b38684c7 admin admin] PATH_INFO: `/v3/projects/e488c18cbc614d649b91ff675a90c8ef/users/f3bdde342aa1404aa01274add936634b/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:37.540186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab9946-1055-4756-aa8e-67b5b38684c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e488c18cbc614d649b91ff675a90c8ef, user_id=f3bdde342aa1404aa01274add936634b, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:37.542951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab9946-1055-4756-aa8e-67b5b38684c7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:37.557416 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3886/7687] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:37 2022] PUT /identity/v3/projects/e488c18cbc614d649b91ff675a90c8ef/users/f3bdde342aa1404aa01274add936634b/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:26:37.568017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c569d6-7239-4079-b9a2-cecbae666eab None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:37.568298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c569d6-7239-4079-b9a2-cecbae666eab None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:37.568593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c569d6-7239-4079-b9a2-cecbae666eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:37.597578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-50c569d6-7239-4079-b9a2-cecbae666eab None None] MFA Rules not processed for user `f3bdde342aa1404aa01274add936634b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:37.598785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-50c569d6-7239-4079-b9a2-cecbae666eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:37.676433 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3802/7688] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:37.687949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab9946-1055-4756-aa8e-67b5b38684c7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:37.691020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-75ab9946-1055-4756-aa8e-67b5b38684c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:37.727059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:37.797706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab9946-1055-4756-aa8e-67b5b38684c7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:37.806655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a3e0f-6f6a-4f27-ad73-5401d1e3fd0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:37.807330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a3e0f-6f6a-4f27-ad73-5401d1e3fd0f service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:37.807668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a3e0f-6f6a-4f27-ad73-5401d1e3fd0f service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:37.807988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a3e0f-6f6a-4f27-ad73-5401d1e3fd0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:37.817616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a3e0f-6f6a-4f27-ad73-5401d1e3fd0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:37.819209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a3e0f-6f6a-4f27-ad73-5401d1e3fd0f service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:37.828883 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3887/7689] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:26:37.873557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c569d6-7239-4079-b9a2-cecbae666eab None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:37.895734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:37.955133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c569d6-7239-4079-b9a2-cecbae666eab None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:37.961002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:37.961646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:37.961912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:37.962178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] PATH_INFO: `/v3/users/f3bdde342aa1404aa01274add936634b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:37.963446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3bdde342aa1404aa01274add936634b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:37.965461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:38.015346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3bdde342aa1404aa01274add936634b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:38.023566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3bdde342aa1404aa01274add936634b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:38.036718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3bdde342aa1404aa01274add936634b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:38.037359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3bdde342aa1404aa01274add936634b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:38.037956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3bdde342aa1404aa01274add936634b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:38.050373 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3803/7690] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:37 2022] DELETE /identity/v3/users/f3bdde342aa1404aa01274add936634b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:38.055021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3178d5-c624-470d-9719-4a26abeacc86 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:38.055508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3178d5-c624-470d-9719-4a26abeacc86 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:38.055626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3178d5-c624-470d-9719-4a26abeacc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:38.076455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-db3178d5-c624-470d-9719-4a26abeacc86 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:38.077631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-db3178d5-c624-470d-9719-4a26abeacc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:38.147299 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3888/7691] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:38.160014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:38.161535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:38.231532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27385f2c-b6cc-455c-bb10-826bc7f5fd83 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:38.236658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db6289-cb1a-4b8a-a19d-4319d4931848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:38.236658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db6289-cb1a-4b8a-a19d-4319d4931848 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:38.236927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db6289-cb1a-4b8a-a19d-4319d4931848 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:38.237209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db6289-cb1a-4b8a-a19d-4319d4931848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:38.247308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db6289-cb1a-4b8a-a19d-4319d4931848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:38.248995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db6289-cb1a-4b8a-a19d-4319d4931848 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:38.258409 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3804/7692] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:26:38.542265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3178d5-c624-470d-9719-4a26abeacc86 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:38.543813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-db3178d5-c624-470d-9719-4a26abeacc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:38.557506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3178d5-c624-470d-9719-4a26abeacc86 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:38.562799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5f396-e616-44f0-a06e-6c0a347dd516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:38.563432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5f396-e616-44f0-a06e-6c0a347dd516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:38.564601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5f396-e616-44f0-a06e-6c0a347dd516 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:38.564871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5f396-e616-44f0-a06e-6c0a347dd516 admin admin] PATH_INFO: `/v3/projects/e488c18cbc614d649b91ff675a90c8ef` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:38.568487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5f396-e616-44f0-a06e-6c0a347dd516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e488c18cbc614d649b91ff675a90c8ef)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:38.571257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5f396-e616-44f0-a06e-6c0a347dd516 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:38.623124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3ae5f396-e616-44f0-a06e-6c0a347dd516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e488c18cbc614d649b91ff675a90c8ef', 'request_id': 'req-3ae5f396-e616-44f0-a06e-6c0a347dd516'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:38.631013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3ae5f396-e616-44f0-a06e-6c0a347dd516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e488c18cbc614d649b91ff675a90c8ef', 'request_id': 'req-3ae5f396-e616-44f0-a06e-6c0a347dd516'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:38.637008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3ae5f396-e616-44f0-a06e-6c0a347dd516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e488c18cbc614d649b91ff675a90c8ef', 'request_id': 'req-3ae5f396-e616-44f0-a06e-6c0a347dd516'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:38.649430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3ae5f396-e616-44f0-a06e-6c0a347dd516 admin admin] The token cache is being invalidate because project e488c18cbc614d649b91ff675a90c8ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:38.649715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3ae5f396-e616-44f0-a06e-6c0a347dd516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:38.650428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3ae5f396-e616-44f0-a06e-6c0a347dd516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:38.652048 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3889/7693] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:38 2022] DELETE /identity/v3/projects/e488c18cbc614d649b91ff675a90c8ef => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:38.659102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea1b42-64ff-4dc9-8c83-6856cb67afe2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:38.659583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea1b42-64ff-4dc9-8c83-6856cb67afe2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:38.659697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea1b42-64ff-4dc9-8c83-6856cb67afe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:38.681807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e4ea1b42-64ff-4dc9-8c83-6856cb67afe2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:38.683021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e4ea1b42-64ff-4dc9-8c83-6856cb67afe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:38.725571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:38.748469 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3805/7694] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:38.758244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41628b2-8b08-4304-b642-266bb95685d4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:38.758484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41628b2-8b08-4304-b642-266bb95685d4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:38.758771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41628b2-8b08-4304-b642-266bb95685d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:38.780933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b41628b2-8b08-4304-b642-266bb95685d4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:38.782139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b41628b2-8b08-4304-b642-266bb95685d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:38.793822 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3890/7695] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:38.800004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea1b42-64ff-4dc9-8c83-6856cb67afe2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:38.814128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea1b42-64ff-4dc9-8c83-6856cb67afe2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:38.819966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794e6b5-c81c-481a-838b-7fc127b3693c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:38.820684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794e6b5-c81c-481a-838b-7fc127b3693c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:38.821021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794e6b5-c81c-481a-838b-7fc127b3693c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:38.821351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794e6b5-c81c-481a-838b-7fc127b3693c admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:38.822175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794e6b5-c81c-481a-838b-7fc127b3693c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:38.822677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794e6b5-c81c-481a-838b-7fc127b3693c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:38.824129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794e6b5-c81c-481a-838b-7fc127b3693c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:38.838914 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3806/7696] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:38.840786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41628b2-8b08-4304-b642-266bb95685d4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:38.851246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41628b2-8b08-4304-b642-266bb95685d4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:38.856508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b29093-ab24-48df-9d74-7ec667578b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:38.857176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b29093-ab24-48df-9d74-7ec667578b51 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:38.857503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b29093-ab24-48df-9d74-7ec667578b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:38.857831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b29093-ab24-48df-9d74-7ec667578b51 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:38.858565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b29093-ab24-48df-9d74-7ec667578b51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:38.860606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b29093-ab24-48df-9d74-7ec667578b51 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:38.891882 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3891/7697] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:38 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:38.892943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794e6b5-c81c-481a-838b-7fc127b3693c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:38.904792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794e6b5-c81c-481a-838b-7fc127b3693c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:38.968449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd955f5-d35f-460d-8ca5-2565d454f357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:38.968700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd955f5-d35f-460d-8ca5-2565d454f357 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:38.969064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd955f5-d35f-460d-8ca5-2565d454f357 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:38.969248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd955f5-d35f-460d-8ca5-2565d454f357 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:38.969883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd955f5-d35f-460d-8ca5-2565d454f357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:38.971866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd955f5-d35f-460d-8ca5-2565d454f357 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:38.994192 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3807/7698] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:38 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:38.998207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b29093-ab24-48df-9d74-7ec667578b51 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:39.008626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b29093-ab24-48df-9d74-7ec667578b51 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:39.013167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec6ccf-be3c-47d7-baf1-c33cec893af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:39.013760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec6ccf-be3c-47d7-baf1-c33cec893af5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:39.014027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec6ccf-be3c-47d7-baf1-c33cec893af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:39.014293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec6ccf-be3c-47d7-baf1-c33cec893af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:39.015242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec6ccf-be3c-47d7-baf1-c33cec893af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:39.016923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec6ccf-be3c-47d7-baf1-c33cec893af5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:39.018295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:39.031913 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3892/7699] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:39.035665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd955f5-d35f-460d-8ca5-2565d454f357 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:39.047338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd955f5-d35f-460d-8ca5-2565d454f357 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:39.052632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de490a36-a206-44ff-a6f3-7a9911358e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:39.053236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de490a36-a206-44ff-a6f3-7a9911358e75 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:39.053505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de490a36-a206-44ff-a6f3-7a9911358e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:39.053769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de490a36-a206-44ff-a6f3-7a9911358e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:39.054405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de490a36-a206-44ff-a6f3-7a9911358e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:39.059012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de490a36-a206-44ff-a6f3-7a9911358e75 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:39.070592 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3808/7700] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:39.074756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec6ccf-be3c-47d7-baf1-c33cec893af5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:39.086541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec6ccf-be3c-47d7-baf1-c33cec893af5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:39.092347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d53ac1-79be-43ee-a7d1-525906bdc6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:39.093867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d53ac1-79be-43ee-a7d1-525906bdc6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:39.094192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d53ac1-79be-43ee-a7d1-525906bdc6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:39.094463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d53ac1-79be-43ee-a7d1-525906bdc6fa admin admin] PATH_INFO: `/v3/projects/4958af50de344beaa5281f706a836cfd/users/1c1b9f7d61144a1baa7bb9b516c7b112/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:39.113173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d53ac1-79be-43ee-a7d1-525906bdc6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4958af50de344beaa5281f706a836cfd, user_id=1c1b9f7d61144a1baa7bb9b516c7b112, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:39.115810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d53ac1-79be-43ee-a7d1-525906bdc6fa admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:39.130958 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3893/7701] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:39 2022] PUT /identity/v3/projects/4958af50de344beaa5281f706a836cfd/users/1c1b9f7d61144a1baa7bb9b516c7b112/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:39.146092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24733d54-99cd-4a32-9567-421367f1e8ee None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:39.146464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24733d54-99cd-4a32-9567-421367f1e8ee None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:39.146832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24733d54-99cd-4a32-9567-421367f1e8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:39.177233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-24733d54-99cd-4a32-9567-421367f1e8ee None None] MFA Rules not processed for user `1c1b9f7d61144a1baa7bb9b516c7b112`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:39.178485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-24733d54-99cd-4a32-9567-421367f1e8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:39.255653 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3809/7702] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:39.260225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d53ac1-79be-43ee-a7d1-525906bdc6fa admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:39.323576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d53ac1-79be-43ee-a7d1-525906bdc6fa admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:39.328358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fa726-4267-40d7-8d99-7e03bfdd18d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:39.328982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fa726-4267-40d7-8d99-7e03bfdd18d7 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:39.329252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fa726-4267-40d7-8d99-7e03bfdd18d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:39.329515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fa726-4267-40d7-8d99-7e03bfdd18d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:39.330126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fa726-4267-40d7-8d99-7e03bfdd18d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:39.332021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fa726-4267-40d7-8d99-7e03bfdd18d7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:39.358900 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3894/7703] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:39 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:39.363374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24733d54-99cd-4a32-9567-421367f1e8ee None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:39.375533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24733d54-99cd-4a32-9567-421367f1e8ee None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:39.413800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:39.439102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f5b00-b165-4889-9903-5fa307bb3696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:39.440067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f5b00-b165-4889-9903-5fa307bb3696 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:39.440343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f5b00-b165-4889-9903-5fa307bb3696 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:39.440645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f5b00-b165-4889-9903-5fa307bb3696 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:39.441281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f5b00-b165-4889-9903-5fa307bb3696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:39.443620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f5b00-b165-4889-9903-5fa307bb3696 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:39.471989 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3810/7704] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:39 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:39.473270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fa726-4267-40d7-8d99-7e03bfdd18d7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:39.485543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fa726-4267-40d7-8d99-7e03bfdd18d7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:39.490488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770fc55-2705-481f-9545-8c221b9e909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:39.491280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770fc55-2705-481f-9545-8c221b9e909d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:39.491820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770fc55-2705-481f-9545-8c221b9e909d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:39.492213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770fc55-2705-481f-9545-8c221b9e909d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:39.492946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770fc55-2705-481f-9545-8c221b9e909d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:39.507767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770fc55-2705-481f-9545-8c221b9e909d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:39.507767 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3895/7705] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:39.511560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f5b00-b165-4889-9903-5fa307bb3696 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:39.522155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f5b00-b165-4889-9903-5fa307bb3696 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:39.527287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0d0d5-2ea5-4025-8472-bbc978d196d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:39.527953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0d0d5-2ea5-4025-8472-bbc978d196d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:39.528227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0d0d5-2ea5-4025-8472-bbc978d196d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:39.528538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0d0d5-2ea5-4025-8472-bbc978d196d3 admin admin] PATH_INFO: `/v3/projects/9274bdbb99d940f9976a19993893233a/users/a937ce6461a54ab488e3644f40c6ba49/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:39.545268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0d0d5-2ea5-4025-8472-bbc978d196d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9274bdbb99d940f9976a19993893233a, user_id=a937ce6461a54ab488e3644f40c6ba49, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:39.548765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0d0d5-2ea5-4025-8472-bbc978d196d3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:39.563700 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3811/7706] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:39 2022] PUT /identity/v3/projects/9274bdbb99d940f9976a19993893233a/users/a937ce6461a54ab488e3644f40c6ba49/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:39.568127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770fc55-2705-481f-9545-8c221b9e909d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:39.578209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770fc55-2705-481f-9545-8c221b9e909d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:39.635556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e30dc-0ae8-4c4a-83a3-7c2393c1b189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:39.636167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e30dc-0ae8-4c4a-83a3-7c2393c1b189 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:39.636478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e30dc-0ae8-4c4a-83a3-7c2393c1b189 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:39.636807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e30dc-0ae8-4c4a-83a3-7c2393c1b189 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:39.637378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e30dc-0ae8-4c4a-83a3-7c2393c1b189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:39.638981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e30dc-0ae8-4c4a-83a3-7c2393c1b189 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:39.649304 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3896/7707] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:39.653698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0d0d5-2ea5-4025-8472-bbc978d196d3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:39.665268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0d0d5-2ea5-4025-8472-bbc978d196d3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:39.670318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41391b4-d3b8-409b-9424-71de0ca44003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:39.671172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41391b4-d3b8-409b-9424-71de0ca44003 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:39.671454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41391b4-d3b8-409b-9424-71de0ca44003 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:39.671716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41391b4-d3b8-409b-9424-71de0ca44003 admin admin] PATH_INFO: `/v3/projects/9274bdbb99d940f9976a19993893233a/users/a937ce6461a54ab488e3644f40c6ba49/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:39.674024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41391b4-d3b8-409b-9424-71de0ca44003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9274bdbb99d940f9976a19993893233a, user_id=a937ce6461a54ab488e3644f40c6ba49, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:39.676743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41391b4-d3b8-409b-9424-71de0ca44003 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:39.688120 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3812/7708] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:39 2022] PUT /identity/v3/projects/9274bdbb99d940f9976a19993893233a/users/a937ce6461a54ab488e3644f40c6ba49/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:39.698795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c6bf5-a14a-4f70-97e1-00be11425e25 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:39.699074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c6bf5-a14a-4f70-97e1-00be11425e25 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:39.699337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c6bf5-a14a-4f70-97e1-00be11425e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:39.729013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4c2c6bf5-a14a-4f70-97e1-00be11425e25 None None] MFA Rules not processed for user `a937ce6461a54ab488e3644f40c6ba49`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:39.730222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4c2c6bf5-a14a-4f70-97e1-00be11425e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:39.761794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:39.800168 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3897/7709] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:39.804644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41391b4-d3b8-409b-9424-71de0ca44003 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:39.816900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41391b4-d3b8-409b-9424-71de0ca44003 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:39.823461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f3876-422a-4f35-9fb4-a2731540d49b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:39.824052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f3876-422a-4f35-9fb4-a2731540d49b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:39.824325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f3876-422a-4f35-9fb4-a2731540d49b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:39.824637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f3876-422a-4f35-9fb4-a2731540d49b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:39.825245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f3876-422a-4f35-9fb4-a2731540d49b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:39.827112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f3876-422a-4f35-9fb4-a2731540d49b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:39.863130 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3813/7710] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:39 2022] POST /identity/v3/projects => generated 351 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:39.867560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c6bf5-a14a-4f70-97e1-00be11425e25 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:39.878551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c6bf5-a14a-4f70-97e1-00be11425e25 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:39.940580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304d157-2ff2-455b-aa63-89f79dccd417 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:39.941356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304d157-2ff2-455b-aa63-89f79dccd417 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:39.941629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304d157-2ff2-455b-aa63-89f79dccd417 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:39.941892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304d157-2ff2-455b-aa63-89f79dccd417 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:39.942845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304d157-2ff2-455b-aa63-89f79dccd417 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:39.944714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304d157-2ff2-455b-aa63-89f79dccd417 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:39.983014 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3898/7711] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:26:39 2022] POST /identity/v3/projects => generated 349 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:39.987787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f3876-422a-4f35-9fb4-a2731540d49b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:39.999284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f3876-422a-4f35-9fb4-a2731540d49b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:40.056637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514308f5-a382-4ffa-b74c-cce07eed23e3 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:40.057263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514308f5-a382-4ffa-b74c-cce07eed23e3 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.057532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514308f5-a382-4ffa-b74c-cce07eed23e3 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.057796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514308f5-a382-4ffa-b74c-cce07eed23e3 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/users/1c1b9f7d61144a1baa7bb9b516c7b112` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.058960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514308f5-a382-4ffa-b74c-cce07eed23e3 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:get_user(user_id=1c1b9f7d61144a1baa7bb9b516c7b112)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.061441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514308f5-a382-4ffa-b74c-cce07eed23e3 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.062815 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3814/7712] 10.222.0.23 () {58 vars in 1221 bytes} [Sat Aug 20 21:26:39 2022] GET /identity/v3/users/1c1b9f7d61144a1baa7bb9b516c7b112 => generated 287 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:40.068277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304d157-2ff2-455b-aa63-89f79dccd417 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:40.079648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304d157-2ff2-455b-aa63-89f79dccd417 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:40.084716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc38c0-bc8a-4d5f-b208-28003bf37b82 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:40.085309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc38c0-bc8a-4d5f-b208-28003bf37b82 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.085580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc38c0-bc8a-4d5f-b208-28003bf37b82 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.085844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc38c0-bc8a-4d5f-b208-28003bf37b82 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.086504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc38c0-bc8a-4d5f-b208-28003bf37b82 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.088333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc38c0-bc8a-4d5f-b208-28003bf37b82 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.106985 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3899/7713] 10.222.0.23 () {60 vars in 1162 bytes} [Sat Aug 20 21:26:40 2022] POST /identity/v3/credentials => generated 429 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:40.111505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514308f5-a382-4ffa-b74c-cce07eed23e3 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:40.122377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514308f5-a382-4ffa-b74c-cce07eed23e3 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:40.128684 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6040197-d654-409a-8844-d3c87746a847 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:40.129318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6040197-d654-409a-8844-d3c87746a847 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.129590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6040197-d654-409a-8844-d3c87746a847 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.129852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6040197-d654-409a-8844-d3c87746a847 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/credentials/7789a66300111a7359aa0a11ba14040f622354130d8f2a5456d5822b4a58e5a4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.132292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6040197-d654-409a-8844-d3c87746a847 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=7789a66300111a7359aa0a11ba14040f622354130d8f2a5456d5822b4a58e5a4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.134150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6040197-d654-409a-8844-d3c87746a847 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.137163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6040197-d654-409a-8844-d3c87746a847 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=7789a66300111a7359aa0a11ba14040f622354130d8f2a5456d5822b4a58e5a4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.138731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6040197-d654-409a-8844-d3c87746a847 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.154642 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3815/7714] 10.222.0.23 () {60 vars in 1358 bytes} [Sat Aug 20 21:26:40 2022] PATCH /identity/v3/credentials/7789a66300111a7359aa0a11ba14040f622354130d8f2a5456d5822b4a58e5a4 => generated 437 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:40.159167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc38c0-bc8a-4d5f-b208-28003bf37b82 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:40.170122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc38c0-bc8a-4d5f-b208-28003bf37b82 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:40.175610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da5ff2-6189-4676-83d8-83c814090892 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:40.176285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da5ff2-6189-4676-83d8-83c814090892 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.176642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da5ff2-6189-4676-83d8-83c814090892 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.176910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da5ff2-6189-4676-83d8-83c814090892 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/credentials/7789a66300111a7359aa0a11ba14040f622354130d8f2a5456d5822b4a58e5a4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.178929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da5ff2-6189-4676-83d8-83c814090892 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:get_credential(credential_id=7789a66300111a7359aa0a11ba14040f622354130d8f2a5456d5822b4a58e5a4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.180696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da5ff2-6189-4676-83d8-83c814090892 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.182835 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3900/7715] 10.222.0.23 () {58 vars in 1335 bytes} [Sat Aug 20 21:26:40 2022] GET /identity/v3/credentials/7789a66300111a7359aa0a11ba14040f622354130d8f2a5456d5822b4a58e5a4 => generated 437 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:40.187248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6040197-d654-409a-8844-d3c87746a847 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:40.198185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6040197-d654-409a-8844-d3c87746a847 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:40.203531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61ba3e-2ead-4795-8a2d-82c7ffd9e14e tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:40.204154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61ba3e-2ead-4795-8a2d-82c7ffd9e14e tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.204459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61ba3e-2ead-4795-8a2d-82c7ffd9e14e tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.204727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61ba3e-2ead-4795-8a2d-82c7ffd9e14e tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/credentials/7789a66300111a7359aa0a11ba14040f622354130d8f2a5456d5822b4a58e5a4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.206675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61ba3e-2ead-4795-8a2d-82c7ffd9e14e tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=7789a66300111a7359aa0a11ba14040f622354130d8f2a5456d5822b4a58e5a4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.208461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61ba3e-2ead-4795-8a2d-82c7ffd9e14e tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.227123 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3816/7716] 10.222.0.23 () {58 vars in 1338 bytes} [Sat Aug 20 21:26:40 2022] DELETE /identity/v3/credentials/7789a66300111a7359aa0a11ba14040f622354130d8f2a5456d5822b4a58e5a4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:26:40.232516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da5ff2-6189-4676-83d8-83c814090892 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:40.243095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da5ff2-6189-4676-83d8-83c814090892 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:40.248208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0794f2-5a08-43d6-b2cd-da784e672f90 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:40.248959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0794f2-5a08-43d6-b2cd-da784e672f90 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.249262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0794f2-5a08-43d6-b2cd-da784e672f90 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.249549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0794f2-5a08-43d6-b2cd-da784e672f90 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.250174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0794f2-5a08-43d6-b2cd-da784e672f90 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.251917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0794f2-5a08-43d6-b2cd-da784e672f90 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.270669 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3901/7717] 10.222.0.23 () {60 vars in 1162 bytes} [Sat Aug 20 21:26:40 2022] POST /identity/v3/credentials => generated 430 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:40.274905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61ba3e-2ead-4795-8a2d-82c7ffd9e14e tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:40.285797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61ba3e-2ead-4795-8a2d-82c7ffd9e14e tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:40.290946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b077d2-5158-4226-8385-9402068de7ad tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:40.291535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b077d2-5158-4226-8385-9402068de7ad tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.291808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b077d2-5158-4226-8385-9402068de7ad tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.292115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b077d2-5158-4226-8385-9402068de7ad tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/credentials` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.292785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b077d2-5158-4226-8385-9402068de7ad tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.294543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b077d2-5158-4226-8385-9402068de7ad tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.314712 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3817/7718] 10.222.0.23 () {60 vars in 1162 bytes} [Sat Aug 20 21:26:40 2022] POST /identity/v3/credentials => generated 430 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:40.318575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0794f2-5a08-43d6-b2cd-da784e672f90 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:40.329662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0794f2-5a08-43d6-b2cd-da784e672f90 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:40.335170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f871587-9485-4f4b-ae2e-8f2f9d14f634 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:40.335785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f871587-9485-4f4b-ae2e-8f2f9d14f634 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.336055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f871587-9485-4f4b-ae2e-8f2f9d14f634 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.336321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f871587-9485-4f4b-ae2e-8f2f9d14f634 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/credentials` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.336935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f871587-9485-4f4b-ae2e-8f2f9d14f634 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:list_credentials()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.338476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f871587-9485-4f4b-ae2e-8f2f9d14f634 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.345860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f871587-9485-4f4b-ae2e-8f2f9d14f634 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.347481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f871587-9485-4f4b-ae2e-8f2f9d14f634 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.349049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f871587-9485-4f4b-ae2e-8f2f9d14f634 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.350617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f871587-9485-4f4b-ae2e-8f2f9d14f634 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.352428 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3902/7719] 10.222.0.23 () {58 vars in 1140 bytes} [Sat Aug 20 21:26:40 2022] GET /identity/v3/credentials => generated 945 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:40.356696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b077d2-5158-4226-8385-9402068de7ad tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:40.369502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b077d2-5158-4226-8385-9402068de7ad tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:40.377154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c16364-b5a9-43a7-b7d7-c2d308497d36 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:40.377778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c16364-b5a9-43a7-b7d7-c2d308497d36 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.378055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c16364-b5a9-43a7-b7d7-c2d308497d36 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.378318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c16364-b5a9-43a7-b7d7-c2d308497d36 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/credentials/b21c91cd6b000a210027bdb6855f6a54df6f90c90ad4b692596c5f01d532ff49` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.380314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c16364-b5a9-43a7-b7d7-c2d308497d36 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=b21c91cd6b000a210027bdb6855f6a54df6f90c90ad4b692596c5f01d532ff49)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.382068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c16364-b5a9-43a7-b7d7-c2d308497d36 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.402377 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3818/7720] 10.222.0.23 () {58 vars in 1338 bytes} [Sat Aug 20 21:26:40 2022] DELETE /identity/v3/credentials/b21c91cd6b000a210027bdb6855f6a54df6f90c90ad4b692596c5f01d532ff49 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:40.407734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f871587-9485-4f4b-ae2e-8f2f9d14f634 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:40.418187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f871587-9485-4f4b-ae2e-8f2f9d14f634 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:40.423354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d3ffb-050a-4c13-b9f4-0aeaf7c53a3b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:40.424004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d3ffb-050a-4c13-b9f4-0aeaf7c53a3b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.424284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d3ffb-050a-4c13-b9f4-0aeaf7c53a3b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.424586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d3ffb-050a-4c13-b9f4-0aeaf7c53a3b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/credentials/c6dbc577eaacfe5e2e5dcafabcfcae1ea7f3b19da60d2a7324f84b80dd254b74` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.426685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d3ffb-050a-4c13-b9f4-0aeaf7c53a3b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=c6dbc577eaacfe5e2e5dcafabcfcae1ea7f3b19da60d2a7324f84b80dd254b74)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.428469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d3ffb-050a-4c13-b9f4-0aeaf7c53a3b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.446476 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3903/7721] 10.222.0.23 () {58 vars in 1338 bytes} [Sat Aug 20 21:26:40 2022] DELETE /identity/v3/credentials/c6dbc577eaacfe5e2e5dcafabcfcae1ea7f3b19da60d2a7324f84b80dd254b74 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:40.451297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c16364-b5a9-43a7-b7d7-c2d308497d36 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:40.461481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c16364-b5a9-43a7-b7d7-c2d308497d36 tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:40.466655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:40.467271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.467585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.468135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/projects/5bc52ca0c23c4a70a99386516497d3ea` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.469169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5bc52ca0c23c4a70a99386516497d3ea)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.471150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.482321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:INSERT=7,keystone:UPDATE=1,keystone:DELETE=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:40.520687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc52ca0c23c4a70a99386516497d3ea', 'request_id': 'req-63b4a92f-f13a-4175-854f-31391f068b8b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.527497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc52ca0c23c4a70a99386516497d3ea', 'request_id': 'req-63b4a92f-f13a-4175-854f-31391f068b8b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.536238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc52ca0c23c4a70a99386516497d3ea', 'request_id': 'req-63b4a92f-f13a-4175-854f-31391f068b8b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.548171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] The token cache is being invalidate because project 5bc52ca0c23c4a70a99386516497d3ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:40.548345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.549168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.550785 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3819/7722] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:40 2022] DELETE /identity/v3/projects/5bc52ca0c23c4a70a99386516497d3ea => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:40.554705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d3ffb-050a-4c13-b9f4-0aeaf7c53a3b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:40.556620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b15d3ffb-050a-4c13-b9f4-0aeaf7c53a3b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:40.570167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d3ffb-050a-4c13-b9f4-0aeaf7c53a3b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:40.626441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:40.633463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a937ce6461a54ab488e3644f40c6ba49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9274bdbb99d940f9976a19993893233a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:40.634218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.634504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.634782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] PATH_INFO: `/v3/projects/8f2f4dff156d4f9e91cd223a8a964cb2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.636188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8f2f4dff156d4f9e91cd223a8a964cb2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.638732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.692640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f2f4dff156d4f9e91cd223a8a964cb2', 'request_id': 'req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.701038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f2f4dff156d4f9e91cd223a8a964cb2', 'request_id': 'req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.707114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f2f4dff156d4f9e91cd223a8a964cb2', 'request_id': 'req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.718927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] The token cache is being invalidate because project 8f2f4dff156d4f9e91cd223a8a964cb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:40.719367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.720295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1ed14fd6-5ecc-4b46-a6b3-003993bceadc tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.722627 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3904/7723] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:40 2022] DELETE /identity/v3/projects/8f2f4dff156d4f9e91cd223a8a964cb2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:40.726723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:40.728494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:40.804414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4a92f-f13a-4175-854f-31391f068b8b tempest-CredentialsTestJSON-818485019 tempest-CredentialsTestJSON-818485019-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:40.866407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542aef0-0e9a-4c57-ac81-3b96f5a8cfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:40.867150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542aef0-0e9a-4c57-ac81-3b96f5a8cfbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.867427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542aef0-0e9a-4c57-ac81-3b96f5a8cfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.867696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542aef0-0e9a-4c57-ac81-3b96f5a8cfbc admin admin] PATH_INFO: `/v3/users/1c1b9f7d61144a1baa7bb9b516c7b112` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.868919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542aef0-0e9a-4c57-ac81-3b96f5a8cfbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c1b9f7d61144a1baa7bb9b516c7b112)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:40.870847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542aef0-0e9a-4c57-ac81-3b96f5a8cfbc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:40.920817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3542aef0-0e9a-4c57-ac81-3b96f5a8cfbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c1b9f7d61144a1baa7bb9b516c7b112'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.923243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:40.930227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3542aef0-0e9a-4c57-ac81-3b96f5a8cfbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c1b9f7d61144a1baa7bb9b516c7b112'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.934792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bde7d-1b75-4660-b8aa-9c9b25c688aa None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.935070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bde7d-1b75-4660-b8aa-9c9b25c688aa None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.935331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bde7d-1b75-4660-b8aa-9c9b25c688aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.945005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3542aef0-0e9a-4c57-ac81-3b96f5a8cfbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c1b9f7d61144a1baa7bb9b516c7b112'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.945725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3542aef0-0e9a-4c57-ac81-3b96f5a8cfbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c1b9f7d61144a1baa7bb9b516c7b112'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.946326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3542aef0-0e9a-4c57-ac81-3b96f5a8cfbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c1b9f7d61144a1baa7bb9b516c7b112'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:40.956166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6b3bde7d-1b75-4660-b8aa-9c9b25c688aa None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:40.957611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6b3bde7d-1b75-4660-b8aa-9c9b25c688aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:40.959685 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3820/7724] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:40 2022] DELETE /identity/v3/users/1c1b9f7d61144a1baa7bb9b516c7b112 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:40.965390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe36bb0-b83d-47a2-9b79-2170dfc94f5d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:40.965667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe36bb0-b83d-47a2-9b79-2170dfc94f5d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:40.965938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe36bb0-b83d-47a2-9b79-2170dfc94f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:40.987670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-efe36bb0-b83d-47a2-9b79-2170dfc94f5d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:40.988919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-efe36bb0-b83d-47a2-9b79-2170dfc94f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:41.021708 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3905/7725] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:41.054125 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3821/7726] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:41.065254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bde7d-1b75-4660-b8aa-9c9b25c688aa None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:41.066919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6b3bde7d-1b75-4660-b8aa-9c9b25c688aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:41.131637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bde7d-1b75-4660-b8aa-9c9b25c688aa None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:41.135503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e0cd8-0bca-48b8-bce7-fa019db00cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:41.136240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e0cd8-0bca-48b8-bce7-fa019db00cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:41.136672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e0cd8-0bca-48b8-bce7-fa019db00cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:41.137039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e0cd8-0bca-48b8-bce7-fa019db00cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:41.148405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e0cd8-0bca-48b8-bce7-fa019db00cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:41.148864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e0cd8-0bca-48b8-bce7-fa019db00cf8 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:41.159476 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3906/7727] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:26:41.440604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a77aa7-6645-45ed-9b6a-57d1c124562c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:41.440887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a77aa7-6645-45ed-9b6a-57d1c124562c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:41.441157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a77aa7-6645-45ed-9b6a-57d1c124562c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:41.461627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c5a77aa7-6645-45ed-9b6a-57d1c124562c None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:41.462767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c5a77aa7-6645-45ed-9b6a-57d1c124562c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:41.471376 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3822/7728] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:41.500482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e0cd8-0bca-48b8-bce7-fa019db00cf8 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:41.501718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fa2e0cd8-0bca-48b8-bce7-fa019db00cf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:41.518637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e0cd8-0bca-48b8-bce7-fa019db00cf8 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:41.525023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e28137-002e-47f7-b3c7-dd3b9a4dd888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:41.525767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e28137-002e-47f7-b3c7-dd3b9a4dd888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:41.526105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e28137-002e-47f7-b3c7-dd3b9a4dd888 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:41.526438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e28137-002e-47f7-b3c7-dd3b9a4dd888 admin admin] PATH_INFO: `/v3/projects/4958af50de344beaa5281f706a836cfd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:41.527718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e28137-002e-47f7-b3c7-dd3b9a4dd888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4958af50de344beaa5281f706a836cfd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:41.529831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e28137-002e-47f7-b3c7-dd3b9a4dd888 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:41.568504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:41.583648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90e28137-002e-47f7-b3c7-dd3b9a4dd888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4958af50de344beaa5281f706a836cfd', 'request_id': 'req-90e28137-002e-47f7-b3c7-dd3b9a4dd888'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:41.592633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90e28137-002e-47f7-b3c7-dd3b9a4dd888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4958af50de344beaa5281f706a836cfd', 'request_id': 'req-90e28137-002e-47f7-b3c7-dd3b9a4dd888'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:41.598720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90e28137-002e-47f7-b3c7-dd3b9a4dd888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4958af50de344beaa5281f706a836cfd', 'request_id': 'req-90e28137-002e-47f7-b3c7-dd3b9a4dd888'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:41.610329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90e28137-002e-47f7-b3c7-dd3b9a4dd888 admin admin] The token cache is being invalidate because project 4958af50de344beaa5281f706a836cfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:41.610613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90e28137-002e-47f7-b3c7-dd3b9a4dd888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:41.611266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90e28137-002e-47f7-b3c7-dd3b9a4dd888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:41.613054 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3907/7729] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:41 2022] DELETE /identity/v3/projects/4958af50de344beaa5281f706a836cfd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:41.617024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a77aa7-6645-45ed-9b6a-57d1c124562c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:41.629080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a77aa7-6645-45ed-9b6a-57d1c124562c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:41.692905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:41.693266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:41.693542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:41.693811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] PATH_INFO: `/v3/users/a937ce6461a54ab488e3644f40c6ba49` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:41.694976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a937ce6461a54ab488e3644f40c6ba49)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:41.696934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:41.748142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a937ce6461a54ab488e3644f40c6ba49'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:41.755210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a937ce6461a54ab488e3644f40c6ba49'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:41.761329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:41.769839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a937ce6461a54ab488e3644f40c6ba49'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:41.771476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a937ce6461a54ab488e3644f40c6ba49'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:41.772494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a937ce6461a54ab488e3644f40c6ba49'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:41.786752 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3823/7730] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:41 2022] DELETE /identity/v3/users/a937ce6461a54ab488e3644f40c6ba49 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:26:41.871772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907ce69-45b9-4cce-9689-16ef681c644a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:41.872076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907ce69-45b9-4cce-9689-16ef681c644a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:41.872509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907ce69-45b9-4cce-9689-16ef681c644a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:41.893941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f907ce69-45b9-4cce-9689-16ef681c644a None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:41.895143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f907ce69-45b9-4cce-9689-16ef681c644a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:41.953129 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3908/7731] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:42.137848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:42.139394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:42.222853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85e1ba-e2c9-4d8c-8558-b7fd3e1e661a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:42.227551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:42.228183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:42.228492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:42.228756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c admin admin] PATH_INFO: `/v3/projects/9274bdbb99d940f9976a19993893233a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:42.229896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9274bdbb99d940f9976a19993893233a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:42.231802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:42.284137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9274bdbb99d940f9976a19993893233a', 'request_id': 'req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:42.290234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9274bdbb99d940f9976a19993893233a', 'request_id': 'req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:42.298579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9274bdbb99d940f9976a19993893233a', 'request_id': 'req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:42.311484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c admin admin] The token cache is being invalidate because project 9274bdbb99d940f9976a19993893233a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:42.311770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:42.312519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8ca4e4f9-ef5d-4e74-a3e0-3af9efd1504c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:42.314283 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3824/7732] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:42 2022] DELETE /identity/v3/projects/9274bdbb99d940f9976a19993893233a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:26:42.319337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb09033-298e-4556-98d2-012c82440951 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:42.319839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb09033-298e-4556-98d2-012c82440951 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:42.319839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb09033-298e-4556-98d2-012c82440951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:42.340064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3cb09033-298e-4556-98d2-012c82440951 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:42.341289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3cb09033-298e-4556-98d2-012c82440951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:42.393126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d869a-4b24-4a98-9824-9aa4cdf8d43d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:42.393605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d869a-4b24-4a98-9824-9aa4cdf8d43d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:42.393932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d869a-4b24-4a98-9824-9aa4cdf8d43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:42.403975 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3909/7733] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:42.412290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d339dea-3c26-4dce-bae5-7f1de2be215a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:42.412679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d339dea-3c26-4dce-bae5-7f1de2be215a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:42.412947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d339dea-3c26-4dce-bae5-7f1de2be215a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:42.416773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-704d869a-4b24-4a98-9824-9aa4cdf8d43d None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:42.418136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-704d869a-4b24-4a98-9824-9aa4cdf8d43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:42.433312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7d339dea-3c26-4dce-bae5-7f1de2be215a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:42.434708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7d339dea-3c26-4dce-bae5-7f1de2be215a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:42.443579 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3910/7734] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:42.447966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d339dea-3c26-4dce-bae5-7f1de2be215a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:42.457434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d339dea-3c26-4dce-bae5-7f1de2be215a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:42.462264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c45565-cdf9-4000-8a14-ac0631e95b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:42.462849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c45565-cdf9-4000-8a14-ac0631e95b35 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:42.463125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c45565-cdf9-4000-8a14-ac0631e95b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:42.463387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c45565-cdf9-4000-8a14-ac0631e95b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:42.464016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c45565-cdf9-4000-8a14-ac0631e95b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:42.464309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c45565-cdf9-4000-8a14-ac0631e95b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:42.465944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c45565-cdf9-4000-8a14-ac0631e95b35 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:42.477842 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3911/7735] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:26:42.482275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c45565-cdf9-4000-8a14-ac0631e95b35 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:42.490125 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3825/7736] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:42.495305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c45565-cdf9-4000-8a14-ac0631e95b35 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:42.500017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc063eb-a6fc-41af-ad9f-6b3e96780900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:42.500656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc063eb-a6fc-41af-ad9f-6b3e96780900 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:42.500925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc063eb-a6fc-41af-ad9f-6b3e96780900 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:42.501186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc063eb-a6fc-41af-ad9f-6b3e96780900 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:42.501797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc063eb-a6fc-41af-ad9f-6b3e96780900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:42.503688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc063eb-a6fc-41af-ad9f-6b3e96780900 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:42.536649 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3912/7737] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:42 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:26:42.540942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d869a-4b24-4a98-9824-9aa4cdf8d43d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:42.552481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d869a-4b24-4a98-9824-9aa4cdf8d43d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:42.561931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:42.612576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613210c1-fa22-49d1-8577-19e29e42358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:42.613579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613210c1-fa22-49d1-8577-19e29e42358d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:42.613994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613210c1-fa22-49d1-8577-19e29e42358d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:42.614465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613210c1-fa22-49d1-8577-19e29e42358d admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:42.615438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613210c1-fa22-49d1-8577-19e29e42358d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:42.617461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613210c1-fa22-49d1-8577-19e29e42358d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:42.642961 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3826/7738] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:42 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:42.647189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc063eb-a6fc-41af-ad9f-6b3e96780900 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:42.659306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc063eb-a6fc-41af-ad9f-6b3e96780900 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:42.664426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015d3fc-937e-4b58-bb4c-29e39bdc9454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:42.665130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015d3fc-937e-4b58-bb4c-29e39bdc9454 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:42.665464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015d3fc-937e-4b58-bb4c-29e39bdc9454 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:42.665788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015d3fc-937e-4b58-bb4c-29e39bdc9454 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:42.666479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015d3fc-937e-4b58-bb4c-29e39bdc9454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:42.668328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015d3fc-937e-4b58-bb4c-29e39bdc9454 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:42.678975 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3913/7739] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:42.683248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613210c1-fa22-49d1-8577-19e29e42358d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:42.694873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613210c1-fa22-49d1-8577-19e29e42358d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:42.699514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c128e7-91a6-4112-8cf5-eb1b78af1b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:42.700129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c128e7-91a6-4112-8cf5-eb1b78af1b30 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:42.700410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c128e7-91a6-4112-8cf5-eb1b78af1b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:42.700695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c128e7-91a6-4112-8cf5-eb1b78af1b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:42.701542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c128e7-91a6-4112-8cf5-eb1b78af1b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:42.703290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c128e7-91a6-4112-8cf5-eb1b78af1b30 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:42.713668 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3827/7740] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:42.718125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015d3fc-937e-4b58-bb4c-29e39bdc9454 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:42.728264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015d3fc-937e-4b58-bb4c-29e39bdc9454 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:42.733354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86ee3b-710e-4ab6-9e3f-6892be119469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:42.734040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86ee3b-710e-4ab6-9e3f-6892be119469 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:42.734341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86ee3b-710e-4ab6-9e3f-6892be119469 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:42.734611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86ee3b-710e-4ab6-9e3f-6892be119469 admin admin] PATH_INFO: `/v3/projects/990945c66e034adeb85da52dc10bee8c/users/8a4c54d51bcc43538a655b3eeea4709a/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:42.744274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:42.753512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86ee3b-710e-4ab6-9e3f-6892be119469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990945c66e034adeb85da52dc10bee8c, user_id=8a4c54d51bcc43538a655b3eeea4709a, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:42.756665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86ee3b-710e-4ab6-9e3f-6892be119469 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:42.770617 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3914/7741] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:42 2022] PUT /identity/v3/projects/990945c66e034adeb85da52dc10bee8c/users/8a4c54d51bcc43538a655b3eeea4709a/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:42.781353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806a934-0acd-4ea9-b51c-e96554787490 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:42.781885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806a934-0acd-4ea9-b51c-e96554787490 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:42.781885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806a934-0acd-4ea9-b51c-e96554787490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:42.810424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c806a934-0acd-4ea9-b51c-e96554787490 None None] MFA Rules not processed for user `8a4c54d51bcc43538a655b3eeea4709a`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:42.811644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c806a934-0acd-4ea9-b51c-e96554787490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:42.870567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03f00a-7b93-47d0-92d1-5e81401cba9d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:42.870809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03f00a-7b93-47d0-92d1-5e81401cba9d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:42.871098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03f00a-7b93-47d0-92d1-5e81401cba9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:42.884780 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3828/7742] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:42.889305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806a934-0acd-4ea9-b51c-e96554787490 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:42.893905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ca03f00a-7b93-47d0-92d1-5e81401cba9d None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:42.895080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ca03f00a-7b93-47d0-92d1-5e81401cba9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:42.950324 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3915/7743] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:42.954740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806a934-0acd-4ea9-b51c-e96554787490 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:42.959797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b4de7-f381-4248-be67-bda83e4506b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:42.960228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b4de7-f381-4248-be67-bda83e4506b3 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:42.960536 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b4de7-f381-4248-be67-bda83e4506b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:42.960805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b4de7-f381-4248-be67-bda83e4506b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:42.961445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b4de7-f381-4248-be67-bda83e4506b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:42.963391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b4de7-f381-4248-be67-bda83e4506b3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:42.997458 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3829/7744] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:42 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:26:43.001855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03f00a-7b93-47d0-92d1-5e81401cba9d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.013047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03f00a-7b93-47d0-92d1-5e81401cba9d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.073544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc77cd-b2ab-4235-94db-232b9444be0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.074155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc77cd-b2ab-4235-94db-232b9444be0f admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.074432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc77cd-b2ab-4235-94db-232b9444be0f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.074722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc77cd-b2ab-4235-94db-232b9444be0f admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.075358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc77cd-b2ab-4235-94db-232b9444be0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.077743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc77cd-b2ab-4235-94db-232b9444be0f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.101481 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3916/7745] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:43 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:26:43.105946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b4de7-f381-4248-be67-bda83e4506b3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.116938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b4de7-f381-4248-be67-bda83e4506b3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.121792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a483c-ccd0-4757-8390-68c2a596cec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.122399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a483c-ccd0-4757-8390-68c2a596cec9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.122699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a483c-ccd0-4757-8390-68c2a596cec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.123089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a483c-ccd0-4757-8390-68c2a596cec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.123883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a483c-ccd0-4757-8390-68c2a596cec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.125662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a483c-ccd0-4757-8390-68c2a596cec9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.127232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:43.141582 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3830/7746] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:43.142643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc77cd-b2ab-4235-94db-232b9444be0f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.153812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc77cd-b2ab-4235-94db-232b9444be0f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.160014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ab7a2-5ace-4409-8499-3ac8fb988f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.160742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ab7a2-5ace-4409-8499-3ac8fb988f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.161018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ab7a2-5ace-4409-8499-3ac8fb988f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.161289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ab7a2-5ace-4409-8499-3ac8fb988f83 admin admin] PATH_INFO: `/v3/projects/97dac2790fa54ffc82a750d113b0659b/users/51f7bfd357ce47ae8c1292f95c5bca1e/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.179239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ab7a2-5ace-4409-8499-3ac8fb988f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97dac2790fa54ffc82a750d113b0659b, user_id=51f7bfd357ce47ae8c1292f95c5bca1e, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.181976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ab7a2-5ace-4409-8499-3ac8fb988f83 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.198461 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3917/7747] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:43 2022] PUT /identity/v3/projects/97dac2790fa54ffc82a750d113b0659b/users/51f7bfd357ce47ae8c1292f95c5bca1e/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:43.202827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a483c-ccd0-4757-8390-68c2a596cec9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.214061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a483c-ccd0-4757-8390-68c2a596cec9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.276384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb1e4d-e712-4ad9-930f-16f3aa142e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.277015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb1e4d-e712-4ad9-930f-16f3aa142e93 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.277289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb1e4d-e712-4ad9-930f-16f3aa142e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.277558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb1e4d-e712-4ad9-930f-16f3aa142e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.278193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb1e4d-e712-4ad9-930f-16f3aa142e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.279780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb1e4d-e712-4ad9-930f-16f3aa142e93 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.290150 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3831/7748] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:26:43.294282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ab7a2-5ace-4409-8499-3ac8fb988f83 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.306280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ab7a2-5ace-4409-8499-3ac8fb988f83 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.311288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dc4c5-edcc-4f03-9641-7e5dcbb13480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.311951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dc4c5-edcc-4f03-9641-7e5dcbb13480 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.312251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dc4c5-edcc-4f03-9641-7e5dcbb13480 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.312559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dc4c5-edcc-4f03-9641-7e5dcbb13480 admin admin] PATH_INFO: `/v3/projects/97dac2790fa54ffc82a750d113b0659b/users/51f7bfd357ce47ae8c1292f95c5bca1e/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.314495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dc4c5-edcc-4f03-9641-7e5dcbb13480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97dac2790fa54ffc82a750d113b0659b, user_id=51f7bfd357ce47ae8c1292f95c5bca1e, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.317805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dc4c5-edcc-4f03-9641-7e5dcbb13480 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.329050 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3918/7749] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:43 2022] PUT /identity/v3/projects/97dac2790fa54ffc82a750d113b0659b/users/51f7bfd357ce47ae8c1292f95c5bca1e/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:43.337646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8f9ec-8134-494f-8d09-99df31ac666e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.338059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8f9ec-8134-494f-8d09-99df31ac666e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.338194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8f9ec-8134-494f-8d09-99df31ac666e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.339764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f817143-cf95-4e47-80b1-4bd120a2d55d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.340185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f817143-cf95-4e47-80b1-4bd120a2d55d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.340511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f817143-cf95-4e47-80b1-4bd120a2d55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.359227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9bf8f9ec-8134-494f-8d09-99df31ac666e None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:43.360539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9bf8f9ec-8134-494f-8d09-99df31ac666e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:43.370930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3f817143-cf95-4e47-80b1-4bd120a2d55d None None] MFA Rules not processed for user `51f7bfd357ce47ae8c1292f95c5bca1e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:43.372293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3f817143-cf95-4e47-80b1-4bd120a2d55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:43.407933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:43.419524 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3832/7750] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:43.447145 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3919/7751] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:43 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:43.452232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8f9ec-8134-494f-8d09-99df31ac666e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.462842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8f9ec-8134-494f-8d09-99df31ac666e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.467397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d481b8-f207-4e99-9672-32922659cbb3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.468007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d481b8-f207-4e99-9672-32922659cbb3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.468279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d481b8-f207-4e99-9672-32922659cbb3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.468573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d481b8-f207-4e99-9672-32922659cbb3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.469468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d481b8-f207-4e99-9672-32922659cbb3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.471107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d481b8-f207-4e99-9672-32922659cbb3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.491252 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3833/7752] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:26:43 2022] POST /identity/v3/domains => generated 276 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:43.495400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f817143-cf95-4e47-80b1-4bd120a2d55d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.506211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f817143-cf95-4e47-80b1-4bd120a2d55d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.565195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bd7cc-820e-4059-9be2-0da96cefbc8d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.565870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bd7cc-820e-4059-9be2-0da96cefbc8d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.566139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bd7cc-820e-4059-9be2-0da96cefbc8d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.566429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bd7cc-820e-4059-9be2-0da96cefbc8d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/b67415b0ffe94085895953c0c5a96779/config` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.567406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7bd7cc-820e-4059-9be2-0da96cefbc8d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=b67415b0ffe94085895953c0c5a96779)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.568505 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-fd7bd7cc-820e-4059-9be2-0da96cefbc8d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:43.568505 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:26:43.568505 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:26:43.569339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7bd7cc-820e-4059-9be2-0da96cefbc8d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.608458 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3920/7753] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:26:43 2022] PUT /identity/v3/domains/b67415b0ffe94085895953c0c5a96779/config => generated 138 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:43.610668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d481b8-f207-4e99-9672-32922659cbb3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.621454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d481b8-f207-4e99-9672-32922659cbb3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.626394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fab543-9687-45a2-bd6e-7775cd0bd64a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.627525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fab543-9687-45a2-bd6e-7775cd0bd64a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.627799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fab543-9687-45a2-bd6e-7775cd0bd64a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.628060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fab543-9687-45a2-bd6e-7775cd0bd64a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/b67415b0ffe94085895953c0c5a96779/config/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.629239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fab543-9687-45a2-bd6e-7775cd0bd64a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=b67415b0ffe94085895953c0c5a96779, group=identity)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.630250 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-75fab543-9687-45a2-bd6e-7775cd0bd64a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:43.630250 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:26:43.630250 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:26:43.631138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fab543-9687-45a2-bd6e-7775cd0bd64a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.643404 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3834/7754] 10.222.0.23 () {58 vars in 1275 bytes} [Sat Aug 20 21:26:43 2022] GET /identity/v3/domains/b67415b0ffe94085895953c0c5a96779/config/identity => generated 45 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Aug 20 21:26:43.647406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bd7cc-820e-4059-9be2-0da96cefbc8d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.658508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bd7cc-820e-4059-9be2-0da96cefbc8d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.663372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7af5f-d7f6-4452-aaa2-22e113b37a91 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.664034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7af5f-d7f6-4452-aaa2-22e113b37a91 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.664307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7af5f-d7f6-4452-aaa2-22e113b37a91 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.664622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7af5f-d7f6-4452-aaa2-22e113b37a91 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/b67415b0ffe94085895953c0c5a96779/config/identity/driver` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.665789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7af5f-d7f6-4452-aaa2-22e113b37a91 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=b67415b0ffe94085895953c0c5a96779, group=identity, option=driver)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.666813 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-f8e7af5f-d7f6-4452-aaa2-22e113b37a91 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:43.666813 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:26:43.666813 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:26:43.667700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7af5f-d7f6-4452-aaa2-22e113b37a91 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.675086 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3921/7755] 10.222.0.23 () {58 vars in 1296 bytes} [Sat Aug 20 21:26:43 2022] GET /identity/v3/domains/b67415b0ffe94085895953c0c5a96779/config/identity/driver => generated 31 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:43.678989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fab543-9687-45a2-bd6e-7775cd0bd64a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.693431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fab543-9687-45a2-bd6e-7775cd0bd64a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.697956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55391c01-8418-4ef0-901f-62daf2e0ceef tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.698748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55391c01-8418-4ef0-901f-62daf2e0ceef tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.699020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55391c01-8418-4ef0-901f-62daf2e0ceef tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.699286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55391c01-8418-4ef0-901f-62daf2e0ceef tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/b67415b0ffe94085895953c0c5a96779/config/ldap` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.700407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55391c01-8418-4ef0-901f-62daf2e0ceef tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=b67415b0ffe94085895953c0c5a96779, group=ldap)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.702712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55391c01-8418-4ef0-901f-62daf2e0ceef tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.708230 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3835/7756] 10.222.0.23 () {58 vars in 1263 bytes} [Sat Aug 20 21:26:43 2022] GET /identity/v3/domains/b67415b0ffe94085895953c0c5a96779/config/ldap => generated 106 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:43.712243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7af5f-d7f6-4452-aaa2-22e113b37a91 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.722879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7af5f-d7f6-4452-aaa2-22e113b37a91 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.727809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379232c6-1f46-46d1-9317-eac9f5cc6b9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.728521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379232c6-1f46-46d1-9317-eac9f5cc6b9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.728794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379232c6-1f46-46d1-9317-eac9f5cc6b9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.729091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379232c6-1f46-46d1-9317-eac9f5cc6b9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/b67415b0ffe94085895953c0c5a96779/config/ldap/url` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.730253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379232c6-1f46-46d1-9317-eac9f5cc6b9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=b67415b0ffe94085895953c0c5a96779, group=ldap, option=url)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.731881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379232c6-1f46-46d1-9317-eac9f5cc6b9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.739929 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3922/7757] 10.222.0.23 () {58 vars in 1275 bytes} [Sat Aug 20 21:26:43 2022] GET /identity/v3/domains/b67415b0ffe94085895953c0c5a96779/config/ldap/url => generated 46 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Aug 20 21:26:43.741275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55391c01-8418-4ef0-901f-62daf2e0ceef tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.751649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55391c01-8418-4ef0-901f-62daf2e0ceef tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.756327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4d05c-86d8-4ef8-82fa-f335148685f7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.757366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4d05c-86d8-4ef8-82fa-f335148685f7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.757643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4d05c-86d8-4ef8-82fa-f335148685f7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.757909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4d05c-86d8-4ef8-82fa-f335148685f7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/b67415b0ffe94085895953c0c5a96779/config/ldap/user_tree_dn` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.759043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4d05c-86d8-4ef8-82fa-f335148685f7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=b67415b0ffe94085895953c0c5a96779, group=ldap, option=user_tree_dn)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.760707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4d05c-86d8-4ef8-82fa-f335148685f7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.767872 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3836/7758] 10.222.0.23 () {58 vars in 1302 bytes} [Sat Aug 20 21:26:43 2022] GET /identity/v3/domains/b67415b0ffe94085895953c0c5a96779/config/ldap/user_tree_dn => generated 63 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:43.772165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379232c6-1f46-46d1-9317-eac9f5cc6b9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.783195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379232c6-1f46-46d1-9317-eac9f5cc6b9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.792127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be80a7f-d054-455d-918e-2d8f5cb34974 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.792941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be80a7f-d054-455d-918e-2d8f5cb34974 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.793195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be80a7f-d054-455d-918e-2d8f5cb34974 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.793478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be80a7f-d054-455d-918e-2d8f5cb34974 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/b67415b0ffe94085895953c0c5a96779/config` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.794059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be80a7f-d054-455d-918e-2d8f5cb34974 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=b67415b0ffe94085895953c0c5a96779)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.795065 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-6be80a7f-d054-455d-918e-2d8f5cb34974 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:43.795065 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:26:43.795065 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:26:43.796365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be80a7f-d054-455d-918e-2d8f5cb34974 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.806306 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3923/7759] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:26:43 2022] DELETE /identity/v3/domains/b67415b0ffe94085895953c0c5a96779/config => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:26:43.810917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4d05c-86d8-4ef8-82fa-f335148685f7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.823641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4d05c-86d8-4ef8-82fa-f335148685f7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.828962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1a54b-c124-433e-9d93-3254f319d4e0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.829599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1a54b-c124-433e-9d93-3254f319d4e0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.829874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1a54b-c124-433e-9d93-3254f319d4e0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.830142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1a54b-c124-433e-9d93-3254f319d4e0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/b67415b0ffe94085895953c0c5a96779` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.831394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1a54b-c124-433e-9d93-3254f319d4e0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=b67415b0ffe94085895953c0c5a96779)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.833079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1a54b-c124-433e-9d93-3254f319d4e0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.857798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a8e1a54b-c124-433e-9d93-3254f319d4e0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'b67415b0ffe94085895953c0c5a96779'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:43.858931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a8e1a54b-c124-433e-9d93-3254f319d4e0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'b67415b0ffe94085895953c0c5a96779'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:43.887196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a8e1a54b-c124-433e-9d93-3254f319d4e0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'b67415b0ffe94085895953c0c5a96779'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:43.887818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a8e1a54b-c124-433e-9d93-3254f319d4e0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'b67415b0ffe94085895953c0c5a96779'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:43.894315 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3837/7760] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:26:43 2022] PATCH /identity/v3/domains/b67415b0ffe94085895953c0c5a96779 => generated 277 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:43.895357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be80a7f-d054-455d-918e-2d8f5cb34974 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:43.897263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6be80a7f-d054-455d-918e-2d8f5cb34974 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:43.910719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be80a7f-d054-455d-918e-2d8f5cb34974 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:43.971789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:43.972456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:43.972732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:43.973004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/b67415b0ffe94085895953c0c5a96779` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:43.985264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=b67415b0ffe94085895953c0c5a96779)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:43.987320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:43.994257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:44.014129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'b67415b0ffe94085895953c0c5a96779'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.025123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'b67415b0ffe94085895953c0c5a96779'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.066227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8f5ee-f3b4-48df-add8-936379d7d9fe None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:44.066502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8f5ee-f3b4-48df-add8-936379d7d9fe None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:44.066773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8f5ee-f3b4-48df-add8-936379d7d9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:44.067587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b67415b0ffe94085895953c0c5a96779', 'request_id': 'req-45aa2ad0-9f15-4624-92ee-0027bc819df1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.077496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b67415b0ffe94085895953c0c5a96779', 'request_id': 'req-45aa2ad0-9f15-4624-92ee-0027bc819df1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.082951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b67415b0ffe94085895953c0c5a96779', 'request_id': 'req-45aa2ad0-9f15-4624-92ee-0027bc819df1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.087449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-63b8f5ee-f3b4-48df-add8-936379d7d9fe None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:44.088676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-63b8f5ee-f3b4-48df-add8-936379d7d9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:44.094867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] The token cache is being invalidate because project b67415b0ffe94085895953c0c5a96779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:44.095150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.095848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.098862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:44.104265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'b67415b0ffe94085895953c0c5a96779', 'request_id': 'req-45aa2ad0-9f15-4624-92ee-0027bc819df1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.104920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'b67415b0ffe94085895953c0c5a96779', 'request_id': 'req-45aa2ad0-9f15-4624-92ee-0027bc819df1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.108808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'b67415b0ffe94085895953c0c5a96779', 'request_id': 'req-45aa2ad0-9f15-4624-92ee-0027bc819df1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.115587 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3924/7761] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:26:43 2022] DELETE /identity/v3/domains/b67415b0ffe94085895953c0c5a96779 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:44.121197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:44.122705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:44.150899 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3838/7762] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:44.191125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa2ad0-9f15-4624-92ee-0027bc819df1 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:44.196513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57960b4-7712-46f4-8dec-d0928c37057b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:44.197297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57960b4-7712-46f4-8dec-d0928c37057b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:44.197571 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57960b4-7712-46f4-8dec-d0928c37057b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:44.197851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57960b4-7712-46f4-8dec-d0928c37057b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:44.198632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57960b4-7712-46f4-8dec-d0928c37057b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:44.200289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57960b4-7712-46f4-8dec-d0928c37057b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:44.219244 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3925/7763] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:26:44 2022] POST /identity/v3/domains => generated 276 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:44.223420 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8f5ee-f3b4-48df-add8-936379d7d9fe None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:44.244867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8f5ee-f3b4-48df-add8-936379d7d9fe None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:44.308333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca332da0-0ec7-4694-9d83-3c35530eecf3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:44.309009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca332da0-0ec7-4694-9d83-3c35530eecf3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:44.309286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca332da0-0ec7-4694-9d83-3c35530eecf3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:44.309556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca332da0-0ec7-4694-9d83-3c35530eecf3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/6189f352e64d4c2aab62ffdfb8f76624/config` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:44.310269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca332da0-0ec7-4694-9d83-3c35530eecf3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=6189f352e64d4c2aab62ffdfb8f76624)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:44.311420 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-ca332da0-0ec7-4694-9d83-3c35530eecf3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:44.311420 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:26:44.311420 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:26:44.312515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca332da0-0ec7-4694-9d83-3c35530eecf3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:44.352622 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3839/7764] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:26:44 2022] PUT /identity/v3/domains/6189f352e64d4c2aab62ffdfb8f76624/config => generated 138 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:26:44.356542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57960b4-7712-46f4-8dec-d0928c37057b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:44.367092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57960b4-7712-46f4-8dec-d0928c37057b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:44.371818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ca93d-1c32-4546-9380-56b100d3f40f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:44.372481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca93d-1c32-4546-9380-56b100d3f40f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:44.372773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca93d-1c32-4546-9380-56b100d3f40f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:44.373034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca93d-1c32-4546-9380-56b100d3f40f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/6189f352e64d4c2aab62ffdfb8f76624/config` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:44.373741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ca93d-1c32-4546-9380-56b100d3f40f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=6189f352e64d4c2aab62ffdfb8f76624)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:44.374750 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-c41ca93d-1c32-4546-9380-56b100d3f40f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:44.374750 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:26:44.374750 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:26:44.375637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ca93d-1c32-4546-9380-56b100d3f40f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:44.400766 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3926/7765] 10.222.0.23 () {60 vars in 1271 bytes} [Sat Aug 20 21:26:44 2022] PATCH /identity/v3/domains/6189f352e64d4c2aab62ffdfb8f76624/config => generated 141 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:26:44.404960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca332da0-0ec7-4694-9d83-3c35530eecf3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:44.415078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca332da0-0ec7-4694-9d83-3c35530eecf3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:44.419780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec2edc-6af0-4484-8a9d-d0bc2b521240 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:44.420442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec2edc-6af0-4484-8a9d-d0bc2b521240 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:44.420781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec2edc-6af0-4484-8a9d-d0bc2b521240 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:44.421049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec2edc-6af0-4484-8a9d-d0bc2b521240 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/6189f352e64d4c2aab62ffdfb8f76624/config` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:44.422193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec2edc-6af0-4484-8a9d-d0bc2b521240 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=6189f352e64d4c2aab62ffdfb8f76624)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:44.423809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec2edc-6af0-4484-8a9d-d0bc2b521240 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:44.428603 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3840/7766] 10.222.0.23 () {58 vars in 1248 bytes} [Sat Aug 20 21:26:44 2022] GET /identity/v3/domains/6189f352e64d4c2aab62ffdfb8f76624/config => generated 141 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:26:44.432732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ca93d-1c32-4546-9380-56b100d3f40f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:44.444045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ca93d-1c32-4546-9380-56b100d3f40f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:44.449508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7348240-9e72-43d6-9e8c-ccf5070eba01 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:44.450135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7348240-9e72-43d6-9e8c-ccf5070eba01 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:44.450407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7348240-9e72-43d6-9e8c-ccf5070eba01 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:44.450673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7348240-9e72-43d6-9e8c-ccf5070eba01 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/6189f352e64d4c2aab62ffdfb8f76624/config` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:44.451267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7348240-9e72-43d6-9e8c-ccf5070eba01 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=6189f352e64d4c2aab62ffdfb8f76624)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:44.453080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7348240-9e72-43d6-9e8c-ccf5070eba01 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:44.464077 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3927/7767] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:26:44 2022] DELETE /identity/v3/domains/6189f352e64d4c2aab62ffdfb8f76624/config => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:26:44.468911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec2edc-6af0-4484-8a9d-d0bc2b521240 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:44.480122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec2edc-6af0-4484-8a9d-d0bc2b521240 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:44.485424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021ebbf-e486-4d36-941c-a05fd0013926 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:44.486060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021ebbf-e486-4d36-941c-a05fd0013926 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:44.486332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021ebbf-e486-4d36-941c-a05fd0013926 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:44.486594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021ebbf-e486-4d36-941c-a05fd0013926 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/6189f352e64d4c2aab62ffdfb8f76624/config` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:44.487822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021ebbf-e486-4d36-941c-a05fd0013926 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=6189f352e64d4c2aab62ffdfb8f76624)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:44.489852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021ebbf-e486-4d36-941c-a05fd0013926 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:44.497982 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-0021ebbf-e486-4d36-941c-a05fd0013926 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Could not find any options in domain configuration for domain 6189f352e64d4c2aab62ffdfb8f76624.: keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain 6189f352e64d4c2aab62ffdfb8f76624. Aug 20 21:26:44.497982 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:26:44.497982 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:26:44.497982 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:26:44.497982 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:26:44.497982 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:26:44.497982 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:26:44.497982 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:26:44.497982 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:26:44.497982 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:26:44.497982 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:26:44.497982 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:26:44.497982 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Aug 20 21:26:44.499108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Aug 20 21:26:44.499108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Aug 20 21:26:44.499108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Aug 20 21:26:44.499108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:26:44.499108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:26:44.499108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Aug 20 21:26:44.499108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Aug 20 21:26:44.499108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain 6189f352e64d4c2aab62ffdfb8f76624. Aug 20 21:26:44.499108 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:26:44.501010 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3841/7768] 10.222.0.23 () {58 vars in 1248 bytes} [Sat Aug 20 21:26:44 2022] GET /identity/v3/domains/6189f352e64d4c2aab62ffdfb8f76624/config => generated 151 bytes in 31 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:26:44.503820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7348240-9e72-43d6-9e8c-ccf5070eba01 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:44.509120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=76,keystone:DELETE=19,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:44.515774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7348240-9e72-43d6-9e8c-ccf5070eba01 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:44.521884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f8b31-af15-48bf-9ed4-a134ab0dec2d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:44.522519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f8b31-af15-48bf-9ed4-a134ab0dec2d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:44.522789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f8b31-af15-48bf-9ed4-a134ab0dec2d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:44.523057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f8b31-af15-48bf-9ed4-a134ab0dec2d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/6189f352e64d4c2aab62ffdfb8f76624/config` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:44.523681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f8b31-af15-48bf-9ed4-a134ab0dec2d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=6189f352e64d4c2aab62ffdfb8f76624)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:44.525472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f8b31-af15-48bf-9ed4-a134ab0dec2d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:44.531806 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3928/7769] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:26:44 2022] DELETE /identity/v3/domains/6189f352e64d4c2aab62ffdfb8f76624/config => generated 0 bytes in 28 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:44.535976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021ebbf-e486-4d36-941c-a05fd0013926 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:44.547268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021ebbf-e486-4d36-941c-a05fd0013926 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:44.552216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a30f98-de9b-4015-9a59-747c78bc999d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:44.552786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a30f98-de9b-4015-9a59-747c78bc999d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:44.553055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a30f98-de9b-4015-9a59-747c78bc999d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:44.553331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a30f98-de9b-4015-9a59-747c78bc999d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/6189f352e64d4c2aab62ffdfb8f76624` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:44.554553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a30f98-de9b-4015-9a59-747c78bc999d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=6189f352e64d4c2aab62ffdfb8f76624)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:44.556137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a30f98-de9b-4015-9a59-747c78bc999d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:44.578577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9a30f98-de9b-4015-9a59-747c78bc999d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '6189f352e64d4c2aab62ffdfb8f76624'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.579308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9a30f98-de9b-4015-9a59-747c78bc999d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '6189f352e64d4c2aab62ffdfb8f76624'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.605585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9a30f98-de9b-4015-9a59-747c78bc999d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '6189f352e64d4c2aab62ffdfb8f76624'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.606229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9a30f98-de9b-4015-9a59-747c78bc999d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '6189f352e64d4c2aab62ffdfb8f76624'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.610650 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3842/7770] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:26:44 2022] PATCH /identity/v3/domains/6189f352e64d4c2aab62ffdfb8f76624 => generated 277 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:44.614986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f8b31-af15-48bf-9ed4-a134ab0dec2d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:44.617006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-609f8b31-af15-48bf-9ed4-a134ab0dec2d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:44.630562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f8b31-af15-48bf-9ed4-a134ab0dec2d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:44.693931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:44.694568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:44.694858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:44.695130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/6189f352e64d4c2aab62ffdfb8f76624` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:44.705884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=6189f352e64d4c2aab62ffdfb8f76624)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:44.707522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:44.730758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '6189f352e64d4c2aab62ffdfb8f76624'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.745181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '6189f352e64d4c2aab62ffdfb8f76624'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.792326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6189f352e64d4c2aab62ffdfb8f76624', 'request_id': 'req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.800630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6189f352e64d4c2aab62ffdfb8f76624', 'request_id': 'req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.806323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6189f352e64d4c2aab62ffdfb8f76624', 'request_id': 'req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.820069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] The token cache is being invalidate because project 6189f352e64d4c2aab62ffdfb8f76624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:44.820354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.821103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.831516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '6189f352e64d4c2aab62ffdfb8f76624', 'request_id': 'req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.832284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '6189f352e64d4c2aab62ffdfb8f76624', 'request_id': 'req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.836304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '6189f352e64d4c2aab62ffdfb8f76624', 'request_id': 'req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:44.844407 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3929/7771] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:26:44 2022] DELETE /identity/v3/domains/6189f352e64d4c2aab62ffdfb8f76624 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:44.851161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a30f98-de9b-4015-9a59-747c78bc999d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:44.862267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a30f98-de9b-4015-9a59-747c78bc999d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:44.893346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:44.923419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134631a8-930e-4495-a258-0be3e5ba114e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:44.924153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134631a8-930e-4495-a258-0be3e5ba114e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:44.924467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134631a8-930e-4495-a258-0be3e5ba114e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:44.924737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134631a8-930e-4495-a258-0be3e5ba114e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:44.925467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134631a8-930e-4495-a258-0be3e5ba114e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:44.927065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134631a8-930e-4495-a258-0be3e5ba114e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:44.948517 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3843/7772] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:26:44 2022] POST /identity/v3/domains => generated 277 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:44.949794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:44.951226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:44.963795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca48c37-b8e8-4c7a-98ed-f7c0e8e564d9 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:44.990852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:45.024488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9377d53-8bb7-4f58-88b3-82ddac970704 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.024488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377d53-8bb7-4f58-88b3-82ddac970704 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.024488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377d53-8bb7-4f58-88b3-82ddac970704 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.024763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377d53-8bb7-4f58-88b3-82ddac970704 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.025105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9377d53-8bb7-4f58-88b3-82ddac970704 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=db19c0f5790d4371b59c2ca3857bc4c8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.026835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9377d53-8bb7-4f58-88b3-82ddac970704 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.064700 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3930/7773] 10.222.0.23 () {60 vars in 1269 bytes} [Sat Aug 20 21:26:44 2022] PUT /identity/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config => generated 138 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:45.069286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134631a8-930e-4495-a258-0be3e5ba114e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.081118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134631a8-930e-4495-a258-0be3e5ba114e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.090506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6550f3-e528-428d-835f-7ecd36a93c60 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.090506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6550f3-e528-428d-835f-7ecd36a93c60 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.090506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6550f3-e528-428d-835f-7ecd36a93c60 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.090918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6550f3-e528-428d-835f-7ecd36a93c60 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.090918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6550f3-e528-428d-835f-7ecd36a93c60 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=db19c0f5790d4371b59c2ca3857bc4c8, group=identity)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.091771 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-0b6550f3-e528-428d-835f-7ecd36a93c60 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:45.091771 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:26:45.091771 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:26:45.092929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6550f3-e528-428d-835f-7ecd36a93c60 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.125959 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3844/7774] 10.222.0.23 () {60 vars in 1297 bytes} [Sat Aug 20 21:26:45 2022] PATCH /identity/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config/identity => generated 176 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:26:45.127323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9377d53-8bb7-4f58-88b3-82ddac970704 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.137878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9377d53-8bb7-4f58-88b3-82ddac970704 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.143327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32aaac-d503-484f-a774-481ebf4abe07 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.143992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32aaac-d503-484f-a774-481ebf4abe07 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.144268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32aaac-d503-484f-a774-481ebf4abe07 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.144577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32aaac-d503-484f-a774-481ebf4abe07 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config/identity/driver` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.145269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32aaac-d503-484f-a774-481ebf4abe07 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=db19c0f5790d4371b59c2ca3857bc4c8, group=identity, option=driver)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.146912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32aaac-d503-484f-a774-481ebf4abe07 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.180465 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3931/7775] 10.222.0.23 () {60 vars in 1318 bytes} [Sat Aug 20 21:26:45 2022] PATCH /identity/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config/identity/driver => generated 177 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:26:45.181901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6550f3-e528-428d-835f-7ecd36a93c60 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.193685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6550f3-e528-428d-835f-7ecd36a93c60 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.199034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35711bea-1097-4900-ab78-f36b87741dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.199694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35711bea-1097-4900-ab78-f36b87741dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.199966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35711bea-1097-4900-ab78-f36b87741dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.200252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35711bea-1097-4900-ab78-f36b87741dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config/identity/driver` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.200883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35711bea-1097-4900-ab78-f36b87741dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=db19c0f5790d4371b59c2ca3857bc4c8, group=identity, option=driver)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.202092 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-35711bea-1097-4900-ab78-f36b87741dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:45.202092 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:26:45.202092 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:26:45.203115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35711bea-1097-4900-ab78-f36b87741dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.220533 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3845/7776] 10.222.0.23 () {58 vars in 1299 bytes} [Sat Aug 20 21:26:45 2022] DELETE /identity/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config/identity/driver => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:45.225035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32aaac-d503-484f-a774-481ebf4abe07 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.240649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32aaac-d503-484f-a774-481ebf4abe07 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.246503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085281b-d05e-4ed7-ab55-ec709b17ec26 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.247418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085281b-d05e-4ed7-ab55-ec709b17ec26 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.247779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085281b-d05e-4ed7-ab55-ec709b17ec26 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.248148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085281b-d05e-4ed7-ab55-ec709b17ec26 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config/identity/driver` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.250177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085281b-d05e-4ed7-ab55-ec709b17ec26 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=db19c0f5790d4371b59c2ca3857bc4c8, group=identity, option=driver)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.251903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085281b-d05e-4ed7-ab55-ec709b17ec26 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.261762 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-1085281b-d05e-4ed7-ab55-ec709b17ec26 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Could not find option driver in group identity in domain configuration for domain db19c0f5790d4371b59c2ca3857bc4c8.: keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain db19c0f5790d4371b59c2ca3857bc4c8. Aug 20 21:26:45.261762 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:26:45.261762 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:26:45.261762 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:26:45.261762 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:26:45.261762 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:26:45.261762 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:26:45.261762 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:26:45.261762 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:26:45.261762 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:26:45.261762 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:26:45.261762 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:26:45.261762 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Aug 20 21:26:45.262860 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Aug 20 21:26:45.262860 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Aug 20 21:26:45.262860 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Aug 20 21:26:45.262860 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:26:45.262860 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:26:45.262860 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Aug 20 21:26:45.262860 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Aug 20 21:26:45.262860 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain db19c0f5790d4371b59c2ca3857bc4c8. Aug 20 21:26:45.262860 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:26:45.264210 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3932/7777] 10.222.0.23 () {58 vars in 1296 bytes} [Sat Aug 20 21:26:45 2022] GET /identity/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config/identity/driver => generated 171 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Aug 20 21:26:45.268325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35711bea-1097-4900-ab78-f36b87741dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.278760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35711bea-1097-4900-ab78-f36b87741dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.284214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95590957-47e4-467b-b9cf-844f84c774b4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.284903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95590957-47e4-467b-b9cf-844f84c774b4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.285178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95590957-47e4-467b-b9cf-844f84c774b4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.285442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95590957-47e4-467b-b9cf-844f84c774b4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.286025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95590957-47e4-467b-b9cf-844f84c774b4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=db19c0f5790d4371b59c2ca3857bc4c8, group=identity)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.287618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95590957-47e4-467b-b9cf-844f84c774b4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.309763 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3846/7778] 10.222.0.23 () {58 vars in 1278 bytes} [Sat Aug 20 21:26:45 2022] DELETE /identity/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config/identity => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:45.314123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085281b-d05e-4ed7-ab55-ec709b17ec26 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.325096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085281b-d05e-4ed7-ab55-ec709b17ec26 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.331162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac5267-f068-4fcf-a7c5-8d1eed608d84 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.331887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac5267-f068-4fcf-a7c5-8d1eed608d84 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.332225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac5267-f068-4fcf-a7c5-8d1eed608d84 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.332584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac5267-f068-4fcf-a7c5-8d1eed608d84 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.333772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac5267-f068-4fcf-a7c5-8d1eed608d84 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=db19c0f5790d4371b59c2ca3857bc4c8, group=identity)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.335466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac5267-f068-4fcf-a7c5-8d1eed608d84 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.345993 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-b4ac5267-f068-4fcf-a7c5-8d1eed608d84 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Could not find group identity in domain configuration for domain db19c0f5790d4371b59c2ca3857bc4c8.: keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain db19c0f5790d4371b59c2ca3857bc4c8. Aug 20 21:26:45.345993 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:26:45.345993 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:26:45.345993 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:26:45.345993 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:26:45.345993 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:26:45.345993 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:26:45.345993 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:26:45.345993 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:26:45.345993 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:26:45.345993 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:26:45.345993 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:26:45.345993 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Aug 20 21:26:45.347100 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Aug 20 21:26:45.347100 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Aug 20 21:26:45.347100 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Aug 20 21:26:45.347100 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:26:45.347100 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:26:45.347100 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Aug 20 21:26:45.347100 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Aug 20 21:26:45.347100 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain db19c0f5790d4371b59c2ca3857bc4c8. Aug 20 21:26:45.347100 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:26:45.348679 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3933/7779] 10.222.0.23 () {58 vars in 1275 bytes} [Sat Aug 20 21:26:45 2022] GET /identity/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config/identity => generated 154 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:26:45.351418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95590957-47e4-467b-b9cf-844f84c774b4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.361735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95590957-47e4-467b-b9cf-844f84c774b4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.366606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316aa5db-f985-4377-9532-e2b7b7b907a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.367245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316aa5db-f985-4377-9532-e2b7b7b907a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.367521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316aa5db-f985-4377-9532-e2b7b7b907a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.367790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316aa5db-f985-4377-9532-e2b7b7b907a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.368365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316aa5db-f985-4377-9532-e2b7b7b907a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=db19c0f5790d4371b59c2ca3857bc4c8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.370016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316aa5db-f985-4377-9532-e2b7b7b907a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.381749 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3847/7780] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:26:45 2022] DELETE /identity/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8/config => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:45.385899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac5267-f068-4fcf-a7c5-8d1eed608d84 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.396470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac5267-f068-4fcf-a7c5-8d1eed608d84 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.406936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1b722-7a30-4717-a5bc-b48ac1d369a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.407638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1b722-7a30-4717-a5bc-b48ac1d369a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.407926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1b722-7a30-4717-a5bc-b48ac1d369a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.408231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1b722-7a30-4717-a5bc-b48ac1d369a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.409576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1b722-7a30-4717-a5bc-b48ac1d369a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=db19c0f5790d4371b59c2ca3857bc4c8)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.411211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1b722-7a30-4717-a5bc-b48ac1d369a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.436210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ce1b722-7a30-4717-a5bc-b48ac1d369a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'db19c0f5790d4371b59c2ca3857bc4c8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.437248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ce1b722-7a30-4717-a5bc-b48ac1d369a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'db19c0f5790d4371b59c2ca3857bc4c8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.464236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ce1b722-7a30-4717-a5bc-b48ac1d369a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'db19c0f5790d4371b59c2ca3857bc4c8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.467050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ce1b722-7a30-4717-a5bc-b48ac1d369a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'db19c0f5790d4371b59c2ca3857bc4c8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.472406 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3934/7781] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:26:45 2022] PATCH /identity/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8 => generated 278 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:26:45.473948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316aa5db-f985-4377-9532-e2b7b7b907a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.475689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-316aa5db-f985-4377-9532-e2b7b7b907a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:45.490120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316aa5db-f985-4377-9532-e2b7b7b907a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.548403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.549045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.549314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.549578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.563094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=db19c0f5790d4371b59c2ca3857bc4c8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.564827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.589269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'db19c0f5790d4371b59c2ca3857bc4c8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.593829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'db19c0f5790d4371b59c2ca3857bc4c8'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.606272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:45.645537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db19c0f5790d4371b59c2ca3857bc4c8', 'request_id': 'req-05b078dc-483d-4c0f-b3e2-c343dfbd380b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.652767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db19c0f5790d4371b59c2ca3857bc4c8', 'request_id': 'req-05b078dc-483d-4c0f-b3e2-c343dfbd380b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.661365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db19c0f5790d4371b59c2ca3857bc4c8', 'request_id': 'req-05b078dc-483d-4c0f-b3e2-c343dfbd380b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.675524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] The token cache is being invalidate because project db19c0f5790d4371b59c2ca3857bc4c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:45.675524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.675524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.683560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'db19c0f5790d4371b59c2ca3857bc4c8', 'request_id': 'req-05b078dc-483d-4c0f-b3e2-c343dfbd380b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.684276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'db19c0f5790d4371b59c2ca3857bc4c8', 'request_id': 'req-05b078dc-483d-4c0f-b3e2-c343dfbd380b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.687807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'db19c0f5790d4371b59c2ca3857bc4c8', 'request_id': 'req-05b078dc-483d-4c0f-b3e2-c343dfbd380b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:45.694057 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3848/7782] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:26:45 2022] DELETE /identity/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:45.699241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1b722-7a30-4717-a5bc-b48ac1d369a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.709936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1b722-7a30-4717-a5bc-b48ac1d369a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.768828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2ae27-8cd3-4e2d-8d89-d60bbdd75b80 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.769445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2ae27-8cd3-4e2d-8d89-d60bbdd75b80 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.769716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2ae27-8cd3-4e2d-8d89-d60bbdd75b80 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.769977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2ae27-8cd3-4e2d-8d89-d60bbdd75b80 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.770535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2ae27-8cd3-4e2d-8d89-d60bbdd75b80 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.776836 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-53f2ae27-8cd3-4e2d-8d89-d60bbdd75b80 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:45.776836 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:26:45.776836 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:26:45.776836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2ae27-8cd3-4e2d-8d89-d60bbdd75b80 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.777614 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3935/7783] 10.222.0.23 () {58 vars in 1173 bytes} [Sat Aug 20 21:26:45 2022] GET /identity/v3/domains/config/default => generated 1527 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:26:45.781340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.782713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:45.795366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b078dc-483d-4c0f-b3e2-c343dfbd380b tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.800847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803e013-997a-4c63-beee-7c8ac25fe891 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.801665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803e013-997a-4c63-beee-7c8ac25fe891 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.801942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803e013-997a-4c63-beee-7c8ac25fe891 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.802204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803e013-997a-4c63-beee-7c8ac25fe891 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/identity/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.802777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803e013-997a-4c63-beee-7c8ac25fe891 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.803775 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-0803e013-997a-4c63-beee-7c8ac25fe891 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:45.803775 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:26:45.803775 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:26:45.804791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803e013-997a-4c63-beee-7c8ac25fe891 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.806061 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3849/7784] 10.222.0.23 () {58 vars in 1200 bytes} [Sat Aug 20 21:26:45 2022] GET /identity/v3/domains/config/identity/default => generated 64 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:45.810309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2ae27-8cd3-4e2d-8d89-d60bbdd75b80 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.821764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2ae27-8cd3-4e2d-8d89-d60bbdd75b80 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.826450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe768c-d8a5-4d8f-badd-e5d4e00d64b6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.827134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe768c-d8a5-4d8f-badd-e5d4e00d64b6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.827411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe768c-d8a5-4d8f-badd-e5d4e00d64b6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.827680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe768c-d8a5-4d8f-badd-e5d4e00d64b6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/identity/driver/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.828267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe768c-d8a5-4d8f-badd-e5d4e00d64b6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=driver)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.829934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe768c-d8a5-4d8f-badd-e5d4e00d64b6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.831176 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3936/7785] 10.222.0.23 () {58 vars in 1221 bytes} [Sat Aug 20 21:26:45 2022] GET /identity/v3/domains/config/identity/driver/default => generated 30 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:45.834937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803e013-997a-4c63-beee-7c8ac25fe891 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.845133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803e013-997a-4c63-beee-7c8ac25fe891 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.849854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179992ae-75ff-4fcc-97f6-ae55038eb7c6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.850496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179992ae-75ff-4fcc-97f6-ae55038eb7c6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.850773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179992ae-75ff-4fcc-97f6-ae55038eb7c6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.851045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179992ae-75ff-4fcc-97f6-ae55038eb7c6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/identity/list_limit/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.851627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179992ae-75ff-4fcc-97f6-ae55038eb7c6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=list_limit)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.853257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179992ae-75ff-4fcc-97f6-ae55038eb7c6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.856411 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3850/7786] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:45 2022] GET /identity/v3/domains/config/identity/list_limit/default => generated 33 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:45.858325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe768c-d8a5-4d8f-badd-e5d4e00d64b6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.868289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe768c-d8a5-4d8f-badd-e5d4e00d64b6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.872976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79827339-cc7e-417c-94fc-792e4d067865 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.873697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827339-cc7e-417c-94fc-792e4d067865 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.874024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827339-cc7e-417c-94fc-792e4d067865 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.874290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827339-cc7e-417c-94fc-792e4d067865 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.874882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79827339-cc7e-417c-94fc-792e4d067865 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.876492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79827339-cc7e-417c-94fc-792e4d067865 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.878002 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3937/7787] 10.222.0.23 () {58 vars in 1188 bytes} [Sat Aug 20 21:26:45 2022] GET /identity/v3/domains/config/ldap/default => generated 1476 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:45.882021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179992ae-75ff-4fcc-97f6-ae55038eb7c6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.894131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179992ae-75ff-4fcc-97f6-ae55038eb7c6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.898951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409836d-c4f5-41ff-8495-4c446780fbcf tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.899759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409836d-c4f5-41ff-8495-4c446780fbcf tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.900333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409836d-c4f5-41ff-8495-4c446780fbcf tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.900627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409836d-c4f5-41ff-8495-4c446780fbcf tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/url/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.901202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409836d-c4f5-41ff-8495-4c446780fbcf tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=url)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.902843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409836d-c4f5-41ff-8495-4c446780fbcf tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.904045 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3851/7788] 10.222.0.23 () {58 vars in 1200 bytes} [Sat Aug 20 21:26:45 2022] GET /identity/v3/domains/config/ldap/url/default => generated 40 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:45.908470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79827339-cc7e-417c-94fc-792e4d067865 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.919149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79827339-cc7e-417c-94fc-792e4d067865 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.924248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d0be7-ddfc-4d0b-a8a2-511112f3dbc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.925004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d0be7-ddfc-4d0b-a8a2-511112f3dbc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.925277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d0be7-ddfc-4d0b-a8a2-511112f3dbc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.925562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d0be7-ddfc-4d0b-a8a2-511112f3dbc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.926137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d0be7-ddfc-4d0b-a8a2-511112f3dbc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.927718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d0be7-ddfc-4d0b-a8a2-511112f3dbc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.928876 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3938/7789] 10.222.0.23 () {58 vars in 1203 bytes} [Sat Aug 20 21:26:45 2022] GET /identity/v3/domains/config/ldap/user/default => generated 27 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:45.932934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409836d-c4f5-41ff-8495-4c446780fbcf tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.943926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409836d-c4f5-41ff-8495-4c446780fbcf tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.948701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c11894-490b-49fc-81a8-af21d8f97afd tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.949339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c11894-490b-49fc-81a8-af21d8f97afd tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.949614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c11894-490b-49fc-81a8-af21d8f97afd tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.949990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c11894-490b-49fc-81a8-af21d8f97afd tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/suffix/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.950596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c11894-490b-49fc-81a8-af21d8f97afd tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=suffix)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.952306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c11894-490b-49fc-81a8-af21d8f97afd tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.953778 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3852/7790] 10.222.0.23 () {58 vars in 1209 bytes} [Sat Aug 20 21:26:45 2022] GET /identity/v3/domains/config/ldap/suffix/default => generated 44 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Aug 20 21:26:45.957551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d0be7-ddfc-4d0b-a8a2-511112f3dbc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.964185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:45.968675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d0be7-ddfc-4d0b-a8a2-511112f3dbc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:45.975603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12398cca-502b-4bc4-b9c8-15e5dbab3420 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:45.976344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12398cca-502b-4bc4-b9c8-15e5dbab3420 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:45.976681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12398cca-502b-4bc4-b9c8-15e5dbab3420 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:45.977037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12398cca-502b-4bc4-b9c8-15e5dbab3420 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/query_scope/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:45.977625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12398cca-502b-4bc4-b9c8-15e5dbab3420 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=query_scope)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:45.979539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12398cca-502b-4bc4-b9c8-15e5dbab3420 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:45.981032 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3939/7791] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:45 2022] GET /identity/v3/domains/config/ldap/query_scope/default => generated 35 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:45.985065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c11894-490b-49fc-81a8-af21d8f97afd tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:45.996025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c11894-490b-49fc-81a8-af21d8f97afd tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.001638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e7298-1036-4859-84ff-bb9a5e542525 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.002245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e7298-1036-4859-84ff-bb9a5e542525 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.002515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e7298-1036-4859-84ff-bb9a5e542525 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.002773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e7298-1036-4859-84ff-bb9a5e542525 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/page_size/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.003373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e7298-1036-4859-84ff-bb9a5e542525 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=page_size)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.005073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e7298-1036-4859-84ff-bb9a5e542525 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.006287 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3853/7792] 10.222.0.23 () {58 vars in 1218 bytes} [Sat Aug 20 21:26:45 2022] GET /identity/v3/domains/config/ldap/page_size/default => generated 29 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.010262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12398cca-502b-4bc4-b9c8-15e5dbab3420 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.020754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12398cca-502b-4bc4-b9c8-15e5dbab3420 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.026275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0362ec7-4b2c-4091-99d6-d6efefa08808 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.027030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0362ec7-4b2c-4091-99d6-d6efefa08808 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.027309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0362ec7-4b2c-4091-99d6-d6efefa08808 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.027568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0362ec7-4b2c-4091-99d6-d6efefa08808 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/alias_dereferencing/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.028142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0362ec7-4b2c-4091-99d6-d6efefa08808 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=alias_dereferencing)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.029838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0362ec7-4b2c-4091-99d6-d6efefa08808 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.033914 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3940/7793] 10.222.0.23 () {58 vars in 1248 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/alias_dereferencing/default => generated 47 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.035373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e7298-1036-4859-84ff-bb9a5e542525 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.046190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e7298-1036-4859-84ff-bb9a5e542525 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.050971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb11f89-4db9-4026-8a79-82036cb9042e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.051616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb11f89-4db9-4026-8a79-82036cb9042e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.051879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb11f89-4db9-4026-8a79-82036cb9042e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.052228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb11f89-4db9-4026-8a79-82036cb9042e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/debug_level/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.052814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb11f89-4db9-4026-8a79-82036cb9042e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=debug_level)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.054386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb11f89-4db9-4026-8a79-82036cb9042e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.055646 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3854/7794] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/debug_level/default => generated 34 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.059587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0362ec7-4b2c-4091-99d6-d6efefa08808 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.071089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0362ec7-4b2c-4091-99d6-d6efefa08808 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.076328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34622a-2611-4215-aad9-af3287f3a0ec tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.077138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34622a-2611-4215-aad9-af3287f3a0ec tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.077415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34622a-2611-4215-aad9-af3287f3a0ec tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.077688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34622a-2611-4215-aad9-af3287f3a0ec tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/chase_referrals/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.078270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34622a-2611-4215-aad9-af3287f3a0ec tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=chase_referrals)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.079867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34622a-2611-4215-aad9-af3287f3a0ec tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.081012 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3941/7795] 10.222.0.23 () {58 vars in 1236 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/chase_referrals/default => generated 38 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.085160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb11f89-4db9-4026-8a79-82036cb9042e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.095674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb11f89-4db9-4026-8a79-82036cb9042e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.100470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d26b2a-7a11-489c-ba48-d353b9c30e00 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.101122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d26b2a-7a11-489c-ba48-d353b9c30e00 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.101392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d26b2a-7a11-489c-ba48-d353b9c30e00 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.101655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d26b2a-7a11-489c-ba48-d353b9c30e00 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_tree_dn/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.102508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d26b2a-7a11-489c-ba48-d353b9c30e00 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_tree_dn)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.104093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d26b2a-7a11-489c-ba48-d353b9c30e00 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.105266 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3855/7796] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_tree_dn/default => generated 35 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.109462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34622a-2611-4215-aad9-af3287f3a0ec tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.119611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34622a-2611-4215-aad9-af3287f3a0ec tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.124758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd4e8d-7a9f-49e9-961d-44814b00daf2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.126331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd4e8d-7a9f-49e9-961d-44814b00daf2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.126331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd4e8d-7a9f-49e9-961d-44814b00daf2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.126331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd4e8d-7a9f-49e9-961d-44814b00daf2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_filter/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.126624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd4e8d-7a9f-49e9-961d-44814b00daf2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_filter)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.128254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd4e8d-7a9f-49e9-961d-44814b00daf2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.129567 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3942/7797] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_filter/default => generated 34 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Aug 20 21:26:46.133460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d26b2a-7a11-489c-ba48-d353b9c30e00 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.144594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d26b2a-7a11-489c-ba48-d353b9c30e00 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.150024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee86dc1-9e91-4397-8ebf-69686a22309c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.150663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee86dc1-9e91-4397-8ebf-69686a22309c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.150939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee86dc1-9e91-4397-8ebf-69686a22309c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.151209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee86dc1-9e91-4397-8ebf-69686a22309c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_objectclass/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.151793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee86dc1-9e91-4397-8ebf-69686a22309c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_objectclass)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.153578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee86dc1-9e91-4397-8ebf-69686a22309c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.154811 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3856/7798] 10.222.0.23 () {58 vars in 1239 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_objectclass/default => generated 50 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.162335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd4e8d-7a9f-49e9-961d-44814b00daf2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.171290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd4e8d-7a9f-49e9-961d-44814b00daf2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.176427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755da8d-443a-400a-8b8d-b70d5babae24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.177144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755da8d-443a-400a-8b8d-b70d5babae24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.177417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755da8d-443a-400a-8b8d-b70d5babae24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.177685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755da8d-443a-400a-8b8d-b70d5babae24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_id_attribute/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.178289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755da8d-443a-400a-8b8d-b70d5babae24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_id_attribute)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.179873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755da8d-443a-400a-8b8d-b70d5babae24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.181083 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3943/7799] 10.222.0.23 () {58 vars in 1242 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_id_attribute/default => generated 40 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.185029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee86dc1-9e91-4397-8ebf-69686a22309c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.195508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee86dc1-9e91-4397-8ebf-69686a22309c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.200085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6cdac-9228-45f8-837d-0179770edbff tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.200750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6cdac-9228-45f8-837d-0179770edbff tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.201021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6cdac-9228-45f8-837d-0179770edbff tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.201285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6cdac-9228-45f8-837d-0179770edbff tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_name_attribute/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.201967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6cdac-9228-45f8-837d-0179770edbff tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_name_attribute)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.203550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6cdac-9228-45f8-837d-0179770edbff tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.204754 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3857/7800] 10.222.0.23 () {58 vars in 1248 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_name_attribute/default => generated 42 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.208664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755da8d-443a-400a-8b8d-b70d5babae24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.219133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755da8d-443a-400a-8b8d-b70d5babae24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.228114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fef38-9459-47e9-90e5-c058735a6dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.228804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fef38-9459-47e9-90e5-c058735a6dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.229081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fef38-9459-47e9-90e5-c058735a6dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.229349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fef38-9459-47e9-90e5-c058735a6dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_mail_attribute/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.229932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756fef38-9459-47e9-90e5-c058735a6dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_mail_attribute)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.231536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756fef38-9459-47e9-90e5-c058735a6dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.235570 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3944/7801] 10.222.0.23 () {58 vars in 1248 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_mail_attribute/default => generated 44 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.236975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6cdac-9228-45f8-837d-0179770edbff tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.247957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6cdac-9228-45f8-837d-0179770edbff tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.252652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c0539-7460-4515-b824-0a3d081f498e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.253289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c0539-7460-4515-b824-0a3d081f498e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.253562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c0539-7460-4515-b824-0a3d081f498e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.253823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c0539-7460-4515-b824-0a3d081f498e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_description_attribute/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.254455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c0539-7460-4515-b824-0a3d081f498e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_description_attribute)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.256032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c0539-7460-4515-b824-0a3d081f498e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.257278 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3858/7802] 10.222.0.23 () {58 vars in 1269 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_description_attribute/default => generated 58 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.261315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fef38-9459-47e9-90e5-c058735a6dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.271796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fef38-9459-47e9-90e5-c058735a6dda tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.276829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429b472-9604-48da-90b5-f517db14346e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.277491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429b472-9604-48da-90b5-f517db14346e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.277751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429b472-9604-48da-90b5-f517db14346e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.278048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429b472-9604-48da-90b5-f517db14346e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_pass_attribute/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.278630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429b472-9604-48da-90b5-f517db14346e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_pass_attribute)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.280332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429b472-9604-48da-90b5-f517db14346e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.284402 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3945/7803] 10.222.0.23 () {58 vars in 1248 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_pass_attribute/default => generated 52 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.285719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c0539-7460-4515-b824-0a3d081f498e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.296351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c0539-7460-4515-b824-0a3d081f498e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.301834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3ce82-2a23-44a6-8be5-be5734e49c9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.302478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3ce82-2a23-44a6-8be5-be5734e49c9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.302761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3ce82-2a23-44a6-8be5-be5734e49c9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.303029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3ce82-2a23-44a6-8be5-be5734e49c9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_attribute/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.303669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3ce82-2a23-44a6-8be5-be5734e49c9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_attribute)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.305457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3ce82-2a23-44a6-8be5-be5734e49c9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.306694 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3859/7804] 10.222.0.23 () {58 vars in 1257 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_enabled_attribute/default => generated 50 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.311234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429b472-9604-48da-90b5-f517db14346e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.322840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429b472-9604-48da-90b5-f517db14346e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.328137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85247737-18a0-4bb0-9b90-860d407eb1fc tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.328946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85247737-18a0-4bb0-9b90-860d407eb1fc tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.329339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85247737-18a0-4bb0-9b90-860d407eb1fc tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.329703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85247737-18a0-4bb0-9b90-860d407eb1fc tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_invert/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.330498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85247737-18a0-4bb0-9b90-860d407eb1fc tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_invert)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.332922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85247737-18a0-4bb0-9b90-860d407eb1fc tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.334073 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3946/7805] 10.222.0.23 () {58 vars in 1248 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_enabled_invert/default => generated 43 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.338126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3ce82-2a23-44a6-8be5-be5734e49c9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.349201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3ce82-2a23-44a6-8be5-be5734e49c9d tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.354079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d0bb0-15aa-4ee8-b6a3-4181829d0340 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.354715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0bb0-15aa-4ee8-b6a3-4181829d0340 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.355013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0bb0-15aa-4ee8-b6a3-4181829d0340 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.355311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0bb0-15aa-4ee8-b6a3-4181829d0340 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_mask/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.355850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d0bb0-15aa-4ee8-b6a3-4181829d0340 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_mask)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.357452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d0bb0-15aa-4ee8-b6a3-4181829d0340 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.359845 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3860/7806] 10.222.0.23 () {58 vars in 1242 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_enabled_mask/default => generated 37 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.362698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85247737-18a0-4bb0-9b90-860d407eb1fc tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.372764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85247737-18a0-4bb0-9b90-860d407eb1fc tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.377621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d2348-b666-48f6-ba31-28801999b699 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.378267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d2348-b666-48f6-ba31-28801999b699 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.378541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d2348-b666-48f6-ba31-28801999b699 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.378809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d2348-b666-48f6-ba31-28801999b699 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_default/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.379422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d2348-b666-48f6-ba31-28801999b699 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_default)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.381074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d2348-b666-48f6-ba31-28801999b699 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.382212 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3947/7807] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_enabled_default/default => generated 45 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.386344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d0bb0-15aa-4ee8-b6a3-4181829d0340 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.396934 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d0bb0-15aa-4ee8-b6a3-4181829d0340 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.402257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b5a04-5d25-4e4f-8874-b7a85c9834d3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.402892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b5a04-5d25-4e4f-8874-b7a85c9834d3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.403165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b5a04-5d25-4e4f-8874-b7a85c9834d3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.403495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b5a04-5d25-4e4f-8874-b7a85c9834d3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_attribute_ignore/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.404081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b5a04-5d25-4e4f-8874-b7a85c9834d3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_attribute_ignore)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.405830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b5a04-5d25-4e4f-8874-b7a85c9834d3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.408416 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3861/7808] 10.222.0.23 () {58 vars in 1254 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_attribute_ignore/default => generated 62 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.411295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d2348-b666-48f6-ba31-28801999b699 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.421830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d2348-b666-48f6-ba31-28801999b699 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.426624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759645bb-90d2-4ef0-b4bf-c22ccd80ca3e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.427248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759645bb-90d2-4ef0-b4bf-c22ccd80ca3e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.427521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759645bb-90d2-4ef0-b4bf-c22ccd80ca3e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.427774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759645bb-90d2-4ef0-b4bf-c22ccd80ca3e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_default_project_id_attribute/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.428431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759645bb-90d2-4ef0-b4bf-c22ccd80ca3e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_default_project_id_attribute)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.430017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759645bb-90d2-4ef0-b4bf-c22ccd80ca3e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.431144 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3948/7809] 10.222.0.23 () {58 vars in 1290 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_default_project_id_attribute/default => generated 56 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.435138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b5a04-5d25-4e4f-8874-b7a85c9834d3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.446184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b5a04-5d25-4e4f-8874-b7a85c9834d3 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.451475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77e991-3416-4ef1-be4e-6cbd4d127d08 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.452135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77e991-3416-4ef1-be4e-6cbd4d127d08 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.452436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77e991-3416-4ef1-be4e-6cbd4d127d08 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.452895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77e991-3416-4ef1-be4e-6cbd4d127d08 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.453499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77e991-3416-4ef1-be4e-6cbd4d127d08 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.455064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77e991-3416-4ef1-be4e-6cbd4d127d08 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.456229 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3862/7810] 10.222.0.23 () {58 vars in 1257 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation/default => generated 46 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.460574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759645bb-90d2-4ef0-b4bf-c22ccd80ca3e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.471315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759645bb-90d2-4ef0-b4bf-c22ccd80ca3e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.475925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b7193-f800-42e1-85a8-2007618a839f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.476592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b7193-f800-42e1-85a8-2007618a839f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.476865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b7193-f800-42e1-85a8-2007618a839f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.477132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b7193-f800-42e1-85a8-2007618a839f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_dn/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.477709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b7193-f800-42e1-85a8-2007618a839f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_dn)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.479305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b7193-f800-42e1-85a8-2007618a839f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.480399 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3949/7811] 10.222.0.23 () {58 vars in 1266 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_dn/default => generated 48 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.484560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77e991-3416-4ef1-be4e-6cbd4d127d08 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.494811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77e991-3416-4ef1-be4e-6cbd4d127d08 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.499860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b862e01-971c-4eaa-8bec-6bdde42c549f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.500544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b862e01-971c-4eaa-8bec-6bdde42c549f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.500823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b862e01-971c-4eaa-8bec-6bdde42c549f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.501092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b862e01-971c-4eaa-8bec-6bdde42c549f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.501675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b862e01-971c-4eaa-8bec-6bdde42c549f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_use_group_config)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.503249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b862e01-971c-4eaa-8bec-6bdde42c549f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.504714 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3863/7812] 10.222.0.23 () {58 vars in 1308 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default => generated 63 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.508497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b7193-f800-42e1-85a8-2007618a839f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.518563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b7193-f800-42e1-85a8-2007618a839f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.523256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d1b30-c755-4391-b491-ef924b1bff24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.523898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d1b30-c755-4391-b491-ef924b1bff24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.524169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d1b30-c755-4391-b491-ef924b1bff24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.524465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d1b30-c755-4391-b491-ef924b1bff24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_additional_attribute_mapping/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.525046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d1b30-c755-4391-b491-ef924b1bff24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_additional_attribute_mapping)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.526954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d1b30-c755-4391-b491-ef924b1bff24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.528020 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3950/7813] 10.222.0.23 () {58 vars in 1290 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/user_additional_attribute_mapping/default => generated 54 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.534940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b862e01-971c-4eaa-8bec-6bdde42c549f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.550033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b862e01-971c-4eaa-8bec-6bdde42c549f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.554876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6d0ef-aa7c-448d-a088-1d3fe674fd04 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.555520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6d0ef-aa7c-448d-a088-1d3fe674fd04 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.555794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6d0ef-aa7c-448d-a088-1d3fe674fd04 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.556065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6d0ef-aa7c-448d-a088-1d3fe674fd04 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_tree_dn/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.556682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6d0ef-aa7c-448d-a088-1d3fe674fd04 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_tree_dn)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.558250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6d0ef-aa7c-448d-a088-1d3fe674fd04 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.562331 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3864/7814] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/group_tree_dn/default => generated 36 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.563469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d1b30-c755-4391-b491-ef924b1bff24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.574440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d1b30-c755-4391-b491-ef924b1bff24 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.579477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fea76-3590-47e1-9ec2-a4ac9e56a407 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.580124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fea76-3590-47e1-9ec2-a4ac9e56a407 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.580440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fea76-3590-47e1-9ec2-a4ac9e56a407 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.580719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fea76-3590-47e1-9ec2-a4ac9e56a407 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_filter/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.581300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fea76-3590-47e1-9ec2-a4ac9e56a407 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_filter)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.582896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fea76-3590-47e1-9ec2-a4ac9e56a407 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.583989 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3951/7815] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/group_filter/default => generated 35 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Aug 20 21:26:46.587982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6d0ef-aa7c-448d-a088-1d3fe674fd04 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.598942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6d0ef-aa7c-448d-a088-1d3fe674fd04 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.604626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7073c4-bb8d-4fc2-8c69-97593e56ba55 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.605266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7073c4-bb8d-4fc2-8c69-97593e56ba55 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.605509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7073c4-bb8d-4fc2-8c69-97593e56ba55 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.605774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7073c4-bb8d-4fc2-8c69-97593e56ba55 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_objectclass/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.606357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7073c4-bb8d-4fc2-8c69-97593e56ba55 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_objectclass)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.608082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7073c4-bb8d-4fc2-8c69-97593e56ba55 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.610065 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3865/7816] 10.222.0.23 () {58 vars in 1242 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/group_objectclass/default => generated 50 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.614080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fea76-3590-47e1-9ec2-a4ac9e56a407 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.624730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fea76-3590-47e1-9ec2-a4ac9e56a407 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.631332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c6b99-4a7d-4aa4-93ca-be53b4267b0f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.631972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c6b99-4a7d-4aa4-93ca-be53b4267b0f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.632400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c6b99-4a7d-4aa4-93ca-be53b4267b0f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.632895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c6b99-4a7d-4aa4-93ca-be53b4267b0f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_id_attribute/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.633836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c6b99-4a7d-4aa4-93ca-be53b4267b0f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_id_attribute)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.635431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c6b99-4a7d-4aa4-93ca-be53b4267b0f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.636576 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3952/7817] 10.222.0.23 () {58 vars in 1245 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/group_id_attribute/default => generated 41 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.641041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7073c4-bb8d-4fc2-8c69-97593e56ba55 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.653591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7073c4-bb8d-4fc2-8c69-97593e56ba55 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.658436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe97d0c-11b8-40d2-8eaa-cc44c9bb0d22 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.659073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe97d0c-11b8-40d2-8eaa-cc44c9bb0d22 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.659345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe97d0c-11b8-40d2-8eaa-cc44c9bb0d22 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.659610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe97d0c-11b8-40d2-8eaa-cc44c9bb0d22 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_name_attribute/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.660223 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe97d0c-11b8-40d2-8eaa-cc44c9bb0d22 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_name_attribute)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.661859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe97d0c-11b8-40d2-8eaa-cc44c9bb0d22 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.663005 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3866/7818] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/group_name_attribute/default => generated 43 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Aug 20 21:26:46.667046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c6b99-4a7d-4aa4-93ca-be53b4267b0f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.679086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c6b99-4a7d-4aa4-93ca-be53b4267b0f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.683904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6e985-584b-4185-aa34-ae6bb4a74491 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.684725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6e985-584b-4185-aa34-ae6bb4a74491 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.685000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6e985-584b-4185-aa34-ae6bb4a74491 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.685270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6e985-584b-4185-aa34-ae6bb4a74491 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_members_are_ids/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.685854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6e985-584b-4185-aa34-ae6bb4a74491 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_members_are_ids)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.688325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6e985-584b-4185-aa34-ae6bb4a74491 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.689461 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3953/7819] 10.222.0.23 () {58 vars in 1254 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/group_members_are_ids/default => generated 45 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.693759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe97d0c-11b8-40d2-8eaa-cc44c9bb0d22 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.704763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe97d0c-11b8-40d2-8eaa-cc44c9bb0d22 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.709569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05dd5d-9b54-4a74-991a-020406d0dc4a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.710221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05dd5d-9b54-4a74-991a-020406d0dc4a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.710495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05dd5d-9b54-4a74-991a-020406d0dc4a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.710818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05dd5d-9b54-4a74-991a-020406d0dc4a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_member_attribute/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.711400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05dd5d-9b54-4a74-991a-020406d0dc4a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_member_attribute)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.713201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05dd5d-9b54-4a74-991a-020406d0dc4a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.714557 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3867/7820] 10.222.0.23 () {58 vars in 1257 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/group_member_attribute/default => generated 49 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.718906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6e985-584b-4185-aa34-ae6bb4a74491 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.729114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6e985-584b-4185-aa34-ae6bb4a74491 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.734455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e61c1-3c3a-4043-873b-da98a6dd90c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.735122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e61c1-3c3a-4043-873b-da98a6dd90c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.735364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e61c1-3c3a-4043-873b-da98a6dd90c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.736227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e61c1-3c3a-4043-873b-da98a6dd90c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_desc_attribute/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.736227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e61c1-3c3a-4043-873b-da98a6dd90c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_desc_attribute)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.737865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e61c1-3c3a-4043-873b-da98a6dd90c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.739214 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3954/7821] 10.222.0.23 () {58 vars in 1251 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/group_desc_attribute/default => generated 52 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Aug 20 21:26:46.743220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05dd5d-9b54-4a74-991a-020406d0dc4a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.753666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05dd5d-9b54-4a74-991a-020406d0dc4a tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.758274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefea2c-5138-47b4-bb3e-4e908d1989c0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.758905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefea2c-5138-47b4-bb3e-4e908d1989c0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.759180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefea2c-5138-47b4-bb3e-4e908d1989c0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.759442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefea2c-5138-47b4-bb3e-4e908d1989c0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_attribute_ignore/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.760423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefea2c-5138-47b4-bb3e-4e908d1989c0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_attribute_ignore)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.762001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefea2c-5138-47b4-bb3e-4e908d1989c0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.763169 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3868/7822] 10.222.0.23 () {58 vars in 1257 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/group_attribute_ignore/default => generated 43 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.768303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e61c1-3c3a-4043-873b-da98a6dd90c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.779041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e61c1-3c3a-4043-873b-da98a6dd90c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.785656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc6688-638a-40fc-82ea-041299a50dea tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.786529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc6688-638a-40fc-82ea-041299a50dea tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.786805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc6688-638a-40fc-82ea-041299a50dea tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.787075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc6688-638a-40fc-82ea-041299a50dea tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_additional_attribute_mapping/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.787660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc6688-638a-40fc-82ea-041299a50dea tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_additional_attribute_mapping)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.789443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc6688-638a-40fc-82ea-041299a50dea tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.793612 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3955/7823] 10.222.0.23 () {58 vars in 1293 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/group_additional_attribute_mapping/default => generated 55 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.794866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefea2c-5138-47b4-bb3e-4e908d1989c0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.805774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefea2c-5138-47b4-bb3e-4e908d1989c0 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.811049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052debe9-2e8b-4402-b07d-63ea31368daa tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.811697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052debe9-2e8b-4402-b07d-63ea31368daa tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.812075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052debe9-2e8b-4402-b07d-63ea31368daa tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.812416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052debe9-2e8b-4402-b07d-63ea31368daa tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertfile/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.813015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052debe9-2e8b-4402-b07d-63ea31368daa tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertfile)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.814614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052debe9-2e8b-4402-b07d-63ea31368daa tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.817385 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3869/7824] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/tls_cacertfile/default => generated 37 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.820272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc6688-638a-40fc-82ea-041299a50dea tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.830984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc6688-638a-40fc-82ea-041299a50dea tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.835597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6d073-0bbc-4df8-bda3-02e420b39140 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.836214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6d073-0bbc-4df8-bda3-02e420b39140 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.836519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6d073-0bbc-4df8-bda3-02e420b39140 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.836784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6d073-0bbc-4df8-bda3-02e420b39140 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertdir/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.837354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6d073-0bbc-4df8-bda3-02e420b39140 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertdir)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.838937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6d073-0bbc-4df8-bda3-02e420b39140 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.840017 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3956/7825] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/tls_cacertdir/default => generated 36 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.844266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052debe9-2e8b-4402-b07d-63ea31368daa tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.855953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052debe9-2e8b-4402-b07d-63ea31368daa tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.862514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f2f8a-7a3a-46c5-8a33-1137c09d7fc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.863197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f2f8a-7a3a-46c5-8a33-1137c09d7fc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.863472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f2f8a-7a3a-46c5-8a33-1137c09d7fc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.863738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f2f8a-7a3a-46c5-8a33-1137c09d7fc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_tls/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.864327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f2f8a-7a3a-46c5-8a33-1137c09d7fc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_tls)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.865961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f2f8a-7a3a-46c5-8a33-1137c09d7fc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.870068 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3870/7826] 10.222.0.23 () {58 vars in 1212 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/use_tls/default => generated 31 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.871431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6d073-0bbc-4df8-bda3-02e420b39140 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.882712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6d073-0bbc-4df8-bda3-02e420b39140 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.890535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84797f80-de81-45e4-a4e5-f38b5eae998c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.891199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84797f80-de81-45e4-a4e5-f38b5eae998c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.891476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84797f80-de81-45e4-a4e5-f38b5eae998c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.891746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84797f80-de81-45e4-a4e5-f38b5eae998c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_req_cert/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.892343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84797f80-de81-45e4-a4e5-f38b5eae998c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_req_cert)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.894214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84797f80-de81-45e4-a4e5-f38b5eae998c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.895359 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3957/7827] 10.222.0.23 () {58 vars in 1227 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/tls_req_cert/default => generated 39 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.899816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f2f8a-7a3a-46c5-8a33-1137c09d7fc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.909949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f2f8a-7a3a-46c5-8a33-1137c09d7fc7 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.915010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86dbdcc-3c4e-4038-a45b-7db2bb7afea4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.915651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dbdcc-3c4e-4038-a45b-7db2bb7afea4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.916007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dbdcc-3c4e-4038-a45b-7db2bb7afea4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.916162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dbdcc-3c4e-4038-a45b-7db2bb7afea4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_pool/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.916973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86dbdcc-3c4e-4038-a45b-7db2bb7afea4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_pool)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.918535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86dbdcc-3c4e-4038-a45b-7db2bb7afea4 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.919897 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3871/7828] 10.222.0.23 () {58 vars in 1215 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/use_pool/default => generated 31 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.924056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84797f80-de81-45e4-a4e5-f38b5eae998c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.933122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11b690-a6e7-46db-a60c-1ff4cbfb8253 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.933395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11b690-a6e7-46db-a60c-1ff4cbfb8253 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.933658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11b690-a6e7-46db-a60c-1ff4cbfb8253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.935146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84797f80-de81-45e4-a4e5-f38b5eae998c tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.940560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9c79d-dc6d-4280-8151-33c4cf9c10a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.941192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9c79d-dc6d-4280-8151-33c4cf9c10a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.941469 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9c79d-dc6d-4280-8151-33c4cf9c10a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.941734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9c79d-dc6d-4280-8151-33c4cf9c10a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_size/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.942630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9c79d-dc6d-4280-8151-33c4cf9c10a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_size)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.944254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9c79d-dc6d-4280-8151-33c4cf9c10a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.948469 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3958/7829] 10.222.0.23 () {58 vars in 1218 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/pool_size/default => generated 30 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:46.949438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9c79d-dc6d-4280-8151-33c4cf9c10a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.954373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-da11b690-a6e7-46db-a60c-1ff4cbfb8253 None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:46.955559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-da11b690-a6e7-46db-a60c-1ff4cbfb8253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:46.959667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9c79d-dc6d-4280-8151-33c4cf9c10a6 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.964597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce77bb-2f46-4517-9c18-058031e49b74 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.965239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce77bb-2f46-4517-9c18-058031e49b74 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.965514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce77bb-2f46-4517-9c18-058031e49b74 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.965808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce77bb-2f46-4517-9c18-058031e49b74 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_max/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.966387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce77bb-2f46-4517-9c18-058031e49b74 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_max)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.968151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce77bb-2f46-4517-9c18-058031e49b74 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.969286 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3959/7830] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/pool_retry_max/default => generated 34 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Aug 20 21:26:46.973186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce77bb-2f46-4517-9c18-058031e49b74 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:46.984424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce77bb-2f46-4517-9c18-058031e49b74 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:46.989553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:46.989706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f26ab-c070-44ce-96ca-20098b852bf8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:46.990397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f26ab-c070-44ce-96ca-20098b852bf8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:46.990862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f26ab-c070-44ce-96ca-20098b852bf8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:46.991042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f26ab-c070-44ce-96ca-20098b852bf8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_delay/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:46.991781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f26ab-c070-44ce-96ca-20098b852bf8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_delay)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:46.993504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f26ab-c070-44ce-96ca-20098b852bf8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:46.995203 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3960/7831] 10.222.0.23 () {58 vars in 1239 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/pool_retry_delay/default => generated 38 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Aug 20 21:26:46.999368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f26ab-c070-44ce-96ca-20098b852bf8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:47.010043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f26ab-c070-44ce-96ca-20098b852bf8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:47.012791 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3872/7832] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:47.015615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3168b-b6b7-4b16-8261-a2c2a7d4fb39 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:47.016251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3168b-b6b7-4b16-8261-a2c2a7d4fb39 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:47.016567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3168b-b6b7-4b16-8261-a2c2a7d4fb39 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:47.017199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3168b-b6b7-4b16-8261-a2c2a7d4fb39 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_timeout/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:47.017786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3168b-b6b7-4b16-8261-a2c2a7d4fb39 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_timeout)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:47.019379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3168b-b6b7-4b16-8261-a2c2a7d4fb39 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:47.023171 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3961/7833] 10.222.0.23 () {58 vars in 1260 bytes} [Sat Aug 20 21:26:46 2022] GET /identity/v3/domains/config/ldap/pool_connection_timeout/default => generated 44 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:47.024260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11b690-a6e7-46db-a60c-1ff4cbfb8253 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:47.035007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11b690-a6e7-46db-a60c-1ff4cbfb8253 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:47.039865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acd4b4-b12f-4d67-bd7a-7873ba7d4c59 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:47.040602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acd4b4-b12f-4d67-bd7a-7873ba7d4c59 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:47.051254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acd4b4-b12f-4d67-bd7a-7873ba7d4c59 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:47.051254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acd4b4-b12f-4d67-bd7a-7873ba7d4c59 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_lifetime/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:47.051254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acd4b4-b12f-4d67-bd7a-7873ba7d4c59 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_lifetime)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:47.051254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acd4b4-b12f-4d67-bd7a-7873ba7d4c59 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:47.051254 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3873/7834] 10.222.0.23 () {58 vars in 1263 bytes} [Sat Aug 20 21:26:47 2022] GET /identity/v3/domains/config/ldap/pool_connection_lifetime/default => generated 46 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:47.051708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3168b-b6b7-4b16-8261-a2c2a7d4fb39 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:47.058978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3168b-b6b7-4b16-8261-a2c2a7d4fb39 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:47.064265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4e5af-4706-4dce-a130-06d6d29f565f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:47.064874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4e5af-4706-4dce-a130-06d6d29f565f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:47.065151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4e5af-4706-4dce-a130-06d6d29f565f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:47.065422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4e5af-4706-4dce-a130-06d6d29f565f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_auth_pool/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:47.066001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4e5af-4706-4dce-a130-06d6d29f565f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_auth_pool)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:47.067907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4e5af-4706-4dce-a130-06d6d29f565f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:47.069173 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3962/7835] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:26:47 2022] GET /identity/v3/domains/config/ldap/use_auth_pool/default => generated 36 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:47.073009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acd4b4-b12f-4d67-bd7a-7873ba7d4c59 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:47.086322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acd4b4-b12f-4d67-bd7a-7873ba7d4c59 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:47.092444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1be599-c78a-49f0-be8e-925e889331c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:47.093406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1be599-c78a-49f0-be8e-925e889331c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:47.093406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1be599-c78a-49f0-be8e-925e889331c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:47.093780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1be599-c78a-49f0-be8e-925e889331c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_size/default` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:47.094276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1be599-c78a-49f0-be8e-925e889331c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_size)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:47.095947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1be599-c78a-49f0-be8e-925e889331c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:47.097905 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3874/7836] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:47 2022] GET /identity/v3/domains/config/ldap/auth_pool_size/default => generated 36 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:47.156234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4e5af-4706-4dce-a130-06d6d29f565f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:47.166474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4e5af-4706-4dce-a130-06d6d29f565f tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:47.173537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84085ca-8201-4a3a-9d93-8d1991c6a227 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:47.174213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84085ca-8201-4a3a-9d93-8d1991c6a227 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:47.174489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84085ca-8201-4a3a-9d93-8d1991c6a227 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:47.174761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84085ca-8201-4a3a-9d93-8d1991c6a227 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_connection_lifetime/default` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:47.175362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84085ca-8201-4a3a-9d93-8d1991c6a227 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_connection_lifetime)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:47.177113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84085ca-8201-4a3a-9d93-8d1991c6a227 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:47.180179 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3963/7837] 10.222.0.23 () {58 vars in 1278 bytes} [Sat Aug 20 21:26:47 2022] GET /identity/v3/domains/config/ldap/auth_pool_connection_lifetime/default => generated 50 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Aug 20 21:26:47.186451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1be599-c78a-49f0-be8e-925e889331c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:47.196875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1be599-c78a-49f0-be8e-925e889331c2 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:47.201584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c2c2b-8578-4eed-9f7d-52883d140a28 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:47.202314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c2c2b-8578-4eed-9f7d-52883d140a28 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:47.202674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c2c2b-8578-4eed-9f7d-52883d140a28 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:47.202862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c2c2b-8578-4eed-9f7d-52883d140a28 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:47.210128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c2c2b-8578-4eed-9f7d-52883d140a28 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-620c2c2b-8578-4eed-9f7d-52883d140a28 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Could not find domain: db19c0f5790d4371b59c2ca3857bc4c8.: keystone.exception.DomainNotFound: Could not find domain: db19c0f5790d4371b59c2ca3857bc4c8. Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: db19c0f5790d4371b59c2ca3857bc4c8. Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:26:47.215086 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:26:47.219773 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:26:47.221063 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: db19c0f5790d4371b59c2ca3857bc4c8. Aug 20 21:26:47.222354 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:26:47.222354 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3875/7838] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:26:47 2022] PATCH /identity/v3/domains/db19c0f5790d4371b59c2ca3857bc4c8 => generated 112 bytes in 32 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:26:47.222535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84085ca-8201-4a3a-9d93-8d1991c6a227 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:47.238856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84085ca-8201-4a3a-9d93-8d1991c6a227 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:47.243739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fdff5-cf26-4e48-ad85-63d4db9b719e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:47.244388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fdff5-cf26-4e48-ad85-63d4db9b719e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:47.244692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fdff5-cf26-4e48-ad85-63d4db9b719e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:47.244956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fdff5-cf26-4e48-ad85-63d4db9b719e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/6189f352e64d4c2aab62ffdfb8f76624` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:47.257105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fdff5-cf26-4e48-ad85-63d4db9b719e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application [None req-510fdff5-cf26-4e48-ad85-63d4db9b719e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Could not find domain: 6189f352e64d4c2aab62ffdfb8f76624.: keystone.exception.DomainNotFound: Could not find domain: 6189f352e64d4c2aab62ffdfb8f76624. Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 6189f352e64d4c2aab62ffdfb8f76624. Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:26:47.262370 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:26:47.263601 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application with Lock( Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:26:47.264758 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 6189f352e64d4c2aab62ffdfb8f76624. Aug 20 21:26:47.265841 ncdv-master-1 devstack@keystone.service[104118]: ERROR keystone.server.flask.application Aug 20 21:26:47.265841 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3964/7839] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:26:47 2022] PATCH /identity/v3/domains/6189f352e64d4c2aab62ffdfb8f76624 => generated 112 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:26:47.268304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c2c2b-8578-4eed-9f7d-52883d140a28 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:47.278889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c2c2b-8578-4eed-9f7d-52883d140a28 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:47.284918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e652e-2713-4474-b58c-093540e691a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f7bfd357ce47ae8c1292f95c5bca1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dac2790fa54ffc82a750d113b0659b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:47.285661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e652e-2713-4474-b58c-093540e691a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:47.285941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e652e-2713-4474-b58c-093540e691a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:47.286209 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e652e-2713-4474-b58c-093540e691a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] PATH_INFO: `/v3/domains/b67415b0ffe94085895953c0c5a96779` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:47.294074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e652e-2713-4474-b58c-093540e691a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-557e652e-2713-4474-b58c-093540e691a8 tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Could not find domain: b67415b0ffe94085895953c0c5a96779.: keystone.exception.DomainNotFound: Could not find domain: b67415b0ffe94085895953c0c5a96779. Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: b67415b0ffe94085895953c0c5a96779. Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:26:47.300305 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:26:47.301708 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 20 21:26:47.302952 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: b67415b0ffe94085895953c0c5a96779. Aug 20 21:26:47.304226 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:26:47.304226 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3876/7840] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:26:47 2022] PATCH /identity/v3/domains/b67415b0ffe94085895953c0c5a96779 => generated 112 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Aug 20 21:26:47.305847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fdff5-cf26-4e48-ad85-63d4db9b719e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:47.307933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-510fdff5-cf26-4e48-ad85-63d4db9b719e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:47.376646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fdff5-cf26-4e48-ad85-63d4db9b719e tempest-DomainConfigurationTestJSON-1170281887 tempest-DomainConfigurationTestJSON-1170281887-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:47.382089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00209aba-1532-42db-a1ce-2d395589f102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:47.382759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00209aba-1532-42db-a1ce-2d395589f102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:47.383049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00209aba-1532-42db-a1ce-2d395589f102 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:47.383450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00209aba-1532-42db-a1ce-2d395589f102 admin admin] PATH_INFO: `/v3/users/8a4c54d51bcc43538a655b3eeea4709a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:47.384672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00209aba-1532-42db-a1ce-2d395589f102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a4c54d51bcc43538a655b3eeea4709a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:47.386628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00209aba-1532-42db-a1ce-2d395589f102 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:47.440349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-00209aba-1532-42db-a1ce-2d395589f102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a4c54d51bcc43538a655b3eeea4709a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:47.442389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:47.449561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-00209aba-1532-42db-a1ce-2d395589f102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a4c54d51bcc43538a655b3eeea4709a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:47.456906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-00209aba-1532-42db-a1ce-2d395589f102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a4c54d51bcc43538a655b3eeea4709a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:47.459319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7ce81-ab02-4867-9811-73ed7e7fb91e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:47.459604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7ce81-ab02-4867-9811-73ed7e7fb91e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:47.460044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7ce81-ab02-4867-9811-73ed7e7fb91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:47.470798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-00209aba-1532-42db-a1ce-2d395589f102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a4c54d51bcc43538a655b3eeea4709a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:47.471544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-00209aba-1532-42db-a1ce-2d395589f102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a4c54d51bcc43538a655b3eeea4709a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:47.477941 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3965/7841] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:47 2022] DELETE /identity/v3/users/8a4c54d51bcc43538a655b3eeea4709a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:47.479964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1be7ce81-ab02-4867-9811-73ed7e7fb91e None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:47.481257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1be7ce81-ab02-4867-9811-73ed7e7fb91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:47.482217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2a31d-fcc4-4f09-b868-e224af045b19 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:47.482499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2a31d-fcc4-4f09-b868-e224af045b19 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:47.482769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2a31d-fcc4-4f09-b868-e224af045b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:47.502710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-dee2a31d-fcc4-4f09-b868-e224af045b19 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:47.503898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dee2a31d-fcc4-4f09-b868-e224af045b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:47.541369 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3877/7842] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:47.573619 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3966/7843] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:47.583671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7ce81-ab02-4867-9811-73ed7e7fb91e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:47.585199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1be7ce81-ab02-4867-9811-73ed7e7fb91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:47.645830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7ce81-ab02-4867-9811-73ed7e7fb91e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:47.649457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f642ae-622f-452f-bfe4-c59c1e4e34f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:47.650060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f642ae-622f-452f-bfe4-c59c1e4e34f6 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:47.650450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f642ae-622f-452f-bfe4-c59c1e4e34f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:47.650715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f642ae-622f-452f-bfe4-c59c1e4e34f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:47.660354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f642ae-622f-452f-bfe4-c59c1e4e34f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:47.661908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f642ae-622f-452f-bfe4-c59c1e4e34f6 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:47.672929 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3878/7844] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:26:47.960762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2a31d-fcc4-4f09-b868-e224af045b19 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:47.962404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dee2a31d-fcc4-4f09-b868-e224af045b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:47.982421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2a31d-fcc4-4f09-b868-e224af045b19 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:47.997442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:47.998135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:47.998526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:47.998780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] PATH_INFO: `/v3/projects/990945c66e034adeb85da52dc10bee8c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:48.000099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=990945c66e034adeb85da52dc10bee8c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:48.002461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:48.053135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990945c66e034adeb85da52dc10bee8c', 'request_id': 'req-0448367d-5b87-429d-a4b8-95ebdccc06ad'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.061243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990945c66e034adeb85da52dc10bee8c', 'request_id': 'req-0448367d-5b87-429d-a4b8-95ebdccc06ad'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.067708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990945c66e034adeb85da52dc10bee8c', 'request_id': 'req-0448367d-5b87-429d-a4b8-95ebdccc06ad'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.080216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] The token cache is being invalidate because project 990945c66e034adeb85da52dc10bee8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:48.080520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.081566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.082660 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3967/7845] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:47 2022] DELETE /identity/v3/projects/990945c66e034adeb85da52dc10bee8c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:48.086820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f642ae-622f-452f-bfe4-c59c1e4e34f6 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:48.096902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f642ae-622f-452f-bfe4-c59c1e4e34f6 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:48.156612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa6f7e-7513-405c-8edf-679600c96f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:48.157351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6f7e-7513-405c-8edf-679600c96f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:48.157613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6f7e-7513-405c-8edf-679600c96f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:48.157905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6f7e-7513-405c-8edf-679600c96f9f admin admin] PATH_INFO: `/v3/users/51f7bfd357ce47ae8c1292f95c5bca1e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:48.159093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa6f7e-7513-405c-8edf-679600c96f9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=51f7bfd357ce47ae8c1292f95c5bca1e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:48.161076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa6f7e-7513-405c-8edf-679600c96f9f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:48.167348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:48.209615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9cfa6f7e-7513-405c-8edf-679600c96f9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51f7bfd357ce47ae8c1292f95c5bca1e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.217788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9cfa6f7e-7513-405c-8edf-679600c96f9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51f7bfd357ce47ae8c1292f95c5bca1e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.240718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9cfa6f7e-7513-405c-8edf-679600c96f9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51f7bfd357ce47ae8c1292f95c5bca1e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.241532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9cfa6f7e-7513-405c-8edf-679600c96f9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51f7bfd357ce47ae8c1292f95c5bca1e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.242192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9cfa6f7e-7513-405c-8edf-679600c96f9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51f7bfd357ce47ae8c1292f95c5bca1e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.255058 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3879/7846] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:48 2022] DELETE /identity/v3/users/51f7bfd357ce47ae8c1292f95c5bca1e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:48.646432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:48.650256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:48.670443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448367d-5b87-429d-a4b8-95ebdccc06ad admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:48.718837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:48.733442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9eade8-4512-4525-96b5-0dba68726993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:48.734081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9eade8-4512-4525-96b5-0dba68726993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:48.734402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9eade8-4512-4525-96b5-0dba68726993 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:48.734672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9eade8-4512-4525-96b5-0dba68726993 admin admin] PATH_INFO: `/v3/projects/97dac2790fa54ffc82a750d113b0659b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:48.735795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9eade8-4512-4525-96b5-0dba68726993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97dac2790fa54ffc82a750d113b0659b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:48.737794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9eade8-4512-4525-96b5-0dba68726993 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:48.785320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1d9eade8-4512-4525-96b5-0dba68726993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97dac2790fa54ffc82a750d113b0659b', 'request_id': 'req-1d9eade8-4512-4525-96b5-0dba68726993'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.793884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1d9eade8-4512-4525-96b5-0dba68726993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97dac2790fa54ffc82a750d113b0659b', 'request_id': 'req-1d9eade8-4512-4525-96b5-0dba68726993'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.800186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1d9eade8-4512-4525-96b5-0dba68726993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97dac2790fa54ffc82a750d113b0659b', 'request_id': 'req-1d9eade8-4512-4525-96b5-0dba68726993'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.811374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1d9eade8-4512-4525-96b5-0dba68726993 admin admin] The token cache is being invalidate because project 97dac2790fa54ffc82a750d113b0659b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:48.811659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1d9eade8-4512-4525-96b5-0dba68726993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.812313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1d9eade8-4512-4525-96b5-0dba68726993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:48.813873 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3968/7847] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:48 2022] DELETE /identity/v3/projects/97dac2790fa54ffc82a750d113b0659b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:48.819124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292e3f-d540-4e50-a106-250088ab7b67 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:48.819404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292e3f-d540-4e50-a106-250088ab7b67 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:48.819673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292e3f-d540-4e50-a106-250088ab7b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:48.841092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-65292e3f-d540-4e50-a106-250088ab7b67 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:48.842390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-65292e3f-d540-4e50-a106-250088ab7b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:48.909006 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3880/7848] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:48.917014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5a24a-29f9-4396-9820-430b5687a172 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:48.917296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5a24a-29f9-4396-9820-430b5687a172 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:48.917540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5a24a-29f9-4396-9820-430b5687a172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:48.938748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-71e5a24a-29f9-4396-9820-430b5687a172 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:48.939942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-71e5a24a-29f9-4396-9820-430b5687a172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:48.950344 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3969/7849] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:48.954834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65292e3f-d540-4e50-a106-250088ab7b67 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:48.964839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65292e3f-d540-4e50-a106-250088ab7b67 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:48.969566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30a492-621e-4d1f-8386-659c476756ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:48.970140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30a492-621e-4d1f-8386-659c476756ee admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:48.970411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30a492-621e-4d1f-8386-659c476756ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:48.970670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30a492-621e-4d1f-8386-659c476756ee admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:48.971307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30a492-621e-4d1f-8386-659c476756ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:48.971602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30a492-621e-4d1f-8386-659c476756ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:48.973232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30a492-621e-4d1f-8386-659c476756ee admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:48.985560 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3881/7850] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:48.989997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5a24a-29f9-4396-9820-430b5687a172 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:48.999958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5a24a-29f9-4396-9820-430b5687a172 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:49.005348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7a628-9f17-41aa-9e0f-f0ba5719826f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:49.005944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7a628-9f17-41aa-9e0f-f0ba5719826f admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:49.006188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7a628-9f17-41aa-9e0f-f0ba5719826f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:49.006455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7a628-9f17-41aa-9e0f-f0ba5719826f admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:49.007084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7a628-9f17-41aa-9e0f-f0ba5719826f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:49.009040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7a628-9f17-41aa-9e0f-f0ba5719826f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:49.037847 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3970/7851] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:48 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:49.042091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30a492-621e-4d1f-8386-659c476756ee admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:49.052856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30a492-621e-4d1f-8386-659c476756ee admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:49.121396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e956618-2bfa-4c79-95fa-a8e76069cfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:49.122112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e956618-2bfa-4c79-95fa-a8e76069cfff admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:49.122214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e956618-2bfa-4c79-95fa-a8e76069cfff admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:49.122518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e956618-2bfa-4c79-95fa-a8e76069cfff admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:49.123140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e956618-2bfa-4c79-95fa-a8e76069cfff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:49.125155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e956618-2bfa-4c79-95fa-a8e76069cfff admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:49.149343 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3882/7852] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:49 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:49.153889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7a628-9f17-41aa-9e0f-f0ba5719826f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:49.164602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7a628-9f17-41aa-9e0f-f0ba5719826f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:49.169432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2b01a-7968-4a73-a87a-0c4a7cf9f641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:49.170020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2b01a-7968-4a73-a87a-0c4a7cf9f641 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:49.170282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2b01a-7968-4a73-a87a-0c4a7cf9f641 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:49.170623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2b01a-7968-4a73-a87a-0c4a7cf9f641 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:49.171352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2b01a-7968-4a73-a87a-0c4a7cf9f641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:49.173010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2b01a-7968-4a73-a87a-0c4a7cf9f641 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:49.183624 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3971/7853] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:26:49.187376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e956618-2bfa-4c79-95fa-a8e76069cfff admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:49.193225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:49.199650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e956618-2bfa-4c79-95fa-a8e76069cfff admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:49.205409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe4dfe-6569-4ffe-8ca9-bf56c42f74ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:49.206007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe4dfe-6569-4ffe-8ca9-bf56c42f74ec admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:49.206280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe4dfe-6569-4ffe-8ca9-bf56c42f74ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:49.206545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe4dfe-6569-4ffe-8ca9-bf56c42f74ec admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:49.207355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe4dfe-6569-4ffe-8ca9-bf56c42f74ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:49.208985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe4dfe-6569-4ffe-8ca9-bf56c42f74ec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:49.219851 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3883/7854] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:26:49.225166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2b01a-7968-4a73-a87a-0c4a7cf9f641 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:49.236331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2b01a-7968-4a73-a87a-0c4a7cf9f641 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:49.241579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc4deb-aa33-4d16-a095-811c51f334d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:49.242233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc4deb-aa33-4d16-a095-811c51f334d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:49.242505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc4deb-aa33-4d16-a095-811c51f334d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:49.242773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc4deb-aa33-4d16-a095-811c51f334d6 admin admin] PATH_INFO: `/v3/projects/e8fae6d0141447e4b42684073fb1f4d6/users/0c7d0408e5f1413985048424fee0168d/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:49.258187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc4deb-aa33-4d16-a095-811c51f334d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8fae6d0141447e4b42684073fb1f4d6, user_id=0c7d0408e5f1413985048424fee0168d, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:49.260897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc4deb-aa33-4d16-a095-811c51f334d6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:49.275263 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3972/7855] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:49 2022] PUT /identity/v3/projects/e8fae6d0141447e4b42684073fb1f4d6/users/0c7d0408e5f1413985048424fee0168d/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:49.286923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578cab4-5cb9-4571-b75b-20515f7f113b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:49.287323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578cab4-5cb9-4571-b75b-20515f7f113b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:49.287691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578cab4-5cb9-4571-b75b-20515f7f113b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:49.320721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0578cab4-5cb9-4571-b75b-20515f7f113b None None] MFA Rules not processed for user `0c7d0408e5f1413985048424fee0168d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:49.321864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0578cab4-5cb9-4571-b75b-20515f7f113b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:49.389995 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3884/7856] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:49.394687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc4deb-aa33-4d16-a095-811c51f334d6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:49.464057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc4deb-aa33-4d16-a095-811c51f334d6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:49.470260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705e057-c91a-4116-9c40-9def61d2d700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:49.471103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705e057-c91a-4116-9c40-9def61d2d700 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:49.471400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705e057-c91a-4116-9c40-9def61d2d700 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:49.471708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705e057-c91a-4116-9c40-9def61d2d700 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:49.472674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705e057-c91a-4116-9c40-9def61d2d700 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:49.475660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705e057-c91a-4116-9c40-9def61d2d700 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:49.505559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:49.512342 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3973/7857] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:49 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:26:49.516544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578cab4-5cb9-4571-b75b-20515f7f113b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:49.527920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578cab4-5cb9-4571-b75b-20515f7f113b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:49.589549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc498d-d6ff-4b84-9ccf-7664b83e56db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:49.590284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc498d-d6ff-4b84-9ccf-7664b83e56db admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:49.590700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc498d-d6ff-4b84-9ccf-7664b83e56db admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:49.591091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc498d-d6ff-4b84-9ccf-7664b83e56db admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:49.592092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc498d-d6ff-4b84-9ccf-7664b83e56db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:49.594263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc498d-d6ff-4b84-9ccf-7664b83e56db admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:49.625796 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3885/7858] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:49 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:49.627604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705e057-c91a-4116-9c40-9def61d2d700 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:49.642264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705e057-c91a-4116-9c40-9def61d2d700 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:49.652881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8adc30-6e7f-49a5-a12f-d835b4d636cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:49.653484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8adc30-6e7f-49a5-a12f-d835b4d636cd admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:49.653765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8adc30-6e7f-49a5-a12f-d835b4d636cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:49.654030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8adc30-6e7f-49a5-a12f-d835b4d636cd admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:49.654693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8adc30-6e7f-49a5-a12f-d835b4d636cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:49.656304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8adc30-6e7f-49a5-a12f-d835b4d636cd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:49.668066 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3974/7859] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:49.672830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc498d-d6ff-4b84-9ccf-7664b83e56db admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:49.683558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc498d-d6ff-4b84-9ccf-7664b83e56db admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:49.689332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff90399-aaea-4308-ab71-152273d526a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:49.690121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff90399-aaea-4308-ab71-152273d526a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:49.690358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff90399-aaea-4308-ab71-152273d526a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:49.690676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff90399-aaea-4308-ab71-152273d526a7 admin admin] PATH_INFO: `/v3/projects/7e994d7cd4a54da2a656411d67253df4/users/2e4c4ab1f9c34025b66661d1e5dc7160/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:49.707742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff90399-aaea-4308-ab71-152273d526a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e994d7cd4a54da2a656411d67253df4, user_id=2e4c4ab1f9c34025b66661d1e5dc7160, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:49.710435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff90399-aaea-4308-ab71-152273d526a7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:49.724202 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3886/7860] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:49 2022] PUT /identity/v3/projects/7e994d7cd4a54da2a656411d67253df4/users/2e4c4ab1f9c34025b66661d1e5dc7160/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:49.728361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8adc30-6e7f-49a5-a12f-d835b4d636cd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:49.739768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8adc30-6e7f-49a5-a12f-d835b4d636cd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:49.801162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd444e-830d-4042-9ad8-2dd660ef7243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:49.801771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd444e-830d-4042-9ad8-2dd660ef7243 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:49.802039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd444e-830d-4042-9ad8-2dd660ef7243 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:49.802301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd444e-830d-4042-9ad8-2dd660ef7243 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:49.803042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd444e-830d-4042-9ad8-2dd660ef7243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:49.804513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd444e-830d-4042-9ad8-2dd660ef7243 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:49.818495 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3975/7861] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:26:49.819734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff90399-aaea-4308-ab71-152273d526a7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:49.831164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff90399-aaea-4308-ab71-152273d526a7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:49.835892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60785b6-8738-4a4b-b0ad-c861f293f83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:49.836582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60785b6-8738-4a4b-b0ad-c861f293f83b admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:49.836859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60785b6-8738-4a4b-b0ad-c861f293f83b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:49.837125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60785b6-8738-4a4b-b0ad-c861f293f83b admin admin] PATH_INFO: `/v3/projects/7e994d7cd4a54da2a656411d67253df4/users/2e4c4ab1f9c34025b66661d1e5dc7160/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:49.839040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60785b6-8738-4a4b-b0ad-c861f293f83b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e994d7cd4a54da2a656411d67253df4, user_id=2e4c4ab1f9c34025b66661d1e5dc7160, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:49.841719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60785b6-8738-4a4b-b0ad-c861f293f83b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:49.853393 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3887/7862] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:49 2022] PUT /identity/v3/projects/7e994d7cd4a54da2a656411d67253df4/users/2e4c4ab1f9c34025b66661d1e5dc7160/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:49.865134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985158c-d1de-4796-8bf3-4a603b7ae0f6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:49.865428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985158c-d1de-4796-8bf3-4a603b7ae0f6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:49.865691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985158c-d1de-4796-8bf3-4a603b7ae0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:49.895301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9985158c-d1de-4796-8bf3-4a603b7ae0f6 None None] MFA Rules not processed for user `2e4c4ab1f9c34025b66661d1e5dc7160`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:49.896527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9985158c-d1de-4796-8bf3-4a603b7ae0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:49.976201 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3976/7863] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:49.980478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60785b6-8738-4a4b-b0ad-c861f293f83b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:49.994669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60785b6-8738-4a4b-b0ad-c861f293f83b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:50.000690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a8cf7-2323-4b04-90af-3aa165d8893f tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4c4ab1f9c34025b66661d1e5dc7160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e994d7cd4a54da2a656411d67253df4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:50.001453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a8cf7-2323-4b04-90af-3aa165d8893f tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:50.001738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a8cf7-2323-4b04-90af-3aa165d8893f tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:50.002008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a8cf7-2323-4b04-90af-3aa165d8893f tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:50.002650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a8cf7-2323-4b04-90af-3aa165d8893f tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:50.004593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a8cf7-2323-4b04-90af-3aa165d8893f tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:50.033706 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3888/7864] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:49 2022] POST /identity/v3/projects => generated 334 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:26:50.037703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985158c-d1de-4796-8bf3-4a603b7ae0f6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:50.047881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985158c-d1de-4796-8bf3-4a603b7ae0f6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:50.106075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740caff-0822-4ff0-abeb-490c19a054a5 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4c4ab1f9c34025b66661d1e5dc7160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e994d7cd4a54da2a656411d67253df4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:50.106597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740caff-0822-4ff0-abeb-490c19a054a5 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:50.106871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740caff-0822-4ff0-abeb-490c19a054a5 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:50.107140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740caff-0822-4ff0-abeb-490c19a054a5 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:50.107749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740caff-0822-4ff0-abeb-490c19a054a5 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:50.109705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740caff-0822-4ff0-abeb-490c19a054a5 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:50.127312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:50.149961 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3977/7865] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:26:50 2022] POST /identity/v3/projects => generated 333 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:50.154230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a8cf7-2323-4b04-90af-3aa165d8893f tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:50.165959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a8cf7-2323-4b04-90af-3aa165d8893f tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:50.174276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:50.225325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf31eac-5961-411a-8f31-049623c78bc3 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4c4ab1f9c34025b66661d1e5dc7160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e994d7cd4a54da2a656411d67253df4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:50.226030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf31eac-5961-411a-8f31-049623c78bc3 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:50.226365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf31eac-5961-411a-8f31-049623c78bc3 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:50.226709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf31eac-5961-411a-8f31-049623c78bc3 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:50.227384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf31eac-5961-411a-8f31-049623c78bc3 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:50.234516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf31eac-5961-411a-8f31-049623c78bc3 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:50.271063 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3889/7866] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:50 2022] POST /identity/v3/projects => generated 358 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:50.275817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740caff-0822-4ff0-abeb-490c19a054a5 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:50.291687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740caff-0822-4ff0-abeb-490c19a054a5 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:50.358508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8071a-b2c9-4ee2-8eed-c75466e99da4 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4c4ab1f9c34025b66661d1e5dc7160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e994d7cd4a54da2a656411d67253df4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:50.359030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8071a-b2c9-4ee2-8eed-c75466e99da4 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:50.359306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8071a-b2c9-4ee2-8eed-c75466e99da4 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:50.359577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8071a-b2c9-4ee2-8eed-c75466e99da4 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:50.360223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8071a-b2c9-4ee2-8eed-c75466e99da4 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:50.362298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8071a-b2c9-4ee2-8eed-c75466e99da4 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:50.380746 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3978/7867] 10.222.0.23 () {58 vars in 1131 bytes} [Sat Aug 20 21:26:50 2022] GET /identity/v3/projects => generated 6119 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:50.386324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf31eac-5961-411a-8f31-049623c78bc3 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:50.398608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf31eac-5961-411a-8f31-049623c78bc3 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:50.403540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18e12d-fe0d-4ce5-8b62-d5a9d1d74391 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4c4ab1f9c34025b66661d1e5dc7160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e994d7cd4a54da2a656411d67253df4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:50.404170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18e12d-fe0d-4ce5-8b62-d5a9d1d74391 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:50.404484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18e12d-fe0d-4ce5-8b62-d5a9d1d74391 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:50.404778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18e12d-fe0d-4ce5-8b62-d5a9d1d74391 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:50.405446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18e12d-fe0d-4ce5-8b62-d5a9d1d74391 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:50.405792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18e12d-fe0d-4ce5-8b62-d5a9d1d74391 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:50.407880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18e12d-fe0d-4ce5-8b62-d5a9d1d74391 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:50.425316 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3890/7868] 10.222.0.23 () {58 vars in 1158 bytes} [Sat Aug 20 21:26:50 2022] GET /identity/v3/projects?enabled=False => generated 445 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:50.430759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8071a-b2c9-4ee2-8eed-c75466e99da4 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:50.442654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8071a-b2c9-4ee2-8eed-c75466e99da4 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:50.449766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce64888-a987-479b-8354-46ca90618d03 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4c4ab1f9c34025b66661d1e5dc7160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e994d7cd4a54da2a656411d67253df4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:50.450532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce64888-a987-479b-8354-46ca90618d03 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:50.450887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce64888-a987-479b-8354-46ca90618d03 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:50.451312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce64888-a987-479b-8354-46ca90618d03 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:50.452246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce64888-a987-479b-8354-46ca90618d03 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Adding query filter params (parent_id=8e8f05d0e9e847208593f32c0b17465f) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:50.452596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce64888-a987-479b-8354-46ca90618d03 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:50.454474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce64888-a987-479b-8354-46ca90618d03 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:50.473168 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3979/7869] 10.222.0.23 () {58 vars in 1216 bytes} [Sat Aug 20 21:26:50 2022] GET /identity/v3/projects?parent_id=8e8f05d0e9e847208593f32c0b17465f => generated 498 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:50.478747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18e12d-fe0d-4ce5-8b62-d5a9d1d74391 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:50.489515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18e12d-fe0d-4ce5-8b62-d5a9d1d74391 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:50.495115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4c4ab1f9c34025b66661d1e5dc7160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e994d7cd4a54da2a656411d67253df4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:50.495874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:50.496154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:50.496404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] PATH_INFO: `/v3/projects/4758ed9657494b0b994e44becffcf4f5` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:50.497729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4758ed9657494b0b994e44becffcf4f5)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:50.499736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:50.551836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4758ed9657494b0b994e44becffcf4f5', 'request_id': 'req-ba082adc-dc28-43d7-a960-e13f01b1924a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.558100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4758ed9657494b0b994e44becffcf4f5', 'request_id': 'req-ba082adc-dc28-43d7-a960-e13f01b1924a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.567897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4758ed9657494b0b994e44becffcf4f5', 'request_id': 'req-ba082adc-dc28-43d7-a960-e13f01b1924a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.579237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] The token cache is being invalidate because project 4758ed9657494b0b994e44becffcf4f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:50.579520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.580272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.582009 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3891/7870] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:50 2022] DELETE /identity/v3/projects/4758ed9657494b0b994e44becffcf4f5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:50.586293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce64888-a987-479b-8354-46ca90618d03 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:50.588290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bce64888-a987-479b-8354-46ca90618d03 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:50.601736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce64888-a987-479b-8354-46ca90618d03 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:50.668340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4c4ab1f9c34025b66661d1e5dc7160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e994d7cd4a54da2a656411d67253df4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:50.669086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:50.669194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:50.669439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] PATH_INFO: `/v3/projects/8e8f05d0e9e847208593f32c0b17465f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:50.670706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8e8f05d0e9e847208593f32c0b17465f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:50.672690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:50.723278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e8f05d0e9e847208593f32c0b17465f', 'request_id': 'req-ed072b14-7eed-499b-9985-5fc585d1ffa8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.732333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e8f05d0e9e847208593f32c0b17465f', 'request_id': 'req-ed072b14-7eed-499b-9985-5fc585d1ffa8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.738338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e8f05d0e9e847208593f32c0b17465f', 'request_id': 'req-ed072b14-7eed-499b-9985-5fc585d1ffa8'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.749320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] The token cache is being invalidate because project 8e8f05d0e9e847208593f32c0b17465f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:50.749599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.750372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.752008 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3980/7871] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:50 2022] DELETE /identity/v3/projects/8e8f05d0e9e847208593f32c0b17465f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:50.756203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:50.766352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba082adc-dc28-43d7-a960-e13f01b1924a tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:50.830080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452361f-01da-403b-a8d4-c9b7a96d2207 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4c4ab1f9c34025b66661d1e5dc7160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e994d7cd4a54da2a656411d67253df4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:50.830746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452361f-01da-403b-a8d4-c9b7a96d2207 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:50.831017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452361f-01da-403b-a8d4-c9b7a96d2207 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:50.831283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452361f-01da-403b-a8d4-c9b7a96d2207 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] PATH_INFO: `/v3/projects/84348651201c412d98450d6f3e9aed1a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:50.832421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452361f-01da-403b-a8d4-c9b7a96d2207 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorizing `identity:delete_project(project_id=84348651201c412d98450d6f3e9aed1a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:50.834387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452361f-01da-403b-a8d4-c9b7a96d2207 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:50.835929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:50.889109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8452361f-01da-403b-a8d4-c9b7a96d2207 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84348651201c412d98450d6f3e9aed1a', 'request_id': 'req-8452361f-01da-403b-a8d4-c9b7a96d2207'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.895978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8452361f-01da-403b-a8d4-c9b7a96d2207 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84348651201c412d98450d6f3e9aed1a', 'request_id': 'req-8452361f-01da-403b-a8d4-c9b7a96d2207'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.905364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8452361f-01da-403b-a8d4-c9b7a96d2207 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84348651201c412d98450d6f3e9aed1a', 'request_id': 'req-8452361f-01da-403b-a8d4-c9b7a96d2207'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.916006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8452361f-01da-403b-a8d4-c9b7a96d2207 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] The token cache is being invalidate because project 84348651201c412d98450d6f3e9aed1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:50.916287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8452361f-01da-403b-a8d4-c9b7a96d2207 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.916969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8452361f-01da-403b-a8d4-c9b7a96d2207 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:50.918654 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3892/7872] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:50 2022] DELETE /identity/v3/projects/84348651201c412d98450d6f3e9aed1a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:26:50.922706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:50.924193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:50.934922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:50.998135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed072b14-7eed-499b-9985-5fc585d1ffa8 tempest-ListProjectsTestJSON-1876014175 tempest-ListProjectsTestJSON-1876014175-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:51.055828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:51.056520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:51.056791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:51.057054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] PATH_INFO: `/v3/users/0c7d0408e5f1413985048424fee0168d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:51.058197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c7d0408e5f1413985048424fee0168d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:51.060115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:51.113077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c7d0408e5f1413985048424fee0168d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.122080 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c7d0408e5f1413985048424fee0168d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.129875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c7d0408e5f1413985048424fee0168d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.142085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c7d0408e5f1413985048424fee0168d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.142781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c7d0408e5f1413985048424fee0168d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.149007 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3981/7873] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:50 2022] DELETE /identity/v3/users/0c7d0408e5f1413985048424fee0168d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:51.153744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0903f-69eb-4e83-9399-55f2c9f54b7d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:51.154024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0903f-69eb-4e83-9399-55f2c9f54b7d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:51.154296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0903f-69eb-4e83-9399-55f2c9f54b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:51.174486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-57c0903f-69eb-4e83-9399-55f2c9f54b7d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:51.175964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-57c0903f-69eb-4e83-9399-55f2c9f54b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:51.249329 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3893/7874] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:51.259783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:51.261500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:51.298716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:51.329933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d1766-4a32-42d1-865b-25fae057d4f1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:51.334320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cfc2e-923f-4d98-9749-d88da9494c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:51.334988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cfc2e-923f-4d98-9749-d88da9494c4b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:51.335512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cfc2e-923f-4d98-9749-d88da9494c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:51.335786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cfc2e-923f-4d98-9749-d88da9494c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:51.346553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cfc2e-923f-4d98-9749-d88da9494c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:51.348413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cfc2e-923f-4d98-9749-d88da9494c4b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:51.358108 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3982/7875] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:26:51.677192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0903f-69eb-4e83-9399-55f2c9f54b7d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:51.678821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-57c0903f-69eb-4e83-9399-55f2c9f54b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:51.692316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0903f-69eb-4e83-9399-55f2c9f54b7d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:51.697905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:51.698485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:51.698755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:51.699020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] PATH_INFO: `/v3/projects/e8fae6d0141447e4b42684073fb1f4d6` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:51.700181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8fae6d0141447e4b42684073fb1f4d6)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:51.702130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:51.754789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8fae6d0141447e4b42684073fb1f4d6', 'request_id': 'req-7bb34b83-17c1-4680-a1b1-977dc9f156a4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.761894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8fae6d0141447e4b42684073fb1f4d6', 'request_id': 'req-7bb34b83-17c1-4680-a1b1-977dc9f156a4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.771945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8fae6d0141447e4b42684073fb1f4d6', 'request_id': 'req-7bb34b83-17c1-4680-a1b1-977dc9f156a4'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.784572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] The token cache is being invalidate because project e8fae6d0141447e4b42684073fb1f4d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:51.784855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.785836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.787498 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3894/7876] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:51 2022] DELETE /identity/v3/projects/e8fae6d0141447e4b42684073fb1f4d6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:51.791699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cfc2e-923f-4d98-9749-d88da9494c4b service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:51.802815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cfc2e-923f-4d98-9749-d88da9494c4b service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:51.862368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6d270-4b05-4d15-b713-34f3eb2e196b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:51.862998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6d270-4b05-4d15-b713-34f3eb2e196b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:51.863268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6d270-4b05-4d15-b713-34f3eb2e196b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:51.863532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6d270-4b05-4d15-b713-34f3eb2e196b admin admin] PATH_INFO: `/v3/users/2e4c4ab1f9c34025b66661d1e5dc7160` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:51.864703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6d270-4b05-4d15-b713-34f3eb2e196b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e4c4ab1f9c34025b66661d1e5dc7160)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:51.866631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6d270-4b05-4d15-b713-34f3eb2e196b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:51.916306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13e6d270-4b05-4d15-b713-34f3eb2e196b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e4c4ab1f9c34025b66661d1e5dc7160'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.925485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13e6d270-4b05-4d15-b713-34f3eb2e196b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e4c4ab1f9c34025b66661d1e5dc7160'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.935286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13e6d270-4b05-4d15-b713-34f3eb2e196b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e4c4ab1f9c34025b66661d1e5dc7160'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.949321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13e6d270-4b05-4d15-b713-34f3eb2e196b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e4c4ab1f9c34025b66661d1e5dc7160'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.950523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-13e6d270-4b05-4d15-b713-34f3eb2e196b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e4c4ab1f9c34025b66661d1e5dc7160'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:51.962447 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3983/7877] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:51 2022] DELETE /identity/v3/users/2e4c4ab1f9c34025b66661d1e5dc7160 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:52.312269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:52.314372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:52.329498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb34b83-17c1-4680-a1b1-977dc9f156a4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:52.343497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:52.410382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:52.411026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:52.411658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:52.411658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb admin admin] PATH_INFO: `/v3/projects/7e994d7cd4a54da2a656411d67253df4` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:52.412723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e994d7cd4a54da2a656411d67253df4)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:52.414637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:52.462041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e994d7cd4a54da2a656411d67253df4', 'request_id': 'req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:52.470024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e994d7cd4a54da2a656411d67253df4', 'request_id': 'req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:52.478098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e994d7cd4a54da2a656411d67253df4', 'request_id': 'req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:52.489197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb admin admin] The token cache is being invalidate because project 7e994d7cd4a54da2a656411d67253df4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:52.489379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:52.490013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1e9b5d33-be6b-41f5-bc11-6657715d9bbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:52.491516 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3895/7878] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:52 2022] DELETE /identity/v3/projects/7e994d7cd4a54da2a656411d67253df4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:52.496604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14abc86-d460-47d1-bde3-09bc81608133 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:52.497181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14abc86-d460-47d1-bde3-09bc81608133 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:52.497541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14abc86-d460-47d1-bde3-09bc81608133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:52.518350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f14abc86-d460-47d1-bde3-09bc81608133 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:52.519532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f14abc86-d460-47d1-bde3-09bc81608133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:52.571904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:52.587156 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3984/7879] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:52.595513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23eb21-1965-4cf3-9039-46493e8906d8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:52.595794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23eb21-1965-4cf3-9039-46493e8906d8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:52.596064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23eb21-1965-4cf3-9039-46493e8906d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:52.616032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ff23eb21-1965-4cf3-9039-46493e8906d8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:52.617259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ff23eb21-1965-4cf3-9039-46493e8906d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:52.626225 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3896/7880] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:52.630281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14abc86-d460-47d1-bde3-09bc81608133 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:52.640554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14abc86-d460-47d1-bde3-09bc81608133 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:52.645559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb93ce-e4e3-467e-a144-d6484484138e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:52.646210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb93ce-e4e3-467e-a144-d6484484138e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:52.646534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb93ce-e4e3-467e-a144-d6484484138e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:52.646806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb93ce-e4e3-467e-a144-d6484484138e admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:52.647453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb93ce-e4e3-467e-a144-d6484484138e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:52.647782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb93ce-e4e3-467e-a144-d6484484138e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:52.649483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb93ce-e4e3-467e-a144-d6484484138e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:52.666845 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3985/7881] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:26:52.668071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23eb21-1965-4cf3-9039-46493e8906d8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:52.680862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23eb21-1965-4cf3-9039-46493e8906d8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:52.686082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e32f9-1663-44d3-b375-f8d890f158f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:52.686806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e32f9-1663-44d3-b375-f8d890f158f2 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:52.687294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e32f9-1663-44d3-b375-f8d890f158f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:52.687756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e32f9-1663-44d3-b375-f8d890f158f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:52.688881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e32f9-1663-44d3-b375-f8d890f158f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:52.692739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e32f9-1663-44d3-b375-f8d890f158f2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:52.727798 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3897/7882] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:52 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:52.729122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb93ce-e4e3-467e-a144-d6484484138e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:52.739281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb93ce-e4e3-467e-a144-d6484484138e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:52.808728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61819f60-da5a-4fd8-a960-d16f599607cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:52.809363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61819f60-da5a-4fd8-a960-d16f599607cf admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:52.809626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61819f60-da5a-4fd8-a960-d16f599607cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:52.809897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61819f60-da5a-4fd8-a960-d16f599607cf admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:52.810521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61819f60-da5a-4fd8-a960-d16f599607cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:52.812707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61819f60-da5a-4fd8-a960-d16f599607cf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:52.835465 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3986/7883] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:52 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:52.839578 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e32f9-1663-44d3-b375-f8d890f158f2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:52.853592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e32f9-1663-44d3-b375-f8d890f158f2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:52.858672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6b428-d99f-47f5-b803-75a2e9c27835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:52.859297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6b428-d99f-47f5-b803-75a2e9c27835 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:52.859556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6b428-d99f-47f5-b803-75a2e9c27835 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:52.859819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6b428-d99f-47f5-b803-75a2e9c27835 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:52.860518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6b428-d99f-47f5-b803-75a2e9c27835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:52.862187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6b428-d99f-47f5-b803-75a2e9c27835 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:52.873330 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3898/7884] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:26:52.877558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61819f60-da5a-4fd8-a960-d16f599607cf admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:52.888247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61819f60-da5a-4fd8-a960-d16f599607cf admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:52.893259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d95eed-abe8-4d0d-8c6c-686e1762c6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:52.893859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d95eed-abe8-4d0d-8c6c-686e1762c6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:52.894133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d95eed-abe8-4d0d-8c6c-686e1762c6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:52.894397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d95eed-abe8-4d0d-8c6c-686e1762c6f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:52.895055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d95eed-abe8-4d0d-8c6c-686e1762c6f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:52.896691 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d95eed-abe8-4d0d-8c6c-686e1762c6f2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:52.907943 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3987/7885] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:26:52.912041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6b428-d99f-47f5-b803-75a2e9c27835 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:52.923165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6b428-d99f-47f5-b803-75a2e9c27835 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:52.928845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a53d5-38e3-4fed-8342-65142d41812a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:52.929501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a53d5-38e3-4fed-8342-65142d41812a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:52.929772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a53d5-38e3-4fed-8342-65142d41812a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:52.930037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a53d5-38e3-4fed-8342-65142d41812a admin admin] PATH_INFO: `/v3/projects/211a6c772cca494e95b8b693ecb0a324/users/fb34d4308e094abc96b911059f3c166b/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:52.955514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a53d5-38e3-4fed-8342-65142d41812a admin admin] RBAC: Authorizing `identity:create_grant(project_id=211a6c772cca494e95b8b693ecb0a324, user_id=fb34d4308e094abc96b911059f3c166b, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:52.958242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a53d5-38e3-4fed-8342-65142d41812a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:52.972553 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3899/7886] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:52 2022] PUT /identity/v3/projects/211a6c772cca494e95b8b693ecb0a324/users/fb34d4308e094abc96b911059f3c166b/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:52.983162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5589c62-c345-40f1-a350-2668637e341c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:52.983460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5589c62-c345-40f1-a350-2668637e341c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:52.983835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5589c62-c345-40f1-a350-2668637e341c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.011911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f5589c62-c345-40f1-a350-2668637e341c None None] MFA Rules not processed for user `fb34d4308e094abc96b911059f3c166b`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:53.013196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f5589c62-c345-40f1-a350-2668637e341c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:53.085794 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3988/7887] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:53.090196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a53d5-38e3-4fed-8342-65142d41812a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:53.136172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:53.162942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a53d5-38e3-4fed-8342-65142d41812a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:53.163516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b5fe4-140a-4cf2-ac42-ba4f13b0c681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:53.164115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b5fe4-140a-4cf2-ac42-ba4f13b0c681 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:53.164422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b5fe4-140a-4cf2-ac42-ba4f13b0c681 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:53.164694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b5fe4-140a-4cf2-ac42-ba4f13b0c681 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.165308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b5fe4-140a-4cf2-ac42-ba4f13b0c681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:53.167194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b5fe4-140a-4cf2-ac42-ba4f13b0c681 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:53.199666 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3900/7888] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:53 2022] POST /identity/v3/projects => generated 363 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:53.200642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5589c62-c345-40f1-a350-2668637e341c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:53.211785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5589c62-c345-40f1-a350-2668637e341c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:53.259883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:53.270782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d40fdc-683e-41ec-a359-2e6e696dc916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:53.271506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d40fdc-683e-41ec-a359-2e6e696dc916 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:53.271836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d40fdc-683e-41ec-a359-2e6e696dc916 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:53.272162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d40fdc-683e-41ec-a359-2e6e696dc916 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.272889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d40fdc-683e-41ec-a359-2e6e696dc916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:53.274862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d40fdc-683e-41ec-a359-2e6e696dc916 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:53.301926 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3989/7889] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:53 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:53.303157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b5fe4-140a-4cf2-ac42-ba4f13b0c681 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:53.313814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b5fe4-140a-4cf2-ac42-ba4f13b0c681 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:53.319034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3c926-d9b1-4bba-9a69-8429c5826ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:53.319624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3c926-d9b1-4bba-9a69-8429c5826ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:53.319896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3c926-d9b1-4bba-9a69-8429c5826ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:53.320163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3c926-d9b1-4bba-9a69-8429c5826ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.320844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3c926-d9b1-4bba-9a69-8429c5826ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:53.322480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3c926-d9b1-4bba-9a69-8429c5826ce5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:53.336862 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3901/7890] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:53.339553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d40fdc-683e-41ec-a359-2e6e696dc916 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:53.351792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d40fdc-683e-41ec-a359-2e6e696dc916 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:53.356976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b197f10-0371-4395-9b7d-690ff57c1a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:53.357822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b197f10-0371-4395-9b7d-690ff57c1a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:53.358096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b197f10-0371-4395-9b7d-690ff57c1a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:53.359164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b197f10-0371-4395-9b7d-690ff57c1a79 admin admin] PATH_INFO: `/v3/projects/ddbd76c1a5754ca599e32c4b8a0a566a/users/21b571064365461b91b8b4cb69d9166e/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.378129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b197f10-0371-4395-9b7d-690ff57c1a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbd76c1a5754ca599e32c4b8a0a566a, user_id=21b571064365461b91b8b4cb69d9166e, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:53.380888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b197f10-0371-4395-9b7d-690ff57c1a79 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:53.395863 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3990/7891] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:53 2022] PUT /identity/v3/projects/ddbd76c1a5754ca599e32c4b8a0a566a/users/21b571064365461b91b8b4cb69d9166e/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:53.399919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3c926-d9b1-4bba-9a69-8429c5826ce5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:53.410828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3c926-d9b1-4bba-9a69-8429c5826ce5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:53.470670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ddcd4-5a4d-483a-b1b8-8b652298bfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:53.471280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ddcd4-5a4d-483a-b1b8-8b652298bfe0 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:53.471555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ddcd4-5a4d-483a-b1b8-8b652298bfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:53.471824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ddcd4-5a4d-483a-b1b8-8b652298bfe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.472634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ddcd4-5a4d-483a-b1b8-8b652298bfe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:53.474253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ddcd4-5a4d-483a-b1b8-8b652298bfe0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:53.485492 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3902/7892] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:53.489531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b197f10-0371-4395-9b7d-690ff57c1a79 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:53.500091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b197f10-0371-4395-9b7d-690ff57c1a79 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:53.504854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0158a6-bf10-4b51-a235-24e317916ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:53.505575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0158a6-bf10-4b51-a235-24e317916ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:53.505906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0158a6-bf10-4b51-a235-24e317916ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:53.506234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0158a6-bf10-4b51-a235-24e317916ba9 admin admin] PATH_INFO: `/v3/projects/ddbd76c1a5754ca599e32c4b8a0a566a/users/21b571064365461b91b8b4cb69d9166e/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.508308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0158a6-bf10-4b51-a235-24e317916ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbd76c1a5754ca599e32c4b8a0a566a, user_id=21b571064365461b91b8b4cb69d9166e, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:53.511177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0158a6-bf10-4b51-a235-24e317916ba9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:53.522699 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3991/7893] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:53 2022] PUT /identity/v3/projects/ddbd76c1a5754ca599e32c4b8a0a566a/users/21b571064365461b91b8b4cb69d9166e/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:53.534830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad37dd-422a-4eac-8401-29a594ed5d85 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:53.535612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad37dd-422a-4eac-8401-29a594ed5d85 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:53.535612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad37dd-422a-4eac-8401-29a594ed5d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.565255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-52ad37dd-422a-4eac-8401-29a594ed5d85 None None] MFA Rules not processed for user `21b571064365461b91b8b4cb69d9166e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:53.566521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-52ad37dd-422a-4eac-8401-29a594ed5d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:53.645423 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3903/7894] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:53.650811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0158a6-bf10-4b51-a235-24e317916ba9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:53.663321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0158a6-bf10-4b51-a235-24e317916ba9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:53.668770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5454c5-3a4f-4e1d-b697-cc2f6caf4a8a tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b571064365461b91b8b4cb69d9166e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:53.669460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5454c5-3a4f-4e1d-b697-cc2f6caf4a8a tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:53.669809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5454c5-3a4f-4e1d-b697-cc2f6caf4a8a tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:53.670140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5454c5-3a4f-4e1d-b697-cc2f6caf4a8a tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.671031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5454c5-3a4f-4e1d-b697-cc2f6caf4a8a tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:53.672688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5454c5-3a4f-4e1d-b697-cc2f6caf4a8a tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:53.695722 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3992/7895] 10.222.0.23 () {60 vars in 1149 bytes} [Sat Aug 20 21:26:53 2022] POST /identity/v3/domains => generated 277 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:53.697056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad37dd-422a-4eac-8401-29a594ed5d85 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:53.708024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad37dd-422a-4eac-8401-29a594ed5d85 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:53.745713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:53.779802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f0689-96fc-4ee4-bede-e781e1e41ced tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b571064365461b91b8b4cb69d9166e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:53.780526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f0689-96fc-4ee4-bede-e781e1e41ced tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:53.780907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f0689-96fc-4ee4-bede-e781e1e41ced tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:53.781279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f0689-96fc-4ee4-bede-e781e1e41ced tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.782009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f0689-96fc-4ee4-bede-e781e1e41ced tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:53.784079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f0689-96fc-4ee4-bede-e781e1e41ced tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:53.820338 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3904/7896] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:53 2022] POST /identity/v3/users => generated 348 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:53.824082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5454c5-3a4f-4e1d-b697-cc2f6caf4a8a tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:53.834726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5454c5-3a4f-4e1d-b697-cc2f6caf4a8a tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:53.839468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cba48-a1bf-40ff-ad4a-cfd071ed9f60 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b571064365461b91b8b4cb69d9166e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:53.840142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cba48-a1bf-40ff-ad4a-cfd071ed9f60 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:53.840526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cba48-a1bf-40ff-ad4a-cfd071ed9f60 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:53.840866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cba48-a1bf-40ff-ad4a-cfd071ed9f60 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.841672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cba48-a1bf-40ff-ad4a-cfd071ed9f60 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:53.844000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cba48-a1bf-40ff-ad4a-cfd071ed9f60 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:53.876428 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3993/7897] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:53 2022] POST /identity/v3/users => generated 324 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:53.878977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f0689-96fc-4ee4-bede-e781e1e41ced tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:53.889489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f0689-96fc-4ee4-bede-e781e1e41ced tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:53.894797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239e785-602a-4973-b31a-ea95a205a545 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b571064365461b91b8b4cb69d9166e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:53.895429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239e785-602a-4973-b31a-ea95a205a545 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:53.895704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239e785-602a-4973-b31a-ea95a205a545 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:53.896006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239e785-602a-4973-b31a-ea95a205a545 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] PATH_INFO: `/v3/users/6e4200a8dfbb43dc9212f43e20864e73` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.911432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239e785-602a-4973-b31a-ea95a205a545 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorizing `identity:get_user(user_id=6e4200a8dfbb43dc9212f43e20864e73)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:53.913616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239e785-602a-4973-b31a-ea95a205a545 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:53.915518 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3905/7898] 10.222.0.23 () {58 vars in 1221 bytes} [Sat Aug 20 21:26:53 2022] GET /identity/v3/users/6e4200a8dfbb43dc9212f43e20864e73 => generated 348 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:53.920634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cba48-a1bf-40ff-ad4a-cfd071ed9f60 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:53.931655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cba48-a1bf-40ff-ad4a-cfd071ed9f60 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:53.936747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddae43-d453-4149-bad0-6f13eddf1e92 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b571064365461b91b8b4cb69d9166e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:53.937410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddae43-d453-4149-bad0-6f13eddf1e92 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:53.937823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddae43-d453-4149-bad0-6f13eddf1e92 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:53.938164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddae43-d453-4149-bad0-6f13eddf1e92 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.938864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ddae43-d453-4149-bad0-6f13eddf1e92 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Adding query filter params (domain_id=230cfbf1e8d948c48894bd11007903cd) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:53.939146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ddae43-d453-4149-bad0-6f13eddf1e92 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:53.941202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ddae43-d453-4149-bad0-6f13eddf1e92 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:53.956171 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3994/7899] 10.222.0.23 () {58 vars in 1207 bytes} [Sat Aug 20 21:26:53 2022] GET /identity/v3/users?domain_id=230cfbf1e8d948c48894bd11007903cd => generated 485 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:53.965303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239e785-602a-4973-b31a-ea95a205a545 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:53.971859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239e785-602a-4973-b31a-ea95a205a545 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:53.976744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486daa6-c054-4a63-83a5-8b9cfb5294b3 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b571064365461b91b8b4cb69d9166e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:53.977363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486daa6-c054-4a63-83a5-8b9cfb5294b3 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:53.977658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486daa6-c054-4a63-83a5-8b9cfb5294b3 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:53.977929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486daa6-c054-4a63-83a5-8b9cfb5294b3 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:53.978794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486daa6-c054-4a63-83a5-8b9cfb5294b3 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:53.980567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486daa6-c054-4a63-83a5-8b9cfb5294b3 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:54.008976 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3906/7900] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:53 2022] GET /identity/v3/users => generated 7749 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:54.014528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddae43-d453-4149-bad0-6f13eddf1e92 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:54.025318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddae43-d453-4149-bad0-6f13eddf1e92 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:54.030107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348528a-ce22-4d0f-b235-a054f3806421 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b571064365461b91b8b4cb69d9166e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:54.030629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348528a-ce22-4d0f-b235-a054f3806421 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:54.030950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348528a-ce22-4d0f-b235-a054f3806421 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:54.031225 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348528a-ce22-4d0f-b235-a054f3806421 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:54.031963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348528a-ce22-4d0f-b235-a054f3806421 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Adding query filter params (name=tempest-test_user-1129038758) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:54.032256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348528a-ce22-4d0f-b235-a054f3806421 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:54.034122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348528a-ce22-4d0f-b235-a054f3806421 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:54.072254 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3995/7901] 10.222.0.23 () {58 vars in 1189 bytes} [Sat Aug 20 21:26:54 2022] GET /identity/v3/users?name=tempest-test_user-1129038758 => generated 476 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:54.077442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486daa6-c054-4a63-83a5-8b9cfb5294b3 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:54.088058 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486daa6-c054-4a63-83a5-8b9cfb5294b3 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:54.093364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556374f5-0430-4b5b-8d74-2a548032777d tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b571064365461b91b8b4cb69d9166e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:54.093957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556374f5-0430-4b5b-8d74-2a548032777d tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:54.094233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556374f5-0430-4b5b-8d74-2a548032777d tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:54.094508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556374f5-0430-4b5b-8d74-2a548032777d tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:54.095206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556374f5-0430-4b5b-8d74-2a548032777d tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:54.095505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556374f5-0430-4b5b-8d74-2a548032777d tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:54.097398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556374f5-0430-4b5b-8d74-2a548032777d tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:54.118197 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3907/7902] 10.222.0.23 () {58 vars in 1149 bytes} [Sat Aug 20 21:26:54 2022] GET /identity/v3/users?enabled=False => generated 432 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:26:54.125068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348528a-ce22-4d0f-b235-a054f3806421 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:54.137243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348528a-ce22-4d0f-b235-a054f3806421 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:54.142181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feac8b-5ecd-4fb0-a44c-064ae052ab08 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b571064365461b91b8b4cb69d9166e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:54.142815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feac8b-5ecd-4fb0-a44c-064ae052ab08 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:54.143091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feac8b-5ecd-4fb0-a44c-064ae052ab08 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:54.143361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feac8b-5ecd-4fb0-a44c-064ae052ab08 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] PATH_INFO: `/v3/users/154c6446df3e4aca8523db8dc8e95162` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:54.158372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feac8b-5ecd-4fb0-a44c-064ae052ab08 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorizing `identity:delete_user(user_id=154c6446df3e4aca8523db8dc8e95162)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:54.160429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feac8b-5ecd-4fb0-a44c-064ae052ab08 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:54.208524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f9feac8b-5ecd-4fb0-a44c-064ae052ab08 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154c6446df3e4aca8523db8dc8e95162'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.216742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f9feac8b-5ecd-4fb0-a44c-064ae052ab08 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154c6446df3e4aca8523db8dc8e95162'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.231461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f9feac8b-5ecd-4fb0-a44c-064ae052ab08 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154c6446df3e4aca8523db8dc8e95162'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.246671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f9feac8b-5ecd-4fb0-a44c-064ae052ab08 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154c6446df3e4aca8523db8dc8e95162'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.247599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f9feac8b-5ecd-4fb0-a44c-064ae052ab08 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154c6446df3e4aca8523db8dc8e95162'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.254721 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3996/7903] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:54 2022] DELETE /identity/v3/users/154c6446df3e4aca8523db8dc8e95162 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:54.258948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556374f5-0430-4b5b-8d74-2a548032777d tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:54.260993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-556374f5-0430-4b5b-8d74-2a548032777d tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:54.274424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556374f5-0430-4b5b-8d74-2a548032777d tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:54.340610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b571064365461b91b8b4cb69d9166e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:54.341333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:54.341672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:54.342009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] PATH_INFO: `/v3/users/6e4200a8dfbb43dc9212f43e20864e73` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:54.343392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorizing `identity:delete_user(user_id=6e4200a8dfbb43dc9212f43e20864e73)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:54.345311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:54.398043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4200a8dfbb43dc9212f43e20864e73'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.406089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4200a8dfbb43dc9212f43e20864e73'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.419604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4200a8dfbb43dc9212f43e20864e73'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.420354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4200a8dfbb43dc9212f43e20864e73'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.421225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4200a8dfbb43dc9212f43e20864e73'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.428813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:54.434904 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3908/7904] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:54 2022] DELETE /identity/v3/users/6e4200a8dfbb43dc9212f43e20864e73 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:54.439136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feac8b-5ecd-4fb0-a44c-064ae052ab08 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:54.450796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feac8b-5ecd-4fb0-a44c-064ae052ab08 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:54.455472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=80,keystone:INSERT=10,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:54.512457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f5bdb-20dd-4eef-8205-7ee6700cf1f5 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b571064365461b91b8b4cb69d9166e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:54.513089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f5bdb-20dd-4eef-8205-7ee6700cf1f5 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:54.513362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f5bdb-20dd-4eef-8205-7ee6700cf1f5 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:54.513628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f5bdb-20dd-4eef-8205-7ee6700cf1f5 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] PATH_INFO: `/v3/domains/230cfbf1e8d948c48894bd11007903cd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:54.514890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f5bdb-20dd-4eef-8205-7ee6700cf1f5 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=230cfbf1e8d948c48894bd11007903cd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:54.516506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f5bdb-20dd-4eef-8205-7ee6700cf1f5 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:54.537208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-746f5bdb-20dd-4eef-8205-7ee6700cf1f5 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '230cfbf1e8d948c48894bd11007903cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.537863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-746f5bdb-20dd-4eef-8205-7ee6700cf1f5 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '230cfbf1e8d948c48894bd11007903cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.562867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-746f5bdb-20dd-4eef-8205-7ee6700cf1f5 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '230cfbf1e8d948c48894bd11007903cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.563484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-746f5bdb-20dd-4eef-8205-7ee6700cf1f5 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '230cfbf1e8d948c48894bd11007903cd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.565958 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3997/7905] 10.222.0.23 () {60 vars in 1249 bytes} [Sat Aug 20 21:26:54 2022] PATCH /identity/v3/domains/230cfbf1e8d948c48894bd11007903cd => generated 278 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:54.573022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:54.574412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:54.587250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965b150-3ed1-46bc-ac44-9bd3bc5f2ef8 tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:54.647177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b571064365461b91b8b4cb69d9166e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:54.647820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:54.648099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:54.648394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] PATH_INFO: `/v3/domains/230cfbf1e8d948c48894bd11007903cd` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:54.659907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=230cfbf1e8d948c48894bd11007903cd)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:54.661621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:54.693131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '230cfbf1e8d948c48894bd11007903cd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.696867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '230cfbf1e8d948c48894bd11007903cd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.760746 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '230cfbf1e8d948c48894bd11007903cd', 'request_id': 'req-791c88e2-5ee0-4164-82ce-caec74e202ce'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.768711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '230cfbf1e8d948c48894bd11007903cd', 'request_id': 'req-791c88e2-5ee0-4164-82ce-caec74e202ce'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.771117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa232c2-f2be-4746-81b7-2b3303c62728 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:54.771404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa232c2-f2be-4746-81b7-2b3303c62728 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:54.771682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa232c2-f2be-4746-81b7-2b3303c62728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:54.780262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '230cfbf1e8d948c48894bd11007903cd', 'request_id': 'req-791c88e2-5ee0-4164-82ce-caec74e202ce'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.791092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] The token cache is being invalidate because project 230cfbf1e8d948c48894bd11007903cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:54.791288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.791923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.794962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ffa232c2-f2be-4746-81b7-2b3303c62728 None None] MFA Rules not processed for user `2bd3b3a2aad74b62aa8af337703f4527`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:54.796524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ffa232c2-f2be-4746-81b7-2b3303c62728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:54.799732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '230cfbf1e8d948c48894bd11007903cd', 'request_id': 'req-791c88e2-5ee0-4164-82ce-caec74e202ce'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.800391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '230cfbf1e8d948c48894bd11007903cd', 'request_id': 'req-791c88e2-5ee0-4164-82ce-caec74e202ce'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.803618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '230cfbf1e8d948c48894bd11007903cd', 'request_id': 'req-791c88e2-5ee0-4164-82ce-caec74e202ce'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.809209 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3909/7906] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:26:54 2022] DELETE /identity/v3/domains/230cfbf1e8d948c48894bd11007903cd => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:54.813355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:54.814672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:54.853730 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3998/7907] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:54.856885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:54.890419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c88e2-5ee0-4164-82ce-caec74e202ce tempest-UsersV3TestJSON-51256595 tempest-UsersV3TestJSON-51256595-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:54.895534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:54.896167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:54.896466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:54.896764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] PATH_INFO: `/v3/users/fb34d4308e094abc96b911059f3c166b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:54.897893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb34d4308e094abc96b911059f3c166b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:54.899800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:54.953443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb34d4308e094abc96b911059f3c166b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.962158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb34d4308e094abc96b911059f3c166b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.975304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb34d4308e094abc96b911059f3c166b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.976004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb34d4308e094abc96b911059f3c166b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.976638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb34d4308e094abc96b911059f3c166b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:54.989565 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3910/7908] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:54 2022] DELETE /identity/v3/users/fb34d4308e094abc96b911059f3c166b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:26:54.994003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae43ca-484d-4742-9625-287359645147 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:54.994290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae43ca-484d-4742-9625-287359645147 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:54.994558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae43ca-484d-4742-9625-287359645147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:55.014089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-faae43ca-484d-4742-9625-287359645147 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:55.015914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-faae43ca-484d-4742-9625-287359645147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:55.081682 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 3999/7909] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:55.091587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:55.093296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:55.155682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0523e7-bb5d-42bc-bdec-4d318a64af4a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:55.159671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114778f-7037-4f54-afd2-f5d37dd0c406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:55.160272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114778f-7037-4f54-afd2-f5d37dd0c406 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:55.160579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114778f-7037-4f54-afd2-f5d37dd0c406 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:55.160815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114778f-7037-4f54-afd2-f5d37dd0c406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:55.170509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114778f-7037-4f54-afd2-f5d37dd0c406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:55.171933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114778f-7037-4f54-afd2-f5d37dd0c406 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:55.181555 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3911/7910] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:26:55.514537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae43ca-484d-4742-9625-287359645147 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:55.516069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-faae43ca-484d-4742-9625-287359645147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:55.529011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae43ca-484d-4742-9625-287359645147 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:55.535324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:55.536083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:55.536485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:55.536853 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] PATH_INFO: `/v3/projects/211a6c772cca494e95b8b693ecb0a324` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:55.538184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] RBAC: Authorizing `identity:delete_project(project_id=211a6c772cca494e95b8b693ecb0a324)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:55.541023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:55.554609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:55.594297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '211a6c772cca494e95b8b693ecb0a324', 'request_id': 'req-b8c46759-8309-427a-b599-930d8ff3e68f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:55.602740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '211a6c772cca494e95b8b693ecb0a324', 'request_id': 'req-b8c46759-8309-427a-b599-930d8ff3e68f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:55.609202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '211a6c772cca494e95b8b693ecb0a324', 'request_id': 'req-b8c46759-8309-427a-b599-930d8ff3e68f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:55.620671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] The token cache is being invalidate because project 211a6c772cca494e95b8b693ecb0a324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:55.620954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:55.621660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:55.623105 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4000/7911] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:55 2022] DELETE /identity/v3/projects/211a6c772cca494e95b8b693ecb0a324 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:55.627342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114778f-7037-4f54-afd2-f5d37dd0c406 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:55.637466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114778f-7037-4f54-afd2-f5d37dd0c406 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:55.696818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:55.704220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebfaae-d547-4af7-988c-47b8f748b048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:55.705060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebfaae-d547-4af7-988c-47b8f748b048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:55.705342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebfaae-d547-4af7-988c-47b8f748b048 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:55.705849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebfaae-d547-4af7-988c-47b8f748b048 admin admin] PATH_INFO: `/v3/users/21b571064365461b91b8b4cb69d9166e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:55.707615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebfaae-d547-4af7-988c-47b8f748b048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21b571064365461b91b8b4cb69d9166e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:55.709856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebfaae-d547-4af7-988c-47b8f748b048 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:55.760993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-47ebfaae-d547-4af7-988c-47b8f748b048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21b571064365461b91b8b4cb69d9166e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:55.771366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-47ebfaae-d547-4af7-988c-47b8f748b048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21b571064365461b91b8b4cb69d9166e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:55.784610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-47ebfaae-d547-4af7-988c-47b8f748b048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21b571064365461b91b8b4cb69d9166e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:55.785285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-47ebfaae-d547-4af7-988c-47b8f748b048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21b571064365461b91b8b4cb69d9166e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:55.785881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-47ebfaae-d547-4af7-988c-47b8f748b048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21b571064365461b91b8b4cb69d9166e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:55.798386 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3912/7912] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:55 2022] DELETE /identity/v3/users/21b571064365461b91b8b4cb69d9166e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:26:56.095968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:56.097867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:56.111732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c46759-8309-427a-b599-930d8ff3e68f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:56.170205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:56.170895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:56.171167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:56.171435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1 admin admin] PATH_INFO: `/v3/projects/ddbd76c1a5754ca599e32c4b8a0a566a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:56.172748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddbd76c1a5754ca599e32c4b8a0a566a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:56.174755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:56.223933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'request_id': 'req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:56.233270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'request_id': 'req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:56.240557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddbd76c1a5754ca599e32c4b8a0a566a', 'request_id': 'req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:56.253248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1 admin admin] The token cache is being invalidate because project ddbd76c1a5754ca599e32c4b8a0a566a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:56.253528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:56.254230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-36b97080-b996-4ecc-9d4f-0ff8d049ebd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:56.255850 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4001/7913] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:56 2022] DELETE /identity/v3/projects/ddbd76c1a5754ca599e32c4b8a0a566a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:26:56.261074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc600226-dbdf-4eb0-b984-4fc6f289b99c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:56.261359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc600226-dbdf-4eb0-b984-4fc6f289b99c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:56.261629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc600226-dbdf-4eb0-b984-4fc6f289b99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:56.281797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fc600226-dbdf-4eb0-b984-4fc6f289b99c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:56.283077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fc600226-dbdf-4eb0-b984-4fc6f289b99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:56.346883 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3913/7914] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:26:56.354902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cdf82-4005-4545-befe-77b04d07a8b2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:56.355172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cdf82-4005-4545-befe-77b04d07a8b2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:56.355435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cdf82-4005-4545-befe-77b04d07a8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:56.376290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1c8cdf82-4005-4545-befe-77b04d07a8b2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:56.377453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1c8cdf82-4005-4545-befe-77b04d07a8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:56.386549 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4002/7915] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:56.391060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc600226-dbdf-4eb0-b984-4fc6f289b99c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:56.401567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc600226-dbdf-4eb0-b984-4fc6f289b99c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:56.407372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a5a34-6b64-47e4-a9a9-90b41cb097cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:56.407966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a5a34-6b64-47e4-a9a9-90b41cb097cd admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:56.408235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a5a34-6b64-47e4-a9a9-90b41cb097cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:56.408532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a5a34-6b64-47e4-a9a9-90b41cb097cd admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:56.409167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a5a34-6b64-47e4-a9a9-90b41cb097cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:56.409464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a5a34-6b64-47e4-a9a9-90b41cb097cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:56.411059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a5a34-6b64-47e4-a9a9-90b41cb097cd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:56.423365 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3914/7916] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:56.427609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cdf82-4005-4545-befe-77b04d07a8b2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:56.437122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cdf82-4005-4545-befe-77b04d07a8b2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:56.442453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3915840-c809-47fc-bb69-56de5c52f990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:56.443055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3915840-c809-47fc-bb69-56de5c52f990 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:56.443330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3915840-c809-47fc-bb69-56de5c52f990 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:56.443631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3915840-c809-47fc-bb69-56de5c52f990 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:56.444258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3915840-c809-47fc-bb69-56de5c52f990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:56.446261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3915840-c809-47fc-bb69-56de5c52f990 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:56.477515 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4003/7917] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:56 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:26:56.482111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a5a34-6b64-47e4-a9a9-90b41cb097cd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:56.493078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a5a34-6b64-47e4-a9a9-90b41cb097cd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:56.554409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f792f9-d5fc-48a2-a89f-6fc9c23a7530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:56.555015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f792f9-d5fc-48a2-a89f-6fc9c23a7530 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:56.555293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f792f9-d5fc-48a2-a89f-6fc9c23a7530 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:56.555567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f792f9-d5fc-48a2-a89f-6fc9c23a7530 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:56.556177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f792f9-d5fc-48a2-a89f-6fc9c23a7530 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:56.558211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f792f9-d5fc-48a2-a89f-6fc9c23a7530 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:56.568524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:56.582889 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3915/7918] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:56 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:56.588032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3915840-c809-47fc-bb69-56de5c52f990 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:56.597686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:56.606738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3915840-c809-47fc-bb69-56de5c52f990 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:56.613807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88f2de-8768-45fc-ad0d-299fa9a97cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:56.614467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88f2de-8768-45fc-ad0d-299fa9a97cac admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:56.614737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88f2de-8768-45fc-ad0d-299fa9a97cac admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:56.615025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88f2de-8768-45fc-ad0d-299fa9a97cac admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:56.615655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88f2de-8768-45fc-ad0d-299fa9a97cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:56.617337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88f2de-8768-45fc-ad0d-299fa9a97cac admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:56.627771 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4004/7919] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:56.631837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f792f9-d5fc-48a2-a89f-6fc9c23a7530 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:56.642836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f792f9-d5fc-48a2-a89f-6fc9c23a7530 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:56.648662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313dc21-7c85-4263-a001-7dab10b9698a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:56.649264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313dc21-7c85-4263-a001-7dab10b9698a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:56.649543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313dc21-7c85-4263-a001-7dab10b9698a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:56.649823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313dc21-7c85-4263-a001-7dab10b9698a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:56.650511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313dc21-7c85-4263-a001-7dab10b9698a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:56.652139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313dc21-7c85-4263-a001-7dab10b9698a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:56.663299 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3916/7920] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:56.667392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88f2de-8768-45fc-ad0d-299fa9a97cac admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:56.678427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88f2de-8768-45fc-ad0d-299fa9a97cac admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:56.683990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9494c-046a-4685-ab67-4d6e8c7b5b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:56.684743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9494c-046a-4685-ab67-4d6e8c7b5b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:56.685059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9494c-046a-4685-ab67-4d6e8c7b5b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:56.685411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9494c-046a-4685-ab67-4d6e8c7b5b11 admin admin] PATH_INFO: `/v3/projects/1c55b70e150f4b3599a24629d70bb7cd/users/9d865b08512b4eed97ed2cd98c7bef01/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:56.701295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9494c-046a-4685-ab67-4d6e8c7b5b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c55b70e150f4b3599a24629d70bb7cd, user_id=9d865b08512b4eed97ed2cd98c7bef01, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:56.704277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9494c-046a-4685-ab67-4d6e8c7b5b11 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:56.721181 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4005/7921] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:56 2022] PUT /identity/v3/projects/1c55b70e150f4b3599a24629d70bb7cd/users/9d865b08512b4eed97ed2cd98c7bef01/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:56.732756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afb326-5c2c-4d05-9862-8bf6d890b571 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:56.733020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afb326-5c2c-4d05-9862-8bf6d890b571 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:56.733285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afb326-5c2c-4d05-9862-8bf6d890b571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:56.765635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-29afb326-5c2c-4d05-9862-8bf6d890b571 None None] MFA Rules not processed for user `9d865b08512b4eed97ed2cd98c7bef01`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:56.766942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-29afb326-5c2c-4d05-9862-8bf6d890b571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:56.855257 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3917/7922] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:56.860416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9494c-046a-4685-ab67-4d6e8c7b5b11 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:56.925160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9494c-046a-4685-ab67-4d6e8c7b5b11 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:56.930312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffabfb8-1fb8-4f17-9b6a-e11e1c0ed56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:56.930915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffabfb8-1fb8-4f17-9b6a-e11e1c0ed56d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:56.931182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffabfb8-1fb8-4f17-9b6a-e11e1c0ed56d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:56.931445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffabfb8-1fb8-4f17-9b6a-e11e1c0ed56d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:56.932052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffabfb8-1fb8-4f17-9b6a-e11e1c0ed56d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:56.934106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffabfb8-1fb8-4f17-9b6a-e11e1c0ed56d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:56.964148 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4006/7923] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:56 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:56.968743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afb326-5c2c-4d05-9862-8bf6d890b571 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:56.979920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afb326-5c2c-4d05-9862-8bf6d890b571 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.039024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae415e-1ef3-4102-ac8d-6838a0942721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.039630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae415e-1ef3-4102-ac8d-6838a0942721 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.039905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae415e-1ef3-4102-ac8d-6838a0942721 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.040171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae415e-1ef3-4102-ac8d-6838a0942721 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.040828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae415e-1ef3-4102-ac8d-6838a0942721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.042761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae415e-1ef3-4102-ac8d-6838a0942721 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.064493 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3918/7924] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:26:56 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:57.068851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffabfb8-1fb8-4f17-9b6a-e11e1c0ed56d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.081211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffabfb8-1fb8-4f17-9b6a-e11e1c0ed56d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.086007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a06b1-54ff-4036-89a6-076d517fd028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.086609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a06b1-54ff-4036-89a6-076d517fd028 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.086884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a06b1-54ff-4036-89a6-076d517fd028 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.087154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a06b1-54ff-4036-89a6-076d517fd028 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.087793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a06b1-54ff-4036-89a6-076d517fd028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.089427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a06b1-54ff-4036-89a6-076d517fd028 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.102078 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4007/7925] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:57.106814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae415e-1ef3-4102-ac8d-6838a0942721 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.117967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae415e-1ef3-4102-ac8d-6838a0942721 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.123634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e5156-a272-4542-91ae-4ae91a43bdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.124468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e5156-a272-4542-91ae-4ae91a43bdf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.124747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e5156-a272-4542-91ae-4ae91a43bdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.125013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e5156-a272-4542-91ae-4ae91a43bdf0 admin admin] PATH_INFO: `/v3/projects/c4e90799386b4fae9bf8a7b4d1000ee8/users/9a920e3b4b324d1c96bc0e044b6c7c76/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.140954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e5156-a272-4542-91ae-4ae91a43bdf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e90799386b4fae9bf8a7b4d1000ee8, user_id=9a920e3b4b324d1c96bc0e044b6c7c76, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.143657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e5156-a272-4542-91ae-4ae91a43bdf0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.157654 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3919/7926] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:57 2022] PUT /identity/v3/projects/c4e90799386b4fae9bf8a7b4d1000ee8/users/9a920e3b4b324d1c96bc0e044b6c7c76/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:57.161776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a06b1-54ff-4036-89a6-076d517fd028 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.172428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a06b1-54ff-4036-89a6-076d517fd028 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.240627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3aeba-6dc6-4db9-83c8-4c28ada16c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.241249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3aeba-6dc6-4db9-83c8-4c28ada16c6f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.241525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3aeba-6dc6-4db9-83c8-4c28ada16c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.241825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3aeba-6dc6-4db9-83c8-4c28ada16c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.242468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3aeba-6dc6-4db9-83c8-4c28ada16c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.244086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3aeba-6dc6-4db9-83c8-4c28ada16c6f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.254467 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4008/7927] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:57.258727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e5156-a272-4542-91ae-4ae91a43bdf0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.270132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e5156-a272-4542-91ae-4ae91a43bdf0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.274946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1d9c5-8daa-40a3-8884-c88cfbb3c439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.275598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1d9c5-8daa-40a3-8884-c88cfbb3c439 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.275877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1d9c5-8daa-40a3-8884-c88cfbb3c439 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.276146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1d9c5-8daa-40a3-8884-c88cfbb3c439 admin admin] PATH_INFO: `/v3/projects/c4e90799386b4fae9bf8a7b4d1000ee8/users/9a920e3b4b324d1c96bc0e044b6c7c76/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.278171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1d9c5-8daa-40a3-8884-c88cfbb3c439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e90799386b4fae9bf8a7b4d1000ee8, user_id=9a920e3b4b324d1c96bc0e044b6c7c76, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.280911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1d9c5-8daa-40a3-8884-c88cfbb3c439 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.291063 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3920/7928] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:26:57 2022] PUT /identity/v3/projects/c4e90799386b4fae9bf8a7b4d1000ee8/users/9a920e3b4b324d1c96bc0e044b6c7c76/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:57.302850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ac5d5-7fcb-4a15-80fd-5b0f4dbf2667 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.303139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ac5d5-7fcb-4a15-80fd-5b0f4dbf2667 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.303395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ac5d5-7fcb-4a15-80fd-5b0f4dbf2667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.333363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cb5ac5d5-7fcb-4a15-80fd-5b0f4dbf2667 None None] MFA Rules not processed for user `9a920e3b4b324d1c96bc0e044b6c7c76`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:57.334568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cb5ac5d5-7fcb-4a15-80fd-5b0f4dbf2667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:57.362776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:57.404793 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4009/7929] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:57.409888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1d9c5-8daa-40a3-8884-c88cfbb3c439 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.420955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1d9c5-8daa-40a3-8884-c88cfbb3c439 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.425915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd3f55-890a-4a2c-806b-10853ae70136 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.426413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd3f55-890a-4a2c-806b-10853ae70136 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.426709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd3f55-890a-4a2c-806b-10853ae70136 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.426977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd3f55-890a-4a2c-806b-10853ae70136 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.427609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd3f55-890a-4a2c-806b-10853ae70136 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.429241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd3f55-890a-4a2c-806b-10853ae70136 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.446413 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3921/7930] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Aug 20 21:26:57 2022] POST /identity/v3/services => generated 240 bytes in 37 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:57.451447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ac5d5-7fcb-4a15-80fd-5b0f4dbf2667 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.461201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ac5d5-7fcb-4a15-80fd-5b0f4dbf2667 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.475685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f823a9-bc29-40e8-89c8-13d94c1dd004 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.476324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f823a9-bc29-40e8-89c8-13d94c1dd004 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.476649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f823a9-bc29-40e8-89c8-13d94c1dd004 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.476912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f823a9-bc29-40e8-89c8-13d94c1dd004 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services/b2051e0a8e7f41228facfe40bd4e30b4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.477507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f823a9-bc29-40e8-89c8-13d94c1dd004 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:delete_service(service_id=b2051e0a8e7f41228facfe40bd4e30b4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.479111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f823a9-bc29-40e8-89c8-13d94c1dd004 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.504316 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4010/7931] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:57 2022] DELETE /identity/v3/services/b2051e0a8e7f41228facfe40bd4e30b4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:57.509506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd3f55-890a-4a2c-806b-10853ae70136 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.521280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd3f55-890a-4a2c-806b-10853ae70136 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.531693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:57.538317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1320bab-5ecc-4b97-b2c6-aa98ea23d9bf tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.538942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1320bab-5ecc-4b97-b2c6-aa98ea23d9bf tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.539241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1320bab-5ecc-4b97-b2c6-aa98ea23d9bf tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.539530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1320bab-5ecc-4b97-b2c6-aa98ea23d9bf tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.540270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1320bab-5ecc-4b97-b2c6-aa98ea23d9bf tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.542036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1320bab-5ecc-4b97-b2c6-aa98ea23d9bf tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.558294 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3922/7932] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:57 2022] POST /identity/v3/services => generated 288 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:57.562757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f823a9-bc29-40e8-89c8-13d94c1dd004 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.574285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f823a9-bc29-40e8-89c8-13d94c1dd004 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.591315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedf8d4-807b-4ba9-812b-f7ad19bed164 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.591948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedf8d4-807b-4ba9-812b-f7ad19bed164 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.592331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedf8d4-807b-4ba9-812b-f7ad19bed164 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.592662 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedf8d4-807b-4ba9-812b-f7ad19bed164 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services/150c3a345aef40aa80cd943c78c77e51` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.593358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedf8d4-807b-4ba9-812b-f7ad19bed164 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:update_service(service_id=150c3a345aef40aa80cd943c78c77e51)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.594403 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-5fedf8d4-807b-4ba9-812b-f7ad19bed164 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:57.594403 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:26:57.594403 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:26:57.595297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedf8d4-807b-4ba9-812b-f7ad19bed164 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.618299 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4011/7933] 10.222.0.23 () {60 vars in 1252 bytes} [Sat Aug 20 21:26:57 2022] PATCH /identity/v3/services/150c3a345aef40aa80cd943c78c77e51 => generated 282 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:57.622669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1320bab-5ecc-4b97-b2c6-aa98ea23d9bf tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.633765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1320bab-5ecc-4b97-b2c6-aa98ea23d9bf tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.649179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44496216-b4e8-43f3-ae74-b1a14993deb0 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.649815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44496216-b4e8-43f3-ae74-b1a14993deb0 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.650098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44496216-b4e8-43f3-ae74-b1a14993deb0 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.650364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44496216-b4e8-43f3-ae74-b1a14993deb0 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services/150c3a345aef40aa80cd943c78c77e51` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.650974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44496216-b4e8-43f3-ae74-b1a14993deb0 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:get_service(service_id=150c3a345aef40aa80cd943c78c77e51)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.651994 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-44496216-b4e8-43f3-ae74-b1a14993deb0 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_service": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:57.651994 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:26:57.651994 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:26:57.652935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44496216-b4e8-43f3-ae74-b1a14993deb0 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.659003 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3923/7934] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:26:57 2022] GET /identity/v3/services/150c3a345aef40aa80cd943c78c77e51 => generated 282 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:57.663337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedf8d4-807b-4ba9-812b-f7ad19bed164 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.675653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedf8d4-807b-4ba9-812b-f7ad19bed164 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.681029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce164dfd-ca87-4405-86fc-79856fa3d735 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.681600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce164dfd-ca87-4405-86fc-79856fa3d735 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.681872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce164dfd-ca87-4405-86fc-79856fa3d735 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.682139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce164dfd-ca87-4405-86fc-79856fa3d735 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services/150c3a345aef40aa80cd943c78c77e51` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.682725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce164dfd-ca87-4405-86fc-79856fa3d735 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:delete_service(service_id=150c3a345aef40aa80cd943c78c77e51)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.684365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce164dfd-ca87-4405-86fc-79856fa3d735 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.707802 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4012/7935] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:57 2022] DELETE /identity/v3/services/150c3a345aef40aa80cd943c78c77e51 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:57.712109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44496216-b4e8-43f3-ae74-b1a14993deb0 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.722944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44496216-b4e8-43f3-ae74-b1a14993deb0 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.737841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba32df-984c-42e0-941c-a7356b69a4f9 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.738453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba32df-984c-42e0-941c-a7356b69a4f9 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.738727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba32df-984c-42e0-941c-a7356b69a4f9 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.738986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba32df-984c-42e0-941c-a7356b69a4f9 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services/150c3a345aef40aa80cd943c78c77e51` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.739569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba32df-984c-42e0-941c-a7356b69a4f9 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:get_service(service_id=150c3a345aef40aa80cd943c78c77e51)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.741204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba32df-984c-42e0-941c-a7356b69a4f9 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application [None req-1eba32df-984c-42e0-941c-a7356b69a4f9 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Could not find service: 150c3a345aef40aa80cd943c78c77e51.: keystone.exception.ServiceNotFound: Could not find service: 150c3a345aef40aa80cd943c78c77e51. Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: 150c3a345aef40aa80cd943c78c77e51. Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 20 21:26:57.750124 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 20 21:26:57.751314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.get_or_create( Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application with Lock( Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self._enter() Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application return self.creator() Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application created_value = creator( Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: 150c3a345aef40aa80cd943c78c77e51. Aug 20 21:26:57.752314 ncdv-master-1 devstack@keystone.service[104119]: ERROR keystone.server.flask.application Aug 20 21:26:57.753432 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3924/7936] 10.222.0.23 () {58 vars in 1230 bytes} [Sat Aug 20 21:26:57 2022] GET /identity/v3/services/150c3a345aef40aa80cd943c78c77e51 => generated 113 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 20 21:26:57.756578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce164dfd-ca87-4405-86fc-79856fa3d735 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.768170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce164dfd-ca87-4405-86fc-79856fa3d735 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.774630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb435a6-b2b4-48b3-999f-ba2c78a5ea75 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.775222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb435a6-b2b4-48b3-999f-ba2c78a5ea75 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.775492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb435a6-b2b4-48b3-999f-ba2c78a5ea75 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.775757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb435a6-b2b4-48b3-999f-ba2c78a5ea75 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.776408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb435a6-b2b4-48b3-999f-ba2c78a5ea75 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.778073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb435a6-b2b4-48b3-999f-ba2c78a5ea75 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.797814 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4013/7937] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:57 2022] POST /identity/v3/services => generated 273 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:57.799686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba32df-984c-42e0-941c-a7356b69a4f9 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.810785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba32df-984c-42e0-941c-a7356b69a4f9 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.830663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822a787-f368-4d32-ad35-1b19cbe83a53 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.830663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822a787-f368-4d32-ad35-1b19cbe83a53 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.830663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822a787-f368-4d32-ad35-1b19cbe83a53 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.831161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822a787-f368-4d32-ad35-1b19cbe83a53 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.831161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822a787-f368-4d32-ad35-1b19cbe83a53 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.831161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822a787-f368-4d32-ad35-1b19cbe83a53 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.854017 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3925/7938] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:57 2022] POST /identity/v3/services => generated 272 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:26:57.860471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb435a6-b2b4-48b3-999f-ba2c78a5ea75 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.872223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb435a6-b2b4-48b3-999f-ba2c78a5ea75 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.894016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afcee1-82d4-42d7-92bb-b87eba590711 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.894611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afcee1-82d4-42d7-92bb-b87eba590711 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.894880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afcee1-82d4-42d7-92bb-b87eba590711 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.895169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afcee1-82d4-42d7-92bb-b87eba590711 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.895770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afcee1-82d4-42d7-92bb-b87eba590711 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.897392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afcee1-82d4-42d7-92bb-b87eba590711 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.915130 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4014/7939] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:26:57 2022] POST /identity/v3/services => generated 275 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:26:57.919547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822a787-f368-4d32-ad35-1b19cbe83a53 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.930750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822a787-f368-4d32-ad35-1b19cbe83a53 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:57.957845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494dd31-6a6a-43ea-8e84-0e808054f0eb tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:57.958314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494dd31-6a6a-43ea-8e84-0e808054f0eb tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:57.958586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494dd31-6a6a-43ea-8e84-0e808054f0eb tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:57.958864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494dd31-6a6a-43ea-8e84-0e808054f0eb tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:57.959479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494dd31-6a6a-43ea-8e84-0e808054f0eb tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:57.960520 ncdv-master-1 devstack@keystone.service[104119]: WARNING py.warnings [None req-a494dd31-6a6a-43ea-8e84-0e808054f0eb tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:57.960520 ncdv-master-1 devstack@keystone.service[104119]: warnings.warn(msg) Aug 20 21:26:57.960520 ncdv-master-1 devstack@keystone.service[104119]: Aug 20 21:26:57.961404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494dd31-6a6a-43ea-8e84-0e808054f0eb tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:57.969730 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3926/7940] 10.222.0.23 () {58 vars in 1131 bytes} [Sat Aug 20 21:26:57 2022] GET /identity/v3/services => generated 2975 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:26:57.975774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afcee1-82d4-42d7-92bb-b87eba590711 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:57.994002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afcee1-82d4-42d7-92bb-b87eba590711 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:58.001270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f62857-34ea-40fe-812c-fa55ffdddb2f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:58.002332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f62857-34ea-40fe-812c-fa55ffdddb2f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:58.002800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f62857-34ea-40fe-812c-fa55ffdddb2f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:58.003280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f62857-34ea-40fe-812c-fa55ffdddb2f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:58.004275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f62857-34ea-40fe-812c-fa55ffdddb2f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-377721435) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:58.004790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f62857-34ea-40fe-812c-fa55ffdddb2f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:58.006093 ncdv-master-1 devstack@keystone.service[104118]: WARNING py.warnings [None req-19f62857-34ea-40fe-812c-fa55ffdddb2f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 20 21:26:58.006093 ncdv-master-1 devstack@keystone.service[104118]: warnings.warn(msg) Aug 20 21:26:58.006093 ncdv-master-1 devstack@keystone.service[104118]: Aug 20 21:26:58.007172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f62857-34ea-40fe-812c-fa55ffdddb2f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:58.013442 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4015/7941] 10.222.0.23 () {58 vars in 1220 bytes} [Sat Aug 20 21:26:57 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-377721435 => generated 415 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:58.018283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494dd31-6a6a-43ea-8e84-0e808054f0eb tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:58.029181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494dd31-6a6a-43ea-8e84-0e808054f0eb tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:58.035681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507b6d8-d526-43c1-8b95-4bb2daf8434e tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:58.036290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507b6d8-d526-43c1-8b95-4bb2daf8434e tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:58.036589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507b6d8-d526-43c1-8b95-4bb2daf8434e tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:58.036854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507b6d8-d526-43c1-8b95-4bb2daf8434e tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:58.037481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507b6d8-d526-43c1-8b95-4bb2daf8434e tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-283011764) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:58.037770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507b6d8-d526-43c1-8b95-4bb2daf8434e tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:58.039415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507b6d8-d526-43c1-8b95-4bb2daf8434e tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:58.044806 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3927/7942] 10.222.0.23 () {58 vars in 1220 bytes} [Sat Aug 20 21:26:58 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-283011764 => generated 414 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:58.049378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f62857-34ea-40fe-812c-fa55ffdddb2f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:58.060670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f62857-34ea-40fe-812c-fa55ffdddb2f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:58.067279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf5b50-dbe7-4860-abf9-83060e125644 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:58.068130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf5b50-dbe7-4860-abf9-83060e125644 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:58.068693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf5b50-dbe7-4860-abf9-83060e125644 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:58.069131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf5b50-dbe7-4860-abf9-83060e125644 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:58.069884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf5b50-dbe7-4860-abf9-83060e125644 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-1331939277) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:26:58.070181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf5b50-dbe7-4860-abf9-83060e125644 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:58.072405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf5b50-dbe7-4860-abf9-83060e125644 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:58.078376 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4016/7943] 10.222.0.23 () {58 vars in 1222 bytes} [Sat Aug 20 21:26:58 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-1331939277 => generated 418 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:26:58.084091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507b6d8-d526-43c1-8b95-4bb2daf8434e tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:58.102306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507b6d8-d526-43c1-8b95-4bb2daf8434e tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:58.109909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623dfd5-7bd2-4f9e-91f5-c69d99c4bf9f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:58.110968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623dfd5-7bd2-4f9e-91f5-c69d99c4bf9f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:58.111407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623dfd5-7bd2-4f9e-91f5-c69d99c4bf9f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:58.111839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623dfd5-7bd2-4f9e-91f5-c69d99c4bf9f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services/973e40e015f44d13bc4edee2a915e949` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:58.112835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623dfd5-7bd2-4f9e-91f5-c69d99c4bf9f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:delete_service(service_id=973e40e015f44d13bc4edee2a915e949)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:58.114864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623dfd5-7bd2-4f9e-91f5-c69d99c4bf9f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:58.143203 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3928/7944] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:58 2022] DELETE /identity/v3/services/973e40e015f44d13bc4edee2a915e949 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:58.147864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf5b50-dbe7-4860-abf9-83060e125644 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:58.162353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf5b50-dbe7-4860-abf9-83060e125644 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:58.179906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0e951-d654-426b-b4ef-e35a742d9226 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:58.180596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0e951-d654-426b-b4ef-e35a742d9226 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:58.180845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0e951-d654-426b-b4ef-e35a742d9226 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:58.181115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0e951-d654-426b-b4ef-e35a742d9226 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services/85eb0da14791442783be487b807d348a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:58.181705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0e951-d654-426b-b4ef-e35a742d9226 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:delete_service(service_id=85eb0da14791442783be487b807d348a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:58.183585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0e951-d654-426b-b4ef-e35a742d9226 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:58.207428 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4017/7945] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:58 2022] DELETE /identity/v3/services/85eb0da14791442783be487b807d348a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:58.212494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623dfd5-7bd2-4f9e-91f5-c69d99c4bf9f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:58.228931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623dfd5-7bd2-4f9e-91f5-c69d99c4bf9f tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:58.245945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb59f2-c1b2-4465-8ca8-bf0cfc77c107 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a920e3b4b324d1c96bc0e044b6c7c76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:58.247446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb59f2-c1b2-4465-8ca8-bf0cfc77c107 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:58.248090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb59f2-c1b2-4465-8ca8-bf0cfc77c107 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:58.248666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb59f2-c1b2-4465-8ca8-bf0cfc77c107 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] PATH_INFO: `/v3/services/b4e0de4ffa1c4f88b2caeafa0a3e1795` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:58.249709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb59f2-c1b2-4465-8ca8-bf0cfc77c107 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorizing `identity:delete_service(service_id=b4e0de4ffa1c4f88b2caeafa0a3e1795)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:58.252472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb59f2-c1b2-4465-8ca8-bf0cfc77c107 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:58.279903 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3929/7946] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:58 2022] DELETE /identity/v3/services/b4e0de4ffa1c4f88b2caeafa0a3e1795 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:58.285512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0e951-d654-426b-b4ef-e35a742d9226 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:58.308102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:58.377205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0e951-d654-426b-b4ef-e35a742d9226 tempest-ServicesTestJSON-2140478979 tempest-ServicesTestJSON-2140478979-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:58.398177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:58.398852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:58.399177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:58.399608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] PATH_INFO: `/v3/users/9d865b08512b4eed97ed2cd98c7bef01` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:58.401076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d865b08512b4eed97ed2cd98c7bef01)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:58.403572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:58.461852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d865b08512b4eed97ed2cd98c7bef01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:58.471370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d865b08512b4eed97ed2cd98c7bef01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:58.482977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d865b08512b4eed97ed2cd98c7bef01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:58.500753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d865b08512b4eed97ed2cd98c7bef01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:58.501928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d865b08512b4eed97ed2cd98c7bef01'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:58.511407 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4018/7947] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:58 2022] DELETE /identity/v3/users/9d865b08512b4eed97ed2cd98c7bef01 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:58.517958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584a5d-c785-47b7-95e3-05e8fa4c8c36 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:58.518348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584a5d-c785-47b7-95e3-05e8fa4c8c36 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:58.518677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584a5d-c785-47b7-95e3-05e8fa4c8c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:58.544225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9a584a5d-c785-47b7-95e3-05e8fa4c8c36 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:58.545646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9a584a5d-c785-47b7-95e3-05e8fa4c8c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:58.630515 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3930/7948] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:58.642543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:58.644238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:58.729680 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a946c-53dc-48b9-b2cf-76c5bc8f2afb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:58.734876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7d0e1-f707-4c61-a653-7e1a98016d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:58.735475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7d0e1-f707-4c61-a653-7e1a98016d43 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:58.735733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7d0e1-f707-4c61-a653-7e1a98016d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:58.736022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7d0e1-f707-4c61-a653-7e1a98016d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:58.749310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7d0e1-f707-4c61-a653-7e1a98016d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:58.751434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7d0e1-f707-4c61-a653-7e1a98016d43 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:58.767888 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4019/7949] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:26:59.104401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a584a5d-c785-47b7-95e3-05e8fa4c8c36 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:59.105999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9a584a5d-c785-47b7-95e3-05e8fa4c8c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:59.122988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a584a5d-c785-47b7-95e3-05e8fa4c8c36 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:59.130828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:59.131553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:59.131711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:59.131956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] PATH_INFO: `/v3/projects/1c55b70e150f4b3599a24629d70bb7cd` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:59.133350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c55b70e150f4b3599a24629d70bb7cd)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:59.135317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:59.137995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:59.189039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c55b70e150f4b3599a24629d70bb7cd', 'request_id': 'req-94381469-e0d4-4cc9-b716-ab898f77970f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.196491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c55b70e150f4b3599a24629d70bb7cd', 'request_id': 'req-94381469-e0d4-4cc9-b716-ab898f77970f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.206095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c55b70e150f4b3599a24629d70bb7cd', 'request_id': 'req-94381469-e0d4-4cc9-b716-ab898f77970f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.217151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] The token cache is being invalidate because project 1c55b70e150f4b3599a24629d70bb7cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:59.217428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.218110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.219925 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3931/7950] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:59 2022] DELETE /identity/v3/projects/1c55b70e150f4b3599a24629d70bb7cd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:59.224326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7d0e1-f707-4c61-a653-7e1a98016d43 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:59.235776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7d0e1-f707-4c61-a653-7e1a98016d43 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:59.244184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:26:59.297470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc08dc-f926-45ad-a0e8-ba10ec8321e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:59.298010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc08dc-f926-45ad-a0e8-ba10ec8321e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:59.298285 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc08dc-f926-45ad-a0e8-ba10ec8321e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:59.298553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc08dc-f926-45ad-a0e8-ba10ec8321e5 admin admin] PATH_INFO: `/v3/users/9a920e3b4b324d1c96bc0e044b6c7c76` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:59.299827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc08dc-f926-45ad-a0e8-ba10ec8321e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a920e3b4b324d1c96bc0e044b6c7c76)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:59.302014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc08dc-f926-45ad-a0e8-ba10ec8321e5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:59.357366 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-08dc08dc-f926-45ad-a0e8-ba10ec8321e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a920e3b4b324d1c96bc0e044b6c7c76'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.366896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-08dc08dc-f926-45ad-a0e8-ba10ec8321e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a920e3b4b324d1c96bc0e044b6c7c76'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.374292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-08dc08dc-f926-45ad-a0e8-ba10ec8321e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a920e3b4b324d1c96bc0e044b6c7c76'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.386454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-08dc08dc-f926-45ad-a0e8-ba10ec8321e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a920e3b4b324d1c96bc0e044b6c7c76'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.387197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-08dc08dc-f926-45ad-a0e8-ba10ec8321e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a920e3b4b324d1c96bc0e044b6c7c76'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.393642 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4020/7951] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:26:59 2022] DELETE /identity/v3/users/9a920e3b4b324d1c96bc0e044b6c7c76 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:59.709473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:26:59.711408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:59.726551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94381469-e0d4-4cc9-b716-ab898f77970f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:26:59.787671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:26:59.788347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:59.788679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:59.788942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08 admin admin] PATH_INFO: `/v3/projects/c4e90799386b4fae9bf8a7b4d1000ee8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:59.790091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e90799386b4fae9bf8a7b4d1000ee8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:26:59.792042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:26:59.839685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'request_id': 'req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.846382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'request_id': 'req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.856140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e90799386b4fae9bf8a7b4d1000ee8', 'request_id': 'req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.867553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08 admin admin] The token cache is being invalidate because project c4e90799386b4fae9bf8a7b4d1000ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:26:59.867832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.868478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ecfcc33d-965d-46f9-b0f9-5f047ecc2d08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:26:59.869992 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3932/7952] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:26:59 2022] DELETE /identity/v3/projects/c4e90799386b4fae9bf8a7b4d1000ee8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:26:59.877291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d193697-a698-4cd6-ac23-267cbc74b0fd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:59.877607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d193697-a698-4cd6-ac23-267cbc74b0fd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:59.877871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d193697-a698-4cd6-ac23-267cbc74b0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:59.898065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4d193697-a698-4cd6-ac23-267cbc74b0fd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:59.899251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4d193697-a698-4cd6-ac23-267cbc74b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:26:59.962208 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4021/7953] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:26:59.970981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f231851-0883-48a3-a252-53670ec20943 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:26:59.971262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f231851-0883-48a3-a252-53670ec20943 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:26:59.971532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f231851-0883-48a3-a252-53670ec20943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:26:59.995387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5f231851-0883-48a3-a252-53670ec20943 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:26:59.997213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5f231851-0883-48a3-a252-53670ec20943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:00.007682 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3933/7954] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:00.012125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d193697-a698-4cd6-ac23-267cbc74b0fd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:00.023207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d193697-a698-4cd6-ac23-267cbc74b0fd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:00.028673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b817d89-c49e-4839-9b0e-818f0f8d97a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:00.029282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b817d89-c49e-4839-9b0e-818f0f8d97a1 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:00.029553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b817d89-c49e-4839-9b0e-818f0f8d97a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:00.029817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b817d89-c49e-4839-9b0e-818f0f8d97a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:00.030468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b817d89-c49e-4839-9b0e-818f0f8d97a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:27:00.030762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b817d89-c49e-4839-9b0e-818f0f8d97a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:00.033031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b817d89-c49e-4839-9b0e-818f0f8d97a1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:00.045339 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4022/7955] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:27:00.049659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f231851-0883-48a3-a252-53670ec20943 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:00.059522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f231851-0883-48a3-a252-53670ec20943 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:00.064961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b790560-1386-4a1f-95d4-2691bc1ca8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:00.065570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b790560-1386-4a1f-95d4-2691bc1ca8b5 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:00.065843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b790560-1386-4a1f-95d4-2691bc1ca8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:00.066085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b790560-1386-4a1f-95d4-2691bc1ca8b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:00.066698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b790560-1386-4a1f-95d4-2691bc1ca8b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:00.068969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b790560-1386-4a1f-95d4-2691bc1ca8b5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:00.085926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:00.105426 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3934/7956] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:00 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:00.109990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b817d89-c49e-4839-9b0e-818f0f8d97a1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:00.120646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b817d89-c49e-4839-9b0e-818f0f8d97a1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:00.134655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:00.189171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62177be5-fd94-4f7a-906b-ad06c5d948f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:00.190010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62177be5-fd94-4f7a-906b-ad06c5d948f0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:00.190363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62177be5-fd94-4f7a-906b-ad06c5d948f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:00.190729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62177be5-fd94-4f7a-906b-ad06c5d948f0 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:00.191652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62177be5-fd94-4f7a-906b-ad06c5d948f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:00.194940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62177be5-fd94-4f7a-906b-ad06c5d948f0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:00.220956 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4023/7957] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:00 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:00.228704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b790560-1386-4a1f-95d4-2691bc1ca8b5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:00.236676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b790560-1386-4a1f-95d4-2691bc1ca8b5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:00.241567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf4693-fb3d-4120-bf97-844157755b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:00.242166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf4693-fb3d-4120-bf97-844157755b31 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:00.242437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf4693-fb3d-4120-bf97-844157755b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:00.242699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf4693-fb3d-4120-bf97-844157755b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:00.243334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf4693-fb3d-4120-bf97-844157755b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:00.244993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf4693-fb3d-4120-bf97-844157755b31 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:00.255595 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3935/7958] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:00.260236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62177be5-fd94-4f7a-906b-ad06c5d948f0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:00.271786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62177be5-fd94-4f7a-906b-ad06c5d948f0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:00.277397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e765359-4bd8-41ef-a145-c53c91350558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:00.278002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e765359-4bd8-41ef-a145-c53c91350558 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:00.278274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e765359-4bd8-41ef-a145-c53c91350558 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:00.278541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e765359-4bd8-41ef-a145-c53c91350558 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:00.279181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e765359-4bd8-41ef-a145-c53c91350558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:00.280841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e765359-4bd8-41ef-a145-c53c91350558 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:00.297512 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4024/7959] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:00.303418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf4693-fb3d-4120-bf97-844157755b31 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:00.318212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf4693-fb3d-4120-bf97-844157755b31 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:00.326681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e7f0-a310-4cd6-821c-2c47d8cacce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:00.327683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e7f0-a310-4cd6-821c-2c47d8cacce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:00.328074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e7f0-a310-4cd6-821c-2c47d8cacce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:00.328355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e7f0-a310-4cd6-821c-2c47d8cacce7 admin admin] PATH_INFO: `/v3/projects/c0bb80f8868a4380a681d601d10dee6e/users/61e3bb0baced4cf6b046ee0744c6b767/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:00.344913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0e7f0-a310-4cd6-821c-2c47d8cacce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0bb80f8868a4380a681d601d10dee6e, user_id=61e3bb0baced4cf6b046ee0744c6b767, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:00.347619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0e7f0-a310-4cd6-821c-2c47d8cacce7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:00.363061 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3936/7960] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:00 2022] PUT /identity/v3/projects/c0bb80f8868a4380a681d601d10dee6e/users/61e3bb0baced4cf6b046ee0744c6b767/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:00.373755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d98764-7a29-4e18-b7a6-6c3bf3d71b32 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:00.374029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d98764-7a29-4e18-b7a6-6c3bf3d71b32 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:00.374291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d98764-7a29-4e18-b7a6-6c3bf3d71b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:00.403860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-67d98764-7a29-4e18-b7a6-6c3bf3d71b32 None None] MFA Rules not processed for user `61e3bb0baced4cf6b046ee0744c6b767`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:00.405104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-67d98764-7a29-4e18-b7a6-6c3bf3d71b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:00.483638 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4025/7961] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:00.495707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e7f0-a310-4cd6-821c-2c47d8cacce7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:00.497554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b6e0e7f0-a310-4cd6-821c-2c47d8cacce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:00.573439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e7f0-a310-4cd6-821c-2c47d8cacce7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:00.578121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1c2ce-a16a-4c19-ba15-4b79df606322 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:00.578729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1c2ce-a16a-4c19-ba15-4b79df606322 service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:00.579007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1c2ce-a16a-4c19-ba15-4b79df606322 service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:00.579267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1c2ce-a16a-4c19-ba15-4b79df606322 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:00.588753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1c2ce-a16a-4c19-ba15-4b79df606322 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:00.590197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1c2ce-a16a-4c19-ba15-4b79df606322 service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:00.602593 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3937/7962] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:00.740943 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0333598-1e85-4b67-9cba-18e67a4b5b91 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:00.741234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0333598-1e85-4b67-9cba-18e67a4b5b91 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:00.741702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0333598-1e85-4b67-9cba-18e67a4b5b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:00.764253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d0333598-1e85-4b67-9cba-18e67a4b5b91 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:00.765552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d0333598-1e85-4b67-9cba-18e67a4b5b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:00.834048 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4026/7963] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:27:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:00.886992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1c2ce-a16a-4c19-ba15-4b79df606322 service glance] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:00.888547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0ad1c2ce-a16a-4c19-ba15-4b79df606322 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:00.983629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1c2ce-a16a-4c19-ba15-4b79df606322 service glance] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:00.989511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a169e-e051-4c4a-9137-144aca1bb5a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:00.990113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a169e-e051-4c4a-9137-144aca1bb5a6 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:00.990383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a169e-e051-4c4a-9137-144aca1bb5a6 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:00.990647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a169e-e051-4c4a-9137-144aca1bb5a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:00.999149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a169e-e051-4c4a-9137-144aca1bb5a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:01.001170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a169e-e051-4c4a-9137-144aca1bb5a6 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:01.009611 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3938/7964] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:27:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:27:01.377421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f3e23-dd3e-4e10-8e73-fca4b83f7e0e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:01.377970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f3e23-dd3e-4e10-8e73-fca4b83f7e0e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:01.378098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f3e23-dd3e-4e10-8e73-fca4b83f7e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:01.401728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-875f3e23-dd3e-4e10-8e73-fca4b83f7e0e None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:01.403652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-875f3e23-dd3e-4e10-8e73-fca4b83f7e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:01.413836 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4027/7965] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:27:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:01.431097 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a169e-e051-4c4a-9137-144aca1bb5a6 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:01.446093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a169e-e051-4c4a-9137-144aca1bb5a6 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:01.452576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039593d-7afa-4aea-a03f-0b01070c5040 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:01.453165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039593d-7afa-4aea-a03f-0b01070c5040 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:01.453434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039593d-7afa-4aea-a03f-0b01070c5040 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:01.453678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039593d-7afa-4aea-a03f-0b01070c5040 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:01.462309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039593d-7afa-4aea-a03f-0b01070c5040 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:01.463896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039593d-7afa-4aea-a03f-0b01070c5040 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:01.472222 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3939/7966] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:27:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:27:01.848027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24046721-2828-49d6-8c69-12a5549d6cb4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:01.848318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24046721-2828-49d6-8c69-12a5549d6cb4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:01.848605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24046721-2828-49d6-8c69-12a5549d6cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:01.869879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-24046721-2828-49d6-8c69-12a5549d6cb4 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:01.871078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-24046721-2828-49d6-8c69-12a5549d6cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:01.879258 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4028/7967] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:27:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:01.892686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039593d-7afa-4aea-a03f-0b01070c5040 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:01.906308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039593d-7afa-4aea-a03f-0b01070c5040 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:01.912961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5346ee1-6388-4f11-b56a-3a6c540a887d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:01.913541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5346ee1-6388-4f11-b56a-3a6c540a887d service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:01.913821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5346ee1-6388-4f11-b56a-3a6c540a887d service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:01.914082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5346ee1-6388-4f11-b56a-3a6c540a887d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:01.923618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5346ee1-6388-4f11-b56a-3a6c540a887d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:01.925259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5346ee1-6388-4f11-b56a-3a6c540a887d service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:01.934098 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3940/7968] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:27:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:27:02.099485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7209612-bfd9-48e0-acd2-0b1dcffe58b0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:02.099846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7209612-bfd9-48e0-acd2-0b1dcffe58b0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:02.100113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7209612-bfd9-48e0-acd2-0b1dcffe58b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:02.119695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f7209612-bfd9-48e0-acd2-0b1dcffe58b0 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:02.120903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f7209612-bfd9-48e0-acd2-0b1dcffe58b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:02.128541 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4029/7969] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:27:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:02.143689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5346ee1-6388-4f11-b56a-3a6c540a887d service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:02.160668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5346ee1-6388-4f11-b56a-3a6c540a887d service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:02.166462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad578df-9993-4e05-8fc5-c4b6ffc09069 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:02.167060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad578df-9993-4e05-8fc5-c4b6ffc09069 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:02.167315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad578df-9993-4e05-8fc5-c4b6ffc09069 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:02.167590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad578df-9993-4e05-8fc5-c4b6ffc09069 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:02.173087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:02.176852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad578df-9993-4e05-8fc5-c4b6ffc09069 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:02.179013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad578df-9993-4e05-8fc5-c4b6ffc09069 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:02.187832 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3941/7970] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:27:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:27:02.510136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dba53a-b9fb-4e78-978f-4765c3e83192 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:02.510634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dba53a-b9fb-4e78-978f-4765c3e83192 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:02.510751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dba53a-b9fb-4e78-978f-4765c3e83192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:02.518594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:02.535753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f6dba53a-b9fb-4e78-978f-4765c3e83192 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:02.537051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f6dba53a-b9fb-4e78-978f-4765c3e83192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:02.546438 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4030/7971] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:27:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:02.559772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad578df-9993-4e05-8fc5-c4b6ffc09069 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:02.579176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad578df-9993-4e05-8fc5-c4b6ffc09069 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:02.585298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5e477-482d-4c8b-84de-87e0f930895a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:02.585904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5e477-482d-4c8b-84de-87e0f930895a service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:02.586178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5e477-482d-4c8b-84de-87e0f930895a service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:02.586447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5e477-482d-4c8b-84de-87e0f930895a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:02.595171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5e477-482d-4c8b-84de-87e0f930895a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:02.596802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5e477-482d-4c8b-84de-87e0f930895a service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:02.605139 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3942/7972] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:27:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:27:02.791534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ba035-967f-422b-88c7-3178a885db3d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:02.792135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ba035-967f-422b-88c7-3178a885db3d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:02.792479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ba035-967f-422b-88c7-3178a885db3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:02.819161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cb1ba035-967f-422b-88c7-3178a885db3d None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:02.820362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cb1ba035-967f-422b-88c7-3178a885db3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:02.828339 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4031/7973] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:27:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:02.840718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5e477-482d-4c8b-84de-87e0f930895a service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:02.854698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5e477-482d-4c8b-84de-87e0f930895a service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:02.860901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88629f27-4cd1-4f62-8c54-3f387fbf7cb7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:02.861496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88629f27-4cd1-4f62-8c54-3f387fbf7cb7 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:02.861783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88629f27-4cd1-4f62-8c54-3f387fbf7cb7 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:02.862035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88629f27-4cd1-4f62-8c54-3f387fbf7cb7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:02.871173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88629f27-4cd1-4f62-8c54-3f387fbf7cb7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:02.872804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88629f27-4cd1-4f62-8c54-3f387fbf7cb7 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:02.881024 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3943/7974] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:27:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:27:03.015597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ba035-967f-422b-88c7-3178a885db3d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:03.084498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ba035-967f-422b-88c7-3178a885db3d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:03.088957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:03.089581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:03.089849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:03.090109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] PATH_INFO: `/v3/users/61e3bb0baced4cf6b046ee0744c6b767` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:03.091268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61e3bb0baced4cf6b046ee0744c6b767)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:03.093234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:03.140843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61e3bb0baced4cf6b046ee0744c6b767'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:03.150212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61e3bb0baced4cf6b046ee0744c6b767'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:03.159158 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61e3bb0baced4cf6b046ee0744c6b767'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:03.172208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61e3bb0baced4cf6b046ee0744c6b767'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:03.173538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61e3bb0baced4cf6b046ee0744c6b767'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:03.180243 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4032/7975] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:03 2022] DELETE /identity/v3/users/61e3bb0baced4cf6b046ee0744c6b767 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:03.184520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea60ff-5ebb-4a80-8651-94474cbd8da4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:03.184973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea60ff-5ebb-4a80-8651-94474cbd8da4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:03.185355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea60ff-5ebb-4a80-8651-94474cbd8da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:03.212393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-acea60ff-5ebb-4a80-8651-94474cbd8da4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:03.213633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-acea60ff-5ebb-4a80-8651-94474cbd8da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:03.285718 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3944/7976] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:03.295980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:03.297511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:03.367105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d3d36-1c0d-42f5-802e-6974f0b5f987 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:03.372097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b105e4-f9c3-4107-a605-6ab41a8b2760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:03.372793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b105e4-f9c3-4107-a605-6ab41a8b2760 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:03.373101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b105e4-f9c3-4107-a605-6ab41a8b2760 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:03.373412 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b105e4-f9c3-4107-a605-6ab41a8b2760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:03.383819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b105e4-f9c3-4107-a605-6ab41a8b2760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:03.385310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b105e4-f9c3-4107-a605-6ab41a8b2760 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:03.395936 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4033/7977] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:27:03.690631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea60ff-5ebb-4a80-8651-94474cbd8da4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:03.692824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-acea60ff-5ebb-4a80-8651-94474cbd8da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:03.710160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea60ff-5ebb-4a80-8651-94474cbd8da4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:03.716001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed90fd6-fbcd-487b-9cd6-293c1058070d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:03.716680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed90fd6-fbcd-487b-9cd6-293c1058070d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:03.716954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed90fd6-fbcd-487b-9cd6-293c1058070d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:03.717240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed90fd6-fbcd-487b-9cd6-293c1058070d admin admin] PATH_INFO: `/v3/projects/c0bb80f8868a4380a681d601d10dee6e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:03.718736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed90fd6-fbcd-487b-9cd6-293c1058070d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0bb80f8868a4380a681d601d10dee6e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:03.720828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed90fd6-fbcd-487b-9cd6-293c1058070d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:03.770844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5ed90fd6-fbcd-487b-9cd6-293c1058070d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0bb80f8868a4380a681d601d10dee6e', 'request_id': 'req-5ed90fd6-fbcd-487b-9cd6-293c1058070d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:03.777061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5ed90fd6-fbcd-487b-9cd6-293c1058070d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0bb80f8868a4380a681d601d10dee6e', 'request_id': 'req-5ed90fd6-fbcd-487b-9cd6-293c1058070d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:03.786198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5ed90fd6-fbcd-487b-9cd6-293c1058070d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0bb80f8868a4380a681d601d10dee6e', 'request_id': 'req-5ed90fd6-fbcd-487b-9cd6-293c1058070d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:03.799481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5ed90fd6-fbcd-487b-9cd6-293c1058070d admin admin] The token cache is being invalidate because project c0bb80f8868a4380a681d601d10dee6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:03.799675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5ed90fd6-fbcd-487b-9cd6-293c1058070d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:03.800704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5ed90fd6-fbcd-487b-9cd6-293c1058070d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:03.802854 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3945/7978] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:03 2022] DELETE /identity/v3/projects/c0bb80f8868a4380a681d601d10dee6e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:03.807705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c36263-1e6d-40de-ae4c-1f96e767fab4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:03.808212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c36263-1e6d-40de-ae4c-1f96e767fab4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:03.808212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c36263-1e6d-40de-ae4c-1f96e767fab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:03.838556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a9c36263-1e6d-40de-ae4c-1f96e767fab4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:03.839765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a9c36263-1e6d-40de-ae4c-1f96e767fab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:03.847153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:03.904150 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4034/7979] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:03.912242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31034e-a446-401b-b0e5-d6e4a26d89a6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:03.912570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31034e-a446-401b-b0e5-d6e4a26d89a6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:03.912843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31034e-a446-401b-b0e5-d6e4a26d89a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:03.934196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ed31034e-a446-401b-b0e5-d6e4a26d89a6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:03.935414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ed31034e-a446-401b-b0e5-d6e4a26d89a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:03.945337 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3946/7980] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:03.949631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c36263-1e6d-40de-ae4c-1f96e767fab4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:03.960805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c36263-1e6d-40de-ae4c-1f96e767fab4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:03.966504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b3542-537c-402a-8213-98eb9979b6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:03.967124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b3542-537c-402a-8213-98eb9979b6ba admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:03.967397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b3542-537c-402a-8213-98eb9979b6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:03.967668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b3542-537c-402a-8213-98eb9979b6ba admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:03.968296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b3542-537c-402a-8213-98eb9979b6ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:27:03.968637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b3542-537c-402a-8213-98eb9979b6ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:03.970741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b3542-537c-402a-8213-98eb9979b6ba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:03.982806 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4035/7981] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:27:03.988301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31034e-a446-401b-b0e5-d6e4a26d89a6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:03.998027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31034e-a446-401b-b0e5-d6e4a26d89a6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:04.003637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304490c-f4ce-47df-8d1c-0c7313ec43aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:04.004230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304490c-f4ce-47df-8d1c-0c7313ec43aa admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:04.004563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304490c-f4ce-47df-8d1c-0c7313ec43aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:04.004843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304490c-f4ce-47df-8d1c-0c7313ec43aa admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:04.005449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304490c-f4ce-47df-8d1c-0c7313ec43aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:04.007377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304490c-f4ce-47df-8d1c-0c7313ec43aa admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:04.039303 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3947/7982] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:03 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:04.045629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b3542-537c-402a-8213-98eb9979b6ba admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:04.058661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b3542-537c-402a-8213-98eb9979b6ba admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:04.144922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7ca7f-8131-4e0a-b9f8-e26da7595a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:04.145556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7ca7f-8131-4e0a-b9f8-e26da7595a9d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:04.145830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7ca7f-8131-4e0a-b9f8-e26da7595a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:04.146090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7ca7f-8131-4e0a-b9f8-e26da7595a9d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:04.146734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7ca7f-8131-4e0a-b9f8-e26da7595a9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:04.148707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7ca7f-8131-4e0a-b9f8-e26da7595a9d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:04.173403 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4036/7983] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:04 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:04.178581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304490c-f4ce-47df-8d1c-0c7313ec43aa admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:04.192843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304490c-f4ce-47df-8d1c-0c7313ec43aa admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:04.197997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986ddf2-cd43-4086-937b-eb9afa0b0a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:04.198617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986ddf2-cd43-4086-937b-eb9afa0b0a3b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:04.198909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986ddf2-cd43-4086-937b-eb9afa0b0a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:04.199160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986ddf2-cd43-4086-937b-eb9afa0b0a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:04.199815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986ddf2-cd43-4086-937b-eb9afa0b0a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:04.201494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986ddf2-cd43-4086-937b-eb9afa0b0a3b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:04.211993 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3948/7984] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:04.216165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7ca7f-8131-4e0a-b9f8-e26da7595a9d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:04.227088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7ca7f-8131-4e0a-b9f8-e26da7595a9d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:04.237350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9706c6e-adbc-4fbc-bca3-6378c6883ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:04.238037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9706c6e-adbc-4fbc-bca3-6378c6883ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:04.238228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9706c6e-adbc-4fbc-bca3-6378c6883ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:04.238490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9706c6e-adbc-4fbc-bca3-6378c6883ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:04.239134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9706c6e-adbc-4fbc-bca3-6378c6883ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:04.240863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9706c6e-adbc-4fbc-bca3-6378c6883ec0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:04.252433 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4037/7985] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:04.256974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986ddf2-cd43-4086-937b-eb9afa0b0a3b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:04.267130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986ddf2-cd43-4086-937b-eb9afa0b0a3b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:04.271787 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c7a45-03a1-4d6f-b256-7f2a267a33b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:04.272482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c7a45-03a1-4d6f-b256-7f2a267a33b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:04.272755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c7a45-03a1-4d6f-b256-7f2a267a33b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:04.273018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c7a45-03a1-4d6f-b256-7f2a267a33b4 admin admin] PATH_INFO: `/v3/projects/87d02844b176418c839f75863a0ac26b/users/6ef87c8ced614c64925a690fe40ec655/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:04.281761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:04.292720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c7a45-03a1-4d6f-b256-7f2a267a33b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d02844b176418c839f75863a0ac26b, user_id=6ef87c8ced614c64925a690fe40ec655, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:04.295674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c7a45-03a1-4d6f-b256-7f2a267a33b4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:04.310297 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3949/7986] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:04 2022] PUT /identity/v3/projects/87d02844b176418c839f75863a0ac26b/users/6ef87c8ced614c64925a690fe40ec655/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:27:04.321239 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22b539-494d-4b0e-9997-0bc4594cbe23 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:04.321512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22b539-494d-4b0e-9997-0bc4594cbe23 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:04.321773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22b539-494d-4b0e-9997-0bc4594cbe23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:04.355641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1a22b539-494d-4b0e-9997-0bc4594cbe23 None None] MFA Rules not processed for user `6ef87c8ced614c64925a690fe40ec655`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:04.357008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1a22b539-494d-4b0e-9997-0bc4594cbe23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:04.424551 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4038/7987] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:04.429276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c7a45-03a1-4d6f-b256-7f2a267a33b4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:04.495849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c7a45-03a1-4d6f-b256-7f2a267a33b4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:04.502449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d036e40-61df-419a-9775-6857b7f9d9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:04.503057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d036e40-61df-419a-9775-6857b7f9d9cc admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:04.503325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d036e40-61df-419a-9775-6857b7f9d9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:04.503584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d036e40-61df-419a-9775-6857b7f9d9cc admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:04.504221 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d036e40-61df-419a-9775-6857b7f9d9cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:04.506230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d036e40-61df-419a-9775-6857b7f9d9cc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:04.536431 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3950/7988] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:04 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:27:04.540057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22b539-494d-4b0e-9997-0bc4594cbe23 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:04.551677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22b539-494d-4b0e-9997-0bc4594cbe23 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:04.555490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:04.612912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b83ac8-b30d-4aa5-b5b5-a3b51dca882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:04.613434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b83ac8-b30d-4aa5-b5b5-a3b51dca882d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:04.613711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b83ac8-b30d-4aa5-b5b5-a3b51dca882d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:04.613979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b83ac8-b30d-4aa5-b5b5-a3b51dca882d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:04.614604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b83ac8-b30d-4aa5-b5b5-a3b51dca882d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:04.616567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b83ac8-b30d-4aa5-b5b5-a3b51dca882d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:04.639708 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4039/7989] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:04 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:04.644071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d036e40-61df-419a-9775-6857b7f9d9cc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:04.654707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d036e40-61df-419a-9775-6857b7f9d9cc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:04.660041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9321247-1109-4ac6-bbf8-4d54bbaeb8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:04.660696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9321247-1109-4ac6-bbf8-4d54bbaeb8da admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:04.660976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9321247-1109-4ac6-bbf8-4d54bbaeb8da admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:04.661250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9321247-1109-4ac6-bbf8-4d54bbaeb8da admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:04.661891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9321247-1109-4ac6-bbf8-4d54bbaeb8da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:04.663560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9321247-1109-4ac6-bbf8-4d54bbaeb8da admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:04.674970 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3951/7990] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:04.679098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b83ac8-b30d-4aa5-b5b5-a3b51dca882d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:04.689936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b83ac8-b30d-4aa5-b5b5-a3b51dca882d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:04.696162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d76a2c-186a-4661-9e30-f9d55cb1ae46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:04.696807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d76a2c-186a-4661-9e30-f9d55cb1ae46 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:04.697081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d76a2c-186a-4661-9e30-f9d55cb1ae46 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:04.697343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d76a2c-186a-4661-9e30-f9d55cb1ae46 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:04.697977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d76a2c-186a-4661-9e30-f9d55cb1ae46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:04.699581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d76a2c-186a-4661-9e30-f9d55cb1ae46 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:04.710267 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4040/7991] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:04.714916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9321247-1109-4ac6-bbf8-4d54bbaeb8da admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:04.726560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9321247-1109-4ac6-bbf8-4d54bbaeb8da admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:04.732197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea2d94-591a-4d28-9c9e-72d9b02aeca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:04.732881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea2d94-591a-4d28-9c9e-72d9b02aeca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:04.733158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea2d94-591a-4d28-9c9e-72d9b02aeca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:04.733429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea2d94-591a-4d28-9c9e-72d9b02aeca9 admin admin] PATH_INFO: `/v3/projects/d78bf3e24d4646f486057715989a3295/users/d4335a5e03e34b658d43c847bf412026/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:04.750489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea2d94-591a-4d28-9c9e-72d9b02aeca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d78bf3e24d4646f486057715989a3295, user_id=d4335a5e03e34b658d43c847bf412026, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:04.753308 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea2d94-591a-4d28-9c9e-72d9b02aeca9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:04.769023 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3952/7992] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:04 2022] PUT /identity/v3/projects/d78bf3e24d4646f486057715989a3295/users/d4335a5e03e34b658d43c847bf412026/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:04.780709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609d3d2-492c-4915-b18a-f9825cdca58b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:04.780990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609d3d2-492c-4915-b18a-f9825cdca58b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:04.781310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609d3d2-492c-4915-b18a-f9825cdca58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:04.811833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e609d3d2-492c-4915-b18a-f9825cdca58b None None] MFA Rules not processed for user `d4335a5e03e34b658d43c847bf412026`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:04.813107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e609d3d2-492c-4915-b18a-f9825cdca58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:04.880406 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4041/7993] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:04.896581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea2d94-591a-4d28-9c9e-72d9b02aeca9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:04.898434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b6ea2d94-591a-4d28-9c9e-72d9b02aeca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:04.960912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea2d94-591a-4d28-9c9e-72d9b02aeca9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:04.965700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9ddba-6cca-402b-8341-a945dccc4879 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:04.966359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9ddba-6cca-402b-8341-a945dccc4879 service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:04.966608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9ddba-6cca-402b-8341-a945dccc4879 service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:04.966889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9ddba-6cca-402b-8341-a945dccc4879 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:05.018943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:05.029365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9ddba-6cca-402b-8341-a945dccc4879 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:05.030869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9ddba-6cca-402b-8341-a945dccc4879 service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:05.040127 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3953/7994] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:27:05.168324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753a814-5118-4987-a746-cb3b75b99b23 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:05.168641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753a814-5118-4987-a746-cb3b75b99b23 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:05.168902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753a814-5118-4987-a746-cb3b75b99b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:05.188517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e753a814-5118-4987-a746-cb3b75b99b23 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:05.189712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e753a814-5118-4987-a746-cb3b75b99b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:05.258810 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4042/7995] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:05.271108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9ddba-6cca-402b-8341-a945dccc4879 service glance] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:05.272731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-77f9ddba-6cca-402b-8341-a945dccc4879 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:05.358386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9ddba-6cca-402b-8341-a945dccc4879 service glance] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:05.364196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cccfaf-8ccb-474b-98e3-3e772c879add service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:05.364886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cccfaf-8ccb-474b-98e3-3e772c879add service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:05.365193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cccfaf-8ccb-474b-98e3-3e772c879add service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:05.365458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cccfaf-8ccb-474b-98e3-3e772c879add service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:05.374145 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cccfaf-8ccb-474b-98e3-3e772c879add service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:05.375730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cccfaf-8ccb-474b-98e3-3e772c879add service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:05.384183 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3954/7996] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:27:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:27:05.567629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753a814-5118-4987-a746-cb3b75b99b23 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:05.569078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e753a814-5118-4987-a746-cb3b75b99b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:05.583226 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753a814-5118-4987-a746-cb3b75b99b23 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:05.587151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168fbc5-87d5-411e-a7e2-2314a661ba54 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:05.587738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168fbc5-87d5-411e-a7e2-2314a661ba54 service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:05.588045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168fbc5-87d5-411e-a7e2-2314a661ba54 service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:05.588503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168fbc5-87d5-411e-a7e2-2314a661ba54 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:05.597833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168fbc5-87d5-411e-a7e2-2314a661ba54 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:05.599451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168fbc5-87d5-411e-a7e2-2314a661ba54 service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:05.608600 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4043/7997] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:05.703387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdb335-de38-4bec-bdc3-73ed1efafae4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:05.703783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdb335-de38-4bec-bdc3-73ed1efafae4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:05.704153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdb335-de38-4bec-bdc3-73ed1efafae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:05.725038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-76fdb335-de38-4bec-bdc3-73ed1efafae4 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:05.726219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-76fdb335-de38-4bec-bdc3-73ed1efafae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:05.733667 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3955/7998] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:05.746336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168fbc5-87d5-411e-a7e2-2314a661ba54 service glance] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:05.760941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168fbc5-87d5-411e-a7e2-2314a661ba54 service glance] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:05.766423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f3362-a324-44e9-bf8a-c66db80cefb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:05.767012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f3362-a324-44e9-bf8a-c66db80cefb9 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:05.767268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f3362-a324-44e9-bf8a-c66db80cefb9 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:05.767539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f3362-a324-44e9-bf8a-c66db80cefb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:05.772861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:05.777111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f3362-a324-44e9-bf8a-c66db80cefb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:05.778772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f3362-a324-44e9-bf8a-c66db80cefb9 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:05.788109 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4044/7999] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:27:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:27:05.905473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdb335-de38-4bec-bdc3-73ed1efafae4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:05.981636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdb335-de38-4bec-bdc3-73ed1efafae4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:05.986450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:05.987098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:05.987368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:05.987635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] PATH_INFO: `/v3/users/6ef87c8ced614c64925a690fe40ec655` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:05.989073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ef87c8ced614c64925a690fe40ec655)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:05.991038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:06.042672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ef87c8ced614c64925a690fe40ec655'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.051638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ef87c8ced614c64925a690fe40ec655'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.065867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ef87c8ced614c64925a690fe40ec655'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.066786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ef87c8ced614c64925a690fe40ec655'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.067745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ef87c8ced614c64925a690fe40ec655'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.072231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:06.082356 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3956/8000] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:05 2022] DELETE /identity/v3/users/6ef87c8ced614c64925a690fe40ec655 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:06.086910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c6d67-4f48-44c5-9692-4cfa05a31c07 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:06.087428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c6d67-4f48-44c5-9692-4cfa05a31c07 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:06.087428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c6d67-4f48-44c5-9692-4cfa05a31c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:06.107709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-621c6d67-4f48-44c5-9692-4cfa05a31c07 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:06.108975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-621c6d67-4f48-44c5-9692-4cfa05a31c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:06.180224 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4045/8001] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:06.190481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:06.192627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:06.255625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abd113-f4e4-4319-b181-50a4afad20fc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:06.259922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bd82b-a742-4584-8ffa-e1e6010ff635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:06.260957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bd82b-a742-4584-8ffa-e1e6010ff635 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:06.261457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bd82b-a742-4584-8ffa-e1e6010ff635 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:06.261939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bd82b-a742-4584-8ffa-e1e6010ff635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:06.274232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bd82b-a742-4584-8ffa-e1e6010ff635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:06.275687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bd82b-a742-4584-8ffa-e1e6010ff635 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:06.285817 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3957/8002] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:06.568330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c6d67-4f48-44c5-9692-4cfa05a31c07 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:06.569887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-621c6d67-4f48-44c5-9692-4cfa05a31c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:06.581984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c6d67-4f48-44c5-9692-4cfa05a31c07 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:06.586672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:06.587276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:06.587548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:06.587811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] PATH_INFO: `/v3/projects/87d02844b176418c839f75863a0ac26b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:06.588948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d02844b176418c839f75863a0ac26b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:06.590920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:06.640269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d02844b176418c839f75863a0ac26b', 'request_id': 'req-a8519541-7811-47d8-b016-2a77d5351fb6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.650015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d02844b176418c839f75863a0ac26b', 'request_id': 'req-a8519541-7811-47d8-b016-2a77d5351fb6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.656693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d02844b176418c839f75863a0ac26b', 'request_id': 'req-a8519541-7811-47d8-b016-2a77d5351fb6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.669688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] The token cache is being invalidate because project 87d02844b176418c839f75863a0ac26b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:06.669981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.673346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.675141 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4046/8003] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:06 2022] DELETE /identity/v3/projects/87d02844b176418c839f75863a0ac26b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:06.679298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bd82b-a742-4584-8ffa-e1e6010ff635 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:06.692808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bd82b-a742-4584-8ffa-e1e6010ff635 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:06.758243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96b6cb-80be-4005-8fc7-47b2728e7edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:06.758986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96b6cb-80be-4005-8fc7-47b2728e7edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:06.759222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96b6cb-80be-4005-8fc7-47b2728e7edc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:06.759490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96b6cb-80be-4005-8fc7-47b2728e7edc admin admin] PATH_INFO: `/v3/users/d4335a5e03e34b658d43c847bf412026` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:06.760739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96b6cb-80be-4005-8fc7-47b2728e7edc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4335a5e03e34b658d43c847bf412026)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:06.762653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96b6cb-80be-4005-8fc7-47b2728e7edc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:06.818882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fe96b6cb-80be-4005-8fc7-47b2728e7edc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4335a5e03e34b658d43c847bf412026'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.827168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fe96b6cb-80be-4005-8fc7-47b2728e7edc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4335a5e03e34b658d43c847bf412026'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.840117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fe96b6cb-80be-4005-8fc7-47b2728e7edc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4335a5e03e34b658d43c847bf412026'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.840927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fe96b6cb-80be-4005-8fc7-47b2728e7edc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4335a5e03e34b658d43c847bf412026'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.841533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-fe96b6cb-80be-4005-8fc7-47b2728e7edc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4335a5e03e34b658d43c847bf412026'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:06.856071 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3958/8004] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:06 2022] DELETE /identity/v3/users/d4335a5e03e34b658d43c847bf412026 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:07.146496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:07.148068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:07.164403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8519541-7811-47d8-b016-2a77d5351fb6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:07.230679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ca03c-271c-4c16-8871-7f6013af940b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:07.231344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ca03c-271c-4c16-8871-7f6013af940b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:07.231597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ca03c-271c-4c16-8871-7f6013af940b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:07.232045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ca03c-271c-4c16-8871-7f6013af940b admin admin] PATH_INFO: `/v3/projects/d78bf3e24d4646f486057715989a3295` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:07.233256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ca03c-271c-4c16-8871-7f6013af940b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d78bf3e24d4646f486057715989a3295)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:07.235236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ca03c-271c-4c16-8871-7f6013af940b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:07.258019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:07.285049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d66ca03c-271c-4c16-8871-7f6013af940b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd78bf3e24d4646f486057715989a3295', 'request_id': 'req-d66ca03c-271c-4c16-8871-7f6013af940b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:07.293795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d66ca03c-271c-4c16-8871-7f6013af940b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd78bf3e24d4646f486057715989a3295', 'request_id': 'req-d66ca03c-271c-4c16-8871-7f6013af940b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:07.299984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d66ca03c-271c-4c16-8871-7f6013af940b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd78bf3e24d4646f486057715989a3295', 'request_id': 'req-d66ca03c-271c-4c16-8871-7f6013af940b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:07.311920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d66ca03c-271c-4c16-8871-7f6013af940b admin admin] The token cache is being invalidate because project d78bf3e24d4646f486057715989a3295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:07.312207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d66ca03c-271c-4c16-8871-7f6013af940b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:07.312942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-d66ca03c-271c-4c16-8871-7f6013af940b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:07.314590 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4047/8005] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:07 2022] DELETE /identity/v3/projects/d78bf3e24d4646f486057715989a3295 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:07.321177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cb000-78dc-4789-bc57-c99ee26ace68 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:07.321456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cb000-78dc-4789-bc57-c99ee26ace68 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:07.321720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cb000-78dc-4789-bc57-c99ee26ace68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:07.342915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c96cb000-78dc-4789-bc57-c99ee26ace68 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:07.344137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c96cb000-78dc-4789-bc57-c99ee26ace68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:07.355155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:07.410411 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3959/8006] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:07.419151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1d31d-45eb-46d1-8cb0-8d40f63b65ef None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:07.419409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1d31d-45eb-46d1-8cb0-8d40f63b65ef None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:07.419703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1d31d-45eb-46d1-8cb0-8d40f63b65ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:07.440028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8ee1d31d-45eb-46d1-8cb0-8d40f63b65ef None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:07.441272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8ee1d31d-45eb-46d1-8cb0-8d40f63b65ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:07.450340 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4048/8007] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:07.457081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cb000-78dc-4789-bc57-c99ee26ace68 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:07.467922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cb000-78dc-4789-bc57-c99ee26ace68 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:07.474131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181cccd-40c4-4b33-a7a4-789b2ef485cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:07.474716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181cccd-40c4-4b33-a7a4-789b2ef485cf admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:07.474983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181cccd-40c4-4b33-a7a4-789b2ef485cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:07.475244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181cccd-40c4-4b33-a7a4-789b2ef485cf admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:07.475875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181cccd-40c4-4b33-a7a4-789b2ef485cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:27:07.476176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181cccd-40c4-4b33-a7a4-789b2ef485cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:07.477879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181cccd-40c4-4b33-a7a4-789b2ef485cf admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:07.490331 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3960/8008] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:27:07.494954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1d31d-45eb-46d1-8cb0-8d40f63b65ef None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:07.504630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1d31d-45eb-46d1-8cb0-8d40f63b65ef None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:07.509413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340586cc-bdc1-4dbf-92be-50a559a34705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:07.510033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340586cc-bdc1-4dbf-92be-50a559a34705 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:07.510323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340586cc-bdc1-4dbf-92be-50a559a34705 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:07.510568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340586cc-bdc1-4dbf-92be-50a559a34705 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:07.511198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340586cc-bdc1-4dbf-92be-50a559a34705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:07.513140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340586cc-bdc1-4dbf-92be-50a559a34705 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:07.542414 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4049/8009] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:07 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:07.547502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181cccd-40c4-4b33-a7a4-789b2ef485cf admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:07.563077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181cccd-40c4-4b33-a7a4-789b2ef485cf admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:07.625104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc627b-6e6c-4acd-aff1-8264ba0687ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:07.625721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc627b-6e6c-4acd-aff1-8264ba0687ec admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:07.625992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc627b-6e6c-4acd-aff1-8264ba0687ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:07.626256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc627b-6e6c-4acd-aff1-8264ba0687ec admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:07.626876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc627b-6e6c-4acd-aff1-8264ba0687ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:07.628842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc627b-6e6c-4acd-aff1-8264ba0687ec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:07.655859 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3961/8010] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:07 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:07.657202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340586cc-bdc1-4dbf-92be-50a559a34705 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:07.668701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340586cc-bdc1-4dbf-92be-50a559a34705 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:07.673989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf20293-7723-4c6b-adbb-3e83fd151cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:07.674724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf20293-7723-4c6b-adbb-3e83fd151cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:07.675004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf20293-7723-4c6b-adbb-3e83fd151cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:07.675336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf20293-7723-4c6b-adbb-3e83fd151cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:07.676021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf20293-7723-4c6b-adbb-3e83fd151cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:07.677833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf20293-7723-4c6b-adbb-3e83fd151cf1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:07.689632 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4050/8011] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:07.694748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc627b-6e6c-4acd-aff1-8264ba0687ec admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:07.705451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc627b-6e6c-4acd-aff1-8264ba0687ec admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:07.710948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabbad0-6eef-4d03-a4cc-0d0a6be79aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:07.711547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabbad0-6eef-4d03-a4cc-0d0a6be79aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:07.711823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabbad0-6eef-4d03-a4cc-0d0a6be79aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:07.712091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabbad0-6eef-4d03-a4cc-0d0a6be79aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:07.712774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eabbad0-6eef-4d03-a4cc-0d0a6be79aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:07.714406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eabbad0-6eef-4d03-a4cc-0d0a6be79aa3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:07.725452 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3962/8012] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:07.729494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf20293-7723-4c6b-adbb-3e83fd151cf1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:07.740489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf20293-7723-4c6b-adbb-3e83fd151cf1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:07.745980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f36bf-bb47-43fb-a163-e2075e669ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:07.746657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f36bf-bb47-43fb-a163-e2075e669ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:07.746930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f36bf-bb47-43fb-a163-e2075e669ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:07.747200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f36bf-bb47-43fb-a163-e2075e669ed4 admin admin] PATH_INFO: `/v3/projects/17d1bec118b3482eaa28a26a2ce8cc63/users/af2a27a865464dc489689ce7815cd4d2/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:07.762504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f36bf-bb47-43fb-a163-e2075e669ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d1bec118b3482eaa28a26a2ce8cc63, user_id=af2a27a865464dc489689ce7815cd4d2, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:07.765274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f36bf-bb47-43fb-a163-e2075e669ed4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:07.781846 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4051/8013] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:07 2022] PUT /identity/v3/projects/17d1bec118b3482eaa28a26a2ce8cc63/users/af2a27a865464dc489689ce7815cd4d2/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:07.793673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23752d5-589d-48b7-8e0f-b6fa4b98bcb9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:07.794210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23752d5-589d-48b7-8e0f-b6fa4b98bcb9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:07.794210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23752d5-589d-48b7-8e0f-b6fa4b98bcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:07.824611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e23752d5-589d-48b7-8e0f-b6fa4b98bcb9 None None] MFA Rules not processed for user `af2a27a865464dc489689ce7815cd4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:07.825926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e23752d5-589d-48b7-8e0f-b6fa4b98bcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:07.898376 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3963/8014] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:07.910068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f36bf-bb47-43fb-a163-e2075e669ed4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:07.912050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5f2f36bf-bb47-43fb-a163-e2075e669ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:07.987414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f36bf-bb47-43fb-a163-e2075e669ed4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:07.991427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a348e-3e9d-4144-acd2-28c6cb16f9b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:07.992038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a348e-3e9d-4144-acd2-28c6cb16f9b0 service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:07.992313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a348e-3e9d-4144-acd2-28c6cb16f9b0 service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:07.992624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a348e-3e9d-4144-acd2-28c6cb16f9b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:08.001328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a348e-3e9d-4144-acd2-28c6cb16f9b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:08.002730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a348e-3e9d-4144-acd2-28c6cb16f9b0 service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:08.012197 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4052/8015] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:08.110290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23752d5-589d-48b7-8e0f-b6fa4b98bcb9 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:08.123965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:08.181126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23752d5-589d-48b7-8e0f-b6fa4b98bcb9 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:08.186083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:08.186730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:08.187001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:08.187262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] PATH_INFO: `/v3/users/af2a27a865464dc489689ce7815cd4d2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:08.188452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=af2a27a865464dc489689ce7815cd4d2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:08.190375 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:08.244903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af2a27a865464dc489689ce7815cd4d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:08.256079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af2a27a865464dc489689ce7815cd4d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:08.269287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af2a27a865464dc489689ce7815cd4d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:08.269990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af2a27a865464dc489689ce7815cd4d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:08.270599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af2a27a865464dc489689ce7815cd4d2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:08.283456 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3964/8016] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:08 2022] DELETE /identity/v3/users/af2a27a865464dc489689ce7815cd4d2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:08.288796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0df8a-ac73-40dc-82da-c4408ba10856 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:08.289341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0df8a-ac73-40dc-82da-c4408ba10856 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:08.289341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0df8a-ac73-40dc-82da-c4408ba10856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:08.311040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9ee0df8a-ac73-40dc-82da-c4408ba10856 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:08.312195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9ee0df8a-ac73-40dc-82da-c4408ba10856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:08.363597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:08.383753 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4053/8017] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:08.393708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:08.395414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:08.467338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c2150-2ded-4bd1-8db0-9197126cad4f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:08.471245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327dfacc-f64f-4494-b2a1-151c41880cf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:08.471853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327dfacc-f64f-4494-b2a1-151c41880cf7 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:08.472127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327dfacc-f64f-4494-b2a1-151c41880cf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:08.472400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327dfacc-f64f-4494-b2a1-151c41880cf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:08.482236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327dfacc-f64f-4494-b2a1-151c41880cf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:08.483701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327dfacc-f64f-4494-b2a1-151c41880cf7 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:08.499896 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3965/8018] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:27:08.785256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0df8a-ac73-40dc-82da-c4408ba10856 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:08.787263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9ee0df8a-ac73-40dc-82da-c4408ba10856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:08.814665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0df8a-ac73-40dc-82da-c4408ba10856 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:08.834598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e94fec-fad2-475d-a023-3ec367a9091b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:08.835434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e94fec-fad2-475d-a023-3ec367a9091b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:08.835923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e94fec-fad2-475d-a023-3ec367a9091b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:08.836309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e94fec-fad2-475d-a023-3ec367a9091b admin admin] PATH_INFO: `/v3/projects/17d1bec118b3482eaa28a26a2ce8cc63` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:08.837609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e94fec-fad2-475d-a023-3ec367a9091b admin admin] RBAC: Authorizing `identity:delete_project(project_id=17d1bec118b3482eaa28a26a2ce8cc63)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:08.839661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e94fec-fad2-475d-a023-3ec367a9091b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:08.939932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49e94fec-fad2-475d-a023-3ec367a9091b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17d1bec118b3482eaa28a26a2ce8cc63', 'request_id': 'req-49e94fec-fad2-475d-a023-3ec367a9091b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:08.960244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49e94fec-fad2-475d-a023-3ec367a9091b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17d1bec118b3482eaa28a26a2ce8cc63', 'request_id': 'req-49e94fec-fad2-475d-a023-3ec367a9091b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:08.969434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49e94fec-fad2-475d-a023-3ec367a9091b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17d1bec118b3482eaa28a26a2ce8cc63', 'request_id': 'req-49e94fec-fad2-475d-a023-3ec367a9091b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:08.983800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49e94fec-fad2-475d-a023-3ec367a9091b admin admin] The token cache is being invalidate because project 17d1bec118b3482eaa28a26a2ce8cc63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:08.984087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49e94fec-fad2-475d-a023-3ec367a9091b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:08.984772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-49e94fec-fad2-475d-a023-3ec367a9091b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:08.986511 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4054/8019] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:08 2022] DELETE /identity/v3/projects/17d1bec118b3482eaa28a26a2ce8cc63 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:08.992646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d76e75-433c-44cc-abfd-57e5de837825 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:08.993128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d76e75-433c-44cc-abfd-57e5de837825 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:08.993226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d76e75-433c-44cc-abfd-57e5de837825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:09.023214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-37d76e75-433c-44cc-abfd-57e5de837825 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:09.025302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-37d76e75-433c-44cc-abfd-57e5de837825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:09.032131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83617c0-e508-483f-a190-40e9c371ced8 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:09.032322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83617c0-e508-483f-a190-40e9c371ced8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:09.032695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83617c0-e508-483f-a190-40e9c371ced8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:09.065914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c83617c0-e508-483f-a190-40e9c371ced8 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:09.067699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c83617c0-e508-483f-a190-40e9c371ced8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:09.068573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:09.121382 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3966/8020] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:09.130607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6d7e9-21e0-493b-b124-71a7f04abd99 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:09.130890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6d7e9-21e0-493b-b124-71a7f04abd99 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:09.131167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6d7e9-21e0-493b-b124-71a7f04abd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:09.152154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-99b6d7e9-21e0-493b-b124-71a7f04abd99 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:09.153409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-99b6d7e9-21e0-493b-b124-71a7f04abd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:09.167356 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3967/8021] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:09.168192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6d7e9-21e0-493b-b124-71a7f04abd99 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:09.185685 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4055/8022] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:27:09.186214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6d7e9-21e0-493b-b124-71a7f04abd99 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:09.193143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd41af6-d68a-4014-b0f8-620c36a5ea7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:09.193755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd41af6-d68a-4014-b0f8-620c36a5ea7c admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:09.194480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd41af6-d68a-4014-b0f8-620c36a5ea7c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:09.194480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd41af6-d68a-4014-b0f8-620c36a5ea7c admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:09.195716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd41af6-d68a-4014-b0f8-620c36a5ea7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:27:09.196013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd41af6-d68a-4014-b0f8-620c36a5ea7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:09.197639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd41af6-d68a-4014-b0f8-620c36a5ea7c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:09.199807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83617c0-e508-483f-a190-40e9c371ced8 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:09.201240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c83617c0-e508-483f-a190-40e9c371ced8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:09.215974 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3968/8023] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:27:09.217732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd41af6-d68a-4014-b0f8-620c36a5ea7c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:09.232894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd41af6-d68a-4014-b0f8-620c36a5ea7c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:09.240246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8361b-0423-498a-9df4-f8ccc86585ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:09.240892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8361b-0423-498a-9df4-f8ccc86585ea admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:09.241198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8361b-0423-498a-9df4-f8ccc86585ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:09.241469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8361b-0423-498a-9df4-f8ccc86585ea admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:09.242117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8361b-0423-498a-9df4-f8ccc86585ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:09.244261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8361b-0423-498a-9df4-f8ccc86585ea admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:09.273358 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3969/8024] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:09 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:27:09.277465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8361b-0423-498a-9df4-f8ccc86585ea admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:09.284810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83617c0-e508-483f-a190-40e9c371ced8 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:09.297526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:09.340602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8361b-0423-498a-9df4-f8ccc86585ea admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:09.345470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4c755-fb0d-4c52-ba21-a1233df8beba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:09.346147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c755-fb0d-4c52-ba21-a1233df8beba admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:09.346538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c755-fb0d-4c52-ba21-a1233df8beba admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:09.346748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c755-fb0d-4c52-ba21-a1233df8beba admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:09.347368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4c755-fb0d-4c52-ba21-a1233df8beba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:09.349824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4c755-fb0d-4c52-ba21-a1233df8beba admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:09.350877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3606b-894c-422d-93d8-8797a289dcda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:09.351485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3606b-894c-422d-93d8-8797a289dcda service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:09.351756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3606b-894c-422d-93d8-8797a289dcda service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:09.352019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3606b-894c-422d-93d8-8797a289dcda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:09.376070 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3970/8025] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:09 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:27:09.377402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4c755-fb0d-4c52-ba21-a1233df8beba admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:09.387496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4c755-fb0d-4c52-ba21-a1233df8beba admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:09.392174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9056e32-6e7c-4b0c-a314-c0c599ad8ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:09.392812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9056e32-6e7c-4b0c-a314-c0c599ad8ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:09.393082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9056e32-6e7c-4b0c-a314-c0c599ad8ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:09.393346 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9056e32-6e7c-4b0c-a314-c0c599ad8ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:09.393981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9056e32-6e7c-4b0c-a314-c0c599ad8ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:09.395671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9056e32-6e7c-4b0c-a314-c0c599ad8ed2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:09.405829 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3971/8026] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:09.409891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9056e32-6e7c-4b0c-a314-c0c599ad8ed2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:09.410147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3606b-894c-422d-93d8-8797a289dcda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:09.411967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3606b-894c-422d-93d8-8797a289dcda service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:09.420629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9056e32-6e7c-4b0c-a314-c0c599ad8ed2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:09.420818 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4056/8027] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:27:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:27:09.425341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10ffaa-2a42-4d1f-a49c-a5246f453e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:09.425824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10ffaa-2a42-4d1f-a49c-a5246f453e9e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:09.426096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10ffaa-2a42-4d1f-a49c-a5246f453e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:09.426443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10ffaa-2a42-4d1f-a49c-a5246f453e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:09.427066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10ffaa-2a42-4d1f-a49c-a5246f453e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:09.428710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10ffaa-2a42-4d1f-a49c-a5246f453e9e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:09.441015 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3972/8028] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:27:09.445794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3606b-894c-422d-93d8-8797a289dcda service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:09.458104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3606b-894c-422d-93d8-8797a289dcda service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:09.464354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d96a7-b29d-4df0-8aac-58593fc00d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:09.465272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d96a7-b29d-4df0-8aac-58593fc00d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:09.465701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d96a7-b29d-4df0-8aac-58593fc00d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:09.469002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d96a7-b29d-4df0-8aac-58593fc00d89 admin admin] PATH_INFO: `/v3/projects/c671ce9b2632444788790480d8f26d82/users/2cac793b251c4082918d7a9f0fdcdcfc/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:09.492206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d96a7-b29d-4df0-8aac-58593fc00d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c671ce9b2632444788790480d8f26d82, user_id=2cac793b251c4082918d7a9f0fdcdcfc, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:09.495628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d96a7-b29d-4df0-8aac-58593fc00d89 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:09.518101 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4057/8029] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:09 2022] PUT /identity/v3/projects/c671ce9b2632444788790480d8f26d82/users/2cac793b251c4082918d7a9f0fdcdcfc/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:09.528851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28345873-e803-4b6a-9382-55beb0331669 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:09.529253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28345873-e803-4b6a-9382-55beb0331669 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:09.529546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28345873-e803-4b6a-9382-55beb0331669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:09.563747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-28345873-e803-4b6a-9382-55beb0331669 None None] MFA Rules not processed for user `2cac793b251c4082918d7a9f0fdcdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:09.565054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-28345873-e803-4b6a-9382-55beb0331669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:09.620197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:09.644183 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3973/8030] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:27:09.655686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d96a7-b29d-4df0-8aac-58593fc00d89 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:09.657382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a17d96a7-b29d-4df0-8aac-58593fc00d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:09.737170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d96a7-b29d-4df0-8aac-58593fc00d89 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:09.742149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b1182-e47b-4218-a6ee-88b3e44ec31f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:09.742850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b1182-e47b-4218-a6ee-88b3e44ec31f service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:09.743132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b1182-e47b-4218-a6ee-88b3e44ec31f service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:09.743397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b1182-e47b-4218-a6ee-88b3e44ec31f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:09.753332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b1182-e47b-4218-a6ee-88b3e44ec31f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:09.755793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b1182-e47b-4218-a6ee-88b3e44ec31f service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:09.766143 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4058/8031] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:10.008091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28345873-e803-4b6a-9382-55beb0331669 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:10.079466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28345873-e803-4b6a-9382-55beb0331669 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:10.084677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:10.085387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:10.085842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:10.086107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] PATH_INFO: `/v3/users/2cac793b251c4082918d7a9f0fdcdcfc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:10.087234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cac793b251c4082918d7a9f0fdcdcfc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:10.089231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:10.139923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cac793b251c4082918d7a9f0fdcdcfc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:10.149970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cac793b251c4082918d7a9f0fdcdcfc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:10.163611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cac793b251c4082918d7a9f0fdcdcfc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:10.164421 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cac793b251c4082918d7a9f0fdcdcfc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:10.165344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cac793b251c4082918d7a9f0fdcdcfc'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:10.180454 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3974/8032] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:10 2022] DELETE /identity/v3/users/2cac793b251c4082918d7a9f0fdcdcfc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:10.185030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221138f6-3fc4-438a-bf31-a2fd4e5bd0ad None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:10.185562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221138f6-3fc4-438a-bf31-a2fd4e5bd0ad None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:10.186075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221138f6-3fc4-438a-bf31-a2fd4e5bd0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:10.211616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-221138f6-3fc4-438a-bf31-a2fd4e5bd0ad None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:10.211988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-221138f6-3fc4-438a-bf31-a2fd4e5bd0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:10.268961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:10.300549 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4059/8033] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:10.313354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:10.314883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:10.402055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfbf7c-1a90-4a6e-b2bc-51c892f9c9a8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:10.407574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2b4cc-17db-42fb-a577-ec1bf26126c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:10.408301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2b4cc-17db-42fb-a577-ec1bf26126c1 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:10.408769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2b4cc-17db-42fb-a577-ec1bf26126c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:10.409122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2b4cc-17db-42fb-a577-ec1bf26126c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:10.424631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2b4cc-17db-42fb-a577-ec1bf26126c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:10.426115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2b4cc-17db-42fb-a577-ec1bf26126c1 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:10.439700 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3975/8034] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:27:10.855114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221138f6-3fc4-438a-bf31-a2fd4e5bd0ad None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:10.856678 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-221138f6-3fc4-438a-bf31-a2fd4e5bd0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:10.882877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221138f6-3fc4-438a-bf31-a2fd4e5bd0ad None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:10.888733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc75160-da12-4a4c-acbb-1ca6fe11241b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:10.889369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc75160-da12-4a4c-acbb-1ca6fe11241b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:10.889661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc75160-da12-4a4c-acbb-1ca6fe11241b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:10.889882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc75160-da12-4a4c-acbb-1ca6fe11241b admin admin] PATH_INFO: `/v3/projects/c671ce9b2632444788790480d8f26d82` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:10.891002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc75160-da12-4a4c-acbb-1ca6fe11241b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c671ce9b2632444788790480d8f26d82)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:10.892941 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc75160-da12-4a4c-acbb-1ca6fe11241b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:10.970526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-adc75160-da12-4a4c-acbb-1ca6fe11241b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c671ce9b2632444788790480d8f26d82', 'request_id': 'req-adc75160-da12-4a4c-acbb-1ca6fe11241b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:10.978372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-adc75160-da12-4a4c-acbb-1ca6fe11241b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c671ce9b2632444788790480d8f26d82', 'request_id': 'req-adc75160-da12-4a4c-acbb-1ca6fe11241b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:10.984245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-adc75160-da12-4a4c-acbb-1ca6fe11241b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c671ce9b2632444788790480d8f26d82', 'request_id': 'req-adc75160-da12-4a4c-acbb-1ca6fe11241b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:10.998816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-adc75160-da12-4a4c-acbb-1ca6fe11241b admin admin] The token cache is being invalidate because project c671ce9b2632444788790480d8f26d82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:10.999049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-adc75160-da12-4a4c-acbb-1ca6fe11241b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:10.999653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-adc75160-da12-4a4c-acbb-1ca6fe11241b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:11.001213 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4060/8035] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:10 2022] DELETE /identity/v3/projects/c671ce9b2632444788790480d8f26d82 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:11.009527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6482e-bc9e-459b-b39d-c4e156f6663e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:11.009527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6482e-bc9e-459b-b39d-c4e156f6663e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:11.009527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6482e-bc9e-459b-b39d-c4e156f6663e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:11.030682 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fdc6482e-bc9e-459b-b39d-c4e156f6663e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:11.031896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fdc6482e-bc9e-459b-b39d-c4e156f6663e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:11.038311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:11.136905 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3976/8036] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:11.140129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554e980-3f20-49fa-b061-1f0a3e11cdba None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:11.140129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554e980-3f20-49fa-b061-1f0a3e11cdba None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:11.140396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554e980-3f20-49fa-b061-1f0a3e11cdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:11.166100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d554e980-3f20-49fa-b061-1f0a3e11cdba None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:11.167349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d554e980-3f20-49fa-b061-1f0a3e11cdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:11.179372 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4061/8037] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:11.183747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6482e-bc9e-459b-b39d-c4e156f6663e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:11.207803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6482e-bc9e-459b-b39d-c4e156f6663e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:11.214177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8db51d-4e30-41b3-903f-c13caa3faeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:11.214790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8db51d-4e30-41b3-903f-c13caa3faeff admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:11.215062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8db51d-4e30-41b3-903f-c13caa3faeff admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:11.215322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8db51d-4e30-41b3-903f-c13caa3faeff admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:11.215982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8db51d-4e30-41b3-903f-c13caa3faeff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:27:11.216325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8db51d-4e30-41b3-903f-c13caa3faeff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:11.218067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8db51d-4e30-41b3-903f-c13caa3faeff admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:11.236018 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3977/8038] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:27:11.240931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554e980-3f20-49fa-b061-1f0a3e11cdba None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:11.251732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554e980-3f20-49fa-b061-1f0a3e11cdba None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:11.257183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1864db8-4585-4d58-aeea-be4cefea547e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:11.257809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1864db8-4585-4d58-aeea-be4cefea547e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:11.258071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1864db8-4585-4d58-aeea-be4cefea547e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:11.258338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1864db8-4585-4d58-aeea-be4cefea547e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:11.258955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1864db8-4585-4d58-aeea-be4cefea547e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:11.260885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1864db8-4585-4d58-aeea-be4cefea547e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:11.297588 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4062/8039] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:11 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:11.301817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8db51d-4e30-41b3-903f-c13caa3faeff admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:11.315387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8db51d-4e30-41b3-903f-c13caa3faeff admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:11.380149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23a376-da17-4d31-934f-24bb47ae2090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:11.380840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23a376-da17-4d31-934f-24bb47ae2090 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:11.381114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23a376-da17-4d31-934f-24bb47ae2090 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:11.381391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23a376-da17-4d31-934f-24bb47ae2090 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:11.382018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23a376-da17-4d31-934f-24bb47ae2090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:11.384099 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23a376-da17-4d31-934f-24bb47ae2090 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:11.412865 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3978/8040] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:11 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:11.413694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1864db8-4585-4d58-aeea-be4cefea547e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:11.425991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1864db8-4585-4d58-aeea-be4cefea547e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:11.431241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c5edd-eb69-4000-a471-bc507517285b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:11.432228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c5edd-eb69-4000-a471-bc507517285b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:11.432738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c5edd-eb69-4000-a471-bc507517285b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:11.433380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c5edd-eb69-4000-a471-bc507517285b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:11.435246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c5edd-eb69-4000-a471-bc507517285b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:11.441282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c5edd-eb69-4000-a471-bc507517285b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:11.464124 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4063/8041] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:11.465978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23a376-da17-4d31-934f-24bb47ae2090 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:11.480003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23a376-da17-4d31-934f-24bb47ae2090 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:11.491374 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fca9d9-526a-4061-9f69-5dba812c81f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:11.492488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fca9d9-526a-4061-9f69-5dba812c81f0 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:11.492488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fca9d9-526a-4061-9f69-5dba812c81f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:11.492897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fca9d9-526a-4061-9f69-5dba812c81f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:11.493658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fca9d9-526a-4061-9f69-5dba812c81f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:11.495355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fca9d9-526a-4061-9f69-5dba812c81f0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:11.508015 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3979/8042] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:11.512790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c5edd-eb69-4000-a471-bc507517285b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:11.530162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c5edd-eb69-4000-a471-bc507517285b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:11.538872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e7911-4cb2-4eae-9f8e-7cf7e10887d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:11.539591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e7911-4cb2-4eae-9f8e-7cf7e10887d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:11.539892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e7911-4cb2-4eae-9f8e-7cf7e10887d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:11.540161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e7911-4cb2-4eae-9f8e-7cf7e10887d7 admin admin] PATH_INFO: `/v3/projects/4db32c682bc64a4292e1bb7d704d492e/users/9feee30e8ccb44a8925e69b9e213555b/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:11.561812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e7911-4cb2-4eae-9f8e-7cf7e10887d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4db32c682bc64a4292e1bb7d704d492e, user_id=9feee30e8ccb44a8925e69b9e213555b, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:11.564539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e7911-4cb2-4eae-9f8e-7cf7e10887d7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:11.588967 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4064/8043] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:11 2022] PUT /identity/v3/projects/4db32c682bc64a4292e1bb7d704d492e/users/9feee30e8ccb44a8925e69b9e213555b/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:27:11.600037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5930d56-ca9e-4f1a-8889-1d18aa0f4127 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:11.600614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5930d56-ca9e-4f1a-8889-1d18aa0f4127 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:11.600926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5930d56-ca9e-4f1a-8889-1d18aa0f4127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:11.632022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e5930d56-ca9e-4f1a-8889-1d18aa0f4127 None None] MFA Rules not processed for user `9feee30e8ccb44a8925e69b9e213555b`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:11.633981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e5930d56-ca9e-4f1a-8889-1d18aa0f4127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:11.726069 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3980/8044] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:11.727488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e7911-4cb2-4eae-9f8e-7cf7e10887d7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:11.792793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:11.797854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e7911-4cb2-4eae-9f8e-7cf7e10887d7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:11.806153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe93ba-ec35-4ec2-9018-834665a44229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:11.806935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe93ba-ec35-4ec2-9018-834665a44229 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:11.807410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe93ba-ec35-4ec2-9018-834665a44229 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:11.808174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe93ba-ec35-4ec2-9018-834665a44229 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:11.809415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe93ba-ec35-4ec2-9018-834665a44229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:11.811562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe93ba-ec35-4ec2-9018-834665a44229 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:11.847878 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4065/8045] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:11 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:11.852438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5930d56-ca9e-4f1a-8889-1d18aa0f4127 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:11.863986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5930d56-ca9e-4f1a-8889-1d18aa0f4127 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:11.868571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:11.961987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ece9a-c2cd-4f26-aa70-76344d12381b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:11.962664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ece9a-c2cd-4f26-aa70-76344d12381b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:11.962947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ece9a-c2cd-4f26-aa70-76344d12381b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:11.963214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ece9a-c2cd-4f26-aa70-76344d12381b admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:11.963981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ece9a-c2cd-4f26-aa70-76344d12381b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:11.965939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ece9a-c2cd-4f26-aa70-76344d12381b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:12.005957 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3981/8046] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:11 2022] POST /identity/v3/users => generated 295 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:27:12.008620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe93ba-ec35-4ec2-9018-834665a44229 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:12.023208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe93ba-ec35-4ec2-9018-834665a44229 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:12.033202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fe972-a0e6-4bc5-a8e4-685231cabde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:12.034057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fe972-a0e6-4bc5-a8e4-685231cabde5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:12.034406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fe972-a0e6-4bc5-a8e4-685231cabde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:12.034750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fe972-a0e6-4bc5-a8e4-685231cabde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:12.035518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fe972-a0e6-4bc5-a8e4-685231cabde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:12.037852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fe972-a0e6-4bc5-a8e4-685231cabde5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:12.058752 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4066/8047] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:12.061944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ece9a-c2cd-4f26-aa70-76344d12381b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:12.072696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ece9a-c2cd-4f26-aa70-76344d12381b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:12.077699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260100f0-6e15-4761-98ea-3d2623a45739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:12.078379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260100f0-6e15-4761-98ea-3d2623a45739 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:12.078656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260100f0-6e15-4761-98ea-3d2623a45739 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:12.078921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260100f0-6e15-4761-98ea-3d2623a45739 admin admin] PATH_INFO: `/v3/projects/0819d94b09bf4a7aaa6b18aa7deca720/users/00c9e8cc80464edd8ff4bda7cbbfac22/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:12.096839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260100f0-6e15-4761-98ea-3d2623a45739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0819d94b09bf4a7aaa6b18aa7deca720, user_id=00c9e8cc80464edd8ff4bda7cbbfac22, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:12.099673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260100f0-6e15-4761-98ea-3d2623a45739 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:12.119077 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3982/8048] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:12 2022] PUT /identity/v3/projects/0819d94b09bf4a7aaa6b18aa7deca720/users/00c9e8cc80464edd8ff4bda7cbbfac22/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:12.123110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fe972-a0e6-4bc5-a8e4-685231cabde5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:12.140407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fe972-a0e6-4bc5-a8e4-685231cabde5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:12.220790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e3620-a294-49fc-bb6d-f80936b2bc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:12.221405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e3620-a294-49fc-bb6d-f80936b2bc3f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:12.221682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e3620-a294-49fc-bb6d-f80936b2bc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:12.221951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e3620-a294-49fc-bb6d-f80936b2bc3f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:12.222588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e3620-a294-49fc-bb6d-f80936b2bc3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:12.224410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e3620-a294-49fc-bb6d-f80936b2bc3f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:12.241434 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4067/8049] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:12.246073 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260100f0-6e15-4761-98ea-3d2623a45739 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:12.258146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260100f0-6e15-4761-98ea-3d2623a45739 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:12.271470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3166a88-41d5-45c7-87b4-d3eef31272d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:12.272273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3166a88-41d5-45c7-87b4-d3eef31272d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:12.272884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3166a88-41d5-45c7-87b4-d3eef31272d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:12.273157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3166a88-41d5-45c7-87b4-d3eef31272d2 admin admin] PATH_INFO: `/v3/projects/0819d94b09bf4a7aaa6b18aa7deca720/users/00c9e8cc80464edd8ff4bda7cbbfac22/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:12.275508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3166a88-41d5-45c7-87b4-d3eef31272d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0819d94b09bf4a7aaa6b18aa7deca720, user_id=00c9e8cc80464edd8ff4bda7cbbfac22, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:12.278249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3166a88-41d5-45c7-87b4-d3eef31272d2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:12.295577 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3983/8050] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:12 2022] PUT /identity/v3/projects/0819d94b09bf4a7aaa6b18aa7deca720/users/00c9e8cc80464edd8ff4bda7cbbfac22/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:12.306383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647b97f-45f4-4515-aa17-00c9b405394a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:12.306654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647b97f-45f4-4515-aa17-00c9b405394a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:12.306918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647b97f-45f4-4515-aa17-00c9b405394a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:12.341541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7647b97f-45f4-4515-aa17-00c9b405394a None None] MFA Rules not processed for user `00c9e8cc80464edd8ff4bda7cbbfac22`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:12.342656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7647b97f-45f4-4515-aa17-00c9b405394a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:12.447735 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4068/8051] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:12.454142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3166a88-41d5-45c7-87b4-d3eef31272d2 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:12.455957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f3166a88-41d5-45c7-87b4-d3eef31272d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:12.527273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3166a88-41d5-45c7-87b4-d3eef31272d2 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:12.531492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1032d3c-987e-4137-97f9-d13b0d1bb93c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:12.532102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1032d3c-987e-4137-97f9-d13b0d1bb93c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:12.532392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1032d3c-987e-4137-97f9-d13b0d1bb93c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:12.532681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1032d3c-987e-4137-97f9-d13b0d1bb93c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:12.544898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1032d3c-987e-4137-97f9-d13b0d1bb93c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:12.546384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1032d3c-987e-4137-97f9-d13b0d1bb93c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:12.564691 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3984/8052] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:14.655325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647b97f-45f4-4515-aa17-00c9b405394a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:14.725802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:14.738956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647b97f-45f4-4515-aa17-00c9b405394a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:14.745599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:14.746319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:14.746658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:14.746989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] PATH_INFO: `/v3/users/9feee30e8ccb44a8925e69b9e213555b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:14.748107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9feee30e8ccb44a8925e69b9e213555b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:14.750307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:14.814278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9feee30e8ccb44a8925e69b9e213555b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:14.826031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9feee30e8ccb44a8925e69b9e213555b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:14.835242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9feee30e8ccb44a8925e69b9e213555b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:14.849854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9feee30e8ccb44a8925e69b9e213555b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:14.850624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9feee30e8ccb44a8925e69b9e213555b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:14.861384 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4069/8053] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:14 2022] DELETE /identity/v3/users/9feee30e8ccb44a8925e69b9e213555b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:14.866127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a66c9b-cce7-42af-8507-9ddf349886b9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:14.866650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a66c9b-cce7-42af-8507-9ddf349886b9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:14.866650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a66c9b-cce7-42af-8507-9ddf349886b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:14.892870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-33a66c9b-cce7-42af-8507-9ddf349886b9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:14.894079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-33a66c9b-cce7-42af-8507-9ddf349886b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:14.976635 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3985/8054] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:14.987142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:14.988870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:15.058608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b6d98-57e4-42e2-9baa-70219234edc1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:15.072317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe408f8-0351-4f3c-b855-544253c3bfac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:15.072981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe408f8-0351-4f3c-b855-544253c3bfac service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:15.073243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe408f8-0351-4f3c-b855-544253c3bfac service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:15.073497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe408f8-0351-4f3c-b855-544253c3bfac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:15.084635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe408f8-0351-4f3c-b855-544253c3bfac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:15.086077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe408f8-0351-4f3c-b855-544253c3bfac service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:15.097434 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4070/8055] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:27:15.439399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a66c9b-cce7-42af-8507-9ddf349886b9 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:15.442246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-33a66c9b-cce7-42af-8507-9ddf349886b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:15.450623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:15.472540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a66c9b-cce7-42af-8507-9ddf349886b9 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:15.479877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:15.480644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:15.480985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:15.481314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] PATH_INFO: `/v3/projects/4db32c682bc64a4292e1bb7d704d492e` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:15.482525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4db32c682bc64a4292e1bb7d704d492e)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:15.484575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:15.543455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4db32c682bc64a4292e1bb7d704d492e', 'request_id': 'req-ead257c6-d06a-41bb-9e17-5c8201d61308'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:15.550212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4db32c682bc64a4292e1bb7d704d492e', 'request_id': 'req-ead257c6-d06a-41bb-9e17-5c8201d61308'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:15.560109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4db32c682bc64a4292e1bb7d704d492e', 'request_id': 'req-ead257c6-d06a-41bb-9e17-5c8201d61308'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:15.573224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] The token cache is being invalidate because project 4db32c682bc64a4292e1bb7d704d492e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:15.573462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:15.574364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:15.576147 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3986/8056] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:15 2022] DELETE /identity/v3/projects/4db32c682bc64a4292e1bb7d704d492e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:15.580413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe408f8-0351-4f3c-b855-544253c3bfac service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:15.591284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe408f8-0351-4f3c-b855-544253c3bfac service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:15.681396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c3fc1-5fe5-418f-a50c-6344519d911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:15.682199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3fc1-5fe5-418f-a50c-6344519d911e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:15.682567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3fc1-5fe5-418f-a50c-6344519d911e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:15.682919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3fc1-5fe5-418f-a50c-6344519d911e admin admin] PATH_INFO: `/v3/users/00c9e8cc80464edd8ff4bda7cbbfac22` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:15.686653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c3fc1-5fe5-418f-a50c-6344519d911e admin admin] RBAC: Authorizing `identity:delete_user(user_id=00c9e8cc80464edd8ff4bda7cbbfac22)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:15.688831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c3fc1-5fe5-418f-a50c-6344519d911e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:15.713036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:15.761474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b34c3fc1-5fe5-418f-a50c-6344519d911e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00c9e8cc80464edd8ff4bda7cbbfac22'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:15.772336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b34c3fc1-5fe5-418f-a50c-6344519d911e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00c9e8cc80464edd8ff4bda7cbbfac22'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:15.781094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b34c3fc1-5fe5-418f-a50c-6344519d911e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00c9e8cc80464edd8ff4bda7cbbfac22'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:15.798762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b34c3fc1-5fe5-418f-a50c-6344519d911e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00c9e8cc80464edd8ff4bda7cbbfac22'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:15.799861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b34c3fc1-5fe5-418f-a50c-6344519d911e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00c9e8cc80464edd8ff4bda7cbbfac22'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:15.811858 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4071/8057] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:15 2022] DELETE /identity/v3/users/00c9e8cc80464edd8ff4bda7cbbfac22 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:16.217706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:16.219310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:16.246032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead257c6-d06a-41bb-9e17-5c8201d61308 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:16.337323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:16.338473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:16.338922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:16.339499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2 admin admin] PATH_INFO: `/v3/projects/0819d94b09bf4a7aaa6b18aa7deca720` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:16.340956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0819d94b09bf4a7aaa6b18aa7deca720)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:16.343526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:16.415203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0819d94b09bf4a7aaa6b18aa7deca720', 'request_id': 'req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:16.424211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0819d94b09bf4a7aaa6b18aa7deca720', 'request_id': 'req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:16.434568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0819d94b09bf4a7aaa6b18aa7deca720', 'request_id': 'req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:16.461485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2 admin admin] The token cache is being invalidate because project 0819d94b09bf4a7aaa6b18aa7deca720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:16.461794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:16.462468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2fb4d65b-4915-44d9-abf7-26f2cfea1fd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:16.464295 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3987/8058] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:16 2022] DELETE /identity/v3/projects/0819d94b09bf4a7aaa6b18aa7deca720 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:16.469284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f06c01-3b78-49b6-8836-b4f772a42c55 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:16.469562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f06c01-3b78-49b6-8836-b4f772a42c55 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:16.469830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f06c01-3b78-49b6-8836-b4f772a42c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:16.492809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-26f06c01-3b78-49b6-8836-b4f772a42c55 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:16.494153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-26f06c01-3b78-49b6-8836-b4f772a42c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:16.593456 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4072/8059] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:27:16.596258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2f458-53c7-458f-a77e-ed523197cef1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:16.596845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2f458-53c7-458f-a77e-ed523197cef1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:16.597381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2f458-53c7-458f-a77e-ed523197cef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:16.619410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c3d2f458-53c7-458f-a77e-ed523197cef1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:16.621173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c3d2f458-53c7-458f-a77e-ed523197cef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:16.634630 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3988/8060] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:16.636330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f06c01-3b78-49b6-8836-b4f772a42c55 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:16.648299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f06c01-3b78-49b6-8836-b4f772a42c55 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:16.655911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9cd88-27b4-450f-9c37-59688e3d1422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:16.656635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9cd88-27b4-450f-9c37-59688e3d1422 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:16.656906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9cd88-27b4-450f-9c37-59688e3d1422 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:16.657172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9cd88-27b4-450f-9c37-59688e3d1422 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:16.657823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9cd88-27b4-450f-9c37-59688e3d1422 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:27:16.658120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9cd88-27b4-450f-9c37-59688e3d1422 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:16.659727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9cd88-27b4-450f-9c37-59688e3d1422 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:16.679340 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4073/8061] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:27:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:27:16.681190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2f458-53c7-458f-a77e-ed523197cef1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:16.696799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2f458-53c7-458f-a77e-ed523197cef1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:16.701976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fc0ac-d242-4b07-b1e8-059a787e1e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:16.702584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fc0ac-d242-4b07-b1e8-059a787e1e62 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:16.702858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fc0ac-d242-4b07-b1e8-059a787e1e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:16.703120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fc0ac-d242-4b07-b1e8-059a787e1e62 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:16.703841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5fc0ac-d242-4b07-b1e8-059a787e1e62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:16.705785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5fc0ac-d242-4b07-b1e8-059a787e1e62 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:16.724663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:16.764439 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3989/8062] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:16 2022] POST /identity/v3/projects => generated 355 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:16.766920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9cd88-27b4-450f-9c37-59688e3d1422 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:16.777477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9cd88-27b4-450f-9c37-59688e3d1422 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:16.851729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73662e94-0841-4462-ae4e-e82ea892fdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:16.852243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662e94-0841-4462-ae4e-e82ea892fdd4 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:16.852545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662e94-0841-4462-ae4e-e82ea892fdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:16.852809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662e94-0841-4462-ae4e-e82ea892fdd4 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:16.853420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73662e94-0841-4462-ae4e-e82ea892fdd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:16.855373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73662e94-0841-4462-ae4e-e82ea892fdd4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:16.887941 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4074/8063] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:16 2022] POST /identity/v3/users => generated 278 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:16.892937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fc0ac-d242-4b07-b1e8-059a787e1e62 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:16.905012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fc0ac-d242-4b07-b1e8-059a787e1e62 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:16.909884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05be389-c26f-4478-b2a8-53ce10e2b5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:16.910514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05be389-c26f-4478-b2a8-53ce10e2b5bc admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:16.910826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05be389-c26f-4478-b2a8-53ce10e2b5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:16.911487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05be389-c26f-4478-b2a8-53ce10e2b5bc admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:16.911659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05be389-c26f-4478-b2a8-53ce10e2b5bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:16.915309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05be389-c26f-4478-b2a8-53ce10e2b5bc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:16.926559 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3990/8064] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:16.930191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73662e94-0841-4462-ae4e-e82ea892fdd4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:16.940725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73662e94-0841-4462-ae4e-e82ea892fdd4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:16.950232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdeba6-fca3-4786-832b-ddd5607b1e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:16.950855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdeba6-fca3-4786-832b-ddd5607b1e1c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:16.951255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdeba6-fca3-4786-832b-ddd5607b1e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:16.951525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdeba6-fca3-4786-832b-ddd5607b1e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:16.952162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdeba6-fca3-4786-832b-ddd5607b1e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:16.953821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdeba6-fca3-4786-832b-ddd5607b1e1c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:16.959838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:16.978365 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4075/8065] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:16.978688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05be389-c26f-4478-b2a8-53ce10e2b5bc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:16.993992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05be389-c26f-4478-b2a8-53ce10e2b5bc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:17.001327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659d61c-20f7-4dd5-9b19-f9175d01a935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:17.002410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659d61c-20f7-4dd5-9b19-f9175d01a935 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:17.003191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659d61c-20f7-4dd5-9b19-f9175d01a935 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:17.003191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659d61c-20f7-4dd5-9b19-f9175d01a935 admin admin] PATH_INFO: `/v3/projects/8a9b3bd8d9d549f69331c8f345a5e96e/users/1424a55edc424313ac4909f4ea4ef087/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:17.023504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659d61c-20f7-4dd5-9b19-f9175d01a935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9b3bd8d9d549f69331c8f345a5e96e, user_id=1424a55edc424313ac4909f4ea4ef087, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:17.026898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659d61c-20f7-4dd5-9b19-f9175d01a935 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:17.046233 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3991/8066] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:16 2022] PUT /identity/v3/projects/8a9b3bd8d9d549f69331c8f345a5e96e/users/1424a55edc424313ac4909f4ea4ef087/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:27:17.058063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910295ac-84cd-4622-ad5f-dee7cb406ebb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:17.058311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910295ac-84cd-4622-ad5f-dee7cb406ebb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:17.058569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910295ac-84cd-4622-ad5f-dee7cb406ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:17.093346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-910295ac-84cd-4622-ad5f-dee7cb406ebb None None] MFA Rules not processed for user `1424a55edc424313ac4909f4ea4ef087`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:17.096075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-910295ac-84cd-4622-ad5f-dee7cb406ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:17.190263 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4076/8067] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:17.191503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659d61c-20f7-4dd5-9b19-f9175d01a935 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:17.269658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659d61c-20f7-4dd5-9b19-f9175d01a935 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:17.275713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686e18b-1cc2-4d26-a137-dc2621eec977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:17.276474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686e18b-1cc2-4d26-a137-dc2621eec977 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:17.276744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686e18b-1cc2-4d26-a137-dc2621eec977 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:17.277012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686e18b-1cc2-4d26-a137-dc2621eec977 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:17.277801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686e18b-1cc2-4d26-a137-dc2621eec977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:17.279755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686e18b-1cc2-4d26-a137-dc2621eec977 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:17.315253 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3992/8068] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:17 2022] POST /identity/v3/projects => generated 355 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:17.316759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910295ac-84cd-4622-ad5f-dee7cb406ebb None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:17.329520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910295ac-84cd-4622-ad5f-dee7cb406ebb None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:17.396756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a59b7-6c40-48dd-8775-ba784243e4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:17.397309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a59b7-6c40-48dd-8775-ba784243e4ef admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:17.397623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a59b7-6c40-48dd-8775-ba784243e4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:17.397893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a59b7-6c40-48dd-8775-ba784243e4ef admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:17.398518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a59b7-6c40-48dd-8775-ba784243e4ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:17.400478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a59b7-6c40-48dd-8775-ba784243e4ef admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:17.427367 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4077/8069] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:17 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:17.431633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686e18b-1cc2-4d26-a137-dc2621eec977 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:17.443357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686e18b-1cc2-4d26-a137-dc2621eec977 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:17.449119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d52ba-87b0-4e6d-b759-84f3bafee7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:17.449734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d52ba-87b0-4e6d-b759-84f3bafee7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:17.450001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d52ba-87b0-4e6d-b759-84f3bafee7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:17.450285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d52ba-87b0-4e6d-b759-84f3bafee7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:17.450910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d52ba-87b0-4e6d-b759-84f3bafee7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:17.452550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d52ba-87b0-4e6d-b759-84f3bafee7f6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:17.466863 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3993/8070] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:17.469530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a59b7-6c40-48dd-8775-ba784243e4ef admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:17.481030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a59b7-6c40-48dd-8775-ba784243e4ef admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:17.485930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1972369-5afd-421b-aaae-9812b40ce381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:17.486877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1972369-5afd-421b-aaae-9812b40ce381 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:17.487247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1972369-5afd-421b-aaae-9812b40ce381 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:17.487608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1972369-5afd-421b-aaae-9812b40ce381 admin admin] PATH_INFO: `/v3/projects/c508074a7ec8498a968a3c2aba21dd70/users/a49e396ccb93458dac5c895293aecdc0/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:17.504519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1972369-5afd-421b-aaae-9812b40ce381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c508074a7ec8498a968a3c2aba21dd70, user_id=a49e396ccb93458dac5c895293aecdc0, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:17.507451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1972369-5afd-421b-aaae-9812b40ce381 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:17.522520 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4078/8071] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:17 2022] PUT /identity/v3/projects/c508074a7ec8498a968a3c2aba21dd70/users/a49e396ccb93458dac5c895293aecdc0/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:27:17.526862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d52ba-87b0-4e6d-b759-84f3bafee7f6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:17.542131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d52ba-87b0-4e6d-b759-84f3bafee7f6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:17.612474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3e0f2-926c-43dc-9b40-d86a6cede1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:17.613085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3e0f2-926c-43dc-9b40-d86a6cede1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:17.613354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3e0f2-926c-43dc-9b40-d86a6cede1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:17.613723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3e0f2-926c-43dc-9b40-d86a6cede1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:17.614354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3e0f2-926c-43dc-9b40-d86a6cede1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:17.615931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3e0f2-926c-43dc-9b40-d86a6cede1b7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:17.629064 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3994/8072] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:27:17.634035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1972369-5afd-421b-aaae-9812b40ce381 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:17.645621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1972369-5afd-421b-aaae-9812b40ce381 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:17.650904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af26be-f13d-4441-9f75-2861ae2b0807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:17.651499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af26be-f13d-4441-9f75-2861ae2b0807 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:17.651769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af26be-f13d-4441-9f75-2861ae2b0807 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:17.652035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af26be-f13d-4441-9f75-2861ae2b0807 admin admin] PATH_INFO: `/v3/projects/c508074a7ec8498a968a3c2aba21dd70/users/a49e396ccb93458dac5c895293aecdc0/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:17.654178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af26be-f13d-4441-9f75-2861ae2b0807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c508074a7ec8498a968a3c2aba21dd70, user_id=a49e396ccb93458dac5c895293aecdc0, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:17.657020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af26be-f13d-4441-9f75-2861ae2b0807 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:17.683083 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4079/8073] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:17 2022] PUT /identity/v3/projects/c508074a7ec8498a968a3c2aba21dd70/users/a49e396ccb93458dac5c895293aecdc0/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:17.683757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3aebb-bdf6-4c1e-9bf9-532a40f3e6fc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:17.684112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3aebb-bdf6-4c1e-9bf9-532a40f3e6fc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:17.684674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3aebb-bdf6-4c1e-9bf9-532a40f3e6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:17.720385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b1b3aebb-bdf6-4c1e-9bf9-532a40f3e6fc None None] MFA Rules not processed for user `a49e396ccb93458dac5c895293aecdc0`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:17.721776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b1b3aebb-bdf6-4c1e-9bf9-532a40f3e6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:17.732271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:17.806987 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3995/8074] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:17 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:27:17.812489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af26be-f13d-4441-9f75-2861ae2b0807 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:17.827807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af26be-f13d-4441-9f75-2861ae2b0807 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:17.832646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714d062-3a07-4c73-b500-b31370246a67 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a49e396ccb93458dac5c895293aecdc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c508074a7ec8498a968a3c2aba21dd70', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:17.833258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714d062-3a07-4c73-b500-b31370246a67 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:17.835567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714d062-3a07-4c73-b500-b31370246a67 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:17.835567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714d062-3a07-4c73-b500-b31370246a67 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:17.835567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714d062-3a07-4c73-b500-b31370246a67 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:27:17.835567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714d062-3a07-4c73-b500-b31370246a67 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:17.836426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714d062-3a07-4c73-b500-b31370246a67 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:17.850666 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4080/8075] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:27:17.854475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3aebb-bdf6-4c1e-9bf9-532a40f3e6fc None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:17.865196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3aebb-bdf6-4c1e-9bf9-532a40f3e6fc None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:17.871105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad91fb3-74f9-45a6-b8be-e18be51594d3 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a49e396ccb93458dac5c895293aecdc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c508074a7ec8498a968a3c2aba21dd70', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:17.872135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad91fb3-74f9-45a6-b8be-e18be51594d3 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:17.872135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad91fb3-74f9-45a6-b8be-e18be51594d3 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:17.872542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad91fb3-74f9-45a6-b8be-e18be51594d3 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:17.873044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad91fb3-74f9-45a6-b8be-e18be51594d3 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:17.874777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad91fb3-74f9-45a6-b8be-e18be51594d3 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:17.910695 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3996/8076] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:17 2022] POST /identity/v3/projects => generated 349 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:17.912105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714d062-3a07-4c73-b500-b31370246a67 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:17.927602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714d062-3a07-4c73-b500-b31370246a67 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:17.965723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:18.006932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bdf33-fcdb-4e3f-8e91-87903863058a tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a49e396ccb93458dac5c895293aecdc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c508074a7ec8498a968a3c2aba21dd70', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:18.007566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bdf33-fcdb-4e3f-8e91-87903863058a tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:18.007833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bdf33-fcdb-4e3f-8e91-87903863058a tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:18.008100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bdf33-fcdb-4e3f-8e91-87903863058a tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:18.008797 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bdf33-fcdb-4e3f-8e91-87903863058a tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:27:18.009095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bdf33-fcdb-4e3f-8e91-87903863058a tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:18.010769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bdf33-fcdb-4e3f-8e91-87903863058a tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:18.027236 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4081/8077] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:27:18.037245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad91fb3-74f9-45a6-b8be-e18be51594d3 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:18.039134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dad91fb3-74f9-45a6-b8be-e18be51594d3 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:18.109190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad91fb3-74f9-45a6-b8be-e18be51594d3 tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:18.121295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6004c6f-d336-458f-9985-968b796fd236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:18.121925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6004c6f-d336-458f-9985-968b796fd236 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:18.122194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6004c6f-d336-458f-9985-968b796fd236 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:18.122452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6004c6f-d336-458f-9985-968b796fd236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:18.139824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6004c6f-d336-458f-9985-968b796fd236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:18.141279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6004c6f-d336-458f-9985-968b796fd236 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:18.154868 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3997/8078] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:18 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:18.370523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bdf33-fcdb-4e3f-8e91-87903863058a tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:18.395095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bdf33-fcdb-4e3f-8e91-87903863058a tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:18.400610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fb049-4ac4-4edc-a196-222147c0c4bf tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a49e396ccb93458dac5c895293aecdc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c508074a7ec8498a968a3c2aba21dd70', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:18.401329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fb049-4ac4-4edc-a196-222147c0c4bf tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:18.401674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fb049-4ac4-4edc-a196-222147c0c4bf tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:18.402026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fb049-4ac4-4edc-a196-222147c0c4bf tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] PATH_INFO: `/v3/projects/5e304fcbbe5a4fa8a06a14bc59e7c0ec` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:18.403198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fb049-4ac4-4edc-a196-222147c0c4bf tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5e304fcbbe5a4fa8a06a14bc59e7c0ec)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:18.405341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fb049-4ac4-4edc-a196-222147c0c4bf tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:18.474638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-254fb049-4ac4-4edc-a196-222147c0c4bf tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e304fcbbe5a4fa8a06a14bc59e7c0ec', 'request_id': 'req-254fb049-4ac4-4edc-a196-222147c0c4bf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:18.484355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-254fb049-4ac4-4edc-a196-222147c0c4bf tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e304fcbbe5a4fa8a06a14bc59e7c0ec', 'request_id': 'req-254fb049-4ac4-4edc-a196-222147c0c4bf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:18.495337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-254fb049-4ac4-4edc-a196-222147c0c4bf tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e304fcbbe5a4fa8a06a14bc59e7c0ec', 'request_id': 'req-254fb049-4ac4-4edc-a196-222147c0c4bf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:18.508132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-254fb049-4ac4-4edc-a196-222147c0c4bf tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] The token cache is being invalidate because project 5e304fcbbe5a4fa8a06a14bc59e7c0ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:18.508468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-254fb049-4ac4-4edc-a196-222147c0c4bf tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:18.512034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-254fb049-4ac4-4edc-a196-222147c0c4bf tempest-QuotasTest-402024189 tempest-QuotasTest-402024189-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:18.514147 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4082/8079] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:18 2022] DELETE /identity/v3/projects/5e304fcbbe5a4fa8a06a14bc59e7c0ec => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:18.622552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6004c6f-d336-458f-9985-968b796fd236 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:18.690697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6004c6f-d336-458f-9985-968b796fd236 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:18.725662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:18.756972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:18.757634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:18.757911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:18.758179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] PATH_INFO: `/v3/users/1424a55edc424313ac4909f4ea4ef087` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:18.759321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1424a55edc424313ac4909f4ea4ef087)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:18.761295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:18.821363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1424a55edc424313ac4909f4ea4ef087'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:18.830119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1424a55edc424313ac4909f4ea4ef087'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:18.843831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1424a55edc424313ac4909f4ea4ef087'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:18.844633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1424a55edc424313ac4909f4ea4ef087'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:18.845244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1424a55edc424313ac4909f4ea4ef087'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:18.865629 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3998/8080] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:18 2022] DELETE /identity/v3/users/1424a55edc424313ac4909f4ea4ef087 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:18.870176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ed91d-2b66-4f19-bf32-e9ce1885a48b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:18.870686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ed91d-2b66-4f19-bf32-e9ce1885a48b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:18.870686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ed91d-2b66-4f19-bf32-e9ce1885a48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:18.901709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-959ed91d-2b66-4f19-bf32-e9ce1885a48b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:18.904251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-959ed91d-2b66-4f19-bf32-e9ce1885a48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:18.970546 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4083/8081] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:27:18.980505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:18.982113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:19.045602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5179b-cb36-4e4c-803a-6f9f7f6e8542 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:19.049205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd93b2-05c8-46bd-9d18-47527b78fb94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:19.049804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd93b2-05c8-46bd-9d18-47527b78fb94 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:19.050075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd93b2-05c8-46bd-9d18-47527b78fb94 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:19.050339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd93b2-05c8-46bd-9d18-47527b78fb94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:19.061069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd93b2-05c8-46bd-9d18-47527b78fb94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:19.062503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd93b2-05c8-46bd-9d18-47527b78fb94 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:19.073681 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 3999/8082] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:19.362421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ed91d-2b66-4f19-bf32-e9ce1885a48b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:19.364631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-959ed91d-2b66-4f19-bf32-e9ce1885a48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:19.377330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ed91d-2b66-4f19-bf32-e9ce1885a48b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:19.391098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:19.391726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:19.391981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:19.392230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] PATH_INFO: `/v3/projects/8a9b3bd8d9d549f69331c8f345a5e96e` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:19.393481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a9b3bd8d9d549f69331c8f345a5e96e)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:19.395703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:19.445900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:19.452923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a9b3bd8d9d549f69331c8f345a5e96e', 'request_id': 'req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:19.461364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a9b3bd8d9d549f69331c8f345a5e96e', 'request_id': 'req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:19.467242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a9b3bd8d9d549f69331c8f345a5e96e', 'request_id': 'req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:19.478584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] The token cache is being invalidate because project 8a9b3bd8d9d549f69331c8f345a5e96e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:19.478866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:19.479514 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:19.481074 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4084/8083] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:19 2022] DELETE /identity/v3/projects/8a9b3bd8d9d549f69331c8f345a5e96e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:19.485391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd93b2-05c8-46bd-9d18-47527b78fb94 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:19.495856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd93b2-05c8-46bd-9d18-47527b78fb94 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:19.552356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:19.559579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4375bbc-9f8c-473b-a155-4c18aa25c1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:19.560348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4375bbc-9f8c-473b-a155-4c18aa25c1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:19.560732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4375bbc-9f8c-473b-a155-4c18aa25c1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:19.561259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4375bbc-9f8c-473b-a155-4c18aa25c1ec admin admin] PATH_INFO: `/v3/users/a49e396ccb93458dac5c895293aecdc0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:19.562664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4375bbc-9f8c-473b-a155-4c18aa25c1ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=a49e396ccb93458dac5c895293aecdc0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:19.565236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4375bbc-9f8c-473b-a155-4c18aa25c1ec admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:19.631692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b4375bbc-9f8c-473b-a155-4c18aa25c1ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a49e396ccb93458dac5c895293aecdc0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:19.642767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b4375bbc-9f8c-473b-a155-4c18aa25c1ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a49e396ccb93458dac5c895293aecdc0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:19.658770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b4375bbc-9f8c-473b-a155-4c18aa25c1ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a49e396ccb93458dac5c895293aecdc0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:19.659730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b4375bbc-9f8c-473b-a155-4c18aa25c1ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a49e396ccb93458dac5c895293aecdc0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:19.660273 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-b4375bbc-9f8c-473b-a155-4c18aa25c1ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a49e396ccb93458dac5c895293aecdc0'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:19.674105 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4000/8084] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:19 2022] DELETE /identity/v3/users/a49e396ccb93458dac5c895293aecdc0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:19.962857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:19.965074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:19.982490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5c8fa-155d-40a6-bb7b-5bbe3a91a7bb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:20.043026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f56688-7795-4988-b2db-e5366ca40165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:20.043590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f56688-7795-4988-b2db-e5366ca40165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.043840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f56688-7795-4988-b2db-e5366ca40165 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.044102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f56688-7795-4988-b2db-e5366ca40165 admin admin] PATH_INFO: `/v3/projects/c508074a7ec8498a968a3c2aba21dd70` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.045249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f56688-7795-4988-b2db-e5366ca40165 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c508074a7ec8498a968a3c2aba21dd70)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:20.047159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f56688-7795-4988-b2db-e5366ca40165 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:20.099861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96f56688-7795-4988-b2db-e5366ca40165 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c508074a7ec8498a968a3c2aba21dd70', 'request_id': 'req-96f56688-7795-4988-b2db-e5366ca40165'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:20.110236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96f56688-7795-4988-b2db-e5366ca40165 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c508074a7ec8498a968a3c2aba21dd70', 'request_id': 'req-96f56688-7795-4988-b2db-e5366ca40165'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:20.115832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96f56688-7795-4988-b2db-e5366ca40165 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c508074a7ec8498a968a3c2aba21dd70', 'request_id': 'req-96f56688-7795-4988-b2db-e5366ca40165'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:20.127259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96f56688-7795-4988-b2db-e5366ca40165 admin admin] The token cache is being invalidate because project c508074a7ec8498a968a3c2aba21dd70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:20.127535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96f56688-7795-4988-b2db-e5366ca40165 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:20.128147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-96f56688-7795-4988-b2db-e5366ca40165 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:20.129800 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4085/8085] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:19 2022] DELETE /identity/v3/projects/c508074a7ec8498a968a3c2aba21dd70 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:20.137933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989292a-695d-4dea-a58e-6769dfe81e5e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.138231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989292a-695d-4dea-a58e-6769dfe81e5e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.138499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989292a-695d-4dea-a58e-6769dfe81e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.162726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8989292a-695d-4dea-a58e-6769dfe81e5e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:20.163966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8989292a-695d-4dea-a58e-6769dfe81e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:20.226486 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4001/8086] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:27:20.234957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cbd86-30a9-4a1d-bd07-af14aee682ce None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.235256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cbd86-30a9-4a1d-bd07-af14aee682ce None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.235507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cbd86-30a9-4a1d-bd07-af14aee682ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.256630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-340cbd86-30a9-4a1d-bd07-af14aee682ce None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:20.257846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-340cbd86-30a9-4a1d-bd07-af14aee682ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:20.267136 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4086/8087] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:20.271838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989292a-695d-4dea-a58e-6769dfe81e5e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:20.282477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989292a-695d-4dea-a58e-6769dfe81e5e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:20.287485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e4507-e8be-42b0-9143-ef5d3367a228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:20.288080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e4507-e8be-42b0-9143-ef5d3367a228 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.288349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e4507-e8be-42b0-9143-ef5d3367a228 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.288654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e4507-e8be-42b0-9143-ef5d3367a228 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.289297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e4507-e8be-42b0-9143-ef5d3367a228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:27:20.289592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e4507-e8be-42b0-9143-ef5d3367a228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:20.291210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e4507-e8be-42b0-9143-ef5d3367a228 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:20.303099 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4002/8088] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:27:20.307800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cbd86-30a9-4a1d-bd07-af14aee682ce None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:20.318183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cbd86-30a9-4a1d-bd07-af14aee682ce None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:20.323241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef09ecd-c387-4bfc-9960-b92d8d00d5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:20.323841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef09ecd-c387-4bfc-9960-b92d8d00d5ca admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.324113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef09ecd-c387-4bfc-9960-b92d8d00d5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.324397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef09ecd-c387-4bfc-9960-b92d8d00d5ca admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.325043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef09ecd-c387-4bfc-9960-b92d8d00d5ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:20.326959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef09ecd-c387-4bfc-9960-b92d8d00d5ca admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:20.357257 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4087/8089] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:20 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:20.361636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e4507-e8be-42b0-9143-ef5d3367a228 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:20.372047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e4507-e8be-42b0-9143-ef5d3367a228 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:20.431389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686401b0-618c-48cf-a7d0-574333f1718d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:20.431986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686401b0-618c-48cf-a7d0-574333f1718d admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.432246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686401b0-618c-48cf-a7d0-574333f1718d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.432535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686401b0-618c-48cf-a7d0-574333f1718d admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.433156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686401b0-618c-48cf-a7d0-574333f1718d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:20.435071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686401b0-618c-48cf-a7d0-574333f1718d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:20.445515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:20.459380 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4003/8090] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:20 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:27:20.463696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef09ecd-c387-4bfc-9960-b92d8d00d5ca admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:20.475148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef09ecd-c387-4bfc-9960-b92d8d00d5ca admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:20.480949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d375f-ebbc-4c6f-b1bd-076baf2ab193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:20.481540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d375f-ebbc-4c6f-b1bd-076baf2ab193 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.481802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d375f-ebbc-4c6f-b1bd-076baf2ab193 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.482068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d375f-ebbc-4c6f-b1bd-076baf2ab193 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.482694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d375f-ebbc-4c6f-b1bd-076baf2ab193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:20.484345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d375f-ebbc-4c6f-b1bd-076baf2ab193 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:20.494865 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4088/8091] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:20.498863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686401b0-618c-48cf-a7d0-574333f1718d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:20.509468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686401b0-618c-48cf-a7d0-574333f1718d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:20.514227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a343a7c-3b96-4d31-989d-a69e4138bd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:20.514831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a343a7c-3b96-4d31-989d-a69e4138bd66 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.515120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a343a7c-3b96-4d31-989d-a69e4138bd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.515387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a343a7c-3b96-4d31-989d-a69e4138bd66 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.516014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a343a7c-3b96-4d31-989d-a69e4138bd66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:20.517698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a343a7c-3b96-4d31-989d-a69e4138bd66 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:20.528495 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4004/8092] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:20.532831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d375f-ebbc-4c6f-b1bd-076baf2ab193 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:20.543551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d375f-ebbc-4c6f-b1bd-076baf2ab193 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:20.549221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3eb50b-c304-45c2-b942-2971f7617aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:20.549860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3eb50b-c304-45c2-b942-2971f7617aae admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.550129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3eb50b-c304-45c2-b942-2971f7617aae admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.550392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3eb50b-c304-45c2-b942-2971f7617aae admin admin] PATH_INFO: `/v3/projects/1e5495b24e204677b6d6cebe9065098c/users/371a84904af14ef78d66938bf9d958ea/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.565367 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3eb50b-c304-45c2-b942-2971f7617aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e5495b24e204677b6d6cebe9065098c, user_id=371a84904af14ef78d66938bf9d958ea, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:20.568333 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3eb50b-c304-45c2-b942-2971f7617aae admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:20.571334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:20.585835 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4089/8093] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:20 2022] PUT /identity/v3/projects/1e5495b24e204677b6d6cebe9065098c/users/371a84904af14ef78d66938bf9d958ea/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:20.602674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0658b0-a860-4882-bb1a-61824863b675 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.603061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0658b0-a860-4882-bb1a-61824863b675 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.603430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0658b0-a860-4882-bb1a-61824863b675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.633155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9a0658b0-a860-4882-bb1a-61824863b675 None None] MFA Rules not processed for user `371a84904af14ef78d66938bf9d958ea`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:20.634327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9a0658b0-a860-4882-bb1a-61824863b675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:20.703599 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4005/8094] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:20.708350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3eb50b-c304-45c2-b942-2971f7617aae admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:20.771141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3eb50b-c304-45c2-b942-2971f7617aae admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:20.775898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cce6e8-3eaf-47b5-bf06-5e471797a46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:20.776554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cce6e8-3eaf-47b5-bf06-5e471797a46e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.776830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cce6e8-3eaf-47b5-bf06-5e471797a46e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.777150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cce6e8-3eaf-47b5-bf06-5e471797a46e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.777771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cce6e8-3eaf-47b5-bf06-5e471797a46e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:20.779709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cce6e8-3eaf-47b5-bf06-5e471797a46e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:20.807557 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4090/8095] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:20 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:20.811770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0658b0-a860-4882-bb1a-61824863b675 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:20.823117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0658b0-a860-4882-bb1a-61824863b675 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:20.887051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7f9ed-9299-4514-b373-60b1ebd92b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:20.887648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7f9ed-9299-4514-b373-60b1ebd92b67 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.887920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7f9ed-9299-4514-b373-60b1ebd92b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.888183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7f9ed-9299-4514-b373-60b1ebd92b67 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.888821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7f9ed-9299-4514-b373-60b1ebd92b67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:20.890766 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7f9ed-9299-4514-b373-60b1ebd92b67 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:20.911483 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4006/8096] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:20 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:27:20.915574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cce6e8-3eaf-47b5-bf06-5e471797a46e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:20.926137 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cce6e8-3eaf-47b5-bf06-5e471797a46e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:20.934517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d43373-ac61-4c23-9f83-7fa8612be26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:20.935424 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d43373-ac61-4c23-9f83-7fa8612be26d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.935986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d43373-ac61-4c23-9f83-7fa8612be26d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.936358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d43373-ac61-4c23-9f83-7fa8612be26d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.937228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d43373-ac61-4c23-9f83-7fa8612be26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:20.938897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d43373-ac61-4c23-9f83-7fa8612be26d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:20.951851 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4091/8097] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:27:20.957045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7f9ed-9299-4514-b373-60b1ebd92b67 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:20.968071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7f9ed-9299-4514-b373-60b1ebd92b67 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:20.974409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f644d-8ea7-4eab-825c-421f157d31eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:20.975089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f644d-8ea7-4eab-825c-421f157d31eb admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:20.975359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f644d-8ea7-4eab-825c-421f157d31eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:20.975622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f644d-8ea7-4eab-825c-421f157d31eb admin admin] PATH_INFO: `/v3/projects/76ef69657c90494fb7af54321bd25d98/users/a2dd8cae6b9241d58d9f38bdeaf46c28/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:20.995103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f644d-8ea7-4eab-825c-421f157d31eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ef69657c90494fb7af54321bd25d98, user_id=a2dd8cae6b9241d58d9f38bdeaf46c28, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:20.998285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f644d-8ea7-4eab-825c-421f157d31eb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:21.015108 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4007/8098] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:20 2022] PUT /identity/v3/projects/76ef69657c90494fb7af54321bd25d98/users/a2dd8cae6b9241d58d9f38bdeaf46c28/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:21.019510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d43373-ac61-4c23-9f83-7fa8612be26d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:21.032799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d43373-ac61-4c23-9f83-7fa8612be26d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:21.095869 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee3548-c948-442b-a093-76a1cc3ebe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:21.096410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee3548-c948-442b-a093-76a1cc3ebe77 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:21.096687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee3548-c948-442b-a093-76a1cc3ebe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:21.096950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee3548-c948-442b-a093-76a1cc3ebe77 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:21.097570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee3548-c948-442b-a093-76a1cc3ebe77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:21.099240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee3548-c948-442b-a093-76a1cc3ebe77 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:21.109728 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4092/8099] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:21.114563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f644d-8ea7-4eab-825c-421f157d31eb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:21.129656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f644d-8ea7-4eab-825c-421f157d31eb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:21.134734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc8c42-a7ab-42c5-a7c2-9114bf7c793e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:21.135424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc8c42-a7ab-42c5-a7c2-9114bf7c793e admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:21.135829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc8c42-a7ab-42c5-a7c2-9114bf7c793e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:21.136082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc8c42-a7ab-42c5-a7c2-9114bf7c793e admin admin] PATH_INFO: `/v3/projects/76ef69657c90494fb7af54321bd25d98/users/a2dd8cae6b9241d58d9f38bdeaf46c28/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:21.138122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc8c42-a7ab-42c5-a7c2-9114bf7c793e admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ef69657c90494fb7af54321bd25d98, user_id=a2dd8cae6b9241d58d9f38bdeaf46c28, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:21.140827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc8c42-a7ab-42c5-a7c2-9114bf7c793e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:21.152605 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4008/8100] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:21 2022] PUT /identity/v3/projects/76ef69657c90494fb7af54321bd25d98/users/a2dd8cae6b9241d58d9f38bdeaf46c28/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:21.163228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ecdbd-bb64-405b-b9d5-818882292661 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:21.163537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ecdbd-bb64-405b-b9d5-818882292661 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:21.163801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ecdbd-bb64-405b-b9d5-818882292661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:21.192638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5f1ecdbd-bb64-405b-b9d5-818882292661 None None] MFA Rules not processed for user `a2dd8cae6b9241d58d9f38bdeaf46c28`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:21.193820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5f1ecdbd-bb64-405b-b9d5-818882292661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:21.246426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:21.265853 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4093/8101] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:27:21.277630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc8c42-a7ab-42c5-a7c2-9114bf7c793e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:21.279641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2acc8c42-a7ab-42c5-a7c2-9114bf7c793e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:21.309567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:21.354929 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc8c42-a7ab-42c5-a7c2-9114bf7c793e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:21.359212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543d106-0d98-45a2-b86e-7fc18afbc161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:21.359813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543d106-0d98-45a2-b86e-7fc18afbc161 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:21.360078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543d106-0d98-45a2-b86e-7fc18afbc161 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:21.360358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543d106-0d98-45a2-b86e-7fc18afbc161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:21.371559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543d106-0d98-45a2-b86e-7fc18afbc161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:21.373006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543d106-0d98-45a2-b86e-7fc18afbc161 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:21.384123 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4009/8102] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:21.847388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ecdbd-bb64-405b-b9d5-818882292661 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:21.919258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ecdbd-bb64-405b-b9d5-818882292661 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:21.924565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:21.925233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:21.925507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:21.925778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] PATH_INFO: `/v3/users/371a84904af14ef78d66938bf9d958ea` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:21.927666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371a84904af14ef78d66938bf9d958ea)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:21.930081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:21.984276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371a84904af14ef78d66938bf9d958ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:21.992546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371a84904af14ef78d66938bf9d958ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:21.999562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371a84904af14ef78d66938bf9d958ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:22.011983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371a84904af14ef78d66938bf9d958ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:22.012677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371a84904af14ef78d66938bf9d958ea'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:22.018650 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4094/8103] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:21 2022] DELETE /identity/v3/users/371a84904af14ef78d66938bf9d958ea => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:22.023406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98ee0b-88db-4f15-87c2-bff282be0ff4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:22.023949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98ee0b-88db-4f15-87c2-bff282be0ff4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:22.023949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98ee0b-88db-4f15-87c2-bff282be0ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:22.043879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fb98ee0b-88db-4f15-87c2-bff282be0ff4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:22.045118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fb98ee0b-88db-4f15-87c2-bff282be0ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:22.112391 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4010/8104] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:22.122802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:22.124484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:22.193739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb934359-afaf-4fe4-ba73-216100ffd324 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:22.197871 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe8a42-e2de-455d-ba0d-1258aa05e086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:22.198489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe8a42-e2de-455d-ba0d-1258aa05e086 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:22.198760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe8a42-e2de-455d-ba0d-1258aa05e086 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:22.199023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe8a42-e2de-455d-ba0d-1258aa05e086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:22.210235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe8a42-e2de-455d-ba0d-1258aa05e086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:22.211689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe8a42-e2de-455d-ba0d-1258aa05e086 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:22.222343 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4095/8105] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:22.506588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98ee0b-88db-4f15-87c2-bff282be0ff4 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:22.508313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fb98ee0b-88db-4f15-87c2-bff282be0ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:22.529511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98ee0b-88db-4f15-87c2-bff282be0ff4 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:22.534350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:22.534990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:22.535310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:22.535717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] PATH_INFO: `/v3/projects/1e5495b24e204677b6d6cebe9065098c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:22.536943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e5495b24e204677b6d6cebe9065098c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:22.539666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:22.588093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e5495b24e204677b6d6cebe9065098c', 'request_id': 'req-d4b78250-9cb4-4022-8c56-06d01743bf23'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:22.594707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e5495b24e204677b6d6cebe9065098c', 'request_id': 'req-d4b78250-9cb4-4022-8c56-06d01743bf23'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:22.602788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e5495b24e204677b6d6cebe9065098c', 'request_id': 'req-d4b78250-9cb4-4022-8c56-06d01743bf23'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:22.615259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] The token cache is being invalidate because project 1e5495b24e204677b6d6cebe9065098c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:22.615259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:22.615259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:22.616156 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4011/8106] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:22 2022] DELETE /identity/v3/projects/1e5495b24e204677b6d6cebe9065098c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:22.620540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe8a42-e2de-455d-ba0d-1258aa05e086 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:22.630927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe8a42-e2de-455d-ba0d-1258aa05e086 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:22.641011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:22.690174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad248a7-22cd-4718-8d02-155222b36b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:22.690800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad248a7-22cd-4718-8d02-155222b36b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:22.691067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad248a7-22cd-4718-8d02-155222b36b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:22.691311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad248a7-22cd-4718-8d02-155222b36b9b admin admin] PATH_INFO: `/v3/users/a2dd8cae6b9241d58d9f38bdeaf46c28` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:22.692910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad248a7-22cd-4718-8d02-155222b36b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2dd8cae6b9241d58d9f38bdeaf46c28)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:22.694833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad248a7-22cd-4718-8d02-155222b36b9b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:22.743848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6ad248a7-22cd-4718-8d02-155222b36b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2dd8cae6b9241d58d9f38bdeaf46c28'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:22.754910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6ad248a7-22cd-4718-8d02-155222b36b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2dd8cae6b9241d58d9f38bdeaf46c28'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:22.762865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6ad248a7-22cd-4718-8d02-155222b36b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2dd8cae6b9241d58d9f38bdeaf46c28'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:22.779002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6ad248a7-22cd-4718-8d02-155222b36b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2dd8cae6b9241d58d9f38bdeaf46c28'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:22.779696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6ad248a7-22cd-4718-8d02-155222b36b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2dd8cae6b9241d58d9f38bdeaf46c28'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:22.786972 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4096/8107] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:22 2022] DELETE /identity/v3/users/a2dd8cae6b9241d58d9f38bdeaf46c28 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:23.106085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:23.107757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:23.121595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b78250-9cb4-4022-8c56-06d01743bf23 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:23.166840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:23.184957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330f69e-140b-4fc2-a1a2-6a2f043135a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:23.185721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330f69e-140b-4fc2-a1a2-6a2f043135a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:23.185992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330f69e-140b-4fc2-a1a2-6a2f043135a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:23.186256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330f69e-140b-4fc2-a1a2-6a2f043135a5 admin admin] PATH_INFO: `/v3/projects/76ef69657c90494fb7af54321bd25d98` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:23.187370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330f69e-140b-4fc2-a1a2-6a2f043135a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ef69657c90494fb7af54321bd25d98)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:23.189629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330f69e-140b-4fc2-a1a2-6a2f043135a5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:23.239436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6330f69e-140b-4fc2-a1a2-6a2f043135a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ef69657c90494fb7af54321bd25d98', 'request_id': 'req-6330f69e-140b-4fc2-a1a2-6a2f043135a5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:23.245698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6330f69e-140b-4fc2-a1a2-6a2f043135a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ef69657c90494fb7af54321bd25d98', 'request_id': 'req-6330f69e-140b-4fc2-a1a2-6a2f043135a5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:23.254023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6330f69e-140b-4fc2-a1a2-6a2f043135a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ef69657c90494fb7af54321bd25d98', 'request_id': 'req-6330f69e-140b-4fc2-a1a2-6a2f043135a5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:23.265041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6330f69e-140b-4fc2-a1a2-6a2f043135a5 admin admin] The token cache is being invalidate because project 76ef69657c90494fb7af54321bd25d98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:23.265325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6330f69e-140b-4fc2-a1a2-6a2f043135a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:23.265962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-6330f69e-140b-4fc2-a1a2-6a2f043135a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:23.267512 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4012/8108] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:23 2022] DELETE /identity/v3/projects/76ef69657c90494fb7af54321bd25d98 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:23.272725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f94f9-a9fc-4f54-91dc-0c32c8b003dd None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:23.273218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f94f9-a9fc-4f54-91dc-0c32c8b003dd None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:23.273218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f94f9-a9fc-4f54-91dc-0c32c8b003dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:23.294565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e91f94f9-a9fc-4f54-91dc-0c32c8b003dd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:23.295827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e91f94f9-a9fc-4f54-91dc-0c32c8b003dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:23.361505 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4097/8109] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:27:23.371081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7e6a6-ce81-4ee1-a047-4e14ff67a726 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:23.371350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7e6a6-ce81-4ee1-a047-4e14ff67a726 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:23.371638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7e6a6-ce81-4ee1-a047-4e14ff67a726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:23.393650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d4d7e6a6-ce81-4ee1-a047-4e14ff67a726 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:23.395200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d4d7e6a6-ce81-4ee1-a047-4e14ff67a726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:23.408276 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4013/8110] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:23.409512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f94f9-a9fc-4f54-91dc-0c32c8b003dd None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:23.420079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f94f9-a9fc-4f54-91dc-0c32c8b003dd None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:23.429917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d658ce-3ea0-4d5b-adb9-dd8d55473be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:23.429917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d658ce-3ea0-4d5b-adb9-dd8d55473be3 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:23.429917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d658ce-3ea0-4d5b-adb9-dd8d55473be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:23.429917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d658ce-3ea0-4d5b-adb9-dd8d55473be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:23.430301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d658ce-3ea0-4d5b-adb9-dd8d55473be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:27:23.430530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d658ce-3ea0-4d5b-adb9-dd8d55473be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:23.432177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d658ce-3ea0-4d5b-adb9-dd8d55473be3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:23.446574 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4098/8111] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:27:23.451071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7e6a6-ce81-4ee1-a047-4e14ff67a726 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:23.460472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7e6a6-ce81-4ee1-a047-4e14ff67a726 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:23.465136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaad583-a52f-45c1-baf0-bd8f818496bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:23.465726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaad583-a52f-45c1-baf0-bd8f818496bc admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:23.465995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaad583-a52f-45c1-baf0-bd8f818496bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:23.466254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaad583-a52f-45c1-baf0-bd8f818496bc admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:23.466858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaad583-a52f-45c1-baf0-bd8f818496bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:23.468829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaad583-a52f-45c1-baf0-bd8f818496bc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:23.504417 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4014/8112] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:23 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:23.506928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d658ce-3ea0-4d5b-adb9-dd8d55473be3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:23.517432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d658ce-3ea0-4d5b-adb9-dd8d55473be3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:23.534535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:23.579416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f16ed7-5c01-4528-80ef-6993a5e35091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:23.580506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f16ed7-5c01-4528-80ef-6993a5e35091 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:23.580857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f16ed7-5c01-4528-80ef-6993a5e35091 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:23.581243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f16ed7-5c01-4528-80ef-6993a5e35091 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:23.581928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f16ed7-5c01-4528-80ef-6993a5e35091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:23.584877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f16ed7-5c01-4528-80ef-6993a5e35091 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:23.609265 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4099/8113] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:23 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:23.611062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaad583-a52f-45c1-baf0-bd8f818496bc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:23.613168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ceaad583-a52f-45c1-baf0-bd8f818496bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:23.613587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f16ed7-5c01-4528-80ef-6993a5e35091 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:23.623834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f16ed7-5c01-4528-80ef-6993a5e35091 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:23.628550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dbd5c-fbbb-4fcb-babb-4a6f464e6d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:23.629289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dbd5c-fbbb-4fcb-babb-4a6f464e6d3d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:23.629568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dbd5c-fbbb-4fcb-babb-4a6f464e6d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:23.629881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dbd5c-fbbb-4fcb-babb-4a6f464e6d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:23.630902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5dbd5c-fbbb-4fcb-babb-4a6f464e6d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:23.632924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5dbd5c-fbbb-4fcb-babb-4a6f464e6d3d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:23.645241 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4100/8114] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:27:23.649062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dbd5c-fbbb-4fcb-babb-4a6f464e6d3d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:23.661314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dbd5c-fbbb-4fcb-babb-4a6f464e6d3d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:23.666807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4851575-ce94-4c3a-999c-26a97d780c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:23.667331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4851575-ce94-4c3a-999c-26a97d780c32 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:23.667608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4851575-ce94-4c3a-999c-26a97d780c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:23.667876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4851575-ce94-4c3a-999c-26a97d780c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:23.668570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4851575-ce94-4c3a-999c-26a97d780c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:23.670372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4851575-ce94-4c3a-999c-26a97d780c32 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:23.683972 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4101/8115] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:27:23.689465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4851575-ce94-4c3a-999c-26a97d780c32 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:23.702680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaad583-a52f-45c1-baf0-bd8f818496bc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:23.703331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4851575-ce94-4c3a-999c-26a97d780c32 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:23.706774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5988be8-cdc0-47cc-9021-f17ac334fc03 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:23.707772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5988be8-cdc0-47cc-9021-f17ac334fc03 service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:23.708053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5988be8-cdc0-47cc-9021-f17ac334fc03 service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:23.708319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5988be8-cdc0-47cc-9021-f17ac334fc03 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:23.709264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84aa9d-25c6-4f77-91e6-3501a8debba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:23.710150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c5988be8-cdc0-47cc-9021-f17ac334fc03 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:23.710475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84aa9d-25c6-4f77-91e6-3501a8debba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:23.710980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84aa9d-25c6-4f77-91e6-3501a8debba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:23.711411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84aa9d-25c6-4f77-91e6-3501a8debba7 admin admin] PATH_INFO: `/v3/projects/45e5999717654f37bc21b474097a689b/users/a032fd5a2c8643fbba4129d1da61af19/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:23.729717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84aa9d-25c6-4f77-91e6-3501a8debba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e5999717654f37bc21b474097a689b, user_id=a032fd5a2c8643fbba4129d1da61af19, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:23.733009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84aa9d-25c6-4f77-91e6-3501a8debba7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:23.749573 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4102/8116] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:23 2022] PUT /identity/v3/projects/45e5999717654f37bc21b474097a689b/users/a032fd5a2c8643fbba4129d1da61af19/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:23.771041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26de39-7bac-4881-bd7b-55b03c535f03 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:23.771317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26de39-7bac-4881-bd7b-55b03c535f03 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:23.771639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26de39-7bac-4881-bd7b-55b03c535f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:23.772172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:23.797769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5988be8-cdc0-47cc-9021-f17ac334fc03 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:23.800944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5988be8-cdc0-47cc-9021-f17ac334fc03 service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:23.806148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8c26de39-7bac-4881-bd7b-55b03c535f03 None None] MFA Rules not processed for user `a032fd5a2c8643fbba4129d1da61af19`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:23.807391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8c26de39-7bac-4881-bd7b-55b03c535f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:23.825982 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4015/8117] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:23.874632 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4103/8118] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:27:23.879427 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5988be8-cdc0-47cc-9021-f17ac334fc03 service glance] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:23.891092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5988be8-cdc0-47cc-9021-f17ac334fc03 service glance] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:23.897478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26de39-7bac-4881-bd7b-55b03c535f03 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:23.898876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8c26de39-7bac-4881-bd7b-55b03c535f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:23.951166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89adc13-b450-4a10-80fa-3b78e69ad376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:23.951803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89adc13-b450-4a10-80fa-3b78e69ad376 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:23.952077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89adc13-b450-4a10-80fa-3b78e69ad376 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:23.952340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89adc13-b450-4a10-80fa-3b78e69ad376 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:23.952997 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89adc13-b450-4a10-80fa-3b78e69ad376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:23.954899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89adc13-b450-4a10-80fa-3b78e69ad376 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:23.957319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26de39-7bac-4881-bd7b-55b03c535f03 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:23.961043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a26ee2-d972-4f76-a10f-1ee0625801bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:23.961663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a26ee2-d972-4f76-a10f-1ee0625801bc service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:23.961930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a26ee2-d972-4f76-a10f-1ee0625801bc service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:23.962191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a26ee2-d972-4f76-a10f-1ee0625801bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:23.964087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-38a26ee2-d972-4f76-a10f-1ee0625801bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:23.971384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:23.977387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a26ee2-d972-4f76-a10f-1ee0625801bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:23.980311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a26ee2-d972-4f76-a10f-1ee0625801bc service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:23.988126 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4016/8119] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:23 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:27:23.991118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89adc13-b450-4a10-80fa-3b78e69ad376 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:23.996254 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4104/8120] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:27:24.060069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89adc13-b450-4a10-80fa-3b78e69ad376 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:24.065135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521352a6-9fa7-448e-ae49-f6456e0cf43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:24.065740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521352a6-9fa7-448e-ae49-f6456e0cf43e admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:24.066018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521352a6-9fa7-448e-ae49-f6456e0cf43e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:24.066288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521352a6-9fa7-448e-ae49-f6456e0cf43e admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:24.067655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521352a6-9fa7-448e-ae49-f6456e0cf43e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:24.069332 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521352a6-9fa7-448e-ae49-f6456e0cf43e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:24.087555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a26ee2-d972-4f76-a10f-1ee0625801bc service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:24.089142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-38a26ee2-d972-4f76-a10f-1ee0625801bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:24.092887 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4017/8121] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:23 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:24.096576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521352a6-9fa7-448e-ae49-f6456e0cf43e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:24.103300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a26ee2-d972-4f76-a10f-1ee0625801bc service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:24.106586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521352a6-9fa7-448e-ae49-f6456e0cf43e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:24.111212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55c9c1-fc01-476e-b2f5-8b2e40b7a42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:24.111851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55c9c1-fc01-476e-b2f5-8b2e40b7a42d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:24.112130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55c9c1-fc01-476e-b2f5-8b2e40b7a42d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:24.112438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55c9c1-fc01-476e-b2f5-8b2e40b7a42d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:24.113205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55c9c1-fc01-476e-b2f5-8b2e40b7a42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:24.114985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55c9c1-fc01-476e-b2f5-8b2e40b7a42d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:24.126139 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4018/8122] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:27:24.131000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55c9c1-fc01-476e-b2f5-8b2e40b7a42d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:24.140807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55c9c1-fc01-476e-b2f5-8b2e40b7a42d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:24.146519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecc93c-fc2f-462f-8456-e85f92ecec20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:24.147168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecc93c-fc2f-462f-8456-e85f92ecec20 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:24.147443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecc93c-fc2f-462f-8456-e85f92ecec20 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:24.147717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecc93c-fc2f-462f-8456-e85f92ecec20 admin admin] PATH_INFO: `/v3/projects/70a0753bf6f941fe8c2628632353ebc6/users/444cf3c4896e4ae586164d4ae0cdba80/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:24.155499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c65704-c431-4774-b032-8dc718f366b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:24.156241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c65704-c431-4774-b032-8dc718f366b3 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:24.156547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c65704-c431-4774-b032-8dc718f366b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:24.156807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c65704-c431-4774-b032-8dc718f366b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:24.158705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-71c65704-c431-4774-b032-8dc718f366b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:24.164863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ecc93c-fc2f-462f-8456-e85f92ecec20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a0753bf6f941fe8c2628632353ebc6, user_id=444cf3c4896e4ae586164d4ae0cdba80, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:24.167488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ecc93c-fc2f-462f-8456-e85f92ecec20 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:24.181406 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4019/8123] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:24 2022] PUT /identity/v3/projects/70a0753bf6f941fe8c2628632353ebc6/users/444cf3c4896e4ae586164d4ae0cdba80/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:24.185277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecc93c-fc2f-462f-8456-e85f92ecec20 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:24.203454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:24.244283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c65704-c431-4774-b032-8dc718f366b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:24.245769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c65704-c431-4774-b032-8dc718f366b3 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:24.253863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecc93c-fc2f-462f-8456-e85f92ecec20 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:24.257749 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4105/8124] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:27:24.259647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eca99c-68ed-461b-b502-6009b7735870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:24.260686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eca99c-68ed-461b-b502-6009b7735870 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:24.261026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eca99c-68ed-461b-b502-6009b7735870 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:24.261360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eca99c-68ed-461b-b502-6009b7735870 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:24.262072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eca99c-68ed-461b-b502-6009b7735870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:24.263779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eca99c-68ed-461b-b502-6009b7735870 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:24.274283 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4020/8125] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:27:24.278454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c65704-c431-4774-b032-8dc718f366b3 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:24.288960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c65704-c431-4774-b032-8dc718f366b3 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:24.293442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef2e58-8561-41bd-857b-f1e470125577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:24.294094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef2e58-8561-41bd-857b-f1e470125577 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:24.294361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef2e58-8561-41bd-857b-f1e470125577 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:24.294621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef2e58-8561-41bd-857b-f1e470125577 admin admin] PATH_INFO: `/v3/projects/70a0753bf6f941fe8c2628632353ebc6/users/444cf3c4896e4ae586164d4ae0cdba80/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:24.296525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef2e58-8561-41bd-857b-f1e470125577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a0753bf6f941fe8c2628632353ebc6, user_id=444cf3c4896e4ae586164d4ae0cdba80, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:24.299181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef2e58-8561-41bd-857b-f1e470125577 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:24.310788 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4106/8126] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:24 2022] PUT /identity/v3/projects/70a0753bf6f941fe8c2628632353ebc6/users/444cf3c4896e4ae586164d4ae0cdba80/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:24.321834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12167249-e6a8-4c11-ab8d-2c7f32cf9360 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:24.322116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12167249-e6a8-4c11-ab8d-2c7f32cf9360 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:24.322384 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12167249-e6a8-4c11-ab8d-2c7f32cf9360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:24.351174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-12167249-e6a8-4c11-ab8d-2c7f32cf9360 None None] MFA Rules not processed for user `444cf3c4896e4ae586164d4ae0cdba80`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:24.352394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-12167249-e6a8-4c11-ab8d-2c7f32cf9360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:24.423956 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4021/8127] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:24.435647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef2e58-8561-41bd-857b-f1e470125577 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:24.437480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-daef2e58-8561-41bd-857b-f1e470125577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:24.504401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef2e58-8561-41bd-857b-f1e470125577 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:24.508071 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce30e4-6c8c-4a93-a162-1c07cf941ade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:24.508717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce30e4-6c8c-4a93-a162-1c07cf941ade service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:24.508991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce30e4-6c8c-4a93-a162-1c07cf941ade service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:24.509258 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce30e4-6c8c-4a93-a162-1c07cf941ade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:24.519327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce30e4-6c8c-4a93-a162-1c07cf941ade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:24.521293 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce30e4-6c8c-4a93-a162-1c07cf941ade service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:24.532016 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4107/8128] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:24.708566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12167249-e6a8-4c11-ab8d-2c7f32cf9360 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:24.777933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12167249-e6a8-4c11-ab8d-2c7f32cf9360 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:24.782745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:24.783391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:24.783714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:24.783992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] PATH_INFO: `/v3/users/a032fd5a2c8643fbba4129d1da61af19` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:24.785182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a032fd5a2c8643fbba4129d1da61af19)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:24.787238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:24.838566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:24.845871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a032fd5a2c8643fbba4129d1da61af19'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:24.856140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a032fd5a2c8643fbba4129d1da61af19'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:24.872189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a032fd5a2c8643fbba4129d1da61af19'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:24.873544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a032fd5a2c8643fbba4129d1da61af19'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:24.874160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a032fd5a2c8643fbba4129d1da61af19'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:24.888793 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4022/8129] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:24 2022] DELETE /identity/v3/users/a032fd5a2c8643fbba4129d1da61af19 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:24.893262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c5c63-5b7d-4c1a-a62a-43e0a9389a6f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:24.893588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c5c63-5b7d-4c1a-a62a-43e0a9389a6f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:24.893939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c5c63-5b7d-4c1a-a62a-43e0a9389a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:24.913558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e41c5c63-5b7d-4c1a-a62a-43e0a9389a6f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:24.914751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e41c5c63-5b7d-4c1a-a62a-43e0a9389a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:24.936746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:24.976446 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4108/8130] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:27:24.986213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:24.987824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:25.082401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834afddd-5387-4b68-922f-87e97300eee1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:25.086869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef9dd5-0005-4aee-9252-b78da829429c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:25.088301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef9dd5-0005-4aee-9252-b78da829429c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:25.088718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef9dd5-0005-4aee-9252-b78da829429c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:25.088893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef9dd5-0005-4aee-9252-b78da829429c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:25.099690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef9dd5-0005-4aee-9252-b78da829429c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:25.101261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef9dd5-0005-4aee-9252-b78da829429c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:25.111769 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4023/8131] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:25.631513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c5c63-5b7d-4c1a-a62a-43e0a9389a6f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:25.633236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e41c5c63-5b7d-4c1a-a62a-43e0a9389a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:25.650906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c5c63-5b7d-4c1a-a62a-43e0a9389a6f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:25.656214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:25.656914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:25.657675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:25.657940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] PATH_INFO: `/v3/projects/45e5999717654f37bc21b474097a689b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:25.659131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45e5999717654f37bc21b474097a689b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:25.672754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:25.725862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45e5999717654f37bc21b474097a689b', 'request_id': 'req-79a57c3a-c5c0-4ebd-a374-299eee337a49'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:25.734255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45e5999717654f37bc21b474097a689b', 'request_id': 'req-79a57c3a-c5c0-4ebd-a374-299eee337a49'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:25.741812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45e5999717654f37bc21b474097a689b', 'request_id': 'req-79a57c3a-c5c0-4ebd-a374-299eee337a49'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:25.755392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] The token cache is being invalidate because project 45e5999717654f37bc21b474097a689b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:25.755720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:25.757004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:25.758512 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4109/8132] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:25 2022] DELETE /identity/v3/projects/45e5999717654f37bc21b474097a689b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:25.762675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef9dd5-0005-4aee-9252-b78da829429c service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:25.773236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef9dd5-0005-4aee-9252-b78da829429c service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:25.833705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfefa5-b6c2-4720-a290-512e70646b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:25.834418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfefa5-b6c2-4720-a290-512e70646b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:25.834765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfefa5-b6c2-4720-a290-512e70646b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:25.835111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfefa5-b6c2-4720-a290-512e70646b5e admin admin] PATH_INFO: `/v3/users/444cf3c4896e4ae586164d4ae0cdba80` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:25.836401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfefa5-b6c2-4720-a290-512e70646b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=444cf3c4896e4ae586164d4ae0cdba80)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:25.838770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfefa5-b6c2-4720-a290-512e70646b5e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:25.899248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1abfefa5-b6c2-4720-a290-512e70646b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '444cf3c4896e4ae586164d4ae0cdba80'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:25.908025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1abfefa5-b6c2-4720-a290-512e70646b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '444cf3c4896e4ae586164d4ae0cdba80'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:25.915529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:25.927632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1abfefa5-b6c2-4720-a290-512e70646b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '444cf3c4896e4ae586164d4ae0cdba80'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:25.928595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1abfefa5-b6c2-4720-a290-512e70646b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '444cf3c4896e4ae586164d4ae0cdba80'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:25.929190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1abfefa5-b6c2-4720-a290-512e70646b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '444cf3c4896e4ae586164d4ae0cdba80'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:25.942472 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4024/8133] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:25 2022] DELETE /identity/v3/users/444cf3c4896e4ae586164d4ae0cdba80 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:26.351597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:26.353036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:26.366647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a57c3a-c5c0-4ebd-a374-299eee337a49 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:26.428667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cba62-5f39-4a58-ab5b-cfddefdec399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:26.429350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cba62-5f39-4a58-ab5b-cfddefdec399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:26.429622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cba62-5f39-4a58-ab5b-cfddefdec399 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:26.429889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cba62-5f39-4a58-ab5b-cfddefdec399 admin admin] PATH_INFO: `/v3/projects/70a0753bf6f941fe8c2628632353ebc6` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:26.431030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cba62-5f39-4a58-ab5b-cfddefdec399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70a0753bf6f941fe8c2628632353ebc6)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:26.433045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cba62-5f39-4a58-ab5b-cfddefdec399 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:26.484401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-914cba62-5f39-4a58-ab5b-cfddefdec399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70a0753bf6f941fe8c2628632353ebc6', 'request_id': 'req-914cba62-5f39-4a58-ab5b-cfddefdec399'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:26.486835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:26.496308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-914cba62-5f39-4a58-ab5b-cfddefdec399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70a0753bf6f941fe8c2628632353ebc6', 'request_id': 'req-914cba62-5f39-4a58-ab5b-cfddefdec399'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:26.505600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-914cba62-5f39-4a58-ab5b-cfddefdec399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70a0753bf6f941fe8c2628632353ebc6', 'request_id': 'req-914cba62-5f39-4a58-ab5b-cfddefdec399'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:26.518341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-914cba62-5f39-4a58-ab5b-cfddefdec399 admin admin] The token cache is being invalidate because project 70a0753bf6f941fe8c2628632353ebc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:26.518624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-914cba62-5f39-4a58-ab5b-cfddefdec399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:26.519537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-914cba62-5f39-4a58-ab5b-cfddefdec399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:26.521607 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4110/8134] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:26 2022] DELETE /identity/v3/projects/70a0753bf6f941fe8c2628632353ebc6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:26.526804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d262e9-9744-4545-ac0f-3693044b4da3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:26.527662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d262e9-9744-4545-ac0f-3693044b4da3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:26.527939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d262e9-9744-4545-ac0f-3693044b4da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:26.549914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-43d262e9-9744-4545-ac0f-3693044b4da3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:26.551201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-43d262e9-9744-4545-ac0f-3693044b4da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:26.612527 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4025/8135] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:26.714509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41eb07b-bfbb-4c55-823c-04a3d78ad306 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:26.715073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41eb07b-bfbb-4c55-823c-04a3d78ad306 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:26.715073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41eb07b-bfbb-4c55-823c-04a3d78ad306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:26.734383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a41eb07b-bfbb-4c55-823c-04a3d78ad306 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:26.735667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a41eb07b-bfbb-4c55-823c-04a3d78ad306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:26.744487 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4111/8136] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:26.749189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d262e9-9744-4545-ac0f-3693044b4da3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:26.761281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d262e9-9744-4545-ac0f-3693044b4da3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:26.766240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e998f-1343-44cf-942d-6ad5c781835f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:26.766843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e998f-1343-44cf-942d-6ad5c781835f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:26.767116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e998f-1343-44cf-942d-6ad5c781835f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:26.767383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e998f-1343-44cf-942d-6ad5c781835f admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:26.768023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e998f-1343-44cf-942d-6ad5c781835f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:27:26.768320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e998f-1343-44cf-942d-6ad5c781835f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:26.770085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e998f-1343-44cf-942d-6ad5c781835f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:26.786310 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4026/8137] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:27:26.787781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41eb07b-bfbb-4c55-823c-04a3d78ad306 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:26.806992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41eb07b-bfbb-4c55-823c-04a3d78ad306 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:26.818682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9cfb8-6914-4b72-a5cd-796e556d2963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:26.819653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9cfb8-6914-4b72-a5cd-796e556d2963 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:26.820554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9cfb8-6914-4b72-a5cd-796e556d2963 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:26.820926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9cfb8-6914-4b72-a5cd-796e556d2963 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:26.822431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9cfb8-6914-4b72-a5cd-796e556d2963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:26.827876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9cfb8-6914-4b72-a5cd-796e556d2963 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:26.859020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b038-c776-4f49-82e6-d8679f69d04d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:26.859280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b038-c776-4f49-82e6-d8679f69d04d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:26.859546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b038-c776-4f49-82e6-d8679f69d04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:26.861003 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4112/8138] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:26 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:26.865263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9cfb8-6914-4b72-a5cd-796e556d2963 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:26.887304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-13a7b038-c776-4f49-82e6-d8679f69d04d None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:26.888679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-13a7b038-c776-4f49-82e6-d8679f69d04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:26.945136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9cfb8-6914-4b72-a5cd-796e556d2963 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:26.950152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839940ed-9502-4375-9a2a-e35b0b960db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:26.950755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839940ed-9502-4375-9a2a-e35b0b960db3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:26.951031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839940ed-9502-4375-9a2a-e35b0b960db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:26.951289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839940ed-9502-4375-9a2a-e35b0b960db3 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:26.951908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839940ed-9502-4375-9a2a-e35b0b960db3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:26.953913 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839940ed-9502-4375-9a2a-e35b0b960db3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:26.962353 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4027/8139] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:27:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:27:26.975125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b038-c776-4f49-82e6-d8679f69d04d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:26.977330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-13a7b038-c776-4f49-82e6-d8679f69d04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:26.981232 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4113/8140] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:26 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:27:26.985313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839940ed-9502-4375-9a2a-e35b0b960db3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:26.996328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839940ed-9502-4375-9a2a-e35b0b960db3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:27.002770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57381652-f4a9-40ce-85d1-1a97e243e74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:27.003591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381652-f4a9-40ce-85d1-1a97e243e74c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:27.003965 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381652-f4a9-40ce-85d1-1a97e243e74c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:27.004769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381652-f4a9-40ce-85d1-1a97e243e74c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:27.004989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57381652-f4a9-40ce-85d1-1a97e243e74c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:27.006642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57381652-f4a9-40ce-85d1-1a97e243e74c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:27.020413 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4114/8141] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:27:27.024809 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57381652-f4a9-40ce-85d1-1a97e243e74c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:27.036296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57381652-f4a9-40ce-85d1-1a97e243e74c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:27.041140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec8284-9c08-4e56-aa46-1e7f38dc40a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:27.041760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec8284-9c08-4e56-aa46-1e7f38dc40a8 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:27.042123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec8284-9c08-4e56-aa46-1e7f38dc40a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:27.042387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec8284-9c08-4e56-aa46-1e7f38dc40a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:27.043048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec8284-9c08-4e56-aa46-1e7f38dc40a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:27.044707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec8284-9c08-4e56-aa46-1e7f38dc40a8 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:27.059176 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4115/8142] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:27:27.060227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec8284-9c08-4e56-aa46-1e7f38dc40a8 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:27.071394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b038-c776-4f49-82e6-d8679f69d04d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:27.071660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec8284-9c08-4e56-aa46-1e7f38dc40a8 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:27.076301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20049244-b79d-4f66-ac23-73fdea38e414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:27.077007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20049244-b79d-4f66-ac23-73fdea38e414 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:27.077274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20049244-b79d-4f66-ac23-73fdea38e414 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:27.077535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20049244-b79d-4f66-ac23-73fdea38e414 admin admin] PATH_INFO: `/v3/projects/f9478f3888bc49dca896c62384001498/users/86bb1c3fc61a4e67a8273553358b181c/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:27.077652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56abc6-5a9e-41e6-a745-36da7d59f4f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:27.078352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56abc6-5a9e-41e6-a745-36da7d59f4f5 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:27.078690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56abc6-5a9e-41e6-a745-36da7d59f4f5 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:27.079036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56abc6-5a9e-41e6-a745-36da7d59f4f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:27.088701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56abc6-5a9e-41e6-a745-36da7d59f4f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:27.090377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56abc6-5a9e-41e6-a745-36da7d59f4f5 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:27.092975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20049244-b79d-4f66-ac23-73fdea38e414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9478f3888bc49dca896c62384001498, user_id=86bb1c3fc61a4e67a8273553358b181c, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:27.097114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20049244-b79d-4f66-ac23-73fdea38e414 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:27.100443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:27.100651 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4028/8143] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:27:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:27:27.115335 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4116/8144] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:27 2022] PUT /identity/v3/projects/f9478f3888bc49dca896c62384001498/users/86bb1c3fc61a4e67a8273553358b181c/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:27.120559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6fab1-8661-4896-a4dd-90413f3b6dab None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:27.120910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6fab1-8661-4896-a4dd-90413f3b6dab None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:27.121263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6fab1-8661-4896-a4dd-90413f3b6dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:27.141914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-24b6fab1-8661-4896-a4dd-90413f3b6dab None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:27.143114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-24b6fab1-8661-4896-a4dd-90413f3b6dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:27.236196 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4029/8145] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:27.246312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20049244-b79d-4f66-ac23-73fdea38e414 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:27.248425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-20049244-b79d-4f66-ac23-73fdea38e414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:27.372415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20049244-b79d-4f66-ac23-73fdea38e414 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:27.385481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74196321-17d4-4db2-8f52-01b3e870cc2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:27.386213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74196321-17d4-4db2-8f52-01b3e870cc2f service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:27.388417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74196321-17d4-4db2-8f52-01b3e870cc2f service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:27.388417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74196321-17d4-4db2-8f52-01b3e870cc2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:27.412664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74196321-17d4-4db2-8f52-01b3e870cc2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:27.416937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74196321-17d4-4db2-8f52-01b3e870cc2f service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:27.431269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:27.458213 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4117/8146] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:28.281891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a054da-11d0-499c-a9dd-9d9bef23eccb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:28.282745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a054da-11d0-499c-a9dd-9d9bef23eccb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:28.282745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a054da-11d0-499c-a9dd-9d9bef23eccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:28.314857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-26a054da-11d0-499c-a9dd-9d9bef23eccb None None] MFA Rules not processed for user `86bb1c3fc61a4e67a8273553358b181c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:28.316642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-26a054da-11d0-499c-a9dd-9d9bef23eccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:28.405018 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4030/8147] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:28.406844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74196321-17d4-4db2-8f52-01b3e870cc2f service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:28.419538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74196321-17d4-4db2-8f52-01b3e870cc2f service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:28.424908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760063f-1231-4e82-8675-d38bd501277c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:28.425825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760063f-1231-4e82-8675-d38bd501277c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:28.426216 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760063f-1231-4e82-8675-d38bd501277c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:28.426834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760063f-1231-4e82-8675-d38bd501277c admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:28.428515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760063f-1231-4e82-8675-d38bd501277c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:28.430899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760063f-1231-4e82-8675-d38bd501277c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:28.473707 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4118/8148] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:28 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:28.474522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a054da-11d0-499c-a9dd-9d9bef23eccb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:28.486649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a054da-11d0-499c-a9dd-9d9bef23eccb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:28.565645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162f148-3460-471e-9d0e-6b4f8ea87612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:28.566531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f148-3460-471e-9d0e-6b4f8ea87612 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:28.567263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f148-3460-471e-9d0e-6b4f8ea87612 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:28.567542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f148-3460-471e-9d0e-6b4f8ea87612 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:28.568260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162f148-3460-471e-9d0e-6b4f8ea87612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:28.570433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162f148-3460-471e-9d0e-6b4f8ea87612 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:28.599392 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4031/8149] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:28 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:27:28.601524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760063f-1231-4e82-8675-d38bd501277c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:28.612245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760063f-1231-4e82-8675-d38bd501277c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:28.621936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24488557-74e0-400f-8bee-8e8131174a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:28.622567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24488557-74e0-400f-8bee-8e8131174a67 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:28.622888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24488557-74e0-400f-8bee-8e8131174a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:28.623114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24488557-74e0-400f-8bee-8e8131174a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:28.628868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24488557-74e0-400f-8bee-8e8131174a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:28.631209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24488557-74e0-400f-8bee-8e8131174a67 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:28.647828 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4119/8150] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:28.652000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162f148-3460-471e-9d0e-6b4f8ea87612 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:28.662733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162f148-3460-471e-9d0e-6b4f8ea87612 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:28.667919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf6c23-eb59-4e25-8c82-a920b2fff744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:28.668647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf6c23-eb59-4e25-8c82-a920b2fff744 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:28.668924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf6c23-eb59-4e25-8c82-a920b2fff744 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:28.669344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf6c23-eb59-4e25-8c82-a920b2fff744 admin admin] PATH_INFO: `/v3/projects/4b14efcdff5540b7a1c0321335522e97/users/c8d9df8245b64813b72b18b44ce2a129/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:28.705165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf6c23-eb59-4e25-8c82-a920b2fff744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b14efcdff5540b7a1c0321335522e97, user_id=c8d9df8245b64813b72b18b44ce2a129, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:28.707941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf6c23-eb59-4e25-8c82-a920b2fff744 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:28.711337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:28.722748 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4032/8151] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:28 2022] PUT /identity/v3/projects/4b14efcdff5540b7a1c0321335522e97/users/c8d9df8245b64813b72b18b44ce2a129/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:27:28.728072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24488557-74e0-400f-8bee-8e8131174a67 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:28.744783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24488557-74e0-400f-8bee-8e8131174a67 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:28.827486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da77ba-1b88-422f-9663-386248ae964e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:28.828002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da77ba-1b88-422f-9663-386248ae964e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:28.828276 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da77ba-1b88-422f-9663-386248ae964e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:28.828581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da77ba-1b88-422f-9663-386248ae964e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:28.829220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da77ba-1b88-422f-9663-386248ae964e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:28.830830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da77ba-1b88-422f-9663-386248ae964e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:28.842572 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4120/8152] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:28.846755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf6c23-eb59-4e25-8c82-a920b2fff744 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:28.858612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf6c23-eb59-4e25-8c82-a920b2fff744 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:28.864410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c5956-d2f7-459e-8866-5509d24182d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:28.865393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c5956-d2f7-459e-8866-5509d24182d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:28.865753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c5956-d2f7-459e-8866-5509d24182d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:28.866088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c5956-d2f7-459e-8866-5509d24182d6 admin admin] PATH_INFO: `/v3/projects/4b14efcdff5540b7a1c0321335522e97/users/c8d9df8245b64813b72b18b44ce2a129/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:28.868252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c5956-d2f7-459e-8866-5509d24182d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b14efcdff5540b7a1c0321335522e97, user_id=c8d9df8245b64813b72b18b44ce2a129, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:28.871070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c5956-d2f7-459e-8866-5509d24182d6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:28.883646 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4033/8153] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:28 2022] PUT /identity/v3/projects/4b14efcdff5540b7a1c0321335522e97/users/c8d9df8245b64813b72b18b44ce2a129/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:29.684742 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421acdb-9d2f-4bcc-9357-c199899f5fdf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:29.685440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421acdb-9d2f-4bcc-9357-c199899f5fdf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:29.685440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421acdb-9d2f-4bcc-9357-c199899f5fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:29.717492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f421acdb-9d2f-4bcc-9357-c199899f5fdf None None] MFA Rules not processed for user `c8d9df8245b64813b72b18b44ce2a129`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:29.718730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f421acdb-9d2f-4bcc-9357-c199899f5fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:29.822850 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4121/8154] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:29.835670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdca0bd-f61d-47cb-a030-d32880a60f92 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:29.836267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdca0bd-f61d-47cb-a030-d32880a60f92 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:29.836267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdca0bd-f61d-47cb-a030-d32880a60f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:29.867469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9bdca0bd-f61d-47cb-a030-d32880a60f92 None None] MFA Rules not processed for user `86bb1c3fc61a4e67a8273553358b181c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:29.868671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9bdca0bd-f61d-47cb-a030-d32880a60f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:29.959799 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4034/8155] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:29.971538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421acdb-9d2f-4bcc-9357-c199899f5fdf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:29.973284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f421acdb-9d2f-4bcc-9357-c199899f5fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:30.064738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:30.093875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421acdb-9d2f-4bcc-9357-c199899f5fdf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:30.099396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542cd26-fcce-46fc-870b-ca6445707223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:30.099999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542cd26-fcce-46fc-870b-ca6445707223 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:30.100265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542cd26-fcce-46fc-870b-ca6445707223 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:30.100557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542cd26-fcce-46fc-870b-ca6445707223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:30.107990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542cd26-fcce-46fc-870b-ca6445707223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:30.109604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542cd26-fcce-46fc-870b-ca6445707223 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:30.118455 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4122/8156] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:34.701294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdca0bd-f61d-47cb-a030-d32880a60f92 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:34.726957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdca0bd-f61d-47cb-a030-d32880a60f92 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:34.736160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde166-f95d-452d-8709-326141dc1ca2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:34.736826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde166-f95d-452d-8709-326141dc1ca2 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:34.737103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde166-f95d-452d-8709-326141dc1ca2 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:34.740743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde166-f95d-452d-8709-326141dc1ca2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:34.742687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cbbde166-f95d-452d-8709-326141dc1ca2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:34.758992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbde166-f95d-452d-8709-326141dc1ca2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:34.760608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbde166-f95d-452d-8709-326141dc1ca2 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:34.774515 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4035/8157] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:39.778253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542cd26-fcce-46fc-870b-ca6445707223 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:39.779894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e542cd26-fcce-46fc-870b-ca6445707223 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:39.865896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542cd26-fcce-46fc-870b-ca6445707223 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:39.872072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0e712-9e3c-450c-a41e-d2056119499e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:39.872723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0e712-9e3c-450c-a41e-d2056119499e service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:39.872972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0e712-9e3c-450c-a41e-d2056119499e service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:39.873273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0e712-9e3c-450c-a41e-d2056119499e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:39.876043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b8b0e712-9e3c-450c-a41e-d2056119499e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:39.892099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0e712-9e3c-450c-a41e-d2056119499e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:39.893737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0e712-9e3c-450c-a41e-d2056119499e service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:39.906581 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4123/8158] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:40.436829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde166-f95d-452d-8709-326141dc1ca2 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:40.438364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cbbde166-f95d-452d-8709-326141dc1ca2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:40.457554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde166-f95d-452d-8709-326141dc1ca2 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:40.463424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b0f70-fd4f-4d0f-88af-5a1bd1d27389 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:40.464020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b0f70-fd4f-4d0f-88af-5a1bd1d27389 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:40.464294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b0f70-fd4f-4d0f-88af-5a1bd1d27389 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:40.464619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b0f70-fd4f-4d0f-88af-5a1bd1d27389 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:40.466444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-501b0f70-fd4f-4d0f-88af-5a1bd1d27389 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:40.480655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b0f70-fd4f-4d0f-88af-5a1bd1d27389 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:40.482235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b0f70-fd4f-4d0f-88af-5a1bd1d27389 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:40.495364 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4036/8159] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:48.628510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e06c4-9234-4f66-beba-97f3642049ce None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:48.629067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e06c4-9234-4f66-beba-97f3642049ce None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:48.629067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e06c4-9234-4f66-beba-97f3642049ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:48.649376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ff9e06c4-9234-4f66-beba-97f3642049ce None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:48.650685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ff9e06c4-9234-4f66-beba-97f3642049ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:48.707013 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4124/8160] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:27:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:48.719700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b0f70-fd4f-4d0f-88af-5a1bd1d27389 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:48.790646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b0f70-fd4f-4d0f-88af-5a1bd1d27389 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:48.796155 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d06dc-f1b4-483d-b2b0-7e7e344a1bdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:48.796788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d06dc-f1b4-483d-b2b0-7e7e344a1bdf service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:48.797061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d06dc-f1b4-483d-b2b0-7e7e344a1bdf service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:48.797324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d06dc-f1b4-483d-b2b0-7e7e344a1bdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:48.806093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d06dc-f1b4-483d-b2b0-7e7e344a1bdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:48.807709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d06dc-f1b4-483d-b2b0-7e7e344a1bdf service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:48.815897 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4037/8161] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:27:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:27:57.040213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e06c4-9234-4f66-beba-97f3642049ce None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:57.113535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:57.118119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e06c4-9234-4f66-beba-97f3642049ce None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:57.123415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:57.124100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:57.124405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:57.124702 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] PATH_INFO: `/v3/users/86bb1c3fc61a4e67a8273553358b181c` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:57.126010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86bb1c3fc61a4e67a8273553358b181c)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:57.128085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:57.180179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86bb1c3fc61a4e67a8273553358b181c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.189880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86bb1c3fc61a4e67a8273553358b181c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.198164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86bb1c3fc61a4e67a8273553358b181c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.211471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86bb1c3fc61a4e67a8273553358b181c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.212307 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86bb1c3fc61a4e67a8273553358b181c'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.219302 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4125/8162] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:57 2022] DELETE /identity/v3/users/86bb1c3fc61a4e67a8273553358b181c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:57.443262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d06dc-f1b4-483d-b2b0-7e7e344a1bdf service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:57.460913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d06dc-f1b4-483d-b2b0-7e7e344a1bdf service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:57.516650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:57.528277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:57.528956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:57.529412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:57.529709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] PATH_INFO: `/v3/projects/f9478f3888bc49dca896c62384001498` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:57.530977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9478f3888bc49dca896c62384001498)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:57.532981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:57.585230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9478f3888bc49dca896c62384001498', 'request_id': 'req-e609cab7-e235-437e-911d-87197c646d75'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.591518 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9478f3888bc49dca896c62384001498', 'request_id': 'req-e609cab7-e235-437e-911d-87197c646d75'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.599651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9478f3888bc49dca896c62384001498', 'request_id': 'req-e609cab7-e235-437e-911d-87197c646d75'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.610905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] The token cache is being invalidate because project f9478f3888bc49dca896c62384001498 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:57.611311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.611963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.613833 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4038/8163] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:57 2022] DELETE /identity/v3/projects/f9478f3888bc49dca896c62384001498 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:57.617911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:57.619332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:57.632483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f974f-b010-48b5-b52c-7542977d5ff0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:57.694611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90610d58-46d9-4f94-8e74-dbe8e6241c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:57.695273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90610d58-46d9-4f94-8e74-dbe8e6241c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:57.695546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90610d58-46d9-4f94-8e74-dbe8e6241c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:57.695814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90610d58-46d9-4f94-8e74-dbe8e6241c8e admin admin] PATH_INFO: `/v3/users/c8d9df8245b64813b72b18b44ce2a129` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:57.697019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90610d58-46d9-4f94-8e74-dbe8e6241c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d9df8245b64813b72b18b44ce2a129)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:57.698966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90610d58-46d9-4f94-8e74-dbe8e6241c8e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:57.749169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90610d58-46d9-4f94-8e74-dbe8e6241c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d9df8245b64813b72b18b44ce2a129'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.758166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90610d58-46d9-4f94-8e74-dbe8e6241c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d9df8245b64813b72b18b44ce2a129'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.766529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90610d58-46d9-4f94-8e74-dbe8e6241c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d9df8245b64813b72b18b44ce2a129'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.778550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:57.780045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90610d58-46d9-4f94-8e74-dbe8e6241c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d9df8245b64813b72b18b44ce2a129'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.780789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-90610d58-46d9-4f94-8e74-dbe8e6241c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d9df8245b64813b72b18b44ce2a129'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:57.788601 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4126/8164] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:57 2022] DELETE /identity/v3/users/c8d9df8245b64813b72b18b44ce2a129 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:58.071408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:58.073043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:58.087164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609cab7-e235-437e-911d-87197c646d75 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:58.173183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:58.174015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:58.174385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:58.174660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b admin admin] PATH_INFO: `/v3/projects/4b14efcdff5540b7a1c0321335522e97` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:58.176315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b14efcdff5540b7a1c0321335522e97)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:58.178361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:58.243013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b14efcdff5540b7a1c0321335522e97', 'request_id': 'req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:58.262744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b14efcdff5540b7a1c0321335522e97', 'request_id': 'req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:58.270500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b14efcdff5540b7a1c0321335522e97', 'request_id': 'req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:58.286984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b admin admin] The token cache is being invalidate because project 4b14efcdff5540b7a1c0321335522e97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:27:58.286984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:58.288910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-458baa43-a0cc-48d9-98b0-8b86f0e71a3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:58.294826 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4039/8165] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:58 2022] DELETE /identity/v3/projects/4b14efcdff5540b7a1c0321335522e97 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:58.299875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4f91f-bd4a-4c01-8fd1-0e9fd448fcf2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:58.301189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4f91f-bd4a-4c01-8fd1-0e9fd448fcf2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:58.301780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4f91f-bd4a-4c01-8fd1-0e9fd448fcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:58.324402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-90d4f91f-bd4a-4c01-8fd1-0e9fd448fcf2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:58.325664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-90d4f91f-bd4a-4c01-8fd1-0e9fd448fcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:58.408078 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4127/8166] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:58.416218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7f338-834a-4f40-8726-9a2731f77418 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:58.416533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7f338-834a-4f40-8726-9a2731f77418 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:58.416806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7f338-834a-4f40-8726-9a2731f77418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:58.440226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1ac7f338-834a-4f40-8726-9a2731f77418 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:58.442523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1ac7f338-834a-4f40-8726-9a2731f77418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:58.453084 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4040/8167] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:58.457652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4f91f-bd4a-4c01-8fd1-0e9fd448fcf2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:58.469841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4f91f-bd4a-4c01-8fd1-0e9fd448fcf2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:58.476199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b4cd9-8c5f-42d0-9a42-db6bd4df2c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:58.477108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4cd9-8c5f-42d0-9a42-db6bd4df2c7d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:58.477559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4cd9-8c5f-42d0-9a42-db6bd4df2c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:58.477820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4cd9-8c5f-42d0-9a42-db6bd4df2c7d admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:58.478665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b4cd9-8c5f-42d0-9a42-db6bd4df2c7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:27:58.479282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b4cd9-8c5f-42d0-9a42-db6bd4df2c7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:58.481223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b4cd9-8c5f-42d0-9a42-db6bd4df2c7d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:58.496498 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4128/8168] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:27:58.500700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7f338-834a-4f40-8726-9a2731f77418 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:58.508021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:58.512638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7f338-834a-4f40-8726-9a2731f77418 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:58.522149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6600e9-c314-4d91-a7ae-4aa8700a61ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:58.523061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6600e9-c314-4d91-a7ae-4aa8700a61ff admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:58.523829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6600e9-c314-4d91-a7ae-4aa8700a61ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:58.526100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6600e9-c314-4d91-a7ae-4aa8700a61ff admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:58.527070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6600e9-c314-4d91-a7ae-4aa8700a61ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:58.530017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6600e9-c314-4d91-a7ae-4aa8700a61ff admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:58.568412 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4041/8169] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:27:58 2022] POST /identity/v3/projects => generated 359 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:27:58.570379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b4cd9-8c5f-42d0-9a42-db6bd4df2c7d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:58.596683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b4cd9-8c5f-42d0-9a42-db6bd4df2c7d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:58.715347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bad05-8e64-40f4-9a19-40d5078771bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:58.716105 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bad05-8e64-40f4-9a19-40d5078771bc admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:58.716481 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bad05-8e64-40f4-9a19-40d5078771bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:58.716872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bad05-8e64-40f4-9a19-40d5078771bc admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:58.717598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47bad05-8e64-40f4-9a19-40d5078771bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:58.720401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47bad05-8e64-40f4-9a19-40d5078771bc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:58.753879 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4129/8170] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:27:58 2022] POST /identity/v3/users => generated 280 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:27:58.756586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6600e9-c314-4d91-a7ae-4aa8700a61ff admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:58.771899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6600e9-c314-4d91-a7ae-4aa8700a61ff admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:58.777943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45f17b-8ad0-47e8-95c3-a6fd30839752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:58.778617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45f17b-8ad0-47e8-95c3-a6fd30839752 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:58.778896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45f17b-8ad0-47e8-95c3-a6fd30839752 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:58.779164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45f17b-8ad0-47e8-95c3-a6fd30839752 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:58.779810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45f17b-8ad0-47e8-95c3-a6fd30839752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:58.781668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45f17b-8ad0-47e8-95c3-a6fd30839752 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:58.796618 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4042/8171] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:27:58.800962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bad05-8e64-40f4-9a19-40d5078771bc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:58.823410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bad05-8e64-40f4-9a19-40d5078771bc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:58.828181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdadb50-9967-4adc-8537-ce3629a07831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:58.828864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdadb50-9967-4adc-8537-ce3629a07831 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:58.829122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdadb50-9967-4adc-8537-ce3629a07831 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:58.829381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdadb50-9967-4adc-8537-ce3629a07831 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:58.830030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdadb50-9967-4adc-8537-ce3629a07831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:58.831686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdadb50-9967-4adc-8537-ce3629a07831 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:58.844504 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4130/8172] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:27:58.848954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45f17b-8ad0-47e8-95c3-a6fd30839752 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:58.860042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45f17b-8ad0-47e8-95c3-a6fd30839752 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:58.865173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf808455-82ea-433d-b40d-09df0aa3c5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:58.865850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf808455-82ea-433d-b40d-09df0aa3c5c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:58.866124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf808455-82ea-433d-b40d-09df0aa3c5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:58.866399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf808455-82ea-433d-b40d-09df0aa3c5c8 admin admin] PATH_INFO: `/v3/projects/2e8426db027b4c988f7d148d264ca46b/users/b815d1e0dc5a49e1bdda0353e68b8cb2/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:58.883795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf808455-82ea-433d-b40d-09df0aa3c5c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e8426db027b4c988f7d148d264ca46b, user_id=b815d1e0dc5a49e1bdda0353e68b8cb2, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:58.886347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf808455-82ea-433d-b40d-09df0aa3c5c8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:58.901234 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4043/8173] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:27:58 2022] PUT /identity/v3/projects/2e8426db027b4c988f7d148d264ca46b/users/b815d1e0dc5a49e1bdda0353e68b8cb2/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:58.913372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd57c36-a00e-40d8-a370-b86b628653d4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:58.913769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd57c36-a00e-40d8-a370-b86b628653d4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:58.914652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd57c36-a00e-40d8-a370-b86b628653d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:58.948312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7dd57c36-a00e-40d8-a370-b86b628653d4 None None] MFA Rules not processed for user `b815d1e0dc5a49e1bdda0353e68b8cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:58.949688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7dd57c36-a00e-40d8-a370-b86b628653d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:59.020570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:27:59.041494 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4131/8174] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:59.078717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf808455-82ea-433d-b40d-09df0aa3c5c8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:59.081395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cf808455-82ea-433d-b40d-09df0aa3c5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:59.157257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf808455-82ea-433d-b40d-09df0aa3c5c8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:59.166500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089d902-8ff9-4f0f-a91f-c29bd1edd7ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:59.167128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089d902-8ff9-4f0f-a91f-c29bd1edd7ed service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:59.167404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089d902-8ff9-4f0f-a91f-c29bd1edd7ed service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:59.167722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089d902-8ff9-4f0f-a91f-c29bd1edd7ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:59.180607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089d902-8ff9-4f0f-a91f-c29bd1edd7ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:59.182113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089d902-8ff9-4f0f-a91f-c29bd1edd7ed service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:59.194968 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4044/8175] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:59 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:59.209993 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd57c36-a00e-40d8-a370-b86b628653d4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:59.300858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd57c36-a00e-40d8-a370-b86b628653d4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:59.305530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:59.308725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:59.311536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:59.311833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] PATH_INFO: `/v3/users/b815d1e0dc5a49e1bdda0353e68b8cb2` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:59.313287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b815d1e0dc5a49e1bdda0353e68b8cb2)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:59.331233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:59.385648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b815d1e0dc5a49e1bdda0353e68b8cb2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:59.395058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b815d1e0dc5a49e1bdda0353e68b8cb2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:59.402710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b815d1e0dc5a49e1bdda0353e68b8cb2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:59.415200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b815d1e0dc5a49e1bdda0353e68b8cb2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:59.416091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b815d1e0dc5a49e1bdda0353e68b8cb2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:27:59.422383 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4132/8176] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:27:59 2022] DELETE /identity/v3/users/b815d1e0dc5a49e1bdda0353e68b8cb2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:27:59.427114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcc526-d53e-4cba-a914-3b8e8895e99f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:59.427459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcc526-d53e-4cba-a914-3b8e8895e99f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:59.427797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcc526-d53e-4cba-a914-3b8e8895e99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:59.449084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-17fcc526-d53e-4cba-a914-3b8e8895e99f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:27:59.450380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-17fcc526-d53e-4cba-a914-3b8e8895e99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:59.511292 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4045/8177] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:27:59.528595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:59.530131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:59.592707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c49e1-6e7e-4cb3-b846-a168f9074247 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:59.602838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b37fdf-736e-4ab2-bb6d-c8f2f5148633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:59.603459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b37fdf-736e-4ab2-bb6d-c8f2f5148633 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:59.603704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b37fdf-736e-4ab2-bb6d-c8f2f5148633 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:59.603998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b37fdf-736e-4ab2-bb6d-c8f2f5148633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:59.614543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b37fdf-736e-4ab2-bb6d-c8f2f5148633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:59.616703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b37fdf-736e-4ab2-bb6d-c8f2f5148633 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:27:59.629488 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4133/8178] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:27:59.931672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fcc526-d53e-4cba-a914-3b8e8895e99f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:27:59.933121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-17fcc526-d53e-4cba-a914-3b8e8895e99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:27:59.957109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fcc526-d53e-4cba-a914-3b8e8895e99f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:27:59.963003 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9350eca-77d6-42a7-93ef-db3b9cd5859b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:27:59.963628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9350eca-77d6-42a7-93ef-db3b9cd5859b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:27:59.963902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9350eca-77d6-42a7-93ef-db3b9cd5859b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:27:59.964651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9350eca-77d6-42a7-93ef-db3b9cd5859b admin admin] PATH_INFO: `/v3/projects/2e8426db027b4c988f7d148d264ca46b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:27:59.966451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9350eca-77d6-42a7-93ef-db3b9cd5859b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e8426db027b4c988f7d148d264ca46b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:27:59.968272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9350eca-77d6-42a7-93ef-db3b9cd5859b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:00.023550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:00.027046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9350eca-77d6-42a7-93ef-db3b9cd5859b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e8426db027b4c988f7d148d264ca46b', 'request_id': 'req-f9350eca-77d6-42a7-93ef-db3b9cd5859b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:00.034890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9350eca-77d6-42a7-93ef-db3b9cd5859b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e8426db027b4c988f7d148d264ca46b', 'request_id': 'req-f9350eca-77d6-42a7-93ef-db3b9cd5859b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:00.046865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9350eca-77d6-42a7-93ef-db3b9cd5859b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e8426db027b4c988f7d148d264ca46b', 'request_id': 'req-f9350eca-77d6-42a7-93ef-db3b9cd5859b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:00.061902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9350eca-77d6-42a7-93ef-db3b9cd5859b admin admin] The token cache is being invalidate because project 2e8426db027b4c988f7d148d264ca46b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:00.062182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9350eca-77d6-42a7-93ef-db3b9cd5859b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:00.062831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9350eca-77d6-42a7-93ef-db3b9cd5859b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:00.069527 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4046/8179] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:27:59 2022] DELETE /identity/v3/projects/2e8426db027b4c988f7d148d264ca46b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:00.070510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c8a38-72a0-4824-b4c3-dff8e5f9d09f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:00.070787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c8a38-72a0-4824-b4c3-dff8e5f9d09f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:00.071054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c8a38-72a0-4824-b4c3-dff8e5f9d09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:00.093350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3a8c8a38-72a0-4824-b4c3-dff8e5f9d09f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:00.094946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3a8c8a38-72a0-4824-b4c3-dff8e5f9d09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:00.105586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:00.187590 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4134/8180] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:28:00.196338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c774d-4451-46e3-a705-544741610a64 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:00.196728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c774d-4451-46e3-a705-544741610a64 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:00.196994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c774d-4451-46e3-a705-544741610a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:00.217636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cc3c774d-4451-46e3-a705-544741610a64 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:00.218835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cc3c774d-4451-46e3-a705-544741610a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:00.234096 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4047/8181] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:28:00.238589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c8a38-72a0-4824-b4c3-dff8e5f9d09f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:00.248560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c8a38-72a0-4824-b4c3-dff8e5f9d09f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:00.253698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5a7c2-fa8b-4553-a9fd-bd58898334c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:00.254398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5a7c2-fa8b-4553-a9fd-bd58898334c0 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:00.254666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5a7c2-fa8b-4553-a9fd-bd58898334c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:00.254927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5a7c2-fa8b-4553-a9fd-bd58898334c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:00.255594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5a7c2-fa8b-4553-a9fd-bd58898334c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:00.255890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5a7c2-fa8b-4553-a9fd-bd58898334c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:00.257607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5a7c2-fa8b-4553-a9fd-bd58898334c0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:00.273109 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4135/8182] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:28:00.274212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c774d-4451-46e3-a705-544741610a64 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:00.283617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c774d-4451-46e3-a705-544741610a64 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:00.288873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d91ad-bc11-4c55-8099-292d12b077c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:00.289475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d91ad-bc11-4c55-8099-292d12b077c0 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:00.289768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d91ad-bc11-4c55-8099-292d12b077c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:00.290036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d91ad-bc11-4c55-8099-292d12b077c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:00.290659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d91ad-bc11-4c55-8099-292d12b077c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:00.292628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d91ad-bc11-4c55-8099-292d12b077c0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:00.323971 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4048/8183] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:00 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:28:00.327877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5a7c2-fa8b-4553-a9fd-bd58898334c0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:00.338650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5a7c2-fa8b-4553-a9fd-bd58898334c0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:00.401947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a82cb9-c6f8-468a-94b6-9f3accf20450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:00.402557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a82cb9-c6f8-468a-94b6-9f3accf20450 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:00.402833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a82cb9-c6f8-468a-94b6-9f3accf20450 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:00.403101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a82cb9-c6f8-468a-94b6-9f3accf20450 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:00.403717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a82cb9-c6f8-468a-94b6-9f3accf20450 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:00.405845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a82cb9-c6f8-468a-94b6-9f3accf20450 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:00.431230 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4136/8184] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:00 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:00.436857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d91ad-bc11-4c55-8099-292d12b077c0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:00.448287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d91ad-bc11-4c55-8099-292d12b077c0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:00.455807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ec45c-d991-4e07-9483-f10d60346335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:00.457198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ec45c-d991-4e07-9483-f10d60346335 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:00.457476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ec45c-d991-4e07-9483-f10d60346335 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:00.457747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ec45c-d991-4e07-9483-f10d60346335 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:00.458407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ec45c-d991-4e07-9483-f10d60346335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:00.460051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ec45c-d991-4e07-9483-f10d60346335 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:00.470511 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4049/8185] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:28:00.474617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a82cb9-c6f8-468a-94b6-9f3accf20450 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:00.485212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a82cb9-c6f8-468a-94b6-9f3accf20450 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:00.490659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039cb34-fecb-47bb-a66f-1ee79336dd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:00.491252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039cb34-fecb-47bb-a66f-1ee79336dd21 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:00.491526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039cb34-fecb-47bb-a66f-1ee79336dd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:00.491789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039cb34-fecb-47bb-a66f-1ee79336dd21 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:00.492444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039cb34-fecb-47bb-a66f-1ee79336dd21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:00.494040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039cb34-fecb-47bb-a66f-1ee79336dd21 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:00.505207 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4137/8186] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:00.509466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ec45c-d991-4e07-9483-f10d60346335 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:00.519803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ec45c-d991-4e07-9483-f10d60346335 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:00.524635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bd710-aedd-4185-8d45-2cadaf1c4bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:00.525278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bd710-aedd-4185-8d45-2cadaf1c4bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:00.525553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bd710-aedd-4185-8d45-2cadaf1c4bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:00.525823 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bd710-aedd-4185-8d45-2cadaf1c4bdc admin admin] PATH_INFO: `/v3/projects/44245ad72a2c45f8aab1acbc8aa0e599/users/28fee42fddc7499ba2084dbeb67b1815/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:00.541295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bd710-aedd-4185-8d45-2cadaf1c4bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=44245ad72a2c45f8aab1acbc8aa0e599, user_id=28fee42fddc7499ba2084dbeb67b1815, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:00.544033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bd710-aedd-4185-8d45-2cadaf1c4bdc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:00.560362 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4050/8187] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:00 2022] PUT /identity/v3/projects/44245ad72a2c45f8aab1acbc8aa0e599/users/28fee42fddc7499ba2084dbeb67b1815/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:00.572549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e1f49-fa99-4c15-9cb1-3c95040a6dfe None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:00.572827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e1f49-fa99-4c15-9cb1-3c95040a6dfe None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:00.573092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e1f49-fa99-4c15-9cb1-3c95040a6dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:00.602593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b87e1f49-fa99-4c15-9cb1-3c95040a6dfe None None] MFA Rules not processed for user `28fee42fddc7499ba2084dbeb67b1815`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:00.603789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b87e1f49-fa99-4c15-9cb1-3c95040a6dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:00.676670 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4138/8188] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:00.688322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bd710-aedd-4185-8d45-2cadaf1c4bdc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:00.690089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2a6bd710-aedd-4185-8d45-2cadaf1c4bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:00.760729 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bd710-aedd-4185-8d45-2cadaf1c4bdc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:00.765549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35c594-8ac5-4835-a0bc-a4450194a7d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:00.766279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35c594-8ac5-4835-a0bc-a4450194a7d0 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:00.766560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35c594-8ac5-4835-a0bc-a4450194a7d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:00.766821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35c594-8ac5-4835-a0bc-a4450194a7d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:00.776235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35c594-8ac5-4835-a0bc-a4450194a7d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:00.777711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35c594-8ac5-4835-a0bc-a4450194a7d0 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:00.787318 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4051/8189] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:02.317881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e1f49-fa99-4c15-9cb1-3c95040a6dfe None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:02.328528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:02.392687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e1f49-fa99-4c15-9cb1-3c95040a6dfe None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:02.399297 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:02.399962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:02.400235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:02.400540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] PATH_INFO: `/v3/users/28fee42fddc7499ba2084dbeb67b1815` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:02.401837 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28fee42fddc7499ba2084dbeb67b1815)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:02.403804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:02.459057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28fee42fddc7499ba2084dbeb67b1815'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:02.472741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28fee42fddc7499ba2084dbeb67b1815'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:02.480919 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28fee42fddc7499ba2084dbeb67b1815'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:02.495268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28fee42fddc7499ba2084dbeb67b1815'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:02.496022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28fee42fddc7499ba2084dbeb67b1815'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:02.502520 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4139/8190] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:02 2022] DELETE /identity/v3/users/28fee42fddc7499ba2084dbeb67b1815 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:02.507440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff05c-5f52-4a12-bc9c-0670dcd36466 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:02.507999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff05c-5f52-4a12-bc9c-0670dcd36466 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:02.508235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff05c-5f52-4a12-bc9c-0670dcd36466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:02.530077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4c8ff05c-5f52-4a12-bc9c-0670dcd36466 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:02.531424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4c8ff05c-5f52-4a12-bc9c-0670dcd36466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:02.596473 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4052/8191] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:28:02.608045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:02.609628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:02.680914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56764953-0dcc-4362-992a-9788b620ce02 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:02.686053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e5912-d24b-489f-96c3-5aa48bbc82d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:02.686920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e5912-d24b-489f-96c3-5aa48bbc82d6 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:02.687474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e5912-d24b-489f-96c3-5aa48bbc82d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:02.687910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e5912-d24b-489f-96c3-5aa48bbc82d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:02.709328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e5912-d24b-489f-96c3-5aa48bbc82d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:02.714097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e5912-d24b-489f-96c3-5aa48bbc82d6 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:02.745188 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4140/8192] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:03.008928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ff05c-5f52-4a12-bc9c-0670dcd36466 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:03.010423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4c8ff05c-5f52-4a12-bc9c-0670dcd36466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:03.048944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:03.201370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ff05c-5f52-4a12-bc9c-0670dcd36466 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:03.211957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f391b2-5c52-4356-b999-cccbcadbdb8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:03.212974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f391b2-5c52-4356-b999-cccbcadbdb8c service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:03.213255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f391b2-5c52-4356-b999-cccbcadbdb8c service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:03.213540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f391b2-5c52-4356-b999-cccbcadbdb8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:03.215861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a0f391b2-5c52-4356-b999-cccbcadbdb8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:03.256962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f391b2-5c52-4356-b999-cccbcadbdb8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:03.258714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f391b2-5c52-4356-b999-cccbcadbdb8c service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:03.292963 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4053/8193] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:03.380381 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e5912-d24b-489f-96c3-5aa48bbc82d6 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:03.381865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-158e5912-d24b-489f-96c3-5aa48bbc82d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:03.395540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e5912-d24b-489f-96c3-5aa48bbc82d6 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:03.400088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13683d5-444a-4405-9ce0-db7b9ce31e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:03.400752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13683d5-444a-4405-9ce0-db7b9ce31e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:03.401026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13683d5-444a-4405-9ce0-db7b9ce31e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:03.401445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13683d5-444a-4405-9ce0-db7b9ce31e2f admin admin] PATH_INFO: `/v3/projects/44245ad72a2c45f8aab1acbc8aa0e599` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:03.403494 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13683d5-444a-4405-9ce0-db7b9ce31e2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=44245ad72a2c45f8aab1acbc8aa0e599)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:03.405631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13683d5-444a-4405-9ce0-db7b9ce31e2f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:03.436933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:03.456466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e13683d5-444a-4405-9ce0-db7b9ce31e2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44245ad72a2c45f8aab1acbc8aa0e599', 'request_id': 'req-e13683d5-444a-4405-9ce0-db7b9ce31e2f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:03.466672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e13683d5-444a-4405-9ce0-db7b9ce31e2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44245ad72a2c45f8aab1acbc8aa0e599', 'request_id': 'req-e13683d5-444a-4405-9ce0-db7b9ce31e2f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:03.472695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e13683d5-444a-4405-9ce0-db7b9ce31e2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44245ad72a2c45f8aab1acbc8aa0e599', 'request_id': 'req-e13683d5-444a-4405-9ce0-db7b9ce31e2f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:03.485592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e13683d5-444a-4405-9ce0-db7b9ce31e2f admin admin] The token cache is being invalidate because project 44245ad72a2c45f8aab1acbc8aa0e599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:03.485880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e13683d5-444a-4405-9ce0-db7b9ce31e2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:03.486538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e13683d5-444a-4405-9ce0-db7b9ce31e2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:03.488165 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4141/8194] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:03 2022] DELETE /identity/v3/projects/44245ad72a2c45f8aab1acbc8aa0e599 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:03.493531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2af1f-70f8-48ad-a844-b847bad460c3 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:03.493808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2af1f-70f8-48ad-a844-b847bad460c3 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:03.494062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2af1f-70f8-48ad-a844-b847bad460c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:03.514226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5ac2af1f-70f8-48ad-a844-b847bad460c3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:03.515431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5ac2af1f-70f8-48ad-a844-b847bad460c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:03.580476 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4054/8195] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:03.588953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb1d6a-5a07-44cf-a9da-9d67f497feb4 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:03.589240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb1d6a-5a07-44cf-a9da-9d67f497feb4 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:03.589482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb1d6a-5a07-44cf-a9da-9d67f497feb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:03.611964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-adfb1d6a-5a07-44cf-a9da-9d67f497feb4 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:03.614017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-adfb1d6a-5a07-44cf-a9da-9d67f497feb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:03.624460 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4142/8196] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:03.626767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2af1f-70f8-48ad-a844-b847bad460c3 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:03.637678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2af1f-70f8-48ad-a844-b847bad460c3 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:03.644064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf7212-22c9-4ac6-93d3-c95d6f143447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:03.644963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf7212-22c9-4ac6-93d3-c95d6f143447 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:03.645242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf7212-22c9-4ac6-93d3-c95d6f143447 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:03.645514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf7212-22c9-4ac6-93d3-c95d6f143447 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:03.646237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf7212-22c9-4ac6-93d3-c95d6f143447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:03.646526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf7212-22c9-4ac6-93d3-c95d6f143447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:03.648459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf7212-22c9-4ac6-93d3-c95d6f143447 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:03.662150 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4055/8197] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:28:03.666547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb1d6a-5a07-44cf-a9da-9d67f497feb4 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:03.679171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb1d6a-5a07-44cf-a9da-9d67f497feb4 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:03.685161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c24f2d-ee75-4f18-bb09-f85ccf273605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:03.686145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c24f2d-ee75-4f18-bb09-f85ccf273605 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:03.686736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c24f2d-ee75-4f18-bb09-f85ccf273605 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:03.687347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c24f2d-ee75-4f18-bb09-f85ccf273605 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:03.688078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c24f2d-ee75-4f18-bb09-f85ccf273605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:03.690385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c24f2d-ee75-4f18-bb09-f85ccf273605 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:03.728762 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4143/8198] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:03 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:03.730028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf7212-22c9-4ac6-93d3-c95d6f143447 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:03.741016 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf7212-22c9-4ac6-93d3-c95d6f143447 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:03.797964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315594d-6497-4a94-9bf5-88707d894ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:03.798572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315594d-6497-4a94-9bf5-88707d894ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:03.798846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315594d-6497-4a94-9bf5-88707d894ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:03.799110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315594d-6497-4a94-9bf5-88707d894ea3 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:03.799741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315594d-6497-4a94-9bf5-88707d894ea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:03.801906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315594d-6497-4a94-9bf5-88707d894ea3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:03.824506 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4056/8199] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:03 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:28:03.828718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c24f2d-ee75-4f18-bb09-f85ccf273605 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:03.839863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c24f2d-ee75-4f18-bb09-f85ccf273605 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:03.844748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6d744-ef93-4ff7-8803-423a48ca1e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:03.845335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6d744-ef93-4ff7-8803-423a48ca1e9d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:03.845604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6d744-ef93-4ff7-8803-423a48ca1e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:03.845883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6d744-ef93-4ff7-8803-423a48ca1e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:03.846525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6d744-ef93-4ff7-8803-423a48ca1e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:03.848178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6d744-ef93-4ff7-8803-423a48ca1e9d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:03.858386 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4144/8200] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:03.862497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315594d-6497-4a94-9bf5-88707d894ea3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:03.873018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315594d-6497-4a94-9bf5-88707d894ea3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:03.878091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcde7aa-cfeb-4226-9eab-9d50c894942f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:03.878688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcde7aa-cfeb-4226-9eab-9d50c894942f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:03.878964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcde7aa-cfeb-4226-9eab-9d50c894942f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:03.879234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcde7aa-cfeb-4226-9eab-9d50c894942f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:03.879873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcde7aa-cfeb-4226-9eab-9d50c894942f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:03.881561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcde7aa-cfeb-4226-9eab-9d50c894942f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:03.892537 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4057/8201] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:03.897247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6d744-ef93-4ff7-8803-423a48ca1e9d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:03.907616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6d744-ef93-4ff7-8803-423a48ca1e9d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:03.913063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663591ce-8245-4a82-95fa-de15e2363c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:03.913723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663591ce-8245-4a82-95fa-de15e2363c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:03.914008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663591ce-8245-4a82-95fa-de15e2363c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:03.914273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663591ce-8245-4a82-95fa-de15e2363c69 admin admin] PATH_INFO: `/v3/projects/99d89979721e427a9c99c7d436978ca4/users/7e7e4f85deee451bbfe697a09695df33/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:03.929807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663591ce-8245-4a82-95fa-de15e2363c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d89979721e427a9c99c7d436978ca4, user_id=7e7e4f85deee451bbfe697a09695df33, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:03.932522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663591ce-8245-4a82-95fa-de15e2363c69 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:03.948395 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4145/8202] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:03 2022] PUT /identity/v3/projects/99d89979721e427a9c99c7d436978ca4/users/7e7e4f85deee451bbfe697a09695df33/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:03.959104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db96e25-7bfd-483d-ae11-b9915daffa15 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:03.959386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db96e25-7bfd-483d-ae11-b9915daffa15 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:03.959661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db96e25-7bfd-483d-ae11-b9915daffa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:03.968511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:03.988690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8db96e25-7bfd-483d-ae11-b9915daffa15 None None] MFA Rules not processed for user `7e7e4f85deee451bbfe697a09695df33`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:03.989813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8db96e25-7bfd-483d-ae11-b9915daffa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:04.060783 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4058/8203] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:04.072556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663591ce-8245-4a82-95fa-de15e2363c69 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:04.074377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-663591ce-8245-4a82-95fa-de15e2363c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:04.149224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663591ce-8245-4a82-95fa-de15e2363c69 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:04.152924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde8f44-90f6-46da-ab53-27d42b98c6ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:04.153531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde8f44-90f6-46da-ab53-27d42b98c6ee service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:04.153805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde8f44-90f6-46da-ab53-27d42b98c6ee service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:04.154059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde8f44-90f6-46da-ab53-27d42b98c6ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:04.163345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde8f44-90f6-46da-ab53-27d42b98c6ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:04.164847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde8f44-90f6-46da-ab53-27d42b98c6ee service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:04.176631 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4146/8204] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:04.202357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db96e25-7bfd-483d-ae11-b9915daffa15 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:04.283228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db96e25-7bfd-483d-ae11-b9915daffa15 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:04.289134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:04.289936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:04.290216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:04.290893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] PATH_INFO: `/v3/users/7e7e4f85deee451bbfe697a09695df33` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:04.291699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e7e4f85deee451bbfe697a09695df33)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:04.293818 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:04.355593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e7e4f85deee451bbfe697a09695df33'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:04.363804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e7e4f85deee451bbfe697a09695df33'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:04.375915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e7e4f85deee451bbfe697a09695df33'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:04.376594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e7e4f85deee451bbfe697a09695df33'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:04.377190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e7e4f85deee451bbfe697a09695df33'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:04.389790 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4059/8205] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:04 2022] DELETE /identity/v3/users/7e7e4f85deee451bbfe697a09695df33 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:04.394580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c615cd-31e8-41fa-8e20-667df840b89b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:04.395127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c615cd-31e8-41fa-8e20-667df840b89b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:04.395127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c615cd-31e8-41fa-8e20-667df840b89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:04.414034 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a5c615cd-31e8-41fa-8e20-667df840b89b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:04.415282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a5c615cd-31e8-41fa-8e20-667df840b89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:04.466119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:04.481221 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4147/8206] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:04.494082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:04.495550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:04.510436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:04.560647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e038c-caf7-4429-b12d-1f4f7068af0c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:04.564546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2d665-c540-46a1-bc3c-0bd69484ea1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:04.565154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2d665-c540-46a1-bc3c-0bd69484ea1b service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:04.565477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2d665-c540-46a1-bc3c-0bd69484ea1b service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:04.565872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2d665-c540-46a1-bc3c-0bd69484ea1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:04.576174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2d665-c540-46a1-bc3c-0bd69484ea1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:04.577645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2d665-c540-46a1-bc3c-0bd69484ea1b service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:04.587770 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4060/8207] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:28:05.033159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c615cd-31e8-41fa-8e20-667df840b89b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:05.038135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a5c615cd-31e8-41fa-8e20-667df840b89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:05.047804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c615cd-31e8-41fa-8e20-667df840b89b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:05.054513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:05.055314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:05.055492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:05.055747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] PATH_INFO: `/v3/projects/99d89979721e427a9c99c7d436978ca4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:05.057002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99d89979721e427a9c99c7d436978ca4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:05.059770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:05.114145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99d89979721e427a9c99c7d436978ca4', 'request_id': 'req-70bf8c45-89d6-4887-922d-4fd1ade8b946'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:05.123856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99d89979721e427a9c99c7d436978ca4', 'request_id': 'req-70bf8c45-89d6-4887-922d-4fd1ade8b946'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:05.133444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99d89979721e427a9c99c7d436978ca4', 'request_id': 'req-70bf8c45-89d6-4887-922d-4fd1ade8b946'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:05.151635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] The token cache is being invalidate because project 99d89979721e427a9c99c7d436978ca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:05.152038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:05.153282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:05.157194 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4148/8208] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:05 2022] DELETE /identity/v3/projects/99d89979721e427a9c99c7d436978ca4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:07.623797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2d665-c540-46a1-bc3c-0bd69484ea1b service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:07.625399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9de2d665-c540-46a1-bc3c-0bd69484ea1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:07.639685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2d665-c540-46a1-bc3c-0bd69484ea1b service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:07.706303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f2028-9211-4e66-9715-d9e59ad19105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:07.707017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f2028-9211-4e66-9715-d9e59ad19105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:07.707199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f2028-9211-4e66-9715-d9e59ad19105 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:07.707467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f2028-9211-4e66-9715-d9e59ad19105 admin admin] PATH_INFO: `/v3/users/2bd3b3a2aad74b62aa8af337703f4527` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:07.708715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f2028-9211-4e66-9715-d9e59ad19105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd3b3a2aad74b62aa8af337703f4527)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:07.710663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f2028-9211-4e66-9715-d9e59ad19105 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:07.766300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-188f2028-9211-4e66-9715-d9e59ad19105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd3b3a2aad74b62aa8af337703f4527'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:07.775741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-188f2028-9211-4e66-9715-d9e59ad19105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd3b3a2aad74b62aa8af337703f4527'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:07.788110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-188f2028-9211-4e66-9715-d9e59ad19105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd3b3a2aad74b62aa8af337703f4527'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:07.788856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-188f2028-9211-4e66-9715-d9e59ad19105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd3b3a2aad74b62aa8af337703f4527'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:07.789501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-188f2028-9211-4e66-9715-d9e59ad19105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd3b3a2aad74b62aa8af337703f4527'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:07.802888 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4061/8209] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:07 2022] DELETE /identity/v3/users/2bd3b3a2aad74b62aa8af337703f4527 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:07.988286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:07.989819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:08.013915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf8c45-89d6-4887-922d-4fd1ade8b946 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:08.071464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47996a9e-06f5-4d28-bce5-3bd757b639e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:08.072121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47996a9e-06f5-4d28-bce5-3bd757b639e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:08.072461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47996a9e-06f5-4d28-bce5-3bd757b639e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:08.072731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47996a9e-06f5-4d28-bce5-3bd757b639e2 admin admin] PATH_INFO: `/v3/projects/0562065ffa3e4c27a03fd19b49c252eb` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:08.073827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47996a9e-06f5-4d28-bce5-3bd757b639e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0562065ffa3e4c27a03fd19b49c252eb)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:08.075752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47996a9e-06f5-4d28-bce5-3bd757b639e2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:08.125745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-47996a9e-06f5-4d28-bce5-3bd757b639e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0562065ffa3e4c27a03fd19b49c252eb', 'request_id': 'req-47996a9e-06f5-4d28-bce5-3bd757b639e2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:08.134325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-47996a9e-06f5-4d28-bce5-3bd757b639e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0562065ffa3e4c27a03fd19b49c252eb', 'request_id': 'req-47996a9e-06f5-4d28-bce5-3bd757b639e2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:08.140408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-47996a9e-06f5-4d28-bce5-3bd757b639e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0562065ffa3e4c27a03fd19b49c252eb', 'request_id': 'req-47996a9e-06f5-4d28-bce5-3bd757b639e2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:08.152247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-47996a9e-06f5-4d28-bce5-3bd757b639e2 admin admin] The token cache is being invalidate because project 0562065ffa3e4c27a03fd19b49c252eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:08.152550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-47996a9e-06f5-4d28-bce5-3bd757b639e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:08.153228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-47996a9e-06f5-4d28-bce5-3bd757b639e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:08.154780 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4149/8210] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:07 2022] DELETE /identity/v3/projects/0562065ffa3e4c27a03fd19b49c252eb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:08.159798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5c47-3f1e-430b-8866-a030c6c33f04 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:08.160171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5c47-3f1e-430b-8866-a030c6c33f04 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:08.160302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5c47-3f1e-430b-8866-a030c6c33f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:08.181143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0d5a5c47-3f1e-430b-8866-a030c6c33f04 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:08.182325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0d5a5c47-3f1e-430b-8866-a030c6c33f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:08.213549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:08.248581 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4062/8211] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:08.257699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fd742-ae92-4244-bda6-6226fc214edc None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:08.257977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fd742-ae92-4244-bda6-6226fc214edc None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:08.258250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fd742-ae92-4244-bda6-6226fc214edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:08.274988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:08.281122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e56fd742-ae92-4244-bda6-6226fc214edc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:08.283208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e56fd742-ae92-4244-bda6-6226fc214edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:08.292737 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4150/8212] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:08.297093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a5c47-3f1e-430b-8866-a030c6c33f04 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:08.307249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a5c47-3f1e-430b-8866-a030c6c33f04 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:08.313551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f4582-df0f-4ad1-80a8-e179cd652229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:08.314354 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f4582-df0f-4ad1-80a8-e179cd652229 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:08.314670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f4582-df0f-4ad1-80a8-e179cd652229 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:08.314980 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f4582-df0f-4ad1-80a8-e179cd652229 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:08.318272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f4582-df0f-4ad1-80a8-e179cd652229 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:08.318659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f4582-df0f-4ad1-80a8-e179cd652229 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:08.320749 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f4582-df0f-4ad1-80a8-e179cd652229 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:08.333370 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4063/8213] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:28:08.338128 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fd742-ae92-4244-bda6-6226fc214edc None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:08.347936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fd742-ae92-4244-bda6-6226fc214edc None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:08.353533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8921825-2f55-4684-93c0-b64cc93eb7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:08.354149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8921825-2f55-4684-93c0-b64cc93eb7ff admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:08.354444 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8921825-2f55-4684-93c0-b64cc93eb7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:08.354710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8921825-2f55-4684-93c0-b64cc93eb7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:08.355346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8921825-2f55-4684-93c0-b64cc93eb7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:08.357335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8921825-2f55-4684-93c0-b64cc93eb7ff admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:08.390011 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4151/8214] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:08 2022] POST /identity/v3/projects => generated 359 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:08.394591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f4582-df0f-4ad1-80a8-e179cd652229 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:08.405083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f4582-df0f-4ad1-80a8-e179cd652229 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:08.466469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318860f-c007-4a2d-918f-1438be8c6020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:08.467074 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318860f-c007-4a2d-918f-1438be8c6020 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:08.467351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318860f-c007-4a2d-918f-1438be8c6020 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:08.467622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318860f-c007-4a2d-918f-1438be8c6020 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:08.468237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318860f-c007-4a2d-918f-1438be8c6020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:08.470293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318860f-c007-4a2d-918f-1438be8c6020 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:08.492970 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4064/8215] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:08 2022] POST /identity/v3/users => generated 280 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:08.497247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8921825-2f55-4684-93c0-b64cc93eb7ff admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:08.508752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8921825-2f55-4684-93c0-b64cc93eb7ff admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:08.516230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af6a02-08dd-48e0-be64-e6fb7d7b1634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:08.517015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af6a02-08dd-48e0-be64-e6fb7d7b1634 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:08.517291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af6a02-08dd-48e0-be64-e6fb7d7b1634 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:08.517562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af6a02-08dd-48e0-be64-e6fb7d7b1634 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:08.519185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af6a02-08dd-48e0-be64-e6fb7d7b1634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:08.520081 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af6a02-08dd-48e0-be64-e6fb7d7b1634 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:08.531812 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4152/8216] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:08.539064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318860f-c007-4a2d-918f-1438be8c6020 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:08.553651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318860f-c007-4a2d-918f-1438be8c6020 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:08.559528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f266609-995b-4f87-95f8-06d232f91ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:08.560397 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f266609-995b-4f87-95f8-06d232f91ddc admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:08.560763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f266609-995b-4f87-95f8-06d232f91ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:08.561187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f266609-995b-4f87-95f8-06d232f91ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:08.562149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f266609-995b-4f87-95f8-06d232f91ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:08.564312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f266609-995b-4f87-95f8-06d232f91ddc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:08.576602 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4065/8217] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:08.580996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af6a02-08dd-48e0-be64-e6fb7d7b1634 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:08.593621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af6a02-08dd-48e0-be64-e6fb7d7b1634 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:08.599796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1843b4-873a-41f9-95d1-a5afa97ae7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:08.600468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1843b4-873a-41f9-95d1-a5afa97ae7e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:08.600749 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1843b4-873a-41f9-95d1-a5afa97ae7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:08.601016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1843b4-873a-41f9-95d1-a5afa97ae7e2 admin admin] PATH_INFO: `/v3/projects/8dcb0e4de2004f719a9110f524017c8d/users/cffdf662616a4b16a76a9a1afb8226fd/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:08.616066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1843b4-873a-41f9-95d1-a5afa97ae7e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dcb0e4de2004f719a9110f524017c8d, user_id=cffdf662616a4b16a76a9a1afb8226fd, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:08.618779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1843b4-873a-41f9-95d1-a5afa97ae7e2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:08.639871 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4153/8218] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:08 2022] PUT /identity/v3/projects/8dcb0e4de2004f719a9110f524017c8d/users/cffdf662616a4b16a76a9a1afb8226fd/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:08.651751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd019b-d54e-4e8f-b0af-dc725723859a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:08.652268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd019b-d54e-4e8f-b0af-dc725723859a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:08.652268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd019b-d54e-4e8f-b0af-dc725723859a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:08.681367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2edd019b-d54e-4e8f-b0af-dc725723859a None None] MFA Rules not processed for user `cffdf662616a4b16a76a9a1afb8226fd`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:08.682556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2edd019b-d54e-4e8f-b0af-dc725723859a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:08.752950 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4066/8219] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:08.792950 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1843b4-873a-41f9-95d1-a5afa97ae7e2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:08.859799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1843b4-873a-41f9-95d1-a5afa97ae7e2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:08.865739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:08.866371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:08.866649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:08.866937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] PATH_INFO: `/v3/users/cffdf662616a4b16a76a9a1afb8226fd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:08.868061 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cffdf662616a4b16a76a9a1afb8226fd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:08.870299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:08.919018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cffdf662616a4b16a76a9a1afb8226fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:08.927414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cffdf662616a4b16a76a9a1afb8226fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:08.938221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cffdf662616a4b16a76a9a1afb8226fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:08.956859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cffdf662616a4b16a76a9a1afb8226fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:08.957640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cffdf662616a4b16a76a9a1afb8226fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:08.964031 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4154/8220] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:08 2022] DELETE /identity/v3/users/cffdf662616a4b16a76a9a1afb8226fd => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:08.968694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123d586-8db8-42e7-91b0-919aa2061f91 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:08.968984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123d586-8db8-42e7-91b0-919aa2061f91 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:08.969256 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123d586-8db8-42e7-91b0-919aa2061f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:08.988415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7123d586-8db8-42e7-91b0-919aa2061f91 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:08.989610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7123d586-8db8-42e7-91b0-919aa2061f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:09.014643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:09.053260 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4067/8221] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:28:09.063199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:09.064698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:09.094129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:09.134545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044796a-f04f-43c9-8255-62735cc6e73e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:09.140526 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6dcab-0df0-4667-9548-5e57666eed62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:09.141581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6dcab-0df0-4667-9548-5e57666eed62 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:09.141916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6dcab-0df0-4667-9548-5e57666eed62 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:09.142236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6dcab-0df0-4667-9548-5e57666eed62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:09.157558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6dcab-0df0-4667-9548-5e57666eed62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:09.159718 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6dcab-0df0-4667-9548-5e57666eed62 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:09.169932 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4155/8222] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:09.471383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123d586-8db8-42e7-91b0-919aa2061f91 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:09.474504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7123d586-8db8-42e7-91b0-919aa2061f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:09.496781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123d586-8db8-42e7-91b0-919aa2061f91 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:09.502225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fed71-05f2-49c9-a852-e081fe8e1e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:09.503458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fed71-05f2-49c9-a852-e081fe8e1e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:09.503887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fed71-05f2-49c9-a852-e081fe8e1e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:09.504153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fed71-05f2-49c9-a852-e081fe8e1e16 admin admin] PATH_INFO: `/v3/projects/8dcb0e4de2004f719a9110f524017c8d` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:09.505358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26fed71-05f2-49c9-a852-e081fe8e1e16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dcb0e4de2004f719a9110f524017c8d)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:09.507305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26fed71-05f2-49c9-a852-e081fe8e1e16 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:09.563461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a26fed71-05f2-49c9-a852-e081fe8e1e16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dcb0e4de2004f719a9110f524017c8d', 'request_id': 'req-a26fed71-05f2-49c9-a852-e081fe8e1e16'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:09.570147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a26fed71-05f2-49c9-a852-e081fe8e1e16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dcb0e4de2004f719a9110f524017c8d', 'request_id': 'req-a26fed71-05f2-49c9-a852-e081fe8e1e16'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:09.578581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a26fed71-05f2-49c9-a852-e081fe8e1e16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dcb0e4de2004f719a9110f524017c8d', 'request_id': 'req-a26fed71-05f2-49c9-a852-e081fe8e1e16'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:09.589020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a26fed71-05f2-49c9-a852-e081fe8e1e16 admin admin] The token cache is being invalidate because project 8dcb0e4de2004f719a9110f524017c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:09.589302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a26fed71-05f2-49c9-a852-e081fe8e1e16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:09.589950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a26fed71-05f2-49c9-a852-e081fe8e1e16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:09.591486 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4068/8223] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:09 2022] DELETE /identity/v3/projects/8dcb0e4de2004f719a9110f524017c8d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:09.600286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec03956-bcc1-4fba-8031-bf3acefedd5b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:09.600599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec03956-bcc1-4fba-8031-bf3acefedd5b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:09.600867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec03956-bcc1-4fba-8031-bf3acefedd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:09.621039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-aec03956-bcc1-4fba-8031-bf3acefedd5b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:09.622254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-aec03956-bcc1-4fba-8031-bf3acefedd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:09.686773 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4156/8224] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:09.695407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420827a-2730-45df-a4b9-19b810b429c8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:09.695909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420827a-2730-45df-a4b9-19b810b429c8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:09.695909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420827a-2730-45df-a4b9-19b810b429c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:09.716936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a420827a-2730-45df-a4b9-19b810b429c8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:09.718117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a420827a-2730-45df-a4b9-19b810b429c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:09.727162 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4069/8225] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:09.733210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec03956-bcc1-4fba-8031-bf3acefedd5b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:09.748765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec03956-bcc1-4fba-8031-bf3acefedd5b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:09.756117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5027d-908f-43a0-8b02-ed8f110d214c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:09.756745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5027d-908f-43a0-8b02-ed8f110d214c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:09.757015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5027d-908f-43a0-8b02-ed8f110d214c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:09.757274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5027d-908f-43a0-8b02-ed8f110d214c admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:09.757910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5027d-908f-43a0-8b02-ed8f110d214c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:09.758203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5027d-908f-43a0-8b02-ed8f110d214c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:09.759799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5027d-908f-43a0-8b02-ed8f110d214c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:09.772172 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4157/8226] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:28:09.776529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420827a-2730-45df-a4b9-19b810b429c8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:09.785931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420827a-2730-45df-a4b9-19b810b429c8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:09.791019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da7e32-3518-4406-861b-581efaf420b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:09.791601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da7e32-3518-4406-861b-581efaf420b7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:09.791869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da7e32-3518-4406-861b-581efaf420b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:09.792128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da7e32-3518-4406-861b-581efaf420b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:09.792783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da7e32-3518-4406-861b-581efaf420b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:09.794643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da7e32-3518-4406-861b-581efaf420b7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:09.824173 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4070/8227] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:09 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:09.828504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5027d-908f-43a0-8b02-ed8f110d214c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:09.839375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5027d-908f-43a0-8b02-ed8f110d214c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:09.906532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58118ff-32e3-4dbe-b989-f5c160e0990e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:09.907146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58118ff-32e3-4dbe-b989-f5c160e0990e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:09.907423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58118ff-32e3-4dbe-b989-f5c160e0990e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:09.907721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58118ff-32e3-4dbe-b989-f5c160e0990e admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:09.908362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58118ff-32e3-4dbe-b989-f5c160e0990e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:09.910577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58118ff-32e3-4dbe-b989-f5c160e0990e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:09.934655 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4158/8228] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:09 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:28:09.939615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da7e32-3518-4406-861b-581efaf420b7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:09.950829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da7e32-3518-4406-861b-581efaf420b7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:09.955542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be483ced-5e4a-49da-b022-8d943f2203e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:09.956139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be483ced-5e4a-49da-b022-8d943f2203e6 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:09.956438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be483ced-5e4a-49da-b022-8d943f2203e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:09.956829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be483ced-5e4a-49da-b022-8d943f2203e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:09.957489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be483ced-5e4a-49da-b022-8d943f2203e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:09.959114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be483ced-5e4a-49da-b022-8d943f2203e6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:09.969733 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4071/8229] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:09.974070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58118ff-32e3-4dbe-b989-f5c160e0990e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:09.984244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58118ff-32e3-4dbe-b989-f5c160e0990e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:09.990175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e416a-a584-4021-a9e2-fc9c6617989e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:09.990761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e416a-a584-4021-a9e2-fc9c6617989e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:09.991030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e416a-a584-4021-a9e2-fc9c6617989e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:09.991287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e416a-a584-4021-a9e2-fc9c6617989e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:09.991915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e416a-a584-4021-a9e2-fc9c6617989e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:09.993551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e416a-a584-4021-a9e2-fc9c6617989e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:10.004477 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4159/8230] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:28:10.010865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be483ced-5e4a-49da-b022-8d943f2203e6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:10.025612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be483ced-5e4a-49da-b022-8d943f2203e6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:10.033020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17811375-31e1-47d7-a776-5678fcca5253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:10.033977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17811375-31e1-47d7-a776-5678fcca5253 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:10.034465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17811375-31e1-47d7-a776-5678fcca5253 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:10.034830 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17811375-31e1-47d7-a776-5678fcca5253 admin admin] PATH_INFO: `/v3/projects/0226f78c48314eb79c23831c68f37b84/users/e2eeacb21f354f4db769a048101e03e0/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:10.049930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17811375-31e1-47d7-a776-5678fcca5253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0226f78c48314eb79c23831c68f37b84, user_id=e2eeacb21f354f4db769a048101e03e0, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:10.052658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17811375-31e1-47d7-a776-5678fcca5253 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:10.067843 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4072/8231] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:10 2022] PUT /identity/v3/projects/0226f78c48314eb79c23831c68f37b84/users/e2eeacb21f354f4db769a048101e03e0/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:10.079391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab38a8a-41f7-43ea-b0c2-454a6e7e5cc3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:10.079952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab38a8a-41f7-43ea-b0c2-454a6e7e5cc3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:10.080059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab38a8a-41f7-43ea-b0c2-454a6e7e5cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:10.108593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fab38a8a-41f7-43ea-b0c2-454a6e7e5cc3 None None] MFA Rules not processed for user `e2eeacb21f354f4db769a048101e03e0`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:10.109725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fab38a8a-41f7-43ea-b0c2-454a6e7e5cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:10.135735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:10.185132 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4160/8232] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:10.190071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17811375-31e1-47d7-a776-5678fcca5253 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:10.200693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17811375-31e1-47d7-a776-5678fcca5253 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:10.205037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba60c40-b5df-4da0-9164-6064a00abe7d tempest-IdentityCatalogTest-2099585242 tempest-IdentityCatalogTest-2099585242-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2eeacb21f354f4db769a048101e03e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0226f78c48314eb79c23831c68f37b84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:10.205644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba60c40-b5df-4da0-9164-6064a00abe7d tempest-IdentityCatalogTest-2099585242 tempest-IdentityCatalogTest-2099585242-project] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:10.205991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba60c40-b5df-4da0-9164-6064a00abe7d tempest-IdentityCatalogTest-2099585242 tempest-IdentityCatalogTest-2099585242-project] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:10.206786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba60c40-b5df-4da0-9164-6064a00abe7d tempest-IdentityCatalogTest-2099585242 tempest-IdentityCatalogTest-2099585242-project] PATH_INFO: `/v3/auth/catalog` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:10.207351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba60c40-b5df-4da0-9164-6064a00abe7d tempest-IdentityCatalogTest-2099585242 tempest-IdentityCatalogTest-2099585242-project] RBAC: Authorizing `identity:get_auth_catalog()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:10.208486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba60c40-b5df-4da0-9164-6064a00abe7d tempest-IdentityCatalogTest-2099585242 tempest-IdentityCatalogTest-2099585242-project] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:10.210234 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4073/8233] 10.222.0.23 () {58 vars in 1143 bytes} [Sat Aug 20 21:28:10 2022] GET /identity/v3/auth/catalog => generated 2658 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:10.215095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab38a8a-41f7-43ea-b0c2-454a6e7e5cc3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:10.290860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab38a8a-41f7-43ea-b0c2-454a6e7e5cc3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:10.296082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:10.296746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:10.297030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:10.297295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] PATH_INFO: `/v3/users/e2eeacb21f354f4db769a048101e03e0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:10.298446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2eeacb21f354f4db769a048101e03e0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:10.300479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:10.348642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2eeacb21f354f4db769a048101e03e0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:10.356879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2eeacb21f354f4db769a048101e03e0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:10.364912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2eeacb21f354f4db769a048101e03e0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:10.377170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2eeacb21f354f4db769a048101e03e0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:10.377994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2eeacb21f354f4db769a048101e03e0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:10.384439 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4161/8234] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:10 2022] DELETE /identity/v3/users/e2eeacb21f354f4db769a048101e03e0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:10.389050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df0bdb-6e12-48d0-b63f-cf779cff2626 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:10.389326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df0bdb-6e12-48d0-b63f-cf779cff2626 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:10.389587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df0bdb-6e12-48d0-b63f-cf779cff2626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:10.409584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-45df0bdb-6e12-48d0-b63f-cf779cff2626 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:10.410814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-45df0bdb-6e12-48d0-b63f-cf779cff2626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:10.458288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:10.477794 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4074/8235] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:10.488049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:10.489757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:10.535860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:10.555007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36ae1a-a361-45eb-b9ce-646460f4dca4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:10.561063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477239e0-30e9-45a9-9444-6f75df21034b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:10.561666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477239e0-30e9-45a9-9444-6f75df21034b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:10.561932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477239e0-30e9-45a9-9444-6f75df21034b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:10.562188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477239e0-30e9-45a9-9444-6f75df21034b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:10.574839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477239e0-30e9-45a9-9444-6f75df21034b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:10.576274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477239e0-30e9-45a9-9444-6f75df21034b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:10.586308 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4162/8236] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:10.868325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df0bdb-6e12-48d0-b63f-cf779cff2626 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:10.870241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-45df0bdb-6e12-48d0-b63f-cf779cff2626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:10.888687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df0bdb-6e12-48d0-b63f-cf779cff2626 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:10.894786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:10.895431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:10.895710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:10.896047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79 admin admin] PATH_INFO: `/v3/projects/0226f78c48314eb79c23831c68f37b84` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:10.897664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0226f78c48314eb79c23831c68f37b84)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:10.899587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:10.960270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0226f78c48314eb79c23831c68f37b84', 'request_id': 'req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:10.967078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0226f78c48314eb79c23831c68f37b84', 'request_id': 'req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:10.975525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0226f78c48314eb79c23831c68f37b84', 'request_id': 'req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:10.986013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79 admin admin] The token cache is being invalidate because project 0226f78c48314eb79c23831c68f37b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:10.986293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:10.986927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-21e8e3b1-c1fb-4d46-b42e-04c4b5eeab79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:10.988764 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4075/8237] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:10 2022] DELETE /identity/v3/projects/0226f78c48314eb79c23831c68f37b84 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:10.994586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3dfb9-97b5-4186-b275-e7573275916a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:10.994863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3dfb9-97b5-4186-b275-e7573275916a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:10.995121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3dfb9-97b5-4186-b275-e7573275916a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.016425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-92d3dfb9-97b5-4186-b275-e7573275916a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:11.017632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-92d3dfb9-97b5-4186-b275-e7573275916a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:11.081199 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4163/8238] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:11.089055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c20d00-f2e5-4ec2-94d8-bb8969164768 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:11.089335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c20d00-f2e5-4ec2-94d8-bb8969164768 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:11.089622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c20d00-f2e5-4ec2-94d8-bb8969164768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.110743 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-59c20d00-f2e5-4ec2-94d8-bb8969164768 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:11.111923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-59c20d00-f2e5-4ec2-94d8-bb8969164768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:11.120015 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4076/8239] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:11.124235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3dfb9-97b5-4186-b275-e7573275916a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:11.134550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3dfb9-97b5-4186-b275-e7573275916a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:11.139816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02f00c-4c30-4fc2-94a1-617f8f94d7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:11.140396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02f00c-4c30-4fc2-94a1-617f8f94d7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:11.140721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02f00c-4c30-4fc2-94a1-617f8f94d7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:11.140975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02f00c-4c30-4fc2-94a1-617f8f94d7f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.141598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02f00c-4c30-4fc2-94a1-617f8f94d7f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:11.141886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02f00c-4c30-4fc2-94a1-617f8f94d7f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:11.143569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02f00c-4c30-4fc2-94a1-617f8f94d7f7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:11.155488 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4164/8240] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:28:11.161619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c20d00-f2e5-4ec2-94d8-bb8969164768 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:11.171309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c20d00-f2e5-4ec2-94d8-bb8969164768 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:11.176892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e9f61-ddbb-4eac-8c3f-eb15a2620933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:11.177540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e9f61-ddbb-4eac-8c3f-eb15a2620933 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:11.177807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e9f61-ddbb-4eac-8c3f-eb15a2620933 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:11.178067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e9f61-ddbb-4eac-8c3f-eb15a2620933 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.178687 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e9f61-ddbb-4eac-8c3f-eb15a2620933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:11.180636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e9f61-ddbb-4eac-8c3f-eb15a2620933 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:11.211719 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4077/8241] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:11 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:11.217603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02f00c-4c30-4fc2-94a1-617f8f94d7f7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:11.228409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02f00c-4c30-4fc2-94a1-617f8f94d7f7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:11.289393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9d627-fea8-4f4b-baaf-55aea2926560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:11.289991 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9d627-fea8-4f4b-baaf-55aea2926560 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:11.290255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9d627-fea8-4f4b-baaf-55aea2926560 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:11.290516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9d627-fea8-4f4b-baaf-55aea2926560 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.291121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9d627-fea8-4f4b-baaf-55aea2926560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:11.293083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9d627-fea8-4f4b-baaf-55aea2926560 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:11.315580 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4165/8242] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:11 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:11.320844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e9f61-ddbb-4eac-8c3f-eb15a2620933 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:11.336095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e9f61-ddbb-4eac-8c3f-eb15a2620933 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:11.342386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72f3c4-c26e-47dd-bd8a-2b2357b32ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:11.342988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72f3c4-c26e-47dd-bd8a-2b2357b32ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:11.343262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72f3c4-c26e-47dd-bd8a-2b2357b32ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:11.343519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72f3c4-c26e-47dd-bd8a-2b2357b32ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.344159 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72f3c4-c26e-47dd-bd8a-2b2357b32ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:11.345875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72f3c4-c26e-47dd-bd8a-2b2357b32ab5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:11.356583 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4078/8243] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:11.360782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9d627-fea8-4f4b-baaf-55aea2926560 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:11.373274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9d627-fea8-4f4b-baaf-55aea2926560 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:11.378320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60337b4-4971-472a-abd3-e32d5454b464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:11.379000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60337b4-4971-472a-abd3-e32d5454b464 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:11.379271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60337b4-4971-472a-abd3-e32d5454b464 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:11.379551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60337b4-4971-472a-abd3-e32d5454b464 admin admin] PATH_INFO: `/v3/projects/0421651eac244fcc9b1f2df030d71ef8/users/14a34161aa1e4cb2a431e1913fbf8779/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.395294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60337b4-4971-472a-abd3-e32d5454b464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0421651eac244fcc9b1f2df030d71ef8, user_id=14a34161aa1e4cb2a431e1913fbf8779, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:11.398001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60337b4-4971-472a-abd3-e32d5454b464 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:11.413889 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4166/8244] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:11 2022] PUT /identity/v3/projects/0421651eac244fcc9b1f2df030d71ef8/users/14a34161aa1e4cb2a431e1913fbf8779/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:28:11.418038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72f3c4-c26e-47dd-bd8a-2b2357b32ab5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:11.429085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72f3c4-c26e-47dd-bd8a-2b2357b32ab5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:11.487546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62646ac-7b17-4139-ad54-7ded80064390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:11.488151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62646ac-7b17-4139-ad54-7ded80064390 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:11.488460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62646ac-7b17-4139-ad54-7ded80064390 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:11.488728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62646ac-7b17-4139-ad54-7ded80064390 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.489359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62646ac-7b17-4139-ad54-7ded80064390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:11.490959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62646ac-7b17-4139-ad54-7ded80064390 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:11.500750 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4079/8245] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:11.506558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60337b4-4971-472a-abd3-e32d5454b464 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:11.524733 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60337b4-4971-472a-abd3-e32d5454b464 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:11.535373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcec8de-ba65-41a4-9aab-63ed5a43df41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:11.536024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcec8de-ba65-41a4-9aab-63ed5a43df41 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:11.536296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcec8de-ba65-41a4-9aab-63ed5a43df41 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:11.536595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcec8de-ba65-41a4-9aab-63ed5a43df41 admin admin] PATH_INFO: `/v3/projects/0421651eac244fcc9b1f2df030d71ef8/users/14a34161aa1e4cb2a431e1913fbf8779/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.538553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcec8de-ba65-41a4-9aab-63ed5a43df41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0421651eac244fcc9b1f2df030d71ef8, user_id=14a34161aa1e4cb2a431e1913fbf8779, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:11.541246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcec8de-ba65-41a4-9aab-63ed5a43df41 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:11.552530 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4167/8246] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:11 2022] PUT /identity/v3/projects/0421651eac244fcc9b1f2df030d71ef8/users/14a34161aa1e4cb2a431e1913fbf8779/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:11.562976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb35b8d-fcd1-4e9a-be6b-f91daba6127d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:11.563257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb35b8d-fcd1-4e9a-be6b-f91daba6127d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:11.563522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb35b8d-fcd1-4e9a-be6b-f91daba6127d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.593819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fbb35b8d-fcd1-4e9a-be6b-f91daba6127d None None] MFA Rules not processed for user `14a34161aa1e4cb2a431e1913fbf8779`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:11.595011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fbb35b8d-fcd1-4e9a-be6b-f91daba6127d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:11.608957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:11.682739 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4080/8247] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:11 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:11.687607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcec8de-ba65-41a4-9aab-63ed5a43df41 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:11.714728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:11.753547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcec8de-ba65-41a4-9aab-63ed5a43df41 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:11.758439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef1465-eb6b-414d-8779-dc6a672bf73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:11.759033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef1465-eb6b-414d-8779-dc6a672bf73a admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:11.759304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef1465-eb6b-414d-8779-dc6a672bf73a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:11.759555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef1465-eb6b-414d-8779-dc6a672bf73a admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.760170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef1465-eb6b-414d-8779-dc6a672bf73a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:11.762112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef1465-eb6b-414d-8779-dc6a672bf73a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:11.792588 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4168/8248] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:11 2022] POST /identity/v3/projects => generated 393 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:28:11.798396 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb35b8d-fcd1-4e9a-be6b-f91daba6127d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:11.809924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb35b8d-fcd1-4e9a-be6b-f91daba6127d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:11.871415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c2701-bd19-492a-af92-eb1c452480ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:11.872116 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c2701-bd19-492a-af92-eb1c452480ae admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:11.872265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c2701-bd19-492a-af92-eb1c452480ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:11.872712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c2701-bd19-492a-af92-eb1c452480ae admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.873616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c2701-bd19-492a-af92-eb1c452480ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:11.876541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c2701-bd19-492a-af92-eb1c452480ae admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:11.902663 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4081/8249] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:11 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:11.907099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef1465-eb6b-414d-8779-dc6a672bf73a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:11.920895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef1465-eb6b-414d-8779-dc6a672bf73a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:11.926752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb9914-7490-4049-8fd0-53ab29efc771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:11.927353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb9914-7490-4049-8fd0-53ab29efc771 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:11.927622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb9914-7490-4049-8fd0-53ab29efc771 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:11.927885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb9914-7490-4049-8fd0-53ab29efc771 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.928556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb9914-7490-4049-8fd0-53ab29efc771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:11.930176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb9914-7490-4049-8fd0-53ab29efc771 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:11.941512 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4169/8250] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:11.947891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c2701-bd19-492a-af92-eb1c452480ae admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:11.961965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c2701-bd19-492a-af92-eb1c452480ae admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:11.967511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0678f-5c18-4f57-b13d-97bb83314100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:11.968107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0678f-5c18-4f57-b13d-97bb83314100 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:11.968398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0678f-5c18-4f57-b13d-97bb83314100 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:11.968671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0678f-5c18-4f57-b13d-97bb83314100 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:11.969306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0678f-5c18-4f57-b13d-97bb83314100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:11.970910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0678f-5c18-4f57-b13d-97bb83314100 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:11.981391 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4082/8251] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:11.986092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb9914-7490-4049-8fd0-53ab29efc771 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:12.001317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb9914-7490-4049-8fd0-53ab29efc771 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:12.008242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b396b38-8b6e-4507-a34b-a533fd3102a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:12.008861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b396b38-8b6e-4507-a34b-a533fd3102a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:12.009131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b396b38-8b6e-4507-a34b-a533fd3102a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:12.009392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b396b38-8b6e-4507-a34b-a533fd3102a3 admin admin] PATH_INFO: `/v3/projects/a15f3d6e7ab54d8fbd094d383959a3b0/users/35bad014cc3c424b8ce03b8efb593835/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:12.026247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b396b38-8b6e-4507-a34b-a533fd3102a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15f3d6e7ab54d8fbd094d383959a3b0, user_id=35bad014cc3c424b8ce03b8efb593835, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:12.028973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b396b38-8b6e-4507-a34b-a533fd3102a3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:12.044661 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4170/8252] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:11 2022] PUT /identity/v3/projects/a15f3d6e7ab54d8fbd094d383959a3b0/users/35bad014cc3c424b8ce03b8efb593835/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:28:12.056072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f65e1-dbe8-4059-a5bd-0d74b870fde7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:12.056350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f65e1-dbe8-4059-a5bd-0d74b870fde7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:12.056647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f65e1-dbe8-4059-a5bd-0d74b870fde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:12.086918 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-719f65e1-dbe8-4059-a5bd-0d74b870fde7 None None] MFA Rules not processed for user `35bad014cc3c424b8ce03b8efb593835`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:12.088133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-719f65e1-dbe8-4059-a5bd-0d74b870fde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:12.153168 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4083/8253] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:12.165012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b396b38-8b6e-4507-a34b-a533fd3102a3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:12.167004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3b396b38-8b6e-4507-a34b-a533fd3102a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:12.226981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b396b38-8b6e-4507-a34b-a533fd3102a3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:12.231230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a0c3c-6bb6-416b-95f6-f1d2e1f7ad2d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:12.231847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a0c3c-6bb6-416b-95f6-f1d2e1f7ad2d service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:12.232101 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a0c3c-6bb6-416b-95f6-f1d2e1f7ad2d service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:12.232365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a0c3c-6bb6-416b-95f6-f1d2e1f7ad2d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:12.301029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a0c3c-6bb6-416b-95f6-f1d2e1f7ad2d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:12.302748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a0c3c-6bb6-416b-95f6-f1d2e1f7ad2d service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:12.318066 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4171/8254] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:12 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:12.948134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f65e1-dbe8-4059-a5bd-0d74b870fde7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:12.949671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-719f65e1-dbe8-4059-a5bd-0d74b870fde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:12.972616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:13.034453 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f65e1-dbe8-4059-a5bd-0d74b870fde7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:13.039714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea536e-8ba1-4e24-b9b3-b5112757b6a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:13.040324 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea536e-8ba1-4e24-b9b3-b5112757b6a2 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:13.040631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea536e-8ba1-4e24-b9b3-b5112757b6a2 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:13.040896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea536e-8ba1-4e24-b9b3-b5112757b6a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:13.042805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e7ea536e-8ba1-4e24-b9b3-b5112757b6a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:13.057879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea536e-8ba1-4e24-b9b3-b5112757b6a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:13.059589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea536e-8ba1-4e24-b9b3-b5112757b6a2 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:13.075362 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4084/8255] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:13.103923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a0c3c-6bb6-416b-95f6-f1d2e1f7ad2d service glance] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:13.129822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:13.170062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a0c3c-6bb6-416b-95f6-f1d2e1f7ad2d service glance] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:13.175734 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:13.176361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:13.176655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:13.176980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] PATH_INFO: `/v3/users/14a34161aa1e4cb2a431e1913fbf8779` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:13.178153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] RBAC: Authorizing `identity:delete_user(user_id=14a34161aa1e4cb2a431e1913fbf8779)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:13.180110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:13.232166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14a34161aa1e4cb2a431e1913fbf8779'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:13.241764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14a34161aa1e4cb2a431e1913fbf8779'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:13.249310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14a34161aa1e4cb2a431e1913fbf8779'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:13.261960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14a34161aa1e4cb2a431e1913fbf8779'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:13.262624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14a34161aa1e4cb2a431e1913fbf8779'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:13.269368 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4172/8256] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:13 2022] DELETE /identity/v3/users/14a34161aa1e4cb2a431e1913fbf8779 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:13.273947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa242c4-c1d2-4f18-a907-a0f0251e396f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:13.274227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa242c4-c1d2-4f18-a907-a0f0251e396f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:13.274493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa242c4-c1d2-4f18-a907-a0f0251e396f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:13.293108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4fa242c4-c1d2-4f18-a907-a0f0251e396f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:13.294302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4fa242c4-c1d2-4f18-a907-a0f0251e396f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:13.355371 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4085/8257] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:13.366135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:13.367767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:13.431543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cc58b-2516-4bf0-aa02-9c0a41f4d63c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:13.435595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01994cb-bb61-4d00-aab3-4ac2e30354d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:13.436202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01994cb-bb61-4d00-aab3-4ac2e30354d0 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:13.436561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01994cb-bb61-4d00-aab3-4ac2e30354d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:13.436823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01994cb-bb61-4d00-aab3-4ac2e30354d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:13.447468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01994cb-bb61-4d00-aab3-4ac2e30354d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:13.448944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01994cb-bb61-4d00-aab3-4ac2e30354d0 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:13.459120 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4173/8258] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:13.755500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa242c4-c1d2-4f18-a907-a0f0251e396f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:13.756976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4fa242c4-c1d2-4f18-a907-a0f0251e396f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:13.770317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa242c4-c1d2-4f18-a907-a0f0251e396f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:13.776171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:13.776892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:13.777161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:13.777424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] PATH_INFO: `/v3/projects/0421651eac244fcc9b1f2df030d71ef8` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:13.778674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0421651eac244fcc9b1f2df030d71ef8)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:13.780821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:13.832666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0421651eac244fcc9b1f2df030d71ef8', 'request_id': 'req-30d4ccaa-d81f-4085-8325-ab5860a73605'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:13.839331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0421651eac244fcc9b1f2df030d71ef8', 'request_id': 'req-30d4ccaa-d81f-4085-8325-ab5860a73605'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:13.848435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0421651eac244fcc9b1f2df030d71ef8', 'request_id': 'req-30d4ccaa-d81f-4085-8325-ab5860a73605'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:13.858847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] The token cache is being invalidate because project 0421651eac244fcc9b1f2df030d71ef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:13.859126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:13.859741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:13.861287 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4086/8259] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:13 2022] DELETE /identity/v3/projects/0421651eac244fcc9b1f2df030d71ef8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:13.865623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01994cb-bb61-4d00-aab3-4ac2e30354d0 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:13.875885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01994cb-bb61-4d00-aab3-4ac2e30354d0 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:13.919049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:13.935996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a872c-c71d-4e8e-a757-e3bbaf850a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:13.936651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a872c-c71d-4e8e-a757-e3bbaf850a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:13.936921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a872c-c71d-4e8e-a757-e3bbaf850a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:13.937182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a872c-c71d-4e8e-a757-e3bbaf850a0a admin admin] PATH_INFO: `/v3/users/35bad014cc3c424b8ce03b8efb593835` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:13.938283 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a872c-c71d-4e8e-a757-e3bbaf850a0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=35bad014cc3c424b8ce03b8efb593835)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:13.940197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a872c-c71d-4e8e-a757-e3bbaf850a0a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:13.994645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4d4a872c-c71d-4e8e-a757-e3bbaf850a0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35bad014cc3c424b8ce03b8efb593835'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:14.004457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4d4a872c-c71d-4e8e-a757-e3bbaf850a0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35bad014cc3c424b8ce03b8efb593835'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:14.011997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4d4a872c-c71d-4e8e-a757-e3bbaf850a0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35bad014cc3c424b8ce03b8efb593835'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:14.025130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4d4a872c-c71d-4e8e-a757-e3bbaf850a0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35bad014cc3c424b8ce03b8efb593835'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:14.025895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-4d4a872c-c71d-4e8e-a757-e3bbaf850a0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35bad014cc3c424b8ce03b8efb593835'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:14.032017 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4174/8260] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:13 2022] DELETE /identity/v3/users/35bad014cc3c424b8ce03b8efb593835 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:14.324527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:14.326106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:14.340596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4ccaa-d81f-4085-8325-ab5860a73605 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:14.377838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:14.411037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101b750-a112-437a-a7df-934be616c6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:14.411037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101b750-a112-437a-a7df-934be616c6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:14.411037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101b750-a112-437a-a7df-934be616c6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:14.411037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101b750-a112-437a-a7df-934be616c6c1 admin admin] PATH_INFO: `/v3/projects/a15f3d6e7ab54d8fbd094d383959a3b0` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:14.411567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101b750-a112-437a-a7df-934be616c6c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a15f3d6e7ab54d8fbd094d383959a3b0)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:14.413352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101b750-a112-437a-a7df-934be616c6c1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:14.461612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4101b750-a112-437a-a7df-934be616c6c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a15f3d6e7ab54d8fbd094d383959a3b0', 'request_id': 'req-4101b750-a112-437a-a7df-934be616c6c1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:14.468244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4101b750-a112-437a-a7df-934be616c6c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a15f3d6e7ab54d8fbd094d383959a3b0', 'request_id': 'req-4101b750-a112-437a-a7df-934be616c6c1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:14.478085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4101b750-a112-437a-a7df-934be616c6c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a15f3d6e7ab54d8fbd094d383959a3b0', 'request_id': 'req-4101b750-a112-437a-a7df-934be616c6c1'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:14.488560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4101b750-a112-437a-a7df-934be616c6c1 admin admin] The token cache is being invalidate because project a15f3d6e7ab54d8fbd094d383959a3b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:14.488844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4101b750-a112-437a-a7df-934be616c6c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:14.489456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4101b750-a112-437a-a7df-934be616c6c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:14.490957 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4087/8261] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:14 2022] DELETE /identity/v3/projects/a15f3d6e7ab54d8fbd094d383959a3b0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:14.495942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd480aea-a8ea-4ec6-8f97-ca1d5907986b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:14.496511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd480aea-a8ea-4ec6-8f97-ca1d5907986b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:14.496511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd480aea-a8ea-4ec6-8f97-ca1d5907986b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:14.516699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cd480aea-a8ea-4ec6-8f97-ca1d5907986b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:14.517884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cd480aea-a8ea-4ec6-8f97-ca1d5907986b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:14.579356 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4175/8262] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:28:14.587378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6d4b2-3b4f-49bb-91ce-f0bf5ea3ff8e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:14.587657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6d4b2-3b4f-49bb-91ce-f0bf5ea3ff8e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:14.587949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6d4b2-3b4f-49bb-91ce-f0bf5ea3ff8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:14.609056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7aa6d4b2-3b4f-49bb-91ce-f0bf5ea3ff8e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:14.610309 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7aa6d4b2-3b4f-49bb-91ce-f0bf5ea3ff8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:14.619497 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4088/8263] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:28:14.624045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd480aea-a8ea-4ec6-8f97-ca1d5907986b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:14.634801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd480aea-a8ea-4ec6-8f97-ca1d5907986b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:14.640735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a8e78-1e66-4e21-aa93-b0f933fdd8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:14.641361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a8e78-1e66-4e21-aa93-b0f933fdd8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:14.641631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a8e78-1e66-4e21-aa93-b0f933fdd8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:14.641894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a8e78-1e66-4e21-aa93-b0f933fdd8c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:14.642544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a8e78-1e66-4e21-aa93-b0f933fdd8c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:14.642863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a8e78-1e66-4e21-aa93-b0f933fdd8c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:14.644588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a8e78-1e66-4e21-aa93-b0f933fdd8c2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:14.656828 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4176/8264] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:28:14.661261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6d4b2-3b4f-49bb-91ce-f0bf5ea3ff8e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:14.671089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6d4b2-3b4f-49bb-91ce-f0bf5ea3ff8e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:14.676968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44898f63-22c9-4463-be75-a4491edc7690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:14.677565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44898f63-22c9-4463-be75-a4491edc7690 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:14.677826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44898f63-22c9-4463-be75-a4491edc7690 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:14.678090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44898f63-22c9-4463-be75-a4491edc7690 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:14.678797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44898f63-22c9-4463-be75-a4491edc7690 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:14.680728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44898f63-22c9-4463-be75-a4491edc7690 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:14.709691 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4089/8265] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:14 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:14.713992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a8e78-1e66-4e21-aa93-b0f933fdd8c2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:14.724036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a8e78-1e66-4e21-aa93-b0f933fdd8c2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:14.771344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:14.784130 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c6d97-3bd8-44c6-9532-aa2ea464431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:14.784755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c6d97-3bd8-44c6-9532-aa2ea464431f admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:14.785028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c6d97-3bd8-44c6-9532-aa2ea464431f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:14.785287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c6d97-3bd8-44c6-9532-aa2ea464431f admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:14.785895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c6d97-3bd8-44c6-9532-aa2ea464431f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:14.787844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c6d97-3bd8-44c6-9532-aa2ea464431f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:14.811929 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4177/8266] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:14 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:14.816341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44898f63-22c9-4463-be75-a4491edc7690 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:14.827112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44898f63-22c9-4463-be75-a4491edc7690 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:14.832193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c9061-dd9b-46d8-b725-068de7f83ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:14.832808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c9061-dd9b-46d8-b725-068de7f83ebd admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:14.833084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c9061-dd9b-46d8-b725-068de7f83ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:14.833351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c9061-dd9b-46d8-b725-068de7f83ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:14.833990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c9061-dd9b-46d8-b725-068de7f83ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:14.835586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c9061-dd9b-46d8-b725-068de7f83ebd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:14.845734 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4090/8267] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:14.850013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c6d97-3bd8-44c6-9532-aa2ea464431f admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:14.860062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c6d97-3bd8-44c6-9532-aa2ea464431f admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:14.865197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fae78e-1ea7-4f51-8a00-0d0914e81dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:14.865840 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fae78e-1ea7-4f51-8a00-0d0914e81dba admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:14.866112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fae78e-1ea7-4f51-8a00-0d0914e81dba admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:14.866374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fae78e-1ea7-4f51-8a00-0d0914e81dba admin admin] PATH_INFO: `/v3/projects/2e6ae71aecf548b98eab04c14bb19db3/users/25767741b659476682acdc171c0feb57/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:14.881754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fae78e-1ea7-4f51-8a00-0d0914e81dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6ae71aecf548b98eab04c14bb19db3, user_id=25767741b659476682acdc171c0feb57, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:14.884591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fae78e-1ea7-4f51-8a00-0d0914e81dba admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:14.899015 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4178/8268] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:14 2022] PUT /identity/v3/projects/2e6ae71aecf548b98eab04c14bb19db3/users/25767741b659476682acdc171c0feb57/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:14.903216 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c9061-dd9b-46d8-b725-068de7f83ebd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:14.914660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c9061-dd9b-46d8-b725-068de7f83ebd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:14.973955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5071138-8173-4f75-8653-077934fb7140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:14.974562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071138-8173-4f75-8653-077934fb7140 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:14.974838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071138-8173-4f75-8653-077934fb7140 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:14.975108 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071138-8173-4f75-8653-077934fb7140 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:14.975726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5071138-8173-4f75-8653-077934fb7140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:14.977364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5071138-8173-4f75-8653-077934fb7140 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:14.987289 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4091/8269] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:14.991683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fae78e-1ea7-4f51-8a00-0d0914e81dba admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:15.002208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fae78e-1ea7-4f51-8a00-0d0914e81dba admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:15.007286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c654a-5f08-4833-b01e-eceb9009888c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:15.007933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c654a-5f08-4833-b01e-eceb9009888c admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:15.008209 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c654a-5f08-4833-b01e-eceb9009888c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:15.008513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c654a-5f08-4833-b01e-eceb9009888c admin admin] PATH_INFO: `/v3/projects/2e6ae71aecf548b98eab04c14bb19db3/users/25767741b659476682acdc171c0feb57/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:15.010531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c654a-5f08-4833-b01e-eceb9009888c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6ae71aecf548b98eab04c14bb19db3, user_id=25767741b659476682acdc171c0feb57, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:15.013232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c654a-5f08-4833-b01e-eceb9009888c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:15.023839 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4179/8270] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:14 2022] PUT /identity/v3/projects/2e6ae71aecf548b98eab04c14bb19db3/users/25767741b659476682acdc171c0feb57/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:15.035432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb44fe-3e70-492c-b421-b5b46970d408 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:15.035713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb44fe-3e70-492c-b421-b5b46970d408 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:15.035983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb44fe-3e70-492c-b421-b5b46970d408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:15.064493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0bcb44fe-3e70-492c-b421-b5b46970d408 None None] MFA Rules not processed for user `25767741b659476682acdc171c0feb57`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:15.066062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0bcb44fe-3e70-492c-b421-b5b46970d408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:15.075534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:15.145122 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4092/8271] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:15.149811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c654a-5f08-4833-b01e-eceb9009888c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:15.228810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c654a-5f08-4833-b01e-eceb9009888c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:15.236591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709a02f-732c-4f57-b9c3-65a4e2763f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:15.237187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709a02f-732c-4f57-b9c3-65a4e2763f9d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:15.238087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709a02f-732c-4f57-b9c3-65a4e2763f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:15.238087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709a02f-732c-4f57-b9c3-65a4e2763f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:15.238769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709a02f-732c-4f57-b9c3-65a4e2763f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:15.240726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709a02f-732c-4f57-b9c3-65a4e2763f9d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:15.270890 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4180/8272] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:15 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:15.275004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb44fe-3e70-492c-b421-b5b46970d408 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:15.285526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb44fe-3e70-492c-b421-b5b46970d408 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:15.344730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de8615-16dd-4d29-8a37-5f0d90687a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:15.345474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de8615-16dd-4d29-8a37-5f0d90687a53 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:15.345679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de8615-16dd-4d29-8a37-5f0d90687a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:15.345834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de8615-16dd-4d29-8a37-5f0d90687a53 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:15.346716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de8615-16dd-4d29-8a37-5f0d90687a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:15.349788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de8615-16dd-4d29-8a37-5f0d90687a53 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:15.380948 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4093/8273] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:15 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:28:15.386043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709a02f-732c-4f57-b9c3-65a4e2763f9d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:15.398486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709a02f-732c-4f57-b9c3-65a4e2763f9d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:15.403757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bf341-3b2c-4942-878b-9694d5c09ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:15.404352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bf341-3b2c-4942-878b-9694d5c09ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:15.404652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bf341-3b2c-4942-878b-9694d5c09ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:15.404906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bf341-3b2c-4942-878b-9694d5c09ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:15.405533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bf341-3b2c-4942-878b-9694d5c09ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:15.407138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bf341-3b2c-4942-878b-9694d5c09ac1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:15.418814 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4181/8274] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:15.423162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de8615-16dd-4d29-8a37-5f0d90687a53 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:15.434085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de8615-16dd-4d29-8a37-5f0d90687a53 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:15.439428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ba7fd-ba41-4a26-84bc-8af5e3f44607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:15.440035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ba7fd-ba41-4a26-84bc-8af5e3f44607 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:15.440307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ba7fd-ba41-4a26-84bc-8af5e3f44607 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:15.440607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ba7fd-ba41-4a26-84bc-8af5e3f44607 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:15.441241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ba7fd-ba41-4a26-84bc-8af5e3f44607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:15.443207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ba7fd-ba41-4a26-84bc-8af5e3f44607 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:15.453525 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4094/8275] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:15.458154 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bf341-3b2c-4942-878b-9694d5c09ac1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:15.468288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bf341-3b2c-4942-878b-9694d5c09ac1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:15.473655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8497348-e8c6-4aa7-8f11-a9883e3d5c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:15.474339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8497348-e8c6-4aa7-8f11-a9883e3d5c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:15.474612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8497348-e8c6-4aa7-8f11-a9883e3d5c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:15.474878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8497348-e8c6-4aa7-8f11-a9883e3d5c4e admin admin] PATH_INFO: `/v3/projects/40effab98cda48c3bb5bba766c9d7da1/users/e8c46899f26b463880aa6f07216268e2/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:15.490359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8497348-e8c6-4aa7-8f11-a9883e3d5c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=40effab98cda48c3bb5bba766c9d7da1, user_id=e8c46899f26b463880aa6f07216268e2, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:15.493060 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8497348-e8c6-4aa7-8f11-a9883e3d5c4e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:15.511594 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4182/8276] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:15 2022] PUT /identity/v3/projects/40effab98cda48c3bb5bba766c9d7da1/users/e8c46899f26b463880aa6f07216268e2/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:15.524426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5f9ab-07c7-4ad6-836c-ec5e77a62903 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:15.524791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5f9ab-07c7-4ad6-836c-ec5e77a62903 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:15.525067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5f9ab-07c7-4ad6-836c-ec5e77a62903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:15.554675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f1c5f9ab-07c7-4ad6-836c-ec5e77a62903 None None] MFA Rules not processed for user `e8c46899f26b463880aa6f07216268e2`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:15.555933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f1c5f9ab-07c7-4ad6-836c-ec5e77a62903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:15.623365 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4095/8277] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:28:15.637655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8497348-e8c6-4aa7-8f11-a9883e3d5c4e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:15.639638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a8497348-e8c6-4aa7-8f11-a9883e3d5c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:15.702515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8497348-e8c6-4aa7-8f11-a9883e3d5c4e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:15.707286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05640f4-18b8-4a48-b8a8-a047cc2ad257 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:15.708056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05640f4-18b8-4a48-b8a8-a047cc2ad257 service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:15.708345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05640f4-18b8-4a48-b8a8-a047cc2ad257 service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:15.708634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05640f4-18b8-4a48-b8a8-a047cc2ad257 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:15.714468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:15.773291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05640f4-18b8-4a48-b8a8-a047cc2ad257 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:15.774722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05640f4-18b8-4a48-b8a8-a047cc2ad257 service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:15.784825 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4183/8278] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:16.397941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5f9ab-07c7-4ad6-836c-ec5e77a62903 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:16.411443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:16.462794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5f9ab-07c7-4ad6-836c-ec5e77a62903 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:16.467487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:16.468119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:16.468366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:16.468701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] PATH_INFO: `/v3/users/25767741b659476682acdc171c0feb57` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:16.469849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25767741b659476682acdc171c0feb57)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:16.471808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:16.523836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25767741b659476682acdc171c0feb57'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:16.534786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25767741b659476682acdc171c0feb57'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:16.547932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25767741b659476682acdc171c0feb57'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:16.548583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25767741b659476682acdc171c0feb57'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:16.549203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25767741b659476682acdc171c0feb57'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:16.564435 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4096/8279] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:16 2022] DELETE /identity/v3/users/25767741b659476682acdc171c0feb57 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:16.569533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed729aa8-9f19-4a28-b43b-e23f3d57760f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:16.570019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed729aa8-9f19-4a28-b43b-e23f3d57760f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:16.570150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed729aa8-9f19-4a28-b43b-e23f3d57760f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:16.589908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ed729aa8-9f19-4a28-b43b-e23f3d57760f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:16.591100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ed729aa8-9f19-4a28-b43b-e23f3d57760f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:16.654893 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4184/8280] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:16.665230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:16.666845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:16.729244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b5cc-a89b-47a9-b903-b1edcaaae624 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:16.733689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c8235-2dae-422f-b022-95a68dcfd9f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:16.734300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c8235-2dae-422f-b022-95a68dcfd9f2 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:16.734571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c8235-2dae-422f-b022-95a68dcfd9f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:16.734834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c8235-2dae-422f-b022-95a68dcfd9f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:16.746036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c8235-2dae-422f-b022-95a68dcfd9f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:16.747474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c8235-2dae-422f-b022-95a68dcfd9f2 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:16.758073 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4097/8281] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:17.043552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed729aa8-9f19-4a28-b43b-e23f3d57760f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:17.045773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ed729aa8-9f19-4a28-b43b-e23f3d57760f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:17.061903 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed729aa8-9f19-4a28-b43b-e23f3d57760f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:17.068868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:17.069503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:17.069769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:17.070160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] PATH_INFO: `/v3/projects/2e6ae71aecf548b98eab04c14bb19db3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:17.071532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6ae71aecf548b98eab04c14bb19db3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:17.073555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:17.123760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6ae71aecf548b98eab04c14bb19db3', 'request_id': 'req-ffde81e0-42de-4886-a084-2162b916632e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.134324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6ae71aecf548b98eab04c14bb19db3', 'request_id': 'req-ffde81e0-42de-4886-a084-2162b916632e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.142467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6ae71aecf548b98eab04c14bb19db3', 'request_id': 'req-ffde81e0-42de-4886-a084-2162b916632e'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.154437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] The token cache is being invalidate because project 2e6ae71aecf548b98eab04c14bb19db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:17.154713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.155356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.156911 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4185/8282] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:17 2022] DELETE /identity/v3/projects/2e6ae71aecf548b98eab04c14bb19db3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:17.161654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c8235-2dae-422f-b022-95a68dcfd9f2 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:17.176090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c8235-2dae-422f-b022-95a68dcfd9f2 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:17.211049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:17.239207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4cd9c-508e-4a6b-a329-25756d0f1fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:17.239847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4cd9c-508e-4a6b-a329-25756d0f1fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:17.240112 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4cd9c-508e-4a6b-a329-25756d0f1fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:17.240391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4cd9c-508e-4a6b-a329-25756d0f1fc0 admin admin] PATH_INFO: `/v3/users/e8c46899f26b463880aa6f07216268e2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:17.241734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4cd9c-508e-4a6b-a329-25756d0f1fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c46899f26b463880aa6f07216268e2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:17.243723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4cd9c-508e-4a6b-a329-25756d0f1fc0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:17.301132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20a4cd9c-508e-4a6b-a329-25756d0f1fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c46899f26b463880aa6f07216268e2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.311270 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20a4cd9c-508e-4a6b-a329-25756d0f1fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c46899f26b463880aa6f07216268e2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.324847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20a4cd9c-508e-4a6b-a329-25756d0f1fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c46899f26b463880aa6f07216268e2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.325579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20a4cd9c-508e-4a6b-a329-25756d0f1fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c46899f26b463880aa6f07216268e2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.326205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-20a4cd9c-508e-4a6b-a329-25756d0f1fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c46899f26b463880aa6f07216268e2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.338500 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4098/8283] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:17 2022] DELETE /identity/v3/users/e8c46899f26b463880aa6f07216268e2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:17.646630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:17.648236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:17.662885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde81e0-42de-4886-a084-2162b916632e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:17.688671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:17.723798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b8e5d-8828-4af2-af03-6217524442a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:17.724466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b8e5d-8828-4af2-af03-6217524442a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:17.724743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b8e5d-8828-4af2-af03-6217524442a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:17.725009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b8e5d-8828-4af2-af03-6217524442a7 admin admin] PATH_INFO: `/v3/projects/40effab98cda48c3bb5bba766c9d7da1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:17.726124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b8e5d-8828-4af2-af03-6217524442a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40effab98cda48c3bb5bba766c9d7da1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:17.728039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b8e5d-8828-4af2-af03-6217524442a7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:17.776328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6c4b8e5d-8828-4af2-af03-6217524442a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40effab98cda48c3bb5bba766c9d7da1', 'request_id': 'req-6c4b8e5d-8828-4af2-af03-6217524442a7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.785669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6c4b8e5d-8828-4af2-af03-6217524442a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40effab98cda48c3bb5bba766c9d7da1', 'request_id': 'req-6c4b8e5d-8828-4af2-af03-6217524442a7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.791525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6c4b8e5d-8828-4af2-af03-6217524442a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40effab98cda48c3bb5bba766c9d7da1', 'request_id': 'req-6c4b8e5d-8828-4af2-af03-6217524442a7'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.803262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6c4b8e5d-8828-4af2-af03-6217524442a7 admin admin] The token cache is being invalidate because project 40effab98cda48c3bb5bba766c9d7da1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:17.803545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6c4b8e5d-8828-4af2-af03-6217524442a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.804244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6c4b8e5d-8828-4af2-af03-6217524442a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:17.805845 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4186/8284] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:17 2022] DELETE /identity/v3/projects/40effab98cda48c3bb5bba766c9d7da1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:17.811055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd24bd-359d-419c-b065-ed233967c708 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:17.811606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd24bd-359d-419c-b065-ed233967c708 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:17.811606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd24bd-359d-419c-b065-ed233967c708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:17.831611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-decd24bd-359d-419c-b065-ed233967c708 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:17.832876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-decd24bd-359d-419c-b065-ed233967c708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:17.895380 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4099/8285] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:17.903448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba52138-42d3-431c-9273-7edeb77e5867 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:17.903613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba52138-42d3-431c-9273-7edeb77e5867 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:17.903940 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba52138-42d3-431c-9273-7edeb77e5867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:17.923739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fba52138-42d3-431c-9273-7edeb77e5867 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:17.925117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fba52138-42d3-431c-9273-7edeb77e5867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:17.933209 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4187/8286] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:17.937544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd24bd-359d-419c-b065-ed233967c708 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:17.947867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd24bd-359d-419c-b065-ed233967c708 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:17.952639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3202c3-a199-45ce-9ba1-b7b2d3898228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:17.953235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3202c3-a199-45ce-9ba1-b7b2d3898228 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:17.953508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3202c3-a199-45ce-9ba1-b7b2d3898228 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:17.953775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3202c3-a199-45ce-9ba1-b7b2d3898228 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:17.954401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3202c3-a199-45ce-9ba1-b7b2d3898228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:17.954720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3202c3-a199-45ce-9ba1-b7b2d3898228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:17.956446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3202c3-a199-45ce-9ba1-b7b2d3898228 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:17.969808 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4100/8287] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:28:17.975210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba52138-42d3-431c-9273-7edeb77e5867 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:17.984049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba52138-42d3-431c-9273-7edeb77e5867 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:17.989922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd475bf-dc60-4cd0-992a-547540a1a0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:17.990530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd475bf-dc60-4cd0-992a-547540a1a0fb admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:17.990836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd475bf-dc60-4cd0-992a-547540a1a0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:17.991121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd475bf-dc60-4cd0-992a-547540a1a0fb admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:17.991743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd475bf-dc60-4cd0-992a-547540a1a0fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:17.993723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd475bf-dc60-4cd0-992a-547540a1a0fb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:18.030301 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4188/8288] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:17 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:18.034676 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3202c3-a199-45ce-9ba1-b7b2d3898228 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:18.045205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3202c3-a199-45ce-9ba1-b7b2d3898228 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:18.086669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:18.105168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504ed75-513c-4d65-90e1-059ed281e726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:18.105767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504ed75-513c-4d65-90e1-059ed281e726 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:18.106044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504ed75-513c-4d65-90e1-059ed281e726 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:18.106311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504ed75-513c-4d65-90e1-059ed281e726 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:18.106924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504ed75-513c-4d65-90e1-059ed281e726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:18.108891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504ed75-513c-4d65-90e1-059ed281e726 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:18.135235 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4101/8289] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:18 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:18.136626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd475bf-dc60-4cd0-992a-547540a1a0fb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:18.147429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd475bf-dc60-4cd0-992a-547540a1a0fb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:18.153208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dba4f5-920f-4606-9e4f-2233ec14fefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:18.153868 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dba4f5-920f-4606-9e4f-2233ec14fefa admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:18.154088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dba4f5-920f-4606-9e4f-2233ec14fefa admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:18.154374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dba4f5-920f-4606-9e4f-2233ec14fefa admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:18.154985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dba4f5-920f-4606-9e4f-2233ec14fefa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:18.156673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dba4f5-920f-4606-9e4f-2233ec14fefa admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:18.167589 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4189/8290] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:18.171467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504ed75-513c-4d65-90e1-059ed281e726 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:18.182630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504ed75-513c-4d65-90e1-059ed281e726 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:18.188429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088af846-45e1-4c80-a8c6-76b8a56ba6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:18.189027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088af846-45e1-4c80-a8c6-76b8a56ba6d4 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:18.189296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088af846-45e1-4c80-a8c6-76b8a56ba6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:18.189561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088af846-45e1-4c80-a8c6-76b8a56ba6d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:18.190413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088af846-45e1-4c80-a8c6-76b8a56ba6d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:18.192394 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088af846-45e1-4c80-a8c6-76b8a56ba6d4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:18.206232 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4102/8291] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:18.210547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dba4f5-920f-4606-9e4f-2233ec14fefa admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:18.221325 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dba4f5-920f-4606-9e4f-2233ec14fefa admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:18.226766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717fbe4-9755-483b-98cb-68ce8cda7ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:18.227452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717fbe4-9755-483b-98cb-68ce8cda7ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:18.227867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717fbe4-9755-483b-98cb-68ce8cda7ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:18.228145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717fbe4-9755-483b-98cb-68ce8cda7ab5 admin admin] PATH_INFO: `/v3/projects/bb4fe2b86e7b4846835290b1da59e42b/users/0dc21474f512473c856476d56247cd97/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:18.248124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717fbe4-9755-483b-98cb-68ce8cda7ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb4fe2b86e7b4846835290b1da59e42b, user_id=0dc21474f512473c856476d56247cd97, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:18.250983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717fbe4-9755-483b-98cb-68ce8cda7ab5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:18.272135 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4190/8292] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:18 2022] PUT /identity/v3/projects/bb4fe2b86e7b4846835290b1da59e42b/users/0dc21474f512473c856476d56247cd97/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:18.283667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76986e-229d-4870-8ef6-b44773d4e6b8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:18.284131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76986e-229d-4870-8ef6-b44773d4e6b8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:18.284553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76986e-229d-4870-8ef6-b44773d4e6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:18.313889 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9e76986e-229d-4870-8ef6-b44773d4e6b8 None None] MFA Rules not processed for user `0dc21474f512473c856476d56247cd97`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:18.315618 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9e76986e-229d-4870-8ef6-b44773d4e6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:18.384702 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4103/8293] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:28:18.408775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717fbe4-9755-483b-98cb-68ce8cda7ab5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:18.411012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3717fbe4-9755-483b-98cb-68ce8cda7ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:18.473332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717fbe4-9755-483b-98cb-68ce8cda7ab5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:18.478180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245ae87-1828-451e-94e5-1b177fed61a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:18.478792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245ae87-1828-451e-94e5-1b177fed61a9 service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:18.479063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245ae87-1828-451e-94e5-1b177fed61a9 service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:18.479327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245ae87-1828-451e-94e5-1b177fed61a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:18.487905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:18.490677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245ae87-1828-451e-94e5-1b177fed61a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:18.492970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245ae87-1828-451e-94e5-1b177fed61a9 service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:18.505031 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4191/8294] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:18.642472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273e480-16d1-47c4-afe5-a7fea899bc3d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:18.642752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273e480-16d1-47c4-afe5-a7fea899bc3d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:18.643015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273e480-16d1-47c4-afe5-a7fea899bc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:18.662875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7273e480-16d1-47c4-afe5-a7fea899bc3d None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:18.664013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7273e480-16d1-47c4-afe5-a7fea899bc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:18.722108 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4104/8295] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:28:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:18.743005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245ae87-1828-451e-94e5-1b177fed61a9 service glance] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:18.744601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3245ae87-1828-451e-94e5-1b177fed61a9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:18.815817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245ae87-1828-451e-94e5-1b177fed61a9 service glance] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:18.821302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc10e5-3292-4454-8c5b-7b04bff12afd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:18.821900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc10e5-3292-4454-8c5b-7b04bff12afd service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:18.822170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc10e5-3292-4454-8c5b-7b04bff12afd service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:18.822442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc10e5-3292-4454-8c5b-7b04bff12afd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:18.830635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc10e5-3292-4454-8c5b-7b04bff12afd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:18.832303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc10e5-3292-4454-8c5b-7b04bff12afd service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:18.840309 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4192/8296] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:28:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:28:19.102949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ac48e-0774-453a-9864-57c262ef7202 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:19.103454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ac48e-0774-453a-9864-57c262ef7202 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:19.103454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ac48e-0774-453a-9864-57c262ef7202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:19.124083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c33ac48e-0774-453a-9864-57c262ef7202 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:19.125704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c33ac48e-0774-453a-9864-57c262ef7202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:19.138046 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4105/8297] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:28:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:19.150677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc10e5-3292-4454-8c5b-7b04bff12afd service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:19.164150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc10e5-3292-4454-8c5b-7b04bff12afd service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:19.170291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80b835-476d-4f87-93d5-8f83442842b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:19.170892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80b835-476d-4f87-93d5-8f83442842b0 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:19.171159 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80b835-476d-4f87-93d5-8f83442842b0 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:19.171415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80b835-476d-4f87-93d5-8f83442842b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:19.180529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80b835-476d-4f87-93d5-8f83442842b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:19.182204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80b835-476d-4f87-93d5-8f83442842b0 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:19.190356 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4193/8298] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:28:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:28:19.456588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdda3ac-2727-42b0-a343-5bb130c0eec7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:19.456855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdda3ac-2727-42b0-a343-5bb130c0eec7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:19.457123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdda3ac-2727-42b0-a343-5bb130c0eec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:19.478685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-2cdda3ac-2727-42b0-a343-5bb130c0eec7 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:19.479955 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2cdda3ac-2727-42b0-a343-5bb130c0eec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:19.488806 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4106/8299] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:28:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:19.503981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80b835-476d-4f87-93d5-8f83442842b0 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:19.519161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80b835-476d-4f87-93d5-8f83442842b0 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:19.524844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bb845-fc98-4491-b98e-8537829e4e01 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:19.525435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bb845-fc98-4491-b98e-8537829e4e01 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:19.525703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bb845-fc98-4491-b98e-8537829e4e01 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:19.525968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bb845-fc98-4491-b98e-8537829e4e01 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:19.534831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8bb845-fc98-4491-b98e-8537829e4e01 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:19.536436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8bb845-fc98-4491-b98e-8537829e4e01 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:19.546828 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4194/8300] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:28:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:28:19.793327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2dc80-6947-4cf7-bf98-8aca42ef538c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:19.793602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2dc80-6947-4cf7-bf98-8aca42ef538c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:19.793866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2dc80-6947-4cf7-bf98-8aca42ef538c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:19.813905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-15e2dc80-6947-4cf7-bf98-8aca42ef538c None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:19.815408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-15e2dc80-6947-4cf7-bf98-8aca42ef538c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:19.823322 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4107/8301] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:28:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:19.835609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bb845-fc98-4491-b98e-8537829e4e01 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:19.849246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bb845-fc98-4491-b98e-8537829e4e01 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:19.854643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb28c62-bbef-4db9-9416-96c64545d199 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:19.855234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb28c62-bbef-4db9-9416-96c64545d199 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:19.855495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb28c62-bbef-4db9-9416-96c64545d199 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:19.855760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb28c62-bbef-4db9-9416-96c64545d199 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:19.863787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb28c62-bbef-4db9-9416-96c64545d199 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:19.865438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb28c62-bbef-4db9-9416-96c64545d199 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:19.873417 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4195/8302] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:28:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:28:20.124144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323f40d-7e8e-4a3a-ace9-7fb692d66e9d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:20.124685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323f40d-7e8e-4a3a-ace9-7fb692d66e9d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:20.124685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323f40d-7e8e-4a3a-ace9-7fb692d66e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:20.145677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8323f40d-7e8e-4a3a-ace9-7fb692d66e9d None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:20.146855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8323f40d-7e8e-4a3a-ace9-7fb692d66e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:20.154241 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4108/8303] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:28:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:20.171899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb28c62-bbef-4db9-9416-96c64545d199 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:20.187568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb28c62-bbef-4db9-9416-96c64545d199 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:20.195489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9575018-e89c-40d6-a280-ba49184de087 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:20.196086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9575018-e89c-40d6-a280-ba49184de087 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:20.196351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9575018-e89c-40d6-a280-ba49184de087 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:20.196646 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9575018-e89c-40d6-a280-ba49184de087 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:20.205251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9575018-e89c-40d6-a280-ba49184de087 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:20.207463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9575018-e89c-40d6-a280-ba49184de087 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:20.215580 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4196/8304] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:28:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:28:20.496792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67717f-5b11-411a-be30-b4d71e0cb68f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:20.497111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67717f-5b11-411a-be30-b4d71e0cb68f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:20.497364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67717f-5b11-411a-be30-b4d71e0cb68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:20.525459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8a67717f-5b11-411a-be30-b4d71e0cb68f None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:20.527382 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a67717f-5b11-411a-be30-b4d71e0cb68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:20.537903 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4109/8305] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:28:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:20.550291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9575018-e89c-40d6-a280-ba49184de087 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:20.568602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9575018-e89c-40d6-a280-ba49184de087 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:20.576057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbe938-2322-4d05-af50-12c47f165eb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:20.576964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbe938-2322-4d05-af50-12c47f165eb8 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:20.577324 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbe938-2322-4d05-af50-12c47f165eb8 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:20.577635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbe938-2322-4d05-af50-12c47f165eb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:20.588650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbbe938-2322-4d05-af50-12c47f165eb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:20.590263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbbe938-2322-4d05-af50-12c47f165eb8 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:20.598490 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4197/8306] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:28:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:28:21.538295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a89ed-6600-4627-bf5b-7e2176550541 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:21.538771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a89ed-6600-4627-bf5b-7e2176550541 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:21.538941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a89ed-6600-4627-bf5b-7e2176550541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:21.558902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-de7a89ed-6600-4627-bf5b-7e2176550541 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:21.560086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-de7a89ed-6600-4627-bf5b-7e2176550541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:21.568938 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4110/8307] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:28:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:21.581099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbe938-2322-4d05-af50-12c47f165eb8 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:21.594459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbe938-2322-4d05-af50-12c47f165eb8 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:21.601092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45decb0-f43e-42a6-b040-63c7bcab02d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:21.601679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45decb0-f43e-42a6-b040-63c7bcab02d4 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:21.601944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45decb0-f43e-42a6-b040-63c7bcab02d4 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:21.602203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45decb0-f43e-42a6-b040-63c7bcab02d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:21.610918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45decb0-f43e-42a6-b040-63c7bcab02d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:21.612642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45decb0-f43e-42a6-b040-63c7bcab02d4 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:21.621398 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4198/8308] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:28:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:28:21.764038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856a5f5-d6dc-4572-a3ff-a8c4537d35ea None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:21.764305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856a5f5-d6dc-4572-a3ff-a8c4537d35ea None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:21.764621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856a5f5-d6dc-4572-a3ff-a8c4537d35ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:21.784656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4856a5f5-d6dc-4572-a3ff-a8c4537d35ea None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:21.785843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4856a5f5-d6dc-4572-a3ff-a8c4537d35ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:21.793581 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4111/8309] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:28:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:21.806550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45decb0-f43e-42a6-b040-63c7bcab02d4 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:21.820417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45decb0-f43e-42a6-b040-63c7bcab02d4 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:21.826558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4463f6-76e7-48bf-abc3-b360d3eeffe6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:21.827169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4463f6-76e7-48bf-abc3-b360d3eeffe6 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:21.827441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4463f6-76e7-48bf-abc3-b360d3eeffe6 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:21.827706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4463f6-76e7-48bf-abc3-b360d3eeffe6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:21.836279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4463f6-76e7-48bf-abc3-b360d3eeffe6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:21.837921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4463f6-76e7-48bf-abc3-b360d3eeffe6 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:21.846113 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4199/8310] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:28:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:28:21.995198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a3244-d42b-4d77-ba49-3b0fa01b8616 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:21.995497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a3244-d42b-4d77-ba49-3b0fa01b8616 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:21.995770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a3244-d42b-4d77-ba49-3b0fa01b8616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:22.022030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ac8a3244-d42b-4d77-ba49-3b0fa01b8616 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:22.023242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ac8a3244-d42b-4d77-ba49-3b0fa01b8616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:22.032141 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4112/8311] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:28:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:22.049611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4463f6-76e7-48bf-abc3-b360d3eeffe6 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:22.068907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4463f6-76e7-48bf-abc3-b360d3eeffe6 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:22.075311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1bb18-cf76-4ae6-9123-3517471e726c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:22.075927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1bb18-cf76-4ae6-9123-3517471e726c service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:22.076192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1bb18-cf76-4ae6-9123-3517471e726c service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:22.076483 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1bb18-cf76-4ae6-9123-3517471e726c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:22.085546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1bb18-cf76-4ae6-9123-3517471e726c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:22.087162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1bb18-cf76-4ae6-9123-3517471e726c service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:22.095192 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4200/8312] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:28:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:28:22.252089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52956-d0f4-4059-a895-070b4fa453bb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:22.252400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52956-d0f4-4059-a895-070b4fa453bb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:22.252874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52956-d0f4-4059-a895-070b4fa453bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:22.260641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:22.275140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e2e52956-d0f4-4059-a895-070b4fa453bb None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:22.276515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e2e52956-d0f4-4059-a895-070b4fa453bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:22.284953 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4113/8313] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:28:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:22.297465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1bb18-cf76-4ae6-9123-3517471e726c service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:22.311432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1bb18-cf76-4ae6-9123-3517471e726c service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:22.317739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbbe79-e0da-4ba4-9d2f-ac366cf58f3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:22.318330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbbe79-e0da-4ba4-9d2f-ac366cf58f3a service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:22.318597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbbe79-e0da-4ba4-9d2f-ac366cf58f3a service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:22.318854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbbe79-e0da-4ba4-9d2f-ac366cf58f3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:22.328229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbbe79-e0da-4ba4-9d2f-ac366cf58f3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:22.329860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbbe79-e0da-4ba4-9d2f-ac366cf58f3a service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:22.338637 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4201/8314] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:28:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:28:22.485483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ce1b6-f1a7-422d-b74c-c172bbfde490 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:22.485767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ce1b6-f1a7-422d-b74c-c172bbfde490 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:22.486087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ce1b6-f1a7-422d-b74c-c172bbfde490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:22.505627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3c4ce1b6-f1a7-422d-b74c-c172bbfde490 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:22.506837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3c4ce1b6-f1a7-422d-b74c-c172bbfde490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:22.514443 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4114/8315] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:28:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:22.526912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbbe79-e0da-4ba4-9d2f-ac366cf58f3a service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:22.543862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbbe79-e0da-4ba4-9d2f-ac366cf58f3a service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:22.553217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb15e5-06a3-40c8-baf8-16f0f488b505 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:22.553899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb15e5-06a3-40c8-baf8-16f0f488b505 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:22.554262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb15e5-06a3-40c8-baf8-16f0f488b505 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:22.554615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb15e5-06a3-40c8-baf8-16f0f488b505 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:22.567644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb15e5-06a3-40c8-baf8-16f0f488b505 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:22.569281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb15e5-06a3-40c8-baf8-16f0f488b505 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:22.577808 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4202/8316] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:28:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:28:22.725435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f544d-7724-4d5e-8c97-00a381aba42e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:22.725937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f544d-7724-4d5e-8c97-00a381aba42e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:22.725937 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f544d-7724-4d5e-8c97-00a381aba42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:22.746131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6a7f544d-7724-4d5e-8c97-00a381aba42e None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:22.747339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6a7f544d-7724-4d5e-8c97-00a381aba42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:22.756046 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4115/8317] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:28:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:22.769064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb15e5-06a3-40c8-baf8-16f0f488b505 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:22.782329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb15e5-06a3-40c8-baf8-16f0f488b505 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:22.788632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e60f6-da5b-4201-aee8-17d87013535b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:22.789227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e60f6-da5b-4201-aee8-17d87013535b service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:22.789498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e60f6-da5b-4201-aee8-17d87013535b service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:22.789758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e60f6-da5b-4201-aee8-17d87013535b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:22.798590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e60f6-da5b-4201-aee8-17d87013535b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:22.800688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e60f6-da5b-4201-aee8-17d87013535b service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:22.805498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:22.809937 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4203/8318] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:28:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:28:22.926183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f544d-7724-4d5e-8c97-00a381aba42e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:22.994506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f544d-7724-4d5e-8c97-00a381aba42e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:22.999253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:22.999876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:23.000141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:23.000439 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] PATH_INFO: `/v3/users/0dc21474f512473c856476d56247cd97` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:23.001586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dc21474f512473c856476d56247cd97)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:23.003713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:23.056154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dc21474f512473c856476d56247cd97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:23.064816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dc21474f512473c856476d56247cd97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:23.079646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dc21474f512473c856476d56247cd97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:23.080524 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dc21474f512473c856476d56247cd97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:23.081866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dc21474f512473c856476d56247cd97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:23.094957 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4116/8319] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:22 2022] DELETE /identity/v3/users/0dc21474f512473c856476d56247cd97 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:23.100364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01988f-798b-4af1-ad24-a07eb9c202e2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:23.100697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01988f-798b-4af1-ad24-a07eb9c202e2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:23.101032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01988f-798b-4af1-ad24-a07eb9c202e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:23.121129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8d01988f-798b-4af1-ad24-a07eb9c202e2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:23.122334 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8d01988f-798b-4af1-ad24-a07eb9c202e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:23.185283 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4204/8320] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:23.197285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:23.198931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:23.264622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47347b-1f0b-4f3b-8f15-283ed1a5b424 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:23.269325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acd814-649a-4b6e-abce-d101d5a0e4fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:23.269931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acd814-649a-4b6e-abce-d101d5a0e4fc service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:23.270206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acd814-649a-4b6e-abce-d101d5a0e4fc service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:23.270472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acd814-649a-4b6e-abce-d101d5a0e4fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:23.279989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acd814-649a-4b6e-abce-d101d5a0e4fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:23.281483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acd814-649a-4b6e-abce-d101d5a0e4fc service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:23.291476 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4117/8321] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:23.581058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01988f-798b-4af1-ad24-a07eb9c202e2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:23.582559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8d01988f-798b-4af1-ad24-a07eb9c202e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:23.595922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01988f-798b-4af1-ad24-a07eb9c202e2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:23.601755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:23.602434 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:23.602778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:23.602974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a admin admin] PATH_INFO: `/v3/projects/bb4fe2b86e7b4846835290b1da59e42b` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:23.604299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb4fe2b86e7b4846835290b1da59e42b)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:23.606353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:23.656027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb4fe2b86e7b4846835290b1da59e42b', 'request_id': 'req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:23.665117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb4fe2b86e7b4846835290b1da59e42b', 'request_id': 'req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:23.670981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb4fe2b86e7b4846835290b1da59e42b', 'request_id': 'req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:23.682280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a admin admin] The token cache is being invalidate because project bb4fe2b86e7b4846835290b1da59e42b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:23.682555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:23.683190 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-10fd1e5a-cad8-4d03-a16d-e5a829c0712a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:23.684770 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4205/8322] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:23 2022] DELETE /identity/v3/projects/bb4fe2b86e7b4846835290b1da59e42b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:23.690376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c40a6-8b71-4b2d-b1bf-894aed063e7f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:23.690643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c40a6-8b71-4b2d-b1bf-894aed063e7f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:23.691045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c40a6-8b71-4b2d-b1bf-894aed063e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:23.702300 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:23.717296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-451c40a6-8b71-4b2d-b1bf-894aed063e7f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:23.723652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-451c40a6-8b71-4b2d-b1bf-894aed063e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:23.788847 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4118/8323] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:23.796935 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6add1d4-efd9-4ddb-b1b7-1e57a02ceaab None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:23.797403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6add1d4-efd9-4ddb-b1b7-1e57a02ceaab None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:23.797523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6add1d4-efd9-4ddb-b1b7-1e57a02ceaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:23.819504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f6add1d4-efd9-4ddb-b1b7-1e57a02ceaab None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:23.820764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f6add1d4-efd9-4ddb-b1b7-1e57a02ceaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:23.829411 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4206/8324] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:23.834650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c40a6-8b71-4b2d-b1bf-894aed063e7f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:23.844700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c40a6-8b71-4b2d-b1bf-894aed063e7f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:23.850143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d68cc0-e52b-47bf-bfe0-ca362978bbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:23.850747 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d68cc0-e52b-47bf-bfe0-ca362978bbda admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:23.850999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d68cc0-e52b-47bf-bfe0-ca362978bbda admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:23.851274 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d68cc0-e52b-47bf-bfe0-ca362978bbda admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:23.851901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d68cc0-e52b-47bf-bfe0-ca362978bbda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:23.852202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d68cc0-e52b-47bf-bfe0-ca362978bbda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:23.853876 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d68cc0-e52b-47bf-bfe0-ca362978bbda admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:23.865805 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4119/8325] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:28:23.870145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6add1d4-efd9-4ddb-b1b7-1e57a02ceaab None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:23.879832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6add1d4-efd9-4ddb-b1b7-1e57a02ceaab None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:23.884726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed23b1-17f7-4813-90b3-d0d1a3d9e282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:23.885318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed23b1-17f7-4813-90b3-d0d1a3d9e282 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:23.885586 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed23b1-17f7-4813-90b3-d0d1a3d9e282 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:23.885846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed23b1-17f7-4813-90b3-d0d1a3d9e282 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:23.886456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed23b1-17f7-4813-90b3-d0d1a3d9e282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:23.888344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed23b1-17f7-4813-90b3-d0d1a3d9e282 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:23.923205 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4207/8326] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:23 2022] POST /identity/v3/projects => generated 359 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:23.928200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d68cc0-e52b-47bf-bfe0-ca362978bbda admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:23.938860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d68cc0-e52b-47bf-bfe0-ca362978bbda admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:24.004143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e6e59-7986-4a7f-ab4f-d15039c92092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:24.004799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e6e59-7986-4a7f-ab4f-d15039c92092 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:24.005070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e6e59-7986-4a7f-ab4f-d15039c92092 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:24.005331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e6e59-7986-4a7f-ab4f-d15039c92092 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:24.006391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e6e59-7986-4a7f-ab4f-d15039c92092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:24.007863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e6e59-7986-4a7f-ab4f-d15039c92092 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:24.036199 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4120/8327] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:23 2022] POST /identity/v3/users => generated 280 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:24.037608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed23b1-17f7-4813-90b3-d0d1a3d9e282 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:24.051529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed23b1-17f7-4813-90b3-d0d1a3d9e282 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:24.056612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bc983-ea73-47a5-85fe-890ce51c23dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:24.057568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bc983-ea73-47a5-85fe-890ce51c23dc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:24.058014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bc983-ea73-47a5-85fe-890ce51c23dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:24.058449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bc983-ea73-47a5-85fe-890ce51c23dc admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:24.059535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bc983-ea73-47a5-85fe-890ce51c23dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:24.061217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bc983-ea73-47a5-85fe-890ce51c23dc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:24.072281 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4208/8328] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:24.077067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e6e59-7986-4a7f-ab4f-d15039c92092 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:24.092311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e6e59-7986-4a7f-ab4f-d15039c92092 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:24.099258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b87ef-0184-422b-9266-ae06f88c5c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:24.099858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b87ef-0184-422b-9266-ae06f88c5c97 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:24.100111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b87ef-0184-422b-9266-ae06f88c5c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:24.100458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b87ef-0184-422b-9266-ae06f88c5c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:24.101040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b87ef-0184-422b-9266-ae06f88c5c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:24.102627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b87ef-0184-422b-9266-ae06f88c5c97 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:24.113925 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4121/8329] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:24.118175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bc983-ea73-47a5-85fe-890ce51c23dc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:24.128891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bc983-ea73-47a5-85fe-890ce51c23dc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:24.133745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46ebe0-5c22-4bde-a0ff-633150f687be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:24.134401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46ebe0-5c22-4bde-a0ff-633150f687be admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:24.134701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46ebe0-5c22-4bde-a0ff-633150f687be admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:24.134961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46ebe0-5c22-4bde-a0ff-633150f687be admin admin] PATH_INFO: `/v3/projects/95b7da3f62e34a4a9e16a76af30b0548/users/0ff3d23fe2ab4b769caab2ed3572aa79/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:24.151040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46ebe0-5c22-4bde-a0ff-633150f687be admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b7da3f62e34a4a9e16a76af30b0548, user_id=0ff3d23fe2ab4b769caab2ed3572aa79, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:24.153732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46ebe0-5c22-4bde-a0ff-633150f687be admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:24.168191 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4209/8330] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:24 2022] PUT /identity/v3/projects/95b7da3f62e34a4a9e16a76af30b0548/users/0ff3d23fe2ab4b769caab2ed3572aa79/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:24.179069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d465e9-7d3f-4791-91aa-0cdaee43426b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:24.179292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d465e9-7d3f-4791-91aa-0cdaee43426b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:24.179560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d465e9-7d3f-4791-91aa-0cdaee43426b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:24.208048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-07d465e9-7d3f-4791-91aa-0cdaee43426b None None] MFA Rules not processed for user `0ff3d23fe2ab4b769caab2ed3572aa79`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:24.209296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-07d465e9-7d3f-4791-91aa-0cdaee43426b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:24.276891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:24.280782 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4122/8331] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:24 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:24.294123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46ebe0-5c22-4bde-a0ff-633150f687be admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:24.319067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:24.364318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46ebe0-5c22-4bde-a0ff-633150f687be admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:24.369251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:24.369879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:24.370150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:24.370415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] PATH_INFO: `/v3/users/0ff3d23fe2ab4b769caab2ed3572aa79` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:24.371544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ff3d23fe2ab4b769caab2ed3572aa79)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:24.373536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:24.428829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ff3d23fe2ab4b769caab2ed3572aa79'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:24.438361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ff3d23fe2ab4b769caab2ed3572aa79'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:24.450791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ff3d23fe2ab4b769caab2ed3572aa79'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:24.464204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ff3d23fe2ab4b769caab2ed3572aa79'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:24.465038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ff3d23fe2ab4b769caab2ed3572aa79'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:24.470977 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4210/8332] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:24 2022] DELETE /identity/v3/users/0ff3d23fe2ab4b769caab2ed3572aa79 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:24.475943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239bc73-e651-412a-bef2-4e3952f9de47 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:24.476230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239bc73-e651-412a-bef2-4e3952f9de47 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:24.476529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239bc73-e651-412a-bef2-4e3952f9de47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:24.497205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8239bc73-e651-412a-bef2-4e3952f9de47 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:24.498495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8239bc73-e651-412a-bef2-4e3952f9de47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:24.560068 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4123/8333] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:28:24.572311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:24.573774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:24.636057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eff007-f240-4ced-9501-4bca7e5b9c99 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:24.640398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54209b-22ea-49f5-979c-ed91ee96d2c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:24.641040 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54209b-22ea-49f5-979c-ed91ee96d2c0 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:24.641311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54209b-22ea-49f5-979c-ed91ee96d2c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:24.641575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54209b-22ea-49f5-979c-ed91ee96d2c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:24.651527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54209b-22ea-49f5-979c-ed91ee96d2c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:24.653038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54209b-22ea-49f5-979c-ed91ee96d2c0 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:24.662927 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4211/8334] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:28:24.946132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239bc73-e651-412a-bef2-4e3952f9de47 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:24.947699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8239bc73-e651-412a-bef2-4e3952f9de47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:24.962119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239bc73-e651-412a-bef2-4e3952f9de47 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:24.968287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:24.968985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:24.969238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:24.969526 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b admin admin] PATH_INFO: `/v3/projects/95b7da3f62e34a4a9e16a76af30b0548` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:24.970788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b admin admin] RBAC: Authorizing `identity:delete_project(project_id=95b7da3f62e34a4a9e16a76af30b0548)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:24.972851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:25.020560 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95b7da3f62e34a4a9e16a76af30b0548', 'request_id': 'req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:25.026936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95b7da3f62e34a4a9e16a76af30b0548', 'request_id': 'req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:25.035135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95b7da3f62e34a4a9e16a76af30b0548', 'request_id': 'req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:25.045601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b admin admin] The token cache is being invalidate because project 95b7da3f62e34a4a9e16a76af30b0548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:25.045865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:25.046477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ef3ab6fd-be56-48eb-b5bc-228ecd88a72b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:25.047983 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4124/8335] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:24 2022] DELETE /identity/v3/projects/95b7da3f62e34a4a9e16a76af30b0548 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:25.052860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294a98b-a355-4f26-9463-0ba97213caa7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:25.053382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294a98b-a355-4f26-9463-0ba97213caa7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:25.053382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294a98b-a355-4f26-9463-0ba97213caa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:25.072820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d294a98b-a355-4f26-9463-0ba97213caa7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:25.074002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d294a98b-a355-4f26-9463-0ba97213caa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:25.114875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:25.137817 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4212/8336] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:25.146880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7671579-4dc1-4224-b80a-60c649f85404 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:25.147142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7671579-4dc1-4224-b80a-60c649f85404 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:25.147417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7671579-4dc1-4224-b80a-60c649f85404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:25.169008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b7671579-4dc1-4224-b80a-60c649f85404 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:25.170199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b7671579-4dc1-4224-b80a-60c649f85404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:25.179679 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4125/8337] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:25.183927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294a98b-a355-4f26-9463-0ba97213caa7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:25.195876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294a98b-a355-4f26-9463-0ba97213caa7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:25.205163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4363c-6072-474c-9e5c-1dc3858ae87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:25.205926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4363c-6072-474c-9e5c-1dc3858ae87c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:25.206194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4363c-6072-474c-9e5c-1dc3858ae87c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:25.206447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4363c-6072-474c-9e5c-1dc3858ae87c admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:25.207078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa4363c-6072-474c-9e5c-1dc3858ae87c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:25.207369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa4363c-6072-474c-9e5c-1dc3858ae87c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:25.208985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa4363c-6072-474c-9e5c-1dc3858ae87c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:25.221644 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4213/8338] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:28:25.226132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7671579-4dc1-4224-b80a-60c649f85404 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:25.235943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7671579-4dc1-4224-b80a-60c649f85404 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:25.241286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b35830-d8bd-4401-982b-adb2954f34e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:25.241894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b35830-d8bd-4401-982b-adb2954f34e0 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:25.242164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b35830-d8bd-4401-982b-adb2954f34e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:25.242429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b35830-d8bd-4401-982b-adb2954f34e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:25.243048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b35830-d8bd-4401-982b-adb2954f34e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:25.245001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b35830-d8bd-4401-982b-adb2954f34e0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:25.278411 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4126/8339] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:25 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:25.281838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4363c-6072-474c-9e5c-1dc3858ae87c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:25.292407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4363c-6072-474c-9e5c-1dc3858ae87c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:25.369489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b1efe-74bf-412b-b5ed-52a18d4b13ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:25.370103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b1efe-74bf-412b-b5ed-52a18d4b13ae admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:25.370375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b1efe-74bf-412b-b5ed-52a18d4b13ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:25.370642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b1efe-74bf-412b-b5ed-52a18d4b13ae admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:25.371268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b1efe-74bf-412b-b5ed-52a18d4b13ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:25.373282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b1efe-74bf-412b-b5ed-52a18d4b13ae admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:25.399890 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4214/8340] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:25 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:25.403764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b35830-d8bd-4401-982b-adb2954f34e0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:25.415063 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b35830-d8bd-4401-982b-adb2954f34e0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:25.421011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f8d02-be3b-437d-908e-9b782d5aa123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:25.421611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f8d02-be3b-437d-908e-9b782d5aa123 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:25.421885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f8d02-be3b-437d-908e-9b782d5aa123 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:25.422154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f8d02-be3b-437d-908e-9b782d5aa123 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:25.422793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f8d02-be3b-437d-908e-9b782d5aa123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:25.424503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f8d02-be3b-437d-908e-9b782d5aa123 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:25.435368 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4127/8341] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:25.439984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b1efe-74bf-412b-b5ed-52a18d4b13ae admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:25.451739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b1efe-74bf-412b-b5ed-52a18d4b13ae admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:25.459127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118264c-3c69-45f9-a777-13107ddbd373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:25.460099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118264c-3c69-45f9-a777-13107ddbd373 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:25.460564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118264c-3c69-45f9-a777-13107ddbd373 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:25.460838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118264c-3c69-45f9-a777-13107ddbd373 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:25.463576 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118264c-3c69-45f9-a777-13107ddbd373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:25.465274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118264c-3c69-45f9-a777-13107ddbd373 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:25.477790 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4215/8342] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:25.482342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f8d02-be3b-437d-908e-9b782d5aa123 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:25.493404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f8d02-be3b-437d-908e-9b782d5aa123 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:25.499163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff398c2-e70e-4bec-be3b-b9035cd40280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:25.499822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff398c2-e70e-4bec-be3b-b9035cd40280 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:25.500095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff398c2-e70e-4bec-be3b-b9035cd40280 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:25.500596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff398c2-e70e-4bec-be3b-b9035cd40280 admin admin] PATH_INFO: `/v3/projects/ab586e49c0644ec699ae4427110782eb/users/0fc78d399d68426ea057f1293da49204/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:25.523414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff398c2-e70e-4bec-be3b-b9035cd40280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab586e49c0644ec699ae4427110782eb, user_id=0fc78d399d68426ea057f1293da49204, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:25.527321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff398c2-e70e-4bec-be3b-b9035cd40280 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:25.543084 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4128/8343] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:25 2022] PUT /identity/v3/projects/ab586e49c0644ec699ae4427110782eb/users/0fc78d399d68426ea057f1293da49204/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:25.554090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbc9f5-7136-4068-84a5-3f5a945766f1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:25.554373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbc9f5-7136-4068-84a5-3f5a945766f1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:25.554631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbc9f5-7136-4068-84a5-3f5a945766f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:25.585627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-63bbc9f5-7136-4068-84a5-3f5a945766f1 None None] MFA Rules not processed for user `0fc78d399d68426ea057f1293da49204`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:25.586945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-63bbc9f5-7136-4068-84a5-3f5a945766f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:25.666248 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4216/8344] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:25.677172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff398c2-e70e-4bec-be3b-b9035cd40280 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:25.679160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-aff398c2-e70e-4bec-be3b-b9035cd40280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:25.703062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:25.744678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff398c2-e70e-4bec-be3b-b9035cd40280 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:25.753172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b013da5-0df5-4f33-a140-b84d678f19b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:25.753829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b013da5-0df5-4f33-a140-b84d678f19b5 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:25.754107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b013da5-0df5-4f33-a140-b84d678f19b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:25.754372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b013da5-0df5-4f33-a140-b84d678f19b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:25.764970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b013da5-0df5-4f33-a140-b84d678f19b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:25.766415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b013da5-0df5-4f33-a140-b84d678f19b5 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:25.775739 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4129/8345] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:25.805256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bbc9f5-7136-4068-84a5-3f5a945766f1 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:25.849784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:25.873415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bbc9f5-7136-4068-84a5-3f5a945766f1 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:25.878449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:25.879075 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:25.879341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:25.879603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] PATH_INFO: `/v3/users/0fc78d399d68426ea057f1293da49204` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:25.880785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc78d399d68426ea057f1293da49204)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:25.882709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:25.933491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc78d399d68426ea057f1293da49204'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:25.943616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc78d399d68426ea057f1293da49204'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:25.951666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc78d399d68426ea057f1293da49204'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:25.964221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc78d399d68426ea057f1293da49204'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:25.964900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc78d399d68426ea057f1293da49204'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:25.970687 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4217/8346] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:25 2022] DELETE /identity/v3/users/0fc78d399d68426ea057f1293da49204 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:25.975289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac6c12-4df6-4d7a-a4ab-48d95a7fb7c8 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:25.975794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac6c12-4df6-4d7a-a4ab-48d95a7fb7c8 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:25.975794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac6c12-4df6-4d7a-a4ab-48d95a7fb7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:25.995389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-51ac6c12-4df6-4d7a-a4ab-48d95a7fb7c8 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:25.996681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-51ac6c12-4df6-4d7a-a4ab-48d95a7fb7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:26.058982 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4130/8347] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:26.071427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:26.073459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:26.145802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c67e7a-c990-4cae-a0e3-856f408c9626 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:26.149896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba98ca-2311-4e45-843c-1755729d0a07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:26.150509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba98ca-2311-4e45-843c-1755729d0a07 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:26.150763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba98ca-2311-4e45-843c-1755729d0a07 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:26.151026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba98ca-2311-4e45-843c-1755729d0a07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:26.161201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba98ca-2311-4e45-843c-1755729d0a07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:26.162637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba98ca-2311-4e45-843c-1755729d0a07 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:26.172363 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4218/8348] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:26.461639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac6c12-4df6-4d7a-a4ab-48d95a7fb7c8 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:26.463525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-51ac6c12-4df6-4d7a-a4ab-48d95a7fb7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:26.476872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac6c12-4df6-4d7a-a4ab-48d95a7fb7c8 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:26.484836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c899a-1196-4ce4-85b5-8c1e896474d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:26.485792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c899a-1196-4ce4-85b5-8c1e896474d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:26.486437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c899a-1196-4ce4-85b5-8c1e896474d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:26.486836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c899a-1196-4ce4-85b5-8c1e896474d7 admin admin] PATH_INFO: `/v3/projects/ab586e49c0644ec699ae4427110782eb` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:26.488220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c899a-1196-4ce4-85b5-8c1e896474d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab586e49c0644ec699ae4427110782eb)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:26.490305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c899a-1196-4ce4-85b5-8c1e896474d7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:26.539206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-548c899a-1196-4ce4-85b5-8c1e896474d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab586e49c0644ec699ae4427110782eb', 'request_id': 'req-548c899a-1196-4ce4-85b5-8c1e896474d7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:26.545993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-548c899a-1196-4ce4-85b5-8c1e896474d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab586e49c0644ec699ae4427110782eb', 'request_id': 'req-548c899a-1196-4ce4-85b5-8c1e896474d7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:26.555198 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-548c899a-1196-4ce4-85b5-8c1e896474d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab586e49c0644ec699ae4427110782eb', 'request_id': 'req-548c899a-1196-4ce4-85b5-8c1e896474d7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:26.567068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-548c899a-1196-4ce4-85b5-8c1e896474d7 admin admin] The token cache is being invalidate because project ab586e49c0644ec699ae4427110782eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:26.567350 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-548c899a-1196-4ce4-85b5-8c1e896474d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:26.567993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-548c899a-1196-4ce4-85b5-8c1e896474d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:26.569566 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4131/8349] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:26 2022] DELETE /identity/v3/projects/ab586e49c0644ec699ae4427110782eb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:26.574535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348ada-36a7-48ad-bc53-e5e61a65cb42 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:26.574834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348ada-36a7-48ad-bc53-e5e61a65cb42 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:26.575168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348ada-36a7-48ad-bc53-e5e61a65cb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:26.596078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c3348ada-36a7-48ad-bc53-e5e61a65cb42 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:26.597596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c3348ada-36a7-48ad-bc53-e5e61a65cb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:26.651199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:26.661687 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4219/8350] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:26.670079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769fb17-f10a-420d-8818-c00e9a019ea5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:26.670478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769fb17-f10a-420d-8818-c00e9a019ea5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:26.670846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769fb17-f10a-420d-8818-c00e9a019ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:26.693231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b769fb17-f10a-420d-8818-c00e9a019ea5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:26.694441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b769fb17-f10a-420d-8818-c00e9a019ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:26.702887 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4132/8351] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:26.707714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348ada-36a7-48ad-bc53-e5e61a65cb42 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:26.718132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348ada-36a7-48ad-bc53-e5e61a65cb42 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:26.724003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2211d4c-0ba4-4a52-9410-d87ccfa0c85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:26.724624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2211d4c-0ba4-4a52-9410-d87ccfa0c85d admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:26.724892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2211d4c-0ba4-4a52-9410-d87ccfa0c85d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:26.725149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2211d4c-0ba4-4a52-9410-d87ccfa0c85d admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:26.725814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2211d4c-0ba4-4a52-9410-d87ccfa0c85d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:26.726106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2211d4c-0ba4-4a52-9410-d87ccfa0c85d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:26.727744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2211d4c-0ba4-4a52-9410-d87ccfa0c85d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:26.740094 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4220/8352] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:28:26.744700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769fb17-f10a-420d-8818-c00e9a019ea5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:26.754631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769fb17-f10a-420d-8818-c00e9a019ea5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:26.760117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326cad1-e433-4e79-b20f-d2b718b0d3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:26.760789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326cad1-e433-4e79-b20f-d2b718b0d3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:26.761042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326cad1-e433-4e79-b20f-d2b718b0d3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:26.761328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326cad1-e433-4e79-b20f-d2b718b0d3d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:26.761941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326cad1-e433-4e79-b20f-d2b718b0d3d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:26.763875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326cad1-e433-4e79-b20f-d2b718b0d3d4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:26.795120 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4133/8353] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:26 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:28:26.799491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2211d4c-0ba4-4a52-9410-d87ccfa0c85d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:26.809901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2211d4c-0ba4-4a52-9410-d87ccfa0c85d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:26.871122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9713d7-a728-4d01-bcf8-c3faa6e75178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:26.871641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9713d7-a728-4d01-bcf8-c3faa6e75178 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:26.871906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9713d7-a728-4d01-bcf8-c3faa6e75178 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:26.872162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9713d7-a728-4d01-bcf8-c3faa6e75178 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:26.872807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9713d7-a728-4d01-bcf8-c3faa6e75178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:26.874758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9713d7-a728-4d01-bcf8-c3faa6e75178 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:26.898570 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4221/8354] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:26 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:26.902971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326cad1-e433-4e79-b20f-d2b718b0d3d4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:26.914267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326cad1-e433-4e79-b20f-d2b718b0d3d4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:26.919840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d655f6-057f-419f-9050-729b63224d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:26.920482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d655f6-057f-419f-9050-729b63224d2a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:26.920765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d655f6-057f-419f-9050-729b63224d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:26.921021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d655f6-057f-419f-9050-729b63224d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:26.921651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d655f6-057f-419f-9050-729b63224d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:26.923286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d655f6-057f-419f-9050-729b63224d2a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:26.933786 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4134/8355] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:28:26.937785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9713d7-a728-4d01-bcf8-c3faa6e75178 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:26.948427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9713d7-a728-4d01-bcf8-c3faa6e75178 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:26.953880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a7c7b-3561-4994-9f06-bc70e3224998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:26.954578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a7c7b-3561-4994-9f06-bc70e3224998 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:26.954856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a7c7b-3561-4994-9f06-bc70e3224998 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:26.955103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a7c7b-3561-4994-9f06-bc70e3224998 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:26.955881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a7c7b-3561-4994-9f06-bc70e3224998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:26.957542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a7c7b-3561-4994-9f06-bc70e3224998 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:26.968233 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4222/8356] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:26.972884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d655f6-057f-419f-9050-729b63224d2a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:26.983780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d655f6-057f-419f-9050-729b63224d2a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:26.989520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19d27d-5677-459e-b0ac-626373934eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:26.990143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19d27d-5677-459e-b0ac-626373934eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:26.990433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19d27d-5677-459e-b0ac-626373934eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:26.990686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19d27d-5677-459e-b0ac-626373934eeb admin admin] PATH_INFO: `/v3/projects/6704a32cf1c54b76b8cf84ed93fd6391/users/6cdaf81bbea34762b069adb7b3b563e5/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:26.993068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:27.009541 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19d27d-5677-459e-b0ac-626373934eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6704a32cf1c54b76b8cf84ed93fd6391, user_id=6cdaf81bbea34762b069adb7b3b563e5, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:27.012212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19d27d-5677-459e-b0ac-626373934eeb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:27.026490 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4135/8357] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:26 2022] PUT /identity/v3/projects/6704a32cf1c54b76b8cf84ed93fd6391/users/6cdaf81bbea34762b069adb7b3b563e5/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:27.131549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5e0b7-9675-4632-9458-85c5cd65f9ee None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:27.132455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5e0b7-9675-4632-9458-85c5cd65f9ee None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:27.132455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5e0b7-9675-4632-9458-85c5cd65f9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:27.173762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-90d5e0b7-9675-4632-9458-85c5cd65f9ee None None] MFA Rules not processed for user `6cdaf81bbea34762b069adb7b3b563e5`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:27.175022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-90d5e0b7-9675-4632-9458-85c5cd65f9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:27.243693 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4223/8358] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:27.255584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19d27d-5677-459e-b0ac-626373934eeb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:27.257436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9b19d27d-5677-459e-b0ac-626373934eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:27.326448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19d27d-5677-459e-b0ac-626373934eeb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:27.330962 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d2eb7-9a95-4492-a94f-79996f32d694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:27.331596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d2eb7-9a95-4492-a94f-79996f32d694 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:27.331864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d2eb7-9a95-4492-a94f-79996f32d694 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:27.332130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d2eb7-9a95-4492-a94f-79996f32d694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:27.345858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d2eb7-9a95-4492-a94f-79996f32d694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:27.348088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d2eb7-9a95-4492-a94f-79996f32d694 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:27.362811 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4136/8359] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:27.923951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5e0b7-9675-4632-9458-85c5cd65f9ee None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:27.985185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:27.992963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5e0b7-9675-4632-9458-85c5cd65f9ee None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:27.998496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:27.999148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:27.999421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:27.999676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] PATH_INFO: `/v3/users/6cdaf81bbea34762b069adb7b3b563e5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:28.000858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cdaf81bbea34762b069adb7b3b563e5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:28.003312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:28.063679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cdaf81bbea34762b069adb7b3b563e5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:28.074743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cdaf81bbea34762b069adb7b3b563e5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:28.084360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cdaf81bbea34762b069adb7b3b563e5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:28.096001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cdaf81bbea34762b069adb7b3b563e5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:28.096736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cdaf81bbea34762b069adb7b3b563e5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:28.102506 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4224/8360] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:27 2022] DELETE /identity/v3/users/6cdaf81bbea34762b069adb7b3b563e5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:28.107195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f13585-fe64-452c-ad78-a8b74c874fbd None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:28.107698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f13585-fe64-452c-ad78-a8b74c874fbd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:28.107819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f13585-fe64-452c-ad78-a8b74c874fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:28.128150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-59f13585-fe64-452c-ad78-a8b74c874fbd None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:28.130714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-59f13585-fe64-452c-ad78-a8b74c874fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:28.219641 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4137/8361] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:28.230532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:28.232420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:28.294615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc91b7-9bf1-4270-a3f1-d299a1fd5dd5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:28.298391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ebe1c-54f8-4eec-bb98-4d0ce4a648f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:28.298990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ebe1c-54f8-4eec-bb98-4d0ce4a648f5 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:28.299266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ebe1c-54f8-4eec-bb98-4d0ce4a648f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:28.299651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ebe1c-54f8-4eec-bb98-4d0ce4a648f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:28.309046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ebe1c-54f8-4eec-bb98-4d0ce4a648f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:28.310507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ebe1c-54f8-4eec-bb98-4d0ce4a648f5 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:28.321269 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4225/8362] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:28.607669 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f13585-fe64-452c-ad78-a8b74c874fbd None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:28.609407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-59f13585-fe64-452c-ad78-a8b74c874fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:28.622113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f13585-fe64-452c-ad78-a8b74c874fbd None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:28.628606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:28.629499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:28.629778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:28.630020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e admin admin] PATH_INFO: `/v3/projects/6704a32cf1c54b76b8cf84ed93fd6391` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:28.631316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6704a32cf1c54b76b8cf84ed93fd6391)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:28.633319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:28.682783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6704a32cf1c54b76b8cf84ed93fd6391', 'request_id': 'req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:28.690553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6704a32cf1c54b76b8cf84ed93fd6391', 'request_id': 'req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:28.700689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6704a32cf1c54b76b8cf84ed93fd6391', 'request_id': 'req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:28.712321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e admin admin] The token cache is being invalidate because project 6704a32cf1c54b76b8cf84ed93fd6391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:28.712701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:28.713327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a3a5761b-4630-4ced-b2f7-06ee91b29d2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:28.714853 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4138/8363] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:28 2022] DELETE /identity/v3/projects/6704a32cf1c54b76b8cf84ed93fd6391 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:28.719966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea2163-3bf8-4487-9324-ea5ee5244e22 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:28.720245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea2163-3bf8-4487-9324-ea5ee5244e22 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:28.720567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea2163-3bf8-4487-9324-ea5ee5244e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:28.741340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-29ea2163-3bf8-4487-9324-ea5ee5244e22 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:28.742542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-29ea2163-3bf8-4487-9324-ea5ee5244e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:28.805052 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4226/8364] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:28.813026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593a4fe-6b33-46dd-8145-8bd7ee1181f1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:28.813301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593a4fe-6b33-46dd-8145-8bd7ee1181f1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:28.813577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593a4fe-6b33-46dd-8145-8bd7ee1181f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:28.835537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9593a4fe-6b33-46dd-8145-8bd7ee1181f1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:28.836782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9593a4fe-6b33-46dd-8145-8bd7ee1181f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:28.845627 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4139/8365] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:28.852946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea2163-3bf8-4487-9324-ea5ee5244e22 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:28.867525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea2163-3bf8-4487-9324-ea5ee5244e22 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:28.874006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ceac3f-fd71-40f9-bf82-8b00b2076480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:28.874618 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceac3f-fd71-40f9-bf82-8b00b2076480 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:28.874886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceac3f-fd71-40f9-bf82-8b00b2076480 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:28.875144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceac3f-fd71-40f9-bf82-8b00b2076480 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:28.875802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ceac3f-fd71-40f9-bf82-8b00b2076480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:28.876094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ceac3f-fd71-40f9-bf82-8b00b2076480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:28.878296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ceac3f-fd71-40f9-bf82-8b00b2076480 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:28.890563 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4227/8366] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:28:28.895919 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593a4fe-6b33-46dd-8145-8bd7ee1181f1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:28.900627 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:28.907841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593a4fe-6b33-46dd-8145-8bd7ee1181f1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:28.915082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c1883-c8f6-455b-a822-dfd7c968c3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:28.915974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c1883-c8f6-455b-a822-dfd7c968c3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:28.916393 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c1883-c8f6-455b-a822-dfd7c968c3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:28.916711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c1883-c8f6-455b-a822-dfd7c968c3e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:28.917639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c1883-c8f6-455b-a822-dfd7c968c3e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:28.919690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c1883-c8f6-455b-a822-dfd7c968c3e0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:28.949250 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4140/8367] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:28 2022] POST /identity/v3/projects => generated 353 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:28.953530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ceac3f-fd71-40f9-bf82-8b00b2076480 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:28.960280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:28.966827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ceac3f-fd71-40f9-bf82-8b00b2076480 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:29.033966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777467a-0f6e-4195-9aaf-c5c110fdc3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:29.034491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777467a-0f6e-4195-9aaf-c5c110fdc3ef admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:29.034763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777467a-0f6e-4195-9aaf-c5c110fdc3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:29.035033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777467a-0f6e-4195-9aaf-c5c110fdc3ef admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:29.035655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777467a-0f6e-4195-9aaf-c5c110fdc3ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:29.037637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777467a-0f6e-4195-9aaf-c5c110fdc3ef admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:29.060663 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4228/8368] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:28 2022] POST /identity/v3/users => generated 277 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:29.064922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c1883-c8f6-455b-a822-dfd7c968c3e0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:29.078805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c1883-c8f6-455b-a822-dfd7c968c3e0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:29.084739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecfe25-3093-4c46-b4e2-c27291a4e084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:29.085343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecfe25-3093-4c46-b4e2-c27291a4e084 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:29.085616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecfe25-3093-4c46-b4e2-c27291a4e084 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:29.085880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecfe25-3093-4c46-b4e2-c27291a4e084 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:29.086520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecfe25-3093-4c46-b4e2-c27291a4e084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:29.088132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecfe25-3093-4c46-b4e2-c27291a4e084 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:29.098801 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4141/8369] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:29.104314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777467a-0f6e-4195-9aaf-c5c110fdc3ef admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:29.117557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777467a-0f6e-4195-9aaf-c5c110fdc3ef admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:29.122862 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f5da0-c7ab-473a-9d36-e2e55f7e65c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:29.123561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f5da0-c7ab-473a-9d36-e2e55f7e65c9 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:29.123842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f5da0-c7ab-473a-9d36-e2e55f7e65c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:29.124099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f5da0-c7ab-473a-9d36-e2e55f7e65c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:29.124765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f5da0-c7ab-473a-9d36-e2e55f7e65c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:29.126361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f5da0-c7ab-473a-9d36-e2e55f7e65c9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:29.137450 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4229/8370] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:29.141607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecfe25-3093-4c46-b4e2-c27291a4e084 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:29.152459 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecfe25-3093-4c46-b4e2-c27291a4e084 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:29.157568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03d811-14c4-49a6-bfe2-7bdae43c78de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:29.158235 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03d811-14c4-49a6-bfe2-7bdae43c78de admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:29.158510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03d811-14c4-49a6-bfe2-7bdae43c78de admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:29.158898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03d811-14c4-49a6-bfe2-7bdae43c78de admin admin] PATH_INFO: `/v3/projects/ba2591111c5d484fbfe10b4cc28ef864/users/75a89f3c2c404a898cb89def1657ce15/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:29.174387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03d811-14c4-49a6-bfe2-7bdae43c78de admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2591111c5d484fbfe10b4cc28ef864, user_id=75a89f3c2c404a898cb89def1657ce15, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:29.177111 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03d811-14c4-49a6-bfe2-7bdae43c78de admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:29.191370 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4142/8371] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:29 2022] PUT /identity/v3/projects/ba2591111c5d484fbfe10b4cc28ef864/users/75a89f3c2c404a898cb89def1657ce15/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:29.201964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5682528-832f-419d-a8a7-ab2508cab249 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:29.202245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5682528-832f-419d-a8a7-ab2508cab249 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:29.202537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5682528-832f-419d-a8a7-ab2508cab249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:29.233026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b5682528-832f-419d-a8a7-ab2508cab249 None None] MFA Rules not processed for user `75a89f3c2c404a898cb89def1657ce15`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:29.234255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b5682528-832f-419d-a8a7-ab2508cab249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:29.300635 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4230/8372] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:29.310855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03d811-14c4-49a6-bfe2-7bdae43c78de admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:29.313318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2a03d811-14c4-49a6-bfe2-7bdae43c78de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:29.376238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03d811-14c4-49a6-bfe2-7bdae43c78de admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:29.380744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899f3b7-e9be-4041-8888-7c517668e4b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:29.381355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899f3b7-e9be-4041-8888-7c517668e4b6 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:29.381630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899f3b7-e9be-4041-8888-7c517668e4b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:29.381898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899f3b7-e9be-4041-8888-7c517668e4b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:29.390662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899f3b7-e9be-4041-8888-7c517668e4b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:29.392071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899f3b7-e9be-4041-8888-7c517668e4b6 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:29.402176 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4143/8373] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:29 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:30.918464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5682528-832f-419d-a8a7-ab2508cab249 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:30.919996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b5682528-832f-419d-a8a7-ab2508cab249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:30.978124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899f3b7-e9be-4041-8888-7c517668e4b6 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:31.007291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5682528-832f-419d-a8a7-ab2508cab249 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:31.012131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491df12-f743-44b8-a34c-cd51226dd3ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:31.012808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491df12-f743-44b8-a34c-cd51226dd3ac service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:31.013074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491df12-f743-44b8-a34c-cd51226dd3ac service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:31.013336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491df12-f743-44b8-a34c-cd51226dd3ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:31.015039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d491df12-f743-44b8-a34c-cd51226dd3ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:31.029463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491df12-f743-44b8-a34c-cd51226dd3ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:31.031069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491df12-f743-44b8-a34c-cd51226dd3ac service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:31.043791 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4231/8374] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:31.047974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899f3b7-e9be-4041-8888-7c517668e4b6 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:31.052606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:31.053242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:31.053520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:31.053788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] PATH_INFO: `/v3/users/75a89f3c2c404a898cb89def1657ce15` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:31.054940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a89f3c2c404a898cb89def1657ce15)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:31.056894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:31.107604 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a89f3c2c404a898cb89def1657ce15'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:31.110636 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:31.117737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a89f3c2c404a898cb89def1657ce15'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:31.131935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a89f3c2c404a898cb89def1657ce15'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:31.132608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a89f3c2c404a898cb89def1657ce15'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:31.133203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a89f3c2c404a898cb89def1657ce15'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:31.146276 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4144/8375] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:30 2022] DELETE /identity/v3/users/75a89f3c2c404a898cb89def1657ce15 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:31.151033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d59082-8eb6-48ad-9ca6-c1e9bf6ba400 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:31.151306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d59082-8eb6-48ad-9ca6-c1e9bf6ba400 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:31.151555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d59082-8eb6-48ad-9ca6-c1e9bf6ba400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:31.159387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:31.175095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c4d59082-8eb6-48ad-9ca6-c1e9bf6ba400 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:31.176292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c4d59082-8eb6-48ad-9ca6-c1e9bf6ba400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:31.236802 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4232/8376] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:31.246651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:31.248357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:31.311871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98876f2-0ea6-48a7-bb21-28621d195759 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:31.316280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20107bf3-59f6-4896-9371-cba92412e85c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:31.317348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20107bf3-59f6-4896-9371-cba92412e85c service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:31.317620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20107bf3-59f6-4896-9371-cba92412e85c service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:31.317914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20107bf3-59f6-4896-9371-cba92412e85c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:31.327982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20107bf3-59f6-4896-9371-cba92412e85c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:31.329449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20107bf3-59f6-4896-9371-cba92412e85c service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:31.339518 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4145/8377] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:31.531841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d59082-8eb6-48ad-9ca6-c1e9bf6ba400 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:31.533765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c4d59082-8eb6-48ad-9ca6-c1e9bf6ba400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:31.547344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d59082-8eb6-48ad-9ca6-c1e9bf6ba400 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:31.553455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6096e6-eaa7-469a-82c2-9d98c9572628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:31.554104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6096e6-eaa7-469a-82c2-9d98c9572628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:31.554357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6096e6-eaa7-469a-82c2-9d98c9572628 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:31.554628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6096e6-eaa7-469a-82c2-9d98c9572628 admin admin] PATH_INFO: `/v3/projects/ba2591111c5d484fbfe10b4cc28ef864` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:31.555909 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6096e6-eaa7-469a-82c2-9d98c9572628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba2591111c5d484fbfe10b4cc28ef864)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:31.558000 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6096e6-eaa7-469a-82c2-9d98c9572628 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:31.607674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2e6096e6-eaa7-469a-82c2-9d98c9572628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba2591111c5d484fbfe10b4cc28ef864', 'request_id': 'req-2e6096e6-eaa7-469a-82c2-9d98c9572628'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:31.615609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2e6096e6-eaa7-469a-82c2-9d98c9572628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba2591111c5d484fbfe10b4cc28ef864', 'request_id': 'req-2e6096e6-eaa7-469a-82c2-9d98c9572628'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:31.621588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2e6096e6-eaa7-469a-82c2-9d98c9572628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba2591111c5d484fbfe10b4cc28ef864', 'request_id': 'req-2e6096e6-eaa7-469a-82c2-9d98c9572628'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:31.632981 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2e6096e6-eaa7-469a-82c2-9d98c9572628 admin admin] The token cache is being invalidate because project ba2591111c5d484fbfe10b4cc28ef864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:31.633264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2e6096e6-eaa7-469a-82c2-9d98c9572628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:31.633901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2e6096e6-eaa7-469a-82c2-9d98c9572628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:31.635440 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4233/8378] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:31 2022] DELETE /identity/v3/projects/ba2591111c5d484fbfe10b4cc28ef864 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:31.640507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad10ef-a470-4c3b-b556-313cd692bbf7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:31.640792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad10ef-a470-4c3b-b556-313cd692bbf7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:31.641060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad10ef-a470-4c3b-b556-313cd692bbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:31.661693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-47ad10ef-a470-4c3b-b556-313cd692bbf7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:31.662908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-47ad10ef-a470-4c3b-b556-313cd692bbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:31.726960 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4146/8379] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:31.737350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a233bf-32f9-4160-af1a-a425dcc32861 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:31.737721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a233bf-32f9-4160-af1a-a425dcc32861 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:31.738074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a233bf-32f9-4160-af1a-a425dcc32861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:31.763884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-48a233bf-32f9-4160-af1a-a425dcc32861 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:31.765118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-48a233bf-32f9-4160-af1a-a425dcc32861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:31.774311 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4234/8380] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:31.779034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad10ef-a470-4c3b-b556-313cd692bbf7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:31.789527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad10ef-a470-4c3b-b556-313cd692bbf7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:31.795261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfb32d-fed5-4d81-bb59-b1227f809bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:31.795860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfb32d-fed5-4d81-bb59-b1227f809bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:31.796133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfb32d-fed5-4d81-bb59-b1227f809bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:31.796456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfb32d-fed5-4d81-bb59-b1227f809bd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:31.797093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfb32d-fed5-4d81-bb59-b1227f809bd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:31.797428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfb32d-fed5-4d81-bb59-b1227f809bd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:31.799015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfb32d-fed5-4d81-bb59-b1227f809bd9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:31.811256 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4147/8381] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:28:31.816077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a233bf-32f9-4160-af1a-a425dcc32861 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:31.825971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a233bf-32f9-4160-af1a-a425dcc32861 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:31.831695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310667b-ea8e-473f-a935-154d9c397700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:31.832340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310667b-ea8e-473f-a935-154d9c397700 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:31.832649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310667b-ea8e-473f-a935-154d9c397700 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:31.832946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310667b-ea8e-473f-a935-154d9c397700 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:31.833598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310667b-ea8e-473f-a935-154d9c397700 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:31.835534 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310667b-ea8e-473f-a935-154d9c397700 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:31.865264 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4235/8382] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:31 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:31.869521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfb32d-fed5-4d81-bb59-b1227f809bd9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:31.880083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfb32d-fed5-4d81-bb59-b1227f809bd9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:31.932238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:31.939888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903542a6-c325-4467-bbe6-f1766b3c6dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:31.940654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903542a6-c325-4467-bbe6-f1766b3c6dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:31.941091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903542a6-c325-4467-bbe6-f1766b3c6dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:31.941358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903542a6-c325-4467-bbe6-f1766b3c6dd1 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:31.942292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903542a6-c325-4467-bbe6-f1766b3c6dd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:31.944253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903542a6-c325-4467-bbe6-f1766b3c6dd1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:31.967091 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4148/8383] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:31 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:31.971879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310667b-ea8e-473f-a935-154d9c397700 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:31.983985 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310667b-ea8e-473f-a935-154d9c397700 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:31.989263 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751f2cf-2b19-4b1a-9197-21ae56ddc4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:31.989852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751f2cf-2b19-4b1a-9197-21ae56ddc4ea admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:31.990122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751f2cf-2b19-4b1a-9197-21ae56ddc4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:31.990389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751f2cf-2b19-4b1a-9197-21ae56ddc4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:31.991025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751f2cf-2b19-4b1a-9197-21ae56ddc4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:31.992666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751f2cf-2b19-4b1a-9197-21ae56ddc4ea admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:32.003243 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4236/8384] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:32.007449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903542a6-c325-4467-bbe6-f1766b3c6dd1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:32.018070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903542a6-c325-4467-bbe6-f1766b3c6dd1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:32.023312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d991f-7984-492a-a589-55d883b82766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:32.024205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d991f-7984-492a-a589-55d883b82766 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:32.024521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d991f-7984-492a-a589-55d883b82766 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:32.024786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d991f-7984-492a-a589-55d883b82766 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:32.025431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d991f-7984-492a-a589-55d883b82766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:32.027012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d991f-7984-492a-a589-55d883b82766 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:32.037802 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4149/8385] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:32.042065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751f2cf-2b19-4b1a-9197-21ae56ddc4ea admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:32.052796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751f2cf-2b19-4b1a-9197-21ae56ddc4ea admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:32.057832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0900c-28aa-471f-886a-6b42320baa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:32.058465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0900c-28aa-471f-886a-6b42320baa89 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:32.058736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0900c-28aa-471f-886a-6b42320baa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:32.058999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0900c-28aa-471f-886a-6b42320baa89 admin admin] PATH_INFO: `/v3/projects/e99fb8d666af439598cb1c482604cb82/users/91e8ff5b3e2b49a786170837ef1c293f/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:32.074107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0900c-28aa-471f-886a-6b42320baa89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e99fb8d666af439598cb1c482604cb82, user_id=91e8ff5b3e2b49a786170837ef1c293f, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:32.076855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0900c-28aa-471f-886a-6b42320baa89 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:32.091049 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4237/8386] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:32 2022] PUT /identity/v3/projects/e99fb8d666af439598cb1c482604cb82/users/91e8ff5b3e2b49a786170837ef1c293f/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:32.101557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875aef65-a98e-4984-bddb-9b6e327458bc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:32.101833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875aef65-a98e-4984-bddb-9b6e327458bc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:32.102100 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875aef65-a98e-4984-bddb-9b6e327458bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:32.131628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-875aef65-a98e-4984-bddb-9b6e327458bc None None] MFA Rules not processed for user `91e8ff5b3e2b49a786170837ef1c293f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:32.132892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-875aef65-a98e-4984-bddb-9b6e327458bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:32.207649 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4150/8387] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:32.212406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0900c-28aa-471f-886a-6b42320baa89 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:32.240393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:32.280092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0900c-28aa-471f-886a-6b42320baa89 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:32.285287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb6698-7b9c-4b33-800e-8f32b1f5b456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:32.285890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6698-7b9c-4b33-800e-8f32b1f5b456 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:32.286233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6698-7b9c-4b33-800e-8f32b1f5b456 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:32.286500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6698-7b9c-4b33-800e-8f32b1f5b456 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:32.287166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb6698-7b9c-4b33-800e-8f32b1f5b456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:32.289103 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb6698-7b9c-4b33-800e-8f32b1f5b456 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:32.319642 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4238/8388] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:32 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:32.325741 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875aef65-a98e-4984-bddb-9b6e327458bc None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:32.343674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875aef65-a98e-4984-bddb-9b6e327458bc None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:32.406956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81a1fc-6978-41b4-96f5-b8be76c8cc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:32.407497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81a1fc-6978-41b4-96f5-b8be76c8cc44 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:32.407753 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81a1fc-6978-41b4-96f5-b8be76c8cc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:32.408017 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81a1fc-6978-41b4-96f5-b8be76c8cc44 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:32.408982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81a1fc-6978-41b4-96f5-b8be76c8cc44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:32.410942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81a1fc-6978-41b4-96f5-b8be76c8cc44 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:32.432780 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4151/8389] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:32 2022] POST /identity/v3/users => generated 301 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:32.436961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb6698-7b9c-4b33-800e-8f32b1f5b456 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:32.448290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb6698-7b9c-4b33-800e-8f32b1f5b456 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:32.454788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b55fb-4f64-4599-acf4-b1b332c64057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:32.455387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b55fb-4f64-4599-acf4-b1b332c64057 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:32.455735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b55fb-4f64-4599-acf4-b1b332c64057 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:32.455994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b55fb-4f64-4599-acf4-b1b332c64057 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:32.456622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b55fb-4f64-4599-acf4-b1b332c64057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:32.458243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b55fb-4f64-4599-acf4-b1b332c64057 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:32.470027 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4239/8390] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:32.474986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81a1fc-6978-41b4-96f5-b8be76c8cc44 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:32.488795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81a1fc-6978-41b4-96f5-b8be76c8cc44 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:32.494152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7a229-3e67-41a7-b2d9-d1598b25ac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:32.495149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7a229-3e67-41a7-b2d9-d1598b25ac69 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:32.495444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7a229-3e67-41a7-b2d9-d1598b25ac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:32.495711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7a229-3e67-41a7-b2d9-d1598b25ac69 admin admin] PATH_INFO: `/v3/projects/5ba89b74e3c3415a9ed9b94a8822cfff/users/64aead9d9aff4b6d9394ed10a54e2424/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:32.513484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7a229-3e67-41a7-b2d9-d1598b25ac69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba89b74e3c3415a9ed9b94a8822cfff, user_id=64aead9d9aff4b6d9394ed10a54e2424, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:32.516193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7a229-3e67-41a7-b2d9-d1598b25ac69 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:32.532643 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4152/8391] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:32 2022] PUT /identity/v3/projects/5ba89b74e3c3415a9ed9b94a8822cfff/users/64aead9d9aff4b6d9394ed10a54e2424/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:32.537835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b55fb-4f64-4599-acf4-b1b332c64057 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:32.552860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b55fb-4f64-4599-acf4-b1b332c64057 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:32.616400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2122e0-08dc-4e68-9534-b05fb2d6f25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:32.616997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2122e0-08dc-4e68-9534-b05fb2d6f25f admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:32.617288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2122e0-08dc-4e68-9534-b05fb2d6f25f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:32.617552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2122e0-08dc-4e68-9534-b05fb2d6f25f admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:32.618182 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2122e0-08dc-4e68-9534-b05fb2d6f25f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:32.619792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2122e0-08dc-4e68-9534-b05fb2d6f25f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:32.629648 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4240/8392] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:32.634169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7a229-3e67-41a7-b2d9-d1598b25ac69 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:32.647559 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7a229-3e67-41a7-b2d9-d1598b25ac69 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:32.652561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d162e6-621a-4bef-817e-0b4c38c72e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:32.653206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d162e6-621a-4bef-817e-0b4c38c72e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:32.653481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d162e6-621a-4bef-817e-0b4c38c72e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:32.653752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d162e6-621a-4bef-817e-0b4c38c72e2f admin admin] PATH_INFO: `/v3/projects/5ba89b74e3c3415a9ed9b94a8822cfff/users/64aead9d9aff4b6d9394ed10a54e2424/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:32.655761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d162e6-621a-4bef-817e-0b4c38c72e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba89b74e3c3415a9ed9b94a8822cfff, user_id=64aead9d9aff4b6d9394ed10a54e2424, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:32.658516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d162e6-621a-4bef-817e-0b4c38c72e2f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:32.669876 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4153/8393] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:32 2022] PUT /identity/v3/projects/5ba89b74e3c3415a9ed9b94a8822cfff/users/64aead9d9aff4b6d9394ed10a54e2424/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:32.680221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99d95b-99b0-47bd-9a19-ec56c5b5ca6d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:32.680531 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99d95b-99b0-47bd-9a19-ec56c5b5ca6d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:32.680792 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99d95b-99b0-47bd-9a19-ec56c5b5ca6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:32.711517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-da99d95b-99b0-47bd-9a19-ec56c5b5ca6d None None] MFA Rules not processed for user `64aead9d9aff4b6d9394ed10a54e2424`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:32.712776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-da99d95b-99b0-47bd-9a19-ec56c5b5ca6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:32.785935 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4241/8394] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:32 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:32.801726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d162e6-621a-4bef-817e-0b4c38c72e2f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:32.803763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-26d162e6-621a-4bef-817e-0b4c38c72e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:32.828277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:32.871694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d162e6-621a-4bef-817e-0b4c38c72e2f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:32.875368 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b28fce-6175-4e88-bec5-eaf93b500646 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:32.875968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b28fce-6175-4e88-bec5-eaf93b500646 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:32.876240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b28fce-6175-4e88-bec5-eaf93b500646 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:32.876535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b28fce-6175-4e88-bec5-eaf93b500646 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:32.886477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b28fce-6175-4e88-bec5-eaf93b500646 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:32.887899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b28fce-6175-4e88-bec5-eaf93b500646 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:32.898485 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4154/8395] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:32 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:33.042551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99d95b-99b0-47bd-9a19-ec56c5b5ca6d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:33.061336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:33.106921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99d95b-99b0-47bd-9a19-ec56c5b5ca6d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:33.111423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:33.112287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:33.112287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:33.112635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] PATH_INFO: `/v3/users/91e8ff5b3e2b49a786170837ef1c293f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:33.113776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91e8ff5b3e2b49a786170837ef1c293f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:33.115716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:33.167476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91e8ff5b3e2b49a786170837ef1c293f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:33.176013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91e8ff5b3e2b49a786170837ef1c293f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:33.184037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91e8ff5b3e2b49a786170837ef1c293f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:33.196990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91e8ff5b3e2b49a786170837ef1c293f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:33.197707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91e8ff5b3e2b49a786170837ef1c293f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:33.203768 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4242/8396] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:33 2022] DELETE /identity/v3/users/91e8ff5b3e2b49a786170837ef1c293f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:33.208573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4e6ff-1323-48c2-952b-c63449ceb302 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:33.209096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4e6ff-1323-48c2-952b-c63449ceb302 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:33.209205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4e6ff-1323-48c2-952b-c63449ceb302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:33.230168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-48f4e6ff-1323-48c2-952b-c63449ceb302 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:33.231656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-48f4e6ff-1323-48c2-952b-c63449ceb302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:33.301320 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4155/8397] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:28:33.313055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:33.315131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:33.380470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd983f-6f6c-44b5-ac37-13506f0b07f6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:33.384382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c546744-0e85-4ec6-9fa9-ebcabc390126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:33.385022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c546744-0e85-4ec6-9fa9-ebcabc390126 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:33.385273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c546744-0e85-4ec6-9fa9-ebcabc390126 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:33.385540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c546744-0e85-4ec6-9fa9-ebcabc390126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:33.396475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c546744-0e85-4ec6-9fa9-ebcabc390126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:33.397963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c546744-0e85-4ec6-9fa9-ebcabc390126 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:33.408436 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4243/8398] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:33.694247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4e6ff-1323-48c2-952b-c63449ceb302 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:33.695839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-48f4e6ff-1323-48c2-952b-c63449ceb302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:33.712109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4e6ff-1323-48c2-952b-c63449ceb302 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:33.717212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:33.717972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:33.718703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:33.718703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] PATH_INFO: `/v3/projects/e99fb8d666af439598cb1c482604cb82` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:33.719811 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e99fb8d666af439598cb1c482604cb82)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:33.722414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:33.775781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e99fb8d666af439598cb1c482604cb82', 'request_id': 'req-7cf826df-2b5a-43c3-8394-1f54ef71df5e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:33.786701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e99fb8d666af439598cb1c482604cb82', 'request_id': 'req-7cf826df-2b5a-43c3-8394-1f54ef71df5e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:33.797509 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e99fb8d666af439598cb1c482604cb82', 'request_id': 'req-7cf826df-2b5a-43c3-8394-1f54ef71df5e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:33.825477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] The token cache is being invalidate because project e99fb8d666af439598cb1c482604cb82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:33.825772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:33.826444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:33.829875 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4156/8399] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:33 2022] DELETE /identity/v3/projects/e99fb8d666af439598cb1c482604cb82 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:33.833106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c546744-0e85-4ec6-9fa9-ebcabc390126 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:33.845984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c546744-0e85-4ec6-9fa9-ebcabc390126 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:33.895627 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:33.939670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f674c-6ace-4d16-9cbd-ee4dbc9f6e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:33.940610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f674c-6ace-4d16-9cbd-ee4dbc9f6e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:33.941014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f674c-6ace-4d16-9cbd-ee4dbc9f6e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:33.941356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f674c-6ace-4d16-9cbd-ee4dbc9f6e52 admin admin] PATH_INFO: `/v3/users/64aead9d9aff4b6d9394ed10a54e2424` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:33.942951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f674c-6ace-4d16-9cbd-ee4dbc9f6e52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64aead9d9aff4b6d9394ed10a54e2424)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:33.945149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f674c-6ace-4d16-9cbd-ee4dbc9f6e52 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:34.037716 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2d0f674c-6ace-4d16-9cbd-ee4dbc9f6e52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64aead9d9aff4b6d9394ed10a54e2424'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:34.050988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2d0f674c-6ace-4d16-9cbd-ee4dbc9f6e52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64aead9d9aff4b6d9394ed10a54e2424'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:34.060969 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2d0f674c-6ace-4d16-9cbd-ee4dbc9f6e52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64aead9d9aff4b6d9394ed10a54e2424'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:34.076527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2d0f674c-6ace-4d16-9cbd-ee4dbc9f6e52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64aead9d9aff4b6d9394ed10a54e2424'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:34.077278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-2d0f674c-6ace-4d16-9cbd-ee4dbc9f6e52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64aead9d9aff4b6d9394ed10a54e2424'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:34.085659 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4244/8400] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:33 2022] DELETE /identity/v3/users/64aead9d9aff4b6d9394ed10a54e2424 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:34.390431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:34.392144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:34.405988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf826df-2b5a-43c3-8394-1f54ef71df5e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:34.451496 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:34.468816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd53ee5-3cf9-45df-88a8-b185dc90956c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:34.469465 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd53ee5-3cf9-45df-88a8-b185dc90956c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:34.469804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd53ee5-3cf9-45df-88a8-b185dc90956c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:34.470036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd53ee5-3cf9-45df-88a8-b185dc90956c admin admin] PATH_INFO: `/v3/projects/5ba89b74e3c3415a9ed9b94a8822cfff` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:34.471588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd53ee5-3cf9-45df-88a8-b185dc90956c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba89b74e3c3415a9ed9b94a8822cfff)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:34.473689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd53ee5-3cf9-45df-88a8-b185dc90956c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:34.527595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9dd53ee5-3cf9-45df-88a8-b185dc90956c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba89b74e3c3415a9ed9b94a8822cfff', 'request_id': 'req-9dd53ee5-3cf9-45df-88a8-b185dc90956c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:34.534391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9dd53ee5-3cf9-45df-88a8-b185dc90956c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba89b74e3c3415a9ed9b94a8822cfff', 'request_id': 'req-9dd53ee5-3cf9-45df-88a8-b185dc90956c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:34.543723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9dd53ee5-3cf9-45df-88a8-b185dc90956c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba89b74e3c3415a9ed9b94a8822cfff', 'request_id': 'req-9dd53ee5-3cf9-45df-88a8-b185dc90956c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:34.555768 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9dd53ee5-3cf9-45df-88a8-b185dc90956c admin admin] The token cache is being invalidate because project 5ba89b74e3c3415a9ed9b94a8822cfff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:34.556050 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9dd53ee5-3cf9-45df-88a8-b185dc90956c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:34.556745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-9dd53ee5-3cf9-45df-88a8-b185dc90956c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:34.558260 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4157/8401] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:34 2022] DELETE /identity/v3/projects/5ba89b74e3c3415a9ed9b94a8822cfff => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:28:34.563302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986d8df-fe4c-4e87-85c5-44fa3074beaf None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:34.563582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986d8df-fe4c-4e87-85c5-44fa3074beaf None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:34.563887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986d8df-fe4c-4e87-85c5-44fa3074beaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:34.584284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6986d8df-fe4c-4e87-85c5-44fa3074beaf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:34.585566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6986d8df-fe4c-4e87-85c5-44fa3074beaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:34.660169 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4245/8402] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:34.668458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02366dde-dd38-4f24-bb61-48ecab59cf68 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:34.668738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02366dde-dd38-4f24-bb61-48ecab59cf68 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:34.669024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02366dde-dd38-4f24-bb61-48ecab59cf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:34.689014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-02366dde-dd38-4f24-bb61-48ecab59cf68 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:34.690601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-02366dde-dd38-4f24-bb61-48ecab59cf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:34.699583 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4158/8403] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:34.704162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986d8df-fe4c-4e87-85c5-44fa3074beaf None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:34.715497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986d8df-fe4c-4e87-85c5-44fa3074beaf None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:34.722552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cebe73-b424-4646-86ab-93661c922e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:34.723435 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cebe73-b424-4646-86ab-93661c922e67 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:34.723842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cebe73-b424-4646-86ab-93661c922e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:34.724176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cebe73-b424-4646-86ab-93661c922e67 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:34.724859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cebe73-b424-4646-86ab-93661c922e67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:34.725175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cebe73-b424-4646-86ab-93661c922e67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:34.726814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cebe73-b424-4646-86ab-93661c922e67 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:34.739539 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4246/8404] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:28:34.744039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02366dde-dd38-4f24-bb61-48ecab59cf68 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:34.754431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02366dde-dd38-4f24-bb61-48ecab59cf68 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:34.759638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620130ea-fe8a-4db0-a71c-97069fbcda9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:34.760340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620130ea-fe8a-4db0-a71c-97069fbcda9c admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:34.760649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620130ea-fe8a-4db0-a71c-97069fbcda9c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:34.760917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620130ea-fe8a-4db0-a71c-97069fbcda9c admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:34.761556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620130ea-fe8a-4db0-a71c-97069fbcda9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:34.763494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620130ea-fe8a-4db0-a71c-97069fbcda9c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:34.798283 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4159/8405] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:34 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:34.800008 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cebe73-b424-4646-86ab-93661c922e67 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:34.810814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cebe73-b424-4646-86ab-93661c922e67 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:34.858072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:34.872408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44491a-370b-4aae-98b9-19bb78190869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:34.873054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44491a-370b-4aae-98b9-19bb78190869 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:34.873331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44491a-370b-4aae-98b9-19bb78190869 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:34.873583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44491a-370b-4aae-98b9-19bb78190869 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:34.874186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44491a-370b-4aae-98b9-19bb78190869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:34.876079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44491a-370b-4aae-98b9-19bb78190869 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:34.903377 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4247/8406] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:34 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:34.904780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620130ea-fe8a-4db0-a71c-97069fbcda9c admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:34.916269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620130ea-fe8a-4db0-a71c-97069fbcda9c admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:34.921703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae207c45-6e45-4da6-b5d8-654a2f6d21a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:34.922325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae207c45-6e45-4da6-b5d8-654a2f6d21a5 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:34.922644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae207c45-6e45-4da6-b5d8-654a2f6d21a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:34.922915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae207c45-6e45-4da6-b5d8-654a2f6d21a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:34.923609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae207c45-6e45-4da6-b5d8-654a2f6d21a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:34.925448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae207c45-6e45-4da6-b5d8-654a2f6d21a5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:34.939562 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4160/8407] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:28:34.940930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44491a-370b-4aae-98b9-19bb78190869 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:34.952121 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44491a-370b-4aae-98b9-19bb78190869 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:34.957525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db24589-6f26-410a-b903-af30b6f137cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:34.958085 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db24589-6f26-410a-b903-af30b6f137cd admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:34.958358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db24589-6f26-410a-b903-af30b6f137cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:34.958624 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db24589-6f26-410a-b903-af30b6f137cd admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:34.959272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db24589-6f26-410a-b903-af30b6f137cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:34.960954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db24589-6f26-410a-b903-af30b6f137cd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:34.972696 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4248/8408] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:34.977268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae207c45-6e45-4da6-b5d8-654a2f6d21a5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:34.989269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae207c45-6e45-4da6-b5d8-654a2f6d21a5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:34.994942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d693da8-4ee8-4457-b5f8-ac61422bd1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:34.995639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d693da8-4ee8-4457-b5f8-ac61422bd1fb admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:34.995911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d693da8-4ee8-4457-b5f8-ac61422bd1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:34.996179 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d693da8-4ee8-4457-b5f8-ac61422bd1fb admin admin] PATH_INFO: `/v3/projects/b3d0a98d05cc4f4abc89db64efa7cef7/users/7dc68facc1e941d8a5f4568d0dde1330/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:35.012449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d693da8-4ee8-4457-b5f8-ac61422bd1fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3d0a98d05cc4f4abc89db64efa7cef7, user_id=7dc68facc1e941d8a5f4568d0dde1330, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:35.015285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d693da8-4ee8-4457-b5f8-ac61422bd1fb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:35.029882 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4161/8409] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:34 2022] PUT /identity/v3/projects/b3d0a98d05cc4f4abc89db64efa7cef7/users/7dc68facc1e941d8a5f4568d0dde1330/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:35.040703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f99923-2211-4eee-9242-b0e47f3f4af7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:35.041014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f99923-2211-4eee-9242-b0e47f3f4af7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:35.041278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f99923-2211-4eee-9242-b0e47f3f4af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:35.069942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-03f99923-2211-4eee-9242-b0e47f3f4af7 None None] MFA Rules not processed for user `7dc68facc1e941d8a5f4568d0dde1330`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:35.071185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-03f99923-2211-4eee-9242-b0e47f3f4af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:35.138712 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4249/8410] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:35.143693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d693da8-4ee8-4457-b5f8-ac61422bd1fb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:35.207639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d693da8-4ee8-4457-b5f8-ac61422bd1fb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:35.212595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d3e24-68b3-4be7-971d-878d8e3469f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:35.213202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d3e24-68b3-4be7-971d-878d8e3469f3 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:35.213470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d3e24-68b3-4be7-971d-878d8e3469f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:35.213755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d3e24-68b3-4be7-971d-878d8e3469f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:35.214363 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d3e24-68b3-4be7-971d-878d8e3469f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:35.216247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d3e24-68b3-4be7-971d-878d8e3469f3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:35.233837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:35.252920 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4162/8411] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:35 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:35.257233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f99923-2211-4eee-9242-b0e47f3f4af7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:35.269249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f99923-2211-4eee-9242-b0e47f3f4af7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:35.334880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae402750-3227-4d1e-b5a6-25b73924e90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:35.335607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae402750-3227-4d1e-b5a6-25b73924e90d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:35.336091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae402750-3227-4d1e-b5a6-25b73924e90d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:35.336455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae402750-3227-4d1e-b5a6-25b73924e90d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:35.337155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae402750-3227-4d1e-b5a6-25b73924e90d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:35.339139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae402750-3227-4d1e-b5a6-25b73924e90d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:35.362066 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4250/8412] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:35 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:35.366500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d3e24-68b3-4be7-971d-878d8e3469f3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:35.377151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d3e24-68b3-4be7-971d-878d8e3469f3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:35.382002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e0872-c109-4a28-8cb4-ad8a53335857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:35.382612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e0872-c109-4a28-8cb4-ad8a53335857 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:35.382908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e0872-c109-4a28-8cb4-ad8a53335857 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:35.383169 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e0872-c109-4a28-8cb4-ad8a53335857 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:35.383804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e0872-c109-4a28-8cb4-ad8a53335857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:35.385477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e0872-c109-4a28-8cb4-ad8a53335857 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:35.396853 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4163/8413] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:35.401317 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae402750-3227-4d1e-b5a6-25b73924e90d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:35.412923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae402750-3227-4d1e-b5a6-25b73924e90d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:35.418541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aef00f-422c-48ab-9890-fdfb8d5de82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:35.419291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aef00f-422c-48ab-9890-fdfb8d5de82d admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:35.419563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aef00f-422c-48ab-9890-fdfb8d5de82d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:35.419824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aef00f-422c-48ab-9890-fdfb8d5de82d admin admin] PATH_INFO: `/v3/projects/1bb99fab52e6445287889f8bb64bc73f/users/d02dbc7f66f746a68d64698b02c75ef3/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:35.437685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aef00f-422c-48ab-9890-fdfb8d5de82d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb99fab52e6445287889f8bb64bc73f, user_id=d02dbc7f66f746a68d64698b02c75ef3, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:35.441277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aef00f-422c-48ab-9890-fdfb8d5de82d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:35.459680 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4251/8414] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:35 2022] PUT /identity/v3/projects/1bb99fab52e6445287889f8bb64bc73f/users/d02dbc7f66f746a68d64698b02c75ef3/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:35.460715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e0872-c109-4a28-8cb4-ad8a53335857 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:35.471095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e0872-c109-4a28-8cb4-ad8a53335857 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:35.535400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f09a2-b018-4873-974d-0dd827fdc48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:35.536029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f09a2-b018-4873-974d-0dd827fdc48f admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:35.536305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f09a2-b018-4873-974d-0dd827fdc48f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:35.536614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f09a2-b018-4873-974d-0dd827fdc48f admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:35.537267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f09a2-b018-4873-974d-0dd827fdc48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:35.538894 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f09a2-b018-4873-974d-0dd827fdc48f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:35.553301 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4164/8415] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:35.554963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aef00f-422c-48ab-9890-fdfb8d5de82d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:35.569249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aef00f-422c-48ab-9890-fdfb8d5de82d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:35.576655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdd5d0-83bf-43b2-919c-ef072fbc435c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:35.577446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdd5d0-83bf-43b2-919c-ef072fbc435c admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:35.577818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdd5d0-83bf-43b2-919c-ef072fbc435c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:35.578194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdd5d0-83bf-43b2-919c-ef072fbc435c admin admin] PATH_INFO: `/v3/projects/1bb99fab52e6445287889f8bb64bc73f/users/d02dbc7f66f746a68d64698b02c75ef3/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:35.580278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdd5d0-83bf-43b2-919c-ef072fbc435c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb99fab52e6445287889f8bb64bc73f, user_id=d02dbc7f66f746a68d64698b02c75ef3, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:35.583387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdd5d0-83bf-43b2-919c-ef072fbc435c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:35.604519 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4252/8416] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:35 2022] PUT /identity/v3/projects/1bb99fab52e6445287889f8bb64bc73f/users/d02dbc7f66f746a68d64698b02c75ef3/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:35.607751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fbb24-97fc-403c-b4ca-4ffbb178eb51 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:35.608015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fbb24-97fc-403c-b4ca-4ffbb178eb51 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:35.608318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fbb24-97fc-403c-b4ca-4ffbb178eb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:35.647543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-635fbb24-97fc-403c-b4ca-4ffbb178eb51 None None] MFA Rules not processed for user `d02dbc7f66f746a68d64698b02c75ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:35.648843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-635fbb24-97fc-403c-b4ca-4ffbb178eb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:35.724287 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4165/8417] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:35.735191 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdd5d0-83bf-43b2-919c-ef072fbc435c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:35.737519 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d0bdd5d0-83bf-43b2-919c-ef072fbc435c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:35.748147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:35.808041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdd5d0-83bf-43b2-919c-ef072fbc435c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:35.811867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a55971-0c7e-4f52-a33e-ac31c1e89992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:35.812538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a55971-0c7e-4f52-a33e-ac31c1e89992 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:35.812816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a55971-0c7e-4f52-a33e-ac31c1e89992 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:35.813160 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a55971-0c7e-4f52-a33e-ac31c1e89992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:35.870946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a55971-0c7e-4f52-a33e-ac31c1e89992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:35.872525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a55971-0c7e-4f52-a33e-ac31c1e89992 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:35.881841 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4253/8418] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:37.478608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fbb24-97fc-403c-b4ca-4ffbb178eb51 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:37.545592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fbb24-97fc-403c-b4ca-4ffbb178eb51 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:37.551147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:37.551798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:37.552064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:37.552327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] PATH_INFO: `/v3/users/7dc68facc1e941d8a5f4568d0dde1330` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:37.553538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc68facc1e941d8a5f4568d0dde1330)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:37.555498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:37.560901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=100 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:37.612979 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc68facc1e941d8a5f4568d0dde1330'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:37.622042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc68facc1e941d8a5f4568d0dde1330'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:37.640629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc68facc1e941d8a5f4568d0dde1330'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:37.641435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc68facc1e941d8a5f4568d0dde1330'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:37.642152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc68facc1e941d8a5f4568d0dde1330'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:37.654598 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4166/8419] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:37 2022] DELETE /identity/v3/users/7dc68facc1e941d8a5f4568d0dde1330 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:37.659451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643126df-a5a6-4c0d-88ca-6de520e2c49c None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:37.659989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643126df-a5a6-4c0d-88ca-6de520e2c49c None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:37.659989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643126df-a5a6-4c0d-88ca-6de520e2c49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:37.680466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-643126df-a5a6-4c0d-88ca-6de520e2c49c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:37.681683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-643126df-a5a6-4c0d-88ca-6de520e2c49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:37.749428 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4254/8420] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:37.759972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:37.761545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:37.826716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64941ffd-74f3-4771-bb8e-67adccb213b6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:37.830815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e52d6-e8de-464b-bf5b-f53c11bf9a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:37.831429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e52d6-e8de-464b-bf5b-f53c11bf9a14 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:37.831699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e52d6-e8de-464b-bf5b-f53c11bf9a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:37.831961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e52d6-e8de-464b-bf5b-f53c11bf9a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:37.843447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e52d6-e8de-464b-bf5b-f53c11bf9a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:37.844991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e52d6-e8de-464b-bf5b-f53c11bf9a14 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:37.854852 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4167/8421] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:38.137601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643126df-a5a6-4c0d-88ca-6de520e2c49c None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:38.139173 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-643126df-a5a6-4c0d-88ca-6de520e2c49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:38.152726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643126df-a5a6-4c0d-88ca-6de520e2c49c None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:38.158863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:38.159524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:38.159878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:38.160147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] PATH_INFO: `/v3/projects/b3d0a98d05cc4f4abc89db64efa7cef7` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:38.161696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3d0a98d05cc4f4abc89db64efa7cef7)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:38.163658 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:38.218805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3d0a98d05cc4f4abc89db64efa7cef7', 'request_id': 'req-f78fc681-e892-4e00-8f97-a5c9d7899f3b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.228489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3d0a98d05cc4f4abc89db64efa7cef7', 'request_id': 'req-f78fc681-e892-4e00-8f97-a5c9d7899f3b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.230543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:38.237713 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3d0a98d05cc4f4abc89db64efa7cef7', 'request_id': 'req-f78fc681-e892-4e00-8f97-a5c9d7899f3b'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.252437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] The token cache is being invalidate because project b3d0a98d05cc4f4abc89db64efa7cef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:38.252774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.253466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.255061 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4255/8422] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:38 2022] DELETE /identity/v3/projects/b3d0a98d05cc4f4abc89db64efa7cef7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:38.259288 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e52d6-e8de-464b-bf5b-f53c11bf9a14 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:38.270028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e52d6-e8de-464b-bf5b-f53c11bf9a14 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:38.330493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b49db-846a-4f33-a18d-1d104334738c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:38.331403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b49db-846a-4f33-a18d-1d104334738c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:38.331685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b49db-846a-4f33-a18d-1d104334738c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:38.331948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b49db-846a-4f33-a18d-1d104334738c admin admin] PATH_INFO: `/v3/users/d02dbc7f66f746a68d64698b02c75ef3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:38.333178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b49db-846a-4f33-a18d-1d104334738c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02dbc7f66f746a68d64698b02c75ef3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:38.335163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b49db-846a-4f33-a18d-1d104334738c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:38.376775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:38.390522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-606b49db-846a-4f33-a18d-1d104334738c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02dbc7f66f746a68d64698b02c75ef3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.399483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-606b49db-846a-4f33-a18d-1d104334738c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02dbc7f66f746a68d64698b02c75ef3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.412485 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-606b49db-846a-4f33-a18d-1d104334738c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02dbc7f66f746a68d64698b02c75ef3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.413170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-606b49db-846a-4f33-a18d-1d104334738c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02dbc7f66f746a68d64698b02c75ef3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.413794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-606b49db-846a-4f33-a18d-1d104334738c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02dbc7f66f746a68d64698b02c75ef3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.426196 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4168/8423] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:38 2022] DELETE /identity/v3/users/d02dbc7f66f746a68d64698b02c75ef3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:38.739357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:38.741546 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:38.757227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fc681-e892-4e00-8f97-a5c9d7899f3b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:38.825721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088b2c1-c7df-4672-8380-d32ff953a8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:38.826352 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088b2c1-c7df-4672-8380-d32ff953a8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:38.826736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088b2c1-c7df-4672-8380-d32ff953a8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:38.827006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088b2c1-c7df-4672-8380-d32ff953a8d6 admin admin] PATH_INFO: `/v3/projects/1bb99fab52e6445287889f8bb64bc73f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:38.828518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088b2c1-c7df-4672-8380-d32ff953a8d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb99fab52e6445287889f8bb64bc73f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:38.830163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088b2c1-c7df-4672-8380-d32ff953a8d6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:38.878750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b088b2c1-c7df-4672-8380-d32ff953a8d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb99fab52e6445287889f8bb64bc73f', 'request_id': 'req-b088b2c1-c7df-4672-8380-d32ff953a8d6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.887174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b088b2c1-c7df-4672-8380-d32ff953a8d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb99fab52e6445287889f8bb64bc73f', 'request_id': 'req-b088b2c1-c7df-4672-8380-d32ff953a8d6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.893508 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b088b2c1-c7df-4672-8380-d32ff953a8d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb99fab52e6445287889f8bb64bc73f', 'request_id': 'req-b088b2c1-c7df-4672-8380-d32ff953a8d6'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.904764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b088b2c1-c7df-4672-8380-d32ff953a8d6 admin admin] The token cache is being invalidate because project 1bb99fab52e6445287889f8bb64bc73f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:38.904954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b088b2c1-c7df-4672-8380-d32ff953a8d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.905612 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b088b2c1-c7df-4672-8380-d32ff953a8d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:38.907122 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4256/8424] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:38 2022] DELETE /identity/v3/projects/1bb99fab52e6445287889f8bb64bc73f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:38.917675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283c067-d77c-4050-aca7-06ef757dfe66 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:38.918258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283c067-d77c-4050-aca7-06ef757dfe66 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:38.918528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283c067-d77c-4050-aca7-06ef757dfe66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:38.943104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c283c067-d77c-4050-aca7-06ef757dfe66 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:38.944297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c283c067-d77c-4050-aca7-06ef757dfe66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:39.035555 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4169/8425] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:39.037309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d823c3-bfcd-46e1-b90d-67ac4c2c6ba1 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:39.037595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d823c3-bfcd-46e1-b90d-67ac4c2c6ba1 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:39.037870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d823c3-bfcd-46e1-b90d-67ac4c2c6ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:39.060996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-76d823c3-bfcd-46e1-b90d-67ac4c2c6ba1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:39.062762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-76d823c3-bfcd-46e1-b90d-67ac4c2c6ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:39.072667 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4257/8426] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:39.076977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283c067-d77c-4050-aca7-06ef757dfe66 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:39.087655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283c067-d77c-4050-aca7-06ef757dfe66 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:39.093327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17990939-15c4-4921-8eee-d7631f19b420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:39.094030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17990939-15c4-4921-8eee-d7631f19b420 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:39.094306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17990939-15c4-4921-8eee-d7631f19b420 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:39.094570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17990939-15c4-4921-8eee-d7631f19b420 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:39.095212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17990939-15c4-4921-8eee-d7631f19b420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:39.095504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17990939-15c4-4921-8eee-d7631f19b420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:39.097663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17990939-15c4-4921-8eee-d7631f19b420 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:39.116668 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4170/8427] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:28:39.117735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d823c3-bfcd-46e1-b90d-67ac4c2c6ba1 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:39.131987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d823c3-bfcd-46e1-b90d-67ac4c2c6ba1 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:39.137192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f385d-cc16-433f-87bb-cf3525654d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:39.137795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f385d-cc16-433f-87bb-cf3525654d3b admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:39.138052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f385d-cc16-433f-87bb-cf3525654d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:39.138347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f385d-cc16-433f-87bb-cf3525654d3b admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:39.138951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f385d-cc16-433f-87bb-cf3525654d3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:39.141049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f385d-cc16-433f-87bb-cf3525654d3b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:39.171119 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4258/8428] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:39 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:39.175478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17990939-15c4-4921-8eee-d7631f19b420 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:39.185991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17990939-15c4-4921-8eee-d7631f19b420 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:39.248263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee63ae-a5ff-4a6a-a163-1b2014ad4d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:39.248905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee63ae-a5ff-4a6a-a163-1b2014ad4d8f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:39.249196 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee63ae-a5ff-4a6a-a163-1b2014ad4d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:39.249533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee63ae-a5ff-4a6a-a163-1b2014ad4d8f admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:39.250146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee63ae-a5ff-4a6a-a163-1b2014ad4d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:39.252047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee63ae-a5ff-4a6a-a163-1b2014ad4d8f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:39.276052 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4171/8429] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:39 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:39.277339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f385d-cc16-433f-87bb-cf3525654d3b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:39.288791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f385d-cc16-433f-87bb-cf3525654d3b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:39.295395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd4ad1-3683-4739-a0ab-35c37c869290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:39.295395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd4ad1-3683-4739-a0ab-35c37c869290 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:39.295721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd4ad1-3683-4739-a0ab-35c37c869290 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:39.296110 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd4ad1-3683-4739-a0ab-35c37c869290 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:39.296863 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd4ad1-3683-4739-a0ab-35c37c869290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:39.298561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd4ad1-3683-4739-a0ab-35c37c869290 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:39.310262 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4259/8430] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:39.316281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee63ae-a5ff-4a6a-a163-1b2014ad4d8f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:39.329240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee63ae-a5ff-4a6a-a163-1b2014ad4d8f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:39.335226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5ea5b-9573-45ac-b58b-cf62fde9b0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:39.335845 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5ea5b-9573-45ac-b58b-cf62fde9b0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:39.336117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5ea5b-9573-45ac-b58b-cf62fde9b0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:39.336547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5ea5b-9573-45ac-b58b-cf62fde9b0b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:39.337206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5ea5b-9573-45ac-b58b-cf62fde9b0b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:39.338844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5ea5b-9573-45ac-b58b-cf62fde9b0b6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:39.351211 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4172/8431] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:39.355609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd4ad1-3683-4739-a0ab-35c37c869290 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:39.365894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd4ad1-3683-4739-a0ab-35c37c869290 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:39.370659 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87ab52-6217-4842-bd7c-676b4e1ed9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:39.371321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87ab52-6217-4842-bd7c-676b4e1ed9a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:39.371597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87ab52-6217-4842-bd7c-676b4e1ed9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:39.371865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87ab52-6217-4842-bd7c-676b4e1ed9a9 admin admin] PATH_INFO: `/v3/projects/ef7f992ccdf44a0cbf670b02e63c5b72/users/f88fc0bbe7f24e3aaea7b231623a80f2/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:39.386980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87ab52-6217-4842-bd7c-676b4e1ed9a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7f992ccdf44a0cbf670b02e63c5b72, user_id=f88fc0bbe7f24e3aaea7b231623a80f2, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:39.389906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87ab52-6217-4842-bd7c-676b4e1ed9a9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:39.404796 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4260/8432] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:39 2022] PUT /identity/v3/projects/ef7f992ccdf44a0cbf670b02e63c5b72/users/f88fc0bbe7f24e3aaea7b231623a80f2/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:39.415648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa99c5-092c-494c-8a3d-bce0e184ba52 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:39.415909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa99c5-092c-494c-8a3d-bce0e184ba52 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:39.416175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa99c5-092c-494c-8a3d-bce0e184ba52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:39.443032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:39.446774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7eaa99c5-092c-494c-8a3d-bce0e184ba52 None None] MFA Rules not processed for user `f88fc0bbe7f24e3aaea7b231623a80f2`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:39.449195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7eaa99c5-092c-494c-8a3d-bce0e184ba52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:39.522738 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4173/8433] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:39.534592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87ab52-6217-4842-bd7c-676b4e1ed9a9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:39.536306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0a87ab52-6217-4842-bd7c-676b4e1ed9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:39.551414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:39.617430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87ab52-6217-4842-bd7c-676b4e1ed9a9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:39.622524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7d7d2-803d-41df-8cff-a834d8b867e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:39.623356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7d7d2-803d-41df-8cff-a834d8b867e9 service cinder] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:39.623622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7d7d2-803d-41df-8cff-a834d8b867e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:39.623887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7d7d2-803d-41df-8cff-a834d8b867e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:39.635104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7d7d2-803d-41df-8cff-a834d8b867e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:39.636399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7d7d2-803d-41df-8cff-a834d8b867e9 service cinder] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:39.647018 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4261/8434] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:39.667339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa99c5-092c-494c-8a3d-bce0e184ba52 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:39.771320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa99c5-092c-494c-8a3d-bce0e184ba52 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:39.778347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:39.779430 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:39.779891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:39.780508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] PATH_INFO: `/v3/users/f88fc0bbe7f24e3aaea7b231623a80f2` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:39.781984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f88fc0bbe7f24e3aaea7b231623a80f2)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:39.784546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:39.862302 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f88fc0bbe7f24e3aaea7b231623a80f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:39.872105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f88fc0bbe7f24e3aaea7b231623a80f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:39.888539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f88fc0bbe7f24e3aaea7b231623a80f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:39.889414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f88fc0bbe7f24e3aaea7b231623a80f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:39.890045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f88fc0bbe7f24e3aaea7b231623a80f2'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:39.907782 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4174/8435] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:39 2022] DELETE /identity/v3/users/f88fc0bbe7f24e3aaea7b231623a80f2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:39.912806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac00e27-6557-4a8e-9db0-b39063935174 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:39.913636 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac00e27-6557-4a8e-9db0-b39063935174 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:39.913918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac00e27-6557-4a8e-9db0-b39063935174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:39.935889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-eac00e27-6557-4a8e-9db0-b39063935174 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:39.938150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-eac00e27-6557-4a8e-9db0-b39063935174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:40.008446 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4262/8436] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:40.019295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:40.021827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:40.059391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:40.111261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5ac3c-9e03-4a68-b2b2-fa9918718958 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:40.120846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb0e32-4c5c-4ed6-8223-f3d87ca0c1c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:40.122013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb0e32-4c5c-4ed6-8223-f3d87ca0c1c1 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:40.122264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb0e32-4c5c-4ed6-8223-f3d87ca0c1c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:40.122534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb0e32-4c5c-4ed6-8223-f3d87ca0c1c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:40.134486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb0e32-4c5c-4ed6-8223-f3d87ca0c1c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:40.136066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb0e32-4c5c-4ed6-8223-f3d87ca0c1c1 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:40.148153 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4175/8437] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:28:40.477300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac00e27-6557-4a8e-9db0-b39063935174 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:40.478875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-eac00e27-6557-4a8e-9db0-b39063935174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:40.494139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac00e27-6557-4a8e-9db0-b39063935174 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:40.500989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:40.501643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:40.501912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:40.502957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4 admin admin] PATH_INFO: `/v3/projects/ef7f992ccdf44a0cbf670b02e63c5b72` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:40.504675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef7f992ccdf44a0cbf670b02e63c5b72)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:40.508575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:40.572565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef7f992ccdf44a0cbf670b02e63c5b72', 'request_id': 'req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:40.582020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef7f992ccdf44a0cbf670b02e63c5b72', 'request_id': 'req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:40.587944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef7f992ccdf44a0cbf670b02e63c5b72', 'request_id': 'req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:40.599382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4 admin admin] The token cache is being invalidate because project ef7f992ccdf44a0cbf670b02e63c5b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:40.599620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:40.600409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-8f810e1a-a12e-4536-bdbd-cfd5ffe962b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:40.602137 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4263/8438] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:40 2022] DELETE /identity/v3/projects/ef7f992ccdf44a0cbf670b02e63c5b72 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:40.607357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dac86-8633-46cf-89d1-793c5e6496b7 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:40.607662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dac86-8633-46cf-89d1-793c5e6496b7 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:40.607922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dac86-8633-46cf-89d1-793c5e6496b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:40.628150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fc0dac86-8633-46cf-89d1-793c5e6496b7 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:40.629410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fc0dac86-8633-46cf-89d1-793c5e6496b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:40.698311 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4176/8439] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:40.708095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a8cf8-a365-44fa-bf18-ed92a8f13931 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:40.708450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a8cf8-a365-44fa-bf18-ed92a8f13931 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:40.708663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a8cf8-a365-44fa-bf18-ed92a8f13931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:40.741861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f61a8cf8-a365-44fa-bf18-ed92a8f13931 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:40.743512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f61a8cf8-a365-44fa-bf18-ed92a8f13931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:40.761984 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4264/8440] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:40.762975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0dac86-8633-46cf-89d1-793c5e6496b7 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:40.774166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0dac86-8633-46cf-89d1-793c5e6496b7 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:40.779299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918f140-f073-4660-8b1f-37d603667441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:40.780053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918f140-f073-4660-8b1f-37d603667441 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:40.780327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918f140-f073-4660-8b1f-37d603667441 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:40.780660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918f140-f073-4660-8b1f-37d603667441 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:40.781335 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918f140-f073-4660-8b1f-37d603667441 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:40.781702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918f140-f073-4660-8b1f-37d603667441 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:40.783317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918f140-f073-4660-8b1f-37d603667441 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:40.799839 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4177/8441] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:28:40.804238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a8cf8-a365-44fa-bf18-ed92a8f13931 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:40.814523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a8cf8-a365-44fa-bf18-ed92a8f13931 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:40.823727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6d149-bfc3-47eb-a499-ee70d822753c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:40.824491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6d149-bfc3-47eb-a499-ee70d822753c admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:40.824842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6d149-bfc3-47eb-a499-ee70d822753c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:40.825775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6d149-bfc3-47eb-a499-ee70d822753c admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:40.826540 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c6d149-bfc3-47eb-a499-ee70d822753c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:40.828613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c6d149-bfc3-47eb-a499-ee70d822753c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:40.869733 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4265/8442] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:40 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:40.871330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918f140-f073-4660-8b1f-37d603667441 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:40.882721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918f140-f073-4660-8b1f-37d603667441 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:40.949722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3533d-e32d-48b1-ade5-51559283499b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:40.950409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3533d-e32d-48b1-ade5-51559283499b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:40.950752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3533d-e32d-48b1-ade5-51559283499b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:40.951089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3533d-e32d-48b1-ade5-51559283499b admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:40.951779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3533d-e32d-48b1-ade5-51559283499b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:40.953824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3533d-e32d-48b1-ade5-51559283499b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:40.977972 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4178/8443] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:40 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:40.982339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6d149-bfc3-47eb-a499-ee70d822753c admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:40.995479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6d149-bfc3-47eb-a499-ee70d822753c admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:41.000897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6d00a-b9ec-4cd5-9cf6-5731881895f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:41.001737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6d00a-b9ec-4cd5-9cf6-5731881895f9 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:41.002012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6d00a-b9ec-4cd5-9cf6-5731881895f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:41.002280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6d00a-b9ec-4cd5-9cf6-5731881895f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:41.011511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6d00a-b9ec-4cd5-9cf6-5731881895f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:41.013525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6d00a-b9ec-4cd5-9cf6-5731881895f9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:41.017164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:41.031365 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4266/8444] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:41.036607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3533d-e32d-48b1-ade5-51559283499b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:41.049883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3533d-e32d-48b1-ade5-51559283499b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:41.055299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb7572-15d7-4dd8-9db8-85e9fead499d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:41.056078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb7572-15d7-4dd8-9db8-85e9fead499d admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:41.056360 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb7572-15d7-4dd8-9db8-85e9fead499d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:41.056659 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb7572-15d7-4dd8-9db8-85e9fead499d admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:41.057304 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb7572-15d7-4dd8-9db8-85e9fead499d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:41.059156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb7572-15d7-4dd8-9db8-85e9fead499d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:41.076661 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4179/8445] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:41.081377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6d00a-b9ec-4cd5-9cf6-5731881895f9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:41.094784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6d00a-b9ec-4cd5-9cf6-5731881895f9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:41.100089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12deb32b-8cab-4535-967f-138cb567e954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:41.101174 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deb32b-8cab-4535-967f-138cb567e954 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:41.101628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deb32b-8cab-4535-967f-138cb567e954 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:41.102212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deb32b-8cab-4535-967f-138cb567e954 admin admin] PATH_INFO: `/v3/projects/12476914d40f47009089fdec72c541d7/users/f3f62913efa44df48a50bcac0ed3f63d/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:41.125642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12deb32b-8cab-4535-967f-138cb567e954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12476914d40f47009089fdec72c541d7, user_id=f3f62913efa44df48a50bcac0ed3f63d, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:41.130097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12deb32b-8cab-4535-967f-138cb567e954 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:41.152217 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4267/8446] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:41 2022] PUT /identity/v3/projects/12476914d40f47009089fdec72c541d7/users/f3f62913efa44df48a50bcac0ed3f63d/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:41.163789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4513421-de8d-478a-84b1-688b215545da None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:41.164103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4513421-de8d-478a-84b1-688b215545da None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:41.164480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4513421-de8d-478a-84b1-688b215545da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:41.196527 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-b4513421-de8d-478a-84b1-688b215545da None None] MFA Rules not processed for user `f3f62913efa44df48a50bcac0ed3f63d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:41.197861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b4513421-de8d-478a-84b1-688b215545da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:41.217806 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:41.277938 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4180/8447] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:41.282933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12deb32b-8cab-4535-967f-138cb567e954 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:41.381563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12deb32b-8cab-4535-967f-138cb567e954 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:41.386361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f627e0-7ea2-4c49-a691-51c3f3ae4ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:41.386974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f627e0-7ea2-4c49-a691-51c3f3ae4ece admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:41.387246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f627e0-7ea2-4c49-a691-51c3f3ae4ece admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:41.387557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f627e0-7ea2-4c49-a691-51c3f3ae4ece admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:41.388143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f627e0-7ea2-4c49-a691-51c3f3ae4ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:41.390100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f627e0-7ea2-4c49-a691-51c3f3ae4ece admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:41.421457 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4268/8448] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:41 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:41.422584 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4513421-de8d-478a-84b1-688b215545da None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:41.434075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4513421-de8d-478a-84b1-688b215545da None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:41.508125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b02dc2-f633-4cbe-9e22-1635c1dede04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:41.509064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b02dc2-f633-4cbe-9e22-1635c1dede04 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:41.509317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b02dc2-f633-4cbe-9e22-1635c1dede04 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:41.509683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b02dc2-f633-4cbe-9e22-1635c1dede04 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:41.510248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b02dc2-f633-4cbe-9e22-1635c1dede04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:41.512359 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b02dc2-f633-4cbe-9e22-1635c1dede04 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:41.551476 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4181/8449] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:41 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:28:41.552808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f627e0-7ea2-4c49-a691-51c3f3ae4ece admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:41.564522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f627e0-7ea2-4c49-a691-51c3f3ae4ece admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:41.570162 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d067288-e779-4145-8057-1d9c021e8ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:41.570983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d067288-e779-4145-8057-1d9c021e8ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:41.571356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d067288-e779-4145-8057-1d9c021e8ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:41.571725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d067288-e779-4145-8057-1d9c021e8ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:41.572767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d067288-e779-4145-8057-1d9c021e8ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:41.583577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d067288-e779-4145-8057-1d9c021e8ea4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:41.596411 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4269/8450] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:41.599563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b02dc2-f633-4cbe-9e22-1635c1dede04 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:41.610271 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b02dc2-f633-4cbe-9e22-1635c1dede04 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:41.615881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c327b10-d124-4ca1-aa95-e303f4733ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:41.616550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c327b10-d124-4ca1-aa95-e303f4733ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:41.616825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c327b10-d124-4ca1-aa95-e303f4733ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:41.617158 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c327b10-d124-4ca1-aa95-e303f4733ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:41.617770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c327b10-d124-4ca1-aa95-e303f4733ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:41.619483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c327b10-d124-4ca1-aa95-e303f4733ae3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:41.630790 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4182/8451] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:41.634970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d067288-e779-4145-8057-1d9c021e8ea4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:41.646327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d067288-e779-4145-8057-1d9c021e8ea4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:41.651135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a19cb-331e-4377-886b-8682aa3bfc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:41.651782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a19cb-331e-4377-886b-8682aa3bfc0b admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:41.652054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a19cb-331e-4377-886b-8682aa3bfc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:41.652321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a19cb-331e-4377-886b-8682aa3bfc0b admin admin] PATH_INFO: `/v3/projects/c60a50e573db4e43b5bcce4d36d7d66b/users/ff4021f31c6d42e7a74ee89a575a0def/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:41.668601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a19cb-331e-4377-886b-8682aa3bfc0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60a50e573db4e43b5bcce4d36d7d66b, user_id=ff4021f31c6d42e7a74ee89a575a0def, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:41.671305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a19cb-331e-4377-886b-8682aa3bfc0b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:41.696507 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4270/8452] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:41 2022] PUT /identity/v3/projects/c60a50e573db4e43b5bcce4d36d7d66b/users/ff4021f31c6d42e7a74ee89a575a0def/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:41.702276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca10181-2ce0-4e74-81f1-1122d206faaf None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:41.702667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca10181-2ce0-4e74-81f1-1122d206faaf None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:41.703287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca10181-2ce0-4e74-81f1-1122d206faaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:41.753478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-6ca10181-2ce0-4e74-81f1-1122d206faaf None None] MFA Rules not processed for user `ff4021f31c6d42e7a74ee89a575a0def`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:41.754916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6ca10181-2ce0-4e74-81f1-1122d206faaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:41.835593 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4183/8453] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:41.840607 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a19cb-331e-4377-886b-8682aa3bfc0b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:41.922271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a19cb-331e-4377-886b-8682aa3bfc0b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:41.927661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50292ba3-dac1-499a-a83d-e0dd9fe90ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:41.928272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50292ba3-dac1-499a-a83d-e0dd9fe90ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:41.928735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50292ba3-dac1-499a-a83d-e0dd9fe90ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:41.928947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50292ba3-dac1-499a-a83d-e0dd9fe90ce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:41.929603 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50292ba3-dac1-499a-a83d-e0dd9fe90ce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:41.931705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50292ba3-dac1-499a-a83d-e0dd9fe90ce5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:41.962803 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4271/8454] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:41 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:41.966817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca10181-2ce0-4e74-81f1-1122d206faaf None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:41.978966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca10181-2ce0-4e74-81f1-1122d206faaf None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:42.015825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:42.052464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8dcec-6874-4a5c-974d-faa2c04f5887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:42.053199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8dcec-6874-4a5c-974d-faa2c04f5887 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:42.053514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8dcec-6874-4a5c-974d-faa2c04f5887 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:42.053829 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8dcec-6874-4a5c-974d-faa2c04f5887 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:42.054486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8dcec-6874-4a5c-974d-faa2c04f5887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:42.056424 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8dcec-6874-4a5c-974d-faa2c04f5887 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:42.079434 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4184/8455] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:41 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:42.084144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50292ba3-dac1-499a-a83d-e0dd9fe90ce5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:42.095066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50292ba3-dac1-499a-a83d-e0dd9fe90ce5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:42.100426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b712a63-cff4-4c46-9ec6-ca72958c0383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:42.101206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b712a63-cff4-4c46-9ec6-ca72958c0383 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:42.101484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b712a63-cff4-4c46-9ec6-ca72958c0383 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:42.101772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b712a63-cff4-4c46-9ec6-ca72958c0383 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:42.102413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b712a63-cff4-4c46-9ec6-ca72958c0383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:42.104027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b712a63-cff4-4c46-9ec6-ca72958c0383 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:42.106466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:42.120537 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4272/8456] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:42.121517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8dcec-6874-4a5c-974d-faa2c04f5887 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:42.133895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8dcec-6874-4a5c-974d-faa2c04f5887 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:42.143791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50fe2d-ea7d-4ace-9ff0-7b6a308dab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:42.144596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50fe2d-ea7d-4ace-9ff0-7b6a308dab83 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:42.144910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50fe2d-ea7d-4ace-9ff0-7b6a308dab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:42.145242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50fe2d-ea7d-4ace-9ff0-7b6a308dab83 admin admin] PATH_INFO: `/v3/projects/f65e82e53217440fae4dd9129ec47f5b/users/f73a1cb53bef43bda5553926754a5fff/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:42.163658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50fe2d-ea7d-4ace-9ff0-7b6a308dab83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f65e82e53217440fae4dd9129ec47f5b, user_id=f73a1cb53bef43bda5553926754a5fff, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:42.166774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50fe2d-ea7d-4ace-9ff0-7b6a308dab83 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:42.180523 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4185/8457] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:42 2022] PUT /identity/v3/projects/f65e82e53217440fae4dd9129ec47f5b/users/f73a1cb53bef43bda5553926754a5fff/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:28:42.184852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b712a63-cff4-4c46-9ec6-ca72958c0383 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:42.195523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b712a63-cff4-4c46-9ec6-ca72958c0383 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:42.266213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66850af2-cd78-4d69-87b7-6cbfd351575a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:42.266570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66850af2-cd78-4d69-87b7-6cbfd351575a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:42.266839 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66850af2-cd78-4d69-87b7-6cbfd351575a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:42.267204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66850af2-cd78-4d69-87b7-6cbfd351575a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:42.267855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66850af2-cd78-4d69-87b7-6cbfd351575a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:42.270123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66850af2-cd78-4d69-87b7-6cbfd351575a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:42.284253 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4273/8458] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:42.285532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50fe2d-ea7d-4ace-9ff0-7b6a308dab83 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:42.296619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50fe2d-ea7d-4ace-9ff0-7b6a308dab83 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:42.302239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaedc68-a6f1-40dd-ada7-b9aa93aa880f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:42.302899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaedc68-a6f1-40dd-ada7-b9aa93aa880f admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:42.303201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaedc68-a6f1-40dd-ada7-b9aa93aa880f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:42.303468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaedc68-a6f1-40dd-ada7-b9aa93aa880f admin admin] PATH_INFO: `/v3/projects/f65e82e53217440fae4dd9129ec47f5b/users/f73a1cb53bef43bda5553926754a5fff/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:42.305418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaedc68-a6f1-40dd-ada7-b9aa93aa880f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f65e82e53217440fae4dd9129ec47f5b, user_id=f73a1cb53bef43bda5553926754a5fff, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:42.308049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaedc68-a6f1-40dd-ada7-b9aa93aa880f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:42.320792 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4186/8459] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:42 2022] PUT /identity/v3/projects/f65e82e53217440fae4dd9129ec47f5b/users/f73a1cb53bef43bda5553926754a5fff/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:42.332611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750926a5-f62f-402d-b672-141eb5ba1bdc None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:42.332896 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750926a5-f62f-402d-b672-141eb5ba1bdc None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:42.333360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750926a5-f62f-402d-b672-141eb5ba1bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:42.363086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-750926a5-f62f-402d-b672-141eb5ba1bdc None None] MFA Rules not processed for user `f73a1cb53bef43bda5553926754a5fff`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:42.364310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-750926a5-f62f-402d-b672-141eb5ba1bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:42.449752 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4274/8460] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:42.462740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaedc68-a6f1-40dd-ada7-b9aa93aa880f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:42.464565 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-beaedc68-a6f1-40dd-ada7-b9aa93aa880f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:42.534328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaedc68-a6f1-40dd-ada7-b9aa93aa880f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:42.538510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dec54f-a021-4332-9cbf-5df5a0119efa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:42.539250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dec54f-a021-4332-9cbf-5df5a0119efa service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:42.539735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dec54f-a021-4332-9cbf-5df5a0119efa service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:42.539999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dec54f-a021-4332-9cbf-5df5a0119efa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:42.600042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dec54f-a021-4332-9cbf-5df5a0119efa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:42.602344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dec54f-a021-4332-9cbf-5df5a0119efa service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:42.612844 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4187/8461] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:45.803033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750926a5-f62f-402d-b672-141eb5ba1bdc None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:45.891417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750926a5-f62f-402d-b672-141eb5ba1bdc None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:45.898151 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:45.899006 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:45.899385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:45.899768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] PATH_INFO: `/v3/users/f3f62913efa44df48a50bcac0ed3f63d` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:45.901193 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f62913efa44df48a50bcac0ed3f63d)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:45.903299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:45.947874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:45.985394 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f62913efa44df48a50bcac0ed3f63d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:46.000495 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f62913efa44df48a50bcac0ed3f63d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:46.009536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f62913efa44df48a50bcac0ed3f63d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:46.024141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f62913efa44df48a50bcac0ed3f63d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:46.025022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f62913efa44df48a50bcac0ed3f63d'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:46.036405 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4275/8462] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:45 2022] DELETE /identity/v3/users/f3f62913efa44df48a50bcac0ed3f63d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:46.040539 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d7442-66f1-4c1a-bd9b-0f8cb23d93e1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:46.041047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d7442-66f1-4c1a-bd9b-0f8cb23d93e1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:46.041141 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d7442-66f1-4c1a-bd9b-0f8cb23d93e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:46.070614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5c0d7442-66f1-4c1a-bd9b-0f8cb23d93e1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:46.071890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5c0d7442-66f1-4c1a-bd9b-0f8cb23d93e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:46.121189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:46.217401 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4188/8463] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:46.231221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:46.235426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:46.333335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec765be-a1ba-4508-bbaa-d9c7f8601839 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:46.340187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3aa57-2232-418f-a6d9-4d9bc2e8246b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:46.341652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3aa57-2232-418f-a6d9-4d9bc2e8246b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:46.341652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3aa57-2232-418f-a6d9-4d9bc2e8246b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:46.341652 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3aa57-2232-418f-a6d9-4d9bc2e8246b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:46.353013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3aa57-2232-418f-a6d9-4d9bc2e8246b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:46.354674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3aa57-2232-418f-a6d9-4d9bc2e8246b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:46.366249 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4276/8464] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:46.771591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d7442-66f1-4c1a-bd9b-0f8cb23d93e1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:46.773643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5c0d7442-66f1-4c1a-bd9b-0f8cb23d93e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:46.789986 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d7442-66f1-4c1a-bd9b-0f8cb23d93e1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:46.795665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:46.796018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:46.796422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:46.796727 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] PATH_INFO: `/v3/projects/12476914d40f47009089fdec72c541d7` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:46.797757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12476914d40f47009089fdec72c541d7)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:46.799930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:46.864789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12476914d40f47009089fdec72c541d7', 'request_id': 'req-4bf28f07-9cc8-40a4-9d30-033dce498de5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:46.871992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12476914d40f47009089fdec72c541d7', 'request_id': 'req-4bf28f07-9cc8-40a4-9d30-033dce498de5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:46.881640 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12476914d40f47009089fdec72c541d7', 'request_id': 'req-4bf28f07-9cc8-40a4-9d30-033dce498de5'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:46.895550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] The token cache is being invalidate because project 12476914d40f47009089fdec72c541d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:46.895938 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:46.896882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:46.902125 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4189/8465] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:46 2022] DELETE /identity/v3/projects/12476914d40f47009089fdec72c541d7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:46.903386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3aa57-2232-418f-a6d9-4d9bc2e8246b service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:46.916938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3aa57-2232-418f-a6d9-4d9bc2e8246b service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:46.985408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca5195-19a4-464e-b8b5-c00c5fb9e070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:46.986248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca5195-19a4-464e-b8b5-c00c5fb9e070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:46.986591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca5195-19a4-464e-b8b5-c00c5fb9e070 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:46.986978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca5195-19a4-464e-b8b5-c00c5fb9e070 admin admin] PATH_INFO: `/v3/users/ff4021f31c6d42e7a74ee89a575a0def` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:46.988171 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca5195-19a4-464e-b8b5-c00c5fb9e070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff4021f31c6d42e7a74ee89a575a0def)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:46.990208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca5195-19a4-464e-b8b5-c00c5fb9e070 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:47.068893 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:47.077774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0eca5195-19a4-464e-b8b5-c00c5fb9e070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff4021f31c6d42e7a74ee89a575a0def'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:47.090461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0eca5195-19a4-464e-b8b5-c00c5fb9e070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff4021f31c6d42e7a74ee89a575a0def'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:47.098989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0eca5195-19a4-464e-b8b5-c00c5fb9e070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff4021f31c6d42e7a74ee89a575a0def'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:47.112879 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0eca5195-19a4-464e-b8b5-c00c5fb9e070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff4021f31c6d42e7a74ee89a575a0def'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:47.113670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0eca5195-19a4-464e-b8b5-c00c5fb9e070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff4021f31c6d42e7a74ee89a575a0def'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:47.122802 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4277/8466] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:46 2022] DELETE /identity/v3/users/ff4021f31c6d42e7a74ee89a575a0def => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:28:47.473201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:47.474912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:47.501506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf28f07-9cc8-40a4-9d30-033dce498de5 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:47.591960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:47.592817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:47.593187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:47.593601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] PATH_INFO: `/v3/projects/c60a50e573db4e43b5bcce4d36d7d66b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:47.594959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60a50e573db4e43b5bcce4d36d7d66b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:47.597327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:47.643653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39172d35-a3b3-4399-8073-7d523700cffe None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:47.643947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39172d35-a3b3-4399-8073-7d523700cffe None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:47.644178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39172d35-a3b3-4399-8073-7d523700cffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:47.671622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-39172d35-a3b3-4399-8073-7d523700cffe None None] MFA Rules not processed for user `f2c2c4da3c954f96af706644f238ed97`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:47.672967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-39172d35-a3b3-4399-8073-7d523700cffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:47.678567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60a50e573db4e43b5bcce4d36d7d66b', 'request_id': 'req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:47.688323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60a50e573db4e43b5bcce4d36d7d66b', 'request_id': 'req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:47.708171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60a50e573db4e43b5bcce4d36d7d66b', 'request_id': 'req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:47.710267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:47.730756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] The token cache is being invalidate because project c60a50e573db4e43b5bcce4d36d7d66b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:47.731152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:47.732007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:47.734994 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4190/8467] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:47 2022] DELETE /identity/v3/projects/c60a50e573db4e43b5bcce4d36d7d66b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:47.740984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:47.743104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:47.822564 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4278/8468] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:47.901140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e55d2c-d4de-47c2-a329-d3a947f5d6c9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:47.932793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:47.934234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:47.936718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:47.937239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] PATH_INFO: `/v3/users/f73a1cb53bef43bda5553926754a5fff` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:47.938813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f73a1cb53bef43bda5553926754a5fff)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:47.941236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:48.020331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f73a1cb53bef43bda5553926754a5fff'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:48.036642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f73a1cb53bef43bda5553926754a5fff'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:48.056927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f73a1cb53bef43bda5553926754a5fff'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:48.057809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f73a1cb53bef43bda5553926754a5fff'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:48.058517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f73a1cb53bef43bda5553926754a5fff'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:48.090994 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4191/8469] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:47 2022] DELETE /identity/v3/users/f73a1cb53bef43bda5553926754a5fff => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:48.296403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39172d35-a3b3-4399-8073-7d523700cffe None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:48.298212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-39172d35-a3b3-4399-8073-7d523700cffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:48.401931 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39172d35-a3b3-4399-8073-7d523700cffe None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:48.478207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35947dfa-0b11-4bc3-a432-f85c20b6c69b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:48.479330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35947dfa-0b11-4bc3-a432-f85c20b6c69b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:48.479614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35947dfa-0b11-4bc3-a432-f85c20b6c69b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:48.479883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35947dfa-0b11-4bc3-a432-f85c20b6c69b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:48.482364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-35947dfa-0b11-4bc3-a432-f85c20b6c69b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:48.498900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35947dfa-0b11-4bc3-a432-f85c20b6c69b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:48.500403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35947dfa-0b11-4bc3-a432-f85c20b6c69b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:48.518512 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4279/8470] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:48.603383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:48.604689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:48.701651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b2f2c-e770-4191-80fa-e853dab9ec86 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:48.706947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf32328-ef13-461a-9d17-0dace9538929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:48.707606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf32328-ef13-461a-9d17-0dace9538929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:48.707877 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf32328-ef13-461a-9d17-0dace9538929 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:48.708143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf32328-ef13-461a-9d17-0dace9538929 admin admin] PATH_INFO: `/v3/projects/f65e82e53217440fae4dd9129ec47f5b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:48.709314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf32328-ef13-461a-9d17-0dace9538929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f65e82e53217440fae4dd9129ec47f5b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:48.711219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf32328-ef13-461a-9d17-0dace9538929 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:48.725457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:48.770432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2bf32328-ef13-461a-9d17-0dace9538929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f65e82e53217440fae4dd9129ec47f5b', 'request_id': 'req-2bf32328-ef13-461a-9d17-0dace9538929'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:48.778006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2bf32328-ef13-461a-9d17-0dace9538929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f65e82e53217440fae4dd9129ec47f5b', 'request_id': 'req-2bf32328-ef13-461a-9d17-0dace9538929'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:48.790172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2bf32328-ef13-461a-9d17-0dace9538929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f65e82e53217440fae4dd9129ec47f5b', 'request_id': 'req-2bf32328-ef13-461a-9d17-0dace9538929'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:48.803336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2bf32328-ef13-461a-9d17-0dace9538929 admin admin] The token cache is being invalidate because project f65e82e53217440fae4dd9129ec47f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:48.803622 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2bf32328-ef13-461a-9d17-0dace9538929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:48.804319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2bf32328-ef13-461a-9d17-0dace9538929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:48.806255 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4192/8471] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:48 2022] DELETE /identity/v3/projects/f65e82e53217440fae4dd9129ec47f5b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:48.811042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db2c88-cebd-4c9a-90aa-176e16050c4b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:48.811330 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db2c88-cebd-4c9a-90aa-176e16050c4b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:48.811601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db2c88-cebd-4c9a-90aa-176e16050c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:48.836468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-24db2c88-cebd-4c9a-90aa-176e16050c4b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:48.837731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-24db2c88-cebd-4c9a-90aa-176e16050c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:48.850401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:48.926915 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4280/8472] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:48.927963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b86ef-4b73-4138-b3f8-b06e6374a23b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:48.928283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b86ef-4b73-4138-b3f8-b06e6374a23b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:48.928602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b86ef-4b73-4138-b3f8-b06e6374a23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:48.958821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1c2b86ef-4b73-4138-b3f8-b06e6374a23b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:48.959842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1c2b86ef-4b73-4138-b3f8-b06e6374a23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:48.976065 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4193/8473] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:28:48.980349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db2c88-cebd-4c9a-90aa-176e16050c4b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:48.990371 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db2c88-cebd-4c9a-90aa-176e16050c4b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:48.994995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21228680-4970-43ad-85e5-f294914cf994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:48.995609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21228680-4970-43ad-85e5-f294914cf994 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:48.995875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21228680-4970-43ad-85e5-f294914cf994 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:48.996133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21228680-4970-43ad-85e5-f294914cf994 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:48.996811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21228680-4970-43ad-85e5-f294914cf994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:48.997106 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21228680-4970-43ad-85e5-f294914cf994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:48.998732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21228680-4970-43ad-85e5-f294914cf994 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:49.025087 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4281/8474] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:28:49.028285 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b86ef-4b73-4138-b3f8-b06e6374a23b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:49.042500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b86ef-4b73-4138-b3f8-b06e6374a23b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:49.050985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8a2a9-5e4b-407a-bfe4-e1590abfd0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:49.051616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8a2a9-5e4b-407a-bfe4-e1590abfd0d0 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:49.052150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8a2a9-5e4b-407a-bfe4-e1590abfd0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:49.052462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8a2a9-5e4b-407a-bfe4-e1590abfd0d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:49.055131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f8a2a9-5e4b-407a-bfe4-e1590abfd0d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:49.059795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f8a2a9-5e4b-407a-bfe4-e1590abfd0d0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:49.097681 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4194/8475] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:49 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:49.099553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21228680-4970-43ad-85e5-f294914cf994 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:49.110577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21228680-4970-43ad-85e5-f294914cf994 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:49.196243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a37cae-b298-45a3-9991-8098e4211953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:49.196962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a37cae-b298-45a3-9991-8098e4211953 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:49.197241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a37cae-b298-45a3-9991-8098e4211953 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:49.197922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a37cae-b298-45a3-9991-8098e4211953 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:49.198187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a37cae-b298-45a3-9991-8098e4211953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:49.200681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a37cae-b298-45a3-9991-8098e4211953 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:49.230767 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4282/8476] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:49 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:28:49.234249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8a2a9-5e4b-407a-bfe4-e1590abfd0d0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:49.249449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8a2a9-5e4b-407a-bfe4-e1590abfd0d0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:49.254726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c49e0b-24f4-4872-8b7c-0ecac5635605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:49.255378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c49e0b-24f4-4872-8b7c-0ecac5635605 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:49.255655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c49e0b-24f4-4872-8b7c-0ecac5635605 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:49.256266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c49e0b-24f4-4872-8b7c-0ecac5635605 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:49.256987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c49e0b-24f4-4872-8b7c-0ecac5635605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:49.258802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c49e0b-24f4-4872-8b7c-0ecac5635605 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:49.270877 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4195/8477] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:49.275962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a37cae-b298-45a3-9991-8098e4211953 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:49.290883 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a37cae-b298-45a3-9991-8098e4211953 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:49.296069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bfe9d-8af6-4573-b5fb-0c9bdf89f110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:49.296835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bfe9d-8af6-4573-b5fb-0c9bdf89f110 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:49.297735 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bfe9d-8af6-4573-b5fb-0c9bdf89f110 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:49.297997 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bfe9d-8af6-4573-b5fb-0c9bdf89f110 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:49.298881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bfe9d-8af6-4573-b5fb-0c9bdf89f110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:49.300542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bfe9d-8af6-4573-b5fb-0c9bdf89f110 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:49.312260 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4283/8478] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:49.317696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c49e0b-24f4-4872-8b7c-0ecac5635605 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:49.331206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c49e0b-24f4-4872-8b7c-0ecac5635605 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:49.336026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079bbb8d-a4e6-40c6-81a7-8f6c5219f287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:49.336726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bbb8d-a4e6-40c6-81a7-8f6c5219f287 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:49.337208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bbb8d-a4e6-40c6-81a7-8f6c5219f287 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:49.337484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bbb8d-a4e6-40c6-81a7-8f6c5219f287 admin admin] PATH_INFO: `/v3/projects/a6dd9d2859534be6b3f27c08959e37d6/users/cefb5b1c226f4e01bfef5aebfd101087/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:49.363909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079bbb8d-a4e6-40c6-81a7-8f6c5219f287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6dd9d2859534be6b3f27c08959e37d6, user_id=cefb5b1c226f4e01bfef5aebfd101087, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:49.367383 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079bbb8d-a4e6-40c6-81a7-8f6c5219f287 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:49.389603 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4196/8479] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:49 2022] PUT /identity/v3/projects/a6dd9d2859534be6b3f27c08959e37d6/users/cefb5b1c226f4e01bfef5aebfd101087/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:28:49.402253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1474e-d9da-4305-8f17-5ba21fab6947 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:49.402549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1474e-d9da-4305-8f17-5ba21fab6947 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:49.402814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1474e-d9da-4305-8f17-5ba21fab6947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:49.445905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-15a1474e-d9da-4305-8f17-5ba21fab6947 None None] MFA Rules not processed for user `cefb5b1c226f4e01bfef5aebfd101087`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:49.447470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-15a1474e-d9da-4305-8f17-5ba21fab6947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:49.532731 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4284/8480] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:49.534299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079bbb8d-a4e6-40c6-81a7-8f6c5219f287 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:49.536547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-079bbb8d-a4e6-40c6-81a7-8f6c5219f287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:49.614562 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079bbb8d-a4e6-40c6-81a7-8f6c5219f287 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:49.618537 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c436ab4-0880-4cca-8996-49d96d12f2a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:49.619278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c436ab4-0880-4cca-8996-49d96d12f2a8 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:49.619874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c436ab4-0880-4cca-8996-49d96d12f2a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:49.619874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c436ab4-0880-4cca-8996-49d96d12f2a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:49.630665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c436ab4-0880-4cca-8996-49d96d12f2a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:49.632224 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c436ab4-0880-4cca-8996-49d96d12f2a8 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:49.648713 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4197/8481] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:53.059456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1474e-d9da-4305-8f17-5ba21fab6947 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:53.073117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:53.129583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1474e-d9da-4305-8f17-5ba21fab6947 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:53.134601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:53.135254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:53.135501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:53.135874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] PATH_INFO: `/v3/users/cefb5b1c226f4e01bfef5aebfd101087` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:53.137051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cefb5b1c226f4e01bfef5aebfd101087)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:53.138975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:53.187344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cefb5b1c226f4e01bfef5aebfd101087'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:53.196267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cefb5b1c226f4e01bfef5aebfd101087'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:53.203274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cefb5b1c226f4e01bfef5aebfd101087'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:53.215422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cefb5b1c226f4e01bfef5aebfd101087'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:53.216135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cefb5b1c226f4e01bfef5aebfd101087'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:53.222545 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4285/8482] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:53 2022] DELETE /identity/v3/users/cefb5b1c226f4e01bfef5aebfd101087 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:53.227662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023aee9-d8aa-428a-bdf2-539991382c65 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:53.228187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023aee9-d8aa-428a-bdf2-539991382c65 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:53.228187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023aee9-d8aa-428a-bdf2-539991382c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:53.248129 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0023aee9-d8aa-428a-bdf2-539991382c65 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:53.249361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0023aee9-d8aa-428a-bdf2-539991382c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:53.281626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:53.312893 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4198/8483] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:53.324725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:53.326265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:53.387921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f407c0-d31e-46cd-b1de-bb0a1d5fd5e9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:53.393228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602b0f6-38ff-4ed0-973b-d5906e6552c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:53.393844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602b0f6-38ff-4ed0-973b-d5906e6552c0 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:53.394117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602b0f6-38ff-4ed0-973b-d5906e6552c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:53.394379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602b0f6-38ff-4ed0-973b-d5906e6552c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:53.403912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602b0f6-38ff-4ed0-973b-d5906e6552c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:53.405380 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602b0f6-38ff-4ed0-973b-d5906e6552c0 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:53.415624 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4286/8484] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:54.122136 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023aee9-d8aa-428a-bdf2-539991382c65 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:54.123732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0023aee9-d8aa-428a-bdf2-539991382c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:54.137701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023aee9-d8aa-428a-bdf2-539991382c65 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:54.144048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f80dc2-5add-4e1b-8490-3464936d0197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:54.144887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f80dc2-5add-4e1b-8490-3464936d0197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:54.145258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f80dc2-5add-4e1b-8490-3464936d0197 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:54.145658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f80dc2-5add-4e1b-8490-3464936d0197 admin admin] PATH_INFO: `/v3/projects/a6dd9d2859534be6b3f27c08959e37d6` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:54.146948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f80dc2-5add-4e1b-8490-3464936d0197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6dd9d2859534be6b3f27c08959e37d6)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:54.149041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f80dc2-5add-4e1b-8490-3464936d0197 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:54.200654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4f80dc2-5add-4e1b-8490-3464936d0197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6dd9d2859534be6b3f27c08959e37d6', 'request_id': 'req-c4f80dc2-5add-4e1b-8490-3464936d0197'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:54.207564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4f80dc2-5add-4e1b-8490-3464936d0197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6dd9d2859534be6b3f27c08959e37d6', 'request_id': 'req-c4f80dc2-5add-4e1b-8490-3464936d0197'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:54.216571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4f80dc2-5add-4e1b-8490-3464936d0197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6dd9d2859534be6b3f27c08959e37d6', 'request_id': 'req-c4f80dc2-5add-4e1b-8490-3464936d0197'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:54.235923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4f80dc2-5add-4e1b-8490-3464936d0197 admin admin] The token cache is being invalidate because project a6dd9d2859534be6b3f27c08959e37d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:54.236686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4f80dc2-5add-4e1b-8490-3464936d0197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:54.237197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-c4f80dc2-5add-4e1b-8490-3464936d0197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:54.239547 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4199/8485] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:54 2022] DELETE /identity/v3/projects/a6dd9d2859534be6b3f27c08959e37d6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:54.244598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75abce4-884a-407b-99b1-6f67394217d6 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:54.245052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75abce4-884a-407b-99b1-6f67394217d6 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:54.245232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75abce4-884a-407b-99b1-6f67394217d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:54.265543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f75abce4-884a-407b-99b1-6f67394217d6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:54.266736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f75abce4-884a-407b-99b1-6f67394217d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:54.298533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:54.333137 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4287/8486] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:54.341104 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b020b7-cbba-429f-87cf-a74d6e317e94 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:54.341385 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b020b7-cbba-429f-87cf-a74d6e317e94 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:54.341656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b020b7-cbba-429f-87cf-a74d6e317e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:54.361896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d6b020b7-cbba-429f-87cf-a74d6e317e94 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:54.363081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d6b020b7-cbba-429f-87cf-a74d6e317e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:54.371537 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4200/8487] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:54.375917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75abce4-884a-407b-99b1-6f67394217d6 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:54.385819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75abce4-884a-407b-99b1-6f67394217d6 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:54.390466 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa664b6-0a2b-4f8c-83d0-9649368a40bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:54.391092 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa664b6-0a2b-4f8c-83d0-9649368a40bc admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:54.391363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa664b6-0a2b-4f8c-83d0-9649368a40bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:54.391634 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa664b6-0a2b-4f8c-83d0-9649368a40bc admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:54.392527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa664b6-0a2b-4f8c-83d0-9649368a40bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:54.392783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa664b6-0a2b-4f8c-83d0-9649368a40bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:54.394591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa664b6-0a2b-4f8c-83d0-9649368a40bc admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:54.406746 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4288/8488] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 20 21:28:54.411758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b020b7-cbba-429f-87cf-a74d6e317e94 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:54.421600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b020b7-cbba-429f-87cf-a74d6e317e94 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:54.427093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890f68c-59a8-4bff-8494-21b9d015fe75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:54.427778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890f68c-59a8-4bff-8494-21b9d015fe75 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:54.428109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890f68c-59a8-4bff-8494-21b9d015fe75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:54.428484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890f68c-59a8-4bff-8494-21b9d015fe75 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:54.429174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890f68c-59a8-4bff-8494-21b9d015fe75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:54.431138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890f68c-59a8-4bff-8494-21b9d015fe75 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:54.465487 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4201/8489] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:54 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:54.466787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa664b6-0a2b-4f8c-83d0-9649368a40bc admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:54.477373 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa664b6-0a2b-4f8c-83d0-9649368a40bc admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:54.537017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef1b69-f621-4501-99d6-06282582c50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:54.537890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef1b69-f621-4501-99d6-06282582c50d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:54.538244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef1b69-f621-4501-99d6-06282582c50d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:54.538613 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef1b69-f621-4501-99d6-06282582c50d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:54.539570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef1b69-f621-4501-99d6-06282582c50d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:54.542942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef1b69-f621-4501-99d6-06282582c50d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:54.569562 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4289/8490] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:54 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:54.573780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890f68c-59a8-4bff-8494-21b9d015fe75 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:54.584773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890f68c-59a8-4bff-8494-21b9d015fe75 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:54.591162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c4396-d004-4091-a1d9-86314d610bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:54.591760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c4396-d004-4091-a1d9-86314d610bcb admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:54.592015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c4396-d004-4091-a1d9-86314d610bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:54.592286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c4396-d004-4091-a1d9-86314d610bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:54.592948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c4396-d004-4091-a1d9-86314d610bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:54.594540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c4396-d004-4091-a1d9-86314d610bcb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:54.605141 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4202/8491] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:28:54.609717 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef1b69-f621-4501-99d6-06282582c50d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:54.620885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef1b69-f621-4501-99d6-06282582c50d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:54.626655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ba62b-a3da-4a73-9236-7151f7350bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:54.627804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ba62b-a3da-4a73-9236-7151f7350bfd admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:54.628074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ba62b-a3da-4a73-9236-7151f7350bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:54.628332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ba62b-a3da-4a73-9236-7151f7350bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:54.629010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ba62b-a3da-4a73-9236-7151f7350bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:54.630633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ba62b-a3da-4a73-9236-7151f7350bfd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:54.642783 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4290/8492] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:54.647647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c4396-d004-4091-a1d9-86314d610bcb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:54.658814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c4396-d004-4091-a1d9-86314d610bcb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:54.664742 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f903f-0005-4819-a9be-10ad70ebb9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:54.665704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f903f-0005-4819-a9be-10ad70ebb9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:54.666024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f903f-0005-4819-a9be-10ad70ebb9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:54.666403 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f903f-0005-4819-a9be-10ad70ebb9e1 admin admin] PATH_INFO: `/v3/projects/8fb996374ee54c27b0a8ce581f9b460f/users/3d60d6a74ef3436daf41b2ef3df79010/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:54.686263 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f903f-0005-4819-a9be-10ad70ebb9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb996374ee54c27b0a8ce581f9b460f, user_id=3d60d6a74ef3436daf41b2ef3df79010, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:54.689009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f903f-0005-4819-a9be-10ad70ebb9e1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:54.707421 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4203/8493] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:54 2022] PUT /identity/v3/projects/8fb996374ee54c27b0a8ce581f9b460f/users/3d60d6a74ef3436daf41b2ef3df79010/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:54.718391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a2cde-724e-45ea-a6ca-7c30015e878a None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:54.718694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a2cde-724e-45ea-a6ca-7c30015e878a None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:54.718955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a2cde-724e-45ea-a6ca-7c30015e878a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:54.749973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ef6a2cde-724e-45ea-a6ca-7c30015e878a None None] MFA Rules not processed for user `3d60d6a74ef3436daf41b2ef3df79010`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:54.751204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ef6a2cde-724e-45ea-a6ca-7c30015e878a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:54.822856 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4291/8494] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:54 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:54.835259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f903f-0005-4819-a9be-10ad70ebb9e1 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:54.837164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1e9f903f-0005-4819-a9be-10ad70ebb9e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:54.897603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f903f-0005-4819-a9be-10ad70ebb9e1 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:54.901585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd50e08-bfe4-496c-a7ce-9345b5b456b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53928836828d45fbb427dd92492cae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:54.902218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd50e08-bfe4-496c-a7ce-9345b5b456b8 service cinder] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:54.902491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd50e08-bfe4-496c-a7ce-9345b5b456b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:54.902756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd50e08-bfe4-496c-a7ce-9345b5b456b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:54.912125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd50e08-bfe4-496c-a7ce-9345b5b456b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:54.913809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd50e08-bfe4-496c-a7ce-9345b5b456b8 service cinder] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:54.918508 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:54.923901 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4204/8495] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:56.642487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a2cde-724e-45ea-a6ca-7c30015e878a None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:56.674558 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:56.708238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a2cde-724e-45ea-a6ca-7c30015e878a None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:56.713245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:56.713874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:56.714136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:56.714378 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] PATH_INFO: `/v3/users/3d60d6a74ef3436daf41b2ef3df79010` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:56.715543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d60d6a74ef3436daf41b2ef3df79010)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:56.717497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:56.768076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d60d6a74ef3436daf41b2ef3df79010'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:56.775457 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d60d6a74ef3436daf41b2ef3df79010'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:56.782332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d60d6a74ef3436daf41b2ef3df79010'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:56.794390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d60d6a74ef3436daf41b2ef3df79010'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:56.795132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d60d6a74ef3436daf41b2ef3df79010'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:56.801421 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4292/8496] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:28:56 2022] DELETE /identity/v3/users/3d60d6a74ef3436daf41b2ef3df79010 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:56.806084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe620801-85df-47a8-8646-c6d23240921a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:56.806605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe620801-85df-47a8-8646-c6d23240921a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:56.806605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe620801-85df-47a8-8646-c6d23240921a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:56.828499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fe620801-85df-47a8-8646-c6d23240921a None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:56.829725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fe620801-85df-47a8-8646-c6d23240921a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:56.891351 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4205/8497] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:56.902608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:56.905113 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:56.980654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d487d3b-8022-4876-83d6-f1630f6b1251 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:56.987573 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee8030-94e0-4f73-96bf-aa2cedfe3d99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:56.988505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee8030-94e0-4f73-96bf-aa2cedfe3d99 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:56.988975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee8030-94e0-4f73-96bf-aa2cedfe3d99 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:56.989461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee8030-94e0-4f73-96bf-aa2cedfe3d99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:57.001979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee8030-94e0-4f73-96bf-aa2cedfe3d99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:57.004743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee8030-94e0-4f73-96bf-aa2cedfe3d99 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:57.021058 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4293/8498] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:28:57.311180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe620801-85df-47a8-8646-c6d23240921a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:57.312760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fe620801-85df-47a8-8646-c6d23240921a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:57.327785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe620801-85df-47a8-8646-c6d23240921a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:57.333577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c5346-c6d5-4c52-81e5-f1eb58a3835b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:57.334295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c5346-c6d5-4c52-81e5-f1eb58a3835b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:57.334859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c5346-c6d5-4c52-81e5-f1eb58a3835b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:57.335132 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c5346-c6d5-4c52-81e5-f1eb58a3835b admin admin] PATH_INFO: `/v3/projects/8fb996374ee54c27b0a8ce581f9b460f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:57.336428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c5346-c6d5-4c52-81e5-f1eb58a3835b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fb996374ee54c27b0a8ce581f9b460f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:57.338407 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c5346-c6d5-4c52-81e5-f1eb58a3835b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:57.395098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-580c5346-c6d5-4c52-81e5-f1eb58a3835b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fb996374ee54c27b0a8ce581f9b460f', 'request_id': 'req-580c5346-c6d5-4c52-81e5-f1eb58a3835b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:57.402803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-580c5346-c6d5-4c52-81e5-f1eb58a3835b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fb996374ee54c27b0a8ce581f9b460f', 'request_id': 'req-580c5346-c6d5-4c52-81e5-f1eb58a3835b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:57.412561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-580c5346-c6d5-4c52-81e5-f1eb58a3835b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fb996374ee54c27b0a8ce581f9b460f', 'request_id': 'req-580c5346-c6d5-4c52-81e5-f1eb58a3835b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:57.424648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-580c5346-c6d5-4c52-81e5-f1eb58a3835b admin admin] The token cache is being invalidate because project 8fb996374ee54c27b0a8ce581f9b460f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:28:57.424926 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-580c5346-c6d5-4c52-81e5-f1eb58a3835b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:57.425602 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-580c5346-c6d5-4c52-81e5-f1eb58a3835b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:28:57.427869 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4206/8499] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:28:57 2022] DELETE /identity/v3/projects/8fb996374ee54c27b0a8ce581f9b460f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:57.433332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eec597-ba3f-486e-a9c0-28b7894dca89 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:57.433663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eec597-ba3f-486e-a9c0-28b7894dca89 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:57.433901 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eec597-ba3f-486e-a9c0-28b7894dca89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:57.455905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-10eec597-ba3f-486e-a9c0-28b7894dca89 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:57.457446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-10eec597-ba3f-486e-a9c0-28b7894dca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:57.519669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:57.555125 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4294/8500] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:57.568748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1fc36-5bd2-4757-bb07-ae2d326ffebf None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:57.569494 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1fc36-5bd2-4757-bb07-ae2d326ffebf None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:57.569939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1fc36-5bd2-4757-bb07-ae2d326ffebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:57.591254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-66a1fc36-5bd2-4757-bb07-ae2d326ffebf None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:57.592572 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-66a1fc36-5bd2-4757-bb07-ae2d326ffebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:57.601750 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4207/8501] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:57.605922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eec597-ba3f-486e-a9c0-28b7894dca89 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:57.616707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eec597-ba3f-486e-a9c0-28b7894dca89 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:57.621770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2aed9-3c89-45de-85b4-49a869dd13d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:57.622796 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2aed9-3c89-45de-85b4-49a869dd13d6 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:57.623135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2aed9-3c89-45de-85b4-49a869dd13d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:57.623472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2aed9-3c89-45de-85b4-49a869dd13d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:57.624195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2aed9-3c89-45de-85b4-49a869dd13d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:28:57.624604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2aed9-3c89-45de-85b4-49a869dd13d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:57.626369 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2aed9-3c89-45de-85b4-49a869dd13d6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:57.641252 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4295/8502] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:28:57.646525 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1fc36-5bd2-4757-bb07-ae2d326ffebf None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:57.656827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1fc36-5bd2-4757-bb07-ae2d326ffebf None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:57.662226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d542a-33a2-48f5-b2a7-06e68ce0a4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:57.663204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d542a-33a2-48f5-b2a7-06e68ce0a4af admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:57.663617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d542a-33a2-48f5-b2a7-06e68ce0a4af admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:57.663983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d542a-33a2-48f5-b2a7-06e68ce0a4af admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:57.664730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d542a-33a2-48f5-b2a7-06e68ce0a4af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:57.668072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d542a-33a2-48f5-b2a7-06e68ce0a4af admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:57.713093 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4208/8503] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:28:57 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:28:57.717962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2aed9-3c89-45de-85b4-49a869dd13d6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:57.730116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2aed9-3c89-45de-85b4-49a869dd13d6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:57.793073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18a5a8-d2e3-4b83-9482-13ebc7906d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:57.793686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18a5a8-d2e3-4b83-9482-13ebc7906d59 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:57.793949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18a5a8-d2e3-4b83-9482-13ebc7906d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:57.794208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18a5a8-d2e3-4b83-9482-13ebc7906d59 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:57.794844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18a5a8-d2e3-4b83-9482-13ebc7906d59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:57.796798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18a5a8-d2e3-4b83-9482-13ebc7906d59 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:57.820558 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4296/8504] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:28:57 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:28:57.824762 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d542a-33a2-48f5-b2a7-06e68ce0a4af admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:57.835923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d542a-33a2-48f5-b2a7-06e68ce0a4af admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:57.840989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7401ded-4ec4-454b-a081-e51960d0c03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:57.841593 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7401ded-4ec4-454b-a081-e51960d0c03a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:57.841869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7401ded-4ec4-454b-a081-e51960d0c03a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:57.842131 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7401ded-4ec4-454b-a081-e51960d0c03a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:57.842778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7401ded-4ec4-454b-a081-e51960d0c03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:57.844433 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7401ded-4ec4-454b-a081-e51960d0c03a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:57.855771 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4209/8505] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:28:57.861063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18a5a8-d2e3-4b83-9482-13ebc7906d59 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:57.872065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18a5a8-d2e3-4b83-9482-13ebc7906d59 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:57.877023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde8a5e-c763-4e5e-8410-6f7bbe1a50ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:57.877625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde8a5e-c763-4e5e-8410-6f7bbe1a50ad admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:57.878138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde8a5e-c763-4e5e-8410-6f7bbe1a50ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:57.878476 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde8a5e-c763-4e5e-8410-6f7bbe1a50ad admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:57.879120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde8a5e-c763-4e5e-8410-6f7bbe1a50ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:57.880790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde8a5e-c763-4e5e-8410-6f7bbe1a50ad admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:57.891594 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4297/8506] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:28:57.896020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7401ded-4ec4-454b-a081-e51960d0c03a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:57.906282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7401ded-4ec4-454b-a081-e51960d0c03a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:57.912172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f78a3-f8a4-4842-86af-0c1d09c49fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:57.912995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f78a3-f8a4-4842-86af-0c1d09c49fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:57.913286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f78a3-f8a4-4842-86af-0c1d09c49fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:57.913552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f78a3-f8a4-4842-86af-0c1d09c49fa0 admin admin] PATH_INFO: `/v3/projects/3d968d1be5234a92a89856c0b73b8367/users/678a391f226b4d42820eaf1bb0ff78d5/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:57.929214 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f78a3-f8a4-4842-86af-0c1d09c49fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d968d1be5234a92a89856c0b73b8367, user_id=678a391f226b4d42820eaf1bb0ff78d5, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:57.931913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f78a3-f8a4-4842-86af-0c1d09c49fa0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:57.946218 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4210/8507] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:28:57 2022] PUT /identity/v3/projects/3d968d1be5234a92a89856c0b73b8367/users/678a391f226b4d42820eaf1bb0ff78d5/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:28:57.951509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f86625-dff5-477f-9d10-8b39195776d3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:57.951791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f86625-dff5-477f-9d10-8b39195776d3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:57.952057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f86625-dff5-477f-9d10-8b39195776d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:57.971866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-44f86625-dff5-477f-9d10-8b39195776d3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:28:57.973291 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-44f86625-dff5-477f-9d10-8b39195776d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:58.033980 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4298/8508] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:28:58.043878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f78a3-f8a4-4842-86af-0c1d09c49fa0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:28:58.045882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d33f78a3-f8a4-4842-86af-0c1d09c49fa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:28:58.069898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:28:58.114961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f78a3-f8a4-4842-86af-0c1d09c49fa0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:28:58.119730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a6d0f-d449-48a9-93f8-ace262f16892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:28:58.120334 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a6d0f-d449-48a9-93f8-ace262f16892 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:28:58.120641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a6d0f-d449-48a9-93f8-ace262f16892 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:28:58.120905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a6d0f-d449-48a9-93f8-ace262f16892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:28:58.130892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a6d0f-d449-48a9-93f8-ace262f16892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:28:58.132486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a6d0f-d449-48a9-93f8-ace262f16892 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:28:58.142505 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4211/8509] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:28:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:29:02.656349 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651b6eb-e226-43f9-b406-333397654806 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:02.656986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651b6eb-e226-43f9-b406-333397654806 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:02.656986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651b6eb-e226-43f9-b406-333397654806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:02.688062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b651b6eb-e226-43f9-b406-333397654806 None None] MFA Rules not processed for user `678a391f226b4d42820eaf1bb0ff78d5`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:29:02.689406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b651b6eb-e226-43f9-b406-333397654806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:02.760393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:29:02.795746 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4299/8510] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:29:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:29:02.800644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a6d0f-d449-48a9-93f8-ace262f16892 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:02.813761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a6d0f-d449-48a9-93f8-ace262f16892 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:02.819858 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e9a3b-7765-4452-9922-9b7f94e00d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:02.820614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e9a3b-7765-4452-9922-9b7f94e00d7e admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:02.821120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e9a3b-7765-4452-9922-9b7f94e00d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:02.821472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e9a3b-7765-4452-9922-9b7f94e00d7e admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:02.822197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e9a3b-7765-4452-9922-9b7f94e00d7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:02.824442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e9a3b-7765-4452-9922-9b7f94e00d7e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:02.865057 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4212/8511] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:29:02 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:29:02.869593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651b6eb-e226-43f9-b406-333397654806 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:02.882670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651b6eb-e226-43f9-b406-333397654806 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:02.947633 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83916a-842e-454a-a5de-e29cc065df02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:02.948254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83916a-842e-454a-a5de-e29cc065df02 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:02.948557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83916a-842e-454a-a5de-e29cc065df02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:02.948819 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83916a-842e-454a-a5de-e29cc065df02 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:02.949436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83916a-842e-454a-a5de-e29cc065df02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:02.951560 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83916a-842e-454a-a5de-e29cc065df02 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:02.981848 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4300/8512] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:29:02 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:29:02.986021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e9a3b-7765-4452-9922-9b7f94e00d7e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:02.998251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e9a3b-7765-4452-9922-9b7f94e00d7e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:03.004793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cfcce-634f-43d9-9342-49ca9fe0b113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:03.005418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cfcce-634f-43d9-9342-49ca9fe0b113 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:03.005693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cfcce-634f-43d9-9342-49ca9fe0b113 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:03.005961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cfcce-634f-43d9-9342-49ca9fe0b113 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:03.006595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cfcce-634f-43d9-9342-49ca9fe0b113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:03.008195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cfcce-634f-43d9-9342-49ca9fe0b113 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:03.020983 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4213/8513] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:29:03.025230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83916a-842e-454a-a5de-e29cc065df02 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:03.035867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83916a-842e-454a-a5de-e29cc065df02 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:03.040785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8354e-e647-446d-bb25-42ee3901f2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:03.041570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8354e-e647-446d-bb25-42ee3901f2c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:03.042049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8354e-e647-446d-bb25-42ee3901f2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:03.042572 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8354e-e647-446d-bb25-42ee3901f2c2 admin admin] PATH_INFO: `/v3/projects/6e1121a798be4fa08ea4b6f90bb16d95/users/e30ec0d6de0e427e8dc76cd7b6e45339/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:03.062219 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8354e-e647-446d-bb25-42ee3901f2c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1121a798be4fa08ea4b6f90bb16d95, user_id=e30ec0d6de0e427e8dc76cd7b6e45339, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:03.065141 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8354e-e647-446d-bb25-42ee3901f2c2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:03.092292 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4301/8514] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:29:03 2022] PUT /identity/v3/projects/6e1121a798be4fa08ea4b6f90bb16d95/users/e30ec0d6de0e427e8dc76cd7b6e45339/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:29:03.093585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cfcce-634f-43d9-9342-49ca9fe0b113 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:03.106455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cfcce-634f-43d9-9342-49ca9fe0b113 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:03.169381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283656dc-6340-476b-b94a-4692e18034da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:03.169987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283656dc-6340-476b-b94a-4692e18034da admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:03.170261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283656dc-6340-476b-b94a-4692e18034da admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:03.170499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283656dc-6340-476b-b94a-4692e18034da admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:03.171185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283656dc-6340-476b-b94a-4692e18034da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:03.172836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283656dc-6340-476b-b94a-4692e18034da admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:03.184061 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4214/8515] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:29:03.188109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8354e-e647-446d-bb25-42ee3901f2c2 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:03.199736 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8354e-e647-446d-bb25-42ee3901f2c2 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:03.204648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bee35c-1bbb-4c91-972c-f7dcdeb12d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:03.205304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bee35c-1bbb-4c91-972c-f7dcdeb12d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:03.205579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bee35c-1bbb-4c91-972c-f7dcdeb12d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:03.205890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bee35c-1bbb-4c91-972c-f7dcdeb12d0a admin admin] PATH_INFO: `/v3/projects/6e1121a798be4fa08ea4b6f90bb16d95/users/e30ec0d6de0e427e8dc76cd7b6e45339/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:03.207916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bee35c-1bbb-4c91-972c-f7dcdeb12d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1121a798be4fa08ea4b6f90bb16d95, user_id=e30ec0d6de0e427e8dc76cd7b6e45339, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:03.210599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bee35c-1bbb-4c91-972c-f7dcdeb12d0a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:03.222177 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4302/8516] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:29:03 2022] PUT /identity/v3/projects/6e1121a798be4fa08ea4b6f90bb16d95/users/e30ec0d6de0e427e8dc76cd7b6e45339/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:29:07.234567 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283656dc-6340-476b-b94a-4692e18034da admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:07.236653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-283656dc-6340-476b-b94a-4692e18034da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:07.436655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283656dc-6340-476b-b94a-4692e18034da admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:07.493968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:29:07.561144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4f880-d1e7-4051-8f2d-28e2f2e4f1a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:07.561783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4f880-d1e7-4051-8f2d-28e2f2e4f1a1 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:07.562130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4f880-d1e7-4051-8f2d-28e2f2e4f1a1 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:07.562399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4f880-d1e7-4051-8f2d-28e2f2e4f1a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:07.567215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fec4f880-d1e7-4051-8f2d-28e2f2e4f1a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:07.582941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4f880-d1e7-4051-8f2d-28e2f2e4f1a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:07.584483 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4f880-d1e7-4051-8f2d-28e2f2e4f1a1 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:07.603082 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4215/8517] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:29:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:29:08.698846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eced79-946b-4167-988b-273b28964e61 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:08.699416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eced79-946b-4167-988b-273b28964e61 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:08.699587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eced79-946b-4167-988b-273b28964e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:08.745179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a9eced79-946b-4167-988b-273b28964e61 None None] MFA Rules not processed for user `e30ec0d6de0e427e8dc76cd7b6e45339`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:29:08.746808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a9eced79-946b-4167-988b-273b28964e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:08.830662 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4303/8518] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:29:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:29:08.844021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4f880-d1e7-4051-8f2d-28e2f2e4f1a1 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:08.845630 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fec4f880-d1e7-4051-8f2d-28e2f2e4f1a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:08.926752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4f880-d1e7-4051-8f2d-28e2f2e4f1a1 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:08.932462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99810418-d7bf-4677-8bdd-80688c3283e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:08.933075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99810418-d7bf-4677-8bdd-80688c3283e1 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:08.933347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99810418-d7bf-4677-8bdd-80688c3283e1 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:08.933615 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99810418-d7bf-4677-8bdd-80688c3283e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:08.943431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99810418-d7bf-4677-8bdd-80688c3283e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:08.945106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99810418-d7bf-4677-8bdd-80688c3283e1 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:08.955833 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4216/8519] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:29:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:29:09.182682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eced79-946b-4167-988b-273b28964e61 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:09.184251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a9eced79-946b-4167-988b-273b28964e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:09.202767 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eced79-946b-4167-988b-273b28964e61 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:09.209737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00bcd8-2265-4907-a70f-1d4b84bd952b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:09.210365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00bcd8-2265-4907-a70f-1d4b84bd952b service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:09.210632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00bcd8-2265-4907-a70f-1d4b84bd952b service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:09.210906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00bcd8-2265-4907-a70f-1d4b84bd952b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:09.212763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5a00bcd8-2265-4907-a70f-1d4b84bd952b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:09.228686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a00bcd8-2265-4907-a70f-1d4b84bd952b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:09.230455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a00bcd8-2265-4907-a70f-1d4b84bd952b service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:09.245886 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4304/8520] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:29:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:29:13.287922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99810418-d7bf-4677-8bdd-80688c3283e1 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:13.316257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99810418-d7bf-4677-8bdd-80688c3283e1 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:13.322702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1056c79-9593-48ef-bd6a-e71199ee9c93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:13.323452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1056c79-9593-48ef-bd6a-e71199ee9c93 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:13.323777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1056c79-9593-48ef-bd6a-e71199ee9c93 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:13.324105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1056c79-9593-48ef-bd6a-e71199ee9c93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:13.325963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c1056c79-9593-48ef-bd6a-e71199ee9c93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:13.341056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1056c79-9593-48ef-bd6a-e71199ee9c93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:13.342851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1056c79-9593-48ef-bd6a-e71199ee9c93 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:13.358492 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4217/8521] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:29:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:29:17.055920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00bcd8-2265-4907-a70f-1d4b84bd952b service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:17.070724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:29:17.124845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00bcd8-2265-4907-a70f-1d4b84bd952b service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:17.129606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:17.130236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:17.130509 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:17.130777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] PATH_INFO: `/v3/users/678a391f226b4d42820eaf1bb0ff78d5` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:17.131938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=678a391f226b4d42820eaf1bb0ff78d5)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:17.133944 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:17.187447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '678a391f226b4d42820eaf1bb0ff78d5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.197097 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '678a391f226b4d42820eaf1bb0ff78d5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.203841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '678a391f226b4d42820eaf1bb0ff78d5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.216847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '678a391f226b4d42820eaf1bb0ff78d5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.217620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '678a391f226b4d42820eaf1bb0ff78d5'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.228969 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4305/8522] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:29:17 2022] DELETE /identity/v3/users/678a391f226b4d42820eaf1bb0ff78d5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:29:17.416966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1056c79-9593-48ef-bd6a-e71199ee9c93 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:17.428022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1056c79-9593-48ef-bd6a-e71199ee9c93 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:17.490503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:17.491163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:17.491440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:17.491671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] PATH_INFO: `/v3/projects/3d968d1be5234a92a89856c0b73b8367` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:17.492840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d968d1be5234a92a89856c0b73b8367)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:17.494758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:17.531849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:29:17.544874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d968d1be5234a92a89856c0b73b8367', 'request_id': 'req-82061c0a-ba43-408c-96e9-3a335901d259'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.551441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d968d1be5234a92a89856c0b73b8367', 'request_id': 'req-82061c0a-ba43-408c-96e9-3a335901d259'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.560118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d968d1be5234a92a89856c0b73b8367', 'request_id': 'req-82061c0a-ba43-408c-96e9-3a335901d259'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.572061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] The token cache is being invalidate because project 3d968d1be5234a92a89856c0b73b8367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:29:17.572339 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.572994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.574491 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4218/8523] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:29:17 2022] DELETE /identity/v3/projects/3d968d1be5234a92a89856c0b73b8367 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:29:17.578672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:17.580120 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:17.594240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fa89e-e03e-4ec0-bdf3-ce967652c3c4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:17.653497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:17.654115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:17.654391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:17.654650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] PATH_INFO: `/v3/users/e30ec0d6de0e427e8dc76cd7b6e45339` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:17.655806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e30ec0d6de0e427e8dc76cd7b6e45339)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:17.657748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:17.694898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:29:17.707420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e30ec0d6de0e427e8dc76cd7b6e45339'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.715674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e30ec0d6de0e427e8dc76cd7b6e45339'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.723127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e30ec0d6de0e427e8dc76cd7b6e45339'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.734711 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e30ec0d6de0e427e8dc76cd7b6e45339'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.735340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e30ec0d6de0e427e8dc76cd7b6e45339'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:17.741963 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4306/8524] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:29:17 2022] DELETE /identity/v3/users/e30ec0d6de0e427e8dc76cd7b6e45339 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:29:17.926160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:17.927888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:17.955589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82061c0a-ba43-408c-96e9-3a335901d259 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:18.017218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:18.018005 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:18.018311 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:18.018575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] PATH_INFO: `/v3/projects/6e1121a798be4fa08ea4b6f90bb16d95` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:18.019703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e1121a798be4fa08ea4b6f90bb16d95)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:18.021800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:18.070650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e1121a798be4fa08ea4b6f90bb16d95', 'request_id': 'req-d8016163-e360-47ba-b25b-81f82784627e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:18.077401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e1121a798be4fa08ea4b6f90bb16d95', 'request_id': 'req-d8016163-e360-47ba-b25b-81f82784627e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:18.087089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e1121a798be4fa08ea4b6f90bb16d95', 'request_id': 'req-d8016163-e360-47ba-b25b-81f82784627e'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:18.098434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] The token cache is being invalidate because project 6e1121a798be4fa08ea4b6f90bb16d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:29:18.098721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:18.099416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:18.101019 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4219/8525] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:29:17 2022] DELETE /identity/v3/projects/6e1121a798be4fa08ea4b6f90bb16d95 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:29:24.297423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:24.299690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:24.417639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a6636-48a5-4bbf-98fb-c27d6fef0c1d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:24.495485 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200b623-35a3-4df1-b983-411a0378703a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:24.496007 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b623-35a3-4df1-b983-411a0378703a service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:24.496277 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b623-35a3-4df1-b983-411a0378703a service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:24.496574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b623-35a3-4df1-b983-411a0378703a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:24.498712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e200b623-35a3-4df1-b983-411a0378703a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:24.519858 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:29:24.580593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200b623-35a3-4df1-b983-411a0378703a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:24.582222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200b623-35a3-4df1-b983-411a0378703a service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:24.602422 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4307/8526] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:29:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:29:28.095146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=67,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:29:29.081690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:29.083299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:29.148535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8016163-e360-47ba-b25b-81f82784627e admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:29.152878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec4c89-bf06-40f9-8daa-d5fe9bcf9430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:29.153512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec4c89-bf06-40f9-8daa-d5fe9bcf9430 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:29.153770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec4c89-bf06-40f9-8daa-d5fe9bcf9430 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:29.154028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec4c89-bf06-40f9-8daa-d5fe9bcf9430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:29.155925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bdec4c89-bf06-40f9-8daa-d5fe9bcf9430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:29.230557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec4c89-bf06-40f9-8daa-d5fe9bcf9430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:29.232172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec4c89-bf06-40f9-8daa-d5fe9bcf9430 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:29.243988 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4220/8527] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:29:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:29:34.599164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=22 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:29:35.948337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200b623-35a3-4df1-b983-411a0378703a service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:35.949967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e200b623-35a3-4df1-b983-411a0378703a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:35.964967 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200b623-35a3-4df1-b983-411a0378703a service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:35.969875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:35.970554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:35.970828 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:35.971241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] PATH_INFO: `/v3/users/15c7508ede01463b9494100198a8d3a0` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:35.972505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15c7508ede01463b9494100198a8d3a0)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:35.974454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:36.028304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15c7508ede01463b9494100198a8d3a0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.037677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15c7508ede01463b9494100198a8d3a0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.045665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15c7508ede01463b9494100198a8d3a0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.063953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15c7508ede01463b9494100198a8d3a0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.064783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15c7508ede01463b9494100198a8d3a0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.073682 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4308/8528] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:29:35 2022] DELETE /identity/v3/users/15c7508ede01463b9494100198a8d3a0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:29:36.269675 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec4c89-bf06-40f9-8daa-d5fe9bcf9430 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:36.282556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec4c89-bf06-40f9-8daa-d5fe9bcf9430 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:36.347310 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:36.347930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:36.348197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:36.348489 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] PATH_INFO: `/v3/projects/5e7dba88703b48f38215fde76a0fb80a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:36.349620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e7dba88703b48f38215fde76a0fb80a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:36.351828 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:36.401473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:29:36.402849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e7dba88703b48f38215fde76a0fb80a', 'request_id': 'req-f5e1b64f-053c-40c5-8225-44a4e58a270d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.409540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e7dba88703b48f38215fde76a0fb80a', 'request_id': 'req-f5e1b64f-053c-40c5-8225-44a4e58a270d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.419372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e7dba88703b48f38215fde76a0fb80a', 'request_id': 'req-f5e1b64f-053c-40c5-8225-44a4e58a270d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.430611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] The token cache is being invalidate because project 5e7dba88703b48f38215fde76a0fb80a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:29:36.430896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.431701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.433449 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4221/8529] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:29:36 2022] DELETE /identity/v3/projects/5e7dba88703b48f38215fde76a0fb80a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:29:36.437704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:36.439547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:36.454975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c985890-8a03-4e08-ab7f-da01c8b90af1 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:36.516477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:36.516964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:36.517230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:36.517488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] PATH_INFO: `/v3/users/b9a588f471204b44aa11ea64a50aaa39` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:36.518684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a588f471204b44aa11ea64a50aaa39)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:36.520648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:36.570058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a588f471204b44aa11ea64a50aaa39'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.578951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a588f471204b44aa11ea64a50aaa39'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.586665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a588f471204b44aa11ea64a50aaa39'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.594383 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:29:36.599388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a588f471204b44aa11ea64a50aaa39'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.600033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a588f471204b44aa11ea64a50aaa39'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.606661 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4309/8530] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:29:36 2022] DELETE /identity/v3/users/b9a588f471204b44aa11ea64a50aaa39 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:29:36.799482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:29:36.801247 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:29:36.815801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1b64f-053c-40c5-8225-44a4e58a270d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:29:36.880522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:29:36.881674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:29:36.881674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:29:36.881933 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] PATH_INFO: `/v3/projects/3c8dee3911674124a15a41dfe181dca1` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:29:36.883082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c8dee3911674124a15a41dfe181dca1)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:29:36.885135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:29:36.934139 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c8dee3911674124a15a41dfe181dca1', 'request_id': 'req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.941043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c8dee3911674124a15a41dfe181dca1', 'request_id': 'req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.951475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c8dee3911674124a15a41dfe181dca1', 'request_id': 'req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.962083 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] The token cache is being invalidate because project 3c8dee3911674124a15a41dfe181dca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:29:36.962361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.962970 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:29:36.964475 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4222/8531] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:29:36 2022] DELETE /identity/v3/projects/3c8dee3911674124a15a41dfe181dca1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:29:46.606046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:29:46.959480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=58,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:30:07.811475 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:30:07.813077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:30:07.912345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f4a3a-3bd5-4212-a90f-9ac19596ebca admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:30:07.962420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083590b6-4af7-451c-9342-633e889f8b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:30:07.963023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083590b6-4af7-451c-9342-633e889f8b04 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:30:07.963289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083590b6-4af7-451c-9342-633e889f8b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:30:07.963551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083590b6-4af7-451c-9342-633e889f8b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:30:07.965422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-083590b6-4af7-451c-9342-633e889f8b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:30:08.040437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083590b6-4af7-451c-9342-633e889f8b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:30:08.041845 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083590b6-4af7-451c-9342-633e889f8b04 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:30:08.055709 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4310/8532] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:30:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:30:08.247808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:30:08.249502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:30:08.264591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a08f2-e2ab-43da-9cb8-c6ddfadd567a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:30:08.268143 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec366d93-53a6-41e8-9902-70166e4965d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:30:08.268773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec366d93-53a6-41e8-9902-70166e4965d8 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:30:08.269057 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec366d93-53a6-41e8-9902-70166e4965d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:30:08.269326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec366d93-53a6-41e8-9902-70166e4965d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:30:08.271000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ec366d93-53a6-41e8-9902-70166e4965d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:30:08.287390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec366d93-53a6-41e8-9902-70166e4965d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:30:08.288875 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec366d93-53a6-41e8-9902-70166e4965d8 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:30:08.302941 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4223/8533] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:30:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:30:08.808015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083590b6-4af7-451c-9342-633e889f8b04 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:30:08.827350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083590b6-4af7-451c-9342-633e889f8b04 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:30:08.832406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec9ec8-f957-4011-be12-3ce8a12abc7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:30:08.833031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec9ec8-f957-4011-be12-3ce8a12abc7b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:30:08.833303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec9ec8-f957-4011-be12-3ce8a12abc7b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:30:08.833566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec9ec8-f957-4011-be12-3ce8a12abc7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:30:08.835847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b1ec9ec8-f957-4011-be12-3ce8a12abc7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:30:08.850800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec9ec8-f957-4011-be12-3ce8a12abc7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:30:08.852834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec9ec8-f957-4011-be12-3ce8a12abc7b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:30:08.864520 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4311/8534] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:30:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:30:09.160082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec366d93-53a6-41e8-9902-70166e4965d8 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:30:09.161761 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ec366d93-53a6-41e8-9902-70166e4965d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:30:09.238532 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec366d93-53a6-41e8-9902-70166e4965d8 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:30:09.244228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa67d35-17a3-44a2-a89b-2455be6ceb3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:30:09.244865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa67d35-17a3-44a2-a89b-2455be6ceb3a service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:30:09.245130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa67d35-17a3-44a2-a89b-2455be6ceb3a service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:30:09.245395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa67d35-17a3-44a2-a89b-2455be6ceb3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:30:09.247157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cfa67d35-17a3-44a2-a89b-2455be6ceb3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:30:09.261228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa67d35-17a3-44a2-a89b-2455be6ceb3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:30:09.262802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa67d35-17a3-44a2-a89b-2455be6ceb3a service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:30:09.275112 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4224/8535] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:30:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:30:18.857960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:30:19.268633 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=38 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:31:07.100403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec9ec8-f957-4011-be12-3ce8a12abc7b service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:31:07.102664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b1ec9ec8-f957-4011-be12-3ce8a12abc7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:31:07.123452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec9ec8-f957-4011-be12-3ce8a12abc7b service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:31:07.129482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4c084-b6c0-4897-90d3-61931d0ea22e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:31:07.130082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4c084-b6c0-4897-90d3-61931d0ea22e service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:31:07.130356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4c084-b6c0-4897-90d3-61931d0ea22e service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:31:07.130623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4c084-b6c0-4897-90d3-61931d0ea22e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:31:07.132461 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b3b4c084-b6c0-4897-90d3-61931d0ea22e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:31:07.191286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4c084-b6c0-4897-90d3-61931d0ea22e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:31:07.192916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4c084-b6c0-4897-90d3-61931d0ea22e service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:31:07.202892 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4312/8536] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:31:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:31:16.941984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa67d35-17a3-44a2-a89b-2455be6ceb3a service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:31:16.943666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cfa67d35-17a3-44a2-a89b-2455be6ceb3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:31:16.959965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa67d35-17a3-44a2-a89b-2455be6ceb3a service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:31:16.964863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f886200-6dd5-42a4-8260-4270bf5bfd39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:31:16.965486 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f886200-6dd5-42a4-8260-4270bf5bfd39 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:31:16.965758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f886200-6dd5-42a4-8260-4270bf5bfd39 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:31:16.966014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f886200-6dd5-42a4-8260-4270bf5bfd39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:31:16.968130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1f886200-6dd5-42a4-8260-4270bf5bfd39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:31:16.982286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f886200-6dd5-42a4-8260-4270bf5bfd39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:31:16.983716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f886200-6dd5-42a4-8260-4270bf5bfd39 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:31:16.995117 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4225/8537] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:31:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:31:17.198102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=28 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:31:23.501536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4c084-b6c0-4897-90d3-61931d0ea22e service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:31:23.504314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b3b4c084-b6c0-4897-90d3-61931d0ea22e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:31:23.525577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4c084-b6c0-4897-90d3-61931d0ea22e service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:31:23.533015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd086763-e996-4e46-b809-24543ad11dbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:31:23.533622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd086763-e996-4e46-b809-24543ad11dbf service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:31:23.533891 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd086763-e996-4e46-b809-24543ad11dbf service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:31:23.534155 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd086763-e996-4e46-b809-24543ad11dbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:31:23.536221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bd086763-e996-4e46-b809-24543ad11dbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:31:23.601196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd086763-e996-4e46-b809-24543ad11dbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:31:23.602802 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd086763-e996-4e46-b809-24543ad11dbf service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:31:23.621832 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4313/8538] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:31:23 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:31:26.988841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:31:33.619241 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=34 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:31:36.790944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78481c47-867b-4463-93df-3b460425e884 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:31:36.792052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78481c47-867b-4463-93df-3b460425e884 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:31:36.792408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78481c47-867b-4463-93df-3b460425e884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:31:36.817243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-78481c47-867b-4463-93df-3b460425e884 None None] MFA Rules not processed for user `f2c2c4da3c954f96af706644f238ed97`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:31:36.818888 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-78481c47-867b-4463-93df-3b460425e884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:31:36.829866 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4226/8539] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:31:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:31:39.516214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd086763-e996-4e46-b809-24543ad11dbf service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:31:39.529552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd086763-e996-4e46-b809-24543ad11dbf service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:31:39.533843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b69dc-e2e0-4951-a9c3-a5e9b7814615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:31:39.534746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b69dc-e2e0-4951-a9c3-a5e9b7814615 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:31:39.535018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b69dc-e2e0-4951-a9c3-a5e9b7814615 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:31:39.535288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b69dc-e2e0-4951-a9c3-a5e9b7814615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:31:39.537116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7f9b69dc-e2e0-4951-a9c3-a5e9b7814615 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:31:39.551235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b69dc-e2e0-4951-a9c3-a5e9b7814615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:31:39.552823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b69dc-e2e0-4951-a9c3-a5e9b7814615 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:31:39.567226 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4314/8540] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:31:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:31:46.026601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78481c47-867b-4463-93df-3b460425e884 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:31:46.044364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78481c47-867b-4463-93df-3b460425e884 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:31:46.050735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7ffe8-67ec-432e-948f-0ba11ba61558 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:31:46.051345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7ffe8-67ec-432e-948f-0ba11ba61558 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:31:46.051606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7ffe8-67ec-432e-948f-0ba11ba61558 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:31:46.051863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7ffe8-67ec-432e-948f-0ba11ba61558 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:31:46.053733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c5f7ffe8-67ec-432e-948f-0ba11ba61558 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:31:46.070174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7ffe8-67ec-432e-948f-0ba11ba61558 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:31:46.072395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7ffe8-67ec-432e-948f-0ba11ba61558 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:31:46.086178 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4227/8541] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:31:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:31:49.558087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:31:56.078780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=9 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:32:25.717725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b69dc-e2e0-4951-a9c3-a5e9b7814615 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:32:25.732784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b69dc-e2e0-4951-a9c3-a5e9b7814615 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:32:25.737119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cfc90-6682-472a-9e89-d38e218f0a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:32:25.737741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cfc90-6682-472a-9e89-d38e218f0a7c service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:32:25.738016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cfc90-6682-472a-9e89-d38e218f0a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:32:25.738312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cfc90-6682-472a-9e89-d38e218f0a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:32:25.740170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-139cfc90-6682-472a-9e89-d38e218f0a7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:32:25.752532 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cfc90-6682-472a-9e89-d38e218f0a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:32:25.754186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cfc90-6682-472a-9e89-d38e218f0a7c service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:32:25.765047 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4315/8542] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:32:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:32:25.853415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7ffe8-67ec-432e-948f-0ba11ba61558 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:32:25.866170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7ffe8-67ec-432e-948f-0ba11ba61558 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:32:25.870732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf48734-83a7-458f-aad8-056f03e36195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:32:25.871328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf48734-83a7-458f-aad8-056f03e36195 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:32:25.871599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf48734-83a7-458f-aad8-056f03e36195 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:32:25.871865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf48734-83a7-458f-aad8-056f03e36195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:32:25.873701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-edf48734-83a7-458f-aad8-056f03e36195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:32:25.889077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf48734-83a7-458f-aad8-056f03e36195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:32:25.890688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf48734-83a7-458f-aad8-056f03e36195 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:32:25.904832 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4228/8543] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:32:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:32:35.759135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:32:35.899331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:32:40.825709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cfc90-6682-472a-9e89-d38e218f0a7c service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:32:40.844282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cfc90-6682-472a-9e89-d38e218f0a7c service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:32:40.850784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101e461-9df5-4dbe-ae8b-4f9b96c1a10c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:32:40.851696 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101e461-9df5-4dbe-ae8b-4f9b96c1a10c service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:32:40.852068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101e461-9df5-4dbe-ae8b-4f9b96c1a10c service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:32:40.852487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101e461-9df5-4dbe-ae8b-4f9b96c1a10c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:32:40.854798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0101e461-9df5-4dbe-ae8b-4f9b96c1a10c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:32:40.871637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101e461-9df5-4dbe-ae8b-4f9b96c1a10c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:32:40.873511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101e461-9df5-4dbe-ae8b-4f9b96c1a10c service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:32:40.887519 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4316/8544] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:32:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:32:50.880262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:32:52.480331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766122b6-c529-4929-97d2-cc2b7d11f269 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:32:52.480331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766122b6-c529-4929-97d2-cc2b7d11f269 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:32:52.481091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766122b6-c529-4929-97d2-cc2b7d11f269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:32:52.503869 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-766122b6-c529-4929-97d2-cc2b7d11f269 None None] MFA Rules not processed for user `f2c2c4da3c954f96af706644f238ed97`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:32:52.505405 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-766122b6-c529-4929-97d2-cc2b7d11f269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:32:52.516818 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4229/8545] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:32:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:32:53.242817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101e461-9df5-4dbe-ae8b-4f9b96c1a10c service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:32:53.244413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0101e461-9df5-4dbe-ae8b-4f9b96c1a10c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:32:53.261685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101e461-9df5-4dbe-ae8b-4f9b96c1a10c service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:32:53.268517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db8f0a-fb78-45d1-bd19-130354707f7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:32:53.269405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db8f0a-fb78-45d1-bd19-130354707f7b service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:32:53.269682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db8f0a-fb78-45d1-bd19-130354707f7b service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:32:53.269930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db8f0a-fb78-45d1-bd19-130354707f7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:32:53.271977 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c5db8f0a-fb78-45d1-bd19-130354707f7b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:32:53.287703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db8f0a-fb78-45d1-bd19-130354707f7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:32:53.289335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db8f0a-fb78-45d1-bd19-130354707f7b service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:32:53.302748 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4317/8546] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:32:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:33:02.498790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:33:03.296196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:33:20.954314 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766122b6-c529-4929-97d2-cc2b7d11f269 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:33:20.970236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766122b6-c529-4929-97d2-cc2b7d11f269 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:33:20.976808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166339fd-75ef-47dd-839e-831e28088ecb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:33:20.977409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166339fd-75ef-47dd-839e-831e28088ecb service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:33:20.977652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166339fd-75ef-47dd-839e-831e28088ecb service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:33:20.977942 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166339fd-75ef-47dd-839e-831e28088ecb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:33:20.979915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-166339fd-75ef-47dd-839e-831e28088ecb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:33:21.009358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166339fd-75ef-47dd-839e-831e28088ecb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:33:21.010928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166339fd-75ef-47dd-839e-831e28088ecb service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:33:21.023423 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4230/8547] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:33:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:33:31.017007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=11 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:34:13.151426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db8f0a-fb78-45d1-bd19-130354707f7b service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:13.165886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db8f0a-fb78-45d1-bd19-130354707f7b service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:13.172707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d8224-2e92-4dfc-918d-6846a537951d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:13.173304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d8224-2e92-4dfc-918d-6846a537951d service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:13.173570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d8224-2e92-4dfc-918d-6846a537951d service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:13.173831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d8224-2e92-4dfc-918d-6846a537951d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:13.175632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d97d8224-2e92-4dfc-918d-6846a537951d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:13.191309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d8224-2e92-4dfc-918d-6846a537951d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:13.193261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d8224-2e92-4dfc-918d-6846a537951d service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:13.206605 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4318/8548] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:34:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:34:23.199835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:34:38.681117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166339fd-75ef-47dd-839e-831e28088ecb service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:38.706954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166339fd-75ef-47dd-839e-831e28088ecb service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:38.713764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e639716-87d6-44f5-b103-f8bfbfa0b62d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:38.714497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e639716-87d6-44f5-b103-f8bfbfa0b62d service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:38.715190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e639716-87d6-44f5-b103-f8bfbfa0b62d service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:38.715548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e639716-87d6-44f5-b103-f8bfbfa0b62d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:38.717552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7e639716-87d6-44f5-b103-f8bfbfa0b62d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:38.749366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e639716-87d6-44f5-b103-f8bfbfa0b62d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:38.751110 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e639716-87d6-44f5-b103-f8bfbfa0b62d service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:38.767381 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4231/8549] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:34:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:34:45.185507 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d8224-2e92-4dfc-918d-6846a537951d service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:45.201376 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d8224-2e92-4dfc-918d-6846a537951d service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:45.208117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85800b-eb1f-43d3-8bb4-62cd9e8e2d93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:45.208770 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85800b-eb1f-43d3-8bb4-62cd9e8e2d93 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:45.209035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85800b-eb1f-43d3-8bb4-62cd9e8e2d93 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:45.209300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85800b-eb1f-43d3-8bb4-62cd9e8e2d93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:45.211385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9e85800b-eb1f-43d3-8bb4-62cd9e8e2d93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:45.227042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85800b-eb1f-43d3-8bb4-62cd9e8e2d93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:45.229211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85800b-eb1f-43d3-8bb4-62cd9e8e2d93 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:45.247377 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4319/8550] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:34:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:34:45.474327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e639716-87d6-44f5-b103-f8bfbfa0b62d service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:45.475859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7e639716-87d6-44f5-b103-f8bfbfa0b62d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:45.567860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e639716-87d6-44f5-b103-f8bfbfa0b62d service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:45.573535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:45.574254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:45.574575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:45.574832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] PATH_INFO: `/v3/users/f2c2c4da3c954f96af706644f238ed97` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:45.576121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2c2c4da3c954f96af706644f238ed97)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:45.578387 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:45.627191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2c2c4da3c954f96af706644f238ed97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:45.635028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2c2c4da3c954f96af706644f238ed97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:45.646813 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2c2c4da3c954f96af706644f238ed97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:45.647437 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2c2c4da3c954f96af706644f238ed97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:45.648032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2c2c4da3c954f96af706644f238ed97'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:45.659920 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4232/8551] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:34:45 2022] DELETE /identity/v3/users/f2c2c4da3c954f96af706644f238ed97 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:34:45.664881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedd9cb-dc7a-4b36-a7f4-8d5ca0f7bc2f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:45.665135 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedd9cb-dc7a-4b36-a7f4-8d5ca0f7bc2f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:45.665406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedd9cb-dc7a-4b36-a7f4-8d5ca0f7bc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:45.695031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-bcedd9cb-dc7a-4b36-a7f4-8d5ca0f7bc2f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:34:45.696268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bcedd9cb-dc7a-4b36-a7f4-8d5ca0f7bc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:45.759113 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4320/8552] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:34:45.771708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:45.773280 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:45.818056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:34:45.854800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a1499-c6a6-4c3a-84b5-cd80edcbb269 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:45.859065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d2d54-58a3-46ed-932c-4b901471c1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:45.859723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d2d54-58a3-46ed-932c-4b901471c1ee service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:45.859991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d2d54-58a3-46ed-932c-4b901471c1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:45.860249 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d2d54-58a3-46ed-932c-4b901471c1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:45.871105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d2d54-58a3-46ed-932c-4b901471c1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:45.872617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d2d54-58a3-46ed-932c-4b901471c1ee service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:45.882739 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4233/8553] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:34:46.061246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedd9cb-dc7a-4b36-a7f4-8d5ca0f7bc2f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:46.062821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-bcedd9cb-dc7a-4b36-a7f4-8d5ca0f7bc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:46.079834 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedd9cb-dc7a-4b36-a7f4-8d5ca0f7bc2f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:46.087400 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:46.088222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:46.088723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:46.088851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] PATH_INFO: `/v3/projects/fa42503d942c47bdb3f3366e3fea0241` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:46.092361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa42503d942c47bdb3f3366e3fea0241)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:46.094243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:46.143820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa42503d942c47bdb3f3366e3fea0241', 'request_id': 'req-3515c571-355c-4fe1-a785-b1e80b2e83bf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.153129 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa42503d942c47bdb3f3366e3fea0241', 'request_id': 'req-3515c571-355c-4fe1-a785-b1e80b2e83bf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.158921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa42503d942c47bdb3f3366e3fea0241', 'request_id': 'req-3515c571-355c-4fe1-a785-b1e80b2e83bf'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.170361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] The token cache is being invalidate because project fa42503d942c47bdb3f3366e3fea0241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:34:46.170645 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.171296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.172919 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4321/8554] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:34:46 2022] DELETE /identity/v3/projects/fa42503d942c47bdb3f3366e3fea0241 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:34:46.177268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d2d54-58a3-46ed-932c-4b901471c1ee service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:46.187810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d2d54-58a3-46ed-932c-4b901471c1ee service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:46.252953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb79f3-1c17-4939-b449-aa3e8905fbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:46.253589 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb79f3-1c17-4939-b449-aa3e8905fbf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:46.253864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb79f3-1c17-4939-b449-aa3e8905fbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:46.254122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb79f3-1c17-4939-b449-aa3e8905fbf0 admin admin] PATH_INFO: `/v3/users/95695afba1af4de1ba73879c85f41d3b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:46.255372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb79f3-1c17-4939-b449-aa3e8905fbf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95695afba1af4de1ba73879c85f41d3b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:46.257352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb79f3-1c17-4939-b449-aa3e8905fbf0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:46.304871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a2bb79f3-1c17-4939-b449-aa3e8905fbf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95695afba1af4de1ba73879c85f41d3b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.314899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a2bb79f3-1c17-4939-b449-aa3e8905fbf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95695afba1af4de1ba73879c85f41d3b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.329087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a2bb79f3-1c17-4939-b449-aa3e8905fbf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95695afba1af4de1ba73879c85f41d3b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.330147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a2bb79f3-1c17-4939-b449-aa3e8905fbf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95695afba1af4de1ba73879c85f41d3b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.330897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a2bb79f3-1c17-4939-b449-aa3e8905fbf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95695afba1af4de1ba73879c85f41d3b'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.348665 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4234/8555] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:34:46 2022] DELETE /identity/v3/users/95695afba1af4de1ba73879c85f41d3b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:34:46.547976 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:46.549575 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:46.565114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515c571-355c-4fe1-a785-b1e80b2e83bf admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:46.630144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27a01a-88ee-441c-a880-babd5dfdd1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:46.630781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27a01a-88ee-441c-a880-babd5dfdd1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:46.631050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27a01a-88ee-441c-a880-babd5dfdd1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:46.631314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27a01a-88ee-441c-a880-babd5dfdd1b0 admin admin] PATH_INFO: `/v3/projects/0f72fd57efba4913982a47f8fa34871f` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:46.632449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27a01a-88ee-441c-a880-babd5dfdd1b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f72fd57efba4913982a47f8fa34871f)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:46.634358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27a01a-88ee-441c-a880-babd5dfdd1b0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:46.641320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:34:46.686518 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1c27a01a-88ee-441c-a880-babd5dfdd1b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f72fd57efba4913982a47f8fa34871f', 'request_id': 'req-1c27a01a-88ee-441c-a880-babd5dfdd1b0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.694980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1c27a01a-88ee-441c-a880-babd5dfdd1b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f72fd57efba4913982a47f8fa34871f', 'request_id': 'req-1c27a01a-88ee-441c-a880-babd5dfdd1b0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.700657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1c27a01a-88ee-441c-a880-babd5dfdd1b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f72fd57efba4913982a47f8fa34871f', 'request_id': 'req-1c27a01a-88ee-441c-a880-babd5dfdd1b0'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.712268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1c27a01a-88ee-441c-a880-babd5dfdd1b0 admin admin] The token cache is being invalidate because project 0f72fd57efba4913982a47f8fa34871f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:34:46.712567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1c27a01a-88ee-441c-a880-babd5dfdd1b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.713284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-1c27a01a-88ee-441c-a880-babd5dfdd1b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:34:46.715070 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4322/8556] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:34:46 2022] DELETE /identity/v3/projects/0f72fd57efba4913982a47f8fa34871f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:34:56.342371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=66,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:34:56.708864 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:34:56.751690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8930dc-3383-45f4-a209-52c6f7625f12 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:56.751963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8930dc-3383-45f4-a209-52c6f7625f12 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:56.752315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8930dc-3383-45f4-a209-52c6f7625f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:56.807065 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cd8930dc-3383-45f4-a209-52c6f7625f12 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:34:56.808181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cd8930dc-3383-45f4-a209-52c6f7625f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:56.872203 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4235/8557] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:34:56.885986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c323ed6-7134-4674-a602-f4d3406c202d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:56.886279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c323ed6-7134-4674-a602-f4d3406c202d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:56.886694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c323ed6-7134-4674-a602-f4d3406c202d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:56.909894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5c323ed6-7134-4674-a602-f4d3406c202d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:34:56.911114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5c323ed6-7134-4674-a602-f4d3406c202d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:56.921229 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4323/8558] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:34:56.926665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8930dc-3383-45f4-a209-52c6f7625f12 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:56.939702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8930dc-3383-45f4-a209-52c6f7625f12 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:56.949276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2cf97-6d01-4ef0-a4a7-ab0c27483f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:56.950115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2cf97-6d01-4ef0-a4a7-ab0c27483f67 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:56.950389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2cf97-6d01-4ef0-a4a7-ab0c27483f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:56.950654 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2cf97-6d01-4ef0-a4a7-ab0c27483f67 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:56.951319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2cf97-6d01-4ef0-a4a7-ab0c27483f67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:34:56.951619 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2cf97-6d01-4ef0-a4a7-ab0c27483f67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:56.953257 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2cf97-6d01-4ef0-a4a7-ab0c27483f67 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:56.965868 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4236/8559] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:34:56.974757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c323ed6-7134-4674-a602-f4d3406c202d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:56.984958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c323ed6-7134-4674-a602-f4d3406c202d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:56.990610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ae610-ab0b-4652-ad3b-29326a8c8686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:56.991220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ae610-ab0b-4652-ad3b-29326a8c8686 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:56.991511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ae610-ab0b-4652-ad3b-29326a8c8686 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:56.991768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ae610-ab0b-4652-ad3b-29326a8c8686 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:56.992415 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ae610-ab0b-4652-ad3b-29326a8c8686 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:56.994305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ae610-ab0b-4652-ad3b-29326a8c8686 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:57.028589 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4324/8560] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:34:56 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:34:57.032885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2cf97-6d01-4ef0-a4a7-ab0c27483f67 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:57.043847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2cf97-6d01-4ef0-a4a7-ab0c27483f67 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:57.102932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e58b90-fec5-4fdc-94d4-efd0a6a6aebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:57.103443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e58b90-fec5-4fdc-94d4-efd0a6a6aebe admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:57.103714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e58b90-fec5-4fdc-94d4-efd0a6a6aebe admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:57.103977 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e58b90-fec5-4fdc-94d4-efd0a6a6aebe admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:57.104609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e58b90-fec5-4fdc-94d4-efd0a6a6aebe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:57.106538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e58b90-fec5-4fdc-94d4-efd0a6a6aebe admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:57.128699 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4237/8561] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:34:57 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:34:57.132953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ae610-ab0b-4652-ad3b-29326a8c8686 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:57.144222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ae610-ab0b-4652-ad3b-29326a8c8686 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:57.149053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d81224-e282-4dd9-836c-a54c869dfb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:57.149655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d81224-e282-4dd9-836c-a54c869dfb7e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:57.149922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d81224-e282-4dd9-836c-a54c869dfb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:57.150188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d81224-e282-4dd9-836c-a54c869dfb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:57.150824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d81224-e282-4dd9-836c-a54c869dfb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:57.152557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d81224-e282-4dd9-836c-a54c869dfb7e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:57.166577 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4325/8562] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:34:57.167996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e58b90-fec5-4fdc-94d4-efd0a6a6aebe admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:57.178689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e58b90-fec5-4fdc-94d4-efd0a6a6aebe admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:57.184295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dfd71-cab7-47f2-a53f-c24dd55223d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:57.184928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dfd71-cab7-47f2-a53f-c24dd55223d1 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:57.185201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dfd71-cab7-47f2-a53f-c24dd55223d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:57.185461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dfd71-cab7-47f2-a53f-c24dd55223d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:57.186101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dfd71-cab7-47f2-a53f-c24dd55223d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:57.187760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dfd71-cab7-47f2-a53f-c24dd55223d1 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:57.199025 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4238/8563] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:34:57.203450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d81224-e282-4dd9-836c-a54c869dfb7e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:57.213644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d81224-e282-4dd9-836c-a54c869dfb7e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:57.218496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf134a6-35c2-4df5-9c66-9110382ee851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:57.219140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf134a6-35c2-4df5-9c66-9110382ee851 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:57.219408 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf134a6-35c2-4df5-9c66-9110382ee851 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:57.219671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf134a6-35c2-4df5-9c66-9110382ee851 admin admin] PATH_INFO: `/v3/projects/ae29a682475b481db2ab78b6fb39a408/users/2d62c37225e84ce1be91b2c3dc325371/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:57.234994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf134a6-35c2-4df5-9c66-9110382ee851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae29a682475b481db2ab78b6fb39a408, user_id=2d62c37225e84ce1be91b2c3dc325371, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:57.237709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf134a6-35c2-4df5-9c66-9110382ee851 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:57.252224 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4326/8564] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:34:57 2022] PUT /identity/v3/projects/ae29a682475b481db2ab78b6fb39a408/users/2d62c37225e84ce1be91b2c3dc325371/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:34:57.266373 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94a427-86b1-4f5e-9f59-b87d08edf406 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:57.266658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94a427-86b1-4f5e-9f59-b87d08edf406 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:57.266899 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94a427-86b1-4f5e-9f59-b87d08edf406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:57.296479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-dc94a427-86b1-4f5e-9f59-b87d08edf406 None None] MFA Rules not processed for user `2d62c37225e84ce1be91b2c3dc325371`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:34:57.297697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dc94a427-86b1-4f5e-9f59-b87d08edf406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:57.331975 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:34:57.369682 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4239/8565] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:34:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:34:57.374895 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf134a6-35c2-4df5-9c66-9110382ee851 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:57.441737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf134a6-35c2-4df5-9c66-9110382ee851 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:57.447596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e2d7c-40d8-49a7-ab5e-6fd9afc52b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:57.448199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e2d7c-40d8-49a7-ab5e-6fd9afc52b81 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:57.448504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e2d7c-40d8-49a7-ab5e-6fd9afc52b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:57.448818 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e2d7c-40d8-49a7-ab5e-6fd9afc52b81 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:57.449491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e2d7c-40d8-49a7-ab5e-6fd9afc52b81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:57.451365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e2d7c-40d8-49a7-ab5e-6fd9afc52b81 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:57.483597 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4327/8566] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:34:57 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:34:57.487649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94a427-86b1-4f5e-9f59-b87d08edf406 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:57.498971 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94a427-86b1-4f5e-9f59-b87d08edf406 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:57.559552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e4004-572c-4c31-a901-a56ace35bd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:57.560149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4004-572c-4c31-a901-a56ace35bd79 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:57.560432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4004-572c-4c31-a901-a56ace35bd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:57.560697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4004-572c-4c31-a901-a56ace35bd79 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:57.561321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e4004-572c-4c31-a901-a56ace35bd79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:57.563220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e4004-572c-4c31-a901-a56ace35bd79 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:57.586340 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4240/8567] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:34:57 2022] POST /identity/v3/users => generated 302 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:34:57.590541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e2d7c-40d8-49a7-ab5e-6fd9afc52b81 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:57.602011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e2d7c-40d8-49a7-ab5e-6fd9afc52b81 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:57.607321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4abecb-ec13-4d8b-84ec-f02c88c2f51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:57.607912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4abecb-ec13-4d8b-84ec-f02c88c2f51b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:57.608167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4abecb-ec13-4d8b-84ec-f02c88c2f51b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:57.608473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4abecb-ec13-4d8b-84ec-f02c88c2f51b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:57.609104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4abecb-ec13-4d8b-84ec-f02c88c2f51b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:57.610710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4abecb-ec13-4d8b-84ec-f02c88c2f51b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:57.621638 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4328/8568] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:34:57.625939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e4004-572c-4c31-a901-a56ace35bd79 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:57.636949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e4004-572c-4c31-a901-a56ace35bd79 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:57.642666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b897d-b00b-429f-8eed-a32ef5720912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:57.643342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b897d-b00b-429f-8eed-a32ef5720912 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:57.643613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b897d-b00b-429f-8eed-a32ef5720912 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:57.643878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b897d-b00b-429f-8eed-a32ef5720912 admin admin] PATH_INFO: `/v3/projects/2fd01fc4762f4b438241355d74aa1ea3/users/763034fb8c354419bf592bbe5954e5ed/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:57.659840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b897d-b00b-429f-8eed-a32ef5720912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd01fc4762f4b438241355d74aa1ea3, user_id=763034fb8c354419bf592bbe5954e5ed, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:57.662577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b897d-b00b-429f-8eed-a32ef5720912 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:57.677437 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4241/8569] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:34:57 2022] PUT /identity/v3/projects/2fd01fc4762f4b438241355d74aa1ea3/users/763034fb8c354419bf592bbe5954e5ed/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:34:57.681384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4abecb-ec13-4d8b-84ec-f02c88c2f51b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:57.691942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4abecb-ec13-4d8b-84ec-f02c88c2f51b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:57.751463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56117b64-6e49-4d9e-884b-8bc217fd2d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:57.752198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56117b64-6e49-4d9e-884b-8bc217fd2d91 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:57.752321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56117b64-6e49-4d9e-884b-8bc217fd2d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:57.752602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56117b64-6e49-4d9e-884b-8bc217fd2d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:57.753230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56117b64-6e49-4d9e-884b-8bc217fd2d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:57.754821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56117b64-6e49-4d9e-884b-8bc217fd2d91 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:57.767100 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4329/8570] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:34:57.771629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b897d-b00b-429f-8eed-a32ef5720912 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:57.783299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b897d-b00b-429f-8eed-a32ef5720912 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:57.788864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0f373-c388-443b-93df-ab1df919d5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:57.789514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0f373-c388-443b-93df-ab1df919d5f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:57.789786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0f373-c388-443b-93df-ab1df919d5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:57.790056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0f373-c388-443b-93df-ab1df919d5f0 admin admin] PATH_INFO: `/v3/projects/2fd01fc4762f4b438241355d74aa1ea3/users/763034fb8c354419bf592bbe5954e5ed/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:57.792134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0f373-c388-443b-93df-ab1df919d5f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd01fc4762f4b438241355d74aa1ea3, user_id=763034fb8c354419bf592bbe5954e5ed, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:57.795306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0f373-c388-443b-93df-ab1df919d5f0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:57.805793 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4242/8571] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:34:57 2022] PUT /identity/v3/projects/2fd01fc4762f4b438241355d74aa1ea3/users/763034fb8c354419bf592bbe5954e5ed/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:34:57.816799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dec639-2648-464b-9258-b0d7f69eba91 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:57.817078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dec639-2648-464b-9258-b0d7f69eba91 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:57.817341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dec639-2648-464b-9258-b0d7f69eba91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:57.832344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:34:57.849578 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-c9dec639-2648-464b-9258-b0d7f69eba91 None None] MFA Rules not processed for user `763034fb8c354419bf592bbe5954e5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:34:57.851019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c9dec639-2648-464b-9258-b0d7f69eba91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:57.938858 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4330/8572] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:34:57 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:34:57.951417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c551d67-7542-48ea-bd45-397ea9969844 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:57.951815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c551d67-7542-48ea-bd45-397ea9969844 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:57.952294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c551d67-7542-48ea-bd45-397ea9969844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:57.976731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-7c551d67-7542-48ea-bd45-397ea9969844 None None] MFA Rules not processed for user `2d62c37225e84ce1be91b2c3dc325371`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:34:57.977936 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7c551d67-7542-48ea-bd45-397ea9969844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:58.044559 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4243/8573] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:34:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:34:58.059730 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dec639-2648-464b-9258-b0d7f69eba91 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:34:58.061368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c9dec639-2648-464b-9258-b0d7f69eba91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:34:58.153961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dec639-2648-464b-9258-b0d7f69eba91 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:34:58.160203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd38be-69bf-46b2-a545-2ae7f6a7c2b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:34:58.160842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd38be-69bf-46b2-a545-2ae7f6a7c2b5 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:34:58.161115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd38be-69bf-46b2-a545-2ae7f6a7c2b5 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:34:58.161404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd38be-69bf-46b2-a545-2ae7f6a7c2b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:34:58.170117 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd38be-69bf-46b2-a545-2ae7f6a7c2b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:34:58.171712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd38be-69bf-46b2-a545-2ae7f6a7c2b5 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:34:58.181116 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4331/8574] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:34:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:35:08.035281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:35:08.043072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c551d67-7542-48ea-bd45-397ea9969844 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:08.045412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7c551d67-7542-48ea-bd45-397ea9969844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:08.106772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c551d67-7542-48ea-bd45-397ea9969844 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:08.110840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39450b7-c01e-47fe-afbd-07cde3ce5c0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:08.111469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39450b7-c01e-47fe-afbd-07cde3ce5c0d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:08.111737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39450b7-c01e-47fe-afbd-07cde3ce5c0d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:08.111999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39450b7-c01e-47fe-afbd-07cde3ce5c0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:08.113857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f39450b7-c01e-47fe-afbd-07cde3ce5c0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:08.129913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39450b7-c01e-47fe-afbd-07cde3ce5c0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:08.131331 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39450b7-c01e-47fe-afbd-07cde3ce5c0d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:08.149020 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4244/8575] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:35:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:35:08.176994 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=65 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:35:09.355955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd38be-69bf-46b2-a545-2ae7f6a7c2b5 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:09.373029 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd38be-69bf-46b2-a545-2ae7f6a7c2b5 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:09.377972 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a529c-a5b4-4211-b5a2-e57c8452dd03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:09.378585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a529c-a5b4-4211-b5a2-e57c8452dd03 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:09.378855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a529c-a5b4-4211-b5a2-e57c8452dd03 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:09.379114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a529c-a5b4-4211-b5a2-e57c8452dd03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:09.381406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9f2a529c-a5b4-4211-b5a2-e57c8452dd03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:09.398670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a529c-a5b4-4211-b5a2-e57c8452dd03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:09.400102 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a529c-a5b4-4211-b5a2-e57c8452dd03 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:09.414049 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4332/8576] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:35:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:35:09.557966 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39450b7-c01e-47fe-afbd-07cde3ce5c0d service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:09.559547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f39450b7-c01e-47fe-afbd-07cde3ce5c0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:09.578658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39450b7-c01e-47fe-afbd-07cde3ce5c0d service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:09.586072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f1d37-a366-478f-a876-c10b44b962ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:09.586652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f1d37-a366-478f-a876-c10b44b962ee service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:09.586892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f1d37-a366-478f-a876-c10b44b962ee service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:09.587163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f1d37-a366-478f-a876-c10b44b962ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:09.596909 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f1d37-a366-478f-a876-c10b44b962ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:09.598816 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f1d37-a366-478f-a876-c10b44b962ee service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:09.611450 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4245/8577] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:35:09 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:35:09.634188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06e1bc-a329-4739-bd16-48ee2f7f62d8 None None] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:09.634471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06e1bc-a329-4739-bd16-48ee2f7f62d8 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:09.634732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06e1bc-a329-4739-bd16-48ee2f7f62d8 None None] PATH_INFO: `/` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:09.636082 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4333/8578] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:35:09.641940 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f1d37-a366-478f-a876-c10b44b962ee service nova] Authenticating service token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:35:09.663492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f1d37-a366-478f-a876-c10b44b962ee service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:09.674133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f1d37-a366-478f-a876-c10b44b962ee service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:09.679303 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca031fa-8cd0-4cd5-93e5-c6f50aa82289 tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '763034fb8c354419bf592bbe5954e5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd01fc4762f4b438241355d74aa1ea3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:09.679930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca031fa-8cd0-4cd5-93e5-c6f50aa82289 tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:09.680202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca031fa-8cd0-4cd5-93e5-c6f50aa82289 tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:09.680520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca031fa-8cd0-4cd5-93e5-c6f50aa82289 tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] PATH_INFO: `/v3/projects/2fd01fc4762f4b438241355d74aa1ea3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:09.681674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca031fa-8cd0-4cd5-93e5-c6f50aa82289 tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] RBAC: Authorizing `identity:get_project(project_id=2fd01fc4762f4b438241355d74aa1ea3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:09.683463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca031fa-8cd0-4cd5-93e5-c6f50aa82289 tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:09.685153 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4246/8579] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:35:09 2022] GET /identity/v3/projects/2fd01fc4762f4b438241355d74aa1ea3 => generated 391 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:35:09.849970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06e1bc-a329-4739-bd16-48ee2f7f62d8 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:09.873720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06e1bc-a329-4739-bd16-48ee2f7f62d8 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:09.877892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77c50c-7123-407b-a37d-84db5cdc6a0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:09.878478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77c50c-7123-407b-a37d-84db5cdc6a0f service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:09.878745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77c50c-7123-407b-a37d-84db5cdc6a0f service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:09.879003 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77c50c-7123-407b-a37d-84db5cdc6a0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:09.881038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9e77c50c-7123-407b-a37d-84db5cdc6a0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:09.893583 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77c50c-7123-407b-a37d-84db5cdc6a0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:09.894999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77c50c-7123-407b-a37d-84db5cdc6a0f service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:09.905705 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4334/8580] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:35:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:35:10.070701 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d265203-5dd2-4fcd-b8d0-1e49635c6dc5 None None] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:10.071041 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d265203-5dd2-4fcd-b8d0-1e49635c6dc5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:10.071392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d265203-5dd2-4fcd-b8d0-1e49635c6dc5 None None] PATH_INFO: `/` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:10.073577 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4247/8581] 10.222.0.23 () {54 vars in 912 bytes} [Sat Aug 20 21:35:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 20 21:35:10.079136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77c50c-7123-407b-a37d-84db5cdc6a0f service neutron] Authenticating service token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Aug 20 21:35:10.104714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77c50c-7123-407b-a37d-84db5cdc6a0f service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:10.118246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77c50c-7123-407b-a37d-84db5cdc6a0f service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:10.124211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e60456-01a0-48fa-8813-207aea37d13a tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '763034fb8c354419bf592bbe5954e5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd01fc4762f4b438241355d74aa1ea3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:10.124854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e60456-01a0-48fa-8813-207aea37d13a tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:10.125125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e60456-01a0-48fa-8813-207aea37d13a tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:10.125392 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e60456-01a0-48fa-8813-207aea37d13a tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] PATH_INFO: `/v3/projects/2fd01fc4762f4b438241355d74aa1ea3` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:10.126585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e60456-01a0-48fa-8813-207aea37d13a tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] RBAC: Authorizing `identity:get_project(project_id=2fd01fc4762f4b438241355d74aa1ea3)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:10.128409 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e60456-01a0-48fa-8813-207aea37d13a tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:10.130061 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4335/8582] 10.222.0.23 () {58 vars in 1443 bytes} [Sat Aug 20 21:35:10 2022] GET /identity/v3/projects/2fd01fc4762f4b438241355d74aa1ea3 => generated 391 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:35:10.294333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d265203-5dd2-4fcd-b8d0-1e49635c6dc5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:10.308842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d265203-5dd2-4fcd-b8d0-1e49635c6dc5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:10.312935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a72ae5-f4e5-43a8-898f-a2d4fb001bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:10.313522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a72ae5-f4e5-43a8-898f-a2d4fb001bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:10.313796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a72ae5-f4e5-43a8-898f-a2d4fb001bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:10.314060 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a72ae5-f4e5-43a8-898f-a2d4fb001bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:10.316125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-14a72ae5-f4e5-43a8-898f-a2d4fb001bb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:10.328706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a72ae5-f4e5-43a8-898f-a2d4fb001bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:10.330128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a72ae5-f4e5-43a8-898f-a2d4fb001bb4 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:10.340641 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4248/8583] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:35:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:35:13.049753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e60456-01a0-48fa-8813-207aea37d13a tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:13.115957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e60456-01a0-48fa-8813-207aea37d13a tempest-ServersAdminNegativeTestJSON-948892815 tempest-ServersAdminNegativeTestJSON-948892815-project-admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:13.121880 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:13.122720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:13.122870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:13.123109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] PATH_INFO: `/v3/users/2d62c37225e84ce1be91b2c3dc325371` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:13.124465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d62c37225e84ce1be91b2c3dc325371)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:13.126496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:13.180156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d62c37225e84ce1be91b2c3dc325371'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.188279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d62c37225e84ce1be91b2c3dc325371'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.195552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d62c37225e84ce1be91b2c3dc325371'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.208055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d62c37225e84ce1be91b2c3dc325371'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.208835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d62c37225e84ce1be91b2c3dc325371'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.215041 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4336/8584] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:35:13 2022] DELETE /identity/v3/users/2d62c37225e84ce1be91b2c3dc325371 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:35:13.219988 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47ec44-2b13-4c82-9477-a827dca88386 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:13.220528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47ec44-2b13-4c82-9477-a827dca88386 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:13.220677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47ec44-2b13-4c82-9477-a827dca88386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:13.241259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0c47ec44-2b13-4c82-9477-a827dca88386 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:35:13.242462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0c47ec44-2b13-4c82-9477-a827dca88386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:13.311728 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4249/8585] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:35:13.322987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:13.324754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:13.386487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:35:13.402018 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5725de0-e827-4dbc-87af-d72339047605 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:13.406028 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16701db4-1b9e-4b8c-b90b-904ac9211301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:13.406621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16701db4-1b9e-4b8c-b90b-904ac9211301 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:13.406889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16701db4-1b9e-4b8c-b90b-904ac9211301 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:13.407147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16701db4-1b9e-4b8c-b90b-904ac9211301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:13.416326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16701db4-1b9e-4b8c-b90b-904ac9211301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:13.417768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16701db4-1b9e-4b8c-b90b-904ac9211301 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:13.427155 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4337/8586] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:35:13.601737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47ec44-2b13-4c82-9477-a827dca88386 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:13.603435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0c47ec44-2b13-4c82-9477-a827dca88386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:13.616170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47ec44-2b13-4c82-9477-a827dca88386 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:13.622352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:13.623341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:13.623613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:13.623871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] PATH_INFO: `/v3/projects/ae29a682475b481db2ab78b6fb39a408` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:13.625621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae29a682475b481db2ab78b6fb39a408)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:13.628127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:13.685330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae29a682475b481db2ab78b6fb39a408', 'request_id': 'req-f9cdd933-deb1-43cf-a5bc-e97842250f7d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.691345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae29a682475b481db2ab78b6fb39a408', 'request_id': 'req-f9cdd933-deb1-43cf-a5bc-e97842250f7d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.700170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae29a682475b481db2ab78b6fb39a408', 'request_id': 'req-f9cdd933-deb1-43cf-a5bc-e97842250f7d'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.702447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:35:13.712725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] The token cache is being invalidate because project ae29a682475b481db2ab78b6fb39a408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:35:13.713013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.713638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.715154 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4250/8587] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:35:13 2022] DELETE /identity/v3/projects/ae29a682475b481db2ab78b6fb39a408 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:35:13.719340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16701db4-1b9e-4b8c-b90b-904ac9211301 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:13.729335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16701db4-1b9e-4b8c-b90b-904ac9211301 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:13.787062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7df2c-4aaa-45f2-b1dd-b9d57ba2c20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:13.787686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7df2c-4aaa-45f2-b1dd-b9d57ba2c20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:13.787953 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7df2c-4aaa-45f2-b1dd-b9d57ba2c20f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:13.788214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7df2c-4aaa-45f2-b1dd-b9d57ba2c20f admin admin] PATH_INFO: `/v3/users/763034fb8c354419bf592bbe5954e5ed` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:13.789341 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7df2c-4aaa-45f2-b1dd-b9d57ba2c20f admin admin] RBAC: Authorizing `identity:delete_user(user_id=763034fb8c354419bf592bbe5954e5ed)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:13.791270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7df2c-4aaa-45f2-b1dd-b9d57ba2c20f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:13.842947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7cb7df2c-4aaa-45f2-b1dd-b9d57ba2c20f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '763034fb8c354419bf592bbe5954e5ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.850570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7cb7df2c-4aaa-45f2-b1dd-b9d57ba2c20f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '763034fb8c354419bf592bbe5954e5ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.857850 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7cb7df2c-4aaa-45f2-b1dd-b9d57ba2c20f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '763034fb8c354419bf592bbe5954e5ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.871667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7cb7df2c-4aaa-45f2-b1dd-b9d57ba2c20f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '763034fb8c354419bf592bbe5954e5ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.872937 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-7cb7df2c-4aaa-45f2-b1dd-b9d57ba2c20f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '763034fb8c354419bf592bbe5954e5ed'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:13.879279 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4338/8588] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:35:13 2022] DELETE /identity/v3/users/763034fb8c354419bf592bbe5954e5ed => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:35:14.173189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:14.174643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:14.189879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdd933-deb1-43cf-a5bc-e97842250f7d admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:14.251416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd600dd5-1dbb-487e-8566-42d7a89b157c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:14.252044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd600dd5-1dbb-487e-8566-42d7a89b157c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:14.252307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd600dd5-1dbb-487e-8566-42d7a89b157c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:14.252605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd600dd5-1dbb-487e-8566-42d7a89b157c admin admin] PATH_INFO: `/v3/projects/2fd01fc4762f4b438241355d74aa1ea3` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:14.253710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd600dd5-1dbb-487e-8566-42d7a89b157c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd01fc4762f4b438241355d74aa1ea3)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:14.255638 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd600dd5-1dbb-487e-8566-42d7a89b157c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:14.303126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd600dd5-1dbb-487e-8566-42d7a89b157c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd01fc4762f4b438241355d74aa1ea3', 'request_id': 'req-cd600dd5-1dbb-487e-8566-42d7a89b157c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:14.309880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd600dd5-1dbb-487e-8566-42d7a89b157c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd01fc4762f4b438241355d74aa1ea3', 'request_id': 'req-cd600dd5-1dbb-487e-8566-42d7a89b157c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:14.320395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd600dd5-1dbb-487e-8566-42d7a89b157c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd01fc4762f4b438241355d74aa1ea3', 'request_id': 'req-cd600dd5-1dbb-487e-8566-42d7a89b157c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:14.331730 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd600dd5-1dbb-487e-8566-42d7a89b157c admin admin] The token cache is being invalidate because project 2fd01fc4762f4b438241355d74aa1ea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:35:14.332008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd600dd5-1dbb-487e-8566-42d7a89b157c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:14.332716 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-cd600dd5-1dbb-487e-8566-42d7a89b157c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:35:14.334285 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4251/8589] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:35:14 2022] DELETE /identity/v3/projects/2fd01fc4762f4b438241355d74aa1ea3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:35:14.340486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d823e68-0d78-4985-b49f-00bc4a287ab0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:14.341084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d823e68-0d78-4985-b49f-00bc4a287ab0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:14.341084 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d823e68-0d78-4985-b49f-00bc4a287ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:14.361830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7d823e68-0d78-4985-b49f-00bc4a287ab0 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:35:14.363215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7d823e68-0d78-4985-b49f-00bc4a287ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:14.427136 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4339/8590] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:35:14.436046 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eadefc-00cf-434c-8b21-a6bf5d719aed None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:14.436329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eadefc-00cf-434c-8b21-a6bf5d719aed None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:14.436637 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eadefc-00cf-434c-8b21-a6bf5d719aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:14.457696 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-88eadefc-00cf-434c-8b21-a6bf5d719aed None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:35:14.458891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-88eadefc-00cf-434c-8b21-a6bf5d719aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:14.467051 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4252/8591] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:35:14.471824 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d823e68-0d78-4985-b49f-00bc4a287ab0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:14.487445 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d823e68-0d78-4985-b49f-00bc4a287ab0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:14.495427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfca4d-dd22-4409-a1a6-082c8fc64b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:14.496223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfca4d-dd22-4409-a1a6-082c8fc64b59 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:14.496600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfca4d-dd22-4409-a1a6-082c8fc64b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:14.497388 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfca4d-dd22-4409-a1a6-082c8fc64b59 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:14.498053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfca4d-dd22-4409-a1a6-082c8fc64b59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:35:14.498379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfca4d-dd22-4409-a1a6-082c8fc64b59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:14.499975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfca4d-dd22-4409-a1a6-082c8fc64b59 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:14.501491 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:35:14.514494 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4340/8592] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:35:14.518885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eadefc-00cf-434c-8b21-a6bf5d719aed None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:14.528320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eadefc-00cf-434c-8b21-a6bf5d719aed None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:14.533283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34518a5-ae45-4ea5-8781-1e8d4d175f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:14.533886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34518a5-ae45-4ea5-8781-1e8d4d175f55 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:14.534176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34518a5-ae45-4ea5-8781-1e8d4d175f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:14.534436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34518a5-ae45-4ea5-8781-1e8d4d175f55 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:14.535079 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34518a5-ae45-4ea5-8781-1e8d4d175f55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:14.536993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34518a5-ae45-4ea5-8781-1e8d4d175f55 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:14.568238 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4253/8593] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:35:14 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:35:14.572637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfca4d-dd22-4409-a1a6-082c8fc64b59 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:14.583329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfca4d-dd22-4409-a1a6-082c8fc64b59 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:14.641350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57d5ab-acdd-48a1-9d89-0059e7903b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:14.641905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57d5ab-acdd-48a1-9d89-0059e7903b97 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:14.642178 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57d5ab-acdd-48a1-9d89-0059e7903b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:14.642443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57d5ab-acdd-48a1-9d89-0059e7903b97 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:14.643069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57d5ab-acdd-48a1-9d89-0059e7903b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:14.645051 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57d5ab-acdd-48a1-9d89-0059e7903b97 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:14.671561 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4341/8594] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:35:14 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:35:14.677039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34518a5-ae45-4ea5-8781-1e8d4d175f55 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:14.688028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34518a5-ae45-4ea5-8781-1e8d4d175f55 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:14.693419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80402ed2-c676-49fd-9ddd-d28214339dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:14.694021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80402ed2-c676-49fd-9ddd-d28214339dcc admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:14.694292 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80402ed2-c676-49fd-9ddd-d28214339dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:14.694908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80402ed2-c676-49fd-9ddd-d28214339dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:14.695543 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80402ed2-c676-49fd-9ddd-d28214339dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:14.697197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80402ed2-c676-49fd-9ddd-d28214339dcc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:14.707979 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4254/8595] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:35:14.712249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57d5ab-acdd-48a1-9d89-0059e7903b97 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:14.723002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57d5ab-acdd-48a1-9d89-0059e7903b97 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:14.728924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23be23-5cd7-405f-8b53-50d5300fbf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:14.729777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23be23-5cd7-405f-8b53-50d5300fbf74 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:14.730187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23be23-5cd7-405f-8b53-50d5300fbf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:14.730522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23be23-5cd7-405f-8b53-50d5300fbf74 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:14.731620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23be23-5cd7-405f-8b53-50d5300fbf74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:14.733292 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23be23-5cd7-405f-8b53-50d5300fbf74 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:14.744863 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4342/8596] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:35:14.749325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80402ed2-c676-49fd-9ddd-d28214339dcc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:14.761095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80402ed2-c676-49fd-9ddd-d28214339dcc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:14.767025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec179856-b8e1-4460-9c03-ba9cb609de31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:14.767678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec179856-b8e1-4460-9c03-ba9cb609de31 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:14.767973 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec179856-b8e1-4460-9c03-ba9cb609de31 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:14.768233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec179856-b8e1-4460-9c03-ba9cb609de31 admin admin] PATH_INFO: `/v3/projects/030c8f33ed3649b7addb5a1464743bda/users/ecf6af54cd2144098aa678d63130351c/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:14.784416 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec179856-b8e1-4460-9c03-ba9cb609de31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=030c8f33ed3649b7addb5a1464743bda, user_id=ecf6af54cd2144098aa678d63130351c, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:14.787127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec179856-b8e1-4460-9c03-ba9cb609de31 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:14.801981 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4255/8597] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:35:14 2022] PUT /identity/v3/projects/030c8f33ed3649b7addb5a1464743bda/users/ecf6af54cd2144098aa678d63130351c/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:35:14.807350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da819e-0b75-487f-af6f-a199dbdecfeb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:14.807649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da819e-0b75-487f-af6f-a199dbdecfeb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:14.807920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da819e-0b75-487f-af6f-a199dbdecfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:14.827681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-50da819e-0b75-487f-af6f-a199dbdecfeb None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:35:14.828908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-50da819e-0b75-487f-af6f-a199dbdecfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:14.891111 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4343/8598] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:35:14.902759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec179856-b8e1-4460-9c03-ba9cb609de31 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:14.905814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ec179856-b8e1-4460-9c03-ba9cb609de31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:14.931739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:35:14.973750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec179856-b8e1-4460-9c03-ba9cb609de31 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:14.977930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ac1f3-60e4-4565-804d-3a46d688a3ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:14.978545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ac1f3-60e4-4565-804d-3a46d688a3ed service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:14.978817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ac1f3-60e4-4565-804d-3a46d688a3ed service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:14.979080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ac1f3-60e4-4565-804d-3a46d688a3ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:14.989683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ac1f3-60e4-4565-804d-3a46d688a3ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:14.991103 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ac1f3-60e4-4565-804d-3a46d688a3ed service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:15.001645 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4256/8599] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:35:19.127827 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53039811-f468-4bb3-9a41-b67bf8d9a608 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:19.128364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53039811-f468-4bb3-9a41-b67bf8d9a608 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:19.128364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53039811-f468-4bb3-9a41-b67bf8d9a608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:19.159949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-53039811-f468-4bb3-9a41-b67bf8d9a608 None None] MFA Rules not processed for user `ecf6af54cd2144098aa678d63130351c`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:35:19.161246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-53039811-f468-4bb3-9a41-b67bf8d9a608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:19.259790 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4344/8600] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:35:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:35:19.273438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8cad2-d3aa-41f7-8028-52bd8950b111 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:19.273438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8cad2-d3aa-41f7-8028-52bd8950b111 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:19.273438 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8cad2-d3aa-41f7-8028-52bd8950b111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:19.294902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-78d8cad2-d3aa-41f7-8028-52bd8950b111 None None] MFA Rules not processed for user `ecf6af54cd2144098aa678d63130351c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:35:19.296195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-78d8cad2-d3aa-41f7-8028-52bd8950b111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:19.304179 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4257/8601] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:35:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:35:19.317639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53039811-f468-4bb3-9a41-b67bf8d9a608 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:19.320037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-53039811-f468-4bb3-9a41-b67bf8d9a608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:19.328600 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:35:19.412708 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53039811-f468-4bb3-9a41-b67bf8d9a608 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:19.418287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578d13b-3e43-491a-92c8-ec68f12c555a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:19.418942 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578d13b-3e43-491a-92c8-ec68f12c555a service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:19.419246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578d13b-3e43-491a-92c8-ec68f12c555a service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:19.419517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578d13b-3e43-491a-92c8-ec68f12c555a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:19.428134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578d13b-3e43-491a-92c8-ec68f12c555a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:19.429781 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578d13b-3e43-491a-92c8-ec68f12c555a service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:19.439633 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4345/8602] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:35:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:35:29.288911 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=21 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:35:29.435250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=33 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:35:32.198989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8cad2-d3aa-41f7-8028-52bd8950b111 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:32.212365 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8cad2-d3aa-41f7-8028-52bd8950b111 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:32.216651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55988181-a739-4013-be11-d9a392ec55af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:32.217253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55988181-a739-4013-be11-d9a392ec55af service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:32.217521 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55988181-a739-4013-be11-d9a392ec55af service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:32.217801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55988181-a739-4013-be11-d9a392ec55af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:32.219683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-55988181-a739-4013-be11-d9a392ec55af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:32.238952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55988181-a739-4013-be11-d9a392ec55af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:32.240472 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55988181-a739-4013-be11-d9a392ec55af service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:32.251916 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4258/8603] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:35:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:35:32.344657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578d13b-3e43-491a-92c8-ec68f12c555a service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:35:32.347096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5578d13b-3e43-491a-92c8-ec68f12c555a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:32.421289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578d13b-3e43-491a-92c8-ec68f12c555a service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:35:32.427049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51626dc1-477e-431b-b314-079937d8e935 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:35:32.427690 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51626dc1-477e-431b-b314-079937d8e935 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:35:32.427952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51626dc1-477e-431b-b314-079937d8e935 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:35:32.428207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51626dc1-477e-431b-b314-079937d8e935 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:35:32.430150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-51626dc1-477e-431b-b314-079937d8e935 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:35:32.447198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51626dc1-477e-431b-b314-079937d8e935 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:35:32.448821 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51626dc1-477e-431b-b314-079937d8e935 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:35:32.461502 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4346/8604] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:35:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:35:42.245594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:35:42.454776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=32 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:36:12.084054 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55988181-a739-4013-be11-d9a392ec55af service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:36:12.101318 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55988181-a739-4013-be11-d9a392ec55af service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:36:12.106025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d6ebc-9199-4679-bdf3-4bd1262d3025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:36:12.106658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d6ebc-9199-4679-bdf3-4bd1262d3025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:12.106932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d6ebc-9199-4679-bdf3-4bd1262d3025 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:12.107197 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d6ebc-9199-4679-bdf3-4bd1262d3025 admin admin] PATH_INFO: `/v3/users/ecf6af54cd2144098aa678d63130351c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:12.108329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4d6ebc-9199-4679-bdf3-4bd1262d3025 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf6af54cd2144098aa678d63130351c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:36:12.110333 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4d6ebc-9199-4679-bdf3-4bd1262d3025 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:36:12.170514 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec4d6ebc-9199-4679-bdf3-4bd1262d3025 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf6af54cd2144098aa678d63130351c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:36:12.179315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec4d6ebc-9199-4679-bdf3-4bd1262d3025 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf6af54cd2144098aa678d63130351c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:36:12.192989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec4d6ebc-9199-4679-bdf3-4bd1262d3025 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf6af54cd2144098aa678d63130351c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:36:12.194001 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec4d6ebc-9199-4679-bdf3-4bd1262d3025 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf6af54cd2144098aa678d63130351c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:36:12.194616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-ec4d6ebc-9199-4679-bdf3-4bd1262d3025 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf6af54cd2144098aa678d63130351c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:36:12.209139 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4259/8605] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:36:12 2022] DELETE /identity/v3/users/ecf6af54cd2144098aa678d63130351c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:36:12.390212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51626dc1-477e-431b-b314-079937d8e935 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:36:12.401397 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51626dc1-477e-431b-b314-079937d8e935 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:36:12.467472 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a5c2f-2b6f-4a3e-9b76-f5159197c311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:36:12.468038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a5c2f-2b6f-4a3e-9b76-f5159197c311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:12.468310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a5c2f-2b6f-4a3e-9b76-f5159197c311 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:12.468626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a5c2f-2b6f-4a3e-9b76-f5159197c311 admin admin] PATH_INFO: `/v3/projects/030c8f33ed3649b7addb5a1464743bda` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:12.469777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a5c2f-2b6f-4a3e-9b76-f5159197c311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=030c8f33ed3649b7addb5a1464743bda)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:36:12.471675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a5c2f-2b6f-4a3e-9b76-f5159197c311 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:36:12.523520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-270a5c2f-2b6f-4a3e-9b76-f5159197c311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '030c8f33ed3649b7addb5a1464743bda', 'request_id': 'req-270a5c2f-2b6f-4a3e-9b76-f5159197c311'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:36:12.536889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-270a5c2f-2b6f-4a3e-9b76-f5159197c311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '030c8f33ed3649b7addb5a1464743bda', 'request_id': 'req-270a5c2f-2b6f-4a3e-9b76-f5159197c311'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:36:12.544681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-270a5c2f-2b6f-4a3e-9b76-f5159197c311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '030c8f33ed3649b7addb5a1464743bda', 'request_id': 'req-270a5c2f-2b6f-4a3e-9b76-f5159197c311'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:36:12.562232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-270a5c2f-2b6f-4a3e-9b76-f5159197c311 admin admin] The token cache is being invalidate because project 030c8f33ed3649b7addb5a1464743bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:36:12.562533 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-270a5c2f-2b6f-4a3e-9b76-f5159197c311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:36:12.563563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-270a5c2f-2b6f-4a3e-9b76-f5159197c311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:36:12.565918 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4347/8606] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:36:12 2022] DELETE /identity/v3/projects/030c8f33ed3649b7addb5a1464743bda => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:36:12.574255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464d5a4-d29a-4b81-abc0-b7ecaceece37 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:12.574553 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464d5a4-d29a-4b81-abc0-b7ecaceece37 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:12.574826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464d5a4-d29a-4b81-abc0-b7ecaceece37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:12.596188 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1464d5a4-d29a-4b81-abc0-b7ecaceece37 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:36:12.597461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1464d5a4-d29a-4b81-abc0-b7ecaceece37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:36:12.660293 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4260/8607] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:36:12.668287 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88bdef-b64b-4796-b568-22343850c94f None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:12.668616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88bdef-b64b-4796-b568-22343850c94f None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:12.668884 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88bdef-b64b-4796-b568-22343850c94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:12.688798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-0d88bdef-b64b-4796-b568-22343850c94f None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:36:12.689992 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0d88bdef-b64b-4796-b568-22343850c94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:36:12.698866 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4348/8608] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:36:12.703230 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464d5a4-d29a-4b81-abc0-b7ecaceece37 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:36:12.713693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464d5a4-d29a-4b81-abc0-b7ecaceece37 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:36:12.719690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e3c02-efb0-4bf2-9dab-3346ff17b3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:36:12.720284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e3c02-efb0-4bf2-9dab-3346ff17b3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:12.720597 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e3c02-efb0-4bf2-9dab-3346ff17b3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:12.720865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e3c02-efb0-4bf2-9dab-3346ff17b3d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:12.721506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e3c02-efb0-4bf2-9dab-3346ff17b3d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:36:12.721804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e3c02-efb0-4bf2-9dab-3346ff17b3d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:36:12.723402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e3c02-efb0-4bf2-9dab-3346ff17b3d9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:36:12.735630 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4261/8609] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:36:12.742109 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88bdef-b64b-4796-b568-22343850c94f None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:36:12.757157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88bdef-b64b-4796-b568-22343850c94f None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:36:12.764550 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bb242-f7af-49a8-a05a-b60af13f0722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:36:12.765145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bb242-f7af-49a8-a05a-b60af13f0722 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:12.765418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bb242-f7af-49a8-a05a-b60af13f0722 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:12.765701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bb242-f7af-49a8-a05a-b60af13f0722 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:12.766314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bb242-f7af-49a8-a05a-b60af13f0722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:36:12.768229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bb242-f7af-49a8-a05a-b60af13f0722 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:36:12.809309 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4349/8610] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:36:12 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:36:12.813490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e3c02-efb0-4bf2-9dab-3346ff17b3d9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:36:12.824621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e3c02-efb0-4bf2-9dab-3346ff17b3d9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:36:12.889428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eab992-d7bc-4e16-b52e-73017e593c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:36:12.890133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eab992-d7bc-4e16-b52e-73017e593c05 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:12.890461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eab992-d7bc-4e16-b52e-73017e593c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:12.890785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eab992-d7bc-4e16-b52e-73017e593c05 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:12.891450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eab992-d7bc-4e16-b52e-73017e593c05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:36:12.893390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eab992-d7bc-4e16-b52e-73017e593c05 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:36:12.901475 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:36:12.920787 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4262/8611] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:36:12 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:36:12.925496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bb242-f7af-49a8-a05a-b60af13f0722 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:36:12.939250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bb242-f7af-49a8-a05a-b60af13f0722 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:36:12.944015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e9912-8b6d-498a-a5df-e2f7182b011b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:36:12.944640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e9912-8b6d-498a-a5df-e2f7182b011b admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:12.944911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e9912-8b6d-498a-a5df-e2f7182b011b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:12.945232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e9912-8b6d-498a-a5df-e2f7182b011b admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:12.946220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e9912-8b6d-498a-a5df-e2f7182b011b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:36:12.948198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e9912-8b6d-498a-a5df-e2f7182b011b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:36:12.958686 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4350/8612] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:36:12.962954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eab992-d7bc-4e16-b52e-73017e593c05 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:36:12.973446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eab992-d7bc-4e16-b52e-73017e593c05 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:36:12.979175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f628-d7c2-4a4e-b1bf-eeb7d906755e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:36:12.979763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f628-d7c2-4a4e-b1bf-eeb7d906755e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:12.980107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f628-d7c2-4a4e-b1bf-eeb7d906755e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:12.980462 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f628-d7c2-4a4e-b1bf-eeb7d906755e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:12.981144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586f628-d7c2-4a4e-b1bf-eeb7d906755e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:36:12.982731 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586f628-d7c2-4a4e-b1bf-eeb7d906755e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:36:12.994064 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4263/8613] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:36:12.999206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e9912-8b6d-498a-a5df-e2f7182b011b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:36:13.011689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e9912-8b6d-498a-a5df-e2f7182b011b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:36:13.017417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc77c2-3304-4dd4-a217-0d8f34e93fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:36:13.018083 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc77c2-3304-4dd4-a217-0d8f34e93fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:13.018355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc77c2-3304-4dd4-a217-0d8f34e93fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:13.018605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc77c2-3304-4dd4-a217-0d8f34e93fcb admin admin] PATH_INFO: `/v3/projects/b84ded171a2e484c92bb740214c1fe77/users/299765af8fe040e695e2284499cb1267/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:13.036574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc77c2-3304-4dd4-a217-0d8f34e93fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b84ded171a2e484c92bb740214c1fe77, user_id=299765af8fe040e695e2284499cb1267, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:36:13.041682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc77c2-3304-4dd4-a217-0d8f34e93fcb admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:36:13.059433 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4351/8614] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:36:12 2022] PUT /identity/v3/projects/b84ded171a2e484c92bb740214c1fe77/users/299765af8fe040e695e2284499cb1267/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:36:13.065431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be152af-91f3-4c75-83be-c9245c9bc3c9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:13.065885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be152af-91f3-4c75-83be-c9245c9bc3c9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:13.066251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be152af-91f3-4c75-83be-c9245c9bc3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:13.096883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-3be152af-91f3-4c75-83be-c9245c9bc3c9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:36:13.098106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-3be152af-91f3-4c75-83be-c9245c9bc3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:36:13.171746 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4264/8615] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:36:13.181918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc77c2-3304-4dd4-a217-0d8f34e93fcb admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:36:13.184039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f4dc77c2-3304-4dd4-a217-0d8f34e93fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:36:13.206250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:36:13.250500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc77c2-3304-4dd4-a217-0d8f34e93fcb admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:36:13.254525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e10f14-9359-43ca-8225-78a128191e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:36:13.255153 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e10f14-9359-43ca-8225-78a128191e2b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:13.255423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e10f14-9359-43ca-8225-78a128191e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:13.255689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e10f14-9359-43ca-8225-78a128191e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:13.265079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e10f14-9359-43ca-8225-78a128191e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:36:13.266497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e10f14-9359-43ca-8225-78a128191e2b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:36:13.275974 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4352/8616] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:36:17.467460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48797202-407a-4c95-8a51-46ee1c2ba4ab None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:17.467460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48797202-407a-4c95-8a51-46ee1c2ba4ab None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:17.468007 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48797202-407a-4c95-8a51-46ee1c2ba4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:17.497928 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-48797202-407a-4c95-8a51-46ee1c2ba4ab None None] MFA Rules not processed for user `299765af8fe040e695e2284499cb1267`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:36:17.499123 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-48797202-407a-4c95-8a51-46ee1c2ba4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:36:17.565781 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4265/8617] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:36:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:36:17.577342 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d558e4-d84c-4a1f-bd93-2052b4297838 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:17.578035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d558e4-d84c-4a1f-bd93-2052b4297838 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:17.578035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d558e4-d84c-4a1f-bd93-2052b4297838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:17.602513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-76d558e4-d84c-4a1f-bd93-2052b4297838 None None] MFA Rules not processed for user `299765af8fe040e695e2284499cb1267`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:36:17.603715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-76d558e4-d84c-4a1f-bd93-2052b4297838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:36:17.611599 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4353/8618] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:36:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:36:17.626930 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48797202-407a-4c95-8a51-46ee1c2ba4ab None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:36:17.628773 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-48797202-407a-4c95-8a51-46ee1c2ba4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:36:17.712703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48797202-407a-4c95-8a51-46ee1c2ba4ab None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:36:17.719673 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4839b-43c0-41dc-b24f-a77be735a249 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:36:17.720284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4839b-43c0-41dc-b24f-a77be735a249 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:17.720598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4839b-43c0-41dc-b24f-a77be735a249 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:17.720866 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4839b-43c0-41dc-b24f-a77be735a249 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:17.726255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:36:17.731883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4839b-43c0-41dc-b24f-a77be735a249 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:36:17.733859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4839b-43c0-41dc-b24f-a77be735a249 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:36:17.743666 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4266/8619] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:36:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:36:21.570449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d558e4-d84c-4a1f-bd93-2052b4297838 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:36:21.584865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d558e4-d84c-4a1f-bd93-2052b4297838 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:36:21.588647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c593e-b84a-49bc-a5ff-981404457598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:36:21.589256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c593e-b84a-49bc-a5ff-981404457598 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:21.589529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c593e-b84a-49bc-a5ff-981404457598 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:21.589812 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c593e-b84a-49bc-a5ff-981404457598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:21.591699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-122c593e-b84a-49bc-a5ff-981404457598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:36:21.604870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c593e-b84a-49bc-a5ff-981404457598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:36:21.606577 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c593e-b84a-49bc-a5ff-981404457598 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:36:21.618535 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4354/8620] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:36:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:36:27.739444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:36:31.612217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=27 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:36:59.902803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4839b-43c0-41dc-b24f-a77be735a249 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:36:59.904284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a6c4839b-43c0-41dc-b24f-a77be735a249 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:36:59.930545 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4839b-43c0-41dc-b24f-a77be735a249 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:36:59.935042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6aff45-52b8-4596-9fbb-3f863abd8901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:36:59.935660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6aff45-52b8-4596-9fbb-3f863abd8901 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:36:59.935932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6aff45-52b8-4596-9fbb-3f863abd8901 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:36:59.936195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6aff45-52b8-4596-9fbb-3f863abd8901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:36:59.938082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8e6aff45-52b8-4596-9fbb-3f863abd8901 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:36:59.951117 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6aff45-52b8-4596-9fbb-3f863abd8901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:36:59.952568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6aff45-52b8-4596-9fbb-3f863abd8901 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:36:59.963531 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4267/8621] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:36:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:37:05.493255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c593e-b84a-49bc-a5ff-981404457598 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:37:05.494591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-122c593e-b84a-49bc-a5ff-981404457598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:37:05.516927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c593e-b84a-49bc-a5ff-981404457598 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:37:05.541431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef460d-4068-43fd-96b5-07bc551f0028 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:37:05.542354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef460d-4068-43fd-96b5-07bc551f0028 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:37:05.542354 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef460d-4068-43fd-96b5-07bc551f0028 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:37:05.542555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef460d-4068-43fd-96b5-07bc551f0028 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:37:05.545460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e6ef460d-4068-43fd-96b5-07bc551f0028 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:37:05.564203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef460d-4068-43fd-96b5-07bc551f0028 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:37:05.566340 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef460d-4068-43fd-96b5-07bc551f0028 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:37:05.582625 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4355/8622] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:37:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:37:06.996469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bfffc-5a73-4b16-9c44-ccba131d1444 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:37:06.997138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bfffc-5a73-4b16-9c44-ccba131d1444 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:37:06.997138 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bfffc-5a73-4b16-9c44-ccba131d1444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:37:07.030896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8a5bfffc-5a73-4b16-9c44-ccba131d1444 None None] MFA Rules not processed for user `299765af8fe040e695e2284499cb1267`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:37:07.032144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8a5bfffc-5a73-4b16-9c44-ccba131d1444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:37:07.040395 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4268/8623] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:37:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:37:15.573843 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:37:17.025203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=13 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:38:01.319350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef460d-4068-43fd-96b5-07bc551f0028 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:38:01.351811 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef460d-4068-43fd-96b5-07bc551f0028 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:38:01.357542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e669eb-7baa-41f6-b787-014fd3fbfd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:38:01.358169 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e669eb-7baa-41f6-b787-014fd3fbfd49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:01.358440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e669eb-7baa-41f6-b787-014fd3fbfd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:01.358704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e669eb-7baa-41f6-b787-014fd3fbfd49 admin admin] PATH_INFO: `/v3/users/299765af8fe040e695e2284499cb1267` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:01.359804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e669eb-7baa-41f6-b787-014fd3fbfd49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=299765af8fe040e695e2284499cb1267)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:38:01.361748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e669eb-7baa-41f6-b787-014fd3fbfd49 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:38:01.412175 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91e669eb-7baa-41f6-b787-014fd3fbfd49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299765af8fe040e695e2284499cb1267'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:38:01.422272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91e669eb-7baa-41f6-b787-014fd3fbfd49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299765af8fe040e695e2284499cb1267'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:38:01.432140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91e669eb-7baa-41f6-b787-014fd3fbfd49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299765af8fe040e695e2284499cb1267'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:38:01.449111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91e669eb-7baa-41f6-b787-014fd3fbfd49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299765af8fe040e695e2284499cb1267'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:38:01.449794 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-91e669eb-7baa-41f6-b787-014fd3fbfd49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299765af8fe040e695e2284499cb1267'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:38:01.456021 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4356/8624] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:38:01 2022] DELETE /identity/v3/users/299765af8fe040e695e2284499cb1267 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:38:01.640493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bfffc-5a73-4b16-9c44-ccba131d1444 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:38:01.655244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bfffc-5a73-4b16-9c44-ccba131d1444 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:38:01.715180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba88b9a-7068-4d31-a41b-cdeb4974ced7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:38:01.715916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba88b9a-7068-4d31-a41b-cdeb4974ced7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:01.716191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba88b9a-7068-4d31-a41b-cdeb4974ced7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:01.716501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba88b9a-7068-4d31-a41b-cdeb4974ced7 admin admin] PATH_INFO: `/v3/projects/b84ded171a2e484c92bb740214c1fe77` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:01.717863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba88b9a-7068-4d31-a41b-cdeb4974ced7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b84ded171a2e484c92bb740214c1fe77)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:38:01.720019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba88b9a-7068-4d31-a41b-cdeb4974ced7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:38:01.779287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eba88b9a-7068-4d31-a41b-cdeb4974ced7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b84ded171a2e484c92bb740214c1fe77', 'request_id': 'req-eba88b9a-7068-4d31-a41b-cdeb4974ced7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:38:01.785957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eba88b9a-7068-4d31-a41b-cdeb4974ced7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b84ded171a2e484c92bb740214c1fe77', 'request_id': 'req-eba88b9a-7068-4d31-a41b-cdeb4974ced7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:38:01.795783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eba88b9a-7068-4d31-a41b-cdeb4974ced7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b84ded171a2e484c92bb740214c1fe77', 'request_id': 'req-eba88b9a-7068-4d31-a41b-cdeb4974ced7'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:38:01.807204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eba88b9a-7068-4d31-a41b-cdeb4974ced7 admin admin] The token cache is being invalidate because project b84ded171a2e484c92bb740214c1fe77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:38:01.807495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eba88b9a-7068-4d31-a41b-cdeb4974ced7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:38:01.808208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eba88b9a-7068-4d31-a41b-cdeb4974ced7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:38:01.809865 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4269/8625] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:38:01 2022] DELETE /identity/v3/projects/b84ded171a2e484c92bb740214c1fe77 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:38:01.815196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f6e5a-349a-4d70-8f6f-f04c71ec5e3e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:01.815504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f6e5a-349a-4d70-8f6f-f04c71ec5e3e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:01.815810 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f6e5a-349a-4d70-8f6f-f04c71ec5e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:01.837512 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fc3f6e5a-349a-4d70-8f6f-f04c71ec5e3e None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:38:01.838725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fc3f6e5a-349a-4d70-8f6f-f04c71ec5e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:38:01.901549 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4357/8626] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:38:01.909685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd86c5-acec-414d-8d3e-1dedea8295ec None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:01.909968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd86c5-acec-414d-8d3e-1dedea8295ec None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:01.910238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd86c5-acec-414d-8d3e-1dedea8295ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:01.929960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9bcd86c5-acec-414d-8d3e-1dedea8295ec None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:38:01.931289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9bcd86c5-acec-414d-8d3e-1dedea8295ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:38:01.939276 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4270/8627] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:38:01.943721 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f6e5a-349a-4d70-8f6f-f04c71ec5e3e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:38:01.954140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f6e5a-349a-4d70-8f6f-f04c71ec5e3e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:38:01.959166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4822b-c403-4c1d-a7b2-8ad3c39285a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:38:01.959761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4822b-c403-4c1d-a7b2-8ad3c39285a5 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:01.960027 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4822b-c403-4c1d-a7b2-8ad3c39285a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:01.960298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4822b-c403-4c1d-a7b2-8ad3c39285a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:01.960946 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4822b-c403-4c1d-a7b2-8ad3c39285a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:38:01.961242 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4822b-c403-4c1d-a7b2-8ad3c39285a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:38:01.962938 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4822b-c403-4c1d-a7b2-8ad3c39285a5 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:38:01.974510 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4358/8628] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:38:01.979429 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd86c5-acec-414d-8d3e-1dedea8295ec None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:38:01.989166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd86c5-acec-414d-8d3e-1dedea8295ec None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:38:01.994732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652401b5-825e-4aeb-8af8-fe503bdeef1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:38:01.995329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652401b5-825e-4aeb-8af8-fe503bdeef1b admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:01.995599 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652401b5-825e-4aeb-8af8-fe503bdeef1b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:01.995860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652401b5-825e-4aeb-8af8-fe503bdeef1b admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:01.996510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652401b5-825e-4aeb-8af8-fe503bdeef1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:38:01.998492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652401b5-825e-4aeb-8af8-fe503bdeef1b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:38:02.029814 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4271/8629] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:38:01 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:38:02.033848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4822b-c403-4c1d-a7b2-8ad3c39285a5 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:38:02.045429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4822b-c403-4c1d-a7b2-8ad3c39285a5 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:38:02.101501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:38:02.113670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87c9e5-1e65-46b9-abaa-aeafe8995d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:38:02.114278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87c9e5-1e65-46b9-abaa-aeafe8995d84 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:02.114554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87c9e5-1e65-46b9-abaa-aeafe8995d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:02.114814 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87c9e5-1e65-46b9-abaa-aeafe8995d84 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:02.115425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87c9e5-1e65-46b9-abaa-aeafe8995d84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:38:02.117385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87c9e5-1e65-46b9-abaa-aeafe8995d84 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:38:02.143648 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4359/8630] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:38:02 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:38:02.145277 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652401b5-825e-4aeb-8af8-fe503bdeef1b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:38:02.155755 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652401b5-825e-4aeb-8af8-fe503bdeef1b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:38:02.160769 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563da76a-9e35-4094-835d-8531f7b7e3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:38:02.161426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563da76a-9e35-4094-835d-8531f7b7e3bc admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:02.161703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563da76a-9e35-4094-835d-8531f7b7e3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:02.161968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563da76a-9e35-4094-835d-8531f7b7e3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:02.162614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563da76a-9e35-4094-835d-8531f7b7e3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:38:02.164276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563da76a-9e35-4094-835d-8531f7b7e3bc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:38:02.175118 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4272/8631] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:38:02.179299 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87c9e5-1e65-46b9-abaa-aeafe8995d84 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:38:02.189595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87c9e5-1e65-46b9-abaa-aeafe8995d84 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:38:02.194700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e2097-f407-4742-8023-60ad13b20c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:38:02.195308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e2097-f407-4742-8023-60ad13b20c62 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:02.195582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e2097-f407-4742-8023-60ad13b20c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:02.195832 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e2097-f407-4742-8023-60ad13b20c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:02.196581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e2097-f407-4742-8023-60ad13b20c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:38:02.198230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e2097-f407-4742-8023-60ad13b20c62 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:38:02.209583 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4360/8632] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:38:02.214225 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563da76a-9e35-4094-835d-8531f7b7e3bc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:38:02.228157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563da76a-9e35-4094-835d-8531f7b7e3bc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:38:02.233513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f6887-ca11-4ea7-b65d-00309cab88c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:38:02.234181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f6887-ca11-4ea7-b65d-00309cab88c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:02.234564 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f6887-ca11-4ea7-b65d-00309cab88c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:02.234715 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f6887-ca11-4ea7-b65d-00309cab88c0 admin admin] PATH_INFO: `/v3/projects/10e1a87a100a4dcb9a30a6c4d35ff429/users/c0447084bdd24f549ecfc88b9cc7b3be/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:02.251732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f6887-ca11-4ea7-b65d-00309cab88c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e1a87a100a4dcb9a30a6c4d35ff429, user_id=c0447084bdd24f549ecfc88b9cc7b3be, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:38:02.254432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f6887-ca11-4ea7-b65d-00309cab88c0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:38:02.272306 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4273/8633] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:38:02 2022] PUT /identity/v3/projects/10e1a87a100a4dcb9a30a6c4d35ff429/users/c0447084bdd24f549ecfc88b9cc7b3be/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:38:02.278306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae31d2-2ebb-4b78-b602-5b8e947d6428 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:02.278626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae31d2-2ebb-4b78-b602-5b8e947d6428 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:02.278987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae31d2-2ebb-4b78-b602-5b8e947d6428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:02.301746 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-3fae31d2-2ebb-4b78-b602-5b8e947d6428 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:38:02.302945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-3fae31d2-2ebb-4b78-b602-5b8e947d6428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:38:02.366523 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4361/8634] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:38:02.383070 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f6887-ca11-4ea7-b65d-00309cab88c0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:38:02.386648 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2c3f6887-ca11-4ea7-b65d-00309cab88c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:38:02.425865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:38:02.459570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f6887-ca11-4ea7-b65d-00309cab88c0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:38:02.463750 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e4468-1812-45fd-88c8-3e34f865c6ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:38:02.464348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e4468-1812-45fd-88c8-3e34f865c6ae service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:02.464660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e4468-1812-45fd-88c8-3e34f865c6ae service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:02.464920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e4468-1812-45fd-88c8-3e34f865c6ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:02.474612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e4468-1812-45fd-88c8-3e34f865c6ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:38:02.476035 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e4468-1812-45fd-88c8-3e34f865c6ae service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:38:02.486110 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4274/8635] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:38:06.553956 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd5aa1-ba70-40a2-8cdf-f073429b54de None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:06.554441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd5aa1-ba70-40a2-8cdf-f073429b54de None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:06.554525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd5aa1-ba70-40a2-8cdf-f073429b54de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:06.587876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-95cd5aa1-ba70-40a2-8cdf-f073429b54de None None] MFA Rules not processed for user `c0447084bdd24f549ecfc88b9cc7b3be`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:38:06.589037 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-95cd5aa1-ba70-40a2-8cdf-f073429b54de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:38:06.667392 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4362/8636] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:38:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:38:06.681996 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e4468-1812-45fd-88c8-3e34f865c6ae service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:38:06.683691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1c5e4468-1812-45fd-88c8-3e34f865c6ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:38:06.795992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e4468-1812-45fd-88c8-3e34f865c6ae service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:38:06.801639 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734bdd2-0ba0-4508-ac1d-c8dea18628b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:38:06.802255 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734bdd2-0ba0-4508-ac1d-c8dea18628b7 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:06.802533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734bdd2-0ba0-4508-ac1d-c8dea18628b7 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:06.802804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734bdd2-0ba0-4508-ac1d-c8dea18628b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:06.812914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734bdd2-0ba0-4508-ac1d-c8dea18628b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:38:06.814535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734bdd2-0ba0-4508-ac1d-c8dea18628b7 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:38:06.825369 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4275/8637] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:38:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:38:09.335765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354690d9-1d4b-4f7a-89cd-8a4828cf2930 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:09.336296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354690d9-1d4b-4f7a-89cd-8a4828cf2930 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:09.336538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354690d9-1d4b-4f7a-89cd-8a4828cf2930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:09.356036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-354690d9-1d4b-4f7a-89cd-8a4828cf2930 None None] MFA Rules not processed for user `c0447084bdd24f549ecfc88b9cc7b3be`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:38:09.357278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-354690d9-1d4b-4f7a-89cd-8a4828cf2930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:38:09.364450 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4363/8638] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:38:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:38:10.052601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734bdd2-0ba0-4508-ac1d-c8dea18628b7 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:38:10.054069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-7734bdd2-0ba0-4508-ac1d-c8dea18628b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:38:10.162254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734bdd2-0ba0-4508-ac1d-c8dea18628b7 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:38:10.167902 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6897d2-3dbc-465c-9819-cc63afaf9df7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:38:10.168574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6897d2-3dbc-465c-9819-cc63afaf9df7 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:10.168850 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6897d2-3dbc-465c-9819-cc63afaf9df7 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:10.169118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6897d2-3dbc-465c-9819-cc63afaf9df7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:10.170879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9a6897d2-3dbc-465c-9819-cc63afaf9df7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:38:10.187861 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6897d2-3dbc-465c-9819-cc63afaf9df7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:38:10.189726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6897d2-3dbc-465c-9819-cc63afaf9df7 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:38:10.205310 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4276/8639] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:38:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:38:19.350504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=73,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:38:20.197264 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=88 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:38:22.145622 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354690d9-1d4b-4f7a-89cd-8a4828cf2930 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:38:22.147419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-354690d9-1d4b-4f7a-89cd-8a4828cf2930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:38:22.166198 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354690d9-1d4b-4f7a-89cd-8a4828cf2930 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:38:22.172544 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f8fe6-2d54-4884-bd31-845f075319d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:38:22.173186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f8fe6-2d54-4884-bd31-845f075319d8 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:22.173455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f8fe6-2d54-4884-bd31-845f075319d8 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:22.173715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f8fe6-2d54-4884-bd31-845f075319d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:22.175706 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a38f8fe6-2d54-4884-bd31-845f075319d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:38:22.191768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f8fe6-2d54-4884-bd31-845f075319d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:38:22.193473 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f8fe6-2d54-4884-bd31-845f075319d8 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:38:22.208859 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4364/8640] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:38:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:38:23.955890 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6897d2-3dbc-465c-9819-cc63afaf9df7 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:38:23.957329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9a6897d2-3dbc-465c-9819-cc63afaf9df7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:38:23.973470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6897d2-3dbc-465c-9819-cc63afaf9df7 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:38:23.979947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce931c-720a-4560-a22a-b04cc94429cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:38:23.980614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce931c-720a-4560-a22a-b04cc94429cd service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:38:23.980884 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce931c-720a-4560-a22a-b04cc94429cd service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:38:23.981150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce931c-720a-4560-a22a-b04cc94429cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:38:23.983118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-89ce931c-720a-4560-a22a-b04cc94429cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:38:23.997780 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce931c-720a-4560-a22a-b04cc94429cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:38:23.999348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce931c-720a-4560-a22a-b04cc94429cd service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:38:24.012788 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4277/8641] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:38:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:38:32.201477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:38:34.005289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:39:17.254640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f8fe6-2d54-4884-bd31-845f075319d8 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:39:17.269295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f8fe6-2d54-4884-bd31-845f075319d8 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:39:17.274756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445012b0-e2fb-465f-9fd5-20d289e84623 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:39:17.275357 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445012b0-e2fb-465f-9fd5-20d289e84623 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:39:17.275629 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445012b0-e2fb-465f-9fd5-20d289e84623 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:39:17.275894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445012b0-e2fb-465f-9fd5-20d289e84623 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:39:17.277755 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-445012b0-e2fb-465f-9fd5-20d289e84623 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:39:17.292835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445012b0-e2fb-465f-9fd5-20d289e84623 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:39:17.294429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445012b0-e2fb-465f-9fd5-20d289e84623 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:39:17.307383 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4365/8642] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:39:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:39:27.300253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:39:28.052478 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce931c-720a-4560-a22a-b04cc94429cd service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:39:28.054119 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-89ce931c-720a-4560-a22a-b04cc94429cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:39:28.080810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce931c-720a-4560-a22a-b04cc94429cd service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:39:28.084944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355f692-ea44-45d3-9222-d4ac7da5d68b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:39:28.085534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355f692-ea44-45d3-9222-d4ac7da5d68b service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:39:28.085803 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355f692-ea44-45d3-9222-d4ac7da5d68b service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:39:28.086064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355f692-ea44-45d3-9222-d4ac7da5d68b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:39:28.087794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1355f692-ea44-45d3-9222-d4ac7da5d68b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:39:28.122734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355f692-ea44-45d3-9222-d4ac7da5d68b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:39:28.124170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355f692-ea44-45d3-9222-d4ac7da5d68b service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:39:28.135914 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4278/8643] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:39:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:39:38.130782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=19 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:39:51.560911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2135fd-fb87-4682-b0f6-e6a31e6aca15 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:39:51.561427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2135fd-fb87-4682-b0f6-e6a31e6aca15 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:39:51.561427 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2135fd-fb87-4682-b0f6-e6a31e6aca15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:39:51.582700 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ca2135fd-fb87-4682-b0f6-e6a31e6aca15 None None] MFA Rules not processed for user `c0447084bdd24f549ecfc88b9cc7b3be`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:39:51.583846 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ca2135fd-fb87-4682-b0f6-e6a31e6aca15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:39:51.595813 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4366/8644] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:39:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:39:54.956275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355f692-ea44-45d3-9222-d4ac7da5d68b service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:39:54.978413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355f692-ea44-45d3-9222-d4ac7da5d68b service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:39:54.984708 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dd469-424a-46be-9fba-f46b19bd2e67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:39:54.985321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dd469-424a-46be-9fba-f46b19bd2e67 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:39:54.985613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dd469-424a-46be-9fba-f46b19bd2e67 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:39:54.985871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dd469-424a-46be-9fba-f46b19bd2e67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:39:54.987764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-838dd469-424a-46be-9fba-f46b19bd2e67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:39:55.003538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838dd469-424a-46be-9fba-f46b19bd2e67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:39:55.005168 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838dd469-424a-46be-9fba-f46b19bd2e67 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:39:55.019258 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4279/8645] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:39:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:40:01.593205 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:40:05.016408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=8 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:41:16.582413 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2135fd-fb87-4682-b0f6-e6a31e6aca15 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:41:16.602541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2135fd-fb87-4682-b0f6-e6a31e6aca15 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:41:16.607115 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51c76d-4146-43bb-8d9e-6872b5a7644e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:41:16.607844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51c76d-4146-43bb-8d9e-6872b5a7644e service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:41:16.608167 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51c76d-4146-43bb-8d9e-6872b5a7644e service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:41:16.608536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51c76d-4146-43bb-8d9e-6872b5a7644e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:41:16.610610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dc51c76d-4146-43bb-8d9e-6872b5a7644e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:41:16.638455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51c76d-4146-43bb-8d9e-6872b5a7644e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:41:16.639907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51c76d-4146-43bb-8d9e-6872b5a7644e service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:41:16.654057 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4367/8646] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:41:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:41:17.855133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dd469-424a-46be-9fba-f46b19bd2e67 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:41:17.877144 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dd469-424a-46be-9fba-f46b19bd2e67 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:41:17.882458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffebc7-edda-4f78-b159-7f7177fc3d59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:41:17.883222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffebc7-edda-4f78-b159-7f7177fc3d59 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:41:17.883499 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffebc7-edda-4f78-b159-7f7177fc3d59 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:41:17.884125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffebc7-edda-4f78-b159-7f7177fc3d59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:41:17.886995 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-afffebc7-edda-4f78-b159-7f7177fc3d59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:41:17.906426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffebc7-edda-4f78-b159-7f7177fc3d59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:41:17.907865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffebc7-edda-4f78-b159-7f7177fc3d59 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:41:17.924033 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4280/8647] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:41:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:41:26.646610 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=13 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:41:26.911127 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51c76d-4146-43bb-8d9e-6872b5a7644e service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:41:26.926908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51c76d-4146-43bb-8d9e-6872b5a7644e service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:41:26.931783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42afc6-d456-450a-a45c-76c35ff72d35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:41:26.932390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42afc6-d456-450a-a45c-76c35ff72d35 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:41:26.932692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42afc6-d456-450a-a45c-76c35ff72d35 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:41:26.932960 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42afc6-d456-450a-a45c-76c35ff72d35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:41:26.934886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c42afc6-d456-450a-a45c-76c35ff72d35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:41:26.963500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42afc6-d456-450a-a45c-76c35ff72d35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:41:26.965769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42afc6-d456-450a-a45c-76c35ff72d35 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:41:26.979927 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4368/8648] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:41:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:41:27.770395 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffebc7-edda-4f78-b159-7f7177fc3d59 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:41:27.782432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffebc7-edda-4f78-b159-7f7177fc3d59 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:41:27.786922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047cce4-7afc-439f-a47d-a752645d7e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:41:27.787533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047cce4-7afc-439f-a47d-a752645d7e80 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:41:27.787810 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047cce4-7afc-439f-a47d-a752645d7e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:41:27.788075 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047cce4-7afc-439f-a47d-a752645d7e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:41:27.790569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9047cce4-7afc-439f-a47d-a752645d7e80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:41:27.803853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047cce4-7afc-439f-a47d-a752645d7e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:41:27.805547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047cce4-7afc-439f-a47d-a752645d7e80 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:41:27.816915 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4281/8649] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:41:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:41:27.899468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42afc6-d456-450a-a45c-76c35ff72d35 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:41:27.914825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42afc6-d456-450a-a45c-76c35ff72d35 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:41:27.919912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca4090-1746-48f2-ad9c-6e555d8e1b35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:41:27.920549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca4090-1746-48f2-ad9c-6e555d8e1b35 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:41:27.920826 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca4090-1746-48f2-ad9c-6e555d8e1b35 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:41:27.921326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca4090-1746-48f2-ad9c-6e555d8e1b35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:41:27.923262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-c4ca4090-1746-48f2-ad9c-6e555d8e1b35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:41:27.939527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca4090-1746-48f2-ad9c-6e555d8e1b35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:41:27.941184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca4090-1746-48f2-ad9c-6e555d8e1b35 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:41:27.955606 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4369/8650] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:41:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:41:37.812697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=15 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:41:37.947860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=17 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:41:44.227745 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ef755-2a01-4790-8f78-24ca92b423a2 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:41:44.228251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ef755-2a01-4790-8f78-24ca92b423a2 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:41:44.228251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ef755-2a01-4790-8f78-24ca92b423a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:41:44.248805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-396ef755-2a01-4790-8f78-24ca92b423a2 None None] MFA Rules not processed for user `c0447084bdd24f549ecfc88b9cc7b3be`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:41:44.250010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-396ef755-2a01-4790-8f78-24ca92b423a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:41:44.257200 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4282/8651] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:41:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:41:44.599359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca4090-1746-48f2-ad9c-6e555d8e1b35 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:41:44.614107 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca4090-1746-48f2-ad9c-6e555d8e1b35 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:41:44.618924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e6ae-4be0-495b-b8b5-3fdd4ab03d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:41:44.619524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e6ae-4be0-495b-b8b5-3fdd4ab03d16 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:41:44.619801 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e6ae-4be0-495b-b8b5-3fdd4ab03d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:41:44.620062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e6ae-4be0-495b-b8b5-3fdd4ab03d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:41:44.622289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6928e6ae-4be0-495b-b8b5-3fdd4ab03d16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:41:44.635088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928e6ae-4be0-495b-b8b5-3fdd4ab03d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:41:44.636741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928e6ae-4be0-495b-b8b5-3fdd4ab03d16 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:41:44.647693 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4370/8652] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:41:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:41:54.243246 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:41:54.641355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:43:09.532991 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ef755-2a01-4790-8f78-24ca92b423a2 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:43:09.550282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ef755-2a01-4790-8f78-24ca92b423a2 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:43:09.557451 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3ea89-58b4-4560-991d-aedb4a94df6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:43:09.558034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3ea89-58b4-4560-991d-aedb4a94df6c service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:43:09.558336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3ea89-58b4-4560-991d-aedb4a94df6c service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:43:09.558588 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3ea89-58b4-4560-991d-aedb4a94df6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:43:09.567044 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3ea89-58b4-4560-991d-aedb4a94df6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:43:09.568737 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3ea89-58b4-4560-991d-aedb4a94df6c service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:43:09.577799 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4283/8653] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:43:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:43:10.065123 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613c373-21d6-4871-a7c9-8dcf6b0dd74d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:43:10.065614 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613c373-21d6-4871-a7c9-8dcf6b0dd74d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:43:10.065739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613c373-21d6-4871-a7c9-8dcf6b0dd74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:43:10.118391 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-8613c373-21d6-4871-a7c9-8dcf6b0dd74d None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:43:10.119656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8613c373-21d6-4871-a7c9-8dcf6b0dd74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:43:10.131383 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4371/8654] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 21:43:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:43:10.146328 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3ea89-58b4-4560-991d-aedb4a94df6c service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:43:10.162380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3ea89-58b4-4560-991d-aedb4a94df6c service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:43:10.168959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0eaa10-5da9-4bbf-a1af-0e18d4f6c76d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:43:10.169575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0eaa10-5da9-4bbf-a1af-0e18d4f6c76d service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:43:10.169844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0eaa10-5da9-4bbf-a1af-0e18d4f6c76d service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:43:10.170106 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0eaa10-5da9-4bbf-a1af-0e18d4f6c76d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:43:10.179015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0eaa10-5da9-4bbf-a1af-0e18d4f6c76d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:43:10.180712 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0eaa10-5da9-4bbf-a1af-0e18d4f6c76d service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:43:10.190763 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4284/8655] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:43:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:43:20.115275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=13 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:43:20.186628 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=12 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:43:28.703520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eec22-95cb-4161-8320-5e9fbb7dc487 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:43:28.704426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eec22-95cb-4161-8320-5e9fbb7dc487 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:43:28.704426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eec22-95cb-4161-8320-5e9fbb7dc487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:43:28.744515 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fd4eec22-95cb-4161-8320-5e9fbb7dc487 None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:43:28.746255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fd4eec22-95cb-4161-8320-5e9fbb7dc487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:43:28.755657 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4372/8656] 10.222.0.23 () {58 vars in 1062 bytes} [Sat Aug 20 21:43:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:43:28.761887 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0eaa10-5da9-4bbf-a1af-0e18d4f6c76d service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:43:28.771357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0eaa10-5da9-4bbf-a1af-0e18d4f6c76d service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:43:28.775423 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f68286-5e9f-4c71-ad3f-250983c9c687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:43:28.776019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f68286-5e9f-4c71-ad3f-250983c9c687 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:43:28.776312 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f68286-5e9f-4c71-ad3f-250983c9c687 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:43:28.776571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f68286-5e9f-4c71-ad3f-250983c9c687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:43:28.778504 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-23f68286-5e9f-4c71-ad3f-250983c9c687 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:43:28.790952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f68286-5e9f-4c71-ad3f-250983c9c687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:43:28.792402 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f68286-5e9f-4c71-ad3f-250983c9c687 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:43:28.803015 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4285/8657] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:43:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:43:30.638207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4eec22-95cb-4161-8320-5e9fbb7dc487 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:43:30.654431 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4eec22-95cb-4161-8320-5e9fbb7dc487 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:43:30.659587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b28d59-86dc-4384-bc20-d083f22cc1ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:43:30.660183 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b28d59-86dc-4384-bc20-d083f22cc1ae service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:43:30.660566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b28d59-86dc-4384-bc20-d083f22cc1ae service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:43:30.660836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b28d59-86dc-4384-bc20-d083f22cc1ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:43:30.671988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b28d59-86dc-4384-bc20-d083f22cc1ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:43:30.673615 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b28d59-86dc-4384-bc20-d083f22cc1ae service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:43:30.686565 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4373/8658] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:43:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:43:35.625905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f68286-5e9f-4c71-ad3f-250983c9c687 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:43:35.645948 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f68286-5e9f-4c71-ad3f-250983c9c687 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:43:35.653605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfaee8-2a43-4de3-8ca5-1f03d7fba1ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:43:35.654173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfaee8-2a43-4de3-8ca5-1f03d7fba1ae service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:43:35.654516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfaee8-2a43-4de3-8ca5-1f03d7fba1ae service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:43:35.654868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfaee8-2a43-4de3-8ca5-1f03d7fba1ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:43:35.657220 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c8dfaee8-2a43-4de3-8ca5-1f03d7fba1ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:43:35.675107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfaee8-2a43-4de3-8ca5-1f03d7fba1ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:43:35.676801 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfaee8-2a43-4de3-8ca5-1f03d7fba1ae service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:43:35.692189 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4286/8659] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:43:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:43:40.679601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=15 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:43:44.268553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1929c1-7e19-43ed-b34b-563d565c4ac7 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:43:44.269076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1929c1-7e19-43ed-b34b-563d565c4ac7 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:43:44.269076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1929c1-7e19-43ed-b34b-563d565c4ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:43:44.289482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7d1929c1-7e19-43ed-b34b-563d565c4ac7 None None] MFA Rules not processed for user `c0447084bdd24f549ecfc88b9cc7b3be`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:43:44.290785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7d1929c1-7e19-43ed-b34b-563d565c4ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:43:44.298196 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4374/8660] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:43:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:43:44.632906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f5df2-2c6b-43ed-97df-66d110bbc272 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:43:44.633445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f5df2-2c6b-43ed-97df-66d110bbc272 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:43:44.633445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f5df2-2c6b-43ed-97df-66d110bbc272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:43:44.653607 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4f8f5df2-2c6b-43ed-97df-66d110bbc272 None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:43:44.654792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4f8f5df2-2c6b-43ed-97df-66d110bbc272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:43:44.661727 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4287/8661] 10.222.0.23 () {58 vars in 1062 bytes} [Sat Aug 20 21:43:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:43:44.668305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1929c1-7e19-43ed-b34b-563d565c4ac7 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:43:44.680091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1929c1-7e19-43ed-b34b-563d565c4ac7 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:43:44.685166 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aade2cb-3082-4981-ba16-241653da50da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:43:44.685775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aade2cb-3082-4981-ba16-241653da50da service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:43:44.686038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aade2cb-3082-4981-ba16-241653da50da service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:43:44.686305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aade2cb-3082-4981-ba16-241653da50da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:43:44.688525 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6aade2cb-3082-4981-ba16-241653da50da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:43:44.701865 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aade2cb-3082-4981-ba16-241653da50da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:43:44.703698 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aade2cb-3082-4981-ba16-241653da50da service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:43:44.715222 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4375/8662] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:43:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:43:45.045330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7559d-6fd0-44d6-8307-d900942d17ef None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:43:45.045603 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7559d-6fd0-44d6-8307-d900942d17ef None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:43:45.045891 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7559d-6fd0-44d6-8307-d900942d17ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:43:45.080546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-31e7559d-6fd0-44d6-8307-d900942d17ef None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:43:45.081779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-31e7559d-6fd0-44d6-8307-d900942d17ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:43:45.090740 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4288/8663] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 21:43:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:43:45.103756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aade2cb-3082-4981-ba16-241653da50da service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:43:45.119361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aade2cb-3082-4981-ba16-241653da50da service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:43:45.124679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90729b-f184-4a2b-b513-3a00551f1fb1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:43:45.125609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90729b-f184-4a2b-b513-3a00551f1fb1 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:43:45.125881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90729b-f184-4a2b-b513-3a00551f1fb1 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:43:45.126138 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90729b-f184-4a2b-b513-3a00551f1fb1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:43:45.134990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90729b-f184-4a2b-b513-3a00551f1fb1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:43:45.136611 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90729b-f184-4a2b-b513-3a00551f1fb1 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:43:45.146710 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4376/8664] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:43:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:43:55.075161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=23 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:43:55.142422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=15 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:43:56.812835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7559d-6fd0-44d6-8307-d900942d17ef None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:43:56.828349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7559d-6fd0-44d6-8307-d900942d17ef None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:43:56.835320 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a23db-4200-4aca-8c8d-b9918818d791 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:43:56.836163 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a23db-4200-4aca-8c8d-b9918818d791 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:43:56.836434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a23db-4200-4aca-8c8d-b9918818d791 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:43:56.836702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a23db-4200-4aca-8c8d-b9918818d791 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:43:56.851377 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a23db-4200-4aca-8c8d-b9918818d791 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:43:56.853061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a23db-4200-4aca-8c8d-b9918818d791 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:43:56.867651 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4289/8665] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:43:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:44:06.860566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:44:23.143569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90729b-f184-4a2b-b513-3a00551f1fb1 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:44:23.160639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90729b-f184-4a2b-b513-3a00551f1fb1 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:44:23.167535 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366b56e-9b25-433d-9414-52328fd7de13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:44:23.168124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366b56e-9b25-433d-9414-52328fd7de13 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:23.168418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366b56e-9b25-433d-9414-52328fd7de13 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:23.168707 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366b56e-9b25-433d-9414-52328fd7de13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:23.180231 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366b56e-9b25-433d-9414-52328fd7de13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:44:23.182145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366b56e-9b25-433d-9414-52328fd7de13 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:44:23.194941 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4377/8666] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:44:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:44:33.188082 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:44:38.332758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a23db-4200-4aca-8c8d-b9918818d791 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:44:38.342468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a23db-4200-4aca-8c8d-b9918818d791 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:44:38.346718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e4e79-8692-45f9-ad17-08d5223512b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:44:38.347313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e4e79-8692-45f9-ad17-08d5223512b4 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:38.347585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e4e79-8692-45f9-ad17-08d5223512b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:38.347854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e4e79-8692-45f9-ad17-08d5223512b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:38.357502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e4e79-8692-45f9-ad17-08d5223512b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:44:38.359217 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e4e79-8692-45f9-ad17-08d5223512b4 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:44:38.369016 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4290/8667] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:44:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:44:41.708687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366b56e-9b25-433d-9414-52328fd7de13 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:44:41.725553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366b56e-9b25-433d-9414-52328fd7de13 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:44:41.731524 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c972d-6c01-43ee-9d89-d762d55a3984 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:44:41.732425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c972d-6c01-43ee-9d89-d762d55a3984 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:41.732807 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c972d-6c01-43ee-9d89-d762d55a3984 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:41.733308 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c972d-6c01-43ee-9d89-d762d55a3984 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:41.736140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0c4c972d-6c01-43ee-9d89-d762d55a3984 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:44:41.752836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c972d-6c01-43ee-9d89-d762d55a3984 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:44:41.755044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c972d-6c01-43ee-9d89-d762d55a3984 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:44:41.770996 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4378/8668] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:44:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:44:42.136125 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e4e79-8692-45f9-ad17-08d5223512b4 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:44:42.146713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e4e79-8692-45f9-ad17-08d5223512b4 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:44:42.154693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9b1ec-1845-47d5-be94-e3107885f24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:44:42.155674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9b1ec-1845-47d5-be94-e3107885f24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:42.155946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9b1ec-1845-47d5-be94-e3107885f24f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:42.156186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9b1ec-1845-47d5-be94-e3107885f24f admin admin] PATH_INFO: `/v3/users/c0447084bdd24f549ecfc88b9cc7b3be` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:42.157498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9b1ec-1845-47d5-be94-e3107885f24f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0447084bdd24f549ecfc88b9cc7b3be)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:44:42.159444 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9b1ec-1845-47d5-be94-e3107885f24f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:44:42.211034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ff9b1ec-1845-47d5-be94-e3107885f24f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0447084bdd24f549ecfc88b9cc7b3be'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:44:42.217649 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ff9b1ec-1845-47d5-be94-e3107885f24f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0447084bdd24f549ecfc88b9cc7b3be'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:44:42.233082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ff9b1ec-1845-47d5-be94-e3107885f24f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0447084bdd24f549ecfc88b9cc7b3be'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:44:42.234452 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ff9b1ec-1845-47d5-be94-e3107885f24f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0447084bdd24f549ecfc88b9cc7b3be'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:44:42.235120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-3ff9b1ec-1845-47d5-be94-e3107885f24f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0447084bdd24f549ecfc88b9cc7b3be'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:44:42.248848 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4291/8669] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:44:42 2022] DELETE /identity/v3/users/c0447084bdd24f549ecfc88b9cc7b3be => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:44:42.449528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c972d-6c01-43ee-9d89-d762d55a3984 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:44:42.466053 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c972d-6c01-43ee-9d89-d762d55a3984 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:44:42.527323 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:44:42.527934 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:42.528220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:42.528497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2 admin admin] PATH_INFO: `/v3/projects/10e1a87a100a4dcb9a30a6c4d35ff429` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:42.529617 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e1a87a100a4dcb9a30a6c4d35ff429)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:44:42.531650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:44:42.585559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e1a87a100a4dcb9a30a6c4d35ff429', 'request_id': 'req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:44:42.594897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e1a87a100a4dcb9a30a6c4d35ff429', 'request_id': 'req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:44:42.600542 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e1a87a100a4dcb9a30a6c4d35ff429', 'request_id': 'req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:44:42.612470 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2 admin admin] The token cache is being invalidate because project 10e1a87a100a4dcb9a30a6c4d35ff429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:44:42.612715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:44:42.613447 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-fcf39ac1-46bc-44c0-9b8a-db8058aed5e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:44:42.615081 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4379/8670] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:44:42 2022] DELETE /identity/v3/projects/10e1a87a100a4dcb9a30a6c4d35ff429 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:44:42.620697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53171480-0f53-4929-871b-7c7399e38a11 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:42.620981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53171480-0f53-4929-871b-7c7399e38a11 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:42.621252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53171480-0f53-4929-871b-7c7399e38a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:42.643092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-53171480-0f53-4929-871b-7c7399e38a11 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:44:42.644296 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-53171480-0f53-4929-871b-7c7399e38a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:44:42.710361 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4292/8671] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:44:42.718847 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacdd6b-6b5a-40cf-b918-3fc1d8be3f23 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:42.719361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacdd6b-6b5a-40cf-b918-3fc1d8be3f23 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:42.719361 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacdd6b-6b5a-40cf-b918-3fc1d8be3f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:42.741235 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-cdacdd6b-6b5a-40cf-b918-3fc1d8be3f23 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:44:42.742430 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cdacdd6b-6b5a-40cf-b918-3fc1d8be3f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:44:42.753374 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4380/8672] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:44:42.757732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53171480-0f53-4929-871b-7c7399e38a11 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:44:42.768555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53171480-0f53-4929-871b-7c7399e38a11 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:44:42.774355 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6c90a-2982-44c9-8f17-cc2537cf81ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:44:42.775085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6c90a-2982-44c9-8f17-cc2537cf81ea admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:42.775358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6c90a-2982-44c9-8f17-cc2537cf81ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:42.775646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6c90a-2982-44c9-8f17-cc2537cf81ea admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:42.776281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6c90a-2982-44c9-8f17-cc2537cf81ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:44:42.776591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6c90a-2982-44c9-8f17-cc2537cf81ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:44:42.778195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6c90a-2982-44c9-8f17-cc2537cf81ea admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:44:42.790211 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4293/8673] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:44:42.794504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacdd6b-6b5a-40cf-b918-3fc1d8be3f23 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:44:42.803861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacdd6b-6b5a-40cf-b918-3fc1d8be3f23 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:44:42.808551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ac381-3b4b-4a94-a676-8c4e65c7e40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:44:42.809139 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ac381-3b4b-4a94-a676-8c4e65c7e40d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:42.809411 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ac381-3b4b-4a94-a676-8c4e65c7e40d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:42.809672 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ac381-3b4b-4a94-a676-8c4e65c7e40d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:42.810311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ac381-3b4b-4a94-a676-8c4e65c7e40d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:44:42.812256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ac381-3b4b-4a94-a676-8c4e65c7e40d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:44:42.841606 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4381/8674] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:44:42 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:44:42.846506 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6c90a-2982-44c9-8f17-cc2537cf81ea admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:44:42.857856 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6c90a-2982-44c9-8f17-cc2537cf81ea admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:44:42.905792 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:44:42.920825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0430f-857e-4e70-aec9-e99db760faf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:44:42.921448 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0430f-857e-4e70-aec9-e99db760faf7 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:42.921720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0430f-857e-4e70-aec9-e99db760faf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:42.921989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0430f-857e-4e70-aec9-e99db760faf7 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:42.922606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0430f-857e-4e70-aec9-e99db760faf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:44:42.924568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0430f-857e-4e70-aec9-e99db760faf7 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:44:42.947347 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4294/8675] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:44:42 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:44:42.951727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ac381-3b4b-4a94-a676-8c4e65c7e40d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:44:42.963004 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ac381-3b4b-4a94-a676-8c4e65c7e40d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:44:42.968393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dae759-0a3d-4291-8ee9-9f0d318bfe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:44:42.968995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dae759-0a3d-4291-8ee9-9f0d318bfe4a admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:42.969238 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dae759-0a3d-4291-8ee9-9f0d318bfe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:42.969499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dae759-0a3d-4291-8ee9-9f0d318bfe4a admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:42.970157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dae759-0a3d-4291-8ee9-9f0d318bfe4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:44:42.971732 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dae759-0a3d-4291-8ee9-9f0d318bfe4a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:44:42.982239 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4382/8676] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:44:42.986796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0430f-857e-4e70-aec9-e99db760faf7 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:44:42.998045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0430f-857e-4e70-aec9-e99db760faf7 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:44:43.004605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da21af3-87c8-4cb9-96d7-ef2280e4a596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:44:43.005207 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da21af3-87c8-4cb9-96d7-ef2280e4a596 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:43.005480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da21af3-87c8-4cb9-96d7-ef2280e4a596 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:43.005759 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da21af3-87c8-4cb9-96d7-ef2280e4a596 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:43.006388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da21af3-87c8-4cb9-96d7-ef2280e4a596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:44:43.007994 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da21af3-87c8-4cb9-96d7-ef2280e4a596 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:44:43.022383 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4295/8677] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:44:43.027223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dae759-0a3d-4291-8ee9-9f0d318bfe4a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:44:43.038079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dae759-0a3d-4291-8ee9-9f0d318bfe4a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:44:43.042626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99062345-c3d1-4e81-9d82-00cf3b85b0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:44:43.043274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99062345-c3d1-4e81-9d82-00cf3b85b0e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:43.043543 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99062345-c3d1-4e81-9d82-00cf3b85b0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:43.043806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99062345-c3d1-4e81-9d82-00cf3b85b0e6 admin admin] PATH_INFO: `/v3/projects/daff5f0d6dbc4a23ac62a0d99075ae56/users/2eeeedcd7ad54c6b9620beb76433a40c/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:43.058968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99062345-c3d1-4e81-9d82-00cf3b85b0e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daff5f0d6dbc4a23ac62a0d99075ae56, user_id=2eeeedcd7ad54c6b9620beb76433a40c, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:44:43.061859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99062345-c3d1-4e81-9d82-00cf3b85b0e6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:44:43.075657 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4383/8678] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:44:43 2022] PUT /identity/v3/projects/daff5f0d6dbc4a23ac62a0d99075ae56/users/2eeeedcd7ad54c6b9620beb76433a40c/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:44:43.081294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5080507-e5aa-4ee8-87f7-6e8ec1bfaaf6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:43.081570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5080507-e5aa-4ee8-87f7-6e8ec1bfaaf6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:43.081834 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5080507-e5aa-4ee8-87f7-6e8ec1bfaaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:43.101161 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f5080507-e5aa-4ee8-87f7-6e8ec1bfaaf6 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:44:43.102371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f5080507-e5aa-4ee8-87f7-6e8ec1bfaaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:44:43.165619 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4296/8679] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:44:43.176251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99062345-c3d1-4e81-9d82-00cf3b85b0e6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:44:43.178059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-99062345-c3d1-4e81-9d82-00cf3b85b0e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:44:43.193685 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:44:43.244011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99062345-c3d1-4e81-9d82-00cf3b85b0e6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:44:43.248210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27b4d9-d396-45a0-9b19-0de517b889a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:44:43.248890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b4d9-d396-45a0-9b19-0de517b889a5 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:43.249186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b4d9-d396-45a0-9b19-0de517b889a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:43.249451 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b4d9-d396-45a0-9b19-0de517b889a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:43.259269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27b4d9-d396-45a0-9b19-0de517b889a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:44:43.260764 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27b4d9-d396-45a0-9b19-0de517b889a5 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:44:43.270893 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4384/8680] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:44:47.120195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca167-ed49-44c6-974c-d2b9a3b5fd1e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:47.120795 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca167-ed49-44c6-974c-d2b9a3b5fd1e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:47.121009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca167-ed49-44c6-974c-d2b9a3b5fd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:47.151846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-f7bca167-ed49-44c6-974c-d2b9a3b5fd1e None None] MFA Rules not processed for user `2eeeedcd7ad54c6b9620beb76433a40c`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:44:47.153744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f7bca167-ed49-44c6-974c-d2b9a3b5fd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:44:47.232546 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4297/8681] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:44:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:44:47.303172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970b270-faa9-4654-8f29-5ecc25b5cc9d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:47.303663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970b270-faa9-4654-8f29-5ecc25b5cc9d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:47.303762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970b270-faa9-4654-8f29-5ecc25b5cc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:47.324089 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9970b270-faa9-4654-8f29-5ecc25b5cc9d None None] MFA Rules not processed for user `2eeeedcd7ad54c6b9620beb76433a40c`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:44:47.325163 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9970b270-faa9-4654-8f29-5ecc25b5cc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:44:47.332743 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4385/8682] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:44:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:44:47.345077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bca167-ed49-44c6-974c-d2b9a3b5fd1e None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:44:47.346702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f7bca167-ed49-44c6-974c-d2b9a3b5fd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:44:47.411841 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:44:47.452671 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bca167-ed49-44c6-974c-d2b9a3b5fd1e None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:44:47.459038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372adfb-d6ea-4d01-bb73-994a17d6c1a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:44:47.459653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372adfb-d6ea-4d01-bb73-994a17d6c1a3 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:47.459924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372adfb-d6ea-4d01-bb73-994a17d6c1a3 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:47.460192 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372adfb-d6ea-4d01-bb73-994a17d6c1a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:47.469219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372adfb-d6ea-4d01-bb73-994a17d6c1a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:44:47.471174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372adfb-d6ea-4d01-bb73-994a17d6c1a3 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:44:47.480614 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4298/8683] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:44:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:44:57.318133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=23 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:44:57.480882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=17 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:44:58.600751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9954c03-a7c5-4024-a6f8-8026f656404b None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:44:58.601347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9954c03-a7c5-4024-a6f8-8026f656404b None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:44:58.601347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9954c03-a7c5-4024-a6f8-8026f656404b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:44:58.620547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f9954c03-a7c5-4024-a6f8-8026f656404b None None] MFA Rules not processed for user `2eeeedcd7ad54c6b9620beb76433a40c`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:44:58.621758 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f9954c03-a7c5-4024-a6f8-8026f656404b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:44:58.629626 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4386/8684] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:44:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:01.936338 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372adfb-d6ea-4d01-bb73-994a17d6c1a3 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:01.955367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372adfb-d6ea-4d01-bb73-994a17d6c1a3 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:01.962480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ef6fa-c722-4cfd-ab90-1687d1a0508c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:01.963605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ef6fa-c722-4cfd-ab90-1687d1a0508c service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:01.963867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ef6fa-c722-4cfd-ab90-1687d1a0508c service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:01.964130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ef6fa-c722-4cfd-ab90-1687d1a0508c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:01.966004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ac0ef6fa-c722-4cfd-ab90-1687d1a0508c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:01.982390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ef6fa-c722-4cfd-ab90-1687d1a0508c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:01.983983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ef6fa-c722-4cfd-ab90-1687d1a0508c service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:01.997656 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4299/8685] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:45:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:45:08.615419 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:11.990621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:20.785926 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9954c03-a7c5-4024-a6f8-8026f656404b None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:20.787885 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f9954c03-a7c5-4024-a6f8-8026f656404b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:20.862760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ef6fa-c722-4cfd-ab90-1687d1a0508c service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:20.879547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9954c03-a7c5-4024-a6f8-8026f656404b None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:20.882410 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ef6fa-c722-4cfd-ab90-1687d1a0508c service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:20.886056 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7c2f4-7e7a-4359-86f6-bab59f19d710 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:20.886683 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7c2f4-7e7a-4359-86f6-bab59f19d710 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:20.886954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7c2f4-7e7a-4359-86f6-bab59f19d710 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:20.887218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7c2f4-7e7a-4359-86f6-bab59f19d710 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:20.889049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d975c3-abb0-4d36-bd74-7a3e85312f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:20.889203 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dae7c2f4-7e7a-4359-86f6-bab59f19d710 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:20.889985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d975c3-abb0-4d36-bd74-7a3e85312f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:20.890261 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d975c3-abb0-4d36-bd74-7a3e85312f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:20.890534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d975c3-abb0-4d36-bd74-7a3e85312f7c admin admin] PATH_INFO: `/v3/users/2eeeedcd7ad54c6b9620beb76433a40c` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:20.891693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d975c3-abb0-4d36-bd74-7a3e85312f7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eeeedcd7ad54c6b9620beb76433a40c)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:20.893677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d975c3-abb0-4d36-bd74-7a3e85312f7c admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:20.905568 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae7c2f4-7e7a-4359-86f6-bab59f19d710 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:20.907751 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae7c2f4-7e7a-4359-86f6-bab59f19d710 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:20.921149 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4387/8686] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:45:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:45:20.949460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-81d975c3-abb0-4d36-bd74-7a3e85312f7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eeeedcd7ad54c6b9620beb76433a40c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:20.959596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-81d975c3-abb0-4d36-bd74-7a3e85312f7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eeeedcd7ad54c6b9620beb76433a40c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:20.972864 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-81d975c3-abb0-4d36-bd74-7a3e85312f7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eeeedcd7ad54c6b9620beb76433a40c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:20.973534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-81d975c3-abb0-4d36-bd74-7a3e85312f7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eeeedcd7ad54c6b9620beb76433a40c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:20.974126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-81d975c3-abb0-4d36-bd74-7a3e85312f7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eeeedcd7ad54c6b9620beb76433a40c'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:20.986111 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4300/8687] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:45:20 2022] DELETE /identity/v3/users/2eeeedcd7ad54c6b9620beb76433a40c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:45:21.166322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7c2f4-7e7a-4359-86f6-bab59f19d710 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:21.182184 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7c2f4-7e7a-4359-86f6-bab59f19d710 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:21.256924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:21.257920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:21.258360 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:21.258745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a admin admin] PATH_INFO: `/v3/projects/daff5f0d6dbc4a23ac62a0d99075ae56` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:21.260651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a admin admin] RBAC: Authorizing `identity:delete_project(project_id=daff5f0d6dbc4a23ac62a0d99075ae56)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:21.263547 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:21.323448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daff5f0d6dbc4a23ac62a0d99075ae56', 'request_id': 'req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:21.333552 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daff5f0d6dbc4a23ac62a0d99075ae56', 'request_id': 'req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:21.339068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daff5f0d6dbc4a23ac62a0d99075ae56', 'request_id': 'req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:21.362218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:21.365592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a admin admin] The token cache is being invalidate because project daff5f0d6dbc4a23ac62a0d99075ae56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:45:21.365902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:21.366816 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-669cc2f6-c884-4fa8-86ba-2ebda9e1a78a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:21.368476 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4388/8688] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:45:21 2022] DELETE /identity/v3/projects/daff5f0d6dbc4a23ac62a0d99075ae56 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:45:21.375053 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3caa1-d9c1-4bb8-acce-9080e2f48df1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:21.375538 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3caa1-d9c1-4bb8-acce-9080e2f48df1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:21.375965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3caa1-d9c1-4bb8-acce-9080e2f48df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:21.436758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-18d3caa1-d9c1-4bb8-acce-9080e2f48df1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:21.437967 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-18d3caa1-d9c1-4bb8-acce-9080e2f48df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:21.503811 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4301/8689] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:21.513382 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549213-b5b6-44b3-bf41-c7f46f940f4d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:21.513666 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549213-b5b6-44b3-bf41-c7f46f940f4d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:21.514032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549213-b5b6-44b3-bf41-c7f46f940f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:21.535314 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5a549213-b5b6-44b3-bf41-c7f46f940f4d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:21.536565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5a549213-b5b6-44b3-bf41-c7f46f940f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:21.546029 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4389/8690] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:21.550529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3caa1-d9c1-4bb8-acce-9080e2f48df1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:21.560814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3caa1-d9c1-4bb8-acce-9080e2f48df1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:21.567585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf6a0e-8664-4a1f-906d-b92240bb3bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:21.568206 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf6a0e-8664-4a1f-906d-b92240bb3bbd admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:21.568522 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf6a0e-8664-4a1f-906d-b92240bb3bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:21.568785 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf6a0e-8664-4a1f-906d-b92240bb3bbd admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:21.569426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf6a0e-8664-4a1f-906d-b92240bb3bbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:45:21.569723 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf6a0e-8664-4a1f-906d-b92240bb3bbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:21.571351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf6a0e-8664-4a1f-906d-b92240bb3bbd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:21.585264 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4302/8691] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:45:21.589739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a549213-b5b6-44b3-bf41-c7f46f940f4d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:21.600063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a549213-b5b6-44b3-bf41-c7f46f940f4d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:21.606026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6874c4d-4d95-4086-baa5-54434b27463e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:21.606619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6874c4d-4d95-4086-baa5-54434b27463e admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:21.606906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6874c4d-4d95-4086-baa5-54434b27463e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:21.607142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6874c4d-4d95-4086-baa5-54434b27463e admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:21.607750 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6874c4d-4d95-4086-baa5-54434b27463e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:21.609692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6874c4d-4d95-4086-baa5-54434b27463e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:21.643303 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4390/8692] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:45:21 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:45:21.648013 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf6a0e-8664-4a1f-906d-b92240bb3bbd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:21.658231 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf6a0e-8664-4a1f-906d-b92240bb3bbd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:21.685343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:21.722287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb3d2c-74c7-412d-90d7-e1b796ea0e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:21.723014 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb3d2c-74c7-412d-90d7-e1b796ea0e02 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:21.723361 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb3d2c-74c7-412d-90d7-e1b796ea0e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:21.723632 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb3d2c-74c7-412d-90d7-e1b796ea0e02 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:21.724278 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb3d2c-74c7-412d-90d7-e1b796ea0e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:21.726219 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb3d2c-74c7-412d-90d7-e1b796ea0e02 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:21.747243 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4303/8693] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:45:21 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:45:21.751726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6874c4d-4d95-4086-baa5-54434b27463e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:21.762590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6874c4d-4d95-4086-baa5-54434b27463e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:21.768348 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85992fe3-5a08-4135-99a5-6afac0c233cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:21.769091 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85992fe3-5a08-4135-99a5-6afac0c233cd admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:21.770058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85992fe3-5a08-4135-99a5-6afac0c233cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:21.770255 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85992fe3-5a08-4135-99a5-6afac0c233cd admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:21.771098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85992fe3-5a08-4135-99a5-6afac0c233cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:21.773442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85992fe3-5a08-4135-99a5-6afac0c233cd admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:21.788317 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4391/8694] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:45:21.792530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb3d2c-74c7-412d-90d7-e1b796ea0e02 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:21.803635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb3d2c-74c7-412d-90d7-e1b796ea0e02 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:21.809418 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b669a3-6f24-44a4-862b-c9c5e98bb471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:21.810018 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b669a3-6f24-44a4-862b-c9c5e98bb471 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:21.810290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b669a3-6f24-44a4-862b-c9c5e98bb471 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:21.810550 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b669a3-6f24-44a4-862b-c9c5e98bb471 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:21.811181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b669a3-6f24-44a4-862b-c9c5e98bb471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:21.812885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b669a3-6f24-44a4-862b-c9c5e98bb471 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:21.824760 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4304/8695] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:45:21.829095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85992fe3-5a08-4135-99a5-6afac0c233cd admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:21.840856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85992fe3-5a08-4135-99a5-6afac0c233cd admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:21.846917 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40047ae5-36a7-4aa1-8e84-d4b885b9c07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:21.847857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40047ae5-36a7-4aa1-8e84-d4b885b9c07b admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:21.847857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40047ae5-36a7-4aa1-8e84-d4b885b9c07b admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:21.848146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40047ae5-36a7-4aa1-8e84-d4b885b9c07b admin admin] PATH_INFO: `/v3/projects/8d88dd9064f94b90aacb6103002aa259/users/32ddc14224d44ab1bf9cde7f142da351/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:21.864602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40047ae5-36a7-4aa1-8e84-d4b885b9c07b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d88dd9064f94b90aacb6103002aa259, user_id=32ddc14224d44ab1bf9cde7f142da351, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:21.867300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40047ae5-36a7-4aa1-8e84-d4b885b9c07b admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:21.882354 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4392/8696] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:45:21 2022] PUT /identity/v3/projects/8d88dd9064f94b90aacb6103002aa259/users/32ddc14224d44ab1bf9cde7f142da351/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:45:21.892885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15717b-966f-4c0c-9238-1231b51a01ad None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:21.893352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15717b-966f-4c0c-9238-1231b51a01ad None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:21.893456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15717b-966f-4c0c-9238-1231b51a01ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:21.922316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8b15717b-966f-4c0c-9238-1231b51a01ad None None] MFA Rules not processed for user `32ddc14224d44ab1bf9cde7f142da351`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:21.923542 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8b15717b-966f-4c0c-9238-1231b51a01ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:21.995656 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4305/8697] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:22.006368 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40047ae5-36a7-4aa1-8e84-d4b885b9c07b admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:22.008396 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-40047ae5-36a7-4aa1-8e84-d4b885b9c07b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:22.067539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40047ae5-36a7-4aa1-8e84-d4b885b9c07b admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:22.071437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae0b7e-3690-4dd2-aea5-e84aca79ed93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:22.072048 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae0b7e-3690-4dd2-aea5-e84aca79ed93 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:22.072316 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae0b7e-3690-4dd2-aea5-e84aca79ed93 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:22.072608 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae0b7e-3690-4dd2-aea5-e84aca79ed93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:22.081549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae0b7e-3690-4dd2-aea5-e84aca79ed93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:22.083142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae0b7e-3690-4dd2-aea5-e84aca79ed93 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:22.092516 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4393/8698] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:45:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:45:31.992728 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:32.088345 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:32.668846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15717b-966f-4c0c-9238-1231b51a01ad None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:32.739337 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15717b-966f-4c0c-9238-1231b51a01ad None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:32.744030 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:32.744695 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:32.744964 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:32.745238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] PATH_INFO: `/v3/users/32ddc14224d44ab1bf9cde7f142da351` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:32.746390 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32ddc14224d44ab1bf9cde7f142da351)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:32.748307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:32.799154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32ddc14224d44ab1bf9cde7f142da351'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:32.813364 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32ddc14224d44ab1bf9cde7f142da351'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:32.826710 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32ddc14224d44ab1bf9cde7f142da351'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:32.827356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32ddc14224d44ab1bf9cde7f142da351'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:32.827965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32ddc14224d44ab1bf9cde7f142da351'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:32.840832 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4306/8699] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:45:32 2022] DELETE /identity/v3/users/32ddc14224d44ab1bf9cde7f142da351 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:45:32.845513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6f27-82d6-4834-9564-c939325ccdf5 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:32.845793 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6f27-82d6-4834-9564-c939325ccdf5 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:32.846064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6f27-82d6-4834-9564-c939325ccdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:32.866063 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-af1c6f27-82d6-4834-9564-c939325ccdf5 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:32.867265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-af1c6f27-82d6-4834-9564-c939325ccdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:32.936393 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4394/8700] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:32.946835 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:32.948327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:33.008849 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165d72a-66bc-4a00-8070-e1fd5b489d88 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:33.013077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e4ebc-1f52-43ef-8e1c-13a966f504a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:33.013881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e4ebc-1f52-43ef-8e1c-13a966f504a4 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:33.014202 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e4ebc-1f52-43ef-8e1c-13a966f504a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:33.014510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e4ebc-1f52-43ef-8e1c-13a966f504a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:33.024819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e4ebc-1f52-43ef-8e1c-13a966f504a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:33.026290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e4ebc-1f52-43ef-8e1c-13a966f504a4 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:33.036272 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4307/8701] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:45:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:45:33.228632 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c6f27-82d6-4834-9564-c939325ccdf5 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:33.231595 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-af1c6f27-82d6-4834-9564-c939325ccdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:33.245963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c6f27-82d6-4834-9564-c939325ccdf5 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:33.252243 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:33.253012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:33.253256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:33.253566 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40 admin admin] PATH_INFO: `/v3/projects/8d88dd9064f94b90aacb6103002aa259` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:33.255156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d88dd9064f94b90aacb6103002aa259)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:33.257023 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:33.315353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d88dd9064f94b90aacb6103002aa259', 'request_id': 'req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:33.324585 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d88dd9064f94b90aacb6103002aa259', 'request_id': 'req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:33.333384 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d88dd9064f94b90aacb6103002aa259', 'request_id': 'req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:33.344679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40 admin admin] The token cache is being invalidate because project 8d88dd9064f94b90aacb6103002aa259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:45:33.344918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:33.345545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-956be4d9-d1bf-482f-bcd2-b2b2010d8e40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:33.347072 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4395/8702] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:45:33 2022] DELETE /identity/v3/projects/8d88dd9064f94b90aacb6103002aa259 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:45:33.352587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0223e71-20ba-44b0-97e0-2b4f640fe54b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:33.352956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0223e71-20ba-44b0-97e0-2b4f640fe54b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:33.353286 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0223e71-20ba-44b0-97e0-2b4f640fe54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:33.373915 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c0223e71-20ba-44b0-97e0-2b4f640fe54b None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:33.375120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c0223e71-20ba-44b0-97e0-2b4f640fe54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:33.393386 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:33.441665 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4308/8703] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:45:33.449620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae0d48-e367-40e3-8b79-444efddd69ca None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:33.449899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae0d48-e367-40e3-8b79-444efddd69ca None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:33.450186 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae0d48-e367-40e3-8b79-444efddd69ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:33.471785 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9bae0d48-e367-40e3-8b79-444efddd69ca None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:33.473059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9bae0d48-e367-40e3-8b79-444efddd69ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:33.481262 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4396/8704] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:33.485614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0223e71-20ba-44b0-97e0-2b4f640fe54b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:33.496000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0223e71-20ba-44b0-97e0-2b4f640fe54b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:33.501406 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654d567-b4b2-4b45-aedf-222661893912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:33.502023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654d567-b4b2-4b45-aedf-222661893912 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:33.502281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654d567-b4b2-4b45-aedf-222661893912 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:33.502558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654d567-b4b2-4b45-aedf-222661893912 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:33.503203 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654d567-b4b2-4b45-aedf-222661893912 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:45:33.503556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654d567-b4b2-4b45-aedf-222661893912 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:33.505704 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654d567-b4b2-4b45-aedf-222661893912 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:33.517955 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4309/8705] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:45:33.522505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae0d48-e367-40e3-8b79-444efddd69ca None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:33.532468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae0d48-e367-40e3-8b79-444efddd69ca None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:33.538429 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf2c01-5178-420e-b461-9ee05bf5ce6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:33.539020 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf2c01-5178-420e-b461-9ee05bf5ce6d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:33.539290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf2c01-5178-420e-b461-9ee05bf5ce6d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:33.539551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf2c01-5178-420e-b461-9ee05bf5ce6d admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:33.540165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf2c01-5178-420e-b461-9ee05bf5ce6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:33.542114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf2c01-5178-420e-b461-9ee05bf5ce6d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:33.572711 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4397/8706] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:45:33 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:45:33.577914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654d567-b4b2-4b45-aedf-222661893912 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:33.594826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654d567-b4b2-4b45-aedf-222661893912 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:33.657284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56c691-ee98-4318-96f3-713c280f49eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:33.657910 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56c691-ee98-4318-96f3-713c280f49eb admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:33.658191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56c691-ee98-4318-96f3-713c280f49eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:33.658456 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56c691-ee98-4318-96f3-713c280f49eb admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:33.659092 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56c691-ee98-4318-96f3-713c280f49eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:33.661122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56c691-ee98-4318-96f3-713c280f49eb admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:33.684068 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4310/8707] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:45:33 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:45:33.689640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf2c01-5178-420e-b461-9ee05bf5ce6d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:33.700493 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf2c01-5178-420e-b461-9ee05bf5ce6d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:33.706455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992c501-2eca-41e6-b8b4-2a38dd633469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:33.707202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992c501-2eca-41e6-b8b4-2a38dd633469 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:33.707498 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992c501-2eca-41e6-b8b4-2a38dd633469 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:33.707757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992c501-2eca-41e6-b8b4-2a38dd633469 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:33.708407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992c501-2eca-41e6-b8b4-2a38dd633469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:33.710059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992c501-2eca-41e6-b8b4-2a38dd633469 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:33.720914 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4398/8708] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:45:33.726130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56c691-ee98-4318-96f3-713c280f49eb admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:33.742826 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56c691-ee98-4318-96f3-713c280f49eb admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:33.751924 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3e11a-0644-49e1-93c4-06c2cb8448a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:33.752863 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3e11a-0644-49e1-93c4-06c2cb8448a6 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:33.753258 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3e11a-0644-49e1-93c4-06c2cb8448a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:33.753725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3e11a-0644-49e1-93c4-06c2cb8448a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:33.754391 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3e11a-0644-49e1-93c4-06c2cb8448a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:33.755992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3e11a-0644-49e1-93c4-06c2cb8448a6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:33.767428 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4311/8709] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:45:33.771881 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992c501-2eca-41e6-b8b4-2a38dd633469 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:33.782739 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992c501-2eca-41e6-b8b4-2a38dd633469 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:33.788830 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2d3e3-87d4-4cb1-9780-d92ecb4f6226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:33.789478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2d3e3-87d4-4cb1-9780-d92ecb4f6226 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:33.789754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2d3e3-87d4-4cb1-9780-d92ecb4f6226 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:33.789996 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2d3e3-87d4-4cb1-9780-d92ecb4f6226 admin admin] PATH_INFO: `/v3/projects/7117bed938ef4a42ad652035e1e6c8d4/users/295360aa948a42ce8eeae9e5a0fe2972/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:33.807249 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2d3e3-87d4-4cb1-9780-d92ecb4f6226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7117bed938ef4a42ad652035e1e6c8d4, user_id=295360aa948a42ce8eeae9e5a0fe2972, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:33.809964 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2d3e3-87d4-4cb1-9780-d92ecb4f6226 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:33.825011 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4399/8710] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:45:33 2022] PUT /identity/v3/projects/7117bed938ef4a42ad652035e1e6c8d4/users/295360aa948a42ce8eeae9e5a0fe2972/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:45:33.838000 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbc2cf-90bc-4c3a-9211-8c6ed5df7e3f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:33.838281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbc2cf-90bc-4c3a-9211-8c6ed5df7e3f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:33.838570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbc2cf-90bc-4c3a-9211-8c6ed5df7e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:33.868426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-fecbc2cf-90bc-4c3a-9211-8c6ed5df7e3f None None] MFA Rules not processed for user `295360aa948a42ce8eeae9e5a0fe2972`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:33.869574 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-fecbc2cf-90bc-4c3a-9211-8c6ed5df7e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:33.945396 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4312/8711] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:33.956118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2d3e3-87d4-4cb1-9780-d92ecb4f6226 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:33.958387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-54a2d3e3-87d4-4cb1-9780-d92ecb4f6226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:33.981599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:34.020087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2d3e3-87d4-4cb1-9780-d92ecb4f6226 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:34.024077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057917e-8029-4b53-8cf4-82fdfde36989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:34.024705 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057917e-8029-4b53-8cf4-82fdfde36989 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:34.024957 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057917e-8029-4b53-8cf4-82fdfde36989 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:34.025220 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057917e-8029-4b53-8cf4-82fdfde36989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:34.033780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057917e-8029-4b53-8cf4-82fdfde36989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:34.035271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057917e-8029-4b53-8cf4-82fdfde36989 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:34.044050 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4400/8712] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:45:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:45:38.399215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbc2cf-90bc-4c3a-9211-8c6ed5df7e3f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:38.420678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:38.469757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbc2cf-90bc-4c3a-9211-8c6ed5df7e3f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:38.475322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:38.475961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:38.476241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:38.476555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] PATH_INFO: `/v3/users/295360aa948a42ce8eeae9e5a0fe2972` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:38.477703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=295360aa948a42ce8eeae9e5a0fe2972)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:38.479642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:38.529927 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '295360aa948a42ce8eeae9e5a0fe2972'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:38.538281 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '295360aa948a42ce8eeae9e5a0fe2972'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:38.550600 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '295360aa948a42ce8eeae9e5a0fe2972'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:38.551243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '295360aa948a42ce8eeae9e5a0fe2972'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:38.551859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '295360aa948a42ce8eeae9e5a0fe2972'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:38.564243 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4313/8713] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:45:38 2022] DELETE /identity/v3/users/295360aa948a42ce8eeae9e5a0fe2972 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:45:38.569069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612513c-2f7c-4888-821c-795e50c968d2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:38.569551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612513c-2f7c-4888-821c-795e50c968d2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:38.569663 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612513c-2f7c-4888-821c-795e50c968d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:38.590630 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7612513c-2f7c-4888-821c-795e50c968d2 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:38.591898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7612513c-2f7c-4888-821c-795e50c968d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:38.652886 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4401/8714] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:38.664034 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:38.665782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:38.730455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55710462-35b3-4b56-924d-e0db0c0741cd admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:38.735266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc449538-36b2-4f5b-8fa7-0ae79e39a83f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:38.735882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc449538-36b2-4f5b-8fa7-0ae79e39a83f service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:38.736154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc449538-36b2-4f5b-8fa7-0ae79e39a83f service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:38.736449 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc449538-36b2-4f5b-8fa7-0ae79e39a83f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:38.749777 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc449538-36b2-4f5b-8fa7-0ae79e39a83f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:38.751272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc449538-36b2-4f5b-8fa7-0ae79e39a83f service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:38.762529 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4314/8715] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:45:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:45:38.957664 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612513c-2f7c-4888-821c-795e50c968d2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:38.963152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7612513c-2f7c-4888-821c-795e50c968d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:38.984556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612513c-2f7c-4888-821c-795e50c968d2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:38.998474 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caa21e-9bbb-40ec-9835-cb78424ac137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:38.999118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caa21e-9bbb-40ec-9835-cb78424ac137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:38.999387 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caa21e-9bbb-40ec-9835-cb78424ac137 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:38.999647 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caa21e-9bbb-40ec-9835-cb78424ac137 admin admin] PATH_INFO: `/v3/projects/7117bed938ef4a42ad652035e1e6c8d4` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:39.001261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94caa21e-9bbb-40ec-9835-cb78424ac137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7117bed938ef4a42ad652035e1e6c8d4)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:39.002986 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94caa21e-9bbb-40ec-9835-cb78424ac137 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:39.056878 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-94caa21e-9bbb-40ec-9835-cb78424ac137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7117bed938ef4a42ad652035e1e6c8d4', 'request_id': 'req-94caa21e-9bbb-40ec-9835-cb78424ac137'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:39.065719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-94caa21e-9bbb-40ec-9835-cb78424ac137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7117bed938ef4a42ad652035e1e6c8d4', 'request_id': 'req-94caa21e-9bbb-40ec-9835-cb78424ac137'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:39.072910 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-94caa21e-9bbb-40ec-9835-cb78424ac137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7117bed938ef4a42ad652035e1e6c8d4', 'request_id': 'req-94caa21e-9bbb-40ec-9835-cb78424ac137'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:39.099923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-94caa21e-9bbb-40ec-9835-cb78424ac137 admin admin] The token cache is being invalidate because project 7117bed938ef4a42ad652035e1e6c8d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:45:39.100351 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-94caa21e-9bbb-40ec-9835-cb78424ac137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:39.101131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-94caa21e-9bbb-40ec-9835-cb78424ac137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:45:39.102966 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4402/8716] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:45:38 2022] DELETE /identity/v3/projects/7117bed938ef4a42ad652035e1e6c8d4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:45:39.108611 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232f51e-36c0-4b7b-9d73-a21e430924cc None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:39.108897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232f51e-36c0-4b7b-9d73-a21e430924cc None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:39.109171 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232f51e-36c0-4b7b-9d73-a21e430924cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:39.130842 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e232f51e-36c0-4b7b-9d73-a21e430924cc None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:39.132059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e232f51e-36c0-4b7b-9d73-a21e430924cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:39.164683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:39.193249 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4315/8717] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:39.202480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546773b-539f-48fa-8f69-fb48bf5b0243 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:39.202788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546773b-539f-48fa-8f69-fb48bf5b0243 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:39.203054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546773b-539f-48fa-8f69-fb48bf5b0243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:39.224521 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b546773b-539f-48fa-8f69-fb48bf5b0243 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:39.225743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b546773b-539f-48fa-8f69-fb48bf5b0243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:39.235616 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4403/8718] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:39.240190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232f51e-36c0-4b7b-9d73-a21e430924cc None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:39.251189 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232f51e-36c0-4b7b-9d73-a21e430924cc None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:39.256944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27550ab8-abea-412c-8bc3-263f6205c5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:39.257571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27550ab8-abea-412c-8bc3-263f6205c5f6 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:39.257862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27550ab8-abea-412c-8bc3-263f6205c5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:39.258093 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27550ab8-abea-412c-8bc3-263f6205c5f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:39.258722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27550ab8-abea-412c-8bc3-263f6205c5f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:45:39.259010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27550ab8-abea-412c-8bc3-263f6205c5f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:39.260617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27550ab8-abea-412c-8bc3-263f6205c5f6 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:39.273658 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4316/8719] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:45:39.278069 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b546773b-539f-48fa-8f69-fb48bf5b0243 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:39.287951 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b546773b-539f-48fa-8f69-fb48bf5b0243 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:39.293161 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd3231-6765-43af-aaf5-b8a0a90218d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:39.293756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd3231-6765-43af-aaf5-b8a0a90218d3 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:39.294031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd3231-6765-43af-aaf5-b8a0a90218d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:39.294313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd3231-6765-43af-aaf5-b8a0a90218d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:39.294928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd3231-6765-43af-aaf5-b8a0a90218d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:39.296873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd3231-6765-43af-aaf5-b8a0a90218d3 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:39.327705 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4404/8720] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:45:39 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:45:39.333555 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27550ab8-abea-412c-8bc3-263f6205c5f6 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:39.346498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27550ab8-abea-412c-8bc3-263f6205c5f6 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:39.420757 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056177a-adc6-4d7e-908d-b8b7ca32d412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:39.421399 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056177a-adc6-4d7e-908d-b8b7ca32d412 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:39.421686 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056177a-adc6-4d7e-908d-b8b7ca32d412 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:39.422032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056177a-adc6-4d7e-908d-b8b7ca32d412 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:39.422882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056177a-adc6-4d7e-908d-b8b7ca32d412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:39.425702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056177a-adc6-4d7e-908d-b8b7ca32d412 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:39.448147 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4317/8721] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:45:39 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:45:39.452402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd3231-6765-43af-aaf5-b8a0a90218d3 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:39.463490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd3231-6765-43af-aaf5-b8a0a90218d3 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:39.469152 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccff07c-9e30-463b-87d9-7a9810d43368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:39.469748 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccff07c-9e30-463b-87d9-7a9810d43368 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:39.470016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccff07c-9e30-463b-87d9-7a9810d43368 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:39.470264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccff07c-9e30-463b-87d9-7a9810d43368 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:39.470890 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccff07c-9e30-463b-87d9-7a9810d43368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:39.472790 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccff07c-9e30-463b-87d9-7a9810d43368 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:39.483705 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4405/8722] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:45:39.490229 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056177a-adc6-4d7e-908d-b8b7ca32d412 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:39.504340 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056177a-adc6-4d7e-908d-b8b7ca32d412 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:39.509692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceab2ae-b781-474a-90b5-da0c30104509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:39.510316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceab2ae-b781-474a-90b5-da0c30104509 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:39.510582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceab2ae-b781-474a-90b5-da0c30104509 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:39.510847 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceab2ae-b781-474a-90b5-da0c30104509 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:39.511482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceab2ae-b781-474a-90b5-da0c30104509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:39.513115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceab2ae-b781-474a-90b5-da0c30104509 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:39.524143 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4318/8723] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:45:39.528468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccff07c-9e30-463b-87d9-7a9810d43368 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:39.539541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccff07c-9e30-463b-87d9-7a9810d43368 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:39.545055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d826646-c8e2-41dd-83e9-379775549697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:39.545738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d826646-c8e2-41dd-83e9-379775549697 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:39.546005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d826646-c8e2-41dd-83e9-379775549697 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:39.546250 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d826646-c8e2-41dd-83e9-379775549697 admin admin] PATH_INFO: `/v3/projects/d7e11effb3274080abe061ae967dee8b/users/60932094d094426a93a0dfa8c05a12fe/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:39.548675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:39.564290 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d826646-c8e2-41dd-83e9-379775549697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7e11effb3274080abe061ae967dee8b, user_id=60932094d094426a93a0dfa8c05a12fe, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:39.567087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d826646-c8e2-41dd-83e9-379775549697 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:39.583254 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4406/8724] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:45:39 2022] PUT /identity/v3/projects/d7e11effb3274080abe061ae967dee8b/users/60932094d094426a93a0dfa8c05a12fe/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:45:39.588477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04410b68-4071-48c5-af6f-ec4228f1612d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:39.588760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04410b68-4071-48c5-af6f-ec4228f1612d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:39.589024 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04410b68-4071-48c5-af6f-ec4228f1612d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:39.610215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-04410b68-4071-48c5-af6f-ec4228f1612d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:39.611447 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-04410b68-4071-48c5-af6f-ec4228f1612d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:39.674365 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4319/8725] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:39.684273 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d826646-c8e2-41dd-83e9-379775549697 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:39.686261 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5d826646-c8e2-41dd-83e9-379775549697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:39.746304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d826646-c8e2-41dd-83e9-379775549697 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:39.750157 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c625ba-7e0a-47e4-baf9-b37885e04d00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:39.750760 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c625ba-7e0a-47e4-baf9-b37885e04d00 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:39.751010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c625ba-7e0a-47e4-baf9-b37885e04d00 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:39.751303 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c625ba-7e0a-47e4-baf9-b37885e04d00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:39.761136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c625ba-7e0a-47e4-baf9-b37885e04d00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:39.762588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c625ba-7e0a-47e4-baf9-b37885e04d00 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:39.772805 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4407/8726] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:45:43.867356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f71dc-d25c-4df7-b9d7-db1b34651fea None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:43.867852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f71dc-d25c-4df7-b9d7-db1b34651fea None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:43.867974 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f71dc-d25c-4df7-b9d7-db1b34651fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:43.899824 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-9b7f71dc-d25c-4df7-b9d7-db1b34651fea None None] MFA Rules not processed for user `60932094d094426a93a0dfa8c05a12fe`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:43.901170 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9b7f71dc-d25c-4df7-b9d7-db1b34651fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:43.952507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:44.005285 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4320/8727] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:44.010068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c625ba-7e0a-47e4-baf9-b37885e04d00 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:44.023460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c625ba-7e0a-47e4-baf9-b37885e04d00 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:44.028927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f549cb7-f70c-41f1-b0fc-a209e44a9916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:44.029701 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f549cb7-f70c-41f1-b0fc-a209e44a9916 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:44.029835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f549cb7-f70c-41f1-b0fc-a209e44a9916 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:44.030077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f549cb7-f70c-41f1-b0fc-a209e44a9916 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:44.030704 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f549cb7-f70c-41f1-b0fc-a209e44a9916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:44.032693 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f549cb7-f70c-41f1-b0fc-a209e44a9916 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:44.063671 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4408/8728] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:45:44 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:45:44.068048 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f71dc-d25c-4df7-b9d7-db1b34651fea None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:44.080702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f71dc-d25c-4df7-b9d7-db1b34651fea None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:44.142646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69c328-ea91-4ce2-82eb-9443842afea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:44.143268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69c328-ea91-4ce2-82eb-9443842afea4 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:44.143531 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69c328-ea91-4ce2-82eb-9443842afea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:44.143798 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69c328-ea91-4ce2-82eb-9443842afea4 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:44.144534 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69c328-ea91-4ce2-82eb-9443842afea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:44.146651 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69c328-ea91-4ce2-82eb-9443842afea4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:44.172155 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4321/8729] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:45:44 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:45:44.173643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f549cb7-f70c-41f1-b0fc-a209e44a9916 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:44.185122 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f549cb7-f70c-41f1-b0fc-a209e44a9916 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:44.189872 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5047af5-a8eb-44fe-8707-15ec5085c016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:44.190487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5047af5-a8eb-44fe-8707-15ec5085c016 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:44.190763 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5047af5-a8eb-44fe-8707-15ec5085c016 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:44.191033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5047af5-a8eb-44fe-8707-15ec5085c016 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:44.191737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5047af5-a8eb-44fe-8707-15ec5085c016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:44.193398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5047af5-a8eb-44fe-8707-15ec5085c016 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:44.205700 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4409/8730] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:45:44.210218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69c328-ea91-4ce2-82eb-9443842afea4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:44.221067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69c328-ea91-4ce2-82eb-9443842afea4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:44.226004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07035639-e683-4fac-b6de-c5d82544322a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:44.226705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07035639-e683-4fac-b6de-c5d82544322a admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:44.226981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07035639-e683-4fac-b6de-c5d82544322a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:44.227276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07035639-e683-4fac-b6de-c5d82544322a admin admin] PATH_INFO: `/v3/projects/de1d860aa0604eccb62ca6cc55e99cbc/users/ccf2e8f707504bd3acc8040bac0000fd/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:44.244178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07035639-e683-4fac-b6de-c5d82544322a admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1d860aa0604eccb62ca6cc55e99cbc, user_id=ccf2e8f707504bd3acc8040bac0000fd, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:44.246932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07035639-e683-4fac-b6de-c5d82544322a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:44.262219 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4322/8731] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:45:44 2022] PUT /identity/v3/projects/de1d860aa0604eccb62ca6cc55e99cbc/users/ccf2e8f707504bd3acc8040bac0000fd/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:45:44.266522 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5047af5-a8eb-44fe-8707-15ec5085c016 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:44.277326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5047af5-a8eb-44fe-8707-15ec5085c016 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:44.335842 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8d7de-e80d-4b4a-81f6-29a8b97d4d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:44.336504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8d7de-e80d-4b4a-81f6-29a8b97d4d0c admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:44.336783 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8d7de-e80d-4b4a-81f6-29a8b97d4d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:44.337054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8d7de-e80d-4b4a-81f6-29a8b97d4d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:44.337694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8d7de-e80d-4b4a-81f6-29a8b97d4d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:44.339313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8d7de-e80d-4b4a-81f6-29a8b97d4d0c admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:44.352438 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4410/8732] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:45:44.354739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07035639-e683-4fac-b6de-c5d82544322a admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:44.366367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07035639-e683-4fac-b6de-c5d82544322a admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:44.371946 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7bca1-fe96-40c2-9f7d-584bc9eb0bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:44.372843 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7bca1-fe96-40c2-9f7d-584bc9eb0bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:44.373127 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7bca1-fe96-40c2-9f7d-584bc9eb0bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:44.373351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7bca1-fe96-40c2-9f7d-584bc9eb0bb9 admin admin] PATH_INFO: `/v3/projects/de1d860aa0604eccb62ca6cc55e99cbc/users/ccf2e8f707504bd3acc8040bac0000fd/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:44.375511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7bca1-fe96-40c2-9f7d-584bc9eb0bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1d860aa0604eccb62ca6cc55e99cbc, user_id=ccf2e8f707504bd3acc8040bac0000fd, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:44.378252 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7bca1-fe96-40c2-9f7d-584bc9eb0bb9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:44.390266 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4323/8733] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:45:44 2022] PUT /identity/v3/projects/de1d860aa0604eccb62ca6cc55e99cbc/users/ccf2e8f707504bd3acc8040bac0000fd/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:45:48.797074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f752d1e-1f0b-43f9-8873-a784d9010ca3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:48.797545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f752d1e-1f0b-43f9-8873-a784d9010ca3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:48.797641 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f752d1e-1f0b-43f9-8873-a784d9010ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:48.826284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5f752d1e-1f0b-43f9-8873-a784d9010ca3 None None] MFA Rules not processed for user `ccf2e8f707504bd3acc8040bac0000fd`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:48.827448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5f752d1e-1f0b-43f9-8873-a784d9010ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:48.869088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:48.901087 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4411/8734] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:48.938157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f838fd-3720-45d4-9bbb-050ca8b4ca4e None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:48.938631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f838fd-3720-45d4-9bbb-050ca8b4ca4e None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:48.938809 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f838fd-3720-45d4-9bbb-050ca8b4ca4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:49.017873 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-21f838fd-3720-45d4-9bbb-050ca8b4ca4e None None] MFA Rules not processed for user `698d5a7fc34a46ba909152414a0860d7`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:49.019482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-21f838fd-3720-45d4-9bbb-050ca8b4ca4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:49.093041 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4324/8735] 10.222.0.23 () {58 vars in 1064 bytes} [Sat Aug 20 21:45:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:49.099859 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f752d1e-1f0b-43f9-8873-a784d9010ca3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:49.110987 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f752d1e-1f0b-43f9-8873-a784d9010ca3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:49.115112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af7266-6078-4fbf-916b-bc0fbe9a275c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:49.115920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af7266-6078-4fbf-916b-bc0fbe9a275c service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:49.116364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af7266-6078-4fbf-916b-bc0fbe9a275c service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:49.116674 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af7266-6078-4fbf-916b-bc0fbe9a275c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:49.206204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af7266-6078-4fbf-916b-bc0fbe9a275c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:49.207669 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af7266-6078-4fbf-916b-bc0fbe9a275c service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:49.228241 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4412/8736] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:45:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:45:49.377744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6a613-13b7-4d56-8aab-e5993f980fd1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:49.378348 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6a613-13b7-4d56-8aab-e5993f980fd1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:49.378820 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6a613-13b7-4d56-8aab-e5993f980fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:49.386804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:45:49.436133 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-91a6a613-13b7-4d56-8aab-e5993f980fd1 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:49.437666 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-91a6a613-13b7-4d56-8aab-e5993f980fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:49.510184 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4325/8737] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:45:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:49.562671 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea252c46-ab01-4f7c-b813-cd0367e4c9ca None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:49.562963 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea252c46-ab01-4f7c-b813-cd0367e4c9ca None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:49.563214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea252c46-ab01-4f7c-b813-cd0367e4c9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:49.621086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ea252c46-ab01-4f7c-b813-cd0367e4c9ca None None] MFA Rules not processed for user `d74f7cdc155a4397ad28a3fbca86f7f9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:49.622467 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ea252c46-ab01-4f7c-b813-cd0367e4c9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:49.704562 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4413/8738] 10.222.0.23 () {58 vars in 1040 bytes} [Sat Aug 20 21:45:49 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:49.712297 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6a613-13b7-4d56-8aab-e5993f980fd1 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:45:49.725167 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6a613-13b7-4d56-8aab-e5993f980fd1 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:45:49.730677 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413660c4-fcb6-4cd3-9474-a0a83de1b266 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:45:49.731287 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413660c4-fcb6-4cd3-9474-a0a83de1b266 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:49.732076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413660c4-fcb6-4cd3-9474-a0a83de1b266 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:49.732076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413660c4-fcb6-4cd3-9474-a0a83de1b266 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:49.755517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413660c4-fcb6-4cd3-9474-a0a83de1b266 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:45:49.757227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413660c4-fcb6-4cd3-9474-a0a83de1b266 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:45:49.774352 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4326/8739] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:45:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 21:45:58.479280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aa00a-1b36-4006-94a3-134e4ecb1ad0 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:45:58.479874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aa00a-1b36-4006-94a3-134e4ecb1ad0 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:45:58.479874 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aa00a-1b36-4006-94a3-134e4ecb1ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:45:58.499229 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2e6aa00a-1b36-4006-94a3-134e4ecb1ad0 None None] MFA Rules not processed for user `60932094d094426a93a0dfa8c05a12fe`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:45:58.500420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2e6aa00a-1b36-4006-94a3-134e4ecb1ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:45:58.508327 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4414/8740] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:45:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:45:59.763672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=44 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:46:00.500954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925dedc-a345-46d5-aeef-010ff034b144 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:46:00.501237 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925dedc-a345-46d5-aeef-010ff034b144 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:46:00.501507 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925dedc-a345-46d5-aeef-010ff034b144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:46:00.522137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c925dedc-a345-46d5-aeef-010ff034b144 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:46:00.523398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c925dedc-a345-46d5-aeef-010ff034b144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:46:00.531714 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4327/8741] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:46:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:46:00.545208 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6aa00a-1b36-4006-94a3-134e4ecb1ad0 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:46:00.557920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6aa00a-1b36-4006-94a3-134e4ecb1ad0 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:46:00.563784 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccec153-d724-4255-8d2d-fd8815d89d0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:46:00.564407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccec153-d724-4255-8d2d-fd8815d89d0d service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:46:00.564723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccec153-d724-4255-8d2d-fd8815d89d0d service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:46:00.564979 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccec153-d724-4255-8d2d-fd8815d89d0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:46:00.573892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccec153-d724-4255-8d2d-fd8815d89d0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:46:00.575486 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccec153-d724-4255-8d2d-fd8815d89d0d service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:46:00.584727 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4415/8742] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:46:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:46:10.516853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:46:10.581064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=93 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:46:18.088893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925dedc-a345-46d5-aeef-010ff034b144 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:46:18.147262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925dedc-a345-46d5-aeef-010ff034b144 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:46:18.151642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38387c4-190d-47c0-8631-b9c7b0ee77ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:46:18.152513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38387c4-190d-47c0-8631-b9c7b0ee77ad service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:46:18.152912 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38387c4-190d-47c0-8631-b9c7b0ee77ad service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:46:18.153243 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38387c4-190d-47c0-8631-b9c7b0ee77ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:46:18.157476 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a38387c4-190d-47c0-8631-b9c7b0ee77ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:46:18.236620 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38387c4-190d-47c0-8631-b9c7b0ee77ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:46:18.238091 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38387c4-190d-47c0-8631-b9c7b0ee77ad service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:46:18.252270 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4328/8743] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:46:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:46:18.446529 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccec153-d724-4255-8d2d-fd8815d89d0d service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:46:18.456681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccec153-d724-4255-8d2d-fd8815d89d0d service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:46:18.460462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e638be-94d3-44cb-af94-0ddc3bde1183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:46:18.461044 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e638be-94d3-44cb-af94-0ddc3bde1183 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:46:18.461309 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e638be-94d3-44cb-af94-0ddc3bde1183 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:46:18.461564 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e638be-94d3-44cb-af94-0ddc3bde1183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:46:18.463280 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-02e638be-94d3-44cb-af94-0ddc3bde1183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:46:18.479596 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e638be-94d3-44cb-af94-0ddc3bde1183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:46:18.481050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e638be-94d3-44cb-af94-0ddc3bde1183 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:46:18.496652 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4416/8744] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:46:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:46:24.188218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d44e1-fe54-486c-9133-9e40d76a5945 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:46:24.188725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d44e1-fe54-486c-9133-9e40d76a5945 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:46:24.188853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d44e1-fe54-486c-9133-9e40d76a5945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:46:24.215643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8c2d44e1-fe54-486c-9133-9e40d76a5945 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:46:24.216831 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8c2d44e1-fe54-486c-9133-9e40d76a5945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:46:24.225604 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4329/8745] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:46:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:46:24.238375 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e638be-94d3-44cb-af94-0ddc3bde1183 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:46:24.251133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e638be-94d3-44cb-af94-0ddc3bde1183 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:46:24.256806 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03756daf-c3e5-453f-bdb4-e1e92609cec6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:46:24.257465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03756daf-c3e5-453f-bdb4-e1e92609cec6 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:46:24.257774 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03756daf-c3e5-453f-bdb4-e1e92609cec6 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:46:24.258041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03756daf-c3e5-453f-bdb4-e1e92609cec6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:46:24.266699 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03756daf-c3e5-453f-bdb4-e1e92609cec6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:46:24.268315 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03756daf-c3e5-453f-bdb4-e1e92609cec6 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:46:24.276528 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4417/8746] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:46:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:46:24.805954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01d705-5177-4944-824f-2cb5f634759b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:46:24.806232 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01d705-5177-4944-824f-2cb5f634759b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:46:24.806501 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01d705-5177-4944-824f-2cb5f634759b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:46:24.825699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8c01d705-5177-4944-824f-2cb5f634759b None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:46:24.826917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8c01d705-5177-4944-824f-2cb5f634759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:46:24.834788 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4330/8747] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:46:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:46:24.849741 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03756daf-c3e5-453f-bdb4-e1e92609cec6 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:46:24.861655 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03756daf-c3e5-453f-bdb4-e1e92609cec6 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:46:24.867321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547008a7-e943-45ac-b2b4-d68f72e4c927 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:46:24.867914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547008a7-e943-45ac-b2b4-d68f72e4c927 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:46:24.868207 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547008a7-e943-45ac-b2b4-d68f72e4c927 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:46:24.868501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547008a7-e943-45ac-b2b4-d68f72e4c927 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:46:24.877590 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547008a7-e943-45ac-b2b4-d68f72e4c927 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:46:24.879168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547008a7-e943-45ac-b2b4-d68f72e4c927 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:46:24.887566 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4418/8748] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:46:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:46:33.430190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c46782-fb18-44e9-a1f1-d67f7b304466 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:46:33.430719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c46782-fb18-44e9-a1f1-d67f7b304466 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:46:33.430719 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c46782-fb18-44e9-a1f1-d67f7b304466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:46:33.451010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-68c46782-fb18-44e9-a1f1-d67f7b304466 None None] MFA Rules not processed for user `60932094d094426a93a0dfa8c05a12fe`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:46:33.452413 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-68c46782-fb18-44e9-a1f1-d67f7b304466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:46:33.460616 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4331/8749] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:46:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:46:34.884254 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=18 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:46:35.694422 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93125-2578-4e78-b973-7751cdf83c83 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:46:35.694697 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93125-2578-4e78-b973-7751cdf83c83 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:46:35.694989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93125-2578-4e78-b973-7751cdf83c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:46:35.715294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e5b93125-2578-4e78-b973-7751cdf83c83 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:46:35.716709 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e5b93125-2578-4e78-b973-7751cdf83c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:46:35.724166 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4419/8750] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:46:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:46:35.737173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c46782-fb18-44e9-a1f1-d67f7b304466 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:46:35.750191 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c46782-fb18-44e9-a1f1-d67f7b304466 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:46:35.755248 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5275ab-68db-4056-a233-d5a8e7364084 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:46:35.755917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5275ab-68db-4056-a233-d5a8e7364084 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:46:35.756242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5275ab-68db-4056-a233-d5a8e7364084 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:46:35.756535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5275ab-68db-4056-a233-d5a8e7364084 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:46:35.765859 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5275ab-68db-4056-a233-d5a8e7364084 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:46:35.767503 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5275ab-68db-4056-a233-d5a8e7364084 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:46:35.776102 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4332/8751] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:46:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:46:45.710164 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:46:45.773445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=69 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:46:56.488825 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968be0e-8ae0-46b9-9871-ce4ca9c99dba None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:46:56.489328 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968be0e-8ae0-46b9-9871-ce4ca9c99dba None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:46:56.489441 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968be0e-8ae0-46b9-9871-ce4ca9c99dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:46:56.520980 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-1968be0e-8ae0-46b9-9871-ce4ca9c99dba None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:46:56.522679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1968be0e-8ae0-46b9-9871-ce4ca9c99dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:46:56.533705 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4420/8752] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 21:46:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:46:56.555557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5275ab-68db-4056-a233-d5a8e7364084 service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:46:56.566778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5275ab-68db-4056-a233-d5a8e7364084 service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:46:56.573076 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a8ab-2f20-4eb5-957b-53e92e12ebaf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:46:56.573664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a8ab-2f20-4eb5-957b-53e92e12ebaf service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:46:56.573941 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a8ab-2f20-4eb5-957b-53e92e12ebaf service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:46:56.574208 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a8ab-2f20-4eb5-957b-53e92e12ebaf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:46:56.583006 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0a8ab-2f20-4eb5-957b-53e92e12ebaf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:46:56.584634 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0a8ab-2f20-4eb5-957b-53e92e12ebaf service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:46:56.593043 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4333/8753] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:46:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 21:47:03.365039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968be0e-8ae0-46b9-9871-ce4ca9c99dba None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:03.435272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968be0e-8ae0-46b9-9871-ce4ca9c99dba None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:03.439835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:03.440488 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:03.440759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:03.441021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] PATH_INFO: `/v3/users/60932094d094426a93a0dfa8c05a12fe` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:03.442145 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=60932094d094426a93a0dfa8c05a12fe)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:03.444057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:03.492195 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60932094d094426a93a0dfa8c05a12fe'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:03.502677 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60932094d094426a93a0dfa8c05a12fe'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:03.510520 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60932094d094426a93a0dfa8c05a12fe'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:03.522822 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60932094d094426a93a0dfa8c05a12fe'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:03.523484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60932094d094426a93a0dfa8c05a12fe'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:03.530209 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4421/8754] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:47:03 2022] DELETE /identity/v3/users/60932094d094426a93a0dfa8c05a12fe => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:47:03.714718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a8ab-2f20-4eb5-957b-53e92e12ebaf service swift] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:03.730796 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a8ab-2f20-4eb5-957b-53e92e12ebaf service swift] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:03.798367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:03.798914 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:03.799186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:03.799454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] PATH_INFO: `/v3/projects/d7e11effb3274080abe061ae967dee8b` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:03.800650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7e11effb3274080abe061ae967dee8b)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:03.802563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:03.860066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7e11effb3274080abe061ae967dee8b', 'request_id': 'req-eb75c844-8f18-4606-931a-6f9f59bd6a42'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:03.868313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7e11effb3274080abe061ae967dee8b', 'request_id': 'req-eb75c844-8f18-4606-931a-6f9f59bd6a42'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:03.878549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7e11effb3274080abe061ae967dee8b', 'request_id': 'req-eb75c844-8f18-4606-931a-6f9f59bd6a42'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:03.889473 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] The token cache is being invalidate because project d7e11effb3274080abe061ae967dee8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:47:03.889758 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:03.891015 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:03.892556 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4334/8755] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:47:03 2022] DELETE /identity/v3/projects/d7e11effb3274080abe061ae967dee8b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:47:03.896649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:03.898180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:47:03.912012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a5789-bb80-43ca-b901-686e19576d3a admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:03.980059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40a560-2851-431a-98c8-a69e3c72d766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:03.980786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40a560-2851-431a-98c8-a69e3c72d766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:03.981054 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40a560-2851-431a-98c8-a69e3c72d766 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:03.981304 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40a560-2851-431a-98c8-a69e3c72d766 admin admin] PATH_INFO: `/v3/users/ccf2e8f707504bd3acc8040bac0000fd` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:03.983336 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40a560-2851-431a-98c8-a69e3c72d766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf2e8f707504bd3acc8040bac0000fd)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:03.985513 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40a560-2851-431a-98c8-a69e3c72d766 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:04.001150 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:47:04.036094 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d40a560-2851-431a-98c8-a69e3c72d766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf2e8f707504bd3acc8040bac0000fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:04.043478 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d40a560-2851-431a-98c8-a69e3c72d766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf2e8f707504bd3acc8040bac0000fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:04.050635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d40a560-2851-431a-98c8-a69e3c72d766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf2e8f707504bd3acc8040bac0000fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:04.062181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d40a560-2851-431a-98c8-a69e3c72d766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf2e8f707504bd3acc8040bac0000fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:04.062835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-6d40a560-2851-431a-98c8-a69e3c72d766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf2e8f707504bd3acc8040bac0000fd'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:04.068710 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4422/8756] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:47:03 2022] DELETE /identity/v3/users/ccf2e8f707504bd3acc8040bac0000fd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:47:04.250987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:04.252688 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:47:04.270210 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75c844-8f18-4606-931a-6f9f59bd6a42 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:04.330290 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:04.331194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:04.331464 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:04.331726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89 admin admin] PATH_INFO: `/v3/projects/de1d860aa0604eccb62ca6cc55e99cbc` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:04.332922 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1d860aa0604eccb62ca6cc55e99cbc)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:04.334821 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:04.355700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:47:04.384176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1d860aa0604eccb62ca6cc55e99cbc', 'request_id': 'req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:04.390414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1d860aa0604eccb62ca6cc55e99cbc', 'request_id': 'req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:04.398932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1d860aa0604eccb62ca6cc55e99cbc', 'request_id': 'req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:04.411066 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89 admin admin] The token cache is being invalidate because project de1d860aa0604eccb62ca6cc55e99cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:47:04.411357 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:04.412010 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5bfe3560-80d6-4f0c-9f18-c4838d0bdf89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:47:04.413564 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4335/8757] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:47:04 2022] DELETE /identity/v3/projects/de1d860aa0604eccb62ca6cc55e99cbc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:47:04.418907 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a44027-9720-4b3a-aeed-9ab9f2d88c68 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:04.419390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a44027-9720-4b3a-aeed-9ab9f2d88c68 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:04.419484 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a44027-9720-4b3a-aeed-9ab9f2d88c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:04.439346 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b2a44027-9720-4b3a-aeed-9ab9f2d88c68 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:47:04.440565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b2a44027-9720-4b3a-aeed-9ab9f2d88c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:47:04.506714 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4423/8758] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:47:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:47:04.514400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010d19d-585b-4932-abf5-c6118d8175a9 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:04.514706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010d19d-585b-4932-abf5-c6118d8175a9 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:04.515062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010d19d-585b-4932-abf5-c6118d8175a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:04.534697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-4010d19d-585b-4932-abf5-c6118d8175a9 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:47:04.535916 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4010d19d-585b-4932-abf5-c6118d8175a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:47:04.544264 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4336/8759] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:47:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:47:04.549465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a44027-9720-4b3a-aeed-9ab9f2d88c68 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:04.563982 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a44027-9720-4b3a-aeed-9ab9f2d88c68 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:04.569813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6153e-e2a3-4c3d-98c3-19612f3d64fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:04.570423 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6153e-e2a3-4c3d-98c3-19612f3d64fe admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:04.570695 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6153e-e2a3-4c3d-98c3-19612f3d64fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:04.570962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6153e-e2a3-4c3d-98c3-19612f3d64fe admin admin] PATH_INFO: `/v3/domains` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:04.571597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6153e-e2a3-4c3d-98c3-19612f3d64fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:47:04.571922 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6153e-e2a3-4c3d-98c3-19612f3d64fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:04.573667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6153e-e2a3-4c3d-98c3-19612f3d64fe admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:04.585818 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4424/8760] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:47:04.590149 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010d19d-585b-4932-abf5-c6118d8175a9 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:04.599580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010d19d-585b-4932-abf5-c6118d8175a9 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:04.604614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaa41f-94bd-4001-8f02-5b71a31e9c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:04.605200 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaa41f-94bd-4001-8f02-5b71a31e9c36 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:04.605469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaa41f-94bd-4001-8f02-5b71a31e9c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:04.605734 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaa41f-94bd-4001-8f02-5b71a31e9c36 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:04.606352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeaa41f-94bd-4001-8f02-5b71a31e9c36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:04.608240 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeaa41f-94bd-4001-8f02-5b71a31e9c36 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:04.645790 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4337/8761] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:47:04 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:47:04.650055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6153e-e2a3-4c3d-98c3-19612f3d64fe admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:04.660719 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6153e-e2a3-4c3d-98c3-19612f3d64fe admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:04.728347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e56ce-cdaf-478e-a7ce-c8ee177fb22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:04.728973 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e56ce-cdaf-478e-a7ce-c8ee177fb22d admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:04.729240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e56ce-cdaf-478e-a7ce-c8ee177fb22d admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:04.729527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e56ce-cdaf-478e-a7ce-c8ee177fb22d admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:04.730142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e56ce-cdaf-478e-a7ce-c8ee177fb22d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:04.732096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e56ce-cdaf-478e-a7ce-c8ee177fb22d admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:04.753714 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4425/8762] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:47:04 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:47:04.757999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaa41f-94bd-4001-8f02-5b71a31e9c36 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:04.768800 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaa41f-94bd-4001-8f02-5b71a31e9c36 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:04.774467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c9550-30a2-4a2c-b916-72dae886e5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:04.775069 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c9550-30a2-4a2c-b916-72dae886e5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:04.775345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c9550-30a2-4a2c-b916-72dae886e5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:04.775612 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c9550-30a2-4a2c-b916-72dae886e5c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:04.776253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c9550-30a2-4a2c-b916-72dae886e5c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:04.777921 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c9550-30a2-4a2c-b916-72dae886e5c9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:04.788924 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4338/8763] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:47:04.793223 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e56ce-cdaf-478e-a7ce-c8ee177fb22d admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:04.803888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e56ce-cdaf-478e-a7ce-c8ee177fb22d admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:04.809479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c417c87-cb27-4d23-a5ef-dee112b26a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:04.810156 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c417c87-cb27-4d23-a5ef-dee112b26a9e admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:04.810417 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c417c87-cb27-4d23-a5ef-dee112b26a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:04.810688 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c417c87-cb27-4d23-a5ef-dee112b26a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:04.811321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c417c87-cb27-4d23-a5ef-dee112b26a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:04.813230 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c417c87-cb27-4d23-a5ef-dee112b26a9e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:04.815442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:47:04.829429 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4426/8764] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:47:04.830678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c9550-30a2-4a2c-b916-72dae886e5c9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:04.841259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c9550-30a2-4a2c-b916-72dae886e5c9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:04.846020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a40922-2769-49fb-bb8e-74df3978c06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:04.846655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a40922-2769-49fb-bb8e-74df3978c06b admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:04.846925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a40922-2769-49fb-bb8e-74df3978c06b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:04.847185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a40922-2769-49fb-bb8e-74df3978c06b admin admin] PATH_INFO: `/v3/projects/eaac203854f04e9ab195ab5a5ebdc9d1/users/988aca1c7805446cafeda57c8ed3cbcd/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:04.862047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a40922-2769-49fb-bb8e-74df3978c06b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaac203854f04e9ab195ab5a5ebdc9d1, user_id=988aca1c7805446cafeda57c8ed3cbcd, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:04.864720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a40922-2769-49fb-bb8e-74df3978c06b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:04.879666 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4339/8765] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:47:04 2022] PUT /identity/v3/projects/eaac203854f04e9ab195ab5a5ebdc9d1/users/988aca1c7805446cafeda57c8ed3cbcd/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:47:04.890724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5479a30-5b4d-4150-9a02-8398db97b148 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:04.891059 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5479a30-5b4d-4150-9a02-8398db97b148 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:04.891449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5479a30-5b4d-4150-9a02-8398db97b148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:04.925364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-f5479a30-5b4d-4150-9a02-8398db97b148 None None] MFA Rules not processed for user `988aca1c7805446cafeda57c8ed3cbcd`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:47:04.926593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-f5479a30-5b4d-4150-9a02-8398db97b148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:47:05.002278 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4427/8766] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:47:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:47:05.007298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a40922-2769-49fb-bb8e-74df3978c06b admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:05.073118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a40922-2769-49fb-bb8e-74df3978c06b admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:05.079498 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb5298-3ae8-4229-be73-4de96ad0f861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:05.080107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb5298-3ae8-4229-be73-4de96ad0f861 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:05.080415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb5298-3ae8-4229-be73-4de96ad0f861 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:05.080681 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb5298-3ae8-4229-be73-4de96ad0f861 admin admin] PATH_INFO: `/v3/projects` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:05.081609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb5298-3ae8-4229-be73-4de96ad0f861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:05.083544 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb5298-3ae8-4229-be73-4de96ad0f861 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:05.113340 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4340/8767] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:47:05 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:47:05.117728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5479a30-5b4d-4150-9a02-8398db97b148 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:05.129011 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5479a30-5b4d-4150-9a02-8398db97b148 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:05.187262 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb276bfc-ab1a-483a-a7f4-5bb0650003d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:05.187861 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb276bfc-ab1a-483a-a7f4-5bb0650003d4 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:05.188132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb276bfc-ab1a-483a-a7f4-5bb0650003d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:05.188401 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb276bfc-ab1a-483a-a7f4-5bb0650003d4 admin admin] PATH_INFO: `/v3/users` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:05.189042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb276bfc-ab1a-483a-a7f4-5bb0650003d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:05.190971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb276bfc-ab1a-483a-a7f4-5bb0650003d4 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:05.214871 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4428/8768] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:47:05 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:47:05.219379 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb5298-3ae8-4229-be73-4de96ad0f861 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:05.231660 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb5298-3ae8-4229-be73-4de96ad0f861 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:05.237118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f4a3c-b16e-4a1a-b0c6-046028f86ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:05.237726 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f4a3c-b16e-4a1a-b0c6-046028f86ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:05.238020 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f4a3c-b16e-4a1a-b0c6-046028f86ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:05.238283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f4a3c-b16e-4a1a-b0c6-046028f86ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:05.238917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f4a3c-b16e-4a1a-b0c6-046028f86ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:05.240569 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f4a3c-b16e-4a1a-b0c6-046028f86ce0 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:05.251907 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4341/8769] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:47:05.256480 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb276bfc-ab1a-483a-a7f4-5bb0650003d4 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:05.266692 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb276bfc-ab1a-483a-a7f4-5bb0650003d4 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:05.272030 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f0657-eaa4-4b2a-9b2c-4f3eb678ce19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:05.272725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f0657-eaa4-4b2a-9b2c-4f3eb678ce19 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:05.272999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f0657-eaa4-4b2a-9b2c-4f3eb678ce19 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:05.273265 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f0657-eaa4-4b2a-9b2c-4f3eb678ce19 admin admin] PATH_INFO: `/v3/projects/406b827bb2494589912975e0d5ec2c56/users/a02c9d866cba44ba91ccebe7e2aa0892/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:05.289134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f0657-eaa4-4b2a-9b2c-4f3eb678ce19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=406b827bb2494589912975e0d5ec2c56, user_id=a02c9d866cba44ba91ccebe7e2aa0892, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:05.291788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f0657-eaa4-4b2a-9b2c-4f3eb678ce19 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:05.307013 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4429/8770] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:47:05 2022] PUT /identity/v3/projects/406b827bb2494589912975e0d5ec2c56/users/a02c9d866cba44ba91ccebe7e2aa0892/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:47:05.311194 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f4a3c-b16e-4a1a-b0c6-046028f86ce0 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:05.322098 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f4a3c-b16e-4a1a-b0c6-046028f86ce0 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:05.327154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:47:05.390187 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f013db-513e-4fb9-95b3-80ae1dde7c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:05.390672 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f013db-513e-4fb9-95b3-80ae1dde7c4e admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:05.391062 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f013db-513e-4fb9-95b3-80ae1dde7c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:05.391327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f013db-513e-4fb9-95b3-80ae1dde7c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:05.391958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f013db-513e-4fb9-95b3-80ae1dde7c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:05.393582 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f013db-513e-4fb9-95b3-80ae1dde7c4e admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:05.403707 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4342/8771] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:47:05.408310 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f0657-eaa4-4b2a-9b2c-4f3eb678ce19 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:05.419143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f0657-eaa4-4b2a-9b2c-4f3eb678ce19 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:05.425009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edb461-b760-4125-9136-b94b58ee4724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:05.425656 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edb461-b760-4125-9136-b94b58ee4724 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:05.425920 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edb461-b760-4125-9136-b94b58ee4724 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:05.426165 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edb461-b760-4125-9136-b94b58ee4724 admin admin] PATH_INFO: `/v3/projects/406b827bb2494589912975e0d5ec2c56/users/a02c9d866cba44ba91ccebe7e2aa0892/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:05.428298 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edb461-b760-4125-9136-b94b58ee4724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=406b827bb2494589912975e0d5ec2c56, user_id=a02c9d866cba44ba91ccebe7e2aa0892, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:05.431114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edb461-b760-4125-9136-b94b58ee4724 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:05.442603 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4430/8772] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:47:05 2022] PUT /identity/v3/projects/406b827bb2494589912975e0d5ec2c56/users/a02c9d866cba44ba91ccebe7e2aa0892/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:47:05.454601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7a6bc-3376-4b04-875c-408bf097a601 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:05.454886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7a6bc-3376-4b04-875c-408bf097a601 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:05.455154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7a6bc-3376-4b04-875c-408bf097a601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:05.484547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-98a7a6bc-3376-4b04-875c-408bf097a601 None None] MFA Rules not processed for user `a02c9d866cba44ba91ccebe7e2aa0892`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:47:05.485738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-98a7a6bc-3376-4b04-875c-408bf097a601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:47:05.557076 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4343/8773] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:47:05 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:47:05.573100 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edb461-b760-4125-9136-b94b58ee4724 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:05.575192 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-33edb461-b760-4125-9136-b94b58ee4724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:47:05.625090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:47:05.661686 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edb461-b760-4125-9136-b94b58ee4724 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:05.667553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ea19d-d660-4269-a03c-a8fcff64b2d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:05.668149 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ea19d-d660-4269-a03c-a8fcff64b2d9 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:05.668464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ea19d-d660-4269-a03c-a8fcff64b2d9 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:05.668725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ea19d-d660-4269-a03c-a8fcff64b2d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:05.743326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ea19d-d660-4269-a03c-a8fcff64b2d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:05.745665 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ea19d-d660-4269-a03c-a8fcff64b2d9 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:05.756806 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4431/8774] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:47:05 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:47:08.589428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7a6bc-3376-4b04-875c-408bf097a601 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:08.591067 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-98a7a6bc-3376-4b04-875c-408bf097a601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:47:08.660558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7a6bc-3376-4b04-875c-408bf097a601 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:08.668042 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac8031-de43-49ef-a13f-4e06c80ada0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:08.668705 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac8031-de43-49ef-a13f-4e06c80ada0a service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:08.668998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac8031-de43-49ef-a13f-4e06c80ada0a service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:08.669262 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac8031-de43-49ef-a13f-4e06c80ada0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:08.680186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac8031-de43-49ef-a13f-4e06c80ada0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:08.681647 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac8031-de43-49ef-a13f-4e06c80ada0a service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:08.692138 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4344/8775] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:47:08 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:47:11.519620 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e01c56-deda-4d51-952a-2e5b18e028e9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:11.520065 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e01c56-deda-4d51-952a-2e5b18e028e9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:11.520143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e01c56-deda-4d51-952a-2e5b18e028e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:11.539561 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-d1e01c56-deda-4d51-952a-2e5b18e028e9 None None] MFA Rules not processed for user `988aca1c7805446cafeda57c8ed3cbcd`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:47:11.540789 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-d1e01c56-deda-4d51-952a-2e5b18e028e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:47:11.547782 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4432/8776] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:47:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:47:11.893336 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac8031-de43-49ef-a13f-4e06c80ada0a service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:47:11.902808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac8031-de43-49ef-a13f-4e06c80ada0a service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:47:11.906497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fe7dc-e6a2-4a88-9aa8-9b005d95184f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:47:11.907080 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fe7dc-e6a2-4a88-9aa8-9b005d95184f service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:47:11.907345 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fe7dc-e6a2-4a88-9aa8-9b005d95184f service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:47:11.907608 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fe7dc-e6a2-4a88-9aa8-9b005d95184f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:47:11.909656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-243fe7dc-e6a2-4a88-9aa8-9b005d95184f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:47:11.922885 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fe7dc-e6a2-4a88-9aa8-9b005d95184f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:47:11.924322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fe7dc-e6a2-4a88-9aa8-9b005d95184f service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:47:11.940629 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4345/8777] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:47:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:47:21.534319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=41 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:47:21.931400 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:48:17.241245 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4396404-4b9c-40ea-a600-161772a9bd4e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:48:17.241786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4396404-4b9c-40ea-a600-161772a9bd4e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:48:17.241786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4396404-4b9c-40ea-a600-161772a9bd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:48:17.262492 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-b4396404-4b9c-40ea-a600-161772a9bd4e None None] MFA Rules not processed for user `988aca1c7805446cafeda57c8ed3cbcd`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:48:17.263867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b4396404-4b9c-40ea-a600-161772a9bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:48:17.271171 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4433/8778] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:48:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:48:19.888867 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fe7dc-e6a2-4a88-9aa8-9b005d95184f service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:48:19.890343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-243fe7dc-e6a2-4a88-9aa8-9b005d95184f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:48:19.904883 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fe7dc-e6a2-4a88-9aa8-9b005d95184f service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:48:19.909740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efd0ec-63d1-461f-8377-b9e2114f9749 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:48:19.910517 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efd0ec-63d1-461f-8377-b9e2114f9749 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:48:19.910874 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efd0ec-63d1-461f-8377-b9e2114f9749 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:48:19.911190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efd0ec-63d1-461f-8377-b9e2114f9749 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:48:19.913330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-58efd0ec-63d1-461f-8377-b9e2114f9749 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:48:19.932299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efd0ec-63d1-461f-8377-b9e2114f9749 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:48:19.933900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efd0ec-63d1-461f-8377-b9e2114f9749 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:48:19.964216 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4346/8779] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:48:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:48:27.256820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=3 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:48:29.941999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:48:35.984761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4396404-4b9c-40ea-a600-161772a9bd4e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:48:35.986266 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-b4396404-4b9c-40ea-a600-161772a9bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:48:36.070968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4396404-4b9c-40ea-a600-161772a9bd4e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:48:36.076363 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccde15-f607-4c26-9c83-9cfc6cde5aa7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:48:36.076988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccde15-f607-4c26-9c83-9cfc6cde5aa7 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:48:36.077256 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccde15-f607-4c26-9c83-9cfc6cde5aa7 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:48:36.077516 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccde15-f607-4c26-9c83-9cfc6cde5aa7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:48:36.079337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ecccde15-f607-4c26-9c83-9cfc6cde5aa7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:48:36.093574 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccde15-f607-4c26-9c83-9cfc6cde5aa7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:48:36.095181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccde15-f607-4c26-9c83-9cfc6cde5aa7 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:48:36.111502 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4434/8780] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:48:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:48:46.102780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=36 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:49:28.195004 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efd0ec-63d1-461f-8377-b9e2114f9749 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:49:28.197094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-58efd0ec-63d1-461f-8377-b9e2114f9749 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:49:28.254086 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efd0ec-63d1-461f-8377-b9e2114f9749 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:49:28.259908 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a59bde-e8a4-4cbf-ba69-ec888e6e52d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:49:28.260535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a59bde-e8a4-4cbf-ba69-ec888e6e52d6 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:49:28.260837 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a59bde-e8a4-4cbf-ba69-ec888e6e52d6 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:49:28.261107 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a59bde-e8a4-4cbf-ba69-ec888e6e52d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:49:28.263059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-02a59bde-e8a4-4cbf-ba69-ec888e6e52d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:49:28.277685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a59bde-e8a4-4cbf-ba69-ec888e6e52d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:49:28.279259 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a59bde-e8a4-4cbf-ba69-ec888e6e52d6 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:49:28.292155 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4347/8781] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:49:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:49:38.285371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=19 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:50:31.540281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccde15-f607-4c26-9c83-9cfc6cde5aa7 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:50:31.601210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccde15-f607-4c26-9c83-9cfc6cde5aa7 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:50:31.607897 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6e4c7-57ee-4c26-aee8-c6cc97c6909a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:50:31.608835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6e4c7-57ee-4c26-aee8-c6cc97c6909a service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:50:31.609362 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6e4c7-57ee-4c26-aee8-c6cc97c6909a service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:50:31.609570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6e4c7-57ee-4c26-aee8-c6cc97c6909a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:50:31.611604 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7ff6e4c7-57ee-4c26-aee8-c6cc97c6909a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:50:31.633021 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6e4c7-57ee-4c26-aee8-c6cc97c6909a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:50:31.634676 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6e4c7-57ee-4c26-aee8-c6cc97c6909a service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:50:31.660144 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4435/8782] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:50:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:50:33.926118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a59bde-e8a4-4cbf-ba69-ec888e6e52d6 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:50:33.927576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-02a59bde-e8a4-4cbf-ba69-ec888e6e52d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:50:33.949435 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a59bde-e8a4-4cbf-ba69-ec888e6e52d6 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:50:33.955095 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3507f3b-b1ad-4cd8-b5d3-5b92c5fb65b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:50:33.955697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3507f3b-b1ad-4cd8-b5d3-5b92c5fb65b0 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:50:33.955969 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3507f3b-b1ad-4cd8-b5d3-5b92c5fb65b0 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:50:33.956241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3507f3b-b1ad-4cd8-b5d3-5b92c5fb65b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:50:33.958657 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a3507f3b-b1ad-4cd8-b5d3-5b92c5fb65b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:50:33.974528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3507f3b-b1ad-4cd8-b5d3-5b92c5fb65b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:50:33.976181 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3507f3b-b1ad-4cd8-b5d3-5b92c5fb65b0 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:50:33.990150 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4348/8783] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:50:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:50:37.977841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a20f2-7775-4df9-9081-5db48f94d0ff None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:50:37.978804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a20f2-7775-4df9-9081-5db48f94d0ff None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:50:37.979288 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a20f2-7775-4df9-9081-5db48f94d0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:50:38.000886 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2b1a20f2-7775-4df9-9081-5db48f94d0ff None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:50:38.002213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2b1a20f2-7775-4df9-9081-5db48f94d0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:50:38.013241 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4436/8784] 10.222.0.30 () {58 vars in 1015 bytes} [Sat Aug 20 21:50:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:50:38.040477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3507f3b-b1ad-4cd8-b5d3-5b92c5fb65b0 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:50:38.041227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a20f2-7775-4df9-9081-5db48f94d0ff None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:50:38.055497 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a20f2-7775-4df9-9081-5db48f94d0ff None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:50:38.057118 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3507f3b-b1ad-4cd8-b5d3-5b92c5fb65b0 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:50:38.061715 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d35870-5845-47f6-a905-20a229723d9c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:50:38.062329 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d35870-5845-47f6-a905-20a229723d9c service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:50:38.062454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d188c1d-67e8-4800-806d-580c7617f515 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:50:38.062601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d35870-5845-47f6-a905-20a229723d9c service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:50:38.063036 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d35870-5845-47f6-a905-20a229723d9c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:50:38.063434 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d188c1d-67e8-4800-806d-580c7617f515 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:50:38.063862 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d188c1d-67e8-4800-806d-580c7617f515 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:50:38.064184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d188c1d-67e8-4800-806d-580c7617f515 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:50:38.065745 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-67d35870-5845-47f6-a905-20a229723d9c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:50:38.075467 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d188c1d-67e8-4800-806d-580c7617f515 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:50:38.077367 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d188c1d-67e8-4800-806d-580c7617f515 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:50:38.087098 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d35870-5845-47f6-a905-20a229723d9c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:50:38.089311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d35870-5845-47f6-a905-20a229723d9c service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:50:38.090702 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4349/8785] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:50:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:50:38.106857 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4437/8786] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:50:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:50:44.390166 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7bf36-cf49-478e-9e9d-c0df2214d9b6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:50:44.391254 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7bf36-cf49-478e-9e9d-c0df2214d9b6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:50:44.391546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7bf36-cf49-478e-9e9d-c0df2214d9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:50:44.421557 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0dc7bf36-cf49-478e-9e9d-c0df2214d9b6 None None] MFA Rules not processed for user `988aca1c7805446cafeda57c8ed3cbcd`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:50:44.423061 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0dc7bf36-cf49-478e-9e9d-c0df2214d9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:50:44.431592 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4350/8787] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:50:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:50:44.839924 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d35870-5845-47f6-a905-20a229723d9c service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:50:44.851651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d35870-5845-47f6-a905-20a229723d9c service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:50:44.855616 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782a58c-ab88-49e1-91e5-d11b488647a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:50:44.856236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782a58c-ab88-49e1-91e5-d11b488647a5 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:50:44.856545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782a58c-ab88-49e1-91e5-d11b488647a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:50:44.856813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782a58c-ab88-49e1-91e5-d11b488647a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:50:44.859012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9782a58c-ab88-49e1-91e5-d11b488647a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:50:44.874747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782a58c-ab88-49e1-91e5-d11b488647a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:50:44.876425 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782a58c-ab88-49e1-91e5-d11b488647a5 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:50:44.893229 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4438/8788] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:50:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:50:46.347900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810e347-9011-44dd-99dd-883d8e6b6e77 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:50:46.348551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810e347-9011-44dd-99dd-883d8e6b6e77 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:50:46.348551 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810e347-9011-44dd-99dd-883d8e6b6e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:50:46.369590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-1810e347-9011-44dd-99dd-883d8e6b6e77 None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:50:46.371109 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-1810e347-9011-44dd-99dd-883d8e6b6e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:50:46.384783 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4351/8789] 10.222.0.23 () {58 vars in 1058 bytes} [Sat Aug 20 21:50:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:50:46.391177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782a58c-ab88-49e1-91e5-d11b488647a5 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:50:46.404448 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782a58c-ab88-49e1-91e5-d11b488647a5 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:50:46.409404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54a461-5468-4a42-8384-911660ba0ae9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:50:46.410180 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54a461-5468-4a42-8384-911660ba0ae9 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:50:46.410541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54a461-5468-4a42-8384-911660ba0ae9 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:50:46.410902 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54a461-5468-4a42-8384-911660ba0ae9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:50:46.412852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ee54a461-5468-4a42-8384-911660ba0ae9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:50:46.426482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54a461-5468-4a42-8384-911660ba0ae9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:50:46.428204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54a461-5468-4a42-8384-911660ba0ae9 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:50:46.439306 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4439/8790] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:50:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:50:48.435151 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810e347-9011-44dd-99dd-883d8e6b6e77 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:50:48.445412 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810e347-9011-44dd-99dd-883d8e6b6e77 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:50:48.449693 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879628d-0280-474c-abd2-c133db681d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:50:48.450321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879628d-0280-474c-abd2-c133db681d01 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:50:48.450594 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879628d-0280-474c-abd2-c133db681d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:50:48.450855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879628d-0280-474c-abd2-c133db681d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:50:48.452871 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f879628d-0280-474c-abd2-c133db681d01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:50:48.467961 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879628d-0280-474c-abd2-c133db681d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:50:48.469789 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879628d-0280-474c-abd2-c133db681d01 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:50:48.482630 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4352/8791] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:50:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:50:53.491642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54a461-5468-4a42-8384-911660ba0ae9 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:50:53.508353 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54a461-5468-4a42-8384-911660ba0ae9 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:50:53.514407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e66138-f4ca-4fdf-b472-e759a6c20e3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:50:53.515014 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e66138-f4ca-4fdf-b472-e759a6c20e3f service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:50:53.515289 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e66138-f4ca-4fdf-b472-e759a6c20e3f service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:50:53.515549 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e66138-f4ca-4fdf-b472-e759a6c20e3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:50:53.517370 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-11e66138-f4ca-4fdf-b472-e759a6c20e3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:50:53.533438 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e66138-f4ca-4fdf-b472-e759a6c20e3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:50:53.535045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e66138-f4ca-4fdf-b472-e759a6c20e3f service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:50:53.560478 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4440/8792] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:50:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:50:58.475794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=26 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:51:03.542112 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=48 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:51:30.141697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0a601-b6f8-497d-a0f0-8feba2058c40 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:51:30.142344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0a601-b6f8-497d-a0f0-8feba2058c40 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:51:30.142344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0a601-b6f8-497d-a0f0-8feba2058c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:51:30.163656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8bb0a601-b6f8-497d-a0f0-8feba2058c40 None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:51:30.164904 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8bb0a601-b6f8-497d-a0f0-8feba2058c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:51:30.192920 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4353/8793] 10.222.0.16 () {58 vars in 1015 bytes} [Sat Aug 20 21:51:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:51:30.206404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e66138-f4ca-4fdf-b472-e759a6c20e3f service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:51:30.229118 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e66138-f4ca-4fdf-b472-e759a6c20e3f service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:51:30.235331 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f1578-8359-437f-81cc-fcaccb553474 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:51:30.235947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f1578-8359-437f-81cc-fcaccb553474 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:51:30.236221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f1578-8359-437f-81cc-fcaccb553474 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:51:30.236517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f1578-8359-437f-81cc-fcaccb553474 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:51:30.245584 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f1578-8359-437f-81cc-fcaccb553474 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:51:30.247177 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f1578-8359-437f-81cc-fcaccb553474 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:51:30.257316 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4441/8794] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:51:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:51:33.129251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0a601-b6f8-497d-a0f0-8feba2058c40 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:51:33.139201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0a601-b6f8-497d-a0f0-8feba2058c40 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:51:33.140095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f1578-8359-437f-81cc-fcaccb553474 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:51:33.143184 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdc791-bb3e-4ce5-ac26-0589fcbe04b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:51:33.143788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdc791-bb3e-4ce5-ac26-0589fcbe04b8 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:51:33.144059 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdc791-bb3e-4ce5-ac26-0589fcbe04b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:51:33.144326 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdc791-bb3e-4ce5-ac26-0589fcbe04b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:51:33.146342 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0afdc791-bb3e-4ce5-ac26-0589fcbe04b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:51:33.149452 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f1578-8359-437f-81cc-fcaccb553474 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:51:33.153236 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ab9f5-5b3e-4900-bf8b-6e435fb3b9a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:51:33.153894 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ab9f5-5b3e-4900-bf8b-6e435fb3b9a4 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:51:33.154322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ab9f5-5b3e-4900-bf8b-6e435fb3b9a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:51:33.154637 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ab9f5-5b3e-4900-bf8b-6e435fb3b9a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:51:33.156418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2f5ab9f5-5b3e-4900-bf8b-6e435fb3b9a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:51:33.172898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ab9f5-5b3e-4900-bf8b-6e435fb3b9a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:51:33.173186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdc791-bb3e-4ce5-ac26-0589fcbe04b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:51:33.174579 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ab9f5-5b3e-4900-bf8b-6e435fb3b9a4 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:51:33.175084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdc791-bb3e-4ce5-ac26-0589fcbe04b8 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:51:33.186103 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4354/8795] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:51:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:51:33.189535 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4442/8796] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:51:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:51:33.272652 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4771ce-027b-4d3b-9b0e-dedb70a2b85a None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:51:33.273176 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4771ce-027b-4d3b-9b0e-dedb70a2b85a None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:51:33.273775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4771ce-027b-4d3b-9b0e-dedb70a2b85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:51:33.295442 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ab4771ce-027b-4d3b-9b0e-dedb70a2b85a None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:51:33.296788 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ab4771ce-027b-4d3b-9b0e-dedb70a2b85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:51:33.306126 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4355/8797] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 21:51:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:51:33.317947 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d00332-9bc9-40da-a8e8-f8aed68128fe None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:51:33.318301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d00332-9bc9-40da-a8e8-f8aed68128fe None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:51:33.318639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d00332-9bc9-40da-a8e8-f8aed68128fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:51:33.339673 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-50d00332-9bc9-40da-a8e8-f8aed68128fe None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:51:33.340914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-50d00332-9bc9-40da-a8e8-f8aed68128fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:51:33.350062 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4443/8798] 10.222.0.23 () {58 vars in 1058 bytes} [Sat Aug 20 21:51:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:51:33.356450 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4771ce-027b-4d3b-9b0e-dedb70a2b85a None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:51:33.367645 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4771ce-027b-4d3b-9b0e-dedb70a2b85a None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:51:33.374137 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4431ded-3365-4328-818a-43ec461a1697 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:51:33.374735 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4431ded-3365-4328-818a-43ec461a1697 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:51:33.375008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4431ded-3365-4328-818a-43ec461a1697 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:51:33.375269 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4431ded-3365-4328-818a-43ec461a1697 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:51:33.384428 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4431ded-3365-4328-818a-43ec461a1697 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:51:33.386215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4431ded-3365-4328-818a-43ec461a1697 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:51:33.397073 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4356/8799] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:51:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:51:34.395244 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d00332-9bc9-40da-a8e8-f8aed68128fe None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:51:34.405728 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d00332-9bc9-40da-a8e8-f8aed68128fe None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:51:34.409759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e838ab-e4c6-4303-b518-b314b162724c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:51:34.410359 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e838ab-e4c6-4303-b518-b314b162724c service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:51:34.410626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e838ab-e4c6-4303-b518-b314b162724c service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:51:34.410889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e838ab-e4c6-4303-b518-b314b162724c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:51:34.412775 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-12e838ab-e4c6-4303-b518-b314b162724c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:51:34.429936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e838ab-e4c6-4303-b518-b314b162724c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:51:34.431364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e838ab-e4c6-4303-b518-b314b162724c service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:51:34.450306 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4444/8800] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:51:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:51:34.606644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4431ded-3365-4328-818a-43ec461a1697 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:51:34.616193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4431ded-3365-4328-818a-43ec461a1697 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:51:34.619960 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67a956-1c50-496f-b343-2f147d9ac7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:51:34.622590 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67a956-1c50-496f-b343-2f147d9ac7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:51:34.623463 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67a956-1c50-496f-b343-2f147d9ac7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:51:34.623775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67a956-1c50-496f-b343-2f147d9ac7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:51:34.625846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5d67a956-1c50-496f-b343-2f147d9ac7a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:51:34.641215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67a956-1c50-496f-b343-2f147d9ac7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:51:34.642650 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67a956-1c50-496f-b343-2f147d9ac7a9 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:51:34.654797 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4357/8801] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:51:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:51:34.717420 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e838ab-e4c6-4303-b518-b314b162724c service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:51:34.729066 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e838ab-e4c6-4303-b518-b314b162724c service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:51:34.735455 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb977863-dc44-4224-9238-c0a3cee4c50f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:51:34.736047 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb977863-dc44-4224-9238-c0a3cee4c50f service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:51:34.736321 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb977863-dc44-4224-9238-c0a3cee4c50f service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:51:34.736648 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb977863-dc44-4224-9238-c0a3cee4c50f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:51:34.738450 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cb977863-dc44-4224-9238-c0a3cee4c50f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:51:34.750339 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb977863-dc44-4224-9238-c0a3cee4c50f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:51:34.751912 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb977863-dc44-4224-9238-c0a3cee4c50f service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:51:34.761682 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4445/8802] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:51:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:51:44.649036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=36 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:51:44.757638 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=30 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:52:32.208284 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67a956-1c50-496f-b343-2f147d9ac7a9 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:52:32.222868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67a956-1c50-496f-b343-2f147d9ac7a9 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:52:32.231643 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f551df7-0800-4972-ae4f-cf5c82e98665 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:52:32.232267 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f551df7-0800-4972-ae4f-cf5c82e98665 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:52:32.232570 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f551df7-0800-4972-ae4f-cf5c82e98665 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:52:32.232802 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f551df7-0800-4972-ae4f-cf5c82e98665 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:52:32.243305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f551df7-0800-4972-ae4f-cf5c82e98665 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:52:32.245814 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f551df7-0800-4972-ae4f-cf5c82e98665 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:52:32.257041 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4358/8803] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:52:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:52:42.252260 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:52:56.149312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36254b44-5d27-417f-b6d8-80799896c23e None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:52:56.149916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36254b44-5d27-417f-b6d8-80799896c23e None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:52:56.149916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36254b44-5d27-417f-b6d8-80799896c23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:52:56.187559 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-36254b44-5d27-417f-b6d8-80799896c23e None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:52:56.188804 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-36254b44-5d27-417f-b6d8-80799896c23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:52:56.199186 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4446/8804] 10.222.0.23 () {58 vars in 1059 bytes} [Sat Aug 20 21:52:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:52:56.205183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec8505-e6ee-4534-95df-0ecf2fd5f7fd None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:52:56.206088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec8505-e6ee-4534-95df-0ecf2fd5f7fd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:52:56.206088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec8505-e6ee-4534-95df-0ecf2fd5f7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:52:56.226490 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-59ec8505-e6ee-4534-95df-0ecf2fd5f7fd None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:52:56.227733 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-59ec8505-e6ee-4534-95df-0ecf2fd5f7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:52:56.238128 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4359/8805] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:52:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:52:56.250126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36254b44-5d27-417f-b6d8-80799896c23e None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:52:56.260142 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36254b44-5d27-417f-b6d8-80799896c23e None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:52:56.265148 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eccd572-7b4e-4208-a3d4-efa5d12985c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:52:56.265866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eccd572-7b4e-4208-a3d4-efa5d12985c4 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:52:56.266202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eccd572-7b4e-4208-a3d4-efa5d12985c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:52:56.266556 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eccd572-7b4e-4208-a3d4-efa5d12985c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:52:56.277694 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eccd572-7b4e-4208-a3d4-efa5d12985c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:52:56.279247 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eccd572-7b4e-4208-a3d4-efa5d12985c4 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:52:56.290822 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4447/8806] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:52:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:52:57.477772 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f73953-e807-4222-99a7-79188c319783 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:52:57.478344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f73953-e807-4222-99a7-79188c319783 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:52:57.478344 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f73953-e807-4222-99a7-79188c319783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:52:57.505965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-81f73953-e807-4222-99a7-79188c319783 None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:52:57.507722 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-81f73953-e807-4222-99a7-79188c319783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:52:57.520518 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4360/8807] 10.222.0.23 () {58 vars in 1059 bytes} [Sat Aug 20 21:52:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:52:57.526443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dca19b-50d1-4f35-8d59-f927f1ec1019 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:52:57.526968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dca19b-50d1-4f35-8d59-f927f1ec1019 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:52:57.526968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dca19b-50d1-4f35-8d59-f927f1ec1019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:52:57.546657 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-11dca19b-50d1-4f35-8d59-f927f1ec1019 None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:52:57.547851 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-11dca19b-50d1-4f35-8d59-f927f1ec1019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:52:57.557760 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4448/8808] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:52:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:52:57.568700 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f73953-e807-4222-99a7-79188c319783 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:52:57.579185 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f73953-e807-4222-99a7-79188c319783 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:52:57.583425 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33f38d-4cf6-4158-b559-dd3a2639371d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:52:57.584045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33f38d-4cf6-4158-b559-dd3a2639371d service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:52:57.584319 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33f38d-4cf6-4158-b559-dd3a2639371d service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:52:57.584614 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33f38d-4cf6-4158-b559-dd3a2639371d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:52:57.596228 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33f38d-4cf6-4158-b559-dd3a2639371d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:52:57.597698 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33f38d-4cf6-4158-b559-dd3a2639371d service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:52:57.609483 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4361/8809] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:52:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:53:01.618966 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dca19b-50d1-4f35-8d59-f927f1ec1019 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:53:01.631536 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dca19b-50d1-4f35-8d59-f927f1ec1019 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:53:01.636949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf918170-ffcd-4301-956f-53857feb4cad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:53:01.637554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf918170-ffcd-4301-956f-53857feb4cad service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:53:01.637817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf918170-ffcd-4301-956f-53857feb4cad service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:53:01.638079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf918170-ffcd-4301-956f-53857feb4cad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:53:01.639927 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-cf918170-ffcd-4301-956f-53857feb4cad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:53:01.655196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf918170-ffcd-4301-956f-53857feb4cad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:53:01.656876 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf918170-ffcd-4301-956f-53857feb4cad service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:53:01.674490 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4449/8810] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:53:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:53:04.967739 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33f38d-4cf6-4158-b559-dd3a2639371d service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:53:04.977029 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33f38d-4cf6-4158-b559-dd3a2639371d service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:53:04.981445 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489e4cc-311f-44fb-b5cd-d4dae42af9b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:53:04.982134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489e4cc-311f-44fb-b5cd-d4dae42af9b0 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:53:04.982474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489e4cc-311f-44fb-b5cd-d4dae42af9b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:53:04.983082 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489e4cc-311f-44fb-b5cd-d4dae42af9b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:53:04.992535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489e4cc-311f-44fb-b5cd-d4dae42af9b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:53:04.994047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489e4cc-311f-44fb-b5cd-d4dae42af9b0 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:53:05.004196 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4362/8811] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:53:04 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:53:08.060557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190e9fb-1dd4-4efa-9261-a45a35c21fda None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:53:08.061032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190e9fb-1dd4-4efa-9261-a45a35c21fda None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:53:08.061268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190e9fb-1dd4-4efa-9261-a45a35c21fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:53:08.082187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5190e9fb-1dd4-4efa-9261-a45a35c21fda None None] MFA Rules not processed for user `988aca1c7805446cafeda57c8ed3cbcd`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:53:08.083723 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5190e9fb-1dd4-4efa-9261-a45a35c21fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:53:08.095385 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4450/8812] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:53:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:53:08.450045 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489e4cc-311f-44fb-b5cd-d4dae42af9b0 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:53:08.460124 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489e4cc-311f-44fb-b5cd-d4dae42af9b0 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:53:08.463947 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12e6f1-32ab-4e57-aabd-14c4feff4166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:53:08.464233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12e6f1-32ab-4e57-aabd-14c4feff4166 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:53:08.464558 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12e6f1-32ab-4e57-aabd-14c4feff4166 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:53:08.465147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12e6f1-32ab-4e57-aabd-14c4feff4166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:53:08.477380 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12e6f1-32ab-4e57-aabd-14c4feff4166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:53:08.478832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12e6f1-32ab-4e57-aabd-14c4feff4166 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:53:08.490999 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4363/8813] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:53:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:53:15.773464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc05ed-47ca-42c1-8829-1ede4ccfef05 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:53:15.774045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc05ed-47ca-42c1-8829-1ede4ccfef05 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:53:15.774045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc05ed-47ca-42c1-8829-1ede4ccfef05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:53:15.806072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e1cc05ed-47ca-42c1-8829-1ede4ccfef05 None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:53:15.807282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e1cc05ed-47ca-42c1-8829-1ede4ccfef05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:53:15.814894 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4451/8814] 10.222.0.30 () {58 vars in 1015 bytes} [Sat Aug 20 21:53:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:53:15.831077 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12e6f1-32ab-4e57-aabd-14c4feff4166 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:53:15.840665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12e6f1-32ab-4e57-aabd-14c4feff4166 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:53:15.845068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf767beb-a604-4d7c-b56f-2480c53daf60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:53:15.845713 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf767beb-a604-4d7c-b56f-2480c53daf60 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:53:15.845984 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf767beb-a604-4d7c-b56f-2480c53daf60 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:53:15.846250 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf767beb-a604-4d7c-b56f-2480c53daf60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:53:15.854540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf767beb-a604-4d7c-b56f-2480c53daf60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:53:15.855985 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf767beb-a604-4d7c-b56f-2480c53daf60 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:53:15.865284 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4364/8815] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:53:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:53:16.476923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037282f-a696-4dc4-8a0a-05194036b903 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:53:16.477302 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037282f-a696-4dc4-8a0a-05194036b903 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:53:16.477465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037282f-a696-4dc4-8a0a-05194036b903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:53:16.520703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-4037282f-a696-4dc4-8a0a-05194036b903 None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:53:16.521948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4037282f-a696-4dc4-8a0a-05194036b903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:53:16.532041 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4452/8816] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 21:53:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:53:16.553275 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf767beb-a604-4d7c-b56f-2480c53daf60 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:53:16.567454 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf767beb-a604-4d7c-b56f-2480c53daf60 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:53:16.573548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975b39e-4762-4d83-9b7d-94ee9f43d942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:53:16.574154 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975b39e-4762-4d83-9b7d-94ee9f43d942 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:53:16.574431 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975b39e-4762-4d83-9b7d-94ee9f43d942 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:53:16.574721 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975b39e-4762-4d83-9b7d-94ee9f43d942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:53:16.584760 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975b39e-4762-4d83-9b7d-94ee9f43d942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:53:16.586366 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975b39e-4762-4d83-9b7d-94ee9f43d942 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:53:16.598352 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4365/8817] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:53:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:53:22.174914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe88eb-77cc-47aa-9f3f-bdb2e8748309 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:53:22.175479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe88eb-77cc-47aa-9f3f-bdb2e8748309 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:53:22.175479 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe88eb-77cc-47aa-9f3f-bdb2e8748309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:53:22.196385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2dbe88eb-77cc-47aa-9f3f-bdb2e8748309 None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:53:22.197606 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2dbe88eb-77cc-47aa-9f3f-bdb2e8748309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:53:22.207114 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4453/8818] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 21:53:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:53:22.230190 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975b39e-4762-4d83-9b7d-94ee9f43d942 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:53:22.242372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975b39e-4762-4d83-9b7d-94ee9f43d942 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:53:22.248906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6e8ce-889a-46bd-a85d-081a51128958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:53:22.249646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6e8ce-889a-46bd-a85d-081a51128958 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:53:22.250236 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6e8ce-889a-46bd-a85d-081a51128958 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:53:22.250579 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6e8ce-889a-46bd-a85d-081a51128958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:53:22.257976 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=40 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:53:22.260283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6e8ce-889a-46bd-a85d-081a51128958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:53:22.262568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6e8ce-889a-46bd-a85d-081a51128958 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:53:22.273514 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4366/8819] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:53:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:53:32.191132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=40 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:53:32.269554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=2 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:54:47.961679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d88912-0c89-453f-bb30-150146a4ba88 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:54:47.962213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d88912-0c89-453f-bb30-150146a4ba88 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:54:47.962213 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d88912-0c89-453f-bb30-150146a4ba88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:54:48.033773 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-e3d88912-0c89-453f-bb30-150146a4ba88 None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:54:48.035002 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-e3d88912-0c89-453f-bb30-150146a4ba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:54:48.060799 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4454/8820] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 21:54:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:54:48.073573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6e8ce-889a-46bd-a85d-081a51128958 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:54:48.097965 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6e8ce-889a-46bd-a85d-081a51128958 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:54:48.104174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7e27d-2703-4d6f-b59f-aaf07146fd3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:54:48.104174 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7e27d-2703-4d6f-b59f-aaf07146fd3a service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:54:48.104797 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7e27d-2703-4d6f-b59f-aaf07146fd3a service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:54:48.105358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7e27d-2703-4d6f-b59f-aaf07146fd3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:54:48.118944 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7e27d-2703-4d6f-b59f-aaf07146fd3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:54:48.120678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7e27d-2703-4d6f-b59f-aaf07146fd3a service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:54:48.134814 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4367/8821] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:54:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:54:55.008959 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bc8d5-3211-435a-b69c-5836f3523f90 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:54:55.009528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bc8d5-3211-435a-b69c-5836f3523f90 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:54:55.009528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bc8d5-3211-435a-b69c-5836f3523f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:54:55.072791 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-382bc8d5-3211-435a-b69c-5836f3523f90 None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:54:55.074482 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-382bc8d5-3211-435a-b69c-5836f3523f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:54:55.085733 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4455/8822] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 21:54:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:54:55.107987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7e27d-2703-4d6f-b59f-aaf07146fd3a service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:54:55.120477 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7e27d-2703-4d6f-b59f-aaf07146fd3a service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:54:55.126740 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4b644-3f28-4ef9-8d34-cd92c0ae4a1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:54:55.127351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4b644-3f28-4ef9-8d34-cd92c0ae4a1a service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:54:55.127805 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4b644-3f28-4ef9-8d34-cd92c0ae4a1a service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:54:55.128064 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4b644-3f28-4ef9-8d34-cd92c0ae4a1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:54:55.137135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4b644-3f28-4ef9-8d34-cd92c0ae4a1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:54:55.138625 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4b644-3f28-4ef9-8d34-cd92c0ae4a1a service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:54:55.149353 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4368/8823] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:54:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:55:01.201682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96c269-4fbb-48d0-91d8-e17d14632273 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:55:01.206720 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96c269-4fbb-48d0-91d8-e17d14632273 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:55:01.206995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96c269-4fbb-48d0-91d8-e17d14632273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:55:01.231095 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6c96c269-4fbb-48d0-91d8-e17d14632273 None None] MFA Rules not processed for user `988aca1c7805446cafeda57c8ed3cbcd`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:55:01.235644 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6c96c269-4fbb-48d0-91d8-e17d14632273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:55:01.245135 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4456/8824] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:55:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:55:01.957440 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf4d1-82cb-44a1-8b07-1d88ce1bd4d5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:55:01.957917 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf4d1-82cb-44a1-8b07-1d88ce1bd4d5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:55:01.958150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf4d1-82cb-44a1-8b07-1d88ce1bd4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:55:02.003568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d20cf4d1-82cb-44a1-8b07-1d88ce1bd4d5 None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:55:02.005568 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d20cf4d1-82cb-44a1-8b07-1d88ce1bd4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:55:02.012850 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4369/8825] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:55:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 21:55:02.025905 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96c269-4fbb-48d0-91d8-e17d14632273 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:55:02.036887 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96c269-4fbb-48d0-91d8-e17d14632273 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:55:02.042215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ee837-174d-4310-8b06-12f7447e5b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:55:02.042870 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ee837-174d-4310-8b06-12f7447e5b35 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:55:02.043144 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ee837-174d-4310-8b06-12f7447e5b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:55:02.043395 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ee837-174d-4310-8b06-12f7447e5b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:55:02.054347 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ee837-174d-4310-8b06-12f7447e5b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:55:02.057458 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ee837-174d-4310-8b06-12f7447e5b35 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:55:02.070221 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4457/8826] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:55:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:55:04.591491 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83ef79-8b64-42d8-a767-4690023df1fb None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:55:04.591952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83ef79-8b64-42d8-a767-4690023df1fb None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:55:04.591952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83ef79-8b64-42d8-a767-4690023df1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:55:04.614147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-dc83ef79-8b64-42d8-a767-4690023df1fb None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:55:04.615455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-dc83ef79-8b64-42d8-a767-4690023df1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:55:04.622665 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4370/8827] 10.222.0.16 () {58 vars in 1015 bytes} [Sat Aug 20 21:55:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:55:04.635025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ee837-174d-4310-8b06-12f7447e5b35 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:55:04.644143 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ee837-174d-4310-8b06-12f7447e5b35 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:55:04.649873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2e9ad-8fff-41d7-b476-f8bad138a7c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:55:04.650557 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2e9ad-8fff-41d7-b476-f8bad138a7c3 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:55:04.650892 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2e9ad-8fff-41d7-b476-f8bad138a7c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:55:04.651214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2e9ad-8fff-41d7-b476-f8bad138a7c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:55:04.673039 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2e9ad-8fff-41d7-b476-f8bad138a7c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:55:04.674921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2e9ad-8fff-41d7-b476-f8bad138a7c3 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:55:04.696207 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4458/8828] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:55:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:55:14.608419 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=27 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:55:14.681761 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=40 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:55:38.949011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83ef79-8b64-42d8-a767-4690023df1fb None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:55:38.962401 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83ef79-8b64-42d8-a767-4690023df1fb None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:55:38.968879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a4900-fa54-40d9-a1a2-048bb7471bcd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:55:38.969513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a4900-fa54-40d9-a1a2-048bb7471bcd service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:55:38.969786 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a4900-fa54-40d9-a1a2-048bb7471bcd service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:55:38.970071 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a4900-fa54-40d9-a1a2-048bb7471bcd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:55:38.985272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a4900-fa54-40d9-a1a2-048bb7471bcd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:55:38.986905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a4900-fa54-40d9-a1a2-048bb7471bcd service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:55:38.999063 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4371/8829] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:55:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:55:48.994656 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:56:34.163757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2e9ad-8fff-41d7-b476-f8bad138a7c3 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:34.176899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2e9ad-8fff-41d7-b476-f8bad138a7c3 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:34.184078 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14c62a-1e6f-4d05-801e-fa40496c0d24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:34.184904 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14c62a-1e6f-4d05-801e-fa40496c0d24 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:34.185196 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14c62a-1e6f-4d05-801e-fa40496c0d24 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:34.185462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14c62a-1e6f-4d05-801e-fa40496c0d24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:34.196358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14c62a-1e6f-4d05-801e-fa40496c0d24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:34.198364 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14c62a-1e6f-4d05-801e-fa40496c0d24 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:34.212297 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4459/8830] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:56:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:56:44.206489 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:56:44.489848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6f891-6f12-4040-a5a7-65550acd7e8f None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:44.490432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6f891-6f12-4040-a5a7-65550acd7e8f None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:44.490432 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6f891-6f12-4040-a5a7-65550acd7e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:44.527852 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-bfc6f891-6f12-4040-a5a7-65550acd7e8f None None] MFA Rules not processed for user `2b04f1e0e8094a14a3c1384faa601aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:56:44.529680 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-bfc6f891-6f12-4040-a5a7-65550acd7e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:44.539385 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4372/8831] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 21:56:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:56:44.554649 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14c62a-1e6f-4d05-801e-fa40496c0d24 service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:44.566270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14c62a-1e6f-4d05-801e-fa40496c0d24 service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:44.573782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc2519-5f16-40d5-83b3-d7a5f258e926 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:44.573782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc2519-5f16-40d5-83b3-d7a5f258e926 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:44.573782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc2519-5f16-40d5-83b3-d7a5f258e926 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:44.574064 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc2519-5f16-40d5-83b3-d7a5f258e926 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:44.585456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc2519-5f16-40d5-83b3-d7a5f258e926 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:44.587768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc2519-5f16-40d5-83b3-d7a5f258e926 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:44.603611 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4460/8832] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:56:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:56:44.826707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6f891-6f12-4040-a5a7-65550acd7e8f None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:44.921037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6f891-6f12-4040-a5a7-65550acd7e8f None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:44.926372 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:44.927052 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:44.927268 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:44.927533 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] PATH_INFO: `/v3/users/988aca1c7805446cafeda57c8ed3cbcd` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:44.928756 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=988aca1c7805446cafeda57c8ed3cbcd)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:44.930903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:44.987026 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '988aca1c7805446cafeda57c8ed3cbcd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:44.994535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '988aca1c7805446cafeda57c8ed3cbcd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.012646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '988aca1c7805446cafeda57c8ed3cbcd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.014090 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '988aca1c7805446cafeda57c8ed3cbcd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.014738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '988aca1c7805446cafeda57c8ed3cbcd'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.026839 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4373/8833] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:56:44 2022] DELETE /identity/v3/users/988aca1c7805446cafeda57c8ed3cbcd => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:56:45.031800 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde9759-a22c-4a1f-9aeb-124f3d335ef3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:45.032077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde9759-a22c-4a1f-9aeb-124f3d335ef3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:45.032337 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde9759-a22c-4a1f-9aeb-124f3d335ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:45.073133 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-ffde9759-a22c-4a1f-9aeb-124f3d335ef3 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:56:45.074405 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ffde9759-a22c-4a1f-9aeb-124f3d335ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:45.139229 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4461/8834] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:56:45.150115 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:45.151683 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:45.205689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:56:45.212987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944a4e3-f4a1-42a5-94cd-9a7ee9c58dfc admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:45.217765 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731c2f6-0f0d-4ddc-97c9-a2f6af7c37ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:45.218307 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731c2f6-0f0d-4ddc-97c9-a2f6af7c37ce service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:45.218573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731c2f6-0f0d-4ddc-97c9-a2f6af7c37ce service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:45.218860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731c2f6-0f0d-4ddc-97c9-a2f6af7c37ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:45.231548 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731c2f6-0f0d-4ddc-97c9-a2f6af7c37ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:45.233028 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731c2f6-0f0d-4ddc-97c9-a2f6af7c37ce service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:45.243107 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4374/8835] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:56:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:56:45.431725 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde9759-a22c-4a1f-9aeb-124f3d335ef3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:45.434762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-ffde9759-a22c-4a1f-9aeb-124f3d335ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:45.448194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde9759-a22c-4a1f-9aeb-124f3d335ef3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:45.453043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:45.453681 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:45.453954 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:45.454227 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] PATH_INFO: `/v3/projects/eaac203854f04e9ab195ab5a5ebdc9d1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:45.455372 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaac203854f04e9ab195ab5a5ebdc9d1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:45.457443 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:45.511393 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaac203854f04e9ab195ab5a5ebdc9d1', 'request_id': 'req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.519670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaac203854f04e9ab195ab5a5ebdc9d1', 'request_id': 'req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.525463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaac203854f04e9ab195ab5a5ebdc9d1', 'request_id': 'req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.537551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] The token cache is being invalidate because project eaac203854f04e9ab195ab5a5ebdc9d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:56:45.537833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.538496 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.540035 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4462/8836] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:56:45 2022] DELETE /identity/v3/projects/eaac203854f04e9ab195ab5a5ebdc9d1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:56:45.544251 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731c2f6-0f0d-4ddc-97c9-a2f6af7c37ce service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:45.554540 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731c2f6-0f0d-4ddc-97c9-a2f6af7c37ce service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:45.617471 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c42e7-be35-429a-a8d4-8a0b3bdf6a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:45.617993 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c42e7-be35-429a-a8d4-8a0b3bdf6a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:45.618272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c42e7-be35-429a-a8d4-8a0b3bdf6a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:45.618535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c42e7-be35-429a-a8d4-8a0b3bdf6a67 admin admin] PATH_INFO: `/v3/users/a02c9d866cba44ba91ccebe7e2aa0892` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:45.619661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c42e7-be35-429a-a8d4-8a0b3bdf6a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a02c9d866cba44ba91ccebe7e2aa0892)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:45.621641 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c42e7-be35-429a-a8d4-8a0b3bdf6a67 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:45.671605 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-903c42e7-be35-429a-a8d4-8a0b3bdf6a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a02c9d866cba44ba91ccebe7e2aa0892'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.679674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-903c42e7-be35-429a-a8d4-8a0b3bdf6a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a02c9d866cba44ba91ccebe7e2aa0892'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.692523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-903c42e7-be35-429a-a8d4-8a0b3bdf6a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a02c9d866cba44ba91ccebe7e2aa0892'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.693241 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-903c42e7-be35-429a-a8d4-8a0b3bdf6a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a02c9d866cba44ba91ccebe7e2aa0892'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.693872 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-903c42e7-be35-429a-a8d4-8a0b3bdf6a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a02c9d866cba44ba91ccebe7e2aa0892'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:45.706940 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4375/8837] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 21:56:45 2022] DELETE /identity/v3/users/a02c9d866cba44ba91ccebe7e2aa0892 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:56:46.004356 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:46.005860 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:46.020527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5f3c5-9f4d-491a-b43e-b6b8ad422518 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:46.073855 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:56:46.084743 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85b1a8-bd19-462a-a4e9-33201e08c874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:46.085581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85b1a8-bd19-462a-a4e9-33201e08c874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:46.085945 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85b1a8-bd19-462a-a4e9-33201e08c874 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:46.086201 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85b1a8-bd19-462a-a4e9-33201e08c874 admin admin] PATH_INFO: `/v3/projects/406b827bb2494589912975e0d5ec2c56` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:46.087436 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85b1a8-bd19-462a-a4e9-33201e08c874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=406b827bb2494589912975e0d5ec2c56)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:46.089402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85b1a8-bd19-462a-a4e9-33201e08c874 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:46.138311 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ab85b1a8-bd19-462a-a4e9-33201e08c874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '406b827bb2494589912975e0d5ec2c56', 'request_id': 'req-ab85b1a8-bd19-462a-a4e9-33201e08c874'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:46.147635 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ab85b1a8-bd19-462a-a4e9-33201e08c874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '406b827bb2494589912975e0d5ec2c56', 'request_id': 'req-ab85b1a8-bd19-462a-a4e9-33201e08c874'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:46.153780 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ab85b1a8-bd19-462a-a4e9-33201e08c874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '406b827bb2494589912975e0d5ec2c56', 'request_id': 'req-ab85b1a8-bd19-462a-a4e9-33201e08c874'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:46.165386 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ab85b1a8-bd19-462a-a4e9-33201e08c874 admin admin] The token cache is being invalidate because project 406b827bb2494589912975e0d5ec2c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 21:56:46.165675 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ab85b1a8-bd19-462a-a4e9-33201e08c874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:46.166358 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-ab85b1a8-bd19-462a-a4e9-33201e08c874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 21:56:46.167815 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4463/8838] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 21:56:46 2022] DELETE /identity/v3/projects/406b827bb2494589912975e0d5ec2c56 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:56:46.173833 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c0e94-124c-47cf-965a-1bda9c3b7c19 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:46.174126 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c0e94-124c-47cf-965a-1bda9c3b7c19 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:46.174415 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c0e94-124c-47cf-965a-1bda9c3b7c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:46.218378 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-955c0e94-124c-47cf-965a-1bda9c3b7c19 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:56:46.219606 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-955c0e94-124c-47cf-965a-1bda9c3b7c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:46.271023 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:56:46.295565 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4376/8839] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:56:46.298114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9907cd-d267-4f20-bc48-ef2a73c2e0fa None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:46.298389 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9907cd-d267-4f20-bc48-ef2a73c2e0fa None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:46.298661 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9907cd-d267-4f20-bc48-ef2a73c2e0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:46.319439 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-9d9907cd-d267-4f20-bc48-ef2a73c2e0fa None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:56:46.320703 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9d9907cd-d267-4f20-bc48-ef2a73c2e0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:46.330233 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4464/8840] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:56:46.335096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c0e94-124c-47cf-965a-1bda9c3b7c19 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:46.345932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c0e94-124c-47cf-965a-1bda9c3b7c19 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:46.351317 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a124-ce78-4a7d-b313-cbf3486802a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:46.351906 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a124-ce78-4a7d-b313-cbf3486802a9 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:46.352183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a124-ce78-4a7d-b313-cbf3486802a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:46.352479 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a124-ce78-4a7d-b313-cbf3486802a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:46.353130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96a124-ce78-4a7d-b313-cbf3486802a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 21:56:46.353436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96a124-ce78-4a7d-b313-cbf3486802a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:46.355051 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96a124-ce78-4a7d-b313-cbf3486802a9 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:46.368308 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4377/8841] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 21:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 21:56:46.373055 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9907cd-d267-4f20-bc48-ef2a73c2e0fa None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:46.385747 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9907cd-d267-4f20-bc48-ef2a73c2e0fa None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:46.390854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83d36c-0eb4-41a5-8fd6-56b4b1336597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:46.391456 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83d36c-0eb4-41a5-8fd6-56b4b1336597 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:46.391731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83d36c-0eb4-41a5-8fd6-56b4b1336597 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:46.392013 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83d36c-0eb4-41a5-8fd6-56b4b1336597 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:46.392653 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83d36c-0eb4-41a5-8fd6-56b4b1336597 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:46.394565 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83d36c-0eb4-41a5-8fd6-56b4b1336597 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:46.426940 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4465/8842] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:56:46 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:56:46.431535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a124-ce78-4a7d-b313-cbf3486802a9 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:46.441895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a124-ce78-4a7d-b313-cbf3486802a9 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:46.499078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe93fc-bfa2-4f1d-8a14-540adbea1cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:46.499707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe93fc-bfa2-4f1d-8a14-540adbea1cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:46.499983 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe93fc-bfa2-4f1d-8a14-540adbea1cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:46.500245 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe93fc-bfa2-4f1d-8a14-540adbea1cf8 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:46.500998 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe93fc-bfa2-4f1d-8a14-540adbea1cf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:46.502907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe93fc-bfa2-4f1d-8a14-540adbea1cf8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:46.524974 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4378/8843] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:56:46 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 21:56:46.529202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83d36c-0eb4-41a5-8fd6-56b4b1336597 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:46.540038 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83d36c-0eb4-41a5-8fd6-56b4b1336597 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:46.545570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f462d-0870-4efe-b089-502319d1b387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:46.546189 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f462d-0870-4efe-b089-502319d1b387 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:46.546487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f462d-0870-4efe-b089-502319d1b387 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:46.546744 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f462d-0870-4efe-b089-502319d1b387 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:46.547399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f462d-0870-4efe-b089-502319d1b387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:46.549050 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f462d-0870-4efe-b089-502319d1b387 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:46.559214 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4466/8844] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:56:46.563323 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe93fc-bfa2-4f1d-8a14-540adbea1cf8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:46.573512 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe93fc-bfa2-4f1d-8a14-540adbea1cf8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:46.579455 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e051a7b-2781-4a17-a563-45334fad7d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:46.580055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e051a7b-2781-4a17-a563-45334fad7d9a admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:46.580299 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e051a7b-2781-4a17-a563-45334fad7d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:46.580626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e051a7b-2781-4a17-a563-45334fad7d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:46.581233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e051a7b-2781-4a17-a563-45334fad7d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:46.582870 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e051a7b-2781-4a17-a563-45334fad7d9a admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:46.594968 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4379/8845] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 21:56:46.599442 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f462d-0870-4efe-b089-502319d1b387 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:46.610350 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f462d-0870-4efe-b089-502319d1b387 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:46.615642 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8f0c4-8312-435b-a05f-f14c550699d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:46.616295 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8f0c4-8312-435b-a05f-f14c550699d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:46.616599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8f0c4-8312-435b-a05f-f14c550699d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:46.616856 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8f0c4-8312-435b-a05f-f14c550699d6 admin admin] PATH_INFO: `/v3/projects/392a5a2f4ca04e8ca5043d37dcef9ef1/users/20d5d31e535644fcb48119d7888aba2f/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:46.631338 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8f0c4-8312-435b-a05f-f14c550699d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392a5a2f4ca04e8ca5043d37dcef9ef1, user_id=20d5d31e535644fcb48119d7888aba2f, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:46.634031 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8f0c4-8312-435b-a05f-f14c550699d6 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:46.649000 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4467/8846] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:56:46 2022] PUT /identity/v3/projects/392a5a2f4ca04e8ca5043d37dcef9ef1/users/20d5d31e535644fcb48119d7888aba2f/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 20 21:56:46.654204 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb50987-bf2f-4230-9977-26090e575ee1 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:46.654482 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb50987-bf2f-4230-9977-26090e575ee1 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:46.654807 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb50987-bf2f-4230-9977-26090e575ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:46.675022 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-ecb50987-bf2f-4230-9977-26090e575ee1 None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:56:46.676222 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-ecb50987-bf2f-4230-9977-26090e575ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:46.742526 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4380/8847] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:56:46.753181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8f0c4-8312-435b-a05f-f14c550699d6 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:46.755844 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-98d8f0c4-8312-435b-a05f-f14c550699d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:46.815867 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8f0c4-8312-435b-a05f-f14c550699d6 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:46.820272 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d4ebc-44d7-44f0-9f98-278976ffc7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:46.820923 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d4ebc-44d7-44f0-9f98-278976ffc7df service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:46.821194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d4ebc-44d7-44f0-9f98-278976ffc7df service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:46.821462 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d4ebc-44d7-44f0-9f98-278976ffc7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:46.831765 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d4ebc-44d7-44f0-9f98-278976ffc7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:46.833246 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d4ebc-44d7-44f0-9f98-278976ffc7df service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:46.843369 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4468/8848] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:56:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:56:51.069153 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9517b-413c-4836-a8df-7ff70e1709a6 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:51.069629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9517b-413c-4836-a8df-7ff70e1709a6 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:51.069629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9517b-413c-4836-a8df-7ff70e1709a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:51.106369 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-12c9517b-413c-4836-a8df-7ff70e1709a6 None None] MFA Rules not processed for user `20d5d31e535644fcb48119d7888aba2f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:56:51.107718 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-12c9517b-413c-4836-a8df-7ff70e1709a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:51.167945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:56:51.195931 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4381/8849] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:56:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:56:51.197111 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d4ebc-44d7-44f0-9f98-278976ffc7df service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:51.208126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d4ebc-44d7-44f0-9f98-278976ffc7df service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:51.213684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d63dd1-6695-45ae-9d74-a0d2bfcac9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:51.214974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d63dd1-6695-45ae-9d74-a0d2bfcac9a0 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:51.215251 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d63dd1-6695-45ae-9d74-a0d2bfcac9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:51.215517 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d63dd1-6695-45ae-9d74-a0d2bfcac9a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:51.216306 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d63dd1-6695-45ae-9d74-a0d2bfcac9a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:51.219124 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d63dd1-6695-45ae-9d74-a0d2bfcac9a0 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:51.248754 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:56:51.266714 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9517b-413c-4836-a8df-7ff70e1709a6 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:51.275059 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4469/8850] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 21:56:51 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:56:51.278706 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9517b-413c-4836-a8df-7ff70e1709a6 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:51.346880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bf2ae-776a-40b5-ab71-e28834bafb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:51.347561 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bf2ae-776a-40b5-ab71-e28834bafb49 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:51.347840 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bf2ae-776a-40b5-ab71-e28834bafb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:51.348102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bf2ae-776a-40b5-ab71-e28834bafb49 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:51.348764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bf2ae-776a-40b5-ab71-e28834bafb49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:51.350707 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bf2ae-776a-40b5-ab71-e28834bafb49 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:51.375632 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4382/8851] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 21:56:51 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 21:56:51.379776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d63dd1-6695-45ae-9d74-a0d2bfcac9a0 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:51.391068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d63dd1-6695-45ae-9d74-a0d2bfcac9a0 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:51.395593 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e6149-3cd5-49f8-b7db-a78e3a7c2bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:51.396194 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e6149-3cd5-49f8-b7db-a78e3a7c2bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:51.396510 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e6149-3cd5-49f8-b7db-a78e3a7c2bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:51.396779 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e6149-3cd5-49f8-b7db-a78e3a7c2bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:51.397418 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e6149-3cd5-49f8-b7db-a78e3a7c2bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:51.399012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e6149-3cd5-49f8-b7db-a78e3a7c2bc9 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:51.410204 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4470/8852] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:56:51.414466 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bf2ae-776a-40b5-ab71-e28834bafb49 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:51.425815 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bf2ae-776a-40b5-ab71-e28834bafb49 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:51.432972 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82450d52-773e-4942-b3a9-197d690f09c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:51.433956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82450d52-773e-4942-b3a9-197d690f09c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:51.434325 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82450d52-773e-4942-b3a9-197d690f09c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:51.434663 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82450d52-773e-4942-b3a9-197d690f09c8 admin admin] PATH_INFO: `/v3/projects/1c9aa95ee38645e98b0a59f7e47199ba/users/eafdcadb96d24557877a98b6a7237c90/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:51.456253 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82450d52-773e-4942-b3a9-197d690f09c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9aa95ee38645e98b0a59f7e47199ba, user_id=eafdcadb96d24557877a98b6a7237c90, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:51.459388 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82450d52-773e-4942-b3a9-197d690f09c8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:51.474933 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4383/8853] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:56:51 2022] PUT /identity/v3/projects/1c9aa95ee38645e98b0a59f7e47199ba/users/eafdcadb96d24557877a98b6a7237c90/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:56:51.479043 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e6149-3cd5-49f8-b7db-a78e3a7c2bc9 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:51.489889 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e6149-3cd5-49f8-b7db-a78e3a7c2bc9 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:51.548414 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43225a40-d847-47b3-98a1-437c9cdb6836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:51.549035 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43225a40-d847-47b3-98a1-437c9cdb6836 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:51.549305 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43225a40-d847-47b3-98a1-437c9cdb6836 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:51.549599 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43225a40-d847-47b3-98a1-437c9cdb6836 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:51.550271 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43225a40-d847-47b3-98a1-437c9cdb6836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:51.552073 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43225a40-d847-47b3-98a1-437c9cdb6836 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:51.562245 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4471/8854] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 21:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 21:56:51.566586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82450d52-773e-4942-b3a9-197d690f09c8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:51.576897 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82450d52-773e-4942-b3a9-197d690f09c8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:51.582500 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a37269-ce2a-4d2f-abc5-42797a752ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:51.583293 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a37269-ce2a-4d2f-abc5-42797a752ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:51.583662 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a37269-ce2a-4d2f-abc5-42797a752ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:51.583925 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a37269-ce2a-4d2f-abc5-42797a752ba8 admin admin] PATH_INFO: `/v3/projects/1c9aa95ee38645e98b0a59f7e47199ba/users/eafdcadb96d24557877a98b6a7237c90/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:51.586667 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a37269-ce2a-4d2f-abc5-42797a752ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9aa95ee38645e98b0a59f7e47199ba, user_id=eafdcadb96d24557877a98b6a7237c90, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:51.589370 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a37269-ce2a-4d2f-abc5-42797a752ba8 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:51.600439 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4384/8855] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 21:56:51 2022] PUT /identity/v3/projects/1c9aa95ee38645e98b0a59f7e47199ba/users/eafdcadb96d24557877a98b6a7237c90/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 21:56:56.169296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb12aa2-0d8f-4b7d-b763-ece099c474e3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:56.169787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb12aa2-0d8f-4b7d-b763-ece099c474e3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:56.170335 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb12aa2-0d8f-4b7d-b763-ece099c474e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:56.230301 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-2fb12aa2-0d8f-4b7d-b763-ece099c474e3 None None] MFA Rules not processed for user `eafdcadb96d24557877a98b6a7237c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:56:56.232322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2fb12aa2-0d8f-4b7d-b763-ece099c474e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:56.314979 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4472/8856] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:56:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:56:56.332954 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a37269-ce2a-4d2f-abc5-42797a752ba8 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:56.335343 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-97a37269-ce2a-4d2f-abc5-42797a752ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:56.449213 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a37269-ce2a-4d2f-abc5-42797a752ba8 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:56.455547 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8391b-3ebd-4832-b6fc-99f6d4972da6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:56.456180 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8391b-3ebd-4832-b6fc-99f6d4972da6 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:56.456523 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8391b-3ebd-4832-b6fc-99f6d4972da6 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:56.456793 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8391b-3ebd-4832-b6fc-99f6d4972da6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:56.498665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:56:56.517901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8391b-3ebd-4832-b6fc-99f6d4972da6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:56.520661 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8391b-3ebd-4832-b6fc-99f6d4972da6 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:56.533012 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4385/8857] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:56:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:56:57.592530 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c390fc-ed9f-4707-8dec-05806abf2ce9 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:57.593093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c390fc-ed9f-4707-8dec-05806abf2ce9 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:57.593454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c390fc-ed9f-4707-8dec-05806abf2ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:57.612928 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-49c390fc-ed9f-4707-8dec-05806abf2ce9 None None] MFA Rules not processed for user `20d5d31e535644fcb48119d7888aba2f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:56:57.614119 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-49c390fc-ed9f-4707-8dec-05806abf2ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:57.621681 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4473/8858] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 21:56:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:56:57.968591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546002c2-c2dd-4f55-a334-f73e88c51e7d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:57.969089 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546002c2-c2dd-4f55-a334-f73e88c51e7d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:57.969347 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546002c2-c2dd-4f55-a334-f73e88c51e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:57.989658 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-546002c2-c2dd-4f55-a334-f73e88c51e7d None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:56:57.990879 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-546002c2-c2dd-4f55-a334-f73e88c51e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:58.045576 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4386/8859] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:56:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:56:58.057911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c390fc-ed9f-4707-8dec-05806abf2ce9 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:58.068841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c390fc-ed9f-4707-8dec-05806abf2ce9 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:58.072670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96545ebe-c39c-4b4b-8f94-374ed2f2072e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:58.073274 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96545ebe-c39c-4b4b-8f94-374ed2f2072e service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:58.073541 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96545ebe-c39c-4b4b-8f94-374ed2f2072e service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:58.073799 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96545ebe-c39c-4b4b-8f94-374ed2f2072e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:58.081385 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96545ebe-c39c-4b4b-8f94-374ed2f2072e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:58.082817 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96545ebe-c39c-4b4b-8f94-374ed2f2072e service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:58.090817 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4474/8860] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:56:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:56:58.431488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546002c2-c2dd-4f55-a334-f73e88c51e7d None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:56:58.433009 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-546002c2-c2dd-4f55-a334-f73e88c51e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:58.514510 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546002c2-c2dd-4f55-a334-f73e88c51e7d None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:56:58.520963 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c8492-945f-48ff-89e8-073c89f6e6bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:56:58.521595 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c8492-945f-48ff-89e8-073c89f6e6bb service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:56:58.521901 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c8492-945f-48ff-89e8-073c89f6e6bb service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:56:58.522142 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c8492-945f-48ff-89e8-073c89f6e6bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:56:58.524279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a22c8492-945f-48ff-89e8-073c89f6e6bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:56:58.536238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c8492-945f-48ff-89e8-073c89f6e6bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:56:58.537848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c8492-945f-48ff-89e8-073c89f6e6bb service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:56:58.547898 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4387/8861] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:56:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:57:08.087968 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=77 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:57:08.543697 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=65 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:57:10.561079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96545ebe-c39c-4b4b-8f94-374ed2f2072e service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:57:10.586379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96545ebe-c39c-4b4b-8f94-374ed2f2072e service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:57:10.591545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18080f-b398-4495-bc1e-6844526368a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:57:10.592147 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18080f-b398-4495-bc1e-6844526368a9 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:57:10.592468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18080f-b398-4495-bc1e-6844526368a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:57:10.592729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18080f-b398-4495-bc1e-6844526368a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:57:10.602813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18080f-b398-4495-bc1e-6844526368a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:57:10.604234 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18080f-b398-4495-bc1e-6844526368a9 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:57:10.614868 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4475/8862] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:57:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 21:57:20.609786 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=10 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:59:39.188164 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c8492-945f-48ff-89e8-073c89f6e6bb service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:59:39.189804 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a22c8492-945f-48ff-89e8-073c89f6e6bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:59:39.280242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c8492-945f-48ff-89e8-073c89f6e6bb service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:59:39.284481 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c4b09-7f93-440b-8de0-25d800ad82d4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:59:39.285085 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c4b09-7f93-440b-8de0-25d800ad82d4 service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:59:39.285356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c4b09-7f93-440b-8de0-25d800ad82d4 service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:59:39.285621 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c4b09-7f93-440b-8de0-25d800ad82d4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:59:39.287563 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5a0c4b09-7f93-440b-8de0-25d800ad82d4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:59:39.302172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c4b09-7f93-440b-8de0-25d800ad82d4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:59:39.303546 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c4b09-7f93-440b-8de0-25d800ad82d4 service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:59:39.315055 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4388/8863] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:59:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:59:39.487096 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18080f-b398-4495-bc1e-6844526368a9 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:59:39.495854 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18080f-b398-4495-bc1e-6844526368a9 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:59:39.500146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc19e9-2d3b-4192-8e22-ef854d3ca324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:59:39.500873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc19e9-2d3b-4192-8e22-ef854d3ca324 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:59:39.501116 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc19e9-2d3b-4192-8e22-ef854d3ca324 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:59:39.501449 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc19e9-2d3b-4192-8e22-ef854d3ca324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:59:39.504900 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0ddc19e9-2d3b-4192-8e22-ef854d3ca324 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:59:39.527313 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc19e9-2d3b-4192-8e22-ef854d3ca324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:59:39.529592 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc19e9-2d3b-4192-8e22-ef854d3ca324 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:59:39.542278 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4476/8864] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:59:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:59:49.309732 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=39 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:59:49.536766 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 21:59:54.641038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c4b09-7f93-440b-8de0-25d800ad82d4 service glance] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:59:54.652968 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c4b09-7f93-440b-8de0-25d800ad82d4 service glance] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:59:54.659298 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398c74e-9725-4408-9b4c-1c080b48d5b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:59:54.659945 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398c74e-9725-4408-9b4c-1c080b48d5b6 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:59:54.660409 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398c74e-9725-4408-9b4c-1c080b48d5b6 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:59:54.660678 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398c74e-9725-4408-9b4c-1c080b48d5b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:59:54.662587 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4398c74e-9725-4408-9b4c-1c080b48d5b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:59:54.677774 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398c74e-9725-4408-9b4c-1c080b48d5b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:59:54.679381 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398c74e-9725-4408-9b4c-1c080b48d5b6 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:59:54.690454 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4389/8865] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 21:59:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 21:59:57.630463 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2b133-b040-474b-9cf9-a292415e5fc2 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:59:57.630975 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2b133-b040-474b-9cf9-a292415e5fc2 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:59:57.631070 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2b133-b040-474b-9cf9-a292415e5fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:59:57.689215 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-7fd2b133-b040-474b-9cf9-a292415e5fc2 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 21:59:57.691058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7fd2b133-b040-474b-9cf9-a292415e5fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:59:57.768825 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4477/8866] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 21:59:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 21:59:57.823987 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398c74e-9725-4408-9b4c-1c080b48d5b6 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 21:59:57.826011 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4398c74e-9725-4408-9b4c-1c080b48d5b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 21:59:57.936529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398c74e-9725-4408-9b4c-1c080b48d5b6 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 21:59:57.943162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2946140-8221-40a6-95c9-a796a048771a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 21:59:57.944033 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2946140-8221-40a6-95c9-a796a048771a service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 21:59:57.944469 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2946140-8221-40a6-95c9-a796a048771a service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 21:59:57.944868 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2946140-8221-40a6-95c9-a796a048771a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 21:59:57.955417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2946140-8221-40a6-95c9-a796a048771a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 21:59:57.957056 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2946140-8221-40a6-95c9-a796a048771a service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 21:59:57.967333 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4390/8867] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 21:59:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 22:00:07.765795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=40 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:00:07.963520 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=47 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:00:08.191058 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2b133-b040-474b-9cf9-a292415e5fc2 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:00:08.192757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7fd2b133-b040-474b-9cf9-a292415e5fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:00:08.204990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2b133-b040-474b-9cf9-a292415e5fc2 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:00:08.210264 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479be25-381d-49b8-9da1-5a5a5a97101a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:00:08.210882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479be25-381d-49b8-9da1-5a5a5a97101a service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:08.211131 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479be25-381d-49b8-9da1-5a5a5a97101a service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:08.211390 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479be25-381d-49b8-9da1-5a5a5a97101a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:08.213200 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-4479be25-381d-49b8-9da1-5a5a5a97101a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:00:08.223267 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479be25-381d-49b8-9da1-5a5a5a97101a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:00:08.225214 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479be25-381d-49b8-9da1-5a5a5a97101a service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:00:08.236586 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4478/8868] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:00:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:00:10.904242 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66241d19-e2bb-4f3e-92cc-d954480e7d7d None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:10.904957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66241d19-e2bb-4f3e-92cc-d954480e7d7d None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:10.904957 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66241d19-e2bb-4f3e-92cc-d954480e7d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:10.925751 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-66241d19-e2bb-4f3e-92cc-d954480e7d7d None None] MFA Rules not processed for user `df852af59e7743f7a8798d9edf8ebd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:00:10.926953 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-66241d19-e2bb-4f3e-92cc-d954480e7d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:00:10.937048 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4391/8869] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 22:00:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:00:10.948631 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479be25-381d-49b8-9da1-5a5a5a97101a service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:00:10.958172 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479be25-381d-49b8-9da1-5a5a5a97101a service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:00:10.962949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f042f9-7ba3-45e5-9d33-2ae8c9ab5dec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:00:10.963538 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f042f9-7ba3-45e5-9d33-2ae8c9ab5dec service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:10.963805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f042f9-7ba3-45e5-9d33-2ae8c9ab5dec service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:10.964068 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f042f9-7ba3-45e5-9d33-2ae8c9ab5dec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:10.974581 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f042f9-7ba3-45e5-9d33-2ae8c9ab5dec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:00:10.976086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f042f9-7ba3-45e5-9d33-2ae8c9ab5dec service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:00:10.987197 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4479/8870] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:00:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 22:00:11.055958 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f81088-3185-440d-a87e-d94c9196c9b5 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:11.056349 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f81088-3185-440d-a87e-d94c9196c9b5 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:11.056709 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f81088-3185-440d-a87e-d94c9196c9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:11.076949 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-a0f81088-3185-440d-a87e-d94c9196c9b5 None None] MFA Rules not processed for user `22bb014e04774c278c99496bf64b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:00:11.078199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-a0f81088-3185-440d-a87e-d94c9196c9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:00:11.086022 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4392/8871] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Aug 20 22:00:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 22:00:11.097820 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f042f9-7ba3-45e5-9d33-2ae8c9ab5dec service glance] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:00:11.107212 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f042f9-7ba3-45e5-9d33-2ae8c9ab5dec service glance] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:00:11.110882 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97613d-dd9b-4633-9812-837f1756ccf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:00:11.111468 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97613d-dd9b-4633-9812-837f1756ccf9 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:11.111738 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97613d-dd9b-4633-9812-837f1756ccf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:11.112001 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97613d-dd9b-4633-9812-837f1756ccf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:11.120090 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a97613d-dd9b-4633-9812-837f1756ccf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:00:11.121838 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a97613d-dd9b-4633-9812-837f1756ccf9 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:00:11.131069 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4480/8872] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:00:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:00:11.209592 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f81088-3185-440d-a87e-d94c9196c9b5 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:00:11.219492 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f81088-3185-440d-a87e-d94c9196c9b5 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:00:11.226289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb727d04-4c92-49ee-81ec-6e9a1b00c340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:00:11.226289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb727d04-4c92-49ee-81ec-6e9a1b00c340 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:11.226289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb727d04-4c92-49ee-81ec-6e9a1b00c340 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:11.226289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb727d04-4c92-49ee-81ec-6e9a1b00c340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:11.227781 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-eb727d04-4c92-49ee-81ec-6e9a1b00c340 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:00:11.244576 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb727d04-4c92-49ee-81ec-6e9a1b00c340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:00:11.246094 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb727d04-4c92-49ee-81ec-6e9a1b00c340 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:00:11.258519 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4393/8873] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:00:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 20 22:00:11.323300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97613d-dd9b-4633-9812-837f1756ccf9 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:00:11.324998 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0a97613d-dd9b-4633-9812-837f1756ccf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:00:11.350989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97613d-dd9b-4633-9812-837f1756ccf9 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:00:11.356019 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9d6e7-02e2-4dc2-84e1-700b926d8e7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:00:11.356679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9d6e7-02e2-4dc2-84e1-700b926d8e7b service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:11.356962 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9d6e7-02e2-4dc2-84e1-700b926d8e7b service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:11.357197 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9d6e7-02e2-4dc2-84e1-700b926d8e7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:11.359062 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9cb9d6e7-02e2-4dc2-84e1-700b926d8e7b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:00:11.370460 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9d6e7-02e2-4dc2-84e1-700b926d8e7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:00:11.372202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9d6e7-02e2-4dc2-84e1-700b926d8e7b service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:00:11.382357 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4481/8874] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:00:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:00:12.248552 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb727d04-4c92-49ee-81ec-6e9a1b00c340 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:00:12.260128 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb727d04-4c92-49ee-81ec-6e9a1b00c340 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:00:12.265738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d8723-d7a2-47d0-b317-cc0815289cb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:00:12.266644 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d8723-d7a2-47d0-b317-cc0815289cb5 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:12.267047 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d8723-d7a2-47d0-b317-cc0815289cb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:12.267414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d8723-d7a2-47d0-b317-cc0815289cb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:12.269460 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-497d8723-d7a2-47d0-b317-cc0815289cb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:00:12.280294 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d8723-d7a2-47d0-b317-cc0815289cb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:00:12.281764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d8723-d7a2-47d0-b317-cc0815289cb5 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:00:12.290041 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4394/8875] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:00:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:00:13.112984 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9d6e7-02e2-4dc2-84e1-700b926d8e7b service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:00:13.126771 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9d6e7-02e2-4dc2-84e1-700b926d8e7b service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:00:13.132181 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758d590-1f22-4fce-8b04-b08278199ac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:00:13.133026 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758d590-1f22-4fce-8b04-b08278199ac7 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:13.133296 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758d590-1f22-4fce-8b04-b08278199ac7 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:13.133554 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758d590-1f22-4fce-8b04-b08278199ac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:13.135259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-0758d590-1f22-4fce-8b04-b08278199ac7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:00:13.147873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758d590-1f22-4fce-8b04-b08278199ac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:00:13.149503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758d590-1f22-4fce-8b04-b08278199ac7 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:00:13.161651 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4482/8876] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:00:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:00:13.845036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c9b6c-e919-4ce2-98db-820e0aad05b4 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:13.845528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c9b6c-e919-4ce2-98db-820e0aad05b4 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:13.845609 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c9b6c-e919-4ce2-98db-820e0aad05b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:13.866358 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8d4c9b6c-e919-4ce2-98db-820e0aad05b4 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:00:13.867794 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8d4c9b6c-e919-4ce2-98db-820e0aad05b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:00:13.875338 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4395/8877] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 22:00:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:00:13.888228 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758d590-1f22-4fce-8b04-b08278199ac7 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:00:13.900045 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758d590-1f22-4fce-8b04-b08278199ac7 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:00:13.905407 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c525e8-12bb-4ff9-ba48-43c32c706ab4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:00:13.906010 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c525e8-12bb-4ff9-ba48-43c32c706ab4 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:13.906282 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c525e8-12bb-4ff9-ba48-43c32c706ab4 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:13.906545 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c525e8-12bb-4ff9-ba48-43c32c706ab4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:13.915326 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c525e8-12bb-4ff9-ba48-43c32c706ab4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:00:13.916952 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c525e8-12bb-4ff9-ba48-43c32c706ab4 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:00:13.925997 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4483/8878] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 22:00:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 22:00:23.861398 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=21 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:00:23.923012 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=40 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:00:29.736978 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd0430-eff0-41c4-a37d-ce914ff53e0b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:29.737493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd0430-eff0-41c4-a37d-ce914ff53e0b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:29.737493 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd0430-eff0-41c4-a37d-ce914ff53e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:29.766316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-67cd0430-eff0-41c4-a37d-ce914ff53e0b None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:00:29.767502 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-67cd0430-eff0-41c4-a37d-ce914ff53e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:00:29.775883 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4396/8879] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 22:00:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 22:00:29.788625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c525e8-12bb-4ff9-ba48-43c32c706ab4 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:00:29.799580 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c525e8-12bb-4ff9-ba48-43c32c706ab4 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:00:29.805729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e326f-607b-45eb-9c69-ac5a7c883992 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:00:29.806320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e326f-607b-45eb-9c69-ac5a7c883992 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:29.806588 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e326f-607b-45eb-9c69-ac5a7c883992 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:29.806849 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e326f-607b-45eb-9c69-ac5a7c883992 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:29.815224 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e326f-607b-45eb-9c69-ac5a7c883992 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:00:29.816835 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e326f-607b-45eb-9c69-ac5a7c883992 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:00:29.824782 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4484/8880] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 22:00:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 22:00:39.758244 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:00:39.821752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:00:43.652322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd0430-eff0-41c4-a37d-ce914ff53e0b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:00:43.690162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd0430-eff0-41c4-a37d-ce914ff53e0b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:00:43.696692 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb0402-3cc9-49a3-a97d-766836868e0f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:00:43.697505 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb0402-3cc9-49a3-a97d-766836868e0f service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:00:43.697779 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb0402-3cc9-49a3-a97d-766836868e0f service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:00:43.698040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb0402-3cc9-49a3-a97d-766836868e0f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:00:43.700322 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e8fb0402-3cc9-49a3-a97d-766836868e0f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:00:43.716530 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb0402-3cc9-49a3-a97d-766836868e0f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:00:43.718121 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb0402-3cc9-49a3-a97d-766836868e0f service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:00:43.727879 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4397/8881] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:00:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:00:53.723898 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=17 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:01:40.155939 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e326f-607b-45eb-9c69-ac5a7c883992 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:01:40.206737 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e326f-607b-45eb-9c69-ac5a7c883992 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:01:40.212866 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1e972-54fe-4ae7-948e-ad41c9912b62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:01:40.213453 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1e972-54fe-4ae7-948e-ad41c9912b62 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:01:40.213726 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1e972-54fe-4ae7-948e-ad41c9912b62 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:01:40.213990 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1e972-54fe-4ae7-948e-ad41c9912b62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:01:40.216077 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-84b1e972-54fe-4ae7-948e-ad41c9912b62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:01:40.248803 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1e972-54fe-4ae7-948e-ad41c9912b62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:01:40.250446 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1e972-54fe-4ae7-948e-ad41c9912b62 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:01:40.260677 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4485/8882] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:01:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:01:45.532234 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb0402-3cc9-49a3-a97d-766836868e0f service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:01:45.534101 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e8fb0402-3cc9-49a3-a97d-766836868e0f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:01:45.582087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb0402-3cc9-49a3-a97d-766836868e0f service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:01:45.588211 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc7a46-3a6e-4653-a259-71db9749d5c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:01:45.588860 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc7a46-3a6e-4653-a259-71db9749d5c0 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:01:45.589135 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc7a46-3a6e-4653-a259-71db9749d5c0 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:01:45.589408 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc7a46-3a6e-4653-a259-71db9749d5c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:01:45.591173 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-6fcc7a46-3a6e-4653-a259-71db9749d5c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:01:45.604140 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc7a46-3a6e-4653-a259-71db9749d5c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:01:45.605783 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc7a46-3a6e-4653-a259-71db9749d5c0 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:01:45.625795 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4398/8883] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:01:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:01:50.256275 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=24 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:01:50.536320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544eba8f-72b2-409b-b1da-c1893bab6fee None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:01:50.536639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544eba8f-72b2-409b-b1da-c1893bab6fee None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:01:50.536914 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544eba8f-72b2-409b-b1da-c1893bab6fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:01:50.556670 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-544eba8f-72b2-409b-b1da-c1893bab6fee None None] MFA Rules not processed for user `20d5d31e535644fcb48119d7888aba2f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:01:50.557916 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-544eba8f-72b2-409b-b1da-c1893bab6fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:01:50.564844 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4486/8884] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 22:01:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:01:52.463738 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc7a46-3a6e-4653-a259-71db9749d5c0 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:01:52.473441 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc7a46-3a6e-4653-a259-71db9749d5c0 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:01:52.477039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147247b5-540f-4b07-b604-b54e5dcb665e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:01:52.477624 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147247b5-540f-4b07-b604-b54e5dcb665e service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:01:52.477895 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147247b5-540f-4b07-b604-b54e5dcb665e service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:01:52.478156 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147247b5-540f-4b07-b604-b54e5dcb665e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:01:52.540646 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147247b5-540f-4b07-b604-b54e5dcb665e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:01:52.542087 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147247b5-540f-4b07-b604-b54e5dcb665e service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:01:52.552942 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4399/8885] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:01:55.851598 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544eba8f-72b2-409b-b1da-c1893bab6fee None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:01:55.864643 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544eba8f-72b2-409b-b1da-c1893bab6fee None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:01:55.870222 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d8879-127f-41a9-b1fa-bb680c96cd4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:01:55.870925 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d8879-127f-41a9-b1fa-bb680c96cd4d service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:01:55.871259 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d8879-127f-41a9-b1fa-bb680c96cd4d service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:01:55.871589 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d8879-127f-41a9-b1fa-bb680c96cd4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:01:55.873762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-8b7d8879-127f-41a9-b1fa-bb680c96cd4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:01:55.886961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d8879-127f-41a9-b1fa-bb680c96cd4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:01:55.888930 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d8879-127f-41a9-b1fa-bb680c96cd4d service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:01:55.901744 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4487/8886] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:01:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:01:59.271497 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147247b5-540f-4b07-b604-b54e5dcb665e service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:01:59.281626 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147247b5-540f-4b07-b604-b54e5dcb665e service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:01:59.287515 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:01:59.288157 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:01:59.288470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:01:59.288767 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] PATH_INFO: `/v3/users/20d5d31e535644fcb48119d7888aba2f` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:01:59.289981 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20d5d31e535644fcb48119d7888aba2f)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:01:59.292038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:01:59.339049 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20d5d31e535644fcb48119d7888aba2f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.347186 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20d5d31e535644fcb48119d7888aba2f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.359583 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20d5d31e535644fcb48119d7888aba2f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.360212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20d5d31e535644fcb48119d7888aba2f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.360854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20d5d31e535644fcb48119d7888aba2f'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.373709 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4400/8887] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 22:01:59 2022] DELETE /identity/v3/users/20d5d31e535644fcb48119d7888aba2f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 22:01:59.582932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d8879-127f-41a9-b1fa-bb680c96cd4d service placement] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:01:59.595016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d8879-127f-41a9-b1fa-bb680c96cd4d service placement] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:01:59.655253 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:01:59.655915 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:01:59.656187 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:01:59.656487 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] PATH_INFO: `/v3/projects/392a5a2f4ca04e8ca5043d37dcef9ef1` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:01:59.657609 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=392a5a2f4ca04e8ca5043d37dcef9ef1)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:01:59.659527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:01:59.714428 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392a5a2f4ca04e8ca5043d37dcef9ef1', 'request_id': 'req-9ecd77e2-97d5-4b11-88c8-51362319fc35'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.723211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392a5a2f4ca04e8ca5043d37dcef9ef1', 'request_id': 'req-9ecd77e2-97d5-4b11-88c8-51362319fc35'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.729016 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392a5a2f4ca04e8ca5043d37dcef9ef1', 'request_id': 'req-9ecd77e2-97d5-4b11-88c8-51362319fc35'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.741168 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] The token cache is being invalidate because project 392a5a2f4ca04e8ca5043d37dcef9ef1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 22:01:59.741355 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.741989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.743481 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4488/8888] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 22:01:59 2022] DELETE /identity/v3/projects/392a5a2f4ca04e8ca5043d37dcef9ef1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 22:01:59.747776 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:01:59.749316 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:01:59.763635 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615c2da-14f3-4648-b59e-de14cd1fbfb4 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:01:59.792160 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:01:59.824088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac69fc-4259-447a-a6fd-d12f700744b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:01:59.824752 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac69fc-4259-447a-a6fd-d12f700744b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:01:59.825025 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac69fc-4259-447a-a6fd-d12f700744b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:01:59.825289 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac69fc-4259-447a-a6fd-d12f700744b4 admin admin] PATH_INFO: `/v3/users/eafdcadb96d24557877a98b6a7237c90` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:01:59.826484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac69fc-4259-447a-a6fd-d12f700744b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eafdcadb96d24557877a98b6a7237c90)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:01:59.828549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac69fc-4259-447a-a6fd-d12f700744b4 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:01:59.882019 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51ac69fc-4259-447a-a6fd-d12f700744b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eafdcadb96d24557877a98b6a7237c90'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.890529 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51ac69fc-4259-447a-a6fd-d12f700744b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eafdcadb96d24557877a98b6a7237c90'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.902808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51ac69fc-4259-447a-a6fd-d12f700744b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eafdcadb96d24557877a98b6a7237c90'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.903458 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51ac69fc-4259-447a-a6fd-d12f700744b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eafdcadb96d24557877a98b6a7237c90'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.904078 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-51ac69fc-4259-447a-a6fd-d12f700744b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eafdcadb96d24557877a98b6a7237c90'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:01:59.917958 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4401/8889] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 22:01:59 2022] DELETE /identity/v3/users/eafdcadb96d24557877a98b6a7237c90 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 22:02:00.131344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:00.133404 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:02:00.146929 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd77e2-97d5-4b11-88c8-51362319fc35 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:00.204563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:00.205188 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:00.205454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:00.205710 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f admin admin] PATH_INFO: `/v3/projects/1c9aa95ee38645e98b0a59f7e47199ba` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:00.206831 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c9aa95ee38645e98b0a59f7e47199ba)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:00.208823 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:00.220252 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:02:00.258268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c9aa95ee38645e98b0a59f7e47199ba', 'request_id': 'req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:02:00.267555 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c9aa95ee38645e98b0a59f7e47199ba', 'request_id': 'req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:02:00.274221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c9aa95ee38645e98b0a59f7e47199ba', 'request_id': 'req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:02:00.285877 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f admin admin] The token cache is being invalidate because project 1c9aa95ee38645e98b0a59f7e47199ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 22:02:00.286093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:02:00.286989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-9301b3d6-546e-4e1b-8a7b-ea57c7645a4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:02:00.289485 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4489/8890] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 22:02:00 2022] DELETE /identity/v3/projects/1c9aa95ee38645e98b0a59f7e47199ba => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 22:02:00.295717 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f9a81-a6b0-428d-8b83-7c050628926c None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:00.296239 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f9a81-a6b0-428d-8b83-7c050628926c None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:00.296352 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f9a81-a6b0-428d-8b83-7c050628926c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:00.318134 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c43f9a81-a6b0-428d-8b83-7c050628926c None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:02:00.319414 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c43f9a81-a6b0-428d-8b83-7c050628926c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:02:00.385416 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4402/8891] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 22:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:02:00.395757 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6dea4-7a9a-4657-bf57-cd000096816d None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:00.396015 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6dea4-7a9a-4657-bf57-cd000096816d None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:00.396312 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6dea4-7a9a-4657-bf57-cd000096816d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:00.418628 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-51f6dea4-7a9a-4657-bf57-cd000096816d None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:02:00.419848 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-51f6dea4-7a9a-4657-bf57-cd000096816d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:02:00.429431 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4490/8892] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 22:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:02:00.433855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f9a81-a6b0-428d-8b83-7c050628926c None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:00.444356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f9a81-a6b0-428d-8b83-7c050628926c None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:00.450055 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f453810-bbe0-40df-9c9e-3d0bc591b214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:00.450655 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f453810-bbe0-40df-9c9e-3d0bc591b214 admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:00.450935 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f453810-bbe0-40df-9c9e-3d0bc591b214 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:00.451201 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f453810-bbe0-40df-9c9e-3d0bc591b214 admin admin] PATH_INFO: `/v3/domains` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:00.451848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f453810-bbe0-40df-9c9e-3d0bc591b214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 20 22:02:00.452150 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f453810-bbe0-40df-9c9e-3d0bc591b214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:00.453817 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f453810-bbe0-40df-9c9e-3d0bc591b214 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:00.466066 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4403/8893] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Aug 20 22:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 20 22:02:00.470582 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6dea4-7a9a-4657-bf57-cd000096816d None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:00.481146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6dea4-7a9a-4657-bf57-cd000096816d None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:00.486626 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d3dad-df7e-45f8-bc3f-9fbf36f92352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:00.487233 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d3dad-df7e-45f8-bc3f-9fbf36f92352 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:00.487503 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d3dad-df7e-45f8-bc3f-9fbf36f92352 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:00.487768 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d3dad-df7e-45f8-bc3f-9fbf36f92352 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:00.488511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d3dad-df7e-45f8-bc3f-9fbf36f92352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:00.490432 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d3dad-df7e-45f8-bc3f-9fbf36f92352 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:00.529779 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4491/8894] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 22:02:00 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 22:02:00.536549 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f453810-bbe0-40df-9c9e-3d0bc591b214 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:00.549199 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f453810-bbe0-40df-9c9e-3d0bc591b214 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:00.593878 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:02:00.624854 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162fb8b-ea02-40ce-9f4e-df7fa7de379f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:00.625461 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162fb8b-ea02-40ce-9f4e-df7fa7de379f admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:00.625736 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162fb8b-ea02-40ce-9f4e-df7fa7de379f admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:00.626002 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162fb8b-ea02-40ce-9f4e-df7fa7de379f admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:00.626623 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162fb8b-ea02-40ce-9f4e-df7fa7de379f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:00.628668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162fb8b-ea02-40ce-9f4e-df7fa7de379f admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:00.658167 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4404/8895] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 22:02:00 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 22:02:00.659402 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d3dad-df7e-45f8-bc3f-9fbf36f92352 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:00.670756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d3dad-df7e-45f8-bc3f-9fbf36f92352 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:00.676327 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c7713-aa68-4aca-83bd-b22790f6c5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:00.677046 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c7713-aa68-4aca-83bd-b22790f6c5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:00.677343 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c7713-aa68-4aca-83bd-b22790f6c5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:00.677605 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c7713-aa68-4aca-83bd-b22790f6c5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:00.678286 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c7713-aa68-4aca-83bd-b22790f6c5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:00.679970 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c7713-aa68-4aca-83bd-b22790f6c5d7 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:00.690545 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4492/8896] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 22:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 20 22:02:00.694989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162fb8b-ea02-40ce-9f4e-df7fa7de379f admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:00.706113 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162fb8b-ea02-40ce-9f4e-df7fa7de379f admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:00.711748 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8b7cc-37e7-4418-9de5-eef0d028412b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:00.712556 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8b7cc-37e7-4418-9de5-eef0d028412b admin admin] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:00.712825 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8b7cc-37e7-4418-9de5-eef0d028412b admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:00.713084 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8b7cc-37e7-4418-9de5-eef0d028412b admin admin] PATH_INFO: `/v3/roles` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:00.713711 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8b7cc-37e7-4418-9de5-eef0d028412b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:00.715306 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8b7cc-37e7-4418-9de5-eef0d028412b admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:00.726067 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4405/8897] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 22:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 22:02:00.730788 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c7713-aa68-4aca-83bd-b22790f6c5d7 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:00.741961 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c7713-aa68-4aca-83bd-b22790f6c5d7 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:00.747210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c83e2f-3091-4a03-9580-c329a3e52dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:00.747873 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c83e2f-3091-4a03-9580-c329a3e52dca admin admin] REQUEST_METHOD: `PUT` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:00.748146 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c83e2f-3091-4a03-9580-c329a3e52dca admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:00.748454 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c83e2f-3091-4a03-9580-c329a3e52dca admin admin] PATH_INFO: `/v3/projects/18ca8f32d6274fb5beb086e22cc9c65a/users/42c5ba9cf6174da0a906beb63397e2d9/roles/5eaf9701f37e480bb87c89f7391d5d3a` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:00.764218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c83e2f-3091-4a03-9580-c329a3e52dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=18ca8f32d6274fb5beb086e22cc9c65a, user_id=42c5ba9cf6174da0a906beb63397e2d9, role_id=5eaf9701f37e480bb87c89f7391d5d3a)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:00.766948 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c83e2f-3091-4a03-9580-c329a3e52dca admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:00.781243 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4493/8898] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 22:02:00 2022] PUT /identity/v3/projects/18ca8f32d6274fb5beb086e22cc9c65a/users/42c5ba9cf6174da0a906beb63397e2d9/roles/5eaf9701f37e480bb87c89f7391d5d3a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 22:02:00.786694 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99f688-0fb4-438b-a08d-1e58258e82ba None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:00.786905 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99f688-0fb4-438b-a08d-1e58258e82ba None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:00.787175 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99f688-0fb4-438b-a08d-1e58258e82ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:00.806679 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-8e99f688-0fb4-438b-a08d-1e58258e82ba None None] MFA Rules not processed for user `4ceb679399ff439a9bfe96f9758e269d`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:02:00.807896 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-8e99f688-0fb4-438b-a08d-1e58258e82ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:02:00.869508 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4406/8899] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 22:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 22:02:00.880042 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c83e2f-3091-4a03-9580-c329a3e52dca admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:00.882067 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-47c83e2f-3091-4a03-9580-c329a3e52dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:02:00.964406 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c83e2f-3091-4a03-9580-c329a3e52dca admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:00.968319 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686fd06-47e4-446c-aead-f7d17a647ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:00.968949 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686fd06-47e4-446c-aead-f7d17a647ddc service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:00.969217 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686fd06-47e4-446c-aead-f7d17a647ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:00.969477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686fd06-47e4-446c-aead-f7d17a647ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:00.979093 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686fd06-47e4-446c-aead-f7d17a647ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:00.980537 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686fd06-47e4-446c-aead-f7d17a647ddc service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:00.990122 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4494/8900] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:02:04.908939 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e738c86-0ba9-4c89-a7a1-04f42d37011b None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:04.909513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e738c86-0ba9-4c89-a7a1-04f42d37011b None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:04.909513 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e738c86-0ba9-4c89-a7a1-04f42d37011b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:04.940021 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-5e738c86-0ba9-4c89-a7a1-04f42d37011b None None] MFA Rules not processed for user `42c5ba9cf6174da0a906beb63397e2d9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:02:04.942182 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-5e738c86-0ba9-4c89-a7a1-04f42d37011b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:02:05.026513 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4407/8901] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 22:02:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:02:05.031104 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686fd06-47e4-446c-aead-f7d17a647ddc service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:05.042136 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686fd06-47e4-446c-aead-f7d17a647ddc service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:05.046933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa08984-33b3-4b6d-894d-131f0ca34e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:05.047553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa08984-33b3-4b6d-894d-131f0ca34e84 admin admin] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:05.048024 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa08984-33b3-4b6d-894d-131f0ca34e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:05.048294 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa08984-33b3-4b6d-894d-131f0ca34e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:05.048988 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa08984-33b3-4b6d-894d-131f0ca34e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:05.050918 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa08984-33b3-4b6d-894d-131f0ca34e84 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:05.075079 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:02:05.084674 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4495/8902] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Aug 20 22:02:05 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 20 22:02:05.089266 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e738c86-0ba9-4c89-a7a1-04f42d37011b None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:05.106577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e738c86-0ba9-4c89-a7a1-04f42d37011b None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:05.156571 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:02:05.177990 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d760a-b8f5-491b-b23b-ab30014f8b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:05.179177 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d760a-b8f5-491b-b23b-ab30014f8b97 admin admin] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:05.179610 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d760a-b8f5-491b-b23b-ab30014f8b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:05.180032 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d760a-b8f5-491b-b23b-ab30014f8b97 admin admin] PATH_INFO: `/v3/users` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:05.180855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d760a-b8f5-491b-b23b-ab30014f8b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:05.183642 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d760a-b8f5-491b-b23b-ab30014f8b97 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:05.207534 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4408/8903] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Aug 20 22:02:05 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 20 22:02:05.212170 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa08984-33b3-4b6d-894d-131f0ca34e84 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:05.226660 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa08984-33b3-4b6d-894d-131f0ca34e84 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:05.234841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4e4e4-1355-4cfe-86a0-97db8b1221ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:05.235729 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4e4e4-1355-4cfe-86a0-97db8b1221ed admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:05.235995 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4e4e4-1355-4cfe-86a0-97db8b1221ed admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:05.236257 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4e4e4-1355-4cfe-86a0-97db8b1221ed admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:05.236983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4e4e4-1355-4cfe-86a0-97db8b1221ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:05.238712 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4e4e4-1355-4cfe-86a0-97db8b1221ed admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:05.252567 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4496/8904] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 22:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 22:02:05.255799 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d760a-b8f5-491b-b23b-ab30014f8b97 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:05.268488 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d760a-b8f5-491b-b23b-ab30014f8b97 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:05.273865 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4e92d-edbb-4fa1-99ec-6073f18e1707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:05.274699 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4e92d-edbb-4fa1-99ec-6073f18e1707 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:05.275178 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4e92d-edbb-4fa1-99ec-6073f18e1707 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:05.275848 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4e92d-edbb-4fa1-99ec-6073f18e1707 admin admin] PATH_INFO: `/v3/projects/6ae35334f62f46279f06c08efd6cac10/users/f6ab9b646ce14585b3fdef23e61a8a56/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:05.292702 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4e92d-edbb-4fa1-99ec-6073f18e1707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae35334f62f46279f06c08efd6cac10, user_id=f6ab9b646ce14585b3fdef23e61a8a56, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:05.295839 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4e92d-edbb-4fa1-99ec-6073f18e1707 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:05.313672 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4409/8905] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 22:02:05 2022] PUT /identity/v3/projects/6ae35334f62f46279f06c08efd6cac10/users/f6ab9b646ce14585b3fdef23e61a8a56/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 22:02:05.317955 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4e4e4-1355-4cfe-86a0-97db8b1221ed admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:05.330752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4e4e4-1355-4cfe-86a0-97db8b1221ed admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:05.397836 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f832d-4f5f-4ec7-a5ad-00e1189b1dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:05.398684 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f832d-4f5f-4ec7-a5ad-00e1189b1dea admin admin] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:05.399022 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f832d-4f5f-4ec7-a5ad-00e1189b1dea admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:05.399398 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f832d-4f5f-4ec7-a5ad-00e1189b1dea admin admin] PATH_INFO: `/v3/roles` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:05.400132 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f832d-4f5f-4ec7-a5ad-00e1189b1dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:05.401936 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f832d-4f5f-4ec7-a5ad-00e1189b1dea admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:05.413942 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4497/8906] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Aug 20 22:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 20 22:02:05.418027 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4e92d-edbb-4fa1-99ec-6073f18e1707 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:05.430356 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4e92d-edbb-4fa1-99ec-6073f18e1707 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:05.435516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c6514-74b1-4bde-8dc6-6c3789d22622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:05.436272 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c6514-74b1-4bde-8dc6-6c3789d22622 admin admin] REQUEST_METHOD: `PUT` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:05.436664 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c6514-74b1-4bde-8dc6-6c3789d22622 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:05.437043 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c6514-74b1-4bde-8dc6-6c3789d22622 admin admin] PATH_INFO: `/v3/projects/6ae35334f62f46279f06c08efd6cac10/users/f6ab9b646ce14585b3fdef23e61a8a56/roles/1e7f1276e60442b0bb784f6e1d0a4647` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:05.439601 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c6514-74b1-4bde-8dc6-6c3789d22622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae35334f62f46279f06c08efd6cac10, user_id=f6ab9b646ce14585b3fdef23e61a8a56, role_id=1e7f1276e60442b0bb784f6e1d0a4647)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:05.442575 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c6514-74b1-4bde-8dc6-6c3789d22622 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:05.455463 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4410/8907] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Aug 20 22:02:05 2022] PUT /identity/v3/projects/6ae35334f62f46279f06c08efd6cac10/users/f6ab9b646ce14585b3fdef23e61a8a56/roles/1e7f1276e60442b0bb784f6e1d0a4647 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 22:02:10.134623 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc8592-d82a-4e47-9dcc-961540d79eae None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:10.135202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc8592-d82a-4e47-9dcc-961540d79eae None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:10.135202 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc8592-d82a-4e47-9dcc-961540d79eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:10.167815 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-a7dc8592-d82a-4e47-9dcc-961540d79eae None None] MFA Rules not processed for user `f6ab9b646ce14585b3fdef23e61a8a56`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:02:10.169076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-a7dc8592-d82a-4e47-9dcc-961540d79eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:02:10.247892 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4498/8908] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 22:02:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:02:10.270195 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c6514-74b1-4bde-8dc6-6c3789d22622 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:10.272327 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-2c7c6514-74b1-4bde-8dc6-6c3789d22622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:02:10.355457 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c6514-74b1-4bde-8dc6-6c3789d22622 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:10.361036 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ecf63-4a8a-42fa-aadd-c83767254a5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:10.361665 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ecf63-4a8a-42fa-aadd-c83767254a5b service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:10.361950 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ecf63-4a8a-42fa-aadd-c83767254a5b service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:10.362205 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ecf63-4a8a-42fa-aadd-c83767254a5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:10.423443 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ecf63-4a8a-42fa-aadd-c83767254a5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:10.425102 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ecf63-4a8a-42fa-aadd-c83767254a5b service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:10.434749 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4411/8909] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:02:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:02:11.487958 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d5b88-e485-4166-8312-8196291adfc3 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:11.488464 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d5b88-e485-4166-8312-8196291adfc3 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:11.488562 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d5b88-e485-4166-8312-8196291adfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:11.507983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-5b3d5b88-e485-4166-8312-8196291adfc3 None None] MFA Rules not processed for user `42c5ba9cf6174da0a906beb63397e2d9`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:02:11.509176 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5b3d5b88-e485-4166-8312-8196291adfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:02:11.521835 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4499/8910] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 22:02:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:02:11.869389 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ecf63-4a8a-42fa-aadd-c83767254a5b service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:11.906631 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:02:11.929215 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ecf63-4a8a-42fa-aadd-c83767254a5b service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:11.933691 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7ff85-42a2-4bd2-a850-b18c3a3fc020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:11.934315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7ff85-42a2-4bd2-a850-b18c3a3fc020 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:11.934586 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7ff85-42a2-4bd2-a850-b18c3a3fc020 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:11.934857 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7ff85-42a2-4bd2-a850-b18c3a3fc020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:11.936674 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-4de7ff85-42a2-4bd2-a850-b18c3a3fc020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:02:11.946495 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de7ff85-42a2-4bd2-a850-b18c3a3fc020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:11.948218 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de7ff85-42a2-4bd2-a850-b18c3a3fc020 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:11.956688 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4412/8911] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:02:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:02:12.318591 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d5b88-e485-4166-8312-8196291adfc3 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:12.320857 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-5b3d5b88-e485-4166-8312-8196291adfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:02:12.408416 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=100 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:02:12.413374 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d5b88-e485-4166-8312-8196291adfc3 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:12.418523 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12236b-4f4d-4ae8-8d1f-1cfda6b84428 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:12.419134 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12236b-4f4d-4ae8-8d1f-1cfda6b84428 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:12.419714 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12236b-4f4d-4ae8-8d1f-1cfda6b84428 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:12.419983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12236b-4f4d-4ae8-8d1f-1cfda6b84428 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:12.421808 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9e12236b-4f4d-4ae8-8d1f-1cfda6b84428 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:02:12.434787 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12236b-4f4d-4ae8-8d1f-1cfda6b84428 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:12.436410 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12236b-4f4d-4ae8-8d1f-1cfda6b84428 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:12.447040 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4500/8912] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:02:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:02:21.953689 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=11 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:02:22.443399 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=4 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:02:23.582404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7ff85-42a2-4bd2-a850-b18c3a3fc020 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:02:23.591951 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7ff85-42a2-4bd2-a850-b18c3a3fc020 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:02:23.595892 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca755e6-70c2-4e7a-a025-5824927339f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:02:23.596519 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca755e6-70c2-4e7a-a025-5824927339f9 service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:02:23.596791 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca755e6-70c2-4e7a-a025-5824927339f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:02:23.597068 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca755e6-70c2-4e7a-a025-5824927339f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:02:23.606516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca755e6-70c2-4e7a-a025-5824927339f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:02:23.607956 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca755e6-70c2-4e7a-a025-5824927339f9 service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:02:23.617691 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4413/8913] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:02:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:02:33.613330 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:04:55.713440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b6488-1891-4d93-b799-17b37a66faeb None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:04:55.714125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b6488-1891-4d93-b799-17b37a66faeb None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:04:55.714125 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b6488-1891-4d93-b799-17b37a66faeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:04:55.762421 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-fb5b6488-1891-4d93-b799-17b37a66faeb None None] MFA Rules not processed for user `698d5a7fc34a46ba909152414a0860d7`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:04:55.763601 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fb5b6488-1891-4d93-b799-17b37a66faeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:04:55.818785 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4501/8914] 10.222.0.23 () {58 vars in 1064 bytes} [Sat Aug 20 22:04:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:04:55.824744 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca755e6-70c2-4e7a-a025-5824927339f9 service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:04:55.834516 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca755e6-70c2-4e7a-a025-5824927339f9 service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:04:55.839436 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805a9c2-b4c4-4ee4-8764-40aa9f1bfc18 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:04:55.840037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805a9c2-b4c4-4ee4-8764-40aa9f1bfc18 service glance] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:04:55.840315 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805a9c2-b4c4-4ee4-8764-40aa9f1bfc18 service glance] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:04:55.840613 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805a9c2-b4c4-4ee4-8764-40aa9f1bfc18 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:04:55.842616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-b805a9c2-b4c4-4ee4-8764-40aa9f1bfc18 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:04:55.876146 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805a9c2-b4c4-4ee4-8764-40aa9f1bfc18 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:04:55.877690 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805a9c2-b4c4-4ee4-8764-40aa9f1bfc18 service glance] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:04:55.888747 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4414/8915] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:04:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:05:04.242248 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b6488-1891-4d93-b799-17b37a66faeb None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:05:04.244086 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-fb5b6488-1891-4d93-b799-17b37a66faeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:04.261437 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b6488-1891-4d93-b799-17b37a66faeb None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:05:04.267667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd11fa-54ce-4a5a-b23a-f71daba3924f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:05:04.268539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd11fa-54ce-4a5a-b23a-f71daba3924f service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:05:04.268971 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd11fa-54ce-4a5a-b23a-f71daba3924f service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:05:04.269505 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd11fa-54ce-4a5a-b23a-f71daba3924f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:05:04.272052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2ccd11fa-54ce-4a5a-b23a-f71daba3924f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:04.288597 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd11fa-54ce-4a5a-b23a-f71daba3924f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:05:04.290279 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd11fa-54ce-4a5a-b23a-f71daba3924f service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:05:04.303072 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4502/8916] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:05:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:05:04.637227 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8eee27-f0a1-4493-bed8-c0493f3a2688 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:05:04.637770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8eee27-f0a1-4493-bed8-c0493f3a2688 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:05:04.637770 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8eee27-f0a1-4493-bed8-c0493f3a2688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:05:04.657484 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-cb8eee27-f0a1-4493-bed8-c0493f3a2688 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:05:04.658668 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-cb8eee27-f0a1-4493-bed8-c0493f3a2688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:04.717318 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4415/8917] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 22:05:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:05:04.769722 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd11fa-54ce-4a5a-b23a-f71daba3924f service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:05:04.771621 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-2ccd11fa-54ce-4a5a-b23a-f71daba3924f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:04.844240 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd11fa-54ce-4a5a-b23a-f71daba3924f service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:05:04.849778 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab2a58-4da5-4bd0-ba27-48936785d7ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:05:04.850377 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab2a58-4da5-4bd0-ba27-48936785d7ae service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:05:04.850639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab2a58-4da5-4bd0-ba27-48936785d7ae service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:05:04.850899 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab2a58-4da5-4bd0-ba27-48936785d7ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:05:04.859199 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab2a58-4da5-4bd0-ba27-48936785d7ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:05:04.860841 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab2a58-4da5-4bd0-ba27-48936785d7ae service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:05:04.869066 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4503/8918] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 22:05:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 22:05:14.710886 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=39 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:05:14.831301 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ab497-044e-4b70-a2e6-a20075f03431 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:05:14.831581 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ab497-044e-4b70-a2e6-a20075f03431 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:05:14.831827 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ab497-044e-4b70-a2e6-a20075f03431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:05:14.852913 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-e81ab497-044e-4b70-a2e6-a20075f03431 None None] MFA Rules not processed for user `42c5ba9cf6174da0a906beb63397e2d9`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:05:14.854096 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e81ab497-044e-4b70-a2e6-a20075f03431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:14.861979 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4416/8919] 10.222.0.23 () {58 vars in 958 bytes} [Sat Aug 20 22:05:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:05:14.865651 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=71 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:05:15.194477 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab2a58-4da5-4bd0-ba27-48936785d7ae service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:05:15.204459 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab2a58-4da5-4bd0-ba27-48936785d7ae service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:05:15.208344 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99360166-0521-43e5-aaa6-131c51d13c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:05:15.208983 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99360166-0521-43e5-aaa6-131c51d13c52 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:05:15.209232 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99360166-0521-43e5-aaa6-131c51d13c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:05:15.209501 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99360166-0521-43e5-aaa6-131c51d13c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:05:15.211679 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-99360166-0521-43e5-aaa6-131c51d13c52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:15.221088 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99360166-0521-43e5-aaa6-131c51d13c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:05:15.222506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99360166-0521-43e5-aaa6-131c51d13c52 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:05:15.230642 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4504/8920] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:05:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:05:15.590943 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ab497-044e-4b70-a2e6-a20075f03431 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:05:15.603341 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ab497-044e-4b70-a2e6-a20075f03431 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:05:15.608838 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992dde4d-172d-4f9b-8e27-f9e6308c9294 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:05:15.609446 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dde4d-172d-4f9b-8e27-f9e6308c9294 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:05:15.609720 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dde4d-172d-4f9b-8e27-f9e6308c9294 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:05:15.610008 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dde4d-172d-4f9b-8e27-f9e6308c9294 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:05:15.611775 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-992dde4d-172d-4f9b-8e27-f9e6308c9294 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:15.611932 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99360166-0521-43e5-aaa6-131c51d13c52 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:05:15.621511 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99360166-0521-43e5-aaa6-131c51d13c52 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:05:15.625759 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ec948-a0cf-4d2b-9d77-bd7ac844cb33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:05:15.626553 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ec948-a0cf-4d2b-9d77-bd7ac844cb33 service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:05:15.626833 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ec948-a0cf-4d2b-9d77-bd7ac844cb33 service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:05:15.627099 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ec948-a0cf-4d2b-9d77-bd7ac844cb33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:05:15.628989 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-7c3ec948-a0cf-4d2b-9d77-bd7ac844cb33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:15.638282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992dde4d-172d-4f9b-8e27-f9e6308c9294 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:05:15.639881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992dde4d-172d-4f9b-8e27-f9e6308c9294 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:05:15.643033 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ec948-a0cf-4d2b-9d77-bd7ac844cb33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:05:15.644762 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ec948-a0cf-4d2b-9d77-bd7ac844cb33 service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:05:15.649724 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4417/8921] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:05:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:05:15.657230 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4505/8922] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:05:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:05:16.995931 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48843-3b33-4fdb-adfa-365d47faf8bd None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:05:16.996426 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48843-3b33-4fdb-adfa-365d47faf8bd None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:05:16.996511 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48843-3b33-4fdb-adfa-365d47faf8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:05:17.016130 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-0eb48843-3b33-4fdb-adfa-365d47faf8bd None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:05:17.017596 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0eb48843-3b33-4fdb-adfa-365d47faf8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:17.025537 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4418/8923] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 22:05:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:05:17.038210 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ec948-a0cf-4d2b-9d77-bd7ac844cb33 service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:05:17.050204 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ec948-a0cf-4d2b-9d77-bd7ac844cb33 service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:05:17.055933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a7935-8e48-4c99-a5bd-e6775d655a43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:05:17.056587 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7935-8e48-4c99-a5bd-e6775d655a43 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:05:17.056852 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7935-8e48-4c99-a5bd-e6775d655a43 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:05:17.057108 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7935-8e48-4c99-a5bd-e6775d655a43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:05:17.066322 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a7935-8e48-4c99-a5bd-e6775d655a43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:05:17.068179 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a7935-8e48-4c99-a5bd-e6775d655a43 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:05:17.078310 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4506/8924] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 22:05:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 22:05:25.285422 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb48843-3b33-4fdb-adfa-365d47faf8bd None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:05:25.296822 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb48843-3b33-4fdb-adfa-365d47faf8bd None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:05:25.301031 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d56bae-c39b-401d-bfea-e4584eb453ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:05:25.301629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d56bae-c39b-401d-bfea-e4584eb453ad service neutron] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:05:25.301900 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d56bae-c39b-401d-bfea-e4584eb453ad service neutron] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:05:25.302162 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d56bae-c39b-401d-bfea-e4584eb453ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:05:25.303923 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-65d56bae-c39b-401d-bfea-e4584eb453ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:25.313573 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d56bae-c39b-401d-bfea-e4584eb453ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:05:25.314999 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d56bae-c39b-401d-bfea-e4584eb453ad service neutron] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:05:25.323236 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4419/8925] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:05:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:05:27.073772 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=18 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:05:35.319812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=23 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:05:35.944888 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a7935-8e48-4c99-a5bd-e6775d655a43 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:05:35.957528 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a7935-8e48-4c99-a5bd-e6775d655a43 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:05:35.964654 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd954b-ef97-4851-8e89-843b6a67ea41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:05:35.965284 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd954b-ef97-4851-8e89-843b6a67ea41 service nova] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:05:35.965551 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd954b-ef97-4851-8e89-843b6a67ea41 service nova] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:05:35.965782 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd954b-ef97-4851-8e89-843b6a67ea41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:05:35.967829 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dafd954b-ef97-4851-8e89-843b6a67ea41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:35.981318 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd954b-ef97-4851-8e89-843b6a67ea41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:05:35.982908 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd954b-ef97-4851-8e89-843b6a67ea41 service nova] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:05:35.994933 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4507/8926] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:05:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:05:45.989921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:05:48.979952 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d56bae-c39b-401d-bfea-e4584eb453ad service neutron] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:05:48.981685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-65d56bae-c39b-401d-bfea-e4584eb453ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:48.998480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d56bae-c39b-401d-bfea-e4584eb453ad service neutron] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:05:49.005238 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026caab-47e9-457e-946d-71734b39fd30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:05:49.005846 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026caab-47e9-457e-946d-71734b39fd30 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:05:49.006114 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026caab-47e9-457e-946d-71734b39fd30 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:05:49.006376 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026caab-47e9-457e-946d-71734b39fd30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:05:49.008279 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-e026caab-47e9-457e-946d-71734b39fd30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:49.019880 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026caab-47e9-457e-946d-71734b39fd30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:05:49.021480 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026caab-47e9-457e-946d-71734b39fd30 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:05:49.031734 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4420/8927] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:05:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:05:51.889440 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd954b-ef97-4851-8e89-843b6a67ea41 service nova] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:05:51.892005 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-dafd954b-ef97-4851-8e89-843b6a67ea41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:51.908490 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd954b-ef97-4851-8e89-843b6a67ea41 service nova] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:05:51.913076 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d142fa2-e9fc-40ef-abcd-99d1fb40416a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:05:51.913667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d142fa2-e9fc-40ef-abcd-99d1fb40416a service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:05:51.913933 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d142fa2-e9fc-40ef-abcd-99d1fb40416a service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:05:51.914185 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d142fa2-e9fc-40ef-abcd-99d1fb40416a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:05:51.916221 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-9d142fa2-e9fc-40ef-abcd-99d1fb40416a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:05:51.926087 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d142fa2-e9fc-40ef-abcd-99d1fb40416a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:05:51.927504 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d142fa2-e9fc-40ef-abcd-99d1fb40416a service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:05:51.936102 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4508/8928] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:05:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:05:59.027411 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:06:01.932426 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=6 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:06:47.149882 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026caab-47e9-457e-946d-71734b39fd30 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:06:47.161528 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026caab-47e9-457e-946d-71734b39fd30 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:06:47.167907 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1cc73-52f8-4995-a780-14e8656fc611 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:06:47.168535 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1cc73-52f8-4995-a780-14e8656fc611 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:06:47.168812 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1cc73-52f8-4995-a780-14e8656fc611 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:06:47.169072 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1cc73-52f8-4995-a780-14e8656fc611 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:06:47.171193 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-0dc1cc73-52f8-4995-a780-14e8656fc611 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:06:47.183591 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc1cc73-52f8-4995-a780-14e8656fc611 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:06:47.185226 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc1cc73-52f8-4995-a780-14e8656fc611 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:06:47.195402 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4421/8929] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:06:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:06:57.191233 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=6 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:07:06.672921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a90a-fa60-49d4-97d8-0ab7652f8002 None None] REQUEST_METHOD: `POST` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:06.673499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a90a-fa60-49d4-97d8-0ab7652f8002 None None] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:06.673499 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a90a-fa60-49d4-97d8-0ab7652f8002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:06.693017 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.auth.core [None req-6a91a90a-fa60-49d4-97d8-0ab7652f8002 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:07:06.694218 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6a91a90a-fa60-49d4-97d8-0ab7652f8002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:07:06.701921 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4509/8930] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Aug 20 22:07:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 20 22:07:06.743764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1cc73-52f8-4995-a780-14e8656fc611 service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:07:06.756903 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1cc73-52f8-4995-a780-14e8656fc611 service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:07:06.762183 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd48f6-5931-4d59-9856-0793487cc764 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:07:06.762724 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd48f6-5931-4d59-9856-0793487cc764 service swift] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:06.763283 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd48f6-5931-4d59-9856-0793487cc764 service swift] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:06.763554 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd48f6-5931-4d59-9856-0793487cc764 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:06.771819 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd48f6-5931-4d59-9856-0793487cc764 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:07:06.773474 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd48f6-5931-4d59-9856-0793487cc764 service swift] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:07:06.781312 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4422/8931] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 22:07:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 22:07:10.631126 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91a90a-fa60-49d4-97d8-0ab7652f8002 None None] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:07:10.633500 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-6a91a90a-fa60-49d4-97d8-0ab7652f8002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:07:10.645795 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91a90a-fa60-49d4-97d8-0ab7652f8002 None None] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:07:10.650032 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4136a2-e04c-49c9-aac1-df13078be85d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698d5a7fc34a46ba909152414a0860d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:07:10.650974 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4136a2-e04c-49c9-aac1-df13078be85d service glance] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:10.651332 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4136a2-e04c-49c9-aac1-df13078be85d service glance] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:10.651667 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4136a2-e04c-49c9-aac1-df13078be85d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:10.662072 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4136a2-e04c-49c9-aac1-df13078be85d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:07:10.663639 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4136a2-e04c-49c9-aac1-df13078be85d service glance] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:07:10.679850 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4510/8932] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:07:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:07:15.916932 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d1191-3138-4545-8b2d-e3864cdf9d26 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:15.917470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d1191-3138-4545-8b2d-e3864cdf9d26 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:15.917470 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d1191-3138-4545-8b2d-e3864cdf9d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:15.938038 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-d75d1191-3138-4545-8b2d-e3864cdf9d26 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:07:15.939371 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-d75d1191-3138-4545-8b2d-e3864cdf9d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:07:15.947372 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4423/8933] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 22:07:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:07:15.960269 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4136a2-e04c-49c9-aac1-df13078be85d service glance] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:07:15.973563 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4136a2-e04c-49c9-aac1-df13078be85d service glance] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:07:15.980114 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233ea4e-d2c0-44ab-b9a6-b23759951393 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:07:15.980769 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233ea4e-d2c0-44ab-b9a6-b23759951393 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:15.981041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233ea4e-d2c0-44ab-b9a6-b23759951393 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:15.981300 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233ea4e-d2c0-44ab-b9a6-b23759951393 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:15.990898 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233ea4e-d2c0-44ab-b9a6-b23759951393 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:07:15.992569 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233ea4e-d2c0-44ab-b9a6-b23759951393 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:07:16.002177 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4511/8934] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 22:07:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 20 22:07:20.995313 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7673061-bace-46d5-ba27-223298cb9127 None None] REQUEST_METHOD: `POST` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:20.995790 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7673061-bace-46d5-ba27-223298cb9127 None None] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:20.995893 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7673061-bace-46d5-ba27-223298cb9127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:21.015585 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.auth.core [None req-c7673061-bace-46d5-ba27-223298cb9127 None None] MFA Rules not processed for user `1547cb99abe642ae86db8de1fa80041f`. Rule list: `[]` (Enabled: `True`). {{(pid=104119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 20 22:07:21.016808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-c7673061-bace-46d5-ba27-223298cb9127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:07:21.024284 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4424/8935] 10.222.0.23 () {58 vars in 967 bytes} [Sat Aug 20 22:07:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 20 22:07:21.037278 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233ea4e-d2c0-44ab-b9a6-b23759951393 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:07:21.048049 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233ea4e-d2c0-44ab-b9a6-b23759951393 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:07:21.053875 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597442f2-ad9c-4efb-b9af-eb64cc136981 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd74f7cdc155a4397ad28a3fbca86f7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:07:21.054471 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597442f2-ad9c-4efb-b9af-eb64cc136981 service swift] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:21.054740 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597442f2-ad9c-4efb-b9af-eb64cc136981 service swift] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:21.054999 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597442f2-ad9c-4efb-b9af-eb64cc136981 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:21.064025 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597442f2-ad9c-4efb-b9af-eb64cc136981 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:07:21.065727 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597442f2-ad9c-4efb-b9af-eb64cc136981 service swift] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:07:21.073924 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4512/8936] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Aug 20 22:07:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 20 22:07:22.423468 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7673061-bace-46d5-ba27-223298cb9127 None None] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:07:22.437616 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7673061-bace-46d5-ba27-223298cb9127 None None] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:07:22.444120 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817c8ef-5734-47c1-b67d-fc1da7c19343 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df852af59e7743f7a8798d9edf8ebd3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:07:22.444764 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817c8ef-5734-47c1-b67d-fc1da7c19343 service nova] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:22.445040 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817c8ef-5734-47c1-b67d-fc1da7c19343 service nova] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:22.445305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817c8ef-5734-47c1-b67d-fc1da7c19343 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:22.447566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-9817c8ef-5734-47c1-b67d-fc1da7c19343 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:07:22.461321 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817c8ef-5734-47c1-b67d-fc1da7c19343 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:07:22.463172 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817c8ef-5734-47c1-b67d-fc1da7c19343 service nova] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:07:22.474222 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4425/8937] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:07:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:07:25.152640 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597442f2-ad9c-4efb-b9af-eb64cc136981 service swift] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:07:25.172365 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597442f2-ad9c-4efb-b9af-eb64cc136981 service swift] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:07:25.176668 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4ecb2-82c8-4dec-a7f8-bfb9cd41dd2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bb014e04774c278c99496bf64b2319', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:07:25.177268 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4ecb2-82c8-4dec-a7f8-bfb9cd41dd2b service neutron] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:25.177539 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4ecb2-82c8-4dec-a7f8-bfb9cd41dd2b service neutron] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:25.177798 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4ecb2-82c8-4dec-a7f8-bfb9cd41dd2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:25.250527 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4ecb2-82c8-4dec-a7f8-bfb9cd41dd2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:07:25.252074 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4ecb2-82c8-4dec-a7f8-bfb9cd41dd2b service neutron] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:07:25.273398 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4513/8938] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:07:28.687992 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817c8ef-5734-47c1-b67d-fc1da7c19343 service nova] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:07:28.694392 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=19 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:07:28.700836 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817c8ef-5734-47c1-b67d-fc1da7c19343 service nova] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:07:28.707351 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31705b32-b9de-4448-9041-aaf07b06082a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:07:28.708329 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31705b32-b9de-4448-9041-aaf07b06082a service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:28.708920 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31705b32-b9de-4448-9041-aaf07b06082a service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:28.709417 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31705b32-b9de-4448-9041-aaf07b06082a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:28.711844 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-31705b32-b9de-4448-9041-aaf07b06082a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:07:28.725853 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31705b32-b9de-4448-9041-aaf07b06082a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:07:28.727725 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31705b32-b9de-4448-9041-aaf07b06082a service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:07:28.739881 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4426/8939] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:07:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:07:32.151753 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4ecb2-82c8-4dec-a7f8-bfb9cd41dd2b service neutron] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:07:32.156752 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=64 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:07:32.164594 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4ecb2-82c8-4dec-a7f8-bfb9cd41dd2b service neutron] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:07:32.170570 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:07:32.171206 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:32.171465 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:32.171731 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] PATH_INFO: `/v3/users/42c5ba9cf6174da0a906beb63397e2d9` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:32.172978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c5ba9cf6174da0a906beb63397e2d9)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:07:32.174921 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:07:32.223724 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c5ba9cf6174da0a906beb63397e2d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.231687 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c5ba9cf6174da0a906beb63397e2d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.239403 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c5ba9cf6174da0a906beb63397e2d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.251379 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c5ba9cf6174da0a906beb63397e2d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.252057 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c5ba9cf6174da0a906beb63397e2d9'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.257839 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4514/8940] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 22:07:32 2022] DELETE /identity/v3/users/42c5ba9cf6174da0a906beb63397e2d9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 22:07:32.447212 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31705b32-b9de-4448-9041-aaf07b06082a service placement] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:07:32.458487 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31705b32-b9de-4448-9041-aaf07b06082a service placement] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:07:32.516081 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:07:32.518404 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:32.518685 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:32.518959 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] PATH_INFO: `/v3/projects/18ca8f32d6274fb5beb086e22cc9c65a` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:32.520122 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18ca8f32d6274fb5beb086e22cc9c65a)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:07:32.522105 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:07:32.571771 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18ca8f32d6274fb5beb086e22cc9c65a', 'request_id': 'req-07083cc2-57a6-4b85-872d-720b027e66d3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.578778 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18ca8f32d6274fb5beb086e22cc9c65a', 'request_id': 'req-07083cc2-57a6-4b85-872d-720b027e66d3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.588276 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18ca8f32d6274fb5beb086e22cc9c65a', 'request_id': 'req-07083cc2-57a6-4b85-872d-720b027e66d3'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.611808 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] The token cache is being invalidate because project 18ca8f32d6274fb5beb086e22cc9c65a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 22:07:32.612088 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.612782 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.614352 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4427/8941] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 22:07:32 2022] DELETE /identity/v3/projects/18ca8f32d6274fb5beb086e22cc9c65a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 22:07:32.618619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:07:32.620052 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:07:32.633260 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39388926-f7b5-4396-89ef-af16180c588e admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:07:32.692625 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:07:32.693281 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:32.693548 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:32.693805 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] PATH_INFO: `/v3/users/f6ab9b646ce14585b3fdef23e61a8a56` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:32.695009 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6ab9b646ce14585b3fdef23e61a8a56)` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:07:32.696978 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:07:32.745813 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6ab9b646ce14585b3fdef23e61a8a56'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.755211 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6ab9b646ce14585b3fdef23e61a8a56'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.762682 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6ab9b646ce14585b3fdef23e61a8a56'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.773320 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:07:32.775777 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6ab9b646ce14585b3fdef23e61a8a56'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.776602 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.notifications [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6ab9b646ce14585b3fdef23e61a8a56'} {{(pid=104118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:32.784123 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4515/8942] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Aug 20 22:07:32 2022] DELETE /identity/v3/users/f6ab9b646ce14585b3fdef23e61a8a56 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 22:07:32.970152 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:07:32.971629 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:07:32.986982 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07083cc2-57a6-4b85-872d-720b027e66d3 admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:07:33.047353 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ceb679399ff439a9bfe96f9758e269d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5bed9a4b90f48f592cdc8d94741f821', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:07:33.048037 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] REQUEST_METHOD: `DELETE` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:07:33.048305 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:07:33.048598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] PATH_INFO: `/v3/projects/6ae35334f62f46279f06c08efd6cac10` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:07:33.049703 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ae35334f62f46279f06c08efd6cac10)` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:07:33.051598 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:07:33.061851 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:07:33.099881 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ae35334f62f46279f06c08efd6cac10', 'request_id': 'req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:33.106282 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ae35334f62f46279f06c08efd6cac10', 'request_id': 'req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:33.116577 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ae35334f62f46279f06c08efd6cac10', 'request_id': 'req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af'} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:33.127580 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] The token cache is being invalidate because project 6ae35334f62f46279f06c08efd6cac10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 20 22:07:33.127855 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:33.128566 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.notifications [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 20 22:07:33.130186 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4428/8943] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Aug 20 22:07:32 2022] DELETE /identity/v3/projects/6ae35334f62f46279f06c08efd6cac10 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 20 22:07:42.787567 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:07:43.124617 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:10:54.955689 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] Authenticating user token {{(pid=104118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:10:54.957270 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:10:55.051619 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3200c3-c037-4000-8cdb-8fb430018f61 admin admin] Validating token access rules against request {{(pid=104118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:10:55.113906 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83ac9d-75d2-4e64-8125-016fba931b71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:10:55.114506 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83ac9d-75d2-4e64-8125-016fba931b71 service placement] REQUEST_METHOD: `GET` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:10:55.114776 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83ac9d-75d2-4e64-8125-016fba931b71 service placement] SCRIPT_NAME: `/identity` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:10:55.115041 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83ac9d-75d2-4e64-8125-016fba931b71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:10:55.116911 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.fernet_utils [None req-1a83ac9d-75d2-4e64-8125-016fba931b71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:10:55.128140 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83ac9d-75d2-4e64-8125-016fba931b71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:10:55.129756 ncdv-master-1 devstack@keystone.service[104118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83ac9d-75d2-4e64-8125-016fba931b71 service placement] RBAC: Authorization granted {{(pid=104118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:10:55.139911 ncdv-master-1 devstack@keystone.service[104118]: [pid: 104118|app: 0|req: 4516/8944] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:10:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:11:05.135650 ncdv-master-1 devstack@keystone.service[104118]: DEBUG dbcounter [-] [104118] Writing DB stats keystone:SELECT=66 {{(pid=104118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 20 22:11:53.134012 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] Authenticating user token {{(pid=104119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 20 22:11:53.135670 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:11:53.196832 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a5f1c-c4cd-491a-8fc0-9570a7c6d4af admin admin] Validating token access rules against request {{(pid=104119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 20 22:11:53.203147 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27853d36-ca7a-4963-9d3d-c2d91b5135d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b04f1e0e8094a14a3c1384faa601aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b6c88840d764c2385aa678da4338213', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 20 22:11:53.203763 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27853d36-ca7a-4963-9d3d-c2d91b5135d8 service placement] REQUEST_METHOD: `GET` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 20 22:11:53.203989 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27853d36-ca7a-4963-9d3d-c2d91b5135d8 service placement] SCRIPT_NAME: `/identity` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 20 22:11:53.204265 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27853d36-ca7a-4963-9d3d-c2d91b5135d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 20 22:11:53.206295 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.fernet_utils [None req-27853d36-ca7a-4963-9d3d-c2d91b5135d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 20 22:11:53.226039 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27853d36-ca7a-4963-9d3d-c2d91b5135d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 20 22:11:53.227653 ncdv-master-1 devstack@keystone.service[104119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27853d36-ca7a-4963-9d3d-c2d91b5135d8 service placement] RBAC: Authorization granted {{(pid=104119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 20 22:11:53.238632 ncdv-master-1 devstack@keystone.service[104119]: [pid: 104119|app: 0|req: 4429/8945] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Aug 20 22:11:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 20 22:12:03.234165 ncdv-master-1 devstack@keystone.service[104119]: DEBUG dbcounter [-] [104119] Writing DB stats keystone:SELECT=24 {{(pid=104119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}